2024-03-29T04:22:20.032 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-29T04:22:20.033 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --verbose --name teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi --description upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119 -- /home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119/orig.config.yaml 2024-03-29T04:22:20.088 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T04:22:20.175 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119 branch: squid description: upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7629119' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.1 ' name: teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 8da23624bc17fe0b1f797227e72a0db0ce7821ba ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8da23624bc17fe0b1f797227e72a0db0ce7821ba kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 8da23624bc17fe0b1f797227e72a0db0ce7821ba owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 4408 sha1: 8da23624bc17fe0b1f797227e72a0db0ce7821ba sleep_before_teardown: 0 subset: 19/32 suite: upgrade suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_8da23624bc17fe0b1f797227e72a0db0ce7821ba/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 8da23624bc17fe0b1f797227e72a0db0ce7821ba targets: smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKhuqw5ByIsVOWv4Bv0KqDWPRK5wkqtiEp0lF0GjNFCvvdH54FCOKPyJTmRhpWF4JlXASZzBBNDsjgxKMNjvZKw= smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHXTdImcwoR3lXy6Keid7Lvx4jVNBDVmUGWuWyLP+wnSrYDEGyLhxMDcmu6M+EGBk1piqUuihrtjh4gZsJTn1BU= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.1 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.1 roleless: true - print: '**** done end installing v18.2.1 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-29_02:08:11 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-29T04:22:20.175 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_8da23624bc17fe0b1f797227e72a0db0ce7821ba/qa; will attempt to use it 2024-03-29T04:22:20.176 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_8da23624bc17fe0b1f797227e72a0db0ce7821ba/qa/tasks 2024-03-29T04:22:20.176 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-29T04:22:20.177 INFO:teuthology.task.internal:Checking packages... 2024-03-29T04:22:20.201 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8da23624bc17fe0b1f797227e72a0db0ce7821ba' 2024-03-29T04:22:20.201 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T04:22:20.201 INFO:teuthology.packaging:ref: None 2024-03-29T04:22:20.201 INFO:teuthology.packaging:tag: None 2024-03-29T04:22:20.201 INFO:teuthology.packaging:branch: squid 2024-03-29T04:22:20.201 INFO:teuthology.packaging:sha1: 8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:22:20.201 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-03-29T04:22:20.488 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1959.g8da23624 2024-03-29T04:22:20.489 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-29T04:22:20.498 INFO:teuthology.task.internal:no buildpackages task found 2024-03-29T04:22:20.498 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-29T04:22:20.540 INFO:teuthology.task.internal:Saving configuration 2024-03-29T04:22:20.562 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-29T04:22:20.570 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T04:22:20.596 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 04:17:19.785464', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKhuqw5ByIsVOWv4Bv0KqDWPRK5wkqtiEp0lF0GjNFCvvdH54FCOKPyJTmRhpWF4JlXASZzBBNDsjgxKMNjvZKw='} 2024-03-29T04:22:20.617 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 04:17:19.783753', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHXTdImcwoR3lXy6Keid7Lvx4jVNBDVmUGWuWyLP+wnSrYDEGyLhxMDcmu6M+EGBk1piqUuihrtjh4gZsJTn1BU='} 2024-03-29T04:22:20.617 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-29T04:22:20.628 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-29T04:22:20.628 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-29T04:22:20.628 INFO:teuthology.run_tasks:Running task console_log... 2024-03-29T04:22:20.794 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8ba8a520d0>, signals=[15]) 2024-03-29T04:22:20.794 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-29T04:22:20.805 INFO:teuthology.task.internal:Opening connections... 2024-03-29T04:22:20.806 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T04:22:20.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:22:20.896 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2024-03-29T04:22:20.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:22:20.983 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-29T04:22:20.995 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-03-29T04:22:21.012 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-03-29T04:22:21.013 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-03-29T04:22:21.069 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-03-29T04:22:21.069 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-03-29T04:22:21.069 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-03-29T04:22:21.069 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-03-29T04:22:21.069 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-03-29T04:22:21.070 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-03-29T04:22:21.070 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-29T04:22:21.070 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-03-29T04:22:21.070 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-03-29T04:22:21.070 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-29T04:22:21.070 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-03-29T04:22:21.070 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-29T04:22:21.070 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-29T04:22:21.070 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-29T04:22:21.071 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-03-29T04:22:21.094 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2024-03-29T04:22:21.111 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2024-03-29T04:22:21.111 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2024-03-29T04:22:21.168 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2024-03-29T04:22:21.168 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="9" 2024-03-29T04:22:21.168 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2024-03-29T04:22:21.168 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2024-03-29T04:22:21.168 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="9" 2024-03-29T04:22:21.168 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el9" 2024-03-29T04:22:21.169 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-29T04:22:21.169 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2024-03-29T04:22:21.169 INFO:teuthology.orchestra.run.smithi152.stdout:LOGO="fedora-logo-icon" 2024-03-29T04:22:21.169 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-29T04:22:21.169 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2024-03-29T04:22:21.169 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-29T04:22:21.169 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-29T04:22:21.169 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-29T04:22:21.169 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2024-03-29T04:22:21.194 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-29T04:22:21.205 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-29T04:22:21.217 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-29T04:22:21.217 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-03-29T04:22:21.219 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2024-03-29T04:22:21.234 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-29T04:22:21.246 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-29T04:22:21.246 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-03-29T04:22:21.276 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2024-03-29T04:22:21.300 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-29T04:22:21.359 INFO:teuthology.run_tasks:Running task kernel... 2024-03-29T04:22:21.378 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-29T04:22:21.378 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-29T04:22:21.378 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T04:22:21.379 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T04:22:21.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:22:21.395 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2024-03-29T04:22:21.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:22:21.396 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-03-29T04:22:21.452 INFO:teuthology.orchestra.run.smithi152.stdout:5.14.0-361.el9.x86_64 2024-03-29T04:22:21.452 INFO:teuthology.task.kernel:Running kernel on smithi152: 5.14.0-361.el9.x86_64 2024-03-29T04:22:21.452 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum install -y kernel 2024-03-29T04:22:21.454 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-361.el9.x86_64 2024-03-29T04:22:21.454 INFO:teuthology.task.kernel:Running kernel on smithi142: 5.14.0-361.el9.x86_64 2024-03-29T04:22:21.454 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum install -y kernel 2024-03-29T04:22:23.504 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-29T04:22:23.590 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-29T04:22:24.946 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.0 MB 00:00 2024-03-29T04:22:25.249 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-03-29T04:22:26.215 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-03-29T04:22:29.183 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-03-29T04:22:30.294 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 1.6 MB/s | 8.0 MB 00:05 2024-03-29T04:22:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-29T04:22:33.950 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.1 MB 00:01 2024-03-29T04:22:35.293 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 16 kB 00:00 2024-03-29T04:22:36.504 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.1 MB 00:00 2024-03-29T04:22:37.327 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-29T04:22:37.921 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 16 kB 00:00 2024-03-29T04:22:39.284 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-29T04:22:42.377 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-29T04:22:44.314 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-29T04:22:44.609 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-29T04:22:44.610 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-29T04:22:44.709 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:22:44.720 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:22:44.720 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-03-29T04:22:44.720 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:22:44.720 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:22:44.721 INFO:teuthology.orchestra.run.smithi152.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-03-29T04:22:44.721 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-03-29T04:22:44.721 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-03-29T04:22:44.721 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-03-29T04:22:44.721 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-03-29T04:22:44.721 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:22:44.721 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:22:44.721 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:22:44.721 INFO:teuthology.orchestra.run.smithi152.stdout:Install 4 Packages 2024-03-29T04:22:44.721 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:22:44.722 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 76 M 2024-03-29T04:22:44.722 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 123 M 2024-03-29T04:22:44.722 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:22:44.895 INFO:teuthology.orchestra.run.smithi152.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-03-29T04:22:45.596 INFO:teuthology.orchestra.run.smithi152.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 17 MB/s | 15 MB 00:00 2024-03-29T04:22:46.063 INFO:teuthology.orchestra.run.smithi152.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 25 MB/s | 34 MB 00:01 2024-03-29T04:22:46.588 INFO:teuthology.orchestra.run.smithi152.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 16 MB/s | 28 MB 00:01 2024-03-29T04:22:46.591 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:22:46.591 INFO:teuthology.orchestra.run.smithi152.stdout:Total 41 MB/s | 76 MB 00:01 2024-03-29T04:22:46.591 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:22:46.637 INFO:teuthology.orchestra.run.smithi142.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-29T04:22:46.638 INFO:teuthology.orchestra.run.smithi142.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-29T04:22:46.736 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:22:46.749 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout:Install 4 Packages 2024-03-29T04:22:46.750 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:22:46.751 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 76 M 2024-03-29T04:22:46.751 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 123 M 2024-03-29T04:22:46.751 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:22:46.915 INFO:teuthology.orchestra.run.smithi142.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-03-29T04:22:46.961 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:22:46.962 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:22:47.448 INFO:teuthology.orchestra.run.smithi142.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-03-29T04:22:47.827 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:22:47.827 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:22:47.907 INFO:teuthology.orchestra.run.smithi142.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 29 MB/s | 34 MB 00:01 2024-03-29T04:22:48.507 INFO:teuthology.orchestra.run.smithi142.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 17 MB/s | 28 MB 00:01 2024-03-29T04:22:48.510 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:22:48.510 INFO:teuthology.orchestra.run.smithi142.stdout:Total 43 MB/s | 76 MB 00:01 2024-03-29T04:22:48.510 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:22:48.881 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:22:48.881 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:22:49.702 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:22:49.702 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:22:50.163 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:22:50.791 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-03-29T04:22:50.875 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T04:22:52.011 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:22:52.687 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-03-29T04:22:52.773 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T04:22:53.638 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T04:22:53.788 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T04:22:55.572 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T04:22:55.714 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T04:22:58.555 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T04:22:58.596 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:23:00.413 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T04:23:00.454 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:23:03.357 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:23:05.195 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:23:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:23:33.122 INFO:teuthology.orchestra.run.smithi152.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T04:23:33.122 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:23:33.165 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:23:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:23:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-03-29T04:23:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T04:23:33.542 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T04:23:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:23:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:23:33.942 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:23:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-5.14.0-434.el9.x86_64 2024-03-29T04:23:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-03-29T04:23:33.942 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-03-29T04:23:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-03-29T04:23:33.943 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:23:33.943 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:23:34.258 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T04:23:34.368 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T04:23:34.368 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T04:23:34.368 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-03-29T04:23:34.369 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-29T04:23:34.369 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-29T04:23:34.369 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-03-29T04:23:34.369 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi152.front.sepia.ceph.com, path=None, version=distro) 2024-03-29T04:23:34.369 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T04:23:34.449 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:23:34.449 INFO:teuthology.orchestra.run.smithi142.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T04:23:34.449 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:23:34.450 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T04:23:34.450 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T04:23:34.451 DEBUG:teuthology.orchestra.run.smithi152:> sudo rpm -qi grub2-tools 2024-03-29T04:23:34.485 INFO:teuthology.orchestra.run.smithi152.stdout:Name : grub2-tools 2024-03-29T04:23:34.485 INFO:teuthology.orchestra.run.smithi152.stdout:Epoch : 1 2024-03-29T04:23:34.485 INFO:teuthology.orchestra.run.smithi152.stdout:Version : 2.06 2024-03-29T04:23:34.485 INFO:teuthology.orchestra.run.smithi152.stdout:Release : 68.el9 2024-03-29T04:23:34.485 INFO:teuthology.orchestra.run.smithi152.stdout:Architecture: x86_64 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:Group : Unspecified 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:Size : 8264254 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:License : GPLv3+ 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:Packager : builder@centos.org 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:Vendor : CentOS 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:Summary : Support tools for GRUB. 2024-03-29T04:23:34.486 INFO:teuthology.orchestra.run.smithi152.stdout:Description : 2024-03-29T04:23:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:23:34.487 INFO:teuthology.orchestra.run.smithi152.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-29T04:23:34.487 INFO:teuthology.orchestra.run.smithi152.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-29T04:23:34.487 INFO:teuthology.orchestra.run.smithi152.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-29T04:23:34.487 INFO:teuthology.orchestra.run.smithi152.stdout:hardware devices. 2024-03-29T04:23:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:23:34.487 INFO:teuthology.orchestra.run.smithi152.stdout:This subpackage provides tools for support of all platforms. 2024-03-29T04:23:34.488 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-29T04:23:34.488 INFO:teuthology.task.kernel:Updating grub on smithi152 to boot 5.14.0-434.el9.x86_64 2024-03-29T04:23:34.488 DEBUG:teuthology.orchestra.run.smithi152:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-29T04:23:34.492 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:23:34.834 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:23:34.834 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-03-29T04:23:34.834 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T04:23:34.834 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T04:23:35.126 INFO:teuthology.orchestra.run.smithi152.stderr:Generating grub configuration file ... 2024-03-29T04:23:35.270 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:23:35.270 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:23:35.270 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:23:35.270 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-5.14.0-434.el9.x86_64 2024-03-29T04:23:35.270 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-03-29T04:23:35.270 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-03-29T04:23:35.270 INFO:teuthology.orchestra.run.smithi142.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-03-29T04:23:35.270 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:23:35.270 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:23:35.600 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T04:23:35.743 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T04:23:35.744 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T04:23:35.744 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-03-29T04:23:35.744 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-29T04:23:35.744 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-29T04:23:35.744 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-03-29T04:23:35.744 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi142.front.sepia.ceph.com, path=None, version=distro) 2024-03-29T04:23:35.744 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T04:23:35.824 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T04:23:35.824 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T04:23:35.824 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -qi grub2-tools 2024-03-29T04:23:35.867 INFO:teuthology.orchestra.run.smithi142.stdout:Name : grub2-tools 2024-03-29T04:23:35.867 INFO:teuthology.orchestra.run.smithi142.stdout:Epoch : 1 2024-03-29T04:23:35.867 INFO:teuthology.orchestra.run.smithi142.stdout:Version : 2.06 2024-03-29T04:23:35.867 INFO:teuthology.orchestra.run.smithi142.stdout:Release : 68.el9 2024-03-29T04:23:35.867 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: x86_64 2024-03-29T04:23:35.867 INFO:teuthology.orchestra.run.smithi142.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-29T04:23:35.867 INFO:teuthology.orchestra.run.smithi142.stdout:Group : Unspecified 2024-03-29T04:23:35.867 INFO:teuthology.orchestra.run.smithi142.stdout:Size : 8264254 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:License : GPLv3+ 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:Packager : builder@centos.org 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:Vendor : CentOS 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:Summary : Support tools for GRUB. 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:Description : 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-29T04:23:35.868 INFO:teuthology.orchestra.run.smithi142.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-29T04:23:35.869 INFO:teuthology.orchestra.run.smithi142.stdout:hardware devices. 2024-03-29T04:23:35.869 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:23:35.869 INFO:teuthology.orchestra.run.smithi142.stdout:This subpackage provides tools for support of all platforms. 2024-03-29T04:23:35.870 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-29T04:23:35.870 INFO:teuthology.task.kernel:Updating grub on smithi142 to boot 5.14.0-434.el9.x86_64 2024-03-29T04:23:35.870 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-29T04:23:36.496 INFO:teuthology.orchestra.run.smithi142.stderr:Generating grub configuration file ... 2024-03-29T04:23:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-29T04:23:36.668 INFO:teuthology.orchestra.run.smithi152.stderr:done 2024-03-29T04:23:36.670 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2024-03-29T04:23:36.685 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.uebrJZb4HT 2024-03-29T04:23:36.685 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uebrJZb4HT 2024-03-29T04:23:36.753 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.uebrJZb4HT 2024-03-29T04:23:36.887 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.uebrJZb4HT is 6KB 2024-03-29T04:23:36.936 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.uebrJZb4HT 2024-03-29T04:23:36.950 DEBUG:teuthology.orchestra.run.smithi152:> sudo /bin/ls /boot/loader/entries || true 2024-03-29T04:23:37.015 INFO:teuthology.orchestra.run.smithi152.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-29T04:23:37.015 INFO:teuthology.orchestra.run.smithi152.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-29T04:23:37.015 INFO:teuthology.orchestra.run.smithi152.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-29T04:23:37.015 INFO:teuthology.orchestra.run.smithi152.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-03-29T04:23:37.016 DEBUG:teuthology.orchestra.run.smithi152:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-03-29T04:23:37.165 DEBUG:teuthology.orchestra.run.smithi152:> sudo shutdown -r now 2024-03-29T04:23:38.025 INFO:teuthology.orchestra.run.smithi142.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-29T04:23:38.055 INFO:teuthology.orchestra.run.smithi142.stderr:done 2024-03-29T04:23:38.057 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-03-29T04:23:38.113 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.hrMyCYsyP4 2024-03-29T04:23:38.113 DEBUG:teuthology.orchestra.run.smithi142:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hrMyCYsyP4 2024-03-29T04:23:38.182 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.hrMyCYsyP4 2024-03-29T04:23:38.318 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.hrMyCYsyP4 is 6KB 2024-03-29T04:23:38.366 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.hrMyCYsyP4 2024-03-29T04:23:38.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo /bin/ls /boot/loader/entries || true 2024-03-29T04:23:38.445 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-29T04:23:38.445 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-29T04:23:38.445 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-29T04:23:38.445 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-03-29T04:23:38.447 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-03-29T04:23:38.600 DEBUG:teuthology.orchestra.run.smithi142:> sudo shutdown -r now 2024-03-29T04:24:07.198 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-29T04:24:07.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-29T04:24:07.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:24:08.604 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-29T04:24:08.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-03-29T04:24:08.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:24:27.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-03-29T04:24:36.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-03-29T04:24:36.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:24:39.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-03-29T04:24:41.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-03-29T04:24:50.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-29T04:24:50.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:24:51.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-03-29T04:24:51.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:25:09.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-03-29T04:25:21.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-29T04:25:21.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:25:23.026 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-03-29T04:25:23.945 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-03-29T04:25:23.945 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-434.el9.x86_64"... 2024-03-29T04:25:23.945 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-03-29T04:25:23.959 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-434.el9.x86_64 2024-03-29T04:25:23.959 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-03-29T04:25:23.959 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T04:25:23.960 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-29T04:25:24.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-03-29T04:25:24.960 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-29T04:25:24.961 DEBUG:teuthology.orchestra.run.smithi142:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-29T04:25:25.031 INFO:teuthology.orchestra.run.smithi142.stdout:ttyS1 2024-03-29T04:25:25.063 DEBUG:teuthology.parallel:result is None 2024-03-29T04:25:39.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-29T04:25:39.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:25:39.595 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-03-29T04:25:39.957 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-29T04:25:39.958 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-434.el9.x86_64"... 2024-03-29T04:25:39.958 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2024-03-29T04:25:39.974 INFO:teuthology.orchestra.run.smithi152.stdout:5.14.0-434.el9.x86_64 2024-03-29T04:25:39.974 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-03-29T04:25:39.975 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T04:25:39.975 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-29T04:25:40.975 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-29T04:25:40.976 DEBUG:teuthology.orchestra.run.smithi152:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-29T04:25:41.062 INFO:teuthology.orchestra.run.smithi152.stdout:ttyS1 2024-03-29T04:25:41.104 DEBUG:teuthology.parallel:result is None 2024-03-29T04:25:41.104 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-29T04:25:41.111 INFO:teuthology.task.internal:Creating test directory... 2024-03-29T04:25:41.111 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T04:25:41.114 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T04:25:41.132 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-29T04:25:41.143 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-29T04:25:41.156 INFO:teuthology.task.internal:Creating archive directory... 2024-03-29T04:25:41.156 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T04:25:41.172 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T04:25:41.227 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-29T04:25:41.238 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-29T04:25:41.238 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T04:25:41.241 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T04:25:41.276 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:25:41.277 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:25:41.290 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:25:41.292 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:25:41.294 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-29T04:25:41.304 INFO:teuthology.task.internal:Configuring sudo... 2024-03-29T04:25:41.304 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T04:25:41.337 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T04:25:41.367 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-29T04:25:41.380 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-29T04:25:41.380 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T04:25:41.407 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T04:25:41.423 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:25:41.504 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:25:41.556 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:25:41.640 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:25:41.681 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:25:41.681 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T04:25:41.750 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:25:41.794 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:25:41.855 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:25:41.898 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:25:41.942 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:25:41.942 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T04:25:42.011 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-03-29T04:25:42.013 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2024-03-29T04:25:42.056 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:25:42.106 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:25:42.385 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-29T04:25:42.393 INFO:teuthology.task.internal:Starting timer... 2024-03-29T04:25:42.393 INFO:teuthology.run_tasks:Running task pcp... 2024-03-29T04:25:42.406 INFO:teuthology.run_tasks:Running task selinux... 2024-03-29T04:25:42.416 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-29T04:25:42.416 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-03-29T04:25:42.478 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-03-29T04:25:42.480 DEBUG:teuthology.orchestra.run.smithi152:> sudo service auditd rotate 2024-03-29T04:25:42.537 INFO:teuthology.orchestra.run.smithi152.stdout:Rotating logs: 2024-03-29T04:25:42.539 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-29T04:25:42.539 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-03-29T04:25:42.570 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-03-29T04:25:42.570 DEBUG:teuthology.orchestra.run.smithi152:> /usr/sbin/getenforce 2024-03-29T04:25:42.601 INFO:teuthology.orchestra.run.smithi152.stdout:Permissive 2024-03-29T04:25:42.601 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi142.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi152.front.sepia.ceph.com': 'permissive'} 2024-03-29T04:25:42.601 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:25:42.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:25:42.641 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:25:42.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:25:42.669 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-29T04:25:42.669 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-03-29T04:25:42.730 DEBUG:teuthology.orchestra.run.smithi152:> sudo /usr/sbin/setenforce permissive 2024-03-29T04:25:42.761 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-29T04:25:42.772 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-29T04:25:42.788 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-29T04:25:42.789 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi142.front.sepia.ceph.com,smithi152.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-29T04:29:55.257 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi142.front.sepia.ceph.com'), Remote(name='ubuntu@smithi152.front.sepia.ceph.com')] 2024-03-29T04:29:55.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-03-29T04:29:55.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:29:55.348 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-03-29T04:29:55.425 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-03-29T04:29:55.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-29T04:29:55.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:29:55.518 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-03-29T04:29:55.597 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-29T04:29:55.597 INFO:teuthology.run_tasks:Running task clock... 2024-03-29T04:29:55.608 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-29T04:29:55.608 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T04:29:55.608 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:29:55.611 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T04:29:55.611 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:29:55.640 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-29T04:29:55.657 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-29T04:29:55.670 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-29T04:29:55.687 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-03-29T04:29:55.689 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-29T04:29:55.701 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-03-29T04:29:55.717 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: ntpd: command not found 2024-03-29T04:29:55.718 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-29T04:29:55.731 INFO:teuthology.orchestra.run.smithi152.stdout:506 Cannot talk to daemon 2024-03-29T04:29:55.734 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-29T04:29:55.748 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-29T04:29:55.764 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-29T04:29:55.789 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:29:55.793 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:29:55.793 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-03-29T04:29:55.793 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:29:55.793 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:29:55.793 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:29:55.793 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:29:55.819 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:29:55.822 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:29:55.822 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-03-29T04:29:55.823 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:29:55.823 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:29:55.823 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:29:55.823 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:29:55.823 INFO:teuthology.run_tasks:Running task install... 2024-03-29T04:29:55.835 DEBUG:teuthology.task.install:project ceph 2024-03-29T04:29:55.835 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8da23624bc17fe0b1f797227e72a0db0ce7821ba'}} 2024-03-29T04:29:55.835 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.1', 'flavor': 'default', 'sha1': '8da23624bc17fe0b1f797227e72a0db0ce7821ba'} 2024-03-29T04:29:55.836 INFO:teuthology.task.install:Using flavor: default 2024-03-29T04:29:55.842 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-29T04:29:55.842 INFO:teuthology.task.install:extra packages: [] 2024-03-29T04:29:55.842 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8da23624bc17fe0b1f797227e72a0db0ce7821ba', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-03-29T04:29:55.843 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-29T04:29:55.843 INFO:teuthology.packaging:ref: None 2024-03-29T04:29:55.843 INFO:teuthology.packaging:tag: v18.2.1 2024-03-29T04:29:55.843 INFO:teuthology.packaging:branch: None 2024-03-29T04:29:55.843 INFO:teuthology.packaging:sha1: 8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:29:57.999 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.1^{} -> 7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-29T04:29:57.999 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-29T04:29:58.002 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8da23624bc17fe0b1f797227e72a0db0ce7821ba', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-03-29T04:29:58.002 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-29T04:29:58.002 INFO:teuthology.packaging:ref: None 2024-03-29T04:29:58.002 INFO:teuthology.packaging:tag: v18.2.1 2024-03-29T04:29:58.002 INFO:teuthology.packaging:branch: None 2024-03-29T04:29:58.002 INFO:teuthology.packaging:sha1: 8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:29:58.003 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-29T04:29:58.230 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-03-29T04:29:58.230 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-03-29T04:29:58.236 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-03-29T04:29:58.236 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-03-29T04:29:58.274 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-29T04:29:58.274 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:29:58.274 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-29T04:29:58.282 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-29T04:29:58.282 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:29:58.282 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-29T04:29:58.308 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-29T04:29:58.308 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-29T04:29:58.309 INFO:teuthology.packaging:ref: None 2024-03-29T04:29:58.309 INFO:teuthology.packaging:tag: v18.2.1 2024-03-29T04:29:58.309 INFO:teuthology.packaging:branch: None 2024-03-29T04:29:58.309 INFO:teuthology.packaging:sha1: 8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:29:58.309 DEBUG:teuthology.orchestra.run.smithi152:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-29T04:29:58.315 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-29T04:29:58.315 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-29T04:29:58.315 INFO:teuthology.packaging:ref: None 2024-03-29T04:29:58.315 INFO:teuthology.packaging:tag: v18.2.1 2024-03-29T04:29:58.315 INFO:teuthology.packaging:branch: None 2024-03-29T04:29:58.316 INFO:teuthology.packaging:sha1: 8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:29:58.316 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-29T04:29:58.379 DEBUG:teuthology.orchestra.run.smithi152:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-29T04:29:58.385 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-29T04:29:58.463 DEBUG:teuthology.orchestra.run.smithi152:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-29T04:29:58.468 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-29T04:29:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:check_obsoletes = 1 2024-03-29T04:29:58.532 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum clean all 2024-03-29T04:29:58.536 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-03-29T04:29:58.537 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-03-29T04:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout:74 files removed 2024-03-29T04:29:58.781 INFO:teuthology.orchestra.run.smithi142.stdout:74 files removed 2024-03-29T04:29:58.796 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw 2024-03-29T04:29:58.808 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw 2024-03-29T04:29:59.226 INFO:teuthology.orchestra.run.smithi152.stdout:ceph packages for x86_64 574 kB/s | 76 kB 00:00 2024-03-29T04:29:59.243 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 557 kB/s | 76 kB 00:00 2024-03-29T04:29:59.427 INFO:teuthology.orchestra.run.smithi152.stdout:ceph noarch packages 62 kB/s | 9.4 kB 00:00 2024-03-29T04:29:59.427 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 69 kB/s | 9.4 kB 00:00 2024-03-29T04:29:59.594 INFO:teuthology.orchestra.run.smithi152.stdout:ceph source packages 17 kB/s | 2.2 kB 00:00 2024-03-29T04:29:59.603 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 16 kB/s | 2.2 kB 00:00 2024-03-29T04:29:59.988 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS-9 - AppStream 19 MB/s | 6.7 MB 00:00 2024-03-29T04:30:00.322 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - AppStream 9.8 MB/s | 6.7 MB 00:00 2024-03-29T04:30:01.506 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2024-03-29T04:30:01.714 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.0 MB 00:00 2024-03-29T04:30:02.313 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-29T04:30:02.896 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.0 MB 00:00 2024-03-29T04:30:07.406 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 19 MB 00:03 2024-03-29T04:30:13.922 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.1 MB 00:03 2024-03-29T04:30:15.373 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-03-29T04:30:16.952 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-29T04:30:21.833 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-03-29T04:30:24.469 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:30:24.472 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:30:24.472 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-03-29T04:30:24.472 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:30:24.472 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:30:24.472 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-03-29T04:30:24.472 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-03-29T04:30:24.472 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-03-29T04:30:24.473 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-03-29T04:30:24.473 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-03-29T04:30:24.473 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-29T04:30:24.473 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-03-29T04:30:24.473 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-03-29T04:30:24.473 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-03-29T04:30:24.473 INFO:teuthology.orchestra.run.smithi142.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-29T04:30:24.473 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-29T04:30:24.473 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-29T04:30:24.473 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-29T04:30:24.473 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-03-29T04:30:24.473 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-29T04:30:24.474 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-03-29T04:30:24.475 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-03-29T04:30:24.475 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-03-29T04:30:24.475 INFO:teuthology.orchestra.run.smithi142.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-29T04:30:24.475 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-29T04:30:24.475 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:24.475 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:30:24.475 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:30:24.475 INFO:teuthology.orchestra.run.smithi142.stdout:Install 28 Packages 2024-03-29T04:30:24.475 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 2 Packages 2024-03-29T04:30:24.475 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:24.476 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 53 M 2024-03-29T04:30:24.476 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:30:24.709 INFO:teuthology.orchestra.run.smithi142.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 28 MB/s | 5.2 MB 00:00 2024-03-29T04:30:24.734 INFO:teuthology.orchestra.run.smithi142.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 964 kB/s | 24 kB 00:00 2024-03-29T04:30:24.776 INFO:teuthology.orchestra.run.smithi142.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 16 MB/s | 658 kB 00:00 2024-03-29T04:30:24.810 INFO:teuthology.orchestra.run.smithi142.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-03-29T04:30:24.918 INFO:teuthology.orchestra.run.smithi142.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 20 MB/s | 7.7 MB 00:00 2024-03-29T04:30:24.943 INFO:teuthology.orchestra.run.smithi142.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-03-29T04:30:24.968 INFO:teuthology.orchestra.run.smithi142.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.9 MB/s | 124 kB 00:00 2024-03-29T04:30:24.994 INFO:teuthology.orchestra.run.smithi142.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 6.3 MB/s | 161 kB 00:00 2024-03-29T04:30:25.019 INFO:teuthology.orchestra.run.smithi142.stdout:(9/30): python3-rados-18.2.1-0.el9.x86_64.rpm 13 MB/s | 321 kB 00:00 2024-03-29T04:30:25.044 INFO:teuthology.orchestra.run.smithi142.stdout:(10/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 12 MB/s | 297 kB 00:00 2024-03-29T04:30:25.070 INFO:teuthology.orchestra.run.smithi142.stdout:(11/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 3.9 MB/s | 99 kB 00:00 2024-03-29T04:30:25.138 INFO:teuthology.orchestra.run.smithi142.stdout:(12/30): librgw2-18.2.1-0.el9.x86_64.rpm 14 MB/s | 4.5 MB 00:00 2024-03-29T04:30:25.278 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 884 kB/s | 19 MB 00:21 2024-03-29T04:30:25.404 INFO:teuthology.orchestra.run.smithi142.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 21 MB/s | 18 MB 00:00 2024-03-29T04:30:25.429 INFO:teuthology.orchestra.run.smithi142.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 295 kB/s | 106 kB 00:00 2024-03-29T04:30:25.445 INFO:teuthology.orchestra.run.smithi142.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-29T04:30:25.470 INFO:teuthology.orchestra.run.smithi142.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 477 kB/s | 159 kB 00:00 2024-03-29T04:30:25.521 INFO:teuthology.orchestra.run.smithi142.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 661 kB 00:00 2024-03-29T04:30:25.549 INFO:teuthology.orchestra.run.smithi142.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.7 MB/s | 282 kB 00:00 2024-03-29T04:30:25.571 INFO:teuthology.orchestra.run.smithi142.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 411 kB/s | 41 kB 00:00 2024-03-29T04:30:25.763 INFO:teuthology.orchestra.run.smithi142.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 459 kB/s | 111 kB 00:00 2024-03-29T04:30:25.830 INFO:teuthology.orchestra.run.smithi142.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-03-29T04:30:25.855 INFO:teuthology.orchestra.run.smithi142.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.1 MB/s | 309 kB 00:00 2024-03-29T04:30:25.897 INFO:teuthology.orchestra.run.smithi142.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 726 kB/s | 49 kB 00:00 2024-03-29T04:30:25.922 INFO:teuthology.orchestra.run.smithi142.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-29T04:30:26.040 INFO:teuthology.orchestra.run.smithi142.stdout:(25/30): libarrow-9.0.0-10.el9.x86_64.rpm 9.4 MB/s | 4.4 MB 00:00 2024-03-29T04:30:26.065 INFO:teuthology.orchestra.run.smithi142.stdout:(26/30): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-29T04:30:26.107 INFO:teuthology.orchestra.run.smithi142.stdout:(27/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-03-29T04:30:26.157 INFO:teuthology.orchestra.run.smithi142.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-03-29T04:30:26.224 INFO:teuthology.orchestra.run.smithi142.stdout:(29/30): librados2-18.2.1-0.el9.x86_64.rpm 21 MB/s | 3.3 MB 00:00 2024-03-29T04:30:26.274 INFO:teuthology.orchestra.run.smithi142.stdout:(30/30): librbd1-18.2.1-0.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2024-03-29T04:30:26.278 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:30:26.279 INFO:teuthology.orchestra.run.smithi142.stdout:Total 29 MB/s | 53 MB 00:01 2024-03-29T04:30:26.279 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:30:26.315 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:30:26.315 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:30:26.767 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:30:26.768 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:30:28.137 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:30:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-03-29T04:30:28.245 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-03-29T04:30:28.296 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-03-29T04:30:28.665 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-03-29T04:30:28.703 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-03-29T04:30:28.794 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-03-29T04:30:28.874 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-03-29T04:30:28.921 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-03-29T04:30:28.983 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-03-29T04:30:29.012 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-03-29T04:30:29.056 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-03-29T04:30:29.121 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-03-29T04:30:29.148 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-03-29T04:30:29.196 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-03-29T04:30:29.303 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-03-29T04:30:29.339 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-03-29T04:30:29.379 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-03-29T04:30:29.411 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-03-29T04:30:29.438 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-03-29T04:30:29.474 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-03-29T04:30:29.507 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-03-29T04:30:29.808 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-03-29T04:30:29.837 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-03-29T04:30:29.910 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-03-29T04:30:29.941 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-03-29T04:30:29.977 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-03-29T04:30:30.220 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-03-29T04:30:30.378 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-03-29T04:30:30.405 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-03-29T04:30:30.479 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-03-29T04:30:30.544 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-03-29T04:30:30.574 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-03-29T04:30:31.788 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 16 kB 00:00 2024-03-29T04:30:32.663 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-29T04:30:32.698 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-29T04:30:32.881 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-03-29T04:30:33.247 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-29T04:30:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-03-29T04:30:33.362 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-03-29T04:30:33.362 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-29T04:30:33.363 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-29T04:30:33.363 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:33.402 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-03-29T04:30:37.832 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-03-29T04:30:40.450 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:30:40.459 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:30:40.459 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-03-29T04:30:40.459 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrading: 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-29T04:30:40.460 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-03-29T04:30:40.461 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-03-29T04:30:40.462 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-29T04:30:40.462 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-03-29T04:30:40.462 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-03-29T04:30:40.462 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-03-29T04:30:40.462 INFO:teuthology.orchestra.run.smithi152.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-29T04:30:40.462 INFO:teuthology.orchestra.run.smithi152.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-29T04:30:40.462 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:30:40.462 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:30:40.462 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:30:40.462 INFO:teuthology.orchestra.run.smithi152.stdout:Install 28 Packages 2024-03-29T04:30:40.462 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrade 2 Packages 2024-03-29T04:30:40.462 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:30:40.463 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 53 M 2024-03-29T04:30:40.463 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:30:41.025 INFO:teuthology.orchestra.run.smithi152.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 26 MB/s | 5.2 MB 00:00 2024-03-29T04:30:41.050 INFO:teuthology.orchestra.run.smithi152.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 965 kB/s | 24 kB 00:00 2024-03-29T04:30:41.092 INFO:teuthology.orchestra.run.smithi152.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 15 MB/s | 658 kB 00:00 2024-03-29T04:30:41.125 INFO:teuthology.orchestra.run.smithi152.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-03-29T04:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 19 MB/s | 7.7 MB 00:00 2024-03-29T04:30:41.259 INFO:teuthology.orchestra.run.smithi152.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-03-29T04:30:41.284 INFO:teuthology.orchestra.run.smithi152.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.9 MB/s | 124 kB 00:00 2024-03-29T04:30:41.318 INFO:teuthology.orchestra.run.smithi152.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 4.7 MB/s | 161 kB 00:00 2024-03-29T04:30:41.352 INFO:teuthology.orchestra.run.smithi152.stdout:(9/30): python3-rados-18.2.1-0.el9.x86_64.rpm 9.4 MB/s | 321 kB 00:00 2024-03-29T04:30:41.369 INFO:teuthology.orchestra.run.smithi152.stdout:(10/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 17 MB/s | 297 kB 00:00 2024-03-29T04:30:41.394 INFO:teuthology.orchestra.run.smithi152.stdout:(11/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 3.9 MB/s | 99 kB 00:00 2024-03-29T04:30:41.453 INFO:teuthology.orchestra.run.smithi152.stdout:(12/30): librgw2-18.2.1-0.el9.x86_64.rpm 14 MB/s | 4.5 MB 00:00 2024-03-29T04:30:41.728 INFO:teuthology.orchestra.run.smithi152.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 20 MB/s | 18 MB 00:00 2024-03-29T04:30:41.745 INFO:teuthology.orchestra.run.smithi152.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 302 kB/s | 106 kB 00:00 2024-03-29T04:30:41.762 INFO:teuthology.orchestra.run.smithi152.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-29T04:30:41.821 INFO:teuthology.orchestra.run.smithi152.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 462 kB/s | 159 kB 00:00 2024-03-29T04:30:41.887 INFO:teuthology.orchestra.run.smithi152.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 661 kB 00:00 2024-03-29T04:30:41.913 INFO:teuthology.orchestra.run.smithi152.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 282 kB 00:00 2024-03-29T04:30:41.938 INFO:teuthology.orchestra.run.smithi152.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 352 kB/s | 41 kB 00:00 2024-03-29T04:30:42.255 INFO:teuthology.orchestra.run.smithi152.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 302 kB/s | 111 kB 00:00 2024-03-29T04:30:42.330 INFO:teuthology.orchestra.run.smithi152.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-03-29T04:30:42.364 INFO:teuthology.orchestra.run.smithi152.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 726 kB/s | 309 kB 00:00 2024-03-29T04:30:42.414 INFO:teuthology.orchestra.run.smithi152.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 578 kB/s | 49 kB 00:00 2024-03-29T04:30:42.440 INFO:teuthology.orchestra.run.smithi152.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 890 kB/s | 67 kB 00:00 2024-03-29T04:30:42.523 INFO:teuthology.orchestra.run.smithi152.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-03-29T04:30:42.607 INFO:teuthology.orchestra.run.smithi152.stdout:(26/30): libarrow-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-03-29T04:30:42.641 INFO:teuthology.orchestra.run.smithi152.stdout:(27/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-03-29T04:30:42.716 INFO:teuthology.orchestra.run.smithi152.stdout:(28/30): librados2-18.2.1-0.el9.x86_64.rpm 30 MB/s | 3.3 MB 00:00 2024-03-29T04:30:42.766 INFO:teuthology.orchestra.run.smithi152.stdout:(29/30): librbd1-18.2.1-0.el9.x86_64.rpm 24 MB/s | 3.0 MB 00:00 2024-03-29T04:30:42.800 INFO:teuthology.orchestra.run.smithi152.stdout:(30/30): thrift-0.15.0-2.el9.x86_64.rpm 5.7 MB/s | 1.6 MB 00:00 2024-03-29T04:30:42.806 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:30:42.806 INFO:teuthology.orchestra.run.smithi152.stdout:Total 23 MB/s | 53 MB 00:02 2024-03-29T04:30:42.806 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:30:42.837 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:30:42.837 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:30:43.024 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-03-29T04:30:43.024 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-03-29T04:30:43.024 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-03-29T04:30:43.024 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-03-29T04:30:43.024 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-03-29T04:30:43.025 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-03-29T04:30:43.025 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2024-03-29T04:30:43.025 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-03-29T04:30:43.025 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:43.278 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:30:43.278 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:30:43.786 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-03-29T04:30:43.810 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-03-29T04:30:43.810 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:30:43.810 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-29T04:30:43.811 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:30:43.811 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:30:43.811 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:43.812 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-29T04:30:43.880 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-29T04:30:43.880 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-29T04:30:44.664 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:30:44.744 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-03-29T04:30:44.779 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-03-29T04:30:44.830 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-03-29T04:30:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-29T04:30:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-03-29T04:30:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-03-29T04:30:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-03-29T04:30:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-03-29T04:30:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-03-29T04:30:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-03-29T04:30:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-03-29T04:30:45.020 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-03-29T04:30:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-03-29T04:30:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-03-29T04:30:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-03-29T04:30:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-03-29T04:30:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-03-29T04:30:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-03-29T04:30:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-03-29T04:30:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-03-29T04:30:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-03-29T04:30:45.022 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-03-29T04:30:45.149 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-03-29T04:30:45.212 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-03-29T04:30:45.287 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-03-29T04:30:45.369 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-03-29T04:30:45.406 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-03-29T04:30:45.452 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-03-29T04:30:45.480 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-03-29T04:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-03-29T04:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-03-29T04:30:45.616 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-03-29T04:30:45.665 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-03-29T04:30:45.772 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-03-29T04:30:45.814 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-03-29T04:30:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-03-29T04:30:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-03-29T04:30:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-03-29T04:30:45.947 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-03-29T04:30:45.947 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:45.947 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-03-29T04:30:45.947 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.947 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:30:45.947 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-29T04:30:45.947 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.947 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.947 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.947 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-29T04:30:45.948 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-29T04:30:45.949 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.949 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.949 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.949 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-29T04:30:45.949 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.949 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.949 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-03-29T04:30:45.949 INFO:teuthology.orchestra.run.smithi142.stdout: re2-1:20211101-3.el9.x86_64 2024-03-29T04:30:45.949 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-29T04:30:45.949 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:45.949 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:30:45.952 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-03-29T04:30:45.984 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-03-29T04:30:46.286 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-03-29T04:30:46.314 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-03-29T04:30:46.335 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-test 2024-03-29T04:30:46.405 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-03-29T04:30:46.444 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-03-29T04:30:46.489 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-03-29T04:30:46.893 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-03-29T04:30:46.904 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:25 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:30:46.925 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-03-29T04:30:46.996 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-03-29T04:30:47.058 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:30:47.058 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:30:47.059 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:30:47.060 INFO:teuthology.orchestra.run.smithi142.stdout:Install 6 Packages 2024-03-29T04:30:47.060 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:47.060 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 41 M 2024-03-29T04:30:47.060 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 167 M 2024-03-29T04:30:47.060 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:30:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-03-29T04:30:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-03-29T04:30:47.227 INFO:teuthology.orchestra.run.smithi142.stdout:(1/6): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.2 MB/s | 165 kB 00:00 2024-03-29T04:30:47.294 INFO:teuthology.orchestra.run.smithi142.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.1 MB/s | 217 kB 00:00 2024-03-29T04:30:47.319 INFO:teuthology.orchestra.run.smithi142.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 300 kB 00:00 2024-03-29T04:30:47.353 INFO:teuthology.orchestra.run.smithi142.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-03-29T04:30:47.436 INFO:teuthology.orchestra.run.smithi142.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2024-03-29T04:30:47.795 INFO:teuthology.orchestra.run.smithi142.stdout:(6/6): ceph-test-18.2.1-0.el9.x86_64.rpm 57 MB/s | 40 MB 00:00 2024-03-29T04:30:47.798 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:30:47.798 INFO:teuthology.orchestra.run.smithi142.stdout:Total 55 MB/s | 41 MB 00:00 2024-03-29T04:30:47.813 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:30:47.834 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:30:47.834 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:30:48.185 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:30:48.185 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:30:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:30:48.796 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-29T04:30:48.830 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-29T04:30:48.862 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-29T04:30:48.905 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-29T04:30:48.932 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-03-29T04:30:49.061 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-29T04:30:49.091 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-29T04:30:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-29T04:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-03-29T04:30:49.810 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-03-29T04:30:49.810 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-29T04:30:49.810 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-29T04:30:49.810 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:30:49.846 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-03-29T04:30:52.673 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-03-29T04:30:52.757 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-03-29T04:30:54.803 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-03-29T04:30:54.803 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/6 2024-03-29T04:30:54.803 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/6 2024-03-29T04:30:54.803 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-29T04:30:54.804 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-29T04:30:54.804 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-29T04:30:55.206 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-29T04:30:55.207 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:55.207 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:30:55.207 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:18.2.1-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-03-29T04:30:55.207 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:18.2.1-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-03-29T04:30:55.207 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-03-29T04:30:55.207 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:55.207 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:30:55.492 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph 2024-03-29T04:30:56.065 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:35 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:30:56.226 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:30:56.229 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:30:56.229 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-03-29T04:30:56.229 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:30:56.229 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:30:56.229 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-03-29T04:30:56.229 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-03-29T04:30:56.229 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-03-29T04:30:56.229 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-03-29T04:30:56.229 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-03-29T04:30:56.229 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-03-29T04:30:56.229 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-29T04:30:56.230 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-29T04:30:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:56.232 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:30:56.232 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:30:56.232 INFO:teuthology.orchestra.run.smithi142.stdout:Install 33 Packages 2024-03-29T04:30:56.232 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:30:56.232 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 30 M 2024-03-29T04:30:56.232 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 110 M 2024-03-29T04:30:56.232 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:30:56.830 INFO:teuthology.orchestra.run.smithi142.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 54 kB/s | 6.4 kB 00:00 2024-03-29T04:30:56.872 INFO:teuthology.orchestra.run.smithi142.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-03-29T04:30:56.931 INFO:teuthology.orchestra.run.smithi142.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 9.6 MB/s | 2.1 MB 00:00 2024-03-29T04:30:56.956 INFO:teuthology.orchestra.run.smithi142.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 9.5 MB/s | 242 kB 00:00 2024-03-29T04:30:57.057 INFO:teuthology.orchestra.run.smithi142.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-03-29T04:30:57.098 INFO:teuthology.orchestra.run.smithi142.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 787 kB/s | 32 kB 00:00 2024-03-29T04:30:57.124 INFO:teuthology.orchestra.run.smithi142.stdout:(7/33): python3-mako-1.1.4-6.el9.noarch.rpm 862 kB/s | 144 kB 00:00 2024-03-29T04:30:57.149 INFO:teuthology.orchestra.run.smithi142.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 946 kB/s | 47 kB 00:00 2024-03-29T04:30:57.174 INFO:teuthology.orchestra.run.smithi142.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 770 kB/s | 39 kB 00:00 2024-03-29T04:30:57.199 INFO:teuthology.orchestra.run.smithi142.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-29T04:30:57.242 INFO:teuthology.orchestra.run.smithi142.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-03-29T04:30:57.267 INFO:teuthology.orchestra.run.smithi142.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-29T04:30:57.292 INFO:teuthology.orchestra.run.smithi142.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-03-29T04:30:57.317 INFO:teuthology.orchestra.run.smithi142.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-03-29T04:30:57.501 INFO:teuthology.orchestra.run.smithi142.stdout:(15/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 28 MB/s | 18 MB 00:00 2024-03-29T04:30:57.693 INFO:teuthology.orchestra.run.smithi142.stdout:(16/33): python3-requests-2.25.1-8.el9.noarch.r 312 kB/s | 125 kB 00:00 2024-03-29T04:30:57.718 INFO:teuthology.orchestra.run.smithi142.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 108 kB/s | 43 kB 00:00 2024-03-29T04:30:57.743 INFO:teuthology.orchestra.run.smithi142.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 791 kB/s | 172 kB 00:00 2024-03-29T04:30:57.785 INFO:teuthology.orchestra.run.smithi142.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 160 kB/s | 11 kB 00:00 2024-03-29T04:30:57.810 INFO:teuthology.orchestra.run.smithi142.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 3.0 MB/s | 358 kB 00:00 2024-03-29T04:30:57.827 INFO:teuthology.orchestra.run.smithi142.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 211 kB/s | 18 kB 00:00 2024-03-29T04:30:57.844 INFO:teuthology.orchestra.run.smithi142.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 396 kB/s | 23 kB 00:00 2024-03-29T04:30:57.869 INFO:teuthology.orchestra.run.smithi142.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 465 kB/s | 19 kB 00:00 2024-03-29T04:30:57.886 INFO:teuthology.orchestra.run.smithi142.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 335 kB/s | 20 kB 00:00 2024-03-29T04:30:57.903 INFO:teuthology.orchestra.run.smithi142.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 786 kB/s | 46 kB 00:00 2024-03-29T04:30:57.928 INFO:teuthology.orchestra.run.smithi142.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-03-29T04:30:57.995 INFO:teuthology.orchestra.run.smithi142.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-03-29T04:30:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 140 kB/s | 16 kB 00:00 2024-03-29T04:30:58.046 INFO:teuthology.orchestra.run.smithi142.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 765 kB/s | 90 kB 00:00 2024-03-29T04:30:58.071 INFO:teuthology.orchestra.run.smithi142.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-03-29T04:30:58.105 INFO:teuthology.orchestra.run.smithi142.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-29T04:30:58.130 INFO:teuthology.orchestra.run.smithi142.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 344 kB/s | 20 kB 00:00 2024-03-29T04:30:58.155 INFO:teuthology.orchestra.run.smithi142.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-03-29T04:30:58.161 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:30:58.161 INFO:teuthology.orchestra.run.smithi142.stdout:Total 15 MB/s | 30 MB 00:01 2024-03-29T04:30:58.173 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:30:58.214 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:30:58.214 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:30:58.581 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:30:58.582 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:30:59.529 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-03-29T04:30:59.529 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /sys 2024-03-29T04:30:59.529 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /proc 2024-03-29T04:30:59.530 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /mnt 2024-03-29T04:30:59.530 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /var/tmp 2024-03-29T04:30:59.530 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /home 2024-03-29T04:30:59.530 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /root 2024-03-29T04:30:59.530 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /tmp 2024-03-29T04:30:59.530 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:30:59.856 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:30:59.924 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-29T04:30:59.956 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-29T04:30:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-29T04:31:00.081 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-29T04:31:00.177 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-29T04:31:00.217 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-29T04:31:00.252 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-03-29T04:31:00.277 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-03-29T04:31:00.277 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:31:00.277 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-29T04:31:00.278 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:31:00.278 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:31:00.278 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:00.279 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-29T04:31:00.288 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-29T04:31:00.343 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-29T04:31:00.363 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-29T04:31:00.363 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-29T04:31:00.390 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-29T04:31:00.438 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-29T04:31:00.496 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-29T04:31:00.688 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-29T04:31:00.735 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-29T04:31:00.771 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-29T04:31:00.808 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-29T04:31:00.838 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-29T04:31:00.915 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-29T04:31:00.949 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-29T04:31:00.982 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-29T04:31:01.023 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-29T04:31:01.057 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-29T04:31:01.092 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-29T04:31:01.181 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-29T04:31:01.223 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-29T04:31:01.294 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-29T04:31:01.434 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-29T04:31:01.461 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-03-29T04:31:01.462 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-03-29T04:31:01.463 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-03-29T04:31:01.463 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-03-29T04:31:01.463 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-03-29T04:31:01.463 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-03-29T04:31:01.463 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-03-29T04:31:01.463 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-03-29T04:31:01.463 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-03-29T04:31:01.463 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-03-29T04:31:01.463 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-03-29T04:31:01.463 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-03-29T04:31:01.464 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-03-29T04:31:01.464 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-03-29T04:31:01.464 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-03-29T04:31:01.464 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-03-29T04:31:01.464 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-03-29T04:31:01.464 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-03-29T04:31:01.464 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-03-29T04:31:01.464 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-03-29T04:31:01.556 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-29T04:31:01.712 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-03-29T04:31:01.744 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-03-29T04:31:01.769 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-03-29T04:31:01.769 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:31:01.769 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-29T04:31:01.770 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:31:01.770 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:31:01.770 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:02.470 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-03-29T04:31:02.470 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:02.470 INFO:teuthology.orchestra.run.smithi152.stdout:Upgraded: 2024-03-29T04:31:02.470 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.470 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:31:02.470 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-29T04:31:02.470 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.470 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-29T04:31:02.471 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: re2-1:20211101-3.el9.x86_64 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-29T04:31:02.472 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:02.473 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:31:02.839 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-test 2024-03-29T04:31:03.338 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-03-29T04:31:03.366 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-03-29T04:31:03.366 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:31:03.367 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-29T04:31:03.367 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:31:03.367 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:31:03.367 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:03.412 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:26 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:31:03.574 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:31:03.574 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:03.574 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:03.575 INFO:teuthology.orchestra.run.smithi152.stdout:Install 6 Packages 2024-03-29T04:31:03.576 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:03.576 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 41 M 2024-03-29T04:31:03.576 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 167 M 2024-03-29T04:31:03.576 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:31:03.814 INFO:teuthology.orchestra.run.smithi152.stdout:(1/6): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.3 MB/s | 165 kB 00:00 2024-03-29T04:31:03.822 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-03-29T04:31:03.846 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-03-29T04:31:03.846 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:31:03.846 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-29T04:31:03.846 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:31:03.846 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:31:03.846 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:03.898 INFO:teuthology.orchestra.run.smithi152.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.0 MB/s | 217 kB 00:00 2024-03-29T04:31:03.939 INFO:teuthology.orchestra.run.smithi152.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.3 MB/s | 300 kB 00:00 2024-03-29T04:31:03.957 INFO:teuthology.orchestra.run.smithi152.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-03-29T04:31:04.048 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-03-29T04:31:04.071 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-03-29T04:31:04.071 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:31:04.072 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-29T04:31:04.072 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:31:04.072 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:31:04.072 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:04.137 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-03-29T04:31:04.398 INFO:teuthology.orchestra.run.smithi152.stdout:(5/6): ceph-test-18.2.1-0.el9.x86_64.rpm 56 MB/s | 40 MB 00:00 2024-03-29T04:31:04.690 INFO:teuthology.orchestra.run.smithi152.stdout:(6/6): jq-1.6-16.el9.x86_64.rpm 249 kB/s | 187 kB 00:00 2024-03-29T04:31:04.693 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:04.694 INFO:teuthology.orchestra.run.smithi152.stdout:Total 36 MB/s | 41 MB 00:01 2024-03-29T04:31:04.709 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:31:04.730 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:31:04.730 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:31:05.083 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:31:05.083 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:31:05.607 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:31:05.684 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-29T04:31:05.734 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-29T04:31:05.765 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-29T04:31:05.807 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-29T04:31:05.835 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-03-29T04:31:05.911 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-03-29T04:31:05.911 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-29T04:31:05.912 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-29T04:31:05.913 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-29T04:31:05.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-29T04:31:05.914 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-29T04:31:06.850 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-29T04:31:06.850 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:06.850 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:06.850 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:06.850 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:06.850 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:06.850 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-03-29T04:31:06.850 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-29T04:31:06.851 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-29T04:31:06.852 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-29T04:31:06.853 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:06.853 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:07.144 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-base 2024-03-29T04:31:07.708 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:46 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:07.808 INFO:teuthology.orchestra.run.smithi142.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:31:07.860 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:07.861 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-03-29T04:31:07.861 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:07.922 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install cephadm 2024-03-29T04:31:08.488 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:47 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:08.638 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:08.639 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:08.639 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-03-29T04:31:08.639 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:08.639 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:31:08.640 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-03-29T04:31:08.640 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:08.640 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:31:08.640 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:08.640 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-03-29T04:31:08.640 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:08.640 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 221 k 2024-03-29T04:31:08.640 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 213 k 2024-03-29T04:31:08.640 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:31:08.714 INFO:teuthology.orchestra.run.smithi142.stdout:cephadm-18.2.1-0.el9.noarch.rpm 2.9 MB/s | 221 kB 00:00 2024-03-29T04:31:08.715 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:08.715 INFO:teuthology.orchestra.run.smithi142.stdout:Total 2.9 MB/s | 221 kB 00:00 2024-03-29T04:31:08.715 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:31:08.720 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:31:08.720 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:31:08.723 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:31:08.723 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:31:08.803 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:31:09.242 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-29T04:31:09.327 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-29T04:31:09.562 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-03-29T04:31:09.634 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-03-29T04:31:09.684 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-29T04:31:09.963 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-29T04:31:09.963 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:09.963 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:09.963 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-03-29T04:31:09.963 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:09.963 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:10.186 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-immutable-object-cache 2024-03-29T04:31:10.759 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:49 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:10.916 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:10.916 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:10.916 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repo Size 2024-03-29T04:31:10.916 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:10.917 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:31:10.917 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-03-29T04:31:10.917 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:10.917 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:31:10.917 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:10.917 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-03-29T04:31:10.917 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:10.917 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 142 k 2024-03-29T04:31:10.917 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 395 k 2024-03-29T04:31:10.917 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:31:10.978 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 2.3 MB/s | 142 kB 00:00 2024-03-29T04:31:10.979 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:10.979 INFO:teuthology.orchestra.run.smithi142.stdout:Total 2.2 MB/s | 142 kB 00:00 2024-03-29T04:31:10.979 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:31:10.986 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:31:10.986 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:31:11.038 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:31:11.038 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:31:11.187 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:31:11.254 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:11.279 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:11.279 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:31:11.279 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-29T04:31:11.279 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-03-29T04:31:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/6 2024-03-29T04:31:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/6 2024-03-29T04:31:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-29T04:31:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-29T04:31:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-29T04:31:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-29T04:31:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:12.093 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:31:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-2:18.2.1-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-03-29T04:31:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite-2:18.2.1-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-03-29T04:31:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-03-29T04:31:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:12.094 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:31:12.222 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:12.222 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:12.222 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:12.222 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:12.222 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:12.222 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:12.358 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph 2024-03-29T04:31:12.456 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr 2024-03-29T04:31:12.927 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:35 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:31:13.042 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:52 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:13.084 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:31:13.086 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:13.086 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-03-29T04:31:13.086 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:13.086 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:31:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-03-29T04:31:13.087 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-03-29T04:31:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-03-29T04:31:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-03-29T04:31:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-03-29T04:31:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-03-29T04:31:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-03-29T04:31:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-29T04:31:13.087 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-29T04:31:13.088 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-29T04:31:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-29T04:31:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-29T04:31:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-29T04:31:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-29T04:31:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-29T04:31:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-29T04:31:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-29T04:31:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-29T04:31:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-29T04:31:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-29T04:31:13.089 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-29T04:31:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-29T04:31:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:13.090 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:31:13.090 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:13.090 INFO:teuthology.orchestra.run.smithi152.stdout:Install 33 Packages 2024-03-29T04:31:13.090 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:13.091 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 30 M 2024-03-29T04:31:13.091 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 110 M 2024-03-29T04:31:13.091 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:31:13.141 INFO:teuthology.orchestra.run.smithi142.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:31:13.193 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:13.194 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-03-29T04:31:13.195 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:13.261 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr-dashboard 2024-03-29T04:31:13.483 INFO:teuthology.orchestra.run.smithi152.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 57 kB/s | 6.4 kB 00:00 2024-03-29T04:31:13.542 INFO:teuthology.orchestra.run.smithi152.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 8.5 MB/s | 1.4 MB 00:00 2024-03-29T04:31:13.592 INFO:teuthology.orchestra.run.smithi152.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 9.5 MB/s | 2.1 MB 00:00 2024-03-29T04:31:13.626 INFO:teuthology.orchestra.run.smithi152.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 7.1 MB/s | 242 kB 00:00 2024-03-29T04:31:13.734 INFO:teuthology.orchestra.run.smithi152.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-03-29T04:31:13.801 INFO:teuthology.orchestra.run.smithi152.stdout:(6/33): python3-mako-1.1.4-6.el9.noarch.rpm 823 kB/s | 144 kB 00:00 2024-03-29T04:31:13.826 INFO:teuthology.orchestra.run.smithi152.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 348 kB/s | 32 kB 00:00 2024-03-29T04:31:13.832 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:52 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:13.851 INFO:teuthology.orchestra.run.smithi152.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 950 kB/s | 47 kB 00:00 2024-03-29T04:31:13.877 INFO:teuthology.orchestra.run.smithi152.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 768 kB/s | 39 kB 00:00 2024-03-29T04:31:13.902 INFO:teuthology.orchestra.run.smithi152.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-29T04:31:13.953 INFO:teuthology.orchestra.run.smithi152.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-03-29T04:31:13.978 INFO:teuthology.orchestra.run.smithi152.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-03-29T04:31:13.982 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:13.983 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-03-29T04:31:13.984 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:13.984 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:31:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-03-29T04:31:13.984 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-03-29T04:31:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-03-29T04:31:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-03-29T04:31:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-29T04:31:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-29T04:31:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-29T04:31:13.984 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-03-29T04:31:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-29T04:31:13.985 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:13.985 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:31:13.985 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:13.985 INFO:teuthology.orchestra.run.smithi142.stdout:Install 7 Packages 2024-03-29T04:31:13.985 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:13.985 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 2.1 M 2024-03-29T04:31:13.985 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 8.5 M 2024-03-29T04:31:13.986 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:31:14.003 INFO:teuthology.orchestra.run.smithi152.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-03-29T04:31:14.028 INFO:teuthology.orchestra.run.smithi152.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-03-29T04:31:14.174 INFO:teuthology.orchestra.run.smithi142.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 230 kB/s | 23 kB 00:00 2024-03-29T04:31:14.199 INFO:teuthology.orchestra.run.smithi142.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 116 kB/s | 15 kB 00:00 2024-03-29T04:31:14.229 INFO:teuthology.orchestra.run.smithi152.stdout:(15/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 26 MB/s | 18 MB 00:00 2024-03-29T04:31:14.241 INFO:teuthology.orchestra.run.smithi142.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 10 MB/s | 1.8 MB 00:00 2024-03-29T04:31:14.291 INFO:teuthology.orchestra.run.smithi142.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 77 kB/s | 9.0 kB 00:00 2024-03-29T04:31:14.350 INFO:teuthology.orchestra.run.smithi142.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 272 kB/s | 41 kB 00:00 2024-03-29T04:31:14.371 INFO:teuthology.orchestra.run.smithi152.stdout:(16/33): python3-requests-2.25.1-8.el9.noarch.r 340 kB/s | 125 kB 00:00 2024-03-29T04:31:14.375 INFO:teuthology.orchestra.run.smithi142.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 230 kB/s | 31 kB 00:00 2024-03-29T04:31:14.450 INFO:teuthology.orchestra.run.smithi142.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.2 MB/s | 188 kB 00:00 2024-03-29T04:31:14.452 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:14.452 INFO:teuthology.orchestra.run.smithi142.stdout:Total 4.4 MB/s | 2.1 MB 00:00 2024-03-29T04:31:14.452 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:31:14.460 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:31:14.460 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:31:14.471 INFO:teuthology.orchestra.run.smithi152.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 98 kB/s | 43 kB 00:00 2024-03-29T04:31:14.520 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:31:14.521 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:31:14.555 INFO:teuthology.orchestra.run.smithi152.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 128 kB/s | 11 kB 00:00 2024-03-29T04:31:14.613 INFO:teuthology.orchestra.run.smithi152.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 479 kB/s | 172 kB 00:00 2024-03-29T04:31:14.638 INFO:teuthology.orchestra.run.smithi152.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 212 kB/s | 18 kB 00:00 2024-03-29T04:31:14.680 INFO:teuthology.orchestra.run.smithi152.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 346 kB/s | 23 kB 00:00 2024-03-29T04:31:14.714 INFO:teuthology.orchestra.run.smithi152.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-03-29T04:31:14.756 INFO:teuthology.orchestra.run.smithi152.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-03-29T04:31:14.781 INFO:teuthology.orchestra.run.smithi152.stdout:(24/33): python3-cherrypy-18.6.1-2.el9.noarch.r 876 kB/s | 358 kB 00:00 2024-03-29T04:31:14.814 INFO:teuthology.orchestra.run.smithi152.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 461 kB/s | 46 kB 00:00 2024-03-29T04:31:14.815 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:31:14.848 INFO:teuthology.orchestra.run.smithi152.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 856 kB/s | 79 kB 00:00 2024-03-29T04:31:14.860 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-29T04:31:14.873 INFO:teuthology.orchestra.run.smithi152.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-03-29T04:31:14.890 INFO:teuthology.orchestra.run.smithi152.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 218 kB/s | 16 kB 00:00 2024-03-29T04:31:14.893 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-29T04:31:14.932 INFO:teuthology.orchestra.run.smithi152.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.0 MB/s | 90 kB 00:00 2024-03-29T04:31:14.940 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-29T04:31:14.949 INFO:teuthology.orchestra.run.smithi152.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-03-29T04:31:14.972 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-29T04:31:15.001 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-03-29T04:31:15.007 INFO:teuthology.orchestra.run.smithi152.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 342 kB/s | 20 kB 00:00 2024-03-29T04:31:15.075 INFO:teuthology.orchestra.run.smithi152.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-03-29T04:31:15.100 INFO:teuthology.orchestra.run.smithi152.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.5 MB/s | 427 kB 00:00 2024-03-29T04:31:15.105 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:15.105 INFO:teuthology.orchestra.run.smithi152.stdout:Total 15 MB/s | 30 MB 00:02 2024-03-29T04:31:15.117 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:31:15.154 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:31:15.154 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:31:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-03-29T04:31:15.270 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-03-29T04:31:15.505 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:31:15.505 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:31:15.814 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-03-29T04:31:15.814 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-03-29T04:31:15.814 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-03-29T04:31:15.815 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-03-29T04:31:15.815 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-29T04:31:15.815 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-29T04:31:15.815 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-29T04:31:16.112 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-29T04:31:16.112 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:16.112 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:16.112 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-03-29T04:31:16.112 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-03-29T04:31:16.112 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-03-29T04:31:16.112 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-29T04:31:16.112 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-29T04:31:16.112 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-29T04:31:16.112 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-29T04:31:16.112 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:16.113 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:16.319 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-29T04:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:31:16.777 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-29T04:31:16.809 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-29T04:31:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-29T04:31:16.886 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:00:55 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:16.942 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-29T04:31:17.029 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-29T04:31:17.039 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:17.040 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-29T04:31:17.041 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-29T04:31:17.042 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-29T04:31:17.042 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-29T04:31:17.042 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-29T04:31:17.042 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:17.042 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:31:17.042 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:17.042 INFO:teuthology.orchestra.run.smithi142.stdout:Install 12 Packages 2024-03-29T04:31:17.042 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:17.043 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 37 M 2024-03-29T04:31:17.043 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 209 M 2024-03-29T04:31:17.043 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:31:17.070 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-29T04:31:17.142 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-29T04:31:17.204 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-29T04:31:17.242 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-29T04:31:17.290 INFO:teuthology.orchestra.run.smithi142.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 253 kB/s | 32 kB 00:00 2024-03-29T04:31:17.290 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-29T04:31:17.353 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-29T04:31:17.398 INFO:teuthology.orchestra.run.smithi142.stdout:(2/12): ceph-mgr-diskprediction-local-18.2.1-0. 32 MB/s | 7.4 MB 00:00 2024-03-29T04:31:17.424 INFO:teuthology.orchestra.run.smithi142.stdout:(3/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 128 kB/s | 17 kB 00:00 2024-03-29T04:31:17.449 INFO:teuthology.orchestra.run.smithi142.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 679 kB/s | 34 kB 00:00 2024-03-29T04:31:17.550 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-29T04:31:17.604 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-29T04:31:17.643 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-29T04:31:17.677 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-29T04:31:17.691 INFO:teuthology.orchestra.run.smithi142.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2024-03-29T04:31:17.708 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-29T04:31:17.758 INFO:teuthology.orchestra.run.smithi142.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 4.8 MB 00:00 2024-03-29T04:31:17.784 INFO:teuthology.orchestra.run.smithi142.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-03-29T04:31:17.784 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-29T04:31:17.818 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-29T04:31:17.860 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-29T04:31:17.867 INFO:teuthology.orchestra.run.smithi142.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.3 MB/s | 3.0 MB 00:00 2024-03-29T04:31:17.900 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-29T04:31:17.935 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-29T04:31:17.969 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-29T04:31:18.067 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-29T04:31:18.100 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-29T04:31:18.171 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-29T04:31:18.201 INFO:teuthology.orchestra.run.smithi142.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-03-29T04:31:18.234 INFO:teuthology.orchestra.run.smithi142.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-03-29T04:31:18.259 INFO:teuthology.orchestra.run.smithi142.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 501 kB/s | 195 kB 00:00 2024-03-29T04:31:18.318 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-29T04:31:18.443 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-29T04:31:18.596 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-03-29T04:31:18.621 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-03-29T04:31:18.645 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-03-29T04:31:18.645 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:31:18.646 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-29T04:31:18.646 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:31:18.646 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:31:18.646 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:19.109 INFO:teuthology.orchestra.run.smithi142.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 270 kB/s | 245 kB 00:00 2024-03-29T04:31:19.115 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:19.115 INFO:teuthology.orchestra.run.smithi142.stdout:Total 18 MB/s | 37 MB 00:02 2024-03-29T04:31:19.159 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:31:19.185 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:31:19.185 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:31:19.624 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:31:19.625 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:31:20.147 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-03-29T04:31:20.173 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-03-29T04:31:20.173 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:31:20.173 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-29T04:31:20.173 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:31:20.173 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:31:20.173 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:20.324 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:31:20.399 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-29T04:31:20.430 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-29T04:31:20.533 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-29T04:31:20.567 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-29T04:31:20.614 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-03-29T04:31:20.637 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-03-29T04:31:20.638 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:31:20.638 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-29T04:31:20.638 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:31:20.638 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:31:20.638 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:20.746 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-29T04:31:20.782 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-29T04:31:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-03-29T04:31:20.863 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-29T04:31:20.871 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-03-29T04:31:20.871 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:31:20.871 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-29T04:31:20.871 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:31:20.871 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:31:20.871 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:20.937 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-03-29T04:31:21.522 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-29T04:31:21.646 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-29T04:31:22.671 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-03-29T04:31:22.671 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-03-29T04:31:22.671 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-03-29T04:31:22.671 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-03-29T04:31:22.671 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-03-29T04:31:22.671 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-03-29T04:31:22.671 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-03-29T04:31:22.671 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-29T04:31:22.671 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-29T04:31:22.671 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-29T04:31:22.672 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-29T04:31:22.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-29T04:31:22.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-29T04:31:22.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-29T04:31:22.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-29T04:31:22.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-29T04:31:22.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-29T04:31:22.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-29T04:31:22.673 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-29T04:31:22.921 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-29T04:31:23.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-29T04:31:23.688 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:23.688 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:31:23.688 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:23.688 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:23.688 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:23.688 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-03-29T04:31:23.688 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:23.688 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:23.688 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-29T04:31:23.688 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-29T04:31:23.688 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-29T04:31:23.689 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-29T04:31:23.689 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-29T04:31:23.689 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-29T04:31:23.689 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-29T04:31:23.689 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-29T04:31:23.689 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-29T04:31:23.689 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-29T04:31:23.689 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-29T04:31:23.689 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-29T04:31:23.689 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-29T04:31:23.689 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-29T04:31:23.689 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply-3.11-14.el9.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:23.690 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:31:23.950 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-29T04:31:23.981 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-03-29T04:31:24.067 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-base 2024-03-29T04:31:24.638 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:47 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:31:24.736 INFO:teuthology.orchestra.run.smithi152.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:31:24.788 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:31:24.789 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-03-29T04:31:24.789 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:31:24.876 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install cephadm 2024-03-29T04:31:25.439 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:48 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:31:25.588 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:31:25.589 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:25.589 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-03-29T04:31:25.589 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:25.590 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:31:25.590 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-03-29T04:31:25.590 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:25.590 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:31:25.590 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:25.590 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-03-29T04:31:25.590 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:25.590 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 221 k 2024-03-29T04:31:25.590 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 213 k 2024-03-29T04:31:25.590 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:31:25.654 INFO:teuthology.orchestra.run.smithi152.stdout:cephadm-18.2.1-0.el9.noarch.rpm 3.4 MB/s | 221 kB 00:00 2024-03-29T04:31:25.655 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:25.655 INFO:teuthology.orchestra.run.smithi152.stdout:Total 3.3 MB/s | 221 kB 00:00 2024-03-29T04:31:25.655 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:31:25.660 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:31:25.660 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:31:25.663 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:31:25.663 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:31:25.736 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:31:26.132 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-29T04:31:26.226 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-29T04:31:26.528 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-03-29T04:31:26.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-03-29T04:31:26.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-29T04:31:26.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-29T04:31:26.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-29T04:31:26.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-29T04:31:26.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-29T04:31:26.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-29T04:31:26.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-29T04:31:26.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-29T04:31:26.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-29T04:31:26.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-29T04:31:26.582 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-29T04:31:26.867 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-29T04:31:26.867 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:26.867 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:31:26.867 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-03-29T04:31:26.867 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:26.867 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:31:26.947 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-29T04:31:26.947 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:26.947 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:26.947 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-03-29T04:31:26.947 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-29T04:31:26.947 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-29T04:31:26.948 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-29T04:31:26.948 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-29T04:31:26.948 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-29T04:31:26.948 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-29T04:31:26.948 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-29T04:31:26.948 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-29T04:31:26.948 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-29T04:31:26.948 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-29T04:31:26.948 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-29T04:31:26.948 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:26.948 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:27.068 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-immutable-object-cache 2024-03-29T04:31:27.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr-rook 2024-03-29T04:31:27.640 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:50 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:31:27.797 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:31:27.797 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:27.797 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2024-03-29T04:31:27.798 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:27.798 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:31:27.798 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-03-29T04:31:27.798 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:27.798 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:31:27.798 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:27.798 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-03-29T04:31:27.798 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:27.798 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 142 k 2024-03-29T04:31:27.798 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 395 k 2024-03-29T04:31:27.798 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:31:27.873 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 1.9 MB/s | 142 kB 00:00 2024-03-29T04:31:27.874 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:27.874 INFO:teuthology.orchestra.run.smithi152.stdout:Total 1.8 MB/s | 142 kB 00:00 2024-03-29T04:31:27.874 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:31:27.881 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:31:27.882 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:31:27.933 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:31:27.933 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:31:27.954 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:06 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:28.068 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:31:28.108 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:28.109 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:28.109 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-03-29T04:31:28.109 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:28.109 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:31:28.109 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-29T04:31:28.110 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-29T04:31:28.111 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:28.111 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:31:28.111 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:28.111 INFO:teuthology.orchestra.run.smithi142.stdout:Install 13 Packages 2024-03-29T04:31:28.111 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:28.111 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 2.0 M 2024-03-29T04:31:28.111 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 26 M 2024-03-29T04:31:28.112 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:31:28.132 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-29T04:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:28.373 INFO:teuthology.orchestra.run.smithi142.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 476 kB/s | 50 kB 00:00 2024-03-29T04:31:28.398 INFO:teuthology.orchestra.run.smithi142.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 187 kB/s | 24 kB 00:00 2024-03-29T04:31:28.415 INFO:teuthology.orchestra.run.smithi142.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 122 kB/s | 18 kB 00:00 2024-03-29T04:31:28.449 INFO:teuthology.orchestra.run.smithi142.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-29T04:31:28.474 INFO:teuthology.orchestra.run.smithi142.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-29T04:31:28.499 INFO:teuthology.orchestra.run.smithi142.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 211 kB 00:00 2024-03-29T04:31:28.516 INFO:teuthology.orchestra.run.smithi142.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 643 kB/s | 43 kB 00:00 2024-03-29T04:31:28.766 INFO:teuthology.orchestra.run.smithi142.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 56 kB/s | 15 kB 00:00 2024-03-29T04:31:28.784 INFO:teuthology.orchestra.run.smithi142.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 113 kB/s | 32 kB 00:00 2024-03-29T04:31:28.920 INFO:teuthology.orchestra.run.smithi142.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 441 kB/s | 59 kB 00:00 2024-03-29T04:31:28.942 INFO:teuthology.orchestra.run.smithi142.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 476 kB/s | 203 kB 00:00 2024-03-29T04:31:29.042 INFO:teuthology.orchestra.run.smithi142.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 736 kB/s | 90 kB 00:00 2024-03-29T04:31:29.086 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:29.086 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:29.086 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:31:29.087 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:29.087 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:29.087 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:31:29.126 INFO:teuthology.orchestra.run.smithi142.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.8 MB/s | 1.0 MB 00:00 2024-03-29T04:31:29.130 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:29.130 INFO:teuthology.orchestra.run.smithi142.stdout:Total 2.0 MB/s | 2.0 MB 00:01 2024-03-29T04:31:29.130 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:31:29.143 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:31:29.143 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:31:29.232 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:31:29.232 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:31:29.326 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr 2024-03-29T04:31:29.736 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:31:29.898 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-29T04:31:29.907 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:52 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:31:29.944 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-29T04:31:30.005 INFO:teuthology.orchestra.run.smithi152.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:31:30.007 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-29T04:31:30.045 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-29T04:31:30.056 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:31:30.058 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-03-29T04:31:30.058 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:31:30.086 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-29T04:31:30.129 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-dashboard 2024-03-29T04:31:30.180 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-29T04:31:30.296 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-29T04:31:30.362 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-29T04:31:30.693 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:53 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:31:30.843 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:31:30.844 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:30.844 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-03-29T04:31:30.844 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:30.844 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:31:30.844 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-03-29T04:31:30.844 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-03-29T04:31:30.844 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-03-29T04:31:30.844 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-03-29T04:31:30.844 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-29T04:31:30.844 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-29T04:31:30.844 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-29T04:31:30.844 INFO:teuthology.orchestra.run.smithi152.stdout:Installing weak dependencies: 2024-03-29T04:31:30.845 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-29T04:31:30.845 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:30.845 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:31:30.845 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:30.845 INFO:teuthology.orchestra.run.smithi152.stdout:Install 7 Packages 2024-03-29T04:31:30.845 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:30.845 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 2.1 M 2024-03-29T04:31:30.846 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 8.5 M 2024-03-29T04:31:30.846 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:31:31.034 INFO:teuthology.orchestra.run.smithi152.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 220 kB/s | 23 kB 00:00 2024-03-29T04:31:31.059 INFO:teuthology.orchestra.run.smithi152.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 112 kB/s | 15 kB 00:00 2024-03-29T04:31:31.086 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-29T04:31:31.109 INFO:teuthology.orchestra.run.smithi152.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 9.7 MB/s | 1.8 MB 00:00 2024-03-29T04:31:31.146 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-29T04:31:31.168 INFO:teuthology.orchestra.run.smithi152.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 67 kB/s | 9.0 kB 00:00 2024-03-29T04:31:31.176 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-29T04:31:31.193 INFO:teuthology.orchestra.run.smithi152.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 367 kB/s | 31 kB 00:00 2024-03-29T04:31:31.218 INFO:teuthology.orchestra.run.smithi152.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 257 kB/s | 41 kB 00:00 2024-03-29T04:31:31.224 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-29T04:31:31.246 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-03-29T04:31:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.7 MB/s | 188 kB 00:00 2024-03-29T04:31:31.278 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:31.279 INFO:teuthology.orchestra.run.smithi152.stdout:Total 4.7 MB/s | 2.1 MB 00:00 2024-03-29T04:31:31.279 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:31:31.286 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:31:31.287 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:31:31.354 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:31:31.354 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:31:31.654 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:31:31.720 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-29T04:31:31.754 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-29T04:31:31.809 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-29T04:31:31.849 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-29T04:31:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-03-29T04:31:32.010 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-03-29T04:31:32.010 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-03-29T04:31:32.010 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-29T04:31:32.011 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-29T04:31:32.011 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-29T04:31:32.011 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-29T04:31:32.011 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-29T04:31:32.011 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-29T04:31:32.011 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-29T04:31:32.011 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-29T04:31:32.011 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-29T04:31:32.011 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-29T04:31:32.011 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-29T04:31:32.114 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-03-29T04:31:32.147 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-03-29T04:31:32.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-29T04:31:32.369 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:32.369 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:32.369 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-03-29T04:31:32.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-29T04:31:32.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-29T04:31:32.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-29T04:31:32.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-29T04:31:32.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-29T04:31:32.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-29T04:31:32.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-29T04:31:32.369 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-29T04:31:32.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-29T04:31:32.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-29T04:31:32.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-29T04:31:32.370 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-29T04:31:32.370 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:32.370 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:32.612 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-mgr-cephadm 2024-03-29T04:31:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-03-29T04:31:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-03-29T04:31:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-03-29T04:31:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-03-29T04:31:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-29T04:31:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-29T04:31:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-29T04:31:32.991 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-29T04:31:32.991 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:32.991 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:31:32.991 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-03-29T04:31:32.991 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-03-29T04:31:32.991 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-03-29T04:31:32.991 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-29T04:31:32.991 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-29T04:31:32.991 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-29T04:31:32.992 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-29T04:31:32.992 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:32.992 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:31:33.187 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:12 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:33.206 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-29T04:31:33.348 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:33.349 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:31:33.350 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:33.350 INFO:teuthology.orchestra.run.smithi142.stdout:Install 6 Packages 2024-03-29T04:31:33.350 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:33.350 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 6.8 M 2024-03-29T04:31:33.350 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 33 M 2024-03-29T04:31:33.350 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:31:33.794 INFO:teuthology.orchestra.run.smithi142.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 1.1 MB/s | 132 kB 00:00 2024-03-29T04:31:33.795 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:56 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:31:33.895 INFO:teuthology.orchestra.run.smithi142.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-03-29T04:31:33.955 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:31:33.956 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-03-29T04:31:33.956 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:33.956 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:31:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-03-29T04:31:33.956 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-03-29T04:31:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-29T04:31:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout:Install 12 Packages 2024-03-29T04:31:33.957 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:33.958 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 37 M 2024-03-29T04:31:33.958 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 209 M 2024-03-29T04:31:33.958 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:31:34.203 INFO:teuthology.orchestra.run.smithi142.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 5.8 MB 00:00 2024-03-29T04:31:34.228 INFO:teuthology.orchestra.run.smithi142.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 173 kB/s | 58 kB 00:00 2024-03-29T04:31:34.259 INFO:teuthology.orchestra.run.smithi152.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 230 kB/s | 32 kB 00:00 2024-03-29T04:31:34.292 INFO:teuthology.orchestra.run.smithi152.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 518 kB/s | 17 kB 00:00 2024-03-29T04:31:34.312 INFO:teuthology.orchestra.run.smithi142.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.0 MB/s | 548 kB 00:00 2024-03-29T04:31:34.384 INFO:teuthology.orchestra.run.smithi152.stdout:(3/12): ceph-mgr-diskprediction-local-18.2.1-0. 28 MB/s | 7.4 MB 00:00 2024-03-29T04:31:34.409 INFO:teuthology.orchestra.run.smithi152.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 292 kB/s | 34 kB 00:00 2024-03-29T04:31:34.495 INFO:teuthology.orchestra.run.smithi142.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 175 kB/s | 51 kB 00:00 2024-03-29T04:31:34.499 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:34.499 INFO:teuthology.orchestra.run.smithi142.stdout:Total 5.9 MB/s | 6.8 MB 00:01 2024-03-29T04:31:34.513 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:31:34.524 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:31:34.524 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:31:34.612 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:31:34.613 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:31:34.926 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:31:35.110 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-29T04:31:35.163 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-29T04:31:35.268 INFO:teuthology.orchestra.run.smithi152.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-03-29T04:31:35.610 INFO:teuthology.orchestra.run.smithi152.stdout:(6/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 637 kB/s | 217 kB 00:00 2024-03-29T04:31:35.683 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-29T04:31:35.774 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-29T04:31:35.835 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-29T04:31:35.859 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-03-29T04:31:35.927 INFO:teuthology.orchestra.run.smithi152.stdout:(7/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 3.1 MB/s | 4.7 MB 00:01 2024-03-29T04:31:36.035 INFO:teuthology.orchestra.run.smithi152.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.9 MB/s | 4.8 MB 00:01 2024-03-29T04:31:36.428 INFO:teuthology.orchestra.run.smithi152.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-03-29T04:31:36.623 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-03-29T04:31:36.623 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-03-29T04:31:36.623 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-29T04:31:36.623 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-29T04:31:36.623 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-29T04:31:36.623 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-29T04:31:36.628 INFO:teuthology.orchestra.run.smithi152.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 331 kB/s | 195 kB 00:00 2024-03-29T04:31:36.653 INFO:teuthology.orchestra.run.smithi152.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-03-29T04:31:36.770 INFO:teuthology.orchestra.run.smithi152.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 954 kB/s | 803 kB 00:00 2024-03-29T04:31:36.772 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:36.772 INFO:teuthology.orchestra.run.smithi152.stdout:Total 13 MB/s | 37 MB 00:02 2024-03-29T04:31:36.817 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:31:36.842 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:31:36.842 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:31:36.915 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-29T04:31:36.915 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:36.915 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:36.915 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-03-29T04:31:36.915 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-29T04:31:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-29T04:31:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-29T04:31:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-29T04:31:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-29T04:31:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:36.916 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:37.184 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-fuse 2024-03-29T04:31:37.269 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:31:37.269 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:31:37.746 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:16 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:37.897 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:37.897 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:37.897 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-03-29T04:31:37.897 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:37.897 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:31:37.898 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-03-29T04:31:37.898 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-03-29T04:31:37.898 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-29T04:31:37.898 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:37.898 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:31:37.898 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:37.898 INFO:teuthology.orchestra.run.smithi142.stdout:Install 2 Packages 2024-03-29T04:31:37.898 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:37.899 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 918 k 2024-03-29T04:31:37.899 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 2.7 M 2024-03-29T04:31:37.899 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:31:37.969 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:31:38.013 INFO:teuthology.orchestra.run.smithi142.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 7.2 MB/s | 839 kB 00:00 2024-03-29T04:31:38.051 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-29T04:31:38.082 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-29T04:31:38.113 INFO:teuthology.orchestra.run.smithi142.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 366 kB/s | 79 kB 00:00 2024-03-29T04:31:38.115 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:38.115 INFO:teuthology.orchestra.run.smithi142.stdout:Total 4.1 MB/s | 918 kB 00:00 2024-03-29T04:31:38.115 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:31:38.136 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:31:38.136 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:31:38.178 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-29T04:31:38.208 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:31:38.209 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:31:38.218 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-29T04:31:38.395 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-29T04:31:38.396 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:31:38.426 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-29T04:31:38.510 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-29T04:31:38.537 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-29T04:31:38.613 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-03-29T04:31:39.159 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-29T04:31:39.287 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-29T04:31:39.369 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-03-29T04:31:39.370 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-03-29T04:31:39.656 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-29T04:31:39.656 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:39.656 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:39.656 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-29T04:31:39.656 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:39.656 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:39.935 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install librados-devel 2024-03-29T04:31:40.508 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:19 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:40.513 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-29T04:31:40.661 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:40.662 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:40.662 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-03-29T04:31:40.662 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:40.662 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:31:40.662 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-03-29T04:31:40.662 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:40.662 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:31:40.662 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:40.662 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-03-29T04:31:40.662 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:40.662 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 127 k 2024-03-29T04:31:40.662 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 456 k 2024-03-29T04:31:40.663 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:31:40.727 INFO:teuthology.orchestra.run.smithi142.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 127 kB 00:00 2024-03-29T04:31:40.727 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:40.728 INFO:teuthology.orchestra.run.smithi142.stdout:Total 1.9 MB/s | 127 kB 00:00 2024-03-29T04:31:40.728 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:31:40.733 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:31:40.733 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:31:40.789 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:31:40.789 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:31:40.930 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:31:41.025 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:41.453 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:41.534 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-29T04:31:41.566 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-03-29T04:31:41.740 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:41.740 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:41.740 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:41.740 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:41.740 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:41.740 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:42.004 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install libcephfs2 2024-03-29T04:31:42.579 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:21 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:42.681 INFO:teuthology.orchestra.run.smithi142.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:31:42.737 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:42.737 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-03-29T04:31:42.738 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:42.795 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install libcephfs-devel 2024-03-29T04:31:43.359 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:22 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:43.511 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:43.511 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-03-29T04:31:43.512 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:43.512 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:31:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-03-29T04:31:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:43.512 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:31:43.512 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:43.512 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-03-29T04:31:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:43.512 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 31 k 2024-03-29T04:31:43.512 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 139 k 2024-03-29T04:31:43.512 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:31:43.580 INFO:teuthology.orchestra.run.smithi142.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 460 kB/s | 31 kB 00:00 2024-03-29T04:31:43.580 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:43.580 INFO:teuthology.orchestra.run.smithi142.stdout:Total 450 kB/s | 31 kB 00:00 2024-03-29T04:31:43.581 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:31:43.584 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:31:43.584 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:31:43.609 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:31:43.609 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:31:43.704 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:31:43.813 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:44.202 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:44.371 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-03-29T04:31:44.372 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-03-29T04:31:44.372 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-29T04:31:44.372 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-29T04:31:44.372 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-29T04:31:44.372 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-29T04:31:44.372 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-29T04:31:44.372 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-29T04:31:44.372 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-29T04:31:44.372 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-29T04:31:44.372 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-29T04:31:44.372 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-29T04:31:44.464 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:44.465 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:44.465 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:44.465 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:44.465 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:44.465 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:44.636 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install librados2 2024-03-29T04:31:44.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-29T04:31:44.862 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-29T04:31:44.863 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-29T04:31:44.863 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:44.863 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:31:45.199 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:24 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:45.251 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-rook 2024-03-29T04:31:45.297 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:31:45.349 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:45.350 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-03-29T04:31:45.351 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:45.411 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install librbd1 2024-03-29T04:31:45.825 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:08 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:31:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:24 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:45.977 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:31:45.978 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:45.978 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-03-29T04:31:45.978 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:45.978 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:31:45.978 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-29T04:31:45.979 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:45.980 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:31:45.980 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:45.980 INFO:teuthology.orchestra.run.smithi152.stdout:Install 13 Packages 2024-03-29T04:31:45.980 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:45.980 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 2.0 M 2024-03-29T04:31:45.980 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 26 M 2024-03-29T04:31:45.980 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:31:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:31:46.121 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:46.121 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-03-29T04:31:46.121 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:46.188 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install python3-rados 2024-03-29T04:31:46.198 INFO:teuthology.orchestra.run.smithi152.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 460 kB/s | 50 kB 00:00 2024-03-29T04:31:46.223 INFO:teuthology.orchestra.run.smithi152.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 182 kB/s | 24 kB 00:00 2024-03-29T04:31:46.248 INFO:teuthology.orchestra.run.smithi152.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 113 kB/s | 18 kB 00:00 2024-03-29T04:31:46.265 INFO:teuthology.orchestra.run.smithi152.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2024-03-29T04:31:46.290 INFO:teuthology.orchestra.run.smithi152.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-29T04:31:46.316 INFO:teuthology.orchestra.run.smithi152.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.1 MB/s | 211 kB 00:00 2024-03-29T04:31:46.341 INFO:teuthology.orchestra.run.smithi152.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 569 kB/s | 43 kB 00:00 2024-03-29T04:31:46.608 INFO:teuthology.orchestra.run.smithi152.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 101 kB/s | 32 kB 00:00 2024-03-29T04:31:46.633 INFO:teuthology.orchestra.run.smithi152.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 51 kB/s | 15 kB 00:00 2024-03-29T04:31:46.757 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:25 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:46.783 INFO:teuthology.orchestra.run.smithi152.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 459 kB/s | 203 kB 00:00 2024-03-29T04:31:46.809 INFO:teuthology.orchestra.run.smithi152.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 337 kB/s | 59 kB 00:00 2024-03-29T04:31:46.858 INFO:teuthology.orchestra.run.smithi142.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:31:46.909 INFO:teuthology.orchestra.run.smithi152.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 717 kB/s | 90 kB 00:00 2024-03-29T04:31:46.911 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:46.912 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-03-29T04:31:46.912 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:46.942 INFO:teuthology.orchestra.run.smithi152.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.1 MB/s | 1.0 MB 00:00 2024-03-29T04:31:46.946 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:46.946 INFO:teuthology.orchestra.run.smithi152.stdout:Total 2.1 MB/s | 2.0 MB 00:00 2024-03-29T04:31:46.946 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:31:46.959 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:31:46.959 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:31:46.973 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install python3-rgw 2024-03-29T04:31:47.049 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:31:47.049 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:31:47.539 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:26 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:47.551 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:31:47.639 INFO:teuthology.orchestra.run.smithi142.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:31:47.691 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:47.692 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-03-29T04:31:47.692 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:47.722 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-29T04:31:47.754 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install python3-cephfs 2024-03-29T04:31:47.785 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-29T04:31:47.849 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-29T04:31:47.886 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-29T04:31:47.927 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-29T04:31:48.020 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-29T04:31:48.129 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-29T04:31:48.191 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-29T04:31:48.316 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:27 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:48.416 INFO:teuthology.orchestra.run.smithi142.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:31:48.469 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:48.470 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-03-29T04:31:48.470 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:48.531 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install python3-rbd 2024-03-29T04:31:48.899 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-29T04:31:48.951 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-29T04:31:48.984 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-29T04:31:49.032 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-29T04:31:49.054 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-03-29T04:31:49.099 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:28 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:49.198 INFO:teuthology.orchestra.run.smithi142.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:31:49.251 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:49.251 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-03-29T04:31:49.252 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:49.315 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install rbd-fuse 2024-03-29T04:31:49.874 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-03-29T04:31:49.874 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-03-29T04:31:49.874 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-29T04:31:49.874 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-29T04:31:49.874 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-29T04:31:49.874 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-29T04:31:49.874 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-29T04:31:49.874 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-29T04:31:49.874 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-29T04:31:49.874 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-29T04:31:49.875 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-29T04:31:49.875 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-29T04:31:49.875 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-29T04:31:49.879 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:28 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:50.029 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:50.030 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:50.030 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-03-29T04:31:50.030 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:50.030 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:31:50.030 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-03-29T04:31:50.030 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:50.030 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:31:50.030 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:50.030 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-03-29T04:31:50.031 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:50.031 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 86 k 2024-03-29T04:31:50.031 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 226 k 2024-03-29T04:31:50.031 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:31:50.100 INFO:teuthology.orchestra.run.smithi142.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.2 MB/s | 86 kB 00:00 2024-03-29T04:31:50.100 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:50.100 INFO:teuthology.orchestra.run.smithi142.stdout:Total 1.2 MB/s | 86 kB 00:00 2024-03-29T04:31:50.101 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:31:50.108 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:31:50.108 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:31:50.162 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:31:50.162 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-29T04:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-03-29T04:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-29T04:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-29T04:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-29T04:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-29T04:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-29T04:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-29T04:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-29T04:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-29T04:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-29T04:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-29T04:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-29T04:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-29T04:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:31:50.306 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:31:50.406 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:50.514 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-cephadm 2024-03-29T04:31:50.786 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:51.072 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:51.072 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:51.072 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:51.072 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:51.072 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:51.072 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:51.081 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:14 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:31:51.233 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:31:51.234 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:51.234 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-03-29T04:31:51.234 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:51.234 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:31:51.235 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-03-29T04:31:51.235 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-03-29T04:31:51.235 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-29T04:31:51.235 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-29T04:31:51.235 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-29T04:31:51.235 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-29T04:31:51.235 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-29T04:31:51.235 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:51.235 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:31:51.235 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:31:51.235 INFO:teuthology.orchestra.run.smithi152.stdout:Install 6 Packages 2024-03-29T04:31:51.235 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:31:51.236 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 6.8 M 2024-03-29T04:31:51.236 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 33 M 2024-03-29T04:31:51.236 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:31:51.300 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install rbd-mirror 2024-03-29T04:31:51.535 INFO:teuthology.orchestra.run.smithi152.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 1.4 MB/s | 132 kB 00:00 2024-03-29T04:31:51.663 INFO:teuthology.orchestra.run.smithi152.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-03-29T04:31:51.738 INFO:teuthology.orchestra.run.smithi152.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-03-29T04:31:51.866 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:30 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:52.021 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:52.022 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:52.022 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-03-29T04:31:52.022 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:52.022 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:31:52.022 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-03-29T04:31:52.022 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:52.022 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:31:52.022 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:52.022 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-03-29T04:31:52.022 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:52.022 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 3.0 M 2024-03-29T04:31:52.023 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 12 M 2024-03-29T04:31:52.023 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:31:52.120 INFO:teuthology.orchestra.run.smithi142.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 31 MB/s | 3.0 MB 00:00 2024-03-29T04:31:52.120 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:52.121 INFO:teuthology.orchestra.run.smithi142.stdout:Total 30 MB/s | 3.0 MB 00:00 2024-03-29T04:31:52.121 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:31:52.129 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:31:52.130 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:31:52.202 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:31:52.202 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:31:52.614 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:31:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:52.703 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:52.703 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:31:52.703 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-29T04:31:52.703 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:31:52.704 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:31:52.704 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:53.804 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:53.804 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:53.804 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:53.804 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:53.804 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:53.804 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:54.038 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install rbd-nbd 2024-03-29T04:31:54.618 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:33 ago on Fri 29 Mar 2024 04:30:21 AM UTC. 2024-03-29T04:31:54.770 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-29T04:31:54.771 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:54.771 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-03-29T04:31:54.771 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:54.771 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-29T04:31:54.771 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-03-29T04:31:54.771 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:54.771 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-29T04:31:54.771 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-29T04:31:54.771 INFO:teuthology.orchestra.run.smithi142.stdout:Install 1 Package 2024-03-29T04:31:54.771 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:54.771 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 171 k 2024-03-29T04:31:54.772 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 494 k 2024-03-29T04:31:54.772 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-29T04:31:54.840 INFO:teuthology.orchestra.run.smithi142.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.4 MB/s | 171 kB 00:00 2024-03-29T04:31:54.840 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:31:54.841 INFO:teuthology.orchestra.run.smithi142.stdout:Total 2.4 MB/s | 171 kB 00:00 2024-03-29T04:31:54.841 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-29T04:31:54.848 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-29T04:31:54.849 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-29T04:31:54.904 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-29T04:31:54.904 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-29T04:31:55.047 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-29T04:31:55.138 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:55.542 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:55.837 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:31:55.837 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:55.837 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-29T04:31:55.838 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-03-29T04:31:55.838 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:31:55.838 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-29T04:31:56.069 DEBUG:teuthology.parallel:result is None 2024-03-29T04:34:02.568 INFO:teuthology.orchestra.run.smithi152.stdout:[MIRROR] python3-asyncssh-2.13.2-5.el9.noarch.rpm: Curl error (28): Timeout was reached for http://epel.stl.us.ssimn.org/9/Everything/x86_64/Packages/p/python3-asyncssh-2.13.2-5.el9.noarch.rpm [Failed to connect to epel.stl.us.ssimn.org port 80: Connection timed out] 2024-03-29T04:34:02.568 INFO:teuthology.orchestra.run.smithi152.stdout:[MIRROR] python3-natsort-7.1.1-5.el9.noarch.rpm: Curl error (28): Timeout was reached for http://epel.stl.us.ssimn.org/9/Everything/x86_64/Packages/p/python3-natsort-7.1.1-5.el9.noarch.rpm [Failed to connect to epel.stl.us.ssimn.org port 80: Connection timed out] 2024-03-29T04:34:02.568 INFO:teuthology.orchestra.run.smithi152.stdout:[MIRROR] python3-typing-extensions-4.4.0-2.el9.noarch.rpm: Curl error (28): Timeout was reached for http://epel.stl.us.ssimn.org/9/Everything/x86_64/Packages/p/python3-typing-extensions-4.4.0-2.el9.noarch.rpm [Failed to connect to epel.stl.us.ssimn.org port 80: Connection timed out] 2024-03-29T04:34:02.808 INFO:teuthology.orchestra.run.smithi152.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 398 B/s | 51 kB 02:11 2024-03-29T04:34:02.833 INFO:teuthology.orchestra.run.smithi152.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 450 B/s | 58 kB 02:11 2024-03-29T04:34:02.984 INFO:teuthology.orchestra.run.smithi152.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.2 kB/s | 548 kB 02:11 2024-03-29T04:34:02.984 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:34:02.984 INFO:teuthology.orchestra.run.smithi152.stdout:Total 53 kB/s | 6.8 MB 02:11 2024-03-29T04:34:02.997 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:34:03.009 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:34:03.009 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:34:03.101 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:34:03.102 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:34:03.396 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:34:03.574 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-29T04:34:03.618 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-29T04:34:04.141 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-29T04:34:04.246 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-29T04:34:04.306 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-29T04:34:04.338 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-03-29T04:34:05.088 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-03-29T04:34:05.088 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-03-29T04:34:05.088 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-29T04:34:05.089 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-29T04:34:05.089 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-29T04:34:05.089 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-29T04:34:05.406 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-29T04:34:05.407 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:05.407 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:34:05.407 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-03-29T04:34:05.407 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-29T04:34:05.407 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-29T04:34:05.407 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-29T04:34:05.407 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-29T04:34:05.407 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-29T04:34:05.407 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:05.407 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:05.664 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-fuse 2024-03-29T04:34:06.230 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:29 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:06.380 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:06.380 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:06.380 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-03-29T04:34:06.381 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:06.381 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:34:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-03-29T04:34:06.381 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-03-29T04:34:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-29T04:34:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:06.381 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:34:06.381 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:06.381 INFO:teuthology.orchestra.run.smithi152.stdout:Install 2 Packages 2024-03-29T04:34:06.381 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:06.382 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 918 k 2024-03-29T04:34:06.382 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 2.7 M 2024-03-29T04:34:06.382 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:34:06.476 INFO:teuthology.orchestra.run.smithi152.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 8.8 MB/s | 839 kB 00:00 2024-03-29T04:34:06.534 INFO:teuthology.orchestra.run.smithi152.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 516 kB/s | 79 kB 00:00 2024-03-29T04:34:06.535 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:34:06.536 INFO:teuthology.orchestra.run.smithi152.stdout:Total 5.8 MB/s | 918 kB 00:00 2024-03-29T04:34:06.536 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:34:06.556 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:34:06.556 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:34:06.627 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:34:06.628 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:34:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:34:06.958 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-29T04:34:07.027 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-03-29T04:34:07.676 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-03-29T04:34:07.677 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-03-29T04:34:07.964 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-29T04:34:07.964 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:07.964 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:34:07.964 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-29T04:34:07.964 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:07.964 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:08.220 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install librados-devel 2024-03-29T04:34:08.797 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:31 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:08.955 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:08.956 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-03-29T04:34:08.956 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:08.956 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:34:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-03-29T04:34:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:08.957 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:34:08.957 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:08.957 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-03-29T04:34:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:08.957 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 127 k 2024-03-29T04:34:08.957 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 456 k 2024-03-29T04:34:08.957 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:34:09.021 INFO:teuthology.orchestra.run.smithi152.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 127 kB 00:00 2024-03-29T04:34:09.022 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:34:09.022 INFO:teuthology.orchestra.run.smithi152.stdout:Total 1.9 MB/s | 127 kB 00:00 2024-03-29T04:34:09.022 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:34:09.028 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:34:09.028 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:34:09.083 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:34:09.083 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:34:09.232 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:34:09.347 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:09.759 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:10.063 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:10.063 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:10.063 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:34:10.063 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-03-29T04:34:10.063 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:10.063 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:10.348 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install libcephfs2 2024-03-29T04:34:10.926 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:33 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:11.029 INFO:teuthology.orchestra.run.smithi152.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:34:11.085 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:11.086 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-03-29T04:34:11.086 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:11.160 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install libcephfs-devel 2024-03-29T04:34:11.728 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:34 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:11.883 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:11.883 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-03-29T04:34:11.883 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:11.883 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:34:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-03-29T04:34:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:11.884 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:34:11.884 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:11.884 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-03-29T04:34:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:11.884 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 31 k 2024-03-29T04:34:11.884 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 139 k 2024-03-29T04:34:11.884 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:34:11.949 INFO:teuthology.orchestra.run.smithi152.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 472 kB/s | 31 kB 00:00 2024-03-29T04:34:11.950 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:34:11.950 INFO:teuthology.orchestra.run.smithi152.stdout:Total 463 kB/s | 31 kB 00:00 2024-03-29T04:34:11.950 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:34:11.954 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:34:11.954 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:34:11.976 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:34:11.976 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:34:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:34:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:12.540 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:12.804 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:12.805 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:12.805 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:34:12.805 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-03-29T04:34:12.805 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:12.805 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:12.997 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install librados2 2024-03-29T04:34:13.562 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:36 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:13.661 INFO:teuthology.orchestra.run.smithi152.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:34:13.713 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:13.714 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-03-29T04:34:13.714 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:13.788 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install librbd1 2024-03-29T04:34:14.346 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:37 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:14.444 INFO:teuthology.orchestra.run.smithi152.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:34:14.497 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:14.498 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-03-29T04:34:14.498 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:14.571 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-rados 2024-03-29T04:34:15.137 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:38 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:15.235 INFO:teuthology.orchestra.run.smithi152.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:34:15.290 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:15.291 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-03-29T04:34:15.291 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:15.363 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-rgw 2024-03-29T04:34:15.930 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:38 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:16.030 INFO:teuthology.orchestra.run.smithi152.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:34:16.083 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:16.084 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-03-29T04:34:16.084 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:16.163 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-cephfs 2024-03-29T04:34:16.729 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:39 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:16.827 INFO:teuthology.orchestra.run.smithi152.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:34:16.883 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:16.883 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-03-29T04:34:16.884 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:16.952 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-rbd 2024-03-29T04:34:17.515 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:40 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:17.615 INFO:teuthology.orchestra.run.smithi152.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-29T04:34:17.668 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:17.669 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-03-29T04:34:17.669 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:17.735 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install rbd-fuse 2024-03-29T04:34:18.301 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:41 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:18.460 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:18.460 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-03-29T04:34:18.460 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:18.460 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:34:18.460 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-03-29T04:34:18.461 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:18.461 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:34:18.461 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:18.461 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-03-29T04:34:18.461 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:18.461 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 86 k 2024-03-29T04:34:18.461 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 226 k 2024-03-29T04:34:18.461 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:34:18.519 INFO:teuthology.orchestra.run.smithi152.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.4 MB/s | 86 kB 00:00 2024-03-29T04:34:18.520 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:34:18.520 INFO:teuthology.orchestra.run.smithi152.stdout:Total 1.4 MB/s | 86 kB 00:00 2024-03-29T04:34:18.520 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:34:18.527 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:34:18.527 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:34:18.580 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:34:18.581 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:34:18.714 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:34:18.825 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:19.230 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:19.518 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:19.518 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:19.518 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:34:19.518 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-03-29T04:34:19.518 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:19.518 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:19.779 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install rbd-mirror 2024-03-29T04:34:20.351 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:43 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:20.509 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:20.509 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:20.509 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-03-29T04:34:20.509 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:20.509 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:34:20.509 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-03-29T04:34:20.509 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:20.510 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:34:20.510 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:20.510 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-03-29T04:34:20.510 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:20.510 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 3.0 M 2024-03-29T04:34:20.510 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 12 M 2024-03-29T04:34:20.511 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:34:20.614 INFO:teuthology.orchestra.run.smithi152.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 29 MB/s | 3.0 MB 00:00 2024-03-29T04:34:20.614 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:34:20.615 INFO:teuthology.orchestra.run.smithi152.stdout:Total 28 MB/s | 3.0 MB 00:00 2024-03-29T04:34:20.615 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:34:20.624 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:34:20.624 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:34:20.697 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:34:20.697 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:34:21.108 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:34:21.165 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:21.189 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:21.189 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:34:21.189 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-29T04:34:21.190 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:34:21.190 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:34:21.190 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:22.313 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:22.314 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:22.314 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:34:22.314 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-03-29T04:34:22.314 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:22.314 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:22.544 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install rbd-nbd 2024-03-29T04:34:23.126 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:46 ago on Fri 29 Mar 2024 04:30:37 AM UTC. 2024-03-29T04:34:23.276 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-29T04:34:23.277 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:23.277 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-03-29T04:34:23.277 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:23.277 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-29T04:34:23.278 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-03-29T04:34:23.278 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:23.278 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-29T04:34:23.278 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-29T04:34:23.278 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2024-03-29T04:34:23.278 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:23.278 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 171 k 2024-03-29T04:34:23.278 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 494 k 2024-03-29T04:34:23.278 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-29T04:34:23.333 INFO:teuthology.orchestra.run.smithi152.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 3.0 MB/s | 171 kB 00:00 2024-03-29T04:34:23.334 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:34:23.334 INFO:teuthology.orchestra.run.smithi152.stdout:Total 2.9 MB/s | 171 kB 00:00 2024-03-29T04:34:23.334 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-29T04:34:23.342 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-29T04:34:23.342 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-29T04:34:23.397 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-29T04:34:23.397 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-29T04:34:23.540 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-29T04:34:23.631 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:24.345 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-29T04:34:24.345 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:24.346 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-29T04:34:24.346 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-03-29T04:34:24.346 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:34:24.346 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-29T04:34:24.621 DEBUG:teuthology.parallel:result is None 2024-03-29T04:34:24.622 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-29T04:34:24.622 INFO:teuthology.packaging:ref: None 2024-03-29T04:34:24.622 INFO:teuthology.packaging:tag: v18.2.1 2024-03-29T04:34:24.622 INFO:teuthology.packaging:branch: None 2024-03-29T04:34:24.622 INFO:teuthology.packaging:sha1: 8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:34:24.623 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-29T04:34:24.841 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-29T04:34:24.866 INFO:teuthology.orchestra.run.smithi142.stdout:18.2.1-0.el9 2024-03-29T04:34:24.867 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-03-29T04:34:24.867 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-03-29T04:34:24.868 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-29T04:34:24.869 INFO:teuthology.packaging:ref: None 2024-03-29T04:34:24.869 INFO:teuthology.packaging:tag: v18.2.1 2024-03-29T04:34:24.869 INFO:teuthology.packaging:branch: None 2024-03-29T04:34:24.869 INFO:teuthology.packaging:sha1: 8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:34:24.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-29T04:34:25.098 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-29T04:34:25.122 INFO:teuthology.orchestra.run.smithi152.stdout:18.2.1-0.el9 2024-03-29T04:34:25.122 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-03-29T04:34:25.122 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-03-29T04:34:25.124 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-29T04:34:25.124 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:34:25.124 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T04:34:25.157 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:34:25.158 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T04:34:25.190 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-29T04:34:25.191 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:34:25.191 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T04:34:25.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T04:34:25.297 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:34:25.297 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T04:34:25.324 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T04:34:25.391 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-29T04:34:25.391 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:34:25.391 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T04:34:25.420 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T04:34:25.488 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:34:25.489 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T04:34:25.515 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T04:34:25.582 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-29T04:34:25.582 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:34:25.582 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T04:34:25.609 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T04:34:25.675 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:34:25.676 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T04:34:25.702 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T04:34:25.769 INFO:teuthology.run_tasks:Running task print... 2024-03-29T04:34:25.779 INFO:teuthology.task.print:**** done install task... 2024-03-29T04:34:25.779 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-29T04:34:25.891 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.1', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '8da23624bc17fe0b1f797227e72a0db0ce7821ba'} 2024-03-29T04:34:25.891 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.1 2024-03-29T04:34:25.891 INFO:tasks.cephadm:Cluster fsid is 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:34:25.891 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-29T04:34:25.892 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-29T04:34:25.892 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi142': '172.21.15.142', 'mon.smithi152': '172.21.15.152'} 2024-03-29T04:34:25.892 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-29T04:34:25.892 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-03-29T04:34:25.920 DEBUG:teuthology.orchestra.run.smithi152:> sudo hostname $(hostname -s) 2024-03-29T04:34:25.947 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-03-29T04:34:25.947 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:34:26.164 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8da23624bc17fe0b1f797227e72a0db0ce7821ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54734/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1959-g8da23624', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1959.g8da23624'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/8da23624bc17fe0b1f797227e72a0db0ce7821ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-28 17:13:08.560511', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/8da23624bc17fe0b1f797227e72a0db0ce7821ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-29T04:34:26.521 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref reef, sha1 c7067060f289900b6b819327c6351d3e278e7383 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-03-29T04:34:26.522 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/reef/c7067060f289900b6b819327c6351d3e278e7383/centos/9/x86_64/flavors/default/cephadm 2024-03-29T04:34:26.522 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/c7067060f289900b6b819327c6351d3e278e7383/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-29T04:34:27.041 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-r--r--. 1 ubuntu ubuntu 215334 Mar 29 04:34 /home/ubuntu/cephtest/cephadm 2024-03-29T04:34:27.041 DEBUG:teuthology.orchestra.run.smithi152:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/c7067060f289900b6b819327c6351d3e278e7383/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-29T04:34:27.560 INFO:teuthology.orchestra.run.smithi152.stdout:-rw-r--r--. 1 ubuntu ubuntu 215334 Mar 29 04:34 /home/ubuntu/cephtest/cephadm 2024-03-29T04:34:27.560 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-29T04:34:27.580 DEBUG:teuthology.orchestra.run.smithi152:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-29T04:34:27.610 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.1 on all hosts... 2024-03-29T04:34:27.610 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-03-29T04:34:27.623 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-03-29T04:34:28.476 INFO:teuthology.orchestra.run.smithi152.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-03-29T04:34:28.592 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-03-29T04:34:46.971 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:34:46.971 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-03-29T04:34:46.971 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-03-29T04:34:46.971 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-03-29T04:34:46.971 INFO:teuthology.orchestra.run.smithi142.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-03-29T04:34:46.971 INFO:teuthology.orchestra.run.smithi142.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-03-29T04:34:46.971 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-03-29T04:34:46.971 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:34:48.947 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-03-29T04:34:48.947 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-03-29T04:34:48.947 INFO:teuthology.orchestra.run.smithi152.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-03-29T04:34:48.947 INFO:teuthology.orchestra.run.smithi152.stdout: "repo_digests": [ 2024-03-29T04:34:48.947 INFO:teuthology.orchestra.run.smithi152.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-03-29T04:34:48.947 INFO:teuthology.orchestra.run.smithi152.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-03-29T04:34:48.947 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-03-29T04:34:48.947 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-03-29T04:34:48.966 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-03-29T04:34:49.000 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph 2024-03-29T04:34:49.030 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-03-29T04:34:49.071 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /etc/ceph 2024-03-29T04:34:49.097 INFO:tasks.cephadm:Writing seed config... 2024-03-29T04:34:49.098 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-29T04:34:49.098 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-29T04:34:49.098 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-29T04:34:49.098 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-29T04:34:49.098 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-29T04:34:49.098 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-29T04:34:49.098 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-29T04:34:49.099 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-29T04:34:49.100 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-03-29T04:34:49.101 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-29T04:34:49.101 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-29T04:34:49.101 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-29T04:34:49.101 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-29T04:34:49.101 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-29T04:34:49.101 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-03-29T04:34:49.101 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-29T04:34:49.102 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:34:49.102 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-29T04:34:49.127 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9f96302c-ed85-11ee-95d0-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-29T04:34:49.127 DEBUG:teuthology.orchestra.run.smithi142:mon.smithi142> sudo journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi142.service 2024-03-29T04:34:49.170 INFO:tasks.cephadm:Bootstrapping... 2024-03-29T04:34:49.170 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 -v bootstrap --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.142 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-29T04:34:49.329 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:34:49.329 INFO:teuthology.orchestra.run.smithi142.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.1', '-v', 'bootstrap', '--fsid', '9f96302c-ed85-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.142', '--skip-admin-label'] 2024-03-29T04:34:49.354 INFO:teuthology.orchestra.run.smithi142.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-29T04:34:49.354 INFO:teuthology.orchestra.run.smithi142.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-29T04:34:49.354 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying podman|docker is present... 2024-03-29T04:34:49.377 INFO:teuthology.orchestra.run.smithi142.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-29T04:34:49.377 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying lvm2 is present... 2024-03-29T04:34:49.377 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying time synchronization is in place... 2024-03-29T04:34:49.384 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-29T04:34:49.384 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-29T04:34:49.390 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-29T04:34:49.390 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout inactive 2024-03-29T04:34:49.396 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout enabled 2024-03-29T04:34:49.403 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout active 2024-03-29T04:34:49.403 INFO:teuthology.orchestra.run.smithi142.stdout:Unit chronyd.service is enabled and running 2024-03-29T04:34:49.403 INFO:teuthology.orchestra.run.smithi142.stdout:Repeating the final host check... 2024-03-29T04:34:49.426 INFO:teuthology.orchestra.run.smithi142.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-29T04:34:49.426 INFO:teuthology.orchestra.run.smithi142.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-29T04:34:49.426 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl is present 2024-03-29T04:34:49.426 INFO:teuthology.orchestra.run.smithi142.stdout:lvcreate is present 2024-03-29T04:34:49.432 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-29T04:34:49.433 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-29T04:34:49.438 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-29T04:34:49.438 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout inactive 2024-03-29T04:34:49.444 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout enabled 2024-03-29T04:34:49.450 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout active 2024-03-29T04:34:49.450 INFO:teuthology.orchestra.run.smithi142.stdout:Unit chronyd.service is enabled and running 2024-03-29T04:34:49.450 INFO:teuthology.orchestra.run.smithi142.stdout:Host looks OK 2024-03-29T04:34:49.450 INFO:teuthology.orchestra.run.smithi142.stdout:Cluster fsid: 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:34:49.450 INFO:teuthology.orchestra.run.smithi142.stdout:Acquiring lock 140636405349056 on /run/cephadm/9f96302c-ed85-11ee-95d0-87774f69a715.lock 2024-03-29T04:34:49.451 INFO:teuthology.orchestra.run.smithi142.stdout:Lock 140636405349056 acquired on /run/cephadm/9f96302c-ed85-11ee-95d0-87774f69a715.lock 2024-03-29T04:34:49.451 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying IP 172.21.15.142 port 3300 ... 2024-03-29T04:34:49.451 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying IP 172.21.15.142 port 6789 ... 2024-03-29T04:34:49.451 INFO:teuthology.orchestra.run.smithi142.stdout:Base mon IP(s) is [172.21.15.142:3300, 172.21.15.142:6789], mon addrv is [v2:172.21.15.142:3300,v1:172.21.15.142:6789] 2024-03-29T04:34:49.455 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.142 metric 100 2024-03-29T04:34:49.455 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.142 metric 100 2024-03-29T04:34:49.457 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-29T04:34:49.457 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-29T04:34:49.457 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1794sec hoplimit 64 pref medium 2024-03-29T04:34:49.459 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-29T04:34:49.459 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-29T04:34:49.459 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-29T04:34:49.460 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-29T04:34:49.460 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-03-29T04:34:49.460 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-29T04:34:49.460 INFO:teuthology.orchestra.run.smithi142.stdout:Mon IP `172.21.15.142` is in CIDR network `172.21.0.0/20` 2024-03-29T04:34:49.460 INFO:teuthology.orchestra.run.smithi142.stdout:Mon IP `172.21.15.142` is in CIDR network `172.21.0.0/20` 2024-03-29T04:34:49.460 INFO:teuthology.orchestra.run.smithi142.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-29T04:34:49.461 INFO:teuthology.orchestra.run.smithi142.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-29T04:34:49.462 INFO:teuthology.orchestra.run.smithi142.stdout:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-03-29T04:34:53.872 INFO:teuthology.orchestra.run.smithi142.stdout:/bin/podman: stdout 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-03-29T04:34:53.873 INFO:teuthology.orchestra.run.smithi142.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.1... 2024-03-29T04:34:53.873 INFO:teuthology.orchestra.run.smithi142.stdout:/bin/podman: stderr Getting image source signatures 2024-03-29T04:34:53.873 INFO:teuthology.orchestra.run.smithi142.stdout:/bin/podman: stderr Copying blob sha256:a733d3c618b71f19c168ebecd1953429dce2c1631835ca182e9551c36dce5989 2024-03-29T04:34:53.873 INFO:teuthology.orchestra.run.smithi142.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-03-29T04:34:53.873 INFO:teuthology.orchestra.run.smithi142.stdout:/bin/podman: stderr Copying config sha256:5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-03-29T04:34:53.873 INFO:teuthology.orchestra.run.smithi142.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-29T04:35:00.330 INFO:teuthology.orchestra.run.smithi142.stdout:ceph: stdout ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-03-29T04:35:00.331 INFO:teuthology.orchestra.run.smithi142.stdout:Ceph version: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-03-29T04:35:00.331 INFO:teuthology.orchestra.run.smithi142.stdout:Extracting ceph user uid/gid from container image... 2024-03-29T04:35:01.303 INFO:teuthology.orchestra.run.smithi142.stdout:stat: stdout 167 167 2024-03-29T04:35:01.303 INFO:teuthology.orchestra.run.smithi142.stdout:Creating initial keys... 2024-03-29T04:35:02.233 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph-authtool: stdout AQD1RAZm3d2NKBAAMuHcRa30Ei8PvVNNlJlsNQ== 2024-03-29T04:35:03.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph-authtool: stdout AQD2RAZmD2ZpIhAAOPKV45CSE8lL1prgyckf8A== 2024-03-29T04:35:03.937 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph-authtool: stdout AQD3RAZmv9QyFhAAjXX9DZFIBQEaqVEyaCDh7Q== 2024-03-29T04:35:03.938 INFO:teuthology.orchestra.run.smithi142.stdout:Creating initial monmap... 2024-03-29T04:35:04.909 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-29T04:35:04.910 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-29T04:35:04.910 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:35:04.910 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-29T04:35:04.910 INFO:teuthology.orchestra.run.smithi142.stdout:monmaptool for smithi142 [v2:172.21.15.142:3300,v1:172.21.15.142:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-29T04:35:04.910 INFO:teuthology.orchestra.run.smithi142.stdout:setting min_mon_release = pacific 2024-03-29T04:35:04.910 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/monmaptool: set fsid to 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:35:04.910 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-29T04:35:04.910 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:35:04.910 INFO:teuthology.orchestra.run.smithi142.stdout:Creating mon... 2024-03-29T04:35:06.316 INFO:teuthology.orchestra.run.smithi142.stdout:create mon.smithi142 on 2024-03-29T04:35:06.528 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-29T04:35:06.663 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-29T04:35:06.850 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9f96302c-ed85-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-9f96302c-ed85-11ee-95d0-87774f69a715.target. 2024-03-29T04:35:06.850 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9f96302c-ed85-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-9f96302c-ed85-11ee-95d0-87774f69a715.target. 2024-03-29T04:35:07.050 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi142 2024-03-29T04:35:07.050 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi142.service: Unit ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi142.service not loaded. 2024-03-29T04:35:07.208 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9f96302c-ed85-11ee-95d0-87774f69a715.target.wants/ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi142.service → /etc/systemd/system/ceph-9f96302c-ed85-11ee-95d0-87774f69a715@.service. 2024-03-29T04:35:07.725 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:07 smithi142 ceph-mon[28654]: mon.smithi142 is new leader, mons smithi142 in quorum (ranks 0) 2024-03-29T04:35:07.738 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-29T04:35:07.739 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout disabled 2024-03-29T04:35:07.745 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-29T04:35:07.745 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout inactive 2024-03-29T04:35:07.745 INFO:teuthology.orchestra.run.smithi142.stdout:firewalld.service is not enabled 2024-03-29T04:35:07.745 INFO:teuthology.orchestra.run.smithi142.stdout:Not possible to enable service . firewalld.service is not available 2024-03-29T04:35:07.745 INFO:teuthology.orchestra.run.smithi142.stdout:Waiting for mon to start... 2024-03-29T04:35:07.745 INFO:teuthology.orchestra.run.smithi142.stdout:Waiting for mon... 2024-03-29T04:35:07.992 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:07 smithi142 bash[28635]: 34d24d6d8873b496bc710e3edb4ba17f1e04ef6bcd54c0621c7b3744ac28081f 2024-03-29T04:35:07.992 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:07 smithi142 systemd[1]: Started Ceph mon.smithi142 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:35:08.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:08 smithi142 ceph-mon[28654]: mon.smithi142 is new leader, mons smithi142 in quorum (ranks 0) 2024-03-29T04:35:08.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:08 smithi142 ceph-mon[28654]: monmap e1: 1 mons at {smithi142=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-29T04:35:08.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:08 smithi142 ceph-mon[28654]: fsmap 2024-03-29T04:35:08.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:08 smithi142 ceph-mon[28654]: osdmap e1: 0 total, 0 up, 0 in 2024-03-29T04:35:08.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:08 smithi142 ceph-mon[28654]: mgrmap e1: no daemons active 2024-03-29T04:35:08.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:08 smithi142 ceph-mon[28654]: from='client.? 172.21.15.142:0/2719286601' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-29T04:35:08.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout cluster: 2024-03-29T04:35:08.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout id: 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:35:08.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-29T04:35:08.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 2024-03-29T04:35:08.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout services: 2024-03-29T04:35:08.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi142 (age 0.557804s) 2024-03-29T04:35:08.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-29T04:35:08.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-29T04:35:08.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 2024-03-29T04:35:08.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout data: 2024-03-29T04:35:08.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-29T04:35:08.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-29T04:35:08.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-29T04:35:08.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout pgs: 2024-03-29T04:35:08.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 2024-03-29T04:35:08.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.183+0000 7f1fa0307700 1 Processor -- start 2024-03-29T04:35:08.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.183+0000 7f1fa0307700 1 -- start start 2024-03-29T04:35:08.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.183+0000 7f1fa0307700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 0x7f1f98105240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:08.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.183+0000 7f1fa0307700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f98079e30 con 0x7f1f98104e20 2024-03-29T04:35:08.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.183+0000 7f1f9e0a3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 0x7f1f98105240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:08.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.183+0000 7f1f9e0a3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 0x7f1f98105240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54004/0 (socket says 172.21.15.142:54004) 2024-03-29T04:35:08.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.183+0000 7f1f9e0a3700 1 -- 172.21.15.142:0/3341135121 learned_addr learned my addr 172.21.15.142:0/3341135121 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:08.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.184+0000 7f1f9e0a3700 1 -- 172.21.15.142:0/3341135121 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98105780 con 0x7f1f98104e20 2024-03-29T04:35:08.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.184+0000 7f1f9e0a3700 1 --2- 172.21.15.142:0/3341135121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 0x7f1f98105240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1f940098a0 tx=0x7f1f94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e4ec2265023f88 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:08.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.184+0000 7f1f9d0a1700 1 -- 172.21.15.142:0/3341135121 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f94004180 con 0x7f1f98104e20 2024-03-29T04:35:08.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.184+0000 7f1f9d0a1700 1 -- 172.21.15.142:0/3341135121 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f940042e0 con 0x7f1f98104e20 2024-03-29T04:35:08.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.184+0000 7f1f9d0a1700 1 -- 172.21.15.142:0/3341135121 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f940036a0 con 0x7f1f98104e20 2024-03-29T04:35:08.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.184+0000 7f1fa0307700 1 -- 172.21.15.142:0/3341135121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 msgr2=0x7f1f98105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:08.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.184+0000 7f1fa0307700 1 --2- 172.21.15.142:0/3341135121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 0x7f1f98105240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1f940098a0 tx=0x7f1f94009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:08.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.185+0000 7f1fa0307700 1 -- 172.21.15.142:0/3341135121 shutdown_connections 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.185+0000 7f1fa0307700 1 --2- 172.21.15.142:0/3341135121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 0x7f1f98105240 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.185+0000 7f1fa0307700 1 -- 172.21.15.142:0/3341135121 >> 172.21.15.142:0/3341135121 conn(0x7f1f98100a20 msgr2=0x7f1f98102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.185+0000 7f1fa0307700 1 -- 172.21.15.142:0/3341135121 shutdown_connections 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.185+0000 7f1fa0307700 1 -- 172.21.15.142:0/3341135121 wait complete. 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.185+0000 7f1fa0307700 1 Processor -- start 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.185+0000 7f1fa0307700 1 -- start start 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.185+0000 7f1fa0307700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 0x7f1f9819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.185+0000 7f1fa0307700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9819c0b0 con 0x7f1f98104e20 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.185+0000 7f1f9e0a3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 0x7f1f9819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.185+0000 7f1f9e0a3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 0x7f1f9819bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54014/0 (socket says 172.21.15.142:54014) 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.185+0000 7f1f9e0a3700 1 -- 172.21.15.142:0/2719286601 learned_addr learned my addr 172.21.15.142:0/2719286601 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.186+0000 7f1f9e0a3700 1 -- 172.21.15.142:0/2719286601 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f94009550 con 0x7f1f98104e20 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.186+0000 7f1f9e0a3700 1 --2- 172.21.15.142:0/2719286601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 0x7f1f9819bb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1f98079270 tx=0x7f1f94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:08.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.186+0000 7f1f8f7fe700 1 -- 172.21.15.142:0/2719286601 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f94003ce0 con 0x7f1f98104e20 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.186+0000 7f1f8f7fe700 1 -- 172.21.15.142:0/2719286601 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f94020070 con 0x7f1f98104e20 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.186+0000 7f1fa0307700 1 -- 172.21.15.142:0/2719286601 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9819c2b0 con 0x7f1f98104e20 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.186+0000 7f1f8f7fe700 1 -- 172.21.15.142:0/2719286601 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f94011630 con 0x7f1f98104e20 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.186+0000 7f1fa0307700 1 -- 172.21.15.142:0/2719286601 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9819c750 con 0x7f1f98104e20 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.187+0000 7f1f8f7fe700 1 -- 172.21.15.142:0/2719286601 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1f94022030 con 0x7f1f98104e20 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.187+0000 7f1f8f7fe700 1 -- 172.21.15.142:0/2719286601 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1f94025d00 con 0x7f1f98104e20 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.187+0000 7f1fa0307700 1 -- 172.21.15.142:0/2719286601 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f981958b0 con 0x7f1f98104e20 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.189+0000 7f1f8f7fe700 1 -- 172.21.15.142:0/2719286601 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f1f94025480 con 0x7f1f98104e20 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.257+0000 7f1fa0307700 1 -- 172.21.15.142:0/2719286601 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f1f9802cee0 con 0x7f1f98104e20 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.257+0000 7f1f8f7fe700 1 -- 172.21.15.142:0/2719286601 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f1f94011c20 con 0x7f1f98104e20 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.258+0000 7f1fa0307700 1 -- 172.21.15.142:0/2719286601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 msgr2=0x7f1f9819bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.258+0000 7f1fa0307700 1 --2- 172.21.15.142:0/2719286601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 0x7f1f9819bb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1f98079270 tx=0x7f1f94004750 comp rx=0 tx=0).stop 2024-03-29T04:35:08.869 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.258+0000 7f1fa0307700 1 -- 172.21.15.142:0/2719286601 shutdown_connections 2024-03-29T04:35:08.870 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.258+0000 7f1fa0307700 1 --2- 172.21.15.142:0/2719286601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f98104e20 0x7f1f9819bb70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:08.870 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.258+0000 7f1fa0307700 1 -- 172.21.15.142:0/2719286601 >> 172.21.15.142:0/2719286601 conn(0x7f1f98100a20 msgr2=0x7f1f981929f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:08.870 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.258+0000 7f1fa0307700 1 -- 172.21.15.142:0/2719286601 shutdown_connections 2024-03-29T04:35:08.870 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:08.258+0000 7f1fa0307700 1 -- 172.21.15.142:0/2719286601 wait complete. 2024-03-29T04:35:08.870 INFO:teuthology.orchestra.run.smithi142.stdout:mon is available 2024-03-29T04:35:08.870 INFO:teuthology.orchestra.run.smithi142.stdout:Assimilating anything we can from ceph.conf... 2024-03-29T04:35:09.881 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 2024-03-29T04:35:09.882 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout [global] 2024-03-29T04:35:09.882 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout fsid = 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:35:09.882 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.142:3300,v1:172.21.15.142:6789] 2024-03-29T04:35:09.882 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-29T04:35:09.882 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-29T04:35:09.882 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-29T04:35:09.882 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-29T04:35:09.882 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 2024-03-29T04:35:09.882 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-29T04:35:09.882 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-29T04:35:09.882 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 2024-03-29T04:35:09.882 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout [osd] 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.295+0000 7f5ccdbab700 1 Processor -- start 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.295+0000 7f5ccdbab700 1 -- start start 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.295+0000 7f5ccdbab700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 0x7f5cc8104c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.295+0000 7f5ccdbab700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8105240 con 0x7f5cc8104850 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.295+0000 7f5cc77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 0x7f5cc8104c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.295+0000 7f5cc77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 0x7f5cc8104c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:38154/0 (socket says 172.21.15.142:38154) 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.295+0000 7f5cc77fe700 1 -- 172.21.15.142:0/2664783845 learned_addr learned my addr 172.21.15.142:0/2664783845 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.296+0000 7f5cc77fe700 1 -- 172.21.15.142:0/2664783845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc81077e0 con 0x7f5cc8104850 2024-03-29T04:35:09.883 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.296+0000 7f5cc77fe700 1 --2- 172.21.15.142:0/2664783845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 0x7f5cc8104c70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80098a0 tx=0x7f5cb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7394e9dea03c42 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.296+0000 7f5cc67fc700 1 -- 172.21.15.142:0/2664783845 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cb8004180 con 0x7f5cc8104850 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.296+0000 7f5cc67fc700 1 -- 172.21.15.142:0/2664783845 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb80042e0 con 0x7f5cc8104850 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.296+0000 7f5cc67fc700 1 -- 172.21.15.142:0/2664783845 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cb80036a0 con 0x7f5cc8104850 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.296+0000 7f5ccdbab700 1 -- 172.21.15.142:0/2664783845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 msgr2=0x7f5cc8104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.296+0000 7f5ccdbab700 1 --2- 172.21.15.142:0/2664783845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 0x7f5cc8104c70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80098a0 tx=0x7f5cb8009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.297+0000 7f5ccdbab700 1 -- 172.21.15.142:0/2664783845 shutdown_connections 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.297+0000 7f5ccdbab700 1 --2- 172.21.15.142:0/2664783845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 0x7f5cc8104c70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.297+0000 7f5ccdbab700 1 -- 172.21.15.142:0/2664783845 >> 172.21.15.142:0/2664783845 conn(0x7f5cc81001a0 msgr2=0x7f5cc81025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.297+0000 7f5ccdbab700 1 -- 172.21.15.142:0/2664783845 shutdown_connections 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.297+0000 7f5ccdbab700 1 -- 172.21.15.142:0/2664783845 wait complete. 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.297+0000 7f5ccdbab700 1 Processor -- start 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.297+0000 7f5ccdbab700 1 -- start start 2024-03-29T04:35:09.884 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5ccdbab700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 0x7f5cc8199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5ccdbab700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8199eb0 con 0x7f5cc8104850 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5cc77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 0x7f5cc8199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5cc77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 0x7f5cc8199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:38166/0 (socket says 172.21.15.142:38166) 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5cc77fe700 1 -- 172.21.15.142:0/1533476462 learned_addr learned my addr 172.21.15.142:0/1533476462 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5cc77fe700 1 -- 172.21.15.142:0/1533476462 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb8009550 con 0x7f5cc8104850 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5cc77fe700 1 --2- 172.21.15.142:0/1533476462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 0x7f5cc8199970 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8004000 tx=0x7f5cb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5cc4ff9700 1 -- 172.21.15.142:0/1533476462 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cb8003ce0 con 0x7f5cc8104850 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5cc4ff9700 1 -- 172.21.15.142:0/1533476462 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb8020070 con 0x7f5cc8104850 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5cc4ff9700 1 -- 172.21.15.142:0/1533476462 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cb8011630 con 0x7f5cc8104850 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5ccdbab700 1 -- 172.21.15.142:0/1533476462 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc819a0b0 con 0x7f5cc8104850 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5ccdbab700 1 -- 172.21.15.142:0/1533476462 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc819a550 con 0x7f5cc8104850 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.298+0000 7f5cc4ff9700 1 -- 172.21.15.142:0/1533476462 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5cb8022030 con 0x7f5cc8104850 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.299+0000 7f5cc4ff9700 1 -- 172.21.15.142:0/1533476462 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5cb8025d40 con 0x7f5cc8104850 2024-03-29T04:35:09.885 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.299+0000 7f5ccdbab700 1 -- 172.21.15.142:0/1533476462 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc8193290 con 0x7f5cc8104850 2024-03-29T04:35:09.886 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.301+0000 7f5cc4ff9700 1 -- 172.21.15.142:0/1533476462 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f5cb8025480 con 0x7f5cc8104850 2024-03-29T04:35:09.886 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.367+0000 7f5ccdbab700 1 -- 172.21.15.142:0/1533476462 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5cc802cf20 con 0x7f5cc8104850 2024-03-29T04:35:09.886 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.370+0000 7f5cc4ff9700 1 -- 172.21.15.142:0/1533476462 <== mon.0 v2:172.21.15.142:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5cb8011bd0 con 0x7f5cc8104850 2024-03-29T04:35:09.886 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.370+0000 7f5cc4ff9700 1 -- 172.21.15.142:0/1533476462 <== mon.0 v2:172.21.15.142:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f5cb8036020 con 0x7f5cc8104850 2024-03-29T04:35:09.886 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.371+0000 7f5ccdbab700 1 -- 172.21.15.142:0/1533476462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 msgr2=0x7f5cc8199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:09.886 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.371+0000 7f5ccdbab700 1 --2- 172.21.15.142:0/1533476462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 0x7f5cc8199970 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8004000 tx=0x7f5cb8004750 comp rx=0 tx=0).stop 2024-03-29T04:35:09.886 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.371+0000 7f5ccdbab700 1 -- 172.21.15.142:0/1533476462 shutdown_connections 2024-03-29T04:35:09.886 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.371+0000 7f5ccdbab700 1 --2- 172.21.15.142:0/1533476462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8104850 0x7f5cc8199970 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:09.886 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.371+0000 7f5ccdbab700 1 -- 172.21.15.142:0/1533476462 >> 172.21.15.142:0/1533476462 conn(0x7f5cc81001a0 msgr2=0x7f5cc818dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:09.886 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.372+0000 7f5ccdbab700 1 -- 172.21.15.142:0/1533476462 shutdown_connections 2024-03-29T04:35:09.886 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:09.372+0000 7f5ccdbab700 1 -- 172.21.15.142:0/1533476462 wait complete. 2024-03-29T04:35:09.886 INFO:teuthology.orchestra.run.smithi142.stdout:Generating new minimal ceph.conf... 2024-03-29T04:35:09.989 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:09 smithi142 ceph-mon[28654]: from='client.? 172.21.15.142:0/1533476462' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-29T04:35:09.989 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:09 smithi142 ceph-mon[28654]: from='client.? 172.21.15.142:0/1533476462' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-29T04:35:11.106 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:10 smithi142 ceph-mon[28654]: from='client.? 172.21.15.142:0/865186417' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:35:11.680 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.311+0000 7f38761ca700 1 Processor -- start 2024-03-29T04:35:11.680 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.311+0000 7f38761ca700 1 -- start start 2024-03-29T04:35:11.681 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.311+0000 7f38761ca700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 0x7f3870105d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:11.681 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.311+0000 7f38761ca700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3870106320 con 0x7f3870105930 2024-03-29T04:35:11.681 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.311+0000 7f386f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 0x7f3870105d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:11.681 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.311+0000 7f386f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 0x7f3870105d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:38178/0 (socket says 172.21.15.142:38178) 2024-03-29T04:35:11.681 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.311+0000 7f386f7fe700 1 -- 172.21.15.142:0/344143140 learned_addr learned my addr 172.21.15.142:0/344143140 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:11.681 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.312+0000 7f386f7fe700 1 -- 172.21.15.142:0/344143140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3870106650 con 0x7f3870105930 2024-03-29T04:35:11.681 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.312+0000 7f386f7fe700 1 --2- 172.21.15.142:0/344143140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 0x7f3870105d50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f386000bb40 tx=0x7f386000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88b1d5e8841a9e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:11.681 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.312+0000 7f386e7fc700 1 -- 172.21.15.142:0/344143140 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f386000dc00 con 0x7f3870105930 2024-03-29T04:35:11.681 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.312+0000 7f386e7fc700 1 -- 172.21.15.142:0/344143140 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38600044a0 con 0x7f3870105930 2024-03-29T04:35:11.681 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.312+0000 7f386e7fc700 1 -- 172.21.15.142:0/344143140 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3860010470 con 0x7f3870105930 2024-03-29T04:35:11.681 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.312+0000 7f38761ca700 1 -- 172.21.15.142:0/344143140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 msgr2=0x7f3870105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:11.682 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.312+0000 7f38761ca700 1 --2- 172.21.15.142:0/344143140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 0x7f3870105d50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f386000bb40 tx=0x7f386000be50 comp rx=0 tx=0).stop 2024-03-29T04:35:11.682 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.312+0000 7f38761ca700 1 -- 172.21.15.142:0/344143140 shutdown_connections 2024-03-29T04:35:11.682 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.312+0000 7f38761ca700 1 --2- 172.21.15.142:0/344143140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 0x7f3870105d50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:11.682 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.312+0000 7f38761ca700 1 -- 172.21.15.142:0/344143140 >> 172.21.15.142:0/344143140 conn(0x7f3870100110 msgr2=0x7f3870102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:11.682 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.312+0000 7f38761ca700 1 -- 172.21.15.142:0/344143140 shutdown_connections 2024-03-29T04:35:11.682 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.312+0000 7f38761ca700 1 -- 172.21.15.142:0/344143140 wait complete. 2024-03-29T04:35:11.682 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.313+0000 7f38761ca700 1 Processor -- start 2024-03-29T04:35:11.682 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.313+0000 7f38761ca700 1 -- start start 2024-03-29T04:35:11.682 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.313+0000 7f38761ca700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 0x7f3870197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:11.682 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.313+0000 7f38761ca700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38701984b0 con 0x7f3870105930 2024-03-29T04:35:11.682 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.314+0000 7f386f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 0x7f3870197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:11.683 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.314+0000 7f386f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 0x7f3870197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:38188/0 (socket says 172.21.15.142:38188) 2024-03-29T04:35:11.683 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.314+0000 7f386f7fe700 1 -- 172.21.15.142:0/865186417 learned_addr learned my addr 172.21.15.142:0/865186417 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:11.683 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.314+0000 7f386f7fe700 1 -- 172.21.15.142:0/865186417 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386000b7f0 con 0x7f3870105930 2024-03-29T04:35:11.683 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.314+0000 7f386f7fe700 1 --2- 172.21.15.142:0/865186417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 0x7f3870197f70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f38600046d0 tx=0x7f3860003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:11.683 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.314+0000 7f386cff9700 1 -- 172.21.15.142:0/865186417 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f386001e070 con 0x7f3870105930 2024-03-29T04:35:11.683 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.314+0000 7f386cff9700 1 -- 172.21.15.142:0/865186417 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3860004260 con 0x7f3870105930 2024-03-29T04:35:11.683 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.314+0000 7f386cff9700 1 -- 172.21.15.142:0/865186417 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38600229c0 con 0x7f3870105930 2024-03-29T04:35:11.683 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.314+0000 7f38761ca700 1 -- 172.21.15.142:0/865186417 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38701986b0 con 0x7f3870105930 2024-03-29T04:35:11.683 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.314+0000 7f38761ca700 1 -- 172.21.15.142:0/865186417 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3870198b50 con 0x7f3870105930 2024-03-29T04:35:11.683 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.314+0000 7f386cff9700 1 -- 172.21.15.142:0/865186417 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f386002c740 con 0x7f3870105930 2024-03-29T04:35:11.683 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.314+0000 7f386cff9700 1 -- 172.21.15.142:0/865186417 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3860019970 con 0x7f3870105930 2024-03-29T04:35:11.684 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.315+0000 7f38761ca700 1 -- 172.21.15.142:0/865186417 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3870191ce0 con 0x7f3870105930 2024-03-29T04:35:11.684 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.317+0000 7f386cff9700 1 -- 172.21.15.142:0/865186417 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f3860027070 con 0x7f3870105930 2024-03-29T04:35:11.684 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.382+0000 7f38761ca700 1 -- 172.21.15.142:0/865186417 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f38700623c0 con 0x7f3870105930 2024-03-29T04:35:11.684 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.382+0000 7f386cff9700 1 -- 172.21.15.142:0/865186417 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f38700623c0 con 0x7f3870105930 2024-03-29T04:35:11.684 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.383+0000 7f38761ca700 1 -- 172.21.15.142:0/865186417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 msgr2=0x7f3870197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:11.684 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.383+0000 7f38761ca700 1 --2- 172.21.15.142:0/865186417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 0x7f3870197f70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f38600046d0 tx=0x7f3860003d70 comp rx=0 tx=0).stop 2024-03-29T04:35:11.684 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.383+0000 7f38761ca700 1 -- 172.21.15.142:0/865186417 shutdown_connections 2024-03-29T04:35:11.684 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.383+0000 7f38761ca700 1 --2- 172.21.15.142:0/865186417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3870105930 0x7f3870197f70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:11.684 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.383+0000 7f38761ca700 1 -- 172.21.15.142:0/865186417 >> 172.21.15.142:0/865186417 conn(0x7f3870100110 msgr2=0x7f387018ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:11.684 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.383+0000 7f38761ca700 1 -- 172.21.15.142:0/865186417 shutdown_connections 2024-03-29T04:35:11.684 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:10.383+0000 7f38761ca700 1 -- 172.21.15.142:0/865186417 wait complete. 2024-03-29T04:35:11.685 INFO:teuthology.orchestra.run.smithi142.stdout:Restarting the monitor... 2024-03-29T04:35:11.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:11 smithi142 systemd[1]: Stopping Ceph mon.smithi142 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:35:12.356 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:11 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[28650]: 2024-03-29T04:35:11.965+0000 7fd2a4fe8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi142 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T04:35:12.356 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:11 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[28650]: 2024-03-29T04:35:11.965+0000 7fd2a4fe8700 -1 mon.smithi142@0(leader) e1 *** Got Signal Terminated *** 2024-03-29T04:35:12.356 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:12 smithi142 podman[28900]: 2024-03-29 04:35:12.160166554 +0000 UTC m=+0.301562569 container died 34d24d6d8873b496bc710e3edb4ba17f1e04ef6bcd54c0621c7b3744ac28081f (image=quay.io/ceph/ceph:v18.2.1, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-03-29T04:35:12.675 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:12 smithi142 podman[28900]: 2024-03-29 04:35:12.48221128 +0000 UTC m=+0.623607294 container cleanup 34d24d6d8873b496bc710e3edb4ba17f1e04ef6bcd54c0621c7b3744ac28081f (image=quay.io/ceph/ceph:v18.2.1, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD) 2024-03-29T04:35:12.676 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:12 smithi142 bash[28900]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142 2024-03-29T04:35:12.949 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:12 smithi142 podman[28915]: 2024-03-29 04:35:12.691433466 +0000 UTC m=+0.528667065 container remove 34d24d6d8873b496bc710e3edb4ba17f1e04ef6bcd54c0621c7b3744ac28081f (image=quay.io/ceph/ceph:v18.2.1, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-29T04:35:12.949 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:12 smithi142 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi142.service: Deactivated successfully. 2024-03-29T04:35:12.949 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:12 smithi142 systemd[1]: Stopped Ceph mon.smithi142 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:35:12.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:12 smithi142 systemd[1]: Starting Ceph mon.smithi142 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:35:13.219 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 podman[29008]: 2024-03-29 04:35:13.01448604 +0000 UTC m=+0.017779930 image pull 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf quay.io/ceph/ceph:v18.2.1 2024-03-29T04:35:13.219 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 podman[29008]: 2024-03-29 04:35:13.125908197 +0000 UTC m=+0.129202080 container create 32c014de54798a63a71db248997c8b52a5423a4f5793bf0b1e5fca634ceb7fad (image=quay.io/ceph/ceph:v18.2.1, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, RELEASE=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-03-29T04:35:13.374 INFO:teuthology.orchestra.run.smithi142.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-29T04:35:13.499 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 podman[29008]: 2024-03-29 04:35:13.311433092 +0000 UTC m=+0.314726982 container init 32c014de54798a63a71db248997c8b52a5423a4f5793bf0b1e5fca634ceb7fad (image=quay.io/ceph/ceph:v18.2.1, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=HEAD, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222) 2024-03-29T04:35:13.499 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 podman[29008]: 2024-03-29 04:35:13.314236807 +0000 UTC m=+0.317530687 container start 32c014de54798a63a71db248997c8b52a5423a4f5793bf0b1e5fca634ceb7fad (image=quay.io/ceph/ceph:v18.2.1, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, RELEASE=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-03-29T04:35:13.499 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: set uid:gid to 167:167 (ceph:ceph) 2024-03-29T04:35:13.499 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-03-29T04:35:13.499 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: pidfile_write: ignore empty --pid-file 2024-03-29T04:35:13.499 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: load: jerasure load: lrc 2024-03-29T04:35:13.499 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: RocksDB version: 7.9.2 2024-03-29T04:35:13.499 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Git sha 0 2024-03-29T04:35:13.499 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Compile date 2023-12-11 22:07:34 2024-03-29T04:35:13.499 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: DB SUMMARY 2024-03-29T04:35:13.499 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: DB Session ID: ZWW83C7FAQ2QAQFZQ63A 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: CURRENT file: CURRENT 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: IDENTITY file: IDENTITY 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi142/store.db dir, Total Num: 1, files: 000008.sst 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi142/store.db: 000009.log size: 98387 ; 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.error_if_exists: 0 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.create_if_missing: 0 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.paranoid_checks: 1 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.env: 0x55a8a99e8720 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.fs: PosixFileSystem 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.info_log: 0x55a8ab6c52e0 2024-03-29T04:35:13.500 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_file_opening_threads: 16 2024-03-29T04:35:13.501 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.statistics: (nil) 2024-03-29T04:35:13.501 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.use_fsync: 0 2024-03-29T04:35:13.502 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_log_file_size: 0 2024-03-29T04:35:13.502 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-29T04:35:13.502 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-29T04:35:13.502 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.keep_log_file_num: 1000 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.recycle_log_file_num: 0 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.allow_fallocate: 1 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.allow_mmap_reads: 0 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.allow_mmap_writes: 0 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.use_direct_reads: 0 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.create_missing_column_families: 0 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.db_log_dir: 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.wal_dir: 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-29T04:35:13.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.advise_random_on_open: 1 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.db_write_buffer_size: 0 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.write_buffer_manager: 0x55a8aa936140 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.rate_limiter: (nil) 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.wal_recovery_mode: 2 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.enable_thread_tracking: 0 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.enable_pipelined_write: 0 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.unordered_write: 0 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-29T04:35:13.504 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.row_cache: None 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.wal_filter: None 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.allow_ingest_behind: 0 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.two_write_queues: 0 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.manual_wal_flush: 0 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.wal_compression: 0 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.atomic_flush: 0 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-29T04:35:13.505 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.log_readahead_size: 0 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.best_efforts_recovery: 0 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.allow_data_in_errors: 0 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.db_host_id: __hostname__ 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_background_jobs: 2 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_background_compactions: -1 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_subcompactions: 1 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-29T04:35:13.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_total_wal_size: 0 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_open_files: -1 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bytes_per_sync: 0 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_readahead_size: 0 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_background_flushes: -1 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Compression algorithms supported: 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: kZSTD supported: 0 2024-03-29T04:35:13.507 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: kXpressCompression supported: 0 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: kLZ4HCCompression supported: 1 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: kZlibCompression supported: 1 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: kSnappyCompression supported: 1 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: kLZ4Compression supported: 1 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: kBZip2Compression supported: 0 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi142/store.db/MANIFEST-000010 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.merge_operator: 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_filter: None 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_filter_factory: None 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.sst_partitioner_factory: None 2024-03-29T04:35:13.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a8ab6c53e0) 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: cache_index_and_filter_blocks: 1 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: pin_top_level_index_and_filter: 1 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: index_type: 0 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: data_block_index_type: 0 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: index_shortening: 1 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: checksum: 4 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: no_block_cache: 0 2024-03-29T04:35:13.509 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache: 0x55a8aa9d71f0 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache_name: BinnedLRUCache 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache_options: 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: capacity : 536870912 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: num_shard_bits : 4 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: strict_capacity_limit : 0 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: high_pri_pool_ratio: 0.000 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache_compressed: (nil) 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: persistent_cache: (nil) 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_size: 4096 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_size_deviation: 10 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_restart_interval: 16 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: index_block_restart_interval: 1 2024-03-29T04:35:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: metadata_block_size: 4096 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: partition_filters: 0 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: use_delta_encoding: 1 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: filter_policy: bloomfilter 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: whole_key_filtering: 1 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: verify_compression: 0 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: read_amp_bytes_per_bit: 0 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: format_version: 5 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: enable_index_compression: 1 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_align: 0 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: max_auto_readahead_size: 262144 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: prepopulate_block_cache: 0 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: initial_auto_readahead_size: 8192 2024-03-29T04:35:13.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.write_buffer_size: 33554432 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_write_buffer_number: 2 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compression: NoCompression 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bottommost_compression: Disabled 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.prefix_extractor: nullptr 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.num_levels: 7 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-29T04:35:13.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compression_opts.level: 32767 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compression_opts.strategy: 0 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compression_opts.enabled: false 2024-03-29T04:35:13.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.target_file_size_base: 67108864 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-29T04:35:13.514 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.arena_block_size: 1048576 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.disable_auto_compactions: 0 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-29T04:35:13.515 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.table_properties_collectors: 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.inplace_update_support: 0 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.bloom_locality: 0 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.max_successive_merges: 0 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-29T04:35:13.516 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.paranoid_file_checks: 0 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.force_consistency_checks: 1 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.report_bg_io_stats: 0 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.ttl: 2592000 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.enable_blob_files: false 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.min_blob_size: 0 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.blob_file_size: 268435456 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-29T04:35:13.517 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.blob_file_starting_level: 0 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi142/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 63462697-da50-4f71-bff3-8fd7e1db5fe3 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711686913342829, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711686913343833, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93631, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91653, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84262, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711686913, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "63462697-da50-4f71-bff3-8fd7e1db5fe3", "db_session_id": "ZWW83C7FAQ2QAQFZQ63A", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711686913343913, "job": 1, "event": "recovery_finished"} 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi142/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a8aaa74000 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: DB pointer 0x55a8aaa60000 2024-03-29T04:35:13.518 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** DB Stats ** 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** Compaction Stats [default] ** 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-29T04:35:13.519 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: L0 2/0 93.31 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Sum 2/0 93.31 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** Compaction Stats [default] ** 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-29T04:35:13.520 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative compaction: 0.00 GB write, 30.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval compaction: 0.00 GB write, 30.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Block cache BinnedLRUCache@0x55a8aa9d71f0#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: starting mon.smithi142 rank 0 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi142 fsid 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: mon.smithi142@-1(???) e1 preinit fsid 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: mon.smithi142@-1(???).mds e1 new map 2024-03-29T04:35:13.521 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: mon.smithi142@-1(???).mds e1 print_map 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: e1 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: legacy client fscid: -1 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: No filesystems configured 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: mon.smithi142@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: mon.smithi142@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: mon.smithi142@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: mon.smithi142@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: mon.smithi142@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: expand_channel_meta expand map: {default=false} 2024-03-29T04:35:13.522 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: expand_channel_meta from 'false' to 'false' 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: expand_channel_meta expanded map: {default=false} 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: expand_channel_meta expand map: {default=info} 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: expand_channel_meta from 'info' to 'info' 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: expand_channel_meta expanded map: {default=info} 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: expand_channel_meta expand map: {default=daemon} 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: expand_channel_meta from 'daemon' to 'daemon' 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: expand_channel_meta expanded map: {default=daemon} 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: expand_channel_meta expand map: {default=debug} 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: expand_channel_meta from 'debug' to 'debug' 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: expand_channel_meta expanded map: {default=debug} 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 bash[29008]: 32c014de54798a63a71db248997c8b52a5423a4f5793bf0b1e5fca634ceb7fad 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 systemd[1]: Started Ceph mon.smithi142 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: mon.smithi142 is new leader, mons smithi142 in quorum (ranks 0) 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: monmap e1: 1 mons at {smithi142=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-29T04:35:13.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: fsmap 2024-03-29T04:35:13.524 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: osdmap e1: 0 total, 0 up, 0 in 2024-03-29T04:35:13.524 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:13 smithi142 ceph-mon[29027]: mgrmap e1: no daemons active 2024-03-29T04:35:14.545 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.883+0000 7f0589c43700 1 Processor -- start 2024-03-29T04:35:14.545 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.883+0000 7f0589c43700 1 -- start start 2024-03-29T04:35:14.545 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.883+0000 7f0589c43700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 0x7f0584105840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:14.545 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.883+0000 7f0589c43700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0584105e10 con 0x7f0584105420 2024-03-29T04:35:14.545 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.883+0000 7f05837fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 0x7f0584105840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:14.545 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.883+0000 7f05837fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 0x7f0584105840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:38202/0 (socket says 172.21.15.142:38202) 2024-03-29T04:35:14.545 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.883+0000 7f05837fe700 1 -- 172.21.15.142:0/2339387559 learned_addr learned my addr 172.21.15.142:0/2339387559 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.884+0000 7f05837fe700 1 -- 172.21.15.142:0/2339387559 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0584106620 con 0x7f0584105420 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.884+0000 7f05837fe700 1 --2- 172.21.15.142:0/2339387559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 0x7f0584105840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f05740098a0 tx=0x7f0574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62a663eda99657 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.884+0000 7f05827fc700 1 -- 172.21.15.142:0/2339387559 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0574004180 con 0x7f0584105420 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.884+0000 7f05827fc700 1 -- 172.21.15.142:0/2339387559 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f05740042e0 con 0x7f0584105420 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.884+0000 7f05827fc700 1 -- 172.21.15.142:0/2339387559 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0574011a50 con 0x7f0584105420 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.884+0000 7f0589c43700 1 -- 172.21.15.142:0/2339387559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 msgr2=0x7f0584105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.884+0000 7f0589c43700 1 --2- 172.21.15.142:0/2339387559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 0x7f0584105840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f05740098a0 tx=0x7f0574009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.885+0000 7f0589c43700 1 -- 172.21.15.142:0/2339387559 shutdown_connections 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.885+0000 7f0589c43700 1 --2- 172.21.15.142:0/2339387559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 0x7f0584105840 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.885+0000 7f0589c43700 1 -- 172.21.15.142:0/2339387559 >> 172.21.15.142:0/2339387559 conn(0x7f0584100980 msgr2=0x7f0584102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.885+0000 7f0589c43700 1 -- 172.21.15.142:0/2339387559 shutdown_connections 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.885+0000 7f0589c43700 1 -- 172.21.15.142:0/2339387559 wait complete. 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.885+0000 7f0589c43700 1 Processor -- start 2024-03-29T04:35:14.546 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.885+0000 7f0589c43700 1 -- start start 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f0589c43700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 0x7f058419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f0589c43700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058419a760 con 0x7f0584105420 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f05837fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 0x7f058419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f05837fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 0x7f058419a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:38210/0 (socket says 172.21.15.142:38210) 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f05837fe700 1 -- 172.21.15.142:0/1457653474 learned_addr learned my addr 172.21.15.142:0/1457653474 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f05837fe700 1 -- 172.21.15.142:0/1457653474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0574009550 con 0x7f0584105420 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f05837fe700 1 --2- 172.21.15.142:0/1457653474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 0x7f058419a220 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0574003de0 tx=0x7f0574011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f0580ff9700 1 -- 172.21.15.142:0/1457653474 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0574003930 con 0x7f0584105420 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f0580ff9700 1 -- 172.21.15.142:0/1457653474 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0574003a90 con 0x7f0584105420 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f0589c43700 1 -- 172.21.15.142:0/1457653474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058419a960 con 0x7f0584105420 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f0580ff9700 1 -- 172.21.15.142:0/1457653474 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05740196c0 con 0x7f0584105420 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f0589c43700 1 -- 172.21.15.142:0/1457653474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058419ae00 con 0x7f0584105420 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.886+0000 7f0580ff9700 1 -- 172.21.15.142:0/1457653474 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0574019c50 con 0x7f0584105420 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.887+0000 7f0580ff9700 1 -- 172.21.15.142:0/1457653474 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f057402c3a0 con 0x7f0584105420 2024-03-29T04:35:14.547 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.887+0000 7f0589c43700 1 -- 172.21.15.142:0/1457653474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0584193f20 con 0x7f0584105420 2024-03-29T04:35:14.548 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.890+0000 7f0580ff9700 1 -- 172.21.15.142:0/1457653474 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f057401abc0 con 0x7f0584105420 2024-03-29T04:35:14.548 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.955+0000 7f0589c43700 1 -- 172.21.15.142:0/1457653474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f05840619a0 con 0x7f0584105420 2024-03-29T04:35:14.548 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.957+0000 7f0580ff9700 1 -- 172.21.15.142:0/1457653474 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0574005a20 con 0x7f0584105420 2024-03-29T04:35:14.548 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.958+0000 7f0589c43700 1 -- 172.21.15.142:0/1457653474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 msgr2=0x7f058419a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:14.548 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.958+0000 7f0589c43700 1 --2- 172.21.15.142:0/1457653474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 0x7f058419a220 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0574003de0 tx=0x7f0574011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:14.548 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.958+0000 7f0589c43700 1 -- 172.21.15.142:0/1457653474 shutdown_connections 2024-03-29T04:35:14.548 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.958+0000 7f0589c43700 1 --2- 172.21.15.142:0/1457653474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0584105420 0x7f058419a220 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:14.548 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.958+0000 7f0589c43700 1 -- 172.21.15.142:0/1457653474 >> 172.21.15.142:0/1457653474 conn(0x7f0584100980 msgr2=0x7f0584101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:14.548 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.958+0000 7f0589c43700 1 -- 172.21.15.142:0/1457653474 shutdown_connections 2024-03-29T04:35:14.548 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:13.958+0000 7f0589c43700 1 -- 172.21.15.142:0/1457653474 wait complete. 2024-03-29T04:35:14.548 INFO:teuthology.orchestra.run.smithi142.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-29T04:35:14.570 INFO:teuthology.orchestra.run.smithi142.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-29T04:35:14.571 INFO:teuthology.orchestra.run.smithi142.stdout:Creating mgr... 2024-03-29T04:35:14.571 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-29T04:35:14.572 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-29T04:35:14.572 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying port 0.0.0.0:8443 ... 2024-03-29T04:35:14.821 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mgr.smithi142.swesrg 2024-03-29T04:35:14.822 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mgr.smithi142.swesrg.service: Unit ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mgr.smithi142.swesrg.service not loaded. 2024-03-29T04:35:14.963 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9f96302c-ed85-11ee-95d0-87774f69a715.target.wants/ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mgr.smithi142.swesrg.service → /etc/systemd/system/ceph-9f96302c-ed85-11ee-95d0-87774f69a715@.service. 2024-03-29T04:35:15.196 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:14 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1457653474' entity='client.admin' 2024-03-29T04:35:15.530 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-29T04:35:15.531 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout disabled 2024-03-29T04:35:15.537 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-29T04:35:15.537 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout inactive 2024-03-29T04:35:15.537 INFO:teuthology.orchestra.run.smithi142.stdout:firewalld.service is not enabled 2024-03-29T04:35:15.538 INFO:teuthology.orchestra.run.smithi142.stdout:Not possible to enable service . firewalld.service is not available 2024-03-29T04:35:15.545 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-29T04:35:15.546 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout disabled 2024-03-29T04:35:15.553 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-29T04:35:15.553 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout inactive 2024-03-29T04:35:15.553 INFO:teuthology.orchestra.run.smithi142.stdout:firewalld.service is not enabled 2024-03-29T04:35:15.553 INFO:teuthology.orchestra.run.smithi142.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-03-29T04:35:15.553 INFO:teuthology.orchestra.run.smithi142.stdout:Waiting for mgr to start... 2024-03-29T04:35:15.553 INFO:teuthology.orchestra.run.smithi142.stdout:Waiting for mgr... 2024-03-29T04:35:16.356 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:16 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/976804796' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-29T04:35:18.161 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 2024-03-29T04:35:18.161 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout { 2024-03-29T04:35:18.161 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "fsid": "9f96302c-ed85-11ee-95d0-87774f69a715", 2024-03-29T04:35:18.161 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "health": { 2024-03-29T04:35:18.161 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-29T04:35:18.161 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-29T04:35:18.161 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-29T04:35:18.161 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:18.161 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-29T04:35:18.161 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-29T04:35:18.161 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 0 2024-03-29T04:35:18.161 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout ], 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "smithi142" 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout ], 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-29T04:35:18.162 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-29T04:35:18.170 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-29T04:35:18.171 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "restful" 2024-03-29T04:35:18.172 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout ], 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "modified": "2024-03-29T04:35:07.701762+0000", 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout } 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.027+0000 7f9d07b55700 1 Processor -- start 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.027+0000 7f9d07b55700 1 -- start start 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.027+0000 7f9d07b55700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 0x7f9d00105890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.027+0000 7f9d07b55700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d00105e60 con 0x7f9d00105470 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.028+0000 7f9d058f1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 0x7f9d00105890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.028+0000 7f9d058f1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 0x7f9d00105890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:38244/0 (socket says 172.21.15.142:38244) 2024-03-29T04:35:18.174 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.028+0000 7f9d058f1700 1 -- 172.21.15.142:0/3358278456 learned_addr learned my addr 172.21.15.142:0/3358278456 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:18.174 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.028+0000 7f9d058f1700 1 -- 172.21.15.142:0/3358278456 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00106670 con 0x7f9d00105470 2024-03-29T04:35:18.174 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.028+0000 7f9d058f1700 1 --2- 172.21.15.142:0/3358278456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 0x7f9d00105890 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0098a0 tx=0x7f9cfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0c8efe74cef408 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:18.174 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.028+0000 7f9d048ef700 1 -- 172.21.15.142:0/3358278456 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cfc004180 con 0x7f9d00105470 2024-03-29T04:35:18.174 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.028+0000 7f9d048ef700 1 -- 172.21.15.142:0/3358278456 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9cfc0042e0 con 0x7f9d00105470 2024-03-29T04:35:18.174 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.029+0000 7f9d048ef700 1 -- 172.21.15.142:0/3358278456 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cfc011a50 con 0x7f9d00105470 2024-03-29T04:35:18.174 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.029+0000 7f9d07b55700 1 -- 172.21.15.142:0/3358278456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 msgr2=0x7f9d00105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:18.174 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.029+0000 7f9d07b55700 1 --2- 172.21.15.142:0/3358278456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 0x7f9d00105890 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0098a0 tx=0x7f9cfc009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:18.174 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.029+0000 7f9d07b55700 1 -- 172.21.15.142:0/3358278456 shutdown_connections 2024-03-29T04:35:18.174 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.029+0000 7f9d07b55700 1 --2- 172.21.15.142:0/3358278456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 0x7f9d00105890 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:18.174 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.029+0000 7f9d07b55700 1 -- 172.21.15.142:0/3358278456 >> 172.21.15.142:0/3358278456 conn(0x7f9d001009f0 msgr2=0x7f9d00102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:18.174 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.029+0000 7f9d07b55700 1 -- 172.21.15.142:0/3358278456 shutdown_connections 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.029+0000 7f9d07b55700 1 -- 172.21.15.142:0/3358278456 wait complete. 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.030+0000 7f9d07b55700 1 Processor -- start 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.030+0000 7f9d07b55700 1 -- start start 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.030+0000 7f9d07b55700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 0x7f9d00074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.030+0000 7f9d07b55700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0019acd0 con 0x7f9d00105470 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.030+0000 7f9d058f1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 0x7f9d00074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.030+0000 7f9d058f1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 0x7f9d00074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:38250/0 (socket says 172.21.15.142:38250) 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.030+0000 7f9d058f1700 1 -- 172.21.15.142:0/976804796 learned_addr learned my addr 172.21.15.142:0/976804796 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.030+0000 7f9d058f1700 1 -- 172.21.15.142:0/976804796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc009550 con 0x7f9d00105470 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.030+0000 7f9d058f1700 1 --2- 172.21.15.142:0/976804796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 0x7f9d00074300 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc000c00 tx=0x7f9cfc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.031+0000 7f9cf6ffd700 1 -- 172.21.15.142:0/976804796 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cfc003b90 con 0x7f9d00105470 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.031+0000 7f9cf6ffd700 1 -- 172.21.15.142:0/976804796 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9cfc0248b0 con 0x7f9d00105470 2024-03-29T04:35:18.175 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.031+0000 7f9d07b55700 1 -- 172.21.15.142:0/976804796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0019f2e0 con 0x7f9d00105470 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.031+0000 7f9cf6ffd700 1 -- 172.21.15.142:0/976804796 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cfc01a8e0 con 0x7f9d00105470 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.031+0000 7f9d07b55700 1 -- 172.21.15.142:0/976804796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0019f640 con 0x7f9d00105470 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.031+0000 7f9cf6ffd700 1 -- 172.21.15.142:0/976804796 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9cfc02d3f0 con 0x7f9d00105470 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.031+0000 7f9cf6ffd700 1 -- 172.21.15.142:0/976804796 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9cfc019400 con 0x7f9d00105470 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.032+0000 7f9d07b55700 1 -- 172.21.15.142:0/976804796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d000623c0 con 0x7f9d00105470 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.034+0000 7f9cf6ffd700 1 -- 172.21.15.142:0/976804796 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f9cfc024a20 con 0x7f9d00105470 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.108+0000 7f9d07b55700 1 -- 172.21.15.142:0/976804796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9d001098a0 con 0x7f9d00105470 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.108+0000 7f9cf6ffd700 1 -- 172.21.15.142:0/976804796 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f9cfc035090 con 0x7f9d00105470 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.109+0000 7f9d07b55700 1 -- 172.21.15.142:0/976804796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 msgr2=0x7f9d00074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.109+0000 7f9d07b55700 1 --2- 172.21.15.142:0/976804796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 0x7f9d00074300 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc000c00 tx=0x7f9cfc011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.109+0000 7f9d07b55700 1 -- 172.21.15.142:0/976804796 shutdown_connections 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.109+0000 7f9d07b55700 1 --2- 172.21.15.142:0/976804796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d00105470 0x7f9d00074300 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.109+0000 7f9d07b55700 1 -- 172.21.15.142:0/976804796 >> 172.21.15.142:0/976804796 conn(0x7f9d001009f0 msgr2=0x7f9d00101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.109+0000 7f9d07b55700 1 -- 172.21.15.142:0/976804796 shutdown_connections 2024-03-29T04:35:18.177 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:16.109+0000 7f9d07b55700 1 -- 172.21.15.142:0/976804796 wait complete. 2024-03-29T04:35:18.177 INFO:teuthology.orchestra.run.smithi142.stdout:mgr not available, waiting (1/15)... 2024-03-29T04:35:20.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:20 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/73541290' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-29T04:35:21.292 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 2024-03-29T04:35:21.292 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout { 2024-03-29T04:35:21.292 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "fsid": "9f96302c-ed85-11ee-95d0-87774f69a715", 2024-03-29T04:35:21.292 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "health": { 2024-03-29T04:35:21.292 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-29T04:35:21.292 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-29T04:35:21.292 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-29T04:35:21.292 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 0 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout ], 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "smithi142" 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout ], 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-29T04:35:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T04:35:21.294 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-29T04:35:21.294 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-29T04:35:21.294 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-29T04:35:21.302 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-29T04:35:21.302 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-29T04:35:21.302 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-29T04:35:21.302 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:21.302 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-29T04:35:21.302 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-29T04:35:21.302 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-29T04:35:21.302 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-29T04:35:21.302 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-29T04:35:21.302 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-29T04:35:21.303 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "restful" 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout ], 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "modified": "2024-03-29T04:35:07.701762+0000", 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout } 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.631+0000 7fc304499700 1 Processor -- start 2024-03-29T04:35:21.304 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.631+0000 7fc304499700 1 -- start start 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.631+0000 7fc304499700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 0x7fc2fc106e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.631+0000 7fc304499700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc107430 con 0x7fc2fc106a40 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.631+0000 7fc302235700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 0x7fc2fc106e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.631+0000 7fc302235700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 0x7fc2fc106e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42510/0 (socket says 172.21.15.142:42510) 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.631+0000 7fc302235700 1 -- 172.21.15.142:0/3157016039 learned_addr learned my addr 172.21.15.142:0/3157016039 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.632+0000 7fc302235700 1 -- 172.21.15.142:0/3157016039 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc107570 con 0x7fc2fc106a40 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.632+0000 7fc302235700 1 --2- 172.21.15.142:0/3157016039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 0x7fc2fc106e60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80098a0 tx=0x7fc2f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727ce89e0325d55a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.632+0000 7fc301233700 1 -- 172.21.15.142:0/3157016039 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2f8004180 con 0x7fc2fc106a40 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.632+0000 7fc301233700 1 -- 172.21.15.142:0/3157016039 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2f80042e0 con 0x7fc2fc106a40 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.632+0000 7fc301233700 1 -- 172.21.15.142:0/3157016039 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2f8011a50 con 0x7fc2fc106a40 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.633+0000 7fc304499700 1 -- 172.21.15.142:0/3157016039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 msgr2=0x7fc2fc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.633+0000 7fc304499700 1 --2- 172.21.15.142:0/3157016039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 0x7fc2fc106e60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80098a0 tx=0x7fc2f8009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.633+0000 7fc304499700 1 -- 172.21.15.142:0/3157016039 shutdown_connections 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.633+0000 7fc304499700 1 --2- 172.21.15.142:0/3157016039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 0x7fc2fc106e60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:21.305 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.633+0000 7fc304499700 1 -- 172.21.15.142:0/3157016039 >> 172.21.15.142:0/3157016039 conn(0x7fc2fc100190 msgr2=0x7fc2fc1025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.633+0000 7fc304499700 1 -- 172.21.15.142:0/3157016039 shutdown_connections 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.633+0000 7fc304499700 1 -- 172.21.15.142:0/3157016039 wait complete. 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.633+0000 7fc304499700 1 Processor -- start 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.633+0000 7fc304499700 1 -- start start 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.633+0000 7fc304499700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 0x7fc2fc19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.634+0000 7fc304499700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc19c910 con 0x7fc2fc106a40 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.634+0000 7fc302235700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 0x7fc2fc19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.634+0000 7fc302235700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 0x7fc2fc19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42514/0 (socket says 172.21.15.142:42514) 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.634+0000 7fc302235700 1 -- 172.21.15.142:0/73541290 learned_addr learned my addr 172.21.15.142:0/73541290 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.634+0000 7fc302235700 1 -- 172.21.15.142:0/73541290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f8009550 con 0x7fc2fc106a40 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.634+0000 7fc302235700 1 --2- 172.21.15.142:0/73541290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 0x7fc2fc19c3d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000c00 tx=0x7fc2f8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.634+0000 7fc2f37fe700 1 -- 172.21.15.142:0/73541290 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2f8003b90 con 0x7fc2fc106a40 2024-03-29T04:35:21.306 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.634+0000 7fc2f37fe700 1 -- 172.21.15.142:0/73541290 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2f80248b0 con 0x7fc2fc106a40 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.634+0000 7fc2f37fe700 1 -- 172.21.15.142:0/73541290 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2f801a8e0 con 0x7fc2fc106a40 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.634+0000 7fc304499700 1 -- 172.21.15.142:0/73541290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc19cb10 con 0x7fc2fc106a40 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.634+0000 7fc304499700 1 -- 172.21.15.142:0/73541290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc19cfb0 con 0x7fc2fc106a40 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.635+0000 7fc304499700 1 -- 172.21.15.142:0/73541290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2fc196130 con 0x7fc2fc106a40 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.635+0000 7fc2f37fe700 1 -- 172.21.15.142:0/73541290 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc2f8024440 con 0x7fc2fc106a40 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.635+0000 7fc2f37fe700 1 -- 172.21.15.142:0/73541290 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc2f8019520 con 0x7fc2fc106a40 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.637+0000 7fc2f37fe700 1 -- 172.21.15.142:0/73541290 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fc2f80246f0 con 0x7fc2fc106a40 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.707+0000 7fc304499700 1 -- 172.21.15.142:0/73541290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc2fc02ce70 con 0x7fc2fc106a40 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.707+0000 7fc2f37fe700 1 -- 172.21.15.142:0/73541290 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fc2f8031030 con 0x7fc2fc106a40 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.708+0000 7fc304499700 1 -- 172.21.15.142:0/73541290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 msgr2=0x7fc2fc19c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.708+0000 7fc304499700 1 --2- 172.21.15.142:0/73541290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 0x7fc2fc19c3d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000c00 tx=0x7fc2f8011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.708+0000 7fc304499700 1 -- 172.21.15.142:0/73541290 shutdown_connections 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.708+0000 7fc304499700 1 --2- 172.21.15.142:0/73541290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc106a40 0x7fc2fc19c3d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:21.307 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.708+0000 7fc304499700 1 -- 172.21.15.142:0/73541290 >> 172.21.15.142:0/73541290 conn(0x7fc2fc100190 msgr2=0x7fc2fc193270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:21.308 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.709+0000 7fc304499700 1 -- 172.21.15.142:0/73541290 shutdown_connections 2024-03-29T04:35:21.308 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:20.709+0000 7fc304499700 1 -- 172.21.15.142:0/73541290 wait complete. 2024-03-29T04:35:21.308 INFO:teuthology.orchestra.run.smithi142.stdout:mgr not available, waiting (2/15)... 2024-03-29T04:35:22.606 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: Activating manager daemon smithi142.swesrg 2024-03-29T04:35:22.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: mgrmap e2: smithi142.swesrg(active, starting, since 0.00288578s) 2024-03-29T04:35:22.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: from='mgr.14100 172.21.15.142:0/1304893453' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T04:35:22.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: from='mgr.14100 172.21.15.142:0/1304893453' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T04:35:22.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: from='mgr.14100 172.21.15.142:0/1304893453' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T04:35:22.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: from='mgr.14100 172.21.15.142:0/1304893453' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:35:22.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: from='mgr.14100 172.21.15.142:0/1304893453' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi142.swesrg", "id": "smithi142.swesrg"}]: dispatch 2024-03-29T04:35:22.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: Manager daemon smithi142.swesrg is now available 2024-03-29T04:35:22.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: from='mgr.14100 172.21.15.142:0/1304893453' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:35:22.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: from='mgr.14100 172.21.15.142:0/1304893453' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/trash_purge_schedule"}]: dispatch 2024-03-29T04:35:22.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: from='mgr.14100 172.21.15.142:0/1304893453' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:22.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: from='mgr.14100 172.21.15.142:0/1304893453' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:22.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:22 smithi142 ceph-mon[29027]: from='mgr.14100 172.21.15.142:0/1304893453' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:24.581 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 2024-03-29T04:35:24.581 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout { 2024-03-29T04:35:24.581 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "fsid": "9f96302c-ed85-11ee-95d0-87774f69a715", 2024-03-29T04:35:24.581 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "health": { 2024-03-29T04:35:24.581 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-29T04:35:24.581 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-29T04:35:24.581 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-29T04:35:24.581 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:24.581 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-29T04:35:24.581 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-29T04:35:24.581 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 0 2024-03-29T04:35:24.581 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout ], 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "smithi142" 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout ], 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-29T04:35:24.582 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-29T04:35:24.583 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-29T04:35:24.591 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-29T04:35:24.591 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:24.592 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "restful" 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout ], 2024-03-29T04:35:24.593 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-29T04:35:24.594 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:24.594 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-29T04:35:24.594 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T04:35:24.594 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "modified": "2024-03-29T04:35:07.701762+0000", 2024-03-29T04:35:24.594 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-29T04:35:24.594 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout }, 2024-03-29T04:35:24.594 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-29T04:35:24.594 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout } 2024-03-29T04:35:24.594 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.735+0000 7f2cc38d8700 1 Processor -- start 2024-03-29T04:35:24.594 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.736+0000 7f2cc38d8700 1 -- start start 2024-03-29T04:35:24.594 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.736+0000 7f2cc38d8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 0x7f2cbc109420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:24.594 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.736+0000 7f2cc38d8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc079e30 con 0x7f2cbc107030 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.736+0000 7f2cc1674700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 0x7f2cbc109420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.736+0000 7f2cc1674700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 0x7f2cbc109420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42588/0 (socket says 172.21.15.142:42588) 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.736+0000 7f2cc1674700 1 -- 172.21.15.142:0/3421243918 learned_addr learned my addr 172.21.15.142:0/3421243918 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.736+0000 7f2cc1674700 1 -- 172.21.15.142:0/3421243918 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc109960 con 0x7f2cbc107030 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.736+0000 7f2cc1674700 1 --2- 172.21.15.142:0/3421243918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 0x7f2cbc109420 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80098a0 tx=0x7f2cb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5efa98c3b48f35 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.737+0000 7f2cb3fff700 1 -- 172.21.15.142:0/3421243918 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2cb8004180 con 0x7f2cbc107030 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.737+0000 7f2cb3fff700 1 -- 172.21.15.142:0/3421243918 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2cb80042e0 con 0x7f2cbc107030 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.737+0000 7f2cb3fff700 1 -- 172.21.15.142:0/3421243918 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2cb8011a50 con 0x7f2cbc107030 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.737+0000 7f2cc38d8700 1 -- 172.21.15.142:0/3421243918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 msgr2=0x7f2cbc109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.737+0000 7f2cc38d8700 1 --2- 172.21.15.142:0/3421243918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 0x7f2cbc109420 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80098a0 tx=0x7f2cb8009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.737+0000 7f2cc38d8700 1 -- 172.21.15.142:0/3421243918 shutdown_connections 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.737+0000 7f2cc38d8700 1 --2- 172.21.15.142:0/3421243918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 0x7f2cbc109420 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.737+0000 7f2cc38d8700 1 -- 172.21.15.142:0/3421243918 >> 172.21.15.142:0/3421243918 conn(0x7f2cbc100a20 msgr2=0x7f2cbc102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:24.595 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.737+0000 7f2cc38d8700 1 -- 172.21.15.142:0/3421243918 shutdown_connections 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.737+0000 7f2cc38d8700 1 -- 172.21.15.142:0/3421243918 wait complete. 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.738+0000 7f2cc38d8700 1 Processor -- start 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.738+0000 7f2cc38d8700 1 -- start start 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.738+0000 7f2cc38d8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 0x7f2cbc19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.738+0000 7f2cc38d8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc19a720 con 0x7f2cbc107030 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.738+0000 7f2cc1674700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 0x7f2cbc19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.738+0000 7f2cc1674700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 0x7f2cbc19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42596/0 (socket says 172.21.15.142:42596) 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.738+0000 7f2cc1674700 1 -- 172.21.15.142:0/981351054 learned_addr learned my addr 172.21.15.142:0/981351054 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.738+0000 7f2cc1674700 1 -- 172.21.15.142:0/981351054 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb8009550 con 0x7f2cbc107030 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.739+0000 7f2cc1674700 1 --2- 172.21.15.142:0/981351054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 0x7f2cbc19a1e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8003f80 tx=0x7f2cb8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.739+0000 7f2cb27fc700 1 -- 172.21.15.142:0/981351054 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2cb8003a80 con 0x7f2cbc107030 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.739+0000 7f2cb27fc700 1 -- 172.21.15.142:0/981351054 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2cb8003be0 con 0x7f2cbc107030 2024-03-29T04:35:24.596 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.739+0000 7f2cb27fc700 1 -- 172.21.15.142:0/981351054 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2cb8019820 con 0x7f2cbc107030 2024-03-29T04:35:24.597 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.739+0000 7f2cc38d8700 1 -- 172.21.15.142:0/981351054 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc19a920 con 0x7f2cbc107030 2024-03-29T04:35:24.597 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.739+0000 7f2cc38d8700 1 -- 172.21.15.142:0/981351054 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc19adc0 con 0x7f2cbc107030 2024-03-29T04:35:24.597 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.740+0000 7f2cb27fc700 1 -- 172.21.15.142:0/981351054 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 3) v1 ==== 45166+0+0 (secure 0 0 0) 0x7f2cb801a430 con 0x7f2cbc107030 2024-03-29T04:35:24.597 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.740+0000 7f2cc38d8700 1 -- 172.21.15.142:0/981351054 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc10a680 con 0x7f2cbc107030 2024-03-29T04:35:24.597 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.740+0000 7f2cb27fc700 1 --2- 172.21.15.142:0/981351054 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f2ca80381a0 0x7f2ca803a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:24.597 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.740+0000 7f2cb27fc700 1 -- 172.21.15.142:0/981351054 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2cb804c240 con 0x7f2cbc107030 2024-03-29T04:35:24.597 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.740+0000 7f2cc0e73700 1 --2- 172.21.15.142:0/981351054 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f2ca80381a0 0x7f2ca803a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:24.597 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.740+0000 7f2cc0e73700 1 --2- 172.21.15.142:0/981351054 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f2ca80381a0 0x7f2ca803a660 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2cac006fd0 tx=0x7f2cac006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:24.597 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:23.745+0000 7f2cb27fc700 1 -- 172.21.15.142:0/981351054 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2cb801acc0 con 0x7f2cbc107030 2024-03-29T04:35:24.597 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.007+0000 7f2cc38d8700 1 -- 172.21.15.142:0/981351054 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2cbc10a8a0 con 0x7f2cbc107030 2024-03-29T04:35:24.597 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.008+0000 7f2cb27fc700 1 -- 172.21.15.142:0/981351054 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f2cb8018350 con 0x7f2cbc107030 2024-03-29T04:35:24.597 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.010+0000 7f2cc38d8700 1 -- 172.21.15.142:0/981351054 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f2ca80381a0 msgr2=0x7f2ca803a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:24.598 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.010+0000 7f2cc38d8700 1 --2- 172.21.15.142:0/981351054 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f2ca80381a0 0x7f2ca803a660 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2cac006fd0 tx=0x7f2cac006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:24.598 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.010+0000 7f2cc38d8700 1 -- 172.21.15.142:0/981351054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 msgr2=0x7f2cbc19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:24.598 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.010+0000 7f2cc38d8700 1 --2- 172.21.15.142:0/981351054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 0x7f2cbc19a1e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8003f80 tx=0x7f2cb8011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:24.598 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.010+0000 7f2cc38d8700 1 -- 172.21.15.142:0/981351054 shutdown_connections 2024-03-29T04:35:24.598 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.010+0000 7f2cc38d8700 1 --2- 172.21.15.142:0/981351054 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f2ca80381a0 0x7f2ca803a660 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:24.598 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.010+0000 7f2cc38d8700 1 --2- 172.21.15.142:0/981351054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cbc107030 0x7f2cbc19a1e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:24.598 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.010+0000 7f2cc38d8700 1 -- 172.21.15.142:0/981351054 >> 172.21.15.142:0/981351054 conn(0x7f2cbc100a20 msgr2=0x7f2cbc1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:24.598 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.010+0000 7f2cc38d8700 1 -- 172.21.15.142:0/981351054 shutdown_connections 2024-03-29T04:35:24.598 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.010+0000 7f2cc38d8700 1 -- 172.21.15.142:0/981351054 wait complete. 2024-03-29T04:35:24.598 INFO:teuthology.orchestra.run.smithi142.stdout:mgr is available 2024-03-29T04:35:24.694 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:24 smithi142 ceph-mon[29027]: mgrmap e3: smithi142.swesrg(active, since 1.00689s) 2024-03-29T04:35:24.694 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:24 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/981351054' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-29T04:35:25.766 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:25 smithi142 ceph-mon[29027]: mgrmap e4: smithi142.swesrg(active, since 2s) 2024-03-29T04:35:25.766 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:25 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2823893698' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-29T04:35:25.775 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 2024-03-29T04:35:25.775 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout [global] 2024-03-29T04:35:25.775 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout fsid = 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:35:25.775 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-29T04:35:25.775 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-29T04:35:25.775 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-29T04:35:25.775 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-29T04:35:25.776 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 2024-03-29T04:35:25.776 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-29T04:35:25.776 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-29T04:35:25.776 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 2024-03-29T04:35:25.776 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout [osd] 2024-03-29T04:35:25.776 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-29T04:35:25.776 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-29T04:35:25.776 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-29T04:35:25.776 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-29T04:35:25.776 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-29T04:35:25.776 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.982+0000 7fe6b0bdf700 1 Processor -- start 2024-03-29T04:35:25.777 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.982+0000 7fe6b0bdf700 1 -- start start 2024-03-29T04:35:25.777 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.983+0000 7fe6b0bdf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 0x7fe6ac106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:25.777 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.983+0000 7fe6b0bdf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac106f50 con 0x7fe6ac106560 2024-03-29T04:35:25.777 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.983+0000 7fe6aa59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 0x7fe6ac106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:25.777 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.983+0000 7fe6aa59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 0x7fe6ac106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42608/0 (socket says 172.21.15.142:42608) 2024-03-29T04:35:25.777 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.983+0000 7fe6aa59c700 1 -- 172.21.15.142:0/2910934022 learned_addr learned my addr 172.21.15.142:0/2910934022 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:25.777 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.983+0000 7fe6aa59c700 1 -- 172.21.15.142:0/2910934022 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac107760 con 0x7fe6ac106560 2024-03-29T04:35:25.777 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.983+0000 7fe6aa59c700 1 --2- 172.21.15.142:0/2910934022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 0x7fe6ac106980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00098a0 tx=0x7fe6a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37304fc9ad061ee server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:25.777 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.983+0000 7fe6a959a700 1 -- 172.21.15.142:0/2910934022 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6a0004180 con 0x7fe6ac106560 2024-03-29T04:35:25.777 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.983+0000 7fe6a959a700 1 -- 172.21.15.142:0/2910934022 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6a00042e0 con 0x7fe6ac106560 2024-03-29T04:35:25.777 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.983+0000 7fe6a959a700 1 -- 172.21.15.142:0/2910934022 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6a0011a50 con 0x7fe6ac106560 2024-03-29T04:35:25.777 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.984+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2910934022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 msgr2=0x7fe6ac106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.984+0000 7fe6b0bdf700 1 --2- 172.21.15.142:0/2910934022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 0x7fe6ac106980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00098a0 tx=0x7fe6a0009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.984+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2910934022 shutdown_connections 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.984+0000 7fe6b0bdf700 1 --2- 172.21.15.142:0/2910934022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 0x7fe6ac106980 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.984+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2910934022 >> 172.21.15.142:0/2910934022 conn(0x7fe6ac101b20 msgr2=0x7fe6ac103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.984+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2910934022 shutdown_connections 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.984+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2910934022 wait complete. 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.985+0000 7fe6b0bdf700 1 Processor -- start 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.985+0000 7fe6b0bdf700 1 -- start start 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.985+0000 7fe6b0bdf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 0x7fe6ac07ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.985+0000 7fe6b0bdf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac07b060 con 0x7fe6ac106560 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.985+0000 7fe6aa59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 0x7fe6ac07ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.985+0000 7fe6aa59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 0x7fe6ac07ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42620/0 (socket says 172.21.15.142:42620) 2024-03-29T04:35:25.778 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.985+0000 7fe6aa59c700 1 -- 172.21.15.142:0/2823893698 learned_addr learned my addr 172.21.15.142:0/2823893698 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.985+0000 7fe6aa59c700 1 -- 172.21.15.142:0/2823893698 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a0009550 con 0x7fe6ac106560 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.985+0000 7fe6aa59c700 1 --2- 172.21.15.142:0/2823893698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 0x7fe6ac07ab20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0000c00 tx=0x7fe6a0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.986+0000 7fe69b7fe700 1 -- 172.21.15.142:0/2823893698 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6a0003b90 con 0x7fe6ac106560 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.986+0000 7fe69b7fe700 1 -- 172.21.15.142:0/2823893698 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6a00248b0 con 0x7fe6ac106560 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.986+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2823893698 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ac079170 con 0x7fe6ac106560 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.986+0000 7fe69b7fe700 1 -- 172.21.15.142:0/2823893698 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6a001a8e0 con 0x7fe6ac106560 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.986+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2823893698 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ac079610 con 0x7fe6ac106560 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.986+0000 7fe69b7fe700 1 -- 172.21.15.142:0/2823893698 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fe6a001ab50 con 0x7fe6ac106560 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.987+0000 7fe69b7fe700 1 --2- 172.21.15.142:0/2823893698 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7fe694038320 0x7fe69403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.987+0000 7fe69b7fe700 1 -- 172.21.15.142:0/2823893698 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe6a004cfb0 con 0x7fe6ac106560 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.987+0000 7fe6a9d9b700 1 --2- 172.21.15.142:0/2823893698 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7fe694038320 0x7fe69403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.987+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2823893698 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ac0623c0 con 0x7fe6ac106560 2024-03-29T04:35:25.779 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.987+0000 7fe6a9d9b700 1 --2- 172.21.15.142:0/2823893698 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7fe694038320 0x7fe69403a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe69c006fd0 tx=0x7fe69c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:24.992+0000 7fe69b7fe700 1 -- 172.21.15.142:0/2823893698 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe6a00237b0 con 0x7fe6ac106560 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:25.178+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2823893698 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe6ac079f90 con 0x7fe6ac106560 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:25.179+0000 7fe69b7fe700 1 -- 172.21.15.142:0/2823893698 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fe6a004b020 con 0x7fe6ac106560 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:25.181+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2823893698 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7fe694038320 msgr2=0x7fe69403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:25.181+0000 7fe6b0bdf700 1 --2- 172.21.15.142:0/2823893698 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7fe694038320 0x7fe69403a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe69c006fd0 tx=0x7fe69c006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:25.181+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2823893698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 msgr2=0x7fe6ac07ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:25.181+0000 7fe6b0bdf700 1 --2- 172.21.15.142:0/2823893698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 0x7fe6ac07ab20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0000c00 tx=0x7fe6a0011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:25.181+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2823893698 shutdown_connections 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:25.181+0000 7fe6b0bdf700 1 --2- 172.21.15.142:0/2823893698 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7fe694038320 0x7fe69403a7e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:25.181+0000 7fe6b0bdf700 1 --2- 172.21.15.142:0/2823893698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6ac106560 0x7fe6ac07ab20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:25.181+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2823893698 >> 172.21.15.142:0/2823893698 conn(0x7fe6ac101b20 msgr2=0x7fe6ac102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:25.181+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2823893698 shutdown_connections 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:25.181+0000 7fe6b0bdf700 1 -- 172.21.15.142:0/2823893698 wait complete. 2024-03-29T04:35:25.780 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling cephadm module... 2024-03-29T04:35:26.747 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:26 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/442410498' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-29T04:35:27.022 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.198+0000 7fa0370e1700 1 Processor -- start 2024-03-29T04:35:27.022 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.198+0000 7fa0370e1700 1 -- start start 2024-03-29T04:35:27.022 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.198+0000 7fa0370e1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 0x7fa030109420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:27.022 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.198+0000 7fa0370e1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa030079e30 con 0x7fa030107030 2024-03-29T04:35:27.022 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.198+0000 7fa034e7d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 0x7fa030109420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:27.022 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.198+0000 7fa034e7d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 0x7fa030109420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42630/0 (socket says 172.21.15.142:42630) 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.198+0000 7fa034e7d700 1 -- 172.21.15.142:0/3366718648 learned_addr learned my addr 172.21.15.142:0/3366718648 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.199+0000 7fa034e7d700 1 -- 172.21.15.142:0/3366718648 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa030109960 con 0x7fa030107030 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.199+0000 7fa034e7d700 1 --2- 172.21.15.142:0/3366718648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 0x7fa030109420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa0240098a0 tx=0x7fa024009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0848a6e195445d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.199+0000 7fa02f7fe700 1 -- 172.21.15.142:0/3366718648 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa024004180 con 0x7fa030107030 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.199+0000 7fa02f7fe700 1 -- 172.21.15.142:0/3366718648 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa0240042e0 con 0x7fa030107030 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.199+0000 7fa02f7fe700 1 -- 172.21.15.142:0/3366718648 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa024011a50 con 0x7fa030107030 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.199+0000 7fa0370e1700 1 -- 172.21.15.142:0/3366718648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 msgr2=0x7fa030109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.199+0000 7fa0370e1700 1 --2- 172.21.15.142:0/3366718648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 0x7fa030109420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa0240098a0 tx=0x7fa024009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.199+0000 7fa0370e1700 1 -- 172.21.15.142:0/3366718648 shutdown_connections 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.199+0000 7fa0370e1700 1 --2- 172.21.15.142:0/3366718648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 0x7fa030109420 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.200+0000 7fa0370e1700 1 -- 172.21.15.142:0/3366718648 >> 172.21.15.142:0/3366718648 conn(0x7fa030100a20 msgr2=0x7fa030102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.200+0000 7fa0370e1700 1 -- 172.21.15.142:0/3366718648 shutdown_connections 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.200+0000 7fa0370e1700 1 -- 172.21.15.142:0/3366718648 wait complete. 2024-03-29T04:35:27.023 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.200+0000 7fa0370e1700 1 Processor -- start 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.200+0000 7fa0370e1700 1 -- start start 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.200+0000 7fa0370e1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 0x7fa03019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.200+0000 7fa0370e1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03019a720 con 0x7fa030107030 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.200+0000 7fa034e7d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 0x7fa03019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.200+0000 7fa034e7d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 0x7fa03019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42642/0 (socket says 172.21.15.142:42642) 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.200+0000 7fa034e7d700 1 -- 172.21.15.142:0/442410498 learned_addr learned my addr 172.21.15.142:0/442410498 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.201+0000 7fa034e7d700 1 -- 172.21.15.142:0/442410498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa024009550 con 0x7fa030107030 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.201+0000 7fa034e7d700 1 --2- 172.21.15.142:0/442410498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 0x7fa03019a1e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa024003f80 tx=0x7fa024011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.201+0000 7fa02dffb700 1 -- 172.21.15.142:0/442410498 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa024003a80 con 0x7fa030107030 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.201+0000 7fa02dffb700 1 -- 172.21.15.142:0/442410498 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa024003be0 con 0x7fa030107030 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.201+0000 7fa0370e1700 1 -- 172.21.15.142:0/442410498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa03019a920 con 0x7fa030107030 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.201+0000 7fa02dffb700 1 -- 172.21.15.142:0/442410498 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa024019820 con 0x7fa030107030 2024-03-29T04:35:27.024 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.201+0000 7fa0370e1700 1 -- 172.21.15.142:0/442410498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03019adc0 con 0x7fa030107030 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.202+0000 7fa02dffb700 1 -- 172.21.15.142:0/442410498 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fa024019aa0 con 0x7fa030107030 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.202+0000 7fa0370e1700 1 -- 172.21.15.142:0/442410498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03010a680 con 0x7fa030107030 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.202+0000 7fa02dffb700 1 --2- 172.21.15.142:0/442410498 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7fa020038280 0x7fa02003a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.202+0000 7fa02dffb700 1 -- 172.21.15.142:0/442410498 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa02404c580 con 0x7fa030107030 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.202+0000 7fa02ffff700 1 --2- 172.21.15.142:0/442410498 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7fa020038280 0x7fa02003a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.203+0000 7fa02ffff700 1 --2- 172.21.15.142:0/442410498 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7fa020038280 0x7fa02003a740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa01c006fd0 tx=0x7fa01c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.207+0000 7fa02dffb700 1 -- 172.21.15.142:0/442410498 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa02402b750 con 0x7fa030107030 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.447+0000 7fa0370e1700 1 -- 172.21.15.142:0/442410498 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fa03019dba0 con 0x7fa030107030 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.499+0000 7fa02dffb700 1 -- 172.21.15.142:0/442410498 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fa024019d50 con 0x7fa030107030 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.500+0000 7fa02dffb700 1 -- 172.21.15.142:0/442410498 <== mon.0 v2:172.21.15.142:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fa024023550 con 0x7fa030107030 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.503+0000 7fa0370e1700 1 -- 172.21.15.142:0/442410498 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7fa020038280 msgr2=0x7fa02003a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.503+0000 7fa0370e1700 1 --2- 172.21.15.142:0/442410498 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7fa020038280 0x7fa02003a740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa01c006fd0 tx=0x7fa01c006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:27.025 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.503+0000 7fa0370e1700 1 -- 172.21.15.142:0/442410498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 msgr2=0x7fa03019a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:27.026 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.503+0000 7fa0370e1700 1 --2- 172.21.15.142:0/442410498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 0x7fa03019a1e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa024003f80 tx=0x7fa024011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:27.026 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.503+0000 7fa0370e1700 1 -- 172.21.15.142:0/442410498 shutdown_connections 2024-03-29T04:35:27.026 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.503+0000 7fa0370e1700 1 --2- 172.21.15.142:0/442410498 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7fa020038280 0x7fa02003a740 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:27.026 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.503+0000 7fa0370e1700 1 --2- 172.21.15.142:0/442410498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa030107030 0x7fa03019a1e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:27.026 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.503+0000 7fa0370e1700 1 -- 172.21.15.142:0/442410498 >> 172.21.15.142:0/442410498 conn(0x7fa030100a20 msgr2=0x7fa0301017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:27.026 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.503+0000 7fa0370e1700 1 -- 172.21.15.142:0/442410498 shutdown_connections 2024-03-29T04:35:27.026 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:26.503+0000 7fa0370e1700 1 -- 172.21.15.142:0/442410498 wait complete. 2024-03-29T04:35:27.531 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:27 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/442410498' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-29T04:35:27.531 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:27 smithi142 ceph-mon[29027]: mgrmap e5: smithi142.swesrg(active, since 4s) 2024-03-29T04:35:28.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout { 2024-03-29T04:35:28.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-29T04:35:28.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-29T04:35:28.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "active_name": "smithi142.swesrg", 2024-03-29T04:35:28.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-29T04:35:28.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout } 2024-03-29T04:35:28.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.491+0000 7f063d822700 1 Processor -- start 2024-03-29T04:35:28.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.491+0000 7f063d822700 1 -- start start 2024-03-29T04:35:28.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.491+0000 7f063d822700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 0x7f0638106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:28.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.491+0000 7f063d822700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638106f50 con 0x7f0638106560 2024-03-29T04:35:28.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.492+0000 7f0636ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 0x7f0638106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:28.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.492+0000 7f0636ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 0x7f0638106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42664/0 (socket says 172.21.15.142:42664) 2024-03-29T04:35:28.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.492+0000 7f0636ffd700 1 -- 172.21.15.142:0/1442797506 learned_addr learned my addr 172.21.15.142:0/1442797506 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:28.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.492+0000 7f0636ffd700 1 -- 172.21.15.142:0/1442797506 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638107760 con 0x7f0638106560 2024-03-29T04:35:28.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.492+0000 7f0636ffd700 1 --2- 172.21.15.142:0/1442797506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 0x7f0638106980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f06280098a0 tx=0x7f0628009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1e94dbd105642e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:28.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.492+0000 7f0635ffb700 1 -- 172.21.15.142:0/1442797506 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0628004180 con 0x7f0638106560 2024-03-29T04:35:28.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.492+0000 7f0635ffb700 1 -- 172.21.15.142:0/1442797506 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f06280042e0 con 0x7f0638106560 2024-03-29T04:35:28.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.492+0000 7f0635ffb700 1 -- 172.21.15.142:0/1442797506 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0628011a50 con 0x7f0638106560 2024-03-29T04:35:28.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.493+0000 7f063d822700 1 -- 172.21.15.142:0/1442797506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 msgr2=0x7f0638106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:28.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.493+0000 7f063d822700 1 --2- 172.21.15.142:0/1442797506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 0x7f0638106980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f06280098a0 tx=0x7f0628009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:28.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.493+0000 7f063d822700 1 -- 172.21.15.142:0/1442797506 shutdown_connections 2024-03-29T04:35:28.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.493+0000 7f063d822700 1 --2- 172.21.15.142:0/1442797506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 0x7f0638106980 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:28.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.493+0000 7f063d822700 1 -- 172.21.15.142:0/1442797506 >> 172.21.15.142:0/1442797506 conn(0x7f0638101b40 msgr2=0x7f0638103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:28.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.493+0000 7f063d822700 1 -- 172.21.15.142:0/1442797506 shutdown_connections 2024-03-29T04:35:28.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.494+0000 7f063d822700 1 -- 172.21.15.142:0/1442797506 wait complete. 2024-03-29T04:35:28.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.494+0000 7f063d822700 1 Processor -- start 2024-03-29T04:35:28.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.494+0000 7f063d822700 1 -- start start 2024-03-29T04:35:28.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.495+0000 7f063d822700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 0x7f063819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:28.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.495+0000 7f063d822700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063819a700 con 0x7f0638106560 2024-03-29T04:35:28.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.495+0000 7f0636ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 0x7f063819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:28.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.495+0000 7f0636ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 0x7f063819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42680/0 (socket says 172.21.15.142:42680) 2024-03-29T04:35:28.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.495+0000 7f0636ffd700 1 -- 172.21.15.142:0/1728393366 learned_addr learned my addr 172.21.15.142:0/1728393366 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.495+0000 7f0636ffd700 1 -- 172.21.15.142:0/1728393366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0628009550 con 0x7f0638106560 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.495+0000 7f0636ffd700 1 --2- 172.21.15.142:0/1728393366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 0x7f063819a1c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0628000c00 tx=0x7f0628011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.495+0000 7f063c820700 1 -- 172.21.15.142:0/1728393366 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0628003b90 con 0x7f0638106560 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.496+0000 7f063c820700 1 -- 172.21.15.142:0/1728393366 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f06280248b0 con 0x7f0638106560 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.496+0000 7f063c820700 1 -- 172.21.15.142:0/1728393366 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f062801a8e0 con 0x7f0638106560 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.496+0000 7f063d822700 1 -- 172.21.15.142:0/1728393366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063819a900 con 0x7f0638106560 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.496+0000 7f063d822700 1 -- 172.21.15.142:0/1728393366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063819ada0 con 0x7f0638106560 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.497+0000 7f063d822700 1 -- 172.21.15.142:0/1728393366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06380623c0 con 0x7f0638106560 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.497+0000 7f063c820700 1 -- 172.21.15.142:0/1728393366 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f0628024a20 con 0x7f0638106560 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.498+0000 7f063c820700 1 --2- 172.21.15.142:0/1728393366 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f0620038370 0x7f062003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.498+0000 7f063c820700 1 -- 172.21.15.142:0/1728393366 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f062804c220 con 0x7f0638106560 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.498+0000 7f06367fc700 1 -- 172.21.15.142:0/1728393366 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f0620038370 msgr2=0x7f062003a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2745119007 2024-03-29T04:35:28.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.498+0000 7f06367fc700 1 --2- 172.21.15.142:0/1728393366 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f0620038370 0x7f062003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.502+0000 7f063c820700 1 -- 172.21.15.142:0/1728393366 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0628024dd0 con 0x7f0638106560 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.699+0000 7f06367fc700 1 -- 172.21.15.142:0/1728393366 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f0620038370 msgr2=0x7f062003a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2745119007 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.699+0000 7f06367fc700 1 --2- 172.21.15.142:0/1728393366 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f0620038370 0x7f062003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.755+0000 7f063d822700 1 -- 172.21.15.142:0/1728393366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f063810a990 con 0x7f0638106560 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.755+0000 7f063c820700 1 -- 172.21.15.142:0/1728393366 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f062802c390 con 0x7f0638106560 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.759+0000 7f063d822700 1 -- 172.21.15.142:0/1728393366 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f0620038370 msgr2=0x7f062003a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.759+0000 7f063d822700 1 --2- 172.21.15.142:0/1728393366 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f0620038370 0x7f062003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.759+0000 7f063d822700 1 -- 172.21.15.142:0/1728393366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 msgr2=0x7f063819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.759+0000 7f063d822700 1 --2- 172.21.15.142:0/1728393366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 0x7f063819a1c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0628000c00 tx=0x7f0628011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.759+0000 7f063d822700 1 -- 172.21.15.142:0/1728393366 shutdown_connections 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.759+0000 7f063d822700 1 --2- 172.21.15.142:0/1728393366 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f0620038370 0x7f062003a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.759+0000 7f063d822700 1 --2- 172.21.15.142:0/1728393366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638106560 0x7f063819a1c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:28.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.759+0000 7f063d822700 1 -- 172.21.15.142:0/1728393366 >> 172.21.15.142:0/1728393366 conn(0x7f0638101b40 msgr2=0x7f0638102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:28.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.759+0000 7f063d822700 1 -- 172.21.15.142:0/1728393366 shutdown_connections 2024-03-29T04:35:28.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:27.759+0000 7f063d822700 1 -- 172.21.15.142:0/1728393366 wait complete. 2024-03-29T04:35:28.344 INFO:teuthology.orchestra.run.smithi142.stdout:Waiting for the mgr to restart... 2024-03-29T04:35:28.344 INFO:teuthology.orchestra.run.smithi142.stdout:Waiting for mgr epoch 5... 2024-03-29T04:35:28.783 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:28 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1728393366' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-29T04:35:33.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: Active manager daemon smithi142.swesrg restarted 2024-03-29T04:35:33.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: Activating manager daemon smithi142.swesrg 2024-03-29T04:35:33.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: osdmap e2: 0 total, 0 up, 0 in 2024-03-29T04:35:33.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: mgrmap e6: smithi142.swesrg(active, starting, since 0.00352942s) 2024-03-29T04:35:33.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:35:33.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi142.swesrg", "id": "smithi142.swesrg"}]: dispatch 2024-03-29T04:35:33.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T04:35:33.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T04:35:33.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T04:35:33.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: Manager daemon smithi142.swesrg is now available 2024-03-29T04:35:33.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:33.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:33.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:35:33.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:35:33.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:35:33.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:35:33.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:33 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/trash_purge_schedule"}]: dispatch 2024-03-29T04:35:34.606 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:34 smithi142 ceph-mon[29027]: Found migration_current of "None". Setting to last migration. 2024-03-29T04:35:34.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:34 smithi142 ceph-mon[29027]: mgrmap e7: smithi142.swesrg(active, since 1.00595s) 2024-03-29T04:35:34.863 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout { 2024-03-29T04:35:34.863 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-29T04:35:34.863 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-29T04:35:34.863 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout } 2024-03-29T04:35:34.863 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.806+0000 7f55289f5700 1 Processor -- start 2024-03-29T04:35:34.863 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.806+0000 7f55289f5700 1 -- start start 2024-03-29T04:35:34.863 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.806+0000 7f55289f5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 0x7f5524105270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:34.863 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.806+0000 7f55289f5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524074600 con 0x7f5524104e50 2024-03-29T04:35:34.863 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.807+0000 7f552259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 0x7f5524105270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.807+0000 7f552259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 0x7f5524105270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33112/0 (socket says 172.21.15.142:33112) 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.807+0000 7f552259c700 1 -- 172.21.15.142:0/803532952 learned_addr learned my addr 172.21.15.142:0/803532952 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.807+0000 7f552259c700 1 -- 172.21.15.142:0/803532952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55241057b0 con 0x7f5524104e50 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.807+0000 7f552259c700 1 --2- 172.21.15.142:0/803532952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 0x7f5524105270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f55140098a0 tx=0x7f5514009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfa8b07eb92ff51 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.807+0000 7f552159a700 1 -- 172.21.15.142:0/803532952 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5514004180 con 0x7f5524104e50 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.807+0000 7f552159a700 1 -- 172.21.15.142:0/803532952 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f55140042e0 con 0x7f5524104e50 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.807+0000 7f552159a700 1 -- 172.21.15.142:0/803532952 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5514011a50 con 0x7f5524104e50 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.808+0000 7f55289f5700 1 -- 172.21.15.142:0/803532952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 msgr2=0x7f5524105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.808+0000 7f55289f5700 1 --2- 172.21.15.142:0/803532952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 0x7f5524105270 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f55140098a0 tx=0x7f5514009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.808+0000 7f55289f5700 1 -- 172.21.15.142:0/803532952 shutdown_connections 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.808+0000 7f55289f5700 1 --2- 172.21.15.142:0/803532952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 0x7f5524105270 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.808+0000 7f55289f5700 1 -- 172.21.15.142:0/803532952 >> 172.21.15.142:0/803532952 conn(0x7f55241009f0 msgr2=0x7f5524102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.808+0000 7f55289f5700 1 -- 172.21.15.142:0/803532952 shutdown_connections 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.808+0000 7f55289f5700 1 -- 172.21.15.142:0/803532952 wait complete. 2024-03-29T04:35:34.864 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.809+0000 7f55289f5700 1 Processor -- start 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.809+0000 7f55289f5700 1 -- start start 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.809+0000 7f55289f5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 0x7f552419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.809+0000 7f55289f5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552419c8c0 con 0x7f5524104e50 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.809+0000 7f552259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 0x7f552419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.809+0000 7f552259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 0x7f552419c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33114/0 (socket says 172.21.15.142:33114) 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.809+0000 7f552259c700 1 -- 172.21.15.142:0/3294740179 learned_addr learned my addr 172.21.15.142:0/3294740179 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.809+0000 7f552259c700 1 -- 172.21.15.142:0/3294740179 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5514009550 con 0x7f5524104e50 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.809+0000 7f552259c700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 0x7f552419c380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5514000c00 tx=0x7f5514011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.810+0000 7f55137fe700 1 -- 172.21.15.142:0/3294740179 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5514003b90 con 0x7f5524104e50 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.810+0000 7f55137fe700 1 -- 172.21.15.142:0/3294740179 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f55140248b0 con 0x7f5524104e50 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.810+0000 7f55289f5700 1 -- 172.21.15.142:0/3294740179 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552419cac0 con 0x7f5524104e50 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.810+0000 7f55137fe700 1 -- 172.21.15.142:0/3294740179 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f551401a8e0 con 0x7f5524104e50 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.810+0000 7f55289f5700 1 -- 172.21.15.142:0/3294740179 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552419cf60 con 0x7f5524104e50 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.811+0000 7f55137fe700 1 -- 172.21.15.142:0/3294740179 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f551401ab50 con 0x7f5524104e50 2024-03-29T04:35:34.865 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.811+0000 7f55137fe700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 0x7f550c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.811+0000 7f55137fe700 1 -- 172.21.15.142:0/3294740179 --> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f550c03af40 con 0x7f550c038370 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.811+0000 7f55137fe700 1 -- 172.21.15.142:0/3294740179 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f551404c0c0 con 0x7f5524104e50 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.811+0000 7f5521d9b700 1 -- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 msgr2=0x7f550c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2745119007 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:28.811+0000 7f5521d9b700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 0x7f550c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:29.012+0000 7f5521d9b700 1 -- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 msgr2=0x7f550c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2745119007 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:29.012+0000 7f5521d9b700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 0x7f550c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:29.413+0000 7f5521d9b700 1 -- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 msgr2=0x7f550c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2745119007 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:29.413+0000 7f5521d9b700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 0x7f550c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:30.214+0000 7f5521d9b700 1 -- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 msgr2=0x7f550c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2745119007 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:30.214+0000 7f5521d9b700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 0x7f550c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:31.815+0000 7f5521d9b700 1 -- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 msgr2=0x7f550c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2745119007 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:31.815+0000 7f5521d9b700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 0x7f550c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:33.217+0000 7f55137fe700 1 -- 172.21.15.142:0/3294740179 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mgrmap(e 6) v1 ==== 45050+0+0 (secure 0 0 0) 0x7f551402d3f0 con 0x7f5524104e50 2024-03-29T04:35:34.866 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:33.217+0000 7f55137fe700 1 -- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 msgr2=0x7f550c03a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:33.217+0000 7f55137fe700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 0x7f550c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.218+0000 7f55137fe700 1 -- 172.21.15.142:0/3294740179 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f5514020d20 con 0x7f5524104e50 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.218+0000 7f55137fe700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f550c03bd70 0x7f550c03e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.218+0000 7f55137fe700 1 -- 172.21.15.142:0/3294740179 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f550c03af40 con 0x7f550c03bd70 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.219+0000 7f5521d9b700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f550c03bd70 0x7f550c03e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.219+0000 7f5521d9b700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f550c03bd70 0x7f550c03e160 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5518003a10 tx=0x7f55180092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.220+0000 7f55137fe700 1 -- 172.21.15.142:0/3294740179 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f550c03af40 con 0x7f550c03bd70 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.226+0000 7f55289f5700 1 -- 172.21.15.142:0/3294740179 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f552419d6c0 con 0x7f550c03bd70 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.226+0000 7f55137fe700 1 -- 172.21.15.142:0/3294740179 <== mgr.14118 v2:172.21.15.142:6800/2742619935 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f552419d6c0 con 0x7f550c03bd70 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.227+0000 7f55289f5700 1 -- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f550c03bd70 msgr2=0x7f550c03e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.227+0000 7f55289f5700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f550c03bd70 0x7f550c03e160 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5518003a10 tx=0x7f55180092b0 comp rx=0 tx=0).stop 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.227+0000 7f55289f5700 1 -- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 msgr2=0x7f552419c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.227+0000 7f55289f5700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 0x7f552419c380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5514000c00 tx=0x7f5514011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:34.867 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.227+0000 7f55289f5700 1 -- 172.21.15.142:0/3294740179 shutdown_connections 2024-03-29T04:35:34.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.227+0000 7f55289f5700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f550c03bd70 0x7f550c03e160 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:34.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.227+0000 7f55289f5700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5524104e50 0x7f552419c380 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:34.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.227+0000 7f55289f5700 1 --2- 172.21.15.142:0/3294740179 >> [v2:172.21.15.142:6800/2745119007,v1:172.21.15.142:6801/2745119007] conn(0x7f550c038370 0x7f550c03a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:34.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.227+0000 7f55289f5700 1 -- 172.21.15.142:0/3294740179 >> 172.21.15.142:0/3294740179 conn(0x7f55241009f0 msgr2=0x7f5524190a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:34.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.227+0000 7f55289f5700 1 -- 172.21.15.142:0/3294740179 shutdown_connections 2024-03-29T04:35:34.868 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:34.227+0000 7f55289f5700 1 -- 172.21.15.142:0/3294740179 wait complete. 2024-03-29T04:35:34.868 INFO:teuthology.orchestra.run.smithi142.stdout:mgr epoch 5 is available 2024-03-29T04:35:34.868 INFO:teuthology.orchestra.run.smithi142.stdout:Setting orchestrator backend to cephadm... 2024-03-29T04:35:35.850 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:35 smithi142 ceph-mon[29027]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-29T04:35:35.850 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:35 smithi142 ceph-mon[29027]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-29T04:35:35.850 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:35 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:35.850 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:35 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:35.850 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:35 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:35:35.850 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:35 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:35.850 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:35 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:35:36.124 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.287+0000 7f3de0f7e700 1 Processor -- start 2024-03-29T04:35:36.124 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.287+0000 7f3de0f7e700 1 -- start start 2024-03-29T04:35:36.124 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.287+0000 7f3de0f7e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 0x7f3ddc109380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.287+0000 7f3de0f7e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc079e90 con 0x7f3ddc106f90 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.287+0000 7f3dda59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 0x7f3ddc109380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.287+0000 7f3dda59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 0x7f3ddc109380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33206/0 (socket says 172.21.15.142:33206) 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.287+0000 7f3dda59c700 1 -- 172.21.15.142:0/3623975674 learned_addr learned my addr 172.21.15.142:0/3623975674 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.287+0000 7f3dda59c700 1 -- 172.21.15.142:0/3623975674 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc1098c0 con 0x7f3ddc106f90 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.288+0000 7f3dda59c700 1 --2- 172.21.15.142:0/3623975674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 0x7f3ddc109380 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0098a0 tx=0x7f3dcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c5856afc62233e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.288+0000 7f3dd959a700 1 -- 172.21.15.142:0/3623975674 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3dcc004180 con 0x7f3ddc106f90 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.288+0000 7f3dd959a700 1 -- 172.21.15.142:0/3623975674 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3dcc0042e0 con 0x7f3ddc106f90 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.288+0000 7f3dd959a700 1 -- 172.21.15.142:0/3623975674 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3dcc011a50 con 0x7f3ddc106f90 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.288+0000 7f3de0f7e700 1 -- 172.21.15.142:0/3623975674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 msgr2=0x7f3ddc109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.288+0000 7f3de0f7e700 1 --2- 172.21.15.142:0/3623975674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 0x7f3ddc109380 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0098a0 tx=0x7f3dcc009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.289+0000 7f3de0f7e700 1 -- 172.21.15.142:0/3623975674 shutdown_connections 2024-03-29T04:35:36.125 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.289+0000 7f3de0f7e700 1 --2- 172.21.15.142:0/3623975674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 0x7f3ddc109380 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.289+0000 7f3de0f7e700 1 -- 172.21.15.142:0/3623975674 >> 172.21.15.142:0/3623975674 conn(0x7f3ddc100980 msgr2=0x7f3ddc102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.289+0000 7f3de0f7e700 1 -- 172.21.15.142:0/3623975674 shutdown_connections 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.289+0000 7f3de0f7e700 1 -- 172.21.15.142:0/3623975674 wait complete. 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.289+0000 7f3de0f7e700 1 Processor -- start 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.289+0000 7f3de0f7e700 1 -- start start 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.289+0000 7f3de0f7e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 0x7f3ddc19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.290+0000 7f3de0f7e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc19a700 con 0x7f3ddc106f90 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.290+0000 7f3dda59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 0x7f3ddc19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.290+0000 7f3dda59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 0x7f3ddc19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33212/0 (socket says 172.21.15.142:33212) 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.290+0000 7f3dda59c700 1 -- 172.21.15.142:0/1530015181 learned_addr learned my addr 172.21.15.142:0/1530015181 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.290+0000 7f3dda59c700 1 -- 172.21.15.142:0/1530015181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc009550 con 0x7f3ddc106f90 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.290+0000 7f3dda59c700 1 --2- 172.21.15.142:0/1530015181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 0x7f3ddc19a1c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc003f80 tx=0x7f3dcc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.290+0000 7f3dcb7fe700 1 -- 172.21.15.142:0/1530015181 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3dcc003a80 con 0x7f3ddc106f90 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.290+0000 7f3de0f7e700 1 -- 172.21.15.142:0/1530015181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ddc19a900 con 0x7f3ddc106f90 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.290+0000 7f3dcb7fe700 1 -- 172.21.15.142:0/1530015181 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3dcc003be0 con 0x7f3ddc106f90 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.290+0000 7f3dcb7fe700 1 -- 172.21.15.142:0/1530015181 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3dcc019820 con 0x7f3ddc106f90 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.290+0000 7f3de0f7e700 1 -- 172.21.15.142:0/1530015181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ddc19ada0 con 0x7f3ddc106f90 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.291+0000 7f3dcb7fe700 1 -- 172.21.15.142:0/1530015181 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f3dcc019a50 con 0x7f3ddc106f90 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.291+0000 7f3dcb7fe700 1 --2- 172.21.15.142:0/1530015181 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f3dc40381f0 0x7f3dc403a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.291+0000 7f3de0f7e700 1 -- 172.21.15.142:0/1530015181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ddc10a640 con 0x7f3ddc106f90 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.291+0000 7f3dcb7fe700 1 -- 172.21.15.142:0/1530015181 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3dcc04c390 con 0x7f3ddc106f90 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.291+0000 7f3dd9d9b700 1 --2- 172.21.15.142:0/1530015181 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f3dc40381f0 0x7f3dc403a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.292+0000 7f3dd9d9b700 1 --2- 172.21.15.142:0/1530015181 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f3dc40381f0 0x7f3dc403a6b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0006fd0 tx=0x7f3dd0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.296+0000 7f3dcb7fe700 1 -- 172.21.15.142:0/1530015181 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3dcc02c3a0 con 0x7f3ddc106f90 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.500+0000 7f3de0f7e700 1 -- 172.21.15.142:0/1530015181 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ddc101e90 con 0x7f3dc40381f0 2024-03-29T04:35:36.127 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.505+0000 7f3dcb7fe700 1 -- 172.21.15.142:0/1530015181 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3ddc101e90 con 0x7f3dc40381f0 2024-03-29T04:35:36.128 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.507+0000 7f3de0f7e700 1 -- 172.21.15.142:0/1530015181 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f3dc40381f0 msgr2=0x7f3dc403a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:36.128 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.507+0000 7f3de0f7e700 1 --2- 172.21.15.142:0/1530015181 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f3dc40381f0 0x7f3dc403a6b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0006fd0 tx=0x7f3dd0006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:36.128 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.507+0000 7f3de0f7e700 1 -- 172.21.15.142:0/1530015181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 msgr2=0x7f3ddc19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:36.128 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.507+0000 7f3de0f7e700 1 --2- 172.21.15.142:0/1530015181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 0x7f3ddc19a1c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc003f80 tx=0x7f3dcc011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:36.128 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.507+0000 7f3de0f7e700 1 -- 172.21.15.142:0/1530015181 shutdown_connections 2024-03-29T04:35:36.128 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.507+0000 7f3de0f7e700 1 --2- 172.21.15.142:0/1530015181 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f3dc40381f0 0x7f3dc403a6b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:36.128 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.507+0000 7f3de0f7e700 1 --2- 172.21.15.142:0/1530015181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ddc106f90 0x7f3ddc19a1c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:36.128 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.507+0000 7f3de0f7e700 1 -- 172.21.15.142:0/1530015181 >> 172.21.15.142:0/1530015181 conn(0x7f3ddc100980 msgr2=0x7f3ddc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:36.128 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.507+0000 7f3de0f7e700 1 -- 172.21.15.142:0/1530015181 shutdown_connections 2024-03-29T04:35:36.128 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:35.507+0000 7f3de0f7e700 1 -- 172.21.15.142:0/1530015181 wait complete. 2024-03-29T04:35:36.755 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:36 smithi142 ceph-mon[29027]: [29/Mar/2024:04:35:34] ENGINE Bus STARTING 2024-03-29T04:35:36.755 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:36 smithi142 ceph-mon[29027]: [29/Mar/2024:04:35:35] ENGINE Serving on https://172.21.15.142:7150 2024-03-29T04:35:36.755 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:36 smithi142 ceph-mon[29027]: [29/Mar/2024:04:35:35] ENGINE Serving on http://172.21.15.142:8765 2024-03-29T04:35:36.755 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:36 smithi142 ceph-mon[29027]: [29/Mar/2024:04:35:35] ENGINE Bus STARTED 2024-03-29T04:35:36.756 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:36 smithi142 ceph-mon[29027]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:36.756 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:36 smithi142 ceph-mon[29027]: mgrmap e8: smithi142.swesrg(active, since 2s) 2024-03-29T04:35:37.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-29T04:35:37.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.570+0000 7f2e9d56f700 1 Processor -- start 2024-03-29T04:35:37.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.570+0000 7f2e9d56f700 1 -- start start 2024-03-29T04:35:37.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.570+0000 7f2e9d56f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 0x7f2e98105880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:37.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.570+0000 7f2e9d56f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e98105e50 con 0x7f2e98105460 2024-03-29T04:35:37.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.570+0000 7f2e96ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 0x7f2e98105880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:37.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.570+0000 7f2e96ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 0x7f2e98105880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33224/0 (socket says 172.21.15.142:33224) 2024-03-29T04:35:37.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.570+0000 7f2e96ffd700 1 -- 172.21.15.142:0/1182626824 learned_addr learned my addr 172.21.15.142:0/1182626824 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:37.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.571+0000 7f2e96ffd700 1 -- 172.21.15.142:0/1182626824 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e98106660 con 0x7f2e98105460 2024-03-29T04:35:37.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.571+0000 7f2e96ffd700 1 --2- 172.21.15.142:0/1182626824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 0x7f2e98105880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2e880098a0 tx=0x7f2e88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b97157ea9b160a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:37.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.571+0000 7f2e95ffb700 1 -- 172.21.15.142:0/1182626824 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e88004180 con 0x7f2e98105460 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.571+0000 7f2e95ffb700 1 -- 172.21.15.142:0/1182626824 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e880042e0 con 0x7f2e98105460 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.571+0000 7f2e95ffb700 1 -- 172.21.15.142:0/1182626824 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e88011a50 con 0x7f2e98105460 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.571+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1182626824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 msgr2=0x7f2e98105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.571+0000 7f2e9d56f700 1 --2- 172.21.15.142:0/1182626824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 0x7f2e98105880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2e880098a0 tx=0x7f2e88009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.571+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1182626824 shutdown_connections 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.571+0000 7f2e9d56f700 1 --2- 172.21.15.142:0/1182626824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 0x7f2e98105880 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.571+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1182626824 >> 172.21.15.142:0/1182626824 conn(0x7f2e98100a20 msgr2=0x7f2e98102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.572+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1182626824 shutdown_connections 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.572+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1182626824 wait complete. 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.572+0000 7f2e9d56f700 1 Processor -- start 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.572+0000 7f2e9d56f700 1 -- start start 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.572+0000 7f2e9d56f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 0x7f2e9819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.572+0000 7f2e9d56f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9819a740 con 0x7f2e98105460 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.572+0000 7f2e96ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 0x7f2e9819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.573+0000 7f2e96ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 0x7f2e9819a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33230/0 (socket says 172.21.15.142:33230) 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.573+0000 7f2e96ffd700 1 -- 172.21.15.142:0/1780168674 learned_addr learned my addr 172.21.15.142:0/1780168674 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.573+0000 7f2e96ffd700 1 -- 172.21.15.142:0/1780168674 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e88009550 con 0x7f2e98105460 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.573+0000 7f2e96ffd700 1 --2- 172.21.15.142:0/1780168674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 0x7f2e9819a200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009520 tx=0x7f2e88011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.573+0000 7f2e7ffff700 1 -- 172.21.15.142:0/1780168674 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e88003a80 con 0x7f2e98105460 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.573+0000 7f2e7ffff700 1 -- 172.21.15.142:0/1780168674 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e88003be0 con 0x7f2e98105460 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.573+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1780168674 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9819a940 con 0x7f2e98105460 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.573+0000 7f2e7ffff700 1 -- 172.21.15.142:0/1780168674 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e880197b0 con 0x7f2e98105460 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.573+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1780168674 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9819ade0 con 0x7f2e98105460 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.574+0000 7f2e7ffff700 1 -- 172.21.15.142:0/1780168674 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f2e88019a70 con 0x7f2e98105460 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.574+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1780168674 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e980623c0 con 0x7f2e98105460 2024-03-29T04:35:37.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.574+0000 7f2e7ffff700 1 --2- 172.21.15.142:0/1780168674 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f2e800382d0 0x7f2e8003a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.574+0000 7f2e7ffff700 1 -- 172.21.15.142:0/1780168674 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2e8804c590 con 0x7f2e98105460 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.574+0000 7f2e967fc700 1 --2- 172.21.15.142:0/1780168674 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f2e800382d0 0x7f2e8003a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.575+0000 7f2e967fc700 1 --2- 172.21.15.142:0/1780168674 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f2e800382d0 0x7f2e8003a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c006fd0 tx=0x7f2e8c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.579+0000 7f2e7ffff700 1 -- 172.21.15.142:0/1780168674 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2e8802b750 con 0x7f2e98105460 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.769+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1780168674 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e98109860 con 0x7f2e800382d0 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.770+0000 7f2e7ffff700 1 -- 172.21.15.142:0/1780168674 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f2e98109860 con 0x7f2e800382d0 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.772+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1780168674 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f2e800382d0 msgr2=0x7f2e8003a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.772+0000 7f2e9d56f700 1 --2- 172.21.15.142:0/1780168674 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f2e800382d0 0x7f2e8003a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c006fd0 tx=0x7f2e8c006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.772+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1780168674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 msgr2=0x7f2e9819a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.772+0000 7f2e9d56f700 1 --2- 172.21.15.142:0/1780168674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 0x7f2e9819a200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009520 tx=0x7f2e88011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.772+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1780168674 shutdown_connections 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.772+0000 7f2e9d56f700 1 --2- 172.21.15.142:0/1780168674 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f2e800382d0 0x7f2e8003a790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:37.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.772+0000 7f2e9d56f700 1 --2- 172.21.15.142:0/1780168674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e98105460 0x7f2e9819a200 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:37.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.772+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1780168674 >> 172.21.15.142:0/1780168674 conn(0x7f2e98100a20 msgr2=0x7f2e98079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:37.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.772+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1780168674 shutdown_connections 2024-03-29T04:35:37.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:36.772+0000 7f2e9d56f700 1 -- 172.21.15.142:0/1780168674 wait complete. 2024-03-29T04:35:37.344 INFO:teuthology.orchestra.run.smithi142.stdout:Generating ssh key... 2024-03-29T04:35:37.857 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:37 smithi142 ceph-mon[29027]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:38.572 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.797+0000 7f43519a8700 1 Processor -- start 2024-03-29T04:35:38.572 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.797+0000 7f43519a8700 1 -- start start 2024-03-29T04:35:38.572 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.797+0000 7f43519a8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 0x7f434c106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:38.572 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.797+0000 7f43519a8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c106f50 con 0x7f434c106560 2024-03-29T04:35:38.572 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.798+0000 7f434affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 0x7f434c106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:38.572 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.798+0000 7f434affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 0x7f434c106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33232/0 (socket says 172.21.15.142:33232) 2024-03-29T04:35:38.572 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.798+0000 7f434affd700 1 -- 172.21.15.142:0/1804650875 learned_addr learned my addr 172.21.15.142:0/1804650875 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:38.572 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.798+0000 7f434affd700 1 -- 172.21.15.142:0/1804650875 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c107760 con 0x7f434c106560 2024-03-29T04:35:38.572 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.798+0000 7f434affd700 1 --2- 172.21.15.142:0/1804650875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 0x7f434c106980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f433c0098a0 tx=0x7f433c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39dfac3a30042a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:38.572 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.798+0000 7f434a7fc700 1 -- 172.21.15.142:0/1804650875 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f433c004180 con 0x7f434c106560 2024-03-29T04:35:38.572 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.798+0000 7f434a7fc700 1 -- 172.21.15.142:0/1804650875 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f433c0042e0 con 0x7f434c106560 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.798+0000 7f434a7fc700 1 -- 172.21.15.142:0/1804650875 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f433c011a50 con 0x7f434c106560 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.799+0000 7f43519a8700 1 -- 172.21.15.142:0/1804650875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 msgr2=0x7f434c106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.799+0000 7f43519a8700 1 --2- 172.21.15.142:0/1804650875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 0x7f434c106980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f433c0098a0 tx=0x7f433c009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.799+0000 7f43519a8700 1 -- 172.21.15.142:0/1804650875 shutdown_connections 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.799+0000 7f43519a8700 1 --2- 172.21.15.142:0/1804650875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 0x7f434c106980 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.799+0000 7f43519a8700 1 -- 172.21.15.142:0/1804650875 >> 172.21.15.142:0/1804650875 conn(0x7f434c101b40 msgr2=0x7f434c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.799+0000 7f43519a8700 1 -- 172.21.15.142:0/1804650875 shutdown_connections 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.799+0000 7f43519a8700 1 -- 172.21.15.142:0/1804650875 wait complete. 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.799+0000 7f43519a8700 1 Processor -- start 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.799+0000 7f43519a8700 1 -- start start 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.800+0000 7f43519a8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 0x7f434c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.800+0000 7f43519a8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c19a700 con 0x7f434c106560 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.800+0000 7f434affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 0x7f434c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.800+0000 7f434affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 0x7f434c19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33246/0 (socket says 172.21.15.142:33246) 2024-03-29T04:35:38.573 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.800+0000 7f434affd700 1 -- 172.21.15.142:0/2883454195 learned_addr learned my addr 172.21.15.142:0/2883454195 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:38.574 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.800+0000 7f434affd700 1 -- 172.21.15.142:0/2883454195 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c009550 con 0x7f434c106560 2024-03-29T04:35:38.574 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.800+0000 7f434affd700 1 --2- 172.21.15.142:0/2883454195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 0x7f434c19a1c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f433c000c00 tx=0x7f433c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:38.574 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.800+0000 7f43509a6700 1 -- 172.21.15.142:0/2883454195 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f433c003b90 con 0x7f434c106560 2024-03-29T04:35:38.574 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.800+0000 7f43509a6700 1 -- 172.21.15.142:0/2883454195 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f433c0248b0 con 0x7f434c106560 2024-03-29T04:35:38.574 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.800+0000 7f43519a8700 1 -- 172.21.15.142:0/2883454195 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434c19a900 con 0x7f434c106560 2024-03-29T04:35:38.574 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.800+0000 7f43509a6700 1 -- 172.21.15.142:0/2883454195 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f433c01a8e0 con 0x7f434c106560 2024-03-29T04:35:38.574 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.800+0000 7f43519a8700 1 -- 172.21.15.142:0/2883454195 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434c19ada0 con 0x7f434c106560 2024-03-29T04:35:38.574 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.801+0000 7f43509a6700 1 -- 172.21.15.142:0/2883454195 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f433c01aba0 con 0x7f434c106560 2024-03-29T04:35:38.574 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.801+0000 7f43519a8700 1 -- 172.21.15.142:0/2883454195 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434c0623c0 con 0x7f434c106560 2024-03-29T04:35:38.575 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.801+0000 7f43509a6700 1 --2- 172.21.15.142:0/2883454195 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f4338038320 0x7f433803a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:38.575 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.801+0000 7f43509a6700 1 -- 172.21.15.142:0/2883454195 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f433c04c430 con 0x7f434c106560 2024-03-29T04:35:38.575 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.801+0000 7f43427fc700 1 --2- 172.21.15.142:0/2883454195 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f4338038320 0x7f433803a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:38.575 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.802+0000 7f43427fc700 1 --2- 172.21.15.142:0/2883454195 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f4338038320 0x7f433803a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4334006fd0 tx=0x7f4334006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:38.575 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.806+0000 7f43509a6700 1 -- 172.21.15.142:0/2883454195 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f433c0233a0 con 0x7f434c106560 2024-03-29T04:35:38.575 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:37.997+0000 7f43519a8700 1 -- 172.21.15.142:0/2883454195 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f434c102f80 con 0x7f4338038320 2024-03-29T04:35:38.575 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.052+0000 7f43509a6700 1 -- 172.21.15.142:0/2883454195 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f434c102f80 con 0x7f4338038320 2024-03-29T04:35:38.575 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.054+0000 7f43519a8700 1 -- 172.21.15.142:0/2883454195 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f4338038320 msgr2=0x7f433803a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:38.575 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.054+0000 7f43519a8700 1 --2- 172.21.15.142:0/2883454195 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f4338038320 0x7f433803a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4334006fd0 tx=0x7f4334006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:38.575 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.054+0000 7f43519a8700 1 -- 172.21.15.142:0/2883454195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 msgr2=0x7f434c19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:38.575 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.054+0000 7f43519a8700 1 --2- 172.21.15.142:0/2883454195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 0x7f434c19a1c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f433c000c00 tx=0x7f433c011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:38.575 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.055+0000 7f43519a8700 1 -- 172.21.15.142:0/2883454195 shutdown_connections 2024-03-29T04:35:38.576 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.055+0000 7f43519a8700 1 --2- 172.21.15.142:0/2883454195 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f4338038320 0x7f433803a7e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:38.576 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.055+0000 7f43519a8700 1 --2- 172.21.15.142:0/2883454195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f434c106560 0x7f434c19a1c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:38.576 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.055+0000 7f43519a8700 1 -- 172.21.15.142:0/2883454195 >> 172.21.15.142:0/2883454195 conn(0x7f434c101b40 msgr2=0x7f434c102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:38.576 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.055+0000 7f43519a8700 1 -- 172.21.15.142:0/2883454195 shutdown_connections 2024-03-29T04:35:38.576 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.055+0000 7f43519a8700 1 -- 172.21.15.142:0/2883454195 wait complete. 2024-03-29T04:35:39.180 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:39 smithi142 ceph-mon[29027]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:39.180 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:39 smithi142 ceph-mon[29027]: Generating ssh key... 2024-03-29T04:35:39.180 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:39 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:39.180 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:39 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:39.754 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyrWIhg0Jw/09phX82LS12t9kKJ9C2aYHfPk9qgesnBPZ3pzdAqdSgTfJNJMzYZlPr0Z+8URVxnSLng35dm/4kmQQLBpSlEO4DCfRRW3Nw0kD6Sxq6ktYpgsrOooobt+gy9M4WjZpIrzGnvNXMxcJZUcHwazDSKxoX2fupAVgB56h/iP/HIU6z0kuPbUIi/hDv0ouK2MJUYTaU4Z+qmMF6CV+acztBRBPambhWvUPgtIX2t1ntiJMHdxWZWHLKhH9pw6Apbc2Jf25Sg324gIdw1oblH2McYxG8m6ob+l/DA/lJBdQfEnY/jJWQmN2j84/McuJS2HlZKgSQTiLy5w1jsNfWqH95IYcBT+CN/FdxrszqNczIw8o9x1x4A/xKdO9XHkxhhsticv0LaxZ/RYroKitTRZ1vwQzVexMDwMixE3f1aDXmmBMgV+X5odW80QTRikb6FoQtm/vdr6jR5saT5KvBaQ8AxApHw9gBPZvJrscbVf/mwKcTY8cCYE+gdD0= ceph-9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:35:39.755 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.996+0000 7fc9b4cc1700 1 Processor -- start 2024-03-29T04:35:39.755 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.996+0000 7fc9b4cc1700 1 -- start start 2024-03-29T04:35:39.755 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.996+0000 7fc9b4cc1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 0x7fc9b0105270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:39.755 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.996+0000 7fc9b4cc1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b0079e30 con 0x7fc9b0104e50 2024-03-29T04:35:39.755 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.996+0000 7fc9ae59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 0x7fc9b0105270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:39.755 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.996+0000 7fc9ae59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 0x7fc9b0105270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42938/0 (socket says 172.21.15.142:42938) 2024-03-29T04:35:39.755 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.996+0000 7fc9ae59c700 1 -- 172.21.15.142:0/1184032311 learned_addr learned my addr 172.21.15.142:0/1184032311 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:39.755 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.996+0000 7fc9ae59c700 1 -- 172.21.15.142:0/1184032311 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b01057b0 con 0x7fc9b0104e50 2024-03-29T04:35:39.755 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.997+0000 7fc9ae59c700 1 --2- 172.21.15.142:0/1184032311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 0x7fc9b0105270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00098a0 tx=0x7fc9a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde3e300010c7490 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:39.755 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.997+0000 7fc9ad59a700 1 -- 172.21.15.142:0/1184032311 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9a0004180 con 0x7fc9b0104e50 2024-03-29T04:35:39.755 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.997+0000 7fc9ad59a700 1 -- 172.21.15.142:0/1184032311 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc9a00042e0 con 0x7fc9b0104e50 2024-03-29T04:35:39.756 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.997+0000 7fc9ad59a700 1 -- 172.21.15.142:0/1184032311 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9a0011a50 con 0x7fc9b0104e50 2024-03-29T04:35:39.756 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.997+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/1184032311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 msgr2=0x7fc9b0105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:39.756 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.997+0000 7fc9b4cc1700 1 --2- 172.21.15.142:0/1184032311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 0x7fc9b0105270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00098a0 tx=0x7fc9a0009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:39.756 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.998+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/1184032311 shutdown_connections 2024-03-29T04:35:39.756 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.998+0000 7fc9b4cc1700 1 --2- 172.21.15.142:0/1184032311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 0x7fc9b0105270 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:39.756 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.998+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/1184032311 >> 172.21.15.142:0/1184032311 conn(0x7fc9b0100a50 msgr2=0x7fc9b0102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:39.756 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.998+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/1184032311 shutdown_connections 2024-03-29T04:35:39.756 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.998+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/1184032311 wait complete. 2024-03-29T04:35:39.756 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.998+0000 7fc9b4cc1700 1 Processor -- start 2024-03-29T04:35:39.756 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.998+0000 7fc9b4cc1700 1 -- start start 2024-03-29T04:35:39.756 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.999+0000 7fc9b4cc1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 0x7fc9b019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:39.756 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.999+0000 7fc9b4cc1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b019c960 con 0x7fc9b0104e50 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.999+0000 7fc9ae59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 0x7fc9b019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.999+0000 7fc9ae59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 0x7fc9b019c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42942/0 (socket says 172.21.15.142:42942) 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.999+0000 7fc9ae59c700 1 -- 172.21.15.142:0/803756718 learned_addr learned my addr 172.21.15.142:0/803756718 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.999+0000 7fc9ae59c700 1 -- 172.21.15.142:0/803756718 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a0009550 con 0x7fc9b0104e50 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.999+0000 7fc9ae59c700 1 --2- 172.21.15.142:0/803756718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 0x7fc9b019c420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0003f80 tx=0x7fc9a0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.999+0000 7fc99f7fe700 1 -- 172.21.15.142:0/803756718 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9a0003a80 con 0x7fc9b0104e50 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.999+0000 7fc99f7fe700 1 -- 172.21.15.142:0/803756718 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc9a0003be0 con 0x7fc9b0104e50 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.999+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/803756718 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b019cb60 con 0x7fc9b0104e50 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.999+0000 7fc99f7fe700 1 -- 172.21.15.142:0/803756718 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9a0019820 con 0x7fc9b0104e50 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:38.999+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/803756718 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b019d000 con 0x7fc9b0104e50 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.000+0000 7fc99f7fe700 1 -- 172.21.15.142:0/803756718 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fc9a0019a50 con 0x7fc9b0104e50 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.000+0000 7fc99f7fe700 1 --2- 172.21.15.142:0/803756718 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fc998038320 0x7fc99803a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:39.757 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.000+0000 7fc99f7fe700 1 -- 172.21.15.142:0/803756718 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc9a004c5c0 con 0x7fc9b0104e50 2024-03-29T04:35:39.758 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.000+0000 7fc9add9b700 1 --2- 172.21.15.142:0/803756718 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fc998038320 0x7fc99803a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:39.758 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.000+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/803756718 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b0195cd0 con 0x7fc9b0104e50 2024-03-29T04:35:39.758 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.001+0000 7fc9add9b700 1 --2- 172.21.15.142:0/803756718 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fc998038320 0x7fc99803a7e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4006fd0 tx=0x7fc9a4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:39.758 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.005+0000 7fc99f7fe700 1 -- 172.21.15.142:0/803756718 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc9a002c3a0 con 0x7fc9b0104e50 2024-03-29T04:35:39.758 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.194+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/803756718 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9b00611d0 con 0x7fc998038320 2024-03-29T04:35:39.758 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.195+0000 7fc99f7fe700 1 -- 172.21.15.142:0/803756718 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fc9b00611d0 con 0x7fc998038320 2024-03-29T04:35:39.758 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.196+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/803756718 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fc998038320 msgr2=0x7fc99803a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:39.758 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.196+0000 7fc9b4cc1700 1 --2- 172.21.15.142:0/803756718 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fc998038320 0x7fc99803a7e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4006fd0 tx=0x7fc9a4006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:39.758 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.196+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/803756718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 msgr2=0x7fc9b019c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:39.758 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.197+0000 7fc9b4cc1700 1 --2- 172.21.15.142:0/803756718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 0x7fc9b019c420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0003f80 tx=0x7fc9a0011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:39.758 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.197+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/803756718 shutdown_connections 2024-03-29T04:35:39.758 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.197+0000 7fc9b4cc1700 1 --2- 172.21.15.142:0/803756718 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fc998038320 0x7fc99803a7e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:39.759 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.197+0000 7fc9b4cc1700 1 --2- 172.21.15.142:0/803756718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b0104e50 0x7fc9b019c420 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:39.759 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.197+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/803756718 >> 172.21.15.142:0/803756718 conn(0x7fc9b0100a50 msgr2=0x7fc9b0190a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:39.759 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.197+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/803756718 shutdown_connections 2024-03-29T04:35:39.759 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:39.197+0000 7fc9b4cc1700 1 -- 172.21.15.142:0/803756718 wait complete. 2024-03-29T04:35:39.759 INFO:teuthology.orchestra.run.smithi142.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-29T04:35:39.759 INFO:teuthology.orchestra.run.smithi142.stdout:Adding key to root@localhost authorized_keys... 2024-03-29T04:35:39.759 INFO:teuthology.orchestra.run.smithi142.stdout:Adding host smithi142... 2024-03-29T04:35:40.350 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:40 smithi142 ceph-mon[29027]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:41.301 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:41 smithi142 ceph-mon[29027]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "addr": "172.21.15.142", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:42.283 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:42 smithi142 ceph-mon[29027]: Deploying cephadm binary to smithi142 2024-03-29T04:35:43.453 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout Added host 'smithi142' with addr '172.21.15.142' 2024-03-29T04:35:43.453 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.164+0000 7f6cf98c6700 1 Processor -- start 2024-03-29T04:35:43.453 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.164+0000 7f6cf98c6700 1 -- start start 2024-03-29T04:35:43.453 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.164+0000 7f6cf98c6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 0x7f6cf4109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:43.454 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.164+0000 7f6cf98c6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4074640 con 0x7f6cf4107020 2024-03-29T04:35:43.454 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.164+0000 7f6cf2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 0x7f6cf4109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:43.454 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.164+0000 7f6cf2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 0x7f6cf4109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42954/0 (socket says 172.21.15.142:42954) 2024-03-29T04:35:43.454 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.164+0000 7f6cf2ffd700 1 -- 172.21.15.142:0/3719794624 learned_addr learned my addr 172.21.15.142:0/3719794624 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:43.454 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.164+0000 7f6cf2ffd700 1 -- 172.21.15.142:0/3719794624 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf4109950 con 0x7f6cf4107020 2024-03-29T04:35:43.454 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.164+0000 7f6cf2ffd700 1 --2- 172.21.15.142:0/3719794624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 0x7f6cf4109410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40098a0 tx=0x7f6ce4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb5c509c65fd370 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:43.454 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.165+0000 7f6cf1ffb700 1 -- 172.21.15.142:0/3719794624 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ce4004180 con 0x7f6cf4107020 2024-03-29T04:35:43.454 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.165+0000 7f6cf1ffb700 1 -- 172.21.15.142:0/3719794624 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ce40042e0 con 0x7f6cf4107020 2024-03-29T04:35:43.454 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.165+0000 7f6cf1ffb700 1 -- 172.21.15.142:0/3719794624 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ce4011a50 con 0x7f6cf4107020 2024-03-29T04:35:43.454 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.165+0000 7f6cf98c6700 1 -- 172.21.15.142:0/3719794624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 msgr2=0x7f6cf4109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:43.454 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.165+0000 7f6cf98c6700 1 --2- 172.21.15.142:0/3719794624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 0x7f6cf4109410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40098a0 tx=0x7f6ce4009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:43.454 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.165+0000 7f6cf98c6700 1 -- 172.21.15.142:0/3719794624 shutdown_connections 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.165+0000 7f6cf98c6700 1 --2- 172.21.15.142:0/3719794624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 0x7f6cf4109410 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.165+0000 7f6cf98c6700 1 -- 172.21.15.142:0/3719794624 >> 172.21.15.142:0/3719794624 conn(0x7f6cf41009f0 msgr2=0x7f6cf4102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.165+0000 7f6cf98c6700 1 -- 172.21.15.142:0/3719794624 shutdown_connections 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.165+0000 7f6cf98c6700 1 -- 172.21.15.142:0/3719794624 wait complete. 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.166+0000 7f6cf98c6700 1 Processor -- start 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.166+0000 7f6cf98c6700 1 -- start start 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.166+0000 7f6cf98c6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 0x7f6cf419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.166+0000 7f6cf98c6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf419a680 con 0x7f6cf4107020 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.166+0000 7f6cf2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 0x7f6cf419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.166+0000 7f6cf2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 0x7f6cf419a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42966/0 (socket says 172.21.15.142:42966) 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.166+0000 7f6cf2ffd700 1 -- 172.21.15.142:0/214869728 learned_addr learned my addr 172.21.15.142:0/214869728 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.166+0000 7f6cf2ffd700 1 -- 172.21.15.142:0/214869728 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4009550 con 0x7f6cf4107020 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.167+0000 7f6cf2ffd700 1 --2- 172.21.15.142:0/214869728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 0x7f6cf419a140 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40119e0 tx=0x7f6ce4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:43.455 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.167+0000 7f6cf88c4700 1 -- 172.21.15.142:0/214869728 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ce4003b90 con 0x7f6cf4107020 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.167+0000 7f6cf88c4700 1 -- 172.21.15.142:0/214869728 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ce401a8a0 con 0x7f6cf4107020 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.167+0000 7f6cf98c6700 1 -- 172.21.15.142:0/214869728 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf419a880 con 0x7f6cf4107020 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.167+0000 7f6cf88c4700 1 -- 172.21.15.142:0/214869728 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ce4019930 con 0x7f6cf4107020 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.167+0000 7f6cf98c6700 1 -- 172.21.15.142:0/214869728 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf419ad20 con 0x7f6cf4107020 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.168+0000 7f6cf88c4700 1 -- 172.21.15.142:0/214869728 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f6ce4019ba0 con 0x7f6cf4107020 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.168+0000 7f6cf88c4700 1 --2- 172.21.15.142:0/214869728 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f6cdc038330 0x7f6cdc03a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.168+0000 7f6cf88c4700 1 -- 172.21.15.142:0/214869728 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6ce404c7d0 con 0x7f6cf4107020 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.168+0000 7f6cf98c6700 1 -- 172.21.15.142:0/214869728 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf410a6a0 con 0x7f6cf4107020 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.168+0000 7f6cf27fc700 1 --2- 172.21.15.142:0/214869728 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f6cdc038330 0x7f6cdc03a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.168+0000 7f6cf27fc700 1 --2- 172.21.15.142:0/214869728 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f6cdc038330 0x7f6cdc03a7f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006fd0 tx=0x7f6ce8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.173+0000 7f6cf88c4700 1 -- 172.21.15.142:0/214869728 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6ce402d440 con 0x7f6cf4107020 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:40.364+0000 7f6cf98c6700 1 -- 172.21.15.142:0/214869728 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi142", "addr": "172.21.15.142", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cf4101e50 con 0x7f6cdc038330 2024-03-29T04:35:43.456 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:42.884+0000 7f6cf88c4700 1 -- 172.21.15.142:0/214869728 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f6cf4101e50 con 0x7f6cdc038330 2024-03-29T04:35:43.457 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:42.886+0000 7f6cf98c6700 1 -- 172.21.15.142:0/214869728 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f6cdc038330 msgr2=0x7f6cdc03a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:43.457 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:42.886+0000 7f6cf98c6700 1 --2- 172.21.15.142:0/214869728 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f6cdc038330 0x7f6cdc03a7f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006fd0 tx=0x7f6ce8006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:43.457 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:42.886+0000 7f6cf98c6700 1 -- 172.21.15.142:0/214869728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 msgr2=0x7f6cf419a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:43.457 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:42.886+0000 7f6cf98c6700 1 --2- 172.21.15.142:0/214869728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 0x7f6cf419a140 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40119e0 tx=0x7f6ce4011ac0 comp rx=0 tx=0).stop 2024-03-29T04:35:43.457 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:42.886+0000 7f6cf98c6700 1 -- 172.21.15.142:0/214869728 shutdown_connections 2024-03-29T04:35:43.457 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:42.886+0000 7f6cf98c6700 1 --2- 172.21.15.142:0/214869728 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f6cdc038330 0x7f6cdc03a7f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:43.457 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:42.886+0000 7f6cf98c6700 1 --2- 172.21.15.142:0/214869728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4107020 0x7f6cf419a140 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:43.457 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:42.887+0000 7f6cf98c6700 1 -- 172.21.15.142:0/214869728 >> 172.21.15.142:0/214869728 conn(0x7f6cf41009f0 msgr2=0x7f6cf4101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:43.457 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:42.887+0000 7f6cf98c6700 1 -- 172.21.15.142:0/214869728 shutdown_connections 2024-03-29T04:35:43.457 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:42.887+0000 7f6cf98c6700 1 -- 172.21.15.142:0/214869728 wait complete. 2024-03-29T04:35:43.457 INFO:teuthology.orchestra.run.smithi142.stdout:Deploying mon service with default placement... 2024-03-29T04:35:44.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:43 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:44.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:43 smithi142 ceph-mon[29027]: Added host smithi142 2024-03-29T04:35:44.056 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:43 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:35:45.357 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:45 smithi142 ceph-mon[29027]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:45.358 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:45 smithi142 ceph-mon[29027]: Saving service mon spec with placement count:5 2024-03-29T04:35:45.358 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:45 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:45.566 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-29T04:35:45.566 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.853+0000 7f0e718bc700 1 Processor -- start 2024-03-29T04:35:45.567 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.853+0000 7f0e718bc700 1 -- start start 2024-03-29T04:35:45.567 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.853+0000 7f0e718bc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 0x7f0e6c106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:45.567 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.853+0000 7f0e718bc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c106f50 con 0x7f0e6c106560 2024-03-29T04:35:45.567 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.853+0000 7f0e6affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 0x7f0e6c106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:45.567 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.854+0000 7f0e6affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 0x7f0e6c106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42980/0 (socket says 172.21.15.142:42980) 2024-03-29T04:35:45.567 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.854+0000 7f0e6affd700 1 -- 172.21.15.142:0/2506078829 learned_addr learned my addr 172.21.15.142:0/2506078829 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:45.567 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.854+0000 7f0e6affd700 1 -- 172.21.15.142:0/2506078829 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c107760 con 0x7f0e6c106560 2024-03-29T04:35:45.567 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.854+0000 7f0e6affd700 1 --2- 172.21.15.142:0/2506078829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 0x7f0e6c106980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0e600098a0 tx=0x7f0e60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6852d1a0896b3a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:45.567 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.854+0000 7f0e69ffb700 1 -- 172.21.15.142:0/2506078829 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e60004180 con 0x7f0e6c106560 2024-03-29T04:35:45.567 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.854+0000 7f0e69ffb700 1 -- 172.21.15.142:0/2506078829 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0e600042e0 con 0x7f0e6c106560 2024-03-29T04:35:45.567 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.854+0000 7f0e69ffb700 1 -- 172.21.15.142:0/2506078829 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e60011a50 con 0x7f0e6c106560 2024-03-29T04:35:45.567 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.855+0000 7f0e718bc700 1 -- 172.21.15.142:0/2506078829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 msgr2=0x7f0e6c106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.855+0000 7f0e718bc700 1 --2- 172.21.15.142:0/2506078829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 0x7f0e6c106980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0e600098a0 tx=0x7f0e60009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.855+0000 7f0e718bc700 1 -- 172.21.15.142:0/2506078829 shutdown_connections 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.855+0000 7f0e718bc700 1 --2- 172.21.15.142:0/2506078829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 0x7f0e6c106980 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.855+0000 7f0e718bc700 1 -- 172.21.15.142:0/2506078829 >> 172.21.15.142:0/2506078829 conn(0x7f0e6c101b40 msgr2=0x7f0e6c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.855+0000 7f0e718bc700 1 -- 172.21.15.142:0/2506078829 shutdown_connections 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.855+0000 7f0e718bc700 1 -- 172.21.15.142:0/2506078829 wait complete. 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.855+0000 7f0e718bc700 1 Processor -- start 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e718bc700 1 -- start start 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e718bc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 0x7f0e6c19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e718bc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c19a730 con 0x7f0e6c106560 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e6affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 0x7f0e6c19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e6affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 0x7f0e6c19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42992/0 (socket says 172.21.15.142:42992) 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e6affd700 1 -- 172.21.15.142:0/1644716095 learned_addr learned my addr 172.21.15.142:0/1644716095 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:45.568 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e6affd700 1 -- 172.21.15.142:0/1644716095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e60009550 con 0x7f0e6c106560 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e6affd700 1 --2- 172.21.15.142:0/1644716095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 0x7f0e6c19a1f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009520 tx=0x7f0e60011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e708ba700 1 -- 172.21.15.142:0/1644716095 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e60003a80 con 0x7f0e6c106560 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e718bc700 1 -- 172.21.15.142:0/1644716095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c19a930 con 0x7f0e6c106560 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e708ba700 1 -- 172.21.15.142:0/1644716095 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0e60003be0 con 0x7f0e6c106560 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e708ba700 1 -- 172.21.15.142:0/1644716095 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e600197b0 con 0x7f0e6c106560 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.856+0000 7f0e718bc700 1 -- 172.21.15.142:0/1644716095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c19add0 con 0x7f0e6c106560 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.857+0000 7f0e708ba700 1 -- 172.21.15.142:0/1644716095 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f0e60019a70 con 0x7f0e6c106560 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.857+0000 7f0e708ba700 1 --2- 172.21.15.142:0/1644716095 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0e540382d0 0x7f0e5403a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.857+0000 7f0e708ba700 1 -- 172.21.15.142:0/1644716095 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0e6004c760 con 0x7f0e6c106560 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.857+0000 7f0e718bc700 1 -- 172.21.15.142:0/1644716095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c0623c0 con 0x7f0e6c106560 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.857+0000 7f0e6a7fc700 1 --2- 172.21.15.142:0/1644716095 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0e540382d0 0x7f0e5403a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.858+0000 7f0e6a7fc700 1 --2- 172.21.15.142:0/1644716095 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0e540382d0 0x7f0e5403a790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c006fd0 tx=0x7f0e5c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:45.569 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:43.862+0000 7f0e708ba700 1 -- 172.21.15.142:0/1644716095 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0e6002c3a0 con 0x7f0e6c106560 2024-03-29T04:35:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:44.069+0000 7f0e718bc700 1 -- 172.21.15.142:0/1644716095 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e6c10a960 con 0x7f0e540382d0 2024-03-29T04:35:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:44.076+0000 7f0e708ba700 1 -- 172.21.15.142:0/1644716095 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0e6c10a960 con 0x7f0e540382d0 2024-03-29T04:35:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:44.079+0000 7f0e718bc700 1 -- 172.21.15.142:0/1644716095 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0e540382d0 msgr2=0x7f0e5403a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:44.079+0000 7f0e718bc700 1 --2- 172.21.15.142:0/1644716095 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0e540382d0 0x7f0e5403a790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c006fd0 tx=0x7f0e5c006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:44.079+0000 7f0e718bc700 1 -- 172.21.15.142:0/1644716095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 msgr2=0x7f0e6c19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:44.079+0000 7f0e718bc700 1 --2- 172.21.15.142:0/1644716095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 0x7f0e6c19a1f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009520 tx=0x7f0e60011ab0 comp rx=0 tx=0).stop 2024-03-29T04:35:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:44.079+0000 7f0e718bc700 1 -- 172.21.15.142:0/1644716095 shutdown_connections 2024-03-29T04:35:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:44.079+0000 7f0e718bc700 1 --2- 172.21.15.142:0/1644716095 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0e540382d0 0x7f0e5403a790 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:44.079+0000 7f0e718bc700 1 --2- 172.21.15.142:0/1644716095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e6c106560 0x7f0e6c19a1f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:44.079+0000 7f0e718bc700 1 -- 172.21.15.142:0/1644716095 >> 172.21.15.142:0/1644716095 conn(0x7f0e6c101b40 msgr2=0x7f0e6c102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:44.079+0000 7f0e718bc700 1 -- 172.21.15.142:0/1644716095 shutdown_connections 2024-03-29T04:35:45.570 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:44.079+0000 7f0e718bc700 1 -- 172.21.15.142:0/1644716095 wait complete. 2024-03-29T04:35:45.571 INFO:teuthology.orchestra.run.smithi142.stdout:Deploying mgr service with default placement... 2024-03-29T04:35:46.461 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:46 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:46.461 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:46 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:47.359 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-29T04:35:47.359 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.001+0000 7f0df1ab0700 1 Processor -- start 2024-03-29T04:35:47.359 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.001+0000 7f0df1ab0700 1 -- start start 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.001+0000 7f0df1ab0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 0x7f0dec1094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.001+0000 7f0df1ab0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec0745a0 con 0x7f0dec1070c0 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.001+0000 7f0deb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 0x7f0dec1094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.001+0000 7f0deb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 0x7f0dec1094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42996/0 (socket says 172.21.15.142:42996) 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.001+0000 7f0deb7fe700 1 -- 172.21.15.142:0/3768084405 learned_addr learned my addr 172.21.15.142:0/3768084405 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.001+0000 7f0deb7fe700 1 -- 172.21.15.142:0/3768084405 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec0746e0 con 0x7f0dec1070c0 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.002+0000 7f0deb7fe700 1 --2- 172.21.15.142:0/3768084405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 0x7f0dec1094b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0098a0 tx=0x7f0ddc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc144b96c08eecc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.002+0000 7f0dea7fc700 1 -- 172.21.15.142:0/3768084405 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ddc004180 con 0x7f0dec1070c0 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.002+0000 7f0dea7fc700 1 -- 172.21.15.142:0/3768084405 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ddc0042e0 con 0x7f0dec1070c0 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.002+0000 7f0dea7fc700 1 -- 172.21.15.142:0/3768084405 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ddc011af0 con 0x7f0dec1070c0 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.002+0000 7f0df1ab0700 1 -- 172.21.15.142:0/3768084405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 msgr2=0x7f0dec1094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.002+0000 7f0df1ab0700 1 --2- 172.21.15.142:0/3768084405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 0x7f0dec1094b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0098a0 tx=0x7f0ddc009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:47.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.002+0000 7f0df1ab0700 1 -- 172.21.15.142:0/3768084405 shutdown_connections 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.002+0000 7f0df1ab0700 1 --2- 172.21.15.142:0/3768084405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 0x7f0dec1094b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.002+0000 7f0df1ab0700 1 -- 172.21.15.142:0/3768084405 >> 172.21.15.142:0/3768084405 conn(0x7f0dec1009f0 msgr2=0x7f0dec102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.002+0000 7f0df1ab0700 1 -- 172.21.15.142:0/3768084405 shutdown_connections 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.002+0000 7f0df1ab0700 1 -- 172.21.15.142:0/3768084405 wait complete. 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.003+0000 7f0df1ab0700 1 Processor -- start 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.003+0000 7f0df1ab0700 1 -- start start 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.003+0000 7f0df1ab0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 0x7f0dec106a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.003+0000 7f0df1ab0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec106f60 con 0x7f0dec1070c0 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.003+0000 7f0deb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 0x7f0dec106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.003+0000 7f0deb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 0x7f0dec106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43004/0 (socket says 172.21.15.142:43004) 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.003+0000 7f0deb7fe700 1 -- 172.21.15.142:0/2128899474 learned_addr learned my addr 172.21.15.142:0/2128899474 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.004+0000 7f0deb7fe700 1 -- 172.21.15.142:0/2128899474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc009550 con 0x7f0dec1070c0 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.004+0000 7f0deb7fe700 1 --2- 172.21.15.142:0/2128899474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 0x7f0dec106a20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc011ad0 tx=0x7f0ddc011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.004+0000 7f0de8ff9700 1 -- 172.21.15.142:0/2128899474 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ddc003ab0 con 0x7f0dec1070c0 2024-03-29T04:35:47.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.004+0000 7f0de8ff9700 1 -- 172.21.15.142:0/2128899474 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ddc003c10 con 0x7f0dec1070c0 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.004+0000 7f0df1ab0700 1 -- 172.21.15.142:0/2128899474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec105070 con 0x7f0dec1070c0 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.004+0000 7f0de8ff9700 1 -- 172.21.15.142:0/2128899474 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ddc0198e0 con 0x7f0dec1070c0 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.004+0000 7f0df1ab0700 1 -- 172.21.15.142:0/2128899474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec105510 con 0x7f0dec1070c0 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.005+0000 7f0de8ff9700 1 -- 172.21.15.142:0/2128899474 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f0ddc019b50 con 0x7f0dec1070c0 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.005+0000 7f0de8ff9700 1 --2- 172.21.15.142:0/2128899474 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0dd4038370 0x7f0dd403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.005+0000 7f0df1ab0700 1 -- 172.21.15.142:0/2128899474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec10a710 con 0x7f0dec1070c0 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.005+0000 7f0de8ff9700 1 -- 172.21.15.142:0/2128899474 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0ddc04c2c0 con 0x7f0dec1070c0 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.005+0000 7f0deaffd700 1 --2- 172.21.15.142:0/2128899474 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0dd4038370 0x7f0dd403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.005+0000 7f0deaffd700 1 --2- 172.21.15.142:0/2128899474 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0dd4038370 0x7f0dd403a830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0de0006fd0 tx=0x7f0de0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.010+0000 7f0de8ff9700 1 -- 172.21.15.142:0/2128899474 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0ddc01a610 con 0x7f0dec1070c0 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.226+0000 7f0df1ab0700 1 -- 172.21.15.142:0/2128899474 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dec101e50 con 0x7f0dd4038370 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.233+0000 7f0de8ff9700 1 -- 172.21.15.142:0/2128899474 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0dec101e50 con 0x7f0dd4038370 2024-03-29T04:35:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.235+0000 7f0df1ab0700 1 -- 172.21.15.142:0/2128899474 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0dd4038370 msgr2=0x7f0dd403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:47.363 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.235+0000 7f0df1ab0700 1 --2- 172.21.15.142:0/2128899474 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0dd4038370 0x7f0dd403a830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0de0006fd0 tx=0x7f0de0006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:47.363 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.235+0000 7f0df1ab0700 1 -- 172.21.15.142:0/2128899474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 msgr2=0x7f0dec106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:47.363 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.235+0000 7f0df1ab0700 1 --2- 172.21.15.142:0/2128899474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 0x7f0dec106a20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc011ad0 tx=0x7f0ddc011bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:47.363 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.235+0000 7f0df1ab0700 1 -- 172.21.15.142:0/2128899474 shutdown_connections 2024-03-29T04:35:47.363 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.235+0000 7f0df1ab0700 1 --2- 172.21.15.142:0/2128899474 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0dd4038370 0x7f0dd403a830 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:47.363 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.235+0000 7f0df1ab0700 1 --2- 172.21.15.142:0/2128899474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dec1070c0 0x7f0dec106a20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:47.363 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.235+0000 7f0df1ab0700 1 -- 172.21.15.142:0/2128899474 >> 172.21.15.142:0/2128899474 conn(0x7f0dec1009f0 msgr2=0x7f0dec101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:47.363 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.235+0000 7f0df1ab0700 1 -- 172.21.15.142:0/2128899474 shutdown_connections 2024-03-29T04:35:47.363 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:46.235+0000 7f0df1ab0700 1 -- 172.21.15.142:0/2128899474 wait complete. 2024-03-29T04:35:47.363 INFO:teuthology.orchestra.run.smithi142.stdout:Deploying crash service with default placement... 2024-03-29T04:35:47.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:47 smithi142 ceph-mon[29027]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:47.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:47 smithi142 ceph-mon[29027]: Saving service mgr spec with placement count:2 2024-03-29T04:35:47.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:47 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:47.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:47 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:48.583 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-29T04:35:48.583 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.821+0000 7fd792b15700 1 Processor -- start 2024-03-29T04:35:48.583 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.821+0000 7fd792b15700 1 -- start start 2024-03-29T04:35:48.583 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.821+0000 7fd792b15700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 0x7fd78c105230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:48.583 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.821+0000 7fd792b15700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c074640 con 0x7fd78c104e10 2024-03-29T04:35:48.583 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.821+0000 7fd7908b1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 0x7fd78c105230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:48.583 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.822+0000 7fd7908b1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 0x7fd78c105230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43010/0 (socket says 172.21.15.142:43010) 2024-03-29T04:35:48.583 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.822+0000 7fd7908b1700 1 -- 172.21.15.142:0/2794239557 learned_addr learned my addr 172.21.15.142:0/2794239557 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:48.583 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.822+0000 7fd7908b1700 1 -- 172.21.15.142:0/2794239557 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c105770 con 0x7fd78c104e10 2024-03-29T04:35:48.584 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.822+0000 7fd7908b1700 1 --2- 172.21.15.142:0/2794239557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 0x7fd78c105230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7800098a0 tx=0x7fd780009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ad996c03b35561 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:48.584 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.822+0000 7fd78b7fe700 1 -- 172.21.15.142:0/2794239557 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd780004180 con 0x7fd78c104e10 2024-03-29T04:35:48.584 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.822+0000 7fd78b7fe700 1 -- 172.21.15.142:0/2794239557 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd7800042e0 con 0x7fd78c104e10 2024-03-29T04:35:48.584 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.822+0000 7fd78b7fe700 1 -- 172.21.15.142:0/2794239557 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd780011af0 con 0x7fd78c104e10 2024-03-29T04:35:48.584 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.823+0000 7fd792b15700 1 -- 172.21.15.142:0/2794239557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 msgr2=0x7fd78c105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:48.584 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.823+0000 7fd792b15700 1 --2- 172.21.15.142:0/2794239557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 0x7fd78c105230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7800098a0 tx=0x7fd780009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:48.584 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.823+0000 7fd792b15700 1 -- 172.21.15.142:0/2794239557 shutdown_connections 2024-03-29T04:35:48.584 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.823+0000 7fd792b15700 1 --2- 172.21.15.142:0/2794239557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 0x7fd78c105230 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:48.584 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.823+0000 7fd792b15700 1 -- 172.21.15.142:0/2794239557 >> 172.21.15.142:0/2794239557 conn(0x7fd78c1009f0 msgr2=0x7fd78c102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:48.584 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.823+0000 7fd792b15700 1 -- 172.21.15.142:0/2794239557 shutdown_connections 2024-03-29T04:35:48.584 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.823+0000 7fd792b15700 1 -- 172.21.15.142:0/2794239557 wait complete. 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.824+0000 7fd792b15700 1 Processor -- start 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.824+0000 7fd792b15700 1 -- start start 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.824+0000 7fd792b15700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 0x7fd78c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.824+0000 7fd792b15700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c19c8f0 con 0x7fd78c104e10 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.824+0000 7fd7908b1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 0x7fd78c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.824+0000 7fd7908b1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 0x7fd78c19c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43024/0 (socket says 172.21.15.142:43024) 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.824+0000 7fd7908b1700 1 -- 172.21.15.142:0/1052859591 learned_addr learned my addr 172.21.15.142:0/1052859591 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.824+0000 7fd7908b1700 1 -- 172.21.15.142:0/1052859591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780009550 con 0x7fd78c104e10 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.825+0000 7fd7908b1700 1 --2- 172.21.15.142:0/1052859591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 0x7fd78c19c3b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd780011ad0 tx=0x7fd780011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.825+0000 7fd789ffb700 1 -- 172.21.15.142:0/1052859591 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd780003ab0 con 0x7fd78c104e10 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.825+0000 7fd789ffb700 1 -- 172.21.15.142:0/1052859591 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd780003c10 con 0x7fd78c104e10 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.825+0000 7fd792b15700 1 -- 172.21.15.142:0/1052859591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78c19caf0 con 0x7fd78c104e10 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.825+0000 7fd789ffb700 1 -- 172.21.15.142:0/1052859591 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7800198e0 con 0x7fd78c104e10 2024-03-29T04:35:48.585 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.825+0000 7fd792b15700 1 -- 172.21.15.142:0/1052859591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78c19cf90 con 0x7fd78c104e10 2024-03-29T04:35:48.586 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.826+0000 7fd789ffb700 1 -- 172.21.15.142:0/1052859591 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fd780019ba0 con 0x7fd78c104e10 2024-03-29T04:35:48.586 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.826+0000 7fd792b15700 1 -- 172.21.15.142:0/1052859591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78c195c40 con 0x7fd78c104e10 2024-03-29T04:35:48.586 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.826+0000 7fd789ffb700 1 --2- 172.21.15.142:0/1052859591 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fd77c038320 0x7fd77c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:48.586 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.826+0000 7fd789ffb700 1 -- 172.21.15.142:0/1052859591 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd78004c160 con 0x7fd78c104e10 2024-03-29T04:35:48.586 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.826+0000 7fd78bfff700 1 --2- 172.21.15.142:0/1052859591 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fd77c038320 0x7fd77c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:48.586 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.827+0000 7fd78bfff700 1 --2- 172.21.15.142:0/1052859591 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fd77c038320 0x7fd77c03a7e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd778006fd0 tx=0x7fd778006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:48.586 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:47.831+0000 7fd789ffb700 1 -- 172.21.15.142:0/1052859591 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd78002b300 con 0x7fd78c104e10 2024-03-29T04:35:48.586 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.046+0000 7fd792b15700 1 -- 172.21.15.142:0/1052859591 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fd78c0611d0 con 0x7fd77c038320 2024-03-29T04:35:48.586 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.050+0000 7fd789ffb700 1 -- 172.21.15.142:0/1052859591 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fd78c0611d0 con 0x7fd77c038320 2024-03-29T04:35:48.586 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.052+0000 7fd792b15700 1 -- 172.21.15.142:0/1052859591 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fd77c038320 msgr2=0x7fd77c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:48.586 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.052+0000 7fd792b15700 1 --2- 172.21.15.142:0/1052859591 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fd77c038320 0x7fd77c03a7e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd778006fd0 tx=0x7fd778006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:48.586 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.052+0000 7fd792b15700 1 -- 172.21.15.142:0/1052859591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 msgr2=0x7fd78c19c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:48.587 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.052+0000 7fd792b15700 1 --2- 172.21.15.142:0/1052859591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 0x7fd78c19c3b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd780011ad0 tx=0x7fd780011bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:48.587 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.052+0000 7fd792b15700 1 -- 172.21.15.142:0/1052859591 shutdown_connections 2024-03-29T04:35:48.587 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.052+0000 7fd792b15700 1 --2- 172.21.15.142:0/1052859591 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fd77c038320 0x7fd77c03a7e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:48.587 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.052+0000 7fd792b15700 1 --2- 172.21.15.142:0/1052859591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd78c104e10 0x7fd78c19c3b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:48.587 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.052+0000 7fd792b15700 1 -- 172.21.15.142:0/1052859591 >> 172.21.15.142:0/1052859591 conn(0x7fd78c1009f0 msgr2=0x7fd78c1909d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:48.587 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.052+0000 7fd792b15700 1 -- 172.21.15.142:0/1052859591 shutdown_connections 2024-03-29T04:35:48.587 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.052+0000 7fd792b15700 1 -- 172.21.15.142:0/1052859591 wait complete. 2024-03-29T04:35:48.587 INFO:teuthology.orchestra.run.smithi142.stdout:Deploying ceph-exporter service with default placement... 2024-03-29T04:35:48.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:48 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:48.969 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:48 smithi142 ceph-mon[29027]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:48.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:48 smithi142 ceph-mon[29027]: Saving service crash spec with placement * 2024-03-29T04:35:48.970 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:48 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:50.335 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-03-29T04:35:50.336 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.990+0000 7fe799f9f700 1 Processor -- start 2024-03-29T04:35:50.336 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.990+0000 7fe799f9f700 1 -- start start 2024-03-29T04:35:50.336 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.990+0000 7fe799f9f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 0x7fe79407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:50.336 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.990+0000 7fe799f9f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79407a8b0 con 0x7fe79407be80 2024-03-29T04:35:50.336 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.991+0000 7fe798f9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 0x7fe79407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:50.336 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.991+0000 7fe798f9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 0x7fe79407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46522/0 (socket says 172.21.15.142:46522) 2024-03-29T04:35:50.336 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.991+0000 7fe798f9d700 1 -- 172.21.15.142:0/4062306422 learned_addr learned my addr 172.21.15.142:0/4062306422 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:50.336 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.991+0000 7fe798f9d700 1 -- 172.21.15.142:0/4062306422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79407a9f0 con 0x7fe79407be80 2024-03-29T04:35:50.336 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.991+0000 7fe798f9d700 1 --2- 172.21.15.142:0/4062306422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 0x7fe79407a2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe7880098a0 tx=0x7fe788009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86bb66d0d7fe094 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:50.336 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.991+0000 7fe7937fe700 1 -- 172.21.15.142:0/4062306422 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe788004180 con 0x7fe79407be80 2024-03-29T04:35:50.336 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.991+0000 7fe7937fe700 1 -- 172.21.15.142:0/4062306422 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe7880042e0 con 0x7fe79407be80 2024-03-29T04:35:50.337 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.991+0000 7fe7937fe700 1 -- 172.21.15.142:0/4062306422 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe788011af0 con 0x7fe79407be80 2024-03-29T04:35:50.337 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.992+0000 7fe799f9f700 1 -- 172.21.15.142:0/4062306422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 msgr2=0x7fe79407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:50.337 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.992+0000 7fe799f9f700 1 --2- 172.21.15.142:0/4062306422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 0x7fe79407a2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe7880098a0 tx=0x7fe788009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:50.337 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.992+0000 7fe799f9f700 1 -- 172.21.15.142:0/4062306422 shutdown_connections 2024-03-29T04:35:50.337 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.992+0000 7fe799f9f700 1 --2- 172.21.15.142:0/4062306422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 0x7fe79407a2e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:50.337 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.992+0000 7fe799f9f700 1 -- 172.21.15.142:0/4062306422 >> 172.21.15.142:0/4062306422 conn(0x7fe794101920 msgr2=0x7fe794103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:50.337 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.992+0000 7fe799f9f700 1 -- 172.21.15.142:0/4062306422 shutdown_connections 2024-03-29T04:35:50.337 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.992+0000 7fe799f9f700 1 -- 172.21.15.142:0/4062306422 wait complete. 2024-03-29T04:35:50.337 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.993+0000 7fe799f9f700 1 Processor -- start 2024-03-29T04:35:50.337 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.993+0000 7fe799f9f700 1 -- start start 2024-03-29T04:35:50.337 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.993+0000 7fe799f9f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 0x7fe79419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:50.337 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.993+0000 7fe799f9f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79419c800 con 0x7fe79407be80 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.993+0000 7fe798f9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 0x7fe79419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.993+0000 7fe798f9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 0x7fe79419c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46534/0 (socket says 172.21.15.142:46534) 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.993+0000 7fe798f9d700 1 -- 172.21.15.142:0/1777775722 learned_addr learned my addr 172.21.15.142:0/1777775722 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.993+0000 7fe798f9d700 1 -- 172.21.15.142:0/1777775722 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788009550 con 0x7fe79407be80 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.993+0000 7fe798f9d700 1 --2- 172.21.15.142:0/1777775722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 0x7fe79419c2c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe788011670 tx=0x7fe788011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.994+0000 7fe791ffb700 1 -- 172.21.15.142:0/1777775722 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7880039c0 con 0x7fe79407be80 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.994+0000 7fe791ffb700 1 -- 172.21.15.142:0/1777775722 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe788003b20 con 0x7fe79407be80 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.994+0000 7fe791ffb700 1 -- 172.21.15.142:0/1777775722 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7880247f0 con 0x7fe79407be80 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.994+0000 7fe799f9f700 1 -- 172.21.15.142:0/1777775722 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79419ca00 con 0x7fe79407be80 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.994+0000 7fe799f9f700 1 -- 172.21.15.142:0/1777775722 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79419cea0 con 0x7fe79407be80 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.995+0000 7fe791ffb700 1 -- 172.21.15.142:0/1777775722 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fe788024a60 con 0x7fe79407be80 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.995+0000 7fe791ffb700 1 --2- 172.21.15.142:0/1777775722 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fe784038330 0x7fe78403a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:50.338 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.995+0000 7fe791ffb700 1 -- 172.21.15.142:0/1777775722 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe78804c250 con 0x7fe79407be80 2024-03-29T04:35:50.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.995+0000 7fe793fff700 1 --2- 172.21.15.142:0/1777775722 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fe784038330 0x7fe78403a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:50.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.995+0000 7fe799f9f700 1 -- 172.21.15.142:0/1777775722 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7940623c0 con 0x7fe79407be80 2024-03-29T04:35:50.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:48.995+0000 7fe793fff700 1 --2- 172.21.15.142:0/1777775722 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fe784038330 0x7fe78403a7f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe780006fd0 tx=0x7fe780006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:50.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.000+0000 7fe791ffb700 1 -- 172.21.15.142:0/1777775722 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe788024d10 con 0x7fe79407be80 2024-03-29T04:35:50.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.210+0000 7fe799f9f700 1 -- 172.21.15.142:0/1777775722 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fe79419f920 con 0x7fe784038330 2024-03-29T04:35:50.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.215+0000 7fe791ffb700 1 -- 172.21.15.142:0/1777775722 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fe79419f920 con 0x7fe784038330 2024-03-29T04:35:50.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.217+0000 7fe799f9f700 1 -- 172.21.15.142:0/1777775722 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fe784038330 msgr2=0x7fe78403a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:50.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.217+0000 7fe799f9f700 1 --2- 172.21.15.142:0/1777775722 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fe784038330 0x7fe78403a7f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe780006fd0 tx=0x7fe780006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:50.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.217+0000 7fe799f9f700 1 -- 172.21.15.142:0/1777775722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 msgr2=0x7fe79419c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:50.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.218+0000 7fe799f9f700 1 --2- 172.21.15.142:0/1777775722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 0x7fe79419c2c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe788011670 tx=0x7fe788011aa0 comp rx=0 tx=0).stop 2024-03-29T04:35:50.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.218+0000 7fe799f9f700 1 -- 172.21.15.142:0/1777775722 shutdown_connections 2024-03-29T04:35:50.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.218+0000 7fe799f9f700 1 --2- 172.21.15.142:0/1777775722 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fe784038330 0x7fe78403a7f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:50.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.218+0000 7fe799f9f700 1 --2- 172.21.15.142:0/1777775722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79407be80 0x7fe79419c2c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:50.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.218+0000 7fe799f9f700 1 -- 172.21.15.142:0/1777775722 >> 172.21.15.142:0/1777775722 conn(0x7fe794101920 msgr2=0x7fe794103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:50.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.218+0000 7fe799f9f700 1 -- 172.21.15.142:0/1777775722 shutdown_connections 2024-03-29T04:35:50.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:49.218+0000 7fe799f9f700 1 -- 172.21.15.142:0/1777775722 wait complete. 2024-03-29T04:35:50.340 INFO:teuthology.orchestra.run.smithi142.stdout:Deploying prometheus service with default placement... 2024-03-29T04:35:50.458 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:50 smithi142 ceph-mon[29027]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:50.458 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:50 smithi142 ceph-mon[29027]: Saving service ceph-exporter spec with placement * 2024-03-29T04:35:50.458 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:50 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:52.027 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.298+0000 7fe905380700 1 Processor -- start 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.298+0000 7fe905380700 1 -- start start 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.298+0000 7fe905380700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 0x7fe900104c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.298+0000 7fe905380700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe900105200 con 0x7fe900104810 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.298+0000 7fe8feffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 0x7fe900104c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.298+0000 7fe8feffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 0x7fe900104c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46548/0 (socket says 172.21.15.142:46548) 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.298+0000 7fe8feffd700 1 -- 172.21.15.142:0/3196771777 learned_addr learned my addr 172.21.15.142:0/3196771777 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.299+0000 7fe8feffd700 1 -- 172.21.15.142:0/3196771777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe900105340 con 0x7fe900104810 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.299+0000 7fe8feffd700 1 --2- 172.21.15.142:0/3196771777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 0x7fe900104c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00098a0 tx=0x7fe8f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d33d7b99860f9c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.299+0000 7fe8fe7fc700 1 -- 172.21.15.142:0/3196771777 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8f0004180 con 0x7fe900104810 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.299+0000 7fe8fe7fc700 1 -- 172.21.15.142:0/3196771777 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe8f00042e0 con 0x7fe900104810 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.299+0000 7fe8fe7fc700 1 -- 172.21.15.142:0/3196771777 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8f0011af0 con 0x7fe900104810 2024-03-29T04:35:52.028 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.300+0000 7fe905380700 1 -- 172.21.15.142:0/3196771777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 msgr2=0x7fe900104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.300+0000 7fe905380700 1 --2- 172.21.15.142:0/3196771777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 0x7fe900104c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00098a0 tx=0x7fe8f0009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.300+0000 7fe905380700 1 -- 172.21.15.142:0/3196771777 shutdown_connections 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.300+0000 7fe905380700 1 --2- 172.21.15.142:0/3196771777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 0x7fe900104c30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.300+0000 7fe905380700 1 -- 172.21.15.142:0/3196771777 >> 172.21.15.142:0/3196771777 conn(0x7fe900100190 msgr2=0x7fe9001025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.300+0000 7fe905380700 1 -- 172.21.15.142:0/3196771777 shutdown_connections 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.300+0000 7fe905380700 1 -- 172.21.15.142:0/3196771777 wait complete. 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.300+0000 7fe905380700 1 Processor -- start 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.301+0000 7fe905380700 1 -- start start 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.301+0000 7fe905380700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 0x7fe9001071f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.301+0000 7fe905380700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe900107730 con 0x7fe900104810 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.301+0000 7fe8feffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 0x7fe9001071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.301+0000 7fe8feffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 0x7fe9001071f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46564/0 (socket says 172.21.15.142:46564) 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.301+0000 7fe8feffd700 1 -- 172.21.15.142:0/547257562 learned_addr learned my addr 172.21.15.142:0/547257562 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:52.029 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.301+0000 7fe8feffd700 1 -- 172.21.15.142:0/547257562 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f0009550 con 0x7fe900104810 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.301+0000 7fe8feffd700 1 --2- 172.21.15.142:0/547257562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 0x7fe9001071f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0011670 tx=0x7fe8f0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.301+0000 7fe8fcff9700 1 -- 172.21.15.142:0/547257562 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8f00039c0 con 0x7fe900104810 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.301+0000 7fe8fcff9700 1 -- 172.21.15.142:0/547257562 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe8f0003b20 con 0x7fe900104810 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.301+0000 7fe8fcff9700 1 -- 172.21.15.142:0/547257562 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8f00247f0 con 0x7fe900104810 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.301+0000 7fe905380700 1 -- 172.21.15.142:0/547257562 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe900105840 con 0x7fe900104810 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.302+0000 7fe905380700 1 -- 172.21.15.142:0/547257562 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe900105ce0 con 0x7fe900104810 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.302+0000 7fe8fcff9700 1 -- 172.21.15.142:0/547257562 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fe8f0024a60 con 0x7fe900104810 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.302+0000 7fe905380700 1 -- 172.21.15.142:0/547257562 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9000623c0 con 0x7fe900104810 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.303+0000 7fe8fcff9700 1 --2- 172.21.15.142:0/547257562 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fe8ec038330 0x7fe8ec03a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.303+0000 7fe8fcff9700 1 -- 172.21.15.142:0/547257562 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe8f004c3d0 con 0x7fe900104810 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.303+0000 7fe8f67fc700 1 --2- 172.21.15.142:0/547257562 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fe8ec038330 0x7fe8ec03a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.303+0000 7fe8f67fc700 1 --2- 172.21.15.142:0/547257562 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fe8ec038330 0x7fe8ec03a7f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8006fd0 tx=0x7fe8e8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:52.030 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.307+0000 7fe8fcff9700 1 -- 172.21.15.142:0/547257562 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe8f0018a50 con 0x7fe900104810 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.507+0000 7fe905380700 1 -- 172.21.15.142:0/547257562 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fe900101650 con 0x7fe8ec038330 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.511+0000 7fe8fcff9700 1 -- 172.21.15.142:0/547257562 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fe900101650 con 0x7fe8ec038330 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.513+0000 7fe905380700 1 -- 172.21.15.142:0/547257562 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fe8ec038330 msgr2=0x7fe8ec03a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.513+0000 7fe905380700 1 --2- 172.21.15.142:0/547257562 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fe8ec038330 0x7fe8ec03a7f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8006fd0 tx=0x7fe8e8006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.513+0000 7fe905380700 1 -- 172.21.15.142:0/547257562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 msgr2=0x7fe9001071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.513+0000 7fe905380700 1 --2- 172.21.15.142:0/547257562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 0x7fe9001071f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0011670 tx=0x7fe8f0011aa0 comp rx=0 tx=0).stop 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.513+0000 7fe905380700 1 -- 172.21.15.142:0/547257562 shutdown_connections 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.513+0000 7fe905380700 1 --2- 172.21.15.142:0/547257562 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fe8ec038330 0x7fe8ec03a7f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.513+0000 7fe905380700 1 --2- 172.21.15.142:0/547257562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe900104810 0x7fe9001071f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.513+0000 7fe905380700 1 -- 172.21.15.142:0/547257562 >> 172.21.15.142:0/547257562 conn(0x7fe900100190 msgr2=0x7fe900100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.513+0000 7fe905380700 1 -- 172.21.15.142:0/547257562 shutdown_connections 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:51.514+0000 7fe905380700 1 -- 172.21.15.142:0/547257562 wait complete. 2024-03-29T04:35:52.031 INFO:teuthology.orchestra.run.smithi142.stdout:Deploying grafana service with default placement... 2024-03-29T04:35:52.857 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:52 smithi142 ceph-mon[29027]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:52.858 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:52 smithi142 ceph-mon[29027]: Saving service prometheus spec with placement count:1 2024-03-29T04:35:52.858 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:52 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:53.233 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-29T04:35:53.233 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.468+0000 7fc75ff7b700 1 Processor -- start 2024-03-29T04:35:53.233 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.469+0000 7fc75ff7b700 1 -- start start 2024-03-29T04:35:53.233 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.469+0000 7fc75ff7b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 0x7fc7581069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:53.233 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.469+0000 7fc75ff7b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc758106f70 con 0x7fc758106580 2024-03-29T04:35:53.233 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.469+0000 7fc75dd17700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 0x7fc7581069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:53.233 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.469+0000 7fc75dd17700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 0x7fc7581069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46566/0 (socket says 172.21.15.142:46566) 2024-03-29T04:35:53.233 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.469+0000 7fc75dd17700 1 -- 172.21.15.142:0/438593694 learned_addr learned my addr 172.21.15.142:0/438593694 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:53.233 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.469+0000 7fc75dd17700 1 -- 172.21.15.142:0/438593694 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc758107780 con 0x7fc758106580 2024-03-29T04:35:53.233 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.469+0000 7fc75dd17700 1 --2- 172.21.15.142:0/438593694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 0x7fc7581069a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7480098a0 tx=0x7fc748009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4cfd2387cdbadd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.470+0000 7fc75cd15700 1 -- 172.21.15.142:0/438593694 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc748004180 con 0x7fc758106580 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.470+0000 7fc75cd15700 1 -- 172.21.15.142:0/438593694 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc7480042e0 con 0x7fc758106580 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.470+0000 7fc75cd15700 1 -- 172.21.15.142:0/438593694 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc748011af0 con 0x7fc758106580 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.470+0000 7fc75ff7b700 1 -- 172.21.15.142:0/438593694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 msgr2=0x7fc7581069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.470+0000 7fc75ff7b700 1 --2- 172.21.15.142:0/438593694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 0x7fc7581069a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7480098a0 tx=0x7fc748009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.470+0000 7fc75ff7b700 1 -- 172.21.15.142:0/438593694 shutdown_connections 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.470+0000 7fc75ff7b700 1 --2- 172.21.15.142:0/438593694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 0x7fc7581069a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.470+0000 7fc75ff7b700 1 -- 172.21.15.142:0/438593694 >> 172.21.15.142:0/438593694 conn(0x7fc758101b00 msgr2=0x7fc758103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.470+0000 7fc75ff7b700 1 -- 172.21.15.142:0/438593694 shutdown_connections 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.470+0000 7fc75ff7b700 1 -- 172.21.15.142:0/438593694 wait complete. 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.471+0000 7fc75ff7b700 1 Processor -- start 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.471+0000 7fc75ff7b700 1 -- start start 2024-03-29T04:35:53.234 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.471+0000 7fc75ff7b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 0x7fc75819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:53.235 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.471+0000 7fc75ff7b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75819a7d0 con 0x7fc758106580 2024-03-29T04:35:53.235 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.471+0000 7fc75dd17700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 0x7fc75819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:53.235 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.471+0000 7fc75dd17700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 0x7fc75819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46570/0 (socket says 172.21.15.142:46570) 2024-03-29T04:35:53.235 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.471+0000 7fc75dd17700 1 -- 172.21.15.142:0/4056822748 learned_addr learned my addr 172.21.15.142:0/4056822748 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:53.235 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.471+0000 7fc75dd17700 1 -- 172.21.15.142:0/4056822748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc748009550 con 0x7fc758106580 2024-03-29T04:35:53.235 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.471+0000 7fc75dd17700 1 --2- 172.21.15.142:0/4056822748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 0x7fc75819a290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc748011670 tx=0x7fc748011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:53.235 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.472+0000 7fc74effd700 1 -- 172.21.15.142:0/4056822748 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7480039c0 con 0x7fc758106580 2024-03-29T04:35:53.235 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.472+0000 7fc74effd700 1 -- 172.21.15.142:0/4056822748 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc748003b20 con 0x7fc758106580 2024-03-29T04:35:53.235 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.472+0000 7fc75ff7b700 1 -- 172.21.15.142:0/4056822748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75819a9d0 con 0x7fc758106580 2024-03-29T04:35:53.235 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.472+0000 7fc74effd700 1 -- 172.21.15.142:0/4056822748 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7480247f0 con 0x7fc758106580 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.472+0000 7fc75ff7b700 1 -- 172.21.15.142:0/4056822748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75819ae70 con 0x7fc758106580 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.472+0000 7fc74effd700 1 -- 172.21.15.142:0/4056822748 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fc748024a60 con 0x7fc758106580 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.473+0000 7fc74effd700 1 --2- 172.21.15.142:0/4056822748 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fc744038330 0x7fc74403a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.473+0000 7fc75ff7b700 1 -- 172.21.15.142:0/4056822748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7580623c0 con 0x7fc758106580 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.473+0000 7fc74effd700 1 -- 172.21.15.142:0/4056822748 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc74804c270 con 0x7fc758106580 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.473+0000 7fc75d516700 1 --2- 172.21.15.142:0/4056822748 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fc744038330 0x7fc74403a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.473+0000 7fc75d516700 1 --2- 172.21.15.142:0/4056822748 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fc744038330 0x7fc74403a7f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc754006fd0 tx=0x7fc754006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.477+0000 7fc74effd700 1 -- 172.21.15.142:0/4056822748 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc748024d10 con 0x7fc758106580 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.677+0000 7fc75ff7b700 1 -- 172.21.15.142:0/4056822748 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fc75810a980 con 0x7fc744038330 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.682+0000 7fc74effd700 1 -- 172.21.15.142:0/4056822748 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fc75810a980 con 0x7fc744038330 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.684+0000 7fc75ff7b700 1 -- 172.21.15.142:0/4056822748 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fc744038330 msgr2=0x7fc74403a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.684+0000 7fc75ff7b700 1 --2- 172.21.15.142:0/4056822748 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fc744038330 0x7fc74403a7f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc754006fd0 tx=0x7fc754006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:53.236 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.684+0000 7fc75ff7b700 1 -- 172.21.15.142:0/4056822748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 msgr2=0x7fc75819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:53.237 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.684+0000 7fc75ff7b700 1 --2- 172.21.15.142:0/4056822748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 0x7fc75819a290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc748011670 tx=0x7fc748011aa0 comp rx=0 tx=0).stop 2024-03-29T04:35:53.237 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.684+0000 7fc75ff7b700 1 -- 172.21.15.142:0/4056822748 shutdown_connections 2024-03-29T04:35:53.237 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.684+0000 7fc75ff7b700 1 --2- 172.21.15.142:0/4056822748 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fc744038330 0x7fc74403a7f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:53.237 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.684+0000 7fc75ff7b700 1 --2- 172.21.15.142:0/4056822748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc758106580 0x7fc75819a290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:53.237 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.684+0000 7fc75ff7b700 1 -- 172.21.15.142:0/4056822748 >> 172.21.15.142:0/4056822748 conn(0x7fc758101b00 msgr2=0x7fc7581031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:53.237 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.685+0000 7fc75ff7b700 1 -- 172.21.15.142:0/4056822748 shutdown_connections 2024-03-29T04:35:53.237 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:52.685+0000 7fc75ff7b700 1 -- 172.21.15.142:0/4056822748 wait complete. 2024-03-29T04:35:53.237 INFO:teuthology.orchestra.run.smithi142.stdout:Deploying node-exporter service with default placement... 2024-03-29T04:35:53.826 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:53 smithi142 ceph-mon[29027]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:53.826 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:53 smithi142 ceph-mon[29027]: Saving service grafana spec with placement count:1 2024-03-29T04:35:53.826 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:53 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:53.826 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:53 smithi142 ceph-mon[29027]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:35:54.414 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-29T04:35:54.414 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.631+0000 7f60b7a13700 1 Processor -- start 2024-03-29T04:35:54.414 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.631+0000 7f60b7a13700 1 -- start start 2024-03-29T04:35:54.414 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.631+0000 7f60b7a13700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 0x7f60b007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:54.414 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.631+0000 7f60b7a13700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b007a8b0 con 0x7f60b007be80 2024-03-29T04:35:54.414 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.631+0000 7f60b57af700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 0x7f60b007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:54.415 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.631+0000 7f60b57af700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 0x7f60b007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46576/0 (socket says 172.21.15.142:46576) 2024-03-29T04:35:54.415 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.631+0000 7f60b57af700 1 -- 172.21.15.142:0/2066144345 learned_addr learned my addr 172.21.15.142:0/2066144345 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:54.415 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.631+0000 7f60b57af700 1 -- 172.21.15.142:0/2066144345 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b007a9f0 con 0x7f60b007be80 2024-03-29T04:35:54.415 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.631+0000 7f60b57af700 1 --2- 172.21.15.142:0/2066144345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 0x7f60b007a2e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f60a00098a0 tx=0x7f60a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b578e8f7542cc561 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:54.415 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.632+0000 7f60a7fff700 1 -- 172.21.15.142:0/2066144345 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60a0004180 con 0x7f60b007be80 2024-03-29T04:35:54.415 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.632+0000 7f60a7fff700 1 -- 172.21.15.142:0/2066144345 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f60a00042e0 con 0x7f60b007be80 2024-03-29T04:35:54.415 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.632+0000 7f60a7fff700 1 -- 172.21.15.142:0/2066144345 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60a0011af0 con 0x7f60b007be80 2024-03-29T04:35:54.415 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.632+0000 7f60b7a13700 1 -- 172.21.15.142:0/2066144345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 msgr2=0x7f60b007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:54.415 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.632+0000 7f60b7a13700 1 --2- 172.21.15.142:0/2066144345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 0x7f60b007a2e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f60a00098a0 tx=0x7f60a0009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:54.415 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.632+0000 7f60b7a13700 1 -- 172.21.15.142:0/2066144345 shutdown_connections 2024-03-29T04:35:54.415 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.632+0000 7f60b7a13700 1 --2- 172.21.15.142:0/2066144345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 0x7f60b007a2e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.632+0000 7f60b7a13700 1 -- 172.21.15.142:0/2066144345 >> 172.21.15.142:0/2066144345 conn(0x7f60b0101b40 msgr2=0x7f60b0103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.632+0000 7f60b7a13700 1 -- 172.21.15.142:0/2066144345 shutdown_connections 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.632+0000 7f60b7a13700 1 -- 172.21.15.142:0/2066144345 wait complete. 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.633+0000 7f60b7a13700 1 Processor -- start 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.633+0000 7f60b7a13700 1 -- start start 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.633+0000 7f60b7a13700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 0x7f60b019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.633+0000 7f60b7a13700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b019c9a0 con 0x7f60b007be80 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.633+0000 7f60b57af700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 0x7f60b019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.633+0000 7f60b57af700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 0x7f60b019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46580/0 (socket says 172.21.15.142:46580) 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.633+0000 7f60b57af700 1 -- 172.21.15.142:0/4137548977 learned_addr learned my addr 172.21.15.142:0/4137548977 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.633+0000 7f60b57af700 1 -- 172.21.15.142:0/4137548977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a0009550 con 0x7f60b007be80 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.633+0000 7f60b57af700 1 --2- 172.21.15.142:0/4137548977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 0x7f60b019c460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f60a0011670 tx=0x7f60a0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:54.416 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.634+0000 7f60a67fc700 1 -- 172.21.15.142:0/4137548977 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60a00039c0 con 0x7f60b007be80 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.634+0000 7f60a67fc700 1 -- 172.21.15.142:0/4137548977 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f60a0003b20 con 0x7f60b007be80 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.634+0000 7f60b7a13700 1 -- 172.21.15.142:0/4137548977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b019cba0 con 0x7f60b007be80 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.634+0000 7f60a67fc700 1 -- 172.21.15.142:0/4137548977 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60a00247f0 con 0x7f60b007be80 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.634+0000 7f60b7a13700 1 -- 172.21.15.142:0/4137548977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b019d040 con 0x7f60b007be80 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.634+0000 7f60a67fc700 1 -- 172.21.15.142:0/4137548977 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f60a0024a60 con 0x7f60b007be80 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.635+0000 7f60a67fc700 1 --2- 172.21.15.142:0/4137548977 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f609c038330 0x7f609c03a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.635+0000 7f60b7a13700 1 -- 172.21.15.142:0/4137548977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b010c920 con 0x7f60b007be80 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.635+0000 7f60a67fc700 1 -- 172.21.15.142:0/4137548977 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f60a004c3c0 con 0x7f60b007be80 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.635+0000 7f60b4fae700 1 --2- 172.21.15.142:0/4137548977 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f609c038330 0x7f609c03a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.635+0000 7f60b4fae700 1 --2- 172.21.15.142:0/4137548977 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f609c038330 0x7f609c03a7f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f60ac006fd0 tx=0x7f60ac006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.640+0000 7f60a67fc700 1 -- 172.21.15.142:0/4137548977 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f60a0024d10 con 0x7f60b007be80 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.839+0000 7f60b7a13700 1 -- 172.21.15.142:0/4137548977 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f60b0103e00 con 0x7f609c038330 2024-03-29T04:35:54.417 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.845+0000 7f60a67fc700 1 -- 172.21.15.142:0/4137548977 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f60b0103e00 con 0x7f609c038330 2024-03-29T04:35:54.418 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.847+0000 7f60b7a13700 1 -- 172.21.15.142:0/4137548977 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f609c038330 msgr2=0x7f609c03a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:54.418 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.847+0000 7f60b7a13700 1 --2- 172.21.15.142:0/4137548977 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f609c038330 0x7f609c03a7f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f60ac006fd0 tx=0x7f60ac006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:54.418 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.847+0000 7f60b7a13700 1 -- 172.21.15.142:0/4137548977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 msgr2=0x7f60b019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:54.418 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.847+0000 7f60b7a13700 1 --2- 172.21.15.142:0/4137548977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 0x7f60b019c460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f60a0011670 tx=0x7f60a0011aa0 comp rx=0 tx=0).stop 2024-03-29T04:35:54.418 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.847+0000 7f60b7a13700 1 -- 172.21.15.142:0/4137548977 shutdown_connections 2024-03-29T04:35:54.418 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.847+0000 7f60b7a13700 1 --2- 172.21.15.142:0/4137548977 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f609c038330 0x7f609c03a7f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:54.418 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.847+0000 7f60b7a13700 1 --2- 172.21.15.142:0/4137548977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b007be80 0x7f60b019c460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:54.418 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.847+0000 7f60b7a13700 1 -- 172.21.15.142:0/4137548977 >> 172.21.15.142:0/4137548977 conn(0x7f60b0101b40 msgr2=0x7f60b01031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:54.418 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.847+0000 7f60b7a13700 1 -- 172.21.15.142:0/4137548977 shutdown_connections 2024-03-29T04:35:54.418 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:53.847+0000 7f60b7a13700 1 -- 172.21.15.142:0/4137548977 wait complete. 2024-03-29T04:35:54.418 INFO:teuthology.orchestra.run.smithi142.stdout:Deploying alertmanager service with default placement... 2024-03-29T04:35:55.052 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:54 smithi142 ceph-mon[29027]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:55.053 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:54 smithi142 ceph-mon[29027]: Saving service node-exporter spec with placement * 2024-03-29T04:35:55.053 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:54 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:55.645 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.855+0000 7f052fe6c700 1 Processor -- start 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.855+0000 7f052fe6c700 1 -- start start 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.855+0000 7f052fe6c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 0x7f05281093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.855+0000 7f052fe6c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0528079e90 con 0x7f0528106fd0 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.855+0000 7f052dc08700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 0x7f05281093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.855+0000 7f052dc08700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 0x7f05281093c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46592/0 (socket says 172.21.15.142:46592) 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.855+0000 7f052dc08700 1 -- 172.21.15.142:0/510735104 learned_addr learned my addr 172.21.15.142:0/510735104 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.856+0000 7f052dc08700 1 -- 172.21.15.142:0/510735104 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0528109900 con 0x7f0528106fd0 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.856+0000 7f052dc08700 1 --2- 172.21.15.142:0/510735104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 0x7f05281093c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f05240098a0 tx=0x7f0524009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb98ad75fb12728 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.856+0000 7f052cc06700 1 -- 172.21.15.142:0/510735104 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0524004180 con 0x7f0528106fd0 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.856+0000 7f052cc06700 1 -- 172.21.15.142:0/510735104 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f05240042e0 con 0x7f0528106fd0 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.856+0000 7f052cc06700 1 -- 172.21.15.142:0/510735104 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0524011af0 con 0x7f0528106fd0 2024-03-29T04:35:55.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.856+0000 7f052fe6c700 1 -- 172.21.15.142:0/510735104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 msgr2=0x7f05281093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.856+0000 7f052fe6c700 1 --2- 172.21.15.142:0/510735104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 0x7f05281093c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f05240098a0 tx=0x7f0524009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.857+0000 7f052fe6c700 1 -- 172.21.15.142:0/510735104 shutdown_connections 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.857+0000 7f052fe6c700 1 --2- 172.21.15.142:0/510735104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 0x7f05281093c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.857+0000 7f052fe6c700 1 -- 172.21.15.142:0/510735104 >> 172.21.15.142:0/510735104 conn(0x7f0528100a20 msgr2=0x7f0528102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.857+0000 7f052fe6c700 1 -- 172.21.15.142:0/510735104 shutdown_connections 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.857+0000 7f052fe6c700 1 -- 172.21.15.142:0/510735104 wait complete. 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.857+0000 7f052fe6c700 1 Processor -- start 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.857+0000 7f052fe6c700 1 -- start start 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.857+0000 7f052fe6c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 0x7f0528106930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.857+0000 7f052fe6c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0528106e70 con 0x7f0528106fd0 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.857+0000 7f052dc08700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 0x7f0528106930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.857+0000 7f052dc08700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 0x7f0528106930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46608/0 (socket says 172.21.15.142:46608) 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.857+0000 7f052dc08700 1 -- 172.21.15.142:0/554438582 learned_addr learned my addr 172.21.15.142:0/554438582 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:55.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.858+0000 7f052dc08700 1 -- 172.21.15.142:0/554438582 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524009550 con 0x7f0528106fd0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.858+0000 7f052dc08700 1 --2- 172.21.15.142:0/554438582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 0x7f0528106930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0524011670 tx=0x7f0524011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.858+0000 7f051effd700 1 -- 172.21.15.142:0/554438582 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0524003a40 con 0x7f0528106fd0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.858+0000 7f051effd700 1 -- 172.21.15.142:0/554438582 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0524003ba0 con 0x7f0528106fd0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.858+0000 7f052fe6c700 1 -- 172.21.15.142:0/554438582 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0528104f80 con 0x7f0528106fd0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.858+0000 7f052fe6c700 1 -- 172.21.15.142:0/554438582 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0528105420 con 0x7f0528106fd0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.858+0000 7f051effd700 1 -- 172.21.15.142:0/554438582 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0524024820 con 0x7f0528106fd0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.859+0000 7f051effd700 1 -- 172.21.15.142:0/554438582 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f0524024a90 con 0x7f0528106fd0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.859+0000 7f052fe6c700 1 -- 172.21.15.142:0/554438582 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050c005320 con 0x7f0528106fd0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.859+0000 7f051effd700 1 --2- 172.21.15.142:0/554438582 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0514038330 0x7f051403a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.859+0000 7f051effd700 1 -- 172.21.15.142:0/554438582 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f052404c290 con 0x7f0528106fd0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.859+0000 7f052d407700 1 --2- 172.21.15.142:0/554438582 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0514038330 0x7f051403a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.860+0000 7f052d407700 1 --2- 172.21.15.142:0/554438582 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0514038330 0x7f051403a7f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0518006fd0 tx=0x7f0518006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:54.864+0000 7f051effd700 1 -- 172.21.15.142:0/554438582 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0524022310 con 0x7f0528106fd0 2024-03-29T04:35:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:55.065+0000 7f052fe6c700 1 -- 172.21.15.142:0/554438582 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f050c000bf0 con 0x7f0514038330 2024-03-29T04:35:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:55.071+0000 7f051effd700 1 -- 172.21.15.142:0/554438582 <== mgr.14118 v2:172.21.15.142:6800/2742619935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f050c000bf0 con 0x7f0514038330 2024-03-29T04:35:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:55.073+0000 7f052fe6c700 1 -- 172.21.15.142:0/554438582 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0514038330 msgr2=0x7f051403a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:55.073+0000 7f052fe6c700 1 --2- 172.21.15.142:0/554438582 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0514038330 0x7f051403a7f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0518006fd0 tx=0x7f0518006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:55.073+0000 7f052fe6c700 1 -- 172.21.15.142:0/554438582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 msgr2=0x7f0528106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:55.073+0000 7f052fe6c700 1 --2- 172.21.15.142:0/554438582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 0x7f0528106930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0524011670 tx=0x7f0524011aa0 comp rx=0 tx=0).stop 2024-03-29T04:35:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:55.073+0000 7f052fe6c700 1 -- 172.21.15.142:0/554438582 shutdown_connections 2024-03-29T04:35:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:55.073+0000 7f052fe6c700 1 --2- 172.21.15.142:0/554438582 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0514038330 0x7f051403a7f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:55.073+0000 7f052fe6c700 1 --2- 172.21.15.142:0/554438582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0528106fd0 0x7f0528106930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:55.073+0000 7f052fe6c700 1 -- 172.21.15.142:0/554438582 >> 172.21.15.142:0/554438582 conn(0x7f0528100a20 msgr2=0x7f05281017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:55.073+0000 7f052fe6c700 1 -- 172.21.15.142:0/554438582 shutdown_connections 2024-03-29T04:35:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:55.073+0000 7f052fe6c700 1 -- 172.21.15.142:0/554438582 wait complete. 2024-03-29T04:35:56.257 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:56 smithi142 ceph-mon[29027]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:35:56.258 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:56 smithi142 ceph-mon[29027]: Saving service alertmanager spec with placement count:1 2024-03-29T04:35:56.258 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:56 smithi142 ceph-mon[29027]: from='mgr.14118 172.21.15.142:0/4186348002' entity='mgr.smithi142.swesrg' 2024-03-29T04:35:56.258 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:56 smithi142 ceph-mon[29027]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:35:56.843 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.066+0000 7f0f933aa700 1 Processor -- start 2024-03-29T04:35:56.844 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.066+0000 7f0f933aa700 1 -- start start 2024-03-29T04:35:56.844 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.067+0000 7f0f933aa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 0x7f0f8c07a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:56.844 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.067+0000 7f0f933aa700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c07a8f0 con 0x7f0f8c07bec0 2024-03-29T04:35:56.844 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.067+0000 7f0f91146700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 0x7f0f8c07a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:56.844 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.067+0000 7f0f91146700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 0x7f0f8c07a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46624/0 (socket says 172.21.15.142:46624) 2024-03-29T04:35:56.844 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.067+0000 7f0f91146700 1 -- 172.21.15.142:0/1393358478 learned_addr learned my addr 172.21.15.142:0/1393358478 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:56.844 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.067+0000 7f0f91146700 1 -- 172.21.15.142:0/1393358478 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c07aa30 con 0x7f0f8c07bec0 2024-03-29T04:35:56.844 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.067+0000 7f0f91146700 1 --2- 172.21.15.142:0/1393358478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 0x7f0f8c07a320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0098a0 tx=0x7f0f7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502052f3b3cbedac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:56.844 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.068+0000 7f0f83fff700 1 -- 172.21.15.142:0/1393358478 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f7c004180 con 0x7f0f8c07bec0 2024-03-29T04:35:56.844 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.068+0000 7f0f83fff700 1 -- 172.21.15.142:0/1393358478 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0f7c0042e0 con 0x7f0f8c07bec0 2024-03-29T04:35:56.845 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.068+0000 7f0f83fff700 1 -- 172.21.15.142:0/1393358478 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f7c011af0 con 0x7f0f8c07bec0 2024-03-29T04:35:56.845 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.068+0000 7f0f933aa700 1 -- 172.21.15.142:0/1393358478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 msgr2=0x7f0f8c07a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:56.845 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.068+0000 7f0f933aa700 1 --2- 172.21.15.142:0/1393358478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 0x7f0f8c07a320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0098a0 tx=0x7f0f7c009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:56.845 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.068+0000 7f0f933aa700 1 -- 172.21.15.142:0/1393358478 shutdown_connections 2024-03-29T04:35:56.845 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.068+0000 7f0f933aa700 1 --2- 172.21.15.142:0/1393358478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 0x7f0f8c07a320 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:56.845 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.068+0000 7f0f933aa700 1 -- 172.21.15.142:0/1393358478 >> 172.21.15.142:0/1393358478 conn(0x7f0f8c101b40 msgr2=0x7f0f8c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:56.845 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.068+0000 7f0f933aa700 1 -- 172.21.15.142:0/1393358478 shutdown_connections 2024-03-29T04:35:56.845 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.068+0000 7f0f933aa700 1 -- 172.21.15.142:0/1393358478 wait complete. 2024-03-29T04:35:56.845 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.069+0000 7f0f933aa700 1 Processor -- start 2024-03-29T04:35:56.845 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.069+0000 7f0f933aa700 1 -- start start 2024-03-29T04:35:56.846 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.069+0000 7f0f933aa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 0x7f0f8c19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:56.846 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.069+0000 7f0f933aa700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c19c940 con 0x7f0f8c07bec0 2024-03-29T04:35:56.846 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.069+0000 7f0f91146700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 0x7f0f8c19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:56.846 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.069+0000 7f0f91146700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 0x7f0f8c19c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46632/0 (socket says 172.21.15.142:46632) 2024-03-29T04:35:56.846 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.069+0000 7f0f91146700 1 -- 172.21.15.142:0/2557997637 learned_addr learned my addr 172.21.15.142:0/2557997637 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:56.846 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.069+0000 7f0f91146700 1 -- 172.21.15.142:0/2557997637 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c009550 con 0x7f0f8c07bec0 2024-03-29T04:35:56.846 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.070+0000 7f0f91146700 1 --2- 172.21.15.142:0/2557997637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 0x7f0f8c19c400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c000c00 tx=0x7f0f7c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:56.846 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.071+0000 7f0f827fc700 1 -- 172.21.15.142:0/2557997637 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f7c003a70 con 0x7f0f8c07bec0 2024-03-29T04:35:56.846 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.071+0000 7f0f827fc700 1 -- 172.21.15.142:0/2557997637 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0f7c003bd0 con 0x7f0f8c07bec0 2024-03-29T04:35:56.846 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.071+0000 7f0f827fc700 1 -- 172.21.15.142:0/2557997637 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f7c0238c0 con 0x7f0f8c07bec0 2024-03-29T04:35:56.847 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.071+0000 7f0f933aa700 1 -- 172.21.15.142:0/2557997637 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c19cb40 con 0x7f0f8c07bec0 2024-03-29T04:35:56.847 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.071+0000 7f0f933aa700 1 -- 172.21.15.142:0/2557997637 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c19cfe0 con 0x7f0f8c07bec0 2024-03-29T04:35:56.847 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.072+0000 7f0f933aa700 1 -- 172.21.15.142:0/2557997637 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8c10c930 con 0x7f0f8c07bec0 2024-03-29T04:35:56.847 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.072+0000 7f0f827fc700 1 -- 172.21.15.142:0/2557997637 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f0f7c023b30 con 0x7f0f8c07bec0 2024-03-29T04:35:56.847 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.073+0000 7f0f827fc700 1 --2- 172.21.15.142:0/2557997637 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0f78038370 0x7f0f7803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:56.847 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.073+0000 7f0f827fc700 1 -- 172.21.15.142:0/2557997637 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0f7c04c1c0 con 0x7f0f8c07bec0 2024-03-29T04:35:56.847 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.073+0000 7f0f90945700 1 --2- 172.21.15.142:0/2557997637 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0f78038370 0x7f0f7803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:56.847 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.073+0000 7f0f90945700 1 --2- 172.21.15.142:0/2557997637 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0f78038370 0x7f0f7803a830 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f88006fd0 tx=0x7f0f88006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:56.847 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.079+0000 7f0f827fc700 1 -- 172.21.15.142:0/2557997637 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0f7c010c20 con 0x7f0f8c07bec0 2024-03-29T04:35:56.848 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.271+0000 7f0f933aa700 1 -- 172.21.15.142:0/2557997637 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f0f8c10cb50 con 0x7f0f8c07bec0 2024-03-29T04:35:56.848 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.275+0000 7f0f827fc700 1 -- 172.21.15.142:0/2557997637 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f0f7c010830 con 0x7f0f8c07bec0 2024-03-29T04:35:56.848 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.278+0000 7f0f933aa700 1 -- 172.21.15.142:0/2557997637 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0f78038370 msgr2=0x7f0f7803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:56.848 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.278+0000 7f0f933aa700 1 --2- 172.21.15.142:0/2557997637 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0f78038370 0x7f0f7803a830 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f88006fd0 tx=0x7f0f88006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:56.848 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.278+0000 7f0f933aa700 1 -- 172.21.15.142:0/2557997637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 msgr2=0x7f0f8c19c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:56.848 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.278+0000 7f0f933aa700 1 --2- 172.21.15.142:0/2557997637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 0x7f0f8c19c400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c000c00 tx=0x7f0f7c011b50 comp rx=0 tx=0).stop 2024-03-29T04:35:56.848 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.278+0000 7f0f933aa700 1 -- 172.21.15.142:0/2557997637 shutdown_connections 2024-03-29T04:35:56.848 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.278+0000 7f0f933aa700 1 --2- 172.21.15.142:0/2557997637 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f0f78038370 0x7f0f7803a830 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:56.848 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.278+0000 7f0f933aa700 1 --2- 172.21.15.142:0/2557997637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f8c07bec0 0x7f0f8c19c400 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:56.848 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.278+0000 7f0f933aa700 1 -- 172.21.15.142:0/2557997637 >> 172.21.15.142:0/2557997637 conn(0x7f0f8c101b40 msgr2=0x7f0f8c102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:56.848 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.278+0000 7f0f933aa700 1 -- 172.21.15.142:0/2557997637 shutdown_connections 2024-03-29T04:35:56.849 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:56.278+0000 7f0f933aa700 1 -- 172.21.15.142:0/2557997637 wait complete. 2024-03-29T04:35:57.430 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:57 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2557997637' entity='client.admin' 2024-03-29T04:35:58.017 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.244+0000 7fb32a182700 1 Processor -- start 2024-03-29T04:35:58.017 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.244+0000 7fb32a182700 1 -- start start 2024-03-29T04:35:58.017 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.244+0000 7fb32a182700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 0x7fb3241093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:58.017 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.244+0000 7fb32a182700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb324079e50 con 0x7fb324107000 2024-03-29T04:35:58.017 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.244+0000 7fb3237fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 0x7fb3241093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:58.017 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.245+0000 7fb3237fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 0x7fb3241093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46642/0 (socket says 172.21.15.142:46642) 2024-03-29T04:35:58.017 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.245+0000 7fb3237fe700 1 -- 172.21.15.142:0/834300374 learned_addr learned my addr 172.21.15.142:0/834300374 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:58.017 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.245+0000 7fb3237fe700 1 -- 172.21.15.142:0/834300374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb324109930 con 0x7fb324107000 2024-03-29T04:35:58.017 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.245+0000 7fb3237fe700 1 --2- 172.21.15.142:0/834300374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 0x7fb3241093f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb3140098a0 tx=0x7fb314009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf21073e866ea7b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:58.018 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.245+0000 7fb3227fc700 1 -- 172.21.15.142:0/834300374 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb314004180 con 0x7fb324107000 2024-03-29T04:35:58.018 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.245+0000 7fb3227fc700 1 -- 172.21.15.142:0/834300374 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb3140042e0 con 0x7fb324107000 2024-03-29T04:35:58.018 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.245+0000 7fb3227fc700 1 -- 172.21.15.142:0/834300374 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb314011af0 con 0x7fb324107000 2024-03-29T04:35:58.018 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.246+0000 7fb32a182700 1 -- 172.21.15.142:0/834300374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 msgr2=0x7fb3241093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:58.018 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.246+0000 7fb32a182700 1 --2- 172.21.15.142:0/834300374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 0x7fb3241093f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb3140098a0 tx=0x7fb314009bb0 comp rx=0 tx=0).stop 2024-03-29T04:35:58.018 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.246+0000 7fb32a182700 1 -- 172.21.15.142:0/834300374 shutdown_connections 2024-03-29T04:35:58.018 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.246+0000 7fb32a182700 1 --2- 172.21.15.142:0/834300374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 0x7fb3241093f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:58.018 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.246+0000 7fb32a182700 1 -- 172.21.15.142:0/834300374 >> 172.21.15.142:0/834300374 conn(0x7fb3241009f0 msgr2=0x7fb324102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:58.018 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.246+0000 7fb32a182700 1 -- 172.21.15.142:0/834300374 shutdown_connections 2024-03-29T04:35:58.018 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.246+0000 7fb32a182700 1 -- 172.21.15.142:0/834300374 wait complete. 2024-03-29T04:35:58.018 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.246+0000 7fb32a182700 1 Processor -- start 2024-03-29T04:35:58.018 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.246+0000 7fb32a182700 1 -- start start 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.247+0000 7fb32a182700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 0x7fb324106960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.247+0000 7fb32a182700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb324106ea0 con 0x7fb324107000 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.247+0000 7fb3237fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 0x7fb324106960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.247+0000 7fb3237fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 0x7fb324106960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46648/0 (socket says 172.21.15.142:46648) 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.247+0000 7fb3237fe700 1 -- 172.21.15.142:0/496747649 learned_addr learned my addr 172.21.15.142:0/496747649 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.247+0000 7fb3237fe700 1 -- 172.21.15.142:0/496747649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb314009550 con 0x7fb324107000 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.247+0000 7fb3237fe700 1 --2- 172.21.15.142:0/496747649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 0x7fb324106960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb314011670 tx=0x7fb314011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.247+0000 7fb320ff9700 1 -- 172.21.15.142:0/496747649 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb314003a40 con 0x7fb324107000 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.247+0000 7fb320ff9700 1 -- 172.21.15.142:0/496747649 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb314003ba0 con 0x7fb324107000 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.247+0000 7fb32a182700 1 -- 172.21.15.142:0/496747649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb324104fb0 con 0x7fb324107000 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.247+0000 7fb320ff9700 1 -- 172.21.15.142:0/496747649 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb314024790 con 0x7fb324107000 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.247+0000 7fb32a182700 1 -- 172.21.15.142:0/496747649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb324105450 con 0x7fb324107000 2024-03-29T04:35:58.019 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.248+0000 7fb320ff9700 1 -- 172.21.15.142:0/496747649 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fb31402d3f0 con 0x7fb324107000 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.248+0000 7fb320ff9700 1 --2- 172.21.15.142:0/496747649 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fb30c038320 0x7fb30c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.248+0000 7fb320ff9700 1 -- 172.21.15.142:0/496747649 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb31404c540 con 0x7fb324107000 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.248+0000 7fb32a182700 1 -- 172.21.15.142:0/496747649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32410a6c0 con 0x7fb324107000 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.248+0000 7fb322ffd700 1 --2- 172.21.15.142:0/496747649 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fb30c038320 0x7fb30c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.249+0000 7fb322ffd700 1 --2- 172.21.15.142:0/496747649 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fb30c038320 0x7fb30c03a7e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb318006fd0 tx=0x7fb318006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.253+0000 7fb320ff9700 1 -- 172.21.15.142:0/496747649 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb314010970 con 0x7fb324107000 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.443+0000 7fb32a182700 1 -- 172.21.15.142:0/496747649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fb3241a0260 con 0x7fb324107000 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.448+0000 7fb320ff9700 1 -- 172.21.15.142:0/496747649 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fb31404beb0 con 0x7fb324107000 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.450+0000 7fb32a182700 1 -- 172.21.15.142:0/496747649 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fb30c038320 msgr2=0x7fb30c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.450+0000 7fb32a182700 1 --2- 172.21.15.142:0/496747649 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fb30c038320 0x7fb30c03a7e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb318006fd0 tx=0x7fb318006e40 comp rx=0 tx=0).stop 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.450+0000 7fb32a182700 1 -- 172.21.15.142:0/496747649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 msgr2=0x7fb324106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.450+0000 7fb32a182700 1 --2- 172.21.15.142:0/496747649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 0x7fb324106960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb314011670 tx=0x7fb314011b50 comp rx=0 tx=0).stop 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.450+0000 7fb32a182700 1 -- 172.21.15.142:0/496747649 shutdown_connections 2024-03-29T04:35:58.020 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.450+0000 7fb32a182700 1 --2- 172.21.15.142:0/496747649 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fb30c038320 0x7fb30c03a7e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:58.021 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.450+0000 7fb32a182700 1 --2- 172.21.15.142:0/496747649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb324107000 0x7fb324106960 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:35:58.021 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.450+0000 7fb32a182700 1 -- 172.21.15.142:0/496747649 >> 172.21.15.142:0/496747649 conn(0x7fb3241009f0 msgr2=0x7fb3241017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:35:58.021 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.451+0000 7fb32a182700 1 -- 172.21.15.142:0/496747649 shutdown_connections 2024-03-29T04:35:58.021 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:57.451+0000 7fb32a182700 1 -- 172.21.15.142:0/496747649 wait complete. 2024-03-29T04:35:58.021 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling the dashboard module... 2024-03-29T04:35:58.607 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:58 smithi142 ceph-mon[29027]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:35:58.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:58 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/496747649' entity='client.admin' 2024-03-29T04:35:59.781 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:35:59 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/576439310' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-29T04:36:00.106 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.458+0000 7f1640fe0700 1 Processor -- start 2024-03-29T04:36:00.106 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.458+0000 7f1640fe0700 1 -- start start 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.459+0000 7f1640fe0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 0x7f163c1069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.459+0000 7f1640fe0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c106f70 con 0x7f163c106580 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.459+0000 7f163a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 0x7f163c1069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.459+0000 7f163a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 0x7f163c1069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46662/0 (socket says 172.21.15.142:46662) 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.459+0000 7f163a59c700 1 -- 172.21.15.142:0/1782632290 learned_addr learned my addr 172.21.15.142:0/1782632290 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.459+0000 7f163a59c700 1 -- 172.21.15.142:0/1782632290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c107780 con 0x7f163c106580 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.459+0000 7f163a59c700 1 --2- 172.21.15.142:0/1782632290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 0x7f163c1069a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1634009e80 tx=0x7f163400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5117127ebf23af7d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.460+0000 7f163959a700 1 -- 172.21.15.142:0/1782632290 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f163400e070 con 0x7f163c106580 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.460+0000 7f163959a700 1 -- 172.21.15.142:0/1782632290 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f163400b8c0 con 0x7f163c106580 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.460+0000 7f163959a700 1 -- 172.21.15.142:0/1782632290 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16340044d0 con 0x7f163c106580 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.460+0000 7f1640fe0700 1 -- 172.21.15.142:0/1782632290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 msgr2=0x7f163c1069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.460+0000 7f1640fe0700 1 --2- 172.21.15.142:0/1782632290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 0x7f163c1069a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1634009e80 tx=0x7f163400b330 comp rx=0 tx=0).stop 2024-03-29T04:36:00.107 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.460+0000 7f1640fe0700 1 -- 172.21.15.142:0/1782632290 shutdown_connections 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.460+0000 7f1640fe0700 1 --2- 172.21.15.142:0/1782632290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 0x7f163c1069a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.460+0000 7f1640fe0700 1 -- 172.21.15.142:0/1782632290 >> 172.21.15.142:0/1782632290 conn(0x7f163c101b00 msgr2=0x7f163c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.460+0000 7f1640fe0700 1 -- 172.21.15.142:0/1782632290 shutdown_connections 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.460+0000 7f1640fe0700 1 -- 172.21.15.142:0/1782632290 wait complete. 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.461+0000 7f1640fe0700 1 Processor -- start 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.461+0000 7f1640fe0700 1 -- start start 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.461+0000 7f1640fe0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 0x7f163c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.461+0000 7f1640fe0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c19a790 con 0x7f163c106580 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.461+0000 7f163a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 0x7f163c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.461+0000 7f163a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 0x7f163c19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35796/0 (socket says 172.21.15.142:35796) 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.461+0000 7f163a59c700 1 -- 172.21.15.142:0/576439310 learned_addr learned my addr 172.21.15.142:0/576439310 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.462+0000 7f163a59c700 1 -- 172.21.15.142:0/576439310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1634009b30 con 0x7f163c106580 2024-03-29T04:36:00.108 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.462+0000 7f163a59c700 1 --2- 172.21.15.142:0/576439310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 0x7f163c19a250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1634004940 tx=0x7f1634004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.462+0000 7f162b7fe700 1 -- 172.21.15.142:0/576439310 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f163400e040 con 0x7f163c106580 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.462+0000 7f162b7fe700 1 -- 172.21.15.142:0/576439310 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f163400bc40 con 0x7f163c106580 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.462+0000 7f1640fe0700 1 -- 172.21.15.142:0/576439310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c19a990 con 0x7f163c106580 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.462+0000 7f162b7fe700 1 -- 172.21.15.142:0/576439310 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1634012670 con 0x7f163c106580 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.462+0000 7f1640fe0700 1 -- 172.21.15.142:0/576439310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c19ae90 con 0x7f163c106580 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.463+0000 7f162b7fe700 1 -- 172.21.15.142:0/576439310 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f1634019040 con 0x7f163c106580 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.463+0000 7f1640fe0700 1 -- 172.21.15.142:0/576439310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c0623c0 con 0x7f163c106580 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.463+0000 7f162b7fe700 1 --2- 172.21.15.142:0/576439310 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f1624038370 0x7f162403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.463+0000 7f162b7fe700 1 -- 172.21.15.142:0/576439310 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f163404bba0 con 0x7f163c106580 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.463+0000 7f1639d9b700 1 --2- 172.21.15.142:0/576439310 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f1624038370 0x7f162403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.463+0000 7f1639d9b700 1 --2- 172.21.15.142:0/576439310 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f1624038370 0x7f162403a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1630006fd0 tx=0x7f1630006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.468+0000 7f162b7fe700 1 -- 172.21.15.142:0/576439310 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1634011750 con 0x7f163c106580 2024-03-29T04:36:00.109 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:58.710+0000 7f1640fe0700 1 -- 172.21.15.142:0/576439310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f163c10a9b0 con 0x7f163c106580 2024-03-29T04:36:00.110 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:59.457+0000 7f162b7fe700 1 -- 172.21.15.142:0/576439310 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f163402ad30 con 0x7f163c106580 2024-03-29T04:36:00.110 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:59.459+0000 7f162b7fe700 1 -- 172.21.15.142:0/576439310 <== mon.0 v2:172.21.15.142:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f163401b430 con 0x7f163c106580 2024-03-29T04:36:00.110 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:59.461+0000 7f1640fe0700 1 -- 172.21.15.142:0/576439310 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f1624038370 msgr2=0x7f162403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:00.110 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:59.461+0000 7f1640fe0700 1 --2- 172.21.15.142:0/576439310 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f1624038370 0x7f162403a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1630006fd0 tx=0x7f1630006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:00.110 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:59.461+0000 7f1640fe0700 1 -- 172.21.15.142:0/576439310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 msgr2=0x7f163c19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:00.110 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:59.461+0000 7f1640fe0700 1 --2- 172.21.15.142:0/576439310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 0x7f163c19a250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1634004940 tx=0x7f1634004a20 comp rx=0 tx=0).stop 2024-03-29T04:36:00.110 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:59.461+0000 7f1640fe0700 1 -- 172.21.15.142:0/576439310 shutdown_connections 2024-03-29T04:36:00.110 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:59.461+0000 7f1640fe0700 1 --2- 172.21.15.142:0/576439310 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f1624038370 0x7f162403a830 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:00.110 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:59.461+0000 7f1640fe0700 1 --2- 172.21.15.142:0/576439310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f163c106580 0x7f163c19a250 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:00.110 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:59.461+0000 7f1640fe0700 1 -- 172.21.15.142:0/576439310 >> 172.21.15.142:0/576439310 conn(0x7f163c101b00 msgr2=0x7f163c1028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:00.110 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:59.461+0000 7f1640fe0700 1 -- 172.21.15.142:0/576439310 shutdown_connections 2024-03-29T04:36:00.110 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:35:59.461+0000 7f1640fe0700 1 -- 172.21.15.142:0/576439310 wait complete. 2024-03-29T04:36:00.663 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:00 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/576439310' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-29T04:36:00.664 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:00 smithi142 ceph-mon[29027]: mgrmap e9: smithi142.swesrg(active, since 26s) 2024-03-29T04:36:01.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout { 2024-03-29T04:36:01.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-29T04:36:01.339 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-29T04:36:01.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "active_name": "smithi142.swesrg", 2024-03-29T04:36:01.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-29T04:36:01.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout } 2024-03-29T04:36:01.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.471+0000 7f3619d68700 1 Processor -- start 2024-03-29T04:36:01.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.471+0000 7f3619d68700 1 -- start start 2024-03-29T04:36:01.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.471+0000 7f3619d68700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 0x7f36141093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:01.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.471+0000 7f3619d68700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3614079e90 con 0x7f3614106fe0 2024-03-29T04:36:01.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.472+0000 7f36137fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 0x7f36141093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:01.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.472+0000 7f36137fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 0x7f36141093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35816/0 (socket says 172.21.15.142:35816) 2024-03-29T04:36:01.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.472+0000 7f36137fe700 1 -- 172.21.15.142:0/3776736453 learned_addr learned my addr 172.21.15.142:0/3776736453 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:01.340 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.472+0000 7f36137fe700 1 -- 172.21.15.142:0/3776736453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614109910 con 0x7f3614106fe0 2024-03-29T04:36:01.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.472+0000 7f36137fe700 1 --2- 172.21.15.142:0/3776736453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 0x7f36141093d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f36040098a0 tx=0x7f3604009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764ca46aea4c0985 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:01.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.472+0000 7f36127fc700 1 -- 172.21.15.142:0/3776736453 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3604004180 con 0x7f3614106fe0 2024-03-29T04:36:01.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.472+0000 7f36127fc700 1 -- 172.21.15.142:0/3776736453 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f36040042e0 con 0x7f3614106fe0 2024-03-29T04:36:01.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.472+0000 7f36127fc700 1 -- 172.21.15.142:0/3776736453 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3604011af0 con 0x7f3614106fe0 2024-03-29T04:36:01.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.473+0000 7f3619d68700 1 -- 172.21.15.142:0/3776736453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 msgr2=0x7f36141093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:01.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.473+0000 7f3619d68700 1 --2- 172.21.15.142:0/3776736453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 0x7f36141093d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f36040098a0 tx=0x7f3604009bb0 comp rx=0 tx=0).stop 2024-03-29T04:36:01.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.473+0000 7f3619d68700 1 -- 172.21.15.142:0/3776736453 shutdown_connections 2024-03-29T04:36:01.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.473+0000 7f3619d68700 1 --2- 172.21.15.142:0/3776736453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 0x7f36141093d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:01.341 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.473+0000 7f3619d68700 1 -- 172.21.15.142:0/3776736453 >> 172.21.15.142:0/3776736453 conn(0x7f36141009f0 msgr2=0x7f3614102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:01.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.473+0000 7f3619d68700 1 -- 172.21.15.142:0/3776736453 shutdown_connections 2024-03-29T04:36:01.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.473+0000 7f3619d68700 1 -- 172.21.15.142:0/3776736453 wait complete. 2024-03-29T04:36:01.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.474+0000 7f3619d68700 1 Processor -- start 2024-03-29T04:36:01.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.474+0000 7f3619d68700 1 -- start start 2024-03-29T04:36:01.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.474+0000 7f3619d68700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 0x7f3614106940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:01.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.474+0000 7f3619d68700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3614106e80 con 0x7f3614106fe0 2024-03-29T04:36:01.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.474+0000 7f36137fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 0x7f3614106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:01.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.474+0000 7f36137fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 0x7f3614106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35828/0 (socket says 172.21.15.142:35828) 2024-03-29T04:36:01.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.474+0000 7f36137fe700 1 -- 172.21.15.142:0/2941943913 learned_addr learned my addr 172.21.15.142:0/2941943913 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:01.342 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.474+0000 7f36137fe700 1 -- 172.21.15.142:0/2941943913 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3604009550 con 0x7f3614106fe0 2024-03-29T04:36:01.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.475+0000 7f36137fe700 1 --2- 172.21.15.142:0/2941943913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 0x7f3614106940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3604011ad0 tx=0x7f3604011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:01.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.475+0000 7f3610ff9700 1 -- 172.21.15.142:0/2941943913 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3604003ab0 con 0x7f3614106fe0 2024-03-29T04:36:01.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.475+0000 7f3610ff9700 1 -- 172.21.15.142:0/2941943913 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3604003c10 con 0x7f3614106fe0 2024-03-29T04:36:01.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.475+0000 7f3619d68700 1 -- 172.21.15.142:0/2941943913 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3614104f90 con 0x7f3614106fe0 2024-03-29T04:36:01.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.475+0000 7f3610ff9700 1 -- 172.21.15.142:0/2941943913 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36040198e0 con 0x7f3614106fe0 2024-03-29T04:36:01.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.475+0000 7f3619d68700 1 -- 172.21.15.142:0/2941943913 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3614105430 con 0x7f3614106fe0 2024-03-29T04:36:01.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.476+0000 7f3610ff9700 1 -- 172.21.15.142:0/2941943913 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f3604019ba0 con 0x7f3614106fe0 2024-03-29T04:36:01.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.476+0000 7f3619d68700 1 -- 172.21.15.142:0/2941943913 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361410a700 con 0x7f3614106fe0 2024-03-29T04:36:01.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.476+0000 7f3610ff9700 1 --2- 172.21.15.142:0/2941943913 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f35fc038370 0x7f35fc03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:01.343 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.476+0000 7f3610ff9700 1 -- 172.21.15.142:0/2941943913 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f360404c190 con 0x7f3614106fe0 2024-03-29T04:36:01.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.476+0000 7f3612ffd700 1 -- 172.21.15.142:0/2941943913 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f35fc038370 msgr2=0x7f35fc03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2742619935 2024-03-29T04:36:01.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.476+0000 7f3612ffd700 1 --2- 172.21.15.142:0/2941943913 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f35fc038370 0x7f35fc03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:36:01.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.481+0000 7f3610ff9700 1 -- 172.21.15.142:0/2941943913 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f360402b300 con 0x7f3614106fe0 2024-03-29T04:36:01.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.677+0000 7f3612ffd700 1 -- 172.21.15.142:0/2941943913 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f35fc038370 msgr2=0x7f35fc03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2742619935 2024-03-29T04:36:01.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.677+0000 7f3612ffd700 1 --2- 172.21.15.142:0/2941943913 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f35fc038370 0x7f35fc03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:36:01.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.726+0000 7f3619d68700 1 -- 172.21.15.142:0/2941943913 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f361410a920 con 0x7f3614106fe0 2024-03-29T04:36:01.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.727+0000 7f3610ff9700 1 -- 172.21.15.142:0/2941943913 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f3604010b70 con 0x7f3614106fe0 2024-03-29T04:36:01.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.729+0000 7f3619d68700 1 -- 172.21.15.142:0/2941943913 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f35fc038370 msgr2=0x7f35fc03a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:36:01.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.729+0000 7f3619d68700 1 --2- 172.21.15.142:0/2941943913 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f35fc038370 0x7f35fc03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:01.344 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.729+0000 7f3619d68700 1 -- 172.21.15.142:0/2941943913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 msgr2=0x7f3614106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:01.345 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.729+0000 7f3619d68700 1 --2- 172.21.15.142:0/2941943913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 0x7f3614106940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3604011ad0 tx=0x7f3604011bb0 comp rx=0 tx=0).stop 2024-03-29T04:36:01.345 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.729+0000 7f3619d68700 1 -- 172.21.15.142:0/2941943913 shutdown_connections 2024-03-29T04:36:01.345 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.729+0000 7f3619d68700 1 --2- 172.21.15.142:0/2941943913 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7f35fc038370 0x7f35fc03a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:01.345 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.729+0000 7f3619d68700 1 --2- 172.21.15.142:0/2941943913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3614106fe0 0x7f3614106940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:01.345 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.729+0000 7f3619d68700 1 -- 172.21.15.142:0/2941943913 >> 172.21.15.142:0/2941943913 conn(0x7f36141009f0 msgr2=0x7f3614101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:01.345 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.729+0000 7f3619d68700 1 -- 172.21.15.142:0/2941943913 shutdown_connections 2024-03-29T04:36:01.345 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:00.729+0000 7f3619d68700 1 -- 172.21.15.142:0/2941943913 wait complete. 2024-03-29T04:36:01.345 INFO:teuthology.orchestra.run.smithi142.stdout:Waiting for the mgr to restart... 2024-03-29T04:36:01.345 INFO:teuthology.orchestra.run.smithi142.stdout:Waiting for mgr epoch 9... 2024-03-29T04:36:01.598 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:01 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2941943913' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-29T04:36:06.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: Active manager daemon smithi142.swesrg restarted 2024-03-29T04:36:06.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: Activating manager daemon smithi142.swesrg 2024-03-29T04:36:06.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: osdmap e3: 0 total, 0 up, 0 in 2024-03-29T04:36:06.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: mgrmap e10: smithi142.swesrg(active, starting, since 0.00346538s) 2024-03-29T04:36:06.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:36:06.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi142.swesrg", "id": "smithi142.swesrg"}]: dispatch 2024-03-29T04:36:06.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T04:36:06.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T04:36:06.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T04:36:06.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: Manager daemon smithi142.swesrg is now available 2024-03-29T04:36:06.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:36:06.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:36:06.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:36:06.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:06 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/trash_purge_schedule"}]: dispatch 2024-03-29T04:36:07.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:07 smithi142 ceph-mon[29027]: [29/Mar/2024:04:36:06] ENGINE Bus STARTING 2024-03-29T04:36:07.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:07 smithi142 ceph-mon[29027]: [29/Mar/2024:04:36:06] ENGINE Serving on https://172.21.15.142:7150 2024-03-29T04:36:07.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:07 smithi142 ceph-mon[29027]: mgrmap e11: smithi142.swesrg(active, since 1.00591s) 2024-03-29T04:36:07.984 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout { 2024-03-29T04:36:07.984 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-29T04:36:07.984 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-29T04:36:07.984 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout } 2024-03-29T04:36:07.984 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.795+0000 7fbab61ec700 1 Processor -- start 2024-03-29T04:36:07.984 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.795+0000 7fbab61ec700 1 -- start start 2024-03-29T04:36:07.984 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.795+0000 7fbab61ec700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 0x7fbab0104c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:07.984 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.795+0000 7fbab61ec700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab0105250 con 0x7fbab0104860 2024-03-29T04:36:07.984 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.795+0000 7fbaaf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 0x7fbab0104c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:07.984 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.795+0000 7fbaaf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 0x7fbab0104c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35832/0 (socket says 172.21.15.142:35832) 2024-03-29T04:36:07.984 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.795+0000 7fbaaf7fe700 1 -- 172.21.15.142:0/1255096191 learned_addr learned my addr 172.21.15.142:0/1255096191 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:07.984 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.796+0000 7fbaaf7fe700 1 -- 172.21.15.142:0/1255096191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab0105390 con 0x7fbab0104860 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.796+0000 7fbaaf7fe700 1 --2- 172.21.15.142:0/1255096191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 0x7fbab0104c80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40098a0 tx=0x7fbaa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e11e9cd5bd21e66 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.796+0000 7fbaae7fc700 1 -- 172.21.15.142:0/1255096191 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbaa4004180 con 0x7fbab0104860 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.796+0000 7fbaae7fc700 1 -- 172.21.15.142:0/1255096191 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbaa40042e0 con 0x7fbab0104860 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.796+0000 7fbaae7fc700 1 -- 172.21.15.142:0/1255096191 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbaa4011af0 con 0x7fbab0104860 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.797+0000 7fbab61ec700 1 -- 172.21.15.142:0/1255096191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 msgr2=0x7fbab0104c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.797+0000 7fbab61ec700 1 --2- 172.21.15.142:0/1255096191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 0x7fbab0104c80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40098a0 tx=0x7fbaa4009bb0 comp rx=0 tx=0).stop 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.797+0000 7fbab61ec700 1 -- 172.21.15.142:0/1255096191 shutdown_connections 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.797+0000 7fbab61ec700 1 --2- 172.21.15.142:0/1255096191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 0x7fbab0104c80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.797+0000 7fbab61ec700 1 -- 172.21.15.142:0/1255096191 >> 172.21.15.142:0/1255096191 conn(0x7fbab0100190 msgr2=0x7fbab01025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.797+0000 7fbab61ec700 1 -- 172.21.15.142:0/1255096191 shutdown_connections 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.797+0000 7fbab61ec700 1 -- 172.21.15.142:0/1255096191 wait complete. 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.797+0000 7fbab61ec700 1 Processor -- start 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.797+0000 7fbab61ec700 1 -- start start 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.798+0000 7fbab61ec700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 0x7fbab019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:07.985 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.798+0000 7fbab61ec700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab019a760 con 0x7fbab0104860 2024-03-29T04:36:07.986 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.798+0000 7fbaaf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 0x7fbab019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:07.986 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.798+0000 7fbaaf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 0x7fbab019a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35844/0 (socket says 172.21.15.142:35844) 2024-03-29T04:36:07.986 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.798+0000 7fbaaf7fe700 1 -- 172.21.15.142:0/494481180 learned_addr learned my addr 172.21.15.142:0/494481180 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:07.986 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.798+0000 7fbaaf7fe700 1 -- 172.21.15.142:0/494481180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa4009550 con 0x7fbab0104860 2024-03-29T04:36:07.986 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.798+0000 7fbaaf7fe700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 0x7fbab019a220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4006ba0 tx=0x7fbaa4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:07.986 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.798+0000 7fbaacff9700 1 -- 172.21.15.142:0/494481180 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbaa4003b00 con 0x7fbab0104860 2024-03-29T04:36:07.986 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.798+0000 7fbaacff9700 1 -- 172.21.15.142:0/494481180 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbaa401a8e0 con 0x7fbab0104860 2024-03-29T04:36:07.986 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.798+0000 7fbaacff9700 1 -- 172.21.15.142:0/494481180 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbaa4019990 con 0x7fbab0104860 2024-03-29T04:36:07.986 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.798+0000 7fbab61ec700 1 -- 172.21.15.142:0/494481180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab019a960 con 0x7fbab0104860 2024-03-29T04:36:07.986 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.798+0000 7fbab61ec700 1 -- 172.21.15.142:0/494481180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab019ae00 con 0x7fbab0104860 2024-03-29T04:36:07.986 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.799+0000 7fbaacff9700 1 -- 172.21.15.142:0/494481180 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fbaa4019c00 con 0x7fbab0104860 2024-03-29T04:36:07.986 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.799+0000 7fbaacff9700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 0x7fba9803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:07.987 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.799+0000 7fbaacff9700 1 -- 172.21.15.142:0/494481180 --> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba9803af40 con 0x7fba98038370 2024-03-29T04:36:07.987 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.800+0000 7fbaacff9700 1 -- 172.21.15.142:0/494481180 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbaa404c7b0 con 0x7fbab0104860 2024-03-29T04:36:07.987 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.800+0000 7fbaaeffd700 1 -- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 msgr2=0x7fba9803a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2742619935 2024-03-29T04:36:07.987 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:01.800+0000 7fbaaeffd700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 0x7fba9803a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:36:07.987 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:02.001+0000 7fbaaeffd700 1 -- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 msgr2=0x7fba9803a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2742619935 2024-03-29T04:36:07.987 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:02.001+0000 7fbaaeffd700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 0x7fba9803a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:36:07.987 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:02.402+0000 7fbaaeffd700 1 -- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 msgr2=0x7fba9803a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2742619935 2024-03-29T04:36:07.987 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:02.402+0000 7fbaaeffd700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 0x7fba9803a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T04:36:07.987 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:03.203+0000 7fbaaeffd700 1 -- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 msgr2=0x7fba9803a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2742619935 2024-03-29T04:36:07.987 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:03.203+0000 7fbaaeffd700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 0x7fba9803a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T04:36:07.987 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:04.804+0000 7fbaaeffd700 1 -- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 msgr2=0x7fba9803a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/2742619935 2024-03-29T04:36:07.987 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:04.804+0000 7fbaaeffd700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 0x7fba9803a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:06.147+0000 7fbaacff9700 1 -- 172.21.15.142:0/494481180 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mgrmap(e 10) v1 ==== 45063+0+0 (secure 0 0 0) 0x7fbaa402d620 con 0x7fbab0104860 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:06.147+0000 7fbaacff9700 1 -- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 msgr2=0x7fba9803a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:06.147+0000 7fbaacff9700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 0x7fba9803a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.148+0000 7fbaacff9700 1 -- 172.21.15.142:0/494481180 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7fbaa402d9a0 con 0x7fbab0104860 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.148+0000 7fbaacff9700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fba9803bde0 0x7fba9803e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.148+0000 7fbaacff9700 1 -- 172.21.15.142:0/494481180 --> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba9803af40 con 0x7fba9803bde0 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.149+0000 7fbaaeffd700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fba9803bde0 0x7fba9803e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.149+0000 7fbaaeffd700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fba9803bde0 0x7fba9803e1d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0003a10 tx=0x7fbaa0009270 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.150+0000 7fbaacff9700 1 -- 172.21.15.142:0/494481180 <== mgr.14162 v2:172.21.15.142:6800/779416791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fba9803af40 con 0x7fba9803bde0 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.156+0000 7fbab61ec700 1 -- 172.21.15.142:0/494481180 --> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fbab019b560 con 0x7fba9803bde0 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.156+0000 7fbaacff9700 1 -- 172.21.15.142:0/494481180 <== mgr.14162 v2:172.21.15.142:6800/779416791 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fbab019b560 con 0x7fba9803bde0 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.156+0000 7fbab61ec700 1 -- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fba9803bde0 msgr2=0x7fba9803e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.156+0000 7fbab61ec700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fba9803bde0 0x7fba9803e1d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0003a10 tx=0x7fbaa0009270 comp rx=0 tx=0).stop 2024-03-29T04:36:07.988 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.156+0000 7fbab61ec700 1 -- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 msgr2=0x7fbab019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:07.989 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.156+0000 7fbab61ec700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 0x7fbab019a220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4006ba0 tx=0x7fbaa4011b50 comp rx=0 tx=0).stop 2024-03-29T04:36:07.989 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.157+0000 7fbab61ec700 1 -- 172.21.15.142:0/494481180 shutdown_connections 2024-03-29T04:36:07.989 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.157+0000 7fbab61ec700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fba9803bde0 0x7fba9803e1d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:07.989 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.157+0000 7fbab61ec700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab0104860 0x7fbab019a220 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:07.989 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.157+0000 7fbab61ec700 1 --2- 172.21.15.142:0/494481180 >> [v2:172.21.15.142:6800/2742619935,v1:172.21.15.142:6801/2742619935] conn(0x7fba98038370 0x7fba9803a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:07.989 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.157+0000 7fbab61ec700 1 -- 172.21.15.142:0/494481180 >> 172.21.15.142:0/494481180 conn(0x7fbab0100190 msgr2=0x7fbab0100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:07.989 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.157+0000 7fbab61ec700 1 -- 172.21.15.142:0/494481180 shutdown_connections 2024-03-29T04:36:07.989 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:07.157+0000 7fbab61ec700 1 -- 172.21.15.142:0/494481180 wait complete. 2024-03-29T04:36:07.989 INFO:teuthology.orchestra.run.smithi142.stdout:mgr epoch 9 is available 2024-03-29T04:36:07.989 INFO:teuthology.orchestra.run.smithi142.stdout:Generating a dashboard self-signed certificate... 2024-03-29T04:36:08.234 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:08 smithi142 ceph-mon[29027]: [29/Mar/2024:04:36:06] ENGINE Serving on http://172.21.15.142:8765 2024-03-29T04:36:08.234 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:08 smithi142 ceph-mon[29027]: [29/Mar/2024:04:36:06] ENGINE Bus STARTED 2024-03-29T04:36:08.234 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:08 smithi142 ceph-mon[29027]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-29T04:36:08.234 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:08 smithi142 ceph-mon[29027]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-29T04:36:08.234 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:08 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:09.254 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-29T04:36:09.254 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.451+0000 7fbc77b9d700 1 Processor -- start 2024-03-29T04:36:09.254 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.451+0000 7fbc77b9d700 1 -- start start 2024-03-29T04:36:09.254 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.451+0000 7fbc77b9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 0x7fbc70104c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:09.254 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.451+0000 7fbc77b9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70105220 con 0x7fbc70104830 2024-03-29T04:36:09.254 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.451+0000 7fbc75939700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 0x7fbc70104c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:09.254 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.451+0000 7fbc75939700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 0x7fbc70104c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35928/0 (socket says 172.21.15.142:35928) 2024-03-29T04:36:09.254 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.451+0000 7fbc75939700 1 -- 172.21.15.142:0/943753333 learned_addr learned my addr 172.21.15.142:0/943753333 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.452+0000 7fbc75939700 1 -- 172.21.15.142:0/943753333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc70105360 con 0x7fbc70104830 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.452+0000 7fbc75939700 1 --2- 172.21.15.142:0/943753333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 0x7fbc70104c50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0098a0 tx=0x7fbc6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da3b3eb5664eb39 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.452+0000 7fbc74937700 1 -- 172.21.15.142:0/943753333 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc6c004180 con 0x7fbc70104830 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.452+0000 7fbc74937700 1 -- 172.21.15.142:0/943753333 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbc6c0042e0 con 0x7fbc70104830 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.452+0000 7fbc74937700 1 -- 172.21.15.142:0/943753333 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc6c011af0 con 0x7fbc70104830 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.453+0000 7fbc77b9d700 1 -- 172.21.15.142:0/943753333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 msgr2=0x7fbc70104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.453+0000 7fbc77b9d700 1 --2- 172.21.15.142:0/943753333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 0x7fbc70104c50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0098a0 tx=0x7fbc6c009bb0 comp rx=0 tx=0).stop 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.453+0000 7fbc77b9d700 1 -- 172.21.15.142:0/943753333 shutdown_connections 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.453+0000 7fbc77b9d700 1 --2- 172.21.15.142:0/943753333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 0x7fbc70104c50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.453+0000 7fbc77b9d700 1 -- 172.21.15.142:0/943753333 >> 172.21.15.142:0/943753333 conn(0x7fbc70100190 msgr2=0x7fbc701025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.453+0000 7fbc77b9d700 1 -- 172.21.15.142:0/943753333 shutdown_connections 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.453+0000 7fbc77b9d700 1 -- 172.21.15.142:0/943753333 wait complete. 2024-03-29T04:36:09.255 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.454+0000 7fbc77b9d700 1 Processor -- start 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.454+0000 7fbc77b9d700 1 -- start start 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.454+0000 7fbc77b9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 0x7fbc7019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.454+0000 7fbc77b9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7019a730 con 0x7fbc70104830 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.454+0000 7fbc75939700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 0x7fbc7019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.454+0000 7fbc75939700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 0x7fbc7019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35932/0 (socket says 172.21.15.142:35932) 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.454+0000 7fbc75939700 1 -- 172.21.15.142:0/4199194951 learned_addr learned my addr 172.21.15.142:0/4199194951 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.454+0000 7fbc75939700 1 -- 172.21.15.142:0/4199194951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c009550 con 0x7fbc70104830 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.455+0000 7fbc75939700 1 --2- 172.21.15.142:0/4199194951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 0x7fbc7019a1f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c000c00 tx=0x7fbc6c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.455+0000 7fbc66ffd700 1 -- 172.21.15.142:0/4199194951 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc6c003a70 con 0x7fbc70104830 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.455+0000 7fbc66ffd700 1 -- 172.21.15.142:0/4199194951 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbc6c003bd0 con 0x7fbc70104830 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.455+0000 7fbc66ffd700 1 -- 172.21.15.142:0/4199194951 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc6c0238c0 con 0x7fbc70104830 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.455+0000 7fbc77b9d700 1 -- 172.21.15.142:0/4199194951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7019a930 con 0x7fbc70104830 2024-03-29T04:36:09.256 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.455+0000 7fbc77b9d700 1 -- 172.21.15.142:0/4199194951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7019add0 con 0x7fbc70104830 2024-03-29T04:36:09.257 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.456+0000 7fbc66ffd700 1 -- 172.21.15.142:0/4199194951 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7fbc6c023b30 con 0x7fbc70104830 2024-03-29T04:36:09.257 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.456+0000 7fbc66ffd700 1 --2- 172.21.15.142:0/4199194951 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fbc5c0382a0 0x7fbc5c03a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:09.257 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.456+0000 7fbc66ffd700 1 -- 172.21.15.142:0/4199194951 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbc6c04d0f0 con 0x7fbc70104830 2024-03-29T04:36:09.257 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.456+0000 7fbc77b9d700 1 -- 172.21.15.142:0/4199194951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc54005320 con 0x7fbc70104830 2024-03-29T04:36:09.257 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.456+0000 7fbc75138700 1 --2- 172.21.15.142:0/4199194951 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fbc5c0382a0 0x7fbc5c03a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:09.257 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.457+0000 7fbc75138700 1 --2- 172.21.15.142:0/4199194951 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fbc5c0382a0 0x7fbc5c03a760 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc60006fd0 tx=0x7fbc60006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:09.257 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.462+0000 7fbc66ffd700 1 -- 172.21.15.142:0/4199194951 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbc6c02b300 con 0x7fbc70104830 2024-03-29T04:36:09.257 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.600+0000 7fbc66ffd700 1 -- 172.21.15.142:0/4199194951 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fbc6c02ad00 con 0x7fbc70104830 2024-03-29T04:36:09.257 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.665+0000 7fbc77b9d700 1 -- 172.21.15.142:0/4199194951 --> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc54000bf0 con 0x7fbc5c0382a0 2024-03-29T04:36:09.257 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.708+0000 7fbc66ffd700 1 -- 172.21.15.142:0/4199194951 <== mgr.14162 v2:172.21.15.142:6800/779416791 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fbc54000bf0 con 0x7fbc5c0382a0 2024-03-29T04:36:09.257 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.710+0000 7fbc77b9d700 1 -- 172.21.15.142:0/4199194951 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fbc5c0382a0 msgr2=0x7fbc5c03a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:09.258 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.710+0000 7fbc77b9d700 1 --2- 172.21.15.142:0/4199194951 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fbc5c0382a0 0x7fbc5c03a760 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc60006fd0 tx=0x7fbc60006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:09.258 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.710+0000 7fbc77b9d700 1 -- 172.21.15.142:0/4199194951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 msgr2=0x7fbc7019a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:09.258 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.710+0000 7fbc77b9d700 1 --2- 172.21.15.142:0/4199194951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 0x7fbc7019a1f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c000c00 tx=0x7fbc6c011b50 comp rx=0 tx=0).stop 2024-03-29T04:36:09.258 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.711+0000 7fbc77b9d700 1 -- 172.21.15.142:0/4199194951 shutdown_connections 2024-03-29T04:36:09.258 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.711+0000 7fbc77b9d700 1 --2- 172.21.15.142:0/4199194951 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fbc5c0382a0 0x7fbc5c03a760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:09.258 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.711+0000 7fbc77b9d700 1 --2- 172.21.15.142:0/4199194951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc70104830 0x7fbc7019a1f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:09.258 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.711+0000 7fbc77b9d700 1 -- 172.21.15.142:0/4199194951 >> 172.21.15.142:0/4199194951 conn(0x7fbc70100190 msgr2=0x7fbc70100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:09.258 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.711+0000 7fbc77b9d700 1 -- 172.21.15.142:0/4199194951 shutdown_connections 2024-03-29T04:36:09.259 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:08.711+0000 7fbc77b9d700 1 -- 172.21.15.142:0/4199194951 wait complete. 2024-03-29T04:36:09.259 INFO:teuthology.orchestra.run.smithi142.stdout:Creating initial admin user... 2024-03-29T04:36:09.632 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:09 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:09.633 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:09 smithi142 ceph-mon[29027]: mgrmap e12: smithi142.swesrg(active, since 2s) 2024-03-29T04:36:09.633 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:09 smithi142 ceph-mon[29027]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:36:09.633 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:09 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:09.633 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:09 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:10.839 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$J3y8KRBuzViJF4gFNqb7d.N/01joXMb1DKy55AvNWVlnyEqLR1FBi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711686970, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-29T04:36:10.839 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.748+0000 7fba071c2700 1 Processor -- start 2024-03-29T04:36:10.839 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.748+0000 7fba071c2700 1 -- start start 2024-03-29T04:36:10.839 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.748+0000 7fba071c2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 0x7fba00105270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:10.839 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.748+0000 7fba071c2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba00074600 con 0x7fba00104e50 2024-03-29T04:36:10.839 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.749+0000 7fba04f5e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 0x7fba00105270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.749+0000 7fba04f5e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 0x7fba00105270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58262/0 (socket says 172.21.15.142:58262) 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.749+0000 7fba04f5e700 1 -- 172.21.15.142:0/3345927801 learned_addr learned my addr 172.21.15.142:0/3345927801 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.749+0000 7fba04f5e700 1 -- 172.21.15.142:0/3345927801 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba001057b0 con 0x7fba00104e50 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.749+0000 7fba04f5e700 1 --2- 172.21.15.142:0/3345927801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 0x7fba00105270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009e80 tx=0x7fb9f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eaa6aa6b1d7c163 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.749+0000 7fb9ff7fe700 1 -- 172.21.15.142:0/3345927801 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9f800e070 con 0x7fba00104e50 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.749+0000 7fb9ff7fe700 1 -- 172.21.15.142:0/3345927801 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb9f800b8c0 con 0x7fba00104e50 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.749+0000 7fb9ff7fe700 1 -- 172.21.15.142:0/3345927801 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9f80044d0 con 0x7fba00104e50 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.750+0000 7fba071c2700 1 -- 172.21.15.142:0/3345927801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 msgr2=0x7fba00105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.750+0000 7fba071c2700 1 --2- 172.21.15.142:0/3345927801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 0x7fba00105270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009e80 tx=0x7fb9f800b330 comp rx=0 tx=0).stop 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.750+0000 7fba071c2700 1 -- 172.21.15.142:0/3345927801 shutdown_connections 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.750+0000 7fba071c2700 1 --2- 172.21.15.142:0/3345927801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 0x7fba00105270 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.750+0000 7fba071c2700 1 -- 172.21.15.142:0/3345927801 >> 172.21.15.142:0/3345927801 conn(0x7fba001009f0 msgr2=0x7fba00102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:10.840 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.750+0000 7fba071c2700 1 -- 172.21.15.142:0/3345927801 shutdown_connections 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.750+0000 7fba071c2700 1 -- 172.21.15.142:0/3345927801 wait complete. 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fba071c2700 1 Processor -- start 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fba071c2700 1 -- start start 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fba071c2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 0x7fba0019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fba071c2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0019c8b0 con 0x7fba00104e50 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fba04f5e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 0x7fba0019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fba04f5e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 0x7fba0019c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58270/0 (socket says 172.21.15.142:58270) 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fba04f5e700 1 -- 172.21.15.142:0/3970929319 learned_addr learned my addr 172.21.15.142:0/3970929319 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fba04f5e700 1 -- 172.21.15.142:0/3970929319 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f8009b30 con 0x7fba00104e50 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fba04f5e700 1 --2- 172.21.15.142:0/3970929319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 0x7fba0019c370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8004070 tx=0x7fb9f80049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fb9fdffb700 1 -- 172.21.15.142:0/3970929319 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9f800e040 con 0x7fba00104e50 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fb9fdffb700 1 -- 172.21.15.142:0/3970929319 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb9f800be00 con 0x7fba00104e50 2024-03-29T04:36:10.841 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fba071c2700 1 -- 172.21.15.142:0/3970929319 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0019cab0 con 0x7fba00104e50 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.751+0000 7fb9fdffb700 1 -- 172.21.15.142:0/3970929319 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9f8012770 con 0x7fba00104e50 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.752+0000 7fba071c2700 1 -- 172.21.15.142:0/3970929319 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0019cfb0 con 0x7fba00104e50 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.753+0000 7fb9fdffb700 1 -- 172.21.15.142:0/3970929319 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fb9f8019040 con 0x7fba00104e50 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.753+0000 7fba071c2700 1 -- 172.21.15.142:0/3970929319 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba00195c30 con 0x7fba00104e50 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.753+0000 7fb9fdffb700 1 --2- 172.21.15.142:0/3970929319 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb9ec038370 0x7fb9ec03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.753+0000 7fb9fdffb700 1 -- 172.21.15.142:0/3970929319 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb9f804bfa0 con 0x7fba00104e50 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.753+0000 7fb9fffff700 1 --2- 172.21.15.142:0/3970929319 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb9ec038370 0x7fb9ec03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.753+0000 7fb9fffff700 1 --2- 172.21.15.142:0/3970929319 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb9ec038370 0x7fb9ec03a830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8006fd0 tx=0x7fb9e8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.758+0000 7fb9fdffb700 1 -- 172.21.15.142:0/3970929319 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb9f8011980 con 0x7fba00104e50 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:09.964+0000 7fba071c2700 1 -- 172.21.15.142:0/3970929319 --> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fba000621e0 con 0x7fb9ec038370 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:10.188+0000 7fb9fdffb700 1 -- 172.21.15.142:0/3970929319 <== mgr.14162 v2:172.21.15.142:6800/779416791 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fba000621e0 con 0x7fb9ec038370 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:10.190+0000 7fba071c2700 1 -- 172.21.15.142:0/3970929319 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb9ec038370 msgr2=0x7fb9ec03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:10.842 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:10.190+0000 7fba071c2700 1 --2- 172.21.15.142:0/3970929319 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb9ec038370 0x7fb9ec03a830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8006fd0 tx=0x7fb9e8006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:10.843 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:10.190+0000 7fba071c2700 1 -- 172.21.15.142:0/3970929319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 msgr2=0x7fba0019c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:10.843 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:10.190+0000 7fba071c2700 1 --2- 172.21.15.142:0/3970929319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 0x7fba0019c370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8004070 tx=0x7fb9f80049f0 comp rx=0 tx=0).stop 2024-03-29T04:36:10.843 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:10.190+0000 7fba071c2700 1 -- 172.21.15.142:0/3970929319 shutdown_connections 2024-03-29T04:36:10.843 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:10.190+0000 7fba071c2700 1 --2- 172.21.15.142:0/3970929319 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb9ec038370 0x7fb9ec03a830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:10.843 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:10.190+0000 7fba071c2700 1 --2- 172.21.15.142:0/3970929319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba00104e50 0x7fba0019c370 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:10.843 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:10.190+0000 7fba071c2700 1 -- 172.21.15.142:0/3970929319 >> 172.21.15.142:0/3970929319 conn(0x7fba001009f0 msgr2=0x7fba00190a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:10.843 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:10.191+0000 7fba071c2700 1 -- 172.21.15.142:0/3970929319 shutdown_connections 2024-03-29T04:36:10.843 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:10.191+0000 7fba071c2700 1 -- 172.21.15.142:0/3970929319 wait complete. 2024-03-29T04:36:10.843 INFO:teuthology.orchestra.run.smithi142.stdout:Fetching dashboard port number... 2024-03-29T04:36:11.437 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:11 smithi142 ceph-mon[29027]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:36:11.437 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:11 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:12.357 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stdout 8443 2024-03-29T04:36:12.357 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.259+0000 7fdb40194700 1 Processor -- start 2024-03-29T04:36:12.357 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.259+0000 7fdb40194700 1 -- start start 2024-03-29T04:36:12.357 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.259+0000 7fdb40194700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 0x7fdb38105270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:12.357 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.259+0000 7fdb40194700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38079e10 con 0x7fdb38104e50 2024-03-29T04:36:12.357 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.259+0000 7fdb3df30700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 0x7fdb38105270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:12.358 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.259+0000 7fdb3df30700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 0x7fdb38105270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58282/0 (socket says 172.21.15.142:58282) 2024-03-29T04:36:12.358 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.259+0000 7fdb3df30700 1 -- 172.21.15.142:0/1623782000 learned_addr learned my addr 172.21.15.142:0/1623782000 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:12.358 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.259+0000 7fdb3df30700 1 -- 172.21.15.142:0/1623782000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb381057b0 con 0x7fdb38104e50 2024-03-29T04:36:12.358 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.260+0000 7fdb3df30700 1 --2- 172.21.15.142:0/1623782000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 0x7fdb38105270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb340098a0 tx=0x7fdb34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fba9fbdebb12b97 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:12.358 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.260+0000 7fdb3cf2e700 1 -- 172.21.15.142:0/1623782000 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb34004180 con 0x7fdb38104e50 2024-03-29T04:36:12.358 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.260+0000 7fdb3cf2e700 1 -- 172.21.15.142:0/1623782000 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb340042e0 con 0x7fdb38104e50 2024-03-29T04:36:12.358 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.260+0000 7fdb3cf2e700 1 -- 172.21.15.142:0/1623782000 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb34011af0 con 0x7fdb38104e50 2024-03-29T04:36:12.358 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.260+0000 7fdb40194700 1 -- 172.21.15.142:0/1623782000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 msgr2=0x7fdb38105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:12.358 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.260+0000 7fdb40194700 1 --2- 172.21.15.142:0/1623782000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 0x7fdb38105270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb340098a0 tx=0x7fdb34009bb0 comp rx=0 tx=0).stop 2024-03-29T04:36:12.358 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.260+0000 7fdb40194700 1 -- 172.21.15.142:0/1623782000 shutdown_connections 2024-03-29T04:36:12.358 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.260+0000 7fdb40194700 1 --2- 172.21.15.142:0/1623782000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 0x7fdb38105270 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:12.358 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.260+0000 7fdb40194700 1 -- 172.21.15.142:0/1623782000 >> 172.21.15.142:0/1623782000 conn(0x7fdb381009f0 msgr2=0x7fdb38102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:12.359 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.261+0000 7fdb40194700 1 -- 172.21.15.142:0/1623782000 shutdown_connections 2024-03-29T04:36:12.359 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.261+0000 7fdb40194700 1 -- 172.21.15.142:0/1623782000 wait complete. 2024-03-29T04:36:12.359 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.261+0000 7fdb40194700 1 Processor -- start 2024-03-29T04:36:12.359 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.261+0000 7fdb40194700 1 -- start start 2024-03-29T04:36:12.359 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.261+0000 7fdb40194700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 0x7fdb3819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:12.359 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.261+0000 7fdb40194700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3819c8e0 con 0x7fdb38104e50 2024-03-29T04:36:12.359 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.261+0000 7fdb3df30700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 0x7fdb3819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:12.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.262+0000 7fdb3df30700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 0x7fdb3819c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58284/0 (socket says 172.21.15.142:58284) 2024-03-29T04:36:12.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.262+0000 7fdb3df30700 1 -- 172.21.15.142:0/1612467113 learned_addr learned my addr 172.21.15.142:0/1612467113 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:12.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.262+0000 7fdb3df30700 1 -- 172.21.15.142:0/1612467113 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb34009550 con 0x7fdb38104e50 2024-03-29T04:36:12.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.262+0000 7fdb3df30700 1 --2- 172.21.15.142:0/1612467113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 0x7fdb3819c3a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb34006ba0 tx=0x7fdb34011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:12.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.262+0000 7fdb2effd700 1 -- 172.21.15.142:0/1612467113 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb34003b00 con 0x7fdb38104e50 2024-03-29T04:36:12.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.262+0000 7fdb2effd700 1 -- 172.21.15.142:0/1612467113 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb3401a8e0 con 0x7fdb38104e50 2024-03-29T04:36:12.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.262+0000 7fdb40194700 1 -- 172.21.15.142:0/1612467113 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3819cae0 con 0x7fdb38104e50 2024-03-29T04:36:12.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.262+0000 7fdb2effd700 1 -- 172.21.15.142:0/1612467113 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb340199c0 con 0x7fdb38104e50 2024-03-29T04:36:12.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.262+0000 7fdb40194700 1 -- 172.21.15.142:0/1612467113 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3819cf80 con 0x7fdb38104e50 2024-03-29T04:36:12.360 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.263+0000 7fdb2effd700 1 -- 172.21.15.142:0/1612467113 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fdb34019c30 con 0x7fdb38104e50 2024-03-29T04:36:12.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.263+0000 7fdb40194700 1 -- 172.21.15.142:0/1612467113 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb38195c30 con 0x7fdb38104e50 2024-03-29T04:36:12.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.263+0000 7fdb2effd700 1 --2- 172.21.15.142:0/1612467113 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fdb24038370 0x7fdb2403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:12.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.264+0000 7fdb2effd700 1 -- 172.21.15.142:0/1612467113 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdb3404c7c0 con 0x7fdb38104e50 2024-03-29T04:36:12.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.264+0000 7fdb3d72f700 1 --2- 172.21.15.142:0/1612467113 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fdb24038370 0x7fdb2403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:12.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.264+0000 7fdb3d72f700 1 --2- 172.21.15.142:0/1612467113 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fdb24038370 0x7fdb2403a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdb28006fd0 tx=0x7fdb28006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:12.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.268+0000 7fdb2effd700 1 -- 172.21.15.142:0/1612467113 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdb3402d440 con 0x7fdb38104e50 2024-03-29T04:36:12.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.465+0000 7fdb40194700 1 -- 172.21.15.142:0/1612467113 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fdb3802cc70 con 0x7fdb38104e50 2024-03-29T04:36:12.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.466+0000 7fdb2effd700 1 -- 172.21.15.142:0/1612467113 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fdb3402b340 con 0x7fdb38104e50 2024-03-29T04:36:12.361 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.468+0000 7fdb40194700 1 -- 172.21.15.142:0/1612467113 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fdb24038370 msgr2=0x7fdb2403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:12.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.468+0000 7fdb40194700 1 --2- 172.21.15.142:0/1612467113 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fdb24038370 0x7fdb2403a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdb28006fd0 tx=0x7fdb28006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:12.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.468+0000 7fdb40194700 1 -- 172.21.15.142:0/1612467113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 msgr2=0x7fdb3819c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:12.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.468+0000 7fdb40194700 1 --2- 172.21.15.142:0/1612467113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 0x7fdb3819c3a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb34006ba0 tx=0x7fdb34011b50 comp rx=0 tx=0).stop 2024-03-29T04:36:12.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.468+0000 7fdb40194700 1 -- 172.21.15.142:0/1612467113 shutdown_connections 2024-03-29T04:36:12.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.468+0000 7fdb40194700 1 --2- 172.21.15.142:0/1612467113 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fdb24038370 0x7fdb2403a830 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:12.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.468+0000 7fdb40194700 1 --2- 172.21.15.142:0/1612467113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb38104e50 0x7fdb3819c3a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:12.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.468+0000 7fdb40194700 1 -- 172.21.15.142:0/1612467113 >> 172.21.15.142:0/1612467113 conn(0x7fdb381009f0 msgr2=0x7fdb38190a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:12.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.468+0000 7fdb40194700 1 -- 172.21.15.142:0/1612467113 shutdown_connections 2024-03-29T04:36:12.362 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:11.468+0000 7fdb40194700 1 -- 172.21.15.142:0/1612467113 wait complete. 2024-03-29T04:36:12.363 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:12 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1612467113' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-29T04:36:12.364 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-29T04:36:12.364 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout disabled 2024-03-29T04:36:12.372 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-29T04:36:12.372 INFO:teuthology.orchestra.run.smithi142.stdout:systemctl: stdout inactive 2024-03-29T04:36:12.372 INFO:teuthology.orchestra.run.smithi142.stdout:firewalld.service is not enabled 2024-03-29T04:36:12.372 INFO:teuthology.orchestra.run.smithi142.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-29T04:36:12.374 INFO:teuthology.orchestra.run.smithi142.stdout:Ceph Dashboard is now available at: 2024-03-29T04:36:12.374 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:36:12.375 INFO:teuthology.orchestra.run.smithi142.stdout: URL: https://smithi142.front.sepia.ceph.com:8443/ 2024-03-29T04:36:12.375 INFO:teuthology.orchestra.run.smithi142.stdout: User: admin 2024-03-29T04:36:12.375 INFO:teuthology.orchestra.run.smithi142.stdout: Password: 8ozshsdzoo 2024-03-29T04:36:12.375 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:36:12.375 INFO:teuthology.orchestra.run.smithi142.stdout:Saving cluster configuration to /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config directory 2024-03-29T04:36:12.375 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling autotune for osd_memory_target 2024-03-29T04:36:13.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:13 smithi142 ceph-mon[29027]: mgrmap e13: smithi142.swesrg(active, since 6s) 2024-03-29T04:36:13.645 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.831+0000 7f5e42204700 1 Processor -- start 2024-03-29T04:36:13.645 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.831+0000 7f5e42204700 1 -- start start 2024-03-29T04:36:13.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.831+0000 7f5e42204700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 0x7f5e3c106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:13.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.831+0000 7f5e42204700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c106f50 con 0x7f5e3c106560 2024-03-29T04:36:13.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.831+0000 7f5e3b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 0x7f5e3c106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:13.646 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.831+0000 7f5e3b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 0x7f5e3c106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58290/0 (socket says 172.21.15.142:58290) 2024-03-29T04:36:13.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.831+0000 7f5e3b7fe700 1 -- 172.21.15.142:0/1462296161 learned_addr learned my addr 172.21.15.142:0/1462296161 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:13.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.831+0000 7f5e3b7fe700 1 -- 172.21.15.142:0/1462296161 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c107760 con 0x7f5e3c106560 2024-03-29T04:36:13.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.832+0000 7f5e3b7fe700 1 --2- 172.21.15.142:0/1462296161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 0x7f5e3c106980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0098a0 tx=0x7f5e2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628cda2b105cbec3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:13.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.832+0000 7f5e3a7fc700 1 -- 172.21.15.142:0/1462296161 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e2c004180 con 0x7f5e3c106560 2024-03-29T04:36:13.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.832+0000 7f5e3a7fc700 1 -- 172.21.15.142:0/1462296161 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5e2c0042e0 con 0x7f5e3c106560 2024-03-29T04:36:13.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.832+0000 7f5e3a7fc700 1 -- 172.21.15.142:0/1462296161 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e2c011af0 con 0x7f5e3c106560 2024-03-29T04:36:13.647 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.832+0000 7f5e42204700 1 -- 172.21.15.142:0/1462296161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 msgr2=0x7f5e3c106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:13.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.832+0000 7f5e42204700 1 --2- 172.21.15.142:0/1462296161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 0x7f5e3c106980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0098a0 tx=0x7f5e2c009bb0 comp rx=0 tx=0).stop 2024-03-29T04:36:13.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.832+0000 7f5e42204700 1 -- 172.21.15.142:0/1462296161 shutdown_connections 2024-03-29T04:36:13.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.832+0000 7f5e42204700 1 --2- 172.21.15.142:0/1462296161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 0x7f5e3c106980 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:13.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.832+0000 7f5e42204700 1 -- 172.21.15.142:0/1462296161 >> 172.21.15.142:0/1462296161 conn(0x7f5e3c101b40 msgr2=0x7f5e3c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:13.648 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.832+0000 7f5e42204700 1 -- 172.21.15.142:0/1462296161 shutdown_connections 2024-03-29T04:36:13.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.832+0000 7f5e42204700 1 -- 172.21.15.142:0/1462296161 wait complete. 2024-03-29T04:36:13.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.833+0000 7f5e42204700 1 Processor -- start 2024-03-29T04:36:13.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.833+0000 7f5e42204700 1 -- start start 2024-03-29T04:36:13.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.833+0000 7f5e42204700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 0x7f5e3c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:13.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.833+0000 7f5e42204700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c19c980 con 0x7f5e3c106560 2024-03-29T04:36:13.649 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.833+0000 7f5e3b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 0x7f5e3c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:13.650 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.833+0000 7f5e3b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 0x7f5e3c19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58296/0 (socket says 172.21.15.142:58296) 2024-03-29T04:36:13.650 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.833+0000 7f5e3b7fe700 1 -- 172.21.15.142:0/244707946 learned_addr learned my addr 172.21.15.142:0/244707946 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:13.650 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.834+0000 7f5e3b7fe700 1 -- 172.21.15.142:0/244707946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c009550 con 0x7f5e3c106560 2024-03-29T04:36:13.650 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.834+0000 7f5e3b7fe700 1 --2- 172.21.15.142:0/244707946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 0x7f5e3c19c440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c011670 tx=0x7f5e2c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:13.650 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.834+0000 7f5e38ff9700 1 -- 172.21.15.142:0/244707946 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e2c0039c0 con 0x7f5e3c106560 2024-03-29T04:36:13.650 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.834+0000 7f5e38ff9700 1 -- 172.21.15.142:0/244707946 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5e2c003b20 con 0x7f5e3c106560 2024-03-29T04:36:13.651 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.834+0000 7f5e42204700 1 -- 172.21.15.142:0/244707946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c19cb80 con 0x7f5e3c106560 2024-03-29T04:36:13.651 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.834+0000 7f5e38ff9700 1 -- 172.21.15.142:0/244707946 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e2c0247f0 con 0x7f5e3c106560 2024-03-29T04:36:13.651 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.834+0000 7f5e42204700 1 -- 172.21.15.142:0/244707946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c19d020 con 0x7f5e3c106560 2024-03-29T04:36:13.651 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.835+0000 7f5e38ff9700 1 -- 172.21.15.142:0/244707946 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f5e2c024ab0 con 0x7f5e3c106560 2024-03-29T04:36:13.651 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.835+0000 7f5e42204700 1 -- 172.21.15.142:0/244707946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c0623c0 con 0x7f5e3c106560 2024-03-29T04:36:13.651 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.835+0000 7f5e38ff9700 1 --2- 172.21.15.142:0/244707946 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f5e240383d0 0x7f5e2403a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:13.652 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.835+0000 7f5e38ff9700 1 -- 172.21.15.142:0/244707946 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5e2c04c780 con 0x7f5e3c106560 2024-03-29T04:36:13.652 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.835+0000 7f5e3affd700 1 --2- 172.21.15.142:0/244707946 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f5e240383d0 0x7f5e2403a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:13.652 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.836+0000 7f5e3affd700 1 --2- 172.21.15.142:0/244707946 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f5e240383d0 0x7f5e2403a890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5e30006fd0 tx=0x7f5e30006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:13.653 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:12.840+0000 7f5e38ff9700 1 -- 172.21.15.142:0/244707946 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5e2c022720 con 0x7f5e3c106560 2024-03-29T04:36:13.653 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:13.026+0000 7f5e42204700 1 -- 172.21.15.142:0/244707946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f5e3c19fe60 con 0x7f5e3c106560 2024-03-29T04:36:13.653 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:13.027+0000 7f5e38ff9700 1 -- 172.21.15.142:0/244707946 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f5e2c02c350 con 0x7f5e3c106560 2024-03-29T04:36:13.653 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:13.029+0000 7f5e42204700 1 -- 172.21.15.142:0/244707946 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f5e240383d0 msgr2=0x7f5e2403a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:13.653 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:13.029+0000 7f5e42204700 1 --2- 172.21.15.142:0/244707946 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f5e240383d0 0x7f5e2403a890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5e30006fd0 tx=0x7f5e30006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:13.653 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:13.029+0000 7f5e42204700 1 -- 172.21.15.142:0/244707946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 msgr2=0x7f5e3c19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:13.654 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:13.029+0000 7f5e42204700 1 --2- 172.21.15.142:0/244707946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 0x7f5e3c19c440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c011670 tx=0x7f5e2c011aa0 comp rx=0 tx=0).stop 2024-03-29T04:36:13.654 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:13.029+0000 7f5e42204700 1 -- 172.21.15.142:0/244707946 shutdown_connections 2024-03-29T04:36:13.654 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:13.029+0000 7f5e42204700 1 --2- 172.21.15.142:0/244707946 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f5e240383d0 0x7f5e2403a890 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:13.654 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:13.029+0000 7f5e42204700 1 --2- 172.21.15.142:0/244707946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c106560 0x7f5e3c19c440 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:13.654 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:13.029+0000 7f5e42204700 1 -- 172.21.15.142:0/244707946 >> 172.21.15.142:0/244707946 conn(0x7f5e3c101b40 msgr2=0x7f5e3c1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:13.655 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:13.029+0000 7f5e42204700 1 -- 172.21.15.142:0/244707946 shutdown_connections 2024-03-29T04:36:13.655 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:13.029+0000 7f5e42204700 1 -- 172.21.15.142:0/244707946 wait complete. 2024-03-29T04:36:14.954 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.073+0000 7f1012232700 1 Processor -- start 2024-03-29T04:36:14.954 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.073+0000 7f1012232700 1 -- start start 2024-03-29T04:36:14.954 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.073+0000 7f1012232700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 0x7f100c109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:14.954 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.073+0000 7f1012232700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c074640 con 0x7f100c107020 2024-03-29T04:36:14.954 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.074+0000 7f100b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 0x7f100c109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:14.954 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.074+0000 7f100b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 0x7f100c109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58302/0 (socket says 172.21.15.142:58302) 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.074+0000 7f100b7fe700 1 -- 172.21.15.142:0/122444486 learned_addr learned my addr 172.21.15.142:0/122444486 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.074+0000 7f100b7fe700 1 -- 172.21.15.142:0/122444486 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c109950 con 0x7f100c107020 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.074+0000 7f100b7fe700 1 --2- 172.21.15.142:0/122444486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 0x7f100c109410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0098a0 tx=0x7f0ffc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7958901166a1be server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.074+0000 7f100affd700 1 -- 172.21.15.142:0/122444486 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ffc004180 con 0x7f100c107020 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.074+0000 7f100affd700 1 -- 172.21.15.142:0/122444486 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ffc0042e0 con 0x7f100c107020 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.074+0000 7f100affd700 1 -- 172.21.15.142:0/122444486 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ffc011af0 con 0x7f100c107020 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.075+0000 7f1012232700 1 -- 172.21.15.142:0/122444486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 msgr2=0x7f100c109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.075+0000 7f1012232700 1 --2- 172.21.15.142:0/122444486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 0x7f100c109410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0098a0 tx=0x7f0ffc009bb0 comp rx=0 tx=0).stop 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.075+0000 7f1012232700 1 -- 172.21.15.142:0/122444486 shutdown_connections 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.075+0000 7f1012232700 1 --2- 172.21.15.142:0/122444486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 0x7f100c109410 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.075+0000 7f1012232700 1 -- 172.21.15.142:0/122444486 >> 172.21.15.142:0/122444486 conn(0x7f100c1009f0 msgr2=0x7f100c102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.075+0000 7f1012232700 1 -- 172.21.15.142:0/122444486 shutdown_connections 2024-03-29T04:36:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.075+0000 7f1012232700 1 -- 172.21.15.142:0/122444486 wait complete. 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.076+0000 7f1012232700 1 Processor -- start 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.076+0000 7f1012232700 1 -- start start 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.076+0000 7f1012232700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 0x7f100c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.076+0000 7f1012232700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c19a6c0 con 0x7f100c107020 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.076+0000 7f100b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 0x7f100c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.076+0000 7f100b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 0x7f100c19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58312/0 (socket says 172.21.15.142:58312) 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.076+0000 7f100b7fe700 1 -- 172.21.15.142:0/180027233 learned_addr learned my addr 172.21.15.142:0/180027233 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.076+0000 7f100b7fe700 1 -- 172.21.15.142:0/180027233 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ffc009550 con 0x7f100c107020 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.076+0000 7f100b7fe700 1 --2- 172.21.15.142:0/180027233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 0x7f100c19a180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc006ba0 tx=0x7f0ffc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.077+0000 7f10097fa700 1 -- 172.21.15.142:0/180027233 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ffc003b00 con 0x7f100c107020 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.077+0000 7f10097fa700 1 -- 172.21.15.142:0/180027233 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ffc01a8e0 con 0x7f100c107020 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.077+0000 7f1012232700 1 -- 172.21.15.142:0/180027233 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c19a8c0 con 0x7f100c107020 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.077+0000 7f10097fa700 1 -- 172.21.15.142:0/180027233 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ffc0199c0 con 0x7f100c107020 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.077+0000 7f1012232700 1 -- 172.21.15.142:0/180027233 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c19ad60 con 0x7f100c107020 2024-03-29T04:36:14.956 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.077+0000 7f10097fa700 1 -- 172.21.15.142:0/180027233 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f0ffc019c80 con 0x7f100c107020 2024-03-29T04:36:14.957 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.078+0000 7f10097fa700 1 --2- 172.21.15.142:0/180027233 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f0ff80383c0 0x7f0ff803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:14.957 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.078+0000 7f10097fa700 1 -- 172.21.15.142:0/180027233 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0ffc04c7e0 con 0x7f100c107020 2024-03-29T04:36:14.957 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.078+0000 7f1002ffd700 1 --2- 172.21.15.142:0/180027233 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f0ff80383c0 0x7f0ff803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:14.957 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.078+0000 7f1012232700 1 -- 172.21.15.142:0/180027233 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100c0623c0 con 0x7f100c107020 2024-03-29T04:36:14.957 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.078+0000 7f1002ffd700 1 --2- 172.21.15.142:0/180027233 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f0ff80383c0 0x7f0ff803a880 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4006fd0 tx=0x7f0ff4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:14.957 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.083+0000 7f10097fa700 1 -- 172.21.15.142:0/180027233 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0ffc02d440 con 0x7f100c107020 2024-03-29T04:36:14.957 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.383+0000 7f1012232700 1 -- 172.21.15.142:0/180027233 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f100c19d720 con 0x7f100c107020 2024-03-29T04:36:14.957 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.386+0000 7f10097fa700 1 -- 172.21.15.142:0/180027233 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f0ffc0172b0 con 0x7f100c107020 2024-03-29T04:36:14.957 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.388+0000 7f1012232700 1 -- 172.21.15.142:0/180027233 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f0ff80383c0 msgr2=0x7f0ff803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.388+0000 7f1012232700 1 --2- 172.21.15.142:0/180027233 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f0ff80383c0 0x7f0ff803a880 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4006fd0 tx=0x7f0ff4006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.388+0000 7f1012232700 1 -- 172.21.15.142:0/180027233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 msgr2=0x7f100c19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.388+0000 7f1012232700 1 --2- 172.21.15.142:0/180027233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 0x7f100c19a180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc006ba0 tx=0x7f0ffc011b50 comp rx=0 tx=0).stop 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.388+0000 7f1012232700 1 -- 172.21.15.142:0/180027233 shutdown_connections 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.388+0000 7f1012232700 1 --2- 172.21.15.142:0/180027233 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f0ff80383c0 0x7f0ff803a880 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.388+0000 7f1012232700 1 --2- 172.21.15.142:0/180027233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c107020 0x7f100c19a180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.388+0000 7f1012232700 1 -- 172.21.15.142:0/180027233 >> 172.21.15.142:0/180027233 conn(0x7f100c1009f0 msgr2=0x7f100c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.388+0000 7f1012232700 1 -- 172.21.15.142:0/180027233 shutdown_connections 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr 2024-03-29T04:36:14.388+0000 7f1012232700 1 -- 172.21.15.142:0/180027233 wait complete. 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:36:14.958 INFO:teuthology.orchestra.run.smithi142.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout:Or, if you are only running a single cluster on this host: 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout: ceph telemetry on 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout:For more information see: 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:36:14.959 INFO:teuthology.orchestra.run.smithi142.stdout:Bootstrap complete. 2024-03-29T04:36:14.987 INFO:tasks.cephadm:Fetching config... 2024-03-29T04:36:14.987 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:36:14.987 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-29T04:36:15.043 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-29T04:36:15.043 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:36:15.043 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-29T04:36:15.098 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-29T04:36:15.098 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:36:15.099 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/keyring of=/dev/stdout 2024-03-29T04:36:15.166 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-29T04:36:15.166 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:36:15.166 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-29T04:36:15.220 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-29T04:36:15.221 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyrWIhg0Jw/09phX82LS12t9kKJ9C2aYHfPk9qgesnBPZ3pzdAqdSgTfJNJMzYZlPr0Z+8URVxnSLng35dm/4kmQQLBpSlEO4DCfRRW3Nw0kD6Sxq6ktYpgsrOooobt+gy9M4WjZpIrzGnvNXMxcJZUcHwazDSKxoX2fupAVgB56h/iP/HIU6z0kuPbUIi/hDv0ouK2MJUYTaU4Z+qmMF6CV+acztBRBPambhWvUPgtIX2t1ntiJMHdxWZWHLKhH9pw6Apbc2Jf25Sg324gIdw1oblH2McYxG8m6ob+l/DA/lJBdQfEnY/jJWQmN2j84/McuJS2HlZKgSQTiLy5w1jsNfWqH95IYcBT+CN/FdxrszqNczIw8o9x1x4A/xKdO9XHkxhhsticv0LaxZ/RYroKitTRZ1vwQzVexMDwMixE3f1aDXmmBMgV+X5odW80QTRikb6FoQtm/vdr6jR5saT5KvBaQ8AxApHw9gBPZvJrscbVf/mwKcTY8cCYE+gdD0= ceph-9f96302c-ed85-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-29T04:36:15.302 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyrWIhg0Jw/09phX82LS12t9kKJ9C2aYHfPk9qgesnBPZ3pzdAqdSgTfJNJMzYZlPr0Z+8URVxnSLng35dm/4kmQQLBpSlEO4DCfRRW3Nw0kD6Sxq6ktYpgsrOooobt+gy9M4WjZpIrzGnvNXMxcJZUcHwazDSKxoX2fupAVgB56h/iP/HIU6z0kuPbUIi/hDv0ouK2MJUYTaU4Z+qmMF6CV+acztBRBPambhWvUPgtIX2t1ntiJMHdxWZWHLKhH9pw6Apbc2Jf25Sg324gIdw1oblH2McYxG8m6ob+l/DA/lJBdQfEnY/jJWQmN2j84/McuJS2HlZKgSQTiLy5w1jsNfWqH95IYcBT+CN/FdxrszqNczIw8o9x1x4A/xKdO9XHkxhhsticv0LaxZ/RYroKitTRZ1vwQzVexMDwMixE3f1aDXmmBMgV+X5odW80QTRikb6FoQtm/vdr6jR5saT5KvBaQ8AxApHw9gBPZvJrscbVf/mwKcTY8cCYE+gdD0= ceph-9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:36:15.315 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyrWIhg0Jw/09phX82LS12t9kKJ9C2aYHfPk9qgesnBPZ3pzdAqdSgTfJNJMzYZlPr0Z+8URVxnSLng35dm/4kmQQLBpSlEO4DCfRRW3Nw0kD6Sxq6ktYpgsrOooobt+gy9M4WjZpIrzGnvNXMxcJZUcHwazDSKxoX2fupAVgB56h/iP/HIU6z0kuPbUIi/hDv0ouK2MJUYTaU4Z+qmMF6CV+acztBRBPambhWvUPgtIX2t1ntiJMHdxWZWHLKhH9pw6Apbc2Jf25Sg324gIdw1oblH2McYxG8m6ob+l/DA/lJBdQfEnY/jJWQmN2j84/McuJS2HlZKgSQTiLy5w1jsNfWqH95IYcBT+CN/FdxrszqNczIw8o9x1x4A/xKdO9XHkxhhsticv0LaxZ/RYroKitTRZ1vwQzVexMDwMixE3f1aDXmmBMgV+X5odW80QTRikb6FoQtm/vdr6jR5saT5KvBaQ8AxApHw9gBPZvJrscbVf/mwKcTY8cCYE+gdD0= ceph-9f96302c-ed85-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-29T04:36:15.361 INFO:teuthology.orchestra.run.smithi152.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyrWIhg0Jw/09phX82LS12t9kKJ9C2aYHfPk9qgesnBPZ3pzdAqdSgTfJNJMzYZlPr0Z+8URVxnSLng35dm/4kmQQLBpSlEO4DCfRRW3Nw0kD6Sxq6ktYpgsrOooobt+gy9M4WjZpIrzGnvNXMxcJZUcHwazDSKxoX2fupAVgB56h/iP/HIU6z0kuPbUIi/hDv0ouK2MJUYTaU4Z+qmMF6CV+acztBRBPambhWvUPgtIX2t1ntiJMHdxWZWHLKhH9pw6Apbc2Jf25Sg324gIdw1oblH2McYxG8m6ob+l/DA/lJBdQfEnY/jJWQmN2j84/McuJS2HlZKgSQTiLy5w1jsNfWqH95IYcBT+CN/FdxrszqNczIw8o9x1x4A/xKdO9XHkxhhsticv0LaxZ/RYroKitTRZ1vwQzVexMDwMixE3f1aDXmmBMgV+X5odW80QTRikb6FoQtm/vdr6jR5saT5KvBaQ8AxApHw9gBPZvJrscbVf/mwKcTY8cCYE+gdD0= ceph-9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:36:15.376 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-29T04:36:15.577 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:36:15.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:15 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/180027233' entity='client.admin' 2024-03-29T04:36:17.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.402+0000 7f2740c2d700 1 -- 172.21.15.142:0/2806236783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f273c101330 msgr2=0x7f273c103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:17.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.402+0000 7f2740c2d700 1 --2- 172.21.15.142:0/2806236783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f273c101330 0x7f273c103720 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2734009910 tx=0x7f2734009c20 comp rx=0 tx=0).stop 2024-03-29T04:36:17.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.402+0000 7f2740c2d700 1 -- 172.21.15.142:0/2806236783 shutdown_connections 2024-03-29T04:36:17.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.402+0000 7f2740c2d700 1 --2- 172.21.15.142:0/2806236783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f273c101330 0x7f273c103720 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:17.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.402+0000 7f2740c2d700 1 -- 172.21.15.142:0/2806236783 >> 172.21.15.142:0/2806236783 conn(0x7f273c0fad20 msgr2=0x7f273c0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:17.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.402+0000 7f2740c2d700 1 -- 172.21.15.142:0/2806236783 shutdown_connections 2024-03-29T04:36:17.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.402+0000 7f2740c2d700 1 -- 172.21.15.142:0/2806236783 wait complete. 2024-03-29T04:36:17.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.403+0000 7f2740c2d700 1 Processor -- start 2024-03-29T04:36:17.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.403+0000 7f2740c2d700 1 -- start start 2024-03-29T04:36:17.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.403+0000 7f2740c2d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f273c101330 0x7f273c195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:17.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.403+0000 7f2740c2d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c195e70 con 0x7f273c101330 2024-03-29T04:36:17.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.403+0000 7f273a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f273c101330 0x7f273c195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:17.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.403+0000 7f273a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f273c101330 0x7f273c195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58332/0 (socket says 172.21.15.142:58332) 2024-03-29T04:36:17.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.403+0000 7f273a59c700 1 -- 172.21.15.142:0/2234750106 learned_addr learned my addr 172.21.15.142:0/2234750106 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:17.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.404+0000 7f273a59c700 1 -- 172.21.15.142:0/2234750106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27340095f0 con 0x7f273c101330 2024-03-29T04:36:17.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.404+0000 7f273a59c700 1 --2- 172.21.15.142:0/2234750106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f273c101330 0x7f273c195930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2734005b40 tx=0x7f2734004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:17.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.404+0000 7f272b7fe700 1 -- 172.21.15.142:0/2234750106 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f273401d070 con 0x7f273c101330 2024-03-29T04:36:17.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.404+0000 7f272b7fe700 1 -- 172.21.15.142:0/2234750106 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f273400be10 con 0x7f273c101330 2024-03-29T04:36:17.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.404+0000 7f2740c2d700 1 -- 172.21.15.142:0/2234750106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c196070 con 0x7f273c101330 2024-03-29T04:36:17.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.404+0000 7f272b7fe700 1 -- 172.21.15.142:0/2234750106 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f273400f460 con 0x7f273c101330 2024-03-29T04:36:17.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.404+0000 7f2740c2d700 1 -- 172.21.15.142:0/2234750106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273c196510 con 0x7f273c101330 2024-03-29T04:36:17.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.405+0000 7f272b7fe700 1 -- 172.21.15.142:0/2234750106 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f273400f6d0 con 0x7f273c101330 2024-03-29T04:36:17.394 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.405+0000 7f272b7fe700 1 --2- 172.21.15.142:0/2234750106 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f27240383c0 0x7f272403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:17.394 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.405+0000 7f272b7fe700 1 -- 172.21.15.142:0/2234750106 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f273404d460 con 0x7f273c101330 2024-03-29T04:36:17.394 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.405+0000 7f2739d9b700 1 --2- 172.21.15.142:0/2234750106 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f27240383c0 0x7f272403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:17.394 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.405+0000 7f2740c2d700 1 -- 172.21.15.142:0/2234750106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273c18f4c0 con 0x7f273c101330 2024-03-29T04:36:17.394 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.405+0000 7f2739d9b700 1 --2- 172.21.15.142:0/2234750106 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f27240383c0 0x7f272403a880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2730006fd0 tx=0x7f2730006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:17.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.410+0000 7f272b7fe700 1 -- 172.21.15.142:0/2234750106 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2734027070 con 0x7f273c101330 2024-03-29T04:36:17.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.600+0000 7f2740c2d700 1 -- 172.21.15.142:0/2234750106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f273c0623c0 con 0x7f273c101330 2024-03-29T04:36:17.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.603+0000 7f272b7fe700 1 -- 172.21.15.142:0/2234750106 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f273402c440 con 0x7f273c101330 2024-03-29T04:36:17.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.605+0000 7f2740c2d700 1 -- 172.21.15.142:0/2234750106 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f27240383c0 msgr2=0x7f272403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:17.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.605+0000 7f2740c2d700 1 --2- 172.21.15.142:0/2234750106 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f27240383c0 0x7f272403a880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2730006fd0 tx=0x7f2730006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:17.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.605+0000 7f2740c2d700 1 -- 172.21.15.142:0/2234750106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f273c101330 msgr2=0x7f273c195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:17.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.605+0000 7f2740c2d700 1 --2- 172.21.15.142:0/2234750106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f273c101330 0x7f273c195930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2734005b40 tx=0x7f2734004740 comp rx=0 tx=0).stop 2024-03-29T04:36:17.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.605+0000 7f2740c2d700 1 -- 172.21.15.142:0/2234750106 shutdown_connections 2024-03-29T04:36:17.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.605+0000 7f2740c2d700 1 --2- 172.21.15.142:0/2234750106 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f27240383c0 0x7f272403a880 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:17.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.605+0000 7f2740c2d700 1 --2- 172.21.15.142:0/2234750106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f273c101330 0x7f273c195930 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:17.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.605+0000 7f2740c2d700 1 -- 172.21.15.142:0/2234750106 >> 172.21.15.142:0/2234750106 conn(0x7f273c0fad20 msgr2=0x7f273c0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:17.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.605+0000 7f2740c2d700 1 -- 172.21.15.142:0/2234750106 shutdown_connections 2024-03-29T04:36:17.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:17.606+0000 7f2740c2d700 1 -- 172.21.15.142:0/2234750106 wait complete. 2024-03-29T04:36:18.827 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:18 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2234750106' entity='client.admin' 2024-03-29T04:36:18.828 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:18 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:18.828 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:18 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:18.828 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:18 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:36:18.828 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:18 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:18.828 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:18 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:36:18.828 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:18 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-29T04:36:18.828 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:18 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:36:18.828 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:18 smithi142 ceph-mon[29027]: Deploying daemon ceph-exporter.smithi142 on smithi142 2024-03-29T04:36:18.854 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-29T04:36:18.854 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-29T04:36:19.658 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:36:22.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d52f9e700 1 -- 172.21.15.142:0/2058575093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d4c0ff790 msgr2=0x7f3d4c0ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:22.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d52f9e700 1 --2- 172.21.15.142:0/2058575093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d4c0ff790 0x7f3d4c0ffbb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009910 tx=0x7f3d40009c20 comp rx=0 tx=0).stop 2024-03-29T04:36:22.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d52f9e700 1 -- 172.21.15.142:0/2058575093 shutdown_connections 2024-03-29T04:36:22.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d52f9e700 1 --2- 172.21.15.142:0/2058575093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d4c0ff790 0x7f3d4c0ffbb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:22.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d52f9e700 1 -- 172.21.15.142:0/2058575093 >> 172.21.15.142:0/2058575093 conn(0x7f3d4c0fad30 msgr2=0x7f3d4c0fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:22.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d52f9e700 1 -- 172.21.15.142:0/2058575093 shutdown_connections 2024-03-29T04:36:22.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d52f9e700 1 -- 172.21.15.142:0/2058575093 wait complete. 2024-03-29T04:36:22.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d52f9e700 1 Processor -- start 2024-03-29T04:36:22.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d52f9e700 1 -- start start 2024-03-29T04:36:22.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d52f9e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d4c0ff790 0x7f3d4c1959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:22.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d52f9e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c195f00 con 0x7f3d4c0ff790 2024-03-29T04:36:22.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d50d3a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d4c0ff790 0x7f3d4c1959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:22.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d50d3a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d4c0ff790 0x7f3d4c1959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37064/0 (socket says 172.21.15.142:37064) 2024-03-29T04:36:22.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.259+0000 7f3d50d3a700 1 -- 172.21.15.142:0/1831550738 learned_addr learned my addr 172.21.15.142:0/1831550738 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:22.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.260+0000 7f3d50d3a700 1 -- 172.21.15.142:0/1831550738 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d400095f0 con 0x7f3d4c0ff790 2024-03-29T04:36:22.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.260+0000 7f3d50d3a700 1 --2- 172.21.15.142:0/1831550738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d4c0ff790 0x7f3d4c1959c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d40005b40 tx=0x7f3d40004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:22.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.260+0000 7f3d49ffb700 1 -- 172.21.15.142:0/1831550738 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d4001d070 con 0x7f3d4c0ff790 2024-03-29T04:36:22.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.260+0000 7f3d49ffb700 1 -- 172.21.15.142:0/1831550738 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3d4000be10 con 0x7f3d4c0ff790 2024-03-29T04:36:22.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.260+0000 7f3d49ffb700 1 -- 172.21.15.142:0/1831550738 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d4000f460 con 0x7f3d4c0ff790 2024-03-29T04:36:22.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.260+0000 7f3d52f9e700 1 -- 172.21.15.142:0/1831550738 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4c196100 con 0x7f3d4c0ff790 2024-03-29T04:36:22.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.260+0000 7f3d52f9e700 1 -- 172.21.15.142:0/1831550738 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c1965a0 con 0x7f3d4c0ff790 2024-03-29T04:36:22.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.261+0000 7f3d49ffb700 1 -- 172.21.15.142:0/1831550738 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f3d400227e0 con 0x7f3d4c0ff790 2024-03-29T04:36:22.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.261+0000 7f3d52f9e700 1 -- 172.21.15.142:0/1831550738 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4c1055d0 con 0x7f3d4c0ff790 2024-03-29T04:36:22.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.261+0000 7f3d49ffb700 1 --2- 172.21.15.142:0/1831550738 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f3d3c038410 0x7f3d3c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:22.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.261+0000 7f3d49ffb700 1 -- 172.21.15.142:0/1831550738 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3d4004c360 con 0x7f3d4c0ff790 2024-03-29T04:36:22.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.261+0000 7f3d4bfff700 1 --2- 172.21.15.142:0/1831550738 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f3d3c038410 0x7f3d3c03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:22.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.262+0000 7f3d4bfff700 1 --2- 172.21.15.142:0/1831550738 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f3d3c038410 0x7f3d3c03a8d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3d38006fd0 tx=0x7f3d38006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:22.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.266+0000 7f3d49ffb700 1 -- 172.21.15.142:0/1831550738 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3d4002ba00 con 0x7f3d4c0ff790 2024-03-29T04:36:22.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.483+0000 7f3d52f9e700 1 -- 172.21.15.142:0/1831550738 --> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d4c0fc630 con 0x7f3d3c038410 2024-03-29T04:36:22.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.486+0000 7f3d49ffb700 1 -- 172.21.15.142:0/1831550738 <== mgr.14162 v2:172.21.15.142:6800/779416791 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3d4c0fc630 con 0x7f3d3c038410 2024-03-29T04:36:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.488+0000 7f3d52f9e700 1 -- 172.21.15.142:0/1831550738 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f3d3c038410 msgr2=0x7f3d3c03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.488+0000 7f3d52f9e700 1 --2- 172.21.15.142:0/1831550738 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f3d3c038410 0x7f3d3c03a8d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3d38006fd0 tx=0x7f3d38006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.488+0000 7f3d52f9e700 1 -- 172.21.15.142:0/1831550738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d4c0ff790 msgr2=0x7f3d4c1959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.488+0000 7f3d52f9e700 1 --2- 172.21.15.142:0/1831550738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d4c0ff790 0x7f3d4c1959c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d40005b40 tx=0x7f3d40004740 comp rx=0 tx=0).stop 2024-03-29T04:36:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.488+0000 7f3d52f9e700 1 -- 172.21.15.142:0/1831550738 shutdown_connections 2024-03-29T04:36:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.488+0000 7f3d52f9e700 1 --2- 172.21.15.142:0/1831550738 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f3d3c038410 0x7f3d3c03a8d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.488+0000 7f3d52f9e700 1 --2- 172.21.15.142:0/1831550738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d4c0ff790 0x7f3d4c1959c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:22.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.488+0000 7f3d52f9e700 1 -- 172.21.15.142:0/1831550738 >> 172.21.15.142:0/1831550738 conn(0x7f3d4c0fad30 msgr2=0x7f3d4c0fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:22.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.489+0000 7f3d52f9e700 1 -- 172.21.15.142:0/1831550738 shutdown_connections 2024-03-29T04:36:22.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:22.489+0000 7f3d52f9e700 1 -- 172.21.15.142:0/1831550738 wait complete. 2024-03-29T04:36:23.233 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi152 2024-03-29T04:36:23.233 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:36:23.233 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/etc/ceph/ceph.conf 2024-03-29T04:36:23.251 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:36:23.251 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:36:23.309 INFO:tasks.cephadm:Adding host smithi152 to orchestrator... 2024-03-29T04:36:23.309 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph orch host add smithi152 2024-03-29T04:36:23.572 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:36:23.756 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:23 smithi142 ceph-mon[29027]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:36:23.757 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:23 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:24.697 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:24 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:24.697 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:24 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:24.697 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:24 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:24.697 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:24 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:24.697 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:24 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:36:24.697 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:24 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-29T04:36:24.697 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:24 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:36:24.698 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:24 smithi142 ceph-mon[29027]: Deploying daemon crash.smithi142 on smithi142 2024-03-29T04:36:25.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.186+0000 7f916309a700 1 -- 172.21.15.142:0/1712706121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f915c0ff7a0 msgr2=0x7f915c0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:25.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.186+0000 7f916309a700 1 --2- 172.21.15.142:0/1712706121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f915c0ff7a0 0x7f915c0ffbc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9150009910 tx=0x7f9150009c20 comp rx=0 tx=0).stop 2024-03-29T04:36:25.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.187+0000 7f916309a700 1 -- 172.21.15.142:0/1712706121 shutdown_connections 2024-03-29T04:36:25.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.187+0000 7f916309a700 1 --2- 172.21.15.142:0/1712706121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f915c0ff7a0 0x7f915c0ffbc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:25.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.187+0000 7f916309a700 1 -- 172.21.15.142:0/1712706121 >> 172.21.15.142:0/1712706121 conn(0x7f915c0fad20 msgr2=0x7f915c0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:25.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.187+0000 7f916309a700 1 -- 172.21.15.142:0/1712706121 shutdown_connections 2024-03-29T04:36:25.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.187+0000 7f916309a700 1 -- 172.21.15.142:0/1712706121 wait complete. 2024-03-29T04:36:25.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.187+0000 7f916309a700 1 Processor -- start 2024-03-29T04:36:25.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.187+0000 7f916309a700 1 -- start start 2024-03-29T04:36:25.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.187+0000 7f916309a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f915c0ff7a0 0x7f915c0693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:25.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.187+0000 7f916309a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c1986e0 con 0x7f915c0ff7a0 2024-03-29T04:36:25.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.187+0000 7f9160e36700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f915c0ff7a0 0x7f915c0693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:25.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.187+0000 7f9160e36700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f915c0ff7a0 0x7f915c0693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37080/0 (socket says 172.21.15.142:37080) 2024-03-29T04:36:25.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.187+0000 7f9160e36700 1 -- 172.21.15.142:0/1729016510 learned_addr learned my addr 172.21.15.142:0/1729016510 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:25.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.188+0000 7f9160e36700 1 -- 172.21.15.142:0/1729016510 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91500095f0 con 0x7f915c0ff7a0 2024-03-29T04:36:25.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.188+0000 7f9160e36700 1 --2- 172.21.15.142:0/1729016510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f915c0ff7a0 0x7f915c0693f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9150005950 tx=0x7f9150009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:25.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.188+0000 7f9159ffb700 1 -- 172.21.15.142:0/1729016510 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f915001d070 con 0x7f915c0ff7a0 2024-03-29T04:36:25.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.188+0000 7f9159ffb700 1 -- 172.21.15.142:0/1729016510 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f915000be10 con 0x7f915c0ff7a0 2024-03-29T04:36:25.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.188+0000 7f9159ffb700 1 -- 172.21.15.142:0/1729016510 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f915000f460 con 0x7f915c0ff7a0 2024-03-29T04:36:25.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.188+0000 7f916309a700 1 -- 172.21.15.142:0/1729016510 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915c198820 con 0x7f915c0ff7a0 2024-03-29T04:36:25.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.188+0000 7f916309a700 1 -- 172.21.15.142:0/1729016510 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915c198cc0 con 0x7f915c0ff7a0 2024-03-29T04:36:25.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.189+0000 7f9159ffb700 1 -- 172.21.15.142:0/1729016510 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f91500227c0 con 0x7f915c0ff7a0 2024-03-29T04:36:25.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.189+0000 7f9159ffb700 1 --2- 172.21.15.142:0/1729016510 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f914c038410 0x7f914c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:25.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.189+0000 7f916309a700 1 -- 172.21.15.142:0/1729016510 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915c18e3c0 con 0x7f915c0ff7a0 2024-03-29T04:36:25.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.189+0000 7f9159ffb700 1 -- 172.21.15.142:0/1729016510 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f915004c4a0 con 0x7f915c0ff7a0 2024-03-29T04:36:25.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.189+0000 7f915bfff700 1 --2- 172.21.15.142:0/1729016510 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f914c038410 0x7f914c03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:25.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.190+0000 7f915bfff700 1 --2- 172.21.15.142:0/1729016510 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f914c038410 0x7f914c03a8d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9148006fd0 tx=0x7f9148006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:25.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.195+0000 7f9159ffb700 1 -- 172.21.15.142:0/1729016510 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f915002b3b0 con 0x7f915c0ff7a0 2024-03-29T04:36:25.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:25.390+0000 7f916309a700 1 -- 172.21.15.142:0/1729016510 --> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi152", "target": ["mon-mgr", ""]}) v1 -- 0x7f915c0611d0 con 0x7f914c038410 2024-03-29T04:36:25.858 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:25 smithi142 ceph-mon[29027]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi152", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:36:27.216 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:27 smithi142 ceph-mon[29027]: Deploying cephadm binary to smithi152 2024-03-29T04:36:27.216 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:27 smithi142 ceph-mon[29027]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:27.216 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:27 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:27.948 INFO:teuthology.orchestra.run.smithi142.stdout:Added host 'smithi152' with addr '172.21.15.152' 2024-03-29T04:36:27.949 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:27.959+0000 7f9159ffb700 1 -- 172.21.15.142:0/1729016510 <== mgr.14162 v2:172.21.15.142:6800/779416791 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f915c0611d0 con 0x7f914c038410 2024-03-29T04:36:27.949 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:27.961+0000 7f916309a700 1 -- 172.21.15.142:0/1729016510 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f914c038410 msgr2=0x7f914c03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:27.949 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:27.961+0000 7f916309a700 1 --2- 172.21.15.142:0/1729016510 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f914c038410 0x7f914c03a8d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9148006fd0 tx=0x7f9148006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:27.949 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:27.961+0000 7f916309a700 1 -- 172.21.15.142:0/1729016510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f915c0ff7a0 msgr2=0x7f915c0693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:27.950 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:27.961+0000 7f916309a700 1 --2- 172.21.15.142:0/1729016510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f915c0ff7a0 0x7f915c0693f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9150005950 tx=0x7f9150009e60 comp rx=0 tx=0).stop 2024-03-29T04:36:27.950 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:27.962+0000 7f916309a700 1 -- 172.21.15.142:0/1729016510 shutdown_connections 2024-03-29T04:36:27.950 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:27.962+0000 7f916309a700 1 --2- 172.21.15.142:0/1729016510 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f914c038410 0x7f914c03a8d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:27.950 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:27.962+0000 7f916309a700 1 --2- 172.21.15.142:0/1729016510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f915c0ff7a0 0x7f915c0693f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:27.950 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:27.962+0000 7f916309a700 1 -- 172.21.15.142:0/1729016510 >> 172.21.15.142:0/1729016510 conn(0x7f915c0fad20 msgr2=0x7f915c0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:27.950 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:27.962+0000 7f916309a700 1 -- 172.21.15.142:0/1729016510 shutdown_connections 2024-03-29T04:36:27.950 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:27.962+0000 7f916309a700 1 -- 172.21.15.142:0/1729016510 wait complete. 2024-03-29T04:36:28.580 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:28 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:28.580 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:28 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:28.580 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:28 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:28.580 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:28 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:28.580 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:28 smithi142 ceph-mon[29027]: Deploying daemon node-exporter.smithi142 on smithi142 2024-03-29T04:36:28.580 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:28 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:28.602 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-29T04:36:28.833 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:36:29.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:29 smithi142 ceph-mon[29027]: Added host smithi152 2024-03-29T04:36:29.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:29 smithi142 ceph-mon[29027]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:31.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.122+0000 7ff0b795c700 1 -- 172.21.15.142:0/961886145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b01008a0 msgr2=0x7ff0b0100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:31.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.122+0000 7ff0b795c700 1 --2- 172.21.15.142:0/961886145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b01008a0 0x7ff0b0100cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009910 tx=0x7ff0a0009c20 comp rx=0 tx=0).stop 2024-03-29T04:36:31.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.122+0000 7ff0b795c700 1 -- 172.21.15.142:0/961886145 shutdown_connections 2024-03-29T04:36:31.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.122+0000 7ff0b795c700 1 --2- 172.21.15.142:0/961886145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b01008a0 0x7ff0b0100cc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:31.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.122+0000 7ff0b795c700 1 -- 172.21.15.142:0/961886145 >> 172.21.15.142:0/961886145 conn(0x7ff0b00fbe60 msgr2=0x7ff0b00fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:31.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.122+0000 7ff0b795c700 1 -- 172.21.15.142:0/961886145 shutdown_connections 2024-03-29T04:36:31.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.122+0000 7ff0b795c700 1 -- 172.21.15.142:0/961886145 wait complete. 2024-03-29T04:36:31.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.123+0000 7ff0b795c700 1 Processor -- start 2024-03-29T04:36:31.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.123+0000 7ff0b795c700 1 -- start start 2024-03-29T04:36:31.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.123+0000 7ff0b795c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b01008a0 0x7ff0b01959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:31.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.123+0000 7ff0b795c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b0195f00 con 0x7ff0b01008a0 2024-03-29T04:36:31.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.123+0000 7ff0b56f8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b01008a0 0x7ff0b01959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:31.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.123+0000 7ff0b56f8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b01008a0 0x7ff0b01959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39952/0 (socket says 172.21.15.142:39952) 2024-03-29T04:36:31.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.123+0000 7ff0b56f8700 1 -- 172.21.15.142:0/4191051063 learned_addr learned my addr 172.21.15.142:0/4191051063 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:31.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.123+0000 7ff0b56f8700 1 -- 172.21.15.142:0/4191051063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a00095f0 con 0x7ff0b01008a0 2024-03-29T04:36:31.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.124+0000 7ff0b56f8700 1 --2- 172.21.15.142:0/4191051063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b01008a0 0x7ff0b01959c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0005b40 tx=0x7ff0a0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:31.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.124+0000 7ff0a67fc700 1 -- 172.21.15.142:0/4191051063 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0a001d070 con 0x7ff0b01008a0 2024-03-29T04:36:31.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.124+0000 7ff0b795c700 1 -- 172.21.15.142:0/4191051063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b0196100 con 0x7ff0b01008a0 2024-03-29T04:36:31.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.124+0000 7ff0a67fc700 1 -- 172.21.15.142:0/4191051063 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff0a000be10 con 0x7ff0b01008a0 2024-03-29T04:36:31.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.124+0000 7ff0a67fc700 1 -- 172.21.15.142:0/4191051063 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0a000f460 con 0x7ff0b01008a0 2024-03-29T04:36:31.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.124+0000 7ff0b795c700 1 -- 172.21.15.142:0/4191051063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b01965a0 con 0x7ff0b01008a0 2024-03-29T04:36:31.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.125+0000 7ff0a67fc700 1 -- 172.21.15.142:0/4191051063 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7ff0a00227e0 con 0x7ff0b01008a0 2024-03-29T04:36:31.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.125+0000 7ff0b795c700 1 -- 172.21.15.142:0/4191051063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b018f4c0 con 0x7ff0b01008a0 2024-03-29T04:36:31.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.125+0000 7ff0a67fc700 1 --2- 172.21.15.142:0/4191051063 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7ff09c0383c0 0x7ff09c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:31.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.125+0000 7ff0a67fc700 1 -- 172.21.15.142:0/4191051063 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff0a004c4a0 con 0x7ff0b01008a0 2024-03-29T04:36:31.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.125+0000 7ff0b4ef7700 1 --2- 172.21.15.142:0/4191051063 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7ff09c0383c0 0x7ff09c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:31.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.125+0000 7ff0b4ef7700 1 --2- 172.21.15.142:0/4191051063 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7ff09c0383c0 0x7ff09c03a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac006fd0 tx=0x7ff0ac006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:31.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.130+0000 7ff0a67fc700 1 -- 172.21.15.142:0/4191051063 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff0a002bba0 con 0x7ff0b01008a0 2024-03-29T04:36:31.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.326+0000 7ff0b795c700 1 -- 172.21.15.142:0/4191051063 --> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff0b00611d0 con 0x7ff09c0383c0 2024-03-29T04:36:31.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.328+0000 7ff0a67fc700 1 -- 172.21.15.142:0/4191051063 <== mgr.14162 v2:172.21.15.142:6800/779416791 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7ff0b00611d0 con 0x7ff09c0383c0 2024-03-29T04:36:31.315 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:36:31.315 INFO:teuthology.orchestra.run.smithi142.stdout:[{"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}, {"addr": "172.21.15.152", "hostname": "smithi152", "labels": [], "status": ""}] 2024-03-29T04:36:31.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.329+0000 7ff0b795c700 1 -- 172.21.15.142:0/4191051063 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7ff09c0383c0 msgr2=0x7ff09c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:31.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.329+0000 7ff0b795c700 1 --2- 172.21.15.142:0/4191051063 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7ff09c0383c0 0x7ff09c03a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac006fd0 tx=0x7ff0ac006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:31.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.330+0000 7ff0b795c700 1 -- 172.21.15.142:0/4191051063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b01008a0 msgr2=0x7ff0b01959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:31.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.330+0000 7ff0b795c700 1 --2- 172.21.15.142:0/4191051063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b01008a0 0x7ff0b01959c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0005b40 tx=0x7ff0a0004740 comp rx=0 tx=0).stop 2024-03-29T04:36:31.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.330+0000 7ff0b795c700 1 -- 172.21.15.142:0/4191051063 shutdown_connections 2024-03-29T04:36:31.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.330+0000 7ff0b795c700 1 --2- 172.21.15.142:0/4191051063 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7ff09c0383c0 0x7ff09c03a880 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:31.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.330+0000 7ff0b795c700 1 --2- 172.21.15.142:0/4191051063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0b01008a0 0x7ff0b01959c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:31.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.330+0000 7ff0b795c700 1 -- 172.21.15.142:0/4191051063 >> 172.21.15.142:0/4191051063 conn(0x7ff0b00fbe60 msgr2=0x7ff0b00fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:31.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.330+0000 7ff0b795c700 1 -- 172.21.15.142:0/4191051063 shutdown_connections 2024-03-29T04:36:31.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:31.330+0000 7ff0b795c700 1 -- 172.21.15.142:0/4191051063 wait complete. 2024-03-29T04:36:31.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:31 smithi142 ceph-mon[29027]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:31.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:31 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:32.098 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-29T04:36:32.099 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-29T04:36:32.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:32 smithi142 ceph-mon[29027]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-29T04:36:32.632 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:36:33.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:33 smithi142 ceph-mon[29027]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:33.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:33 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:33.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:33 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:33.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:33 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:33.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:33 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:34.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.169+0000 7f03b28b9700 1 -- 172.21.15.142:0/3180051224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03ac0ff7a0 msgr2=0x7f03ac0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:34.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.169+0000 7f03b28b9700 1 --2- 172.21.15.142:0/3180051224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03ac0ff7a0 0x7f03ac0ffbc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f039c009910 tx=0x7f039c009c20 comp rx=0 tx=0).stop 2024-03-29T04:36:34.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.169+0000 7f03b28b9700 1 -- 172.21.15.142:0/3180051224 shutdown_connections 2024-03-29T04:36:34.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.169+0000 7f03b28b9700 1 --2- 172.21.15.142:0/3180051224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03ac0ff7a0 0x7f03ac0ffbc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:34.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.169+0000 7f03b28b9700 1 -- 172.21.15.142:0/3180051224 >> 172.21.15.142:0/3180051224 conn(0x7f03ac0fad20 msgr2=0x7f03ac0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:34.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.169+0000 7f03b28b9700 1 -- 172.21.15.142:0/3180051224 shutdown_connections 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.169+0000 7f03b28b9700 1 -- 172.21.15.142:0/3180051224 wait complete. 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.169+0000 7f03b28b9700 1 Processor -- start 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.169+0000 7f03b28b9700 1 -- start start 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.170+0000 7f03b28b9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03ac0ff7a0 0x7f03ac0693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.170+0000 7f03b28b9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac198700 con 0x7f03ac0ff7a0 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.170+0000 7f03abfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03ac0ff7a0 0x7f03ac0693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.170+0000 7f03abfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03ac0ff7a0 0x7f03ac0693d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39964/0 (socket says 172.21.15.142:39964) 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.170+0000 7f03abfff700 1 -- 172.21.15.142:0/3093265125 learned_addr learned my addr 172.21.15.142:0/3093265125 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.170+0000 7f03abfff700 1 -- 172.21.15.142:0/3093265125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0095f0 con 0x7f03ac0ff7a0 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.170+0000 7f03abfff700 1 --2- 172.21.15.142:0/3093265125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03ac0ff7a0 0x7f03ac0693d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f039c005950 tx=0x7f039c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.170+0000 7f03a97fa700 1 -- 172.21.15.142:0/3093265125 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f039c01d070 con 0x7f03ac0ff7a0 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.170+0000 7f03a97fa700 1 -- 172.21.15.142:0/3093265125 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f039c00be10 con 0x7f03ac0ff7a0 2024-03-29T04:36:34.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.170+0000 7f03b28b9700 1 -- 172.21.15.142:0/3093265125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac198840 con 0x7f03ac0ff7a0 2024-03-29T04:36:34.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.170+0000 7f03a97fa700 1 -- 172.21.15.142:0/3093265125 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f039c00f460 con 0x7f03ac0ff7a0 2024-03-29T04:36:34.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.170+0000 7f03b28b9700 1 -- 172.21.15.142:0/3093265125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac198ce0 con 0x7f03ac0ff7a0 2024-03-29T04:36:34.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.171+0000 7f03a97fa700 1 -- 172.21.15.142:0/3093265125 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f039c00f690 con 0x7f03ac0ff7a0 2024-03-29T04:36:34.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.171+0000 7f03a97fa700 1 --2- 172.21.15.142:0/3093265125 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f03940383c0 0x7f039403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:34.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.171+0000 7f03b28b9700 1 -- 172.21.15.142:0/3093265125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac18e3c0 con 0x7f03ac0ff7a0 2024-03-29T04:36:34.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.171+0000 7f03a97fa700 1 -- 172.21.15.142:0/3093265125 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f039c04c6c0 con 0x7f03ac0ff7a0 2024-03-29T04:36:34.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.171+0000 7f03ab7fe700 1 --2- 172.21.15.142:0/3093265125 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f03940383c0 0x7f039403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:34.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.172+0000 7f03ab7fe700 1 --2- 172.21.15.142:0/3093265125 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f03940383c0 0x7f039403a880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f03a0006fd0 tx=0x7f03a0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:34.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.176+0000 7f03a97fa700 1 -- 172.21.15.142:0/3093265125 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f039c027070 con 0x7f03ac0ff7a0 2024-03-29T04:36:34.394 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:34.406+0000 7f03b28b9700 1 -- 172.21.15.142:0/3093265125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f03ac02cc70 con 0x7f03ac0ff7a0 2024-03-29T04:36:34.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:34 smithi142 ceph-mon[29027]: Deploying daemon alertmanager.smithi142 on smithi142 2024-03-29T04:36:35.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:35.380+0000 7f03a97fa700 1 -- 172.21.15.142:0/3093265125 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f039c02c440 con 0x7f03ac0ff7a0 2024-03-29T04:36:35.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:35.382+0000 7f03b28b9700 1 -- 172.21.15.142:0/3093265125 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f03940383c0 msgr2=0x7f039403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:35.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:35.382+0000 7f03b28b9700 1 --2- 172.21.15.142:0/3093265125 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f03940383c0 0x7f039403a880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f03a0006fd0 tx=0x7f03a0006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:35.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:35.382+0000 7f03b28b9700 1 -- 172.21.15.142:0/3093265125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03ac0ff7a0 msgr2=0x7f03ac0693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:35.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:35.382+0000 7f03b28b9700 1 --2- 172.21.15.142:0/3093265125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03ac0ff7a0 0x7f03ac0693d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f039c005950 tx=0x7f039c009e60 comp rx=0 tx=0).stop 2024-03-29T04:36:35.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:35.382+0000 7f03b28b9700 1 -- 172.21.15.142:0/3093265125 shutdown_connections 2024-03-29T04:36:35.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:35.382+0000 7f03b28b9700 1 --2- 172.21.15.142:0/3093265125 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f03940383c0 0x7f039403a880 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:35.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:35.382+0000 7f03b28b9700 1 --2- 172.21.15.142:0/3093265125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03ac0ff7a0 0x7f03ac0693d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:35.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:35.382+0000 7f03b28b9700 1 -- 172.21.15.142:0/3093265125 >> 172.21.15.142:0/3093265125 conn(0x7f03ac0fad20 msgr2=0x7f03ac0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:35.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:35.382+0000 7f03b28b9700 1 -- 172.21.15.142:0/3093265125 shutdown_connections 2024-03-29T04:36:35.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:36:35.382+0000 7f03b28b9700 1 -- 172.21.15.142:0/3093265125 wait complete. 2024-03-29T04:36:35.371 INFO:teuthology.orchestra.run.smithi142.stderr:adjusted tunables profile to default 2024-03-29T04:36:35.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:35 smithi142 ceph-mon[29027]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:35.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:35 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3093265125' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-29T04:36:36.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:36 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3093265125' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-29T04:36:36.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:36 smithi142 ceph-mon[29027]: osdmap e4: 0 total, 0 up, 0 in 2024-03-29T04:36:36.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:36 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:36.645 INFO:tasks.cephadm:Adding mon.smithi142 on smithi142 2024-03-29T04:36:36.645 INFO:tasks.cephadm:Adding mon.smithi152 on smithi152 2024-03-29T04:36:36.645 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph orch apply mon '2;smithi142:172.21.15.142=smithi142;smithi152:172.21.15.152=smithi152' 2024-03-29T04:36:36.848 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:36.894 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:37.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:37 smithi142 ceph-mon[29027]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:38.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.399+0000 7f7693010700 1 -- 172.21.15.152:0/2751928129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c100d30 msgr2=0x7f768c101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:38.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.399+0000 7f7693010700 1 --2- 172.21.15.152:0/2751928129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c100d30 0x7f768c101150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7680009910 tx=0x7f7680009c20 comp rx=0 tx=0).stop 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.400+0000 7f7693010700 1 -- 172.21.15.152:0/2751928129 shutdown_connections 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.400+0000 7f7693010700 1 --2- 172.21.15.152:0/2751928129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c100d30 0x7f768c101150 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.400+0000 7f7693010700 1 -- 172.21.15.152:0/2751928129 >> 172.21.15.152:0/2751928129 conn(0x7f768c0fa4a0 msgr2=0x7f768c0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.400+0000 7f7693010700 1 -- 172.21.15.152:0/2751928129 shutdown_connections 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.400+0000 7f7693010700 1 -- 172.21.15.152:0/2751928129 wait complete. 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.400+0000 7f7693010700 1 Processor -- start 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.400+0000 7f7693010700 1 -- start start 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.400+0000 7f7693010700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c100d30 0x7f768c194880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.400+0000 7f7693010700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c194dc0 con 0x7f768c100d30 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.401+0000 7f7690dac700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c100d30 0x7f768c194880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.401+0000 7f7690dac700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c100d30 0x7f768c194880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:52322/0 (socket says 172.21.15.152:52322) 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.401+0000 7f7690dac700 1 -- 172.21.15.152:0/2226914844 learned_addr learned my addr 172.21.15.152:0/2226914844 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:36:38.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.401+0000 7f7690dac700 1 -- 172.21.15.152:0/2226914844 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76800095f0 con 0x7f768c100d30 2024-03-29T04:36:38.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.401+0000 7f7690dac700 1 --2- 172.21.15.152:0/2226914844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c100d30 0x7f768c194880 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7680005b40 tx=0x7f7680004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:38.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.401+0000 7f7689ffb700 1 -- 172.21.15.152:0/2226914844 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f768001f070 con 0x7f768c100d30 2024-03-29T04:36:38.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.401+0000 7f7689ffb700 1 -- 172.21.15.152:0/2226914844 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f768000be10 con 0x7f768c100d30 2024-03-29T04:36:38.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.401+0000 7f7693010700 1 -- 172.21.15.152:0/2226914844 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768c194fc0 con 0x7f768c100d30 2024-03-29T04:36:38.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.401+0000 7f7689ffb700 1 -- 172.21.15.152:0/2226914844 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7680024620 con 0x7f768c100d30 2024-03-29T04:36:38.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.402+0000 7f7693010700 1 -- 172.21.15.152:0/2226914844 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768c195460 con 0x7f768c100d30 2024-03-29T04:36:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.402+0000 7f7689ffb700 1 -- 172.21.15.152:0/2226914844 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f768000b960 con 0x7f768c100d30 2024-03-29T04:36:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.402+0000 7f7689ffb700 1 --2- 172.21.15.152:0/2226914844 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f767c0383c0 0x7f767c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.402+0000 7f7693010700 1 -- 172.21.15.152:0/2226914844 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768c1055e0 con 0x7f768c100d30 2024-03-29T04:36:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.402+0000 7f7689ffb700 1 -- 172.21.15.152:0/2226914844 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f768004e5b0 con 0x7f768c100d30 2024-03-29T04:36:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.403+0000 7f768bfff700 1 --2- 172.21.15.152:0/2226914844 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f767c0383c0 0x7f767c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:38.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.403+0000 7f768bfff700 1 --2- 172.21.15.152:0/2226914844 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f767c0383c0 0x7f767c03a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7678006fd0 tx=0x7f7678006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:38.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.408+0000 7f7689ffb700 1 -- 172.21.15.152:0/2226914844 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7680029020 con 0x7f768c100d30 2024-03-29T04:36:38.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.607+0000 7f7693010700 1 -- 172.21.15.152:0/2226914844 --> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi142:172.21.15.142=smithi142;smithi152:172.21.15.152=smithi152", "target": ["mon-mgr", ""]}) v1 -- 0x7f768c0fea10 con 0x7f767c0383c0 2024-03-29T04:36:38.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.614+0000 7f7689ffb700 1 -- 172.21.15.152:0/2226914844 <== mgr.14162 v2:172.21.15.142:6800/779416791 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f768c0fea10 con 0x7f767c0383c0 2024-03-29T04:36:38.602 INFO:teuthology.orchestra.run.smithi152.stdout:Scheduled mon update... 2024-03-29T04:36:38.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.616+0000 7f7693010700 1 -- 172.21.15.152:0/2226914844 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f767c0383c0 msgr2=0x7f767c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:38.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.616+0000 7f7693010700 1 --2- 172.21.15.152:0/2226914844 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f767c0383c0 0x7f767c03a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7678006fd0 tx=0x7f7678006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:38.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.616+0000 7f7693010700 1 -- 172.21.15.152:0/2226914844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c100d30 msgr2=0x7f768c194880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:38.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.616+0000 7f7693010700 1 --2- 172.21.15.152:0/2226914844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c100d30 0x7f768c194880 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7680005b40 tx=0x7f7680004740 comp rx=0 tx=0).stop 2024-03-29T04:36:38.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.616+0000 7f7693010700 1 -- 172.21.15.152:0/2226914844 shutdown_connections 2024-03-29T04:36:38.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.616+0000 7f7693010700 1 --2- 172.21.15.152:0/2226914844 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f767c0383c0 0x7f767c03a880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:38.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.616+0000 7f7693010700 1 --2- 172.21.15.152:0/2226914844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f768c100d30 0x7f768c194880 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:38.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.616+0000 7f7693010700 1 -- 172.21.15.152:0/2226914844 >> 172.21.15.152:0/2226914844 conn(0x7f768c0fa4a0 msgr2=0x7f768c0fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:38.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.616+0000 7f7693010700 1 -- 172.21.15.152:0/2226914844 shutdown_connections 2024-03-29T04:36:38.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:38.616+0000 7f7693010700 1 -- 172.21.15.152:0/2226914844 wait complete. 2024-03-29T04:36:39.259 DEBUG:teuthology.orchestra.run.smithi152:mon.smithi152> sudo journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi152.service 2024-03-29T04:36:39.262 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:36:39.262 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:36:39.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:39 smithi142 ceph-mon[29027]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:39.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:39 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:39.651 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:39.696 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:40.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:40 smithi142 ceph-mon[29027]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi142:172.21.15.142=smithi142;smithi152:172.21.15.152=smithi152", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:36:40.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:40 smithi142 ceph-mon[29027]: Saving service mon spec with placement smithi142:172.21.15.142=smithi142;smithi152:172.21.15.152=smithi152;count:2 2024-03-29T04:36:41.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.164+0000 7f6a98001700 1 -- 172.21.15.152:0/1341762379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900ff7a0 msgr2=0x7f6a900ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:41.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.164+0000 7f6a98001700 1 --2- 172.21.15.152:0/1341762379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900ff7a0 0x7f6a900ffbc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009910 tx=0x7f6a80009c20 comp rx=0 tx=0).stop 2024-03-29T04:36:41.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.164+0000 7f6a98001700 1 -- 172.21.15.152:0/1341762379 shutdown_connections 2024-03-29T04:36:41.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.164+0000 7f6a98001700 1 --2- 172.21.15.152:0/1341762379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900ff7a0 0x7f6a900ffbc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:41.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.164+0000 7f6a98001700 1 -- 172.21.15.152:0/1341762379 >> 172.21.15.152:0/1341762379 conn(0x7f6a900fad20 msgr2=0x7f6a900fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:41.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.164+0000 7f6a98001700 1 -- 172.21.15.152:0/1341762379 shutdown_connections 2024-03-29T04:36:41.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.164+0000 7f6a98001700 1 -- 172.21.15.152:0/1341762379 wait complete. 2024-03-29T04:36:41.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.165+0000 7f6a98001700 1 Processor -- start 2024-03-29T04:36:41.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.165+0000 7f6a98001700 1 -- start start 2024-03-29T04:36:41.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.165+0000 7f6a98001700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900ff7a0 0x7f6a900693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:41.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.165+0000 7f6a98001700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a90198700 con 0x7f6a900ff7a0 2024-03-29T04:36:41.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.165+0000 7f6a95d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900ff7a0 0x7f6a900693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:41.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.165+0000 7f6a95d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900ff7a0 0x7f6a900693d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:52344/0 (socket says 172.21.15.152:52344) 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.165+0000 7f6a95d9d700 1 -- 172.21.15.152:0/391397452 learned_addr learned my addr 172.21.15.152:0/391397452 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.166+0000 7f6a95d9d700 1 -- 172.21.15.152:0/391397452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a800095f0 con 0x7f6a900ff7a0 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.166+0000 7f6a95d9d700 1 --2- 172.21.15.152:0/391397452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900ff7a0 0x7f6a900693d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a80005b40 tx=0x7f6a80004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.166+0000 7f6a86ffd700 1 -- 172.21.15.152:0/391397452 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a8001d070 con 0x7f6a900ff7a0 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.166+0000 7f6a98001700 1 -- 172.21.15.152:0/391397452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a90198840 con 0x7f6a900ff7a0 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.166+0000 7f6a86ffd700 1 -- 172.21.15.152:0/391397452 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a8000be10 con 0x7f6a900ff7a0 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.166+0000 7f6a86ffd700 1 -- 172.21.15.152:0/391397452 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a8000f460 con 0x7f6a900ff7a0 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.166+0000 7f6a98001700 1 -- 172.21.15.152:0/391397452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a90198ce0 con 0x7f6a900ff7a0 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.167+0000 7f6a86ffd700 1 -- 172.21.15.152:0/391397452 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f6a8000f6d0 con 0x7f6a900ff7a0 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.167+0000 7f6a86ffd700 1 --2- 172.21.15.152:0/391397452 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f6a7c0383c0 0x7f6a7c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.167+0000 7f6a86ffd700 1 -- 172.21.15.152:0/391397452 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6a8004c480 con 0x7f6a900ff7a0 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.167+0000 7f6a98001700 1 -- 172.21.15.152:0/391397452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9018e3c0 con 0x7f6a900ff7a0 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.167+0000 7f6a9559c700 1 --2- 172.21.15.152:0/391397452 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f6a7c0383c0 0x7f6a7c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:41.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.168+0000 7f6a9559c700 1 --2- 172.21.15.152:0/391397452 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f6a7c0383c0 0x7f6a7c03a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c006fd0 tx=0x7f6a8c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:41.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.173+0000 7f6a86ffd700 1 -- 172.21.15.152:0/391397452 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6a80027070 con 0x7f6a900ff7a0 2024-03-29T04:36:41.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.435+0000 7f6a98001700 1 -- 172.21.15.152:0/391397452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6a9002cc70 con 0x7f6a900ff7a0 2024-03-29T04:36:41.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.435+0000 7f6a86ffd700 1 -- 172.21.15.152:0/391397452 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f6a8002c440 con 0x7f6a900ff7a0 2024-03-29T04:36:41.423 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:36:41.424 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:36:41.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.437+0000 7f6a98001700 1 -- 172.21.15.152:0/391397452 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f6a7c0383c0 msgr2=0x7f6a7c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:41.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.437+0000 7f6a98001700 1 --2- 172.21.15.152:0/391397452 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f6a7c0383c0 0x7f6a7c03a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c006fd0 tx=0x7f6a8c006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:41.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.437+0000 7f6a98001700 1 -- 172.21.15.152:0/391397452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900ff7a0 msgr2=0x7f6a900693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:41.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.437+0000 7f6a98001700 1 --2- 172.21.15.152:0/391397452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900ff7a0 0x7f6a900693d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a80005b40 tx=0x7f6a80004740 comp rx=0 tx=0).stop 2024-03-29T04:36:41.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.437+0000 7f6a98001700 1 -- 172.21.15.152:0/391397452 shutdown_connections 2024-03-29T04:36:41.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.437+0000 7f6a98001700 1 --2- 172.21.15.152:0/391397452 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f6a7c0383c0 0x7f6a7c03a880 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:41.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.437+0000 7f6a98001700 1 --2- 172.21.15.152:0/391397452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a900ff7a0 0x7f6a900693d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:41.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.437+0000 7f6a98001700 1 -- 172.21.15.152:0/391397452 >> 172.21.15.152:0/391397452 conn(0x7f6a900fad20 msgr2=0x7f6a900fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:41.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.437+0000 7f6a98001700 1 -- 172.21.15.152:0/391397452 shutdown_connections 2024-03-29T04:36:41.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:41.437+0000 7f6a98001700 1 -- 172.21.15.152:0/391397452 wait complete. 2024-03-29T04:36:41.426 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:36:41.550 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:41 smithi142 ceph-mon[29027]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:42.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:42 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/391397452' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:36:42.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:42 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:42.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:42 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:42.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:42 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:42.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:42 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:42.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:42 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:42.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:42 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:42.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:42 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:42.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:42 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:42.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:42 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-29T04:36:42.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:42 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:42.962 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:36:42.962 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:36:43.166 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:43.211 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:43.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:43 smithi142 ceph-mon[29027]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:43.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:43 smithi142 ceph-mon[29027]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-29T04:36:43.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:43 smithi142 ceph-mon[29027]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-29T04:36:43.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:43 smithi142 ceph-mon[29027]: Deploying daemon grafana.smithi142 on smithi142 2024-03-29T04:36:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.645+0000 7f85c5d76700 1 -- 172.21.15.152:0/2668475173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c0101320 msgr2=0x7f85c0103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.645+0000 7f85c5d76700 1 --2- 172.21.15.152:0/2668475173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c0101320 0x7f85c0103710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009910 tx=0x7f85b0009c20 comp rx=0 tx=0).stop 2024-03-29T04:36:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.645+0000 7f85c5d76700 1 -- 172.21.15.152:0/2668475173 shutdown_connections 2024-03-29T04:36:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.645+0000 7f85c5d76700 1 --2- 172.21.15.152:0/2668475173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c0101320 0x7f85c0103710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:44.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.645+0000 7f85c5d76700 1 -- 172.21.15.152:0/2668475173 >> 172.21.15.152:0/2668475173 conn(0x7f85c00fad30 msgr2=0x7f85c00fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.645+0000 7f85c5d76700 1 -- 172.21.15.152:0/2668475173 shutdown_connections 2024-03-29T04:36:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.645+0000 7f85c5d76700 1 -- 172.21.15.152:0/2668475173 wait complete. 2024-03-29T04:36:44.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.645+0000 7f85c5d76700 1 Processor -- start 2024-03-29T04:36:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.646+0000 7f85c5d76700 1 -- start start 2024-03-29T04:36:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.646+0000 7f85c5d76700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c0101320 0x7f85c0195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.646+0000 7f85c5d76700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c0195e80 con 0x7f85c0101320 2024-03-29T04:36:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.646+0000 7f85bf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c0101320 0x7f85c0195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.646+0000 7f85bf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c0101320 0x7f85c0195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:52354/0 (socket says 172.21.15.152:52354) 2024-03-29T04:36:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.646+0000 7f85bf7fe700 1 -- 172.21.15.152:0/2809034917 learned_addr learned my addr 172.21.15.152:0/2809034917 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:36:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.646+0000 7f85bf7fe700 1 -- 172.21.15.152:0/2809034917 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b00095f0 con 0x7f85c0101320 2024-03-29T04:36:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.646+0000 7f85bf7fe700 1 --2- 172.21.15.152:0/2809034917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c0101320 0x7f85c0195940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f85b0005b40 tx=0x7f85b0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.646+0000 7f85bcff9700 1 -- 172.21.15.152:0/2809034917 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85b001d070 con 0x7f85c0101320 2024-03-29T04:36:44.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.647+0000 7f85bcff9700 1 -- 172.21.15.152:0/2809034917 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f85b000be10 con 0x7f85c0101320 2024-03-29T04:36:44.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.647+0000 7f85c5d76700 1 -- 172.21.15.152:0/2809034917 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c0196080 con 0x7f85c0101320 2024-03-29T04:36:44.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.647+0000 7f85bcff9700 1 -- 172.21.15.152:0/2809034917 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85b000f460 con 0x7f85c0101320 2024-03-29T04:36:44.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.647+0000 7f85c5d76700 1 -- 172.21.15.152:0/2809034917 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c0196520 con 0x7f85c0101320 2024-03-29T04:36:44.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.647+0000 7f85bcff9700 1 -- 172.21.15.152:0/2809034917 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f85b000f5c0 con 0x7f85c0101320 2024-03-29T04:36:44.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.647+0000 7f85c5d76700 1 -- 172.21.15.152:0/2809034917 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c018f4b0 con 0x7f85c0101320 2024-03-29T04:36:44.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.648+0000 7f85bcff9700 1 --2- 172.21.15.152:0/2809034917 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f85a8038410 0x7f85a803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:44.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.648+0000 7f85bcff9700 1 -- 172.21.15.152:0/2809034917 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f85b004c550 con 0x7f85c0101320 2024-03-29T04:36:44.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.648+0000 7f85beffd700 1 --2- 172.21.15.152:0/2809034917 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f85a8038410 0x7f85a803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:44.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.649+0000 7f85beffd700 1 --2- 172.21.15.152:0/2809034917 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f85a8038410 0x7f85a803a8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f85b4006fd0 tx=0x7f85b4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:44.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.653+0000 7f85bcff9700 1 -- 172.21.15.152:0/2809034917 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f85b002a360 con 0x7f85c0101320 2024-03-29T04:36:44.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.916+0000 7f85c5d76700 1 -- 172.21.15.152:0/2809034917 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f85c002cc70 con 0x7f85c0101320 2024-03-29T04:36:44.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.916+0000 7f85bcff9700 1 -- 172.21.15.152:0/2809034917 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f85b0027030 con 0x7f85c0101320 2024-03-29T04:36:44.905 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:36:44.905 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:36:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.918+0000 7f85c5d76700 1 -- 172.21.15.152:0/2809034917 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f85a8038410 msgr2=0x7f85a803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.918+0000 7f85c5d76700 1 --2- 172.21.15.152:0/2809034917 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f85a8038410 0x7f85a803a8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f85b4006fd0 tx=0x7f85b4006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.918+0000 7f85c5d76700 1 -- 172.21.15.152:0/2809034917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c0101320 msgr2=0x7f85c0195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.918+0000 7f85c5d76700 1 --2- 172.21.15.152:0/2809034917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c0101320 0x7f85c0195940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f85b0005b40 tx=0x7f85b0004740 comp rx=0 tx=0).stop 2024-03-29T04:36:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.918+0000 7f85c5d76700 1 -- 172.21.15.152:0/2809034917 shutdown_connections 2024-03-29T04:36:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.918+0000 7f85c5d76700 1 --2- 172.21.15.152:0/2809034917 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f85a8038410 0x7f85a803a8d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.918+0000 7f85c5d76700 1 --2- 172.21.15.152:0/2809034917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c0101320 0x7f85c0195940 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.918+0000 7f85c5d76700 1 -- 172.21.15.152:0/2809034917 >> 172.21.15.152:0/2809034917 conn(0x7f85c00fad30 msgr2=0x7f85c00fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.919+0000 7f85c5d76700 1 -- 172.21.15.152:0/2809034917 shutdown_connections 2024-03-29T04:36:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:44.919+0000 7f85c5d76700 1 -- 172.21.15.152:0/2809034917 wait complete. 2024-03-29T04:36:44.908 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:36:45.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:45 smithi142 ceph-mon[29027]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:45.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:45 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/2809034917' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:36:46.533 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:36:46.533 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:36:46.748 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:46.800 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:47.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:47 smithi142 ceph-mon[29027]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:47.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:47 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:48.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.193+0000 7fb96f53e700 1 -- 172.21.15.152:0/2556533807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9681008b0 msgr2=0x7fb968100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:48.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.193+0000 7fb96f53e700 1 --2- 172.21.15.152:0/2556533807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9681008b0 0x7fb968100cd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb96000b0a0 tx=0x7fb96000b3b0 comp rx=0 tx=0).stop 2024-03-29T04:36:48.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.193+0000 7fb96f53e700 1 -- 172.21.15.152:0/2556533807 shutdown_connections 2024-03-29T04:36:48.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.193+0000 7fb96f53e700 1 --2- 172.21.15.152:0/2556533807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9681008b0 0x7fb968100cd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.193+0000 7fb96f53e700 1 -- 172.21.15.152:0/2556533807 >> 172.21.15.152:0/2556533807 conn(0x7fb9680fbe30 msgr2=0x7fb9680fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.193+0000 7fb96f53e700 1 -- 172.21.15.152:0/2556533807 shutdown_connections 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.193+0000 7fb96f53e700 1 -- 172.21.15.152:0/2556533807 wait complete. 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.194+0000 7fb96f53e700 1 Processor -- start 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.194+0000 7fb96f53e700 1 -- start start 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.194+0000 7fb96f53e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9681008b0 0x7fb9681959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.194+0000 7fb96f53e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb968195f00 con 0x7fb9681008b0 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.194+0000 7fb96d2da700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9681008b0 0x7fb9681959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.194+0000 7fb96d2da700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9681008b0 0x7fb9681959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:54838/0 (socket says 172.21.15.152:54838) 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.194+0000 7fb96d2da700 1 -- 172.21.15.152:0/1923903374 learned_addr learned my addr 172.21.15.152:0/1923903374 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.195+0000 7fb96d2da700 1 -- 172.21.15.152:0/1923903374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb960009d00 con 0x7fb9681008b0 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.195+0000 7fb96d2da700 1 --2- 172.21.15.152:0/1923903374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9681008b0 0x7fb9681959c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb96000b830 tx=0x7fb96000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:48.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.195+0000 7fb9667fc700 1 -- 172.21.15.152:0/1923903374 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb96000e040 con 0x7fb9681008b0 2024-03-29T04:36:48.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.195+0000 7fb9667fc700 1 -- 172.21.15.152:0/1923903374 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb960012af0 con 0x7fb9681008b0 2024-03-29T04:36:48.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.195+0000 7fb96f53e700 1 -- 172.21.15.152:0/1923903374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb968196100 con 0x7fb9681008b0 2024-03-29T04:36:48.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.195+0000 7fb9667fc700 1 -- 172.21.15.152:0/1923903374 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb96001d9f0 con 0x7fb9681008b0 2024-03-29T04:36:48.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.195+0000 7fb96f53e700 1 -- 172.21.15.152:0/1923903374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb968196520 con 0x7fb9681008b0 2024-03-29T04:36:48.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.196+0000 7fb96f53e700 1 -- 172.21.15.152:0/1923903374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96818f4d0 con 0x7fb9681008b0 2024-03-29T04:36:48.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.196+0000 7fb9667fc700 1 -- 172.21.15.152:0/1923903374 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fb960012c60 con 0x7fb9681008b0 2024-03-29T04:36:48.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.196+0000 7fb9667fc700 1 --2- 172.21.15.152:0/1923903374 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb9500383c0 0x7fb95003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:48.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.196+0000 7fb9667fc700 1 -- 172.21.15.152:0/1923903374 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb96004c950 con 0x7fb9681008b0 2024-03-29T04:36:48.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.196+0000 7fb96cad9700 1 --2- 172.21.15.152:0/1923903374 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb9500383c0 0x7fb95003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:48.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.197+0000 7fb96cad9700 1 --2- 172.21.15.152:0/1923903374 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb9500383c0 0x7fb95003a880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb95c006fd0 tx=0x7fb95c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:48.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.201+0000 7fb9667fc700 1 -- 172.21.15.152:0/1923903374 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb96001c5c0 con 0x7fb9681008b0 2024-03-29T04:36:48.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.463+0000 7fb96f53e700 1 -- 172.21.15.152:0/1923903374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb9680623c0 con 0x7fb9681008b0 2024-03-29T04:36:48.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.464+0000 7fb9667fc700 1 -- 172.21.15.152:0/1923903374 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fb96001c3e0 con 0x7fb9681008b0 2024-03-29T04:36:48.452 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:36:48.452 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:36:48.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.466+0000 7fb96f53e700 1 -- 172.21.15.152:0/1923903374 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb9500383c0 msgr2=0x7fb95003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:48.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.466+0000 7fb96f53e700 1 --2- 172.21.15.152:0/1923903374 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb9500383c0 0x7fb95003a880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb95c006fd0 tx=0x7fb95c006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:48.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.466+0000 7fb96f53e700 1 -- 172.21.15.152:0/1923903374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9681008b0 msgr2=0x7fb9681959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.466+0000 7fb96f53e700 1 --2- 172.21.15.152:0/1923903374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9681008b0 0x7fb9681959c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb96000b830 tx=0x7fb96000bd40 comp rx=0 tx=0).stop 2024-03-29T04:36:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.466+0000 7fb96f53e700 1 -- 172.21.15.152:0/1923903374 shutdown_connections 2024-03-29T04:36:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.466+0000 7fb96f53e700 1 --2- 172.21.15.152:0/1923903374 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb9500383c0 0x7fb95003a880 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.466+0000 7fb96f53e700 1 --2- 172.21.15.152:0/1923903374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9681008b0 0x7fb9681959c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.466+0000 7fb96f53e700 1 -- 172.21.15.152:0/1923903374 >> 172.21.15.152:0/1923903374 conn(0x7fb9680fbe30 msgr2=0x7fb9680fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.466+0000 7fb96f53e700 1 -- 172.21.15.152:0/1923903374 shutdown_connections 2024-03-29T04:36:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:48.466+0000 7fb96f53e700 1 -- 172.21.15.152:0/1923903374 wait complete. 2024-03-29T04:36:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:36:49.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:49 smithi142 ceph-mon[29027]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:49.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:49 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/1923903374' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:36:50.041 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:36:50.042 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:36:50.245 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:50.290 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:51.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:51 smithi142 ceph-mon[29027]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:51.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.686+0000 7fdc095b4700 1 -- 172.21.15.152:0/3208710361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040ff7a0 msgr2=0x7fdc040ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:51.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.686+0000 7fdc095b4700 1 --2- 172.21.15.152:0/3208710361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040ff7a0 0x7fdc040ffbc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009910 tx=0x7fdbf4009c20 comp rx=0 tx=0).stop 2024-03-29T04:36:51.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.686+0000 7fdc095b4700 1 -- 172.21.15.152:0/3208710361 shutdown_connections 2024-03-29T04:36:51.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.686+0000 7fdc095b4700 1 --2- 172.21.15.152:0/3208710361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040ff7a0 0x7fdc040ffbc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:51.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.686+0000 7fdc095b4700 1 -- 172.21.15.152:0/3208710361 >> 172.21.15.152:0/3208710361 conn(0x7fdc040fad20 msgr2=0x7fdc040fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.686+0000 7fdc095b4700 1 -- 172.21.15.152:0/3208710361 shutdown_connections 2024-03-29T04:36:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.686+0000 7fdc095b4700 1 -- 172.21.15.152:0/3208710361 wait complete. 2024-03-29T04:36:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.687+0000 7fdc095b4700 1 Processor -- start 2024-03-29T04:36:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.687+0000 7fdc095b4700 1 -- start start 2024-03-29T04:36:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.687+0000 7fdc095b4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040ff7a0 0x7fdc04195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.687+0000 7fdc095b4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc04195ed0 con 0x7fdc040ff7a0 2024-03-29T04:36:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.687+0000 7fdc02ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040ff7a0 0x7fdc04195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.687+0000 7fdc02ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040ff7a0 0x7fdc04195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:54856/0 (socket says 172.21.15.152:54856) 2024-03-29T04:36:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.687+0000 7fdc02ffd700 1 -- 172.21.15.152:0/1905098060 learned_addr learned my addr 172.21.15.152:0/1905098060 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:36:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.687+0000 7fdc02ffd700 1 -- 172.21.15.152:0/1905098060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf40095f0 con 0x7fdc040ff7a0 2024-03-29T04:36:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.688+0000 7fdc02ffd700 1 --2- 172.21.15.152:0/1905098060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040ff7a0 0x7fdc04195990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4005b40 tx=0x7fdbf4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:51.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.688+0000 7fdc00ff9700 1 -- 172.21.15.152:0/1905098060 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbf401d070 con 0x7fdc040ff7a0 2024-03-29T04:36:51.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.688+0000 7fdc095b4700 1 -- 172.21.15.152:0/1905098060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc041960d0 con 0x7fdc040ff7a0 2024-03-29T04:36:51.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.688+0000 7fdc00ff9700 1 -- 172.21.15.152:0/1905098060 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdbf400be10 con 0x7fdc040ff7a0 2024-03-29T04:36:51.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.688+0000 7fdc00ff9700 1 -- 172.21.15.152:0/1905098060 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbf400f460 con 0x7fdc040ff7a0 2024-03-29T04:36:51.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.688+0000 7fdc095b4700 1 -- 172.21.15.152:0/1905098060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc04196570 con 0x7fdc040ff7a0 2024-03-29T04:36:51.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.689+0000 7fdc00ff9700 1 -- 172.21.15.152:0/1905098060 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fdbf400b960 con 0x7fdc040ff7a0 2024-03-29T04:36:51.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.689+0000 7fdc095b4700 1 -- 172.21.15.152:0/1905098060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0418f3d0 con 0x7fdc040ff7a0 2024-03-29T04:36:51.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.689+0000 7fdc00ff9700 1 --2- 172.21.15.152:0/1905098060 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fdbf00383c0 0x7fdbf003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:51.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.689+0000 7fdc00ff9700 1 -- 172.21.15.152:0/1905098060 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdbf404c4b0 con 0x7fdc040ff7a0 2024-03-29T04:36:51.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.689+0000 7fdbfa7fc700 1 --2- 172.21.15.152:0/1905098060 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fdbf00383c0 0x7fdbf003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:51.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.690+0000 7fdbfa7fc700 1 --2- 172.21.15.152:0/1905098060 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fdbf00383c0 0x7fdbf003a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdbec006fd0 tx=0x7fdbec006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:51.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.695+0000 7fdc00ff9700 1 -- 172.21.15.152:0/1905098060 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdbf402a360 con 0x7fdc040ff7a0 2024-03-29T04:36:51.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.957+0000 7fdc095b4700 1 -- 172.21.15.152:0/1905098060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdc0402cc70 con 0x7fdc040ff7a0 2024-03-29T04:36:51.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.958+0000 7fdc00ff9700 1 -- 172.21.15.152:0/1905098060 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fdbf4027070 con 0x7fdc040ff7a0 2024-03-29T04:36:51.947 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:36:51.947 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:36:51.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.960+0000 7fdc095b4700 1 -- 172.21.15.152:0/1905098060 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fdbf00383c0 msgr2=0x7fdbf003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:51.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.960+0000 7fdc095b4700 1 --2- 172.21.15.152:0/1905098060 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fdbf00383c0 0x7fdbf003a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdbec006fd0 tx=0x7fdbec006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:51.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.960+0000 7fdc095b4700 1 -- 172.21.15.152:0/1905098060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040ff7a0 msgr2=0x7fdc04195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:51.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.960+0000 7fdc095b4700 1 --2- 172.21.15.152:0/1905098060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040ff7a0 0x7fdc04195990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4005b40 tx=0x7fdbf4004740 comp rx=0 tx=0).stop 2024-03-29T04:36:51.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.960+0000 7fdc095b4700 1 -- 172.21.15.152:0/1905098060 shutdown_connections 2024-03-29T04:36:51.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.960+0000 7fdc095b4700 1 --2- 172.21.15.152:0/1905098060 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fdbf00383c0 0x7fdbf003a880 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:51.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.960+0000 7fdc095b4700 1 --2- 172.21.15.152:0/1905098060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040ff7a0 0x7fdc04195990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:51.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.960+0000 7fdc095b4700 1 -- 172.21.15.152:0/1905098060 >> 172.21.15.152:0/1905098060 conn(0x7fdc040fad20 msgr2=0x7fdc04068810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:51.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.960+0000 7fdc095b4700 1 -- 172.21.15.152:0/1905098060 shutdown_connections 2024-03-29T04:36:51.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:51.960+0000 7fdc095b4700 1 -- 172.21.15.152:0/1905098060 wait complete. 2024-03-29T04:36:51.950 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:36:52.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:52 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/1905098060' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:36:53.536 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:36:53.536 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:36:53.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:53 smithi142 ceph-mon[29027]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:53.742 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:53.788 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.193+0000 7fe87064f700 1 -- 172.21.15.152:0/725879184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8681013f0 msgr2=0x7fe8681037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.193+0000 7fe87064f700 1 --2- 172.21.15.152:0/725879184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8681013f0 0x7fe8681037e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe864009910 tx=0x7fe864009c20 comp rx=0 tx=0).stop 2024-03-29T04:36:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.193+0000 7fe87064f700 1 -- 172.21.15.152:0/725879184 shutdown_connections 2024-03-29T04:36:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.193+0000 7fe87064f700 1 --2- 172.21.15.152:0/725879184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8681013f0 0x7fe8681037e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.193+0000 7fe87064f700 1 -- 172.21.15.152:0/725879184 >> 172.21.15.152:0/725879184 conn(0x7fe8680fad20 msgr2=0x7fe8680fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.193+0000 7fe87064f700 1 -- 172.21.15.152:0/725879184 shutdown_connections 2024-03-29T04:36:55.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.193+0000 7fe87064f700 1 -- 172.21.15.152:0/725879184 wait complete. 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.194+0000 7fe87064f700 1 Processor -- start 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.194+0000 7fe87064f700 1 -- start start 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.194+0000 7fe87064f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8681013f0 0x7fe868195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.194+0000 7fe87064f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe868195eb0 con 0x7fe8681013f0 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.194+0000 7fe86e3eb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8681013f0 0x7fe868195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.194+0000 7fe86e3eb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8681013f0 0x7fe868195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:54872/0 (socket says 172.21.15.152:54872) 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.194+0000 7fe86e3eb700 1 -- 172.21.15.152:0/2030193435 learned_addr learned my addr 172.21.15.152:0/2030193435 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.194+0000 7fe86e3eb700 1 -- 172.21.15.152:0/2030193435 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8640095f0 con 0x7fe8681013f0 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.194+0000 7fe86e3eb700 1 --2- 172.21.15.152:0/2030193435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8681013f0 0x7fe868195970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe864005b40 tx=0x7fe864004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.194+0000 7fe85f7fe700 1 -- 172.21.15.152:0/2030193435 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe86401d070 con 0x7fe8681013f0 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.195+0000 7fe87064f700 1 -- 172.21.15.152:0/2030193435 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8681960b0 con 0x7fe8681013f0 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.195+0000 7fe85f7fe700 1 -- 172.21.15.152:0/2030193435 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe86400be10 con 0x7fe8681013f0 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.195+0000 7fe85f7fe700 1 -- 172.21.15.152:0/2030193435 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe86400f460 con 0x7fe8681013f0 2024-03-29T04:36:55.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.195+0000 7fe87064f700 1 -- 172.21.15.152:0/2030193435 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe868196550 con 0x7fe8681013f0 2024-03-29T04:36:55.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.196+0000 7fe85f7fe700 1 -- 172.21.15.152:0/2030193435 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fe86400b960 con 0x7fe8681013f0 2024-03-29T04:36:55.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.196+0000 7fe87064f700 1 -- 172.21.15.152:0/2030193435 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86818f4d0 con 0x7fe8681013f0 2024-03-29T04:36:55.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.196+0000 7fe85f7fe700 1 --2- 172.21.15.152:0/2030193435 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fe854038410 0x7fe85403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:55.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.196+0000 7fe85f7fe700 1 -- 172.21.15.152:0/2030193435 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe86404c420 con 0x7fe8681013f0 2024-03-29T04:36:55.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.196+0000 7fe86dbea700 1 --2- 172.21.15.152:0/2030193435 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fe854038410 0x7fe85403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:55.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.196+0000 7fe86dbea700 1 --2- 172.21.15.152:0/2030193435 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fe854038410 0x7fe85403a8d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe858006fd0 tx=0x7fe858006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:55.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.201+0000 7fe85f7fe700 1 -- 172.21.15.152:0/2030193435 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe86402bb60 con 0x7fe8681013f0 2024-03-29T04:36:55.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.462+0000 7fe87064f700 1 -- 172.21.15.152:0/2030193435 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe86802cc70 con 0x7fe8681013f0 2024-03-29T04:36:55.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.484+0000 7fe85f7fe700 1 -- 172.21.15.152:0/2030193435 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fe864027070 con 0x7fe8681013f0 2024-03-29T04:36:55.472 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:36:55.473 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:36:55.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.486+0000 7fe87064f700 1 -- 172.21.15.152:0/2030193435 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fe854038410 msgr2=0x7fe85403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:55.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.486+0000 7fe87064f700 1 --2- 172.21.15.152:0/2030193435 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fe854038410 0x7fe85403a8d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe858006fd0 tx=0x7fe858006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:55.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.486+0000 7fe87064f700 1 -- 172.21.15.152:0/2030193435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8681013f0 msgr2=0x7fe868195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:55.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.486+0000 7fe87064f700 1 --2- 172.21.15.152:0/2030193435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8681013f0 0x7fe868195970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe864005b40 tx=0x7fe864004740 comp rx=0 tx=0).stop 2024-03-29T04:36:55.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.486+0000 7fe87064f700 1 -- 172.21.15.152:0/2030193435 shutdown_connections 2024-03-29T04:36:55.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.486+0000 7fe87064f700 1 --2- 172.21.15.152:0/2030193435 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fe854038410 0x7fe85403a8d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:55.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.486+0000 7fe87064f700 1 --2- 172.21.15.152:0/2030193435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8681013f0 0x7fe868195970 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:55.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.486+0000 7fe87064f700 1 -- 172.21.15.152:0/2030193435 >> 172.21.15.152:0/2030193435 conn(0x7fe8680fad20 msgr2=0x7fe8680fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:55.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.486+0000 7fe87064f700 1 -- 172.21.15.152:0/2030193435 shutdown_connections 2024-03-29T04:36:55.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:55.486+0000 7fe87064f700 1 -- 172.21.15.152:0/2030193435 wait complete. 2024-03-29T04:36:55.475 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:36:55.577 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:55 smithi142 ceph-mon[29027]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:56.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:56 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/2030193435' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:36:56.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:56 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:56.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:56 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:56.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:56 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:56.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:56 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:56.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:56 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:56.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:56 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:56.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:56 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:56.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:56 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:56.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:56 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:36:57.040 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:36:57.040 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:36:57.241 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:57.288 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:36:57.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:57 smithi142 ceph-mon[29027]: Deploying daemon prometheus.smithi142 on smithi142 2024-03-29T04:36:57.610 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:57 smithi142 ceph-mon[29027]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:58.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.681+0000 7fbd5157d700 1 -- 172.21.15.152:0/2253931383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd4c0ff7a0 msgr2=0x7fbd4c0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:58.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.681+0000 7fbd5157d700 1 --2- 172.21.15.152:0/2253931383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd4c0ff7a0 0x7fbd4c0ffbc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009910 tx=0x7fbd3c009c20 comp rx=0 tx=0).stop 2024-03-29T04:36:58.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.681+0000 7fbd5157d700 1 -- 172.21.15.152:0/2253931383 shutdown_connections 2024-03-29T04:36:58.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.681+0000 7fbd5157d700 1 --2- 172.21.15.152:0/2253931383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd4c0ff7a0 0x7fbd4c0ffbc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.681+0000 7fbd5157d700 1 -- 172.21.15.152:0/2253931383 >> 172.21.15.152:0/2253931383 conn(0x7fbd4c0fad20 msgr2=0x7fbd4c0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.681+0000 7fbd5157d700 1 -- 172.21.15.152:0/2253931383 shutdown_connections 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.681+0000 7fbd5157d700 1 -- 172.21.15.152:0/2253931383 wait complete. 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.681+0000 7fbd5157d700 1 Processor -- start 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.682+0000 7fbd5157d700 1 -- start start 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.682+0000 7fbd5157d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd4c0ff7a0 0x7fbd4c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.682+0000 7fbd5157d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c1986a0 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.682+0000 7fbd4affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd4c0ff7a0 0x7fbd4c069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.682+0000 7fbd4affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd4c0ff7a0 0x7fbd4c069430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:43526/0 (socket says 172.21.15.152:43526) 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.682+0000 7fbd4affd700 1 -- 172.21.15.152:0/3367199776 learned_addr learned my addr 172.21.15.152:0/3367199776 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.682+0000 7fbd4affd700 1 -- 172.21.15.152:0/3367199776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c0095f0 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.682+0000 7fbd4affd700 1 --2- 172.21.15.152:0/3367199776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd4c0ff7a0 0x7fbd4c069430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c005b40 tx=0x7fbd3c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:58.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.683+0000 7fbd33fff700 1 -- 172.21.15.152:0/3367199776 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd3c01d070 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.683+0000 7fbd5157d700 1 -- 172.21.15.152:0/3367199776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4c1987e0 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.683+0000 7fbd33fff700 1 -- 172.21.15.152:0/3367199776 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbd3c00be10 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.683+0000 7fbd33fff700 1 -- 172.21.15.152:0/3367199776 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd3c00f460 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.683+0000 7fbd5157d700 1 -- 172.21.15.152:0/3367199776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c198c80 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.684+0000 7fbd33fff700 1 -- 172.21.15.152:0/3367199776 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fbd3c00f6d0 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.684+0000 7fbd5157d700 1 -- 172.21.15.152:0/3367199776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4c18e3c0 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.684+0000 7fbd33fff700 1 --2- 172.21.15.152:0/3367199776 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fbd340383c0 0x7fbd3403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:36:58.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.684+0000 7fbd33fff700 1 -- 172.21.15.152:0/3367199776 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbd3c04c500 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.684+0000 7fbd4a7fc700 1 --2- 172.21.15.152:0/3367199776 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fbd340383c0 0x7fbd3403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:36:58.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.684+0000 7fbd4a7fc700 1 --2- 172.21.15.152:0/3367199776 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fbd340383c0 0x7fbd3403a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbd40006fd0 tx=0x7fbd40006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:36:58.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.691+0000 7fbd33fff700 1 -- 172.21.15.152:0/3367199776 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbd3c020550 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.957+0000 7fbd5157d700 1 -- 172.21.15.152:0/3367199776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbd4c02cc70 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.957+0000 7fbd33fff700 1 -- 172.21.15.152:0/3367199776 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fbd3c027020 con 0x7fbd4c0ff7a0 2024-03-29T04:36:58.946 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:36:58.946 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:36:58.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.959+0000 7fbd5157d700 1 -- 172.21.15.152:0/3367199776 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fbd340383c0 msgr2=0x7fbd3403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:58.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.959+0000 7fbd5157d700 1 --2- 172.21.15.152:0/3367199776 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fbd340383c0 0x7fbd3403a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbd40006fd0 tx=0x7fbd40006e40 comp rx=0 tx=0).stop 2024-03-29T04:36:58.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.959+0000 7fbd5157d700 1 -- 172.21.15.152:0/3367199776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd4c0ff7a0 msgr2=0x7fbd4c069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:36:58.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.959+0000 7fbd5157d700 1 --2- 172.21.15.152:0/3367199776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd4c0ff7a0 0x7fbd4c069430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c005b40 tx=0x7fbd3c004740 comp rx=0 tx=0).stop 2024-03-29T04:36:58.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.959+0000 7fbd5157d700 1 -- 172.21.15.152:0/3367199776 shutdown_connections 2024-03-29T04:36:58.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.959+0000 7fbd5157d700 1 --2- 172.21.15.152:0/3367199776 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fbd340383c0 0x7fbd3403a880 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:58.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.959+0000 7fbd5157d700 1 --2- 172.21.15.152:0/3367199776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd4c0ff7a0 0x7fbd4c069430 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:36:58.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.959+0000 7fbd5157d700 1 -- 172.21.15.152:0/3367199776 >> 172.21.15.152:0/3367199776 conn(0x7fbd4c0fad20 msgr2=0x7fbd4c0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:36:58.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.959+0000 7fbd5157d700 1 -- 172.21.15.152:0/3367199776 shutdown_connections 2024-03-29T04:36:58.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:36:58.959+0000 7fbd5157d700 1 -- 172.21.15.152:0/3367199776 wait complete. 2024-03-29T04:36:58.949 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:36:59.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:59 smithi142 ceph-mon[29027]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:36:59.610 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:36:59 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/3367199776' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:00.543 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:37:00.544 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:37:00.748 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:00.794 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:01.859 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:01 smithi142 ceph-mon[29027]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:02.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.180+0000 7f5ac3399700 1 -- 172.21.15.152:0/3080047737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5abc0ff790 msgr2=0x7f5abc0ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:02.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.180+0000 7f5ac3399700 1 --2- 172.21.15.152:0/3080047737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5abc0ff790 0x7f5abc0ffbb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009910 tx=0x7f5ab8009c20 comp rx=0 tx=0).stop 2024-03-29T04:37:02.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.180+0000 7f5ac3399700 1 -- 172.21.15.152:0/3080047737 shutdown_connections 2024-03-29T04:37:02.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.180+0000 7f5ac3399700 1 --2- 172.21.15.152:0/3080047737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5abc0ff790 0x7f5abc0ffbb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:02.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.180+0000 7f5ac3399700 1 -- 172.21.15.152:0/3080047737 >> 172.21.15.152:0/3080047737 conn(0x7f5abc0fad30 msgr2=0x7f5abc0fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:02.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.180+0000 7f5ac3399700 1 -- 172.21.15.152:0/3080047737 shutdown_connections 2024-03-29T04:37:02.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.180+0000 7f5ac3399700 1 -- 172.21.15.152:0/3080047737 wait complete. 2024-03-29T04:37:02.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.180+0000 7f5ac3399700 1 Processor -- start 2024-03-29T04:37:02.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.180+0000 7f5ac3399700 1 -- start start 2024-03-29T04:37:02.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.181+0000 7f5ac3399700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5abc0ff790 0x7f5abc194890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:02.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.181+0000 7f5ac3399700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc194dd0 con 0x7f5abc0ff790 2024-03-29T04:37:02.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.181+0000 7f5ac1135700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5abc0ff790 0x7f5abc194890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:02.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.181+0000 7f5ac1135700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5abc0ff790 0x7f5abc194890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:43530/0 (socket says 172.21.15.152:43530) 2024-03-29T04:37:02.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.181+0000 7f5ac1135700 1 -- 172.21.15.152:0/3585324199 learned_addr learned my addr 172.21.15.152:0/3585324199 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.181+0000 7f5ac1135700 1 -- 172.21.15.152:0/3585324199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab80095f0 con 0x7f5abc0ff790 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.181+0000 7f5ac1135700 1 --2- 172.21.15.152:0/3585324199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5abc0ff790 0x7f5abc194890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005b40 tx=0x7f5ab8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.181+0000 7f5ab27fc700 1 -- 172.21.15.152:0/3585324199 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ab801d070 con 0x7f5abc0ff790 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.181+0000 7f5ac3399700 1 -- 172.21.15.152:0/3585324199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc194fd0 con 0x7f5abc0ff790 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.181+0000 7f5ab27fc700 1 -- 172.21.15.152:0/3585324199 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5ab800be10 con 0x7f5abc0ff790 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.181+0000 7f5ac3399700 1 -- 172.21.15.152:0/3585324199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc195470 con 0x7f5abc0ff790 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.181+0000 7f5ab27fc700 1 -- 172.21.15.152:0/3585324199 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ab800f460 con 0x7f5abc0ff790 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.182+0000 7f5ab27fc700 1 -- 172.21.15.152:0/3585324199 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f5ab800b960 con 0x7f5abc0ff790 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.182+0000 7f5ac3399700 1 -- 172.21.15.152:0/3585324199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc18e3b0 con 0x7f5abc0ff790 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.182+0000 7f5ab27fc700 1 --2- 172.21.15.152:0/3585324199 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f5aa80383c0 0x7f5aa803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.182+0000 7f5ab27fc700 1 -- 172.21.15.152:0/3585324199 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5ab804c370 con 0x7f5abc0ff790 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.183+0000 7f5ac0934700 1 --2- 172.21.15.152:0/3585324199 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f5aa80383c0 0x7f5aa803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:02.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.183+0000 7f5ac0934700 1 --2- 172.21.15.152:0/3585324199 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f5aa80383c0 0x7f5aa803a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5aac006fd0 tx=0x7f5aac006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:02.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.188+0000 7f5ab27fc700 1 -- 172.21.15.152:0/3585324199 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5ab80227e0 con 0x7f5abc0ff790 2024-03-29T04:37:02.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.452+0000 7f5ac3399700 1 -- 172.21.15.152:0/3585324199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5abc02cc70 con 0x7f5abc0ff790 2024-03-29T04:37:02.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.453+0000 7f5ab27fc700 1 -- 172.21.15.152:0/3585324199 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f5ab8027030 con 0x7f5abc0ff790 2024-03-29T04:37:02.442 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:37:02.442 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:37:02.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.455+0000 7f5ac3399700 1 -- 172.21.15.152:0/3585324199 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f5aa80383c0 msgr2=0x7f5aa803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:02.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.455+0000 7f5ac3399700 1 --2- 172.21.15.152:0/3585324199 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f5aa80383c0 0x7f5aa803a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5aac006fd0 tx=0x7f5aac006e40 comp rx=0 tx=0).stop 2024-03-29T04:37:02.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.455+0000 7f5ac3399700 1 -- 172.21.15.152:0/3585324199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5abc0ff790 msgr2=0x7f5abc194890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:02.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.455+0000 7f5ac3399700 1 --2- 172.21.15.152:0/3585324199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5abc0ff790 0x7f5abc194890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005b40 tx=0x7f5ab8004740 comp rx=0 tx=0).stop 2024-03-29T04:37:02.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.455+0000 7f5ac3399700 1 -- 172.21.15.152:0/3585324199 shutdown_connections 2024-03-29T04:37:02.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.455+0000 7f5ac3399700 1 --2- 172.21.15.152:0/3585324199 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f5aa80383c0 0x7f5aa803a880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:02.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.455+0000 7f5ac3399700 1 --2- 172.21.15.152:0/3585324199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5abc0ff790 0x7f5abc194890 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:02.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.455+0000 7f5ac3399700 1 -- 172.21.15.152:0/3585324199 >> 172.21.15.152:0/3585324199 conn(0x7f5abc0fad30 msgr2=0x7f5abc0fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:02.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.455+0000 7f5ac3399700 1 -- 172.21.15.152:0/3585324199 shutdown_connections 2024-03-29T04:37:02.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:02.455+0000 7f5ac3399700 1 -- 172.21.15.152:0/3585324199 wait complete. 2024-03-29T04:37:02.445 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:37:03.496 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:03 smithi142 ceph-mon[29027]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:03.496 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:03 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/3585324199' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:03.984 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:37:03.985 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:37:04.188 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:04.234 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:05.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.714+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/395467999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc0ff7a0 msgr2=0x7fd4cc0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:05.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.714+0000 7fd4d2a1f700 1 --2- 172.21.15.152:0/395467999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc0ff7a0 0x7fd4cc0ffbc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009910 tx=0x7fd4bc009c20 comp rx=0 tx=0).stop 2024-03-29T04:37:05.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.714+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/395467999 shutdown_connections 2024-03-29T04:37:05.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.714+0000 7fd4d2a1f700 1 --2- 172.21.15.152:0/395467999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc0ff7a0 0x7fd4cc0ffbc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:05.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.714+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/395467999 >> 172.21.15.152:0/395467999 conn(0x7fd4cc0fad20 msgr2=0x7fd4cc0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:05.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.714+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/395467999 shutdown_connections 2024-03-29T04:37:05.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.714+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/395467999 wait complete. 2024-03-29T04:37:05.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.715+0000 7fd4d2a1f700 1 Processor -- start 2024-03-29T04:37:05.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.715+0000 7fd4d2a1f700 1 -- start start 2024-03-29T04:37:05.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.715+0000 7fd4d2a1f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc0ff7a0 0x7fd4cc193770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:05.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.715+0000 7fd4d2a1f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc193cb0 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.715+0000 7fd4cbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc0ff7a0 0x7fd4cc193770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:05.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.715+0000 7fd4cbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc0ff7a0 0x7fd4cc193770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:43554/0 (socket says 172.21.15.152:43554) 2024-03-29T04:37:05.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.715+0000 7fd4cbfff700 1 -- 172.21.15.152:0/576788658 learned_addr learned my addr 172.21.15.152:0/576788658 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:37:05.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.715+0000 7fd4cbfff700 1 -- 172.21.15.152:0/576788658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc0095f0 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.716+0000 7fd4cbfff700 1 --2- 172.21.15.152:0/576788658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc0ff7a0 0x7fd4cc193770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc005b40 tx=0x7fd4bc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:05.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.716+0000 7fd4c97fa700 1 -- 172.21.15.152:0/576788658 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4bc01d070 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.716+0000 7fd4c97fa700 1 -- 172.21.15.152:0/576788658 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd4bc00be10 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.716+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/576788658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc193eb0 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.716+0000 7fd4c97fa700 1 -- 172.21.15.152:0/576788658 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4bc00f460 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.716+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/576788658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc194350 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.717+0000 7fd4c97fa700 1 -- 172.21.15.152:0/576788658 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fd4bc00f690 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.717+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/576788658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4cc18d2b0 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.717+0000 7fd4c97fa700 1 --2- 172.21.15.152:0/576788658 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fd4b403a620 0x7fd4b403cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.717+0000 7fd4c97fa700 1 -- 172.21.15.152:0/576788658 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd4bc04c480 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.717+0000 7fd4cb7fe700 1 --2- 172.21.15.152:0/576788658 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fd4b403a620 0x7fd4b403cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:05.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.717+0000 7fd4cb7fe700 1 --2- 172.21.15.152:0/576788658 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fd4b403a620 0x7fd4b403cae0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0006fd0 tx=0x7fd4c0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:05.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.722+0000 7fd4c97fa700 1 -- 172.21.15.152:0/576788658 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd4bc02c7c0 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.799 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:05 smithi142 ceph-mon[29027]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:05.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.982+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/576788658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd4cc02cc70 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.983+0000 7fd4c97fa700 1 -- 172.21.15.152:0/576788658 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fd4bc027020 con 0x7fd4cc0ff7a0 2024-03-29T04:37:05.971 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:37:05.972 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:37:05.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.985+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/576788658 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fd4b403a620 msgr2=0x7fd4b403cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:05.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.985+0000 7fd4d2a1f700 1 --2- 172.21.15.152:0/576788658 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fd4b403a620 0x7fd4b403cae0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0006fd0 tx=0x7fd4c0006e40 comp rx=0 tx=0).stop 2024-03-29T04:37:05.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.985+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/576788658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc0ff7a0 msgr2=0x7fd4cc193770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:05.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.985+0000 7fd4d2a1f700 1 --2- 172.21.15.152:0/576788658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc0ff7a0 0x7fd4cc193770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc005b40 tx=0x7fd4bc004740 comp rx=0 tx=0).stop 2024-03-29T04:37:05.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.985+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/576788658 shutdown_connections 2024-03-29T04:37:05.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.985+0000 7fd4d2a1f700 1 --2- 172.21.15.152:0/576788658 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fd4b403a620 0x7fd4b403cae0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:05.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.985+0000 7fd4d2a1f700 1 --2- 172.21.15.152:0/576788658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc0ff7a0 0x7fd4cc193770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:05.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.985+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/576788658 >> 172.21.15.152:0/576788658 conn(0x7fd4cc0fad20 msgr2=0x7fd4cc0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:05.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.985+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/576788658 shutdown_connections 2024-03-29T04:37:05.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:05.985+0000 7fd4d2a1f700 1 -- 172.21.15.152:0/576788658 wait complete. 2024-03-29T04:37:05.974 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:37:06.610 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:06 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/576788658' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:07.556 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:37:07.557 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:37:07.610 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:07 smithi142 ceph-mon[29027]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:07.759 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:07.805 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:09.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.229+0000 7f56f5cae700 1 -- 172.21.15.152:0/137325672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f0101330 msgr2=0x7f56f0103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:09.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.229+0000 7f56f5cae700 1 --2- 172.21.15.152:0/137325672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f0101330 0x7f56f0103720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009910 tx=0x7f56e0009c20 comp rx=0 tx=0).stop 2024-03-29T04:37:09.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.229+0000 7f56f5cae700 1 -- 172.21.15.152:0/137325672 shutdown_connections 2024-03-29T04:37:09.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.229+0000 7f56f5cae700 1 --2- 172.21.15.152:0/137325672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f0101330 0x7f56f0103720 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:09.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.229+0000 7f56f5cae700 1 -- 172.21.15.152:0/137325672 >> 172.21.15.152:0/137325672 conn(0x7f56f00fad20 msgr2=0x7f56f00fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:09.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.229+0000 7f56f5cae700 1 -- 172.21.15.152:0/137325672 shutdown_connections 2024-03-29T04:37:09.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.229+0000 7f56f5cae700 1 -- 172.21.15.152:0/137325672 wait complete. 2024-03-29T04:37:09.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.230+0000 7f56f5cae700 1 Processor -- start 2024-03-29T04:37:09.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.230+0000 7f56f5cae700 1 -- start start 2024-03-29T04:37:09.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.230+0000 7f56f5cae700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f0101330 0x7f56f0100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:09.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.230+0000 7f56f5cae700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f01011d0 con 0x7f56f0101330 2024-03-29T04:37:09.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.230+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f0101330 0x7f56f0100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:09.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.230+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f0101330 0x7f56f0100c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:34710/0 (socket says 172.21.15.152:34710) 2024-03-29T04:37:09.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.230+0000 7f56ef7fe700 1 -- 172.21.15.152:0/3370555703 learned_addr learned my addr 172.21.15.152:0/3370555703 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:37:09.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.231+0000 7f56ef7fe700 1 -- 172.21.15.152:0/3370555703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e00095f0 con 0x7f56f0101330 2024-03-29T04:37:09.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.231+0000 7f56ef7fe700 1 --2- 172.21.15.152:0/3370555703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f0101330 0x7f56f0100c90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f56e0005b40 tx=0x7f56e0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:09.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.231+0000 7f56ed7fa700 1 -- 172.21.15.152:0/3370555703 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56e001d070 con 0x7f56f0101330 2024-03-29T04:37:09.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.231+0000 7f56f5cae700 1 -- 172.21.15.152:0/3370555703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f00ff2e0 con 0x7f56f0101330 2024-03-29T04:37:09.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.231+0000 7f56ed7fa700 1 -- 172.21.15.152:0/3370555703 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f56e000be10 con 0x7f56f0101330 2024-03-29T04:37:09.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.231+0000 7f56ed7fa700 1 -- 172.21.15.152:0/3370555703 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56e000f460 con 0x7f56f0101330 2024-03-29T04:37:09.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.231+0000 7f56f5cae700 1 -- 172.21.15.152:0/3370555703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f00ff780 con 0x7f56f0101330 2024-03-29T04:37:09.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.232+0000 7f56ed7fa700 1 -- 172.21.15.152:0/3370555703 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f56e00227e0 con 0x7f56f0101330 2024-03-29T04:37:09.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.232+0000 7f56f5cae700 1 -- 172.21.15.152:0/3370555703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f00623c0 con 0x7f56f0101330 2024-03-29T04:37:09.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.232+0000 7f56ed7fa700 1 --2- 172.21.15.152:0/3370555703 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f56dc0383c0 0x7f56dc03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:09.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.232+0000 7f56ed7fa700 1 -- 172.21.15.152:0/3370555703 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f56e004c580 con 0x7f56f0101330 2024-03-29T04:37:09.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.232+0000 7f56e6ffd700 1 --2- 172.21.15.152:0/3370555703 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f56dc0383c0 0x7f56dc03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:09.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.233+0000 7f56e6ffd700 1 --2- 172.21.15.152:0/3370555703 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f56dc0383c0 0x7f56dc03a880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f56d8006fd0 tx=0x7f56d8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:09.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.237+0000 7f56ed7fa700 1 -- 172.21.15.152:0/3370555703 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f56e002bba0 con 0x7f56f0101330 2024-03-29T04:37:09.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.501+0000 7f56f5cae700 1 -- 172.21.15.152:0/3370555703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f56f018eea0 con 0x7f56f0101330 2024-03-29T04:37:09.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.501+0000 7f56ed7fa700 1 -- 172.21.15.152:0/3370555703 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f56e0027020 con 0x7f56f0101330 2024-03-29T04:37:09.491 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:37:09.491 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:37:09.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.503+0000 7f56f5cae700 1 -- 172.21.15.152:0/3370555703 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f56dc0383c0 msgr2=0x7f56dc03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:09.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.503+0000 7f56f5cae700 1 --2- 172.21.15.152:0/3370555703 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f56dc0383c0 0x7f56dc03a880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f56d8006fd0 tx=0x7f56d8006e40 comp rx=0 tx=0).stop 2024-03-29T04:37:09.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.503+0000 7f56f5cae700 1 -- 172.21.15.152:0/3370555703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f0101330 msgr2=0x7f56f0100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:09.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.503+0000 7f56f5cae700 1 --2- 172.21.15.152:0/3370555703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f0101330 0x7f56f0100c90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f56e0005b40 tx=0x7f56e0004740 comp rx=0 tx=0).stop 2024-03-29T04:37:09.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.504+0000 7f56f5cae700 1 -- 172.21.15.152:0/3370555703 shutdown_connections 2024-03-29T04:37:09.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.504+0000 7f56f5cae700 1 --2- 172.21.15.152:0/3370555703 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f56dc0383c0 0x7f56dc03a880 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:09.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.504+0000 7f56f5cae700 1 --2- 172.21.15.152:0/3370555703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56f0101330 0x7f56f0100c90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:09.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.504+0000 7f56f5cae700 1 -- 172.21.15.152:0/3370555703 >> 172.21.15.152:0/3370555703 conn(0x7f56f00fad20 msgr2=0x7f56f00fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:09.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.504+0000 7f56f5cae700 1 -- 172.21.15.152:0/3370555703 shutdown_connections 2024-03-29T04:37:09.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:09.504+0000 7f56f5cae700 1 -- 172.21.15.152:0/3370555703 wait complete. 2024-03-29T04:37:09.494 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:37:09.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:09 smithi142 ceph-mon[29027]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:10.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:10 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/3370555703' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:10.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:10 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:10.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:10 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:10.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:10 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:10.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:10 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-29T04:37:11.163 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:37:11.163 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:37:11.367 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:11.414 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:11.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:11 smithi142 ceph-mon[29027]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:11.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:11 smithi142 ceph-mon[29027]: from='mgr.14162 172.21.15.142:0/1312753681' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-29T04:37:11.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:11 smithi142 ceph-mon[29027]: mgrmap e14: smithi142.swesrg(active, since 64s) 2024-03-29T04:37:12.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.906+0000 7f8d2e39b700 1 -- 172.21.15.152:0/1326958453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d280ff7a0 msgr2=0x7f8d280ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:12.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.906+0000 7f8d2e39b700 1 --2- 172.21.15.152:0/1326958453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d280ff7a0 0x7f8d280ffbc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009910 tx=0x7f8d18009c20 comp rx=0 tx=0).stop 2024-03-29T04:37:12.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.906+0000 7f8d2e39b700 1 -- 172.21.15.152:0/1326958453 shutdown_connections 2024-03-29T04:37:12.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.906+0000 7f8d2e39b700 1 --2- 172.21.15.152:0/1326958453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d280ff7a0 0x7f8d280ffbc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:12.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.906+0000 7f8d2e39b700 1 -- 172.21.15.152:0/1326958453 >> 172.21.15.152:0/1326958453 conn(0x7f8d280fad20 msgr2=0x7f8d280fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:12.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.906+0000 7f8d2e39b700 1 -- 172.21.15.152:0/1326958453 shutdown_connections 2024-03-29T04:37:12.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.906+0000 7f8d2e39b700 1 -- 172.21.15.152:0/1326958453 wait complete. 2024-03-29T04:37:12.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.907+0000 7f8d2e39b700 1 Processor -- start 2024-03-29T04:37:12.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.907+0000 7f8d2e39b700 1 -- start start 2024-03-29T04:37:12.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.907+0000 7f8d2e39b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d280ff7a0 0x7f8d28069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:12.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.907+0000 7f8d2e39b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d28198710 con 0x7f8d280ff7a0 2024-03-29T04:37:12.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.907+0000 7f8d27fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d280ff7a0 0x7f8d28069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:12.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.907+0000 7f8d27fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d280ff7a0 0x7f8d28069390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:34728/0 (socket says 172.21.15.152:34728) 2024-03-29T04:37:12.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.907+0000 7f8d27fff700 1 -- 172.21.15.152:0/229414201 learned_addr learned my addr 172.21.15.152:0/229414201 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:37:12.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.908+0000 7f8d27fff700 1 -- 172.21.15.152:0/229414201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d180095f0 con 0x7f8d280ff7a0 2024-03-29T04:37:12.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.908+0000 7f8d27fff700 1 --2- 172.21.15.152:0/229414201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d280ff7a0 0x7f8d28069390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8d18005950 tx=0x7f8d18009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:12.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.908+0000 7f8d257fa700 1 -- 172.21.15.152:0/229414201 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d1801d070 con 0x7f8d280ff7a0 2024-03-29T04:37:12.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.908+0000 7f8d257fa700 1 -- 172.21.15.152:0/229414201 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8d1800be10 con 0x7f8d280ff7a0 2024-03-29T04:37:12.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.908+0000 7f8d257fa700 1 -- 172.21.15.152:0/229414201 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d1800f460 con 0x7f8d280ff7a0 2024-03-29T04:37:12.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.908+0000 7f8d2e39b700 1 -- 172.21.15.152:0/229414201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d28198850 con 0x7f8d280ff7a0 2024-03-29T04:37:12.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.908+0000 7f8d2e39b700 1 -- 172.21.15.152:0/229414201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d28198cf0 con 0x7f8d280ff7a0 2024-03-29T04:37:12.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.909+0000 7f8d257fa700 1 -- 172.21.15.152:0/229414201 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 14) v1 ==== 45354+0+0 (secure 0 0 0) 0x7f8d180227c0 con 0x7f8d280ff7a0 2024-03-29T04:37:12.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.909+0000 7f8d2e39b700 1 -- 172.21.15.152:0/229414201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2818e3c0 con 0x7f8d280ff7a0 2024-03-29T04:37:12.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.909+0000 7f8d257fa700 1 --2- 172.21.15.152:0/229414201 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f8d10038410 0x7f8d1003a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:12.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.909+0000 7f8d257fa700 1 -- 172.21.15.152:0/229414201 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8d1804c490 con 0x7f8d280ff7a0 2024-03-29T04:37:12.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.909+0000 7f8d277fe700 1 -- 172.21.15.152:0/229414201 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f8d10038410 msgr2=0x7f8d1003a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/779416791 2024-03-29T04:37:12.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.909+0000 7f8d277fe700 1 --2- 172.21.15.152:0/229414201 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f8d10038410 0x7f8d1003a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:37:12.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:12.914+0000 7f8d257fa700 1 -- 172.21.15.152:0/229414201 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8d1802bba0 con 0x7f8d280ff7a0 2024-03-29T04:37:13.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.110+0000 7f8d277fe700 1 -- 172.21.15.152:0/229414201 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f8d10038410 msgr2=0x7f8d1003a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/779416791 2024-03-29T04:37:13.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.110+0000 7f8d277fe700 1 --2- 172.21.15.152:0/229414201 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f8d10038410 0x7f8d1003a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:37:13.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.181+0000 7f8d2e39b700 1 -- 172.21.15.152:0/229414201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8d2802cc70 con 0x7f8d280ff7a0 2024-03-29T04:37:13.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.181+0000 7f8d257fa700 1 -- 172.21.15.152:0/229414201 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f8d18027030 con 0x7f8d280ff7a0 2024-03-29T04:37:13.170 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:37:13.171 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:37:13.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.183+0000 7f8d2e39b700 1 -- 172.21.15.152:0/229414201 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f8d10038410 msgr2=0x7f8d1003a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:37:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.183+0000 7f8d2e39b700 1 --2- 172.21.15.152:0/229414201 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f8d10038410 0x7f8d1003a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.183+0000 7f8d2e39b700 1 -- 172.21.15.152:0/229414201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d280ff7a0 msgr2=0x7f8d28069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.183+0000 7f8d2e39b700 1 --2- 172.21.15.152:0/229414201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d280ff7a0 0x7f8d28069390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8d18005950 tx=0x7f8d18009e60 comp rx=0 tx=0).stop 2024-03-29T04:37:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.183+0000 7f8d2e39b700 1 -- 172.21.15.152:0/229414201 shutdown_connections 2024-03-29T04:37:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.183+0000 7f8d2e39b700 1 --2- 172.21.15.152:0/229414201 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7f8d10038410 0x7f8d1003a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.183+0000 7f8d2e39b700 1 --2- 172.21.15.152:0/229414201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d280ff7a0 0x7f8d28069390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.183+0000 7f8d2e39b700 1 -- 172.21.15.152:0/229414201 >> 172.21.15.152:0/229414201 conn(0x7f8d280fad20 msgr2=0x7f8d280fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.183+0000 7f8d2e39b700 1 -- 172.21.15.152:0/229414201 shutdown_connections 2024-03-29T04:37:13.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:13.183+0000 7f8d2e39b700 1 -- 172.21.15.152:0/229414201 wait complete. 2024-03-29T04:37:13.174 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:37:13.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:13 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/229414201' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:14.784 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:37:14.785 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:37:14.987 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:15.032 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:16.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.493+0000 7fb67adc5700 1 -- 172.21.15.152:0/3344245216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6741008b0 msgr2=0x7fb674100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:16.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.493+0000 7fb67adc5700 1 --2- 172.21.15.152:0/3344245216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6741008b0 0x7fb674100cd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb668009910 tx=0x7fb668009c20 comp rx=0 tx=0).stop 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.493+0000 7fb67adc5700 1 -- 172.21.15.152:0/3344245216 shutdown_connections 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.493+0000 7fb67adc5700 1 --2- 172.21.15.152:0/3344245216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6741008b0 0x7fb674100cd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.493+0000 7fb67adc5700 1 -- 172.21.15.152:0/3344245216 >> 172.21.15.152:0/3344245216 conn(0x7fb6740fbe30 msgr2=0x7fb6740fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.493+0000 7fb67adc5700 1 -- 172.21.15.152:0/3344245216 shutdown_connections 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.493+0000 7fb67adc5700 1 -- 172.21.15.152:0/3344245216 wait complete. 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.493+0000 7fb67adc5700 1 Processor -- start 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.493+0000 7fb67adc5700 1 -- start start 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.493+0000 7fb67adc5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6741008b0 0x7fb6741959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.493+0000 7fb67adc5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb674195f20 con 0x7fb6741008b0 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.494+0000 7fb678b61700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6741008b0 0x7fb6741959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.494+0000 7fb678b61700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6741008b0 0x7fb6741959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:34748/0 (socket says 172.21.15.152:34748) 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.494+0000 7fb678b61700 1 -- 172.21.15.152:0/861670427 learned_addr learned my addr 172.21.15.152:0/861670427 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.494+0000 7fb678b61700 1 -- 172.21.15.152:0/861670427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6680095f0 con 0x7fb6741008b0 2024-03-29T04:37:16.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.494+0000 7fb678b61700 1 --2- 172.21.15.152:0/861670427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6741008b0 0x7fb6741959e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb668005b40 tx=0x7fb668004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:16.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.494+0000 7fb671ffb700 1 -- 172.21.15.152:0/861670427 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb66801d070 con 0x7fb6741008b0 2024-03-29T04:37:16.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.494+0000 7fb67adc5700 1 -- 172.21.15.152:0/861670427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb674196120 con 0x7fb6741008b0 2024-03-29T04:37:16.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.494+0000 7fb671ffb700 1 -- 172.21.15.152:0/861670427 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb66800be10 con 0x7fb6741008b0 2024-03-29T04:37:16.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.494+0000 7fb671ffb700 1 -- 172.21.15.152:0/861670427 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb66800f460 con 0x7fb6741008b0 2024-03-29T04:37:16.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.495+0000 7fb67adc5700 1 -- 172.21.15.152:0/861670427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6741965c0 con 0x7fb6741008b0 2024-03-29T04:37:16.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.495+0000 7fb67adc5700 1 -- 172.21.15.152:0/861670427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67418f4d0 con 0x7fb6741008b0 2024-03-29T04:37:16.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.495+0000 7fb671ffb700 1 -- 172.21.15.152:0/861670427 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 14) v1 ==== 45354+0+0 (secure 0 0 0) 0x7fb66800b960 con 0x7fb6741008b0 2024-03-29T04:37:16.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.496+0000 7fb671ffb700 1 --2- 172.21.15.152:0/861670427 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb664038410 0x7fb66403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:16.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.496+0000 7fb671ffb700 1 -- 172.21.15.152:0/861670427 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb66804c430 con 0x7fb6741008b0 2024-03-29T04:37:16.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.496+0000 7fb673fff700 1 -- 172.21.15.152:0/861670427 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb664038410 msgr2=0x7fb66403a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/779416791 2024-03-29T04:37:16.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.496+0000 7fb673fff700 1 --2- 172.21.15.152:0/861670427 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb664038410 0x7fb66403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:37:16.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.500+0000 7fb671ffb700 1 -- 172.21.15.152:0/861670427 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb66802a360 con 0x7fb6741008b0 2024-03-29T04:37:16.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.697+0000 7fb673fff700 1 -- 172.21.15.152:0/861670427 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb664038410 msgr2=0x7fb66403a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.142:6800/779416791 2024-03-29T04:37:16.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.697+0000 7fb673fff700 1 --2- 172.21.15.152:0/861670427 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb664038410 0x7fb66403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:37:16.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.769+0000 7fb67adc5700 1 -- 172.21.15.152:0/861670427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb67402cc70 con 0x7fb6741008b0 2024-03-29T04:37:16.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.770+0000 7fb671ffb700 1 -- 172.21.15.152:0/861670427 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fb668027070 con 0x7fb6741008b0 2024-03-29T04:37:16.759 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:37:16.759 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:37:16.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.772+0000 7fb67adc5700 1 -- 172.21.15.152:0/861670427 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb664038410 msgr2=0x7fb66403a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:37:16.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.772+0000 7fb67adc5700 1 --2- 172.21.15.152:0/861670427 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb664038410 0x7fb66403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:16.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.772+0000 7fb67adc5700 1 -- 172.21.15.152:0/861670427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6741008b0 msgr2=0x7fb6741959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:16.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.772+0000 7fb67adc5700 1 --2- 172.21.15.152:0/861670427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6741008b0 0x7fb6741959e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb668005b40 tx=0x7fb668004740 comp rx=0 tx=0).stop 2024-03-29T04:37:16.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.772+0000 7fb67adc5700 1 -- 172.21.15.152:0/861670427 shutdown_connections 2024-03-29T04:37:16.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.772+0000 7fb67adc5700 1 --2- 172.21.15.152:0/861670427 >> [v2:172.21.15.142:6800/779416791,v1:172.21.15.142:6801/779416791] conn(0x7fb664038410 0x7fb66403a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:16.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.772+0000 7fb67adc5700 1 --2- 172.21.15.152:0/861670427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6741008b0 0x7fb6741959e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:16.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.772+0000 7fb67adc5700 1 -- 172.21.15.152:0/861670427 >> 172.21.15.152:0/861670427 conn(0x7fb6740fbe30 msgr2=0x7fb6740fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:16.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.772+0000 7fb67adc5700 1 -- 172.21.15.152:0/861670427 shutdown_connections 2024-03-29T04:37:16.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:16.772+0000 7fb67adc5700 1 -- 172.21.15.152:0/861670427 wait complete. 2024-03-29T04:37:16.762 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:37:17.110 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:16 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/861670427' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:17.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: Active manager daemon smithi142.swesrg restarted 2024-03-29T04:37:17.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: Activating manager daemon smithi142.swesrg 2024-03-29T04:37:17.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: osdmap e5: 0 total, 0 up, 0 in 2024-03-29T04:37:17.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: mgrmap e15: smithi142.swesrg(active, starting, since 0.00321223s) 2024-03-29T04:37:17.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:37:17.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi142.swesrg", "id": "smithi142.swesrg"}]: dispatch 2024-03-29T04:37:17.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T04:37:17.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T04:37:17.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T04:37:17.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: Manager daemon smithi142.swesrg is now available 2024-03-29T04:37:17.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:37:17.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:37:17.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:17.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:37:17.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:37:17.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/trash_purge_schedule"}]: dispatch 2024-03-29T04:37:18.314 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:37:18.314 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:37:18.509 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:18.557 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:19.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:19 smithi142 ceph-mon[29027]: [29/Mar/2024:04:37:18] ENGINE Bus STARTING 2024-03-29T04:37:19.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:19 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:19.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:19 smithi142 ceph-mon[29027]: [29/Mar/2024:04:37:18] ENGINE Serving on https://172.21.15.142:7150 2024-03-29T04:37:19.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:19 smithi142 ceph-mon[29027]: [29/Mar/2024:04:37:18] ENGINE Serving on http://172.21.15.142:8765 2024-03-29T04:37:19.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:19 smithi142 ceph-mon[29027]: [29/Mar/2024:04:37:18] ENGINE Bus STARTED 2024-03-29T04:37:19.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:19 smithi142 ceph-mon[29027]: mgrmap e16: smithi142.swesrg(active, since 1.00556s) 2024-03-29T04:37:19.557 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:19 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:20.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.080+0000 7f81d93f1700 1 -- 172.21.15.152:0/3099132749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d41013f0 msgr2=0x7f81d41037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:20.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.080+0000 7f81d93f1700 1 --2- 172.21.15.152:0/3099132749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d41013f0 0x7f81d41037e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009910 tx=0x7f81c4009c20 comp rx=0 tx=0).stop 2024-03-29T04:37:20.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.080+0000 7f81d93f1700 1 -- 172.21.15.152:0/3099132749 shutdown_connections 2024-03-29T04:37:20.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.080+0000 7f81d93f1700 1 --2- 172.21.15.152:0/3099132749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d41013f0 0x7f81d41037e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:20.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.080+0000 7f81d93f1700 1 -- 172.21.15.152:0/3099132749 >> 172.21.15.152:0/3099132749 conn(0x7f81d40fad20 msgr2=0x7f81d40fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:20.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.080+0000 7f81d93f1700 1 -- 172.21.15.152:0/3099132749 shutdown_connections 2024-03-29T04:37:20.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.080+0000 7f81d93f1700 1 -- 172.21.15.152:0/3099132749 wait complete. 2024-03-29T04:37:20.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.081+0000 7f81d93f1700 1 Processor -- start 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.081+0000 7f81d93f1700 1 -- start start 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.081+0000 7f81d93f1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d41013f0 0x7f81d4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.081+0000 7f81d93f1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d41980c0 con 0x7f81d41013f0 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.082+0000 7f81d2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d41013f0 0x7f81d4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.082+0000 7f81d2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d41013f0 0x7f81d4197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:59646/0 (socket says 172.21.15.152:59646) 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.082+0000 7f81d2ffd700 1 -- 172.21.15.152:0/3816660024 learned_addr learned my addr 172.21.15.152:0/3816660024 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.082+0000 7f81d2ffd700 1 -- 172.21.15.152:0/3816660024 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c40095f0 con 0x7f81d41013f0 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.082+0000 7f81d2ffd700 1 --2- 172.21.15.152:0/3816660024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d41013f0 0x7f81d4197b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f81c4005b40 tx=0x7f81c4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.083+0000 7f81bbfff700 1 -- 172.21.15.152:0/3816660024 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81c401d070 con 0x7f81d41013f0 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.083+0000 7f81bbfff700 1 -- 172.21.15.152:0/3816660024 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f81c400be10 con 0x7f81d41013f0 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.083+0000 7f81d93f1700 1 -- 172.21.15.152:0/3816660024 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d41982c0 con 0x7f81d41013f0 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.083+0000 7f81bbfff700 1 -- 172.21.15.152:0/3816660024 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81c400f460 con 0x7f81d41013f0 2024-03-29T04:37:20.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.083+0000 7f81d93f1700 1 -- 172.21.15.152:0/3816660024 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d4198760 con 0x7f81d41013f0 2024-03-29T04:37:20.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.084+0000 7f81d93f1700 1 -- 172.21.15.152:0/3816660024 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d41918d0 con 0x7f81d41013f0 2024-03-29T04:37:20.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.084+0000 7f81bbfff700 1 -- 172.21.15.152:0/3816660024 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 45204+0+0 (secure 0 0 0) 0x7f81c400f6d0 con 0x7f81d41013f0 2024-03-29T04:37:20.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.084+0000 7f81bbfff700 1 --2- 172.21.15.152:0/3816660024 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f81bc038270 0x7f81bc03a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:20.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.084+0000 7f81bbfff700 1 -- 172.21.15.152:0/3816660024 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f81c404c380 con 0x7f81d41013f0 2024-03-29T04:37:20.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.084+0000 7f81d27fc700 1 --2- 172.21.15.152:0/3816660024 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f81bc038270 0x7f81bc03a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:20.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.085+0000 7f81d27fc700 1 --2- 172.21.15.152:0/3816660024 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f81bc038270 0x7f81bc03a730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f81c8006fd0 tx=0x7f81c8006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:20.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.089+0000 7f81bbfff700 1 -- 172.21.15.152:0/3816660024 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f81c402a360 con 0x7f81d41013f0 2024-03-29T04:37:20.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.253+0000 7f81bbfff700 1 -- 172.21.15.152:0/3816660024 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f81c40227e0 con 0x7f81d41013f0 2024-03-29T04:37:20.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.370+0000 7f81d93f1700 1 -- 172.21.15.152:0/3816660024 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f81d402cc70 con 0x7f81d41013f0 2024-03-29T04:37:20.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.371+0000 7f81bbfff700 1 -- 172.21.15.152:0/3816660024 <== mon.0 v2:172.21.15.142:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f81c4027070 con 0x7f81d41013f0 2024-03-29T04:37:20.360 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:37:20.360 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:37:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.374+0000 7f81d93f1700 1 -- 172.21.15.152:0/3816660024 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f81bc038270 msgr2=0x7f81bc03a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.374+0000 7f81d93f1700 1 --2- 172.21.15.152:0/3816660024 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f81bc038270 0x7f81bc03a730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f81c8006fd0 tx=0x7f81c8006e40 comp rx=0 tx=0).stop 2024-03-29T04:37:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.374+0000 7f81d93f1700 1 -- 172.21.15.152:0/3816660024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d41013f0 msgr2=0x7f81d4197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.374+0000 7f81d93f1700 1 --2- 172.21.15.152:0/3816660024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d41013f0 0x7f81d4197b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f81c4005b40 tx=0x7f81c4004740 comp rx=0 tx=0).stop 2024-03-29T04:37:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.374+0000 7f81d93f1700 1 -- 172.21.15.152:0/3816660024 shutdown_connections 2024-03-29T04:37:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.374+0000 7f81d93f1700 1 --2- 172.21.15.152:0/3816660024 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f81bc038270 0x7f81bc03a730 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.374+0000 7f81d93f1700 1 --2- 172.21.15.152:0/3816660024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d41013f0 0x7f81d4197b80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.374+0000 7f81d93f1700 1 -- 172.21.15.152:0/3816660024 >> 172.21.15.152:0/3816660024 conn(0x7f81d40fad20 msgr2=0x7f81d40fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.374+0000 7f81d93f1700 1 -- 172.21.15.152:0/3816660024 shutdown_connections 2024-03-29T04:37:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:20.374+0000 7f81d93f1700 1 -- 172.21.15.152:0/3816660024 wait complete. 2024-03-29T04:37:20.364 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:37:21.110 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:20 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:21.110 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:20 smithi142 ceph-mon[29027]: mgrmap e17: smithi142.swesrg(active, since 2s) 2024-03-29T04:37:21.110 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:20 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/3816660024' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:22.662 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:37:22.662 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:37:23.110 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:23.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:23.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:23.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:23.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:37:23.313 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:23.359 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T04:37:24.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:24.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:24.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:24.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:24.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:37:24.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:24.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:37:24.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-29T04:37:24.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-29T04:37:24.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:37:24.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:37:24.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:37:24.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:37:24.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:24.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:24.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:24.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:24.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:24.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi152", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:37:24.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi152", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-29T04:37:24.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:25.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.048+0000 7fc021a20700 1 -- 172.21.15.152:0/2105918109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c106830 msgr2=0x7fc01c106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.048+0000 7fc021a20700 1 --2- 172.21.15.152:0/2105918109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c106830 0x7fc01c106c10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc010009910 tx=0x7fc010009c20 comp rx=0 tx=0).stop 2024-03-29T04:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.048+0000 7fc021a20700 1 -- 172.21.15.152:0/2105918109 shutdown_connections 2024-03-29T04:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.048+0000 7fc021a20700 1 --2- 172.21.15.152:0/2105918109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c106830 0x7fc01c106c10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.048+0000 7fc021a20700 1 -- 172.21.15.152:0/2105918109 >> 172.21.15.152:0/2105918109 conn(0x7fc01c0fc380 msgr2=0x7fc01c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.048+0000 7fc021a20700 1 -- 172.21.15.152:0/2105918109 shutdown_connections 2024-03-29T04:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.048+0000 7fc021a20700 1 -- 172.21.15.152:0/2105918109 wait complete. 2024-03-29T04:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.048+0000 7fc021a20700 1 Processor -- start 2024-03-29T04:37:25.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.049+0000 7fc021a20700 1 -- start start 2024-03-29T04:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.049+0000 7fc021a20700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c106830 0x7fc01c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.049+0000 7fc021a20700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c075060 con 0x7fc01c106830 2024-03-29T04:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.049+0000 7fc01affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c106830 0x7fc01c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.049+0000 7fc01affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c106830 0x7fc01c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:59656/0 (socket says 172.21.15.152:59656) 2024-03-29T04:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.049+0000 7fc01affd700 1 -- 172.21.15.152:0/1623689759 learned_addr learned my addr 172.21.15.152:0/1623689759 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.049+0000 7fc01affd700 1 -- 172.21.15.152:0/1623689759 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0100095f0 con 0x7fc01c106830 2024-03-29T04:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.049+0000 7fc01affd700 1 --2- 172.21.15.152:0/1623689759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c106830 0x7fc01c073090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc010005b40 tx=0x7fc010004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.049+0000 7fc020a1e700 1 -- 172.21.15.152:0/1623689759 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc01001d070 con 0x7fc01c106830 2024-03-29T04:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.050+0000 7fc021a20700 1 -- 172.21.15.152:0/1623689759 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01c073690 con 0x7fc01c106830 2024-03-29T04:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.050+0000 7fc020a1e700 1 -- 172.21.15.152:0/1623689759 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc0100044d0 con 0x7fc01c106830 2024-03-29T04:37:25.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.050+0000 7fc020a1e700 1 -- 172.21.15.152:0/1623689759 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc010003910 con 0x7fc01c106830 2024-03-29T04:37:25.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.050+0000 7fc021a20700 1 -- 172.21.15.152:0/1623689759 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c073b10 con 0x7fc01c106830 2024-03-29T04:37:25.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.050+0000 7fc020a1e700 1 -- 172.21.15.152:0/1623689759 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fc010003b90 con 0x7fc01c106830 2024-03-29T04:37:25.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.051+0000 7fc020a1e700 1 --2- 172.21.15.152:0/1623689759 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc0040384b0 0x7fc00403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:25.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.051+0000 7fc021a20700 1 -- 172.21.15.152:0/1623689759 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01c104ae0 con 0x7fc01c106830 2024-03-29T04:37:25.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.051+0000 7fc020a1e700 1 -- 172.21.15.152:0/1623689759 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fc01004c4a0 con 0x7fc01c106830 2024-03-29T04:37:25.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.051+0000 7fc01a7fc700 1 --2- 172.21.15.152:0/1623689759 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc0040384b0 0x7fc00403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:25.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.051+0000 7fc01a7fc700 1 --2- 172.21.15.152:0/1623689759 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc0040384b0 0x7fc00403a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc00c006fd0 tx=0x7fc00c006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:25.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.056+0000 7fc020a1e700 1 -- 172.21.15.152:0/1623689759 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc01002a950 con 0x7fc01c106830 2024-03-29T04:37:25.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.330+0000 7fc021a20700 1 -- 172.21.15.152:0/1623689759 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc01c0623c0 con 0x7fc01c106830 2024-03-29T04:37:25.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.331+0000 7fc020a1e700 1 -- 172.21.15.152:0/1623689759 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fc01002a360 con 0x7fc01c106830 2024-03-29T04:37:25.320 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:37:25.321 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:37:25.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.333+0000 7fc021a20700 1 -- 172.21.15.152:0/1623689759 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc0040384b0 msgr2=0x7fc00403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:25.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.333+0000 7fc021a20700 1 --2- 172.21.15.152:0/1623689759 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc0040384b0 0x7fc00403a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc00c006fd0 tx=0x7fc00c006e40 comp rx=0 tx=0).stop 2024-03-29T04:37:25.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.333+0000 7fc021a20700 1 -- 172.21.15.152:0/1623689759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c106830 msgr2=0x7fc01c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:25.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.333+0000 7fc021a20700 1 --2- 172.21.15.152:0/1623689759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c106830 0x7fc01c073090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc010005b40 tx=0x7fc010004c80 comp rx=0 tx=0).stop 2024-03-29T04:37:25.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.333+0000 7fc021a20700 1 -- 172.21.15.152:0/1623689759 shutdown_connections 2024-03-29T04:37:25.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.333+0000 7fc021a20700 1 --2- 172.21.15.152:0/1623689759 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc0040384b0 0x7fc00403a970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:25.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.333+0000 7fc021a20700 1 --2- 172.21.15.152:0/1623689759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c106830 0x7fc01c073090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:25.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.333+0000 7fc021a20700 1 -- 172.21.15.152:0/1623689759 >> 172.21.15.152:0/1623689759 conn(0x7fc01c0fc380 msgr2=0x7fc01c108bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:25.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.333+0000 7fc021a20700 1 -- 172.21.15.152:0/1623689759 shutdown_connections 2024-03-29T04:37:25.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:25.333+0000 7fc021a20700 1 -- 172.21.15.152:0/1623689759 wait complete. 2024-03-29T04:37:25.323 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:37:25.610 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:25 smithi142 ceph-mon[29027]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:37:25.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:25 smithi142 ceph-mon[29027]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:37:25.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:25 smithi142 ceph-mon[29027]: Deploying daemon ceph-exporter.smithi152 on smithi152 2024-03-29T04:37:26.610 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:26 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/1623689759' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:27.496 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:37:27.496 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:37:27.700 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:37:28.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:28.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:28.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:28.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:28.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi152", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:37:28.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi152", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-29T04:37:28.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:28.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:28 smithi142 ceph-mon[29027]: Deploying daemon crash.smithi152 on smithi152 2024-03-29T04:37:30.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.128+0000 7f0edbc6c700 1 -- 172.21.15.152:0/196920676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed40ff0f0 msgr2=0x7f0ed40ff4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:30.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.128+0000 7f0edbc6c700 1 --2- 172.21.15.152:0/196920676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed40ff0f0 0x7f0ed40ff4d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009910 tx=0x7f0ed0009c20 comp rx=0 tx=0).stop 2024-03-29T04:37:30.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.129+0000 7f0edbc6c700 1 -- 172.21.15.152:0/196920676 shutdown_connections 2024-03-29T04:37:30.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.129+0000 7f0edbc6c700 1 --2- 172.21.15.152:0/196920676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed40ff0f0 0x7f0ed40ff4d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:30.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.129+0000 7f0edbc6c700 1 -- 172.21.15.152:0/196920676 >> 172.21.15.152:0/196920676 conn(0x7f0ed4075e80 msgr2=0x7f0ed4076290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:30.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.129+0000 7f0edbc6c700 1 -- 172.21.15.152:0/196920676 shutdown_connections 2024-03-29T04:37:30.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.129+0000 7f0edbc6c700 1 -- 172.21.15.152:0/196920676 wait complete. 2024-03-29T04:37:30.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.129+0000 7f0edbc6c700 1 Processor -- start 2024-03-29T04:37:30.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.130+0000 7f0edbc6c700 1 -- start start 2024-03-29T04:37:30.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.130+0000 7f0edbc6c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed40ff0f0 0x7f0ed4195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:30.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.130+0000 7f0edbc6c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed4196270 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.130+0000 7f0ed9a08700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed40ff0f0 0x7f0ed4195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:30.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.130+0000 7f0ed9a08700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed40ff0f0 0x7f0ed4195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:41146/0 (socket says 172.21.15.152:41146) 2024-03-29T04:37:30.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.130+0000 7f0ed9a08700 1 -- 172.21.15.152:0/2796478581 learned_addr learned my addr 172.21.15.152:0/2796478581 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:37:30.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.130+0000 7f0ed9a08700 1 -- 172.21.15.152:0/2796478581 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed00095f0 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.130+0000 7f0ed9a08700 1 --2- 172.21.15.152:0/2796478581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed40ff0f0 0x7f0ed4195d30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0005b40 tx=0x7f0ed0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:30.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.131+0000 7f0ecaffd700 1 -- 172.21.15.152:0/2796478581 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ed001d070 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.131+0000 7f0ecaffd700 1 -- 172.21.15.152:0/2796478581 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ed00044d0 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.131+0000 7f0edbc6c700 1 -- 172.21.15.152:0/2796478581 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed4196470 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.131+0000 7f0ecaffd700 1 -- 172.21.15.152:0/2796478581 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ed0003930 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.131+0000 7f0edbc6c700 1 -- 172.21.15.152:0/2796478581 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed4196830 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.132+0000 7f0edbc6c700 1 -- 172.21.15.152:0/2796478581 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed40623c0 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.132+0000 7f0ecaffd700 1 -- 172.21.15.152:0/2796478581 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f0ed000b960 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.132+0000 7f0ecaffd700 1 --2- 172.21.15.152:0/2796478581 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0ec00384b0 0x7f0ec003a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:30.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.132+0000 7f0ecaffd700 1 -- 172.21.15.152:0/2796478581 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f0ed004c5d0 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.132+0000 7f0ed9207700 1 --2- 172.21.15.152:0/2796478581 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0ec00384b0 0x7f0ec003a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:30.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.133+0000 7f0ed9207700 1 --2- 172.21.15.152:0/2796478581 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0ec00384b0 0x7f0ec003a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4006fd0 tx=0x7f0ec4006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:30.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.137+0000 7f0ecaffd700 1 -- 172.21.15.152:0/2796478581 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0ed002a540 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.433+0000 7f0edbc6c700 1 -- 172.21.15.152:0/2796478581 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0ed418fb80 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.433+0000 7f0ecaffd700 1 -- 172.21.15.152:0/2796478581 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f0ed002a360 con 0x7f0ed40ff0f0 2024-03-29T04:37:30.424 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:37:30.424 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:37:30.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.436+0000 7f0edbc6c700 1 -- 172.21.15.152:0/2796478581 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0ec00384b0 msgr2=0x7f0ec003a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:30.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.436+0000 7f0edbc6c700 1 --2- 172.21.15.152:0/2796478581 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0ec00384b0 0x7f0ec003a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4006fd0 tx=0x7f0ec4006e40 comp rx=0 tx=0).stop 2024-03-29T04:37:30.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.436+0000 7f0edbc6c700 1 -- 172.21.15.152:0/2796478581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed40ff0f0 msgr2=0x7f0ed4195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:30.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.436+0000 7f0edbc6c700 1 --2- 172.21.15.152:0/2796478581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed40ff0f0 0x7f0ed4195d30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0005b40 tx=0x7f0ed0004c80 comp rx=0 tx=0).stop 2024-03-29T04:37:30.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.436+0000 7f0edbc6c700 1 -- 172.21.15.152:0/2796478581 shutdown_connections 2024-03-29T04:37:30.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.436+0000 7f0edbc6c700 1 --2- 172.21.15.152:0/2796478581 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0ec00384b0 0x7f0ec003a970 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:30.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.436+0000 7f0edbc6c700 1 --2- 172.21.15.152:0/2796478581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ed40ff0f0 0x7f0ed4195d30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:30.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.436+0000 7f0edbc6c700 1 -- 172.21.15.152:0/2796478581 >> 172.21.15.152:0/2796478581 conn(0x7f0ed4075e80 msgr2=0x7f0ed4106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:30.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.436+0000 7f0edbc6c700 1 -- 172.21.15.152:0/2796478581 shutdown_connections 2024-03-29T04:37:30.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:30.436+0000 7f0edbc6c700 1 -- 172.21.15.152:0/2796478581 wait complete. 2024-03-29T04:37:30.426 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:37:30.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:30 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/2796478581' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:32.052 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:37:32.052 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:37:32.271 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:37:32.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:32.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:32.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:32.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:32.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:32 smithi142 ceph-mon[29027]: Deploying daemon node-exporter.smithi152 on smithi152 2024-03-29T04:37:33.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:33.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:37:33.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.840+0000 7f25b5364700 1 -- 172.21.15.152:0/1707259517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b0105740 msgr2=0x7f25b0105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:33.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.840+0000 7f25b5364700 1 --2- 172.21.15.152:0/1707259517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b0105740 0x7f25b0105b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f25a000b0a0 tx=0x7f25a000b3b0 comp rx=0 tx=0).stop 2024-03-29T04:37:33.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.840+0000 7f25b5364700 1 -- 172.21.15.152:0/1707259517 shutdown_connections 2024-03-29T04:37:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.840+0000 7f25b5364700 1 --2- 172.21.15.152:0/1707259517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b0105740 0x7f25b0105b20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.840+0000 7f25b5364700 1 -- 172.21.15.152:0/1707259517 >> 172.21.15.152:0/1707259517 conn(0x7f25b00fb250 msgr2=0x7f25b00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.840+0000 7f25b5364700 1 -- 172.21.15.152:0/1707259517 shutdown_connections 2024-03-29T04:37:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.840+0000 7f25b5364700 1 -- 172.21.15.152:0/1707259517 wait complete. 2024-03-29T04:37:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.840+0000 7f25b5364700 1 Processor -- start 2024-03-29T04:37:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.840+0000 7f25b5364700 1 -- start start 2024-03-29T04:37:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.841+0000 7f25b5364700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b0105740 0x7f25b0196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.841+0000 7f25b5364700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b01970d0 con 0x7f25b0105740 2024-03-29T04:37:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.841+0000 7f25aeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b0105740 0x7f25b0196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.841+0000 7f25aeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b0105740 0x7f25b0196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:41168/0 (socket says 172.21.15.152:41168) 2024-03-29T04:37:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.841+0000 7f25aeffd700 1 -- 172.21.15.152:0/1412435411 learned_addr learned my addr 172.21.15.152:0/1412435411 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:37:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.841+0000 7f25aeffd700 1 -- 172.21.15.152:0/1412435411 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0009d00 con 0x7f25b0105740 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.841+0000 7f25aeffd700 1 --2- 172.21.15.152:0/1412435411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b0105740 0x7f25b0196b90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f25a000d010 tx=0x7f25a0003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.841+0000 7f25a7fff700 1 -- 172.21.15.152:0/1412435411 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25a000e040 con 0x7f25b0105740 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.841+0000 7f25b5364700 1 -- 172.21.15.152:0/1412435411 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b01972d0 con 0x7f25b0105740 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.841+0000 7f25b5364700 1 -- 172.21.15.152:0/1412435411 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b0197750 con 0x7f25b0105740 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.841+0000 7f25a7fff700 1 -- 172.21.15.152:0/1412435411 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f25a00044f0 con 0x7f25b0105740 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.841+0000 7f25a7fff700 1 -- 172.21.15.152:0/1412435411 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25a001d900 con 0x7f25b0105740 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.842+0000 7f25b5364700 1 -- 172.21.15.152:0/1412435411 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b01039f0 con 0x7f25b0105740 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.842+0000 7f25a7fff700 1 -- 172.21.15.152:0/1412435411 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f25a0004660 con 0x7f25b0105740 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.843+0000 7f25a7fff700 1 --2- 172.21.15.152:0/1412435411 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f259c0384b0 0x7f259c03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.843+0000 7f25a7fff700 1 -- 172.21.15.152:0/1412435411 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f25a004c440 con 0x7f25b0105740 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.843+0000 7f25ae7fc700 1 --2- 172.21.15.152:0/1412435411 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f259c0384b0 0x7f259c03a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.843+0000 7f25ae7fc700 1 --2- 172.21.15.152:0/1412435411 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f259c0384b0 0x7f259c03a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2598006fd0 tx=0x7f2598006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:33.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:33.848+0000 7f25a7fff700 1 -- 172.21.15.152:0/1412435411 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f25a0025540 con 0x7f25b0105740 2024-03-29T04:37:34.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:34.120+0000 7f25b5364700 1 -- 172.21.15.152:0/1412435411 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f25b00623c0 con 0x7f25b0105740 2024-03-29T04:37:34.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:34.121+0000 7f25a7fff700 1 -- 172.21.15.152:0/1412435411 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f25a0025360 con 0x7f25b0105740 2024-03-29T04:37:34.111 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:37:34.111 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:37:34.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:34.123+0000 7f25b5364700 1 -- 172.21.15.152:0/1412435411 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f259c0384b0 msgr2=0x7f259c03a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:34.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:34.123+0000 7f25b5364700 1 --2- 172.21.15.152:0/1412435411 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f259c0384b0 0x7f259c03a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2598006fd0 tx=0x7f2598006e40 comp rx=0 tx=0).stop 2024-03-29T04:37:34.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:34.123+0000 7f25b5364700 1 -- 172.21.15.152:0/1412435411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b0105740 msgr2=0x7f25b0196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:34.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:34.123+0000 7f25b5364700 1 --2- 172.21.15.152:0/1412435411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b0105740 0x7f25b0196b90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f25a000d010 tx=0x7f25a0003ad0 comp rx=0 tx=0).stop 2024-03-29T04:37:34.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:34.123+0000 7f25b5364700 1 -- 172.21.15.152:0/1412435411 shutdown_connections 2024-03-29T04:37:34.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:34.123+0000 7f25b5364700 1 --2- 172.21.15.152:0/1412435411 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f259c0384b0 0x7f259c03a970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:34.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:34.123+0000 7f25b5364700 1 --2- 172.21.15.152:0/1412435411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25b0105740 0x7f25b0196b90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:34.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:34.123+0000 7f25b5364700 1 -- 172.21.15.152:0/1412435411 >> 172.21.15.152:0/1412435411 conn(0x7f25b00fb250 msgr2=0x7f25b0107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:34.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:34.123+0000 7f25b5364700 1 -- 172.21.15.152:0/1412435411 shutdown_connections 2024-03-29T04:37:34.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:34.123+0000 7f25b5364700 1 -- 172.21.15.152:0/1412435411 wait complete. 2024-03-29T04:37:34.114 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:37:35.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:34 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/1412435411' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:36.641 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:37:36.642 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:37:37.361 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:37 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:37.361 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:37 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:37.361 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:37 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:37.361 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:37 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:37.361 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:37 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi152.usoolh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:37:37.361 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:37 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi152.usoolh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-29T04:37:37.361 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:37 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:37:37.361 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:37 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:37.361 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:37 smithi142 ceph-mon[29027]: Deploying daemon mgr.smithi152.usoolh on smithi152 2024-03-29T04:37:37.618 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:37:38.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:38 smithi142 ceph-mon[29027]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:38.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:39.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.126+0000 7fb7615d4700 1 -- 172.21.15.152:0/736862115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c0ff120 msgr2=0x7fb75c0ff500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:39.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.126+0000 7fb7615d4700 1 --2- 172.21.15.152:0/736862115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c0ff120 0x7fb75c0ff500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009910 tx=0x7fb74c009c20 comp rx=0 tx=0).stop 2024-03-29T04:37:39.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.126+0000 7fb7615d4700 1 -- 172.21.15.152:0/736862115 shutdown_connections 2024-03-29T04:37:39.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.126+0000 7fb7615d4700 1 --2- 172.21.15.152:0/736862115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c0ff120 0x7fb75c0ff500 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:39.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.126+0000 7fb7615d4700 1 -- 172.21.15.152:0/736862115 >> 172.21.15.152:0/736862115 conn(0x7fb75c075df0 msgr2=0x7fb75c076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:39.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.126+0000 7fb7615d4700 1 -- 172.21.15.152:0/736862115 shutdown_connections 2024-03-29T04:37:39.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.126+0000 7fb7615d4700 1 -- 172.21.15.152:0/736862115 wait complete. 2024-03-29T04:37:39.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.126+0000 7fb7615d4700 1 Processor -- start 2024-03-29T04:37:39.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.126+0000 7fb7615d4700 1 -- start start 2024-03-29T04:37:39.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.127+0000 7fb7615d4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c0ff120 0x7fb75c100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:39.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.127+0000 7fb7615d4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c102b30 con 0x7fb75c0ff120 2024-03-29T04:37:39.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.127+0000 7fb75affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c0ff120 0x7fb75c100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:39.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.127+0000 7fb75affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c0ff120 0x7fb75c100bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.152:46566/0 (socket says 172.21.15.152:46566) 2024-03-29T04:37:39.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.127+0000 7fb75affd700 1 -- 172.21.15.152:0/514137911 learned_addr learned my addr 172.21.15.152:0/514137911 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:37:39.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.127+0000 7fb75affd700 1 -- 172.21.15.152:0/514137911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c0095f0 con 0x7fb75c0ff120 2024-03-29T04:37:39.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.127+0000 7fb75affd700 1 --2- 172.21.15.152:0/514137911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c0ff120 0x7fb75c100bb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb74c005b40 tx=0x7fb74c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:39.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.127+0000 7fb743fff700 1 -- 172.21.15.152:0/514137911 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb74c01d070 con 0x7fb75c0ff120 2024-03-29T04:37:39.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.127+0000 7fb7615d4700 1 -- 172.21.15.152:0/514137911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75c1010f0 con 0x7fb75c0ff120 2024-03-29T04:37:39.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.127+0000 7fb743fff700 1 -- 172.21.15.152:0/514137911 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb74c0044d0 con 0x7fb75c0ff120 2024-03-29T04:37:39.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.127+0000 7fb7615d4700 1 -- 172.21.15.152:0/514137911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75c101570 con 0x7fb75c0ff120 2024-03-29T04:37:39.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.127+0000 7fb743fff700 1 -- 172.21.15.152:0/514137911 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb74c003930 con 0x7fb75c0ff120 2024-03-29T04:37:39.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.128+0000 7fb7615d4700 1 -- 172.21.15.152:0/514137911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75c0623c0 con 0x7fb75c0ff120 2024-03-29T04:37:39.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.128+0000 7fb743fff700 1 -- 172.21.15.152:0/514137911 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fb74c003c70 con 0x7fb75c0ff120 2024-03-29T04:37:39.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.129+0000 7fb743fff700 1 --2- 172.21.15.152:0/514137911 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb7440384b0 0x7fb74403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:39.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.129+0000 7fb743fff700 1 -- 172.21.15.152:0/514137911 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb74c04c4f0 con 0x7fb75c0ff120 2024-03-29T04:37:39.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.129+0000 7fb75a7fc700 1 --2- 172.21.15.152:0/514137911 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb7440384b0 0x7fb74403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:39.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.129+0000 7fb75a7fc700 1 --2- 172.21.15.152:0/514137911 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb7440384b0 0x7fb74403a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb750006fd0 tx=0x7fb750006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:39.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.134+0000 7fb743fff700 1 -- 172.21.15.152:0/514137911 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb74c02a540 con 0x7fb75c0ff120 2024-03-29T04:37:39.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.402+0000 7fb7615d4700 1 -- 172.21.15.152:0/514137911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb75c101940 con 0x7fb75c0ff120 2024-03-29T04:37:39.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.403+0000 7fb743fff700 1 -- 172.21.15.152:0/514137911 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fb74c02a360 con 0x7fb75c0ff120 2024-03-29T04:37:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:37:39.395 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:35:04.327053Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T04:37:39.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.405+0000 7fb7615d4700 1 -- 172.21.15.152:0/514137911 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb7440384b0 msgr2=0x7fb74403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:39.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.405+0000 7fb7615d4700 1 --2- 172.21.15.152:0/514137911 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb7440384b0 0x7fb74403a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb750006fd0 tx=0x7fb750006e40 comp rx=0 tx=0).stop 2024-03-29T04:37:39.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.405+0000 7fb7615d4700 1 -- 172.21.15.152:0/514137911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c0ff120 msgr2=0x7fb75c100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:39.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.405+0000 7fb7615d4700 1 --2- 172.21.15.152:0/514137911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c0ff120 0x7fb75c100bb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb74c005b40 tx=0x7fb74c004c80 comp rx=0 tx=0).stop 2024-03-29T04:37:39.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.405+0000 7fb7615d4700 1 -- 172.21.15.152:0/514137911 shutdown_connections 2024-03-29T04:37:39.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.405+0000 7fb7615d4700 1 --2- 172.21.15.152:0/514137911 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb7440384b0 0x7fb74403a970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:39.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.405+0000 7fb7615d4700 1 --2- 172.21.15.152:0/514137911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c0ff120 0x7fb75c100bb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:39.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.405+0000 7fb7615d4700 1 -- 172.21.15.152:0/514137911 >> 172.21.15.152:0/514137911 conn(0x7fb75c075df0 msgr2=0x7fb75c106c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:39.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.405+0000 7fb7615d4700 1 -- 172.21.15.152:0/514137911 shutdown_connections 2024-03-29T04:37:39.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:39.405+0000 7fb7615d4700 1 -- 172.21.15.152:0/514137911 wait complete. 2024-03-29T04:37:39.397 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-29T04:37:40.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:39 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/514137911' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:40.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:39 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:40.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:39 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:41.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:40 smithi142 ceph-mon[29027]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:41.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:40 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:41.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:40 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:41.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:40 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:37:41.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:40 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:41.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:40 smithi142 ceph-mon[29027]: Deploying daemon mon.smithi152 on smithi152 2024-03-29T04:37:41.175 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T04:37:41.175 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-29T04:37:42.475 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi152/config 2024-03-29T04:37:43.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:42 smithi142 ceph-mon[29027]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:43.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:43 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:44.316 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:44 smithi152 ceph-mon[37177]: expand_channel_meta expanded map: {default=debug} 2024-03-29T04:37:44.588 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:44 smithi152 bash[37130]: defcc70499d901aaa1b677159d548a85d392b6552f960844aa4202bae16020e3 2024-03-29T04:37:44.588 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:44 smithi152 systemd[1]: Started Ceph mon.smithi152 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:37:49.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:37:49.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: mon.smithi142 calling monitor election 2024-03-29T04:37:49.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:49.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:49.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:49.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:49.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: mon.smithi152 calling monitor election 2024-03-29T04:37:49.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/2970912792' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/crt"}]: dispatch 2024-03-29T04:37:49.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:49.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:49.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:37:49.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:49.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:49.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: mon.smithi142 is new leader, mons smithi142,smithi152 in quorum (ranks 0,1) 2024-03-29T04:37:49.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: monmap e2: 2 mons at {smithi142=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],smithi152=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-29T04:37:49.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: fsmap 2024-03-29T04:37:49.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: osdmap e5: 0 total, 0 up, 0 in 2024-03-29T04:37:49.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: mgrmap e17: smithi142.swesrg(active, since 31s) 2024-03-29T04:37:49.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: Standby manager daemon smithi152.usoolh started 2024-03-29T04:37:49.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/2970912792' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:37:49.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: overall HEALTH_OK 2024-03-29T04:37:49.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/2970912792' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/key"}]: dispatch 2024-03-29T04:37:49.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/2970912792' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:37:49.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:49.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:49.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:49.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:49.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:49.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:37:49.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: mon.smithi142 calling monitor election 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: mon.smithi152 calling monitor election 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/2970912792' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/crt"}]: dispatch 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: mon.smithi142 is new leader, mons smithi142,smithi152 in quorum (ranks 0,1) 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: monmap e2: 2 mons at {smithi142=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],smithi152=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-29T04:37:49.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: fsmap 2024-03-29T04:37:49.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: osdmap e5: 0 total, 0 up, 0 in 2024-03-29T04:37:49.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: mgrmap e17: smithi142.swesrg(active, since 31s) 2024-03-29T04:37:49.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: Standby manager daemon smithi152.usoolh started 2024-03-29T04:37:49.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/2970912792' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:37:49.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: overall HEALTH_OK 2024-03-29T04:37:49.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/2970912792' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/key"}]: dispatch 2024-03-29T04:37:49.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/2970912792' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:37:49.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:49.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:49.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:49.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:49.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:50.438 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:50 smithi152 ceph-mon[37177]: mgrmap e18: smithi142.swesrg(active, since 31s), standbys: smithi152.usoolh 2024-03-29T04:37:50.439 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:50 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi152.usoolh", "id": "smithi152.usoolh"}]: dispatch 2024-03-29T04:37:50.439 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:50 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:37:50.439 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:50 smithi152 ceph-mon[37177]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:50.439 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:50 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:50.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.825+0000 7f597e927700 1 -- 172.21.15.152:0/3694465977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59600062a0 msgr2=0x7f5960008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:50.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.825+0000 7f597e927700 1 --2- 172.21.15.152:0/3694465977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59600062a0 0x7f5960008710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f59781019f0 tx=0x7f596800fe90 comp rx=0 tx=0).stop 2024-03-29T04:37:50.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.826+0000 7f597e927700 1 -- 172.21.15.152:0/3694465977 shutdown_connections 2024-03-29T04:37:50.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.826+0000 7f597e927700 1 --2- 172.21.15.152:0/3694465977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59600062a0 0x7f5960008710 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:50.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.826+0000 7f597e927700 1 -- 172.21.15.152:0/3694465977 >> 172.21.15.152:0/3694465977 conn(0x7f59780fc400 msgr2=0x7f59780fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:50.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.826+0000 7f597e927700 1 -- 172.21.15.152:0/3694465977 shutdown_connections 2024-03-29T04:37:50.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.826+0000 7f597e927700 1 -- 172.21.15.152:0/3694465977 wait complete. 2024-03-29T04:37:50.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.826+0000 7f597e927700 1 Processor -- start 2024-03-29T04:37:50.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.826+0000 7f597e927700 1 -- start start 2024-03-29T04:37:50.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f597e927700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59600062a0 0x7f5978195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:50.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f597e927700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5978196520 0x7f597819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:50.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f597e927700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5978196b40 con 0x7f59600062a0 2024-03-29T04:37:50.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f597e927700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5978196cb0 con 0x7f5978196520 2024-03-29T04:37:50.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f59777fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5978196520 0x7f597819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:50.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f59777fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5978196520 0x7f597819a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37976/0 (socket says 172.21.15.152:37976) 2024-03-29T04:37:50.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f59777fe700 1 -- 172.21.15.152:0/1762653272 learned_addr learned my addr 172.21.15.152:0/1762653272 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:37:50.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f5977fff700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59600062a0 0x7f5978195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:50.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f59777fe700 1 -- 172.21.15.152:0/1762653272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5978196520 msgr2=0x7f597819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-29T04:37:50.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f59777fe700 1 -- 172.21.15.152:0/1762653272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5978196520 msgr2=0x7f597819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-29T04:37:50.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f59777fe700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5978196520 0x7f597819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:37:50.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f59777fe700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5978196520 0x7f597819a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:37:50.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f5977fff700 1 -- 172.21.15.152:0/1762653272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5978196520 msgr2=0x7f597819a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:37:50.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f5977fff700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5978196520 0x7f597819a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:50.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.827+0000 7f5977fff700 1 -- 172.21.15.152:0/1762653272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596800f8e0 con 0x7f59600062a0 2024-03-29T04:37:50.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.828+0000 7f5977fff700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59600062a0 0x7f5978195fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f59781019f0 tx=0x7f596801d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:50.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.828+0000 7f59757fa700 1 -- 172.21.15.152:0/1762653272 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5968005da0 con 0x7f59600062a0 2024-03-29T04:37:50.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.828+0000 7f59757fa700 1 -- 172.21.15.152:0/1762653272 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f596801dc10 con 0x7f59600062a0 2024-03-29T04:37:50.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.828+0000 7f597e927700 1 -- 172.21.15.152:0/1762653272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597819af90 con 0x7f59600062a0 2024-03-29T04:37:50.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.828+0000 7f59757fa700 1 -- 172.21.15.152:0/1762653272 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5968011830 con 0x7f59600062a0 2024-03-29T04:37:50.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.828+0000 7f597e927700 1 -- 172.21.15.152:0/1762653272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597819b4e0 con 0x7f59600062a0 2024-03-29T04:37:50.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.829+0000 7f597e927700 1 -- 172.21.15.152:0/1762653272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59780623c0 con 0x7f59600062a0 2024-03-29T04:37:50.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.829+0000 7f59757fa700 1 -- 172.21.15.152:0/1762653272 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f5968003680 con 0x7f59600062a0 2024-03-29T04:37:50.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.830+0000 7f59757fa700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f596406c560 0x7f596406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:50.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.830+0000 7f59757fa700 1 -- 172.21.15.152:0/1762653272 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f596808d690 con 0x7f59600062a0 2024-03-29T04:37:50.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.830+0000 7f59777fe700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f596406c560 0x7f596406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:50.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.830+0000 7f59777fe700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f596406c560 0x7f596406ea20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f596c009180 tx=0x7f596c008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:50.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:50.835+0000 7f59757fa700 1 -- 172.21.15.152:0/1762653272 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5968059000 con 0x7f59600062a0 2024-03-29T04:37:50.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:50 smithi142 ceph-mon[29027]: mgrmap e18: smithi142.swesrg(active, since 31s), standbys: smithi152.usoolh 2024-03-29T04:37:50.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:50 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi152.usoolh", "id": "smithi152.usoolh"}]: dispatch 2024-03-29T04:37:50.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:50 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:37:50.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:50 smithi142 ceph-mon[29027]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:50.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:50 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:37:51.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.110+0000 7f597e927700 1 -- 172.21.15.152:0/1762653272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f59781a0c30 con 0x7f59600062a0 2024-03-29T04:37:51.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.110+0000 7f59757fa700 1 -- 172.21.15.152:0/1762653272 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1026 (secure 0 0 0) 0x7f5968035420 con 0x7f59600062a0 2024-03-29T04:37:51.101 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-29T04:37:51.101 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":2,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","modified":"2024-03-29T04:37:44.323694Z","created":"2024-03-29T04:35:04.327053Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi142","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi152","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-29T04:37:51.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.112+0000 7f597e927700 1 -- 172.21.15.152:0/1762653272 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f596406c560 msgr2=0x7f596406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:51.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.112+0000 7f597e927700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f596406c560 0x7f596406ea20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f596c009180 tx=0x7f596c008040 comp rx=0 tx=0).stop 2024-03-29T04:37:51.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.112+0000 7f597e927700 1 -- 172.21.15.152:0/1762653272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59600062a0 msgr2=0x7f5978195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:51.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.112+0000 7f597e927700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59600062a0 0x7f5978195fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f59781019f0 tx=0x7f596801d9b0 comp rx=0 tx=0).stop 2024-03-29T04:37:51.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.112+0000 7f597e927700 1 -- 172.21.15.152:0/1762653272 shutdown_connections 2024-03-29T04:37:51.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.112+0000 7f597e927700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f596406c560 0x7f596406ea20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:51.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.112+0000 7f597e927700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59600062a0 0x7f5978195fe0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:51.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.112+0000 7f597e927700 1 --2- 172.21.15.152:0/1762653272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5978196520 0x7f597819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:51.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.112+0000 7f597e927700 1 -- 172.21.15.152:0/1762653272 >> 172.21.15.152:0/1762653272 conn(0x7f59780fc400 msgr2=0x7f5978102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:51.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.113+0000 7f597e927700 1 -- 172.21.15.152:0/1762653272 shutdown_connections 2024-03-29T04:37:51.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:37:51.113+0000 7f597e927700 1 -- 172.21.15.152:0/1762653272 wait complete. 2024-03-29T04:37:51.104 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 2 2024-03-29T04:37:51.392 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:51 smithi152 ceph-mon[37177]: from='client.? 172.21.15.152:0/1762653272' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:51.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:51 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/1762653272' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T04:37:52.102 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-29T04:37:52.102 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-29T04:37:52.317 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:37:52.706 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.706 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.706 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:52.706 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:37:52.706 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-29T04:37:52.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-29T04:37:52.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:52.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:37:52.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:37:52.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:37:52.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T04:37:52.707 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:52.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:52.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:37:52.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-29T04:37:52.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-29T04:37:52.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:52.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:37:52.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:37:52.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:52.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:37:52.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T04:37:52.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:53.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:53 smithi142 ceph-mon[29027]: Reconfiguring mon.smithi142 (unknown last config time)... 2024-03-29T04:37:53.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:53 smithi142 ceph-mon[29027]: Reconfiguring daemon mon.smithi142 on smithi142 2024-03-29T04:37:53.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:53 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:54.030 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:53 smithi152 ceph-mon[37177]: Reconfiguring mon.smithi142 (unknown last config time)... 2024-03-29T04:37:54.030 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:53 smithi152 ceph-mon[37177]: Reconfiguring daemon mon.smithi142 on smithi142 2024-03-29T04:37:54.030 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:53 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:54.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.481+0000 7fbaf05c6700 1 -- 172.21.15.142:0/1039795708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae81019f0 msgr2=0x7fbae8101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:54.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.481+0000 7fbaf05c6700 1 --2- 172.21.15.142:0/1039795708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae81019f0 0x7fbae8101dd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009910 tx=0x7fbae4009c20 comp rx=0 tx=0).stop 2024-03-29T04:37:54.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.481+0000 7fbaf05c6700 1 -- 172.21.15.142:0/1039795708 shutdown_connections 2024-03-29T04:37:54.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.481+0000 7fbaf05c6700 1 --2- 172.21.15.142:0/1039795708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae81019f0 0x7fbae8101dd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:54.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.481+0000 7fbaf05c6700 1 -- 172.21.15.142:0/1039795708 >> 172.21.15.142:0/1039795708 conn(0x7fbae80fb240 msgr2=0x7fbae80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:54.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.481+0000 7fbaf05c6700 1 -- 172.21.15.142:0/1039795708 shutdown_connections 2024-03-29T04:37:54.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.481+0000 7fbaf05c6700 1 -- 172.21.15.142:0/1039795708 wait complete. 2024-03-29T04:37:54.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaf05c6700 1 Processor -- start 2024-03-29T04:37:54.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaf05c6700 1 -- start start 2024-03-29T04:37:54.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaf05c6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae81019f0 0x7fbae8195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:54.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaf05c6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae8196440 0x7fbae819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:54.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaf05c6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae8196a60 con 0x7fbae81019f0 2024-03-29T04:37:54.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaf05c6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae8196bd0 con 0x7fbae8196440 2024-03-29T04:37:54.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaee362700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae81019f0 0x7fbae8195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:54.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaee362700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae81019f0 0x7fbae8195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:49184/0 (socket says 172.21.15.142:49184) 2024-03-29T04:37:54.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaee362700 1 -- 172.21.15.142:0/62890018 learned_addr learned my addr 172.21.15.142:0/62890018 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:37:54.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaee362700 1 -- 172.21.15.142:0/62890018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae8196440 msgr2=0x7fbae819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:54.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaee362700 1 --2- 172.21.15.142:0/62890018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae8196440 0x7fbae819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:54.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaee362700 1 -- 172.21.15.142:0/62890018 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae40095f0 con 0x7fbae81019f0 2024-03-29T04:37:54.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbaee362700 1 --2- 172.21.15.142:0/62890018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae81019f0 0x7fbae8195f00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbae4005b40 tx=0x7fbae400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:54.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.482+0000 7fbadf7fe700 1 -- 172.21.15.142:0/62890018 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbae401d070 con 0x7fbae81019f0 2024-03-29T04:37:54.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.483+0000 7fbadf7fe700 1 -- 172.21.15.142:0/62890018 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbae4005140 con 0x7fbae81019f0 2024-03-29T04:37:54.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.483+0000 7fbadf7fe700 1 -- 172.21.15.142:0/62890018 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbae4017440 con 0x7fbae81019f0 2024-03-29T04:37:54.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.483+0000 7fbaf05c6700 1 -- 172.21.15.142:0/62890018 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae819ae50 con 0x7fbae81019f0 2024-03-29T04:37:54.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.483+0000 7fbaf05c6700 1 -- 172.21.15.142:0/62890018 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae819b3a0 con 0x7fbae81019f0 2024-03-29T04:37:54.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.484+0000 7fbaf05c6700 1 -- 172.21.15.142:0/62890018 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae8108000 con 0x7fbae81019f0 2024-03-29T04:37:54.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.484+0000 7fbadf7fe700 1 -- 172.21.15.142:0/62890018 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fbae400f6f0 con 0x7fbae81019f0 2024-03-29T04:37:54.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.484+0000 7fbadf7fe700 1 --2- 172.21.15.142:0/62890018 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbad406c5c0 0x7fbad406ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:37:54.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.484+0000 7fbadf7fe700 1 -- 172.21.15.142:0/62890018 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fbae408bc10 con 0x7fbae81019f0 2024-03-29T04:37:54.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.484+0000 7fbaedb61700 1 --2- 172.21.15.142:0/62890018 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbad406c5c0 0x7fbad406ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:37:54.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.485+0000 7fbaedb61700 1 --2- 172.21.15.142:0/62890018 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbad406c5c0 0x7fbad406ea80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbad80098e0 tx=0x7fbad8006dd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:37:54.479 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.489+0000 7fbadf7fe700 1 -- 172.21.15.142:0/62890018 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbae40574d0 con 0x7fbae81019f0 2024-03-29T04:37:54.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.684+0000 7fbaf05c6700 1 -- 172.21.15.142:0/62890018 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fbae819b660 con 0x7fbae81019f0 2024-03-29T04:37:54.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.684+0000 7fbadf7fe700 1 -- 172.21.15.142:0/62890018 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7fbae408b3d0 con 0x7fbae81019f0 2024-03-29T04:37:54.675 INFO:teuthology.orchestra.run.smithi142.stdout:# minimal ceph.conf for 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:37:54.675 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-03-29T04:37:54.675 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:37:54.675 INFO:teuthology.orchestra.run.smithi142.stdout: mon_host = [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-29T04:37:54.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.686+0000 7fbaf05c6700 1 -- 172.21.15.142:0/62890018 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbad406c5c0 msgr2=0x7fbad406ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:54.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.686+0000 7fbaf05c6700 1 --2- 172.21.15.142:0/62890018 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbad406c5c0 0x7fbad406ea80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbad80098e0 tx=0x7fbad8006dd0 comp rx=0 tx=0).stop 2024-03-29T04:37:54.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.686+0000 7fbaf05c6700 1 -- 172.21.15.142:0/62890018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae81019f0 msgr2=0x7fbae8195f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:37:54.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.686+0000 7fbaf05c6700 1 --2- 172.21.15.142:0/62890018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae81019f0 0x7fbae8195f00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbae4005b40 tx=0x7fbae400bda0 comp rx=0 tx=0).stop 2024-03-29T04:37:54.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.686+0000 7fbaf05c6700 1 -- 172.21.15.142:0/62890018 shutdown_connections 2024-03-29T04:37:54.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.686+0000 7fbaf05c6700 1 --2- 172.21.15.142:0/62890018 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbad406c5c0 0x7fbad406ea80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:54.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.686+0000 7fbaf05c6700 1 --2- 172.21.15.142:0/62890018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbae81019f0 0x7fbae8195f00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:54.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.686+0000 7fbaf05c6700 1 --2- 172.21.15.142:0/62890018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae8196440 0x7fbae819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:37:54.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.686+0000 7fbaf05c6700 1 -- 172.21.15.142:0/62890018 >> 172.21.15.142:0/62890018 conn(0x7fbae80fb240 msgr2=0x7fbae8105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:37:54.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.687+0000 7fbaf05c6700 1 -- 172.21.15.142:0/62890018 shutdown_connections 2024-03-29T04:37:54.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:37:54.687+0000 7fbaf05c6700 1 -- 172.21.15.142:0/62890018 wait complete. 2024-03-29T04:37:54.826 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:54 smithi142 ceph-mon[29027]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:55.030 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:54 smithi152 ceph-mon[37177]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:55.383 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-29T04:37:55.383 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:37:55.383 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-03-29T04:37:55.414 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:37:55.414 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:37:55.482 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:37:55.482 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.conf 2024-03-29T04:37:55.516 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:37:55.517 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:37:55.587 INFO:tasks.cephadm:Deploying OSDs... 2024-03-29T04:37:55.588 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:37:55.588 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T04:37:55.603 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T04:37:55.603 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-03-29T04:37:55.658 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T04:37:55.658 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:37:55.658 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-29T04:37:55.658 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:37:55.658 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:37:55.658 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-03-29 04:37:05.993619383 +0000 2024-03-29T04:37:55.658 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-03-29 04:29:33.156479961 +0000 2024-03-29T04:37:55.658 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-03-29 04:29:33.156479961 +0000 2024-03-29T04:37:55.658 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-03-29 04:29:33.156479961 +0000 2024-03-29T04:37:55.658 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T04:37:55.729 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-03-29T04:37:55.729 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-03-29T04:37:55.729 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000198256 s, 2.6 MB/s 2024-03-29T04:37:55.730 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T04:37:55.780 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:55 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/62890018' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:55.780 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:55 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:55.780 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:55 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:55.780 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:55 smithi152 ceph-mon[37177]: Reconfiguring mgr.smithi142.swesrg (unknown last config time)... 2024-03-29T04:37:55.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:55 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.swesrg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:37:55.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:55 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:37:55.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:55 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:55.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:55 smithi152 ceph-mon[37177]: Reconfiguring daemon mgr.smithi142.swesrg on smithi142 2024-03-29T04:37:55.787 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-03-29T04:37:55.844 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T04:37:55.844 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:37:55.844 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-29T04:37:55.844 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:37:55.844 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:37:55.844 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-03-29 04:37:05.993619383 +0000 2024-03-29T04:37:55.844 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-03-29 04:29:33.502473771 +0000 2024-03-29T04:37:55.844 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-03-29 04:29:33.502473771 +0000 2024-03-29T04:37:55.844 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-03-29 04:29:33.502473771 +0000 2024-03-29T04:37:55.845 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T04:37:55.905 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:55 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/62890018' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:55.905 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:55 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:55.906 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:55 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:55.906 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:55 smithi142 ceph-mon[29027]: Reconfiguring mgr.smithi142.swesrg (unknown last config time)... 2024-03-29T04:37:55.906 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:55 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.swesrg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:37:55.906 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:55 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:37:55.906 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:55 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:55.906 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:55 smithi142 ceph-mon[29027]: Reconfiguring daemon mgr.smithi142.swesrg on smithi142 2024-03-29T04:37:55.911 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-03-29T04:37:55.911 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-03-29T04:37:55.911 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000213146 s, 2.4 MB/s 2024-03-29T04:37:55.912 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T04:37:55.967 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-03-29T04:37:56.022 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T04:37:56.022 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:37:56.022 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-29T04:37:56.022 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:37:56.022 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:37:56.022 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-03-29 04:37:05.993619383 +0000 2024-03-29T04:37:56.022 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-03-29 04:29:33.834467832 +0000 2024-03-29T04:37:56.022 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-03-29 04:29:33.834467832 +0000 2024-03-29T04:37:56.022 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-03-29 04:29:33.834467832 +0000 2024-03-29T04:37:56.022 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T04:37:56.087 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-03-29T04:37:56.088 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-03-29T04:37:56.088 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.00017634 s, 2.9 MB/s 2024-03-29T04:37:56.089 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T04:37:56.144 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-03-29T04:37:56.199 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T04:37:56.199 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:37:56.199 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-29T04:37:56.200 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:37:56.200 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:37:56.200 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-03-29 04:37:05.994619365 +0000 2024-03-29T04:37:56.200 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-03-29 04:29:34.175461731 +0000 2024-03-29T04:37:56.200 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-03-29 04:29:34.175461731 +0000 2024-03-29T04:37:56.200 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-03-29 04:29:34.175461731 +0000 2024-03-29T04:37:56.200 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T04:37:56.265 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-03-29T04:37:56.265 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-03-29T04:37:56.265 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000191482 s, 2.7 MB/s 2024-03-29T04:37:56.267 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T04:37:56.323 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:37:56.323 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T04:37:56.342 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T04:37:56.342 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2024-03-29T04:37:56.398 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T04:37:56.398 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:37:56.398 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-29T04:37:56.398 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:37:56.398 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:37:56.398 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-03-29 04:37:42.801861903 +0000 2024-03-29T04:37:56.398 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-29 04:29:32.347567625 +0000 2024-03-29T04:37:56.398 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-29 04:29:32.347567625 +0000 2024-03-29T04:37:56.399 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-03-29 04:29:32.347567625 +0000 2024-03-29T04:37:56.399 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T04:37:56.465 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-03-29T04:37:56.465 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-03-29T04:37:56.465 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000237822 s, 2.2 MB/s 2024-03-29T04:37:56.467 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T04:37:56.523 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2024-03-29T04:37:56.579 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T04:37:56.579 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:37:56.579 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-29T04:37:56.580 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:37:56.580 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:37:56.580 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-03-29 04:37:42.801861903 +0000 2024-03-29T04:37:56.580 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-29 04:29:32.701563359 +0000 2024-03-29T04:37:56.580 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-29 04:29:32.701563359 +0000 2024-03-29T04:37:56.580 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-03-29 04:29:32.701563359 +0000 2024-03-29T04:37:56.580 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T04:37:56.646 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-03-29T04:37:56.647 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-03-29T04:37:56.647 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000205222 s, 2.5 MB/s 2024-03-29T04:37:56.648 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T04:37:56.704 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2024-03-29T04:37:56.759 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T04:37:56.759 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:37:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-29T04:37:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:37:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:37:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-03-29 04:37:42.802861891 +0000 2024-03-29T04:37:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-29 04:29:33.057559065 +0000 2024-03-29T04:37:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-29 04:29:33.057559065 +0000 2024-03-29T04:37:56.760 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-03-29 04:29:33.057559065 +0000 2024-03-29T04:37:56.761 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T04:37:56.824 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-03-29T04:37:56.824 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-03-29T04:37:56.825 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000228856 s, 2.2 MB/s 2024-03-29T04:37:56.826 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T04:37:56.882 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2024-03-29T04:37:56.937 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T04:37:56.937 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:37:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-29T04:37:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:37:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:37:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-03-29 04:37:42.802861891 +0000 2024-03-29T04:37:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-29 04:29:33.408554830 +0000 2024-03-29T04:37:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-29 04:29:33.408554830 +0000 2024-03-29T04:37:56.937 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-03-29 04:29:33.408554830 +0000 2024-03-29T04:37:56.938 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T04:37:57.000 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:56 smithi152 ceph-mon[37177]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:57.000 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:56 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:57.000 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:56 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:57.000 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:56 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:37:57.000 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:56 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:57.002 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-03-29T04:37:57.002 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-03-29T04:37:57.003 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000191385 s, 2.7 MB/s 2024-03-29T04:37:57.003 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T04:37:57.059 INFO:tasks.cephadm:Deploying osd.0 on smithi142 with /dev/vg_nvme/lv_4... 2024-03-29T04:37:57.060 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-29T04:37:57.187 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:56 smithi142 ceph-mon[29027]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:57.187 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:56 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:57.188 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:56 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:57.188 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:56 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:37:57.188 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:56 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:58.030 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:37:58.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:57 smithi142 ceph-mon[29027]: Reconfiguring ceph-exporter.smithi142 (monmap changed)... 2024-03-29T04:37:58.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:57 smithi142 ceph-mon[29027]: Reconfiguring daemon ceph-exporter.smithi142 on smithi142 2024-03-29T04:37:58.280 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:57 smithi152 ceph-mon[37177]: Reconfiguring ceph-exporter.smithi142 (monmap changed)... 2024-03-29T04:37:58.280 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:57 smithi152 ceph-mon[37177]: Reconfiguring daemon ceph-exporter.smithi142 on smithi142 2024-03-29T04:37:59.163 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:58 smithi142 ceph-mon[29027]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:59.163 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:58 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:59.163 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:58 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:59.163 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:58 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:37:59.163 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:58 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:37:59.281 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:58 smithi152 ceph-mon[37177]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:37:59.281 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:58 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:59.281 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:58 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:37:59.281 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:58 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:37:59.281 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:58 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:00.144 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:59 smithi142 ceph-mon[29027]: Reconfiguring crash.smithi142 (monmap changed)... 2024-03-29T04:38:00.144 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:37:59 smithi142 ceph-mon[29027]: Reconfiguring daemon crash.smithi142 on smithi142 2024-03-29T04:38:00.280 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:59 smithi152 ceph-mon[37177]: Reconfiguring crash.smithi142 (monmap changed)... 2024-03-29T04:38:00.281 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:37:59 smithi152 ceph-mon[37177]: Reconfiguring daemon crash.smithi142 on smithi142 2024-03-29T04:38:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-29T04:38:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-29T04:38:00.905 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-03-29T04:38:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-03-29T04:38:00.905 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013284 s, 789 MB/s 2024-03-29T04:38:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-03-29T04:38:01.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:00 smithi142 ceph-mon[29027]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:01.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:00 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:01.112 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:00 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:01.280 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:00 smithi152 ceph-mon[37177]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:01.281 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:00 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:01.281 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:00 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:01.658 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2024-03-29T04:38:02.148 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:01 smithi142 ceph-mon[29027]: Reconfiguring alertmanager.smithi142 (dependencies changed)... 2024-03-29T04:38:02.148 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:01 smithi142 ceph-mon[29027]: Reconfiguring daemon alertmanager.smithi142 on smithi142 2024-03-29T04:38:02.280 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:01 smithi152 ceph-mon[37177]: Reconfiguring alertmanager.smithi142 (dependencies changed)... 2024-03-29T04:38:02.280 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:01 smithi152 ceph-mon[37177]: Reconfiguring daemon alertmanager.smithi142 on smithi142 2024-03-29T04:38:02.986 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:02 smithi142 ceph-mon[29027]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:02.986 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:02 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:38:03.281 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:02 smithi152 ceph-mon[37177]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:03.281 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:02 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:38:03.742 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:38:04.612 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:04 smithi142 ceph-mon[29027]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:04.780 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:04 smithi152 ceph-mon[37177]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:05.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 -- 172.21.15.142:0/642978285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288101dc0 msgr2=0x7fb288102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:38:05.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 --2- 172.21.15.142:0/642978285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288101dc0 0x7fb288102220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb284009910 tx=0x7fb284009c20 comp rx=0 tx=0).stop 2024-03-29T04:38:05.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 -- 172.21.15.142:0/642978285 shutdown_connections 2024-03-29T04:38:05.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 --2- 172.21.15.142:0/642978285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288101dc0 0x7fb288102220 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:38:05.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 --2- 172.21.15.142:0/642978285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb288100bc0 0x7fb288100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:38:05.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 -- 172.21.15.142:0/642978285 >> 172.21.15.142:0/642978285 conn(0x7fb2880fc140 msgr2=0x7fb2880fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:38:05.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 -- 172.21.15.142:0/642978285 shutdown_connections 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 -- 172.21.15.142:0/642978285 wait complete. 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 Processor -- start 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 -- start start 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb288100bc0 0x7fb288196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288101dc0 0x7fb2881968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb288196ed0 con 0x7fb288101dc0 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.833+0000 7fb28fe50700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb288197010 con 0x7fb288100bc0 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb28d3eb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288101dc0 0x7fb2881968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb28d3eb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288101dc0 0x7fb2881968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45500/0 (socket says 172.21.15.142:45500) 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb28d3eb700 1 -- 172.21.15.142:0/3328449191 learned_addr learned my addr 172.21.15.142:0/3328449191 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb28dbec700 1 --2- 172.21.15.142:0/3328449191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb288100bc0 0x7fb288196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb28d3eb700 1 -- 172.21.15.142:0/3328449191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb288100bc0 msgr2=0x7fb288196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb28d3eb700 1 --2- 172.21.15.142:0/3328449191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb288100bc0 0x7fb288196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb28d3eb700 1 -- 172.21.15.142:0/3328449191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840095f0 con 0x7fb288101dc0 2024-03-29T04:38:05.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb28d3eb700 1 --2- 172.21.15.142:0/3328449191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288101dc0 0x7fb2881968b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb28400b5c0 tx=0x7fb28400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:38:05.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb27effd700 1 -- 172.21.15.142:0/3328449191 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb28401d070 con 0x7fb288101dc0 2024-03-29T04:38:05.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb27effd700 1 -- 172.21.15.142:0/3328449191 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb28400f460 con 0x7fb288101dc0 2024-03-29T04:38:05.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb27effd700 1 -- 172.21.15.142:0/3328449191 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb284021640 con 0x7fb288101dc0 2024-03-29T04:38:05.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb28fe50700 1 -- 172.21.15.142:0/3328449191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28819ba60 con 0x7fb288101dc0 2024-03-29T04:38:05.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.834+0000 7fb28fe50700 1 -- 172.21.15.142:0/3328449191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28819bf50 con 0x7fb288101dc0 2024-03-29T04:38:05.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.835+0000 7fb28fe50700 1 -- 172.21.15.142:0/3328449191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb288109720 con 0x7fb288101dc0 2024-03-29T04:38:05.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.836+0000 7fb27effd700 1 -- 172.21.15.142:0/3328449191 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fb28400fa60 con 0x7fb288101dc0 2024-03-29T04:38:05.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.836+0000 7fb27effd700 1 --2- 172.21.15.142:0/3328449191 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb27406c470 0x7fb27406e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:38:05.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.836+0000 7fb27effd700 1 -- 172.21.15.142:0/3328449191 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb28408ba00 con 0x7fb288101dc0 2024-03-29T04:38:05.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.836+0000 7fb28dbec700 1 --2- 172.21.15.142:0/3328449191 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb27406c470 0x7fb27406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:38:05.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.836+0000 7fb28dbec700 1 --2- 172.21.15.142:0/3328449191 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb27406c470 0x7fb27406e930 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb288101c20 tx=0x7fb27800b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:38:05.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:05.840+0000 7fb27effd700 1 -- 172.21.15.142:0/3328449191 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb28405ad50 con 0x7fb288101dc0 2024-03-29T04:38:06.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:06.042+0000 7fb28fe50700 1 -- 172.21.15.142:0/3328449191 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb288106710 con 0x7fb27406c470 2024-03-29T04:38:07.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:06 smithi152 ceph-mon[37177]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:07.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:06 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T04:38:07.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:06 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T04:38:07.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:06 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:07.092 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:06 smithi142 ceph-mon[29027]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:07.093 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:06 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T04:38:07.093 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:06 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T04:38:07.093 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:06 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:07.769 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:07 smithi142 ceph-mon[29027]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:38:07.770 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:07.770 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:08.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:07 smithi152 ceph-mon[37177]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:38:08.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:08.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:08.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:08 smithi142 ceph-mon[29027]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:08.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:08 smithi142 ceph-mon[29027]: Reconfiguring grafana.smithi142 (dependencies changed)... 2024-03-29T04:38:08.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:08 smithi142 ceph-mon[29027]: Reconfiguring daemon grafana.smithi142 on smithi142 2024-03-29T04:38:09.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:08 smithi152 ceph-mon[37177]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:09.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:08 smithi152 ceph-mon[37177]: Reconfiguring grafana.smithi142 (dependencies changed)... 2024-03-29T04:38:09.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:08 smithi152 ceph-mon[37177]: Reconfiguring daemon grafana.smithi142 on smithi142 2024-03-29T04:38:10.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:09 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/4140068886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d908de1a-bec3-4238-959a-f3f550431ae8"}]: dispatch 2024-03-29T04:38:10.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:09 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/4140068886' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d908de1a-bec3-4238-959a-f3f550431ae8"}]': finished 2024-03-29T04:38:10.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:09 smithi152 ceph-mon[37177]: osdmap e6: 1 total, 0 up, 1 in 2024-03-29T04:38:10.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:09 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:38:10.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:09 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2405427490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T04:38:10.112 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:09 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/4140068886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d908de1a-bec3-4238-959a-f3f550431ae8"}]: dispatch 2024-03-29T04:38:10.112 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:09 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/4140068886' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d908de1a-bec3-4238-959a-f3f550431ae8"}]': finished 2024-03-29T04:38:10.112 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:09 smithi142 ceph-mon[29027]: osdmap e6: 1 total, 0 up, 1 in 2024-03-29T04:38:10.112 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:09 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:38:10.112 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:09 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2405427490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T04:38:11.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:10 smithi152 ceph-mon[37177]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:11.112 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:10 smithi142 ceph-mon[29027]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:12.902 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:12 smithi142 ceph-mon[29027]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:13.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:12 smithi152 ceph-mon[37177]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:14.729 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:14 smithi142 ceph-mon[29027]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:14.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:14 smithi152 ceph-mon[37177]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:16.771 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:16 smithi142 ceph-mon[29027]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:17.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:16 smithi152 ceph-mon[37177]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:17.754 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:38:18.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:17 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:38:18.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:18 smithi142 ceph-mon[29027]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:18.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:18 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:18.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:18 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:19.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:18 smithi152 ceph-mon[37177]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:19.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:18 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:19.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:18 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:19.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:19 smithi142 ceph-mon[29027]: Reconfiguring prometheus.smithi142 (dependencies changed)... 2024-03-29T04:38:19.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:19 smithi142 ceph-mon[29027]: Reconfiguring daemon prometheus.smithi142 on smithi142 2024-03-29T04:38:19.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:19 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-29T04:38:19.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:19 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:20.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:19 smithi152 ceph-mon[37177]: Reconfiguring prometheus.smithi142 (dependencies changed)... 2024-03-29T04:38:20.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:19 smithi152 ceph-mon[37177]: Reconfiguring daemon prometheus.smithi142 on smithi142 2024-03-29T04:38:20.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:19 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-29T04:38:20.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:19 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:20.757 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:20 smithi142 ceph-mon[29027]: Deploying daemon osd.0 on smithi142 2024-03-29T04:38:20.757 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:20 smithi142 ceph-mon[29027]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:21.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:20 smithi152 ceph-mon[37177]: Deploying daemon osd.0 on smithi142 2024-03-29T04:38:21.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:20 smithi152 ceph-mon[37177]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:22.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:22 smithi142 ceph-mon[29027]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:23.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:22 smithi152 ceph-mon[37177]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:24.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:24 smithi142 ceph-mon[29027]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:24.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:24.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:24.848 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:24 smithi142 ceph-mon[29027]: Reconfiguring ceph-exporter.smithi152 (monmap changed)... 2024-03-29T04:38:24.848 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi152", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:38:24.848 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:24.848 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:24 smithi142 ceph-mon[29027]: Reconfiguring daemon ceph-exporter.smithi152 on smithi152 2024-03-29T04:38:25.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:24 smithi152 ceph-mon[37177]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:25.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:25.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:25.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:24 smithi152 ceph-mon[37177]: Reconfiguring ceph-exporter.smithi152 (monmap changed)... 2024-03-29T04:38:25.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi152", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:38:25.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:25.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:24 smithi152 ceph-mon[37177]: Reconfiguring daemon ceph-exporter.smithi152 on smithi152 2024-03-29T04:38:26.252 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:26 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:26.252 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:26 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:26.252 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:26 smithi152 ceph-mon[37177]: Reconfiguring crash.smithi152 (monmap changed)... 2024-03-29T04:38:26.252 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:26 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi152", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:38:26.252 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:26 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:26.252 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:26 smithi152 ceph-mon[37177]: Reconfiguring daemon crash.smithi152 on smithi152 2024-03-29T04:38:26.252 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:26 smithi152 ceph-mon[37177]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:26.362 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:26 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:26.362 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:26 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:26.362 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:26 smithi142 ceph-mon[29027]: Reconfiguring crash.smithi152 (monmap changed)... 2024-03-29T04:38:26.363 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:26 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi152", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:38:26.363 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:26 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:26.363 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:26 smithi142 ceph-mon[29027]: Reconfiguring daemon crash.smithi152 on smithi152 2024-03-29T04:38:26.363 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:26 smithi142 ceph-mon[29027]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:27.722 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:27 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:27.722 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:27 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:27.722 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:27 smithi152 ceph-mon[37177]: Reconfiguring mgr.smithi152.usoolh (monmap changed)... 2024-03-29T04:38:27.722 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:27 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi152.usoolh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:38:27.723 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:27 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:38:27.723 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:27 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:27.723 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:27 smithi152 ceph-mon[37177]: Reconfiguring daemon mgr.smithi152.usoolh on smithi152 2024-03-29T04:38:27.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:27 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:27.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:27 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:27.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:27 smithi142 ceph-mon[29027]: Reconfiguring mgr.smithi152.usoolh (monmap changed)... 2024-03-29T04:38:27.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:27 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi152.usoolh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:38:27.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:27 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:38:27.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:27 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:27.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:27 smithi142 ceph-mon[29027]: Reconfiguring daemon mgr.smithi152.usoolh on smithi152 2024-03-29T04:38:28.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:28 smithi142 ceph-mon[29027]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:28.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:28.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:28.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:38:28.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T04:38:28.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:29.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:28 smithi152 ceph-mon[37177]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:29.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:28 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:29.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:28 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:29.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:28 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:38:29.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:28 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T04:38:29.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:28 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:29.790 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: Reconfiguring mon.smithi152 (monmap changed)... 2024-03-29T04:38:29.790 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: Reconfiguring daemon mon.smithi152 on smithi152 2024-03-29T04:38:29.790 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:29.790 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:29.790 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:29.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:29.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T04:38:29.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi142.front.sepia.ceph.com:9093"}]: dispatch 2024-03-29T04:38:29.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:29.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T04:38:29.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi142.front.sepia.ceph.com:3000"}]: dispatch 2024-03-29T04:38:29.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:29.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:38:29.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi142.front.sepia.ceph.com:9095"}]: dispatch 2024-03-29T04:38:29.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:29.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:38:30.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: Reconfiguring mon.smithi152 (monmap changed)... 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: Reconfiguring daemon mon.smithi152 on smithi152 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi142.front.sepia.ceph.com:9093"}]: dispatch 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi142.front.sepia.ceph.com:3000"}]: dispatch 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:38:30.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi142.front.sepia.ceph.com:9095"}]: dispatch 2024-03-29T04:38:30.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:30.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:38:30.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:30 smithi142 ceph-mon[29027]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T04:38:30.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:30 smithi142 ceph-mon[29027]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi142.front.sepia.ceph.com:9093"}]: dispatch 2024-03-29T04:38:30.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:30 smithi142 ceph-mon[29027]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T04:38:30.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:30 smithi142 ceph-mon[29027]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi142.front.sepia.ceph.com:3000"}]: dispatch 2024-03-29T04:38:30.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:30 smithi142 ceph-mon[29027]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:38:30.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:30 smithi142 ceph-mon[29027]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi142.front.sepia.ceph.com:9095"}]: dispatch 2024-03-29T04:38:30.978 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:30 smithi142 ceph-mon[29027]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:31.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:30 smithi152 ceph-mon[37177]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T04:38:31.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:30 smithi152 ceph-mon[37177]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi142.front.sepia.ceph.com:9093"}]: dispatch 2024-03-29T04:38:31.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:30 smithi152 ceph-mon[37177]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T04:38:31.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:30 smithi152 ceph-mon[37177]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi142.front.sepia.ceph.com:3000"}]: dispatch 2024-03-29T04:38:31.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:30 smithi152 ceph-mon[37177]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:38:31.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:30 smithi152 ceph-mon[37177]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi142.front.sepia.ceph.com:9095"}]: dispatch 2024-03-29T04:38:31.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:30 smithi152 ceph-mon[37177]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:32.074 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 0 on host 'smithi142' 2024-03-29T04:38:32.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:32.082+0000 7fb27effd700 1 -- 172.21.15.142:0/3328449191 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb288106710 con 0x7fb27406c470 2024-03-29T04:38:32.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:32.084+0000 7fb28fe50700 1 -- 172.21.15.142:0/3328449191 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb27406c470 msgr2=0x7fb27406e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:38:32.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:32.084+0000 7fb28fe50700 1 --2- 172.21.15.142:0/3328449191 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb27406c470 0x7fb27406e930 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb288101c20 tx=0x7fb27800b410 comp rx=0 tx=0).stop 2024-03-29T04:38:32.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:32.084+0000 7fb28fe50700 1 -- 172.21.15.142:0/3328449191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288101dc0 msgr2=0x7fb2881968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:38:32.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:32.084+0000 7fb28fe50700 1 --2- 172.21.15.142:0/3328449191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288101dc0 0x7fb2881968b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb28400b5c0 tx=0x7fb28400bd70 comp rx=0 tx=0).stop 2024-03-29T04:38:32.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:32.084+0000 7fb28fe50700 1 -- 172.21.15.142:0/3328449191 shutdown_connections 2024-03-29T04:38:32.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:32.084+0000 7fb28fe50700 1 --2- 172.21.15.142:0/3328449191 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb27406c470 0x7fb27406e930 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:38:32.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:32.084+0000 7fb28fe50700 1 --2- 172.21.15.142:0/3328449191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb288100bc0 0x7fb288196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:38:32.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:32.084+0000 7fb28fe50700 1 --2- 172.21.15.142:0/3328449191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288101dc0 0x7fb2881968b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:38:32.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:32.084+0000 7fb28fe50700 1 -- 172.21.15.142:0/3328449191 >> 172.21.15.142:0/3328449191 conn(0x7fb2880fc140 msgr2=0x7fb288104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:38:32.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:32.084+0000 7fb28fe50700 1 -- 172.21.15.142:0/3328449191 shutdown_connections 2024-03-29T04:38:32.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:32.084+0000 7fb28fe50700 1 -- 172.21.15.142:0/3328449191 wait complete. 2024-03-29T04:38:32.677 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:32.677 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:32.677 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:32 smithi142 ceph-mon[29027]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:32.677 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:32.677 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:32.698 DEBUG:teuthology.orchestra.run.smithi142:osd.0> sudo journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.0.service 2024-03-29T04:38:32.700 INFO:tasks.cephadm:Deploying osd.1 on smithi142 with /dev/vg_nvme/lv_3... 2024-03-29T04:38:32.701 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-29T04:38:32.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:32.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:32.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:32 smithi152 ceph-mon[37177]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:32.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:32.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:33.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:33 smithi152 ceph-mon[37177]: from='osd.0 [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-29T04:38:33.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:38:33.809 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:33 smithi142 ceph-mon[29027]: from='osd.0 [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-29T04:38:33.809 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:38:34.209 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:38:34.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:34 smithi142 ceph-mon[29027]: from='osd.0 [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-29T04:38:34.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:34 smithi142 ceph-mon[29027]: osdmap e7: 1 total, 0 up, 1 in 2024-03-29T04:38:34.574 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:34 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:38:34.574 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:34 smithi142 ceph-mon[29027]: from='osd.0 [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-29T04:38:34.574 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:34 smithi142 ceph-mon[29027]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:34.574 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:38:34 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0[48057]: 2024-03-29T04:38:34.484+0000 7f70de63b700 -1 osd.0 0 waiting for initial osdmap 2024-03-29T04:38:34.574 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:38:34 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0[48057]: 2024-03-29T04:38:34.488+0000 7f70d842c700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:38:34.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:34 smithi152 ceph-mon[37177]: from='osd.0 [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-29T04:38:34.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:34 smithi152 ceph-mon[37177]: osdmap e7: 1 total, 0 up, 1 in 2024-03-29T04:38:34.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:34 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:38:34.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:34 smithi152 ceph-mon[37177]: from='osd.0 [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-29T04:38:34.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:34 smithi152 ceph-mon[37177]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:35.780 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:35 smithi142 ceph-mon[29027]: from='osd.0 [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-29T04:38:35.780 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:35 smithi142 ceph-mon[29027]: osdmap e8: 1 total, 0 up, 1 in 2024-03-29T04:38:35.780 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:35 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:38:35.780 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:35 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:38:35.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:35 smithi152 ceph-mon[37177]: from='osd.0 [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-29T04:38:35.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:35 smithi152 ceph-mon[37177]: osdmap e8: 1 total, 0 up, 1 in 2024-03-29T04:38:35.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:35 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:38:35.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:35 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:38:36.627 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-29T04:38:36.627 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-29T04:38:36.627 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-03-29T04:38:36.627 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-03-29T04:38:36.627 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124995 s, 839 MB/s 2024-03-29T04:38:36.628 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-03-29T04:38:36.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:36 smithi152 ceph-mon[37177]: purged_snaps scrub starts 2024-03-29T04:38:36.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:36 smithi152 ceph-mon[37177]: purged_snaps scrub ok 2024-03-29T04:38:36.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:36 smithi152 ceph-mon[37177]: osd.0 [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973] boot 2024-03-29T04:38:36.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:36 smithi152 ceph-mon[37177]: osdmap e9: 1 total, 1 up, 1 in 2024-03-29T04:38:36.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:36 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:38:36.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:36 smithi152 ceph-mon[37177]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:36.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:36 smithi142 ceph-mon[29027]: purged_snaps scrub starts 2024-03-29T04:38:36.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:36 smithi142 ceph-mon[29027]: purged_snaps scrub ok 2024-03-29T04:38:36.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:36 smithi142 ceph-mon[29027]: osd.0 [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973] boot 2024-03-29T04:38:36.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:36 smithi142 ceph-mon[29027]: osdmap e9: 1 total, 1 up, 1 in 2024-03-29T04:38:36.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:36 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:38:36.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:36 smithi142 ceph-mon[29027]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T04:38:37.522 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2024-03-29T04:38:37.741 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:37 smithi142 ceph-mon[29027]: osdmap e10: 1 total, 1 up, 1 in 2024-03-29T04:38:37.753 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:38:37.781 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:37 smithi152 ceph-mon[37177]: osdmap e10: 1 total, 1 up, 1 in 2024-03-29T04:38:38.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:38 smithi142 ceph-mon[29027]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:38.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:38.816 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:39.031 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:38 smithi152 ceph-mon[37177]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:39.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:39.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:39.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.352+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/4014426488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80fee80 msgr2=0x7fe4c80ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:38:39.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.352+0000 7fe4cd1f9700 1 --2- 172.21.15.142:0/4014426488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80fee80 0x7fe4c80ff2a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009910 tx=0x7fe4c0009c20 comp rx=0 tx=0).stop 2024-03-29T04:38:39.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.352+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/4014426488 shutdown_connections 2024-03-29T04:38:39.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.352+0000 7fe4cd1f9700 1 --2- 172.21.15.142:0/4014426488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c8101e60 0x7fe4c81022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:38:39.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.352+0000 7fe4cd1f9700 1 --2- 172.21.15.142:0/4014426488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80fee80 0x7fe4c80ff2a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:38:39.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.352+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/4014426488 >> 172.21.15.142:0/4014426488 conn(0x7fe4c80fa7c0 msgr2=0x7fe4c80fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:38:39.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.352+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/4014426488 shutdown_connections 2024-03-29T04:38:39.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.352+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/4014426488 wait complete. 2024-03-29T04:38:39.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.353+0000 7fe4cd1f9700 1 Processor -- start 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.353+0000 7fe4cd1f9700 1 -- start start 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.354+0000 7fe4cd1f9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80fee80 0x7fe4c8196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.354+0000 7fe4cd1f9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c8101e60 0x7fe4c81968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.354+0000 7fe4cd1f9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c8196ef0 con 0x7fe4c80fee80 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.354+0000 7fe4cd1f9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c8197030 con 0x7fe4c8101e60 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.354+0000 7fe4c6d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80fee80 0x7fe4c8196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.354+0000 7fe4c6d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80fee80 0x7fe4c8196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:50596/0 (socket says 172.21.15.142:50596) 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.354+0000 7fe4c6d9d700 1 -- 172.21.15.142:0/1726212941 learned_addr learned my addr 172.21.15.142:0/1726212941 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.354+0000 7fe4c659c700 1 --2- 172.21.15.142:0/1726212941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c8101e60 0x7fe4c81968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.354+0000 7fe4c6d9d700 1 -- 172.21.15.142:0/1726212941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c8101e60 msgr2=0x7fe4c81968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.354+0000 7fe4c6d9d700 1 --2- 172.21.15.142:0/1726212941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c8101e60 0x7fe4c81968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.354+0000 7fe4c6d9d700 1 -- 172.21.15.142:0/1726212941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c00095f0 con 0x7fe4c80fee80 2024-03-29T04:38:39.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.354+0000 7fe4c6d9d700 1 --2- 172.21.15.142:0/1726212941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80fee80 0x7fe4c8196390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009fd0 tx=0x7fe4c000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:38:39.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.355+0000 7fe4affff700 1 -- 172.21.15.142:0/1726212941 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c001d070 con 0x7fe4c80fee80 2024-03-29T04:38:39.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.355+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/1726212941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c819ba80 con 0x7fe4c80fee80 2024-03-29T04:38:39.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.355+0000 7fe4affff700 1 -- 172.21.15.142:0/1726212941 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe4c0005dc0 con 0x7fe4c80fee80 2024-03-29T04:38:39.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.355+0000 7fe4affff700 1 -- 172.21.15.142:0/1726212941 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c0003ae0 con 0x7fe4c80fee80 2024-03-29T04:38:39.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.355+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/1726212941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c819bf90 con 0x7fe4c80fee80 2024-03-29T04:38:39.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.356+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/1726212941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c8109720 con 0x7fe4c80fee80 2024-03-29T04:38:39.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.356+0000 7fe4affff700 1 -- 172.21.15.142:0/1726212941 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fe4c0003d60 con 0x7fe4c80fee80 2024-03-29T04:38:39.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.357+0000 7fe4affff700 1 --2- 172.21.15.142:0/1726212941 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe4b006c470 0x7fe4b006e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:38:39.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.357+0000 7fe4affff700 1 -- 172.21.15.142:0/1726212941 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7fe4c008be60 con 0x7fe4c80fee80 2024-03-29T04:38:39.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.357+0000 7fe4c659c700 1 --2- 172.21.15.142:0/1726212941 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe4b006c470 0x7fe4b006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:38:39.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.357+0000 7fe4c659c700 1 --2- 172.21.15.142:0/1726212941 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe4b006c470 0x7fe4b006e930 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009550 tx=0x7fe4bc0091b0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:38:39.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.361+0000 7fe4affff700 1 -- 172.21.15.142:0/1726212941 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe4c005af50 con 0x7fe4c80fee80 2024-03-29T04:38:39.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:39.573+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/1726212941 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4c81067b0 con 0x7fe4b006c470 2024-03-29T04:38:39.789 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:39 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T04:38:39.789 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:39 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T04:38:39.789 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:39 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:40.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:39 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T04:38:40.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:39 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T04:38:40.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:39 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:40.660 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:40 smithi142 ceph-mon[29027]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:38:40.661 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:40 smithi142 ceph-mon[29027]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:41.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:40 smithi152 ceph-mon[37177]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:38:41.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:40 smithi152 ceph-mon[37177]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:42.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:42 smithi142 ceph-mon[29027]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:42.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:42 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:42.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:42 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:42.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:42 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:38:43.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:42 smithi152 ceph-mon[37177]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:43.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:42 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:43.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:42 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:43.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:42 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:38:43.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:43 smithi152 ceph-mon[37177]: Detected new or changed devices on smithi142 2024-03-29T04:38:43.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:43 smithi152 ceph-mon[37177]: Adjusting osd_memory_target on smithi142 to 11840M 2024-03-29T04:38:43.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:43 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:43.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:43 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:43.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:43 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:38:43.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:43 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:43.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:43 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:38:43.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:43 smithi142 ceph-mon[29027]: Detected new or changed devices on smithi142 2024-03-29T04:38:43.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:43 smithi142 ceph-mon[29027]: Adjusting osd_memory_target on smithi142 to 11840M 2024-03-29T04:38:43.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:43 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:43.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:43 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:43.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:43 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:38:43.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:43 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:43.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:43 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:38:44.795 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:44 smithi142 ceph-mon[29027]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:44.795 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:44 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1670322641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717e7135-21ab-4d33-a81e-65da6d5186c2"}]: dispatch 2024-03-29T04:38:44.795 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:44 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1670322641' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "717e7135-21ab-4d33-a81e-65da6d5186c2"}]': finished 2024-03-29T04:38:44.795 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:44 smithi142 ceph-mon[29027]: osdmap e11: 2 total, 1 up, 2 in 2024-03-29T04:38:44.795 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:44 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:38:44.795 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:44 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/20020672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T04:38:45.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:44 smithi152 ceph-mon[37177]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:45.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:44 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1670322641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717e7135-21ab-4d33-a81e-65da6d5186c2"}]: dispatch 2024-03-29T04:38:45.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:44 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1670322641' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "717e7135-21ab-4d33-a81e-65da6d5186c2"}]': finished 2024-03-29T04:38:45.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:44 smithi152 ceph-mon[37177]: osdmap e11: 2 total, 1 up, 2 in 2024-03-29T04:38:45.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:44 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:38:45.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:44 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/20020672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T04:38:46.113 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:45 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:46.113 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:45 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:46.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:45 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:46.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:45 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:47.113 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:46 smithi142 ceph-mon[29027]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:47.183 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:46 smithi152 ceph-mon[37177]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:48.381 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:48.382 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:48.382 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:48.382 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:38:48.382 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:48.382 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:48 smithi152 ceph-mon[37177]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:48.382 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:38:48.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:48.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:48.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:48.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:38:48.426 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:48.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:48 smithi142 ceph-mon[29027]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:48.427 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:38:50.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:50 smithi142 ceph-mon[29027]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:51.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:50 smithi152 ceph-mon[37177]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:51.775 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:51 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-29T04:38:51.776 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:51 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:52.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:51 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-29T04:38:52.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:51 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:38:52.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:52 smithi142 ceph-mon[29027]: Deploying daemon osd.1 on smithi142 2024-03-29T04:38:52.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:52 smithi142 ceph-mon[29027]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:53.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:52 smithi152 ceph-mon[37177]: Deploying daemon osd.1 on smithi142 2024-03-29T04:38:53.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:52 smithi152 ceph-mon[37177]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:54.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:54 smithi142 ceph-mon[29027]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:54.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:54 smithi152 ceph-mon[37177]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:56.988 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:56 smithi142 ceph-mon[29027]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:56.988 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:56 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:56.989 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:56 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:56.989 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:56 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:38:57.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:56 smithi152 ceph-mon[37177]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:57.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:56 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:57.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:56 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:57.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:56 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:38:58.874 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:58.881+0000 7fe4affff700 1 -- 172.21.15.142:0/1726212941 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe4c81067b0 con 0x7fe4b006c470 2024-03-29T04:38:58.874 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 1 on host 'smithi142' 2024-03-29T04:38:58.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:58.884+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/1726212941 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe4b006c470 msgr2=0x7fe4b006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:38:58.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:58.884+0000 7fe4cd1f9700 1 --2- 172.21.15.142:0/1726212941 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe4b006c470 0x7fe4b006e930 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009550 tx=0x7fe4bc0091b0 comp rx=0 tx=0).stop 2024-03-29T04:38:58.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:58.884+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/1726212941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80fee80 msgr2=0x7fe4c8196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:38:58.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:58.884+0000 7fe4cd1f9700 1 --2- 172.21.15.142:0/1726212941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80fee80 0x7fe4c8196390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009fd0 tx=0x7fe4c000bd70 comp rx=0 tx=0).stop 2024-03-29T04:38:58.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:58.884+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/1726212941 shutdown_connections 2024-03-29T04:38:58.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:58.884+0000 7fe4cd1f9700 1 --2- 172.21.15.142:0/1726212941 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe4b006c470 0x7fe4b006e930 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:38:58.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:58.884+0000 7fe4cd1f9700 1 --2- 172.21.15.142:0/1726212941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4c80fee80 0x7fe4c8196390 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:38:58.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:58.884+0000 7fe4cd1f9700 1 --2- 172.21.15.142:0/1726212941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4c8101e60 0x7fe4c81968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:38:58.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:58.884+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/1726212941 >> 172.21.15.142:0/1726212941 conn(0x7fe4c80fa7c0 msgr2=0x7fe4c8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:38:58.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:58.884+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/1726212941 shutdown_connections 2024-03-29T04:38:58.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:38:58.884+0000 7fe4cd1f9700 1 -- 172.21.15.142:0/1726212941 wait complete. 2024-03-29T04:38:59.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:58 smithi152 ceph-mon[37177]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:59.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:58 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:59.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:58 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:59.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:58 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:59.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:58 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:59.363 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:58 smithi142 ceph-mon[29027]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:38:59.363 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:58 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:59.363 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:58 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:59.363 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:58 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:59.363 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:58 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:38:59.747 DEBUG:teuthology.orchestra.run.smithi142:osd.1> sudo journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.1.service 2024-03-29T04:38:59.750 INFO:tasks.cephadm:Deploying osd.2 on smithi142 with /dev/vg_nvme/lv_2... 2024-03-29T04:38:59.750 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-29T04:39:00.113 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:38:59 smithi142 ceph-mon[29027]: from='osd.1 [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-29T04:39:00.113 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:38:59 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[55551]: 2024-03-29T04:38:59.855+0000 7fd43fc87640 -1 osd.1 0 log_to_monitors true 2024-03-29T04:39:00.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:38:59 smithi152 ceph-mon[37177]: from='osd.1 [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-29T04:39:00.644 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:39:01.113 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:39:00 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[55551]: 2024-03-29T04:39:00.901+0000 7fd434aea700 -1 osd.1 0 waiting for initial osdmap 2024-03-29T04:39:01.114 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:39:00 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[55551]: 2024-03-29T04:39:00.905+0000 7fd42f0dc700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:39:01.114 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:00 smithi142 ceph-mon[29027]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:39:01.114 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:00 smithi142 ceph-mon[29027]: from='osd.1 [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-29T04:39:01.114 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:00 smithi142 ceph-mon[29027]: osdmap e12: 2 total, 1 up, 2 in 2024-03-29T04:39:01.114 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:00 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:39:01.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:00 smithi142 ceph-mon[29027]: from='osd.1 [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-29T04:39:01.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:00 smithi152 ceph-mon[37177]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:39:01.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:00 smithi152 ceph-mon[37177]: from='osd.1 [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-29T04:39:01.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:00 smithi152 ceph-mon[37177]: osdmap e12: 2 total, 1 up, 2 in 2024-03-29T04:39:01.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:00 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:39:01.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:00 smithi152 ceph-mon[37177]: from='osd.1 [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-29T04:39:02.113 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:01 smithi142 ceph-mon[29027]: from='osd.1 [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-29T04:39:02.113 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:01 smithi142 ceph-mon[29027]: osdmap e13: 2 total, 1 up, 2 in 2024-03-29T04:39:02.113 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:01 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:39:02.113 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:01 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:39:02.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:01 smithi152 ceph-mon[37177]: from='osd.1 [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-29T04:39:02.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:01 smithi152 ceph-mon[37177]: osdmap e13: 2 total, 1 up, 2 in 2024-03-29T04:39:02.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:01 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:39:02.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:01 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:39:03.000 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:02 smithi142 ceph-mon[29027]: purged_snaps scrub starts 2024-03-29T04:39:03.000 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:02 smithi142 ceph-mon[29027]: purged_snaps scrub ok 2024-03-29T04:39:03.000 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:02 smithi142 ceph-mon[29027]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:39:03.000 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:02 smithi142 ceph-mon[29027]: osd.1 [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047] boot 2024-03-29T04:39:03.000 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:02 smithi142 ceph-mon[29027]: osdmap e14: 2 total, 2 up, 2 in 2024-03-29T04:39:03.000 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:02 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:39:03.000 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:02 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:39:03.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:02 smithi152 ceph-mon[37177]: purged_snaps scrub starts 2024-03-29T04:39:03.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:02 smithi152 ceph-mon[37177]: purged_snaps scrub ok 2024-03-29T04:39:03.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:02 smithi152 ceph-mon[37177]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-29T04:39:03.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:02 smithi152 ceph-mon[37177]: osd.1 [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047] boot 2024-03-29T04:39:03.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:02 smithi152 ceph-mon[37177]: osdmap e14: 2 total, 2 up, 2 in 2024-03-29T04:39:03.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:02 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:39:03.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:02 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:39:03.840 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-29T04:39:03.841 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-29T04:39:03.841 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-03-29T04:39:03.841 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-03-29T04:39:03.841 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0142286 s, 737 MB/s 2024-03-29T04:39:03.841 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-03-29T04:39:04.095 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:03 smithi142 ceph-mon[29027]: osdmap e15: 2 total, 2 up, 2 in 2024-03-29T04:39:04.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:03 smithi152 ceph-mon[37177]: osdmap e15: 2 total, 2 up, 2 in 2024-03-29T04:39:04.734 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2024-03-29T04:39:04.947 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:39:05.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:04 smithi142 ceph-mon[29027]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:05.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:04 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:05.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:04 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:05.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:04 smithi152 ceph-mon[37177]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:05.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:04 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:05.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:04 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:07.113 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:06 smithi142 ceph-mon[29027]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:07.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:06 smithi152 ceph-mon[37177]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.360+0000 7fe64e62a700 1 -- 172.21.15.142:0/1113988561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe648101de0 msgr2=0x7fe648102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:39:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.360+0000 7fe64e62a700 1 --2- 172.21.15.142:0/1113988561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe648101de0 0x7fe648102240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009910 tx=0x7fe63c009c20 comp rx=0 tx=0).stop 2024-03-29T04:39:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.360+0000 7fe64e62a700 1 -- 172.21.15.142:0/1113988561 shutdown_connections 2024-03-29T04:39:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.360+0000 7fe64e62a700 1 --2- 172.21.15.142:0/1113988561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe648101de0 0x7fe648102240 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.360+0000 7fe64e62a700 1 --2- 172.21.15.142:0/1113988561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe648100be0 0x7fe648101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.360+0000 7fe64e62a700 1 -- 172.21.15.142:0/1113988561 >> 172.21.15.142:0/1113988561 conn(0x7fe6480fc140 msgr2=0x7fe6480fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:39:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.360+0000 7fe64e62a700 1 -- 172.21.15.142:0/1113988561 shutdown_connections 2024-03-29T04:39:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.360+0000 7fe64e62a700 1 -- 172.21.15.142:0/1113988561 wait complete. 2024-03-29T04:39:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.361+0000 7fe64e62a700 1 Processor -- start 2024-03-29T04:39:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.361+0000 7fe64e62a700 1 -- start start 2024-03-29T04:39:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.361+0000 7fe64e62a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe648100be0 0x7fe648074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:39:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.361+0000 7fe64e62a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe648101de0 0x7fe648073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.361+0000 7fe64e62a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6480736c0 con 0x7fe648100be0 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.361+0000 7fe64e62a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe648073800 con 0x7fe648101de0 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.361+0000 7fe647fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe648100be0 0x7fe648074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.362+0000 7fe647fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe648100be0 0x7fe648074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57212/0 (socket says 172.21.15.142:57212) 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.362+0000 7fe647fff700 1 -- 172.21.15.142:0/96155265 learned_addr learned my addr 172.21.15.142:0/96155265 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.362+0000 7fe6477fe700 1 --2- 172.21.15.142:0/96155265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe648101de0 0x7fe648073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.362+0000 7fe647fff700 1 -- 172.21.15.142:0/96155265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe648101de0 msgr2=0x7fe648073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.362+0000 7fe647fff700 1 --2- 172.21.15.142:0/96155265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe648101de0 0x7fe648073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.362+0000 7fe647fff700 1 -- 172.21.15.142:0/96155265 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0095f0 con 0x7fe648100be0 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.362+0000 7fe647fff700 1 --2- 172.21.15.142:0/96155265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe648100be0 0x7fe648074b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c8f0 tx=0x7fe63800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.362+0000 7fe6457fa700 1 -- 172.21.15.142:0/96155265 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6380078e0 con 0x7fe648100be0 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.362+0000 7fe6457fa700 1 -- 172.21.15.142:0/96155265 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe63800f450 con 0x7fe648100be0 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.362+0000 7fe64e62a700 1 -- 172.21.15.142:0/96155265 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe648073ae0 con 0x7fe648100be0 2024-03-29T04:39:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.362+0000 7fe6457fa700 1 -- 172.21.15.142:0/96155265 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe638018610 con 0x7fe648100be0 2024-03-29T04:39:07.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.362+0000 7fe64e62a700 1 -- 172.21.15.142:0/96155265 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64819e3f0 con 0x7fe648100be0 2024-03-29T04:39:07.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.363+0000 7fe64e62a700 1 -- 172.21.15.142:0/96155265 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe648109740 con 0x7fe648100be0 2024-03-29T04:39:07.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.364+0000 7fe6457fa700 1 -- 172.21.15.142:0/96155265 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fe63800f5c0 con 0x7fe648100be0 2024-03-29T04:39:07.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.364+0000 7fe6457fa700 1 --2- 172.21.15.142:0/96155265 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe63006c460 0x7fe63006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:39:07.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.364+0000 7fe6457fa700 1 -- 172.21.15.142:0/96155265 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7fe63808a9f0 con 0x7fe648100be0 2024-03-29T04:39:07.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.364+0000 7fe6477fe700 1 --2- 172.21.15.142:0/96155265 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe63006c460 0x7fe63006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:39:07.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.364+0000 7fe6477fe700 1 --2- 172.21.15.142:0/96155265 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe63006c460 0x7fe63006e920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe63c006010 tx=0x7fe63c00b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:39:07.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.369+0000 7fe6457fa700 1 -- 172.21.15.142:0/96155265 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe638059930 con 0x7fe648100be0 2024-03-29T04:39:07.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:07.574+0000 7fe64e62a700 1 -- 172.21.15.142:0/96155265 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe64819e9a0 con 0x7fe63006c460 2024-03-29T04:39:08.060 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T04:39:08.060 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T04:39:08.060 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:08.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T04:39:08.282 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T04:39:08.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:09.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:08 smithi142 ceph-mon[29027]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:39:09.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:08 smithi142 ceph-mon[29027]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:09.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:08 smithi152 ceph-mon[37177]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:39:09.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:08 smithi152 ceph-mon[37177]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:10.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:10 smithi152 ceph-mon[37177]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:10.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:10 smithi152 ceph-mon[37177]: Detected new or changed devices on smithi142 2024-03-29T04:39:10.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:10 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:10.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:10 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:10.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:10 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:39:10.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:10 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:39:10.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:10 smithi152 ceph-mon[37177]: Adjusting osd_memory_target on smithi142 to 5920M 2024-03-29T04:39:10.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:10 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:10.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:10 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:10.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:10 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:39:10.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:10 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:10.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:10 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:39:10.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:10 smithi142 ceph-mon[29027]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:10.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:10 smithi142 ceph-mon[29027]: Detected new or changed devices on smithi142 2024-03-29T04:39:10.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:10 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:10.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:10 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:10.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:10 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:39:10.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:10 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:39:10.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:10 smithi142 ceph-mon[29027]: Adjusting osd_memory_target on smithi142 to 5920M 2024-03-29T04:39:10.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:10 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:10.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:10 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:10.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:10 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:39:10.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:10 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:10.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:10 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:39:11.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:11 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2828758621' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "977032d2-3c29-4c68-ae66-73d024ab94ff"}]: dispatch 2024-03-29T04:39:11.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:11 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2828758621' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "977032d2-3c29-4c68-ae66-73d024ab94ff"}]': finished 2024-03-29T04:39:11.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:11 smithi142 ceph-mon[29027]: osdmap e16: 3 total, 2 up, 3 in 2024-03-29T04:39:11.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:11 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:39:11.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:11 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2467904804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T04:39:11.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:11 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:12.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:11 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2828758621' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "977032d2-3c29-4c68-ae66-73d024ab94ff"}]: dispatch 2024-03-29T04:39:12.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:11 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2828758621' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "977032d2-3c29-4c68-ae66-73d024ab94ff"}]': finished 2024-03-29T04:39:12.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:11 smithi152 ceph-mon[37177]: osdmap e16: 3 total, 2 up, 3 in 2024-03-29T04:39:12.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:11 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:39:12.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:11 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2467904804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T04:39:12.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:11 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:13.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:12 smithi152 ceph-mon[37177]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:13.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:12 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:13.113 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:12 smithi142 ceph-mon[29027]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:13.113 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:12 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:14.699 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:14 smithi142 ceph-mon[29027]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:14.700 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:14 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:14.700 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:14 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:14.700 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:14 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:14.700 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:14 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:39:14.700 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:14 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:14.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:14 smithi152 ceph-mon[37177]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:14.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:14 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:14.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:14 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:14.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:14 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:14.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:14 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:39:14.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:14 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:16.695 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:16 smithi142 ceph-mon[29027]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:17.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:16 smithi152 ceph-mon[37177]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:18.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:17 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:39:18.113 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:39:19.032 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:18 smithi152 ceph-mon[37177]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:19.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:18 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-29T04:39:19.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:18 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:19.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:18 smithi152 ceph-mon[37177]: Deploying daemon osd.2 on smithi142 2024-03-29T04:39:19.114 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:18 smithi142 ceph-mon[29027]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:19.114 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:18 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-29T04:39:19.114 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:18 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:19.114 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:18 smithi142 ceph-mon[29027]: Deploying daemon osd.2 on smithi142 2024-03-29T04:39:21.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:20 smithi152 ceph-mon[37177]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:21.085 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:20 smithi142 ceph-mon[29027]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:22.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:22 smithi142 ceph-mon[29027]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:23.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:22 smithi152 ceph-mon[37177]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:24.533 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:24.533 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:24.533 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:39:24.533 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:24 smithi152 ceph-mon[37177]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:24.614 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:24.614 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:24.614 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:39:24.614 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:24 smithi142 ceph-mon[29027]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:26.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:26.479+0000 7fe6457fa700 1 -- 172.21.15.142:0/96155265 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe64819e9a0 con 0x7fe63006c460 2024-03-29T04:39:26.473 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 2 on host 'smithi142' 2024-03-29T04:39:26.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:26.482+0000 7fe64e62a700 1 -- 172.21.15.142:0/96155265 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe63006c460 msgr2=0x7fe63006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:39:26.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:26.482+0000 7fe64e62a700 1 --2- 172.21.15.142:0/96155265 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe63006c460 0x7fe63006e920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe63c006010 tx=0x7fe63c00b540 comp rx=0 tx=0).stop 2024-03-29T04:39:26.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:26.482+0000 7fe64e62a700 1 -- 172.21.15.142:0/96155265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe648100be0 msgr2=0x7fe648074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:39:26.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:26.482+0000 7fe64e62a700 1 --2- 172.21.15.142:0/96155265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe648100be0 0x7fe648074b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c8f0 tx=0x7fe63800cc00 comp rx=0 tx=0).stop 2024-03-29T04:39:26.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:26.482+0000 7fe64e62a700 1 -- 172.21.15.142:0/96155265 shutdown_connections 2024-03-29T04:39:26.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:26.482+0000 7fe64e62a700 1 --2- 172.21.15.142:0/96155265 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe63006c460 0x7fe63006e920 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:26.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:26.482+0000 7fe64e62a700 1 --2- 172.21.15.142:0/96155265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe648100be0 0x7fe648074b30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:26.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:26.482+0000 7fe64e62a700 1 --2- 172.21.15.142:0/96155265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe648101de0 0x7fe648073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:26.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:26.482+0000 7fe64e62a700 1 -- 172.21.15.142:0/96155265 >> 172.21.15.142:0/96155265 conn(0x7fe6480fc140 msgr2=0x7fe648105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:39:26.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:26.482+0000 7fe64e62a700 1 -- 172.21.15.142:0/96155265 shutdown_connections 2024-03-29T04:39:26.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:39:26.482+0000 7fe64e62a700 1 -- 172.21.15.142:0/96155265 wait complete. 2024-03-29T04:39:26.769 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:26 smithi142 ceph-mon[29027]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:26.769 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:26 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:26.769 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:26 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:27.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:26 smithi152 ceph-mon[37177]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:27.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:26 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:27.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:26 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:27.090 DEBUG:teuthology.orchestra.run.smithi142:osd.2> sudo journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.2.service 2024-03-29T04:39:27.092 INFO:tasks.cephadm:Deploying osd.3 on smithi152 with /dev/vg_nvme/lv_4... 2024-03-29T04:39:27.092 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-29T04:39:27.309 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi152/config 2024-03-29T04:39:27.682 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:27 smithi152 ceph-mon[37177]: from='osd.2 [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-29T04:39:27.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:27 smithi142 ceph-mon[29027]: from='osd.2 [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-29T04:39:28.942 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:39:28 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2[63214]: 2024-03-29T04:39:28.665+0000 7fc4218cc700 -1 osd.2 0 waiting for initial osdmap 2024-03-29T04:39:28.942 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:39:28 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2[63214]: 2024-03-29T04:39:28.669+0000 7fc419eba700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:39:28.942 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:28 smithi142 ceph-mon[29027]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:28.942 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:28 smithi142 ceph-mon[29027]: from='osd.2 [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-29T04:39:28.942 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:28 smithi142 ceph-mon[29027]: osdmap e17: 3 total, 2 up, 3 in 2024-03-29T04:39:28.942 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:39:28.942 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:28 smithi142 ceph-mon[29027]: from='osd.2 [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-29T04:39:29.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:28 smithi152 ceph-mon[37177]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:29.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:28 smithi152 ceph-mon[37177]: from='osd.2 [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-29T04:39:29.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:28 smithi152 ceph-mon[37177]: osdmap e17: 3 total, 2 up, 3 in 2024-03-29T04:39:29.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:28 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:39:29.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:28 smithi152 ceph-mon[37177]: from='osd.2 [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-29T04:39:29.085 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-29T04:39:29.085 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-29T04:39:29.085 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-03-29T04:39:29.085 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-03-29T04:39:29.085 INFO:teuthology.orchestra.run.smithi152.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113775 s, 922 MB/s 2024-03-29T04:39:29.085 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 2024-03-29T04:39:29.085 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-03-29T04:39:29.722 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:29 smithi152 ceph-mon[37177]: from='osd.2 [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-29T04:39:29.722 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:29 smithi152 ceph-mon[37177]: osdmap e18: 3 total, 2 up, 3 in 2024-03-29T04:39:29.722 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:39:29.722 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:29 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:39:29.745 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi152:vg_nvme/lv_4 2024-03-29T04:39:29.765 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:29 smithi142 ceph-mon[29027]: from='osd.2 [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-29T04:39:29.765 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:29 smithi142 ceph-mon[29027]: osdmap e18: 3 total, 2 up, 3 in 2024-03-29T04:39:29.765 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:39:29.765 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:29 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:39:29.950 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi152/config 2024-03-29T04:39:30.894 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:30 smithi142 ceph-mon[29027]: purged_snaps scrub starts 2024-03-29T04:39:30.894 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:30 smithi142 ceph-mon[29027]: purged_snaps scrub ok 2024-03-29T04:39:30.894 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:30 smithi142 ceph-mon[29027]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:30.894 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:30 smithi142 ceph-mon[29027]: osd.2 [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059] boot 2024-03-29T04:39:30.894 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:30 smithi142 ceph-mon[29027]: osdmap e19: 3 total, 3 up, 3 in 2024-03-29T04:39:30.894 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:30 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:39:30.894 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:30 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:30.894 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:30 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:31.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:30 smithi152 ceph-mon[37177]: purged_snaps scrub starts 2024-03-29T04:39:31.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:30 smithi152 ceph-mon[37177]: purged_snaps scrub ok 2024-03-29T04:39:31.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:30 smithi152 ceph-mon[37177]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-29T04:39:31.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:30 smithi152 ceph-mon[37177]: osd.2 [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059] boot 2024-03-29T04:39:31.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:30 smithi152 ceph-mon[37177]: osdmap e19: 3 total, 3 up, 3 in 2024-03-29T04:39:31.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:30 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:39:31.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:30 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:31.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:30 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:31.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.509+0000 7f1518981700 1 -- 172.21.15.152:0/3071612598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514100cd0 msgr2=0x7f1514101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:39:31.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.509+0000 7f1518981700 1 --2- 172.21.15.152:0/3071612598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514100cd0 0x7f1514101130 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009910 tx=0x7f14fc009c20 comp rx=0 tx=0).stop 2024-03-29T04:39:31.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.509+0000 7f1518981700 1 -- 172.21.15.152:0/3071612598 shutdown_connections 2024-03-29T04:39:31.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.509+0000 7f1518981700 1 --2- 172.21.15.152:0/3071612598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514100cd0 0x7f1514101130 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:31.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.509+0000 7f1518981700 1 --2- 172.21.15.152:0/3071612598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15140ffad0 0x7f15140ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:31.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.509+0000 7f1518981700 1 -- 172.21.15.152:0/3071612598 >> 172.21.15.152:0/3071612598 conn(0x7f15140fb050 msgr2=0x7f15140fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:39:31.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.509+0000 7f1518981700 1 -- 172.21.15.152:0/3071612598 shutdown_connections 2024-03-29T04:39:31.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.509+0000 7f1518981700 1 -- 172.21.15.152:0/3071612598 wait complete. 2024-03-29T04:39:31.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.509+0000 7f1518981700 1 Processor -- start 2024-03-29T04:39:31.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f1518981700 1 -- start start 2024-03-29T04:39:31.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f1518981700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15140ffad0 0x7f1514196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:39:31.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f1518981700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514100cd0 0x7f15141968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:39:31.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f1518981700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1514196ef0 con 0x7f15140ffad0 2024-03-29T04:39:31.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f1518981700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1514197030 con 0x7f1514100cd0 2024-03-29T04:39:31.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f1509d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514100cd0 0x7f15141968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:39:31.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f1509d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514100cd0 0x7f15141968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55846/0 (socket says 172.21.15.152:55846) 2024-03-29T04:39:31.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f1509d9b700 1 -- 172.21.15.152:0/1461207842 learned_addr learned my addr 172.21.15.152:0/1461207842 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:39:31.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f151259c700 1 --2- 172.21.15.152:0/1461207842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15140ffad0 0x7f1514196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:39:31.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f1509d9b700 1 -- 172.21.15.152:0/1461207842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15140ffad0 msgr2=0x7f1514196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:39:31.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f1509d9b700 1 --2- 172.21.15.152:0/1461207842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15140ffad0 0x7f1514196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:31.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f1509d9b700 1 -- 172.21.15.152:0/1461207842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc0095f0 con 0x7f1514100cd0 2024-03-29T04:39:31.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f1509d9b700 1 --2- 172.21.15.152:0/1461207842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514100cd0 0x7f15141968d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f14fc006010 tx=0x7f14fc00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:39:31.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.510+0000 7f150bfff700 1 -- 172.21.15.152:0/1461207842 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14fc01d070 con 0x7f1514100cd0 2024-03-29T04:39:31.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.511+0000 7f1518981700 1 -- 172.21.15.152:0/1461207842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151419ba30 con 0x7f1514100cd0 2024-03-29T04:39:31.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.511+0000 7f150bfff700 1 -- 172.21.15.152:0/1461207842 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f14fc00f460 con 0x7f1514100cd0 2024-03-29T04:39:31.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.511+0000 7f150bfff700 1 -- 172.21.15.152:0/1461207842 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14fc021640 con 0x7f1514100cd0 2024-03-29T04:39:31.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.511+0000 7f1518981700 1 -- 172.21.15.152:0/1461207842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151419bf20 con 0x7f1514100cd0 2024-03-29T04:39:31.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.512+0000 7f1518981700 1 -- 172.21.15.152:0/1461207842 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1514108630 con 0x7f1514100cd0 2024-03-29T04:39:31.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.512+0000 7f150bfff700 1 -- 172.21.15.152:0/1461207842 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f14fc00fab0 con 0x7f1514100cd0 2024-03-29T04:39:31.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.512+0000 7f150bfff700 1 --2- 172.21.15.152:0/1461207842 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f150006c460 0x7f150006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:39:31.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.512+0000 7f150bfff700 1 -- 172.21.15.152:0/1461207842 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2846+0+0 (secure 0 0 0) 0x7f14fc08bf70 con 0x7f1514100cd0 2024-03-29T04:39:31.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.513+0000 7f151259c700 1 --2- 172.21.15.152:0/1461207842 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f150006c460 0x7f150006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:39:31.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.513+0000 7f151259c700 1 --2- 172.21.15.152:0/1461207842 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f150006c460 0x7f150006e920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1514100b30 tx=0x7f150400b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:39:31.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.517+0000 7f150bfff700 1 -- 172.21.15.152:0/1461207842 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f14fc0bf0e0 con 0x7f1514100cd0 2024-03-29T04:39:31.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:31.710+0000 7f1518981700 1 -- 172.21.15.152:0/1461207842 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f151419c620 con 0x7f150006c460 2024-03-29T04:39:32.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:31 smithi152 ceph-mon[37177]: osdmap e20: 3 total, 3 up, 3 in 2024-03-29T04:39:32.109 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:31 smithi142 ceph-mon[29027]: osdmap e20: 3 total, 3 up, 3 in 2024-03-29T04:39:33.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:32 smithi152 ceph-mon[37177]: pgmap v75: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:33.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-29T04:39:33.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:32 smithi152 ceph-mon[37177]: from='client.24139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:39:33.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T04:39:33.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T04:39:33.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:33.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-29T04:39:33.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:32 smithi152 ceph-mon[37177]: osdmap e21: 3 total, 3 up, 3 in 2024-03-29T04:39:33.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-29T04:39:33.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:39:33.038 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:32 smithi142 ceph-mon[29027]: pgmap v75: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:33.038 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-29T04:39:33.038 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:32 smithi142 ceph-mon[29027]: from='client.24139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:39:33.038 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T04:39:33.039 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T04:39:33.039 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:33.039 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-29T04:39:33.039 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:32 smithi142 ceph-mon[29027]: osdmap e21: 3 total, 3 up, 3 in 2024-03-29T04:39:33.039 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-29T04:39:33.039 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:39:33.744 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:39:33 smithi142 sudo[67407]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-29T04:39:33.745 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:39:33 smithi142 sudo[67407]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T04:39:33.745 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:39:33 smithi142 sudo[67407]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T04:39:33.745 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:39:33 smithi142 sudo[67407]: pam_unix(sudo:session): session closed for user root 2024-03-29T04:39:34.040 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:39:33 smithi142 sudo[67491]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-29T04:39:34.040 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:39:33 smithi142 sudo[67491]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T04:39:34.040 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:39:33 smithi142 sudo[67491]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T04:39:34.040 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:39:33 smithi142 sudo[67491]: pam_unix(sudo:session): session closed for user root 2024-03-29T04:39:34.040 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-29T04:39:34.041 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: osdmap e22: 3 total, 3 up, 3 in 2024-03-29T04:39:34.041 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:34.041 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:34.041 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:39:34.041 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:39:34.041 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:39:34.041 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:34.041 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:34.041 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:39:34.041 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:34.041 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:39:34.041 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:34.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-29T04:39:34.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: osdmap e22: 3 total, 3 up, 3 in 2024-03-29T04:39:34.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:34.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:34.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:39:34.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:39:34.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:39:34.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:34.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:34.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:39:34.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:34.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:39:34.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:34.300 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67583]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-29T04:39:34.301 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67583]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T04:39:34.301 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67583]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T04:39:34.301 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67583]: pam_unix(sudo:session): session closed for user root 2024-03-29T04:39:34.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:34.464+0000 7f150bfff700 1 -- 172.21.15.152:0/1461207842 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f14fc057640 con 0x7f1514100cd0 2024-03-29T04:39:34.556 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67591]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-29T04:39:34.556 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67591]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T04:39:34.556 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67591]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T04:39:34.556 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67591]: pam_unix(sudo:session): session closed for user root 2024-03-29T04:39:34.832 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67608]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-29T04:39:34.832 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67608]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T04:39:34.833 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67608]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T04:39:34.833 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67608]: pam_unix(sudo:session): session closed for user root 2024-03-29T04:39:35.083 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67664]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-29T04:39:35.083 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67664]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T04:39:35.083 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67664]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T04:39:35.083 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:39:34 smithi142 sudo[67664]: pam_unix(sudo:session): session closed for user root 2024-03-29T04:39:35.345 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 sudo[67715]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-29T04:39:35.346 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 sudo[67715]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T04:39:35.346 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 sudo[67715]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T04:39:35.346 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 sudo[67715]: pam_unix(sudo:session): session closed for user root 2024-03-29T04:39:35.614 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 sudo[67740]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-29T04:39:35.614 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 sudo[67740]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T04:39:35.614 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 sudo[67740]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T04:39:35.614 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 sudo[67740]: pam_unix(sudo:session): session closed for user root 2024-03-29T04:39:35.614 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 ceph-mon[29027]: Detected new or changed devices on smithi142 2024-03-29T04:39:35.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 ceph-mon[29027]: Adjusting osd_memory_target on smithi142 to 3946M 2024-03-29T04:39:35.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 ceph-mon[29027]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:35.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 ceph-mon[29027]: osdmap e23: 3 total, 3 up, 3 in 2024-03-29T04:39:35.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/86414833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39ae7173-3a83-4ae1-9866-5a0da908171c"}]: dispatch 2024-03-29T04:39:35.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 ceph-mon[29027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39ae7173-3a83-4ae1-9866-5a0da908171c"}]: dispatch 2024-03-29T04:39:35.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 ceph-mon[29027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39ae7173-3a83-4ae1-9866-5a0da908171c"}]': finished 2024-03-29T04:39:35.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 ceph-mon[29027]: osdmap e24: 4 total, 3 up, 4 in 2024-03-29T04:39:35.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:39:35.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 ceph-mon[29027]: mgrmap e19: smithi142.swesrg(active, since 2m), standbys: smithi152.usoolh 2024-03-29T04:39:35.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:35 smithi142 ceph-mon[29027]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-29T04:39:35.867 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 sudo[41129]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-29T04:39:35.867 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 sudo[41129]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T04:39:35.867 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 sudo[41129]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T04:39:35.867 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 sudo[41129]: pam_unix(sudo:session): session closed for user root 2024-03-29T04:39:36.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 sudo[41138]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-29T04:39:36.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 sudo[41138]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T04:39:36.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 sudo[41138]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T04:39:36.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 sudo[41138]: pam_unix(sudo:session): session closed for user root 2024-03-29T04:39:36.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 ceph-mon[37177]: Detected new or changed devices on smithi142 2024-03-29T04:39:36.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 ceph-mon[37177]: Adjusting osd_memory_target on smithi142 to 3946M 2024-03-29T04:39:36.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 ceph-mon[37177]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:36.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 ceph-mon[37177]: osdmap e23: 3 total, 3 up, 3 in 2024-03-29T04:39:36.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 ceph-mon[37177]: from='client.? 172.21.15.152:0/86414833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39ae7173-3a83-4ae1-9866-5a0da908171c"}]: dispatch 2024-03-29T04:39:36.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 ceph-mon[37177]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39ae7173-3a83-4ae1-9866-5a0da908171c"}]: dispatch 2024-03-29T04:39:36.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 ceph-mon[37177]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39ae7173-3a83-4ae1-9866-5a0da908171c"}]': finished 2024-03-29T04:39:36.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 ceph-mon[37177]: osdmap e24: 4 total, 3 up, 4 in 2024-03-29T04:39:36.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:39:36.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 ceph-mon[37177]: mgrmap e19: smithi142.swesrg(active, since 2m), standbys: smithi152.usoolh 2024-03-29T04:39:36.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:35 smithi152 ceph-mon[37177]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-29T04:39:36.614 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:36 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/951221652' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T04:39:36.614 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:36 smithi142 ceph-mon[29027]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-29T04:39:36.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:36 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:39:36.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:36 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:39:36.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:36 smithi142 ceph-mon[29027]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-29T04:39:36.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:36 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:39:36.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:36 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:39:36.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:36 smithi142 ceph-mon[29027]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:36.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:36 smithi142 ceph-mon[29027]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-29T04:39:36.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:36 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:36.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:36 smithi152 ceph-mon[37177]: from='client.? 172.21.15.152:0/951221652' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T04:39:36.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:36 smithi152 ceph-mon[37177]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-29T04:39:36.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:36 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:39:36.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:36 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:39:36.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:36 smithi152 ceph-mon[37177]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-29T04:39:36.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:36 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:39:36.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:36 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:39:36.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:36 smithi152 ceph-mon[37177]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:36.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:36 smithi152 ceph-mon[37177]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-29T04:39:36.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:36 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:39.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:38 smithi152 ceph-mon[37177]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:39.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:39.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:39.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:39.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:39:39.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:39.364 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:38 smithi142 ceph-mon[29027]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:39.365 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:39.365 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:39.365 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:39.365 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:39:39.365 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:41.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:40 smithi152 ceph-mon[37177]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:41.364 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:40 smithi142 ceph-mon[29027]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:42.277 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:41 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-29T04:39:42.277 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:41 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:42.364 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:41 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-29T04:39:42.364 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:41 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:43.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:42 smithi152 ceph-mon[37177]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:43.033 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:42 smithi152 ceph-mon[37177]: Deploying daemon osd.3 on smithi152 2024-03-29T04:39:43.364 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:42 smithi142 ceph-mon[29027]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:43.364 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:42 smithi142 ceph-mon[29027]: Deploying daemon osd.3 on smithi152 2024-03-29T04:39:44.739 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:44 smithi152 ceph-mon[37177]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:44.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:44 smithi142 ceph-mon[29027]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:46.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:46 smithi152 ceph-mon[37177]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:47.114 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:46 smithi142 ceph-mon[29027]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:48.283 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:48.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:48.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:39:48.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:48 smithi152 ceph-mon[37177]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:48.284 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:39:48.364 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:48.364 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:48.365 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:39:48.365 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:48 smithi142 ceph-mon[29027]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:48.365 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:39:50.297 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 3 on host 'smithi152' 2024-03-29T04:39:50.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:50.303+0000 7f150bfff700 1 -- 172.21.15.152:0/1461207842 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f151419c620 con 0x7f150006c460 2024-03-29T04:39:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:50.306+0000 7f1518981700 1 -- 172.21.15.152:0/1461207842 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f150006c460 msgr2=0x7f150006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:39:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:50.306+0000 7f1518981700 1 --2- 172.21.15.152:0/1461207842 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f150006c460 0x7f150006e920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1514100b30 tx=0x7f150400b410 comp rx=0 tx=0).stop 2024-03-29T04:39:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:50.306+0000 7f1518981700 1 -- 172.21.15.152:0/1461207842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514100cd0 msgr2=0x7f15141968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:39:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:50.306+0000 7f1518981700 1 --2- 172.21.15.152:0/1461207842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514100cd0 0x7f15141968d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f14fc006010 tx=0x7f14fc00bd70 comp rx=0 tx=0).stop 2024-03-29T04:39:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:50.306+0000 7f1518981700 1 -- 172.21.15.152:0/1461207842 shutdown_connections 2024-03-29T04:39:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:50.306+0000 7f1518981700 1 --2- 172.21.15.152:0/1461207842 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f150006c460 0x7f150006e920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:50.306+0000 7f1518981700 1 --2- 172.21.15.152:0/1461207842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15140ffad0 0x7f1514196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:50.306+0000 7f1518981700 1 --2- 172.21.15.152:0/1461207842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1514100cd0 0x7f15141968d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:50.306+0000 7f1518981700 1 -- 172.21.15.152:0/1461207842 >> 172.21.15.152:0/1461207842 conn(0x7f15140fb050 msgr2=0x7f1514068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:39:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:50.306+0000 7f1518981700 1 -- 172.21.15.152:0/1461207842 shutdown_connections 2024-03-29T04:39:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:50.306+0000 7f1518981700 1 -- 172.21.15.152:0/1461207842 wait complete. 2024-03-29T04:39:50.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:50 smithi152 ceph-mon[37177]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:50.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:50 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:50.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:50 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:51.114 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:50 smithi142 ceph-mon[29027]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:51.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:50 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:51.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:50 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:51.532 DEBUG:teuthology.orchestra.run.smithi152:osd.3> sudo journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.3.service 2024-03-29T04:39:51.534 INFO:tasks.cephadm:Deploying osd.4 on smithi152 with /dev/vg_nvme/lv_3... 2024-03-29T04:39:51.535 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-29T04:39:51.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:51 smithi152 ceph-mon[37177]: from='osd.3 [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T04:39:51.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:51 smithi152 ceph-mon[37177]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T04:39:51.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:51 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:51.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:51 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:51.784 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi152/config 2024-03-29T04:39:51.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:51 smithi142 ceph-mon[29027]: from='osd.3 [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T04:39:51.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:51 smithi142 ceph-mon[29027]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T04:39:51.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:51 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:51.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:51 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:52.934 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:52 smithi152 ceph-mon[37177]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:52.935 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:52 smithi152 ceph-mon[37177]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-29T04:39:52.935 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:52 smithi152 ceph-mon[37177]: osdmap e25: 4 total, 3 up, 4 in 2024-03-29T04:39:52.935 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:52 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:39:52.935 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:52 smithi152 ceph-mon[37177]: from='osd.3 [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:39:52.935 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:52 smithi152 ceph-mon[37177]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:39:52.935 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:39:52 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3[43139]: 2024-03-29T04:39:52.676+0000 7fc8a3173700 -1 osd.3 0 waiting for initial osdmap 2024-03-29T04:39:52.935 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:39:52 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3[43139]: 2024-03-29T04:39:52.682+0000 7fc89d765700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:39:53.114 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:52 smithi142 ceph-mon[29027]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:53.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:52 smithi142 ceph-mon[29027]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-29T04:39:53.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:52 smithi142 ceph-mon[29027]: osdmap e25: 4 total, 3 up, 4 in 2024-03-29T04:39:53.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:52 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:39:53.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:52 smithi142 ceph-mon[29027]: from='osd.3 [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:39:53.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:52 smithi142 ceph-mon[29027]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:39:53.976 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:53 smithi152 ceph-mon[37177]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-29T04:39:53.976 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:53 smithi152 ceph-mon[37177]: osdmap e26: 4 total, 3 up, 4 in 2024-03-29T04:39:53.977 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:53 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:39:53.977 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:53 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:39:53.977 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:53 smithi152 ceph-mon[37177]: osd.3 [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166] boot 2024-03-29T04:39:53.977 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:53 smithi152 ceph-mon[37177]: osdmap e27: 4 total, 4 up, 4 in 2024-03-29T04:39:53.977 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:53 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:39:54.114 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:53 smithi142 ceph-mon[29027]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-29T04:39:54.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:53 smithi142 ceph-mon[29027]: osdmap e26: 4 total, 3 up, 4 in 2024-03-29T04:39:54.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:53 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:39:54.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:53 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:39:54.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:53 smithi142 ceph-mon[29027]: osd.3 [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166] boot 2024-03-29T04:39:54.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:53 smithi142 ceph-mon[29027]: osdmap e27: 4 total, 4 up, 4 in 2024-03-29T04:39:54.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:53 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:39:54.370 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-29T04:39:54.370 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-29T04:39:54.370 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-03-29T04:39:54.370 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-03-29T04:39:54.370 INFO:teuthology.orchestra.run.smithi152.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0140898 s, 744 MB/s 2024-03-29T04:39:54.370 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 2024-03-29T04:39:54.370 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-03-29T04:39:54.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:54 smithi152 ceph-mon[37177]: purged_snaps scrub starts 2024-03-29T04:39:54.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:54 smithi152 ceph-mon[37177]: purged_snaps scrub ok 2024-03-29T04:39:54.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:54 smithi152 ceph-mon[37177]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:54.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:54 smithi152 ceph-mon[37177]: osdmap e28: 4 total, 4 up, 4 in 2024-03-29T04:39:54.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:54 smithi142 ceph-mon[29027]: purged_snaps scrub starts 2024-03-29T04:39:54.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:54 smithi142 ceph-mon[29027]: purged_snaps scrub ok 2024-03-29T04:39:54.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:54 smithi142 ceph-mon[29027]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-29T04:39:54.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:54 smithi142 ceph-mon[29027]: osdmap e28: 4 total, 4 up, 4 in 2024-03-29T04:39:55.134 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi152:vg_nvme/lv_3 2024-03-29T04:39:55.640 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi152/config 2024-03-29T04:39:56.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:56 smithi152 ceph-mon[37177]: osdmap e29: 4 total, 4 up, 4 in 2024-03-29T04:39:56.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:56 smithi152 ceph-mon[37177]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:39:56.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:56 smithi142 ceph-mon[29027]: osdmap e29: 4 total, 4 up, 4 in 2024-03-29T04:39:56.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:56 smithi142 ceph-mon[29027]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:39:57.653 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: osdmap e30: 4 total, 4 up, 4 in 2024-03-29T04:39:57.653 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: Detected new or changed devices on smithi152 2024-03-29T04:39:57.653 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:57.653 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:57.653 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:39:57.653 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: Adjusting osd_memory_target on smithi152 to 14912M 2024-03-29T04:39:57.653 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:57.653 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:57.654 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:39:57.654 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:57.654 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:39:57.654 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:57.654 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:39:57.654 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:57 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:57.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: osdmap e30: 4 total, 4 up, 4 in 2024-03-29T04:39:57.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: Detected new or changed devices on smithi152 2024-03-29T04:39:57.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:57.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:57.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:39:57.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: Adjusting osd_memory_target on smithi152 to 14912M 2024-03-29T04:39:57.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:57.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:57.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:39:57.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:57.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:39:57.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:57.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:39:57.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:57 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:39:57.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.872+0000 7fc37043f700 1 -- 172.21.15.152:0/2023458101 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc368100cb0 msgr2=0x7fc368101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:39:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.872+0000 7fc37043f700 1 --2- 172.21.15.152:0/2023458101 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc368100cb0 0x7fc368101110 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc364009870 tx=0x7fc364009b80 comp rx=0 tx=0).stop 2024-03-29T04:39:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.873+0000 7fc37043f700 1 -- 172.21.15.152:0/2023458101 shutdown_connections 2024-03-29T04:39:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.873+0000 7fc37043f700 1 --2- 172.21.15.152:0/2023458101 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc368100cb0 0x7fc368101110 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.873+0000 7fc37043f700 1 --2- 172.21.15.152:0/2023458101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3680ffab0 0x7fc3680ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.873+0000 7fc37043f700 1 -- 172.21.15.152:0/2023458101 >> 172.21.15.152:0/2023458101 conn(0x7fc3680fb010 msgr2=0x7fc3680fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:39:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.873+0000 7fc37043f700 1 -- 172.21.15.152:0/2023458101 shutdown_connections 2024-03-29T04:39:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.873+0000 7fc37043f700 1 -- 172.21.15.152:0/2023458101 wait complete. 2024-03-29T04:39:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.873+0000 7fc37043f700 1 Processor -- start 2024-03-29T04:39:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.873+0000 7fc37043f700 1 -- start start 2024-03-29T04:39:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.873+0000 7fc37043f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3680ffab0 0x7fc368198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:39:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.873+0000 7fc37043f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc368100cb0 0x7fc368198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:39:57.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.873+0000 7fc37043f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681990b0 con 0x7fc368100cb0 2024-03-29T04:39:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.873+0000 7fc37043f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681991f0 con 0x7fc3680ffab0 2024-03-29T04:39:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc36e1db700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3680ffab0 0x7fc368198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:39:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc36e1db700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3680ffab0 0x7fc368198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47428/0 (socket says 172.21.15.152:47428) 2024-03-29T04:39:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc36e1db700 1 -- 172.21.15.152:0/91475386 learned_addr learned my addr 172.21.15.152:0/91475386 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:39:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc36d9da700 1 --2- 172.21.15.152:0/91475386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc368100cb0 0x7fc368198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:39:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc36e1db700 1 -- 172.21.15.152:0/91475386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc368100cb0 msgr2=0x7fc368198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:39:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc36e1db700 1 --2- 172.21.15.152:0/91475386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc368100cb0 0x7fc368198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:39:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc36e1db700 1 -- 172.21.15.152:0/91475386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc364009520 con 0x7fc3680ffab0 2024-03-29T04:39:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc36e1db700 1 --2- 172.21.15.152:0/91475386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3680ffab0 0x7fc368198550 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc35800e970 tx=0x7fc35800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:39:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc35f7fe700 1 -- 172.21.15.152:0/91475386 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3580041f0 con 0x7fc3680ffab0 2024-03-29T04:39:57.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc37043f700 1 -- 172.21.15.152:0/91475386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36819dca0 con 0x7fc3680ffab0 2024-03-29T04:39:57.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc35f7fe700 1 -- 172.21.15.152:0/91475386 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc35800cc70 con 0x7fc3680ffab0 2024-03-29T04:39:57.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc35f7fe700 1 -- 172.21.15.152:0/91475386 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc358003cb0 con 0x7fc3680ffab0 2024-03-29T04:39:57.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.874+0000 7fc37043f700 1 -- 172.21.15.152:0/91475386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36819e220 con 0x7fc3680ffab0 2024-03-29T04:39:57.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.875+0000 7fc37043f700 1 -- 172.21.15.152:0/91475386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc368108610 con 0x7fc3680ffab0 2024-03-29T04:39:57.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.876+0000 7fc35f7fe700 1 -- 172.21.15.152:0/91475386 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc35800c7c0 con 0x7fc3680ffab0 2024-03-29T04:39:57.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.876+0000 7fc35f7fe700 1 --2- 172.21.15.152:0/91475386 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc35406c4e0 0x7fc35406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:39:57.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.876+0000 7fc35f7fe700 1 -- 172.21.15.152:0/91475386 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3774+0+0 (secure 0 0 0) 0x7fc358014070 con 0x7fc3680ffab0 2024-03-29T04:39:57.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.876+0000 7fc36d9da700 1 --2- 172.21.15.152:0/91475386 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc35406c4e0 0x7fc35406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:39:57.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.877+0000 7fc36d9da700 1 --2- 172.21.15.152:0/91475386 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc35406c4e0 0x7fc35406e9a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc36400b6f0 tx=0x7fc364019040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:39:57.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:57.881+0000 7fc35f7fe700 1 -- 172.21.15.152:0/91475386 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc358059cb0 con 0x7fc3680ffab0 2024-03-29T04:39:58.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:39:58.079+0000 7fc37043f700 1 -- 172.21.15.152:0/91475386 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc368069500 con 0x7fc35406c4e0 2024-03-29T04:39:58.668 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:58 smithi152 ceph-mon[37177]: osdmap e31: 4 total, 4 up, 4 in 2024-03-29T04:39:58.669 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:58 smithi152 ceph-mon[37177]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:39:58.669 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:58 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T04:39:58.669 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:58 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T04:39:58.669 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:58 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:58.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:58 smithi142 ceph-mon[29027]: osdmap e31: 4 total, 4 up, 4 in 2024-03-29T04:39:58.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:58 smithi142 ceph-mon[29027]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:39:58.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:58 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T04:39:58.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:58 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T04:39:58.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:58 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:39:59.748 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:39:59 smithi152 ceph-mon[37177]: from='client.24161 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:39:59.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:39:59 smithi142 ceph-mon[29027]: from='client.24161 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:40:00.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:00 smithi152 ceph-mon[37177]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-29T04:40:00.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:00 smithi152 ceph-mon[37177]: overall HEALTH_OK 2024-03-29T04:40:00.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:00 smithi142 ceph-mon[29027]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-29T04:40:00.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:00 smithi142 ceph-mon[29027]: overall HEALTH_OK 2024-03-29T04:40:01.534 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:01 smithi152 ceph-mon[37177]: from='client.? 172.21.15.152:0/2927881919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7b5fab7-0f30-4756-a481-15ad64b9e691"}]: dispatch 2024-03-29T04:40:01.534 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:01 smithi152 ceph-mon[37177]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7b5fab7-0f30-4756-a481-15ad64b9e691"}]: dispatch 2024-03-29T04:40:01.534 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:01 smithi152 ceph-mon[37177]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7b5fab7-0f30-4756-a481-15ad64b9e691"}]': finished 2024-03-29T04:40:01.534 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:01 smithi152 ceph-mon[37177]: osdmap e32: 5 total, 4 up, 5 in 2024-03-29T04:40:01.534 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:01 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:40:01.534 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:01 smithi152 ceph-mon[37177]: from='client.? 172.21.15.152:0/1226401619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T04:40:01.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:01 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/2927881919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7b5fab7-0f30-4756-a481-15ad64b9e691"}]: dispatch 2024-03-29T04:40:01.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:01 smithi142 ceph-mon[29027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7b5fab7-0f30-4756-a481-15ad64b9e691"}]: dispatch 2024-03-29T04:40:01.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:01 smithi142 ceph-mon[29027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7b5fab7-0f30-4756-a481-15ad64b9e691"}]': finished 2024-03-29T04:40:01.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:01 smithi142 ceph-mon[29027]: osdmap e32: 5 total, 4 up, 5 in 2024-03-29T04:40:01.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:01 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:40:01.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:01 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/1226401619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T04:40:02.768 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:02 smithi152 ceph-mon[37177]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-29T04:40:02.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:02 smithi142 ceph-mon[29027]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-29T04:40:03.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:03 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:40:03.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:03 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:40:04.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:04 smithi152 ceph-mon[37177]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-29T04:40:04.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:04 smithi142 ceph-mon[29027]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-29T04:40:06.762 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:06 smithi152 ceph-mon[37177]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-29T04:40:07.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:06 smithi142 ceph-mon[29027]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-29T04:40:09.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:08 smithi152 ceph-mon[37177]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-29T04:40:09.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:08 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-29T04:40:09.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:08 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:09.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:08 smithi152 ceph-mon[37177]: Deploying daemon osd.4 on smithi152 2024-03-29T04:40:09.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:08 smithi142 ceph-mon[29027]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-29T04:40:09.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:08 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-29T04:40:09.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:08 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:09.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:08 smithi142 ceph-mon[29027]: Deploying daemon osd.4 on smithi152 2024-03-29T04:40:11.005 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:10 smithi152 ceph-mon[37177]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:40:11.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:10 smithi142 ceph-mon[29027]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:40:12.965 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:12 smithi152 ceph-mon[37177]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:40:13.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:12 smithi142 ceph-mon[29027]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:40:14.512 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:14 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:14.512 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:14 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:14.512 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:14 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:40:14.512 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:14 smithi152 ceph-mon[37177]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:40:14.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:14 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:14.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:14 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:14.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:14 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:40:14.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:14 smithi142 ceph-mon[29027]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:40:16.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:16 smithi152 ceph-mon[37177]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:40:17.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:16 smithi142 ceph-mon[29027]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:40:17.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:17.190+0000 7fc35f7fe700 1 -- 172.21.15.152:0/91475386 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc368069500 con 0x7fc35406c4e0 2024-03-29T04:40:17.186 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 4 on host 'smithi152' 2024-03-29T04:40:17.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:17.193+0000 7fc37043f700 1 -- 172.21.15.152:0/91475386 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc35406c4e0 msgr2=0x7fc35406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:17.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:17.193+0000 7fc37043f700 1 --2- 172.21.15.152:0/91475386 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc35406c4e0 0x7fc35406e9a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc36400b6f0 tx=0x7fc364019040 comp rx=0 tx=0).stop 2024-03-29T04:40:17.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:17.193+0000 7fc37043f700 1 -- 172.21.15.152:0/91475386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3680ffab0 msgr2=0x7fc368198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:17.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:17.193+0000 7fc37043f700 1 --2- 172.21.15.152:0/91475386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3680ffab0 0x7fc368198550 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc35800e970 tx=0x7fc35800ec80 comp rx=0 tx=0).stop 2024-03-29T04:40:17.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:17.193+0000 7fc37043f700 1 -- 172.21.15.152:0/91475386 shutdown_connections 2024-03-29T04:40:17.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:17.193+0000 7fc37043f700 1 --2- 172.21.15.152:0/91475386 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc35406c4e0 0x7fc35406e9a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:17.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:17.193+0000 7fc37043f700 1 --2- 172.21.15.152:0/91475386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3680ffab0 0x7fc368198550 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:17.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:17.193+0000 7fc37043f700 1 --2- 172.21.15.152:0/91475386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc368100cb0 0x7fc368198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:17.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:17.193+0000 7fc37043f700 1 -- 172.21.15.152:0/91475386 >> 172.21.15.152:0/91475386 conn(0x7fc3680fb010 msgr2=0x7fc368068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:17.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:17.193+0000 7fc37043f700 1 -- 172.21.15.152:0/91475386 shutdown_connections 2024-03-29T04:40:17.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:17.193+0000 7fc37043f700 1 -- 172.21.15.152:0/91475386 wait complete. 2024-03-29T04:40:17.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:17 smithi152 ceph-mon[37177]: from='osd.4 [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T04:40:17.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:17 smithi152 ceph-mon[37177]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T04:40:17.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:17 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:17.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:17 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:17.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:17 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:40:17.822 DEBUG:teuthology.orchestra.run.smithi152:osd.4> sudo journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.4.service 2024-03-29T04:40:17.824 INFO:tasks.cephadm:Deploying osd.5 on smithi152 with /dev/vg_nvme/lv_2... 2024-03-29T04:40:17.825 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-29T04:40:18.085 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi152/config 2024-03-29T04:40:18.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:17 smithi142 ceph-mon[29027]: from='osd.4 [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T04:40:18.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:17 smithi142 ceph-mon[29027]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T04:40:18.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:18.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:18.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:40:18.784 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:18 smithi152 ceph-mon[37177]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:40:18.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:18 smithi152 ceph-mon[37177]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-29T04:40:18.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:18 smithi152 ceph-mon[37177]: osdmap e33: 5 total, 4 up, 5 in 2024-03-29T04:40:18.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:18 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:40:18.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:18 smithi152 ceph-mon[37177]: from='osd.4 [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:40:18.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:18 smithi152 ceph-mon[37177]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:40:18.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:18 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:18.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:18 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:18.785 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:40:18 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[50068]: 2024-03-29T04:40:18.674+0000 7f36c1c0a700 -1 osd.4 0 waiting for initial osdmap 2024-03-29T04:40:18.785 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:40:18 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[50068]: 2024-03-29T04:40:18.683+0000 7f36be200700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:40:19.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:18 smithi142 ceph-mon[29027]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:40:19.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:18 smithi142 ceph-mon[29027]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-29T04:40:19.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:18 smithi142 ceph-mon[29027]: osdmap e33: 5 total, 4 up, 5 in 2024-03-29T04:40:19.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:18 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:40:19.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:18 smithi142 ceph-mon[29027]: from='osd.4 [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:40:19.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:18 smithi142 ceph-mon[29027]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:40:19.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:18 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:19.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:18 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:19.740 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:19 smithi152 ceph-mon[37177]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-29T04:40:19.740 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:19 smithi152 ceph-mon[37177]: osdmap e34: 5 total, 4 up, 5 in 2024-03-29T04:40:19.740 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:19 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:40:19.740 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:19 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:40:20.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:19 smithi142 ceph-mon[29027]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-29T04:40:20.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:19 smithi142 ceph-mon[29027]: osdmap e34: 5 total, 4 up, 5 in 2024-03-29T04:40:20.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:19 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:40:20.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:19 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:40:20.388 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-29T04:40:20.388 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-29T04:40:20.388 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-03-29T04:40:20.388 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-03-29T04:40:20.388 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126301 s, 830 MB/s 2024-03-29T04:40:20.388 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-03-29T04:40:20.762 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:20 smithi152 ceph-mon[37177]: purged_snaps scrub starts 2024-03-29T04:40:20.762 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:20 smithi152 ceph-mon[37177]: purged_snaps scrub ok 2024-03-29T04:40:20.762 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:20 smithi152 ceph-mon[37177]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:40:20.762 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:20 smithi152 ceph-mon[37177]: osd.4 [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639] boot 2024-03-29T04:40:20.762 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:20 smithi152 ceph-mon[37177]: osdmap e35: 5 total, 5 up, 5 in 2024-03-29T04:40:20.762 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:20 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:40:21.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:20 smithi142 ceph-mon[29027]: purged_snaps scrub starts 2024-03-29T04:40:21.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:20 smithi142 ceph-mon[29027]: purged_snaps scrub ok 2024-03-29T04:40:21.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:20 smithi142 ceph-mon[29027]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-29T04:40:21.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:20 smithi142 ceph-mon[29027]: osd.4 [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639] boot 2024-03-29T04:40:21.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:20 smithi142 ceph-mon[29027]: osdmap e35: 5 total, 5 up, 5 in 2024-03-29T04:40:21.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:20 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:40:21.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:21 smithi142 ceph-mon[29027]: osdmap e36: 5 total, 5 up, 5 in 2024-03-29T04:40:21.920 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:21 smithi152 ceph-mon[37177]: osdmap e36: 5 total, 5 up, 5 in 2024-03-29T04:40:22.017 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi152:vg_nvme/lv_2 2024-03-29T04:40:22.753 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi152/config 2024-03-29T04:40:22.875 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:22 smithi152 ceph-mon[37177]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:22.875 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:22 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:22.875 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:22 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:22.875 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:22 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:40:22.875 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:22 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:40:22.875 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:22 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:22.876 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:22 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:22.876 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:22 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:40:23.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:22 smithi142 ceph-mon[29027]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:23.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:23.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:23.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:40:23.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:40:23.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:23.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:23.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:40:23.848 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:23 smithi152 ceph-mon[37177]: Detected new or changed devices on smithi152 2024-03-29T04:40:23.848 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:23 smithi152 ceph-mon[37177]: Adjusting osd_memory_target on smithi152 to 7456M 2024-03-29T04:40:23.849 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:23 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:23.849 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:23 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:40:23.849 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:23 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:23.849 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:23 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:40:23.849 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:23 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:24.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:23 smithi142 ceph-mon[29027]: Detected new or changed devices on smithi152 2024-03-29T04:40:24.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:23 smithi142 ceph-mon[29027]: Adjusting osd_memory_target on smithi152 to 7456M 2024-03-29T04:40:24.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:23 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:24.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:23 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:40:24.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:23 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:24.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:23 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:40:24.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:23 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:24.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.337+0000 7fe9354ef700 1 -- 172.21.15.152:0/2941542119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe930100c10 msgr2=0x7fe930101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.337+0000 7fe9354ef700 1 --2- 172.21.15.152:0/2941542119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe930100c10 0x7fe930101030 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe924009910 tx=0x7fe924009c20 comp rx=0 tx=0).stop 2024-03-29T04:40:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.337+0000 7fe9354ef700 1 -- 172.21.15.152:0/2941542119 shutdown_connections 2024-03-29T04:40:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.337+0000 7fe9354ef700 1 --2- 172.21.15.152:0/2941542119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe930101e10 0x7fe930102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.337+0000 7fe9354ef700 1 --2- 172.21.15.152:0/2941542119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe930100c10 0x7fe930101030 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.337+0000 7fe9354ef700 1 -- 172.21.15.152:0/2941542119 >> 172.21.15.152:0/2941542119 conn(0x7fe9300fc1b0 msgr2=0x7fe9300fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.337+0000 7fe9354ef700 1 -- 172.21.15.152:0/2941542119 shutdown_connections 2024-03-29T04:40:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.337+0000 7fe9354ef700 1 -- 172.21.15.152:0/2941542119 wait complete. 2024-03-29T04:40:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe9354ef700 1 Processor -- start 2024-03-29T04:40:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe9354ef700 1 -- start start 2024-03-29T04:40:24.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe9354ef700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe930100c10 0x7fe930196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe9354ef700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe930101e10 0x7fe9301968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe9354ef700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe930196ef0 con 0x7fe930101e10 2024-03-29T04:40:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe9354ef700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe930197030 con 0x7fe930100c10 2024-03-29T04:40:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe92effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe930100c10 0x7fe930196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe92effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe930100c10 0x7fe930196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46456/0 (socket says 172.21.15.152:46456) 2024-03-29T04:40:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe92effd700 1 -- 172.21.15.152:0/1281307117 learned_addr learned my addr 172.21.15.152:0/1281307117 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:40:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe92e7fc700 1 --2- 172.21.15.152:0/1281307117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe930101e10 0x7fe9301968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe92effd700 1 -- 172.21.15.152:0/1281307117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe930101e10 msgr2=0x7fe9301968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe92effd700 1 --2- 172.21.15.152:0/1281307117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe930101e10 0x7fe9301968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe92effd700 1 -- 172.21.15.152:0/1281307117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9240095f0 con 0x7fe930100c10 2024-03-29T04:40:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.338+0000 7fe92e7fc700 1 --2- 172.21.15.152:0/1281307117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe930101e10 0x7fe9301968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:40:24.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.339+0000 7fe92effd700 1 --2- 172.21.15.152:0/1281307117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe930100c10 0x7fe930196390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe92400bda0 tx=0x7fe92400be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:40:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.339+0000 7fe917fff700 1 -- 172.21.15.152:0/1281307117 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe92401d070 con 0x7fe930100c10 2024-03-29T04:40:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.339+0000 7fe917fff700 1 -- 172.21.15.152:0/1281307117 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe92400f460 con 0x7fe930100c10 2024-03-29T04:40:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.339+0000 7fe9354ef700 1 -- 172.21.15.152:0/1281307117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93019ba80 con 0x7fe930100c10 2024-03-29T04:40:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.339+0000 7fe917fff700 1 -- 172.21.15.152:0/1281307117 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe924021640 con 0x7fe930100c10 2024-03-29T04:40:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.339+0000 7fe9354ef700 1 -- 172.21.15.152:0/1281307117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93019bf70 con 0x7fe930100c10 2024-03-29T04:40:24.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.340+0000 7fe9354ef700 1 -- 172.21.15.152:0/1281307117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe930109770 con 0x7fe930100c10 2024-03-29T04:40:24.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.340+0000 7fe917fff700 1 -- 172.21.15.152:0/1281307117 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe92400fa00 con 0x7fe930100c10 2024-03-29T04:40:24.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.341+0000 7fe917fff700 1 --2- 172.21.15.152:0/1281307117 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe91806c4e0 0x7fe91806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:24.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.341+0000 7fe917fff700 1 -- 172.21.15.152:0/1281307117 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7fe92408c630 con 0x7fe930100c10 2024-03-29T04:40:24.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.341+0000 7fe92e7fc700 1 --2- 172.21.15.152:0/1281307117 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe91806c4e0 0x7fe91806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:24.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.341+0000 7fe92e7fc700 1 --2- 172.21.15.152:0/1281307117 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe91806c4e0 0x7fe91806e9a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe920005fd0 tx=0x7fe920005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:40:24.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.345+0000 7fe917fff700 1 -- 172.21.15.152:0/1281307117 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe92405ad90 con 0x7fe930100c10 2024-03-29T04:40:24.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:24.544+0000 7fe9354ef700 1 -- 172.21.15.152:0/1281307117 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe930106760 con 0x7fe91806c4e0 2024-03-29T04:40:25.003 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:24 smithi152 ceph-mon[37177]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:25.004 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T04:40:25.004 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T04:40:25.004 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:25.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:24 smithi142 ceph-mon[29027]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:25.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T04:40:25.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T04:40:25.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:25.927 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:25 smithi152 ceph-mon[37177]: from='client.24179 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:40:26.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:25 smithi142 ceph-mon[29027]: from='client.24179 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:40:27.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:26 smithi152 ceph-mon[37177]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:27.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:26 smithi142 ceph-mon[29027]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:28.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:27 smithi152 ceph-mon[37177]: from='client.? 172.21.15.152:0/1476394623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ed3caba-b201-4ad7-b055-3dc40610ab1f"}]: dispatch 2024-03-29T04:40:28.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:27 smithi152 ceph-mon[37177]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ed3caba-b201-4ad7-b055-3dc40610ab1f"}]: dispatch 2024-03-29T04:40:28.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:27 smithi152 ceph-mon[37177]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ed3caba-b201-4ad7-b055-3dc40610ab1f"}]': finished 2024-03-29T04:40:28.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:27 smithi152 ceph-mon[37177]: osdmap e37: 6 total, 5 up, 6 in 2024-03-29T04:40:28.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:27 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:40:28.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:27 smithi152 ceph-mon[37177]: from='client.? 172.21.15.152:0/1441479859' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T04:40:28.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:27 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/1476394623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ed3caba-b201-4ad7-b055-3dc40610ab1f"}]: dispatch 2024-03-29T04:40:28.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:27 smithi142 ceph-mon[29027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ed3caba-b201-4ad7-b055-3dc40610ab1f"}]: dispatch 2024-03-29T04:40:28.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:27 smithi142 ceph-mon[29027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ed3caba-b201-4ad7-b055-3dc40610ab1f"}]': finished 2024-03-29T04:40:28.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:27 smithi142 ceph-mon[29027]: osdmap e37: 6 total, 5 up, 6 in 2024-03-29T04:40:28.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:27 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:40:28.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:27 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/1441479859' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T04:40:29.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:28 smithi152 ceph-mon[37177]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:29.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:28 smithi142 ceph-mon[29027]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:31.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:30 smithi152 ceph-mon[37177]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:31.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:30 smithi142 ceph-mon[29027]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:33.027 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:32 smithi152 ceph-mon[37177]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:33.027 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:40:33.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:32 smithi142 ceph-mon[29027]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:33.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:40:34.665 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:34 smithi152 ceph-mon[37177]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:34.666 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:34 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-29T04:40:34.666 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:34 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:34.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:34 smithi142 ceph-mon[29027]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:34.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:34 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-29T04:40:34.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:34 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:35.534 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:35 smithi152 ceph-mon[37177]: Deploying daemon osd.5 on smithi152 2024-03-29T04:40:35.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:35 smithi142 ceph-mon[29027]: Deploying daemon osd.5 on smithi152 2024-03-29T04:40:36.462 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:36 smithi152 ceph-mon[37177]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:36.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:36 smithi142 ceph-mon[29027]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:38.950 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:38 smithi152 ceph-mon[37177]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:39.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:38 smithi142 ceph-mon[29027]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:41.115 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:40 smithi142 ceph-mon[29027]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:41.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:40 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:41.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:40 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:40:41.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:40 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:41.129 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:40 smithi152 ceph-mon[37177]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:41.129 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:40 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:41.129 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:40 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:40:41.129 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:40 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:42.957 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:42 smithi152 ceph-mon[37177]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:43.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:42 smithi142 ceph-mon[29027]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:43.605 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 5 on host 'smithi152' 2024-03-29T04:40:43.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:43.610+0000 7fe917fff700 1 -- 172.21.15.152:0/1281307117 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe930106760 con 0x7fe91806c4e0 2024-03-29T04:40:43.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:43.613+0000 7fe9354ef700 1 -- 172.21.15.152:0/1281307117 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe91806c4e0 msgr2=0x7fe91806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:43.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:43.613+0000 7fe9354ef700 1 --2- 172.21.15.152:0/1281307117 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe91806c4e0 0x7fe91806e9a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe920005fd0 tx=0x7fe920005ee0 comp rx=0 tx=0).stop 2024-03-29T04:40:43.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:43.613+0000 7fe9354ef700 1 -- 172.21.15.152:0/1281307117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe930100c10 msgr2=0x7fe930196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:43.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:43.613+0000 7fe9354ef700 1 --2- 172.21.15.152:0/1281307117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe930100c10 0x7fe930196390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe92400bda0 tx=0x7fe92400be80 comp rx=0 tx=0).stop 2024-03-29T04:40:43.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:43.613+0000 7fe9354ef700 1 -- 172.21.15.152:0/1281307117 shutdown_connections 2024-03-29T04:40:43.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:43.613+0000 7fe9354ef700 1 --2- 172.21.15.152:0/1281307117 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe91806c4e0 0x7fe91806e9a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:43.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:43.613+0000 7fe9354ef700 1 --2- 172.21.15.152:0/1281307117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe930100c10 0x7fe930196390 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:43.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:43.613+0000 7fe9354ef700 1 --2- 172.21.15.152:0/1281307117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe930101e10 0x7fe9301968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:43.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:43.613+0000 7fe9354ef700 1 -- 172.21.15.152:0/1281307117 >> 172.21.15.152:0/1281307117 conn(0x7fe9300fc1b0 msgr2=0x7fe930105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:43.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:43.613+0000 7fe9354ef700 1 -- 172.21.15.152:0/1281307117 shutdown_connections 2024-03-29T04:40:43.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:40:43.613+0000 7fe9354ef700 1 -- 172.21.15.152:0/1281307117 wait complete. 2024-03-29T04:40:44.261 DEBUG:teuthology.orchestra.run.smithi152:osd.5> sudo journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.5.service 2024-03-29T04:40:44.264 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-29T04:40:44.264 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-29T04:40:44.487 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:40:44.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:44 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:44.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:44 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:44.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:44 smithi142 ceph-mon[29027]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:44.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:44 smithi142 ceph-mon[29027]: from='osd.5 [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T04:40:44.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:44 smithi142 ceph-mon[29027]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T04:40:44.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:44 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:44.848 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:44 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:45.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:44 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:45.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:44 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:45.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:44 smithi152 ceph-mon[37177]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:45.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:44 smithi152 ceph-mon[37177]: from='osd.5 [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T04:40:45.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:44 smithi152 ceph-mon[37177]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T04:40:45.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:44 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:45.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:44 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:45.695 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:45 smithi152 ceph-mon[37177]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-29T04:40:45.695 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:45 smithi152 ceph-mon[37177]: osdmap e38: 6 total, 5 up, 6 in 2024-03-29T04:40:45.695 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:45 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:40:45.695 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:45 smithi152 ceph-mon[37177]: from='osd.5 [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:40:45.695 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:45 smithi152 ceph-mon[37177]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:40:45.695 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:40:45 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[57008]: 2024-03-29T04:40:45.628+0000 7ff716b19700 -1 osd.5 0 waiting for initial osdmap 2024-03-29T04:40:45.695 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:40:45 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[57008]: 2024-03-29T04:40:45.636+0000 7ff70f908700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:40:45.908 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:45 smithi142 ceph-mon[29027]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-29T04:40:45.908 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:45 smithi142 ceph-mon[29027]: osdmap e38: 6 total, 5 up, 6 in 2024-03-29T04:40:45.908 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:45 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:40:45.908 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:45 smithi142 ceph-mon[29027]: from='osd.5 [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:40:45.908 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:45 smithi142 ceph-mon[29027]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:40:46.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.131+0000 7efc7c653700 1 -- 172.21.15.142:0/1866234585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc740ffab0 msgr2=0x7efc740ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:46.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.131+0000 7efc7c653700 1 --2- 172.21.15.142:0/1866234585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc740ffab0 0x7efc740ffed0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efc64009910 tx=0x7efc64009c20 comp rx=0 tx=0).stop 2024-03-29T04:40:46.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.131+0000 7efc7c653700 1 -- 172.21.15.142:0/1866234585 shutdown_connections 2024-03-29T04:40:46.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.131+0000 7efc7c653700 1 --2- 172.21.15.142:0/1866234585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc74100cb0 0x7efc74101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:46.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.131+0000 7efc7c653700 1 --2- 172.21.15.142:0/1866234585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc740ffab0 0x7efc740ffed0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:46.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.131+0000 7efc7c653700 1 -- 172.21.15.142:0/1866234585 >> 172.21.15.142:0/1866234585 conn(0x7efc740fb030 msgr2=0x7efc740fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:46.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.131+0000 7efc7c653700 1 -- 172.21.15.142:0/1866234585 shutdown_connections 2024-03-29T04:40:46.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.131+0000 7efc7c653700 1 -- 172.21.15.142:0/1866234585 wait complete. 2024-03-29T04:40:46.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc7c653700 1 Processor -- start 2024-03-29T04:40:46.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc7c653700 1 -- start start 2024-03-29T04:40:46.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc7c653700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc740ffab0 0x7efc741963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:46.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc7c653700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc74100cb0 0x7efc74196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc7c653700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc74196f50 con 0x7efc74100cb0 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc7c653700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc74197090 con 0x7efc740ffab0 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc79bee700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc74100cb0 0x7efc74196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc79bee700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc74100cb0 0x7efc74196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54150/0 (socket says 172.21.15.142:54150) 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc79bee700 1 -- 172.21.15.142:0/814386960 learned_addr learned my addr 172.21.15.142:0/814386960 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc7a3ef700 1 --2- 172.21.15.142:0/814386960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc740ffab0 0x7efc741963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc79bee700 1 -- 172.21.15.142:0/814386960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc740ffab0 msgr2=0x7efc741963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc79bee700 1 --2- 172.21.15.142:0/814386960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc740ffab0 0x7efc741963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.132+0000 7efc79bee700 1 -- 172.21.15.142:0/814386960 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc640095f0 con 0x7efc74100cb0 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.133+0000 7efc79bee700 1 --2- 172.21.15.142:0/814386960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc74100cb0 0x7efc74196930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efc7000b4b0 tx=0x7efc7000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.133+0000 7efc6b7fe700 1 -- 172.21.15.142:0/814386960 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc70011800 con 0x7efc74100cb0 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.133+0000 7efc6b7fe700 1 -- 172.21.15.142:0/814386960 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efc70011e10 con 0x7efc74100cb0 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.133+0000 7efc7c653700 1 -- 172.21.15.142:0/814386960 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7419bb40 con 0x7efc74100cb0 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.133+0000 7efc6b7fe700 1 -- 172.21.15.142:0/814386960 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc7000f2e0 con 0x7efc74100cb0 2024-03-29T04:40:46.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.133+0000 7efc7c653700 1 -- 172.21.15.142:0/814386960 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7419c110 con 0x7efc74100cb0 2024-03-29T04:40:46.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.134+0000 7efc7c653700 1 -- 172.21.15.142:0/814386960 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc74108690 con 0x7efc74100cb0 2024-03-29T04:40:46.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.134+0000 7efc6b7fe700 1 -- 172.21.15.142:0/814386960 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7efc70010380 con 0x7efc74100cb0 2024-03-29T04:40:46.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.135+0000 7efc6b7fe700 1 --2- 172.21.15.142:0/814386960 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efc6006c4e0 0x7efc6006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:46.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.135+0000 7efc6b7fe700 1 -- 172.21.15.142:0/814386960 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4358+0+0 (secure 0 0 0) 0x7efc7008af70 con 0x7efc74100cb0 2024-03-29T04:40:46.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.135+0000 7efc7a3ef700 1 --2- 172.21.15.142:0/814386960 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efc6006c4e0 0x7efc6006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:46.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.135+0000 7efc7a3ef700 1 --2- 172.21.15.142:0/814386960 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efc6006c4e0 0x7efc6006e9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc64006010 tx=0x7efc6400b560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:40:46.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.139+0000 7efc6b7fe700 1 -- 172.21.15.142:0/814386960 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efc700be0e0 con 0x7efc74100cb0 2024-03-29T04:40:46.333 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.338+0000 7efc7c653700 1 -- 172.21.15.142:0/814386960 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7efc74108880 con 0x7efc74100cb0 2024-03-29T04:40:46.333 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.339+0000 7efc6b7fe700 1 -- 172.21.15.142:0/814386960 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7efc700150a0 con 0x7efc74100cb0 2024-03-29T04:40:46.334 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:40:46.339 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.341+0000 7efc7c653700 1 -- 172.21.15.142:0/814386960 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efc6006c4e0 msgr2=0x7efc6006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:46.339 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.341+0000 7efc7c653700 1 --2- 172.21.15.142:0/814386960 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efc6006c4e0 0x7efc6006e9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc64006010 tx=0x7efc6400b560 comp rx=0 tx=0).stop 2024-03-29T04:40:46.339 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.341+0000 7efc7c653700 1 -- 172.21.15.142:0/814386960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc74100cb0 msgr2=0x7efc74196930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:46.339 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.341+0000 7efc7c653700 1 --2- 172.21.15.142:0/814386960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc74100cb0 0x7efc74196930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efc7000b4b0 tx=0x7efc7000b7c0 comp rx=0 tx=0).stop 2024-03-29T04:40:46.339 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.341+0000 7efc7c653700 1 -- 172.21.15.142:0/814386960 shutdown_connections 2024-03-29T04:40:46.339 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.341+0000 7efc7c653700 1 --2- 172.21.15.142:0/814386960 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efc6006c4e0 0x7efc6006e9a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:46.339 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.341+0000 7efc7c653700 1 --2- 172.21.15.142:0/814386960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc740ffab0 0x7efc741963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:46.340 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.341+0000 7efc7c653700 1 --2- 172.21.15.142:0/814386960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc74100cb0 0x7efc74196930 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:46.340 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.341+0000 7efc7c653700 1 -- 172.21.15.142:0/814386960 >> 172.21.15.142:0/814386960 conn(0x7efc740fb030 msgr2=0x7efc74068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:46.340 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.341+0000 7efc7c653700 1 -- 172.21.15.142:0/814386960 shutdown_connections 2024-03-29T04:40:46.340 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:46.341+0000 7efc7c653700 1 -- 172.21.15.142:0/814386960 wait complete. 2024-03-29T04:40:46.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:46 smithi152 ceph-mon[37177]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:46.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:46 smithi152 ceph-mon[37177]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-29T04:40:46.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:46 smithi152 ceph-mon[37177]: osdmap e39: 6 total, 5 up, 6 in 2024-03-29T04:40:46.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:46 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:40:46.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:46 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:40:46.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:46 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/814386960' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-29T04:40:46.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:46 smithi142 ceph-mon[29027]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:46.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:46 smithi142 ceph-mon[29027]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-29T04:40:46.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:46 smithi142 ceph-mon[29027]: osdmap e39: 6 total, 5 up, 6 in 2024-03-29T04:40:46.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:46 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:40:46.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:46 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:40:46.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:46 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/814386960' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-29T04:40:47.008 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":39,"num_osds":6,"num_up_osds":5,"osd_up_since":1711687219,"num_in_osds":6,"osd_in_since":1711687227,"num_remapped_pgs":0} 2024-03-29T04:40:47.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:47 smithi142 ceph-mon[29027]: purged_snaps scrub starts 2024-03-29T04:40:47.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:47 smithi142 ceph-mon[29027]: purged_snaps scrub ok 2024-03-29T04:40:47.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:47 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:40:47.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:47 smithi142 ceph-mon[29027]: osd.5 [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269] boot 2024-03-29T04:40:47.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:47 smithi142 ceph-mon[29027]: osdmap e40: 6 total, 6 up, 6 in 2024-03-29T04:40:47.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:47 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:40:47.980 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:47 smithi152 ceph-mon[37177]: purged_snaps scrub starts 2024-03-29T04:40:47.980 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:47 smithi152 ceph-mon[37177]: purged_snaps scrub ok 2024-03-29T04:40:47.980 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:47 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:40:47.980 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:47 smithi152 ceph-mon[37177]: osd.5 [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269] boot 2024-03-29T04:40:47.980 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:47 smithi152 ceph-mon[37177]: osdmap e40: 6 total, 6 up, 6 in 2024-03-29T04:40:47.981 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:47 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:40:48.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-29T04:40:48.223 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:40:48.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:48.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:40:48.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: osdmap e41: 6 total, 6 up, 6 in 2024-03-29T04:40:48.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:48.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:48.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:40:48.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:40:48.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:40:48.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:48.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:48.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:40:48.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:48.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:40:48.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-29T04:40:48.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:40:48.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: osdmap e41: 6 total, 6 up, 6 in 2024-03-29T04:40:48.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:48.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:48.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:40:48.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:40:48.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:40:48.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:48.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:48.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:40:48.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:48.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:40:49.653 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:49 smithi142 ceph-mon[29027]: Detected new or changed devices on smithi152 2024-03-29T04:40:49.653 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:49 smithi142 ceph-mon[29027]: Adjusting osd_memory_target on smithi152 to 4970M 2024-03-29T04:40:49.653 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:49.653 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:40:49.653 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:50.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.036+0000 7fbcbdfef700 1 -- 172.21.15.142:0/718173518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8100cc0 msgr2=0x7fbcb8101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:50.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.036+0000 7fbcbdfef700 1 --2- 172.21.15.142:0/718173518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8100cc0 0x7fbcb8101120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009910 tx=0x7fbcac009c20 comp rx=0 tx=0).stop 2024-03-29T04:40:50.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.036+0000 7fbcbdfef700 1 -- 172.21.15.142:0/718173518 shutdown_connections 2024-03-29T04:40:50.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.036+0000 7fbcbdfef700 1 --2- 172.21.15.142:0/718173518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8100cc0 0x7fbcb8101120 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:50.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.036+0000 7fbcbdfef700 1 --2- 172.21.15.142:0/718173518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb80ffac0 0x7fbcb80ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:50.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.036+0000 7fbcbdfef700 1 -- 172.21.15.142:0/718173518 >> 172.21.15.142:0/718173518 conn(0x7fbcb80fb060 msgr2=0x7fbcb80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:50.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.036+0000 7fbcbdfef700 1 -- 172.21.15.142:0/718173518 shutdown_connections 2024-03-29T04:40:50.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.036+0000 7fbcbdfef700 1 -- 172.21.15.142:0/718173518 wait complete. 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.037+0000 7fbcbdfef700 1 Processor -- start 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.037+0000 7fbcbdfef700 1 -- start start 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.037+0000 7fbcbdfef700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb80ffac0 0x7fbcb8196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.037+0000 7fbcbdfef700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb8100cc0 0x7fbcb81968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.037+0000 7fbcbdfef700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8196ed0 con 0x7fbcb80ffac0 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.037+0000 7fbcbdfef700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8197010 con 0x7fbcb8100cc0 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.037+0000 7fbcb77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb80ffac0 0x7fbcb8196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.037+0000 7fbcb77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb80ffac0 0x7fbcb8196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:50664/0 (socket says 172.21.15.142:50664) 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.037+0000 7fbcb77fe700 1 -- 172.21.15.142:0/1368937734 learned_addr learned my addr 172.21.15.142:0/1368937734 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.038+0000 7fbcb6ffd700 1 --2- 172.21.15.142:0/1368937734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb8100cc0 0x7fbcb81968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.038+0000 7fbcb77fe700 1 -- 172.21.15.142:0/1368937734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb8100cc0 msgr2=0x7fbcb81968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.038+0000 7fbcb77fe700 1 --2- 172.21.15.142:0/1368937734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb8100cc0 0x7fbcb81968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:50.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.038+0000 7fbcb77fe700 1 -- 172.21.15.142:0/1368937734 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac0095f0 con 0x7fbcb80ffac0 2024-03-29T04:40:50.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.038+0000 7fbcb77fe700 1 --2- 172.21.15.142:0/1368937734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb80ffac0 0x7fbcb8196370 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbca800d900 tx=0x7fbca800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:40:50.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.038+0000 7fbcb4ff9700 1 -- 172.21.15.142:0/1368937734 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbca80041f0 con 0x7fbcb80ffac0 2024-03-29T04:40:50.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.038+0000 7fbcbdfef700 1 -- 172.21.15.142:0/1368937734 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb819bac0 con 0x7fbcb80ffac0 2024-03-29T04:40:50.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.038+0000 7fbcb4ff9700 1 -- 172.21.15.142:0/1368937734 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbca8004350 con 0x7fbcb80ffac0 2024-03-29T04:40:50.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.038+0000 7fbcb4ff9700 1 -- 172.21.15.142:0/1368937734 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbca8003da0 con 0x7fbcb80ffac0 2024-03-29T04:40:50.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.038+0000 7fbcbdfef700 1 -- 172.21.15.142:0/1368937734 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb819c010 con 0x7fbcb80ffac0 2024-03-29T04:40:50.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:49 smithi152 ceph-mon[37177]: Detected new or changed devices on smithi152 2024-03-29T04:40:50.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:49 smithi152 ceph-mon[37177]: Adjusting osd_memory_target on smithi152 to 4970M 2024-03-29T04:40:50.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:40:50.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:40:50.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:40:50.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.039+0000 7fbcbdfef700 1 -- 172.21.15.142:0/1368937734 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb8108620 con 0x7fbcb80ffac0 2024-03-29T04:40:50.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.039+0000 7fbcb4ff9700 1 -- 172.21.15.142:0/1368937734 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbca8009730 con 0x7fbcb80ffac0 2024-03-29T04:40:50.037 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.040+0000 7fbcb4ff9700 1 --2- 172.21.15.142:0/1368937734 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbca006c4e0 0x7fbca006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:50.037 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.040+0000 7fbcb4ff9700 1 -- 172.21.15.142:0/1368937734 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbca808b650 con 0x7fbcb80ffac0 2024-03-29T04:40:50.037 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.040+0000 7fbcb6ffd700 1 --2- 172.21.15.142:0/1368937734 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbca006c4e0 0x7fbca006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:50.037 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.040+0000 7fbcb6ffd700 1 --2- 172.21.15.142:0/1368937734 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbca006c4e0 0x7fbca006e9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbcac005b40 tx=0x7fbcac005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:40:50.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.044+0000 7fbcb4ff9700 1 -- 172.21.15.142:0/1368937734 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbca8059c00 con 0x7fbcb80ffac0 2024-03-29T04:40:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.237+0000 7fbcbdfef700 1 -- 172.21.15.142:0/1368937734 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fbcb8108810 con 0x7fbcb80ffac0 2024-03-29T04:40:50.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.237+0000 7fbcb4ff9700 1 -- 172.21.15.142:0/1368937734 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7fbca8059790 con 0x7fbcb80ffac0 2024-03-29T04:40:50.232 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:40:50.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.239+0000 7fbcbdfef700 1 -- 172.21.15.142:0/1368937734 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbca006c4e0 msgr2=0x7fbca006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:50.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.239+0000 7fbcbdfef700 1 --2- 172.21.15.142:0/1368937734 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbca006c4e0 0x7fbca006e9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbcac005b40 tx=0x7fbcac005a90 comp rx=0 tx=0).stop 2024-03-29T04:40:50.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.239+0000 7fbcbdfef700 1 -- 172.21.15.142:0/1368937734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb80ffac0 msgr2=0x7fbcb8196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:50.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.239+0000 7fbcbdfef700 1 --2- 172.21.15.142:0/1368937734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb80ffac0 0x7fbcb8196370 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbca800d900 tx=0x7fbca800dcc0 comp rx=0 tx=0).stop 2024-03-29T04:40:50.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.239+0000 7fbcbdfef700 1 -- 172.21.15.142:0/1368937734 shutdown_connections 2024-03-29T04:40:50.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.239+0000 7fbcbdfef700 1 --2- 172.21.15.142:0/1368937734 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbca006c4e0 0x7fbca006e9a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:50.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.239+0000 7fbcbdfef700 1 --2- 172.21.15.142:0/1368937734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb80ffac0 0x7fbcb8196370 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.239+0000 7fbcbdfef700 1 --2- 172.21.15.142:0/1368937734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb8100cc0 0x7fbcb81968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.239+0000 7fbcbdfef700 1 -- 172.21.15.142:0/1368937734 >> 172.21.15.142:0/1368937734 conn(0x7fbcb80fb060 msgr2=0x7fbcb8068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.239+0000 7fbcbdfef700 1 -- 172.21.15.142:0/1368937734 shutdown_connections 2024-03-29T04:40:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:50.240+0000 7fbcbdfef700 1 -- 172.21.15.142:0/1368937734 wait complete. 2024-03-29T04:40:50.865 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:50 smithi142 ceph-mon[29027]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:40:50.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:50 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1368937734' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-29T04:40:50.914 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1711687246,"num_in_osds":6,"osd_in_since":1711687227,"num_remapped_pgs":0} 2024-03-29T04:40:50.915 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-29T04:40:51.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:50 smithi152 ceph-mon[37177]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:40:51.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:50 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1368937734' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-29T04:40:51.119 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:40:52.873 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.878+0000 7f41fd7e4700 1 -- 172.21.15.142:0/2268422263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80ff510 msgr2=0x7f41f80ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:52.873 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.878+0000 7f41fd7e4700 1 --2- 172.21.15.142:0/2268422263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80ff510 0x7f41f80ff8f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009910 tx=0x7f41e8009c20 comp rx=0 tx=0).stop 2024-03-29T04:40:52.874 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.878+0000 7f41fd7e4700 1 -- 172.21.15.142:0/2268422263 shutdown_connections 2024-03-29T04:40:52.874 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.878+0000 7f41fd7e4700 1 --2- 172.21.15.142:0/2268422263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41f80ffec0 0x7f41f8103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:52.874 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.878+0000 7f41fd7e4700 1 --2- 172.21.15.142:0/2268422263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80ff510 0x7f41f80ff8f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:52.874 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.878+0000 7f41fd7e4700 1 -- 172.21.15.142:0/2268422263 >> 172.21.15.142:0/2268422263 conn(0x7f41f8074d80 msgr2=0x7f41f8075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:52.874 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.878+0000 7f41fd7e4700 1 -- 172.21.15.142:0/2268422263 shutdown_connections 2024-03-29T04:40:52.874 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.879+0000 7f41fd7e4700 1 -- 172.21.15.142:0/2268422263 wait complete. 2024-03-29T04:40:52.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:52 smithi142 ceph-mon[29027]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:40:52.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.879+0000 7f41fd7e4700 1 Processor -- start 2024-03-29T04:40:52.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.879+0000 7f41fd7e4700 1 -- start start 2024-03-29T04:40:52.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41fd7e4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80ff510 0x7f41f819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:52.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41fd7e4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41f80ffec0 0x7f41f819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:52.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41fd7e4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f819d160 con 0x7f41f80ff510 2024-03-29T04:40:52.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41fd7e4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f8196610 con 0x7f41f80ffec0 2024-03-29T04:40:52.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80ff510 0x7f41f819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:52.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80ff510 0x7f41f819c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:50672/0 (socket says 172.21.15.142:50672) 2024-03-29T04:40:52.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41f6ffd700 1 -- 172.21.15.142:0/1329859634 learned_addr learned my addr 172.21.15.142:0/1329859634 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:40:52.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41effff700 1 --2- 172.21.15.142:0/1329859634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41f80ffec0 0x7f41f819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:52.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41f6ffd700 1 -- 172.21.15.142:0/1329859634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41f80ffec0 msgr2=0x7f41f819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:52.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41f6ffd700 1 --2- 172.21.15.142:0/1329859634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41f80ffec0 0x7f41f819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:52.877 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41f6ffd700 1 -- 172.21.15.142:0/1329859634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e80095f0 con 0x7f41f80ff510 2024-03-29T04:40:52.877 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41f6ffd700 1 --2- 172.21.15.142:0/1329859634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80ff510 0x7f41f819c590 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f41e8006010 tx=0x7f41e800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:40:52.877 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41f4ff9700 1 -- 172.21.15.142:0/1329859634 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41e801d070 con 0x7f41f80ff510 2024-03-29T04:40:52.877 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41f4ff9700 1 -- 172.21.15.142:0/1329859634 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f41e8004fa0 con 0x7f41f80ff510 2024-03-29T04:40:52.877 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41fd7e4700 1 -- 172.21.15.142:0/1329859634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f8196890 con 0x7f41f80ff510 2024-03-29T04:40:52.877 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41f4ff9700 1 -- 172.21.15.142:0/1329859634 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41e800f690 con 0x7f41f80ff510 2024-03-29T04:40:52.877 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.880+0000 7f41fd7e4700 1 -- 172.21.15.142:0/1329859634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f8196d80 con 0x7f41f80ff510 2024-03-29T04:40:52.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.882+0000 7f41fd7e4700 1 -- 172.21.15.142:0/1329859634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f804ea90 con 0x7f41f80ff510 2024-03-29T04:40:52.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.882+0000 7f41f4ff9700 1 -- 172.21.15.142:0/1329859634 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f41e8022470 con 0x7f41f80ff510 2024-03-29T04:40:52.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.882+0000 7f41f4ff9700 1 --2- 172.21.15.142:0/1329859634 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f41e406c4e0 0x7f41e406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:52.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.882+0000 7f41f4ff9700 1 -- 172.21.15.142:0/1329859634 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f41e808cbb0 con 0x7f41f80ff510 2024-03-29T04:40:52.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.882+0000 7f41effff700 1 --2- 172.21.15.142:0/1329859634 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f41e406c4e0 0x7f41e406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:52.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.883+0000 7f41effff700 1 --2- 172.21.15.142:0/1329859634 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f41e406c4e0 0x7f41e406e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f41f8105e20 tx=0x7f41e0009190 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:40:52.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:52.887+0000 7f41f4ff9700 1 -- 172.21.15.142:0/1329859634 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f41e8093080 con 0x7f41f80ff510 2024-03-29T04:40:53.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:52 smithi152 ceph-mon[37177]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:40:53.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.074+0000 7f41fd7e4700 1 -- 172.21.15.142:0/1329859634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41f8197bf0 con 0x7f41f80ff510 2024-03-29T04:40:53.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.075+0000 7f41f4ff9700 1 -- 172.21.15.142:0/1329859634 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11372 (secure 0 0 0) 0x7f41e805b1e0 con 0x7f41f80ff510 2024-03-29T04:40:53.070 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:40:53.070 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":41,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","created":"2024-03-29T04:35:07.701328+0000","modified":"2024-03-29T04:40:47.629046+0000","last_up_change":"2024-03-29T04:40:46.622246+0000","last_in_change":"2024-03-29T04:40:27.009053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:39:31.891786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d908de1a-bec3-4238-959a-f3f550431ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6803","nonce":3710621973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6805","nonce":3710621973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6809","nonce":3710621973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6807","nonce":3710621973}]},"public_addr":"172.21.15.142:6803/3710621973","cluster_addr":"172.21.15.142:6805/3710621973","heartbeat_back_addr":"172.21.15.142:6809/3710621973","heartbeat_front_addr":"172.21.15.142:6807/3710621973","state":["exists","up"]},{"osd":1,"uuid":"717e7135-21ab-4d33-a81e-65da6d5186c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6811","nonce":1500472047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6813","nonce":1500472047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6817","nonce":1500472047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6815","nonce":1500472047}]},"public_addr":"172.21.15.142:6811/1500472047","cluster_addr":"172.21.15.142:6813/1500472047","heartbeat_back_addr":"172.21.15.142:6817/1500472047","heartbeat_front_addr":"172.21.15.142:6815/1500472047","state":["exists","up"]},{"osd":2,"uuid":"977032d2-3c29-4c68-ae66-73d024ab94ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6819","nonce":1815862059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6821","nonce":1815862059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6825","nonce":1815862059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6823","nonce":1815862059}]},"public_addr":"172.21.15.142:6819/1815862059","cluster_addr":"172.21.15.142:6821/1815862059","heartbeat_back_addr":"172.21.15.142:6825/1815862059","heartbeat_front_addr":"172.21.15.142:6823/1815862059","state":["exists","up"]},{"osd":3,"uuid":"39ae7173-3a83-4ae1-9866-5a0da908171c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6801","nonce":3854389166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6803","nonce":3854389166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6807","nonce":3854389166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6805","nonce":3854389166}]},"public_addr":"172.21.15.152:6801/3854389166","cluster_addr":"172.21.15.152:6803/3854389166","heartbeat_back_addr":"172.21.15.152:6807/3854389166","heartbeat_front_addr":"172.21.15.152:6805/3854389166","state":["exists","up"]},{"osd":4,"uuid":"f7b5fab7-0f30-4756-a481-15ad64b9e691","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6809","nonce":2813629639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6811","nonce":2813629639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6815","nonce":2813629639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6813","nonce":2813629639}]},"public_addr":"172.21.15.152:6809/2813629639","cluster_addr":"172.21.15.152:6811/2813629639","heartbeat_back_addr":"172.21.15.152:6815/2813629639","heartbeat_front_addr":"172.21.15.152:6813/2813629639","state":["exists","up"]},{"osd":5,"uuid":"8ed3caba-b201-4ad7-b055-3dc40610ab1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6817","nonce":48435269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6819","nonce":48435269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6823","nonce":48435269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6821","nonce":48435269}]},"public_addr":"172.21.15.152:6817/48435269","cluster_addr":"172.21.15.152:6819/48435269","heartbeat_back_addr":"172.21.15.152:6823/48435269","heartbeat_front_addr":"172.21.15.152:6821/48435269","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:38:33.605811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:00.911691+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:28.052973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:51.974988+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:18.104282+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:44.747877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6801/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:6800/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/3338740178":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/586295950":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/672130862":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/2545215192":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/966342870":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6800/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6801/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/2666008640":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/3751399855":"2024-03-30T04:35:33.215645+0000","172.21.15.142:6800/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:6801/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/1935271872":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/390665991":"2024-03-30T04:36:06.145588+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:40:53.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.077+0000 7f41fd7e4700 1 -- 172.21.15.142:0/1329859634 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f41e406c4e0 msgr2=0x7f41e406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:53.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.077+0000 7f41fd7e4700 1 --2- 172.21.15.142:0/1329859634 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f41e406c4e0 0x7f41e406e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f41f8105e20 tx=0x7f41e0009190 comp rx=0 tx=0).stop 2024-03-29T04:40:53.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.077+0000 7f41fd7e4700 1 -- 172.21.15.142:0/1329859634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80ff510 msgr2=0x7f41f819c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:53.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.077+0000 7f41fd7e4700 1 --2- 172.21.15.142:0/1329859634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80ff510 0x7f41f819c590 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f41e8006010 tx=0x7f41e800bd70 comp rx=0 tx=0).stop 2024-03-29T04:40:53.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.077+0000 7f41fd7e4700 1 -- 172.21.15.142:0/1329859634 shutdown_connections 2024-03-29T04:40:53.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.077+0000 7f41fd7e4700 1 --2- 172.21.15.142:0/1329859634 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f41e406c4e0 0x7f41e406e9a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:53.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.077+0000 7f41fd7e4700 1 --2- 172.21.15.142:0/1329859634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f80ff510 0x7f41f819c590 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:53.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.077+0000 7f41fd7e4700 1 --2- 172.21.15.142:0/1329859634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41f80ffec0 0x7f41f819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:53.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.077+0000 7f41fd7e4700 1 -- 172.21.15.142:0/1329859634 >> 172.21.15.142:0/1329859634 conn(0x7f41f8074d80 msgr2=0x7f41f8103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:53.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.077+0000 7f41fd7e4700 1 -- 172.21.15.142:0/1329859634 shutdown_connections 2024-03-29T04:40:53.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:53.077+0000 7f41fd7e4700 1 -- 172.21.15.142:0/1329859634 wait complete. 2024-03-29T04:40:53.968 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:53 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1329859634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T04:40:53.993 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:39:31.891786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:40:53.993 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-29T04:40:54.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:53 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1329859634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T04:40:54.203 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:40:55.016 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:54 smithi142 ceph-mon[29027]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:40:55.034 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:54 smithi152 ceph-mon[37177]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:40:55.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.885+0000 7ff1d3a79700 1 -- 172.21.15.142:0/133243852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc100dd0 msgr2=0x7ff1cc1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:55.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.885+0000 7ff1d3a79700 1 --2- 172.21.15.142:0/133243852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc100dd0 0x7ff1cc1011b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009910 tx=0x7ff1c8009c20 comp rx=0 tx=0).stop 2024-03-29T04:40:55.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.885+0000 7ff1d3a79700 1 -- 172.21.15.142:0/133243852 shutdown_connections 2024-03-29T04:40:55.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.885+0000 7ff1d3a79700 1 --2- 172.21.15.142:0/133243852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1cc101780 0x7ff1cc1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:55.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.885+0000 7ff1d3a79700 1 --2- 172.21.15.142:0/133243852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc100dd0 0x7ff1cc1011b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.885+0000 7ff1d3a79700 1 -- 172.21.15.142:0/133243852 >> 172.21.15.142:0/133243852 conn(0x7ff1cc0fc640 msgr2=0x7ff1cc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.886+0000 7ff1d3a79700 1 -- 172.21.15.142:0/133243852 shutdown_connections 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.886+0000 7ff1d3a79700 1 -- 172.21.15.142:0/133243852 wait complete. 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.886+0000 7ff1d3a79700 1 Processor -- start 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.886+0000 7ff1d3a79700 1 -- start start 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d3a79700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc100dd0 0x7ff1cc19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d3a79700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1cc101780 0x7ff1cc19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d3a79700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc19f360 con 0x7ff1cc100dd0 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d3a79700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc198810 con 0x7ff1cc101780 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d1815700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc100dd0 0x7ff1cc19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d1815700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc100dd0 0x7ff1cc19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:50684/0 (socket says 172.21.15.142:50684) 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d1815700 1 -- 172.21.15.142:0/450449046 learned_addr learned my addr 172.21.15.142:0/450449046 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:40:55.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d1014700 1 --2- 172.21.15.142:0/450449046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1cc101780 0x7ff1cc19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:55.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d1815700 1 -- 172.21.15.142:0/450449046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1cc101780 msgr2=0x7ff1cc19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:55.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d1815700 1 --2- 172.21.15.142:0/450449046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1cc101780 0x7ff1cc19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:55.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d1815700 1 -- 172.21.15.142:0/450449046 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c80095f0 con 0x7ff1cc100dd0 2024-03-29T04:40:55.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d1815700 1 --2- 172.21.15.142:0/450449046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc100dd0 0x7ff1cc19e790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8006010 tx=0x7ff1c800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:40:55.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1c2ffd700 1 -- 172.21.15.142:0/450449046 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1c801d070 con 0x7ff1cc100dd0 2024-03-29T04:40:55.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1c2ffd700 1 -- 172.21.15.142:0/450449046 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff1c8004fa0 con 0x7ff1cc100dd0 2024-03-29T04:40:55.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d3a79700 1 -- 172.21.15.142:0/450449046 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc198a90 con 0x7ff1cc100dd0 2024-03-29T04:40:55.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1c2ffd700 1 -- 172.21.15.142:0/450449046 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1c800f460 con 0x7ff1cc100dd0 2024-03-29T04:40:55.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.887+0000 7ff1d3a79700 1 -- 172.21.15.142:0/450449046 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc198f80 con 0x7ff1cc100dd0 2024-03-29T04:40:55.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.888+0000 7ff1d3a79700 1 -- 172.21.15.142:0/450449046 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc104f50 con 0x7ff1cc100dd0 2024-03-29T04:40:55.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.889+0000 7ff1c2ffd700 1 -- 172.21.15.142:0/450449046 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff1c8022880 con 0x7ff1cc100dd0 2024-03-29T04:40:55.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.889+0000 7ff1c2ffd700 1 --2- 172.21.15.142:0/450449046 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7ff1b806c4e0 0x7ff1b806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:55.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.889+0000 7ff1c2ffd700 1 -- 172.21.15.142:0/450449046 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff1c808c990 con 0x7ff1cc100dd0 2024-03-29T04:40:55.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.889+0000 7ff1d1014700 1 --2- 172.21.15.142:0/450449046 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7ff1b806c4e0 0x7ff1b806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:55.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.890+0000 7ff1d1014700 1 --2- 172.21.15.142:0/450449046 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7ff1b806c4e0 0x7ff1b806e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0fcd80 tx=0x7ff1bc00a300 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:40:55.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:55.894+0000 7ff1c2ffd700 1 -- 172.21.15.142:0/450449046 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff1c805b070 con 0x7ff1cc100dd0 2024-03-29T04:40:56.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.079+0000 7ff1d3a79700 1 -- 172.21.15.142:0/450449046 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1cc104930 con 0x7ff1cc100dd0 2024-03-29T04:40:56.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.080+0000 7ff1c2ffd700 1 -- 172.21.15.142:0/450449046 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff1c80270a0 con 0x7ff1cc100dd0 2024-03-29T04:40:56.075 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 1 2024-03-29T04:40:56.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.083+0000 7ff1d3a79700 1 -- 172.21.15.142:0/450449046 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7ff1b806c4e0 msgr2=0x7ff1b806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:56.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.083+0000 7ff1d3a79700 1 --2- 172.21.15.142:0/450449046 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7ff1b806c4e0 0x7ff1b806e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0fcd80 tx=0x7ff1bc00a300 comp rx=0 tx=0).stop 2024-03-29T04:40:56.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.083+0000 7ff1d3a79700 1 -- 172.21.15.142:0/450449046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc100dd0 msgr2=0x7ff1cc19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:56.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.083+0000 7ff1d3a79700 1 --2- 172.21.15.142:0/450449046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc100dd0 0x7ff1cc19e790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8006010 tx=0x7ff1c800bd70 comp rx=0 tx=0).stop 2024-03-29T04:40:56.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.083+0000 7ff1d3a79700 1 -- 172.21.15.142:0/450449046 shutdown_connections 2024-03-29T04:40:56.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.083+0000 7ff1d3a79700 1 --2- 172.21.15.142:0/450449046 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7ff1b806c4e0 0x7ff1b806e9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:56.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.083+0000 7ff1d3a79700 1 --2- 172.21.15.142:0/450449046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc100dd0 0x7ff1cc19e790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:56.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.083+0000 7ff1d3a79700 1 --2- 172.21.15.142:0/450449046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1cc101780 0x7ff1cc19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:56.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.083+0000 7ff1d3a79700 1 -- 172.21.15.142:0/450449046 >> 172.21.15.142:0/450449046 conn(0x7ff1cc0fc640 msgr2=0x7ff1cc0fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:56.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.083+0000 7ff1d3a79700 1 -- 172.21.15.142:0/450449046 shutdown_connections 2024-03-29T04:40:56.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:56.083+0000 7ff1d3a79700 1 -- 172.21.15.142:0/450449046 wait complete. 2024-03-29T04:40:56.837 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:56 smithi142 ceph-mon[29027]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:40:56.837 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:56 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/450449046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T04:40:56.839 INFO:tasks.cephadm:Setting up client nodes... 2024-03-29T04:40:56.839 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-29T04:40:57.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:56 smithi152 ceph-mon[37177]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:40:57.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:56 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/450449046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T04:40:57.046 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:40:58.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.733+0000 7fedf6595700 1 -- 172.21.15.142:0/1950988307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf0100bc0 msgr2=0x7fedf0100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:58.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.733+0000 7fedf6595700 1 --2- 172.21.15.142:0/1950988307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf0100bc0 0x7fedf0100fe0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fede0009910 tx=0x7fede0009c20 comp rx=0 tx=0).stop 2024-03-29T04:40:58.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.733+0000 7fedf6595700 1 -- 172.21.15.142:0/1950988307 shutdown_connections 2024-03-29T04:40:58.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.733+0000 7fedf6595700 1 --2- 172.21.15.142:0/1950988307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedf0101dc0 0x7fedf0102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:58.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.733+0000 7fedf6595700 1 --2- 172.21.15.142:0/1950988307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf0100bc0 0x7fedf0100fe0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:58.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.733+0000 7fedf6595700 1 -- 172.21.15.142:0/1950988307 >> 172.21.15.142:0/1950988307 conn(0x7fedf00fc140 msgr2=0x7fedf00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:58.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.733+0000 7fedf6595700 1 -- 172.21.15.142:0/1950988307 shutdown_connections 2024-03-29T04:40:58.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.733+0000 7fedf6595700 1 -- 172.21.15.142:0/1950988307 wait complete. 2024-03-29T04:40:58.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.734+0000 7fedf6595700 1 Processor -- start 2024-03-29T04:40:58.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.734+0000 7fedf6595700 1 -- start start 2024-03-29T04:40:58.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.734+0000 7fedf6595700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf0100bc0 0x7fedf0196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:58.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.734+0000 7fedf6595700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedf0101dc0 0x7fedf01968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:58.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.734+0000 7fedf6595700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0196ef0 con 0x7fedf0100bc0 2024-03-29T04:40:58.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.734+0000 7fedf6595700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0197030 con 0x7fedf0101dc0 2024-03-29T04:40:58.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.734+0000 7fedeffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf0100bc0 0x7fedf0196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:58.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.734+0000 7fedeffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf0100bc0 0x7fedf0196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54692/0 (socket says 172.21.15.142:54692) 2024-03-29T04:40:58.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.734+0000 7fedeffff700 1 -- 172.21.15.142:0/182737482 learned_addr learned my addr 172.21.15.142:0/182737482 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:40:58.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.735+0000 7fedef7fe700 1 --2- 172.21.15.142:0/182737482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedf0101dc0 0x7fedf01968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:58.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.735+0000 7fedeffff700 1 -- 172.21.15.142:0/182737482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedf0101dc0 msgr2=0x7fedf01968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:58.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.735+0000 7fedeffff700 1 --2- 172.21.15.142:0/182737482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedf0101dc0 0x7fedf01968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:58.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.735+0000 7fedeffff700 1 -- 172.21.15.142:0/182737482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede00095f0 con 0x7fedf0100bc0 2024-03-29T04:40:58.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.735+0000 7fedef7fe700 1 --2- 172.21.15.142:0/182737482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedf0101dc0 0x7fedf01968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:40:58.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.735+0000 7fedeffff700 1 --2- 172.21.15.142:0/182737482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf0100bc0 0x7fedf0196390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fede000bda0 tx=0x7fede000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:40:58.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.735+0000 7feded7fa700 1 -- 172.21.15.142:0/182737482 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede001d070 con 0x7fedf0100bc0 2024-03-29T04:40:58.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.735+0000 7feded7fa700 1 -- 172.21.15.142:0/182737482 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fede0004c50 con 0x7fedf0100bc0 2024-03-29T04:40:58.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.735+0000 7fedf6595700 1 -- 172.21.15.142:0/182737482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf019ba80 con 0x7fedf0100bc0 2024-03-29T04:40:58.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.735+0000 7feded7fa700 1 -- 172.21.15.142:0/182737482 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede000f690 con 0x7fedf0100bc0 2024-03-29T04:40:58.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.735+0000 7fedf6595700 1 -- 172.21.15.142:0/182737482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf019bf70 con 0x7fedf0100bc0 2024-03-29T04:40:58.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.736+0000 7fedf6595700 1 -- 172.21.15.142:0/182737482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf0109720 con 0x7fedf0100bc0 2024-03-29T04:40:58.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.736+0000 7feded7fa700 1 -- 172.21.15.142:0/182737482 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fede00229b0 con 0x7fedf0100bc0 2024-03-29T04:40:58.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.737+0000 7feded7fa700 1 --2- 172.21.15.142:0/182737482 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fedd806c4e0 0x7fedd806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:40:58.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.737+0000 7feded7fa700 1 -- 172.21.15.142:0/182737482 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fede008cc40 con 0x7fedf0100bc0 2024-03-29T04:40:58.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.737+0000 7fedef7fe700 1 --2- 172.21.15.142:0/182737482 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fedd806c4e0 0x7fedd806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:40:58.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.737+0000 7fedef7fe700 1 --2- 172.21.15.142:0/182737482 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fedd806c4e0 0x7fedd806e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fede4005950 tx=0x7fede4005f50 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:40:58.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:58.741+0000 7feded7fa700 1 -- 172.21.15.142:0/182737482 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fede005b2a0 con 0x7fedf0100bc0 2024-03-29T04:40:58.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:58 smithi142 ceph-mon[29027]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:40:58.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.004+0000 7fedf6595700 1 -- 172.21.15.142:0/182737482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fedf0073690 con 0x7fedf0100bc0 2024-03-29T04:40:59.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.008+0000 7feded7fa700 1 -- 172.21.15.142:0/182737482 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fede005ae30 con 0x7fedf0100bc0 2024-03-29T04:40:59.003 INFO:teuthology.orchestra.run.smithi142.stdout:[client.0] 2024-03-29T04:40:59.003 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBbRgZmffNWABAAHaPPBaeoTrl9eww4Q7je6g== 2024-03-29T04:40:59.005 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.010+0000 7fedf6595700 1 -- 172.21.15.142:0/182737482 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fedd806c4e0 msgr2=0x7fedd806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:59.005 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.010+0000 7fedf6595700 1 --2- 172.21.15.142:0/182737482 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fedd806c4e0 0x7fedd806e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fede4005950 tx=0x7fede4005f50 comp rx=0 tx=0).stop 2024-03-29T04:40:59.005 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.010+0000 7fedf6595700 1 -- 172.21.15.142:0/182737482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf0100bc0 msgr2=0x7fedf0196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:40:59.005 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.010+0000 7fedf6595700 1 --2- 172.21.15.142:0/182737482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf0100bc0 0x7fedf0196390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fede000bda0 tx=0x7fede000be80 comp rx=0 tx=0).stop 2024-03-29T04:40:59.005 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.010+0000 7fedf6595700 1 -- 172.21.15.142:0/182737482 shutdown_connections 2024-03-29T04:40:59.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.010+0000 7fedf6595700 1 --2- 172.21.15.142:0/182737482 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fedd806c4e0 0x7fedd806e9a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:59.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.010+0000 7fedf6595700 1 --2- 172.21.15.142:0/182737482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf0100bc0 0x7fedf0196390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:59.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.010+0000 7fedf6595700 1 --2- 172.21.15.142:0/182737482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedf0101dc0 0x7fedf01968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:40:59.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.010+0000 7fedf6595700 1 -- 172.21.15.142:0/182737482 >> 172.21.15.142:0/182737482 conn(0x7fedf00fc140 msgr2=0x7fedf0104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:40:59.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.010+0000 7fedf6595700 1 -- 172.21.15.142:0/182737482 shutdown_connections 2024-03-29T04:40:59.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:40:59.010+0000 7fedf6595700 1 -- 172.21.15.142:0/182737482 wait complete. 2024-03-29T04:40:59.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:58 smithi152 ceph-mon[37177]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:40:59.719 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:40:59.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-29T04:40:59.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-29T04:40:59.762 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-29T04:40:59.981 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi152/config 2024-03-29T04:41:00.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:59 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/182737482' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-29T04:41:00.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:40:59 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/182737482' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-29T04:41:00.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:59 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/182737482' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-29T04:41:00.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:40:59 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/182737482' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-29T04:41:01.285 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:01 smithi152 ceph-mon[37177]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:01.366 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:01 smithi142 ceph-mon[29027]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:01.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.525+0000 7f7770c2f700 1 -- 172.21.15.152:0/2382856907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f776c0ffab0 msgr2=0x7f776c0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:01.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.525+0000 7f7770c2f700 1 --2- 172.21.15.152:0/2382856907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f776c0ffab0 0x7f776c0ffed0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f775c009910 tx=0x7f775c009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:01.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.525+0000 7f7770c2f700 1 -- 172.21.15.152:0/2382856907 shutdown_connections 2024-03-29T04:41:01.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.525+0000 7f7770c2f700 1 --2- 172.21.15.152:0/2382856907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f776c100cb0 0x7f776c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:01.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.525+0000 7f7770c2f700 1 --2- 172.21.15.152:0/2382856907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f776c0ffab0 0x7f776c0ffed0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:01.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.525+0000 7f7770c2f700 1 -- 172.21.15.152:0/2382856907 >> 172.21.15.152:0/2382856907 conn(0x7f776c0fb030 msgr2=0x7f776c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:01.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.525+0000 7f7770c2f700 1 -- 172.21.15.152:0/2382856907 shutdown_connections 2024-03-29T04:41:01.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.525+0000 7f7770c2f700 1 -- 172.21.15.152:0/2382856907 wait complete. 2024-03-29T04:41:01.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.526+0000 7f7770c2f700 1 Processor -- start 2024-03-29T04:41:01.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.526+0000 7f7770c2f700 1 -- start start 2024-03-29T04:41:01.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.526+0000 7f7770c2f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f776c0ffab0 0x7f776c068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:01.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.526+0000 7f7770c2f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f776c100cb0 0x7f776c19b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:01.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.526+0000 7f7770c2f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c19b710 con 0x7f776c0ffab0 2024-03-29T04:41:01.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.526+0000 7f7770c2f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c19b850 con 0x7f776c100cb0 2024-03-29T04:41:01.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.526+0000 7f7769d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f776c100cb0 0x7f776c19b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:01.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.526+0000 7f7769d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f776c100cb0 0x7f776c19b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37270/0 (socket says 172.21.15.152:37270) 2024-03-29T04:41:01.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.526+0000 7f7769d9b700 1 -- 172.21.15.152:0/960321802 learned_addr learned my addr 172.21.15.152:0/960321802 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-29T04:41:01.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.526+0000 7f776a59c700 1 --2- 172.21.15.152:0/960321802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f776c0ffab0 0x7f776c068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:01.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.527+0000 7f7769d9b700 1 -- 172.21.15.152:0/960321802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f776c0ffab0 msgr2=0x7f776c068f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:01.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.527+0000 7f7769d9b700 1 --2- 172.21.15.152:0/960321802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f776c0ffab0 0x7f776c068f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:01.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.527+0000 7f7769d9b700 1 -- 172.21.15.152:0/960321802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c0095f0 con 0x7f776c100cb0 2024-03-29T04:41:01.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.527+0000 7f776a59c700 1 --2- 172.21.15.152:0/960321802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f776c0ffab0 0x7f776c068f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:41:01.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.527+0000 7f7769d9b700 1 --2- 172.21.15.152:0/960321802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f776c100cb0 0x7f776c19b140 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f776000b4e0 tx=0x7f776000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:01.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.527+0000 7f775b7fe700 1 -- 172.21.15.152:0/960321802 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7760011800 con 0x7f776c100cb0 2024-03-29T04:41:01.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.527+0000 7f775b7fe700 1 -- 172.21.15.152:0/960321802 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7760011e10 con 0x7f776c100cb0 2024-03-29T04:41:01.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.527+0000 7f7770c2f700 1 -- 172.21.15.152:0/960321802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776c19bb30 con 0x7f776c100cb0 2024-03-29T04:41:01.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.527+0000 7f775b7fe700 1 -- 172.21.15.152:0/960321802 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f776000f2e0 con 0x7f776c100cb0 2024-03-29T04:41:01.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.527+0000 7f7770c2f700 1 -- 172.21.15.152:0/960321802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776c19c100 con 0x7f776c100cb0 2024-03-29T04:41:01.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.528+0000 7f7770c2f700 1 -- 172.21.15.152:0/960321802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776c108690 con 0x7f776c100cb0 2024-03-29T04:41:01.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.528+0000 7f775b7fe700 1 -- 172.21.15.152:0/960321802 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f7760010380 con 0x7f776c100cb0 2024-03-29T04:41:01.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.529+0000 7f775b7fe700 1 --2- 172.21.15.152:0/960321802 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f775406c4e0 0x7f775406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:01.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.529+0000 7f775b7fe700 1 -- 172.21.15.152:0/960321802 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f776008b0f0 con 0x7f776c100cb0 2024-03-29T04:41:01.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.529+0000 7f776a59c700 1 --2- 172.21.15.152:0/960321802 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f775406c4e0 0x7f775406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:01.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.530+0000 7f776a59c700 1 --2- 172.21.15.152:0/960321802 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f775406c4e0 0x7f775406e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f775c006010 tx=0x7f775c00b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:01.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.533+0000 7f775b7fe700 1 -- 172.21.15.152:0/960321802 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f77600596a0 con 0x7f776c100cb0 2024-03-29T04:41:01.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.797+0000 7f7770c2f700 1 -- 172.21.15.152:0/960321802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f776c19c770 con 0x7f776c100cb0 2024-03-29T04:41:01.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.803+0000 7f775b7fe700 1 -- 172.21.15.152:0/960321802 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f7760059230 con 0x7f776c100cb0 2024-03-29T04:41:01.798 INFO:teuthology.orchestra.run.smithi152.stdout:[client.1] 2024-03-29T04:41:01.798 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBdRgZmoyunLxAAdAmIkbcqqjLdXvMD5fbDPA== 2024-03-29T04:41:01.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.805+0000 7f7770c2f700 1 -- 172.21.15.152:0/960321802 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f775406c4e0 msgr2=0x7f775406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:01.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.805+0000 7f7770c2f700 1 --2- 172.21.15.152:0/960321802 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f775406c4e0 0x7f775406e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f775c006010 tx=0x7f775c00b540 comp rx=0 tx=0).stop 2024-03-29T04:41:01.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.805+0000 7f7770c2f700 1 -- 172.21.15.152:0/960321802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f776c100cb0 msgr2=0x7f776c19b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:01.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.805+0000 7f7770c2f700 1 --2- 172.21.15.152:0/960321802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f776c100cb0 0x7f776c19b140 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f776000b4e0 tx=0x7f776000b8a0 comp rx=0 tx=0).stop 2024-03-29T04:41:01.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.805+0000 7f7770c2f700 1 -- 172.21.15.152:0/960321802 shutdown_connections 2024-03-29T04:41:01.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.805+0000 7f7770c2f700 1 --2- 172.21.15.152:0/960321802 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f775406c4e0 0x7f775406e9a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:01.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.805+0000 7f7770c2f700 1 --2- 172.21.15.152:0/960321802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f776c0ffab0 0x7f776c068f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:01.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.805+0000 7f7770c2f700 1 --2- 172.21.15.152:0/960321802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f776c100cb0 0x7f776c19b140 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:01.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.805+0000 7f7770c2f700 1 -- 172.21.15.152:0/960321802 >> 172.21.15.152:0/960321802 conn(0x7f776c0fb030 msgr2=0x7f776c103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:01.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.805+0000 7f7770c2f700 1 -- 172.21.15.152:0/960321802 shutdown_connections 2024-03-29T04:41:01.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:41:01.805+0000 7f7770c2f700 1 -- 172.21.15.152:0/960321802 wait complete. 2024-03-29T04:41:02.132 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:02 smithi152 ceph-mon[37177]: from='client.? 172.21.15.152:0/960321802' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-29T04:41:02.132 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:02 smithi152 ceph-mon[37177]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-29T04:41:02.132 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:02 smithi152 ceph-mon[37177]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-29T04:41:02.366 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:02 smithi142 ceph-mon[29027]: from='client.? 172.21.15.152:0/960321802' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-29T04:41:02.366 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:02 smithi142 ceph-mon[29027]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-29T04:41:02.366 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:02 smithi142 ceph-mon[29027]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-29T04:41:02.420 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:41:02.420 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-29T04:41:02.420 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-29T04:41:02.462 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-29T04:41:02.463 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-29T04:41:02.463 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-29T04:41:02.682 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:03.100 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:03 smithi142 ceph-mon[29027]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:03.100 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:03 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:41:03.285 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:03 smithi152 ceph-mon[37177]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:03.285 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:03 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:41:04.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.432+0000 7efdb71c7700 1 -- 172.21.15.142:0/1128825400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb0100de0 msgr2=0x7efdb01011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:04.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.432+0000 7efdb71c7700 1 --2- 172.21.15.142:0/1128825400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb0100de0 0x7efdb01011c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efda4009910 tx=0x7efda4009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:04.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.432+0000 7efdb71c7700 1 -- 172.21.15.142:0/1128825400 shutdown_connections 2024-03-29T04:41:04.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.432+0000 7efdb71c7700 1 --2- 172.21.15.142:0/1128825400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdb0101790 0x7efdb01057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:04.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.432+0000 7efdb71c7700 1 --2- 172.21.15.142:0/1128825400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb0100de0 0x7efdb01011c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.432+0000 7efdb71c7700 1 -- 172.21.15.142:0/1128825400 >> 172.21.15.142:0/1128825400 conn(0x7efdb00fc670 msgr2=0x7efdb00fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.432+0000 7efdb71c7700 1 -- 172.21.15.142:0/1128825400 shutdown_connections 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.432+0000 7efdb71c7700 1 -- 172.21.15.142:0/1128825400 wait complete. 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb71c7700 1 Processor -- start 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb71c7700 1 -- start start 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb71c7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb0100de0 0x7efdb019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb71c7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdb0101790 0x7efdb019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb71c7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb019d150 con 0x7efdb0100de0 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb71c7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb0196600 con 0x7efdb0101790 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb4f63700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb0100de0 0x7efdb019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb4f63700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb0100de0 0x7efdb019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54718/0 (socket says 172.21.15.142:54718) 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb4f63700 1 -- 172.21.15.142:0/3200744397 learned_addr learned my addr 172.21.15.142:0/3200744397 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:04.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdaffff700 1 --2- 172.21.15.142:0/3200744397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdb0101790 0x7efdb019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:04.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb4f63700 1 -- 172.21.15.142:0/3200744397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdb0101790 msgr2=0x7efdb019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:04.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb4f63700 1 --2- 172.21.15.142:0/3200744397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdb0101790 0x7efdb019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:04.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb4f63700 1 -- 172.21.15.142:0/3200744397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda40095f0 con 0x7efdb0100de0 2024-03-29T04:41:04.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdaffff700 1 --2- 172.21.15.142:0/3200744397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdb0101790 0x7efdb019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:41:04.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.433+0000 7efdb4f63700 1 --2- 172.21.15.142:0/3200744397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb0100de0 0x7efdb019c580 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efda400b5c0 tx=0x7efda400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:04.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.434+0000 7efdadffb700 1 -- 172.21.15.142:0/3200744397 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efda401d070 con 0x7efdb0100de0 2024-03-29T04:41:04.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.434+0000 7efdadffb700 1 -- 172.21.15.142:0/3200744397 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efda4022470 con 0x7efdb0100de0 2024-03-29T04:41:04.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.434+0000 7efdb71c7700 1 -- 172.21.15.142:0/3200744397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb0196880 con 0x7efdb0100de0 2024-03-29T04:41:04.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.434+0000 7efdadffb700 1 -- 172.21.15.142:0/3200744397 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efda400f670 con 0x7efdb0100de0 2024-03-29T04:41:04.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.434+0000 7efdb71c7700 1 -- 172.21.15.142:0/3200744397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb0196d70 con 0x7efdb0100de0 2024-03-29T04:41:04.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.435+0000 7efdb71c7700 1 -- 172.21.15.142:0/3200744397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb0109180 con 0x7efdb0100de0 2024-03-29T04:41:04.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.435+0000 7efdadffb700 1 -- 172.21.15.142:0/3200744397 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7efda400f890 con 0x7efdb0100de0 2024-03-29T04:41:04.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.435+0000 7efdadffb700 1 --2- 172.21.15.142:0/3200744397 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efda006c4e0 0x7efda006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:04.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.436+0000 7efdadffb700 1 -- 172.21.15.142:0/3200744397 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7efda408cb00 con 0x7efdb0100de0 2024-03-29T04:41:04.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.436+0000 7efdaffff700 1 --2- 172.21.15.142:0/3200744397 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efda006c4e0 0x7efda006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:04.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.436+0000 7efdaffff700 1 --2- 172.21.15.142:0/3200744397 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efda006c4e0 0x7efda006e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efdb01059d0 tx=0x7efd9c008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:04.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.440+0000 7efdadffb700 1 -- 172.21.15.142:0/3200744397 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efda405b250 con 0x7efdb0100de0 2024-03-29T04:41:04.616 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:04 smithi142 ceph-mon[29027]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:04.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.685+0000 7efdb71c7700 1 -- 172.21.15.142:0/3200744397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7efdb002ccc0 con 0x7efdb0100de0 2024-03-29T04:41:04.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.688+0000 7efdadffb700 1 -- 172.21.15.142:0/3200744397 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173047 (secure 0 0 0) 0x7efda4027090 con 0x7efdb0100de0 2024-03-29T04:41:04.686 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:41:04.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.691+0000 7efdb71c7700 1 -- 172.21.15.142:0/3200744397 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efda006c4e0 msgr2=0x7efda006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:04.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.691+0000 7efdb71c7700 1 --2- 172.21.15.142:0/3200744397 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efda006c4e0 0x7efda006e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efdb01059d0 tx=0x7efd9c008040 comp rx=0 tx=0).stop 2024-03-29T04:41:04.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.691+0000 7efdb71c7700 1 -- 172.21.15.142:0/3200744397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb0100de0 msgr2=0x7efdb019c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:04.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.691+0000 7efdb71c7700 1 --2- 172.21.15.142:0/3200744397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb0100de0 0x7efdb019c580 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efda400b5c0 tx=0x7efda400be00 comp rx=0 tx=0).stop 2024-03-29T04:41:04.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.691+0000 7efdb71c7700 1 -- 172.21.15.142:0/3200744397 shutdown_connections 2024-03-29T04:41:04.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.691+0000 7efdb71c7700 1 --2- 172.21.15.142:0/3200744397 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efda006c4e0 0x7efda006e9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:04.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.691+0000 7efdb71c7700 1 --2- 172.21.15.142:0/3200744397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb0100de0 0x7efdb019c580 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:04.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.691+0000 7efdb71c7700 1 --2- 172.21.15.142:0/3200744397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdb0101790 0x7efdb019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:04.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.691+0000 7efdb71c7700 1 -- 172.21.15.142:0/3200744397 >> 172.21.15.142:0/3200744397 conn(0x7efdb00fc670 msgr2=0x7efdb00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:04.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.691+0000 7efdb71c7700 1 -- 172.21.15.142:0/3200744397 shutdown_connections 2024-03-29T04:41:04.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:04.691+0000 7efdb71c7700 1 -- 172.21.15.142:0/3200744397 wait complete. 2024-03-29T04:41:04.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:04 smithi152 ceph-mon[37177]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:05.316 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":19,"active_gid":14215,"active_name":"smithi142.swesrg","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2395947042},{"type":"v1","addr":"172.21.15.142:6801","nonce":2395947042}]},"active_addr":"172.21.15.142:6801/2395947042","active_change":"2024-03-29T04:37:17.579286+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14238,"name":"smithi152.usoolh","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.142:8443/","prometheus":"http://172.21.15.142:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":2370069599}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":484532431}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":3750656759}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":1851570418}]}]} 2024-03-29T04:41:05.320 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-29T04:41:05.320 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-29T04:41:05.320 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-29T04:41:05.527 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:05.616 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:05 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3200744397' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-29T04:41:05.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:05 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3200744397' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-29T04:41:06.602 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:06 smithi142 ceph-mon[29027]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:06.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:06 smithi152 ceph-mon[37177]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:07.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.094+0000 7f76bf60d700 1 -- 172.21.15.142:0/2359478257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76b81052c0 msgr2=0x7f76b81056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:07.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.094+0000 7f76bf60d700 1 --2- 172.21.15.142:0/2359478257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76b81052c0 0x7f76b81056a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009910 tx=0x7f76a8009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:07.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.094+0000 7f76bf60d700 1 -- 172.21.15.142:0/2359478257 shutdown_connections 2024-03-29T04:41:07.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.094+0000 7f76bf60d700 1 --2- 172.21.15.142:0/2359478257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76b80ff2b0 0x7f76b80ff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:07.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.094+0000 7f76bf60d700 1 --2- 172.21.15.142:0/2359478257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76b81052c0 0x7f76b81056a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:07.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.094+0000 7f76bf60d700 1 -- 172.21.15.142:0/2359478257 >> 172.21.15.142:0/2359478257 conn(0x7f76b8074e00 msgr2=0x7f76b8075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:07.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.094+0000 7f76bf60d700 1 -- 172.21.15.142:0/2359478257 shutdown_connections 2024-03-29T04:41:07.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.094+0000 7f76bf60d700 1 -- 172.21.15.142:0/2359478257 wait complete. 2024-03-29T04:41:07.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.094+0000 7f76bf60d700 1 Processor -- start 2024-03-29T04:41:07.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.094+0000 7f76bf60d700 1 -- start start 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bf60d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76b80ff2b0 0x7f76b81953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bf60d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76b81052c0 0x7f76b8195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bf60d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b8195e40 con 0x7f76b80ff2b0 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bf60d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b8195fb0 con 0x7f76b81052c0 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bd3a9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76b80ff2b0 0x7f76b81953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bd3a9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76b80ff2b0 0x7f76b81953c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54734/0 (socket says 172.21.15.142:54734) 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bd3a9700 1 -- 172.21.15.142:0/30464002 learned_addr learned my addr 172.21.15.142:0/30464002 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bcba8700 1 --2- 172.21.15.142:0/30464002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76b81052c0 0x7f76b8195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bd3a9700 1 -- 172.21.15.142:0/30464002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76b81052c0 msgr2=0x7f76b8195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bd3a9700 1 --2- 172.21.15.142:0/30464002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76b81052c0 0x7f76b8195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bd3a9700 1 -- 172.21.15.142:0/30464002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80095f0 con 0x7f76b80ff2b0 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bd3a9700 1 --2- 172.21.15.142:0/30464002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76b80ff2b0 0x7f76b81953c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f76a800fa00 tx=0x7f76a800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:07.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76ae7fc700 1 -- 172.21.15.142:0/30464002 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76a801d070 con 0x7f76b80ff2b0 2024-03-29T04:41:07.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bf60d700 1 -- 172.21.15.142:0/30464002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b8196230 con 0x7f76b80ff2b0 2024-03-29T04:41:07.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76ae7fc700 1 -- 172.21.15.142:0/30464002 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f76a8005640 con 0x7f76b80ff2b0 2024-03-29T04:41:07.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76ae7fc700 1 -- 172.21.15.142:0/30464002 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76a80226a0 con 0x7f76b80ff2b0 2024-03-29T04:41:07.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.095+0000 7f76bf60d700 1 -- 172.21.15.142:0/30464002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b81a2cb0 con 0x7f76b80ff2b0 2024-03-29T04:41:07.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.096+0000 7f76bf60d700 1 -- 172.21.15.142:0/30464002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b804ea90 con 0x7f76b80ff2b0 2024-03-29T04:41:07.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.097+0000 7f76ae7fc700 1 -- 172.21.15.142:0/30464002 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f76a8003680 con 0x7f76b80ff2b0 2024-03-29T04:41:07.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.097+0000 7f76ae7fc700 1 --2- 172.21.15.142:0/30464002 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f76a406c4e0 0x7f76a406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:07.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.097+0000 7f76ae7fc700 1 -- 172.21.15.142:0/30464002 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f76a808c7e0 con 0x7f76b80ff2b0 2024-03-29T04:41:07.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.097+0000 7f76bcba8700 1 --2- 172.21.15.142:0/30464002 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f76a406c4e0 0x7f76a406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:07.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.098+0000 7f76bcba8700 1 --2- 172.21.15.142:0/30464002 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f76a406c4e0 0x7f76a406e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76b80fe870 tx=0x7f76b4009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:07.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.102+0000 7f76ae7fc700 1 -- 172.21.15.142:0/30464002 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f76a805ad90 con 0x7f76b80ff2b0 2024-03-29T04:41:07.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.293+0000 7f76bf60d700 1 -- 172.21.15.142:0/30464002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76b80fe500 con 0x7f76b80ff2b0 2024-03-29T04:41:07.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.294+0000 7f76ae7fc700 1 -- 172.21.15.142:0/30464002 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11372 (secure 0 0 0) 0x7f76a8003e60 con 0x7f76b80ff2b0 2024-03-29T04:41:07.289 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:41:07.290 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":41,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","created":"2024-03-29T04:35:07.701328+0000","modified":"2024-03-29T04:40:47.629046+0000","last_up_change":"2024-03-29T04:40:46.622246+0000","last_in_change":"2024-03-29T04:40:27.009053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:39:31.891786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d908de1a-bec3-4238-959a-f3f550431ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6803","nonce":3710621973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6805","nonce":3710621973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6809","nonce":3710621973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6807","nonce":3710621973}]},"public_addr":"172.21.15.142:6803/3710621973","cluster_addr":"172.21.15.142:6805/3710621973","heartbeat_back_addr":"172.21.15.142:6809/3710621973","heartbeat_front_addr":"172.21.15.142:6807/3710621973","state":["exists","up"]},{"osd":1,"uuid":"717e7135-21ab-4d33-a81e-65da6d5186c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6811","nonce":1500472047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6813","nonce":1500472047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6817","nonce":1500472047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6815","nonce":1500472047}]},"public_addr":"172.21.15.142:6811/1500472047","cluster_addr":"172.21.15.142:6813/1500472047","heartbeat_back_addr":"172.21.15.142:6817/1500472047","heartbeat_front_addr":"172.21.15.142:6815/1500472047","state":["exists","up"]},{"osd":2,"uuid":"977032d2-3c29-4c68-ae66-73d024ab94ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6819","nonce":1815862059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6821","nonce":1815862059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6825","nonce":1815862059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6823","nonce":1815862059}]},"public_addr":"172.21.15.142:6819/1815862059","cluster_addr":"172.21.15.142:6821/1815862059","heartbeat_back_addr":"172.21.15.142:6825/1815862059","heartbeat_front_addr":"172.21.15.142:6823/1815862059","state":["exists","up"]},{"osd":3,"uuid":"39ae7173-3a83-4ae1-9866-5a0da908171c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6801","nonce":3854389166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6803","nonce":3854389166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6807","nonce":3854389166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6805","nonce":3854389166}]},"public_addr":"172.21.15.152:6801/3854389166","cluster_addr":"172.21.15.152:6803/3854389166","heartbeat_back_addr":"172.21.15.152:6807/3854389166","heartbeat_front_addr":"172.21.15.152:6805/3854389166","state":["exists","up"]},{"osd":4,"uuid":"f7b5fab7-0f30-4756-a481-15ad64b9e691","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6809","nonce":2813629639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6811","nonce":2813629639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6815","nonce":2813629639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6813","nonce":2813629639}]},"public_addr":"172.21.15.152:6809/2813629639","cluster_addr":"172.21.15.152:6811/2813629639","heartbeat_back_addr":"172.21.15.152:6815/2813629639","heartbeat_front_addr":"172.21.15.152:6813/2813629639","state":["exists","up"]},{"osd":5,"uuid":"8ed3caba-b201-4ad7-b055-3dc40610ab1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6817","nonce":48435269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6819","nonce":48435269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6823","nonce":48435269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6821","nonce":48435269}]},"public_addr":"172.21.15.152:6817/48435269","cluster_addr":"172.21.15.152:6819/48435269","heartbeat_back_addr":"172.21.15.152:6823/48435269","heartbeat_front_addr":"172.21.15.152:6821/48435269","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:38:33.605811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:00.911691+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:28.052973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:51.974988+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:18.104282+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:44.747877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6801/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:6800/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/3338740178":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/586295950":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/672130862":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/2545215192":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/966342870":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6800/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6801/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/2666008640":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/3751399855":"2024-03-30T04:35:33.215645+0000","172.21.15.142:6800/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:6801/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/1935271872":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/390665991":"2024-03-30T04:36:06.145588+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:41:07.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.296+0000 7f76bf60d700 1 -- 172.21.15.142:0/30464002 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f76a406c4e0 msgr2=0x7f76a406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:07.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.296+0000 7f76bf60d700 1 --2- 172.21.15.142:0/30464002 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f76a406c4e0 0x7f76a406e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f76b80fe870 tx=0x7f76b4009260 comp rx=0 tx=0).stop 2024-03-29T04:41:07.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.297+0000 7f76bf60d700 1 -- 172.21.15.142:0/30464002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76b80ff2b0 msgr2=0x7f76b81953c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:07.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.297+0000 7f76bf60d700 1 --2- 172.21.15.142:0/30464002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76b80ff2b0 0x7f76b81953c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f76a800fa00 tx=0x7f76a800fae0 comp rx=0 tx=0).stop 2024-03-29T04:41:07.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.297+0000 7f76bf60d700 1 -- 172.21.15.142:0/30464002 shutdown_connections 2024-03-29T04:41:07.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.297+0000 7f76bf60d700 1 --2- 172.21.15.142:0/30464002 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f76a406c4e0 0x7f76a406e9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:07.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.297+0000 7f76bf60d700 1 --2- 172.21.15.142:0/30464002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76b80ff2b0 0x7f76b81953c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:07.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.297+0000 7f76bf60d700 1 --2- 172.21.15.142:0/30464002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76b81052c0 0x7f76b8195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:07.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.297+0000 7f76bf60d700 1 -- 172.21.15.142:0/30464002 >> 172.21.15.142:0/30464002 conn(0x7f76b8074e00 msgr2=0x7f76b81038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:07.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.297+0000 7f76bf60d700 1 -- 172.21.15.142:0/30464002 shutdown_connections 2024-03-29T04:41:07.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:07.297+0000 7f76bf60d700 1 -- 172.21.15.142:0/30464002 wait complete. 2024-03-29T04:41:07.616 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:07 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/30464002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T04:41:07.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:07 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/30464002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T04:41:08.042 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-29T04:41:08.043 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-29T04:41:08.251 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:08.695 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:08 smithi142 ceph-mon[29027]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:08.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:08 smithi152 ceph-mon[37177]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:09.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.897+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/3962318886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c24100da0 msgr2=0x7f8c24101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.897+0000 7f8c2c3e3700 1 --2- 172.21.15.142:0/3962318886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c24100da0 0x7f8c24101180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009910 tx=0x7f8c20009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.897+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/3962318886 shutdown_connections 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.897+0000 7f8c2c3e3700 1 --2- 172.21.15.142:0/3962318886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c24101750 0x7f8c241057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.897+0000 7f8c2c3e3700 1 --2- 172.21.15.142:0/3962318886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c24100da0 0x7f8c24101180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.897+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/3962318886 >> 172.21.15.142:0/3962318886 conn(0x7f8c240fc650 msgr2=0x7f8c240fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.898+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/3962318886 shutdown_connections 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.898+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/3962318886 wait complete. 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.898+0000 7f8c2c3e3700 1 Processor -- start 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.898+0000 7f8c2c3e3700 1 -- start start 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.898+0000 7f8c2c3e3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c24100da0 0x7f8c24073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.898+0000 7f8c2c3e3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c24101750 0x7f8c240735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.898+0000 7f8c2c3e3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c24073ba0 con 0x7f8c24100da0 2024-03-29T04:41:09.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.898+0000 7f8c2c3e3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c24073d10 con 0x7f8c24101750 2024-03-29T04:41:09.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.898+0000 7f8c2a17f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c24100da0 0x7f8c24073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:09.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.898+0000 7f8c2a17f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c24100da0 0x7f8c24073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55212/0 (socket says 172.21.15.142:55212) 2024-03-29T04:41:09.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.898+0000 7f8c2a17f700 1 -- 172.21.15.142:0/1015918354 learned_addr learned my addr 172.21.15.142:0/1015918354 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:09.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.899+0000 7f8c2a17f700 1 -- 172.21.15.142:0/1015918354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c24101750 msgr2=0x7f8c240735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:09.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.899+0000 7f8c2a17f700 1 --2- 172.21.15.142:0/1015918354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c24101750 0x7f8c240735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:09.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.899+0000 7f8c2a17f700 1 -- 172.21.15.142:0/1015918354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c200095f0 con 0x7f8c24100da0 2024-03-29T04:41:09.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.899+0000 7f8c2a17f700 1 --2- 172.21.15.142:0/1015918354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c24100da0 0x7f8c24073090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c20006010 tx=0x7f8c2000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:09.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.899+0000 7f8c1b7fe700 1 -- 172.21.15.142:0/1015918354 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c2001d070 con 0x7f8c24100da0 2024-03-29T04:41:09.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.899+0000 7f8c1b7fe700 1 -- 172.21.15.142:0/1015918354 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8c2000f460 con 0x7f8c24100da0 2024-03-29T04:41:09.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.899+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/1015918354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c241a2ab0 con 0x7f8c24100da0 2024-03-29T04:41:09.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.899+0000 7f8c1b7fe700 1 -- 172.21.15.142:0/1015918354 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c20021640 con 0x7f8c24100da0 2024-03-29T04:41:09.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.899+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/1015918354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c241a2e60 con 0x7f8c24100da0 2024-03-29T04:41:09.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.900+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/1015918354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c24109140 con 0x7f8c24100da0 2024-03-29T04:41:09.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.901+0000 7f8c1b7fe700 1 -- 172.21.15.142:0/1015918354 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f8c2000faf0 con 0x7f8c24100da0 2024-03-29T04:41:09.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.901+0000 7f8c1b7fe700 1 --2- 172.21.15.142:0/1015918354 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8c1006c4e0 0x7f8c1006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:09.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.901+0000 7f8c1b7fe700 1 -- 172.21.15.142:0/1015918354 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8c2008c7b0 con 0x7f8c24100da0 2024-03-29T04:41:09.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.901+0000 7f8c2997e700 1 --2- 172.21.15.142:0/1015918354 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8c1006c4e0 0x7f8c1006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:09.898 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.901+0000 7f8c2997e700 1 --2- 172.21.15.142:0/1015918354 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8c1006c4e0 0x7f8c1006e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8c24074880 tx=0x7f8c14006d20 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:09.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:09.906+0000 7f8c1b7fe700 1 -- 172.21.15.142:0/1015918354 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8c20057350 con 0x7f8c24100da0 2024-03-29T04:41:10.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.095+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/1015918354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c24108b20 con 0x7f8c24100da0 2024-03-29T04:41:10.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.096+0000 7f8c1b7fe700 1 -- 172.21.15.142:0/1015918354 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11372 (secure 0 0 0) 0x7f8c2005a970 con 0x7f8c24100da0 2024-03-29T04:41:10.091 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:41:10.091 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":41,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","created":"2024-03-29T04:35:07.701328+0000","modified":"2024-03-29T04:40:47.629046+0000","last_up_change":"2024-03-29T04:40:46.622246+0000","last_in_change":"2024-03-29T04:40:27.009053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:39:31.891786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d908de1a-bec3-4238-959a-f3f550431ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6803","nonce":3710621973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6805","nonce":3710621973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6809","nonce":3710621973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6807","nonce":3710621973}]},"public_addr":"172.21.15.142:6803/3710621973","cluster_addr":"172.21.15.142:6805/3710621973","heartbeat_back_addr":"172.21.15.142:6809/3710621973","heartbeat_front_addr":"172.21.15.142:6807/3710621973","state":["exists","up"]},{"osd":1,"uuid":"717e7135-21ab-4d33-a81e-65da6d5186c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6811","nonce":1500472047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6813","nonce":1500472047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6817","nonce":1500472047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6815","nonce":1500472047}]},"public_addr":"172.21.15.142:6811/1500472047","cluster_addr":"172.21.15.142:6813/1500472047","heartbeat_back_addr":"172.21.15.142:6817/1500472047","heartbeat_front_addr":"172.21.15.142:6815/1500472047","state":["exists","up"]},{"osd":2,"uuid":"977032d2-3c29-4c68-ae66-73d024ab94ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6819","nonce":1815862059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6821","nonce":1815862059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6825","nonce":1815862059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6823","nonce":1815862059}]},"public_addr":"172.21.15.142:6819/1815862059","cluster_addr":"172.21.15.142:6821/1815862059","heartbeat_back_addr":"172.21.15.142:6825/1815862059","heartbeat_front_addr":"172.21.15.142:6823/1815862059","state":["exists","up"]},{"osd":3,"uuid":"39ae7173-3a83-4ae1-9866-5a0da908171c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6801","nonce":3854389166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6803","nonce":3854389166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6807","nonce":3854389166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6805","nonce":3854389166}]},"public_addr":"172.21.15.152:6801/3854389166","cluster_addr":"172.21.15.152:6803/3854389166","heartbeat_back_addr":"172.21.15.152:6807/3854389166","heartbeat_front_addr":"172.21.15.152:6805/3854389166","state":["exists","up"]},{"osd":4,"uuid":"f7b5fab7-0f30-4756-a481-15ad64b9e691","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6809","nonce":2813629639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6811","nonce":2813629639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6815","nonce":2813629639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6813","nonce":2813629639}]},"public_addr":"172.21.15.152:6809/2813629639","cluster_addr":"172.21.15.152:6811/2813629639","heartbeat_back_addr":"172.21.15.152:6815/2813629639","heartbeat_front_addr":"172.21.15.152:6813/2813629639","state":["exists","up"]},{"osd":5,"uuid":"8ed3caba-b201-4ad7-b055-3dc40610ab1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6817","nonce":48435269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6819","nonce":48435269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6823","nonce":48435269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6821","nonce":48435269}]},"public_addr":"172.21.15.152:6817/48435269","cluster_addr":"172.21.15.152:6819/48435269","heartbeat_back_addr":"172.21.15.152:6823/48435269","heartbeat_front_addr":"172.21.15.152:6821/48435269","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:38:33.605811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:00.911691+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:28.052973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:51.974988+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:18.104282+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:44.747877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6801/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:6800/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/3338740178":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/586295950":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/672130862":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/2545215192":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/966342870":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6800/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6801/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/2666008640":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/3751399855":"2024-03-30T04:35:33.215645+0000","172.21.15.142:6800/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:6801/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/1935271872":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/390665991":"2024-03-30T04:36:06.145588+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:41:10.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.099+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/1015918354 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8c1006c4e0 msgr2=0x7f8c1006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:10.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.099+0000 7f8c2c3e3700 1 --2- 172.21.15.142:0/1015918354 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8c1006c4e0 0x7f8c1006e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8c24074880 tx=0x7f8c14006d20 comp rx=0 tx=0).stop 2024-03-29T04:41:10.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.099+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/1015918354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c24100da0 msgr2=0x7f8c24073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:10.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.099+0000 7f8c2c3e3700 1 --2- 172.21.15.142:0/1015918354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c24100da0 0x7f8c24073090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c20006010 tx=0x7f8c2000be20 comp rx=0 tx=0).stop 2024-03-29T04:41:10.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.099+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/1015918354 shutdown_connections 2024-03-29T04:41:10.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.099+0000 7f8c2c3e3700 1 --2- 172.21.15.142:0/1015918354 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8c1006c4e0 0x7f8c1006e9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:10.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.099+0000 7f8c2c3e3700 1 --2- 172.21.15.142:0/1015918354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c24100da0 0x7f8c24073090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:10.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.099+0000 7f8c2c3e3700 1 --2- 172.21.15.142:0/1015918354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c24101750 0x7f8c240735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:10.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.099+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/1015918354 >> 172.21.15.142:0/1015918354 conn(0x7f8c240fc650 msgr2=0x7f8c240fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:10.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.099+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/1015918354 shutdown_connections 2024-03-29T04:41:10.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:10.099+0000 7f8c2c3e3700 1 -- 172.21.15.142:0/1015918354 wait complete. 2024-03-29T04:41:10.796 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-29T04:41:10.796 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-29T04:41:10.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-29T04:41:10.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-29T04:41:10.798 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-29T04:41:10.798 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-29T04:41:11.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:10 smithi152 ceph-mon[37177]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:11.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:10 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1015918354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T04:41:11.110 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:11.111 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:11.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:10 smithi142 ceph-mon[29027]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:11.116 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:10 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1015918354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T04:41:11.162 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:12.849 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:12 smithi142 ceph-mon[29027]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:13.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:12 smithi152 ceph-mon[37177]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:13.209 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:13.710 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:13.710 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:14.705 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:14 smithi142 ceph-mon[29027]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:14.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:14 smithi152 ceph-mon[37177]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:16.697 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:16 smithi142 ceph-mon[29027]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:17.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.039+0000 7fab80f18700 1 -- 172.21.15.142:0/421645500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab7c0ff530 msgr2=0x7fab7c0ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:17.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.039+0000 7fab80f18700 1 --2- 172.21.15.142:0/421645500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab7c0ff530 0x7fab7c0ff910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009910 tx=0x7fab6c009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:17.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.039+0000 7fab80f18700 1 -- 172.21.15.142:0/421645500 shutdown_connections 2024-03-29T04:41:17.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.039+0000 7fab80f18700 1 --2- 172.21.15.142:0/421645500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab7c0ffee0 0x7fab7c105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:17.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.039+0000 7fab80f18700 1 --2- 172.21.15.142:0/421645500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab7c0ff530 0x7fab7c0ff910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:17.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.039+0000 7fab80f18700 1 -- 172.21.15.142:0/421645500 >> 172.21.15.142:0/421645500 conn(0x7fab7c074d50 msgr2=0x7fab7c075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:17.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.039+0000 7fab80f18700 1 -- 172.21.15.142:0/421645500 shutdown_connections 2024-03-29T04:41:17.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.039+0000 7fab80f18700 1 -- 172.21.15.142:0/421645500 wait complete. 2024-03-29T04:41:17.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:16 smithi152 ceph-mon[37177]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:17.037 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.040+0000 7fab80f18700 1 Processor -- start 2024-03-29T04:41:17.037 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.040+0000 7fab80f18700 1 -- start start 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.040+0000 7fab80f18700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab7c0ff530 0x7fab7c19c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.040+0000 7fab80f18700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab7c0ffee0 0x7fab7c19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.040+0000 7fab80f18700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c19d1b0 con 0x7fab7c0ff530 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.040+0000 7fab80f18700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c196660 con 0x7fab7c0ffee0 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab7a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab7c0ff530 0x7fab7c19c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab7a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab7c0ff530 0x7fab7c19c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55242/0 (socket says 172.21.15.142:55242) 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab7a59c700 1 -- 172.21.15.142:0/2413729876 learned_addr learned my addr 172.21.15.142:0/2413729876 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab73fff700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab7c0ffee0 0x7fab7c19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab7a59c700 1 -- 172.21.15.142:0/2413729876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab7c0ffee0 msgr2=0x7fab7c19cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab7a59c700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab7c0ffee0 0x7fab7c19cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab7a59c700 1 -- 172.21.15.142:0/2413729876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c0095f0 con 0x7fab7c0ff530 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab7a59c700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab7c0ff530 0x7fab7c19c5e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab6c006010 tx=0x7fab6c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:17.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab737fe700 1 -- 172.21.15.142:0/2413729876 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6c01d070 con 0x7fab7c0ff530 2024-03-29T04:41:17.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab737fe700 1 -- 172.21.15.142:0/2413729876 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fab6c00f460 con 0x7fab7c0ff530 2024-03-29T04:41:17.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab80f18700 1 -- 172.21.15.142:0/2413729876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7c1968e0 con 0x7fab7c0ff530 2024-03-29T04:41:17.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab737fe700 1 -- 172.21.15.142:0/2413729876 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6c017630 con 0x7fab7c0ff530 2024-03-29T04:41:17.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.041+0000 7fab80f18700 1 -- 172.21.15.142:0/2413729876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7c196dd0 con 0x7fab7c0ff530 2024-03-29T04:41:17.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.042+0000 7fab80f18700 1 -- 172.21.15.142:0/2413729876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab7c10e0c0 con 0x7fab7c0ff530 2024-03-29T04:41:17.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.043+0000 7fab737fe700 1 -- 172.21.15.142:0/2413729876 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fab6c00f5d0 con 0x7fab7c0ff530 2024-03-29T04:41:17.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.043+0000 7fab737fe700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fab6806c4e0 0x7fab6806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:17.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.043+0000 7fab737fe700 1 -- 172.21.15.142:0/2413729876 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fab6c08ce10 con 0x7fab7c0ff530 2024-03-29T04:41:17.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.043+0000 7fab73fff700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fab6806c4e0 0x7fab6806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:17.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.043+0000 7fab737fe700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166] conn(0x7fab680721c0 0x7fab680745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:17.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.043+0000 7fab737fe700 1 -- 172.21.15.142:0/2413729876 --> [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab68074c90 con 0x7fab680721c0 2024-03-29T04:41:17.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.043+0000 7fab737fe700 1 -- 172.21.15.142:0/2413729876 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab6c08d270 con 0x7fab7c0ff530 2024-03-29T04:41:17.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.044+0000 7fab7ad9d700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166] conn(0x7fab680721c0 0x7fab680745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:17.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.044+0000 7fab73fff700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fab6806c4e0 0x7fab6806e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab7c1077d0 tx=0x7fab6400b3c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:17.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.044+0000 7fab7ad9d700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166] conn(0x7fab680721c0 0x7fab680745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:17.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.045+0000 7fab737fe700 1 -- 172.21.15.142:0/2413729876 <== osd.3 v2:172.21.15.152:6800/3854389166 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fab68074c90 con 0x7fab680721c0 2024-03-29T04:41:17.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.060+0000 7fab80f18700 1 -- 172.21.15.142:0/2413729876 --> [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab7c04ea90 con 0x7fab680721c0 2024-03-29T04:41:17.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab737fe700 1 -- 172.21.15.142:0/2413729876 <== osd.3 v2:172.21.15.152:6800/3854389166 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fab7c04ea90 con 0x7fab680721c0 2024-03-29T04:41:17.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 -- 172.21.15.142:0/2413729876 >> [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166] conn(0x7fab680721c0 msgr2=0x7fab680745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:17.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166] conn(0x7fab680721c0 0x7fab680745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:17.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 -- 172.21.15.142:0/2413729876 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fab6806c4e0 msgr2=0x7fab6806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:17.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fab6806c4e0 0x7fab6806e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab7c1077d0 tx=0x7fab6400b3c0 comp rx=0 tx=0).stop 2024-03-29T04:41:17.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 -- 172.21.15.142:0/2413729876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab7c0ff530 msgr2=0x7fab7c19c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:17.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab7c0ff530 0x7fab7c19c5e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab6c006010 tx=0x7fab6c00bd70 comp rx=0 tx=0).stop 2024-03-29T04:41:17.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 -- 172.21.15.142:0/2413729876 shutdown_connections 2024-03-29T04:41:17.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.152:6800/3854389166,v1:172.21.15.152:6801/3854389166] conn(0x7fab680721c0 0x7fab680745e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:17.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fab6806c4e0 0x7fab6806e9a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:17.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab7c0ff530 0x7fab7c19c5e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:17.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 --2- 172.21.15.142:0/2413729876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab7c0ffee0 0x7fab7c19cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:17.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 -- 172.21.15.142:0/2413729876 >> 172.21.15.142:0/2413729876 conn(0x7fab7c074d50 msgr2=0x7fab7c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:17.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 -- 172.21.15.142:0/2413729876 shutdown_connections 2024-03-29T04:41:17.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:17.061+0000 7fab80f18700 1 -- 172.21.15.142:0/2413729876 wait complete. 2024-03-29T04:41:17.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:41:17.913 INFO:teuthology.orchestra.run.smithi142.stdout:115964117011 2024-03-29T04:41:17.913 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-29T04:41:18.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:17 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:41:18.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:18 smithi142 ceph-mon[29027]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:19.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:18 smithi152 ceph-mon[37177]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:19.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.768+0000 7f9a06197700 1 -- 172.21.15.142:0/2699380193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a000ffbd0 msgr2=0x7f9a00100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:19.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.768+0000 7f9a06197700 1 --2- 172.21.15.142:0/2699380193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a000ffbd0 0x7f9a00100050 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009910 tx=0x7f99f4009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:19.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.768+0000 7f9a06197700 1 -- 172.21.15.142:0/2699380193 shutdown_connections 2024-03-29T04:41:19.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.768+0000 7f9a06197700 1 --2- 172.21.15.142:0/2699380193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a000ffbd0 0x7f9a00100050 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:19.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.768+0000 7f9a06197700 1 --2- 172.21.15.142:0/2699380193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a00105a40 0x7f9a00105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:19.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.768+0000 7f9a06197700 1 -- 172.21.15.142:0/2699380193 >> 172.21.15.142:0/2699380193 conn(0x7f9a000755a0 msgr2=0x7f9a000759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:19.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.768+0000 7f9a06197700 1 -- 172.21.15.142:0/2699380193 shutdown_connections 2024-03-29T04:41:19.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.768+0000 7f9a06197700 1 -- 172.21.15.142:0/2699380193 wait complete. 2024-03-29T04:41:19.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.769+0000 7f9a06197700 1 Processor -- start 2024-03-29T04:41:19.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.769+0000 7f9a06197700 1 -- start start 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.769+0000 7f9a06197700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a000ffbd0 0x7f9a00195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.769+0000 7f9a06197700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a00105a40 0x7f9a00195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.769+0000 7f9a06197700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a001961b0 con 0x7f9a00105a40 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.769+0000 7f9a06197700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a001962f0 con 0x7f9a000ffbd0 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f99feffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a00105a40 0x7f9a00195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f99feffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a00105a40 0x7f9a00195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33174/0 (socket says 172.21.15.142:33174) 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f99feffd700 1 -- 172.21.15.142:0/33634034 learned_addr learned my addr 172.21.15.142:0/33634034 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f99ff7fe700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a000ffbd0 0x7f9a00195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f99feffd700 1 -- 172.21.15.142:0/33634034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a000ffbd0 msgr2=0x7f9a00195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f99feffd700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a000ffbd0 0x7f9a00195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f99feffd700 1 -- 172.21.15.142:0/33634034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f40095f0 con 0x7f9a00105a40 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f99feffd700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a00105a40 0x7f9a00195b60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f99f4006010 tx=0x7f99f400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:19.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f99fcff9700 1 -- 172.21.15.142:0/33634034 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99f401d070 con 0x7f9a00105a40 2024-03-29T04:41:19.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f99fcff9700 1 -- 172.21.15.142:0/33634034 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f99f4004fa0 con 0x7f9a00105a40 2024-03-29T04:41:19.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f99fcff9700 1 -- 172.21.15.142:0/33634034 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99f400f690 con 0x7f9a00105a40 2024-03-29T04:41:19.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f9a06197700 1 -- 172.21.15.142:0/33634034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0019a0e0 con 0x7f9a00105a40 2024-03-29T04:41:19.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.770+0000 7f9a06197700 1 -- 172.21.15.142:0/33634034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0019a580 con 0x7f9a00105a40 2024-03-29T04:41:19.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.771+0000 7f9a06197700 1 -- 172.21.15.142:0/33634034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a0010cf40 con 0x7f9a00105a40 2024-03-29T04:41:19.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.772+0000 7f99fcff9700 1 -- 172.21.15.142:0/33634034 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f99f4022470 con 0x7f9a00105a40 2024-03-29T04:41:19.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.772+0000 7f99fcff9700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f99e806c4e0 0x7f99e806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:19.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.772+0000 7f99ff7fe700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f99e806c4e0 0x7f99e806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:19.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.772+0000 7f99fcff9700 1 -- 172.21.15.142:0/33634034 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f99f408cc20 con 0x7f9a00105a40 2024-03-29T04:41:19.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.772+0000 7f99fcff9700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639] conn(0x7f99e80721c0 0x7f99e80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:19.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.772+0000 7f99fcff9700 1 -- 172.21.15.142:0/33634034 --> [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99e8074c90 con 0x7f99e80721c0 2024-03-29T04:41:19.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.772+0000 7f99fcff9700 1 -- 172.21.15.142:0/33634034 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99f4092300 con 0x7f9a00105a40 2024-03-29T04:41:19.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.773+0000 7f99ff7fe700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f99e806c4e0 0x7f99e806e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f99f000ac50 tx=0x7f99f000a380 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:19.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.773+0000 7f99fffff700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639] conn(0x7f99e80721c0 0x7f99e80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:19.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.773+0000 7f99fffff700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639] conn(0x7f99e80721c0 0x7f99e80745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:19.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.774+0000 7f99fcff9700 1 -- 172.21.15.142:0/33634034 <== osd.4 v2:172.21.15.152:6808/2813629639 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f99e8074c90 con 0x7f99e80721c0 2024-03-29T04:41:19.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.797+0000 7f9a06197700 1 -- 172.21.15.142:0/33634034 --> [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9a0004ea90 con 0x7f99e80721c0 2024-03-29T04:41:19.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f99fcff9700 1 -- 172.21.15.142:0/33634034 <== osd.4 v2:172.21.15.152:6808/2813629639 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9a0004ea90 con 0x7f99e80721c0 2024-03-29T04:41:19.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 -- 172.21.15.142:0/33634034 >> [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639] conn(0x7f99e80721c0 msgr2=0x7f99e80745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:19.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639] conn(0x7f99e80721c0 0x7f99e80745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:19.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 -- 172.21.15.142:0/33634034 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f99e806c4e0 msgr2=0x7f99e806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:19.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f99e806c4e0 0x7f99e806e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f99f000ac50 tx=0x7f99f000a380 comp rx=0 tx=0).stop 2024-03-29T04:41:19.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 -- 172.21.15.142:0/33634034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a00105a40 msgr2=0x7f9a00195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:19.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a00105a40 0x7f9a00195b60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f99f4006010 tx=0x7f99f400bd70 comp rx=0 tx=0).stop 2024-03-29T04:41:19.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 -- 172.21.15.142:0/33634034 shutdown_connections 2024-03-29T04:41:19.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.152:6808/2813629639,v1:172.21.15.152:6809/2813629639] conn(0x7f99e80721c0 0x7f99e80745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:19.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f99e806c4e0 0x7f99e806e9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:19.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a000ffbd0 0x7f9a00195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:19.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 --2- 172.21.15.142:0/33634034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a00105a40 0x7f9a00195b60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:19.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 -- 172.21.15.142:0/33634034 >> 172.21.15.142:0/33634034 conn(0x7f9a000755a0 msgr2=0x7f9a000fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:19.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 -- 172.21.15.142:0/33634034 shutdown_connections 2024-03-29T04:41:19.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:19.798+0000 7f9a06197700 1 -- 172.21.15.142:0/33634034 wait complete. 2024-03-29T04:41:20.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.010+0000 7f4b525da700 1 -- 172.21.15.142:0/1740724303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b4c101d60 msgr2=0x7f4b4c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:20.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.010+0000 7f4b525da700 1 --2- 172.21.15.142:0/1740724303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b4c101d60 0x7f4b4c102140 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009940 tx=0x7f4b40009c50 comp rx=0 tx=0).stop 2024-03-29T04:41:20.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.010+0000 7f4b525da700 1 -- 172.21.15.142:0/1740724303 shutdown_connections 2024-03-29T04:41:20.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.010+0000 7f4b525da700 1 --2- 172.21.15.142:0/1740724303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c102680 0x7f4b4c10ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:20.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.010+0000 7f4b525da700 1 --2- 172.21.15.142:0/1740724303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b4c101d60 0x7f4b4c102140 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:20.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.010+0000 7f4b525da700 1 -- 172.21.15.142:0/1740724303 >> 172.21.15.142:0/1740724303 conn(0x7f4b4c0fb570 msgr2=0x7f4b4c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.010+0000 7f4b525da700 1 -- 172.21.15.142:0/1740724303 shutdown_connections 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.010+0000 7f4b525da700 1 -- 172.21.15.142:0/1740724303 wait complete. 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.010+0000 7f4b525da700 1 Processor -- start 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.010+0000 7f4b525da700 1 -- start start 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.011+0000 7f4b525da700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b4c101d60 0x7f4b4c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.011+0000 7f4b525da700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c102680 0x7f4b4c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.011+0000 7f4b525da700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c19d100 con 0x7f4b4c101d60 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.011+0000 7f4b525da700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c1965b0 con 0x7f4b4c102680 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.011+0000 7f4b4bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b4c101d60 0x7f4b4c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.011+0000 7f4b4bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b4c101d60 0x7f4b4c19c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33202/0 (socket says 172.21.15.142:33202) 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.011+0000 7f4b4bfff700 1 -- 172.21.15.142:0/92369792 learned_addr learned my addr 172.21.15.142:0/92369792 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.011+0000 7f4b4b7fe700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c102680 0x7f4b4c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.011+0000 7f4b4bfff700 1 -- 172.21.15.142:0/92369792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c102680 msgr2=0x7f4b4c19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:20.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.011+0000 7f4b4bfff700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c102680 0x7f4b4c19ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:20.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.011+0000 7f4b4bfff700 1 -- 172.21.15.142:0/92369792 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b400095f0 con 0x7f4b4c101d60 2024-03-29T04:41:20.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.011+0000 7f4b4bfff700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b4c101d60 0x7f4b4c19c530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4b400055f0 tx=0x7f4b40005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:20.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.012+0000 7f4b497fa700 1 -- 172.21.15.142:0/92369792 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b4001d070 con 0x7f4b4c101d60 2024-03-29T04:41:20.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.012+0000 7f4b497fa700 1 -- 172.21.15.142:0/92369792 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4b4000f460 con 0x7f4b4c101d60 2024-03-29T04:41:20.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.012+0000 7f4b525da700 1 -- 172.21.15.142:0/92369792 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4c196830 con 0x7f4b4c101d60 2024-03-29T04:41:20.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.012+0000 7f4b497fa700 1 -- 172.21.15.142:0/92369792 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b40021640 con 0x7f4b4c101d60 2024-03-29T04:41:20.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.012+0000 7f4b525da700 1 -- 172.21.15.142:0/92369792 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4c196cd0 con 0x7f4b4c101d60 2024-03-29T04:41:20.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.012+0000 7f4b525da700 1 -- 172.21.15.142:0/92369792 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b4c10e080 con 0x7f4b4c101d60 2024-03-29T04:41:20.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.014+0000 7f4b497fa700 1 -- 172.21.15.142:0/92369792 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f4b4000fab0 con 0x7f4b4c101d60 2024-03-29T04:41:20.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.014+0000 7f4b497fa700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4b3406c4e0 0x7f4b3406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:20.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.014+0000 7f4b497fa700 1 -- 172.21.15.142:0/92369792 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4b4008cfd0 con 0x7f4b4c101d60 2024-03-29T04:41:20.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.014+0000 7f4b4b7fe700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4b3406c4e0 0x7f4b3406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:20.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.014+0000 7f4b497fa700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973] conn(0x7f4b340721c0 0x7f4b340745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:20.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.014+0000 7f4b497fa700 1 -- 172.21.15.142:0/92369792 --> [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b34074c90 con 0x7f4b340721c0 2024-03-29T04:41:20.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.014+0000 7f4b497fa700 1 -- 172.21.15.142:0/92369792 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b4008d430 con 0x7f4b4c101d60 2024-03-29T04:41:20.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.014+0000 7f4b50b77700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973] conn(0x7f4b340721c0 0x7f4b340745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:20.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.015+0000 7f4b4b7fe700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4b3406c4e0 0x7f4b3406e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0fcc40 tx=0x7f4b3c00b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:20.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.015+0000 7f4b50b77700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973] conn(0x7f4b340721c0 0x7f4b340745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:20.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.016+0000 7f4b497fa700 1 -- 172.21.15.142:0/92369792 <== osd.0 v2:172.21.15.142:6802/3710621973 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4b34074c90 con 0x7f4b340721c0 2024-03-29T04:41:20.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 -- 172.21.15.142:0/92369792 --> [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b4c04ea90 con 0x7f4b340721c0 2024-03-29T04:41:20.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b497fa700 1 -- 172.21.15.142:0/92369792 <== osd.0 v2:172.21.15.142:6802/3710621973 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4b4c04ea90 con 0x7f4b340721c0 2024-03-29T04:41:20.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 -- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973] conn(0x7f4b340721c0 msgr2=0x7f4b340745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:20.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973] conn(0x7f4b340721c0 0x7f4b340745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:20.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 -- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4b3406c4e0 msgr2=0x7f4b3406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:20.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4b3406c4e0 0x7f4b3406e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0fcc40 tx=0x7f4b3c00b410 comp rx=0 tx=0).stop 2024-03-29T04:41:20.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 -- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b4c101d60 msgr2=0x7f4b4c19c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:20.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b4c101d60 0x7f4b4c19c530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4b400055f0 tx=0x7f4b40005620 comp rx=0 tx=0).stop 2024-03-29T04:41:20.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 -- 172.21.15.142:0/92369792 shutdown_connections 2024-03-29T04:41:20.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:6802/3710621973,v1:172.21.15.142:6803/3710621973] conn(0x7f4b340721c0 0x7f4b340745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:20.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4b3406c4e0 0x7f4b3406e9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:20.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b4c101d60 0x7f4b4c19c530 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:20.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 --2- 172.21.15.142:0/92369792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c102680 0x7f4b4c19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:20.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.033+0000 7f4b525da700 1 -- 172.21.15.142:0/92369792 >> 172.21.15.142:0/92369792 conn(0x7f4b4c0fb570 msgr2=0x7f4b4c1053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:20.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.034+0000 7f4b525da700 1 -- 172.21.15.142:0/92369792 shutdown_connections 2024-03-29T04:41:20.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:20.034+0000 7f4b525da700 1 -- 172.21.15.142:0/92369792 wait complete. 2024-03-29T04:41:20.944 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:20 smithi142 ceph-mon[29027]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:21.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:20 smithi152 ceph-mon[37177]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:21.276 INFO:teuthology.orchestra.run.smithi142.stdout:150323855375 2024-03-29T04:41:21.276 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-29T04:41:21.385 INFO:teuthology.orchestra.run.smithi142.stdout:38654705700 2024-03-29T04:41:21.385 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-29T04:41:21.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.906+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/3484080771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0068670 msgr2=0x7f8bd0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:21.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.906+0000 7f8bd5c9d700 1 --2- 172.21.15.142:0/3484080771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0068670 0x7f8bd0068a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009910 tx=0x7f8bc0009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:21.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.906+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/3484080771 shutdown_connections 2024-03-29T04:41:21.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.906+0000 7f8bd5c9d700 1 --2- 172.21.15.142:0/3484080771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bd0069020 0x7f8bd0105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:21.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.906+0000 7f8bd5c9d700 1 --2- 172.21.15.142:0/3484080771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0068670 0x7f8bd0068a50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:21.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.906+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/3484080771 >> 172.21.15.142:0/3484080771 conn(0x7f8bd00755a0 msgr2=0x7f8bd00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.906+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/3484080771 shutdown_connections 2024-03-29T04:41:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.906+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/3484080771 wait complete. 2024-03-29T04:41:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bd5c9d700 1 Processor -- start 2024-03-29T04:41:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bd5c9d700 1 -- start start 2024-03-29T04:41:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bd5c9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0068670 0x7f8bd00ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bd5c9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bd0069020 0x7f8bd01001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bd5c9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd0100720 con 0x7f8bd0068670 2024-03-29T04:41:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bd5c9d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd0100860 con 0x7f8bd0069020 2024-03-29T04:41:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bcf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0068670 0x7f8bd00ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bcf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0068670 0x7f8bd00ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33226/0 (socket says 172.21.15.142:33226) 2024-03-29T04:41:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bcf7fe700 1 -- 172.21.15.142:0/4276376203 learned_addr learned my addr 172.21.15.142:0/4276376203 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bc6ffd700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bd0069020 0x7f8bd01001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:21.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bcf7fe700 1 -- 172.21.15.142:0/4276376203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bd0069020 msgr2=0x7f8bd01001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:21.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bcf7fe700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bd0069020 0x7f8bd01001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:21.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bcf7fe700 1 -- 172.21.15.142:0/4276376203 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc00095f0 con 0x7f8bd0068670 2024-03-29T04:41:21.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.907+0000 7f8bcf7fe700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0068670 0x7f8bd00ffca0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0006010 tx=0x7f8bc000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:21.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.908+0000 7f8bcd7fa700 1 -- 172.21.15.142:0/4276376203 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bc001d070 con 0x7f8bd0068670 2024-03-29T04:41:21.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.908+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/4276376203 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd0100ae0 con 0x7f8bd0068670 2024-03-29T04:41:21.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.908+0000 7f8bcd7fa700 1 -- 172.21.15.142:0/4276376203 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8bc0004f50 con 0x7f8bd0068670 2024-03-29T04:41:21.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.908+0000 7f8bcd7fa700 1 -- 172.21.15.142:0/4276376203 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bc0021410 con 0x7f8bd0068670 2024-03-29T04:41:21.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.908+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/4276376203 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd01a2c40 con 0x7f8bd0068670 2024-03-29T04:41:21.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.908+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/4276376203 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8bd010e080 con 0x7f8bd0068670 2024-03-29T04:41:21.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.909+0000 7f8bcd7fa700 1 -- 172.21.15.142:0/4276376203 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f8bc000f460 con 0x7f8bd0068670 2024-03-29T04:41:21.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.909+0000 7f8bcd7fa700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8bbc06c4e0 0x7f8bbc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:21.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.910+0000 7f8bcd7fa700 1 -- 172.21.15.142:0/4276376203 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8bc008c6d0 con 0x7f8bd0068670 2024-03-29T04:41:21.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.910+0000 7f8bc6ffd700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8bbc06c4e0 0x7f8bbc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:21.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.910+0000 7f8bcd7fa700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269] conn(0x7f8bbc0721c0 0x7f8bbc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:21.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.910+0000 7f8bcd7fa700 1 -- 172.21.15.142:0/4276376203 --> [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8bbc074c90 con 0x7f8bbc0721c0 2024-03-29T04:41:21.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.910+0000 7f8bcd7fa700 1 -- 172.21.15.142:0/4276376203 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bc008eda0 con 0x7f8bd0068670 2024-03-29T04:41:21.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.910+0000 7f8bc6ffd700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8bbc06c4e0 0x7f8bbc06e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005950 tx=0x7f8bb80058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:21.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.910+0000 7f8bcffff700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269] conn(0x7f8bbc0721c0 0x7f8bbc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:21.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.910+0000 7f8bcffff700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269] conn(0x7f8bbc0721c0 0x7f8bbc0745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:21.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.911+0000 7f8bcd7fa700 1 -- 172.21.15.142:0/4276376203 <== osd.5 v2:172.21.15.152:6816/48435269 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8bbc074c90 con 0x7f8bbc0721c0 2024-03-29T04:41:21.921 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.926+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/4276376203 --> [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8bd004ea90 con 0x7f8bbc0721c0 2024-03-29T04:41:21.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bcd7fa700 1 -- 172.21.15.142:0/4276376203 <== osd.5 v2:172.21.15.152:6816/48435269 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8bd004ea90 con 0x7f8bbc0721c0 2024-03-29T04:41:21.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/4276376203 >> [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269] conn(0x7f8bbc0721c0 msgr2=0x7f8bbc0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:21.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269] conn(0x7f8bbc0721c0 0x7f8bbc0745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:21.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/4276376203 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8bbc06c4e0 msgr2=0x7f8bbc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:21.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8bbc06c4e0 0x7f8bbc06e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005950 tx=0x7f8bb80058e0 comp rx=0 tx=0).stop 2024-03-29T04:41:21.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/4276376203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0068670 msgr2=0x7f8bd00ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:21.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0068670 0x7f8bd00ffca0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0006010 tx=0x7f8bc000be20 comp rx=0 tx=0).stop 2024-03-29T04:41:21.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/4276376203 shutdown_connections 2024-03-29T04:41:21.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.152:6816/48435269,v1:172.21.15.152:6817/48435269] conn(0x7f8bbc0721c0 0x7f8bbc0745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:21.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8bbc06c4e0 0x7f8bbc06e9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:21.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd0068670 0x7f8bd00ffca0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:21.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 --2- 172.21.15.142:0/4276376203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bd0069020 0x7f8bd01001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:21.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/4276376203 >> 172.21.15.142:0/4276376203 conn(0x7f8bd00755a0 msgr2=0x7f8bd00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:21.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/4276376203 shutdown_connections 2024-03-29T04:41:21.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:21.927+0000 7f8bd5c9d700 1 -- 172.21.15.142:0/4276376203 wait complete. 2024-03-29T04:41:22.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.607+0000 7fa88ce24700 1 -- 172.21.15.142:0/1662752393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa888105a40 msgr2=0x7fa888105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:22.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.607+0000 7fa88ce24700 1 --2- 172.21.15.142:0/1662752393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa888105a40 0x7fa888105e20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa878009910 tx=0x7fa878009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:22.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.607+0000 7fa88ce24700 1 -- 172.21.15.142:0/1662752393 shutdown_connections 2024-03-29T04:41:22.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.607+0000 7fa88ce24700 1 --2- 172.21.15.142:0/1662752393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8880ffa40 0x7fa8880ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.607+0000 7fa88ce24700 1 --2- 172.21.15.142:0/1662752393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa888105a40 0x7fa888105e20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.607+0000 7fa88ce24700 1 -- 172.21.15.142:0/1662752393 >> 172.21.15.142:0/1662752393 conn(0x7fa8880fb550 msgr2=0x7fa8880fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:22.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.607+0000 7fa88ce24700 1 -- 172.21.15.142:0/1662752393 shutdown_connections 2024-03-29T04:41:22.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.607+0000 7fa88ce24700 1 -- 172.21.15.142:0/1662752393 wait complete. 2024-03-29T04:41:22.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.608+0000 7fa88ce24700 1 Processor -- start 2024-03-29T04:41:22.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.608+0000 7fa88ce24700 1 -- start start 2024-03-29T04:41:22.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.608+0000 7fa88ce24700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8880ffa40 0x7fa8881956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:22.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.608+0000 7fa88ce24700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa888105a40 0x7fa888195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:22.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.608+0000 7fa88ce24700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa888196280 con 0x7fa888105a40 2024-03-29T04:41:22.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.608+0000 7fa88ce24700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa888198fb0 con 0x7fa8880ffa40 2024-03-29T04:41:22.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.608+0000 7fa885d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa888105a40 0x7fa888195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:22.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.609+0000 7fa885d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa888105a40 0x7fa888195bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33234/0 (socket says 172.21.15.142:33234) 2024-03-29T04:41:22.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.609+0000 7fa885d9b700 1 -- 172.21.15.142:0/283718813 learned_addr learned my addr 172.21.15.142:0/283718813 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:22.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.609+0000 7fa88659c700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8880ffa40 0x7fa8881956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:22.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.609+0000 7fa885d9b700 1 -- 172.21.15.142:0/283718813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8880ffa40 msgr2=0x7fa8881956b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:22.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.609+0000 7fa885d9b700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8880ffa40 0x7fa8881956b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.609+0000 7fa885d9b700 1 -- 172.21.15.142:0/283718813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8780095f0 con 0x7fa888105a40 2024-03-29T04:41:22.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.609+0000 7fa885d9b700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa888105a40 0x7fa888195bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00b4b0 tx=0x7fa87c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:22.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.609+0000 7fa8777fe700 1 -- 172.21.15.142:0/283718813 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa87c011800 con 0x7fa888105a40 2024-03-29T04:41:22.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.609+0000 7fa8777fe700 1 -- 172.21.15.142:0/283718813 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa87c011e10 con 0x7fa888105a40 2024-03-29T04:41:22.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.609+0000 7fa8777fe700 1 -- 172.21.15.142:0/283718813 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa87c00f2e0 con 0x7fa888105a40 2024-03-29T04:41:22.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.609+0000 7fa88ce24700 1 -- 172.21.15.142:0/283718813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa888199290 con 0x7fa888105a40 2024-03-29T04:41:22.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.609+0000 7fa88ce24700 1 -- 172.21.15.142:0/283718813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8881997e0 con 0x7fa888105a40 2024-03-29T04:41:22.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.610+0000 7fa88ce24700 1 -- 172.21.15.142:0/283718813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa88810cfc0 con 0x7fa888105a40 2024-03-29T04:41:22.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.611+0000 7fa8777fe700 1 -- 172.21.15.142:0/283718813 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa87c010380 con 0x7fa888105a40 2024-03-29T04:41:22.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.611+0000 7fa8777fe700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fa87006c4e0 0x7fa87006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:22.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.611+0000 7fa88659c700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fa87006c4e0 0x7fa87006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:22.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.611+0000 7fa8777fe700 1 -- 172.21.15.142:0/283718813 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa87c08afa0 con 0x7fa888105a40 2024-03-29T04:41:22.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.611+0000 7fa8777fe700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059] conn(0x7fa8700721c0 0x7fa8700745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:22.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.611+0000 7fa8777fe700 1 -- 172.21.15.142:0/283718813 --> [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa870074c90 con 0x7fa8700721c0 2024-03-29T04:41:22.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.611+0000 7fa8777fe700 1 -- 172.21.15.142:0/283718813 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa87c08d670 con 0x7fa888105a40 2024-03-29T04:41:22.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.611+0000 7fa886d9d700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059] conn(0x7fa8700721c0 0x7fa8700745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:22.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.611+0000 7fa88659c700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fa87006c4e0 0x7fa87006e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa878005f50 tx=0x7fa87801a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:22.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.612+0000 7fa886d9d700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059] conn(0x7fa8700721c0 0x7fa8700745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:22.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.612+0000 7fa8777fe700 1 -- 172.21.15.142:0/283718813 <== osd.2 v2:172.21.15.142:6818/1815862059 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa870074c90 con 0x7fa8700721c0 2024-03-29T04:41:22.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.627+0000 7fa88ce24700 1 -- 172.21.15.142:0/283718813 --> [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa88804ea90 con 0x7fa8700721c0 2024-03-29T04:41:22.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa8777fe700 1 -- 172.21.15.142:0/283718813 <== osd.2 v2:172.21.15.142:6818/1815862059 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa88804ea90 con 0x7fa8700721c0 2024-03-29T04:41:22.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 -- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059] conn(0x7fa8700721c0 msgr2=0x7fa8700745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:22.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059] conn(0x7fa8700721c0 0x7fa8700745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 -- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fa87006c4e0 msgr2=0x7fa87006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:22.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fa87006c4e0 0x7fa87006e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa878005f50 tx=0x7fa87801a040 comp rx=0 tx=0).stop 2024-03-29T04:41:22.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 -- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa888105a40 msgr2=0x7fa888195bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:22.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa888105a40 0x7fa888195bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00b4b0 tx=0x7fa87c00b7c0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 -- 172.21.15.142:0/283718813 shutdown_connections 2024-03-29T04:41:22.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:6818/1815862059,v1:172.21.15.142:6819/1815862059] conn(0x7fa8700721c0 0x7fa8700745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fa87006c4e0 0x7fa87006e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8880ffa40 0x7fa8881956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 --2- 172.21.15.142:0/283718813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa888105a40 0x7fa888195bf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 -- 172.21.15.142:0/283718813 >> 172.21.15.142:0/283718813 conn(0x7fa8880fb550 msgr2=0x7fa8880fbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:22.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 -- 172.21.15.142:0/283718813 shutdown_connections 2024-03-29T04:41:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.628+0000 7fa88ce24700 1 -- 172.21.15.142:0/283718813 wait complete. 2024-03-29T04:41:22.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.836+0000 7efe86640700 1 -- 172.21.15.142:0/864007846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe800ff050 msgr2=0x7efe800ff4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:22.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.836+0000 7efe86640700 1 --2- 172.21.15.142:0/864007846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe800ff050 0x7efe800ff4d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efe74009910 tx=0x7efe74009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:22.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.836+0000 7efe86640700 1 -- 172.21.15.142:0/864007846 shutdown_connections 2024-03-29T04:41:22.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.836+0000 7efe86640700 1 --2- 172.21.15.142:0/864007846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe800ff050 0x7efe800ff4d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.836+0000 7efe86640700 1 --2- 172.21.15.142:0/864007846 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe80105b20 0x7efe80105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.836+0000 7efe86640700 1 -- 172.21.15.142:0/864007846 >> 172.21.15.142:0/864007846 conn(0x7efe80075e30 msgr2=0x7efe800fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:22.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.836+0000 7efe86640700 1 -- 172.21.15.142:0/864007846 shutdown_connections 2024-03-29T04:41:22.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.837+0000 7efe86640700 1 -- 172.21.15.142:0/864007846 wait complete. 2024-03-29T04:41:22.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.837+0000 7efe86640700 1 Processor -- start 2024-03-29T04:41:22.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.837+0000 7efe86640700 1 -- start start 2024-03-29T04:41:22.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe86640700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe800ff050 0x7efe80100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:22.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe86640700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe80105b20 0x7efe80100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:22.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe86640700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe80100cb0 con 0x7efe80105b20 2024-03-29T04:41:22.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe86640700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe80100df0 con 0x7efe800ff050 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe7f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe80105b20 0x7efe80100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe7f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe80105b20 0x7efe80100770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33246/0 (socket says 172.21.15.142:33246) 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe7f7fe700 1 -- 172.21.15.142:0/3268012313 learned_addr learned my addr 172.21.15.142:0/3268012313 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe7ffff700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe800ff050 0x7efe80100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe7f7fe700 1 -- 172.21.15.142:0/3268012313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe800ff050 msgr2=0x7efe80100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe7f7fe700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe800ff050 0x7efe80100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe7f7fe700 1 -- 172.21.15.142:0/3268012313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe740095f0 con 0x7efe80105b20 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe7f7fe700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe80105b20 0x7efe80100770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efe740092e0 tx=0x7efe7400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe7d7fa700 1 -- 172.21.15.142:0/3268012313 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe7401d070 con 0x7efe80105b20 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe7d7fa700 1 -- 172.21.15.142:0/3268012313 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efe7400f460 con 0x7efe80105b20 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe86640700 1 -- 172.21.15.142:0/3268012313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe80101070 con 0x7efe80105b20 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe7d7fa700 1 -- 172.21.15.142:0/3268012313 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe74005e20 con 0x7efe80105b20 2024-03-29T04:41:22.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.838+0000 7efe86640700 1 -- 172.21.15.142:0/3268012313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe801a2ca0 con 0x7efe80105b20 2024-03-29T04:41:22.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.839+0000 7efe86640700 1 -- 172.21.15.142:0/3268012313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efe8010e0a0 con 0x7efe80105b20 2024-03-29T04:41:22.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.840+0000 7efe7d7fa700 1 -- 172.21.15.142:0/3268012313 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7efe7400fa40 con 0x7efe80105b20 2024-03-29T04:41:22.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.840+0000 7efe7d7fa700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efe6806c4e0 0x7efe6806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:22.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.840+0000 7efe7d7fa700 1 -- 172.21.15.142:0/3268012313 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7efe7408c8e0 con 0x7efe80105b20 2024-03-29T04:41:22.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.840+0000 7efe7ffff700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efe6806c4e0 0x7efe6806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:22.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.840+0000 7efe7d7fa700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047] conn(0x7efe680721c0 0x7efe680745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:22.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.840+0000 7efe7d7fa700 1 -- 172.21.15.142:0/3268012313 --> [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe68074c90 con 0x7efe680721c0 2024-03-29T04:41:22.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.840+0000 7efe7d7fa700 1 -- 172.21.15.142:0/3268012313 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe740912a0 con 0x7efe80105b20 2024-03-29T04:41:22.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.841+0000 7efe84bdd700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047] conn(0x7efe680721c0 0x7efe680745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:22.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.841+0000 7efe7ffff700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efe6806c4e0 0x7efe6806e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe70007c60 tx=0x7efe700073d0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:22.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.841+0000 7efe84bdd700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047] conn(0x7efe680721c0 0x7efe680745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:22.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.842+0000 7efe7d7fa700 1 -- 172.21.15.142:0/3268012313 <== osd.1 v2:172.21.15.142:6810/1500472047 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efe68074c90 con 0x7efe680721c0 2024-03-29T04:41:22.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.857+0000 7efe86640700 1 -- 172.21.15.142:0/3268012313 --> [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efe8004ea90 con 0x7efe680721c0 2024-03-29T04:41:22.852 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:22 smithi142 ceph-mon[29027]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:22.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.857+0000 7efe7d7fa700 1 -- 172.21.15.142:0/3268012313 <== osd.1 v2:172.21.15.142:6810/1500472047 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efe8004ea90 con 0x7efe680721c0 2024-03-29T04:41:22.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.857+0000 7efe86640700 1 -- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047] conn(0x7efe680721c0 msgr2=0x7efe680745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:22.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.857+0000 7efe86640700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047] conn(0x7efe680721c0 0x7efe680745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.858+0000 7efe86640700 1 -- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efe6806c4e0 msgr2=0x7efe6806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:22.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.858+0000 7efe86640700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efe6806c4e0 0x7efe6806e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe70007c60 tx=0x7efe700073d0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.858+0000 7efe86640700 1 -- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe80105b20 msgr2=0x7efe80100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:22.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.858+0000 7efe86640700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe80105b20 0x7efe80100770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efe740092e0 tx=0x7efe7400bd70 comp rx=0 tx=0).stop 2024-03-29T04:41:22.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.858+0000 7efe86640700 1 -- 172.21.15.142:0/3268012313 shutdown_connections 2024-03-29T04:41:22.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.858+0000 7efe86640700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:6810/1500472047,v1:172.21.15.142:6811/1500472047] conn(0x7efe680721c0 0x7efe680745e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.858+0000 7efe86640700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7efe6806c4e0 0x7efe6806e9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.858+0000 7efe86640700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe800ff050 0x7efe80100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.858+0000 7efe86640700 1 --2- 172.21.15.142:0/3268012313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe80105b20 0x7efe80100770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:22.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.858+0000 7efe86640700 1 -- 172.21.15.142:0/3268012313 >> 172.21.15.142:0/3268012313 conn(0x7efe80075e30 msgr2=0x7efe801044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:22.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.858+0000 7efe86640700 1 -- 172.21.15.142:0/3268012313 shutdown_connections 2024-03-29T04:41:22.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:22.858+0000 7efe86640700 1 -- 172.21.15.142:0/3268012313 wait complete. 2024-03-29T04:41:23.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:22 smithi152 ceph-mon[37177]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:23.547 INFO:teuthology.orchestra.run.smithi142.stdout:81604378650 2024-03-29T04:41:23.547 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-29T04:41:24.186 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:24.186 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:24.186 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:24.210 INFO:teuthology.orchestra.run.smithi142.stdout:171798691849 2024-03-29T04:41:24.210 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-29T04:41:24.616 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:24 smithi142 ceph-mon[29027]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:24.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:24 smithi152 ceph-mon[37177]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:25.171 INFO:teuthology.orchestra.run.smithi142.stdout:60129542174 2024-03-29T04:41:25.172 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-29T04:41:25.655 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:26.208 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:26.711 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:26 smithi142 ceph-mon[29027]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:27.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:26 smithi152 ceph-mon[37177]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:29.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:28 smithi152 ceph-mon[37177]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:29.051 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:28 smithi142 ceph-mon[29027]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:29.710 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:30.806 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:30 smithi142 ceph-mon[29027]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:31.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:30 smithi152 ceph-mon[37177]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.114+0000 7f7181812700 1 -- 172.21.15.142:0/3276689305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c069020 msgr2=0x7f717c105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.114+0000 7f7181812700 1 --2- 172.21.15.142:0/3276689305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c069020 0x7f717c105790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7164009910 tx=0x7f7164009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.114+0000 7f7181812700 1 -- 172.21.15.142:0/3276689305 shutdown_connections 2024-03-29T04:41:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.114+0000 7f7181812700 1 --2- 172.21.15.142:0/3276689305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c069020 0x7f717c105790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.114+0000 7f7181812700 1 --2- 172.21.15.142:0/3276689305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f717c068670 0x7f717c068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.114+0000 7f7181812700 1 -- 172.21.15.142:0/3276689305 >> 172.21.15.142:0/3276689305 conn(0x7f717c0755a0 msgr2=0x7f717c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.114+0000 7f7181812700 1 -- 172.21.15.142:0/3276689305 shutdown_connections 2024-03-29T04:41:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.114+0000 7f7181812700 1 -- 172.21.15.142:0/3276689305 wait complete. 2024-03-29T04:41:31.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.115+0000 7f7181812700 1 Processor -- start 2024-03-29T04:41:31.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.115+0000 7f7181812700 1 -- start start 2024-03-29T04:41:31.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.115+0000 7f7181812700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c068670 0x7f717c0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:31.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.115+0000 7f7181812700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f717c069020 0x7f717c1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:31.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.115+0000 7f7181812700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c100720 con 0x7f717c068670 2024-03-29T04:41:31.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.115+0000 7f7181812700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c100860 con 0x7f717c069020 2024-03-29T04:41:31.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.115+0000 7f717affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c068670 0x7f717c0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.115+0000 7f717affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c068670 0x7f717c0ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53240/0 (socket says 172.21.15.142:53240) 2024-03-29T04:41:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.115+0000 7f717affd700 1 -- 172.21.15.142:0/3229627658 learned_addr learned my addr 172.21.15.142:0/3229627658 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.115+0000 7f71727fc700 1 --2- 172.21.15.142:0/3229627658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f717c069020 0x7f717c1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.116+0000 7f717affd700 1 -- 172.21.15.142:0/3229627658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f717c069020 msgr2=0x7f717c1001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.116+0000 7f717affd700 1 --2- 172.21.15.142:0/3229627658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f717c069020 0x7f717c1001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.116+0000 7f717affd700 1 -- 172.21.15.142:0/3229627658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71640095f0 con 0x7f717c068670 2024-03-29T04:41:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.116+0000 7f717affd700 1 --2- 172.21.15.142:0/3229627658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c068670 0x7f717c0ffca0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f716c00c8a0 tx=0x7f716c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.116+0000 7f7180810700 1 -- 172.21.15.142:0/3229627658 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f716c00cea0 con 0x7f717c068670 2024-03-29T04:41:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.116+0000 7f7181812700 1 -- 172.21.15.142:0/3229627658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717c1a28d0 con 0x7f717c068670 2024-03-29T04:41:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.116+0000 7f7180810700 1 -- 172.21.15.142:0/3229627658 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f716c004830 con 0x7f717c068670 2024-03-29T04:41:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.116+0000 7f7180810700 1 -- 172.21.15.142:0/3229627658 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f716c0055b0 con 0x7f717c068670 2024-03-29T04:41:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.116+0000 7f7181812700 1 -- 172.21.15.142:0/3229627658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717c1a2ce0 con 0x7f717c068670 2024-03-29T04:41:31.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.117+0000 7f7181812700 1 -- 172.21.15.142:0/3229627658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717c1090a0 con 0x7f717c068670 2024-03-29T04:41:31.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.117+0000 7f7180810700 1 -- 172.21.15.142:0/3229627658 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f716c007ad0 con 0x7f717c068670 2024-03-29T04:41:31.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.118+0000 7f7180810700 1 --2- 172.21.15.142:0/3229627658 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f716806c4e0 0x7f716806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:31.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.118+0000 7f7180810700 1 -- 172.21.15.142:0/3229627658 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f716c08b240 con 0x7f717c068670 2024-03-29T04:41:31.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.118+0000 7f71727fc700 1 --2- 172.21.15.142:0/3229627658 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f716806c4e0 0x7f716806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:31.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.118+0000 7f71727fc700 1 --2- 172.21.15.142:0/3229627658 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f716806c4e0 0x7f716806e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f717c101490 tx=0x7f716401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:31.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.123+0000 7f7180810700 1 -- 172.21.15.142:0/3229627658 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f716c055d60 con 0x7f717c068670 2024-03-29T04:41:31.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.322+0000 7f7181812700 1 -- 172.21.15.142:0/3229627658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f717c04ea90 con 0x7f717c068670 2024-03-29T04:41:31.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.323+0000 7f7180810700 1 -- 172.21.15.142:0/3229627658 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f716c059380 con 0x7f717c068670 2024-03-29T04:41:31.319 INFO:teuthology.orchestra.run.smithi142.stdout:38654705702 2024-03-29T04:41:31.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.325+0000 7f7181812700 1 -- 172.21.15.142:0/3229627658 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f716806c4e0 msgr2=0x7f716806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:31.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.325+0000 7f7181812700 1 --2- 172.21.15.142:0/3229627658 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f716806c4e0 0x7f716806e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f717c101490 tx=0x7f716401a040 comp rx=0 tx=0).stop 2024-03-29T04:41:31.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.325+0000 7f7181812700 1 -- 172.21.15.142:0/3229627658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c068670 msgr2=0x7f717c0ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:31.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.325+0000 7f7181812700 1 --2- 172.21.15.142:0/3229627658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c068670 0x7f717c0ffca0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f716c00c8a0 tx=0x7f716c00cc60 comp rx=0 tx=0).stop 2024-03-29T04:41:31.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.325+0000 7f7181812700 1 -- 172.21.15.142:0/3229627658 shutdown_connections 2024-03-29T04:41:31.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.325+0000 7f7181812700 1 --2- 172.21.15.142:0/3229627658 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f716806c4e0 0x7f716806e9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.325+0000 7f7181812700 1 --2- 172.21.15.142:0/3229627658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c068670 0x7f717c0ffca0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.325+0000 7f7181812700 1 --2- 172.21.15.142:0/3229627658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f717c069020 0x7f717c1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.325+0000 7f7181812700 1 -- 172.21.15.142:0/3229627658 >> 172.21.15.142:0/3229627658 conn(0x7f717c0755a0 msgr2=0x7f717c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:31.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.325+0000 7f7181812700 1 -- 172.21.15.142:0/3229627658 shutdown_connections 2024-03-29T04:41:31.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.325+0000 7f7181812700 1 -- 172.21.15.142:0/3229627658 wait complete. 2024-03-29T04:41:31.720 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:31 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3229627658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-29T04:41:31.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.734+0000 7fe1f12fa700 1 -- 172.21.15.142:0/3742666420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ec105a40 msgr2=0x7fe1ec105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:31.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.734+0000 7fe1f12fa700 1 --2- 172.21.15.142:0/3742666420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ec105a40 0x7fe1ec105e20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009910 tx=0x7fe1dc009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:31.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.734+0000 7fe1f12fa700 1 -- 172.21.15.142:0/3742666420 shutdown_connections 2024-03-29T04:41:31.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.734+0000 7fe1f12fa700 1 --2- 172.21.15.142:0/3742666420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ec0ffa40 0x7fe1ec0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.734+0000 7fe1f12fa700 1 --2- 172.21.15.142:0/3742666420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ec105a40 0x7fe1ec105e20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.734+0000 7fe1f12fa700 1 -- 172.21.15.142:0/3742666420 >> 172.21.15.142:0/3742666420 conn(0x7fe1ec0fb570 msgr2=0x7fe1ec0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:31.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.734+0000 7fe1f12fa700 1 -- 172.21.15.142:0/3742666420 shutdown_connections 2024-03-29T04:41:31.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.734+0000 7fe1f12fa700 1 -- 172.21.15.142:0/3742666420 wait complete. 2024-03-29T04:41:31.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.734+0000 7fe1f12fa700 1 Processor -- start 2024-03-29T04:41:31.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1f12fa700 1 -- start start 2024-03-29T04:41:31.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1f12fa700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ec0ffa40 0x7fe1ec1953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:31.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1f12fa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ec105a40 0x7fe1ec1958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:31.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1f12fa700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec195e30 con 0x7fe1ec105a40 2024-03-29T04:41:31.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1f12fa700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec195fa0 con 0x7fe1ec0ffa40 2024-03-29T04:41:31.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1ea7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ec105a40 0x7fe1ec1958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:31.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1ea7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ec105a40 0x7fe1ec1958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53252/0 (socket says 172.21.15.142:53252) 2024-03-29T04:41:31.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1ea7fc700 1 -- 172.21.15.142:0/879013977 learned_addr learned my addr 172.21.15.142:0/879013977 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:31.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1eaffd700 1 --2- 172.21.15.142:0/879013977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ec0ffa40 0x7fe1ec1953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:31.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1ea7fc700 1 -- 172.21.15.142:0/879013977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ec0ffa40 msgr2=0x7fe1ec1953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:31.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1ea7fc700 1 --2- 172.21.15.142:0/879013977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ec0ffa40 0x7fe1ec1953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1ea7fc700 1 -- 172.21.15.142:0/879013977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0095f0 con 0x7fe1ec105a40 2024-03-29T04:41:31.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1eaffd700 1 --2- 172.21.15.142:0/879013977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ec0ffa40 0x7fe1ec1953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:41:31.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.735+0000 7fe1ea7fc700 1 --2- 172.21.15.142:0/879013977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ec105a40 0x7fe1ec1958f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000d8d0 tx=0x7fe1e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:31.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.736+0000 7fe1d3fff700 1 -- 172.21.15.142:0/879013977 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1e0009940 con 0x7fe1ec105a40 2024-03-29T04:41:31.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.736+0000 7fe1f12fa700 1 -- 172.21.15.142:0/879013977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ec1a2a20 con 0x7fe1ec105a40 2024-03-29T04:41:31.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.736+0000 7fe1d3fff700 1 -- 172.21.15.142:0/879013977 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe1e0010460 con 0x7fe1ec105a40 2024-03-29T04:41:31.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.736+0000 7fe1d3fff700 1 -- 172.21.15.142:0/879013977 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1e000f5f0 con 0x7fe1ec105a40 2024-03-29T04:41:31.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.736+0000 7fe1f12fa700 1 -- 172.21.15.142:0/879013977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ec1a2d80 con 0x7fe1ec105a40 2024-03-29T04:41:31.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.737+0000 7fe1f12fa700 1 -- 172.21.15.142:0/879013977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ec108040 con 0x7fe1ec105a40 2024-03-29T04:41:31.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.737+0000 7fe1d3fff700 1 -- 172.21.15.142:0/879013977 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe1e0009aa0 con 0x7fe1ec105a40 2024-03-29T04:41:31.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.737+0000 7fe1d3fff700 1 --2- 172.21.15.142:0/879013977 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe1d406c4e0 0x7fe1d406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:31.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.737+0000 7fe1eaffd700 1 --2- 172.21.15.142:0/879013977 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe1d406c4e0 0x7fe1d406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:31.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.737+0000 7fe1d3fff700 1 -- 172.21.15.142:0/879013977 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe1e008b470 con 0x7fe1ec105a40 2024-03-29T04:41:31.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.738+0000 7fe1eaffd700 1 --2- 172.21.15.142:0/879013977 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe1d406c4e0 0x7fe1d406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00b5c0 tx=0x7fe1dc0053a0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:31.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.742+0000 7fe1d3fff700 1 -- 172.21.15.142:0/879013977 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe1e0059a20 con 0x7fe1ec105a40 2024-03-29T04:41:31.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.927+0000 7fb0b740c700 1 -- 172.21.15.142:0/2639104289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b0105b50 msgr2=0x7fb0b0105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:31.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.927+0000 7fb0b740c700 1 --2- 172.21.15.142:0/2639104289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b0105b50 0x7fb0b0105f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009910 tx=0x7fb0a0009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:31.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.927+0000 7fb0b740c700 1 -- 172.21.15.142:0/2639104289 shutdown_connections 2024-03-29T04:41:31.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.927+0000 7fb0b740c700 1 --2- 172.21.15.142:0/2639104289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b00fef70 0x7fb0b00ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.927+0000 7fb0b740c700 1 --2- 172.21.15.142:0/2639104289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b0105b50 0x7fb0b0105f30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.927+0000 7fb0b740c700 1 -- 172.21.15.142:0/2639104289 >> 172.21.15.142:0/2639104289 conn(0x7fb0b0075e30 msgr2=0x7fb0b00fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.927+0000 7fb0b740c700 1 -- 172.21.15.142:0/2639104289 shutdown_connections 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.927+0000 7fb0b740c700 1 -- 172.21.15.142:0/2639104289 wait complete. 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.927+0000 7fb0b740c700 1 Processor -- start 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.927+0000 7fb0b740c700 1 -- start start 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b740c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b00fef70 0x7fb0b019c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b740c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b0105b50 0x7fb0b019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b740c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b019d140 con 0x7fb0b00fef70 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b740c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b01965f0 con 0x7fb0b0105b50 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b51a8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b00fef70 0x7fb0b019c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b51a8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b00fef70 0x7fb0b019c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53270/0 (socket says 172.21.15.142:53270) 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b51a8700 1 -- 172.21.15.142:0/182382446 learned_addr learned my addr 172.21.15.142:0/182382446 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b49a7700 1 --2- 172.21.15.142:0/182382446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b0105b50 0x7fb0b019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:31.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b51a8700 1 -- 172.21.15.142:0/182382446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b0105b50 msgr2=0x7fb0b019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:31.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b51a8700 1 --2- 172.21.15.142:0/182382446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b0105b50 0x7fb0b019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b51a8700 1 -- 172.21.15.142:0/182382446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a00095f0 con 0x7fb0b00fef70 2024-03-29T04:41:31.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b51a8700 1 --2- 172.21.15.142:0/182382446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b00fef70 0x7fb0b019c570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0006010 tx=0x7fb0a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:31.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0a67fc700 1 -- 172.21.15.142:0/182382446 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0a001d070 con 0x7fb0b00fef70 2024-03-29T04:41:31.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0a67fc700 1 -- 172.21.15.142:0/182382446 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb0a0004f50 con 0x7fb0b00fef70 2024-03-29T04:41:31.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b740c700 1 -- 172.21.15.142:0/182382446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b0196870 con 0x7fb0b00fef70 2024-03-29T04:41:31.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0a67fc700 1 -- 172.21.15.142:0/182382446 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0a0021410 con 0x7fb0b00fef70 2024-03-29T04:41:31.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.928+0000 7fb0b740c700 1 -- 172.21.15.142:0/182382446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b0196d60 con 0x7fb0b00fef70 2024-03-29T04:41:31.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.929+0000 7fb0b740c700 1 -- 172.21.15.142:0/182382446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b0109120 con 0x7fb0b00fef70 2024-03-29T04:41:31.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.930+0000 7fb0a67fc700 1 -- 172.21.15.142:0/182382446 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb0a000f460 con 0x7fb0b00fef70 2024-03-29T04:41:31.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.930+0000 7fb0a67fc700 1 --2- 172.21.15.142:0/182382446 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb09c06c4e0 0x7fb09c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:31.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.930+0000 7fb0a67fc700 1 -- 172.21.15.142:0/182382446 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb0a008c730 con 0x7fb0b00fef70 2024-03-29T04:41:31.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.930+0000 7fb0b49a7700 1 --2- 172.21.15.142:0/182382446 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb09c06c4e0 0x7fb09c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:31.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.931+0000 7fb0b49a7700 1 --2- 172.21.15.142:0/182382446 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb09c06c4e0 0x7fb09c06e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac005950 tx=0x7fb0ac0058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:31.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.935+0000 7fb0a67fc700 1 -- 172.21.15.142:0/182382446 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb0a005ace0 con 0x7fb0b00fef70 2024-03-29T04:41:31.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.942+0000 7fe1f12fa700 1 -- 172.21.15.142:0/879013977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe1ec04ea90 con 0x7fe1ec105a40 2024-03-29T04:41:31.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.943+0000 7fe1d3fff700 1 -- 172.21.15.142:0/879013977 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe1e00595b0 con 0x7fe1ec105a40 2024-03-29T04:41:31.938 INFO:teuthology.orchestra.run.smithi142.stdout:150323855378 2024-03-29T04:41:31.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.945+0000 7fe1f12fa700 1 -- 172.21.15.142:0/879013977 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe1d406c4e0 msgr2=0x7fe1d406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:31.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.945+0000 7fe1f12fa700 1 --2- 172.21.15.142:0/879013977 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe1d406c4e0 0x7fe1d406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00b5c0 tx=0x7fe1dc0053a0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.945+0000 7fe1f12fa700 1 -- 172.21.15.142:0/879013977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ec105a40 msgr2=0x7fe1ec1958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:31.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.945+0000 7fe1f12fa700 1 --2- 172.21.15.142:0/879013977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ec105a40 0x7fe1ec1958f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000d8d0 tx=0x7fe1e000dc90 comp rx=0 tx=0).stop 2024-03-29T04:41:31.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.945+0000 7fe1f12fa700 1 -- 172.21.15.142:0/879013977 shutdown_connections 2024-03-29T04:41:31.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.945+0000 7fe1f12fa700 1 --2- 172.21.15.142:0/879013977 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe1d406c4e0 0x7fe1d406e9a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.945+0000 7fe1f12fa700 1 --2- 172.21.15.142:0/879013977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1ec0ffa40 0x7fe1ec1953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.945+0000 7fe1f12fa700 1 --2- 172.21.15.142:0/879013977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1ec105a40 0x7fe1ec1958f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:31.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.945+0000 7fe1f12fa700 1 -- 172.21.15.142:0/879013977 >> 172.21.15.142:0/879013977 conn(0x7fe1ec0fb570 msgr2=0x7fe1ec0fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:31.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.945+0000 7fe1f12fa700 1 -- 172.21.15.142:0/879013977 shutdown_connections 2024-03-29T04:41:31.942 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:31.945+0000 7fe1f12fa700 1 -- 172.21.15.142:0/879013977 wait complete. 2024-03-29T04:41:32.035 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:31 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3229627658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-29T04:41:32.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.129+0000 7fb0b740c700 1 -- 172.21.15.142:0/182382446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb0b004ea90 con 0x7fb0b00fef70 2024-03-29T04:41:32.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.130+0000 7fb0a67fc700 1 -- 172.21.15.142:0/182382446 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0a005a870 con 0x7fb0b00fef70 2024-03-29T04:41:32.125 INFO:teuthology.orchestra.run.smithi142.stdout:115964117014 2024-03-29T04:41:32.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.132+0000 7fb0b740c700 1 -- 172.21.15.142:0/182382446 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb09c06c4e0 msgr2=0x7fb09c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:32.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.132+0000 7fb0b740c700 1 --2- 172.21.15.142:0/182382446 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb09c06c4e0 0x7fb09c06e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac005950 tx=0x7fb0ac0058e0 comp rx=0 tx=0).stop 2024-03-29T04:41:32.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.132+0000 7fb0b740c700 1 -- 172.21.15.142:0/182382446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b00fef70 msgr2=0x7fb0b019c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:32.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.132+0000 7fb0b740c700 1 --2- 172.21.15.142:0/182382446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b00fef70 0x7fb0b019c570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0006010 tx=0x7fb0a000be20 comp rx=0 tx=0).stop 2024-03-29T04:41:32.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.132+0000 7fb0b740c700 1 -- 172.21.15.142:0/182382446 shutdown_connections 2024-03-29T04:41:32.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.132+0000 7fb0b740c700 1 --2- 172.21.15.142:0/182382446 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb09c06c4e0 0x7fb09c06e9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:32.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.132+0000 7fb0b740c700 1 --2- 172.21.15.142:0/182382446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b00fef70 0x7fb0b019c570 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:32.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.132+0000 7fb0b740c700 1 --2- 172.21.15.142:0/182382446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0b0105b50 0x7fb0b019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:32.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.132+0000 7fb0b740c700 1 -- 172.21.15.142:0/182382446 >> 172.21.15.142:0/182382446 conn(0x7fb0b0075e30 msgr2=0x7fb0b0104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:32.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.132+0000 7fb0b740c700 1 -- 172.21.15.142:0/182382446 shutdown_connections 2024-03-29T04:41:32.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:32.132+0000 7fb0b740c700 1 -- 172.21.15.142:0/182382446 wait complete. 2024-03-29T04:41:32.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:32 smithi142 ceph-mon[29027]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:32.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:32 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/879013977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-29T04:41:32.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:32 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/182382446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-29T04:41:32.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:41:32.899 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855375 got 150323855378 for osd.4 2024-03-29T04:41:32.900 DEBUG:teuthology.parallel:result is None 2024-03-29T04:41:33.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:32 smithi152 ceph-mon[37177]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:33.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:32 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/879013977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-29T04:41:33.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:32 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/182382446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-29T04:41:33.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:41:33.842 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705700 got 38654705702 for osd.0 2024-03-29T04:41:33.842 DEBUG:teuthology.parallel:result is None 2024-03-29T04:41:34.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.058+0000 7f24edd35700 1 -- 172.21.15.142:0/580123871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24e8101770 msgr2=0x7f24e81057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:34.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.058+0000 7f24edd35700 1 --2- 172.21.15.142:0/580123871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24e8101770 0x7f24e81057c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009910 tx=0x7f24d0009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:34.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.058+0000 7f24edd35700 1 -- 172.21.15.142:0/580123871 shutdown_connections 2024-03-29T04:41:34.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.058+0000 7f24edd35700 1 --2- 172.21.15.142:0/580123871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24e8101770 0x7f24e81057c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:34.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.058+0000 7f24edd35700 1 --2- 172.21.15.142:0/580123871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e8100dc0 0x7f24e81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:34.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.058+0000 7f24edd35700 1 -- 172.21.15.142:0/580123871 >> 172.21.15.142:0/580123871 conn(0x7f24e80fc650 msgr2=0x7f24e80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:34.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.058+0000 7f24edd35700 1 -- 172.21.15.142:0/580123871 shutdown_connections 2024-03-29T04:41:34.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.058+0000 7f24edd35700 1 -- 172.21.15.142:0/580123871 wait complete. 2024-03-29T04:41:34.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.058+0000 7f24edd35700 1 Processor -- start 2024-03-29T04:41:34.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24edd35700 1 -- start start 2024-03-29T04:41:34.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24edd35700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24e8100dc0 0x7f24e819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:34.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24edd35700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e8101770 0x7f24e819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24edd35700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e819d190 con 0x7f24e8100dc0 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24edd35700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e8196640 con 0x7f24e8101770 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24e77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24e8100dc0 0x7f24e819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24e77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24e8100dc0 0x7f24e819c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53278/0 (socket says 172.21.15.142:53278) 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24e77fe700 1 -- 172.21.15.142:0/1368195566 learned_addr learned my addr 172.21.15.142:0/1368195566 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24e6ffd700 1 --2- 172.21.15.142:0/1368195566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e8101770 0x7f24e819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24e77fe700 1 -- 172.21.15.142:0/1368195566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e8101770 msgr2=0x7f24e819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24e77fe700 1 --2- 172.21.15.142:0/1368195566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e8101770 0x7f24e819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24e77fe700 1 -- 172.21.15.142:0/1368195566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00095f0 con 0x7f24e8100dc0 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24e6ffd700 1 --2- 172.21.15.142:0/1368195566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e8101770 0x7f24e819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.059+0000 7f24e77fe700 1 --2- 172.21.15.142:0/1368195566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24e8100dc0 0x7f24e819c5c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f24e000ba50 tx=0x7f24e000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.060+0000 7f24e4ff9700 1 -- 172.21.15.142:0/1368195566 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24e000ca00 con 0x7f24e8100dc0 2024-03-29T04:41:34.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.060+0000 7f24e4ff9700 1 -- 172.21.15.142:0/1368195566 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f24e000cb60 con 0x7f24e8100dc0 2024-03-29T04:41:34.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.060+0000 7f24edd35700 1 -- 172.21.15.142:0/1368195566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e8196920 con 0x7f24e8100dc0 2024-03-29T04:41:34.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.060+0000 7f24e4ff9700 1 -- 172.21.15.142:0/1368195566 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24e0007580 con 0x7f24e8100dc0 2024-03-29T04:41:34.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.060+0000 7f24edd35700 1 -- 172.21.15.142:0/1368195566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e8196e70 con 0x7f24e8100dc0 2024-03-29T04:41:34.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.061+0000 7f24edd35700 1 -- 172.21.15.142:0/1368195566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e8109110 con 0x7f24e8100dc0 2024-03-29T04:41:34.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.061+0000 7f24e4ff9700 1 -- 172.21.15.142:0/1368195566 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f24e000ccd0 con 0x7f24e8100dc0 2024-03-29T04:41:34.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.062+0000 7f24e4ff9700 1 --2- 172.21.15.142:0/1368195566 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f24d406c4e0 0x7f24d406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:34.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.062+0000 7f24e4ff9700 1 -- 172.21.15.142:0/1368195566 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f24e0006e40 con 0x7f24e8100dc0 2024-03-29T04:41:34.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.062+0000 7f24e6ffd700 1 --2- 172.21.15.142:0/1368195566 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f24d406c4e0 0x7f24d406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:34.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.062+0000 7f24e6ffd700 1 --2- 172.21.15.142:0/1368195566 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f24d406c4e0 0x7f24d406e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24d000b5c0 tx=0x7f24d0005480 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:34.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.066+0000 7f24e4ff9700 1 -- 172.21.15.142:0/1368195566 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f24e0059050 con 0x7f24e8100dc0 2024-03-29T04:41:34.263 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.267+0000 7f24edd35700 1 -- 172.21.15.142:0/1368195566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f24e804ea90 con 0x7f24e8100dc0 2024-03-29T04:41:34.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.268+0000 7f24e4ff9700 1 -- 172.21.15.142:0/1368195566 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f24e804ea90 con 0x7f24e8100dc0 2024-03-29T04:41:34.264 INFO:teuthology.orchestra.run.smithi142.stdout:81604378652 2024-03-29T04:41:34.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.270+0000 7f24edd35700 1 -- 172.21.15.142:0/1368195566 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f24d406c4e0 msgr2=0x7f24d406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:34.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.270+0000 7f24edd35700 1 --2- 172.21.15.142:0/1368195566 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f24d406c4e0 0x7f24d406e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24d000b5c0 tx=0x7f24d0005480 comp rx=0 tx=0).stop 2024-03-29T04:41:34.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.270+0000 7f24edd35700 1 -- 172.21.15.142:0/1368195566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24e8100dc0 msgr2=0x7f24e819c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:34.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.270+0000 7f24edd35700 1 --2- 172.21.15.142:0/1368195566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24e8100dc0 0x7f24e819c5c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f24e000ba50 tx=0x7f24e000be10 comp rx=0 tx=0).stop 2024-03-29T04:41:34.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.270+0000 7f24edd35700 1 -- 172.21.15.142:0/1368195566 shutdown_connections 2024-03-29T04:41:34.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.270+0000 7f24edd35700 1 --2- 172.21.15.142:0/1368195566 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f24d406c4e0 0x7f24d406e9a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:34.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.270+0000 7f24edd35700 1 --2- 172.21.15.142:0/1368195566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24e8100dc0 0x7f24e819c5c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:34.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.270+0000 7f24edd35700 1 --2- 172.21.15.142:0/1368195566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24e8101770 0x7f24e819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:34.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.270+0000 7f24edd35700 1 -- 172.21.15.142:0/1368195566 >> 172.21.15.142:0/1368195566 conn(0x7f24e80fc650 msgr2=0x7f24e80fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:34.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.270+0000 7f24edd35700 1 -- 172.21.15.142:0/1368195566 shutdown_connections 2024-03-29T04:41:34.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:34.270+0000 7f24edd35700 1 -- 172.21.15.142:0/1368195566 wait complete. 2024-03-29T04:41:34.608 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117011 got 115964117014 for osd.3 2024-03-29T04:41:34.608 DEBUG:teuthology.parallel:result is None 2024-03-29T04:41:34.697 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:34 smithi142 ceph-mon[29027]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:34.698 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:34 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1368195566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-29T04:41:34.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:34 smithi152 ceph-mon[37177]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:34.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:34 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1368195566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-29T04:41:35.697 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378650 got 81604378652 for osd.2 2024-03-29T04:41:35.697 DEBUG:teuthology.parallel:result is None 2024-03-29T04:41:36.943 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:36 smithi142 ceph-mon[29027]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:36.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.987+0000 7fbba2d30700 1 -- 172.21.15.142:0/3967089392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb9c101d00 msgr2=0x7fbb9c1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:36.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.987+0000 7fbba2d30700 1 --2- 172.21.15.142:0/3967089392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb9c101d00 0x7fbb9c1020e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009910 tx=0x7fbb90009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:36.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.987+0000 7fbba2d30700 1 -- 172.21.15.142:0/3967089392 shutdown_connections 2024-03-29T04:41:36.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.987+0000 7fbba2d30700 1 --2- 172.21.15.142:0/3967089392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb9c102620 0x7fbb9c10ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:36.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.987+0000 7fbba2d30700 1 --2- 172.21.15.142:0/3967089392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb9c101d00 0x7fbb9c1020e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:36.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.987+0000 7fbba2d30700 1 -- 172.21.15.142:0/3967089392 >> 172.21.15.142:0/3967089392 conn(0x7fbb9c0fb570 msgr2=0x7fbb9c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.987+0000 7fbba2d30700 1 -- 172.21.15.142:0/3967089392 shutdown_connections 2024-03-29T04:41:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.988+0000 7fbba2d30700 1 -- 172.21.15.142:0/3967089392 wait complete. 2024-03-29T04:41:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.988+0000 7fbba2d30700 1 Processor -- start 2024-03-29T04:41:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.988+0000 7fbba2d30700 1 -- start start 2024-03-29T04:41:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.988+0000 7fbba2d30700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb9c101d00 0x7fbb9c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.988+0000 7fbba2d30700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb9c102620 0x7fbb9c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.988+0000 7fbba2d30700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c19d070 con 0x7fbb9c102620 2024-03-29T04:41:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.988+0000 7fbba2d30700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c1965b0 con 0x7fbb9c101d00 2024-03-29T04:41:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.988+0000 7fbb9bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb9c102620 0x7fbb9c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.988+0000 7fbb9bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb9c102620 0x7fbb9c19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53308/0 (socket says 172.21.15.142:53308) 2024-03-29T04:41:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.988+0000 7fbb9bfff700 1 -- 172.21.15.142:0/2044066226 learned_addr learned my addr 172.21.15.142:0/2044066226 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.989+0000 7fbba0acc700 1 --2- 172.21.15.142:0/2044066226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb9c101d00 0x7fbb9c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.989+0000 7fbb9bfff700 1 -- 172.21.15.142:0/2044066226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb9c101d00 msgr2=0x7fbb9c19c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.989+0000 7fbb9bfff700 1 --2- 172.21.15.142:0/2044066226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb9c101d00 0x7fbb9c19c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.989+0000 7fbb9bfff700 1 -- 172.21.15.142:0/2044066226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb900095f0 con 0x7fbb9c102620 2024-03-29T04:41:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.989+0000 7fbb9bfff700 1 --2- 172.21.15.142:0/2044066226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb9c102620 0x7fbb9c19ca70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800c8f0 tx=0x7fbb8800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.989+0000 7fbb99ffb700 1 -- 172.21.15.142:0/2044066226 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb880078a0 con 0x7fbb9c102620 2024-03-29T04:41:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.989+0000 7fbb99ffb700 1 -- 172.21.15.142:0/2044066226 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbb8800f450 con 0x7fbb9c102620 2024-03-29T04:41:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.989+0000 7fbba2d30700 1 -- 172.21.15.142:0/2044066226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9c196890 con 0x7fbb9c102620 2024-03-29T04:41:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.989+0000 7fbb99ffb700 1 -- 172.21.15.142:0/2044066226 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb88018630 con 0x7fbb9c102620 2024-03-29T04:41:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.989+0000 7fbba2d30700 1 -- 172.21.15.142:0/2044066226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9c196d90 con 0x7fbb9c102620 2024-03-29T04:41:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.991+0000 7fbba2d30700 1 -- 172.21.15.142:0/2044066226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9c108210 con 0x7fbb9c102620 2024-03-29T04:41:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.991+0000 7fbb99ffb700 1 -- 172.21.15.142:0/2044066226 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbb8800f5c0 con 0x7fbb9c102620 2024-03-29T04:41:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.991+0000 7fbb99ffb700 1 --2- 172.21.15.142:0/2044066226 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbb8c06c4e0 0x7fbb8c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.991+0000 7fbb99ffb700 1 -- 172.21.15.142:0/2044066226 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbb8808b3a0 con 0x7fbb9c102620 2024-03-29T04:41:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.991+0000 7fbba0acc700 1 --2- 172.21.15.142:0/2044066226 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbb8c06c4e0 0x7fbb8c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.992+0000 7fbba0acc700 1 --2- 172.21.15.142:0/2044066226 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbb8c06c4e0 0x7fbb8c06e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbb90006010 tx=0x7fbb9001a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:36.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.996+0000 7fbb99ffb700 1 -- 172.21.15.142:0/2044066226 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbb88059950 con 0x7fbb9c102620 2024-03-29T04:41:37.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:36 smithi152 ceph-mon[37177]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:37.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.891+0000 7f656da98700 1 -- 172.21.15.142:0/1839758729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65680ffa40 msgr2=0x7f65680ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:37.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.891+0000 7f656da98700 1 --2- 172.21.15.142:0/1839758729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65680ffa40 0x7f65680ffec0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f655c009910 tx=0x7f655c009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:37.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.891+0000 7f656da98700 1 -- 172.21.15.142:0/1839758729 shutdown_connections 2024-03-29T04:41:37.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.891+0000 7f656da98700 1 --2- 172.21.15.142:0/1839758729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65680ffa40 0x7f65680ffec0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:37.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.891+0000 7f656da98700 1 --2- 172.21.15.142:0/1839758729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6568105a40 0x7f6568105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:37.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.891+0000 7f656da98700 1 -- 172.21.15.142:0/1839758729 >> 172.21.15.142:0/1839758729 conn(0x7f65680fb570 msgr2=0x7f65680fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.891+0000 7f656da98700 1 -- 172.21.15.142:0/1839758729 shutdown_connections 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.891+0000 7f656da98700 1 -- 172.21.15.142:0/1839758729 wait complete. 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.892+0000 7f656da98700 1 Processor -- start 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.892+0000 7f656da98700 1 -- start start 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.892+0000 7f656da98700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65680ffa40 0x7f656819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.892+0000 7f656da98700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6568105a40 0x7f656819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.892+0000 7f656da98700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656819d0c0 con 0x7f6568105a40 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.892+0000 7f656da98700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6568196570 con 0x7f65680ffa40 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.892+0000 7f6566ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6568105a40 0x7f656819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.892+0000 7f6566ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6568105a40 0x7f656819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53292/0 (socket says 172.21.15.142:53292) 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.892+0000 7f6566ffd700 1 -- 172.21.15.142:0/1976444860 learned_addr learned my addr 172.21.15.142:0/1976444860 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.893+0000 7f65677fe700 1 --2- 172.21.15.142:0/1976444860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65680ffa40 0x7f656819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:37.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.893+0000 7f6566ffd700 1 -- 172.21.15.142:0/1976444860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65680ffa40 msgr2=0x7f656819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.893+0000 7f6566ffd700 1 --2- 172.21.15.142:0/1976444860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65680ffa40 0x7f656819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.893+0000 7f6566ffd700 1 -- 172.21.15.142:0/1976444860 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c0095f0 con 0x7f6568105a40 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.893+0000 7f6566ffd700 1 --2- 172.21.15.142:0/1976444860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6568105a40 0x7f656819ca30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f655c006010 tx=0x7f655c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.893+0000 7f6564ff9700 1 -- 172.21.15.142:0/1976444860 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f655c01d070 con 0x7f6568105a40 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.893+0000 7f656da98700 1 -- 172.21.15.142:0/1976444860 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65681967f0 con 0x7f6568105a40 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.893+0000 7f6564ff9700 1 -- 172.21.15.142:0/1976444860 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f655c00f460 con 0x7f6568105a40 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.893+0000 7f6564ff9700 1 -- 172.21.15.142:0/1976444860 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f655c017630 con 0x7f6568105a40 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.893+0000 7f656da98700 1 -- 172.21.15.142:0/1976444860 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6568196ce0 con 0x7f6568105a40 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.894+0000 7f656da98700 1 -- 172.21.15.142:0/1976444860 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656804ea90 con 0x7f6568105a40 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.895+0000 7f6564ff9700 1 -- 172.21.15.142:0/1976444860 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f655c003a60 con 0x7f6568105a40 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.895+0000 7f6564ff9700 1 --2- 172.21.15.142:0/1976444860 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f655006c4e0 0x7f655006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.895+0000 7f65677fe700 1 --2- 172.21.15.142:0/1976444860 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f655006c4e0 0x7f655006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:37.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.895+0000 7f6564ff9700 1 -- 172.21.15.142:0/1976444860 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f655c08c930 con 0x7f6568105a40 2024-03-29T04:41:37.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.896+0000 7f65677fe700 1 --2- 172.21.15.142:0/1976444860 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f655006c4e0 0x7f655006e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6558004630 tx=0x7f655800b3c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:37.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:36.900+0000 7f6564ff9700 1 -- 172.21.15.142:0/1976444860 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f655c05aee0 con 0x7f6568105a40 2024-03-29T04:41:37.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.123+0000 7f656da98700 1 -- 172.21.15.142:0/1976444860 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6568197160 con 0x7f6568105a40 2024-03-29T04:41:37.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.124+0000 7f6564ff9700 1 -- 172.21.15.142:0/1976444860 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f655c05aa70 con 0x7f6568105a40 2024-03-29T04:41:37.120 INFO:teuthology.orchestra.run.smithi142.stdout:60129542177 2024-03-29T04:41:37.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.126+0000 7f656da98700 1 -- 172.21.15.142:0/1976444860 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f655006c4e0 msgr2=0x7f655006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:37.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.126+0000 7f656da98700 1 --2- 172.21.15.142:0/1976444860 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f655006c4e0 0x7f655006e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6558004630 tx=0x7f655800b3c0 comp rx=0 tx=0).stop 2024-03-29T04:41:37.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.126+0000 7f656da98700 1 -- 172.21.15.142:0/1976444860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6568105a40 msgr2=0x7f656819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:37.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.126+0000 7f656da98700 1 --2- 172.21.15.142:0/1976444860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6568105a40 0x7f656819ca30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f655c006010 tx=0x7f655c00bd70 comp rx=0 tx=0).stop 2024-03-29T04:41:37.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.126+0000 7f656da98700 1 -- 172.21.15.142:0/1976444860 shutdown_connections 2024-03-29T04:41:37.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.126+0000 7f656da98700 1 --2- 172.21.15.142:0/1976444860 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f655006c4e0 0x7f655006e9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:37.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.126+0000 7f656da98700 1 --2- 172.21.15.142:0/1976444860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65680ffa40 0x7f656819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:37.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.126+0000 7f656da98700 1 --2- 172.21.15.142:0/1976444860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6568105a40 0x7f656819ca30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:37.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.126+0000 7f656da98700 1 -- 172.21.15.142:0/1976444860 >> 172.21.15.142:0/1976444860 conn(0x7f65680fb570 msgr2=0x7f6568103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:37.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.126+0000 7f656da98700 1 -- 172.21.15.142:0/1976444860 shutdown_connections 2024-03-29T04:41:37.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.126+0000 7f656da98700 1 -- 172.21.15.142:0/1976444860 wait complete. 2024-03-29T04:41:37.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.203+0000 7fbba2d30700 1 -- 172.21.15.142:0/2044066226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fbb9c04ea90 con 0x7fbb9c102620 2024-03-29T04:41:37.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.203+0000 7fbb99ffb700 1 -- 172.21.15.142:0/2044066226 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb880594e0 con 0x7fbb9c102620 2024-03-29T04:41:37.199 INFO:teuthology.orchestra.run.smithi142.stdout:171798691852 2024-03-29T04:41:37.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.206+0000 7fbba2d30700 1 -- 172.21.15.142:0/2044066226 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbb8c06c4e0 msgr2=0x7fbb8c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:37.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.206+0000 7fbba2d30700 1 --2- 172.21.15.142:0/2044066226 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbb8c06c4e0 0x7fbb8c06e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbb90006010 tx=0x7fbb9001a040 comp rx=0 tx=0).stop 2024-03-29T04:41:37.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.206+0000 7fbba2d30700 1 -- 172.21.15.142:0/2044066226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb9c102620 msgr2=0x7fbb9c19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:37.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.206+0000 7fbba2d30700 1 --2- 172.21.15.142:0/2044066226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb9c102620 0x7fbb9c19ca70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800c8f0 tx=0x7fbb8800cc00 comp rx=0 tx=0).stop 2024-03-29T04:41:37.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.206+0000 7fbba2d30700 1 -- 172.21.15.142:0/2044066226 shutdown_connections 2024-03-29T04:41:37.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.206+0000 7fbba2d30700 1 --2- 172.21.15.142:0/2044066226 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbb8c06c4e0 0x7fbb8c06e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:37.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.206+0000 7fbba2d30700 1 --2- 172.21.15.142:0/2044066226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb9c101d00 0x7fbb9c19c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:37.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.206+0000 7fbba2d30700 1 --2- 172.21.15.142:0/2044066226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb9c102620 0x7fbb9c19ca70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:37.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.206+0000 7fbba2d30700 1 -- 172.21.15.142:0/2044066226 >> 172.21.15.142:0/2044066226 conn(0x7fbb9c0fb570 msgr2=0x7fbb9c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:37.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.206+0000 7fbba2d30700 1 -- 172.21.15.142:0/2044066226 shutdown_connections 2024-03-29T04:41:37.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:37.206+0000 7fbba2d30700 1 -- 172.21.15.142:0/2044066226 wait complete. 2024-03-29T04:41:37.745 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:37 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1976444860' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-29T04:41:37.745 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:37 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2044066226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-29T04:41:37.986 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691849 got 171798691852 for osd.5 2024-03-29T04:41:37.990 DEBUG:teuthology.parallel:result is None 2024-03-29T04:41:38.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:37 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1976444860' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-29T04:41:38.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:37 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2044066226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-29T04:41:38.504 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542177 for osd.1 2024-03-29T04:41:38.504 DEBUG:teuthology.parallel:result is None 2024-03-29T04:41:38.504 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-29T04:41:38.505 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-29T04:41:38.721 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:38.837 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:38 smithi142 ceph-mon[29027]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:39.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:38 smithi152 ceph-mon[37177]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:40.288 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.233+0000 7f03d94b8700 1 -- 172.21.15.142:0/3106614652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03d4068670 msgr2=0x7f03d4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:40.288 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.233+0000 7f03d94b8700 1 --2- 172.21.15.142:0/3106614652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03d4068670 0x7f03d4068a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f03c400b0a0 tx=0x7f03c400b3b0 comp rx=0 tx=0).stop 2024-03-29T04:41:40.288 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.233+0000 7f03d94b8700 1 -- 172.21.15.142:0/3106614652 shutdown_connections 2024-03-29T04:41:40.288 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.233+0000 7f03d94b8700 1 --2- 172.21.15.142:0/3106614652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d4069020 0x7f03d4105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:40.288 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.233+0000 7f03d94b8700 1 --2- 172.21.15.142:0/3106614652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03d4068670 0x7f03d4068a50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:40.288 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.233+0000 7f03d94b8700 1 -- 172.21.15.142:0/3106614652 >> 172.21.15.142:0/3106614652 conn(0x7f03d40755a0 msgr2=0x7f03d40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:40.288 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.234+0000 7f03d94b8700 1 -- 172.21.15.142:0/3106614652 shutdown_connections 2024-03-29T04:41:40.288 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.234+0000 7f03d94b8700 1 -- 172.21.15.142:0/3106614652 wait complete. 2024-03-29T04:41:40.288 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.234+0000 7f03d94b8700 1 Processor -- start 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.234+0000 7f03d94b8700 1 -- start start 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.234+0000 7f03d94b8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d4068670 0x7f03d419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.234+0000 7f03d94b8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03d4069020 0x7f03d419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.234+0000 7f03d94b8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d419d220 con 0x7f03d4069020 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.234+0000 7f03d94b8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4196740 con 0x7f03d4068670 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03d27fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03d4069020 0x7f03d419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03d27fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03d4069020 0x7f03d419cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35110/0 (socket says 172.21.15.142:35110) 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03d27fc700 1 -- 172.21.15.142:0/3182133798 learned_addr learned my addr 172.21.15.142:0/3182133798 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03d2ffd700 1 --2- 172.21.15.142:0/3182133798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d4068670 0x7f03d419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03d27fc700 1 -- 172.21.15.142:0/3182133798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d4068670 msgr2=0x7f03d419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03d27fc700 1 --2- 172.21.15.142:0/3182133798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d4068670 0x7f03d419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03d27fc700 1 -- 172.21.15.142:0/3182133798 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c4009d00 con 0x7f03d4069020 2024-03-29T04:41:40.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03d27fc700 1 --2- 172.21.15.142:0/3182133798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03d4069020 0x7f03d419cb90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00b480 tx=0x7f03bc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:40.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03cbfff700 1 -- 172.21.15.142:0/3182133798 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03bc011800 con 0x7f03d4069020 2024-03-29T04:41:40.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03cbfff700 1 -- 172.21.15.142:0/3182133798 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f03bc011e10 con 0x7f03d4069020 2024-03-29T04:41:40.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03cbfff700 1 -- 172.21.15.142:0/3182133798 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03bc010ea0 con 0x7f03d4069020 2024-03-29T04:41:40.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03d94b8700 1 -- 172.21.15.142:0/3182133798 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d4196a20 con 0x7f03d4069020 2024-03-29T04:41:40.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.235+0000 7f03d94b8700 1 -- 172.21.15.142:0/3182133798 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d4196f20 con 0x7f03d4069020 2024-03-29T04:41:40.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.236+0000 7f03d94b8700 1 -- 172.21.15.142:0/3182133798 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d4109130 con 0x7f03d4069020 2024-03-29T04:41:40.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.237+0000 7f03cbfff700 1 -- 172.21.15.142:0/3182133798 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f03bc010380 con 0x7f03d4069020 2024-03-29T04:41:40.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.237+0000 7f03cbfff700 1 --2- 172.21.15.142:0/3182133798 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f03c006c4e0 0x7f03c006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:40.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.237+0000 7f03d2ffd700 1 --2- 172.21.15.142:0/3182133798 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f03c006c4e0 0x7f03c006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:40.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.237+0000 7f03cbfff700 1 -- 172.21.15.142:0/3182133798 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f03bc08b050 con 0x7f03d4069020 2024-03-29T04:41:40.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.238+0000 7f03d2ffd700 1 --2- 172.21.15.142:0/3182133798 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f03c006c4e0 0x7f03c006e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000f80 tx=0x7f03c40060c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:40.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.242+0000 7f03cbfff700 1 -- 172.21.15.142:0/3182133798 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f03bc059600 con 0x7f03d4069020 2024-03-29T04:41:40.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.428+0000 7f03d94b8700 1 -- 172.21.15.142:0/3182133798 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03d4197240 con 0x7f03c006c4e0 2024-03-29T04:41:40.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.429+0000 7f03cbfff700 1 -- 172.21.15.142:0/3182133798 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28050 (secure 0 0 0) 0x7f03d4197240 con 0x7f03c006c4e0 2024-03-29T04:41:40.425 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:41:40.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.431+0000 7f03d94b8700 1 -- 172.21.15.142:0/3182133798 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f03c006c4e0 msgr2=0x7f03c006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:40.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.431+0000 7f03d94b8700 1 --2- 172.21.15.142:0/3182133798 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f03c006c4e0 0x7f03c006e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000f80 tx=0x7f03c40060c0 comp rx=0 tx=0).stop 2024-03-29T04:41:40.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.431+0000 7f03d94b8700 1 -- 172.21.15.142:0/3182133798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03d4069020 msgr2=0x7f03d419cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:40.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.431+0000 7f03d94b8700 1 --2- 172.21.15.142:0/3182133798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03d4069020 0x7f03d419cb90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00b480 tx=0x7f03bc00b790 comp rx=0 tx=0).stop 2024-03-29T04:41:40.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.431+0000 7f03d94b8700 1 -- 172.21.15.142:0/3182133798 shutdown_connections 2024-03-29T04:41:40.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.431+0000 7f03d94b8700 1 --2- 172.21.15.142:0/3182133798 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f03c006c4e0 0x7f03c006e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:40.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.431+0000 7f03d94b8700 1 --2- 172.21.15.142:0/3182133798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d4068670 0x7f03d419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:40.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.431+0000 7f03d94b8700 1 --2- 172.21.15.142:0/3182133798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03d4069020 0x7f03d419cb90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:40.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.431+0000 7f03d94b8700 1 -- 172.21.15.142:0/3182133798 >> 172.21.15.142:0/3182133798 conn(0x7f03d40755a0 msgr2=0x7f03d40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:40.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.432+0000 7f03d94b8700 1 -- 172.21.15.142:0/3182133798 shutdown_connections 2024-03-29T04:41:40.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:40.432+0000 7f03d94b8700 1 -- 172.21.15.142:0/3182133798 wait complete. 2024-03-29T04:41:40.428 INFO:teuthology.orchestra.run.smithi142.stderr:dumped all 2024-03-29T04:41:41.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:40 smithi152 ceph-mon[37177]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:41.117 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:40 smithi142 ceph-mon[29027]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:41.149 INFO:teuthology.orchestra.run.smithi142.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-03-29T04:41:39.627012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002127"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":185,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:40:46.630015+0000","last_change":"2024-03-29T04:40:19.680217+0000","last_active":"2024-03-29T04:40:46.630015+0000","last_peered":"2024-03-29T04:40:46.630015+0000","last_clean":"2024-03-29T04:40:46.630015+0000","last_became_active":"2024-03-29T04:40:19.679928+0000","last_became_peered":"2024-03-29T04:40:19.679928+0000","last_unstale":"2024-03-29T04:40:46.630015+0000","last_undegraded":"2024-03-29T04:40:46.630015+0000","last_fullsized":"2024-03-29T04:40:46.630015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:39:31.941952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:39:31.941952+0000","last_clean_scrub_stamp":"2024-03-29T04:39:31.941952+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:56:45.534919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]},{"osd":4,"up_from":35,"seq":150323855379,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 04:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.441},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Mar 29 04:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.44700000000000001}]},{"osd":2,"last update":"Fri Mar 29 04:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Mar 29 04:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.33700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 04:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Mar 29 04:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.47899999999999998}]},{"osd":2,"last update":"Fri Mar 29 04:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.505},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Mar 29 04:41:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.432}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 04:41:34 2024","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.32400000000000001,"15min":0.32900000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.44400000000000001,"5min":0.497,"15min":0.497},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.33600000000000002,"15min":0.34300000000000003},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.48999999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.247}]},{"osd":1,"last update":"Fri Mar 29 04:41:34 2024","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35699999999999998,"15min":0.36499999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51200000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.308,"5min":0.34499999999999997,"15min":0.35099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.47799999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.316}]},{"osd":3,"last update":"Fri Mar 29 04:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Fri Mar 29 04:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.42899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 04:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33400000000000002,"15min":0.33300000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.51300000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33800000000000002,"15min":0.33500000000000002},"min":{"1min":0.248,"5min":0.217,"15min":0.217},"max":{"1min":0.497,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Mar 29 04:41:33 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.433,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.374,"15min":0.376},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.432}]},{"osd":3,"last update":"Fri Mar 29 04:40:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":18.658000000000001,"5min":18.658000000000001,"15min":18.658000000000001},"last":0.623}]},{"osd":4,"last update":"Fri Mar 29 04:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 04:41:04 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.25900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.432,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.35199999999999998},"min":{"1min":0.25900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.47099999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.245}]},{"osd":2,"last update":"Fri Mar 29 04:41:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.35399999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46600000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.36099999999999999,"15min":0.36299999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.28199999999999997}]},{"osd":3,"last update":"Fri Mar 29 04:40:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.04,"5min":3.04,"15min":3.04},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Mar 29 04:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:41:41.151 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-29T04:41:41.355 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:41.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:41 smithi142 ceph-mon[29027]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-29T04:41:42.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:41 smithi152 ceph-mon[37177]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-29T04:41:42.742 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:42 smithi142 ceph-mon[29027]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:42.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.947+0000 7f9099d88700 1 -- 172.21.15.142:0/1494280713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9094100fd0 msgr2=0x7f9094104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:42.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.947+0000 7f9099d88700 1 --2- 172.21.15.142:0/1494280713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9094100fd0 0x7f9094104720 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9088009910 tx=0x7f9088009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:42.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.947+0000 7f9099d88700 1 -- 172.21.15.142:0/1494280713 shutdown_connections 2024-03-29T04:41:42.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.947+0000 7f9099d88700 1 --2- 172.21.15.142:0/1494280713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9094100fd0 0x7f9094104720 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:42.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.947+0000 7f9099d88700 1 --2- 172.21.15.142:0/1494280713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9094100620 0x7f9094100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:42.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.947+0000 7f9099d88700 1 -- 172.21.15.142:0/1494280713 >> 172.21.15.142:0/1494280713 conn(0x7f9094074e00 msgr2=0x7f9094075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:42.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.947+0000 7f9099d88700 1 -- 172.21.15.142:0/1494280713 shutdown_connections 2024-03-29T04:41:42.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.947+0000 7f9099d88700 1 -- 172.21.15.142:0/1494280713 wait complete. 2024-03-29T04:41:42.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.948+0000 7f9099d88700 1 Processor -- start 2024-03-29T04:41:42.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.948+0000 7f9099d88700 1 -- start start 2024-03-29T04:41:42.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.948+0000 7f9099d88700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9094100620 0x7f90941017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:42.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.948+0000 7f9099d88700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9094100fd0 0x7f9094101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:42.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.948+0000 7f9099d88700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9094102240 con 0x7f9094100fd0 2024-03-29T04:41:42.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.948+0000 7f9099d88700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9094102380 con 0x7f9094100620 2024-03-29T04:41:42.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.948+0000 7f9092ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9094100fd0 0x7f9094101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:42.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.948+0000 7f9092ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9094100fd0 0x7f9094101d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35136/0 (socket says 172.21.15.142:35136) 2024-03-29T04:41:42.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.948+0000 7f9092ffd700 1 -- 172.21.15.142:0/3856324606 learned_addr learned my addr 172.21.15.142:0/3856324606 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:42.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.949+0000 7f90937fe700 1 --2- 172.21.15.142:0/3856324606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9094100620 0x7f90941017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:42.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.949+0000 7f9092ffd700 1 -- 172.21.15.142:0/3856324606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9094100620 msgr2=0x7f90941017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:42.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.949+0000 7f9092ffd700 1 --2- 172.21.15.142:0/3856324606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9094100620 0x7f90941017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:42.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.949+0000 7f9092ffd700 1 -- 172.21.15.142:0/3856324606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90880095f0 con 0x7f9094100fd0 2024-03-29T04:41:42.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.949+0000 7f90937fe700 1 --2- 172.21.15.142:0/3856324606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9094100620 0x7f90941017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:41:42.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.949+0000 7f9092ffd700 1 --2- 172.21.15.142:0/3856324606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9094100fd0 0x7f9094101d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f90880092e0 tx=0x7f908800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:42.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.949+0000 7f9090ff9700 1 -- 172.21.15.142:0/3856324606 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f908801d070 con 0x7f9094100fd0 2024-03-29T04:41:42.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.949+0000 7f9099d88700 1 -- 172.21.15.142:0/3856324606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9094102600 con 0x7f9094100fd0 2024-03-29T04:41:42.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.949+0000 7f9090ff9700 1 -- 172.21.15.142:0/3856324606 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9088022470 con 0x7f9094100fd0 2024-03-29T04:41:42.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.949+0000 7f9090ff9700 1 -- 172.21.15.142:0/3856324606 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f908800f690 con 0x7f9094100fd0 2024-03-29T04:41:42.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.949+0000 7f9099d88700 1 -- 172.21.15.142:0/3856324606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90941a2cb0 con 0x7f9094100fd0 2024-03-29T04:41:42.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.950+0000 7f9099d88700 1 -- 172.21.15.142:0/3856324606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90941090c0 con 0x7f9094100fd0 2024-03-29T04:41:42.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.951+0000 7f9090ff9700 1 -- 172.21.15.142:0/3856324606 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f90880225e0 con 0x7f9094100fd0 2024-03-29T04:41:42.949 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.951+0000 7f9090ff9700 1 --2- 172.21.15.142:0/3856324606 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f907c06c4e0 0x7f907c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:42.949 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.951+0000 7f9090ff9700 1 -- 172.21.15.142:0/3856324606 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f908808cb00 con 0x7f9094100fd0 2024-03-29T04:41:42.949 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.951+0000 7f90937fe700 1 --2- 172.21.15.142:0/3856324606 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f907c06c4e0 0x7f907c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:42.949 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.951+0000 7f90937fe700 1 --2- 172.21.15.142:0/3856324606 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f907c06c4e0 0x7f907c06e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9084007900 tx=0x7f9084008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:42.951 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:42.955+0000 7f9090ff9700 1 -- 172.21.15.142:0/3856324606 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f90880576d0 con 0x7f9094100fd0 2024-03-29T04:41:43.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:42 smithi152 ceph-mon[37177]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:43.135 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.139+0000 7f9099d88700 1 -- 172.21.15.142:0/3856324606 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f909402cd30 con 0x7f907c06c4e0 2024-03-29T04:41:43.136 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.140+0000 7f9090ff9700 1 -- 172.21.15.142:0/3856324606 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28119 (secure 0 0 0) 0x7f909402cd30 con 0x7f907c06c4e0 2024-03-29T04:41:43.136 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:41:43.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.142+0000 7f9099d88700 1 -- 172.21.15.142:0/3856324606 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f907c06c4e0 msgr2=0x7f907c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:43.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.142+0000 7f9099d88700 1 --2- 172.21.15.142:0/3856324606 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f907c06c4e0 0x7f907c06e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9084007900 tx=0x7f9084008040 comp rx=0 tx=0).stop 2024-03-29T04:41:43.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.142+0000 7f9099d88700 1 -- 172.21.15.142:0/3856324606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9094100fd0 msgr2=0x7f9094101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:43.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.142+0000 7f9099d88700 1 --2- 172.21.15.142:0/3856324606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9094100fd0 0x7f9094101d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f90880092e0 tx=0x7f908800be20 comp rx=0 tx=0).stop 2024-03-29T04:41:43.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.142+0000 7f9099d88700 1 -- 172.21.15.142:0/3856324606 shutdown_connections 2024-03-29T04:41:43.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.142+0000 7f9099d88700 1 --2- 172.21.15.142:0/3856324606 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f907c06c4e0 0x7f907c06e9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:43.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.142+0000 7f9099d88700 1 --2- 172.21.15.142:0/3856324606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9094100620 0x7f90941017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:43.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.142+0000 7f9099d88700 1 --2- 172.21.15.142:0/3856324606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9094100fd0 0x7f9094101d00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:43.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.142+0000 7f9099d88700 1 -- 172.21.15.142:0/3856324606 >> 172.21.15.142:0/3856324606 conn(0x7f9094074e00 msgr2=0x7f9094103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:43.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.143+0000 7f9099d88700 1 -- 172.21.15.142:0/3856324606 shutdown_connections 2024-03-29T04:41:43.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:43.143+0000 7f9099d88700 1 -- 172.21.15.142:0/3856324606 wait complete. 2024-03-29T04:41:43.140 INFO:teuthology.orchestra.run.smithi142.stderr:dumped all 2024-03-29T04:41:43.778 INFO:teuthology.orchestra.run.smithi142.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-03-29T04:41:41.627473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002290"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":185,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T04:40:46.630015+0000","last_change":"2024-03-29T04:40:19.680217+0000","last_active":"2024-03-29T04:40:46.630015+0000","last_peered":"2024-03-29T04:40:46.630015+0000","last_clean":"2024-03-29T04:40:46.630015+0000","last_became_active":"2024-03-29T04:40:19.679928+0000","last_became_peered":"2024-03-29T04:40:19.679928+0000","last_unstale":"2024-03-29T04:40:46.630015+0000","last_undegraded":"2024-03-29T04:40:46.630015+0000","last_fullsized":"2024-03-29T04:40:46.630015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:39:31.941952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:39:31.941952+0000","last_clean_scrub_stamp":"2024-03-29T04:39:31.941952+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:56:45.534919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]},{"osd":4,"up_from":35,"seq":150323855380,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 04:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.42499999999999999}]},{"osd":1,"last update":"Fri Mar 29 04:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.46899999999999997}]},{"osd":2,"last update":"Fri Mar 29 04:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.37},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.48799999999999999}]},{"osd":3,"last update":"Fri Mar 29 04:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.39100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 04:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Mar 29 04:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.47899999999999998}]},{"osd":2,"last update":"Fri Mar 29 04:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.505},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Mar 29 04:41:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.432}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 04:41:34 2024","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.32400000000000001,"15min":0.32900000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.44400000000000001,"5min":0.497,"15min":0.497},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.33600000000000002,"15min":0.34300000000000003},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.48999999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Mar 29 04:41:34 2024","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35699999999999998,"15min":0.36499999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51200000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.308,"5min":0.34499999999999997,"15min":0.35099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.47799999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.33600000000000002}]},{"osd":3,"last update":"Fri Mar 29 04:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Mar 29 04:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 04:41:07 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33400000000000002,"15min":0.33300000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.51300000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33800000000000002,"15min":0.33500000000000002},"min":{"1min":0.248,"5min":0.217,"15min":0.217},"max":{"1min":0.497,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.28199999999999997}]},{"osd":2,"last update":"Fri Mar 29 04:41:33 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.433,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.374,"15min":0.376},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.30599999999999999}]},{"osd":3,"last update":"Fri Mar 29 04:40:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":18.658000000000001,"5min":18.658000000000001,"15min":18.658000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Fri Mar 29 04:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 04:41:04 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35399999999999998,"15min":0.35599999999999998},"min":{"1min":0.25900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.432,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.35199999999999998},"min":{"1min":0.25900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.47099999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.245}]},{"osd":2,"last update":"Fri Mar 29 04:41:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.35399999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.46600000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.36099999999999999,"15min":0.36299999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.28199999999999997}]},{"osd":3,"last update":"Fri Mar 29 04:40:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.04,"5min":3.04,"15min":3.04},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Mar 29 04:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:41:43.779 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-29T04:41:43.779 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-29T04:41:43.779 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-29T04:41:43.779 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-29T04:41:43.983 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:44.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:44 smithi152 ceph-mon[37177]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-29T04:41:44.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:44 smithi152 ceph-mon[37177]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:44.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:44 smithi142 ceph-mon[29027]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-29T04:41:44.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:44 smithi142 ceph-mon[29027]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:45.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.533+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/192617986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd880ffbd0 msgr2=0x7fbd88100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:45.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.533+0000 7fbd8ed4b700 1 --2- 172.21.15.142:0/192617986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd880ffbd0 0x7fbd88100050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009910 tx=0x7fbd7c009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:45.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.533+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/192617986 shutdown_connections 2024-03-29T04:41:45.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.533+0000 7fbd8ed4b700 1 --2- 172.21.15.142:0/192617986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd880ffbd0 0x7fbd88100050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:45.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.533+0000 7fbd8ed4b700 1 --2- 172.21.15.142:0/192617986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd88105a40 0x7fbd88105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:45.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.533+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/192617986 >> 172.21.15.142:0/192617986 conn(0x7fbd880755a0 msgr2=0x7fbd880759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:45.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.533+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/192617986 shutdown_connections 2024-03-29T04:41:45.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.533+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/192617986 wait complete. 2024-03-29T04:41:45.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd8ed4b700 1 Processor -- start 2024-03-29T04:41:45.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd8ed4b700 1 -- start start 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd8ed4b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd880ffbd0 0x7fbd8819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd8ed4b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd88105a40 0x7fbd8819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd8ed4b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8819d140 con 0x7fbd88105a40 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd8ed4b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd881965f0 con 0x7fbd880ffbd0 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd87fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd88105a40 0x7fbd8819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd87fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd88105a40 0x7fbd8819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35148/0 (socket says 172.21.15.142:35148) 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd87fff700 1 -- 172.21.15.142:0/2373554490 learned_addr learned my addr 172.21.15.142:0/2373554490 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd8cae7700 1 --2- 172.21.15.142:0/2373554490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd880ffbd0 0x7fbd8819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd87fff700 1 -- 172.21.15.142:0/2373554490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd880ffbd0 msgr2=0x7fbd8819c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd87fff700 1 --2- 172.21.15.142:0/2373554490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd880ffbd0 0x7fbd8819c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.534+0000 7fbd87fff700 1 -- 172.21.15.142:0/2373554490 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0095f0 con 0x7fbd88105a40 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.535+0000 7fbd87fff700 1 --2- 172.21.15.142:0/2373554490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd88105a40 0x7fbd8819cab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c006010 tx=0x7fbd7c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.535+0000 7fbd85ffb700 1 -- 172.21.15.142:0/2373554490 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd7c01d070 con 0x7fbd88105a40 2024-03-29T04:41:45.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.535+0000 7fbd85ffb700 1 -- 172.21.15.142:0/2373554490 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbd7c00f460 con 0x7fbd88105a40 2024-03-29T04:41:45.533 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.535+0000 7fbd85ffb700 1 -- 172.21.15.142:0/2373554490 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd7c021640 con 0x7fbd88105a40 2024-03-29T04:41:45.533 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.535+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/2373554490 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd88196870 con 0x7fbd88105a40 2024-03-29T04:41:45.533 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.535+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/2373554490 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd88196d10 con 0x7fbd88105a40 2024-03-29T04:41:45.533 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.536+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/2373554490 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd88109070 con 0x7fbd88105a40 2024-03-29T04:41:45.534 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.536+0000 7fbd85ffb700 1 -- 172.21.15.142:0/2373554490 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbd7c00faf0 con 0x7fbd88105a40 2024-03-29T04:41:45.534 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.537+0000 7fbd85ffb700 1 --2- 172.21.15.142:0/2373554490 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbd7006c4e0 0x7fbd7006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:45.534 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.537+0000 7fbd85ffb700 1 -- 172.21.15.142:0/2373554490 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbd7c08c870 con 0x7fbd88105a40 2024-03-29T04:41:45.534 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.537+0000 7fbd8cae7700 1 --2- 172.21.15.142:0/2373554490 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbd7006c4e0 0x7fbd7006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:45.534 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.537+0000 7fbd8cae7700 1 --2- 172.21.15.142:0/2373554490 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbd7006c4e0 0x7fbd7006e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd80007dc0 tx=0x7fbd800072f0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:45.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.541+0000 7fbd85ffb700 1 -- 172.21.15.142:0/2373554490 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbd7c05ae20 con 0x7fbd88105a40 2024-03-29T04:41:45.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.772+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/2373554490 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbd8804ea90 con 0x7fbd88105a40 2024-03-29T04:41:45.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.773+0000 7fbd85ffb700 1 -- 172.21.15.142:0/2373554490 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fbd7c05a9b0 con 0x7fbd88105a40 2024-03-29T04:41:45.769 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:41:45.769 INFO:teuthology.orchestra.run.smithi142.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-29T04:41:45.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.775+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/2373554490 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbd7006c4e0 msgr2=0x7fbd7006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:45.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.775+0000 7fbd8ed4b700 1 --2- 172.21.15.142:0/2373554490 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbd7006c4e0 0x7fbd7006e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd80007dc0 tx=0x7fbd800072f0 comp rx=0 tx=0).stop 2024-03-29T04:41:45.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.775+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/2373554490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd88105a40 msgr2=0x7fbd8819cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:45.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.775+0000 7fbd8ed4b700 1 --2- 172.21.15.142:0/2373554490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd88105a40 0x7fbd8819cab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c006010 tx=0x7fbd7c00be20 comp rx=0 tx=0).stop 2024-03-29T04:41:45.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.775+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/2373554490 shutdown_connections 2024-03-29T04:41:45.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.775+0000 7fbd8ed4b700 1 --2- 172.21.15.142:0/2373554490 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbd7006c4e0 0x7fbd7006e9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:45.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.775+0000 7fbd8ed4b700 1 --2- 172.21.15.142:0/2373554490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd880ffbd0 0x7fbd8819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:45.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.775+0000 7fbd8ed4b700 1 --2- 172.21.15.142:0/2373554490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd88105a40 0x7fbd8819cab0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:45.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.775+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/2373554490 >> 172.21.15.142:0/2373554490 conn(0x7fbd880755a0 msgr2=0x7fbd88068610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:45.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.775+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/2373554490 shutdown_connections 2024-03-29T04:41:45.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:45.775+0000 7fbd8ed4b700 1 -- 172.21.15.142:0/2373554490 wait complete. 2024-03-29T04:41:46.475 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-29T04:41:46.475 INFO:tasks.cephadm:Setup complete, yielding 2024-03-29T04:41:46.475 INFO:teuthology.run_tasks:Running task print... 2024-03-29T04:41:46.484 INFO:teuthology.task.print:**** done end installing v18.2.1 cephadm ... 2024-03-29T04:41:46.485 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T04:41:46.495 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T04:41:46.496 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-29T04:41:46.700 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:46.819 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:46 smithi142 ceph-mon[29027]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:46.820 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:46 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2373554490' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-29T04:41:47.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:46 smithi152 ceph-mon[37177]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:47.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:46 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2373554490' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-29T04:41:47.824 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:47 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:41:48.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:47 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:41:48.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.372+0000 7f2118acb700 1 -- 172.21.15.142:0/3253864917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2114100bb0 msgr2=0x7f2114100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:48.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.372+0000 7f2118acb700 1 --2- 172.21.15.142:0/3253864917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2114100bb0 0x7f2114100fd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2108009910 tx=0x7f2108009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:48.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.372+0000 7f2118acb700 1 -- 172.21.15.142:0/3253864917 shutdown_connections 2024-03-29T04:41:48.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.372+0000 7f2118acb700 1 --2- 172.21.15.142:0/3253864917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2114101db0 0x7f2114102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:48.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.372+0000 7f2118acb700 1 --2- 172.21.15.142:0/3253864917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2114100bb0 0x7f2114100fd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:48.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.372+0000 7f2118acb700 1 -- 172.21.15.142:0/3253864917 >> 172.21.15.142:0/3253864917 conn(0x7f21140fc110 msgr2=0x7f21140fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:48.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.372+0000 7f2118acb700 1 -- 172.21.15.142:0/3253864917 shutdown_connections 2024-03-29T04:41:48.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.372+0000 7f2118acb700 1 -- 172.21.15.142:0/3253864917 wait complete. 2024-03-29T04:41:48.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.373+0000 7f2118acb700 1 Processor -- start 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.373+0000 7f2118acb700 1 -- start start 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.373+0000 7f2118acb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2114100bb0 0x7f2114196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.373+0000 7f2118acb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2114101db0 0x7f21141968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.373+0000 7f2118acb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2114196ef0 con 0x7f2114101db0 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.373+0000 7f2118acb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2114197030 con 0x7f2114100bb0 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.373+0000 7f2111d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2114101db0 0x7f21141968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.373+0000 7f2111d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2114101db0 0x7f21141968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35170/0 (socket says 172.21.15.142:35170) 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.373+0000 7f2111d9b700 1 -- 172.21.15.142:0/1942900276 learned_addr learned my addr 172.21.15.142:0/1942900276 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.373+0000 7f211259c700 1 --2- 172.21.15.142:0/1942900276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2114100bb0 0x7f2114196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.374+0000 7f2111d9b700 1 -- 172.21.15.142:0/1942900276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2114100bb0 msgr2=0x7f2114196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.374+0000 7f2111d9b700 1 --2- 172.21.15.142:0/1942900276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2114100bb0 0x7f2114196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.374+0000 7f2111d9b700 1 -- 172.21.15.142:0/1942900276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21080095f0 con 0x7f2114101db0 2024-03-29T04:41:48.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.374+0000 7f2111d9b700 1 --2- 172.21.15.142:0/1942900276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2114101db0 0x7f21141968d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f210400c8a0 tx=0x7f210400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:48.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.374+0000 7f21037fe700 1 -- 172.21.15.142:0/1942900276 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21040078e0 con 0x7f2114101db0 2024-03-29T04:41:48.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.374+0000 7f2118acb700 1 -- 172.21.15.142:0/1942900276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211419bae0 con 0x7f2114101db0 2024-03-29T04:41:48.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.374+0000 7f21037fe700 1 -- 172.21.15.142:0/1942900276 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f210400f450 con 0x7f2114101db0 2024-03-29T04:41:48.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.374+0000 7f21037fe700 1 -- 172.21.15.142:0/1942900276 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f210400e5e0 con 0x7f2114101db0 2024-03-29T04:41:48.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.374+0000 7f2118acb700 1 -- 172.21.15.142:0/1942900276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211419c030 con 0x7f2114101db0 2024-03-29T04:41:48.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.375+0000 7f2118acb700 1 -- 172.21.15.142:0/1942900276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2114109710 con 0x7f2114101db0 2024-03-29T04:41:48.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.375+0000 7f21037fe700 1 -- 172.21.15.142:0/1942900276 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2104007a40 con 0x7f2114101db0 2024-03-29T04:41:48.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.376+0000 7f21037fe700 1 --2- 172.21.15.142:0/1942900276 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f20fc06c4e0 0x7f20fc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:48.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.376+0000 7f21037fe700 1 -- 172.21.15.142:0/1942900276 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f210408b4b0 con 0x7f2114101db0 2024-03-29T04:41:48.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.376+0000 7f211259c700 1 --2- 172.21.15.142:0/1942900276 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f20fc06c4e0 0x7f20fc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:48.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.376+0000 7f211259c700 1 --2- 172.21.15.142:0/1942900276 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f20fc06c4e0 0x7f20fc06e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2108006010 tx=0x7f210800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:48.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.380+0000 7f21037fe700 1 -- 172.21.15.142:0/1942900276 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2104059a60 con 0x7f2114101db0 2024-03-29T04:41:48.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.570+0000 7f2118acb700 1 -- 172.21.15.142:0/1942900276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f2114109900 con 0x7f2114101db0 2024-03-29T04:41:48.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.576+0000 7f21037fe700 1 -- 172.21.15.142:0/1942900276 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f21040595f0 con 0x7f2114101db0 2024-03-29T04:41:48.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.578+0000 7f2118acb700 1 -- 172.21.15.142:0/1942900276 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f20fc06c4e0 msgr2=0x7f20fc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:48.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.578+0000 7f2118acb700 1 --2- 172.21.15.142:0/1942900276 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f20fc06c4e0 0x7f20fc06e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2108006010 tx=0x7f210800b540 comp rx=0 tx=0).stop 2024-03-29T04:41:48.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.578+0000 7f2118acb700 1 -- 172.21.15.142:0/1942900276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2114101db0 msgr2=0x7f21141968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:48.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.578+0000 7f2118acb700 1 --2- 172.21.15.142:0/1942900276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2114101db0 0x7f21141968d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f210400c8a0 tx=0x7f210400cbb0 comp rx=0 tx=0).stop 2024-03-29T04:41:48.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.578+0000 7f2118acb700 1 -- 172.21.15.142:0/1942900276 shutdown_connections 2024-03-29T04:41:48.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.578+0000 7f2118acb700 1 --2- 172.21.15.142:0/1942900276 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f20fc06c4e0 0x7f20fc06e9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:48.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.578+0000 7f2118acb700 1 --2- 172.21.15.142:0/1942900276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2114100bb0 0x7f2114196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:48.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.578+0000 7f2118acb700 1 --2- 172.21.15.142:0/1942900276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2114101db0 0x7f21141968d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:48.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.578+0000 7f2118acb700 1 -- 172.21.15.142:0/1942900276 >> 172.21.15.142:0/1942900276 conn(0x7f21140fc110 msgr2=0x7f2114104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:48.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.578+0000 7f2118acb700 1 -- 172.21.15.142:0/1942900276 shutdown_connections 2024-03-29T04:41:48.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:48.578+0000 7f2118acb700 1 -- 172.21.15.142:0/1942900276 wait complete. 2024-03-29T04:41:48.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:48 smithi152 ceph-mon[37177]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:48.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:48 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1942900276' entity='client.admin' 2024-03-29T04:41:48.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:41:48.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:48 smithi142 ceph-mon[29027]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:48.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:48 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1942900276' entity='client.admin' 2024-03-29T04:41:48.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:41:49.246 INFO:teuthology.run_tasks:Running task print... 2024-03-29T04:41:49.255 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-29T04:41:49.255 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T04:41:49.264 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T04:41:49.264 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph orch status' 2024-03-29T04:41:49.475 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:49.836 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:41:49.836 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:41:49.836 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:49 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:41:50.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:41:50.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:41:50.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:49 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:41:50.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:50 smithi142 ceph-mon[29027]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:51.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:50 smithi152 ceph-mon[37177]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:51.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.046+0000 7fc9be873700 1 -- 172.21.15.142:0/1320778780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b80ffab0 msgr2=0x7fc9b80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:51.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.046+0000 7fc9be873700 1 --2- 172.21.15.142:0/1320778780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b80ffab0 0x7fc9b80ffed0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009910 tx=0x7fc9a0009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:51.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.046+0000 7fc9be873700 1 -- 172.21.15.142:0/1320778780 shutdown_connections 2024-03-29T04:41:51.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.046+0000 7fc9be873700 1 --2- 172.21.15.142:0/1320778780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9b8100cb0 0x7fc9b8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:51.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.046+0000 7fc9be873700 1 --2- 172.21.15.142:0/1320778780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b80ffab0 0x7fc9b80ffed0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:51.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.046+0000 7fc9be873700 1 -- 172.21.15.142:0/1320778780 >> 172.21.15.142:0/1320778780 conn(0x7fc9b80fb030 msgr2=0x7fc9b80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:51.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.046+0000 7fc9be873700 1 -- 172.21.15.142:0/1320778780 shutdown_connections 2024-03-29T04:41:51.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.046+0000 7fc9be873700 1 -- 172.21.15.142:0/1320778780 wait complete. 2024-03-29T04:41:51.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.046+0000 7fc9be873700 1 Processor -- start 2024-03-29T04:41:51.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9be873700 1 -- start start 2024-03-29T04:41:51.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9be873700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9b80ffab0 0x7fc9b81952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:51.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9be873700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8100cb0 0x7fc9b8195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:51.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9be873700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b8195e40 con 0x7fc9b8100cb0 2024-03-29T04:41:51.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9be873700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b8195f80 con 0x7fc9b80ffab0 2024-03-29T04:41:51.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9b77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8100cb0 0x7fc9b8195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:51.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9b77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8100cb0 0x7fc9b8195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53160/0 (socket says 172.21.15.142:53160) 2024-03-29T04:41:51.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9b77fe700 1 -- 172.21.15.142:0/2554172034 learned_addr learned my addr 172.21.15.142:0/2554172034 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:51.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9b7fff700 1 --2- 172.21.15.142:0/2554172034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9b80ffab0 0x7fc9b81952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:51.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9b77fe700 1 -- 172.21.15.142:0/2554172034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9b80ffab0 msgr2=0x7fc9b81952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:51.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9b77fe700 1 --2- 172.21.15.142:0/2554172034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9b80ffab0 0x7fc9b81952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:51.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9b77fe700 1 -- 172.21.15.142:0/2554172034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a00095f0 con 0x7fc9b8100cb0 2024-03-29T04:41:51.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.047+0000 7fc9b77fe700 1 --2- 172.21.15.142:0/2554172034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8100cb0 0x7fc9b8195820 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800b4b0 tx=0x7fc9a800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:51.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.048+0000 7fc9b57fa700 1 -- 172.21.15.142:0/2554172034 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9a8011800 con 0x7fc9b8100cb0 2024-03-29T04:41:51.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.048+0000 7fc9b57fa700 1 -- 172.21.15.142:0/2554172034 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc9a8011e10 con 0x7fc9b8100cb0 2024-03-29T04:41:51.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.048+0000 7fc9be873700 1 -- 172.21.15.142:0/2554172034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b819aa30 con 0x7fc9b8100cb0 2024-03-29T04:41:51.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.048+0000 7fc9b57fa700 1 -- 172.21.15.142:0/2554172034 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9a800f2e0 con 0x7fc9b8100cb0 2024-03-29T04:41:51.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.048+0000 7fc9be873700 1 -- 172.21.15.142:0/2554172034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b819b000 con 0x7fc9b8100cb0 2024-03-29T04:41:51.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.049+0000 7fc9be873700 1 -- 172.21.15.142:0/2554172034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b8108690 con 0x7fc9b8100cb0 2024-03-29T04:41:51.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.049+0000 7fc9b57fa700 1 -- 172.21.15.142:0/2554172034 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc9a8011960 con 0x7fc9b8100cb0 2024-03-29T04:41:51.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.049+0000 7fc9b57fa700 1 --2- 172.21.15.142:0/2554172034 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc9a406c4e0 0x7fc9a406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:51.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.050+0000 7fc9b7fff700 1 --2- 172.21.15.142:0/2554172034 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc9a406c4e0 0x7fc9a406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:51.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.050+0000 7fc9b57fa700 1 -- 172.21.15.142:0/2554172034 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc9a808b010 con 0x7fc9b8100cb0 2024-03-29T04:41:51.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.050+0000 7fc9b7fff700 1 --2- 172.21.15.142:0/2554172034 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc9a406c4e0 0x7fc9a406e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0006010 tx=0x7fc9a0005280 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:51.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.054+0000 7fc9b57fa700 1 -- 172.21.15.142:0/2554172034 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc9a8059640 con 0x7fc9b8100cb0 2024-03-29T04:41:51.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.260+0000 7fc9be873700 1 -- 172.21.15.142:0/2554172034 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9b8105600 con 0x7fc9a406c4e0 2024-03-29T04:41:51.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.261+0000 7fc9b57fa700 1 -- 172.21.15.142:0/2554172034 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc9b8105600 con 0x7fc9a406c4e0 2024-03-29T04:41:51.257 INFO:teuthology.orchestra.run.smithi142.stdout:Backend: cephadm 2024-03-29T04:41:51.257 INFO:teuthology.orchestra.run.smithi142.stdout:Available: Yes 2024-03-29T04:41:51.257 INFO:teuthology.orchestra.run.smithi142.stdout:Paused: No 2024-03-29T04:41:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.263+0000 7fc9be873700 1 -- 172.21.15.142:0/2554172034 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc9a406c4e0 msgr2=0x7fc9a406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.263+0000 7fc9be873700 1 --2- 172.21.15.142:0/2554172034 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc9a406c4e0 0x7fc9a406e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0006010 tx=0x7fc9a0005280 comp rx=0 tx=0).stop 2024-03-29T04:41:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.263+0000 7fc9be873700 1 -- 172.21.15.142:0/2554172034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8100cb0 msgr2=0x7fc9b8195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.263+0000 7fc9be873700 1 --2- 172.21.15.142:0/2554172034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8100cb0 0x7fc9b8195820 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800b4b0 tx=0x7fc9a800b7c0 comp rx=0 tx=0).stop 2024-03-29T04:41:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.263+0000 7fc9be873700 1 -- 172.21.15.142:0/2554172034 shutdown_connections 2024-03-29T04:41:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.263+0000 7fc9be873700 1 --2- 172.21.15.142:0/2554172034 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc9a406c4e0 0x7fc9a406e9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.263+0000 7fc9be873700 1 --2- 172.21.15.142:0/2554172034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9b80ffab0 0x7fc9b81952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.263+0000 7fc9be873700 1 --2- 172.21.15.142:0/2554172034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9b8100cb0 0x7fc9b8195820 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.263+0000 7fc9be873700 1 -- 172.21.15.142:0/2554172034 >> 172.21.15.142:0/2554172034 conn(0x7fc9b80fb030 msgr2=0x7fc9b8103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.263+0000 7fc9be873700 1 -- 172.21.15.142:0/2554172034 shutdown_connections 2024-03-29T04:41:51.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:51.263+0000 7fc9be873700 1 -- 172.21.15.142:0/2554172034 wait complete. 2024-03-29T04:41:51.961 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-29T04:41:52.170 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:52.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:52 smithi142 ceph-mon[29027]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:41:52.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:52 smithi142 ceph-mon[29027]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:53.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:52 smithi152 ceph-mon[37177]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:41:53.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:52 smithi152 ceph-mon[37177]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:53.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.895+0000 7fc81590e700 1 -- 172.21.15.142:0/402122875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc810101db0 msgr2=0x7fc810102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:53.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.895+0000 7fc81590e700 1 --2- 172.21.15.142:0/402122875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc810101db0 0x7fc810102210 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc800009910 tx=0x7fc800009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:53.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.895+0000 7fc81590e700 1 -- 172.21.15.142:0/402122875 shutdown_connections 2024-03-29T04:41:53.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.895+0000 7fc81590e700 1 --2- 172.21.15.142:0/402122875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc810101db0 0x7fc810102210 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:53.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.895+0000 7fc81590e700 1 --2- 172.21.15.142:0/402122875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc810100bb0 0x7fc810100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:53.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.895+0000 7fc81590e700 1 -- 172.21.15.142:0/402122875 >> 172.21.15.142:0/402122875 conn(0x7fc8100fc110 msgr2=0x7fc8100fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:53.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.895+0000 7fc81590e700 1 -- 172.21.15.142:0/402122875 shutdown_connections 2024-03-29T04:41:53.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.895+0000 7fc81590e700 1 -- 172.21.15.142:0/402122875 wait complete. 2024-03-29T04:41:53.898 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.895+0000 7fc81590e700 1 Processor -- start 2024-03-29T04:41:53.898 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc81590e700 1 -- start start 2024-03-29T04:41:53.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc81590e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc810100bb0 0x7fc810196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:53.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc81590e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc810101db0 0x7fc8101968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:53.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc81590e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810196ec0 con 0x7fc810101db0 2024-03-29T04:41:53.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc81590e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810197000 con 0x7fc810100bb0 2024-03-29T04:41:53.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc80e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc810101db0 0x7fc8101968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:53.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc80e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc810101db0 0x7fc8101968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53176/0 (socket says 172.21.15.142:53176) 2024-03-29T04:41:53.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc80e7fc700 1 -- 172.21.15.142:0/3754534704 learned_addr learned my addr 172.21.15.142:0/3754534704 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:53.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc80e7fc700 1 -- 172.21.15.142:0/3754534704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc810100bb0 msgr2=0x7fc810196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:53.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc80e7fc700 1 --2- 172.21.15.142:0/3754534704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc810100bb0 0x7fc810196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:53.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc80e7fc700 1 -- 172.21.15.142:0/3754534704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8000095f0 con 0x7fc810101db0 2024-03-29T04:41:53.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc80e7fc700 1 --2- 172.21.15.142:0/3754534704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc810101db0 0x7fc8101968a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc800006010 tx=0x7fc80000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:53.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc81490c700 1 -- 172.21.15.142:0/3754534704 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc80001d070 con 0x7fc810101db0 2024-03-29T04:41:53.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc81490c700 1 -- 172.21.15.142:0/3754534704 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc80000f460 con 0x7fc810101db0 2024-03-29T04:41:53.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.896+0000 7fc81590e700 1 -- 172.21.15.142:0/3754534704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81019ba50 con 0x7fc810101db0 2024-03-29T04:41:53.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.897+0000 7fc81490c700 1 -- 172.21.15.142:0/3754534704 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc800017630 con 0x7fc810101db0 2024-03-29T04:41:53.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.897+0000 7fc81590e700 1 -- 172.21.15.142:0/3754534704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81019bf40 con 0x7fc810101db0 2024-03-29T04:41:53.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.898+0000 7fc81590e700 1 -- 172.21.15.142:0/3754534704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc810109710 con 0x7fc810101db0 2024-03-29T04:41:53.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.898+0000 7fc81490c700 1 -- 172.21.15.142:0/3754534704 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc800003a60 con 0x7fc810101db0 2024-03-29T04:41:53.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.898+0000 7fc81490c700 1 --2- 172.21.15.142:0/3754534704 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc7fc06c4e0 0x7fc7fc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:53.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.898+0000 7fc81490c700 1 -- 172.21.15.142:0/3754534704 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc80008c950 con 0x7fc810101db0 2024-03-29T04:41:53.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.898+0000 7fc80effd700 1 --2- 172.21.15.142:0/3754534704 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc7fc06c4e0 0x7fc7fc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:53.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.899+0000 7fc80effd700 1 --2- 172.21.15.142:0/3754534704 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc7fc06c4e0 0x7fc7fc06e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc810101c10 tx=0x7fc7f8009470 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:53.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:53.903+0000 7fc81490c700 1 -- 172.21.15.142:0/3754534704 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc80005af80 con 0x7fc810101db0 2024-03-29T04:41:54.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.103+0000 7fc81590e700 1 -- 172.21.15.142:0/3754534704 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc810106700 con 0x7fc7fc06c4e0 2024-03-29T04:41:54.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.112+0000 7fc81490c700 1 -- 172.21.15.142:0/3754534704 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2860 (secure 0 0 0) 0x7fc810106700 con 0x7fc7fc06c4e0 2024-03-29T04:41:54.108 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:41:54.109 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (3m) 2m ago 5m 26.0M - 0.25.0 c8568f914cd2 d4a3e12f4047 2024-03-29T04:41:54.109 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (5m) 2m ago 5m 7033k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:41:54.109 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (4m) 69s ago 4m 6962k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:41:54.109 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (5m) 2m ago 5m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:41:54.109 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (4m) 69s ago 4m 6660k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:41:54.109 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (3m) 2m ago 4m 79.7M - 9.4.7 954c08fa6188 aba58e552b7c 2024-03-29T04:41:54.109 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:9283,8765,8443 running (6m) 2m ago 6m 481M - 18.2.1 5be31c24972a a5f21a9cfe24 2024-03-29T04:41:54.109 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (4m) 69s ago 4m 440M - 18.2.1 5be31c24972a c6124d88151c 2024-03-29T04:41:54.109 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (6m) 2m ago 6m 44.6M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:41:54.109 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (4m) 69s ago 4m 41.0M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:41:54.109 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (5m) 2m ago 5m 15.8M - 1.5.0 0da6a335fe13 91b95c929db2 2024-03-29T04:41:54.109 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (4m) 69s ago 4m 15.0M - 1.5.0 0da6a335fe13 97e90e5b337b 2024-03-29T04:41:54.110 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (3m) 2m ago 3m 40.5M 3946M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:41:54.110 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (2m) 2m ago 2m 45.1M 3946M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:41:54.110 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (2m) 2m ago 2m 41.6M 3946M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:41:54.110 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (2m) 69s ago 2m 41.1M 4970M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:41:54.110 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (101s) 69s ago 101s 39.1M 4970M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:41:54.110 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (74s) 69s ago 74s 11.5M 4970M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:41:54.110 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (3m) 2m ago 4m 38.1M - 2.43.0 a07b618ecd1d c6c39d3c30f6 2024-03-29T04:41:54.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.114+0000 7fc81590e700 1 -- 172.21.15.142:0/3754534704 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc7fc06c4e0 msgr2=0x7fc7fc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:54.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.114+0000 7fc81590e700 1 --2- 172.21.15.142:0/3754534704 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc7fc06c4e0 0x7fc7fc06e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc810101c10 tx=0x7fc7f8009470 comp rx=0 tx=0).stop 2024-03-29T04:41:54.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.114+0000 7fc81590e700 1 -- 172.21.15.142:0/3754534704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc810101db0 msgr2=0x7fc8101968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:54.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.114+0000 7fc81590e700 1 --2- 172.21.15.142:0/3754534704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc810101db0 0x7fc8101968a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc800006010 tx=0x7fc80000be20 comp rx=0 tx=0).stop 2024-03-29T04:41:54.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.114+0000 7fc81590e700 1 -- 172.21.15.142:0/3754534704 shutdown_connections 2024-03-29T04:41:54.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.114+0000 7fc81590e700 1 --2- 172.21.15.142:0/3754534704 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc7fc06c4e0 0x7fc7fc06e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:54.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.114+0000 7fc81590e700 1 --2- 172.21.15.142:0/3754534704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc810100bb0 0x7fc810196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:54.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.114+0000 7fc81590e700 1 --2- 172.21.15.142:0/3754534704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc810101db0 0x7fc8101968a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:54.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.114+0000 7fc81590e700 1 -- 172.21.15.142:0/3754534704 >> 172.21.15.142:0/3754534704 conn(0x7fc8100fc110 msgr2=0x7fc810104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:54.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.114+0000 7fc81590e700 1 -- 172.21.15.142:0/3754534704 shutdown_connections 2024-03-29T04:41:54.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:54.114+0000 7fc81590e700 1 -- 172.21.15.142:0/3754534704 wait complete. 2024-03-29T04:41:54.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:54 smithi152 ceph-mon[37177]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:54.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:54 smithi142 ceph-mon[29027]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:54.890 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-29T04:41:55.093 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:55.530 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:55 smithi142 ceph-mon[29027]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:41:55.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:55 smithi152 ceph-mon[37177]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:41:56.526 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:56 smithi142 ceph-mon[29027]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:56.786 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:56 smithi152 ceph-mon[37177]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:56.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.866+0000 7feb471bf700 1 -- 172.21.15.142:0/114459902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb400ffae0 msgr2=0x7feb400fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:56.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.866+0000 7feb471bf700 1 --2- 172.21.15.142:0/114459902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb400ffae0 0x7feb400fff00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feb34009910 tx=0x7feb34009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.866+0000 7feb471bf700 1 -- 172.21.15.142:0/114459902 shutdown_connections 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.866+0000 7feb471bf700 1 --2- 172.21.15.142:0/114459902 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb40100ce0 0x7feb40101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.866+0000 7feb471bf700 1 --2- 172.21.15.142:0/114459902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb400ffae0 0x7feb400fff00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.866+0000 7feb471bf700 1 -- 172.21.15.142:0/114459902 >> 172.21.15.142:0/114459902 conn(0x7feb400fb080 msgr2=0x7feb400fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.866+0000 7feb471bf700 1 -- 172.21.15.142:0/114459902 shutdown_connections 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.866+0000 7feb471bf700 1 -- 172.21.15.142:0/114459902 wait complete. 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb471bf700 1 Processor -- start 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb471bf700 1 -- start start 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb471bf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb400ffae0 0x7feb40103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb471bf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb40100ce0 0x7feb401020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb471bf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40102600 con 0x7feb400ffae0 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb471bf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40102740 con 0x7feb40100ce0 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb44f5b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb400ffae0 0x7feb40103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:56.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb44f5b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb400ffae0 0x7feb40103a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53204/0 (socket says 172.21.15.142:53204) 2024-03-29T04:41:56.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb44f5b700 1 -- 172.21.15.142:0/1079997588 learned_addr learned my addr 172.21.15.142:0/1079997588 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:56.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb3ffff700 1 --2- 172.21.15.142:0/1079997588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb40100ce0 0x7feb401020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:56.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb44f5b700 1 -- 172.21.15.142:0/1079997588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb40100ce0 msgr2=0x7feb401020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:56.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb44f5b700 1 --2- 172.21.15.142:0/1079997588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb40100ce0 0x7feb401020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:56.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.867+0000 7feb44f5b700 1 -- 172.21.15.142:0/1079997588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb340095f0 con 0x7feb400ffae0 2024-03-29T04:41:56.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.868+0000 7feb44f5b700 1 --2- 172.21.15.142:0/1079997588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb400ffae0 0x7feb40103a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feb3400bda0 tx=0x7feb3400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:56.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.868+0000 7feb3dffb700 1 -- 172.21.15.142:0/1079997588 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb3401d070 con 0x7feb400ffae0 2024-03-29T04:41:56.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.868+0000 7feb3dffb700 1 -- 172.21.15.142:0/1079997588 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feb34022470 con 0x7feb400ffae0 2024-03-29T04:41:56.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.868+0000 7feb3dffb700 1 -- 172.21.15.142:0/1079997588 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb3400f650 con 0x7feb400ffae0 2024-03-29T04:41:56.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.868+0000 7feb471bf700 1 -- 172.21.15.142:0/1079997588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb401029c0 con 0x7feb400ffae0 2024-03-29T04:41:56.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.868+0000 7feb471bf700 1 -- 172.21.15.142:0/1079997588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb40102eb0 con 0x7feb400ffae0 2024-03-29T04:41:56.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.869+0000 7feb471bf700 1 -- 172.21.15.142:0/1079997588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb40108640 con 0x7feb400ffae0 2024-03-29T04:41:56.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.869+0000 7feb3dffb700 1 -- 172.21.15.142:0/1079997588 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7feb34004af0 con 0x7feb400ffae0 2024-03-29T04:41:56.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.870+0000 7feb3dffb700 1 --2- 172.21.15.142:0/1079997588 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7feb3006c4e0 0x7feb3006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:56.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.870+0000 7feb3dffb700 1 -- 172.21.15.142:0/1079997588 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7feb3408cb50 con 0x7feb400ffae0 2024-03-29T04:41:56.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.870+0000 7feb3ffff700 1 --2- 172.21.15.142:0/1079997588 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7feb3006c4e0 0x7feb3006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:56.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.870+0000 7feb3ffff700 1 --2- 172.21.15.142:0/1079997588 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7feb3006c4e0 0x7feb3006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb2c005950 tx=0x7feb2c00a300 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:56.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:56.874+0000 7feb3dffb700 1 -- 172.21.15.142:0/1079997588 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7feb3405b1b0 con 0x7feb400ffae0 2024-03-29T04:41:57.067 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.071+0000 7feb471bf700 1 -- 172.21.15.142:0/1079997588 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7feb400694e0 con 0x7feb3006c4e0 2024-03-29T04:41:57.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.075+0000 7feb3dffb700 1 -- 172.21.15.142:0/1079997588 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1310 (secure 0 0 0) 0x7feb400694e0 con 0x7feb3006c4e0 2024-03-29T04:41:57.072 INFO:teuthology.orchestra.run.smithi142.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-29T04:41:57.072 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-03-29T04:41:57.072 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter 2/2 2m ago 6m * 2024-03-29T04:41:57.072 INFO:teuthology.orchestra.run.smithi142.stdout:crash 2/2 2m ago 6m * 2024-03-29T04:41:57.072 INFO:teuthology.orchestra.run.smithi142.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-03-29T04:41:57.072 INFO:teuthology.orchestra.run.smithi142.stdout:mgr 2/2 2m ago 6m count:2 2024-03-29T04:41:57.072 INFO:teuthology.orchestra.run.smithi142.stdout:mon 2/2 2m ago 5m smithi142:172.21.15.142=smithi142;smithi152:172.21.15.152=smithi152;count:2 2024-03-29T04:41:57.072 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-03-29T04:41:57.072 INFO:teuthology.orchestra.run.smithi142.stdout:osd 6 2m ago - 2024-03-29T04:41:57.072 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-03-29T04:41:57.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.078+0000 7feb471bf700 1 -- 172.21.15.142:0/1079997588 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7feb3006c4e0 msgr2=0x7feb3006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:57.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.078+0000 7feb471bf700 1 --2- 172.21.15.142:0/1079997588 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7feb3006c4e0 0x7feb3006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb2c005950 tx=0x7feb2c00a300 comp rx=0 tx=0).stop 2024-03-29T04:41:57.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.078+0000 7feb471bf700 1 -- 172.21.15.142:0/1079997588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb400ffae0 msgr2=0x7feb40103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:57.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.078+0000 7feb471bf700 1 --2- 172.21.15.142:0/1079997588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb400ffae0 0x7feb40103a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feb3400bda0 tx=0x7feb3400bdd0 comp rx=0 tx=0).stop 2024-03-29T04:41:57.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.078+0000 7feb471bf700 1 -- 172.21.15.142:0/1079997588 shutdown_connections 2024-03-29T04:41:57.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.078+0000 7feb471bf700 1 --2- 172.21.15.142:0/1079997588 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7feb3006c4e0 0x7feb3006e9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:57.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.078+0000 7feb471bf700 1 --2- 172.21.15.142:0/1079997588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb400ffae0 0x7feb40103a70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:57.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.078+0000 7feb471bf700 1 --2- 172.21.15.142:0/1079997588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb40100ce0 0x7feb401020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:57.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.078+0000 7feb471bf700 1 -- 172.21.15.142:0/1079997588 >> 172.21.15.142:0/1079997588 conn(0x7feb400fb080 msgr2=0x7feb40068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:57.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.078+0000 7feb471bf700 1 -- 172.21.15.142:0/1079997588 shutdown_connections 2024-03-29T04:41:57.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:57.078+0000 7feb471bf700 1 -- 172.21.15.142:0/1079997588 wait complete. 2024-03-29T04:41:57.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-29T04:41:57.950 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:41:59.031 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:58 smithi142 ceph-mon[29027]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:41:59.031 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:41:58 smithi142 ceph-mon[29027]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:59.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:58 smithi152 ceph-mon[37177]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:41:59.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:41:58 smithi152 ceph-mon[37177]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:41:59.555 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.557+0000 7f4c5959b700 1 -- 172.21.15.142:0/976715352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c54103fb0 msgr2=0x7f4c541063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:59.555 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.557+0000 7f4c5959b700 1 --2- 172.21.15.142:0/976715352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c54103fb0 0x7f4c541063a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009910 tx=0x7f4c48009c20 comp rx=0 tx=0).stop 2024-03-29T04:41:59.555 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.557+0000 7f4c5959b700 1 -- 172.21.15.142:0/976715352 shutdown_connections 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.557+0000 7f4c5959b700 1 --2- 172.21.15.142:0/976715352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c54103fb0 0x7f4c541063a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.557+0000 7f4c5959b700 1 --2- 172.21.15.142:0/976715352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c54101680 0x7f4c54103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.557+0000 7f4c5959b700 1 -- 172.21.15.142:0/976715352 >> 172.21.15.142:0/976715352 conn(0x7f4c540fb0b0 msgr2=0x7f4c540fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.557+0000 7f4c5959b700 1 -- 172.21.15.142:0/976715352 shutdown_connections 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.557+0000 7f4c5959b700 1 -- 172.21.15.142:0/976715352 wait complete. 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.558+0000 7f4c5959b700 1 Processor -- start 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.558+0000 7f4c5959b700 1 -- start start 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.558+0000 7f4c5959b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c54101680 0x7f4c54196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.558+0000 7f4c5959b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c54103fb0 0x7f4c54196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.558+0000 7f4c5959b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54196e90 con 0x7f4c54103fb0 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.558+0000 7f4c5959b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54196fd0 con 0x7f4c54101680 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.558+0000 7f4c527fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c54103fb0 0x7f4c54196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.558+0000 7f4c527fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c54103fb0 0x7f4c54196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:48168/0 (socket says 172.21.15.142:48168) 2024-03-29T04:41:59.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.558+0000 7f4c527fc700 1 -- 172.21.15.142:0/4239088331 learned_addr learned my addr 172.21.15.142:0/4239088331 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:41:59.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.558+0000 7f4c52ffd700 1 --2- 172.21.15.142:0/4239088331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c54101680 0x7f4c54196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:59.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.559+0000 7f4c527fc700 1 -- 172.21.15.142:0/4239088331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c54101680 msgr2=0x7f4c54196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:59.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.559+0000 7f4c527fc700 1 --2- 172.21.15.142:0/4239088331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c54101680 0x7f4c54196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:59.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.559+0000 7f4c527fc700 1 -- 172.21.15.142:0/4239088331 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c480095f0 con 0x7f4c54103fb0 2024-03-29T04:41:59.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.559+0000 7f4c527fc700 1 --2- 172.21.15.142:0/4239088331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c54103fb0 0x7f4c54196870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800b5c0 tx=0x7f4c4800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:59.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.559+0000 7f4c3bfff700 1 -- 172.21.15.142:0/4239088331 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4801d070 con 0x7f4c54103fb0 2024-03-29T04:41:59.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.559+0000 7f4c5959b700 1 -- 172.21.15.142:0/4239088331 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5419ba20 con 0x7f4c54103fb0 2024-03-29T04:41:59.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.559+0000 7f4c3bfff700 1 -- 172.21.15.142:0/4239088331 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4c48022470 con 0x7f4c54103fb0 2024-03-29T04:41:59.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.559+0000 7f4c3bfff700 1 -- 172.21.15.142:0/4239088331 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4800f690 con 0x7f4c54103fb0 2024-03-29T04:41:59.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.559+0000 7f4c5959b700 1 -- 172.21.15.142:0/4239088331 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5419bf10 con 0x7f4c54103fb0 2024-03-29T04:41:59.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.560+0000 7f4c5959b700 1 -- 172.21.15.142:0/4239088331 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c54190560 con 0x7f4c54103fb0 2024-03-29T04:41:59.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.560+0000 7f4c3bfff700 1 -- 172.21.15.142:0/4239088331 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f4c480225e0 con 0x7f4c54103fb0 2024-03-29T04:41:59.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.561+0000 7f4c3bfff700 1 --2- 172.21.15.142:0/4239088331 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4c3c06c4e0 0x7f4c3c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:41:59.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.561+0000 7f4c52ffd700 1 --2- 172.21.15.142:0/4239088331 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4c3c06c4e0 0x7f4c3c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:41:59.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.561+0000 7f4c3bfff700 1 -- 172.21.15.142:0/4239088331 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4c4808ccd0 con 0x7f4c54103fb0 2024-03-29T04:41:59.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.561+0000 7f4c52ffd700 1 --2- 172.21.15.142:0/4239088331 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4c3c06c4e0 0x7f4c3c06e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c44005950 tx=0x7f4c4400a300 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:41:59.561 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.565+0000 7f4c3bfff700 1 -- 172.21.15.142:0/4239088331 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4c4805b280 con 0x7f4c54103fb0 2024-03-29T04:41:59.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.762+0000 7f4c5959b700 1 -- 172.21.15.142:0/4239088331 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c540611d0 con 0x7f4c3c06c4e0 2024-03-29T04:41:59.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.763+0000 7f4c3bfff700 1 -- 172.21.15.142:0/4239088331 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f4c540611d0 con 0x7f4c3c06c4e0 2024-03-29T04:41:59.759 INFO:teuthology.orchestra.run.smithi142.stdout:HOST ADDR LABELS STATUS 2024-03-29T04:41:59.759 INFO:teuthology.orchestra.run.smithi142.stdout:smithi142 172.21.15.142 2024-03-29T04:41:59.760 INFO:teuthology.orchestra.run.smithi142.stdout:smithi152 172.21.15.152 2024-03-29T04:41:59.760 INFO:teuthology.orchestra.run.smithi142.stdout:2 hosts in cluster 2024-03-29T04:41:59.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.765+0000 7f4c5959b700 1 -- 172.21.15.142:0/4239088331 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4c3c06c4e0 msgr2=0x7f4c3c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:59.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.765+0000 7f4c5959b700 1 --2- 172.21.15.142:0/4239088331 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4c3c06c4e0 0x7f4c3c06e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c44005950 tx=0x7f4c4400a300 comp rx=0 tx=0).stop 2024-03-29T04:41:59.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.765+0000 7f4c5959b700 1 -- 172.21.15.142:0/4239088331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c54103fb0 msgr2=0x7f4c54196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:41:59.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.765+0000 7f4c5959b700 1 --2- 172.21.15.142:0/4239088331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c54103fb0 0x7f4c54196870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800b5c0 tx=0x7f4c4800bd70 comp rx=0 tx=0).stop 2024-03-29T04:41:59.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.765+0000 7f4c5959b700 1 -- 172.21.15.142:0/4239088331 shutdown_connections 2024-03-29T04:41:59.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.765+0000 7f4c5959b700 1 --2- 172.21.15.142:0/4239088331 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4c3c06c4e0 0x7f4c3c06e9a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:59.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.765+0000 7f4c5959b700 1 --2- 172.21.15.142:0/4239088331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c54101680 0x7f4c54196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:59.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.765+0000 7f4c5959b700 1 --2- 172.21.15.142:0/4239088331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c54103fb0 0x7f4c54196870 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:41:59.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.765+0000 7f4c5959b700 1 -- 172.21.15.142:0/4239088331 >> 172.21.15.142:0/4239088331 conn(0x7f4c540fb0b0 msgr2=0x7f4c540fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:41:59.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.766+0000 7f4c5959b700 1 -- 172.21.15.142:0/4239088331 shutdown_connections 2024-03-29T04:41:59.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:41:59.766+0000 7f4c5959b700 1 -- 172.21.15.142:0/4239088331 wait complete. 2024-03-29T04:42:00.367 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-29T04:42:00.572 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:00.934 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:00 smithi142 ceph-mon[29027]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:00.934 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:00 smithi142 ceph-mon[29027]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:42:01.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:00 smithi152 ceph-mon[37177]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:01.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:00 smithi152 ceph-mon[37177]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:42:02.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.029+0000 7f8130eb4700 1 -- 172.21.15.142:0/2292852312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f812c100ce0 msgr2=0x7f812c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:02.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.029+0000 7f8130eb4700 1 --2- 172.21.15.142:0/2292852312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f812c100ce0 0x7f812c101140 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8120009910 tx=0x7f8120009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:02.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.029+0000 7f8130eb4700 1 -- 172.21.15.142:0/2292852312 shutdown_connections 2024-03-29T04:42:02.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.029+0000 7f8130eb4700 1 --2- 172.21.15.142:0/2292852312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f812c100ce0 0x7f812c101140 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:02.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.029+0000 7f8130eb4700 1 --2- 172.21.15.142:0/2292852312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f812c0ffae0 0x7f812c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:02.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.029+0000 7f8130eb4700 1 -- 172.21.15.142:0/2292852312 >> 172.21.15.142:0/2292852312 conn(0x7f812c0fb080 msgr2=0x7f812c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:02.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.029+0000 7f8130eb4700 1 -- 172.21.15.142:0/2292852312 shutdown_connections 2024-03-29T04:42:02.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.029+0000 7f8130eb4700 1 -- 172.21.15.142:0/2292852312 wait complete. 2024-03-29T04:42:02.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.030+0000 7f8130eb4700 1 Processor -- start 2024-03-29T04:42:02.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.030+0000 7f8130eb4700 1 -- start start 2024-03-29T04:42:02.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.030+0000 7f8130eb4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f812c0ffae0 0x7f812c196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:02.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.030+0000 7f8130eb4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f812c100ce0 0x7f812c196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:02.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.030+0000 7f8130eb4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c196fa0 con 0x7f812c0ffae0 2024-03-29T04:42:02.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.030+0000 7f8130eb4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c1970e0 con 0x7f812c100ce0 2024-03-29T04:42:02.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.030+0000 7f812a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f812c0ffae0 0x7f812c196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:02.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.030+0000 7f812a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f812c0ffae0 0x7f812c196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:48176/0 (socket says 172.21.15.142:48176) 2024-03-29T04:42:02.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.030+0000 7f812a59c700 1 -- 172.21.15.142:0/3716591379 learned_addr learned my addr 172.21.15.142:0/3716591379 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:02.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.031+0000 7f8129d9b700 1 --2- 172.21.15.142:0/3716591379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f812c100ce0 0x7f812c196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:02.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.031+0000 7f812a59c700 1 -- 172.21.15.142:0/3716591379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f812c100ce0 msgr2=0x7f812c196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:02.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.031+0000 7f812a59c700 1 --2- 172.21.15.142:0/3716591379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f812c100ce0 0x7f812c196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:02.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.031+0000 7f812a59c700 1 -- 172.21.15.142:0/3716591379 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81200095f0 con 0x7f812c0ffae0 2024-03-29T04:42:02.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.031+0000 7f812a59c700 1 --2- 172.21.15.142:0/3716591379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f812c0ffae0 0x7f812c196440 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f811c00b480 tx=0x7f811c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:02.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.031+0000 7f811b7fe700 1 -- 172.21.15.142:0/3716591379 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f811c011800 con 0x7f812c0ffae0 2024-03-29T04:42:02.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.031+0000 7f811b7fe700 1 -- 172.21.15.142:0/3716591379 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f811c011e10 con 0x7f812c0ffae0 2024-03-29T04:42:02.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.031+0000 7f8130eb4700 1 -- 172.21.15.142:0/3716591379 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c19bb40 con 0x7f812c0ffae0 2024-03-29T04:42:02.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.031+0000 7f811b7fe700 1 -- 172.21.15.142:0/3716591379 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f811c010ea0 con 0x7f812c0ffae0 2024-03-29T04:42:02.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.031+0000 7f8130eb4700 1 -- 172.21.15.142:0/3716591379 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c19c090 con 0x7f812c0ffae0 2024-03-29T04:42:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.032+0000 7f8130eb4700 1 -- 172.21.15.142:0/3716591379 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812c1086e0 con 0x7f812c0ffae0 2024-03-29T04:42:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.033+0000 7f811b7fe700 1 -- 172.21.15.142:0/3716591379 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f811c010380 con 0x7f812c0ffae0 2024-03-29T04:42:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.033+0000 7f811b7fe700 1 --2- 172.21.15.142:0/3716591379 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f811406c4e0 0x7f811406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.033+0000 7f8129d9b700 1 --2- 172.21.15.142:0/3716591379 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f811406c4e0 0x7f811406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.033+0000 7f811b7fe700 1 -- 172.21.15.142:0/3716591379 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f811c08b060 con 0x7f812c0ffae0 2024-03-29T04:42:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.034+0000 7f8129d9b700 1 --2- 172.21.15.142:0/3716591379 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f811406c4e0 0x7f811406e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8120006010 tx=0x7f812000b560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:02.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.038+0000 7f811b7fe700 1 -- 172.21.15.142:0/3716591379 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f811c0596c0 con 0x7f812c0ffae0 2024-03-29T04:42:02.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.231+0000 7f8130eb4700 1 -- 172.21.15.142:0/3716591379 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f812c0694e0 con 0x7f811406c4e0 2024-03-29T04:42:02.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.232+0000 7f811b7fe700 1 -- 172.21.15.142:0/3716591379 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f812c0694e0 con 0x7f811406c4e0 2024-03-29T04:42:02.228 INFO:teuthology.orchestra.run.smithi142.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-29T04:42:02.229 INFO:teuthology.orchestra.run.smithi142.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-29T04:42:02.229 INFO:teuthology.orchestra.run.smithi142.stdout:smithi152 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JT400BGN 372G No 73s ago Has a FileSystem, LVM detected 2024-03-29T04:42:02.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.234+0000 7f8130eb4700 1 -- 172.21.15.142:0/3716591379 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f811406c4e0 msgr2=0x7f811406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:02.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.234+0000 7f8130eb4700 1 --2- 172.21.15.142:0/3716591379 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f811406c4e0 0x7f811406e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8120006010 tx=0x7f812000b560 comp rx=0 tx=0).stop 2024-03-29T04:42:02.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.234+0000 7f8130eb4700 1 -- 172.21.15.142:0/3716591379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f812c0ffae0 msgr2=0x7f812c196440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:02.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.234+0000 7f8130eb4700 1 --2- 172.21.15.142:0/3716591379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f812c0ffae0 0x7f812c196440 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f811c00b480 tx=0x7f811c00b790 comp rx=0 tx=0).stop 2024-03-29T04:42:02.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.234+0000 7f8130eb4700 1 -- 172.21.15.142:0/3716591379 shutdown_connections 2024-03-29T04:42:02.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.234+0000 7f8130eb4700 1 --2- 172.21.15.142:0/3716591379 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f811406c4e0 0x7f811406e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:02.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.234+0000 7f8130eb4700 1 --2- 172.21.15.142:0/3716591379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f812c0ffae0 0x7f812c196440 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:02.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.234+0000 7f8130eb4700 1 --2- 172.21.15.142:0/3716591379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f812c100ce0 0x7f812c196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:02.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.234+0000 7f8130eb4700 1 -- 172.21.15.142:0/3716591379 >> 172.21.15.142:0/3716591379 conn(0x7f812c0fb080 msgr2=0x7f812c068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:02.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.234+0000 7f8130eb4700 1 -- 172.21.15.142:0/3716591379 shutdown_connections 2024-03-29T04:42:02.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:02.234+0000 7f8130eb4700 1 -- 172.21.15.142:0/3716591379 wait complete. 2024-03-29T04:42:02.891 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T04:42:02.901 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T04:42:02.901 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-29T04:42:03.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:02 smithi152 ceph-mon[37177]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:03.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:02 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:42:03.106 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:03.117 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:02 smithi142 ceph-mon[29027]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:03.117 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:02 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:42:03.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:03 smithi142 ceph-mon[29027]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:42:04.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:03 smithi152 ceph-mon[37177]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:42:04.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.771+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/427124048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b98101db0 msgr2=0x7f9b98102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:04.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.771+0000 7f9b9e5f4700 1 --2- 172.21.15.142:0/427124048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b98101db0 0x7f9b98102210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009910 tx=0x7f9b80009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:04.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.771+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/427124048 shutdown_connections 2024-03-29T04:42:04.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.771+0000 7f9b9e5f4700 1 --2- 172.21.15.142:0/427124048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b98101db0 0x7f9b98102210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:04.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.771+0000 7f9b9e5f4700 1 --2- 172.21.15.142:0/427124048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b98100bb0 0x7f9b98100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:04.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.771+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/427124048 >> 172.21.15.142:0/427124048 conn(0x7f9b980fc110 msgr2=0x7f9b980fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:04.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.771+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/427124048 shutdown_connections 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.771+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/427124048 wait complete. 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b9e5f4700 1 Processor -- start 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b9e5f4700 1 -- start start 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b9e5f4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b98100bb0 0x7f9b98074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b9e5f4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b98101db0 0x7f9b98073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b9e5f4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980736c0 con 0x7f9b98101db0 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b9e5f4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98073800 con 0x7f9b98100bb0 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b977fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b98101db0 0x7f9b98073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b977fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b98101db0 0x7f9b98073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:48200/0 (socket says 172.21.15.142:48200) 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b977fe700 1 -- 172.21.15.142:0/905006679 learned_addr learned my addr 172.21.15.142:0/905006679 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b97fff700 1 --2- 172.21.15.142:0/905006679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b98100bb0 0x7f9b98074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b977fe700 1 -- 172.21.15.142:0/905006679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b98100bb0 msgr2=0x7f9b98074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b977fe700 1 --2- 172.21.15.142:0/905006679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b98100bb0 0x7f9b98074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:04.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b977fe700 1 -- 172.21.15.142:0/905006679 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b800095f0 con 0x7f9b98101db0 2024-03-29T04:42:04.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.772+0000 7f9b977fe700 1 --2- 172.21.15.142:0/905006679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b98101db0 0x7f9b98073180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b80005b40 tx=0x7f9b8000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:04.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.773+0000 7f9b957fa700 1 -- 172.21.15.142:0/905006679 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b8001d070 con 0x7f9b98101db0 2024-03-29T04:42:04.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.773+0000 7f9b957fa700 1 -- 172.21.15.142:0/905006679 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9b80022470 con 0x7f9b98101db0 2024-03-29T04:42:04.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.773+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/905006679 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b98073a80 con 0x7f9b98101db0 2024-03-29T04:42:04.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.773+0000 7f9b957fa700 1 -- 172.21.15.142:0/905006679 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b8000f690 con 0x7f9b98101db0 2024-03-29T04:42:04.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.773+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/905006679 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b98073f70 con 0x7f9b98101db0 2024-03-29T04:42:04.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.774+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/905006679 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b98109710 con 0x7f9b98101db0 2024-03-29T04:42:04.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.774+0000 7f9b957fa700 1 -- 172.21.15.142:0/905006679 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9b8000f7f0 con 0x7f9b98101db0 2024-03-29T04:42:04.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.774+0000 7f9b957fa700 1 --2- 172.21.15.142:0/905006679 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f9b8406c4e0 0x7f9b8406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:04.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.774+0000 7f9b957fa700 1 -- 172.21.15.142:0/905006679 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9b8008cba0 con 0x7f9b98101db0 2024-03-29T04:42:04.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.775+0000 7f9b97fff700 1 --2- 172.21.15.142:0/905006679 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f9b8406c4e0 0x7f9b8406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:04.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.775+0000 7f9b97fff700 1 --2- 172.21.15.142:0/905006679 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f9b8406c4e0 0x7f9b8406e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b88005fd0 tx=0x7f9b88005e50 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:04.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.779+0000 7f9b957fa700 1 -- 172.21.15.142:0/905006679 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9b8005b2f0 con 0x7f9b98101db0 2024-03-29T04:42:04.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:04 smithi142 ceph-mon[29027]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:04.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:04.995+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/905006679 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b98106700 con 0x7f9b8406c4e0 2024-03-29T04:42:05.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:04 smithi152 ceph-mon[37177]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:05.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:05 smithi142 ceph-mon[29027]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:42:05.971 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:05 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-29T04:42:06.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:05 smithi152 ceph-mon[37177]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:42:06.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:05 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-29T04:42:06.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:06.733+0000 7f9b957fa700 1 -- 172.21.15.142:0/905006679 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9b98106700 con 0x7f9b8406c4e0 2024-03-29T04:42:06.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:06.736+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/905006679 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f9b8406c4e0 msgr2=0x7f9b8406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:06.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:06.736+0000 7f9b9e5f4700 1 --2- 172.21.15.142:0/905006679 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f9b8406c4e0 0x7f9b8406e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b88005fd0 tx=0x7f9b88005e50 comp rx=0 tx=0).stop 2024-03-29T04:42:06.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:06.736+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/905006679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b98101db0 msgr2=0x7f9b98073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:06.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:06.736+0000 7f9b9e5f4700 1 --2- 172.21.15.142:0/905006679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b98101db0 0x7f9b98073180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b80005b40 tx=0x7f9b8000be20 comp rx=0 tx=0).stop 2024-03-29T04:42:06.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:06.736+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/905006679 shutdown_connections 2024-03-29T04:42:06.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:06.736+0000 7f9b9e5f4700 1 --2- 172.21.15.142:0/905006679 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f9b8406c4e0 0x7f9b8406e9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:06.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:06.736+0000 7f9b9e5f4700 1 --2- 172.21.15.142:0/905006679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b98100bb0 0x7f9b98074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:06.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:06.736+0000 7f9b9e5f4700 1 --2- 172.21.15.142:0/905006679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b98101db0 0x7f9b98073180 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:06.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:06.736+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/905006679 >> 172.21.15.142:0/905006679 conn(0x7f9b980fc110 msgr2=0x7f9b98104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:06.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:06.736+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/905006679 shutdown_connections 2024-03-29T04:42:06.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:06.736+0000 7f9b9e5f4700 1 -- 172.21.15.142:0/905006679 wait complete. 2024-03-29T04:42:07.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:06 smithi152 ceph-mon[37177]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:07.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:06 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-29T04:42:07.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:06 smithi152 ceph-mon[37177]: osdmap e42: 6 total, 6 up, 6 in 2024-03-29T04:42:07.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:06 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-29T04:42:07.107 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:06 smithi142 ceph-mon[29027]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:07.107 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:06 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-29T04:42:07.107 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:06 smithi142 ceph-mon[29027]: osdmap e42: 6 total, 6 up, 6 in 2024-03-29T04:42:07.108 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:06 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-29T04:42:07.108 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:06 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[29023]: 2024-03-29T04:42:06.708+0000 7f1cb1a73700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:42:07.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-29T04:42:07.710 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:07.834 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-29T04:42:07.834 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: osdmap e43: 6 total, 6 up, 6 in 2024-03-29T04:42:07.834 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-29T04:42:07.834 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: osdmap e44: 6 total, 6 up, 6 in 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: fsmap cephfs:0 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: Saving service mds.cephfs spec with placement count:4 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.wgmndr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.wgmndr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T04:42:07.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:07 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:08.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-29T04:42:08.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: osdmap e43: 6 total, 6 up, 6 in 2024-03-29T04:42:08.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-29T04:42:08.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:42:08.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-29T04:42:08.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: osdmap e44: 6 total, 6 up, 6 in 2024-03-29T04:42:08.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-29T04:42:08.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: fsmap cephfs:0 2024-03-29T04:42:08.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: Saving service mds.cephfs spec with placement count:4 2024-03-29T04:42:08.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:08.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:42:08.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:08.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:42:08.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:08.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.wgmndr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:42:08.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.wgmndr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T04:42:08.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:07 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:09.007 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:08 smithi142 ceph-mon[29027]: pgmap v177: 65 pgs: 3 creating+peering, 1 active+clean, 61 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:09.007 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:08 smithi142 ceph-mon[29027]: Deploying daemon mds.cephfs.smithi142.wgmndr on smithi142 2024-03-29T04:42:09.008 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:08 smithi142 ceph-mon[29027]: osdmap e45: 6 total, 6 up, 6 in 2024-03-29T04:42:09.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:08 smithi152 ceph-mon[37177]: pgmap v177: 65 pgs: 3 creating+peering, 1 active+clean, 61 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:09.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:08 smithi152 ceph-mon[37177]: Deploying daemon mds.cephfs.smithi142.wgmndr on smithi142 2024-03-29T04:42:09.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:08 smithi152 ceph-mon[37177]: osdmap e45: 6 total, 6 up, 6 in 2024-03-29T04:42:09.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.542+0000 7f7561507700 1 -- 172.21.15.142:0/4026671616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c100cd0 msgr2=0x7f755c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:09.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.542+0000 7f7561507700 1 --2- 172.21.15.142:0/4026671616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c100cd0 0x7f755c101130 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7550009910 tx=0x7f7550009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:09.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.542+0000 7f7561507700 1 -- 172.21.15.142:0/4026671616 shutdown_connections 2024-03-29T04:42:09.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.542+0000 7f7561507700 1 --2- 172.21.15.142:0/4026671616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c100cd0 0x7f755c101130 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:09.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.542+0000 7f7561507700 1 --2- 172.21.15.142:0/4026671616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f755c0ffad0 0x7f755c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:09.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.542+0000 7f7561507700 1 -- 172.21.15.142:0/4026671616 >> 172.21.15.142:0/4026671616 conn(0x7f755c0fb050 msgr2=0x7f755c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:09.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.542+0000 7f7561507700 1 -- 172.21.15.142:0/4026671616 shutdown_connections 2024-03-29T04:42:09.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.542+0000 7f7561507700 1 -- 172.21.15.142:0/4026671616 wait complete. 2024-03-29T04:42:09.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.543+0000 7f7561507700 1 Processor -- start 2024-03-29T04:42:09.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.543+0000 7f7561507700 1 -- start start 2024-03-29T04:42:09.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.543+0000 7f7561507700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f755c0ffad0 0x7f755c1952f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:09.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.543+0000 7f7561507700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c100cd0 0x7f755c195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:09.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.543+0000 7f7561507700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c195e50 con 0x7f755c100cd0 2024-03-29T04:42:09.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.543+0000 7f7561507700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c195f90 con 0x7f755c0ffad0 2024-03-29T04:42:09.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.543+0000 7f755a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c100cd0 0x7f755c195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:09.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.543+0000 7f755a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c100cd0 0x7f755c195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56534/0 (socket says 172.21.15.142:56534) 2024-03-29T04:42:09.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.543+0000 7f755a7fc700 1 -- 172.21.15.142:0/3155977563 learned_addr learned my addr 172.21.15.142:0/3155977563 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:09.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.544+0000 7f755a7fc700 1 -- 172.21.15.142:0/3155977563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f755c0ffad0 msgr2=0x7f755c1952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:09.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.544+0000 7f755affd700 1 --2- 172.21.15.142:0/3155977563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f755c0ffad0 0x7f755c1952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:09.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.544+0000 7f755a7fc700 1 --2- 172.21.15.142:0/3155977563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f755c0ffad0 0x7f755c1952f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:09.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.544+0000 7f755a7fc700 1 -- 172.21.15.142:0/3155977563 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75500095f0 con 0x7f755c100cd0 2024-03-29T04:42:09.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.544+0000 7f755a7fc700 1 --2- 172.21.15.142:0/3155977563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c100cd0 0x7f755c195830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7550006010 tx=0x7f755000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:09.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.544+0000 7f7543fff700 1 -- 172.21.15.142:0/3155977563 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f755001d070 con 0x7f755c100cd0 2024-03-29T04:42:09.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.544+0000 7f7561507700 1 -- 172.21.15.142:0/3155977563 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c19a9e0 con 0x7f755c100cd0 2024-03-29T04:42:09.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.544+0000 7f7543fff700 1 -- 172.21.15.142:0/3155977563 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f755000f460 con 0x7f755c100cd0 2024-03-29T04:42:09.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.544+0000 7f7543fff700 1 -- 172.21.15.142:0/3155977563 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7550021640 con 0x7f755c100cd0 2024-03-29T04:42:09.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.544+0000 7f7561507700 1 -- 172.21.15.142:0/3155977563 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c19af50 con 0x7f755c100cd0 2024-03-29T04:42:09.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.545+0000 7f7561507700 1 -- 172.21.15.142:0/3155977563 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755c1086d0 con 0x7f755c100cd0 2024-03-29T04:42:09.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.545+0000 7f7543fff700 1 -- 172.21.15.142:0/3155977563 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f755000fab0 con 0x7f755c100cd0 2024-03-29T04:42:09.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.546+0000 7f7543fff700 1 --2- 172.21.15.142:0/3155977563 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f754406c4e0 0x7f754406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:09.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.546+0000 7f7543fff700 1 -- 172.21.15.142:0/3155977563 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f755008ccb0 con 0x7f755c100cd0 2024-03-29T04:42:09.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.546+0000 7f755affd700 1 --2- 172.21.15.142:0/3155977563 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f754406c4e0 0x7f754406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:09.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.546+0000 7f755affd700 1 --2- 172.21.15.142:0/3155977563 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f754406c4e0 0x7f754406e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f754c006fd0 tx=0x7f754c008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:09.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.551+0000 7f7543fff700 1 -- 172.21.15.142:0/3155977563 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f755005b040 con 0x7f755c100cd0 2024-03-29T04:42:09.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.790+0000 7f7561507700 1 -- 172.21.15.142:0/3155977563 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f755c19b910 con 0x7f755c100cd0 2024-03-29T04:42:09.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.790+0000 7f7543fff700 1 -- 172.21.15.142:0/3155977563 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1093 (secure 0 0 0) 0x7f7550026460 con 0x7f755c100cd0 2024-03-29T04:42:09.787 INFO:teuthology.orchestra.run.smithi142.stdout:e2 2024-03-29T04:42:09.787 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:42:09.787 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:42:09.787 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:42:09.787 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:42:09.787 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:42:09.787 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:42:09.787 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 2 2024-03-29T04:42:09.787 INFO:teuthology.orchestra.run.smithi142.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T04:42:09.787 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:42:09.787 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:42:06.709246+0000 2024-03-29T04:42:09.788 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:42:09.788 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:42:09.788 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:42:09.788 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:42:09.788 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:42:09.788 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:42:09.788 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:42:09.788 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 0 2024-03-29T04:42:09.788 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:42:09.788 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 1 2024-03-29T04:42:09.788 INFO:teuthology.orchestra.run.smithi142.stdout:in 2024-03-29T04:42:09.789 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T04:42:09.789 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T04:42:09.789 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:42:09.789 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:42:09.789 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:42:09.789 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:42:09.789 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:42:09.789 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:42:09.790 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:42:09.790 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 0 2024-03-29T04:42:09.790 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:42:09.790 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:42:09.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.793+0000 7f7561507700 1 -- 172.21.15.142:0/3155977563 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f754406c4e0 msgr2=0x7f754406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:09.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.793+0000 7f7561507700 1 --2- 172.21.15.142:0/3155977563 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f754406c4e0 0x7f754406e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f754c006fd0 tx=0x7f754c008040 comp rx=0 tx=0).stop 2024-03-29T04:42:09.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.793+0000 7f7561507700 1 -- 172.21.15.142:0/3155977563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c100cd0 msgr2=0x7f755c195830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:09.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.793+0000 7f7561507700 1 --2- 172.21.15.142:0/3155977563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c100cd0 0x7f755c195830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7550006010 tx=0x7f755000be20 comp rx=0 tx=0).stop 2024-03-29T04:42:09.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.793+0000 7f7561507700 1 -- 172.21.15.142:0/3155977563 shutdown_connections 2024-03-29T04:42:09.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.793+0000 7f7561507700 1 --2- 172.21.15.142:0/3155977563 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f754406c4e0 0x7f754406e9a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:09.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.793+0000 7f7561507700 1 --2- 172.21.15.142:0/3155977563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f755c0ffad0 0x7f755c1952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:09.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.793+0000 7f7561507700 1 --2- 172.21.15.142:0/3155977563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c100cd0 0x7f755c195830 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:09.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.793+0000 7f7561507700 1 -- 172.21.15.142:0/3155977563 >> 172.21.15.142:0/3155977563 conn(0x7f755c0fb050 msgr2=0x7f755c103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:09.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.793+0000 7f7561507700 1 -- 172.21.15.142:0/3155977563 shutdown_connections 2024-03-29T04:42:09.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:09.793+0000 7f7561507700 1 -- 172.21.15.142:0/3155977563 wait complete. 2024-03-29T04:42:09.793 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 2 2024-03-29T04:42:09.979 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:09 smithi142 ceph-mon[29027]: osdmap e46: 6 total, 6 up, 6 in 2024-03-29T04:42:10.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:09 smithi152 ceph-mon[37177]: osdmap e46: 6 total, 6 up, 6 in 2024-03-29T04:42:10.607 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T04:42:10.617 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T04:42:10.618 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-03-29T04:42:10.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:10 smithi142 ceph-mon[29027]: pgmap v180: 65 pgs: 3 creating+activating, 11 creating+peering, 30 active+clean, 21 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:10.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:10 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3155977563' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:42:11.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:10 smithi152 ceph-mon[37177]: pgmap v180: 65 pgs: 3 creating+activating, 11 creating+peering, 30 active+clean, 21 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:11.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:10 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3155977563' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:42:11.167 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:12.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.964+0000 7fbdaaeef700 1 -- 172.21.15.142:0/4154102553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbda40ffad0 msgr2=0x7fbda40ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.964+0000 7fbdaaeef700 1 --2- 172.21.15.142:0/4154102553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbda40ffad0 0x7fbda40ffef0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009910 tx=0x7fbd98009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.964+0000 7fbdaaeef700 1 -- 172.21.15.142:0/4154102553 shutdown_connections 2024-03-29T04:42:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.964+0000 7fbdaaeef700 1 --2- 172.21.15.142:0/4154102553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbda4100cd0 0x7fbda4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.964+0000 7fbdaaeef700 1 --2- 172.21.15.142:0/4154102553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbda40ffad0 0x7fbda40ffef0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.964+0000 7fbdaaeef700 1 -- 172.21.15.142:0/4154102553 >> 172.21.15.142:0/4154102553 conn(0x7fbda40fb050 msgr2=0x7fbda40fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.964+0000 7fbdaaeef700 1 -- 172.21.15.142:0/4154102553 shutdown_connections 2024-03-29T04:42:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.964+0000 7fbdaaeef700 1 -- 172.21.15.142:0/4154102553 wait complete. 2024-03-29T04:42:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbdaaeef700 1 Processor -- start 2024-03-29T04:42:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbdaaeef700 1 -- start start 2024-03-29T04:42:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbdaaeef700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbda40ffad0 0x7fbda4103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbdaaeef700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbda4100cd0 0x7fbda41020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:12.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbdaaeef700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda41025f0 con 0x7fbda40ffad0 2024-03-29T04:42:12.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbdaaeef700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4102730 con 0x7fbda4100cd0 2024-03-29T04:42:12.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbda8c8b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbda40ffad0 0x7fbda4103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:12.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbda8c8b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbda40ffad0 0x7fbda4103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56544/0 (socket says 172.21.15.142:56544) 2024-03-29T04:42:12.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbda8c8b700 1 -- 172.21.15.142:0/3359466460 learned_addr learned my addr 172.21.15.142:0/3359466460 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:12.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbda3fff700 1 --2- 172.21.15.142:0/3359466460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbda4100cd0 0x7fbda41020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:12.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbda8c8b700 1 -- 172.21.15.142:0/3359466460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbda4100cd0 msgr2=0x7fbda41020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:12.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbda8c8b700 1 --2- 172.21.15.142:0/3359466460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbda4100cd0 0x7fbda41020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:12.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.965+0000 7fbda8c8b700 1 -- 172.21.15.142:0/3359466460 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd980095f0 con 0x7fbda40ffad0 2024-03-29T04:42:12.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.966+0000 7fbda8c8b700 1 --2- 172.21.15.142:0/3359466460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbda40ffad0 0x7fbda4103a60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800bda0 tx=0x7fbd9800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:12.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.966+0000 7fbda1ffb700 1 -- 172.21.15.142:0/3359466460 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd9801d070 con 0x7fbda40ffad0 2024-03-29T04:42:12.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.966+0000 7fbdaaeef700 1 -- 172.21.15.142:0/3359466460 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda41029b0 con 0x7fbda40ffad0 2024-03-29T04:42:12.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.966+0000 7fbda1ffb700 1 -- 172.21.15.142:0/3359466460 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbd9800f460 con 0x7fbda40ffad0 2024-03-29T04:42:12.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.966+0000 7fbda1ffb700 1 -- 172.21.15.142:0/3359466460 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd98021620 con 0x7fbda40ffad0 2024-03-29T04:42:12.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.966+0000 7fbdaaeef700 1 -- 172.21.15.142:0/3359466460 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda4102ea0 con 0x7fbda40ffad0 2024-03-29T04:42:12.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.967+0000 7fbdaaeef700 1 -- 172.21.15.142:0/3359466460 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda4108650 con 0x7fbda40ffad0 2024-03-29T04:42:12.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.967+0000 7fbda1ffb700 1 -- 172.21.15.142:0/3359466460 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbd9800f5d0 con 0x7fbda40ffad0 2024-03-29T04:42:12.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.968+0000 7fbda1ffb700 1 --2- 172.21.15.142:0/3359466460 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbd9406c4e0 0x7fbd9406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:12.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.968+0000 7fbda3fff700 1 --2- 172.21.15.142:0/3359466460 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbd9406c4e0 0x7fbd9406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:12.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.968+0000 7fbda1ffb700 1 -- 172.21.15.142:0/3359466460 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fbd9808cac0 con 0x7fbda40ffad0 2024-03-29T04:42:12.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.968+0000 7fbda3fff700 1 --2- 172.21.15.142:0/3359466460 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbd9406c4e0 0x7fbd9406e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd90004630 tx=0x7fbd9000b3c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:12.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:12.972+0000 7fbda1ffb700 1 -- 172.21.15.142:0/3359466460 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbd9805ad30 con 0x7fbda40ffad0 2024-03-29T04:42:13.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:12 smithi152 ceph-mon[37177]: pgmap v181: 65 pgs: 3 creating+activating, 8 creating+peering, 54 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:13.117 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:12 smithi142 ceph-mon[29027]: pgmap v181: 65 pgs: 3 creating+activating, 8 creating+peering, 54 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:13.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.211+0000 7fbdaaeef700 1 -- 172.21.15.142:0/3359466460 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7fbda419e850 con 0x7fbda40ffad0 2024-03-29T04:42:13.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.432+0000 7fbda1ffb700 1 -- 172.21.15.142:0/3359466460 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v3) v1 ==== 105+0+0 (secure 0 0 0) 0x7fbd9805a8c0 con 0x7fbda40ffad0 2024-03-29T04:42:13.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.434+0000 7fbdaaeef700 1 -- 172.21.15.142:0/3359466460 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbd9406c4e0 msgr2=0x7fbd9406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:13.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.434+0000 7fbdaaeef700 1 --2- 172.21.15.142:0/3359466460 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbd9406c4e0 0x7fbd9406e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd90004630 tx=0x7fbd9000b3c0 comp rx=0 tx=0).stop 2024-03-29T04:42:13.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.434+0000 7fbdaaeef700 1 -- 172.21.15.142:0/3359466460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbda40ffad0 msgr2=0x7fbda4103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:13.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.434+0000 7fbdaaeef700 1 --2- 172.21.15.142:0/3359466460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbda40ffad0 0x7fbda4103a60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800bda0 tx=0x7fbd9800bdd0 comp rx=0 tx=0).stop 2024-03-29T04:42:13.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.435+0000 7fbdaaeef700 1 -- 172.21.15.142:0/3359466460 shutdown_connections 2024-03-29T04:42:13.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.435+0000 7fbdaaeef700 1 --2- 172.21.15.142:0/3359466460 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbd9406c4e0 0x7fbd9406e9a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:13.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.435+0000 7fbdaaeef700 1 --2- 172.21.15.142:0/3359466460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbda40ffad0 0x7fbda4103a60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:13.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.435+0000 7fbdaaeef700 1 --2- 172.21.15.142:0/3359466460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbda4100cd0 0x7fbda41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:13.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.435+0000 7fbdaaeef700 1 -- 172.21.15.142:0/3359466460 >> 172.21.15.142:0/3359466460 conn(0x7fbda40fb050 msgr2=0x7fbda4068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:13.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.435+0000 7fbdaaeef700 1 -- 172.21.15.142:0/3359466460 shutdown_connections 2024-03-29T04:42:13.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:13.435+0000 7fbdaaeef700 1 -- 172.21.15.142:0/3359466460 wait complete. 2024-03-29T04:42:14.016 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3359466460' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-29T04:42:14.016 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: daemon mds.cephfs.smithi142.wgmndr assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-29T04:42:14.016 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-29T04:42:14.016 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3359466460' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-29T04:42:14.016 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.142:6826/2071436089,v1:172.21.15.142:6827/2071436089] up:boot 2024-03-29T04:42:14.017 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: fsmap cephfs:1 {0=cephfs.smithi142.wgmndr=up:creating} 2024-03-29T04:42:14.017 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:42:14.017 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: daemon mds.cephfs.smithi142.wgmndr is now active in filesystem cephfs as rank 0 2024-03-29T04:42:14.017 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:14.017 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:14.017 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:14.017 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.uhebsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:42:14.017 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.uhebsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T04:42:14.017 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:13 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:14.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3359466460' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: daemon mds.cephfs.smithi142.wgmndr assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3359466460' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.142:6826/2071436089,v1:172.21.15.142:6827/2071436089] up:boot 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: fsmap cephfs:1 {0=cephfs.smithi142.wgmndr=up:creating} 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: daemon mds.cephfs.smithi142.wgmndr is now active in filesystem cephfs as rank 0 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.uhebsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.uhebsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T04:42:14.073 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:13 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:14.094 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T04:42:14.103 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T04:42:14.104 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-03-29T04:42:14.312 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:15.020 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:14 smithi142 ceph-mon[29027]: Deploying daemon mds.cephfs.smithi152.uhebsi on smithi152 2024-03-29T04:42:15.020 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:14 smithi142 ceph-mon[29027]: pgmap v182: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:15.020 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:14 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.142:6826/2071436089,v1:172.21.15.142:6827/2071436089] up:active 2024-03-29T04:42:15.020 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:14 smithi142 ceph-mon[29027]: fsmap cephfs:1 {0=cephfs.smithi142.wgmndr=up:active} 2024-03-29T04:42:15.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:14 smithi152 ceph-mon[37177]: Deploying daemon mds.cephfs.smithi152.uhebsi on smithi152 2024-03-29T04:42:15.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:14 smithi152 ceph-mon[37177]: pgmap v182: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-29T04:42:15.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:14 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.142:6826/2071436089,v1:172.21.15.142:6827/2071436089] up:active 2024-03-29T04:42:15.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:14 smithi152 ceph-mon[37177]: fsmap cephfs:1 {0=cephfs.smithi142.wgmndr=up:active} 2024-03-29T04:42:15.879 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.881+0000 7f2a86ba5700 1 -- 172.21.15.142:0/2545063168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a80100cb0 msgr2=0x7f2a80101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:15.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.881+0000 7f2a86ba5700 1 --2- 172.21.15.142:0/2545063168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a80100cb0 0x7f2a80101110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009910 tx=0x7f2a6c009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:15.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.881+0000 7f2a86ba5700 1 -- 172.21.15.142:0/2545063168 shutdown_connections 2024-03-29T04:42:15.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.881+0000 7f2a86ba5700 1 --2- 172.21.15.142:0/2545063168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a80100cb0 0x7f2a80101110 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:15.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.881+0000 7f2a86ba5700 1 --2- 172.21.15.142:0/2545063168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a800ffab0 0x7f2a800ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:15.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.881+0000 7f2a86ba5700 1 -- 172.21.15.142:0/2545063168 >> 172.21.15.142:0/2545063168 conn(0x7f2a800fb030 msgr2=0x7f2a800fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:15.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.881+0000 7f2a86ba5700 1 -- 172.21.15.142:0/2545063168 shutdown_connections 2024-03-29T04:42:15.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.882+0000 7f2a86ba5700 1 -- 172.21.15.142:0/2545063168 wait complete. 2024-03-29T04:42:15.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.882+0000 7f2a86ba5700 1 Processor -- start 2024-03-29T04:42:15.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.882+0000 7f2a86ba5700 1 -- start start 2024-03-29T04:42:15.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.882+0000 7f2a86ba5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a800ffab0 0x7f2a80196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:15.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.882+0000 7f2a86ba5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a80100cb0 0x7f2a80196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:15.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.882+0000 7f2a86ba5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80196ea0 con 0x7f2a800ffab0 2024-03-29T04:42:15.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.882+0000 7f2a86ba5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80196fe0 con 0x7f2a80100cb0 2024-03-29T04:42:15.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.882+0000 7f2a84941700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a800ffab0 0x7f2a80196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:15.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.882+0000 7f2a84941700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a800ffab0 0x7f2a80196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56558/0 (socket says 172.21.15.142:56558) 2024-03-29T04:42:15.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.882+0000 7f2a84941700 1 -- 172.21.15.142:0/3761733009 learned_addr learned my addr 172.21.15.142:0/3761733009 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:15.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.882+0000 7f2a7ffff700 1 --2- 172.21.15.142:0/3761733009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a80100cb0 0x7f2a80196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:15.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.883+0000 7f2a84941700 1 -- 172.21.15.142:0/3761733009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a80100cb0 msgr2=0x7f2a80196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:15.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.883+0000 7f2a84941700 1 --2- 172.21.15.142:0/3761733009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a80100cb0 0x7f2a80196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:15.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.883+0000 7f2a84941700 1 -- 172.21.15.142:0/3761733009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c0095f0 con 0x7f2a800ffab0 2024-03-29T04:42:15.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.883+0000 7f2a84941700 1 --2- 172.21.15.142:0/3761733009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a800ffab0 0x7f2a80196340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400d900 tx=0x7f2a7400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:15.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.883+0000 7f2a7dffb700 1 -- 172.21.15.142:0/3761733009 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a740041f0 con 0x7f2a800ffab0 2024-03-29T04:42:15.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.883+0000 7f2a7dffb700 1 -- 172.21.15.142:0/3761733009 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2a74004350 con 0x7f2a800ffab0 2024-03-29T04:42:15.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.883+0000 7f2a86ba5700 1 -- 172.21.15.142:0/3761733009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8019ba90 con 0x7f2a800ffab0 2024-03-29T04:42:15.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.883+0000 7f2a7dffb700 1 -- 172.21.15.142:0/3761733009 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a74003d20 con 0x7f2a800ffab0 2024-03-29T04:42:15.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.884+0000 7f2a86ba5700 1 -- 172.21.15.142:0/3761733009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8019bfe0 con 0x7f2a800ffab0 2024-03-29T04:42:15.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.885+0000 7f2a7dffb700 1 -- 172.21.15.142:0/3761733009 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2a74009730 con 0x7f2a800ffab0 2024-03-29T04:42:15.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.885+0000 7f2a86ba5700 1 -- 172.21.15.142:0/3761733009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a80108610 con 0x7f2a800ffab0 2024-03-29T04:42:15.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.885+0000 7f2a7dffb700 1 --2- 172.21.15.142:0/3761733009 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f2a7006c4e0 0x7f2a7006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:15.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.885+0000 7f2a7ffff700 1 --2- 172.21.15.142:0/3761733009 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f2a7006c4e0 0x7f2a7006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:15.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.885+0000 7f2a7dffb700 1 -- 172.21.15.142:0/3761733009 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f2a7408ce00 con 0x7f2a800ffab0 2024-03-29T04:42:15.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.885+0000 7f2a7ffff700 1 --2- 172.21.15.142:0/3761733009 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f2a7006c4e0 0x7f2a7006e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c006010 tx=0x7f2a6c00b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:15.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:15.890+0000 7f2a7dffb700 1 -- 172.21.15.142:0/3761733009 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2a74090060 con 0x7f2a800ffab0 2024-03-29T04:42:16.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.128+0000 7f2a86ba5700 1 -- 172.21.15.142:0/3761733009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f2a80108830 con 0x7f2a800ffab0 2024-03-29T04:42:16.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.739+0000 7f2a7dffb700 1 -- 172.21.15.142:0/3761733009 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v5) v1 ==== 121+0+0 (secure 0 0 0) 0x7f2a74058e20 con 0x7f2a800ffab0 2024-03-29T04:42:16.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.741+0000 7f2a86ba5700 1 -- 172.21.15.142:0/3761733009 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f2a7006c4e0 msgr2=0x7f2a7006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:16.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.741+0000 7f2a86ba5700 1 --2- 172.21.15.142:0/3761733009 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f2a7006c4e0 0x7f2a7006e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c006010 tx=0x7f2a6c00b540 comp rx=0 tx=0).stop 2024-03-29T04:42:16.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.741+0000 7f2a86ba5700 1 -- 172.21.15.142:0/3761733009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a800ffab0 msgr2=0x7f2a80196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:16.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.741+0000 7f2a86ba5700 1 --2- 172.21.15.142:0/3761733009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a800ffab0 0x7f2a80196340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400d900 tx=0x7f2a7400dc10 comp rx=0 tx=0).stop 2024-03-29T04:42:16.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.742+0000 7f2a86ba5700 1 -- 172.21.15.142:0/3761733009 shutdown_connections 2024-03-29T04:42:16.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.742+0000 7f2a86ba5700 1 --2- 172.21.15.142:0/3761733009 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f2a7006c4e0 0x7f2a7006e9a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:16.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.742+0000 7f2a86ba5700 1 --2- 172.21.15.142:0/3761733009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a800ffab0 0x7f2a80196340 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:16.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.742+0000 7f2a86ba5700 1 --2- 172.21.15.142:0/3761733009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a80100cb0 0x7f2a80196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:16.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.742+0000 7f2a86ba5700 1 -- 172.21.15.142:0/3761733009 >> 172.21.15.142:0/3761733009 conn(0x7f2a800fb030 msgr2=0x7f2a80068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:16.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.742+0000 7f2a86ba5700 1 -- 172.21.15.142:0/3761733009 shutdown_connections 2024-03-29T04:42:16.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:16.742+0000 7f2a86ba5700 1 -- 172.21.15.142:0/3761733009 wait complete. 2024-03-29T04:42:17.036 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:16 smithi142 ceph-mon[29027]: pgmap v183: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 383 B/s wr, 1 op/s 2024-03-29T04:42:17.036 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:16 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3761733009' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-29T04:42:17.036 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:16 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:17.036 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:16 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:17.036 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:16 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:17.037 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:16 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.bmkjzf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:42:17.037 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:16 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.bmkjzf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T04:42:17.037 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:16 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:17.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:16 smithi152 ceph-mon[37177]: pgmap v183: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 383 B/s wr, 1 op/s 2024-03-29T04:42:17.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:16 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3761733009' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-29T04:42:17.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:16 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:17.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:16 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:17.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:16 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:17.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:16 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.bmkjzf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:42:17.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:16 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.bmkjzf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T04:42:17.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:16 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:17.878 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T04:42:17.888 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T04:42:17.889 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-03-29T04:42:18.001 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:17 smithi142 ceph-mon[29027]: Deploying daemon mds.cephfs.smithi142.bmkjzf on smithi142 2024-03-29T04:42:18.001 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:17 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] up:boot 2024-03-29T04:42:18.001 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:17 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3761733009' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-29T04:42:18.001 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:17 smithi142 ceph-mon[29027]: daemon mds.cephfs.smithi152.uhebsi assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-29T04:42:18.001 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:17 smithi142 ceph-mon[29027]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-29T04:42:18.001 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:17 smithi142 ceph-mon[29027]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-29T04:42:18.002 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:17 smithi142 ceph-mon[29027]: fsmap cephfs:1 {0=cephfs.smithi142.wgmndr=up:active} 1 up:standby 2024-03-29T04:42:18.002 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:42:18.002 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:17 smithi142 ceph-mon[29027]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi152.uhebsi=up:creating} 2024-03-29T04:42:18.002 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:17 smithi142 ceph-mon[29027]: daemon mds.cephfs.smithi152.uhebsi is now active in filesystem cephfs as rank 1 2024-03-29T04:42:18.002 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:18.002 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:42:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:17 smithi152 ceph-mon[37177]: Deploying daemon mds.cephfs.smithi142.bmkjzf on smithi142 2024-03-29T04:42:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:17 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] up:boot 2024-03-29T04:42:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:17 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3761733009' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-29T04:42:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:17 smithi152 ceph-mon[37177]: daemon mds.cephfs.smithi152.uhebsi assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-29T04:42:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:17 smithi152 ceph-mon[37177]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-29T04:42:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:17 smithi152 ceph-mon[37177]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-29T04:42:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:17 smithi152 ceph-mon[37177]: fsmap cephfs:1 {0=cephfs.smithi142.wgmndr=up:active} 1 up:standby 2024-03-29T04:42:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:17 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:42:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:17 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi152.uhebsi=up:creating} 2024-03-29T04:42:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:17 smithi152 ceph-mon[37177]: daemon mds.cephfs.smithi152.uhebsi is now active in filesystem cephfs as rank 1 2024-03-29T04:42:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:17 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:17 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:42:18.440 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:18.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:18 smithi142 ceph-mon[29027]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s wr, 4 op/s 2024-03-29T04:42:18.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:18 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] up:active 2024-03-29T04:42:18.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:18 smithi142 ceph-mon[29027]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi152.uhebsi=up:active} 2024-03-29T04:42:19.036 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:18 smithi152 ceph-mon[37177]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s wr, 4 op/s 2024-03-29T04:42:19.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:18 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] up:active 2024-03-29T04:42:19.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:18 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi152.uhebsi=up:active} 2024-03-29T04:42:20.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.147+0000 7fc70b332700 1 -- 172.21.15.142:0/126064268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc704101eb0 msgr2=0x7fc704102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:20.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.147+0000 7fc70b332700 1 --2- 172.21.15.142:0/126064268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc704101eb0 0x7fc704102310 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009910 tx=0x7fc6f4009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:20.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.147+0000 7fc70b332700 1 -- 172.21.15.142:0/126064268 shutdown_connections 2024-03-29T04:42:20.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.147+0000 7fc70b332700 1 --2- 172.21.15.142:0/126064268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc704101eb0 0x7fc704102310 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:20.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.147+0000 7fc70b332700 1 --2- 172.21.15.142:0/126064268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7040feed0 0x7fc7040ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:20.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.147+0000 7fc70b332700 1 -- 172.21.15.142:0/126064268 >> 172.21.15.142:0/126064268 conn(0x7fc7040fa810 msgr2=0x7fc7040fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:20.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.148+0000 7fc70b332700 1 -- 172.21.15.142:0/126064268 shutdown_connections 2024-03-29T04:42:20.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.148+0000 7fc70b332700 1 -- 172.21.15.142:0/126064268 wait complete. 2024-03-29T04:42:20.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.148+0000 7fc70b332700 1 Processor -- start 2024-03-29T04:42:20.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.148+0000 7fc70b332700 1 -- start start 2024-03-29T04:42:20.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.148+0000 7fc70b332700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7040feed0 0x7fc704101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:20.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.148+0000 7fc70b332700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc704101eb0 0x7fc7040ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:20.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.148+0000 7fc70b332700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7041003a0 con 0x7fc7040feed0 2024-03-29T04:42:20.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.148+0000 7fc70b332700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7041004e0 con 0x7fc704101eb0 2024-03-29T04:42:20.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.148+0000 7fc7090ce700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7040feed0 0x7fc704101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:20.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.148+0000 7fc7090ce700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7040feed0 0x7fc704101810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:50700/0 (socket says 172.21.15.142:50700) 2024-03-29T04:42:20.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.148+0000 7fc7090ce700 1 -- 172.21.15.142:0/3917337511 learned_addr learned my addr 172.21.15.142:0/3917337511 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:20.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.149+0000 7fc7088cd700 1 --2- 172.21.15.142:0/3917337511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc704101eb0 0x7fc7040ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:20.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.149+0000 7fc7090ce700 1 -- 172.21.15.142:0/3917337511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc704101eb0 msgr2=0x7fc7040ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:20.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.149+0000 7fc7090ce700 1 --2- 172.21.15.142:0/3917337511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc704101eb0 0x7fc7040ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:20.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.149+0000 7fc7090ce700 1 -- 172.21.15.142:0/3917337511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f40095f0 con 0x7fc7040feed0 2024-03-29T04:42:20.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.149+0000 7fc7088cd700 1 --2- 172.21.15.142:0/3917337511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc704101eb0 0x7fc7040ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:42:20.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.149+0000 7fc7090ce700 1 --2- 172.21.15.142:0/3917337511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7040feed0 0x7fc704101810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc70000b480 tx=0x7fc70000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:20.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.149+0000 7fc6fa7fc700 1 -- 172.21.15.142:0/3917337511 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7000118c0 con 0x7fc7040feed0 2024-03-29T04:42:20.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.149+0000 7fc70b332700 1 -- 172.21.15.142:0/3917337511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7041007c0 con 0x7fc7040feed0 2024-03-29T04:42:20.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.149+0000 7fc6fa7fc700 1 -- 172.21.15.142:0/3917337511 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc700010380 con 0x7fc7040feed0 2024-03-29T04:42:20.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.149+0000 7fc70b332700 1 -- 172.21.15.142:0/3917337511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70419e510 con 0x7fc7040feed0 2024-03-29T04:42:20.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.149+0000 7fc6fa7fc700 1 -- 172.21.15.142:0/3917337511 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc70000f4f0 con 0x7fc7040feed0 2024-03-29T04:42:20.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.150+0000 7fc70b332700 1 -- 172.21.15.142:0/3917337511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc704109810 con 0x7fc7040feed0 2024-03-29T04:42:20.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.150+0000 7fc6fa7fc700 1 -- 172.21.15.142:0/3917337511 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc7000104f0 con 0x7fc7040feed0 2024-03-29T04:42:20.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.151+0000 7fc6fa7fc700 1 --2- 172.21.15.142:0/3917337511 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc6f006c4e0 0x7fc6f006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:20.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.151+0000 7fc6fa7fc700 1 -- 172.21.15.142:0/3917337511 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc70008b4a0 con 0x7fc7040feed0 2024-03-29T04:42:20.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.151+0000 7fc7088cd700 1 --2- 172.21.15.142:0/3917337511 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc6f006c4e0 0x7fc6f006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:20.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.151+0000 7fc7088cd700 1 --2- 172.21.15.142:0/3917337511 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc6f006c4e0 0x7fc6f006e9a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4006010 tx=0x7fc6f401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:20.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.155+0000 7fc6fa7fc700 1 -- 172.21.15.142:0/3917337511 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc700059710 con 0x7fc7040feed0 2024-03-29T04:42:20.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.394+0000 7fc70b332700 1 -- 172.21.15.142:0/3917337511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7fc70419eb20 con 0x7fc7040feed0 2024-03-29T04:42:20.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.749+0000 7fc6fa7fc700 1 -- 172.21.15.142:0/3917337511 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v8) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc7000592a0 con 0x7fc7040feed0 2024-03-29T04:42:20.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.752+0000 7fc70b332700 1 -- 172.21.15.142:0/3917337511 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc6f006c4e0 msgr2=0x7fc6f006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:20.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.752+0000 7fc70b332700 1 --2- 172.21.15.142:0/3917337511 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc6f006c4e0 0x7fc6f006e9a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4006010 tx=0x7fc6f401a040 comp rx=0 tx=0).stop 2024-03-29T04:42:20.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.752+0000 7fc70b332700 1 -- 172.21.15.142:0/3917337511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7040feed0 msgr2=0x7fc704101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:20.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.752+0000 7fc70b332700 1 --2- 172.21.15.142:0/3917337511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7040feed0 0x7fc704101810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc70000b480 tx=0x7fc70000b840 comp rx=0 tx=0).stop 2024-03-29T04:42:20.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.752+0000 7fc70b332700 1 -- 172.21.15.142:0/3917337511 shutdown_connections 2024-03-29T04:42:20.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.752+0000 7fc70b332700 1 --2- 172.21.15.142:0/3917337511 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc6f006c4e0 0x7fc6f006e9a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:20.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.752+0000 7fc70b332700 1 --2- 172.21.15.142:0/3917337511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7040feed0 0x7fc704101810 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:20.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.752+0000 7fc70b332700 1 --2- 172.21.15.142:0/3917337511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc704101eb0 0x7fc7040ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:20.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.752+0000 7fc70b332700 1 -- 172.21.15.142:0/3917337511 >> 172.21.15.142:0/3917337511 conn(0x7fc7040fa810 msgr2=0x7fc7041050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:20.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.753+0000 7fc70b332700 1 -- 172.21.15.142:0/3917337511 shutdown_connections 2024-03-29T04:42:20.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:20.753+0000 7fc70b332700 1 -- 172.21.15.142:0/3917337511 wait complete. 2024-03-29T04:42:20.750 INFO:teuthology.orchestra.run.smithi142.stderr:inline data disabled 2024-03-29T04:42:20.851 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:20 smithi142 ceph-mon[29027]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s wr, 7 op/s 2024-03-29T04:42:20.851 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:20 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3917337511' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-29T04:42:20.851 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:20 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:20.851 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:20 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:20.851 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:20 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:20.851 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:20 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.hiilum", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:42:20.851 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:20 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.hiilum", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T04:42:20.851 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:20 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:21.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:20 smithi152 ceph-mon[37177]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s wr, 7 op/s 2024-03-29T04:42:21.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:20 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3917337511' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-29T04:42:21.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:20 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:21.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:20 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:21.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:20 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:21.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:20 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.hiilum", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:42:21.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:20 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.hiilum", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T04:42:21.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:20 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:21.367 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T04:42:21.378 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T04:42:21.378 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-29T04:42:21.596 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:21.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:21 smithi142 ceph-mon[29027]: Deploying daemon mds.cephfs.smithi152.hiilum on smithi152 2024-03-29T04:42:21.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:21 smithi142 ceph-mon[29027]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-29T04:42:21.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:21 smithi142 ceph-mon[29027]: Cluster is now healthy 2024-03-29T04:42:21.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:21 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3917337511' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-29T04:42:21.965 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:21 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] up:boot 2024-03-29T04:42:21.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:21 smithi142 ceph-mon[29027]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby 2024-03-29T04:42:21.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:21 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T04:42:21.966 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:21 smithi142 ceph-mon[29027]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby-replay 2024-03-29T04:42:22.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:21 smithi152 ceph-mon[37177]: Deploying daemon mds.cephfs.smithi152.hiilum on smithi152 2024-03-29T04:42:22.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:21 smithi152 ceph-mon[37177]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-29T04:42:22.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:21 smithi152 ceph-mon[37177]: Cluster is now healthy 2024-03-29T04:42:22.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:21 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3917337511' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-29T04:42:22.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:21 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] up:boot 2024-03-29T04:42:22.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:21 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby 2024-03-29T04:42:22.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:21 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T04:42:22.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:21 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby-replay 2024-03-29T04:42:22.924 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:22 smithi142 ceph-mon[29027]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-03-29T04:42:22.924 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:22 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:22.960 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:22 smithi152 ceph-mon[37177]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-03-29T04:42:22.961 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:22 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:23.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.168+0000 7fd87186d700 1 -- 172.21.15.142:0/3651762787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd86c101db0 msgr2=0x7fd86c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:23.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.168+0000 7fd87186d700 1 --2- 172.21.15.142:0/3651762787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd86c101db0 0x7fd86c102210 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd854009910 tx=0x7fd854009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:23.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.168+0000 7fd87186d700 1 -- 172.21.15.142:0/3651762787 shutdown_connections 2024-03-29T04:42:23.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.168+0000 7fd87186d700 1 --2- 172.21.15.142:0/3651762787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd86c101db0 0x7fd86c102210 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:23.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.168+0000 7fd87186d700 1 --2- 172.21.15.142:0/3651762787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd86c100bb0 0x7fd86c100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:23.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.168+0000 7fd87186d700 1 -- 172.21.15.142:0/3651762787 >> 172.21.15.142:0/3651762787 conn(0x7fd86c0fc110 msgr2=0x7fd86c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:23.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.168+0000 7fd87186d700 1 -- 172.21.15.142:0/3651762787 shutdown_connections 2024-03-29T04:42:23.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.168+0000 7fd87186d700 1 -- 172.21.15.142:0/3651762787 wait complete. 2024-03-29T04:42:23.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.168+0000 7fd87186d700 1 Processor -- start 2024-03-29T04:42:23.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd87186d700 1 -- start start 2024-03-29T04:42:23.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd87186d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd86c100bb0 0x7fd86c196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:23.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd87186d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd86c101db0 0x7fd86c1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd87186d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c196ec0 con 0x7fd86c101db0 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd87186d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c197000 con 0x7fd86c100bb0 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd86a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd86c101db0 0x7fd86c1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd86a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd86c101db0 0x7fd86c1968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:50730/0 (socket says 172.21.15.142:50730) 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd86a7fc700 1 -- 172.21.15.142:0/4164634270 learned_addr learned my addr 172.21.15.142:0/4164634270 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd86affd700 1 --2- 172.21.15.142:0/4164634270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd86c100bb0 0x7fd86c196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd86a7fc700 1 -- 172.21.15.142:0/4164634270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd86c100bb0 msgr2=0x7fd86c196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd86a7fc700 1 --2- 172.21.15.142:0/4164634270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd86c100bb0 0x7fd86c196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd86a7fc700 1 -- 172.21.15.142:0/4164634270 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540095f0 con 0x7fd86c101db0 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.169+0000 7fd86a7fc700 1 --2- 172.21.15.142:0/4164634270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd86c101db0 0x7fd86c1968a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd854000c00 tx=0x7fd85400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.170+0000 7fd87086b700 1 -- 172.21.15.142:0/4164634270 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd85401d070 con 0x7fd86c101db0 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.170+0000 7fd87186d700 1 -- 172.21.15.142:0/4164634270 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86c19ba50 con 0x7fd86c101db0 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.170+0000 7fd87086b700 1 -- 172.21.15.142:0/4164634270 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd854005dc0 con 0x7fd86c101db0 2024-03-29T04:42:23.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.170+0000 7fd87086b700 1 -- 172.21.15.142:0/4164634270 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8540226a0 con 0x7fd86c101db0 2024-03-29T04:42:23.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.170+0000 7fd87186d700 1 -- 172.21.15.142:0/4164634270 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86c19bf40 con 0x7fd86c101db0 2024-03-29T04:42:23.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.171+0000 7fd87186d700 1 -- 172.21.15.142:0/4164634270 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86c109710 con 0x7fd86c101db0 2024-03-29T04:42:23.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.171+0000 7fd87086b700 1 -- 172.21.15.142:0/4164634270 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fd854022800 con 0x7fd86c101db0 2024-03-29T04:42:23.170 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.171+0000 7fd87086b700 1 --2- 172.21.15.142:0/4164634270 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fd85806c4e0 0x7fd85806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:23.170 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.172+0000 7fd86affd700 1 --2- 172.21.15.142:0/4164634270 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fd85806c4e0 0x7fd85806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:23.170 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.172+0000 7fd87086b700 1 -- 172.21.15.142:0/4164634270 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fd85408cd40 con 0x7fd86c101db0 2024-03-29T04:42:23.170 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.172+0000 7fd86affd700 1 --2- 172.21.15.142:0/4164634270 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fd85806c4e0 0x7fd85806e9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd86c101c10 tx=0x7fd86400b040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.176+0000 7fd87086b700 1 -- 172.21.15.142:0/4164634270 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd85405b020 con 0x7fd86c101db0 2024-03-29T04:42:23.404 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.407+0000 7fd87186d700 1 -- 172.21.15.142:0/4164634270 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd86c109900 con 0x7fd86c101db0 2024-03-29T04:42:23.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.408+0000 7fd87086b700 1 -- 172.21.15.142:0/4164634270 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 9 v9) v1 ==== 75+0+1627 (secure 0 0 0) 0x7fd854055e60 con 0x7fd86c101db0 2024-03-29T04:42:23.405 INFO:teuthology.orchestra.run.smithi142.stdout:e9 2024-03-29T04:42:23.405 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:42:23.405 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:42:23.405 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:42:23.405 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:42:23.405 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:42:23.405 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:42:23.405 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 9 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:42:20.750934+0000 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 0 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:42:23.406 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:up {0=14478,1=24275} 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{0:14478} state up:active seq 2 addr [v2:172.21.15.142:6826/2071436089,v1:172.21.15.142:6827/2071436089] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{0:14494} state up:standby-replay seq 1 addr [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{1:24275} state up:active seq 2 addr [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:42:23.407 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:42:23.408 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:42:23.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.410+0000 7fd87186d700 1 -- 172.21.15.142:0/4164634270 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fd85806c4e0 msgr2=0x7fd85806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:23.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.410+0000 7fd87186d700 1 --2- 172.21.15.142:0/4164634270 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fd85806c4e0 0x7fd85806e9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd86c101c10 tx=0x7fd86400b040 comp rx=0 tx=0).stop 2024-03-29T04:42:23.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.410+0000 7fd87186d700 1 -- 172.21.15.142:0/4164634270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd86c101db0 msgr2=0x7fd86c1968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:23.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.410+0000 7fd87186d700 1 --2- 172.21.15.142:0/4164634270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd86c101db0 0x7fd86c1968a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd854000c00 tx=0x7fd85400f690 comp rx=0 tx=0).stop 2024-03-29T04:42:23.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.410+0000 7fd87186d700 1 -- 172.21.15.142:0/4164634270 shutdown_connections 2024-03-29T04:42:23.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.410+0000 7fd87186d700 1 --2- 172.21.15.142:0/4164634270 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fd85806c4e0 0x7fd85806e9a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:23.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.410+0000 7fd87186d700 1 --2- 172.21.15.142:0/4164634270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd86c100bb0 0x7fd86c196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:23.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.410+0000 7fd87186d700 1 --2- 172.21.15.142:0/4164634270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd86c101db0 0x7fd86c1968a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:23.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.410+0000 7fd87186d700 1 -- 172.21.15.142:0/4164634270 >> 172.21.15.142:0/4164634270 conn(0x7fd86c0fc110 msgr2=0x7fd86c104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:23.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.410+0000 7fd87186d700 1 -- 172.21.15.142:0/4164634270 shutdown_connections 2024-03-29T04:42:23.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:23.410+0000 7fd87186d700 1 -- 172.21.15.142:0/4164634270 wait complete. 2024-03-29T04:42:23.409 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 9 2024-03-29T04:42:24.030 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-29T04:42:24.537 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:24.537 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:24.537 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:24.538 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:24 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/4164634270' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:42:24.538 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:24.538 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:24.538 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:24 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] up:boot 2024-03-29T04:42:24.538 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:24 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:42:24.538 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T04:42:24.538 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:24 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:42:24.538 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:24 smithi152 ceph-mon[37177]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 255 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-03-29T04:42:24.553 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:24.671 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:24.672 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:24.672 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:24.672 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:24 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/4164634270' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:42:24.672 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:24.672 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:24.672 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:24 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] up:boot 2024-03-29T04:42:24.672 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:24 smithi142 ceph-mon[29027]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:42:24.672 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T04:42:24.672 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:24 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:42:24.673 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:24 smithi142 ceph-mon[29027]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 255 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-03-29T04:42:26.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.436+0000 7f364d07e700 1 -- 172.21.15.142:0/2300059886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648101640 msgr2=0x7f3648103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:26.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.436+0000 7f364d07e700 1 --2- 172.21.15.142:0/2300059886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648101640 0x7f3648103a30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3638009910 tx=0x7f3638009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:26.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.436+0000 7f364d07e700 1 -- 172.21.15.142:0/2300059886 shutdown_connections 2024-03-29T04:42:26.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.436+0000 7f364d07e700 1 --2- 172.21.15.142:0/2300059886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3648103f70 0x7f3648106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:26.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.436+0000 7f364d07e700 1 --2- 172.21.15.142:0/2300059886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648101640 0x7f3648103a30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:26.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.436+0000 7f364d07e700 1 -- 172.21.15.142:0/2300059886 >> 172.21.15.142:0/2300059886 conn(0x7f36480fb030 msgr2=0x7f36480fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:26.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.436+0000 7f364d07e700 1 -- 172.21.15.142:0/2300059886 shutdown_connections 2024-03-29T04:42:26.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.436+0000 7f364d07e700 1 -- 172.21.15.142:0/2300059886 wait complete. 2024-03-29T04:42:26.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.436+0000 7f364d07e700 1 Processor -- start 2024-03-29T04:42:26.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f364d07e700 1 -- start start 2024-03-29T04:42:26.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f364d07e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648101640 0x7f3648196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f364d07e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3648103f70 0x7f3648196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f364d07e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3648196e70 con 0x7f3648101640 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f364d07e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3648196fb0 con 0x7f3648103f70 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f3646d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648101640 0x7f3648196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f3646d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648101640 0x7f3648196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:50766/0 (socket says 172.21.15.142:50766) 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f3646d9d700 1 -- 172.21.15.142:0/1541229155 learned_addr learned my addr 172.21.15.142:0/1541229155 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f364659c700 1 --2- 172.21.15.142:0/1541229155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3648103f70 0x7f3648196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f3646d9d700 1 -- 172.21.15.142:0/1541229155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3648103f70 msgr2=0x7f3648196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f3646d9d700 1 --2- 172.21.15.142:0/1541229155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3648103f70 0x7f3648196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f3646d9d700 1 -- 172.21.15.142:0/1541229155 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36380095f0 con 0x7f3648101640 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f3646d9d700 1 --2- 172.21.15.142:0/1541229155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648101640 0x7f3648196310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3638006010 tx=0x7f363800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.437+0000 7f362ffff700 1 -- 172.21.15.142:0/1541229155 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f363801d070 con 0x7f3648101640 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.438+0000 7f362ffff700 1 -- 172.21.15.142:0/1541229155 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f363800f460 con 0x7f3648101640 2024-03-29T04:42:26.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.438+0000 7f364d07e700 1 -- 172.21.15.142:0/1541229155 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364819ba00 con 0x7f3648101640 2024-03-29T04:42:26.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.438+0000 7f362ffff700 1 -- 172.21.15.142:0/1541229155 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3638021640 con 0x7f3648101640 2024-03-29T04:42:26.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.438+0000 7f364d07e700 1 -- 172.21.15.142:0/1541229155 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364819bef0 con 0x7f3648101640 2024-03-29T04:42:26.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.439+0000 7f364d07e700 1 -- 172.21.15.142:0/1541229155 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3648190540 con 0x7f3648101640 2024-03-29T04:42:26.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.439+0000 7f362ffff700 1 -- 172.21.15.142:0/1541229155 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f363800fab0 con 0x7f3648101640 2024-03-29T04:42:26.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.439+0000 7f362ffff700 1 --2- 172.21.15.142:0/1541229155 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f363006c4e0 0x7f363006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:26.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.440+0000 7f362ffff700 1 -- 172.21.15.142:0/1541229155 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f363808cc50 con 0x7f3648101640 2024-03-29T04:42:26.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.440+0000 7f364659c700 1 --2- 172.21.15.142:0/1541229155 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f363006c4e0 0x7f363006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:26.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.440+0000 7f364659c700 1 --2- 172.21.15.142:0/1541229155 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f363006c4e0 0x7f363006e9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f363c005950 tx=0x7f363c00a300 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:26.441 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.445+0000 7f362ffff700 1 -- 172.21.15.142:0/1541229155 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f363805ae60 con 0x7f3648101640 2024-03-29T04:42:26.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:26 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] up:standby-replay 2024-03-29T04:42:26.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:26 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] up:active 2024-03-29T04:42:26.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:26 smithi142 ceph-mon[29027]: Dropping low affinity active daemon mds.cephfs.smithi142.wgmndr in favor of higher affinity standby. 2024-03-29T04:42:26.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:26 smithi142 ceph-mon[29027]: Replacing daemon mds.cephfs.smithi142.wgmndr as rank 0 with standby daemon mds.cephfs.smithi142.bmkjzf 2024-03-29T04:42:26.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:26 smithi142 ceph-mon[29027]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-29T04:42:26.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:26 smithi142 ceph-mon[29027]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:42:26.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:26 smithi142 ceph-mon[29027]: osdmap e47: 6 total, 6 up, 6 in 2024-03-29T04:42:26.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:26 smithi142 ceph-mon[29027]: fsmap cephfs:2/2 {0=cephfs.smithi142.bmkjzf=up:replay,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby-replay 2024-03-29T04:42:26.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:26 smithi142 ceph-mon[29027]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-03-29T04:42:26.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:26 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:26.618 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:26 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:26.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.688+0000 7f364d07e700 1 -- 172.21.15.142:0/1541229155 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f364802cc70 con 0x7f3648101640 2024-03-29T04:42:26.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.689+0000 7f362ffff700 1 -- 172.21.15.142:0/1541229155 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4773 (secure 0 0 0) 0x7f363805a9f0 con 0x7f3648101640 2024-03-29T04:42:26.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.691+0000 7f364d07e700 1 -- 172.21.15.142:0/1541229155 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f363006c4e0 msgr2=0x7f363006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:26.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.691+0000 7f364d07e700 1 --2- 172.21.15.142:0/1541229155 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f363006c4e0 0x7f363006e9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f363c005950 tx=0x7f363c00a300 comp rx=0 tx=0).stop 2024-03-29T04:42:26.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.691+0000 7f364d07e700 1 -- 172.21.15.142:0/1541229155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648101640 msgr2=0x7f3648196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:26.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.691+0000 7f364d07e700 1 --2- 172.21.15.142:0/1541229155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648101640 0x7f3648196310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3638006010 tx=0x7f363800bd70 comp rx=0 tx=0).stop 2024-03-29T04:42:26.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.691+0000 7f364d07e700 1 -- 172.21.15.142:0/1541229155 shutdown_connections 2024-03-29T04:42:26.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.691+0000 7f364d07e700 1 --2- 172.21.15.142:0/1541229155 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f363006c4e0 0x7f363006e9a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:26.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.691+0000 7f364d07e700 1 --2- 172.21.15.142:0/1541229155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648101640 0x7f3648196310 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:26.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.691+0000 7f364d07e700 1 --2- 172.21.15.142:0/1541229155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3648103f70 0x7f3648196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:26.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.691+0000 7f364d07e700 1 -- 172.21.15.142:0/1541229155 >> 172.21.15.142:0/1541229155 conn(0x7f36480fb030 msgr2=0x7f36480fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:26.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.692+0000 7f364d07e700 1 -- 172.21.15.142:0/1541229155 shutdown_connections 2024-03-29T04:42:26.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:26.692+0000 7f364d07e700 1 -- 172.21.15.142:0/1541229155 wait complete. 2024-03-29T04:42:26.690 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 13 2024-03-29T04:42:26.700 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:42:26.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:26 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] up:standby-replay 2024-03-29T04:42:26.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:26 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] up:active 2024-03-29T04:42:26.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:26 smithi152 ceph-mon[37177]: Dropping low affinity active daemon mds.cephfs.smithi142.wgmndr in favor of higher affinity standby. 2024-03-29T04:42:26.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:26 smithi152 ceph-mon[37177]: Replacing daemon mds.cephfs.smithi142.wgmndr as rank 0 with standby daemon mds.cephfs.smithi142.bmkjzf 2024-03-29T04:42:26.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:26 smithi152 ceph-mon[37177]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-29T04:42:26.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:26 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:42:26.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:26 smithi152 ceph-mon[37177]: osdmap e47: 6 total, 6 up, 6 in 2024-03-29T04:42:26.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:26 smithi152 ceph-mon[37177]: fsmap cephfs:2/2 {0=cephfs.smithi142.bmkjzf=up:replay,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby-replay 2024-03-29T04:42:26.788 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:26 smithi152 ceph-mon[37177]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-03-29T04:42:26.788 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:26 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:26.788 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:26 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:27.570 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:27 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] up:boot 2024-03-29T04:42:27.571 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:27 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] up:resolve 2024-03-29T04:42:27.571 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:27 smithi142 ceph-mon[29027]: fsmap cephfs:2/2 {0=cephfs.smithi142.bmkjzf=up:resolve,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby-replay 1 up:standby 2024-03-29T04:42:27.571 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:27 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:42:27.571 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:27 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1541229155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-29T04:42:27.593 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-29T04:42:27.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:27 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] up:boot 2024-03-29T04:42:27.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:27 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] up:resolve 2024-03-29T04:42:27.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:27 smithi152 ceph-mon[37177]: fsmap cephfs:2/2 {0=cephfs.smithi142.bmkjzf=up:resolve,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby-replay 1 up:standby 2024-03-29T04:42:27.788 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:27 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:42:27.788 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:27 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1541229155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-29T04:42:27.993 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:28.688 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:28 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] up:reconnect 2024-03-29T04:42:28.688 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:28 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] up:standby-replay 2024-03-29T04:42:28.688 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:28 smithi142 ceph-mon[29027]: fsmap cephfs:2/2 {0=cephfs.smithi142.bmkjzf=up:reconnect,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby-replay 1 up:standby 2024-03-29T04:42:28.688 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:28 smithi142 ceph-mon[29027]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 716 B/s wr, 5 op/s 2024-03-29T04:42:28.688 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:28 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:28.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:28 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] up:reconnect 2024-03-29T04:42:28.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:28 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] up:standby-replay 2024-03-29T04:42:28.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:28 smithi152 ceph-mon[37177]: fsmap cephfs:2/2 {0=cephfs.smithi142.bmkjzf=up:reconnect,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby-replay 1 up:standby 2024-03-29T04:42:28.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:28 smithi152 ceph-mon[37177]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 716 B/s wr, 5 op/s 2024-03-29T04:42:28.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:28 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:29.776 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:29 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] up:rejoin 2024-03-29T04:42:29.776 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:29 smithi142 ceph-mon[29027]: fsmap cephfs:2/2 {0=cephfs.smithi142.bmkjzf=up:rejoin,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby-replay 1 up:standby 2024-03-29T04:42:29.776 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:29 smithi142 ceph-mon[29027]: daemon mds.cephfs.smithi142.bmkjzf is now active in filesystem cephfs as rank 0 2024-03-29T04:42:29.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:29 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] up:rejoin 2024-03-29T04:42:29.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:29 smithi152 ceph-mon[37177]: fsmap cephfs:2/2 {0=cephfs.smithi142.bmkjzf=up:rejoin,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby-replay 1 up:standby 2024-03-29T04:42:29.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:29 smithi152 ceph-mon[37177]: daemon mds.cephfs.smithi142.bmkjzf is now active in filesystem cephfs as rank 0 2024-03-29T04:42:30.526 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:30 smithi142 ceph-mon[29027]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-29T04:42:30.526 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:30 smithi142 ceph-mon[29027]: Cluster is now healthy 2024-03-29T04:42:30.526 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:30 smithi142 ceph-mon[29027]: mds.? [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] up:active 2024-03-29T04:42:30.526 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:30 smithi142 ceph-mon[29027]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby-replay 1 up:standby 2024-03-29T04:42:30.526 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:30 smithi142 ceph-mon[29027]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:42:30.526 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:30 smithi142 ceph-mon[29027]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-29T04:42:30.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:30 smithi152 ceph-mon[37177]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-29T04:42:30.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:30 smithi152 ceph-mon[37177]: Cluster is now healthy 2024-03-29T04:42:30.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:30 smithi152 ceph-mon[37177]: mds.? [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] up:active 2024-03-29T04:42:30.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:30 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 1 up:standby-replay 1 up:standby 2024-03-29T04:42:30.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:30 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:42:30.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:30 smithi152 ceph-mon[37177]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-29T04:42:31.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.348+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/1192151484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4101dc0 msgr2=0x7f0dd4102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:31.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.348+0000 7f0dd9b6d700 1 --2- 172.21.15.142:0/1192151484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4101dc0 0x7f0dd4102220 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009910 tx=0x7f0dbc009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:31.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.348+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/1192151484 shutdown_connections 2024-03-29T04:42:31.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.348+0000 7f0dd9b6d700 1 --2- 172.21.15.142:0/1192151484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4101dc0 0x7f0dd4102220 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:31.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.348+0000 7f0dd9b6d700 1 --2- 172.21.15.142:0/1192151484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd4100bc0 0x7f0dd4100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:31.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.348+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/1192151484 >> 172.21.15.142:0/1192151484 conn(0x7f0dd40fc140 msgr2=0x7f0dd40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:31.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.348+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/1192151484 shutdown_connections 2024-03-29T04:42:31.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.348+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/1192151484 wait complete. 2024-03-29T04:42:31.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd9b6d700 1 Processor -- start 2024-03-29T04:42:31.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd9b6d700 1 -- start start 2024-03-29T04:42:31.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd9b6d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd4100bc0 0x7f0dd4196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:31.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd9b6d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4101dc0 0x7f0dd41968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:31.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd9b6d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd4196e40 con 0x7f0dd4101dc0 2024-03-29T04:42:31.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd9b6d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd4196f80 con 0x7f0dd4100bc0 2024-03-29T04:42:31.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4101dc0 0x7f0dd41968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:31.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4101dc0 0x7f0dd41968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33858/0 (socket says 172.21.15.142:33858) 2024-03-29T04:42:31.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd2ffd700 1 -- 172.21.15.142:0/3044061686 learned_addr learned my addr 172.21.15.142:0/3044061686 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:31.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd37fe700 1 --2- 172.21.15.142:0/3044061686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd4100bc0 0x7f0dd4196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:31.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd2ffd700 1 -- 172.21.15.142:0/3044061686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd4100bc0 msgr2=0x7f0dd4196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:31.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd2ffd700 1 --2- 172.21.15.142:0/3044061686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd4100bc0 0x7f0dd4196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:31.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.349+0000 7f0dd2ffd700 1 -- 172.21.15.142:0/3044061686 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc0095f0 con 0x7f0dd4101dc0 2024-03-29T04:42:31.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.350+0000 7f0dd2ffd700 1 --2- 172.21.15.142:0/3044061686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4101dc0 0x7f0dd41968b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00bd70 tx=0x7f0dbc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:31.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.350+0000 7f0dd0ff9700 1 -- 172.21.15.142:0/3044061686 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dbc01d070 con 0x7f0dd4101dc0 2024-03-29T04:42:31.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.350+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/3044061686 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd419b9e0 con 0x7f0dd4101dc0 2024-03-29T04:42:31.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.350+0000 7f0dd0ff9700 1 -- 172.21.15.142:0/3044061686 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0dbc022470 con 0x7f0dd4101dc0 2024-03-29T04:42:31.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.350+0000 7f0dd0ff9700 1 -- 172.21.15.142:0/3044061686 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dbc00f670 con 0x7f0dd4101dc0 2024-03-29T04:42:31.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.350+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/3044061686 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd419bea0 con 0x7f0dd4101dc0 2024-03-29T04:42:31.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.351+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/3044061686 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd4109720 con 0x7f0dd4101dc0 2024-03-29T04:42:31.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.351+0000 7f0dd0ff9700 1 -- 172.21.15.142:0/3044061686 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0dbc022a50 con 0x7f0dd4101dc0 2024-03-29T04:42:31.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.352+0000 7f0dd0ff9700 1 --2- 172.21.15.142:0/3044061686 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0dc006c4e0 0x7f0dc006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:31.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.352+0000 7f0dd0ff9700 1 -- 172.21.15.142:0/3044061686 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0dbc08cf40 con 0x7f0dd4101dc0 2024-03-29T04:42:31.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.352+0000 7f0dd37fe700 1 --2- 172.21.15.142:0/3044061686 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0dc006c4e0 0x7f0dc006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:31.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.352+0000 7f0dd37fe700 1 --2- 172.21.15.142:0/3044061686 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0dc006c4e0 0x7f0dc006e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4101c20 tx=0x7f0dc4009040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:31.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.356+0000 7f0dd0ff9700 1 -- 172.21.15.142:0/3044061686 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0dbc0c00e0 con 0x7f0dd4101dc0 2024-03-29T04:42:31.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.655+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/3044061686 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f0dd419c510 con 0x7f0dd4101dc0 2024-03-29T04:42:31.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.655+0000 7f0dd0ff9700 1 -- 172.21.15.142:0/3044061686 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+83 (secure 0 0 0) 0x7f0dbc027070 con 0x7f0dd4101dc0 2024-03-29T04:42:31.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.658+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/3044061686 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0dc006c4e0 msgr2=0x7f0dc006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:31.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.658+0000 7f0dd9b6d700 1 --2- 172.21.15.142:0/3044061686 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0dc006c4e0 0x7f0dc006e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4101c20 tx=0x7f0dc4009040 comp rx=0 tx=0).stop 2024-03-29T04:42:31.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.658+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/3044061686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4101dc0 msgr2=0x7f0dd41968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:31.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.658+0000 7f0dd9b6d700 1 --2- 172.21.15.142:0/3044061686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4101dc0 0x7f0dd41968b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00bd70 tx=0x7f0dbc00be50 comp rx=0 tx=0).stop 2024-03-29T04:42:31.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.658+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/3044061686 shutdown_connections 2024-03-29T04:42:31.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.658+0000 7f0dd9b6d700 1 --2- 172.21.15.142:0/3044061686 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0dc006c4e0 0x7f0dc006e9a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:31.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.658+0000 7f0dd9b6d700 1 --2- 172.21.15.142:0/3044061686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd4100bc0 0x7f0dd4196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:31.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.658+0000 7f0dd9b6d700 1 --2- 172.21.15.142:0/3044061686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0dd4101dc0 0x7f0dd41968b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:31.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.658+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/3044061686 >> 172.21.15.142:0/3044061686 conn(0x7f0dd40fc140 msgr2=0x7f0dd4104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:31.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.658+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/3044061686 shutdown_connections 2024-03-29T04:42:31.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:31.658+0000 7f0dd9b6d700 1 -- 172.21.15.142:0/3044061686 wait complete. 2024-03-29T04:42:31.665 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:42:32.260 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-29T04:42:32.272 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:42:32.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.396+0000 7f879faa4640 1 -- 172.21.15.142:0/3674611273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87980ff000 msgr2=0x7f87980ff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:32.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.396+0000 7f879faa4640 1 --2- 172.21.15.142:0/3674611273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87980ff000 0x7f87980ff420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8788009880 tx=0x7f878802bce0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.396+0000 7f879faa4640 1 -- 172.21.15.142:0/3674611273 shutdown_connections 2024-03-29T04:42:32.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.396+0000 7f879faa4640 1 --2- 172.21.15.142:0/3674611273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87981001f0 0x7f8798100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.396+0000 7f879faa4640 1 --2- 172.21.15.142:0/3674611273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87980ff000 0x7f87980ff420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.396+0000 7f879faa4640 1 -- 172.21.15.142:0/3674611273 >> 172.21.15.142:0/3674611273 conn(0x7f87980fa570 msgr2=0x7f87980fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:32.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.396+0000 7f879faa4640 1 -- 172.21.15.142:0/3674611273 shutdown_connections 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.396+0000 7f879faa4640 1 -- 172.21.15.142:0/3674611273 wait complete. 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.397+0000 7f879faa4640 1 Processor -- start 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.397+0000 7f879faa4640 1 -- start start 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.397+0000 7f879faa4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87980ff000 0x7f8798068fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.397+0000 7f879faa4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87981001f0 0x7f879819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.397+0000 7f879faa4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879819abd0 con 0x7f87980ff000 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.397+0000 7f879faa4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879819ad40 con 0x7f87981001f0 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.397+0000 7f879d840640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87980ff000 0x7f8798068fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.397+0000 7f879d840640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87980ff000 0x7f8798068fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33868/0 (socket says 172.21.15.142:33868) 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.397+0000 7f879d840640 1 -- 172.21.15.142:0/1213344717 learned_addr learned my addr 172.21.15.142:0/1213344717 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.397+0000 7f879d03f640 1 --2- 172.21.15.142:0/1213344717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87981001f0 0x7f879819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.398+0000 7f879d840640 1 -- 172.21.15.142:0/1213344717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87981001f0 msgr2=0x7f879819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.398+0000 7f879d840640 1 --2- 172.21.15.142:0/1213344717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87981001f0 0x7f879819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.398+0000 7f879d840640 1 -- 172.21.15.142:0/1213344717 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8788009560 con 0x7f87980ff000 2024-03-29T04:42:32.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.398+0000 7f879d840640 1 --2- 172.21.15.142:0/1213344717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87980ff000 0x7f8798068fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8788005ab0 tx=0x7f8788004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:32.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.398+0000 7f878effd640 1 -- 172.21.15.142:0/1213344717 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8788004420 con 0x7f87980ff000 2024-03-29T04:42:32.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.398+0000 7f878effd640 1 -- 172.21.15.142:0/1213344717 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f87880054b0 con 0x7f87980ff000 2024-03-29T04:42:32.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.398+0000 7f879faa4640 1 -- 172.21.15.142:0/1213344717 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879819afc0 con 0x7f87980ff000 2024-03-29T04:42:32.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.398+0000 7f878effd640 1 -- 172.21.15.142:0/1213344717 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8788003e90 con 0x7f87980ff000 2024-03-29T04:42:32.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.398+0000 7f879faa4640 1 -- 172.21.15.142:0/1213344717 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879819b440 con 0x7f87980ff000 2024-03-29T04:42:32.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.399+0000 7f879faa4640 1 -- 172.21.15.142:0/1213344717 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8798107d70 con 0x7f87980ff000 2024-03-29T04:42:32.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.399+0000 7f878effd640 1 -- 172.21.15.142:0/1213344717 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f87880035f0 con 0x7f87980ff000 2024-03-29T04:42:32.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.400+0000 7f878effd640 1 --2- 172.21.15.142:0/1213344717 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f878406c500 0x7f878406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:32.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.400+0000 7f878effd640 1 -- 172.21.15.142:0/1213344717 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f87880adbe0 con 0x7f87980ff000 2024-03-29T04:42:32.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.400+0000 7f879d03f640 1 --2- 172.21.15.142:0/1213344717 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f878406c500 0x7f878406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:32.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.400+0000 7f879d03f640 1 --2- 172.21.15.142:0/1213344717 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f878406c500 0x7f878406e9c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8798068d10 tx=0x7f8794005f90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:32.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.405+0000 7f878effd640 1 -- 172.21.15.142:0/1213344717 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8788078410 con 0x7f87980ff000 2024-03-29T04:42:32.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.588+0000 7f879faa4640 1 -- 172.21.15.142:0/1213344717 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f879819b7a0 con 0x7f87980ff000 2024-03-29T04:42:32.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.590+0000 7f878effd640 1 -- 172.21.15.142:0/1213344717 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15177 (secure 0 0 0) 0x7f878809b090 con 0x7f87980ff000 2024-03-29T04:42:32.587 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:42:32.587 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":47,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","created":"2024-03-29T04:35:07.701328+0000","modified":"2024-03-29T04:42:25.447308+0000","last_up_change":"2024-03-29T04:40:46.622246+0000","last_in_change":"2024-03-29T04:40:27.009053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:39:31.891786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-29T04:42:05.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-29T04:42:05.937588+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d908de1a-bec3-4238-959a-f3f550431ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6803","nonce":3710621973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6805","nonce":3710621973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6809","nonce":3710621973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3710621973},{"type":"v1","addr":"172.21.15.142:6807","nonce":3710621973}]},"public_addr":"172.21.15.142:6803/3710621973","cluster_addr":"172.21.15.142:6805/3710621973","heartbeat_back_addr":"172.21.15.142:6809/3710621973","heartbeat_front_addr":"172.21.15.142:6807/3710621973","state":["exists","up"]},{"osd":1,"uuid":"717e7135-21ab-4d33-a81e-65da6d5186c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6811","nonce":1500472047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6813","nonce":1500472047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6817","nonce":1500472047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1500472047},{"type":"v1","addr":"172.21.15.142:6815","nonce":1500472047}]},"public_addr":"172.21.15.142:6811/1500472047","cluster_addr":"172.21.15.142:6813/1500472047","heartbeat_back_addr":"172.21.15.142:6817/1500472047","heartbeat_front_addr":"172.21.15.142:6815/1500472047","state":["exists","up"]},{"osd":2,"uuid":"977032d2-3c29-4c68-ae66-73d024ab94ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6819","nonce":1815862059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6821","nonce":1815862059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6825","nonce":1815862059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":1815862059},{"type":"v1","addr":"172.21.15.142:6823","nonce":1815862059}]},"public_addr":"172.21.15.142:6819/1815862059","cluster_addr":"172.21.15.142:6821/1815862059","heartbeat_back_addr":"172.21.15.142:6825/1815862059","heartbeat_front_addr":"172.21.15.142:6823/1815862059","state":["exists","up"]},{"osd":3,"uuid":"39ae7173-3a83-4ae1-9866-5a0da908171c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6801","nonce":3854389166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6803","nonce":3854389166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6807","nonce":3854389166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":3854389166},{"type":"v1","addr":"172.21.15.152:6805","nonce":3854389166}]},"public_addr":"172.21.15.152:6801/3854389166","cluster_addr":"172.21.15.152:6803/3854389166","heartbeat_back_addr":"172.21.15.152:6807/3854389166","heartbeat_front_addr":"172.21.15.152:6805/3854389166","state":["exists","up"]},{"osd":4,"uuid":"f7b5fab7-0f30-4756-a481-15ad64b9e691","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6809","nonce":2813629639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6811","nonce":2813629639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6815","nonce":2813629639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2813629639},{"type":"v1","addr":"172.21.15.152:6813","nonce":2813629639}]},"public_addr":"172.21.15.152:6809/2813629639","cluster_addr":"172.21.15.152:6811/2813629639","heartbeat_back_addr":"172.21.15.152:6815/2813629639","heartbeat_front_addr":"172.21.15.152:6813/2813629639","state":["exists","up"]},{"osd":5,"uuid":"8ed3caba-b201-4ad7-b055-3dc40610ab1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6817","nonce":48435269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6819","nonce":48435269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6823","nonce":48435269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":48435269},{"type":"v1","addr":"172.21.15.152:6821","nonce":48435269}]},"public_addr":"172.21.15.152:6817/48435269","cluster_addr":"172.21.15.152:6819/48435269","heartbeat_back_addr":"172.21.15.152:6823/48435269","heartbeat_front_addr":"172.21.15.152:6821/48435269","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:38:33.605811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:00.911691+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:28.052973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:51.974988+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:18.104282+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:44.747877+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6827/2071436089":"2024-03-30T04:42:25.447288+0000","172.21.15.142:6800/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/1935271872":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/3751399855":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/2666008640":"2024-03-30T04:35:33.215645+0000","172.21.15.142:6800/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6801/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6826/2071436089":"2024-03-30T04:42:25.447288+0000","172.21.15.142:0/966342870":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/390665991":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/2545215192":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/672130862":"2024-03-30T04:37:17.579177+0000","172.21.15.142:6801/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/586295950":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/3338740178":"2024-03-30T04:37:17.579177+0000","172.21.15.142:6800/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:6801/779416791":"2024-03-30T04:37:17.579177+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:42:32.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.592+0000 7f879faa4640 1 -- 172.21.15.142:0/1213344717 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f878406c500 msgr2=0x7f878406e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:32.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.592+0000 7f879faa4640 1 --2- 172.21.15.142:0/1213344717 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f878406c500 0x7f878406e9c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8798068d10 tx=0x7f8794005f90 comp rx=0 tx=0).stop 2024-03-29T04:42:32.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.592+0000 7f879faa4640 1 -- 172.21.15.142:0/1213344717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87980ff000 msgr2=0x7f8798068fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:32.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.592+0000 7f879faa4640 1 --2- 172.21.15.142:0/1213344717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87980ff000 0x7f8798068fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8788005ab0 tx=0x7f8788004e30 comp rx=0 tx=0).stop 2024-03-29T04:42:32.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.592+0000 7f879faa4640 1 -- 172.21.15.142:0/1213344717 shutdown_connections 2024-03-29T04:42:32.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.592+0000 7f879faa4640 1 --2- 172.21.15.142:0/1213344717 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f878406c500 0x7f878406e9c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.592+0000 7f879faa4640 1 --2- 172.21.15.142:0/1213344717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87981001f0 0x7f879819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.592+0000 7f879faa4640 1 --2- 172.21.15.142:0/1213344717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87980ff000 0x7f8798068fa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.592+0000 7f879faa4640 1 -- 172.21.15.142:0/1213344717 >> 172.21.15.142:0/1213344717 conn(0x7f87980fa570 msgr2=0x7f8798103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:32.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.592+0000 7f879faa4640 1 -- 172.21.15.142:0/1213344717 shutdown_connections 2024-03-29T04:42:32.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.592+0000 7f879faa4640 1 -- 172.21.15.142:0/1213344717 wait complete. 2024-03-29T04:42:32.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:39:31.891786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-29T04:42:05.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-29T04:42:05.937588+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:42:32.600 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:42:32.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.712+0000 7fe802d0a640 1 -- 172.21.15.142:0/1574892998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7fc100180 msgr2=0x7fe7fc100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:32.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.712+0000 7fe802d0a640 1 --2- 172.21.15.142:0/1574892998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7fc100180 0x7fe7fc100600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009880 tx=0x7fe7e402bce0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.712+0000 7fe802d0a640 1 -- 172.21.15.142:0/1574892998 shutdown_connections 2024-03-29T04:42:32.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.712+0000 7fe802d0a640 1 --2- 172.21.15.142:0/1574892998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7fc100180 0x7fe7fc100600 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.712+0000 7fe802d0a640 1 --2- 172.21.15.142:0/1574892998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc0fef90 0x7fe7fc0ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.712+0000 7fe802d0a640 1 -- 172.21.15.142:0/1574892998 >> 172.21.15.142:0/1574892998 conn(0x7fe7fc0fa4e0 msgr2=0x7fe7fc0fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:32.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.712+0000 7fe802d0a640 1 -- 172.21.15.142:0/1574892998 shutdown_connections 2024-03-29T04:42:32.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.712+0000 7fe802d0a640 1 -- 172.21.15.142:0/1574892998 wait complete. 2024-03-29T04:42:32.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe802d0a640 1 Processor -- start 2024-03-29T04:42:32.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe802d0a640 1 -- start start 2024-03-29T04:42:32.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe802d0a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc0fef90 0x7fe7fc067ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:32.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe802d0a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7fc100180 0x7fe7fc19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe802d0a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc19ac80 con 0x7fe7fc0fef90 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe802d0a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc19adc0 con 0x7fe7fc100180 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe800aa6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc0fef90 0x7fe7fc067ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe800aa6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc0fef90 0x7fe7fc067ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33876/0 (socket says 172.21.15.142:33876) 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe800aa6640 1 -- 172.21.15.142:0/855697526 learned_addr learned my addr 172.21.15.142:0/855697526 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe7fbfff640 1 --2- 172.21.15.142:0/855697526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7fc100180 0x7fe7fc19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe800aa6640 1 -- 172.21.15.142:0/855697526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7fc100180 msgr2=0x7fe7fc19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe800aa6640 1 --2- 172.21.15.142:0/855697526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7fc100180 0x7fe7fc19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe800aa6640 1 -- 172.21.15.142:0/855697526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e4009560 con 0x7fe7fc0fef90 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.713+0000 7fe800aa6640 1 --2- 172.21.15.142:0/855697526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc0fef90 0x7fe7fc067ec0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400eb40 tx=0x7fe7f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.714+0000 7fe7f9ffb640 1 -- 172.21.15.142:0/855697526 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f4003970 con 0x7fe7fc0fef90 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.714+0000 7fe802d0a640 1 -- 172.21.15.142:0/855697526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc19b070 con 0x7fe7fc0fef90 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.714+0000 7fe7f9ffb640 1 -- 172.21.15.142:0/855697526 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe7f4004030 con 0x7fe7fc0fef90 2024-03-29T04:42:32.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.714+0000 7fe7f9ffb640 1 -- 172.21.15.142:0/855697526 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f4019780 con 0x7fe7fc0fef90 2024-03-29T04:42:32.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.714+0000 7fe802d0a640 1 -- 172.21.15.142:0/855697526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc19b5a0 con 0x7fe7fc0fef90 2024-03-29T04:42:32.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.715+0000 7fe802d0a640 1 -- 172.21.15.142:0/855697526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc107b70 con 0x7fe7fc0fef90 2024-03-29T04:42:32.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.715+0000 7fe7f9ffb640 1 -- 172.21.15.142:0/855697526 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe7f4003ad0 con 0x7fe7fc0fef90 2024-03-29T04:42:32.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.716+0000 7fe7f9ffb640 1 --2- 172.21.15.142:0/855697526 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe7e806c450 0x7fe7e806e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:32.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.716+0000 7fe7f9ffb640 1 -- 172.21.15.142:0/855697526 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe7f4014070 con 0x7fe7fc0fef90 2024-03-29T04:42:32.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.716+0000 7fe7fbfff640 1 --2- 172.21.15.142:0/855697526 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe7e806c450 0x7fe7e806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:32.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.716+0000 7fe7fbfff640 1 --2- 172.21.15.142:0/855697526 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe7e806c450 0x7fe7e806e910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4006010 tx=0x7fe7e402c700 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:32.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.721+0000 7fe7f9ffb640 1 -- 172.21.15.142:0/855697526 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe7f4056a70 con 0x7fe7fc0fef90 2024-03-29T04:42:32.831 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:32.831 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:32.831 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:32.831 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:42:32.831 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:32.831 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:32 smithi142 ceph-mon[29027]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-29T04:42:32.831 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:32 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3044061686' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-29T04:42:32.831 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:42:32.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.895+0000 7fe802d0a640 1 -- 172.21.15.142:0/855697526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7fc19b820 con 0x7fe7fc0fef90 2024-03-29T04:42:32.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.896+0000 7fe7f9ffb640 1 -- 172.21.15.142:0/855697526 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7f405a090 con 0x7fe7fc0fef90 2024-03-29T04:42:32.893 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 1 2024-03-29T04:42:32.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.898+0000 7fe802d0a640 1 -- 172.21.15.142:0/855697526 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe7e806c450 msgr2=0x7fe7e806e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:32.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.898+0000 7fe802d0a640 1 --2- 172.21.15.142:0/855697526 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe7e806c450 0x7fe7e806e910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4006010 tx=0x7fe7e402c700 comp rx=0 tx=0).stop 2024-03-29T04:42:32.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.898+0000 7fe802d0a640 1 -- 172.21.15.142:0/855697526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc0fef90 msgr2=0x7fe7fc067ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:32.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.898+0000 7fe802d0a640 1 --2- 172.21.15.142:0/855697526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc0fef90 0x7fe7fc067ec0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400eb40 tx=0x7fe7f400c2d0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.898+0000 7fe802d0a640 1 -- 172.21.15.142:0/855697526 shutdown_connections 2024-03-29T04:42:32.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.898+0000 7fe802d0a640 1 --2- 172.21.15.142:0/855697526 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe7e806c450 0x7fe7e806e910 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.898+0000 7fe802d0a640 1 --2- 172.21.15.142:0/855697526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7fc100180 0x7fe7fc19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.898+0000 7fe802d0a640 1 --2- 172.21.15.142:0/855697526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc0fef90 0x7fe7fc067ec0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:32.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.898+0000 7fe802d0a640 1 -- 172.21.15.142:0/855697526 >> 172.21.15.142:0/855697526 conn(0x7fe7fc0fa4e0 msgr2=0x7fe7fc1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:32.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.898+0000 7fe802d0a640 1 -- 172.21.15.142:0/855697526 shutdown_connections 2024-03-29T04:42:32.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:32.898+0000 7fe802d0a640 1 -- 172.21.15.142:0/855697526 wait complete. 2024-03-29T04:42:32.906 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-29T04:42:33.002 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:33.002 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:33.002 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:33.002 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:42:33.002 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:33.002 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:32 smithi152 ceph-mon[37177]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-29T04:42:33.002 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:32 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3044061686' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-29T04:42:33.003 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:42:33.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.016+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/1943287975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c041000b0 msgr2=0x7f8c041004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.016+0000 7f8c0a3ce640 1 --2- 172.21.15.142:0/1943287975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c041000b0 0x7f8c041004d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009880 tx=0x7f8bf402bce0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.016+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/1943287975 shutdown_connections 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.016+0000 7f8c0a3ce640 1 --2- 172.21.15.142:0/1943287975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c041012a0 0x7f8c04101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.016+0000 7f8c0a3ce640 1 --2- 172.21.15.142:0/1943287975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c041000b0 0x7f8c041004d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.016+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/1943287975 >> 172.21.15.142:0/1943287975 conn(0x7f8c040fb620 msgr2=0x7f8c040fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.016+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/1943287975 shutdown_connections 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.016+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/1943287975 wait complete. 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.016+0000 7f8c0a3ce640 1 Processor -- start 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c0a3ce640 1 -- start start 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c0a3ce640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c041000b0 0x7f8c04195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c0a3ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c041012a0 0x7f8c04195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c0a3ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04196490 con 0x7f8c041012a0 2024-03-29T04:42:33.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c0a3ce640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04196600 con 0x7f8c041000b0 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c037fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c041012a0 0x7f8c04195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c037fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c041012a0 0x7f8c04195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33890/0 (socket says 172.21.15.142:33890) 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c037fe640 1 -- 172.21.15.142:0/4201332785 learned_addr learned my addr 172.21.15.142:0/4201332785 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c03fff640 1 --2- 172.21.15.142:0/4201332785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c041000b0 0x7f8c04195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c037fe640 1 -- 172.21.15.142:0/4201332785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c041000b0 msgr2=0x7f8c04195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c037fe640 1 --2- 172.21.15.142:0/4201332785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c041000b0 0x7f8c04195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c037fe640 1 -- 172.21.15.142:0/4201332785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf4009560 con 0x7f8c041012a0 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c037fe640 1 --2- 172.21.15.142:0/4201332785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c041012a0 0x7f8c04195ec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800d8d0 tx=0x7f8bf800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c017fa640 1 -- 172.21.15.142:0/4201332785 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf8009c00 con 0x7f8c041012a0 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.017+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/4201332785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0419b090 con 0x7f8c041012a0 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.018+0000 7f8c017fa640 1 -- 172.21.15.142:0/4201332785 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8bf80044d0 con 0x7f8c041012a0 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.018+0000 7f8c017fa640 1 -- 172.21.15.142:0/4201332785 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf80049d0 con 0x7f8c041012a0 2024-03-29T04:42:33.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.018+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/4201332785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0419b5f0 con 0x7f8c041012a0 2024-03-29T04:42:33.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.018+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/4201332785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c04108de0 con 0x7f8c041012a0 2024-03-29T04:42:33.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.019+0000 7f8c017fa640 1 -- 172.21.15.142:0/4201332785 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f8bf8004020 con 0x7f8c041012a0 2024-03-29T04:42:33.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.019+0000 7f8c017fa640 1 --2- 172.21.15.142:0/4201332785 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8bec06c450 0x7f8bec06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:33.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.019+0000 7f8c03fff640 1 --2- 172.21.15.142:0/4201332785 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8bec06c450 0x7f8bec06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:33.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.019+0000 7f8c017fa640 1 -- 172.21.15.142:0/4201332785 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8bf808b490 con 0x7f8c041012a0 2024-03-29T04:42:33.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.020+0000 7f8c03fff640 1 --2- 172.21.15.142:0/4201332785 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8bec06c450 0x7f8bec06e910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4000c00 tx=0x7f8bf4005fd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:33.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.024+0000 7f8c017fa640 1 -- 172.21.15.142:0/4201332785 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8bf8055c90 con 0x7f8c041012a0 2024-03-29T04:42:33.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.198+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/4201332785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f8c0419ba70 con 0x7f8c041012a0 2024-03-29T04:42:33.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.198+0000 7f8c017fa640 1 -- 172.21.15.142:0/4201332785 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8bf80592b0 con 0x7f8c041012a0 2024-03-29T04:42:33.196 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 32 2024-03-29T04:42:33.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.200+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/4201332785 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8bec06c450 msgr2=0x7f8bec06e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:33.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.200+0000 7f8c0a3ce640 1 --2- 172.21.15.142:0/4201332785 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8bec06c450 0x7f8bec06e910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4000c00 tx=0x7f8bf4005fd0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.200+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/4201332785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c041012a0 msgr2=0x7f8c04195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:33.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.200+0000 7f8c0a3ce640 1 --2- 172.21.15.142:0/4201332785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c041012a0 0x7f8c04195ec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800d8d0 tx=0x7f8bf800dd90 comp rx=0 tx=0).stop 2024-03-29T04:42:33.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.201+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/4201332785 shutdown_connections 2024-03-29T04:42:33.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.201+0000 7f8c0a3ce640 1 --2- 172.21.15.142:0/4201332785 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8bec06c450 0x7f8bec06e910 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.201+0000 7f8c0a3ce640 1 --2- 172.21.15.142:0/4201332785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c041012a0 0x7f8c04195ec0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.201+0000 7f8c0a3ce640 1 --2- 172.21.15.142:0/4201332785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c041000b0 0x7f8c04195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.201+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/4201332785 >> 172.21.15.142:0/4201332785 conn(0x7f8c040fb620 msgr2=0x7f8c041044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:33.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.201+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/4201332785 shutdown_connections 2024-03-29T04:42:33.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.201+0000 7f8c0a3ce640 1 -- 172.21.15.142:0/4201332785 wait complete. 2024-03-29T04:42:33.210 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-29T04:42:33.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 -- 172.21.15.142:0/1639440299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06bc0fef90 msgr2=0x7f06bc0ff3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:33.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 --2- 172.21.15.142:0/1639440299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06bc0fef90 0x7f06bc0ff3b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009880 tx=0x7f06b802bce0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 -- 172.21.15.142:0/1639440299 shutdown_connections 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 --2- 172.21.15.142:0/1639440299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06bc100180 0x7f06bc100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 --2- 172.21.15.142:0/1639440299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06bc0fef90 0x7f06bc0ff3b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 -- 172.21.15.142:0/1639440299 >> 172.21.15.142:0/1639440299 conn(0x7f06bc0fa4e0 msgr2=0x7f06bc0fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 -- 172.21.15.142:0/1639440299 shutdown_connections 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 -- 172.21.15.142:0/1639440299 wait complete. 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 Processor -- start 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 -- start start 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06bc0fef90 0x7f06bc194880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06bc100180 0x7f06bc194dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc195300 con 0x7f06bc0fef90 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.326+0000 7f06c40f2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc195470 con 0x7f06bc100180 2024-03-29T04:42:33.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06c1e8e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06bc0fef90 0x7f06bc194880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:33.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06c1e8e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06bc0fef90 0x7f06bc194880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33910/0 (socket says 172.21.15.142:33910) 2024-03-29T04:42:33.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06c1e8e640 1 -- 172.21.15.142:0/1743294333 learned_addr learned my addr 172.21.15.142:0/1743294333 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:33.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06c168d640 1 --2- 172.21.15.142:0/1743294333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06bc100180 0x7f06bc194dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:33.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06c1e8e640 1 -- 172.21.15.142:0/1743294333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06bc100180 msgr2=0x7f06bc194dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:33.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06c1e8e640 1 --2- 172.21.15.142:0/1743294333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06bc100180 0x7f06bc194dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06c1e8e640 1 -- 172.21.15.142:0/1743294333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b8009560 con 0x7f06bc0fef90 2024-03-29T04:42:33.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06c1e8e640 1 --2- 172.21.15.142:0/1743294333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06bc0fef90 0x7f06bc194880 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f06b8000c00 tx=0x7f06b8004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:33.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06b2ffd640 1 -- 172.21.15.142:0/1743294333 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06b8004520 con 0x7f06bc0fef90 2024-03-29T04:42:33.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06b2ffd640 1 -- 172.21.15.142:0/1743294333 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f06b80054b0 con 0x7f06bc0fef90 2024-03-29T04:42:33.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06c40f2640 1 -- 172.21.15.142:0/1743294333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc199ee0 con 0x7f06bc0fef90 2024-03-29T04:42:33.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06b2ffd640 1 -- 172.21.15.142:0/1743294333 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06b8034440 con 0x7f06bc0fef90 2024-03-29T04:42:33.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.327+0000 7f06c40f2640 1 -- 172.21.15.142:0/1743294333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc19a360 con 0x7f06bc0fef90 2024-03-29T04:42:33.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.328+0000 7f06c40f2640 1 -- 172.21.15.142:0/1743294333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06bc107d00 con 0x7f06bc0fef90 2024-03-29T04:42:33.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.329+0000 7f06b2ffd640 1 -- 172.21.15.142:0/1743294333 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f06b8034670 con 0x7f06bc0fef90 2024-03-29T04:42:33.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.329+0000 7f06b2ffd640 1 --2- 172.21.15.142:0/1743294333 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f06a806c450 0x7f06a806e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:33.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.329+0000 7f06b2ffd640 1 -- 172.21.15.142:0/1743294333 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f06b80ade00 con 0x7f06bc0fef90 2024-03-29T04:42:33.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.329+0000 7f06c168d640 1 --2- 172.21.15.142:0/1743294333 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f06a806c450 0x7f06a806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:33.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.329+0000 7f06c168d640 1 --2- 172.21.15.142:0/1743294333 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f06a806c450 0x7f06a806e910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f06bc195d90 tx=0x7f06ac00a2d0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:33.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.334+0000 7f06b2ffd640 1 -- 172.21.15.142:0/1743294333 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f06b8078740 con 0x7f06bc0fef90 2024-03-29T04:42:33.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.517+0000 7f06c40f2640 1 -- 172.21.15.142:0/1743294333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f06bc19a550 con 0x7f06bc0fef90 2024-03-29T04:42:33.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.518+0000 7f06b2ffd640 1 -- 172.21.15.142:0/1743294333 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f06b804b0b0 con 0x7f06bc0fef90 2024-03-29T04:42:33.515 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 32 2024-03-29T04:42:33.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.520+0000 7f06c40f2640 1 -- 172.21.15.142:0/1743294333 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f06a806c450 msgr2=0x7f06a806e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:33.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.520+0000 7f06c40f2640 1 --2- 172.21.15.142:0/1743294333 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f06a806c450 0x7f06a806e910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f06bc195d90 tx=0x7f06ac00a2d0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.520+0000 7f06c40f2640 1 -- 172.21.15.142:0/1743294333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06bc0fef90 msgr2=0x7f06bc194880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:33.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.520+0000 7f06c40f2640 1 --2- 172.21.15.142:0/1743294333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06bc0fef90 0x7f06bc194880 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f06b8000c00 tx=0x7f06b8004f00 comp rx=0 tx=0).stop 2024-03-29T04:42:33.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.520+0000 7f06c40f2640 1 -- 172.21.15.142:0/1743294333 shutdown_connections 2024-03-29T04:42:33.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.520+0000 7f06c40f2640 1 --2- 172.21.15.142:0/1743294333 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f06a806c450 0x7f06a806e910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.520+0000 7f06c40f2640 1 --2- 172.21.15.142:0/1743294333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06bc100180 0x7f06bc194dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.520+0000 7f06c40f2640 1 --2- 172.21.15.142:0/1743294333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06bc0fef90 0x7f06bc194880 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.520+0000 7f06c40f2640 1 -- 172.21.15.142:0/1743294333 >> 172.21.15.142:0/1743294333 conn(0x7f06bc0fa4e0 msgr2=0x7f06bc1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:33.518 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.520+0000 7f06c40f2640 1 -- 172.21.15.142:0/1743294333 shutdown_connections 2024-03-29T04:42:33.518 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.520+0000 7f06c40f2640 1 -- 172.21.15.142:0/1743294333 wait complete. 2024-03-29T04:42:33.528 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:42:33.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.637+0000 7ff07c159640 1 -- 172.21.15.142:0/3109270637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0741001a0 msgr2=0x7ff074100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:33.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.637+0000 7ff07c159640 1 --2- 172.21.15.142:0/3109270637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0741001a0 0x7ff074100620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff064009880 tx=0x7ff06402bce0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.637+0000 7ff07c159640 1 -- 172.21.15.142:0/3109270637 shutdown_connections 2024-03-29T04:42:33.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.637+0000 7ff07c159640 1 --2- 172.21.15.142:0/3109270637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0741001a0 0x7ff074100620 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.637+0000 7ff07c159640 1 --2- 172.21.15.142:0/3109270637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0740fefb0 0x7ff0740ff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.637+0000 7ff07c159640 1 -- 172.21.15.142:0/3109270637 >> 172.21.15.142:0/3109270637 conn(0x7ff0740fa4e0 msgr2=0x7ff0740fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:33.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.638+0000 7ff07c159640 1 -- 172.21.15.142:0/3109270637 shutdown_connections 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.638+0000 7ff07c159640 1 -- 172.21.15.142:0/3109270637 wait complete. 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.638+0000 7ff07c159640 1 Processor -- start 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.638+0000 7ff07c159640 1 -- start start 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.638+0000 7ff07c159640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0740fefb0 0x7ff074195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.638+0000 7ff07c159640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0741001a0 0x7ff074195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.638+0000 7ff07c159640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff074196390 con 0x7ff0741001a0 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.638+0000 7ff07c159640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff074196500 con 0x7ff0740fefb0 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.638+0000 7ff0796f4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0741001a0 0x7ff074195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.638+0000 7ff0796f4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0741001a0 0x7ff074195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33920/0 (socket says 172.21.15.142:33920) 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.638+0000 7ff0796f4640 1 -- 172.21.15.142:0/2398717399 learned_addr learned my addr 172.21.15.142:0/2398717399 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.638+0000 7ff079ef5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0740fefb0 0x7ff074195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.639+0000 7ff0796f4640 1 -- 172.21.15.142:0/2398717399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0740fefb0 msgr2=0x7ff074195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:33.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.639+0000 7ff0796f4640 1 --2- 172.21.15.142:0/2398717399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0740fefb0 0x7ff074195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.639+0000 7ff0796f4640 1 -- 172.21.15.142:0/2398717399 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff064009560 con 0x7ff0741001a0 2024-03-29T04:42:33.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.639+0000 7ff0796f4640 1 --2- 172.21.15.142:0/2398717399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0741001a0 0x7ff074195dc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff064009230 tx=0x7ff06402cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:33.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.639+0000 7ff06affd640 1 -- 172.21.15.142:0/2398717399 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff064004420 con 0x7ff0741001a0 2024-03-29T04:42:33.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.639+0000 7ff07c159640 1 -- 172.21.15.142:0/2398717399 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07419af30 con 0x7ff0741001a0 2024-03-29T04:42:33.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.639+0000 7ff06affd640 1 -- 172.21.15.142:0/2398717399 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff0640054b0 con 0x7ff0741001a0 2024-03-29T04:42:33.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.639+0000 7ff06affd640 1 -- 172.21.15.142:0/2398717399 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff064034440 con 0x7ff0741001a0 2024-03-29T04:42:33.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.639+0000 7ff07c159640 1 -- 172.21.15.142:0/2398717399 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07419b400 con 0x7ff0741001a0 2024-03-29T04:42:33.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.640+0000 7ff07c159640 1 -- 172.21.15.142:0/2398717399 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff074107b90 con 0x7ff0741001a0 2024-03-29T04:42:33.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.640+0000 7ff06affd640 1 -- 172.21.15.142:0/2398717399 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff0640037d0 con 0x7ff0741001a0 2024-03-29T04:42:33.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.641+0000 7ff06affd640 1 --2- 172.21.15.142:0/2398717399 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7ff06006c510 0x7ff06006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:33.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.641+0000 7ff06affd640 1 -- 172.21.15.142:0/2398717399 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff0640adbf0 con 0x7ff0741001a0 2024-03-29T04:42:33.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.641+0000 7ff079ef5640 1 --2- 172.21.15.142:0/2398717399 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7ff06006c510 0x7ff06006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:33.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.641+0000 7ff079ef5640 1 --2- 172.21.15.142:0/2398717399 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7ff06006c510 0x7ff06006e9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff074100000 tx=0x7ff070005f90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:33.642 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.645+0000 7ff06affd640 1 -- 172.21.15.142:0/2398717399 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff0640784a0 con 0x7ff0741001a0 2024-03-29T04:42:33.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.857+0000 7ff07c159640 1 -- 172.21.15.142:0/2398717399 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff074107da0 con 0x7ff0741001a0 2024-03-29T04:42:33.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.858+0000 7ff06affd640 1 -- 172.21.15.142:0/2398717399 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4779 (secure 0 0 0) 0x7ff06407bac0 con 0x7ff0741001a0 2024-03-29T04:42:33.855 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:42:33.855 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:42:29.477923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14494,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14494":{"gid":14494,"name":"cephfs.smithi142.bmkjzf","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.142:6829/1941073852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1941073852},{"type":"v1","addr":"172.21.15.142:6829","nonce":1941073852}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14506":{"gid":14506,"name":"cephfs.smithi142.wgmndr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6827/1415100534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1415100534},{"type":"v1","addr":"172.21.15.142:6827","nonce":1415100534}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi152.uhebsi","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.152:6825/832699894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":832699894},{"type":"v1","addr":"172.21.15.152:6825","nonce":832699894}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24287":{"gid":24287,"name":"cephfs.smithi152.hiilum","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.152:6827/3411697908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3411697908},{"type":"v1","addr":"172.21.15.152:6827","nonce":3411697908}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T04:42:33.855 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 17 2024-03-29T04:42:33.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.860+0000 7ff07c159640 1 -- 172.21.15.142:0/2398717399 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7ff06006c510 msgr2=0x7ff06006e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:33.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.860+0000 7ff07c159640 1 --2- 172.21.15.142:0/2398717399 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7ff06006c510 0x7ff06006e9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff074100000 tx=0x7ff070005f90 comp rx=0 tx=0).stop 2024-03-29T04:42:33.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.860+0000 7ff07c159640 1 -- 172.21.15.142:0/2398717399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0741001a0 msgr2=0x7ff074195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:33.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.860+0000 7ff07c159640 1 --2- 172.21.15.142:0/2398717399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0741001a0 0x7ff074195dc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff064009230 tx=0x7ff06402cec0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.860+0000 7ff07c159640 1 -- 172.21.15.142:0/2398717399 shutdown_connections 2024-03-29T04:42:33.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.860+0000 7ff07c159640 1 --2- 172.21.15.142:0/2398717399 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7ff06006c510 0x7ff06006e9d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.860+0000 7ff07c159640 1 --2- 172.21.15.142:0/2398717399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0741001a0 0x7ff074195dc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.860+0000 7ff07c159640 1 --2- 172.21.15.142:0/2398717399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0740fefb0 0x7ff074195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:33.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.860+0000 7ff07c159640 1 -- 172.21.15.142:0/2398717399 >> 172.21.15.142:0/2398717399 conn(0x7ff0740fa4e0 msgr2=0x7ff074067350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:33.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.860+0000 7ff07c159640 1 -- 172.21.15.142:0/2398717399 shutdown_connections 2024-03-29T04:42:33.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:33.860+0000 7ff07c159640 1 -- 172.21.15.142:0/2398717399 wait complete. 2024-03-29T04:42:33.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:33 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1213344717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T04:42:33.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:33.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:33 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:42:33.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:33 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/855697526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T04:42:33.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:33 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/4201332785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T04:42:33.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:33 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1743294333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T04:42:33.867 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 50} 2024-03-29T04:42:33.868 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-29T04:42:33.896 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-29T04:42:33.896 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-03-29T04:42:33.896 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-29T04:42:33.896 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:42:33.896 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:42:33.896 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-29T04:42:33.896 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:42:33.896 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:42:33.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:33.897 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-03-29T04:42:33.897 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:33 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1213344717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T04:42:33.898 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:33.898 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:33 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:42:33.898 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:33 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/855697526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T04:42:33.898 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:33 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/4201332785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T04:42:33.898 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:33 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1743294333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T04:42:33.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:33.913 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2024-03-29T04:42:33.983 INFO:teuthology.orchestra.run.smithi142.stderr:Cannot find device "ceph-brx" 2024-03-29T04:42:33.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:42:33.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:33.985 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-03-29T04:42:34.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:34.002 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link delete ceph-brx 2024-03-29T04:42:34.075 INFO:teuthology.orchestra.run.smithi152.stderr:Cannot find device "ceph-brx" 2024-03-29T04:42:34.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:42:34.077 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-29T04:42:34.077 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-29T04:42:34.829 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:34.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:34 smithi142 ceph-mon[29027]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-29T04:42:34.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:34 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:34.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:34 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:34.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:34 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2398717399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-29T04:42:35.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:34 smithi152 ceph-mon[37177]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-29T04:42:35.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:34 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:35.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:34 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:35.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:34 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2398717399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-29T04:42:37.118 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:36 smithi142 ceph-mon[29027]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-29T04:42:37.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:36 smithi152 ceph-mon[37177]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-29T04:42:37.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.766+0000 7f78d89f0700 1 -- 172.21.15.142:0/2825515799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d4101c70 msgr2=0x7f78d4102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:37.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.766+0000 7f78d89f0700 1 --2- 172.21.15.142:0/2825515799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d4101c70 0x7f78d4102050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009910 tx=0x7f78c4009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:37.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.766+0000 7f78d89f0700 1 -- 172.21.15.142:0/2825515799 shutdown_connections 2024-03-29T04:42:37.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.766+0000 7f78d89f0700 1 --2- 172.21.15.142:0/2825515799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78d4102620 0x7f78d410ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:37.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.766+0000 7f78d89f0700 1 --2- 172.21.15.142:0/2825515799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d4101c70 0x7f78d4102050 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:37.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.766+0000 7f78d89f0700 1 -- 172.21.15.142:0/2825515799 >> 172.21.15.142:0/2825515799 conn(0x7f78d40fb550 msgr2=0x7f78d40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:37.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.766+0000 7f78d89f0700 1 -- 172.21.15.142:0/2825515799 shutdown_connections 2024-03-29T04:42:37.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.766+0000 7f78d89f0700 1 -- 172.21.15.142:0/2825515799 wait complete. 2024-03-29T04:42:37.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.767+0000 7f78d89f0700 1 Processor -- start 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.767+0000 7f78d89f0700 1 -- start start 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.767+0000 7f78d89f0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78d4101c70 0x7f78d419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.767+0000 7f78d89f0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d4102620 0x7f78d419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.767+0000 7f78d89f0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d419d0c0 con 0x7f78d4102620 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.767+0000 7f78d89f0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d4196570 con 0x7f78d4101c70 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.767+0000 7f78d1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d4102620 0x7f78d419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.767+0000 7f78d1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d4102620 0x7f78d419ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33928/0 (socket says 172.21.15.142:33928) 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.767+0000 7f78d1d9b700 1 -- 172.21.15.142:0/3014499639 learned_addr learned my addr 172.21.15.142:0/3014499639 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.767+0000 7f78d259c700 1 --2- 172.21.15.142:0/3014499639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78d4101c70 0x7f78d419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.768+0000 7f78d1d9b700 1 -- 172.21.15.142:0/3014499639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78d4101c70 msgr2=0x7f78d419c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.768+0000 7f78d1d9b700 1 --2- 172.21.15.142:0/3014499639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78d4101c70 0x7f78d419c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.768+0000 7f78d1d9b700 1 -- 172.21.15.142:0/3014499639 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c40095f0 con 0x7f78d4102620 2024-03-29T04:42:37.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.768+0000 7f78d1d9b700 1 --2- 172.21.15.142:0/3014499639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d4102620 0x7f78d419ca30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f78c800c8f0 tx=0x7f78c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:37.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.768+0000 7f78c37fe700 1 -- 172.21.15.142:0/3014499639 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78c8007a50 con 0x7f78d4102620 2024-03-29T04:42:37.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.768+0000 7f78c37fe700 1 -- 172.21.15.142:0/3014499639 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f78c8007bb0 con 0x7f78d4102620 2024-03-29T04:42:37.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.768+0000 7f78c37fe700 1 -- 172.21.15.142:0/3014499639 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78c8018630 con 0x7f78d4102620 2024-03-29T04:42:37.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.768+0000 7f78d89f0700 1 -- 172.21.15.142:0/3014499639 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d4196850 con 0x7f78d4102620 2024-03-29T04:42:37.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.768+0000 7f78d89f0700 1 -- 172.21.15.142:0/3014499639 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d4196da0 con 0x7f78d4102620 2024-03-29T04:42:37.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.769+0000 7f78d89f0700 1 -- 172.21.15.142:0/3014499639 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d4108210 con 0x7f78d4102620 2024-03-29T04:42:37.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.769+0000 7f78c37fe700 1 -- 172.21.15.142:0/3014499639 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f78c8007d20 con 0x7f78d4102620 2024-03-29T04:42:37.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.770+0000 7f78c37fe700 1 --2- 172.21.15.142:0/3014499639 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f78bc06c4e0 0x7f78bc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:37.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.770+0000 7f78d259c700 1 --2- 172.21.15.142:0/3014499639 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f78bc06c4e0 0x7f78bc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:37.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.770+0000 7f78c37fe700 1 -- 172.21.15.142:0/3014499639 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f78c808b850 con 0x7f78d4102620 2024-03-29T04:42:37.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.770+0000 7f78d259c700 1 --2- 172.21.15.142:0/3014499639 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f78bc06c4e0 0x7f78bc06e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f78c4006010 tx=0x7f78c401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:37.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:37.774+0000 7f78c37fe700 1 -- 172.21.15.142:0/3014499639 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f78c8059a60 con 0x7f78d4102620 2024-03-29T04:42:38.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.039+0000 7f78d89f0700 1 -- 172.21.15.142:0/3014499639 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f78d404ea90 con 0x7f78d4102620 2024-03-29T04:42:38.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.039+0000 7f78c37fe700 1 -- 172.21.15.142:0/3014499639 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f78c80595f0 con 0x7f78d4102620 2024-03-29T04:42:38.037 INFO:teuthology.orchestra.run.smithi142.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-29T04:42:38.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.041+0000 7f78d89f0700 1 -- 172.21.15.142:0/3014499639 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f78bc06c4e0 msgr2=0x7f78bc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:38.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.041+0000 7f78d89f0700 1 --2- 172.21.15.142:0/3014499639 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f78bc06c4e0 0x7f78bc06e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f78c4006010 tx=0x7f78c401a040 comp rx=0 tx=0).stop 2024-03-29T04:42:38.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.041+0000 7f78d89f0700 1 -- 172.21.15.142:0/3014499639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d4102620 msgr2=0x7f78d419ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:38.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.041+0000 7f78d89f0700 1 --2- 172.21.15.142:0/3014499639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d4102620 0x7f78d419ca30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f78c800c8f0 tx=0x7f78c800ccb0 comp rx=0 tx=0).stop 2024-03-29T04:42:38.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.041+0000 7f78d89f0700 1 -- 172.21.15.142:0/3014499639 shutdown_connections 2024-03-29T04:42:38.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.041+0000 7f78d89f0700 1 --2- 172.21.15.142:0/3014499639 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f78bc06c4e0 0x7f78bc06e9a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:38.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.041+0000 7f78d89f0700 1 --2- 172.21.15.142:0/3014499639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78d4101c70 0x7f78d419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:38.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.041+0000 7f78d89f0700 1 --2- 172.21.15.142:0/3014499639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d4102620 0x7f78d419ca30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:38.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.042+0000 7f78d89f0700 1 -- 172.21.15.142:0/3014499639 >> 172.21.15.142:0/3014499639 conn(0x7f78d40fb550 msgr2=0x7f78d4105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:38.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.042+0000 7f78d89f0700 1 -- 172.21.15.142:0/3014499639 shutdown_connections 2024-03-29T04:42:38.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38.042+0000 7f78d89f0700 1 -- 172.21.15.142:0/3014499639 wait complete. 2024-03-29T04:42:38.658 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:42:38.658 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:42:38.658 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-03-29T04:42:38.658 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T04:42:38.658 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:42:38.658 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:42:38.658 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:42:38.658 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:42:38.658 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:42:38.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:38.659 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-03-29T04:42:38.674 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:42:38.674 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:42:38.674 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:42:38.675 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-03-29T04:42:38.676 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:42:38.676 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42156sec preferred_lft 42156sec 2024-03-29T04:42:38.676 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-03-29T04:42:38.676 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:42:38.676 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:42:38.677 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:42:38.677 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-03-29T04:42:38.677 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:42:38.677 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip addr flush dev ceph-brx 2024-03-29T04:42:38.677 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set ceph-brx up 2024-03-29T04:42:38.677 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:42:38.677 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-03-29T04:42:38.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:42:38.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:42:38.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:38.873 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:42:38.888 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:38 smithi142 ceph-mon[29027]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-29T04:42:38.889 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:38 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3014499639' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-29T04:42:38.889 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:38.889 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:38.889 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:38.889 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:42:38.890 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:38.900 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-03-29T04:42:38.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:38.902 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-03-29T04:42:38.994 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-03-29T04:42:38.994 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:42:38.995 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:42:38.995 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:42:38.995 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:42:38.995 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:42:38.995 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-03-29T04:42:38.995 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:42:38.995 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:42:38.996 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:42:38.996 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-03-29T04:42:39.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:42:39.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:42:39.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:39.263 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-03-29T04:42:39.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:39.278 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list-id 2024-03-29T04:42:39.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:38 smithi152 ceph-mon[37177]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-29T04:42:39.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:38 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3014499639' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-29T04:42:39.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:39.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:39.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:42:39.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:42:39.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:42:39.334 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:42:39.334 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-03-29T04:42:39.334 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:42:39.334 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-29T04:42:39.334 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-03-29T04:42:39.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:42:39.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:42:39.456 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-29T04:42:39.457 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:42:39.457 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-03-29T04:42:39.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-29T04:42:39.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:42:39.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-29T04:42:39.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-29T04:42:39.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-29T04:42:39.457 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-03-29T04:42:39.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:42:39.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:42:39.595 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:42:39.595 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-03-29T04:42:39.595 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-03-29T04:42:39.595 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:42:39.595 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-03-29T04:42:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:42:39.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:42:39.677 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T04:42:39.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:42:39.677 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T04:42:39.691 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:42:39.692 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:42:39.692 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:42:39.747 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-03-29T04:42:39.816 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:42:39.873 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-03-29T04:42:39.873 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-03-29T04:42:39.873 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-03-29T04:42:39.873 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-03-29T04:42:39.874 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-03-29T04:42:39.874 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-03-29T04:42:39.874 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-03-29T04:42:39.874 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-03-29T04:42:39.874 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-03-29T04:42:39.874 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-03-29T04:42:39.874 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-03-29T04:42:39.874 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/selinux 2024-03-29T04:42:39.874 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:42:39.874 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-03-29T04:42:39.874 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-03-29T04:42:39.874 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-03-29T04:42:39.875 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-03-29T04:42:39.875 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:42:39.875 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-03-29T04:42:39.875 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:42:39.875 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-03-29T04:42:39.875 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:42:39.875 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:42:39.875 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-03-29T04:42:39.875 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-03-29T04:42:39.875 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay 2024-03-29T04:42:39.875 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/67893173437ba39bbc689fe2cd6358a9cec92412d694e43be015558751d6fa62/merged 2024-03-29T04:42:39.875 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/cdc5553bbf39eff0a8182bba11af0470097c3437b250f95d0dbefe363f66ffd4/merged 2024-03-29T04:42:39.876 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/0 2024-03-29T04:42:39.876 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:42:39.876 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/c5f273f93bbe3ab78cd6e8857ca5b50443843ac5f923a6409dcc6b232930a2f4/merged 2024-03-29T04:42:39.876 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/b21095f716281ddb37972cddf6af38cfbf0a11205f0c96f3c4033fc89a0fba5a/merged 2024-03-29T04:42:39.876 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/760c1e7cd6c4fb65402184c6ccd599d63af2f1088872eff74fb4bb34e65ae24b/merged 2024-03-29T04:42:39.876 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/3c305d80eb1bd502355f8c34b483a818dbbb468ab755d5bf9455442d6f340263/merged 2024-03-29T04:42:39.876 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/bafae82edbc6be46e16257686e1eef06d3a40761eb79de8318a8e4ea8abf9824/merged 2024-03-29T04:42:39.877 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/212b4bf44a159b1b0b14b5a46e92c5a66b762868681d20c1ad36268437d9f593/merged 2024-03-29T04:42:39.877 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/18c501dfc8da99665d26e9111f7279feaa23163a60c94c3b1c205b103417ffc6/merged 2024-03-29T04:42:39.877 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/1254adfac8d0addc1dd846b8ed16f1d50a51d8286ee094f3c6dbd1b93ea11f52/merged 2024-03-29T04:42:39.877 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/caf8b8370dcb6781caba2c55dd3635a0390df5ce10523c6d9aa54036361c88ad/merged 2024-03-29T04:42:39.877 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/ebd2790c7808f36999ede3f8d0244b36241145d3c0f9c4a99198a827b6808534/merged 2024-03-29T04:42:39.877 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/a29350e99eaded298fd861ab10551972be39aeb693afa32fd4114191a0ba80a6/merged 2024-03-29T04:42:39.878 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-03-29T04:42:39.878 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:42:39.878 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:42:39.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:39.878 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-03-29T04:42:39.929 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:42:39.929 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T04:42:39.971 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-03-29T04:42:39.998 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:42:40.057 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-03-29T04:42:40.057 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-03-29T04:42:40.057 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-03-29T04:42:40.057 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-03-29T04:42:40.057 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-03-29T04:42:40.057 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-03-29T04:42:40.057 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/selinux 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-03-29T04:42:40.058 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/67893173437ba39bbc689fe2cd6358a9cec92412d694e43be015558751d6fa62/merged 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/cdc5553bbf39eff0a8182bba11af0470097c3437b250f95d0dbefe363f66ffd4/merged 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/0 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/c5f273f93bbe3ab78cd6e8857ca5b50443843ac5f923a6409dcc6b232930a2f4/merged 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/b21095f716281ddb37972cddf6af38cfbf0a11205f0c96f3c4033fc89a0fba5a/merged 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/760c1e7cd6c4fb65402184c6ccd599d63af2f1088872eff74fb4bb34e65ae24b/merged 2024-03-29T04:42:40.059 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/3c305d80eb1bd502355f8c34b483a818dbbb468ab755d5bf9455442d6f340263/merged 2024-03-29T04:42:40.060 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/bafae82edbc6be46e16257686e1eef06d3a40761eb79de8318a8e4ea8abf9824/merged 2024-03-29T04:42:40.060 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/212b4bf44a159b1b0b14b5a46e92c5a66b762868681d20c1ad36268437d9f593/merged 2024-03-29T04:42:40.060 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/18c501dfc8da99665d26e9111f7279feaa23163a60c94c3b1c205b103417ffc6/merged 2024-03-29T04:42:40.060 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/1254adfac8d0addc1dd846b8ed16f1d50a51d8286ee094f3c6dbd1b93ea11f52/merged 2024-03-29T04:42:40.060 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/caf8b8370dcb6781caba2c55dd3635a0390df5ce10523c6d9aa54036361c88ad/merged 2024-03-29T04:42:40.060 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/ebd2790c7808f36999ede3f8d0244b36241145d3c0f9c4a99198a827b6808534/merged 2024-03-29T04:42:40.060 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/containers/storage/overlay/a29350e99eaded298fd861ab10551972be39aeb693afa32fd4114191a0ba80a6/merged 2024-03-29T04:42:40.060 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-03-29T04:42:40.060 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:42:40.060 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:42:40.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:40.061 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-03-29T04:42:40.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi142.stderr:2024-03-29T04:42:40.075+0000 7f90821f9480 -1 init, newargv = 0x5649e72b7d10 newargc=15 2024-03-29T04:42:40.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi142.stderr:ceph-fuse[77265]: starting ceph client 2024-03-29T04:42:40.080 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi142.stderr:ceph-fuse[77265]: starting fuse 2024-03-29T04:42:40.089 INFO:teuthology.orchestra.run.smithi142.stdout:137 2024-03-29T04:42:40.089 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [137] 2024-03-29T04:42:40.089 DEBUG:teuthology.orchestra.run.smithi142:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:42:40.089 DEBUG:teuthology.orchestra.run.smithi142:> import glob 2024-03-29T04:42:40.090 DEBUG:teuthology.orchestra.run.smithi142:> import re 2024-03-29T04:42:40.090 DEBUG:teuthology.orchestra.run.smithi142:> import os 2024-03-29T04:42:40.090 DEBUG:teuthology.orchestra.run.smithi142:> import subprocess 2024-03-29T04:42:40.090 DEBUG:teuthology.orchestra.run.smithi142:> 2024-03-29T04:42:40.090 DEBUG:teuthology.orchestra.run.smithi142:> def _find_admin_socket(client_name): 2024-03-29T04:42:40.090 DEBUG:teuthology.orchestra.run.smithi142:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T04:42:40.090 DEBUG:teuthology.orchestra.run.smithi142:> files = glob.glob(asok_path) 2024-03-29T04:42:40.090 DEBUG:teuthology.orchestra.run.smithi142:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T04:42:40.090 DEBUG:teuthology.orchestra.run.smithi142:> 2024-03-29T04:42:40.091 DEBUG:teuthology.orchestra.run.smithi142:> # Given a non-glob path, it better be there 2024-03-29T04:42:40.091 DEBUG:teuthology.orchestra.run.smithi142:> if "*" not in asok_path: 2024-03-29T04:42:40.091 DEBUG:teuthology.orchestra.run.smithi142:> assert(len(files) == 1) 2024-03-29T04:42:40.091 DEBUG:teuthology.orchestra.run.smithi142:> return files[0] 2024-03-29T04:42:40.091 DEBUG:teuthology.orchestra.run.smithi142:> 2024-03-29T04:42:40.091 DEBUG:teuthology.orchestra.run.smithi142:> for f in files: 2024-03-29T04:42:40.091 DEBUG:teuthology.orchestra.run.smithi142:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:42:40.091 DEBUG:teuthology.orchestra.run.smithi142:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:42:40.091 DEBUG:teuthology.orchestra.run.smithi142:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:42:40.092 DEBUG:teuthology.orchestra.run.smithi142:> contents = proc_f.read() 2024-03-29T04:42:40.092 DEBUG:teuthology.orchestra.run.smithi142:> if mountpoint in contents: 2024-03-29T04:42:40.092 DEBUG:teuthology.orchestra.run.smithi142:> return f 2024-03-29T04:42:40.092 DEBUG:teuthology.orchestra.run.smithi142:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:42:40.092 DEBUG:teuthology.orchestra.run.smithi142:> 2024-03-29T04:42:40.092 DEBUG:teuthology.orchestra.run.smithi142:> print(_find_admin_socket("client.0")) 2024-03-29T04:42:40.092 DEBUG:teuthology.orchestra.run.smithi142:> ' 2024-03-29T04:42:40.210 INFO:teuthology.orchestra.run.smithi142.stdout:/var/run/ceph/ceph-client.0.77265.asok 2024-03-29T04:42:40.213 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:42:40.220 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.77265.asok 2024-03-29T04:42:40.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:40.220 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.77265.asok status 2024-03-29T04:42:40.360 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:42:40.360 INFO:teuthology.orchestra.run.smithi142.stdout: "metadata": { 2024-03-29T04:42:40.360 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-03-29T04:42:40.360 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-03-29T04:42:40.360 INFO:teuthology.orchestra.run.smithi142.stdout: "entity_id": "0", 2024-03-29T04:42:40.360 INFO:teuthology.orchestra.run.smithi142.stdout: "hostname": "smithi142", 2024-03-29T04:42:40.360 INFO:teuthology.orchestra.run.smithi142.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T04:42:40.360 INFO:teuthology.orchestra.run.smithi142.stdout: "pid": "77265", 2024-03-29T04:42:40.360 INFO:teuthology.orchestra.run.smithi142.stdout: "root": "/" 2024-03-29T04:42:40.360 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_count": 0, 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_pinned_count": 0, 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: "id": 14542, 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: "inst": { 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: "name": { 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "client", 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: "num": 14542 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 2208957537 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:42:40.361 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 2208957537 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: "inst_str": "client.14542 192.168.0.1:0/2208957537", 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: "addr_str": "192.168.0.1:0/2208957537", 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: "inode_count": 1, 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_epoch": 17, 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch": 47, 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: "blocklisted": false, 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout: "fs_name": "cephfs" 2024-03-29T04:42:40.362 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:42:40.368 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-29T04:42:40.577 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:40.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:40 smithi142 ceph-mon[29027]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-29T04:42:41.287 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:40 smithi152 ceph-mon[37177]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-29T04:42:42.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.225+0000 7f2a21974700 1 -- 172.21.15.142:0/3701324669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c073a80 msgr2=0x7f2a1c10cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:42.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.225+0000 7f2a21974700 1 --2- 172.21.15.142:0/3701324669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c073a80 0x7f2a1c10cec0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009910 tx=0x7f2a10009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:42.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.225+0000 7f2a21974700 1 -- 172.21.15.142:0/3701324669 shutdown_connections 2024-03-29T04:42:42.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.225+0000 7f2a21974700 1 --2- 172.21.15.142:0/3701324669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c073a80 0x7f2a1c10cec0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:42.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.225+0000 7f2a21974700 1 --2- 172.21.15.142:0/3701324669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a1c073160 0x7f2a1c073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:42.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.225+0000 7f2a21974700 1 -- 172.21.15.142:0/3701324669 >> 172.21.15.142:0/3701324669 conn(0x7f2a1c0fc650 msgr2=0x7f2a1c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:42.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.226+0000 7f2a21974700 1 -- 172.21.15.142:0/3701324669 shutdown_connections 2024-03-29T04:42:42.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.226+0000 7f2a21974700 1 -- 172.21.15.142:0/3701324669 wait complete. 2024-03-29T04:42:42.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.226+0000 7f2a21974700 1 Processor -- start 2024-03-29T04:42:42.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a21974700 1 -- start start 2024-03-29T04:42:42.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a21974700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c073160 0x7f2a1c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:42.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a21974700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a1c073a80 0x7f2a1c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:42.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a21974700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c19f350 con 0x7f2a1c073160 2024-03-29T04:42:42.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a21974700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c198800 con 0x7f2a1c073a80 2024-03-29T04:42:42.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a1affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c073160 0x7f2a1c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:42.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a1affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c073160 0x7f2a1c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46328/0 (socket says 172.21.15.142:46328) 2024-03-29T04:42:42.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a1affd700 1 -- 172.21.15.142:0/3868377725 learned_addr learned my addr 172.21.15.142:0/3868377725 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:42.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a1a7fc700 1 --2- 172.21.15.142:0/3868377725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a1c073a80 0x7f2a1c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:42.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a1affd700 1 -- 172.21.15.142:0/3868377725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a1c073a80 msgr2=0x7f2a1c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:42.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a1affd700 1 --2- 172.21.15.142:0/3868377725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a1c073a80 0x7f2a1c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:42.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a1affd700 1 -- 172.21.15.142:0/3868377725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a100095f0 con 0x7f2a1c073160 2024-03-29T04:42:42.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.227+0000 7f2a1a7fc700 1 --2- 172.21.15.142:0/3868377725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a1c073a80 0x7f2a1c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:42:42.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.228+0000 7f2a1affd700 1 --2- 172.21.15.142:0/3868377725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c073160 0x7f2a1c19e780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00c8a0 tx=0x7f2a0c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:42.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.228+0000 7f2a20972700 1 -- 172.21.15.142:0/3868377725 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a0c00cea0 con 0x7f2a1c073160 2024-03-29T04:42:42.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.228+0000 7f2a21974700 1 -- 172.21.15.142:0/3868377725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c198ae0 con 0x7f2a1c073160 2024-03-29T04:42:42.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.228+0000 7f2a21974700 1 -- 172.21.15.142:0/3868377725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c199030 con 0x7f2a1c073160 2024-03-29T04:42:42.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.228+0000 7f2a20972700 1 -- 172.21.15.142:0/3868377725 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2a0c004830 con 0x7f2a1c073160 2024-03-29T04:42:42.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.228+0000 7f2a20972700 1 -- 172.21.15.142:0/3868377725 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a0c0055b0 con 0x7f2a1c073160 2024-03-29T04:42:42.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.229+0000 7f2a21974700 1 -- 172.21.15.142:0/3868377725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a08005320 con 0x7f2a1c073160 2024-03-29T04:42:42.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.229+0000 7f2a20972700 1 -- 172.21.15.142:0/3868377725 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2a0c007ad0 con 0x7f2a1c073160 2024-03-29T04:42:42.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.230+0000 7f2a20972700 1 --2- 172.21.15.142:0/3868377725 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f2a0406c4e0 0x7f2a0406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:42.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.230+0000 7f2a20972700 1 -- 172.21.15.142:0/3868377725 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2a0c08b530 con 0x7f2a1c073160 2024-03-29T04:42:42.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.230+0000 7f2a1a7fc700 1 --2- 172.21.15.142:0/3868377725 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f2a0406c4e0 0x7f2a0406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:42.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.230+0000 7f2a1a7fc700 1 --2- 172.21.15.142:0/3868377725 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f2a0406c4e0 0x7f2a0406e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b5c0 tx=0x7f2a1001a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:42.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.234+0000 7f2a20972700 1 -- 172.21.15.142:0/3868377725 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2a0c0597c0 con 0x7f2a1c073160 2024-03-29T04:42:42.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.463+0000 7f2a21974700 1 -- 172.21.15.142:0/3868377725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2a08006200 con 0x7f2a1c073160 2024-03-29T04:42:42.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.463+0000 7f2a20972700 1 -- 172.21.15.142:0/3868377725 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f2a0c059350 con 0x7f2a1c073160 2024-03-29T04:42:42.461 INFO:teuthology.orchestra.run.smithi142.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-29T04:42:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.465+0000 7f2a21974700 1 -- 172.21.15.142:0/3868377725 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f2a0406c4e0 msgr2=0x7f2a0406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.465+0000 7f2a21974700 1 --2- 172.21.15.142:0/3868377725 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f2a0406c4e0 0x7f2a0406e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b5c0 tx=0x7f2a1001a040 comp rx=0 tx=0).stop 2024-03-29T04:42:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.465+0000 7f2a21974700 1 -- 172.21.15.142:0/3868377725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c073160 msgr2=0x7f2a1c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.465+0000 7f2a21974700 1 --2- 172.21.15.142:0/3868377725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c073160 0x7f2a1c19e780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00c8a0 tx=0x7f2a0c00cc60 comp rx=0 tx=0).stop 2024-03-29T04:42:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.465+0000 7f2a21974700 1 -- 172.21.15.142:0/3868377725 shutdown_connections 2024-03-29T04:42:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.465+0000 7f2a21974700 1 --2- 172.21.15.142:0/3868377725 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f2a0406c4e0 0x7f2a0406e9a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.465+0000 7f2a21974700 1 --2- 172.21.15.142:0/3868377725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c073160 0x7f2a1c19e780 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.466+0000 7f2a21974700 1 --2- 172.21.15.142:0/3868377725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a1c073a80 0x7f2a1c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.466+0000 7f2a21974700 1 -- 172.21.15.142:0/3868377725 >> 172.21.15.142:0/3868377725 conn(0x7f2a1c0fc650 msgr2=0x7f2a1c107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.466+0000 7f2a21974700 1 -- 172.21.15.142:0/3868377725 shutdown_connections 2024-03-29T04:42:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:42.466+0000 7f2a21974700 1 -- 172.21.15.142:0/3868377725 wait complete. 2024-03-29T04:42:43.060 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:42 smithi142 ceph-mon[29027]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-29T04:42:43.060 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:42 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3868377725' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-29T04:42:43.082 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:42:43.082 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:42:43.082 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-03-29T04:42:43.082 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-29T04:42:43.082 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:42:43.082 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:42:43.082 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:42:43.082 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:42:43.082 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:42:43.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:43.083 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-03-29T04:42:43.101 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:42:43.101 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:42:43.102 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:42:43.102 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:42:43.102 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-03-29T04:42:43.102 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:42:43.102 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:42:43.103 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:42:43.103 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-03-29T04:42:43.103 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:42:43.103 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:42:43.103 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-03-29T04:42:43.103 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:42:43.103 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:42:43.103 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:42:43.103 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:42:43.103 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:42:43.103 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42164sec preferred_lft 42164sec 2024-03-29T04:42:43.103 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link 2024-03-29T04:42:43.104 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:42:43.104 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:42:43.104 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:42:43.104 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-03-29T04:42:43.104 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:42:43.104 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip addr flush dev ceph-brx 2024-03-29T04:42:43.104 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set ceph-brx up 2024-03-29T04:42:43.104 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:42:43.104 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-03-29T04:42:43.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:42:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:42:43.209 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:42 smithi152 ceph-mon[37177]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-29T04:42:43.209 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:42 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3868377725' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-29T04:42:43.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:42:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:42:43.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:43.316 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:42:43.345 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-03-29T04:42:43.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:43.346 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-03-29T04:42:43.412 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-03-29T04:42:43.413 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:42:43.413 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:42:43.413 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:42:43.413 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:42:43.413 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:42:43.414 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-03-29T04:42:43.414 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:42:43.414 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:42:43.414 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:42:43.414 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-03-29T04:42:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:42:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:42:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:42:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:42:43.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:43.708 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-03-29T04:42:43.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:43.724 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list-id 2024-03-29T04:42:43.779 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:42:43.779 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-03-29T04:42:43.780 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:42:43.780 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-29T04:42:43.780 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-03-29T04:42:43.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:42:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:42:43.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:42:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:42:43.902 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-29T04:42:43.902 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:42:43.902 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-03-29T04:42:43.902 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-29T04:42:43.902 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:42:43.902 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-29T04:42:43.902 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-29T04:42:43.902 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-29T04:42:43.903 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-03-29T04:42:43.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:42:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:42:44.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:42:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:42:44.056 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:42:44.056 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-03-29T04:42:44.056 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-03-29T04:42:44.056 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:42:44.056 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-03-29T04:42:44.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:42:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:42:44.132 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:42:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:42:44.138 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-29T04:42:44.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:42:44.138 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-29T04:42:44.160 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:42:44.160 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:42:44.160 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:42:44.216 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-03-29T04:42:44.283 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:42:44.339 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-03-29T04:42:44.340 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/0 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay 2024-03-29T04:42:44.341 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/4b0516bfe2dd4104669ed7116e266f8d629c843153a9cd66fb5fa7fef9fcfe72/merged 2024-03-29T04:42:44.342 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/4aece537bfa0a0c07475e30b6454a78b3a28995407de820d42d0a809cd08a5b6/merged 2024-03-29T04:42:44.342 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/a082222342448eb87217596c2c83f23255a1bbce09ee053ac51489552f6af79e/merged 2024-03-29T04:42:44.342 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/13022cfebb5255a4021f38c306fb71bfedf49bff1eba17a991c7808b2f208e34/merged 2024-03-29T04:42:44.342 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/9856ebcc08da219aa800f295bbdd2f1df26420579b22c16a8f15ba7ebf73dc77/merged 2024-03-29T04:42:44.342 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/11950406ad093a95f221f1e60e7dc41715d224ba5a6ba05e68a0c74a20f0be4d/merged 2024-03-29T04:42:44.342 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/c349e7861816e3035e6f6096109c82282eb388b617625bd1f78821846d12db32/merged 2024-03-29T04:42:44.342 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/f2d590f6b4be53c27ecffbcf341c60306f29548508ce5812aece11b096385187/merged 2024-03-29T04:42:44.342 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/6e35651de5f9e04025e9a22455cf38273d4b9b5e1208fa157b83197446887400/merged 2024-03-29T04:42:44.342 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/aee602aa56187d6c2db5d4817d7675ed571287f84edb93172ac1b307bb1f4fcf/merged 2024-03-29T04:42:44.342 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-03-29T04:42:44.342 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:42:44.342 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:42:44.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:44.343 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-03-29T04:42:44.394 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:42:44.395 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-29T04:42:44.437 DEBUG:teuthology.orchestra.run.smithi152:> sudo modprobe fuse 2024-03-29T04:42:44.464 DEBUG:teuthology.orchestra.run.smithi152:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:42:44.520 INFO:teuthology.orchestra.run.smithi152.stdout:/proc 2024-03-29T04:42:44.520 INFO:teuthology.orchestra.run.smithi152.stdout:/sys 2024-03-29T04:42:44.521 INFO:teuthology.orchestra.run.smithi152.stdout:/dev 2024-03-29T04:42:44.521 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/security 2024-03-29T04:42:44.521 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/shm 2024-03-29T04:42:44.521 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/pts 2024-03-29T04:42:44.521 INFO:teuthology.orchestra.run.smithi152.stdout:/run 2024-03-29T04:42:44.521 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/cgroup 2024-03-29T04:42:44.521 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/pstore 2024-03-29T04:42:44.521 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/bpf 2024-03-29T04:42:44.521 INFO:teuthology.orchestra.run.smithi152.stdout:/ 2024-03-29T04:42:44.521 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/selinux 2024-03-29T04:42:44.521 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/hugepages 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/dev/mqueue 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/debug 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/tracing 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/fs/fuse/connections 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/sys/kernel/config 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/1000 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/ceph 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/run/user/0 2024-03-29T04:42:44.522 INFO:teuthology.orchestra.run.smithi152.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/4b0516bfe2dd4104669ed7116e266f8d629c843153a9cd66fb5fa7fef9fcfe72/merged 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/4aece537bfa0a0c07475e30b6454a78b3a28995407de820d42d0a809cd08a5b6/merged 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/a082222342448eb87217596c2c83f23255a1bbce09ee053ac51489552f6af79e/merged 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/13022cfebb5255a4021f38c306fb71bfedf49bff1eba17a991c7808b2f208e34/merged 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/9856ebcc08da219aa800f295bbdd2f1df26420579b22c16a8f15ba7ebf73dc77/merged 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/11950406ad093a95f221f1e60e7dc41715d224ba5a6ba05e68a0c74a20f0be4d/merged 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/c349e7861816e3035e6f6096109c82282eb388b617625bd1f78821846d12db32/merged 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/f2d590f6b4be53c27ecffbcf341c60306f29548508ce5812aece11b096385187/merged 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/6e35651de5f9e04025e9a22455cf38273d4b9b5e1208fa157b83197446887400/merged 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/var/lib/containers/storage/overlay/aee602aa56187d6c2db5d4817d7675ed571287f84edb93172ac1b307bb1f4fcf/merged 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:42:44.523 INFO:teuthology.orchestra.run.smithi152.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:42:44.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:44.524 DEBUG:teuthology.orchestra.run.smithi152:> ls /sys/fs/fuse/connections 2024-03-29T04:42:44.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi152.stderr:2024-03-29T04:42:44.557+0000 7f41ae841480 -1 init, newargv = 0x55c46bd31d30 newargc=15 2024-03-29T04:42:44.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi152.stderr:ceph-fuse[63726]: starting ceph client 2024-03-29T04:42:44.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi152.stderr:ceph-fuse[63726]: starting fuse 2024-03-29T04:42:44.570 INFO:teuthology.orchestra.run.smithi152.stdout:126 2024-03-29T04:42:44.570 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [126] 2024-03-29T04:42:44.570 DEBUG:teuthology.orchestra.run.smithi152:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:42:44.571 DEBUG:teuthology.orchestra.run.smithi152:> import glob 2024-03-29T04:42:44.571 DEBUG:teuthology.orchestra.run.smithi152:> import re 2024-03-29T04:42:44.571 DEBUG:teuthology.orchestra.run.smithi152:> import os 2024-03-29T04:42:44.571 DEBUG:teuthology.orchestra.run.smithi152:> import subprocess 2024-03-29T04:42:44.571 DEBUG:teuthology.orchestra.run.smithi152:> 2024-03-29T04:42:44.571 DEBUG:teuthology.orchestra.run.smithi152:> def _find_admin_socket(client_name): 2024-03-29T04:42:44.571 DEBUG:teuthology.orchestra.run.smithi152:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-29T04:42:44.571 DEBUG:teuthology.orchestra.run.smithi152:> files = glob.glob(asok_path) 2024-03-29T04:42:44.571 DEBUG:teuthology.orchestra.run.smithi152:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> # Given a non-glob path, it better be there 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> if "*" not in asok_path: 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> assert(len(files) == 1) 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> return files[0] 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> for f in files: 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> contents = proc_f.read() 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> if mountpoint in contents: 2024-03-29T04:42:44.572 DEBUG:teuthology.orchestra.run.smithi152:> return f 2024-03-29T04:42:44.573 DEBUG:teuthology.orchestra.run.smithi152:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:42:44.573 DEBUG:teuthology.orchestra.run.smithi152:> 2024-03-29T04:42:44.573 DEBUG:teuthology.orchestra.run.smithi152:> print(_find_admin_socket("client.1")) 2024-03-29T04:42:44.573 DEBUG:teuthology.orchestra.run.smithi152:> ' 2024-03-29T04:42:44.690 INFO:teuthology.orchestra.run.smithi152.stdout:/var/run/ceph/ceph-client.1.63726.asok 2024-03-29T04:42:44.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-29T04:42:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:42:44.700 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:44 smithi152 ceph-mon[37177]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-29T04:42:44.700 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.63726.asok 2024-03-29T04:42:44.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:44.701 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.63726.asok status 2024-03-29T04:42:44.836 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-03-29T04:42:44.836 INFO:teuthology.orchestra.run.smithi152.stdout: "metadata": { 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "entity_id": "1", 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": "smithi152", 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "pid": "63726", 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "root": "/" 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_count": 0, 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "dentry_pinned_count": 0, 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24323, 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "inst": { 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "name": { 2024-03-29T04:42:44.837 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "client", 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: "num": 24323 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 1101610675 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": { 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 1101610675 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-29T04:42:44.838 INFO:teuthology.orchestra.run.smithi152.stdout: "inst_str": "client.24323 192.168.0.1:0/1101610675", 2024-03-29T04:42:44.839 INFO:teuthology.orchestra.run.smithi152.stdout: "addr_str": "192.168.0.1:0/1101610675", 2024-03-29T04:42:44.839 INFO:teuthology.orchestra.run.smithi152.stdout: "inode_count": 1, 2024-03-29T04:42:44.839 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_epoch": 17, 2024-03-29T04:42:44.839 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch": 47, 2024-03-29T04:42:44.839 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:42:44.839 INFO:teuthology.orchestra.run.smithi152.stdout: "blocklisted": false, 2024-03-29T04:42:44.839 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs" 2024-03-29T04:42:44.839 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-03-29T04:42:44.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:44.845 DEBUG:teuthology.orchestra.run.smithi142:> stat --file-system '--printf=%T 2024-03-29T04:42:44.845 DEBUG:teuthology.orchestra.run.smithi142:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T04:42:44.865 INFO:teuthology.orchestra.run.smithi142.stdout:fuseblk 2024-03-29T04:42:44.865 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T04:42:44.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:44.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:42:44.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:44 smithi142 ceph-mon[29027]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-29T04:42:44.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:44.904 DEBUG:teuthology.orchestra.run.smithi152:> stat --file-system '--printf=%T 2024-03-29T04:42:44.904 DEBUG:teuthology.orchestra.run.smithi152:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-29T04:42:44.920 INFO:teuthology.orchestra.run.smithi152.stdout:fuseblk 2024-03-29T04:42:44.920 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-29T04:42:44.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:42:44.920 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:42:45.584 INFO:teuthology.run_tasks:Running task print... 2024-03-29T04:42:45.596 INFO:teuthology.task.print:**** done client 2024-03-29T04:42:45.597 INFO:teuthology.run_tasks:Running task parallel... 2024-03-29T04:42:45.607 INFO:teuthology.task.parallel:starting parallel... 2024-03-29T04:42:45.607 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-29T04:42:45.608 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-29T04:42:45.608 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T04:42:45.609 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-29T04:42:45.609 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-29T04:42:45.609 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-29T04:42:45.613 INFO:tasks.workunit:Pulling workunits from ref 8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:42:45.613 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-29T04:42:45.613 INFO:tasks.workunit:timeout=3h 2024-03-29T04:42:45.613 INFO:tasks.workunit:cleanup=True 2024-03-29T04:42:45.613 DEBUG:teuthology.orchestra.run.smithi142:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-29T04:42:45.634 INFO:teuthology.orchestra.run.smithi142.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-29T04:42:45.634 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-29T04:42:45.634 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 89h/137d Inode: 1 Links: 2 2024-03-29T04:42:45.634 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:42:45.634 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-29T04:42:45.634 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-29T04:42:45.634 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-03-29 04:42:13.431604282 +0000 2024-03-29T04:42:45.634 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-03-29 04:42:44.903765177 +0000 2024-03-29T04:42:45.635 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-03-29T04:42:45.635 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-29T04:42:45.635 DEBUG:teuthology.orchestra.run.smithi142:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-29T04:42:45.708 DEBUG:teuthology.orchestra.run.smithi152:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-29T04:42:45.727 INFO:teuthology.orchestra.run.smithi152.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-29T04:42:45.727 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-29T04:42:45.727 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 7eh/126d Inode: 1 Links: 3 2024-03-29T04:42:45.727 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:42:45.727 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-29T04:42:45.727 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-29T04:42:45.727 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-29 04:42:45.706752734 +0000 2024-03-29T04:42:45.727 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-29 04:42:45.706752734 +0000 2024-03-29T04:42:45.727 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2024-03-29T04:42:45.728 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-29T04:42:45.728 DEBUG:teuthology.orchestra.run.smithi152:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-29T04:42:45.801 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:42:45.801 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:42:45.816 INFO:tasks.workunit.client.0.smithi142.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-29T04:42:45.838 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:45.857 INFO:tasks.workunit.client.1.smithi152.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-29T04:42:46.956 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:46 smithi142 ceph-mon[29027]: pgmap v199: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-29T04:42:47.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:46 smithi152 ceph-mon[37177]: pgmap v199: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-29T04:42:47.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.406+0000 7f45dabc7700 1 -- 172.21.15.142:0/850962370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4101710 msgr2=0x7f45d4103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:47.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.406+0000 7f45dabc7700 1 --2- 172.21.15.142:0/850962370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4101710 0x7f45d4103b00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009910 tx=0x7f45c8009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:47.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.406+0000 7f45dabc7700 1 -- 172.21.15.142:0/850962370 shutdown_connections 2024-03-29T04:42:47.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.406+0000 7f45dabc7700 1 --2- 172.21.15.142:0/850962370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45d4104040 0x7f45d4106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:47.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.406+0000 7f45dabc7700 1 --2- 172.21.15.142:0/850962370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4101710 0x7f45d4103b00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:47.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.406+0000 7f45dabc7700 1 -- 172.21.15.142:0/850962370 >> 172.21.15.142:0/850962370 conn(0x7f45d40fb080 msgr2=0x7f45d40fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:47.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.406+0000 7f45dabc7700 1 -- 172.21.15.142:0/850962370 shutdown_connections 2024-03-29T04:42:47.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.406+0000 7f45dabc7700 1 -- 172.21.15.142:0/850962370 wait complete. 2024-03-29T04:42:47.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45dabc7700 1 Processor -- start 2024-03-29T04:42:47.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45dabc7700 1 -- start start 2024-03-29T04:42:47.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45dabc7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45d4101710 0x7f45d41963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:47.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45dabc7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4104040 0x7f45d41968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:47.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45dabc7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d4196f10 con 0x7f45d4104040 2024-03-29T04:42:47.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45dabc7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d4197050 con 0x7f45d4101710 2024-03-29T04:42:47.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45d3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4104040 0x7f45d41968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:47.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45d3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4104040 0x7f45d41968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46336/0 (socket says 172.21.15.142:46336) 2024-03-29T04:42:47.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45d3fff700 1 -- 172.21.15.142:0/3886964226 learned_addr learned my addr 172.21.15.142:0/3886964226 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:47.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45d8963700 1 --2- 172.21.15.142:0/3886964226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45d4101710 0x7f45d41963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:47.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45d3fff700 1 -- 172.21.15.142:0/3886964226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45d4101710 msgr2=0x7f45d41963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:47.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45d3fff700 1 --2- 172.21.15.142:0/3886964226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45d4101710 0x7f45d41963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:47.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45d3fff700 1 -- 172.21.15.142:0/3886964226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c80095f0 con 0x7f45d4104040 2024-03-29T04:42:47.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.407+0000 7f45d3fff700 1 --2- 172.21.15.142:0/3886964226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4104040 0x7f45d41968f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f45c000d8d0 tx=0x7f45c000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:47.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.408+0000 7f45d1ffb700 1 -- 172.21.15.142:0/3886964226 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45c00098a0 con 0x7f45d4104040 2024-03-29T04:42:47.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.408+0000 7f45dabc7700 1 -- 172.21.15.142:0/3886964226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d419bab0 con 0x7f45d4104040 2024-03-29T04:42:47.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.408+0000 7f45d1ffb700 1 -- 172.21.15.142:0/3886964226 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f45c0010460 con 0x7f45d4104040 2024-03-29T04:42:47.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.408+0000 7f45d1ffb700 1 -- 172.21.15.142:0/3886964226 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45c000f5f0 con 0x7f45d4104040 2024-03-29T04:42:47.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.408+0000 7f45dabc7700 1 -- 172.21.15.142:0/3886964226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d419c000 con 0x7f45d4104040 2024-03-29T04:42:47.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.409+0000 7f45dabc7700 1 -- 172.21.15.142:0/3886964226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d4190560 con 0x7f45d4104040 2024-03-29T04:42:47.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.409+0000 7f45d1ffb700 1 -- 172.21.15.142:0/3886964226 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f45c0009a00 con 0x7f45d4104040 2024-03-29T04:42:47.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.409+0000 7f45d1ffb700 1 --2- 172.21.15.142:0/3886964226 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f45c406c4e0 0x7f45c406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:47.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.409+0000 7f45d1ffb700 1 -- 172.21.15.142:0/3886964226 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f45c008b730 con 0x7f45d4104040 2024-03-29T04:42:47.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.410+0000 7f45d8963700 1 --2- 172.21.15.142:0/3886964226 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f45c406c4e0 0x7f45c406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:47.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.410+0000 7f45d8963700 1 --2- 172.21.15.142:0/3886964226 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f45c406c4e0 0x7f45c406e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f45c8006010 tx=0x7f45c800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:47.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.414+0000 7f45d1ffb700 1 -- 172.21.15.142:0/3886964226 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f45c0059940 con 0x7f45d4104040 2024-03-29T04:42:47.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.606+0000 7f45dabc7700 1 -- 172.21.15.142:0/3886964226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f45d402cc70 con 0x7f45d4104040 2024-03-29T04:42:47.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.606+0000 7f45d1ffb700 1 -- 172.21.15.142:0/3886964226 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f45c00594d0 con 0x7f45d4104040 2024-03-29T04:42:47.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.609+0000 7f45dabc7700 1 -- 172.21.15.142:0/3886964226 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f45c406c4e0 msgr2=0x7f45c406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:47.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.609+0000 7f45dabc7700 1 --2- 172.21.15.142:0/3886964226 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f45c406c4e0 0x7f45c406e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f45c8006010 tx=0x7f45c800b540 comp rx=0 tx=0).stop 2024-03-29T04:42:47.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.609+0000 7f45dabc7700 1 -- 172.21.15.142:0/3886964226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4104040 msgr2=0x7f45d41968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:47.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.609+0000 7f45dabc7700 1 --2- 172.21.15.142:0/3886964226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4104040 0x7f45d41968f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f45c000d8d0 tx=0x7f45c000dbe0 comp rx=0 tx=0).stop 2024-03-29T04:42:47.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.609+0000 7f45dabc7700 1 -- 172.21.15.142:0/3886964226 shutdown_connections 2024-03-29T04:42:47.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.609+0000 7f45dabc7700 1 --2- 172.21.15.142:0/3886964226 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f45c406c4e0 0x7f45c406e9a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:47.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.609+0000 7f45dabc7700 1 --2- 172.21.15.142:0/3886964226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45d4101710 0x7f45d41963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:47.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.609+0000 7f45dabc7700 1 --2- 172.21.15.142:0/3886964226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4104040 0x7f45d41968f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:47.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.609+0000 7f45dabc7700 1 -- 172.21.15.142:0/3886964226 >> 172.21.15.142:0/3886964226 conn(0x7f45d40fb080 msgr2=0x7f45d40fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:47.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.609+0000 7f45dabc7700 1 -- 172.21.15.142:0/3886964226 shutdown_connections 2024-03-29T04:42:47.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:47.609+0000 7f45dabc7700 1 -- 172.21.15.142:0/3886964226 wait complete. 2024-03-29T04:42:47.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:47 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:42:47.948 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:47 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:42:48.311 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-29T04:42:48.560 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:49.002 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:48 smithi142 ceph-mon[29027]: pgmap v200: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-03-29T04:42:49.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:48 smithi152 ceph-mon[37177]: pgmap v200: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-03-29T04:42:50.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.224+0000 7fbcba2ce700 1 -- 172.21.15.142:0/1875864644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb40feca0 msgr2=0x7fbcb40ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:50.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.224+0000 7fbcba2ce700 1 --2- 172.21.15.142:0/1875864644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb40feca0 0x7fbcb40ff100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009910 tx=0x7fbca8009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:50.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.224+0000 7fbcba2ce700 1 -- 172.21.15.142:0/1875864644 shutdown_connections 2024-03-29T04:42:50.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.224+0000 7fbcba2ce700 1 --2- 172.21.15.142:0/1875864644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb40feca0 0x7fbcb40ff100 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:50.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.224+0000 7fbcba2ce700 1 --2- 172.21.15.142:0/1875864644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb4102220 0x7fbcb4102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:50.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.225+0000 7fbcba2ce700 1 -- 172.21.15.142:0/1875864644 >> 172.21.15.142:0/1875864644 conn(0x7fbcb40fa800 msgr2=0x7fbcb40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:50.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.225+0000 7fbcba2ce700 1 -- 172.21.15.142:0/1875864644 shutdown_connections 2024-03-29T04:42:50.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.225+0000 7fbcba2ce700 1 -- 172.21.15.142:0/1875864644 wait complete. 2024-03-29T04:42:50.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.225+0000 7fbcba2ce700 1 Processor -- start 2024-03-29T04:42:50.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.225+0000 7fbcba2ce700 1 -- start start 2024-03-29T04:42:50.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.225+0000 7fbcba2ce700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb40feca0 0x7fbcb4101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:50.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.225+0000 7fbcba2ce700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb4102220 0x7fbcb40fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:50.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.225+0000 7fbcba2ce700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb41004b0 con 0x7fbcb40feca0 2024-03-29T04:42:50.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.225+0000 7fbcba2ce700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb41005f0 con 0x7fbcb4102220 2024-03-29T04:42:50.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcb3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb40feca0 0x7fbcb4101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:50.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcb3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb40feca0 0x7fbcb4101920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43908/0 (socket says 172.21.15.142:43908) 2024-03-29T04:42:50.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcb3fff700 1 -- 172.21.15.142:0/360429540 learned_addr learned my addr 172.21.15.142:0/360429540 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:50.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcb37fe700 1 --2- 172.21.15.142:0/360429540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb4102220 0x7fbcb40fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:50.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcb3fff700 1 -- 172.21.15.142:0/360429540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb4102220 msgr2=0x7fbcb40fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:50.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcb3fff700 1 --2- 172.21.15.142:0/360429540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb4102220 0x7fbcb40fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:50.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcb3fff700 1 -- 172.21.15.142:0/360429540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80095f0 con 0x7fbcb40feca0 2024-03-29T04:42:50.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcb3fff700 1 --2- 172.21.15.142:0/360429540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb40feca0 0x7fbcb4101920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbca400b480 tx=0x7fbca400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:50.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcb17fa700 1 -- 172.21.15.142:0/360429540 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbca40118c0 con 0x7fbcb40feca0 2024-03-29T04:42:50.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcb17fa700 1 -- 172.21.15.142:0/360429540 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbca4010380 con 0x7fbcb40feca0 2024-03-29T04:42:50.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcba2ce700 1 -- 172.21.15.142:0/360429540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb41008d0 con 0x7fbcb40feca0 2024-03-29T04:42:50.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcb17fa700 1 -- 172.21.15.142:0/360429540 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbca400f4f0 con 0x7fbcb40feca0 2024-03-29T04:42:50.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.226+0000 7fbcba2ce700 1 -- 172.21.15.142:0/360429540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb419e460 con 0x7fbcb40feca0 2024-03-29T04:42:50.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.227+0000 7fbcba2ce700 1 -- 172.21.15.142:0/360429540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb4109800 con 0x7fbcb40feca0 2024-03-29T04:42:50.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.228+0000 7fbcb17fa700 1 -- 172.21.15.142:0/360429540 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbca40104f0 con 0x7fbcb40feca0 2024-03-29T04:42:50.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.228+0000 7fbcb17fa700 1 --2- 172.21.15.142:0/360429540 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbc9c06c4e0 0x7fbc9c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:50.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.228+0000 7fbcb17fa700 1 -- 172.21.15.142:0/360429540 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fbca408b4b0 con 0x7fbcb40feca0 2024-03-29T04:42:50.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.228+0000 7fbcb37fe700 1 --2- 172.21.15.142:0/360429540 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbc9c06c4e0 0x7fbc9c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:50.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.229+0000 7fbcb37fe700 1 --2- 172.21.15.142:0/360429540 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbc9c06c4e0 0x7fbc9c06e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbca80059d0 tx=0x7fbca8005920 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:50.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.233+0000 7fbcb17fa700 1 -- 172.21.15.142:0/360429540 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbca40596c0 con 0x7fbcb40feca0 2024-03-29T04:42:50.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.421+0000 7fbcba2ce700 1 -- 172.21.15.142:0/360429540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fbcb41099f0 con 0x7fbcb40feca0 2024-03-29T04:42:50.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.422+0000 7fbcb17fa700 1 -- 172.21.15.142:0/360429540 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7fbcb41099f0 con 0x7fbcb40feca0 2024-03-29T04:42:50.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.424+0000 7fbcba2ce700 1 -- 172.21.15.142:0/360429540 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbc9c06c4e0 msgr2=0x7fbc9c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:50.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.424+0000 7fbcba2ce700 1 --2- 172.21.15.142:0/360429540 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbc9c06c4e0 0x7fbc9c06e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbca80059d0 tx=0x7fbca8005920 comp rx=0 tx=0).stop 2024-03-29T04:42:50.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.424+0000 7fbcba2ce700 1 -- 172.21.15.142:0/360429540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb40feca0 msgr2=0x7fbcb4101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:50.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.424+0000 7fbcba2ce700 1 --2- 172.21.15.142:0/360429540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb40feca0 0x7fbcb4101920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbca400b480 tx=0x7fbca400b840 comp rx=0 tx=0).stop 2024-03-29T04:42:50.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.424+0000 7fbcba2ce700 1 -- 172.21.15.142:0/360429540 shutdown_connections 2024-03-29T04:42:50.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.424+0000 7fbcba2ce700 1 --2- 172.21.15.142:0/360429540 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fbc9c06c4e0 0x7fbc9c06e9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:50.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.424+0000 7fbcba2ce700 1 --2- 172.21.15.142:0/360429540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb40feca0 0x7fbcb4101920 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:50.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.424+0000 7fbcba2ce700 1 --2- 172.21.15.142:0/360429540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb4102220 0x7fbcb40fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:50.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.424+0000 7fbcba2ce700 1 -- 172.21.15.142:0/360429540 >> 172.21.15.142:0/360429540 conn(0x7fbcb40fa800 msgr2=0x7fbcb41051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:50.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.424+0000 7fbcba2ce700 1 -- 172.21.15.142:0/360429540 shutdown_connections 2024-03-29T04:42:50.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:50.424+0000 7fbcba2ce700 1 -- 172.21.15.142:0/360429540 wait complete. 2024-03-29T04:42:50.986 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:50 smithi142 ceph-mon[29027]: pgmap v201: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-03-29T04:42:51.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:50 smithi152 ceph-mon[37177]: pgmap v201: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-03-29T04:42:51.126 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-29T04:42:51.337 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:52.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.906+0000 7fef3bacc700 1 -- 172.21.15.142:0/3695732767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34100bc0 msgr2=0x7fef34100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:52.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.906+0000 7fef3bacc700 1 --2- 172.21.15.142:0/3695732767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34100bc0 0x7fef34100fe0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fef30009910 tx=0x7fef30009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:52.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.906+0000 7fef3bacc700 1 -- 172.21.15.142:0/3695732767 shutdown_connections 2024-03-29T04:42:52.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.906+0000 7fef3bacc700 1 --2- 172.21.15.142:0/3695732767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef34101dc0 0x7fef34102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:52.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.906+0000 7fef3bacc700 1 --2- 172.21.15.142:0/3695732767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34100bc0 0x7fef34100fe0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:52.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.906+0000 7fef3bacc700 1 -- 172.21.15.142:0/3695732767 >> 172.21.15.142:0/3695732767 conn(0x7fef340fc140 msgr2=0x7fef340fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:52.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.906+0000 7fef3bacc700 1 -- 172.21.15.142:0/3695732767 shutdown_connections 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.906+0000 7fef3bacc700 1 -- 172.21.15.142:0/3695732767 wait complete. 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef3bacc700 1 Processor -- start 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef3bacc700 1 -- start start 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef3bacc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef34100bc0 0x7fef34074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef3bacc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34101dc0 0x7fef34073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef3bacc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef340736c0 con 0x7fef34101dc0 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef3bacc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef34073800 con 0x7fef34100bc0 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef39067700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34101dc0 0x7fef34073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef39067700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34101dc0 0x7fef34073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43922/0 (socket says 172.21.15.142:43922) 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef39067700 1 -- 172.21.15.142:0/1629621002 learned_addr learned my addr 172.21.15.142:0/1629621002 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef39868700 1 --2- 172.21.15.142:0/1629621002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef34100bc0 0x7fef34074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef39067700 1 -- 172.21.15.142:0/1629621002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef34100bc0 msgr2=0x7fef34074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:52.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef39067700 1 --2- 172.21.15.142:0/1629621002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef34100bc0 0x7fef34074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:52.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef39067700 1 -- 172.21.15.142:0/1629621002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef300095f0 con 0x7fef34101dc0 2024-03-29T04:42:52.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef39067700 1 --2- 172.21.15.142:0/1629621002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34101dc0 0x7fef34073180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fef2400b4b0 tx=0x7fef2400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:52.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.907+0000 7fef2affd700 1 -- 172.21.15.142:0/1629621002 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef24011800 con 0x7fef34101dc0 2024-03-29T04:42:52.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.908+0000 7fef3bacc700 1 -- 172.21.15.142:0/1629621002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef34073ae0 con 0x7fef34101dc0 2024-03-29T04:42:52.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.908+0000 7fef2affd700 1 -- 172.21.15.142:0/1629621002 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fef24011e10 con 0x7fef34101dc0 2024-03-29T04:42:52.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.908+0000 7fef2affd700 1 -- 172.21.15.142:0/1629621002 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef2400f2e0 con 0x7fef34101dc0 2024-03-29T04:42:52.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.908+0000 7fef3bacc700 1 -- 172.21.15.142:0/1629621002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3419e450 con 0x7fef34101dc0 2024-03-29T04:42:52.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.909+0000 7fef3bacc700 1 -- 172.21.15.142:0/1629621002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef341097a0 con 0x7fef34101dc0 2024-03-29T04:42:52.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.909+0000 7fef2affd700 1 -- 172.21.15.142:0/1629621002 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fef24010380 con 0x7fef34101dc0 2024-03-29T04:42:52.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.909+0000 7fef2affd700 1 --2- 172.21.15.142:0/1629621002 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fef2006c4e0 0x7fef2006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:52.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.910+0000 7fef2affd700 1 -- 172.21.15.142:0/1629621002 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fef2408b430 con 0x7fef34101dc0 2024-03-29T04:42:52.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.910+0000 7fef39868700 1 --2- 172.21.15.142:0/1629621002 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fef2006c4e0 0x7fef2006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:52.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.910+0000 7fef39868700 1 --2- 172.21.15.142:0/1629621002 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fef2006c4e0 0x7fef2006e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fef30006010 tx=0x7fef3000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:52.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:52.914+0000 7fef2affd700 1 -- 172.21.15.142:0/1629621002 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fef24059640 con 0x7fef34101dc0 2024-03-29T04:42:53.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:52 smithi152 ceph-mon[37177]: pgmap v202: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-29T04:42:53.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.104+0000 7fef3bacc700 1 -- 172.21.15.142:0/1629621002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fef3419ea00 con 0x7fef34101dc0 2024-03-29T04:42:53.102 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:52 smithi142 ceph-mon[29027]: pgmap v202: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-29T04:42:53.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.105+0000 7fef2affd700 1 -- 172.21.15.142:0/1629621002 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fef240591d0 con 0x7fef34101dc0 2024-03-29T04:42:53.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.107+0000 7fef3bacc700 1 -- 172.21.15.142:0/1629621002 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fef2006c4e0 msgr2=0x7fef2006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:53.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.107+0000 7fef3bacc700 1 --2- 172.21.15.142:0/1629621002 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fef2006c4e0 0x7fef2006e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fef30006010 tx=0x7fef3000b540 comp rx=0 tx=0).stop 2024-03-29T04:42:53.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.107+0000 7fef3bacc700 1 -- 172.21.15.142:0/1629621002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34101dc0 msgr2=0x7fef34073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:53.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.107+0000 7fef3bacc700 1 --2- 172.21.15.142:0/1629621002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34101dc0 0x7fef34073180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fef2400b4b0 tx=0x7fef2400b7c0 comp rx=0 tx=0).stop 2024-03-29T04:42:53.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.107+0000 7fef3bacc700 1 -- 172.21.15.142:0/1629621002 shutdown_connections 2024-03-29T04:42:53.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.107+0000 7fef3bacc700 1 --2- 172.21.15.142:0/1629621002 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fef2006c4e0 0x7fef2006e9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:53.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.107+0000 7fef3bacc700 1 --2- 172.21.15.142:0/1629621002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef34100bc0 0x7fef34074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:53.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.107+0000 7fef3bacc700 1 --2- 172.21.15.142:0/1629621002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34101dc0 0x7fef34073180 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:53.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.107+0000 7fef3bacc700 1 -- 172.21.15.142:0/1629621002 >> 172.21.15.142:0/1629621002 conn(0x7fef340fc140 msgr2=0x7fef34104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:53.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.107+0000 7fef3bacc700 1 -- 172.21.15.142:0/1629621002 shutdown_connections 2024-03-29T04:42:53.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:53.107+0000 7fef3bacc700 1 -- 172.21.15.142:0/1629621002 wait complete. 2024-03-29T04:42:53.817 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-03-29T04:42:54.026 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:42:54.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:54 smithi152 ceph-mon[37177]: pgmap v203: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-29T04:42:54.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:54 smithi142 ceph-mon[29027]: pgmap v203: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-29T04:42:55.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.790+0000 7f3462932700 1 -- 172.21.15.142:0/1939154086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345c101e00 msgr2=0x7f345c102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:55.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.790+0000 7f3462932700 1 --2- 172.21.15.142:0/1939154086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345c101e00 0x7f345c102260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3450009910 tx=0x7f3450009c20 comp rx=0 tx=0).stop 2024-03-29T04:42:55.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.790+0000 7f3462932700 1 -- 172.21.15.142:0/1939154086 shutdown_connections 2024-03-29T04:42:55.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.790+0000 7f3462932700 1 --2- 172.21.15.142:0/1939154086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345c101e00 0x7f345c102260 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.790+0000 7f3462932700 1 --2- 172.21.15.142:0/1939154086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345c100c00 0x7f345c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.790+0000 7f3462932700 1 -- 172.21.15.142:0/1939154086 >> 172.21.15.142:0/1939154086 conn(0x7f345c0fc180 msgr2=0x7f345c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.790+0000 7f3462932700 1 -- 172.21.15.142:0/1939154086 shutdown_connections 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.790+0000 7f3462932700 1 -- 172.21.15.142:0/1939154086 wait complete. 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.790+0000 7f3462932700 1 Processor -- start 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f3462932700 1 -- start start 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f3462932700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345c100c00 0x7f345c196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f3462932700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345c101e00 0x7f345c196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f3462932700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c196f80 con 0x7f345c101e00 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f3462932700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c1970c0 con 0x7f345c100c00 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f345b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345c101e00 0x7f345c196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f345b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345c101e00 0x7f345c196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43940/0 (socket says 172.21.15.142:43940) 2024-03-29T04:42:55.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f345b7fe700 1 -- 172.21.15.142:0/771742627 learned_addr learned my addr 172.21.15.142:0/771742627 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:42:55.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f345bfff700 1 --2- 172.21.15.142:0/771742627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345c100c00 0x7f345c196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:55.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f345b7fe700 1 -- 172.21.15.142:0/771742627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345c100c00 msgr2=0x7f345c196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:42:55.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f345b7fe700 1 --2- 172.21.15.142:0/771742627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345c100c00 0x7f345c196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:42:55.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f345b7fe700 1 -- 172.21.15.142:0/771742627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500095f0 con 0x7f345c101e00 2024-03-29T04:42:55.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.791+0000 7f345bfff700 1 --2- 172.21.15.142:0/771742627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345c100c00 0x7f345c196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:42:55.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.792+0000 7f345b7fe700 1 --2- 172.21.15.142:0/771742627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345c101e00 0x7f345c196960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3450006010 tx=0x7f345000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:55.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.792+0000 7f34597fa700 1 -- 172.21.15.142:0/771742627 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f345001d070 con 0x7f345c101e00 2024-03-29T04:42:55.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.792+0000 7f3462932700 1 -- 172.21.15.142:0/771742627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345c19bb10 con 0x7f345c101e00 2024-03-29T04:42:55.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.792+0000 7f34597fa700 1 -- 172.21.15.142:0/771742627 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f345000f460 con 0x7f345c101e00 2024-03-29T04:42:55.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.792+0000 7f34597fa700 1 -- 172.21.15.142:0/771742627 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3450021640 con 0x7f345c101e00 2024-03-29T04:42:55.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.792+0000 7f3462932700 1 -- 172.21.15.142:0/771742627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345c19c080 con 0x7f345c101e00 2024-03-29T04:42:55.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.793+0000 7f3462932700 1 -- 172.21.15.142:0/771742627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345c109800 con 0x7f345c101e00 2024-03-29T04:42:55.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.793+0000 7f34597fa700 1 -- 172.21.15.142:0/771742627 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f345000fab0 con 0x7f345c101e00 2024-03-29T04:42:55.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.793+0000 7f34597fa700 1 --2- 172.21.15.142:0/771742627 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f344406c4e0 0x7f344406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:42:55.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.794+0000 7f34597fa700 1 -- 172.21.15.142:0/771742627 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f345008cd20 con 0x7f345c101e00 2024-03-29T04:42:55.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.794+0000 7f345bfff700 1 --2- 172.21.15.142:0/771742627 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f344406c4e0 0x7f344406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:42:55.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.794+0000 7f345bfff700 1 --2- 172.21.15.142:0/771742627 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f344406c4e0 0x7f344406e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f345c101c60 tx=0x7f344c008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:42:55.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:55.798+0000 7f34597fa700 1 -- 172.21.15.142:0/771742627 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f345005af30 con 0x7f345c101e00 2024-03-29T04:42:56.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:42:56.005+0000 7f3462932700 1 -- 172.21.15.142:0/771742627 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f345c106800 con 0x7f344406c4e0 2024-03-29T04:42:56.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:56 smithi142 ceph-mon[29027]: pgmap v204: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-29T04:42:56.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:56 smithi142 ceph-mon[29027]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:42:57.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:56 smithi152 ceph-mon[37177]: pgmap v204: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-29T04:42:57.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:56 smithi152 ceph-mon[37177]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:42:59.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:42:58 smithi152 ceph-mon[37177]: pgmap v205: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-29T04:42:59.118 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:42:58 smithi142 ceph-mon[29027]: pgmap v205: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-29T04:43:01.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:00 smithi152 ceph-mon[37177]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-29T04:43:01.118 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:00 smithi142 ceph-mon[29027]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-29T04:43:03.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:02 smithi152 ceph-mon[37177]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 3 op/s 2024-03-29T04:43:03.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:02 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:43:03.118 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:02 smithi142 ceph-mon[29027]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 3 op/s 2024-03-29T04:43:03.118 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:02 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:43:04.537 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:04 smithi152 ceph-mon[37177]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-29T04:43:04.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:04 smithi142 ceph-mon[29027]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-29T04:43:06.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:06 smithi142 ceph-mon[29027]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-29T04:43:07.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:06 smithi152 ceph-mon[37177]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-29T04:43:09.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:08 smithi152 ceph-mon[37177]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-29T04:43:09.118 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:08 smithi142 ceph-mon[29027]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-29T04:43:11.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:10 smithi152 ceph-mon[37177]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-29T04:43:11.118 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:10 smithi142 ceph-mon[29027]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-29T04:43:13.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:12 smithi152 ceph-mon[37177]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:43:13.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:12 smithi142 ceph-mon[29027]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:43:14.788 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:14 smithi152 ceph-mon[37177]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:14.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:14 smithi142 ceph-mon[29027]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:17.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:16 smithi152 ceph-mon[37177]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:17.118 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:16 smithi142 ceph-mon[29027]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:18.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:17 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:43:18.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:43:19.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:18 smithi152 ceph-mon[37177]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:43:19.118 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:18 smithi142 ceph-mon[29027]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:43:21.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:20 smithi152 ceph-mon[37177]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:21.118 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:20 smithi142 ceph-mon[29027]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:23.037 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:22 smithi152 ceph-mon[37177]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:43:23.118 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:22 smithi142 ceph-mon[29027]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:43:24.788 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:24 smithi152 ceph-mon[37177]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:24.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:24 smithi142 ceph-mon[29027]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:27.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:26 smithi152 ceph-mon[37177]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:27.118 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:26 smithi142 ceph-mon[29027]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:29.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:28 smithi152 ceph-mon[37177]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:43:29.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:28 smithi142 ceph-mon[29027]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:43:31.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:30 smithi152 ceph-mon[37177]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:31.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:30 smithi142 ceph-mon[29027]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:33.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:32 smithi152 ceph-mon[37177]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:43:33.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:43:33.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:32 smithi142 ceph-mon[29027]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:43:33.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:43:34.788 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:34 smithi152 ceph-mon[37177]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:34.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:34 smithi142 ceph-mon[29027]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:37.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:36 smithi152 ceph-mon[37177]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:37.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:36 smithi142 ceph-mon[29027]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:39.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:38 smithi152 ceph-mon[37177]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:43:39.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:43:39.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:38 smithi142 ceph-mon[29027]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:43:39.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:43:41.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:40 smithi152 ceph-mon[37177]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:41.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:40 smithi142 ceph-mon[29027]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:43.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:42 smithi152 ceph-mon[37177]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:43:43.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:42 smithi142 ceph-mon[29027]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:43:44.733 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:44 smithi142 ceph-mon[29027]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:44.733 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:44 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:43:44.733 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:44 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:43:44.733 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:44 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:43:44.750 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:44 smithi152 ceph-mon[37177]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:44.750 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:44 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:43:44.750 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:44 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:43:44.750 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:44 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:43:46.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:46.152+0000 7f34597fa700 1 -- 172.21.15.142:0/771742627 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f345c106800 con 0x7f344406c4e0 2024-03-29T04:43:46.151 INFO:teuthology.orchestra.run.smithi142.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:43:46.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:46.155+0000 7f3462932700 1 -- 172.21.15.142:0/771742627 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f344406c4e0 msgr2=0x7f344406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:46.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:46.155+0000 7f3462932700 1 --2- 172.21.15.142:0/771742627 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f344406c4e0 0x7f344406e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f345c101c60 tx=0x7f344c008040 comp rx=0 tx=0).stop 2024-03-29T04:43:46.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:46.155+0000 7f3462932700 1 -- 172.21.15.142:0/771742627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345c101e00 msgr2=0x7f345c196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:46.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:46.155+0000 7f3462932700 1 --2- 172.21.15.142:0/771742627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345c101e00 0x7f345c196960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3450006010 tx=0x7f345000be20 comp rx=0 tx=0).stop 2024-03-29T04:43:46.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:46.155+0000 7f3462932700 1 -- 172.21.15.142:0/771742627 shutdown_connections 2024-03-29T04:43:46.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:46.155+0000 7f3462932700 1 --2- 172.21.15.142:0/771742627 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f344406c4e0 0x7f344406e9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:46.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:46.155+0000 7f3462932700 1 --2- 172.21.15.142:0/771742627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345c100c00 0x7f345c196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:46.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:46.155+0000 7f3462932700 1 --2- 172.21.15.142:0/771742627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f345c101e00 0x7f345c196960 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:46.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:46.155+0000 7f3462932700 1 -- 172.21.15.142:0/771742627 >> 172.21.15.142:0/771742627 conn(0x7f345c0fc180 msgr2=0x7f345c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:43:46.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:46.155+0000 7f3462932700 1 -- 172.21.15.142:0/771742627 shutdown_connections 2024-03-29T04:43:46.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:46.155+0000 7f3462932700 1 -- 172.21.15.142:0/771742627 wait complete. 2024-03-29T04:43:46.747 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:46 smithi142 ceph-mon[29027]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:46.747 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:46 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:43:46.747 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:46 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:43:46.747 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:46 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:43:46.747 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:46 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:43:46.747 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:46 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:43:46.782 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-03-29T04:43:47.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:46 smithi152 ceph-mon[37177]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:47.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:46 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:43:47.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:46 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:43:47.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:46 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:43:47.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:46 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:43:47.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:46 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:43:47.535 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:43:48.015 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:47 smithi142 ceph-mon[29027]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:43:48.016 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:47 smithi142 ceph-mon[29027]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:43:48.016 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:47 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:43:48.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:47 smithi152 ceph-mon[37177]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:43:48.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:47 smithi152 ceph-mon[37177]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:43:48.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:47 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:43:49.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:48 smithi152 ceph-mon[37177]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:43:49.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:43:49.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:43:49.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:48 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:43:49.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:48 smithi142 ceph-mon[29027]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:43:49.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:43:49.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:43:49.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:48 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:43:49.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.722+0000 7f9e7ce81700 1 -- 172.21.15.142:0/786824560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e780ffab0 msgr2=0x7f9e780ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:49.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.722+0000 7f9e7ce81700 1 --2- 172.21.15.142:0/786824560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e780ffab0 0x7f9e780ffed0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009910 tx=0x7f9e6c009c20 comp rx=0 tx=0).stop 2024-03-29T04:43:49.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.722+0000 7f9e7ce81700 1 -- 172.21.15.142:0/786824560 shutdown_connections 2024-03-29T04:43:49.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.722+0000 7f9e7ce81700 1 --2- 172.21.15.142:0/786824560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e78100cb0 0x7f9e78101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:49.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.722+0000 7f9e7ce81700 1 --2- 172.21.15.142:0/786824560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e780ffab0 0x7f9e780ffed0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:49.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.722+0000 7f9e7ce81700 1 -- 172.21.15.142:0/786824560 >> 172.21.15.142:0/786824560 conn(0x7f9e780fb050 msgr2=0x7f9e780fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:43:49.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.722+0000 7f9e7ce81700 1 -- 172.21.15.142:0/786824560 shutdown_connections 2024-03-29T04:43:49.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.722+0000 7f9e7ce81700 1 -- 172.21.15.142:0/786824560 wait complete. 2024-03-29T04:43:49.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.723+0000 7f9e7ce81700 1 Processor -- start 2024-03-29T04:43:49.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.723+0000 7f9e7ce81700 1 -- start start 2024-03-29T04:43:49.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.724+0000 7f9e7ce81700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e780ffab0 0x7f9e78194150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:49.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.724+0000 7f9e7ce81700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e78100cb0 0x7f9e78194690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:49.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.724+0000 7f9e7ce81700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78194c60 con 0x7f9e78100cb0 2024-03-29T04:43:49.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.724+0000 7f9e7ce81700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78194da0 con 0x7f9e780ffab0 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.724+0000 7f9e75d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e78100cb0 0x7f9e78194690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.724+0000 7f9e7659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e780ffab0 0x7f9e78194150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.724+0000 7f9e75d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e78100cb0 0x7f9e78194690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33380/0 (socket says 172.21.15.142:33380) 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.724+0000 7f9e75d9b700 1 -- 172.21.15.142:0/4086552477 learned_addr learned my addr 172.21.15.142:0/4086552477 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.724+0000 7f9e7659c700 1 -- 172.21.15.142:0/4086552477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e78100cb0 msgr2=0x7f9e78194690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.724+0000 7f9e7659c700 1 --2- 172.21.15.142:0/4086552477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e78100cb0 0x7f9e78194690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.724+0000 7f9e7659c700 1 -- 172.21.15.142:0/4086552477 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c0095f0 con 0x7f9e780ffab0 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.724+0000 7f9e75d9b700 1 --2- 172.21.15.142:0/4086552477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e78100cb0 0x7f9e78194690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.725+0000 7f9e7659c700 1 --2- 172.21.15.142:0/4086552477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e780ffab0 0x7f9e78194150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c000c00 tx=0x7f9e6c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.725+0000 7f9e677fe700 1 -- 172.21.15.142:0/4086552477 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e6c01d070 con 0x7f9e780ffab0 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.725+0000 7f9e7ce81700 1 -- 172.21.15.142:0/4086552477 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7806a870 con 0x7f9e780ffab0 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.725+0000 7f9e677fe700 1 -- 172.21.15.142:0/4086552477 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9e6c005dc0 con 0x7f9e780ffab0 2024-03-29T04:43:49.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.725+0000 7f9e677fe700 1 -- 172.21.15.142:0/4086552477 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e6c022470 con 0x7f9e780ffab0 2024-03-29T04:43:49.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.725+0000 7f9e7ce81700 1 -- 172.21.15.142:0/4086552477 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7806ad30 con 0x7f9e780ffab0 2024-03-29T04:43:49.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.726+0000 7f9e7ce81700 1 -- 172.21.15.142:0/4086552477 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e78107500 con 0x7f9e780ffab0 2024-03-29T04:43:49.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.726+0000 7f9e677fe700 1 -- 172.21.15.142:0/4086552477 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9e6c0050d0 con 0x7f9e780ffab0 2024-03-29T04:43:49.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.727+0000 7f9e677fe700 1 --2- 172.21.15.142:0/4086552477 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f9e6006c4e0 0x7f9e6006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:49.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.727+0000 7f9e677fe700 1 -- 172.21.15.142:0/4086552477 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9e6c08cdd0 con 0x7f9e780ffab0 2024-03-29T04:43:49.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.727+0000 7f9e75d9b700 1 --2- 172.21.15.142:0/4086552477 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f9e6006c4e0 0x7f9e6006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:49.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.728+0000 7f9e75d9b700 1 --2- 172.21.15.142:0/4086552477 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f9e6006c4e0 0x7f9e6006e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800b3c0 tx=0x7f9e6800d040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:43:49.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:49.736+0000 7f9e677fe700 1 -- 172.21.15.142:0/4086552477 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9e6c05af40 con 0x7f9e780ffab0 2024-03-29T04:43:49.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:49 smithi142 ceph-mon[29027]: Upgrade: Target is version 19.0.0-1959-g8da23624 (unknown) 2024-03-29T04:43:49.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:49 smithi142 ceph-mon[29027]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28'] 2024-03-29T04:43:49.870 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:49 smithi142 ceph-mon[29027]: Upgrade: Need to upgrade myself (mgr.smithi142.swesrg) 2024-03-29T04:43:49.870 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:49 smithi142 ceph-mon[29027]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 on smithi152 2024-03-29T04:43:50.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.010+0000 7f9e7ce81700 1 -- 172.21.15.142:0/4086552477 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e78105600 con 0x7f9e6006c4e0 2024-03-29T04:43:50.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.011+0000 7f9e677fe700 1 -- 172.21.15.142:0/4086552477 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f9e78105600 con 0x7f9e6006c4e0 2024-03-29T04:43:50.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.013+0000 7f9e7ce81700 1 -- 172.21.15.142:0/4086552477 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f9e6006c4e0 msgr2=0x7f9e6006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:50.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.013+0000 7f9e7ce81700 1 --2- 172.21.15.142:0/4086552477 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f9e6006c4e0 0x7f9e6006e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800b3c0 tx=0x7f9e6800d040 comp rx=0 tx=0).stop 2024-03-29T04:43:50.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.013+0000 7f9e7ce81700 1 -- 172.21.15.142:0/4086552477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e780ffab0 msgr2=0x7f9e78194150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:50.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.013+0000 7f9e7ce81700 1 --2- 172.21.15.142:0/4086552477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e780ffab0 0x7f9e78194150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c000c00 tx=0x7f9e6c00fa00 comp rx=0 tx=0).stop 2024-03-29T04:43:50.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.013+0000 7f9e7ce81700 1 -- 172.21.15.142:0/4086552477 shutdown_connections 2024-03-29T04:43:50.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.013+0000 7f9e7ce81700 1 --2- 172.21.15.142:0/4086552477 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f9e6006c4e0 0x7f9e6006e9a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.013+0000 7f9e7ce81700 1 --2- 172.21.15.142:0/4086552477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e780ffab0 0x7f9e78194150 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.013+0000 7f9e7ce81700 1 --2- 172.21.15.142:0/4086552477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e78100cb0 0x7f9e78194690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.013+0000 7f9e7ce81700 1 -- 172.21.15.142:0/4086552477 >> 172.21.15.142:0/4086552477 conn(0x7f9e780fb050 msgr2=0x7f9e78103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:43:50.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.013+0000 7f9e7ce81700 1 -- 172.21.15.142:0/4086552477 shutdown_connections 2024-03-29T04:43:50.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.013+0000 7f9e7ce81700 1 -- 172.21.15.142:0/4086552477 wait complete. 2024-03-29T04:43:50.023 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:43:50.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:49 smithi152 ceph-mon[37177]: Upgrade: Target is version 19.0.0-1959-g8da23624 (unknown) 2024-03-29T04:43:50.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:49 smithi152 ceph-mon[37177]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28'] 2024-03-29T04:43:50.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:49 smithi152 ceph-mon[37177]: Upgrade: Need to upgrade myself (mgr.smithi142.swesrg) 2024-03-29T04:43:50.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:49 smithi152 ceph-mon[37177]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 on smithi152 2024-03-29T04:43:50.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.162+0000 7fe9ae65d700 1 -- 172.21.15.142:0/3434276510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9a8100bb0 msgr2=0x7fe9a8100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:50.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.162+0000 7fe9ae65d700 1 --2- 172.21.15.142:0/3434276510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9a8100bb0 0x7fe9a8100fd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe998009910 tx=0x7fe998009c20 comp rx=0 tx=0).stop 2024-03-29T04:43:50.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.162+0000 7fe9ae65d700 1 -- 172.21.15.142:0/3434276510 shutdown_connections 2024-03-29T04:43:50.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.162+0000 7fe9ae65d700 1 --2- 172.21.15.142:0/3434276510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9a8101db0 0x7fe9a8102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.162+0000 7fe9ae65d700 1 --2- 172.21.15.142:0/3434276510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9a8100bb0 0x7fe9a8100fd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.162+0000 7fe9ae65d700 1 -- 172.21.15.142:0/3434276510 >> 172.21.15.142:0/3434276510 conn(0x7fe9a80fc110 msgr2=0x7fe9a80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:43:50.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.163+0000 7fe9ae65d700 1 -- 172.21.15.142:0/3434276510 shutdown_connections 2024-03-29T04:43:50.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.163+0000 7fe9ae65d700 1 -- 172.21.15.142:0/3434276510 wait complete. 2024-03-29T04:43:50.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.164+0000 7fe9ae65d700 1 Processor -- start 2024-03-29T04:43:50.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.164+0000 7fe9ae65d700 1 -- start start 2024-03-29T04:43:50.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.164+0000 7fe9ae65d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9a8100bb0 0x7fe9a8074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:50.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.164+0000 7fe9ae65d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9a8101db0 0x7fe9a8073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:50.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.164+0000 7fe9ae65d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80736c0 con 0x7fe9a8100bb0 2024-03-29T04:43:50.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.165+0000 7fe9ae65d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a8073800 con 0x7fe9a8101db0 2024-03-29T04:43:50.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.165+0000 7fe9a7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9a8100bb0 0x7fe9a8074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:50.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.165+0000 7fe9a7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9a8100bb0 0x7fe9a8074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33390/0 (socket says 172.21.15.142:33390) 2024-03-29T04:43:50.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.165+0000 7fe9a7fff700 1 -- 172.21.15.142:0/1098017890 learned_addr learned my addr 172.21.15.142:0/1098017890 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:43:50.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.165+0000 7fe9a77fe700 1 --2- 172.21.15.142:0/1098017890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9a8101db0 0x7fe9a8073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:50.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.165+0000 7fe9a7fff700 1 -- 172.21.15.142:0/1098017890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9a8101db0 msgr2=0x7fe9a8073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:50.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.165+0000 7fe9a7fff700 1 --2- 172.21.15.142:0/1098017890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9a8101db0 0x7fe9a8073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.165+0000 7fe9a7fff700 1 -- 172.21.15.142:0/1098017890 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9980095f0 con 0x7fe9a8100bb0 2024-03-29T04:43:50.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.166+0000 7fe9a77fe700 1 --2- 172.21.15.142:0/1098017890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9a8101db0 0x7fe9a8073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:43:50.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.166+0000 7fe9a7fff700 1 --2- 172.21.15.142:0/1098017890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9a8100bb0 0x7fe9a8074b30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe998000c00 tx=0x7fe99800fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:43:50.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.166+0000 7fe9a57fa700 1 -- 172.21.15.142:0/1098017890 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe99801d070 con 0x7fe9a8100bb0 2024-03-29T04:43:50.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.166+0000 7fe9a57fa700 1 -- 172.21.15.142:0/1098017890 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe998005640 con 0x7fe9a8100bb0 2024-03-29T04:43:50.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.166+0000 7fe9a57fa700 1 -- 172.21.15.142:0/1098017890 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9980226a0 con 0x7fe9a8100bb0 2024-03-29T04:43:50.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.167+0000 7fe9ae65d700 1 -- 172.21.15.142:0/1098017890 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a8073a80 con 0x7fe9a8100bb0 2024-03-29T04:43:50.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.167+0000 7fe9ae65d700 1 -- 172.21.15.142:0/1098017890 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a8073f70 con 0x7fe9a8100bb0 2024-03-29T04:43:50.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.168+0000 7fe9ae65d700 1 -- 172.21.15.142:0/1098017890 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a8109710 con 0x7fe9a8100bb0 2024-03-29T04:43:50.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.169+0000 7fe9a57fa700 1 -- 172.21.15.142:0/1098017890 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe998003680 con 0x7fe9a8100bb0 2024-03-29T04:43:50.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.173+0000 7fe9a57fa700 1 --2- 172.21.15.142:0/1098017890 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe99006c530 0x7fe99006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:50.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.173+0000 7fe9a57fa700 1 -- 172.21.15.142:0/1098017890 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe99808c7c0 con 0x7fe9a8100bb0 2024-03-29T04:43:50.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.173+0000 7fe9a77fe700 1 --2- 172.21.15.142:0/1098017890 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe99006c530 0x7fe99006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:50.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.174+0000 7fe9a77fe700 1 --2- 172.21.15.142:0/1098017890 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe99006c530 0x7fe99006e9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009be0 tx=0x7fe99c009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:43:50.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.178+0000 7fe9a57fa700 1 -- 172.21.15.142:0/1098017890 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe99805aaf0 con 0x7fe9a8100bb0 2024-03-29T04:43:50.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.395+0000 7fe9ae65d700 1 -- 172.21.15.142:0/1098017890 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9a8106700 con 0x7fe99006c530 2024-03-29T04:43:50.394 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.396+0000 7fe9a57fa700 1 -- 172.21.15.142:0/1098017890 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fe9a8106700 con 0x7fe99006c530 2024-03-29T04:43:50.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.399+0000 7fe9ae65d700 1 -- 172.21.15.142:0/1098017890 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe99006c530 msgr2=0x7fe99006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:50.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.399+0000 7fe9ae65d700 1 --2- 172.21.15.142:0/1098017890 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe99006c530 0x7fe99006e9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009be0 tx=0x7fe99c009260 comp rx=0 tx=0).stop 2024-03-29T04:43:50.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.399+0000 7fe9ae65d700 1 -- 172.21.15.142:0/1098017890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9a8100bb0 msgr2=0x7fe9a8074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:50.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.399+0000 7fe9ae65d700 1 --2- 172.21.15.142:0/1098017890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9a8100bb0 0x7fe9a8074b30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe998000c00 tx=0x7fe99800fb00 comp rx=0 tx=0).stop 2024-03-29T04:43:50.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.400+0000 7fe9ae65d700 1 -- 172.21.15.142:0/1098017890 shutdown_connections 2024-03-29T04:43:50.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.400+0000 7fe9ae65d700 1 --2- 172.21.15.142:0/1098017890 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe99006c530 0x7fe99006e9f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.400+0000 7fe9ae65d700 1 --2- 172.21.15.142:0/1098017890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9a8100bb0 0x7fe9a8074b30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.400+0000 7fe9ae65d700 1 --2- 172.21.15.142:0/1098017890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9a8101db0 0x7fe9a8073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.400+0000 7fe9ae65d700 1 -- 172.21.15.142:0/1098017890 >> 172.21.15.142:0/1098017890 conn(0x7fe9a80fc110 msgr2=0x7fe9a8104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:43:50.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.400+0000 7fe9ae65d700 1 -- 172.21.15.142:0/1098017890 shutdown_connections 2024-03-29T04:43:50.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.400+0000 7fe9ae65d700 1 -- 172.21.15.142:0/1098017890 wait complete. 2024-03-29T04:43:50.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.510+0000 7f5faf0fd700 1 -- 172.21.15.142:0/386115990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fa80ffd50 msgr2=0x7f5fa81001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:50.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.510+0000 7f5faf0fd700 1 --2- 172.21.15.142:0/386115990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fa80ffd50 0x7f5fa81001b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009910 tx=0x7f5f94009c20 comp rx=0 tx=0).stop 2024-03-29T04:43:50.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.510+0000 7f5faf0fd700 1 -- 172.21.15.142:0/386115990 shutdown_connections 2024-03-29T04:43:50.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.510+0000 7f5faf0fd700 1 --2- 172.21.15.142:0/386115990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fa80ffd50 0x7f5fa81001b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.510+0000 7f5faf0fd700 1 --2- 172.21.15.142:0/386115990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fa81010a0 0x7f5fa81014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.510+0000 7f5faf0fd700 1 -- 172.21.15.142:0/386115990 >> 172.21.15.142:0/386115990 conn(0x7f5fa80fa7c0 msgr2=0x7f5fa80fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:43:50.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.510+0000 7f5faf0fd700 1 -- 172.21.15.142:0/386115990 shutdown_connections 2024-03-29T04:43:50.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.510+0000 7f5faf0fd700 1 -- 172.21.15.142:0/386115990 wait complete. 2024-03-29T04:43:50.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.511+0000 7f5faf0fd700 1 Processor -- start 2024-03-29T04:43:50.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.511+0000 7f5faf0fd700 1 -- start start 2024-03-29T04:43:50.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.512+0000 7f5faf0fd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fa80ffd50 0x7f5fa8196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:50.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.512+0000 7f5faf0fd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fa81010a0 0x7f5fa81968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:50.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.512+0000 7f5faf0fd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa8196ed0 con 0x7f5fa80ffd50 2024-03-29T04:43:50.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.512+0000 7f5faf0fd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa8197010 con 0x7f5fa81010a0 2024-03-29T04:43:50.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.512+0000 7f5face99700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fa80ffd50 0x7f5fa8196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:50.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.512+0000 7f5face99700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fa80ffd50 0x7f5fa8196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33412/0 (socket says 172.21.15.142:33412) 2024-03-29T04:43:50.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.512+0000 7f5face99700 1 -- 172.21.15.142:0/2733744334 learned_addr learned my addr 172.21.15.142:0/2733744334 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:43:50.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.512+0000 7f5fa7fff700 1 --2- 172.21.15.142:0/2733744334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fa81010a0 0x7f5fa81968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:50.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.512+0000 7f5face99700 1 -- 172.21.15.142:0/2733744334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fa81010a0 msgr2=0x7f5fa81968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:50.512 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.512+0000 7f5face99700 1 --2- 172.21.15.142:0/2733744334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fa81010a0 0x7f5fa81968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.512 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.512+0000 7f5face99700 1 -- 172.21.15.142:0/2733744334 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f940095f0 con 0x7f5fa80ffd50 2024-03-29T04:43:50.512 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.512+0000 7f5face99700 1 --2- 172.21.15.142:0/2733744334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fa80ffd50 0x7f5fa8196370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00e8f0 tx=0x7f5f9c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:43:50.512 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.513+0000 7f5fa5ffb700 1 -- 172.21.15.142:0/2733744334 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f9c009790 con 0x7f5fa80ffd50 2024-03-29T04:43:50.512 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.513+0000 7f5faf0fd700 1 -- 172.21.15.142:0/2733744334 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa8101fe0 con 0x7f5fa80ffd50 2024-03-29T04:43:50.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.513+0000 7f5fa5ffb700 1 -- 172.21.15.142:0/2733744334 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5f9c00c810 con 0x7f5fa80ffd50 2024-03-29T04:43:50.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.513+0000 7f5fa5ffb700 1 -- 172.21.15.142:0/2733744334 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f9c003d20 con 0x7f5fa80ffd50 2024-03-29T04:43:50.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.513+0000 7f5faf0fd700 1 -- 172.21.15.142:0/2733744334 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa8102530 con 0x7f5fa80ffd50 2024-03-29T04:43:50.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.514+0000 7f5faf0fd700 1 -- 172.21.15.142:0/2733744334 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa8108620 con 0x7f5fa80ffd50 2024-03-29T04:43:50.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.515+0000 7f5fa5ffb700 1 -- 172.21.15.142:0/2733744334 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f5f9c0041f0 con 0x7f5fa80ffd50 2024-03-29T04:43:50.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.515+0000 7f5fa5ffb700 1 --2- 172.21.15.142:0/2733744334 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f5f9806c4e0 0x7f5f9806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:50.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.515+0000 7f5fa5ffb700 1 -- 172.21.15.142:0/2733744334 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5f9c014070 con 0x7f5fa80ffd50 2024-03-29T04:43:50.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.516+0000 7f5fa7fff700 1 --2- 172.21.15.142:0/2733744334 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f5f9806c4e0 0x7f5f9806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:50.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.516+0000 7f5fa7fff700 1 --2- 172.21.15.142:0/2733744334 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f5f9806c4e0 0x7f5f9806e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009fd0 tx=0x7f5f94005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:43:50.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.523+0000 7f5fa5ffb700 1 -- 172.21.15.142:0/2733744334 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5f9c059d70 con 0x7f5fa80ffd50 2024-03-29T04:43:50.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:50 smithi142 ceph-mon[29027]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:50.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:50 smithi142 ceph-mon[29027]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:43:50.874 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.876+0000 7f5faf0fd700 1 -- 172.21.15.142:0/2733744334 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fa80ff350 con 0x7f5f9806c4e0 2024-03-29T04:43:50.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.888+0000 7f5fa5ffb700 1 -- 172.21.15.142:0/2733744334 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f5fa80ff350 con 0x7f5f9806c4e0 2024-03-29T04:43:50.887 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:43:50.887 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (5m) 72s ago 7m 25.1M - 0.25.0 c8568f914cd2 d4a3e12f4047 2024-03-29T04:43:50.887 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (7m) 72s ago 7m 7512k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:43:50.887 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (6m) 77s ago 6m 7432k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:43:50.887 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (7m) 72s ago 7m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:43:50.887 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (6m) 77s ago 6m 6660k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:43:50.887 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (5m) 72s ago 6m 85.8M - 9.4.7 954c08fa6188 aba58e552b7c 2024-03-29T04:43:50.887 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (90s) 72s ago 90s 17.7M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (97s) 72s ago 97s 12.8M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (88s) 77s ago 87s 17.3M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (95s) 77s ago 94s 17.4M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:9283,8765,8443 running (8m) 72s ago 8m 489M - 18.2.1 5be31c24972a a5f21a9cfe24 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (6m) 77s ago 6m 441M - 18.2.1 5be31c24972a c6124d88151c 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (8m) 72s ago 8m 46.5M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (6m) 77s ago 6m 35.2M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (7m) 72s ago 7m 16.2M - 1.5.0 0da6a335fe13 91b95c929db2 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (6m) 77s ago 6m 15.0M - 1.5.0 0da6a335fe13 97e90e5b337b 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (5m) 72s ago 5m 48.8M 3946M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (4m) 72s ago 4m 51.1M 3946M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (4m) 72s ago 4m 49.1M 3946M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:43:50.888 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (4m) 77s ago 4m 45.8M 4970M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:43:50.889 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (3m) 77s ago 3m 44.2M 4970M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:43:50.889 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (3m) 77s ago 3m 43.8M 4970M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:43:50.889 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (5m) 72s ago 6m 58.7M - 2.43.0 a07b618ecd1d c6c39d3c30f6 2024-03-29T04:43:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.892+0000 7f5faf0fd700 1 -- 172.21.15.142:0/2733744334 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f5f9806c4e0 msgr2=0x7f5f9806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.892+0000 7f5faf0fd700 1 --2- 172.21.15.142:0/2733744334 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f5f9806c4e0 0x7f5f9806e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009fd0 tx=0x7f5f94005940 comp rx=0 tx=0).stop 2024-03-29T04:43:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.892+0000 7f5faf0fd700 1 -- 172.21.15.142:0/2733744334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fa80ffd50 msgr2=0x7f5fa8196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.892+0000 7f5faf0fd700 1 --2- 172.21.15.142:0/2733744334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fa80ffd50 0x7f5fa8196370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00e8f0 tx=0x7f5f9c00ec00 comp rx=0 tx=0).stop 2024-03-29T04:43:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.892+0000 7f5faf0fd700 1 -- 172.21.15.142:0/2733744334 shutdown_connections 2024-03-29T04:43:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.892+0000 7f5faf0fd700 1 --2- 172.21.15.142:0/2733744334 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f5f9806c4e0 0x7f5f9806e9a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.892+0000 7f5faf0fd700 1 --2- 172.21.15.142:0/2733744334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fa80ffd50 0x7f5fa8196370 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.892+0000 7f5faf0fd700 1 --2- 172.21.15.142:0/2733744334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fa81010a0 0x7f5fa81968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:50.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.892+0000 7f5faf0fd700 1 -- 172.21.15.142:0/2733744334 >> 172.21.15.142:0/2733744334 conn(0x7f5fa80fa7c0 msgr2=0x7f5fa80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:43:50.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.892+0000 7f5faf0fd700 1 -- 172.21.15.142:0/2733744334 shutdown_connections 2024-03-29T04:43:50.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:50.892+0000 7f5faf0fd700 1 -- 172.21.15.142:0/2733744334 wait complete. 2024-03-29T04:43:51.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.008+0000 7fe086599700 1 -- 172.21.15.142:0/3873544694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe080100440 msgr2=0x7fe0801008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:51.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.008+0000 7fe086599700 1 --2- 172.21.15.142:0/3873544694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe080100440 0x7fe0801008a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe070009910 tx=0x7fe070009c20 comp rx=0 tx=0).stop 2024-03-29T04:43:51.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.009+0000 7fe086599700 1 -- 172.21.15.142:0/3873544694 shutdown_connections 2024-03-29T04:43:51.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.009+0000 7fe086599700 1 --2- 172.21.15.142:0/3873544694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe080100440 0x7fe0801008a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.009+0000 7fe086599700 1 --2- 172.21.15.142:0/3873544694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0800ff240 0x7fe0800ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.009+0000 7fe086599700 1 -- 172.21.15.142:0/3873544694 >> 172.21.15.142:0/3873544694 conn(0x7fe0800fa7c0 msgr2=0x7fe0800fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:43:51.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.009+0000 7fe086599700 1 -- 172.21.15.142:0/3873544694 shutdown_connections 2024-03-29T04:43:51.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.009+0000 7fe086599700 1 -- 172.21.15.142:0/3873544694 wait complete. 2024-03-29T04:43:51.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.009+0000 7fe086599700 1 Processor -- start 2024-03-29T04:43:51.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.010+0000 7fe086599700 1 -- start start 2024-03-29T04:43:51.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.010+0000 7fe086599700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0800ff240 0x7fe080196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:51.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.010+0000 7fe086599700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe080100440 0x7fe0801968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.010+0000 7fe086599700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe080196ef0 con 0x7fe0800ff240 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.010+0000 7fe086599700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe080197030 con 0x7fe080100440 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.010+0000 7fe07ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0800ff240 0x7fe080196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.010+0000 7fe07ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0800ff240 0x7fe080196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33440/0 (socket says 172.21.15.142:33440) 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.010+0000 7fe07ffff700 1 -- 172.21.15.142:0/2700448094 learned_addr learned my addr 172.21.15.142:0/2700448094 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.011+0000 7fe07ffff700 1 -- 172.21.15.142:0/2700448094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe080100440 msgr2=0x7fe0801968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.011+0000 7fe07f7fe700 1 --2- 172.21.15.142:0/2700448094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe080100440 0x7fe0801968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.011+0000 7fe07ffff700 1 --2- 172.21.15.142:0/2700448094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe080100440 0x7fe0801968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.011+0000 7fe07ffff700 1 -- 172.21.15.142:0/2700448094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0700095f0 con 0x7fe0800ff240 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.011+0000 7fe07f7fe700 1 --2- 172.21.15.142:0/2700448094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe080100440 0x7fe0801968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.011+0000 7fe07ffff700 1 --2- 172.21.15.142:0/2700448094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0800ff240 0x7fe080196390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe06800e8f0 tx=0x7fe06800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.011+0000 7fe07d7fa700 1 -- 172.21.15.142:0/2700448094 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe068009790 con 0x7fe0800ff240 2024-03-29T04:43:51.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.011+0000 7fe086599700 1 -- 172.21.15.142:0/2700448094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08019bae0 con 0x7fe0800ff240 2024-03-29T04:43:51.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.011+0000 7fe086599700 1 -- 172.21.15.142:0/2700448094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08019c030 con 0x7fe0800ff240 2024-03-29T04:43:51.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.011+0000 7fe07d7fa700 1 -- 172.21.15.142:0/2700448094 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe0680044d0 con 0x7fe0800ff240 2024-03-29T04:43:51.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.011+0000 7fe07d7fa700 1 -- 172.21.15.142:0/2700448094 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe068003da0 con 0x7fe0800ff240 2024-03-29T04:43:51.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.013+0000 7fe086599700 1 -- 172.21.15.142:0/2700448094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe060005320 con 0x7fe0800ff240 2024-03-29T04:43:51.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.014+0000 7fe07d7fa700 1 -- 172.21.15.142:0/2700448094 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe068004020 con 0x7fe0800ff240 2024-03-29T04:43:51.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.014+0000 7fe07d7fa700 1 --2- 172.21.15.142:0/2700448094 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe06c06c4e0 0x7fe06c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:51.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.014+0000 7fe07d7fa700 1 -- 172.21.15.142:0/2700448094 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe068014070 con 0x7fe0800ff240 2024-03-29T04:43:51.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.015+0000 7fe07f7fe700 1 --2- 172.21.15.142:0/2700448094 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe06c06c4e0 0x7fe06c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:51.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.016+0000 7fe07f7fe700 1 --2- 172.21.15.142:0/2700448094 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe06c06c4e0 0x7fe06c06e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe070000c00 tx=0x7fe070009f70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:43:51.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.022+0000 7fe07d7fa700 1 -- 172.21.15.142:0/2700448094 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe068059e70 con 0x7fe0800ff240 2024-03-29T04:43:51.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:50 smithi152 ceph-mon[37177]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:51.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:50 smithi152 ceph-mon[37177]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:43:51.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.385+0000 7fe086599700 1 -- 172.21.15.142:0/2700448094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe060006200 con 0x7fe0800ff240 2024-03-29T04:43:51.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.386+0000 7fe07d7fa700 1 -- 172.21.15.142:0/2700448094 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fe068059a00 con 0x7fe0800ff240 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:43:51.386 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:43:51.387 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:43:51.387 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-03-29T04:43:51.387 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:43:51.387 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:43:51.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.389+0000 7fe086599700 1 -- 172.21.15.142:0/2700448094 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe06c06c4e0 msgr2=0x7fe06c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:51.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.389+0000 7fe086599700 1 --2- 172.21.15.142:0/2700448094 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe06c06c4e0 0x7fe06c06e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe070000c00 tx=0x7fe070009f70 comp rx=0 tx=0).stop 2024-03-29T04:43:51.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.389+0000 7fe086599700 1 -- 172.21.15.142:0/2700448094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0800ff240 msgr2=0x7fe080196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:51.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.389+0000 7fe086599700 1 --2- 172.21.15.142:0/2700448094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0800ff240 0x7fe080196390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe06800e8f0 tx=0x7fe06800ecb0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.389+0000 7fe086599700 1 -- 172.21.15.142:0/2700448094 shutdown_connections 2024-03-29T04:43:51.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.389+0000 7fe086599700 1 --2- 172.21.15.142:0/2700448094 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fe06c06c4e0 0x7fe06c06e9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.389+0000 7fe086599700 1 --2- 172.21.15.142:0/2700448094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0800ff240 0x7fe080196390 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.389+0000 7fe086599700 1 --2- 172.21.15.142:0/2700448094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe080100440 0x7fe0801968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.389+0000 7fe086599700 1 -- 172.21.15.142:0/2700448094 >> 172.21.15.142:0/2700448094 conn(0x7fe0800fa7c0 msgr2=0x7fe080103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:43:51.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.390+0000 7fe086599700 1 -- 172.21.15.142:0/2700448094 shutdown_connections 2024-03-29T04:43:51.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.390+0000 7fe086599700 1 -- 172.21.15.142:0/2700448094 wait complete. 2024-03-29T04:43:51.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.543+0000 7fad71ba4700 1 -- 172.21.15.142:0/3683094818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad6c0ffab0 msgr2=0x7fad6c0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:51.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.543+0000 7fad71ba4700 1 --2- 172.21.15.142:0/3683094818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad6c0ffab0 0x7fad6c0ffed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009910 tx=0x7fad5c009c20 comp rx=0 tx=0).stop 2024-03-29T04:43:51.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.543+0000 7fad71ba4700 1 -- 172.21.15.142:0/3683094818 shutdown_connections 2024-03-29T04:43:51.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.543+0000 7fad71ba4700 1 --2- 172.21.15.142:0/3683094818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad6c100cb0 0x7fad6c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.543+0000 7fad71ba4700 1 --2- 172.21.15.142:0/3683094818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad6c0ffab0 0x7fad6c0ffed0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.543+0000 7fad71ba4700 1 -- 172.21.15.142:0/3683094818 >> 172.21.15.142:0/3683094818 conn(0x7fad6c0fb030 msgr2=0x7fad6c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:43:51.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.543+0000 7fad71ba4700 1 -- 172.21.15.142:0/3683094818 shutdown_connections 2024-03-29T04:43:51.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.543+0000 7fad71ba4700 1 -- 172.21.15.142:0/3683094818 wait complete. 2024-03-29T04:43:51.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.544+0000 7fad71ba4700 1 Processor -- start 2024-03-29T04:43:51.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.544+0000 7fad71ba4700 1 -- start start 2024-03-29T04:43:51.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.544+0000 7fad71ba4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad6c0ffab0 0x7fad6c194120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:51.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.544+0000 7fad71ba4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad6c100cb0 0x7fad6c194660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:51.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.544+0000 7fad71ba4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c194c80 con 0x7fad6c0ffab0 2024-03-29T04:43:51.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.544+0000 7fad71ba4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c194dc0 con 0x7fad6c100cb0 2024-03-29T04:43:51.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.544+0000 7fad6b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad6c0ffab0 0x7fad6c194120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:51.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.544+0000 7fad6b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad6c0ffab0 0x7fad6c194120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33460/0 (socket says 172.21.15.142:33460) 2024-03-29T04:43:51.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.544+0000 7fad6b7fe700 1 -- 172.21.15.142:0/731239682 learned_addr learned my addr 172.21.15.142:0/731239682 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:43:51.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.545+0000 7fad6b7fe700 1 -- 172.21.15.142:0/731239682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad6c100cb0 msgr2=0x7fad6c194660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:51.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.545+0000 7fad6b7fe700 1 --2- 172.21.15.142:0/731239682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad6c100cb0 0x7fad6c194660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.545+0000 7fad6b7fe700 1 -- 172.21.15.142:0/731239682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c0095f0 con 0x7fad6c0ffab0 2024-03-29T04:43:51.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.545+0000 7fad6b7fe700 1 --2- 172.21.15.142:0/731239682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad6c0ffab0 0x7fad6c194120 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00bda0 tx=0x7fad5c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:43:51.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.545+0000 7fad68ff9700 1 -- 172.21.15.142:0/731239682 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad5c01d070 con 0x7fad6c0ffab0 2024-03-29T04:43:51.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.545+0000 7fad68ff9700 1 -- 172.21.15.142:0/731239682 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fad5c00f460 con 0x7fad6c0ffab0 2024-03-29T04:43:51.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.545+0000 7fad71ba4700 1 -- 172.21.15.142:0/731239682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6c19dc20 con 0x7fad6c0ffab0 2024-03-29T04:43:51.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.545+0000 7fad68ff9700 1 -- 172.21.15.142:0/731239682 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad5c021640 con 0x7fad6c0ffab0 2024-03-29T04:43:51.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.545+0000 7fad71ba4700 1 -- 172.21.15.142:0/731239682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6c19e110 con 0x7fad6c0ffab0 2024-03-29T04:43:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.547+0000 7fad68ff9700 1 -- 172.21.15.142:0/731239682 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fad5c00f5d0 con 0x7fad6c0ffab0 2024-03-29T04:43:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.548+0000 7fad71ba4700 1 -- 172.21.15.142:0/731239682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6c107500 con 0x7fad6c0ffab0 2024-03-29T04:43:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.548+0000 7fad68ff9700 1 --2- 172.21.15.142:0/731239682 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fad5406c4e0 0x7fad5406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:43:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.548+0000 7fad6affd700 1 --2- 172.21.15.142:0/731239682 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fad5406c4e0 0x7fad5406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:43:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.548+0000 7fad68ff9700 1 -- 172.21.15.142:0/731239682 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fad5c08cb90 con 0x7fad6c0ffab0 2024-03-29T04:43:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.549+0000 7fad6affd700 1 --2- 172.21.15.142:0/731239682 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fad5406c4e0 0x7fad5406e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fad60005950 tx=0x7fad600058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:43:51.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.558+0000 7fad68ff9700 1 -- 172.21.15.142:0/731239682 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fad5c05aec0 con 0x7fad6c0ffab0 2024-03-29T04:43:51.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:51 smithi142 ceph-mon[29027]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:43:51.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:51 smithi142 ceph-mon[29027]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:43:51.870 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:51 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2700448094' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:43:51.976 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.978+0000 7fad71ba4700 1 -- 172.21.15.142:0/731239682 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad6c105600 con 0x7fad5406c4e0 2024-03-29T04:43:51.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.980+0000 7fad68ff9700 1 -- 172.21.15.142:0/731239682 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fad6c105600 con 0x7fad5406c4e0 2024-03-29T04:43:51.978 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:43:51.979 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:43:51.979 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:43:51.979 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T04:43:51.979 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [], 2024-03-29T04:43:51.979 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "0/2 daemons upgraded", 2024-03-29T04:43:51.979 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 image on host smithi152", 2024-03-29T04:43:51.979 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:43:51.980 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:43:51.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.983+0000 7fad71ba4700 1 -- 172.21.15.142:0/731239682 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fad5406c4e0 msgr2=0x7fad5406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:51.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.983+0000 7fad71ba4700 1 --2- 172.21.15.142:0/731239682 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fad5406c4e0 0x7fad5406e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fad60005950 tx=0x7fad600058e0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.983+0000 7fad71ba4700 1 -- 172.21.15.142:0/731239682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad6c0ffab0 msgr2=0x7fad6c194120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:43:51.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.983+0000 7fad71ba4700 1 --2- 172.21.15.142:0/731239682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad6c0ffab0 0x7fad6c194120 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00bda0 tx=0x7fad5c00be80 comp rx=0 tx=0).stop 2024-03-29T04:43:51.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.983+0000 7fad71ba4700 1 -- 172.21.15.142:0/731239682 shutdown_connections 2024-03-29T04:43:51.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.983+0000 7fad71ba4700 1 --2- 172.21.15.142:0/731239682 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fad5406c4e0 0x7fad5406e9a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.983+0000 7fad71ba4700 1 --2- 172.21.15.142:0/731239682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad6c0ffab0 0x7fad6c194120 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.983+0000 7fad71ba4700 1 --2- 172.21.15.142:0/731239682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad6c100cb0 0x7fad6c194660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:43:51.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.983+0000 7fad71ba4700 1 -- 172.21.15.142:0/731239682 >> 172.21.15.142:0/731239682 conn(0x7fad6c0fb030 msgr2=0x7fad6c103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:43:51.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.985+0000 7fad71ba4700 1 -- 172.21.15.142:0/731239682 shutdown_connections 2024-03-29T04:43:51.983 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:43:51.985+0000 7fad71ba4700 1 -- 172.21.15.142:0/731239682 wait complete. 2024-03-29T04:43:52.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:51 smithi152 ceph-mon[37177]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:43:52.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:51 smithi152 ceph-mon[37177]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:43:52.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:51 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2700448094' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:43:53.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:52 smithi152 ceph-mon[37177]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:43:53.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:52 smithi152 ceph-mon[37177]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:43:53.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:52 smithi142 ceph-mon[29027]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:43:53.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:52 smithi142 ceph-mon[29027]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:43:54.788 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:54 smithi152 ceph-mon[37177]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:54.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:54 smithi142 ceph-mon[29027]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:55.118 INFO:tasks.workunit.client.1.smithi152.stderr:Updating files: 96% (12029/12435) Updating files: 97% (12062/12435) Updating files: 98% (12187/12435) Updating files: 99% (12311/12435) Updating files: 100% (12435/12435) Updating files: 100% (12435/12435), done. 2024-03-29T04:43:55.370 INFO:tasks.workunit.client.1.smithi152.stderr:Note: switching to '8da23624bc17fe0b1f797227e72a0db0ce7821ba'. 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr: 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr:state without impacting any branches by switching back to a branch. 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr: 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr: 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr: git switch -c 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr: 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr:Or undo this operation with: 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr: 2024-03-29T04:43:55.371 INFO:tasks.workunit.client.1.smithi152.stderr: git switch - 2024-03-29T04:43:55.372 INFO:tasks.workunit.client.1.smithi152.stderr: 2024-03-29T04:43:55.372 INFO:tasks.workunit.client.1.smithi152.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-29T04:43:55.372 INFO:tasks.workunit.client.1.smithi152.stderr: 2024-03-29T04:43:55.372 INFO:tasks.workunit.client.1.smithi152.stderr:HEAD is now at 8da23624bc1 Merge pull request #56502 from rhcs-dashboard/wip-65166-squid 2024-03-29T04:43:55.374 DEBUG:teuthology.orchestra.run.smithi152:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-29T04:43:55.442 INFO:tasks.workunit.client.1.smithi152.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-29T04:43:55.443 INFO:tasks.workunit.client.1.smithi152.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-29T04:43:55.444 INFO:tasks.workunit.client.1.smithi152.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-29T04:43:56.267 INFO:tasks.workunit.client.1.smithi152.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-29T04:43:56.336 INFO:tasks.workunit.client.1.smithi152.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-29T04:43:56.376 INFO:tasks.workunit.client.1.smithi152.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-29T04:43:56.377 INFO:tasks.workunit.client.1.smithi152.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-29T04:43:56.377 INFO:tasks.workunit.client.1.smithi152.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-29T04:43:56.419 INFO:tasks.workunit.client.1.smithi152.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-29T04:43:56.422 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-29T04:43:56.422 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-29T04:43:56.478 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-29T04:43:56.480 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-29T04:43:56.480 DEBUG:teuthology.orchestra.run.smithi152:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8da23624bc17fe0b1f797227e72a0db0ce7821ba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-29T04:43:56.543 INFO:tasks.workunit.client.1.smithi152.stderr:+ mkdir -p fsstress 2024-03-29T04:43:56.545 INFO:tasks.workunit.client.1.smithi152.stderr:+ pushd fsstress 2024-03-29T04:43:56.546 INFO:tasks.workunit.client.1.smithi152.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-29T04:43:56.547 INFO:tasks.workunit.client.1.smithi152.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-29T04:43:57.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:56 smithi152 ceph-mon[37177]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:57.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:56 smithi142 ceph-mon[29027]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:43:57.214 INFO:tasks.workunit.client.1.smithi152.stderr:+ tar xzf ltp-full.tgz 2024-03-29T04:43:58.162 INFO:tasks.workunit.client.0.smithi142.stderr:Updating files: 91% (11368/12435) Updating files: 92% (11441/12435) Updating files: 93% (11565/12435) Updating files: 94% (11689/12435) Updating files: 95% (11814/12435) Updating files: 96% (11938/12435) Updating files: 97% (12062/12435) Updating files: 98% (12187/12435) Updating files: 99% (12311/12435) Updating files: 100% (12435/12435) Updating files: 100% (12435/12435), done. 2024-03-29T04:43:58.491 INFO:tasks.workunit.client.0.smithi142.stderr:Note: switching to '8da23624bc17fe0b1f797227e72a0db0ce7821ba'. 2024-03-29T04:43:58.491 INFO:tasks.workunit.client.0.smithi142.stderr: 2024-03-29T04:43:58.491 INFO:tasks.workunit.client.0.smithi142.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-29T04:43:58.491 INFO:tasks.workunit.client.0.smithi142.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-29T04:43:58.491 INFO:tasks.workunit.client.0.smithi142.stderr:state without impacting any branches by switching back to a branch. 2024-03-29T04:43:58.491 INFO:tasks.workunit.client.0.smithi142.stderr: 2024-03-29T04:43:58.491 INFO:tasks.workunit.client.0.smithi142.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-29T04:43:58.492 INFO:tasks.workunit.client.0.smithi142.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-29T04:43:58.492 INFO:tasks.workunit.client.0.smithi142.stderr: 2024-03-29T04:43:58.492 INFO:tasks.workunit.client.0.smithi142.stderr: git switch -c 2024-03-29T04:43:58.492 INFO:tasks.workunit.client.0.smithi142.stderr: 2024-03-29T04:43:58.492 INFO:tasks.workunit.client.0.smithi142.stderr:Or undo this operation with: 2024-03-29T04:43:58.492 INFO:tasks.workunit.client.0.smithi142.stderr: 2024-03-29T04:43:58.492 INFO:tasks.workunit.client.0.smithi142.stderr: git switch - 2024-03-29T04:43:58.492 INFO:tasks.workunit.client.0.smithi142.stderr: 2024-03-29T04:43:58.492 INFO:tasks.workunit.client.0.smithi142.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-29T04:43:58.492 INFO:tasks.workunit.client.0.smithi142.stderr: 2024-03-29T04:43:58.492 INFO:tasks.workunit.client.0.smithi142.stderr:HEAD is now at 8da23624bc1 Merge pull request #56502 from rhcs-dashboard/wip-65166-squid 2024-03-29T04:43:58.495 DEBUG:teuthology.orchestra.run.smithi142:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-29T04:43:58.582 INFO:tasks.workunit.client.0.smithi142.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-29T04:43:58.584 INFO:tasks.workunit.client.0.smithi142.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-29T04:43:58.584 INFO:tasks.workunit.client.0.smithi142.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-29T04:43:59.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:43:58 smithi152 ceph-mon[37177]: pgmap v235: 65 pgs: 65 active+clean; 4.7 MiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 356 KiB/s wr, 7 op/s 2024-03-29T04:43:59.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:43:58 smithi142 ceph-mon[29027]: pgmap v235: 65 pgs: 65 active+clean; 4.7 MiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 356 KiB/s wr, 7 op/s 2024-03-29T04:44:01.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:00 smithi152 ceph-mon[37177]: pgmap v236: 65 pgs: 65 active+clean; 12 MiB data, 168 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s rd, 1012 KiB/s wr, 20 op/s 2024-03-29T04:44:01.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:00 smithi142 ceph-mon[29027]: pgmap v236: 65 pgs: 65 active+clean; 12 MiB data, 168 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s rd, 1012 KiB/s wr, 20 op/s 2024-03-29T04:44:03.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:02 smithi152 ceph-mon[37177]: pgmap v237: 65 pgs: 65 active+clean; 14 MiB data, 194 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s rd, 1.2 MiB/s wr, 63 op/s 2024-03-29T04:44:03.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:02 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:44:03.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:02 smithi142 ceph-mon[29027]: pgmap v237: 65 pgs: 65 active+clean; 14 MiB data, 194 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s rd, 1.2 MiB/s wr, 63 op/s 2024-03-29T04:44:03.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:02 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:44:04.639 INFO:tasks.workunit.client.0.smithi142.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-29T04:44:04.788 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:04 smithi152 ceph-mon[37177]: pgmap v238: 65 pgs: 65 active+clean; 17 MiB data, 231 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-03-29T04:44:04.826 INFO:tasks.workunit.client.0.smithi142.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-29T04:44:04.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:04 smithi142 ceph-mon[29027]: pgmap v238: 65 pgs: 65 active+clean; 17 MiB data, 231 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-03-29T04:44:04.929 INFO:tasks.workunit.client.0.smithi142.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-29T04:44:04.931 INFO:tasks.workunit.client.0.smithi142.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-29T04:44:04.931 INFO:tasks.workunit.client.0.smithi142.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-29T04:44:04.998 INFO:tasks.workunit.client.0.smithi142.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-29T04:44:05.001 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T04:44:05.001 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-29T04:44:05.017 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-29T04:44:05.018 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-29T04:44:05.019 DEBUG:teuthology.orchestra.run.smithi142:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8da23624bc17fe0b1f797227e72a0db0ce7821ba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-29T04:44:05.082 INFO:tasks.workunit.client.0.smithi142.stderr:+ mkdir -p fsstress 2024-03-29T04:44:05.087 INFO:tasks.workunit.client.0.smithi142.stderr:+ pushd fsstress 2024-03-29T04:44:05.087 INFO:tasks.workunit.client.0.smithi142.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-29T04:44:05.087 INFO:tasks.workunit.client.0.smithi142.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-29T04:44:05.863 INFO:tasks.workunit.client.0.smithi142.stderr:+ tar xzf ltp-full.tgz 2024-03-29T04:44:07.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:06 smithi152 ceph-mon[37177]: pgmap v239: 65 pgs: 65 active+clean; 21 MiB data, 243 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 1.8 MiB/s wr, 150 op/s 2024-03-29T04:44:07.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:06 smithi142 ceph-mon[29027]: pgmap v239: 65 pgs: 65 active+clean; 21 MiB data, 243 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 1.8 MiB/s wr, 150 op/s 2024-03-29T04:44:09.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:08 smithi152 ceph-mon[37177]: pgmap v240: 65 pgs: 65 active+clean; 34 MiB data, 291 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-03-29T04:44:09.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:08 smithi142 ceph-mon[29027]: pgmap v240: 65 pgs: 65 active+clean; 34 MiB data, 291 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-03-29T04:44:11.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:10 smithi152 ceph-mon[37177]: pgmap v241: 65 pgs: 65 active+clean; 47 MiB data, 335 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 301 op/s 2024-03-29T04:44:11.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:10 smithi142 ceph-mon[29027]: pgmap v241: 65 pgs: 65 active+clean; 47 MiB data, 335 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 3.7 MiB/s wr, 301 op/s 2024-03-29T04:44:11.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:11 smithi142 ceph-mon[29027]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:44:12.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:11 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:44:13.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:12 smithi152 ceph-mon[37177]: pgmap v242: 65 pgs: 65 active+clean; 57 MiB data, 398 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 363 op/s 2024-03-29T04:44:13.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:12 smithi142 ceph-mon[29027]: pgmap v242: 65 pgs: 65 active+clean; 57 MiB data, 398 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 363 op/s 2024-03-29T04:44:14.788 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:14 smithi152 ceph-mon[37177]: pgmap v243: 65 pgs: 65 active+clean; 64 MiB data, 431 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 396 op/s 2024-03-29T04:44:14.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:14 smithi142 ceph-mon[29027]: pgmap v243: 65 pgs: 65 active+clean; 64 MiB data, 431 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 396 op/s 2024-03-29T04:44:17.038 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:16 smithi152 ceph-mon[37177]: pgmap v244: 65 pgs: 65 active+clean; 69 MiB data, 453 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 379 op/s 2024-03-29T04:44:17.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:16 smithi142 ceph-mon[29027]: pgmap v244: 65 pgs: 65 active+clean; 69 MiB data, 453 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 379 op/s 2024-03-29T04:44:17.869 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:44:18.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:17 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:44:19.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:18 smithi152 ceph-mon[37177]: pgmap v245: 65 pgs: 65 active+clean; 77 MiB data, 500 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 470 op/s 2024-03-29T04:44:19.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:18 smithi142 ceph-mon[29027]: pgmap v245: 65 pgs: 65 active+clean; 77 MiB data, 500 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 470 op/s 2024-03-29T04:44:21.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:20 smithi152 ceph-mon[37177]: pgmap v246: 65 pgs: 65 active+clean; 85 MiB data, 492 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 441 op/s 2024-03-29T04:44:21.119 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:20 smithi142 ceph-mon[29027]: pgmap v246: 65 pgs: 65 active+clean; 85 MiB data, 492 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 441 op/s 2024-03-29T04:44:22.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.149+0000 7f783f320700 1 -- 172.21.15.142:0/603291472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7838100cb0 msgr2=0x7f7838101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:22.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.149+0000 7f783f320700 1 --2- 172.21.15.142:0/603291472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7838100cb0 0x7f7838101110 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7828009910 tx=0x7f7828009c20 comp rx=0 tx=0).stop 2024-03-29T04:44:22.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.149+0000 7f783f320700 1 -- 172.21.15.142:0/603291472 shutdown_connections 2024-03-29T04:44:22.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.149+0000 7f783f320700 1 --2- 172.21.15.142:0/603291472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7838100cb0 0x7f7838101110 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:22.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.149+0000 7f783f320700 1 --2- 172.21.15.142:0/603291472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78380ffab0 0x7f78380ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:22.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.149+0000 7f783f320700 1 -- 172.21.15.142:0/603291472 >> 172.21.15.142:0/603291472 conn(0x7f78380fb030 msgr2=0x7f78380fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:22.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.149+0000 7f783f320700 1 -- 172.21.15.142:0/603291472 shutdown_connections 2024-03-29T04:44:22.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.149+0000 7f783f320700 1 -- 172.21.15.142:0/603291472 wait complete. 2024-03-29T04:44:22.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.150+0000 7f783f320700 1 Processor -- start 2024-03-29T04:44:22.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.150+0000 7f783f320700 1 -- start start 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.150+0000 7f783f320700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78380ffab0 0x7f7838198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.150+0000 7f783f320700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7838100cb0 0x7f7838198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.150+0000 7f783f320700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78381990b0 con 0x7f78380ffab0 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.150+0000 7f783f320700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78381991f0 con 0x7f7838100cb0 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.150+0000 7f783d0bc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78380ffab0 0x7f7838198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.150+0000 7f783d0bc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78380ffab0 0x7f7838198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39400/0 (socket says 172.21.15.142:39400) 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.150+0000 7f783d0bc700 1 -- 172.21.15.142:0/3220369391 learned_addr learned my addr 172.21.15.142:0/3220369391 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.150+0000 7f783c8bb700 1 --2- 172.21.15.142:0/3220369391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7838100cb0 0x7f7838198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.151+0000 7f783d0bc700 1 -- 172.21.15.142:0/3220369391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7838100cb0 msgr2=0x7f7838198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.151+0000 7f783d0bc700 1 --2- 172.21.15.142:0/3220369391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7838100cb0 0x7f7838198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.151+0000 7f783d0bc700 1 -- 172.21.15.142:0/3220369391 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78280095f0 con 0x7f78380ffab0 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.151+0000 7f783d0bc700 1 --2- 172.21.15.142:0/3220369391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78380ffab0 0x7f7838198550 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f783400e8f0 tx=0x7f783400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.151+0000 7f782e7fc700 1 -- 172.21.15.142:0/3220369391 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7834009790 con 0x7f78380ffab0 2024-03-29T04:44:22.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.151+0000 7f783f320700 1 -- 172.21.15.142:0/3220369391 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783819dca0 con 0x7f78380ffab0 2024-03-29T04:44:22.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.151+0000 7f782e7fc700 1 -- 172.21.15.142:0/3220369391 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f783400c810 con 0x7f78380ffab0 2024-03-29T04:44:22.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.151+0000 7f783f320700 1 -- 172.21.15.142:0/3220369391 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783819e1a0 con 0x7f78380ffab0 2024-03-29T04:44:22.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.151+0000 7f782e7fc700 1 -- 172.21.15.142:0/3220369391 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7834003d40 con 0x7f78380ffab0 2024-03-29T04:44:22.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.153+0000 7f783f320700 1 -- 172.21.15.142:0/3220369391 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7838108610 con 0x7f78380ffab0 2024-03-29T04:44:22.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.153+0000 7f782e7fc700 1 -- 172.21.15.142:0/3220369391 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f78340041f0 con 0x7f78380ffab0 2024-03-29T04:44:22.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.154+0000 7f782e7fc700 1 --2- 172.21.15.142:0/3220369391 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f782406c4e0 0x7f782406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:22.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.154+0000 7f782e7fc700 1 -- 172.21.15.142:0/3220369391 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7834014070 con 0x7f78380ffab0 2024-03-29T04:44:22.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.154+0000 7f783c8bb700 1 --2- 172.21.15.142:0/3220369391 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f782406c4e0 0x7f782406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:22.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.155+0000 7f783c8bb700 1 --2- 172.21.15.142:0/3220369391 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f782406c4e0 0x7f782406e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f782800b6f0 tx=0x7f7828005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:22.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.161+0000 7f782e7fc700 1 -- 172.21.15.142:0/3220369391 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7834059d30 con 0x7f78380ffab0 2024-03-29T04:44:22.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.473+0000 7f783f320700 1 -- 172.21.15.142:0/3220369391 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7838068b20 con 0x7f782406c4e0 2024-03-29T04:44:22.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.474+0000 7f782e7fc700 1 -- 172.21.15.142:0/3220369391 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f7838068b20 con 0x7f782406c4e0 2024-03-29T04:44:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.477+0000 7f783f320700 1 -- 172.21.15.142:0/3220369391 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f782406c4e0 msgr2=0x7f782406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.477+0000 7f783f320700 1 --2- 172.21.15.142:0/3220369391 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f782406c4e0 0x7f782406e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f782800b6f0 tx=0x7f7828005a90 comp rx=0 tx=0).stop 2024-03-29T04:44:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.477+0000 7f783f320700 1 -- 172.21.15.142:0/3220369391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78380ffab0 msgr2=0x7f7838198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.477+0000 7f783f320700 1 --2- 172.21.15.142:0/3220369391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78380ffab0 0x7f7838198550 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f783400e8f0 tx=0x7f783400ec00 comp rx=0 tx=0).stop 2024-03-29T04:44:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.477+0000 7f783f320700 1 -- 172.21.15.142:0/3220369391 shutdown_connections 2024-03-29T04:44:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.477+0000 7f783f320700 1 --2- 172.21.15.142:0/3220369391 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f782406c4e0 0x7f782406e9a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.477+0000 7f783f320700 1 --2- 172.21.15.142:0/3220369391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78380ffab0 0x7f7838198550 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.477+0000 7f783f320700 1 --2- 172.21.15.142:0/3220369391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7838100cb0 0x7f7838198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:22.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.477+0000 7f783f320700 1 -- 172.21.15.142:0/3220369391 >> 172.21.15.142:0/3220369391 conn(0x7f78380fb030 msgr2=0x7f7838068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:22.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.478+0000 7f783f320700 1 -- 172.21.15.142:0/3220369391 shutdown_connections 2024-03-29T04:44:22.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.478+0000 7f783f320700 1 -- 172.21.15.142:0/3220369391 wait complete. 2024-03-29T04:44:22.491 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:44:22.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.624+0000 7f9011d4b700 1 -- 172.21.15.142:0/911045118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c100cb0 msgr2=0x7f900c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:22.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.624+0000 7f9011d4b700 1 --2- 172.21.15.142:0/911045118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c100cb0 0x7f900c101110 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009910 tx=0x7f8ff4009c20 comp rx=0 tx=0).stop 2024-03-29T04:44:22.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.624+0000 7f9011d4b700 1 -- 172.21.15.142:0/911045118 shutdown_connections 2024-03-29T04:44:22.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.624+0000 7f9011d4b700 1 --2- 172.21.15.142:0/911045118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c100cb0 0x7f900c101110 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:22.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.624+0000 7f9011d4b700 1 --2- 172.21.15.142:0/911045118 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c0ffab0 0x7f900c0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:22.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.624+0000 7f9011d4b700 1 -- 172.21.15.142:0/911045118 >> 172.21.15.142:0/911045118 conn(0x7f900c0fb030 msgr2=0x7f900c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:22.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.624+0000 7f9011d4b700 1 -- 172.21.15.142:0/911045118 shutdown_connections 2024-03-29T04:44:22.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.624+0000 7f9011d4b700 1 -- 172.21.15.142:0/911045118 wait complete. 2024-03-29T04:44:22.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.625+0000 7f9011d4b700 1 Processor -- start 2024-03-29T04:44:22.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.625+0000 7f9011d4b700 1 -- start start 2024-03-29T04:44:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.625+0000 7f9011d4b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c0ffab0 0x7f900c071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.625+0000 7f9011d4b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c100cb0 0x7f900c072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.625+0000 7f9011d4b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c072650 con 0x7f900c100cb0 2024-03-29T04:44:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.625+0000 7f9011d4b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1a62e0 con 0x7f900c0ffab0 2024-03-29T04:44:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.625+0000 7f9003fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c100cb0 0x7f900c072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.625+0000 7f9003fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c100cb0 0x7f900c072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39416/0 (socket says 172.21.15.142:39416) 2024-03-29T04:44:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.625+0000 7f9003fff700 1 -- 172.21.15.142:0/1607884655 learned_addr learned my addr 172.21.15.142:0/1607884655 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:44:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.625+0000 7f900b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c0ffab0 0x7f900c071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.626+0000 7f900b7fe700 1 -- 172.21.15.142:0/1607884655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c100cb0 msgr2=0x7f900c072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.626+0000 7f900b7fe700 1 --2- 172.21.15.142:0/1607884655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c100cb0 0x7f900c072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.626+0000 7f900b7fe700 1 -- 172.21.15.142:0/1607884655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff40095f0 con 0x7f900c0ffab0 2024-03-29T04:44:22.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.626+0000 7f900b7fe700 1 --2- 172.21.15.142:0/1607884655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c0ffab0 0x7f900c071af0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00e940 tx=0x7f8ffc00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:22.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.626+0000 7f90097fa700 1 -- 172.21.15.142:0/1607884655 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ffc0041f0 con 0x7f900c0ffab0 2024-03-29T04:44:22.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.626+0000 7f9011d4b700 1 -- 172.21.15.142:0/1607884655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c1a64e0 con 0x7f900c0ffab0 2024-03-29T04:44:22.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.626+0000 7f9011d4b700 1 -- 172.21.15.142:0/1607884655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c1a69e0 con 0x7f900c0ffab0 2024-03-29T04:44:22.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.626+0000 7f90097fa700 1 -- 172.21.15.142:0/1607884655 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8ffc004350 con 0x7f900c0ffab0 2024-03-29T04:44:22.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.626+0000 7f90097fa700 1 -- 172.21.15.142:0/1607884655 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ffc003c80 con 0x7f900c0ffab0 2024-03-29T04:44:22.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.628+0000 7f9011d4b700 1 -- 172.21.15.142:0/1607884655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec005320 con 0x7f900c0ffab0 2024-03-29T04:44:22.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.628+0000 7f90097fa700 1 -- 172.21.15.142:0/1607884655 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f8ffc018430 con 0x7f900c0ffab0 2024-03-29T04:44:22.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.629+0000 7f90097fa700 1 --2- 172.21.15.142:0/1607884655 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8ff806c530 0x7f8ff806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:22.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.629+0000 7f90097fa700 1 -- 172.21.15.142:0/1607884655 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8ffc014070 con 0x7f900c0ffab0 2024-03-29T04:44:22.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.629+0000 7f9003fff700 1 --2- 172.21.15.142:0/1607884655 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8ff806c530 0x7f8ff806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:22.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.630+0000 7f9003fff700 1 --2- 172.21.15.142:0/1607884655 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8ff806c530 0x7f8ff806e9f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009fd0 tx=0x7f8ff4005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:22.633 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.634+0000 7f90097fa700 1 -- 172.21.15.142:0/1607884655 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8ffc0563b0 con 0x7f900c0ffab0 2024-03-29T04:44:22.870 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:22 smithi142 ceph-mon[29027]: pgmap v247: 65 pgs: 65 active+clean; 89 MiB data, 501 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 458 op/s 2024-03-29T04:44:22.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.936+0000 7f9011d4b700 1 -- 172.21.15.142:0/1607884655 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fec000bf0 con 0x7f8ff806c530 2024-03-29T04:44:22.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.938+0000 7f90097fa700 1 -- 172.21.15.142:0/1607884655 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f8fec000bf0 con 0x7f8ff806c530 2024-03-29T04:44:22.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.940+0000 7f9011d4b700 1 -- 172.21.15.142:0/1607884655 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8ff806c530 msgr2=0x7f8ff806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:22.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.940+0000 7f9011d4b700 1 --2- 172.21.15.142:0/1607884655 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8ff806c530 0x7f8ff806e9f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009fd0 tx=0x7f8ff4005940 comp rx=0 tx=0).stop 2024-03-29T04:44:22.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.940+0000 7f9011d4b700 1 -- 172.21.15.142:0/1607884655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c0ffab0 msgr2=0x7f900c071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:22.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.940+0000 7f9011d4b700 1 --2- 172.21.15.142:0/1607884655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c0ffab0 0x7f900c071af0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00e940 tx=0x7f8ffc00ec50 comp rx=0 tx=0).stop 2024-03-29T04:44:22.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.940+0000 7f9011d4b700 1 -- 172.21.15.142:0/1607884655 shutdown_connections 2024-03-29T04:44:22.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.940+0000 7f9011d4b700 1 --2- 172.21.15.142:0/1607884655 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8ff806c530 0x7f8ff806e9f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:22.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.940+0000 7f9011d4b700 1 --2- 172.21.15.142:0/1607884655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c0ffab0 0x7f900c071af0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:22.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.940+0000 7f9011d4b700 1 --2- 172.21.15.142:0/1607884655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f900c100cb0 0x7f900c072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:22.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.940+0000 7f9011d4b700 1 -- 172.21.15.142:0/1607884655 >> 172.21.15.142:0/1607884655 conn(0x7f900c0fb030 msgr2=0x7f900c105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:22.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.940+0000 7f9011d4b700 1 -- 172.21.15.142:0/1607884655 shutdown_connections 2024-03-29T04:44:22.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:22.940+0000 7f9011d4b700 1 -- 172.21.15.142:0/1607884655 wait complete. 2024-03-29T04:44:23.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:22 smithi152 ceph-mon[37177]: pgmap v247: 65 pgs: 65 active+clean; 89 MiB data, 501 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 458 op/s 2024-03-29T04:44:23.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.092+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2426575025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb40ff240 msgr2=0x7f3cb40ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:23.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.092+0000 7f3cbb5b8700 1 --2- 172.21.15.142:0/2426575025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb40ff240 0x7f3cb40ff660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009870 tx=0x7f3cb0009b80 comp rx=0 tx=0).stop 2024-03-29T04:44:23.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.092+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2426575025 shutdown_connections 2024-03-29T04:44:23.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.092+0000 7f3cbb5b8700 1 --2- 172.21.15.142:0/2426575025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb4100440 0x7f3cb41008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.092+0000 7f3cbb5b8700 1 --2- 172.21.15.142:0/2426575025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb40ff240 0x7f3cb40ff660 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.092+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2426575025 >> 172.21.15.142:0/2426575025 conn(0x7f3cb40fa7c0 msgr2=0x7f3cb40fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:23.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.092+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2426575025 shutdown_connections 2024-03-29T04:44:23.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.093+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2426575025 wait complete. 2024-03-29T04:44:23.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.093+0000 7f3cbb5b8700 1 Processor -- start 2024-03-29T04:44:23.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.093+0000 7f3cbb5b8700 1 -- start start 2024-03-29T04:44:23.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.093+0000 7f3cbb5b8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb40ff240 0x7f3cb4194210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:23.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.093+0000 7f3cbb5b8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb4100440 0x7f3cb4194750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:23.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.093+0000 7f3cbb5b8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4194d70 con 0x7f3cb4100440 2024-03-29T04:44:23.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.093+0000 7f3cbb5b8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4194eb0 con 0x7f3cb40ff240 2024-03-29T04:44:23.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.094+0000 7f3cb8b53700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb4100440 0x7f3cb4194750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:23.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.094+0000 7f3cb9354700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb40ff240 0x7f3cb4194210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:23.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.094+0000 7f3cb8b53700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb4100440 0x7f3cb4194750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39426/0 (socket says 172.21.15.142:39426) 2024-03-29T04:44:23.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.094+0000 7f3cb8b53700 1 -- 172.21.15.142:0/2328592047 learned_addr learned my addr 172.21.15.142:0/2328592047 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:44:23.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.094+0000 7f3cb9354700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb40ff240 0x7f3cb4194210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.142:59400/0 (socket says 172.21.15.142:59400) 2024-03-29T04:44:23.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.094+0000 7f3cb9354700 1 -- 172.21.15.142:0/2328592047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb4100440 msgr2=0x7f3cb4194750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:23.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.094+0000 7f3cb9354700 1 --2- 172.21.15.142:0/2328592047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb4100440 0x7f3cb4194750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.094+0000 7f3cb9354700 1 -- 172.21.15.142:0/2328592047 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb0009520 con 0x7f3cb40ff240 2024-03-29T04:44:23.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.094+0000 7f3cb9354700 1 --2- 172.21.15.142:0/2328592047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb40ff240 0x7f3cb4194210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009870 tx=0x7f3cb000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:23.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.094+0000 7f3caa7fc700 1 -- 172.21.15.142:0/2328592047 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cb001c070 con 0x7f3cb40ff240 2024-03-29T04:44:23.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.094+0000 7f3caa7fc700 1 -- 172.21.15.142:0/2328592047 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3cb000fd60 con 0x7f3cb40ff240 2024-03-29T04:44:23.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.094+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2328592047 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb4199900 con 0x7f3cb40ff240 2024-03-29T04:44:23.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.095+0000 7f3caa7fc700 1 -- 172.21.15.142:0/2328592047 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cb0022ea0 con 0x7f3cb40ff240 2024-03-29T04:44:23.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.095+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2328592047 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb4199e70 con 0x7f3cb40ff240 2024-03-29T04:44:23.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.096+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2328592047 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb4107530 con 0x7f3cb40ff240 2024-03-29T04:44:23.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.096+0000 7f3caa7fc700 1 -- 172.21.15.142:0/2328592047 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f3cb0020070 con 0x7f3cb40ff240 2024-03-29T04:44:23.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.097+0000 7f3caa7fc700 1 --2- 172.21.15.142:0/2328592047 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f3ca00708f0 0x7f3ca0072db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:23.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.097+0000 7f3cb8b53700 1 --2- 172.21.15.142:0/2328592047 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f3ca00708f0 0x7f3ca0072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:23.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.097+0000 7f3caa7fc700 1 -- 172.21.15.142:0/2328592047 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3cb008cbf0 con 0x7f3cb40ff240 2024-03-29T04:44:23.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.098+0000 7f3cb8b53700 1 --2- 172.21.15.142:0/2328592047 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f3ca00708f0 0x7f3ca0072db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400bd90 tx=0x7f3ca400b4c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:23.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.102+0000 7f3caa7fc700 1 -- 172.21.15.142:0/2328592047 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3cb005ae80 con 0x7f3cb40ff240 2024-03-29T04:44:23.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.384+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2328592047 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cb419a540 con 0x7f3ca00708f0 2024-03-29T04:44:23.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.396+0000 7f3caa7fc700 1 -- 172.21.15.142:0/2328592047 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3456 (secure 0 0 0) 0x7f3cb419a540 con 0x7f3ca00708f0 2024-03-29T04:44:23.395 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:44:23.395 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (6m) 105s ago 7m 25.1M - 0.25.0 c8568f914cd2 d4a3e12f4047 2024-03-29T04:44:23.395 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (8m) 105s ago 7m 7512k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:44:23.395 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (6m) 109s ago 6m 7432k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (7m) 105s ago 7m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (6m) 109s ago 6m 6660k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (6m) 105s ago 7m 85.8M - 9.4.7 954c08fa6188 aba58e552b7c 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (2m) 105s ago 2m 17.7M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (2m) 105s ago 2m 12.8M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (2m) 109s ago 2m 17.3M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (2m) 109s ago 2m 17.4M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:9283,8765,8443 running (9m) 105s ago 9m 489M - 18.2.1 5be31c24972a a5f21a9cfe24 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (6m) 109s ago 6m 441M - 18.2.1 5be31c24972a c6124d88151c 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (9m) 105s ago 9m 46.5M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (6m) 109s ago 6m 35.2M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (7m) 105s ago 7m 16.2M - 1.5.0 0da6a335fe13 91b95c929db2 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (6m) 109s ago 6m 15.0M - 1.5.0 0da6a335fe13 97e90e5b337b 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (5m) 105s ago 5m 48.8M 3946M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (5m) 105s ago 5m 51.1M 3946M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:44:23.396 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (5m) 105s ago 5m 49.1M 3946M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:44:23.397 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (4m) 109s ago 4m 45.8M 4970M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:44:23.397 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (4m) 109s ago 4m 44.2M 4970M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:44:23.397 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (3m) 109s ago 3m 43.8M 4970M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:44:23.397 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (6m) 105s ago 7m 58.7M - 2.43.0 a07b618ecd1d c6c39d3c30f6 2024-03-29T04:44:23.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.398+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2328592047 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f3ca00708f0 msgr2=0x7f3ca0072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:23.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.398+0000 7f3cbb5b8700 1 --2- 172.21.15.142:0/2328592047 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f3ca00708f0 0x7f3ca0072db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400bd90 tx=0x7f3ca400b4c0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.398+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2328592047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb40ff240 msgr2=0x7f3cb4194210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:23.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.398+0000 7f3cbb5b8700 1 --2- 172.21.15.142:0/2328592047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb40ff240 0x7f3cb4194210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009870 tx=0x7f3cb000f690 comp rx=0 tx=0).stop 2024-03-29T04:44:23.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.398+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2328592047 shutdown_connections 2024-03-29T04:44:23.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.398+0000 7f3cbb5b8700 1 --2- 172.21.15.142:0/2328592047 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f3ca00708f0 0x7f3ca0072db0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.398+0000 7f3cbb5b8700 1 --2- 172.21.15.142:0/2328592047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb40ff240 0x7f3cb4194210 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.399+0000 7f3cbb5b8700 1 --2- 172.21.15.142:0/2328592047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb4100440 0x7f3cb4194750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.399+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2328592047 >> 172.21.15.142:0/2328592047 conn(0x7f3cb40fa7c0 msgr2=0x7f3cb4103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:23.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.399+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2328592047 shutdown_connections 2024-03-29T04:44:23.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.399+0000 7f3cbb5b8700 1 -- 172.21.15.142:0/2328592047 wait complete. 2024-03-29T04:44:23.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.538+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/4200531914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b38100ce0 msgr2=0x7f8b38101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:23.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.538+0000 7f8b3f9d8700 1 --2- 172.21.15.142:0/4200531914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b38100ce0 0x7f8b38101140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009910 tx=0x7f8b28009c20 comp rx=0 tx=0).stop 2024-03-29T04:44:23.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.538+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/4200531914 shutdown_connections 2024-03-29T04:44:23.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.538+0000 7f8b3f9d8700 1 --2- 172.21.15.142:0/4200531914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b38100ce0 0x7f8b38101140 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.538+0000 7f8b3f9d8700 1 --2- 172.21.15.142:0/4200531914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b380ffae0 0x7f8b380fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.538+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/4200531914 >> 172.21.15.142:0/4200531914 conn(0x7f8b380fb080 msgr2=0x7f8b380fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:23.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.539+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/4200531914 shutdown_connections 2024-03-29T04:44:23.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.539+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/4200531914 wait complete. 2024-03-29T04:44:23.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.539+0000 7f8b3f9d8700 1 Processor -- start 2024-03-29T04:44:23.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.539+0000 7f8b3f9d8700 1 -- start start 2024-03-29T04:44:23.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.540+0000 7f8b3f9d8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b380ffae0 0x7f8b38069890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:23.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.540+0000 7f8b3f9d8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b38100ce0 0x7f8b3819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:23.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.540+0000 7f8b3f9d8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3819b6c0 con 0x7f8b38100ce0 2024-03-29T04:44:23.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.540+0000 7f8b3f9d8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3819b800 con 0x7f8b380ffae0 2024-03-29T04:44:23.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.540+0000 7f8b3cf73700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b38100ce0 0x7f8b3819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:23.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.540+0000 7f8b3cf73700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b38100ce0 0x7f8b3819b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39446/0 (socket says 172.21.15.142:39446) 2024-03-29T04:44:23.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.540+0000 7f8b3cf73700 1 -- 172.21.15.142:0/2054779063 learned_addr learned my addr 172.21.15.142:0/2054779063 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:44:23.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.540+0000 7f8b3d774700 1 --2- 172.21.15.142:0/2054779063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b380ffae0 0x7f8b38069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:23.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.540+0000 7f8b3cf73700 1 -- 172.21.15.142:0/2054779063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b380ffae0 msgr2=0x7f8b38069890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:23.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.540+0000 7f8b3cf73700 1 --2- 172.21.15.142:0/2054779063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b380ffae0 0x7f8b38069890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.540+0000 7f8b3cf73700 1 -- 172.21.15.142:0/2054779063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280095f0 con 0x7f8b38100ce0 2024-03-29T04:44:23.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.540+0000 7f8b3cf73700 1 --2- 172.21.15.142:0/2054779063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b38100ce0 0x7f8b3819b0f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b28000c00 tx=0x7f8b2800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:23.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.541+0000 7f8b2e7fc700 1 -- 172.21.15.142:0/2054779063 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b2801d070 con 0x7f8b38100ce0 2024-03-29T04:44:23.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.541+0000 7f8b2e7fc700 1 -- 172.21.15.142:0/2054779063 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8b28005dc0 con 0x7f8b38100ce0 2024-03-29T04:44:23.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.541+0000 7f8b2e7fc700 1 -- 172.21.15.142:0/2054779063 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b280226a0 con 0x7f8b38100ce0 2024-03-29T04:44:23.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.541+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/2054779063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3819ba80 con 0x7f8b38100ce0 2024-03-29T04:44:23.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.541+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/2054779063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b38195180 con 0x7f8b38100ce0 2024-03-29T04:44:23.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.542+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/2054779063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b38108660 con 0x7f8b38100ce0 2024-03-29T04:44:23.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.543+0000 7f8b2e7fc700 1 -- 172.21.15.142:0/2054779063 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f8b2800faf0 con 0x7f8b38100ce0 2024-03-29T04:44:23.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.544+0000 7f8b2e7fc700 1 --2- 172.21.15.142:0/2054779063 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8b2406c4e0 0x7f8b2406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:23.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.544+0000 7f8b2e7fc700 1 -- 172.21.15.142:0/2054779063 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8b2808d030 con 0x7f8b38100ce0 2024-03-29T04:44:23.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.544+0000 7f8b3d774700 1 --2- 172.21.15.142:0/2054779063 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8b2406c4e0 0x7f8b2406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:23.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.545+0000 7f8b3d774700 1 --2- 172.21.15.142:0/2054779063 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8b2406c4e0 0x7f8b2406e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8b38100b40 tx=0x7f8b34009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:23.548 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.549+0000 7f8b2e7fc700 1 -- 172.21.15.142:0/2054779063 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8b2805b2c0 con 0x7f8b38100ce0 2024-03-29T04:44:23.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.957+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/2054779063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8b38108880 con 0x7f8b38100ce0 2024-03-29T04:44:23.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.958+0000 7f8b2e7fc700 1 -- 172.21.15.142:0/2054779063 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f8b28003680 con 0x7f8b38100ce0 2024-03-29T04:44:23.958 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:44:23.958 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:44:23.958 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:44:23.958 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:44:23.958 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:44:23.958 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:44:23.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:44:23.959 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:44:23.959 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:44:23.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:44:23.959 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:44:23.959 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:44:23.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:44:23.959 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:44:23.959 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-03-29T04:44:23.959 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:44:23.959 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:44:23.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.961+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/2054779063 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8b2406c4e0 msgr2=0x7f8b2406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:23.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.961+0000 7f8b3f9d8700 1 --2- 172.21.15.142:0/2054779063 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8b2406c4e0 0x7f8b2406e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8b38100b40 tx=0x7f8b34009260 comp rx=0 tx=0).stop 2024-03-29T04:44:23.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.961+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/2054779063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b38100ce0 msgr2=0x7f8b3819b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:23.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.961+0000 7f8b3f9d8700 1 --2- 172.21.15.142:0/2054779063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b38100ce0 0x7f8b3819b0f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b28000c00 tx=0x7f8b2800f690 comp rx=0 tx=0).stop 2024-03-29T04:44:23.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.961+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/2054779063 shutdown_connections 2024-03-29T04:44:23.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.961+0000 7f8b3f9d8700 1 --2- 172.21.15.142:0/2054779063 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f8b2406c4e0 0x7f8b2406e9a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.961+0000 7f8b3f9d8700 1 --2- 172.21.15.142:0/2054779063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b380ffae0 0x7f8b38069890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.961+0000 7f8b3f9d8700 1 --2- 172.21.15.142:0/2054779063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b38100ce0 0x7f8b3819b0f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:23.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.961+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/2054779063 >> 172.21.15.142:0/2054779063 conn(0x7f8b380fb080 msgr2=0x7f8b38103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:23.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.961+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/2054779063 shutdown_connections 2024-03-29T04:44:23.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:23.962+0000 7f8b3f9d8700 1 -- 172.21.15.142:0/2054779063 wait complete. 2024-03-29T04:44:24.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:23 smithi152 ceph-mon[37177]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:24.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:23 smithi152 ceph-mon[37177]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:24.111 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.113+0000 7f0285e4b700 1 -- 172.21.15.142:0/3496147963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0280100cb0 msgr2=0x7f0280101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:24.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.113+0000 7f0285e4b700 1 --2- 172.21.15.142:0/3496147963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0280100cb0 0x7f0280101110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0274009990 tx=0x7f0274009ca0 comp rx=0 tx=0).stop 2024-03-29T04:44:24.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.113+0000 7f0285e4b700 1 -- 172.21.15.142:0/3496147963 shutdown_connections 2024-03-29T04:44:24.112 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:23 smithi142 ceph-mon[29027]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:24.112 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:23 smithi142 ceph-mon[29027]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:24.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.113+0000 7f0285e4b700 1 --2- 172.21.15.142:0/3496147963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0280100cb0 0x7f0280101110 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.113+0000 7f0285e4b700 1 --2- 172.21.15.142:0/3496147963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02800ffab0 0x7f02800ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.113+0000 7f0285e4b700 1 -- 172.21.15.142:0/3496147963 >> 172.21.15.142:0/3496147963 conn(0x7f02800fb050 msgr2=0x7f02800fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.113+0000 7f0285e4b700 1 -- 172.21.15.142:0/3496147963 shutdown_connections 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.113+0000 7f0285e4b700 1 -- 172.21.15.142:0/3496147963 wait complete. 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.114+0000 7f0285e4b700 1 Processor -- start 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.114+0000 7f0285e4b700 1 -- start start 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.114+0000 7f0285e4b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02800ffab0 0x7f0280198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.114+0000 7f0285e4b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0280100cb0 0x7f0280198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.114+0000 7f0285e4b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280199090 con 0x7f0280100cb0 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.114+0000 7f0285e4b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02801991d0 con 0x7f02800ffab0 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.114+0000 7f027effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0280100cb0 0x7f0280198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.114+0000 7f027f7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02800ffab0 0x7f0280198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:24.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.114+0000 7f027f7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02800ffab0 0x7f0280198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.142:59428/0 (socket says 172.21.15.142:59428) 2024-03-29T04:44:24.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.114+0000 7f027f7fe700 1 -- 172.21.15.142:0/3677491804 learned_addr learned my addr 172.21.15.142:0/3677491804 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:44:24.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.115+0000 7f027f7fe700 1 -- 172.21.15.142:0/3677491804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0280100cb0 msgr2=0x7f0280198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:24.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.115+0000 7f027f7fe700 1 --2- 172.21.15.142:0/3677491804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0280100cb0 0x7f0280198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:24.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.115+0000 7f027f7fe700 1 -- 172.21.15.142:0/3677491804 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02740095f0 con 0x7f02800ffab0 2024-03-29T04:44:24.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.115+0000 7f027effd700 1 --2- 172.21.15.142:0/3677491804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0280100cb0 0x7f0280198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:44:24.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.115+0000 7f027f7fe700 1 --2- 172.21.15.142:0/3677491804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02800ffab0 0x7f0280198530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f027000e8f0 tx=0x7f027000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:24.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.115+0000 7f027cff9700 1 -- 172.21.15.142:0/3677491804 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0270009790 con 0x7f02800ffab0 2024-03-29T04:44:24.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.115+0000 7f027cff9700 1 -- 172.21.15.142:0/3677491804 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f02700044d0 con 0x7f02800ffab0 2024-03-29T04:44:24.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.115+0000 7f027cff9700 1 -- 172.21.15.142:0/3677491804 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0270003da0 con 0x7f02800ffab0 2024-03-29T04:44:24.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.115+0000 7f0285e4b700 1 -- 172.21.15.142:0/3677491804 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028019dc80 con 0x7f02800ffab0 2024-03-29T04:44:24.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.115+0000 7f0285e4b700 1 -- 172.21.15.142:0/3677491804 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028019e120 con 0x7f02800ffab0 2024-03-29T04:44:24.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.117+0000 7f0285e4b700 1 -- 172.21.15.142:0/3677491804 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0280107570 con 0x7f02800ffab0 2024-03-29T04:44:24.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.118+0000 7f027cff9700 1 -- 172.21.15.142:0/3677491804 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0270004020 con 0x7f02800ffab0 2024-03-29T04:44:24.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.118+0000 7f027cff9700 1 --2- 172.21.15.142:0/3677491804 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f026806c530 0x7f026806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:24.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.118+0000 7f027cff9700 1 -- 172.21.15.142:0/3677491804 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0270014070 con 0x7f02800ffab0 2024-03-29T04:44:24.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.118+0000 7f027effd700 1 --2- 172.21.15.142:0/3677491804 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f026806c530 0x7f026806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:24.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.119+0000 7f027effd700 1 --2- 172.21.15.142:0/3677491804 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f026806c530 0x7f026806e9f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0274000c00 tx=0x7f0274005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:24.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.123+0000 7f027cff9700 1 -- 172.21.15.142:0/3677491804 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0270059f00 con 0x7f02800ffab0 2024-03-29T04:44:24.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.436+0000 7f0285e4b700 1 -- 172.21.15.142:0/3677491804 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0280105600 con 0x7f026806c530 2024-03-29T04:44:24.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.437+0000 7f027cff9700 1 -- 172.21.15.142:0/3677491804 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f0280105600 con 0x7f026806c530 2024-03-29T04:44:24.436 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:44:24.436 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:44:24.436 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:44:24.436 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T04:44:24.436 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [], 2024-03-29T04:44:24.436 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "0/2 daemons upgraded", 2024-03-29T04:44:24.436 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 image on host smithi152", 2024-03-29T04:44:24.436 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:44:24.436 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:44:24.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.440+0000 7f0285e4b700 1 -- 172.21.15.142:0/3677491804 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f026806c530 msgr2=0x7f026806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:24.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.440+0000 7f0285e4b700 1 --2- 172.21.15.142:0/3677491804 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f026806c530 0x7f026806e9f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0274000c00 tx=0x7f0274005a90 comp rx=0 tx=0).stop 2024-03-29T04:44:24.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.440+0000 7f0285e4b700 1 -- 172.21.15.142:0/3677491804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02800ffab0 msgr2=0x7f0280198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:24.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.440+0000 7f0285e4b700 1 --2- 172.21.15.142:0/3677491804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02800ffab0 0x7f0280198530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f027000e8f0 tx=0x7f027000ecb0 comp rx=0 tx=0).stop 2024-03-29T04:44:24.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.440+0000 7f0285e4b700 1 -- 172.21.15.142:0/3677491804 shutdown_connections 2024-03-29T04:44:24.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.440+0000 7f0285e4b700 1 --2- 172.21.15.142:0/3677491804 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f026806c530 0x7f026806e9f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:24.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.440+0000 7f0285e4b700 1 --2- 172.21.15.142:0/3677491804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02800ffab0 0x7f0280198530 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:24.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.440+0000 7f0285e4b700 1 --2- 172.21.15.142:0/3677491804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0280100cb0 0x7f0280198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:24.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.441+0000 7f0285e4b700 1 -- 172.21.15.142:0/3677491804 >> 172.21.15.142:0/3677491804 conn(0x7f02800fb050 msgr2=0x7f0280103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:24.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.441+0000 7f0285e4b700 1 -- 172.21.15.142:0/3677491804 shutdown_connections 2024-03-29T04:44:24.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:24.441+0000 7f0285e4b700 1 -- 172.21.15.142:0/3677491804 wait complete. 2024-03-29T04:44:25.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:24 smithi152 ceph-mon[37177]: from='client.24357 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:25.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:24 smithi152 ceph-mon[37177]: pgmap v248: 65 pgs: 65 active+clean; 101 MiB data, 530 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 478 op/s 2024-03-29T04:44:25.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:24 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2054779063' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:44:25.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:24 smithi142 ceph-mon[29027]: from='client.24357 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:25.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:24 smithi142 ceph-mon[29027]: pgmap v248: 65 pgs: 65 active+clean; 101 MiB data, 530 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 478 op/s 2024-03-29T04:44:25.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:24 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2054779063' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:44:26.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:25 smithi152 ceph-mon[37177]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:26.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:25 smithi142 ceph-mon[29027]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:27.021 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:26 smithi142 ceph-mon[29027]: pgmap v249: 65 pgs: 65 active+clean; 107 MiB data, 556 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 448 op/s 2024-03-29T04:44:27.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:26 smithi152 ceph-mon[37177]: pgmap v249: 65 pgs: 65 active+clean; 107 MiB data, 556 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 448 op/s 2024-03-29T04:44:29.014 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:28 smithi152 ceph-mon[37177]: pgmap v250: 65 pgs: 65 active+clean; 119 MiB data, 630 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 548 op/s 2024-03-29T04:44:29.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:28 smithi142 ceph-mon[29027]: pgmap v250: 65 pgs: 65 active+clean; 119 MiB data, 630 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 548 op/s 2024-03-29T04:44:31.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:30 smithi152 ceph-mon[37177]: pgmap v251: 65 pgs: 65 active+clean; 131 MiB data, 676 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 503 op/s 2024-03-29T04:44:31.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:30 smithi142 ceph-mon[29027]: pgmap v251: 65 pgs: 65 active+clean; 131 MiB data, 676 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 503 op/s 2024-03-29T04:44:33.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:32 smithi152 ceph-mon[37177]: pgmap v252: 65 pgs: 65 active+clean; 141 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 516 op/s 2024-03-29T04:44:33.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:44:33.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:32 smithi142 ceph-mon[29027]: pgmap v252: 65 pgs: 65 active+clean; 141 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.9 MiB/s wr, 516 op/s 2024-03-29T04:44:33.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:44:34.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:34 smithi152 ceph-mon[37177]: pgmap v253: 65 pgs: 65 active+clean; 148 MiB data, 732 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 530 op/s 2024-03-29T04:44:34.870 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:34 smithi142 ceph-mon[29027]: pgmap v253: 65 pgs: 65 active+clean; 148 MiB data, 732 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 530 op/s 2024-03-29T04:44:37.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:36 smithi152 ceph-mon[37177]: pgmap v254: 65 pgs: 65 active+clean; 150 MiB data, 732 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-03-29T04:44:37.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:36 smithi142 ceph-mon[29027]: pgmap v254: 65 pgs: 65 active+clean; 150 MiB data, 732 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-03-29T04:44:39.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:38 smithi152 ceph-mon[37177]: pgmap v255: 65 pgs: 65 active+clean; 164 MiB data, 805 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 550 op/s 2024-03-29T04:44:39.080 INFO:tasks.workunit.client.1.smithi152.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-29T04:44:39.089 INFO:tasks.workunit.client.1.smithi152.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-29T04:44:39.090 INFO:tasks.workunit.client.1.smithi152.stderr:+ make 2024-03-29T04:44:39.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:38 smithi142 ceph-mon[29027]: pgmap v255: 65 pgs: 65 active+clean; 164 MiB data, 805 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 550 op/s 2024-03-29T04:44:39.758 INFO:tasks.workunit.client.1.smithi152.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-29T04:44:40.283 INFO:tasks.workunit.client.1.smithi152.stderr:++ readlink -f fsstress 2024-03-29T04:44:40.285 INFO:tasks.workunit.client.1.smithi152.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-29T04:44:40.285 INFO:tasks.workunit.client.1.smithi152.stderr:+ popd 2024-03-29T04:44:40.286 INFO:tasks.workunit.client.1.smithi152.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-29T04:44:40.286 INFO:tasks.workunit.client.1.smithi152.stderr:+ popd 2024-03-29T04:44:40.287 INFO:tasks.workunit.client.1.smithi152.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-29T04:44:40.287 INFO:tasks.workunit.client.1.smithi152.stderr:++ mktemp -d -p . 2024-03-29T04:44:40.470 INFO:tasks.workunit.client.1.smithi152.stderr:+ T=./tmp.MnuWEfH4w7 2024-03-29T04:44:40.470 INFO:tasks.workunit.client.1.smithi152.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.MnuWEfH4w7 -l 1 -n 1000 -p 10 -v 2024-03-29T04:44:40.474 INFO:tasks.workunit.client.1.smithi152.stdout:seed = 1711341729 2024-03-29T04:44:40.479 INFO:tasks.workunit.client.1.smithi152.stdout:0/0: rename - no filename 2024-03-29T04:44:40.480 INFO:tasks.workunit.client.1.smithi152.stdout:2/0: write - no filename 2024-03-29T04:44:40.480 INFO:tasks.workunit.client.1.smithi152.stdout:2/1: read - no filename 2024-03-29T04:44:40.481 INFO:tasks.workunit.client.1.smithi152.stdout:0/1: mknod c0 0 2024-03-29T04:44:40.481 INFO:tasks.workunit.client.1.smithi152.stdout:0/2: dread - no filename 2024-03-29T04:44:40.481 INFO:tasks.workunit.client.1.smithi152.stdout:0/3: fsync - no filename 2024-03-29T04:44:40.482 INFO:tasks.workunit.client.1.smithi152.stdout:1/0: dwrite - no filename 2024-03-29T04:44:40.482 INFO:tasks.workunit.client.1.smithi152.stdout:1/1: chown . 2099983436 1 2024-03-29T04:44:40.482 INFO:tasks.workunit.client.1.smithi152.stdout:1/2: dwrite - no filename 2024-03-29T04:44:40.482 INFO:tasks.workunit.client.1.smithi152.stdout:1/3: dread - no filename 2024-03-29T04:44:40.482 INFO:tasks.workunit.client.1.smithi152.stdout:1/4: dread - no filename 2024-03-29T04:44:40.483 INFO:tasks.workunit.client.1.smithi152.stdout:0/4: mknod c1 0 2024-03-29T04:44:40.483 INFO:tasks.workunit.client.1.smithi152.stdout:0/5: truncate - no filename 2024-03-29T04:44:40.483 INFO:tasks.workunit.client.1.smithi152.stdout:0/6: dwrite - no filename 2024-03-29T04:44:40.486 INFO:tasks.workunit.client.1.smithi152.stdout:3/0: read - no filename 2024-03-29T04:44:40.487 INFO:tasks.workunit.client.1.smithi152.stdout:3/1: chown . 52812 1 2024-03-29T04:44:40.487 INFO:tasks.workunit.client.1.smithi152.stdout:3/2: dread - no filename 2024-03-29T04:44:40.487 INFO:tasks.workunit.client.1.smithi152.stdout:2/2: mkdir d0 0 2024-03-29T04:44:40.487 INFO:tasks.workunit.client.1.smithi152.stdout:3/3: chown . 772108 1 2024-03-29T04:44:40.487 INFO:tasks.workunit.client.1.smithi152.stdout:3/4: chown . 342 1 2024-03-29T04:44:40.487 INFO:tasks.workunit.client.1.smithi152.stdout:3/5: write - no filename 2024-03-29T04:44:40.487 INFO:tasks.workunit.client.1.smithi152.stdout:3/6: write - no filename 2024-03-29T04:44:40.487 INFO:tasks.workunit.client.1.smithi152.stdout:3/7: dwrite - no filename 2024-03-29T04:44:40.487 INFO:tasks.workunit.client.1.smithi152.stdout:3/8: truncate - no filename 2024-03-29T04:44:40.487 INFO:tasks.workunit.client.1.smithi152.stdout:3/9: dread - no filename 2024-03-29T04:44:40.487 INFO:tasks.workunit.client.1.smithi152.stdout:3/10: dwrite - no filename 2024-03-29T04:44:40.487 INFO:tasks.workunit.client.1.smithi152.stdout:3/11: dread - no filename 2024-03-29T04:44:40.488 INFO:tasks.workunit.client.1.smithi152.stdout:3/12: chown . 790193 1 2024-03-29T04:44:40.488 INFO:tasks.workunit.client.1.smithi152.stdout:0/7: mkdir d2 0 2024-03-29T04:44:40.489 INFO:tasks.workunit.client.1.smithi152.stdout:7/0: chown . 58826162 1 2024-03-29T04:44:40.490 INFO:tasks.workunit.client.1.smithi152.stdout:1/5: mknod c0 0 2024-03-29T04:44:40.490 INFO:tasks.workunit.client.1.smithi152.stdout:1/6: read - no filename 2024-03-29T04:44:40.494 INFO:tasks.workunit.client.1.smithi152.stdout:1/7: mknod c1 0 2024-03-29T04:44:40.494 INFO:tasks.workunit.client.1.smithi152.stdout:3/13: getdents . 0 2024-03-29T04:44:40.495 INFO:tasks.workunit.client.1.smithi152.stdout:0/8: creat d2/f3 x:0 0 0 2024-03-29T04:44:40.496 INFO:tasks.workunit.client.1.smithi152.stdout:7/1: symlink l0 0 2024-03-29T04:44:40.497 INFO:tasks.workunit.client.1.smithi152.stdout:5/0: fsync - no filename 2024-03-29T04:44:40.497 INFO:tasks.workunit.client.1.smithi152.stdout:1/8: mknod c2 0 2024-03-29T04:44:40.500 INFO:tasks.workunit.client.1.smithi152.stdout:7/2: rename l0 to l1 0 2024-03-29T04:44:40.500 INFO:tasks.workunit.client.1.smithi152.stdout:7/3: write - no filename 2024-03-29T04:44:40.504 INFO:tasks.workunit.client.1.smithi152.stdout:7/4: mknod c2 0 2024-03-29T04:44:40.505 INFO:tasks.workunit.client.1.smithi152.stdout:5/1: creat f0 x:0 0 0 2024-03-29T04:44:40.508 INFO:tasks.workunit.client.1.smithi152.stdout:1/9: link c0 c3 0 2024-03-29T04:44:40.508 INFO:tasks.workunit.client.1.smithi152.stdout:1/10: write - no filename 2024-03-29T04:44:40.508 INFO:tasks.workunit.client.1.smithi152.stdout:1/11: truncate - no filename 2024-03-29T04:44:40.508 INFO:tasks.workunit.client.1.smithi152.stdout:1/12: write - no filename 2024-03-29T04:44:40.508 INFO:tasks.workunit.client.1.smithi152.stdout:1/13: write - no filename 2024-03-29T04:44:40.508 INFO:tasks.workunit.client.1.smithi152.stdout:1/14: fsync - no filename 2024-03-29T04:44:40.508 INFO:tasks.workunit.client.1.smithi152.stdout:1/15: dread - no filename 2024-03-29T04:44:40.509 INFO:tasks.workunit.client.1.smithi152.stdout:9/0: getdents . 0 2024-03-29T04:44:40.510 INFO:tasks.workunit.client.1.smithi152.stdout:1/16: mknod c4 0 2024-03-29T04:44:40.511 INFO:tasks.workunit.client.1.smithi152.stdout:6/0: creat f0 x:0 0 0 2024-03-29T04:44:40.513 INFO:tasks.workunit.client.1.smithi152.stdout:6/1: chown f0 10482 1 2024-03-29T04:44:40.514 INFO:tasks.workunit.client.1.smithi152.stdout:1/17: mkdir d5 0 2024-03-29T04:44:40.515 INFO:tasks.workunit.client.1.smithi152.stdout:8/0: symlink l0 0 2024-03-29T04:44:40.517 INFO:tasks.workunit.client.1.smithi152.stdout:9/1: creat f0 x:0 0 0 2024-03-29T04:44:40.517 INFO:tasks.workunit.client.1.smithi152.stdout:9/2: dread - f0 zero size 2024-03-29T04:44:40.517 INFO:tasks.workunit.client.1.smithi152.stdout:6/2: mkdir d1 0 2024-03-29T04:44:40.518 INFO:tasks.workunit.client.1.smithi152.stdout:9/3: read - f0 zero size 2024-03-29T04:44:40.518 INFO:tasks.workunit.client.1.smithi152.stdout:6/3: truncate f0 419265 0 2024-03-29T04:44:40.519 INFO:tasks.workunit.client.1.smithi152.stdout:8/1: mkdir d1 0 2024-03-29T04:44:40.520 INFO:tasks.workunit.client.1.smithi152.stdout:1/18: creat d5/f6 x:0 0 0 2024-03-29T04:44:40.521 INFO:tasks.workunit.client.1.smithi152.stdout:9/4: creat f1 x:0 0 0 2024-03-29T04:44:40.521 INFO:tasks.workunit.client.1.smithi152.stdout:1/19: truncate d5/f6 344233 0 2024-03-29T04:44:40.522 INFO:tasks.workunit.client.1.smithi152.stdout:6/4: mkdir d1/d2 0 2024-03-29T04:44:40.523 INFO:tasks.workunit.client.1.smithi152.stdout:8/2: mkdir d1/d2 0 2024-03-29T04:44:40.525 INFO:tasks.workunit.client.1.smithi152.stdout:1/20: mkdir d5/d7 0 2024-03-29T04:44:40.525 INFO:tasks.workunit.client.1.smithi152.stdout:9/5: chown f0 1 1 2024-03-29T04:44:40.525 INFO:tasks.workunit.client.1.smithi152.stdout:6/5: rename d1/d2 to d1/d3 0 2024-03-29T04:44:40.526 INFO:tasks.workunit.client.1.smithi152.stdout:9/6: chown f0 689217872 1 2024-03-29T04:44:40.527 INFO:tasks.workunit.client.1.smithi152.stdout:8/3: mkdir d1/d2/d3 0 2024-03-29T04:44:40.527 INFO:tasks.workunit.client.1.smithi152.stdout:8/4: dwrite - no filename 2024-03-29T04:44:40.527 INFO:tasks.workunit.client.1.smithi152.stdout:9/7: write f1 [911859,21973] 0 2024-03-29T04:44:40.529 INFO:tasks.workunit.client.1.smithi152.stdout:1/21: mkdir d5/d7/d8 0 2024-03-29T04:44:40.529 INFO:tasks.workunit.client.1.smithi152.stdout:6/6: dread f0 [0,4194304] 0 2024-03-29T04:44:40.534 INFO:tasks.workunit.client.1.smithi152.stdout:1/22: mkdir d5/d7/d9 0 2024-03-29T04:44:40.534 INFO:tasks.workunit.client.1.smithi152.stdout:6/7: symlink d1/l4 0 2024-03-29T04:44:40.534 INFO:tasks.workunit.client.1.smithi152.stdout:1/23: mknod d5/ca 0 2024-03-29T04:44:40.534 INFO:tasks.workunit.client.1.smithi152.stdout:6/8: fdatasync f0 0 2024-03-29T04:44:40.535 INFO:tasks.workunit.client.1.smithi152.stdout:1/24: creat d5/d7/d8/fb x:0 0 0 2024-03-29T04:44:40.536 INFO:tasks.workunit.client.1.smithi152.stdout:6/9: creat d1/f5 x:0 0 0 2024-03-29T04:44:40.538 INFO:tasks.workunit.client.1.smithi152.stdout:6/10: dread f0 [0,4194304] 0 2024-03-29T04:44:40.542 INFO:tasks.workunit.client.1.smithi152.stdout:6/11: rename d1/l4 to d1/d3/l6 0 2024-03-29T04:44:40.545 INFO:tasks.workunit.client.1.smithi152.stdout:6/12: unlink d1/f5 0 2024-03-29T04:44:40.546 INFO:tasks.workunit.client.1.smithi152.stdout:1/25: dwrite d5/d7/d8/fb [0,4194304] 0 2024-03-29T04:44:40.548 INFO:tasks.workunit.client.1.smithi152.stdout:6/13: creat d1/d3/f7 x:0 0 0 2024-03-29T04:44:40.549 INFO:tasks.workunit.client.1.smithi152.stdout:1/26: symlink d5/d7/d8/lc 0 2024-03-29T04:44:40.550 INFO:tasks.workunit.client.1.smithi152.stdout:6/14: creat d1/f8 x:0 0 0 2024-03-29T04:44:40.551 INFO:tasks.workunit.client.1.smithi152.stdout:6/15: fdatasync d1/f8 0 2024-03-29T04:44:40.551 INFO:tasks.workunit.client.1.smithi152.stdout:6/16: stat d1/f8 0 2024-03-29T04:44:40.633 INFO:tasks.workunit.client.1.smithi152.stdout:1/27: dread d5/f6 [0,4194304] 0 2024-03-29T04:44:40.638 INFO:tasks.workunit.client.1.smithi152.stdout:1/28: rename c0 to d5/d7/d9/cd 0 2024-03-29T04:44:40.640 INFO:tasks.workunit.client.1.smithi152.stdout:1/29: creat d5/d7/fe x:0 0 0 2024-03-29T04:44:40.641 INFO:tasks.workunit.client.1.smithi152.stdout:1/30: fdatasync d5/d7/d8/fb 0 2024-03-29T04:44:40.642 INFO:tasks.workunit.client.1.smithi152.stdout:1/31: rename d5 to d5/d7/d9/df 22 2024-03-29T04:44:40.645 INFO:tasks.workunit.client.1.smithi152.stdout:1/32: unlink d5/f6 0 2024-03-29T04:44:40.646 INFO:tasks.workunit.client.1.smithi152.stdout:1/33: truncate d5/d7/fe 682317 0 2024-03-29T04:44:40.708 INFO:tasks.workunit.client.1.smithi152.stdout:0/9: rmdir d2 39 2024-03-29T04:44:40.709 INFO:tasks.workunit.client.1.smithi152.stdout:0/10: rmdir d2 39 2024-03-29T04:44:40.712 INFO:tasks.workunit.client.1.smithi152.stdout:0/11: creat d2/f4 x:0 0 0 2024-03-29T04:44:40.712 INFO:tasks.workunit.client.1.smithi152.stdout:7/5: getdents . 0 2024-03-29T04:44:40.713 INFO:tasks.workunit.client.1.smithi152.stdout:7/6: fdatasync - no filename 2024-03-29T04:44:40.713 INFO:tasks.workunit.client.1.smithi152.stdout:0/12: stat d2/f4 0 2024-03-29T04:44:40.713 INFO:tasks.workunit.client.1.smithi152.stdout:0/13: dread - d2/f4 zero size 2024-03-29T04:44:40.714 INFO:tasks.workunit.client.1.smithi152.stdout:0/14: stat c0 0 2024-03-29T04:44:40.714 INFO:tasks.workunit.client.1.smithi152.stdout:7/7: mkdir d3 0 2024-03-29T04:44:40.714 INFO:tasks.workunit.client.1.smithi152.stdout:7/8: dread - no filename 2024-03-29T04:44:40.724 INFO:tasks.workunit.client.1.smithi152.stdout:6/17: readlink d1/d3/l6 0 2024-03-29T04:44:40.724 INFO:tasks.workunit.client.1.smithi152.stdout:0/15: dwrite d2/f4 [0,4194304] 0 2024-03-29T04:44:40.725 INFO:tasks.workunit.client.1.smithi152.stdout:0/16: write d2/f4 [4105750,28506] 0 2024-03-29T04:44:40.735 INFO:tasks.workunit.client.1.smithi152.stdout:1/34: unlink d5/d7/d9/cd 0 2024-03-29T04:44:40.737 INFO:tasks.workunit.client.1.smithi152.stdout:6/18: dwrite d1/f8 [0,4194304] 0 2024-03-29T04:44:40.742 INFO:tasks.workunit.client.1.smithi152.stdout:1/35: creat d5/d7/d9/f10 x:0 0 0 2024-03-29T04:44:40.742 INFO:tasks.workunit.client.1.smithi152.stdout:6/19: mkdir d1/d3/d9 0 2024-03-29T04:44:40.742 INFO:tasks.workunit.client.1.smithi152.stdout:6/20: write f0 [719923,121143] 0 2024-03-29T04:44:40.744 INFO:tasks.workunit.client.1.smithi152.stdout:6/21: truncate d1/f8 4465702 0 2024-03-29T04:44:40.746 INFO:tasks.workunit.client.1.smithi152.stdout:6/22: readlink d1/d3/l6 0 2024-03-29T04:44:40.748 INFO:tasks.workunit.client.1.smithi152.stdout:0/17: fdatasync d2/f4 0 2024-03-29T04:44:40.748 INFO:tasks.workunit.client.1.smithi152.stdout:6/23: truncate d1/d3/f7 209106 0 2024-03-29T04:44:40.751 INFO:tasks.workunit.client.1.smithi152.stdout:0/18: creat d2/f5 x:0 0 0 2024-03-29T04:44:40.756 INFO:tasks.workunit.client.1.smithi152.stdout:0/19: write d2/f4 [404330,25148] 0 2024-03-29T04:44:40.756 INFO:tasks.workunit.client.1.smithi152.stdout:0/20: readlink - no filename 2024-03-29T04:44:40.759 INFO:tasks.workunit.client.1.smithi152.stdout:0/21: symlink d2/l6 0 2024-03-29T04:44:40.759 INFO:tasks.workunit.client.1.smithi152.stdout:6/24: dwrite f0 [0,4194304] 0 2024-03-29T04:44:40.760 INFO:tasks.workunit.client.1.smithi152.stdout:0/22: chown d2 1 1 2024-03-29T04:44:40.761 INFO:tasks.workunit.client.1.smithi152.stdout:0/23: readlink d2/l6 0 2024-03-29T04:44:40.763 INFO:tasks.workunit.client.1.smithi152.stdout:6/25: symlink d1/d3/d9/la 0 2024-03-29T04:44:40.768 INFO:tasks.workunit.client.1.smithi152.stdout:6/26: creat d1/d3/d9/fb x:0 0 0 2024-03-29T04:44:40.770 INFO:tasks.workunit.client.1.smithi152.stdout:6/27: chown d1/f8 324602394 1 2024-03-29T04:44:40.772 INFO:tasks.workunit.client.1.smithi152.stdout:0/24: dwrite d2/f5 [0,4194304] 0 2024-03-29T04:44:40.776 INFO:tasks.workunit.client.1.smithi152.stdout:0/25: rename d2/f3 to d2/f7 0 2024-03-29T04:44:40.779 INFO:tasks.workunit.client.1.smithi152.stdout:6/28: dread d1/f8 [0,4194304] 0 2024-03-29T04:44:40.781 INFO:tasks.workunit.client.1.smithi152.stdout:0/26: creat d2/f8 x:0 0 0 2024-03-29T04:44:40.781 INFO:tasks.workunit.client.1.smithi152.stdout:6/29: stat d1/d3/d9 0 2024-03-29T04:44:40.782 INFO:tasks.workunit.client.1.smithi152.stdout:0/27: chown d2/l6 176784255 1 2024-03-29T04:44:40.783 INFO:tasks.workunit.client.1.smithi152.stdout:0/28: write d2/f4 [3273989,10020] 0 2024-03-29T04:44:40.784 INFO:tasks.workunit.client.1.smithi152.stdout:6/30: symlink d1/lc 0 2024-03-29T04:44:40.784 INFO:tasks.workunit.client.1.smithi152.stdout:0/29: read - d2/f8 zero size 2024-03-29T04:44:40.785 INFO:tasks.workunit.client.1.smithi152.stdout:6/31: creat d1/fd x:0 0 0 2024-03-29T04:44:40.786 INFO:tasks.workunit.client.1.smithi152.stdout:0/30: creat d2/f9 x:0 0 0 2024-03-29T04:44:40.787 INFO:tasks.workunit.client.1.smithi152.stdout:0/31: chown d2/f5 37516 1 2024-03-29T04:44:40.789 INFO:tasks.workunit.client.1.smithi152.stdout:6/32: rename d1/d3/f7 to d1/fe 0 2024-03-29T04:44:40.789 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:40 smithi152 ceph-mon[37177]: pgmap v256: 65 pgs: 65 active+clean; 178 MiB data, 856 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 483 op/s 2024-03-29T04:44:40.790 INFO:tasks.workunit.client.1.smithi152.stdout:0/32: mkdir d2/da 0 2024-03-29T04:44:40.790 INFO:tasks.workunit.client.1.smithi152.stdout:6/33: mkdir d1/d3/df 0 2024-03-29T04:44:40.791 INFO:tasks.workunit.client.1.smithi152.stdout:0/33: chown d2/f7 6090 1 2024-03-29T04:44:40.793 INFO:tasks.workunit.client.1.smithi152.stdout:6/34: creat d1/d3/df/f10 x:0 0 0 2024-03-29T04:44:40.794 INFO:tasks.workunit.client.1.smithi152.stdout:6/35: fsync d1/d3/d9/fb 0 2024-03-29T04:44:40.796 INFO:tasks.workunit.client.1.smithi152.stdout:6/36: creat d1/d3/d9/f11 x:0 0 0 2024-03-29T04:44:40.799 INFO:tasks.workunit.client.1.smithi152.stdout:6/37: write d1/fe [681587,46871] 0 2024-03-29T04:44:40.802 INFO:tasks.workunit.client.1.smithi152.stdout:6/38: symlink d1/d3/d9/l12 0 2024-03-29T04:44:40.802 INFO:tasks.workunit.client.1.smithi152.stdout:6/39: stat d1/d3/l6 0 2024-03-29T04:44:40.804 INFO:tasks.workunit.client.1.smithi152.stdout:0/34: truncate d2/f4 3085821 0 2024-03-29T04:44:40.805 INFO:tasks.workunit.client.1.smithi152.stdout:0/35: readlink d2/l6 0 2024-03-29T04:44:40.813 INFO:tasks.workunit.client.1.smithi152.stdout:0/36: dwrite d2/f9 [0,4194304] 0 2024-03-29T04:44:40.824 INFO:tasks.workunit.client.1.smithi152.stdout:0/37: dwrite d2/f9 [4194304,4194304] 0 2024-03-29T04:44:40.829 INFO:tasks.workunit.client.1.smithi152.stdout:0/38: creat d2/da/fb x:0 0 0 2024-03-29T04:44:40.830 INFO:tasks.workunit.client.1.smithi152.stdout:0/39: write d2/f5 [2208295,126959] 0 2024-03-29T04:44:40.834 INFO:tasks.workunit.client.1.smithi152.stdout:0/40: symlink d2/da/lc 0 2024-03-29T04:44:40.838 INFO:tasks.workunit.client.1.smithi152.stdout:0/41: mknod d2/cd 0 2024-03-29T04:44:40.839 INFO:tasks.workunit.client.1.smithi152.stdout:0/42: read - d2/da/fb zero size 2024-03-29T04:44:40.842 INFO:tasks.workunit.client.1.smithi152.stdout:0/43: symlink d2/da/le 0 2024-03-29T04:44:40.847 INFO:tasks.workunit.client.1.smithi152.stdout:0/44: dread d2/f5 [0,4194304] 0 2024-03-29T04:44:40.850 INFO:tasks.workunit.client.1.smithi152.stdout:0/45: mkdir d2/da/df 0 2024-03-29T04:44:40.852 INFO:tasks.workunit.client.1.smithi152.stdout:0/46: mknod d2/da/c10 0 2024-03-29T04:44:40.860 INFO:tasks.workunit.client.1.smithi152.stdout:0/47: dwrite d2/f8 [0,4194304] 0 2024-03-29T04:44:40.866 INFO:tasks.workunit.client.1.smithi152.stdout:0/48: mkdir d2/da/df/d11 0 2024-03-29T04:44:40.870 INFO:tasks.workunit.client.1.smithi152.stdout:0/49: dread d2/f9 [0,4194304] 0 2024-03-29T04:44:40.872 INFO:tasks.workunit.client.1.smithi152.stdout:0/50: truncate d2/da/fb 112154 0 2024-03-29T04:44:40.883 INFO:tasks.workunit.client.1.smithi152.stdout:0/51: dwrite d2/f7 [0,4194304] 0 2024-03-29T04:44:40.885 INFO:tasks.workunit.client.1.smithi152.stdout:0/52: symlink d2/da/l12 0 2024-03-29T04:44:40.888 INFO:tasks.workunit.client.1.smithi152.stdout:0/53: link d2/da/le d2/da/df/d11/l13 0 2024-03-29T04:44:40.890 INFO:tasks.workunit.client.1.smithi152.stdout:0/54: write d2/f5 [2381191,127960] 0 2024-03-29T04:44:40.894 INFO:tasks.workunit.client.1.smithi152.stdout:0/55: mknod d2/da/df/c14 0 2024-03-29T04:44:40.897 INFO:tasks.workunit.client.1.smithi152.stdout:0/56: mknod d2/da/df/c15 0 2024-03-29T04:44:40.897 INFO:tasks.workunit.client.1.smithi152.stdout:0/57: read d2/f8 [805416,54284] 0 2024-03-29T04:44:40.901 INFO:tasks.workunit.client.1.smithi152.stdout:0/58: mknod d2/da/df/c16 0 2024-03-29T04:44:40.911 INFO:tasks.workunit.client.1.smithi152.stdout:0/59: rename d2/da/df/d11 to d2/da/d17 0 2024-03-29T04:44:40.913 INFO:tasks.workunit.client.1.smithi152.stdout:0/60: symlink d2/l18 0 2024-03-29T04:44:40.914 INFO:tasks.workunit.client.1.smithi152.stdout:0/61: write d2/f5 [3488572,120276] 0 2024-03-29T04:44:40.916 INFO:tasks.workunit.client.1.smithi152.stdout:0/62: chown d2/da/d17/l13 195923 1 2024-03-29T04:44:40.919 INFO:tasks.workunit.client.1.smithi152.stdout:0/63: dread d2/f4 [0,4194304] 0 2024-03-29T04:44:40.920 INFO:tasks.workunit.client.1.smithi152.stdout:0/64: chown d2/da/df/c15 3176 1 2024-03-29T04:44:40.922 INFO:tasks.workunit.client.1.smithi152.stdout:0/65: mknod d2/da/c19 0 2024-03-29T04:44:40.924 INFO:tasks.workunit.client.1.smithi152.stdout:0/66: mknod d2/da/c1a 0 2024-03-29T04:44:40.926 INFO:tasks.workunit.client.1.smithi152.stdout:0/67: mkdir d2/d1b 0 2024-03-29T04:44:40.927 INFO:tasks.workunit.client.1.smithi152.stdout:0/68: mknod d2/c1c 0 2024-03-29T04:44:40.927 INFO:tasks.workunit.client.1.smithi152.stdout:0/69: readlink d2/l6 0 2024-03-29T04:44:40.930 INFO:tasks.workunit.client.1.smithi152.stdout:0/70: rename d2/da/c1a to d2/da/c1d 0 2024-03-29T04:44:40.941 INFO:tasks.workunit.client.1.smithi152.stdout:0/71: dwrite d2/f4 [0,4194304] 0 2024-03-29T04:44:40.952 INFO:tasks.workunit.client.1.smithi152.stdout:0/72: truncate d2/f7 1114742 0 2024-03-29T04:44:40.957 INFO:tasks.workunit.client.1.smithi152.stdout:0/73: dread d2/f4 [0,4194304] 0 2024-03-29T04:44:40.964 INFO:tasks.workunit.client.1.smithi152.stdout:0/74: dwrite d2/f5 [0,4194304] 0 2024-03-29T04:44:40.975 INFO:tasks.workunit.client.1.smithi152.stdout:0/75: rmdir d2 39 2024-03-29T04:44:40.985 INFO:tasks.workunit.client.1.smithi152.stdout:0/76: truncate d2/f7 691990 0 2024-03-29T04:44:41.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:40 smithi142 ceph-mon[29027]: pgmap v256: 65 pgs: 65 active+clean; 178 MiB data, 856 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 483 op/s 2024-03-29T04:44:43.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:42 smithi152 ceph-mon[37177]: pgmap v257: 65 pgs: 65 active+clean; 198 MiB data, 885 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 474 op/s 2024-03-29T04:44:43.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:42 smithi142 ceph-mon[29027]: pgmap v257: 65 pgs: 65 active+clean; 198 MiB data, 885 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 474 op/s 2024-03-29T04:44:44.789 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:44 smithi152 ceph-mon[37177]: pgmap v258: 65 pgs: 65 active+clean; 217 MiB data, 915 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 455 op/s 2024-03-29T04:44:44.870 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:44 smithi142 ceph-mon[29027]: pgmap v258: 65 pgs: 65 active+clean; 217 MiB data, 915 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 455 op/s 2024-03-29T04:44:47.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:46 smithi152 ceph-mon[37177]: pgmap v259: 65 pgs: 65 active+clean; 226 MiB data, 954 MiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 7.5 MiB/s wr, 391 op/s 2024-03-29T04:44:47.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:46 smithi142 ceph-mon[29027]: pgmap v259: 65 pgs: 65 active+clean; 226 MiB data, 954 MiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 7.5 MiB/s wr, 391 op/s 2024-03-29T04:44:48.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:47 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:44:48.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:47 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:44:49.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:48 smithi152 ceph-mon[37177]: pgmap v260: 65 pgs: 65 active+clean; 239 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 8.5 MiB/s wr, 431 op/s 2024-03-29T04:44:49.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:48 smithi142 ceph-mon[29027]: pgmap v260: 65 pgs: 65 active+clean; 239 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 8.5 MiB/s wr, 431 op/s 2024-03-29T04:44:49.958 INFO:tasks.workunit.client.0.smithi142.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-29T04:44:49.966 INFO:tasks.workunit.client.0.smithi142.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-29T04:44:49.970 INFO:tasks.workunit.client.0.smithi142.stderr:+ make 2024-03-29T04:44:50.209 INFO:tasks.workunit.client.0.smithi142.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-29T04:44:50.564 INFO:tasks.workunit.client.0.smithi142.stderr:++ readlink -f fsstress 2024-03-29T04:44:50.566 INFO:tasks.workunit.client.0.smithi142.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-29T04:44:50.566 INFO:tasks.workunit.client.0.smithi142.stderr:+ popd 2024-03-29T04:44:50.567 INFO:tasks.workunit.client.0.smithi142.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-29T04:44:50.567 INFO:tasks.workunit.client.0.smithi142.stderr:+ popd 2024-03-29T04:44:50.567 INFO:tasks.workunit.client.0.smithi142.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-29T04:44:50.567 INFO:tasks.workunit.client.0.smithi142.stderr:++ mktemp -d -p . 2024-03-29T04:44:50.570 INFO:tasks.workunit.client.0.smithi142.stderr:+ T=./tmp.847eakZwIb 2024-03-29T04:44:50.570 INFO:tasks.workunit.client.0.smithi142.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.847eakZwIb -l 1 -n 1000 -p 10 -v 2024-03-29T04:44:50.573 INFO:tasks.workunit.client.0.smithi142.stdout:seed = 1712230149 2024-03-29T04:44:50.577 INFO:tasks.workunit.client.0.smithi142.stdout:0/0: unlink - no file 2024-03-29T04:44:50.577 INFO:tasks.workunit.client.0.smithi142.stdout:0/1: unlink - no file 2024-03-29T04:44:50.577 INFO:tasks.workunit.client.0.smithi142.stdout:0/2: truncate - no filename 2024-03-29T04:44:50.577 INFO:tasks.workunit.client.0.smithi142.stdout:0/3: fdatasync - no filename 2024-03-29T04:44:50.577 INFO:tasks.workunit.client.0.smithi142.stdout:0/4: chown . 61528 1 2024-03-29T04:44:50.578 INFO:tasks.workunit.client.0.smithi142.stdout:0/5: mknod c0 0 2024-03-29T04:44:50.578 INFO:tasks.workunit.client.0.smithi142.stdout:0/6: fsync - no filename 2024-03-29T04:44:50.578 INFO:tasks.workunit.client.0.smithi142.stdout:0/7: dwrite - no filename 2024-03-29T04:44:50.579 INFO:tasks.workunit.client.0.smithi142.stdout:2/0: chown . 38625736 1 2024-03-29T04:44:50.579 INFO:tasks.workunit.client.0.smithi142.stdout:2/1: dread - no filename 2024-03-29T04:44:50.580 INFO:tasks.workunit.client.0.smithi142.stdout:2/2: dread - no filename 2024-03-29T04:44:50.580 INFO:tasks.workunit.client.0.smithi142.stdout:0/8: mknod c1 0 2024-03-29T04:44:50.580 INFO:tasks.workunit.client.0.smithi142.stdout:0/9: dread - no filename 2024-03-29T04:44:50.582 INFO:tasks.workunit.client.0.smithi142.stdout:1/0: mknod c0 0 2024-03-29T04:44:50.582 INFO:tasks.workunit.client.0.smithi142.stdout:1/1: write - no filename 2024-03-29T04:44:50.582 INFO:tasks.workunit.client.0.smithi142.stdout:1/2: write - no filename 2024-03-29T04:44:50.582 INFO:tasks.workunit.client.0.smithi142.stdout:0/10: creat f2 x:0 0 0 2024-03-29T04:44:50.583 INFO:tasks.workunit.client.0.smithi142.stdout:2/3: getdents . 0 2024-03-29T04:44:50.583 INFO:tasks.workunit.client.0.smithi142.stdout:2/4: rename - no filename 2024-03-29T04:44:50.584 INFO:tasks.workunit.client.0.smithi142.stdout:4/0: read - no filename 2024-03-29T04:44:50.584 INFO:tasks.workunit.client.0.smithi142.stdout:4/1: write - no filename 2024-03-29T04:44:50.584 INFO:tasks.workunit.client.0.smithi142.stdout:4/2: truncate - no filename 2024-03-29T04:44:50.584 INFO:tasks.workunit.client.0.smithi142.stdout:4/3: dwrite - no filename 2024-03-29T04:44:50.584 INFO:tasks.workunit.client.0.smithi142.stdout:4/4: rename - no filename 2024-03-29T04:44:50.584 INFO:tasks.workunit.client.0.smithi142.stdout:4/5: dread - no filename 2024-03-29T04:44:50.585 INFO:tasks.workunit.client.0.smithi142.stdout:0/11: symlink l3 0 2024-03-29T04:44:50.586 INFO:tasks.workunit.client.0.smithi142.stdout:3/0: symlink l0 0 2024-03-29T04:44:50.586 INFO:tasks.workunit.client.0.smithi142.stdout:3/1: write - no filename 2024-03-29T04:44:50.586 INFO:tasks.workunit.client.0.smithi142.stdout:3/2: write - no filename 2024-03-29T04:44:50.586 INFO:tasks.workunit.client.0.smithi142.stdout:3/3: read - no filename 2024-03-29T04:44:50.587 INFO:tasks.workunit.client.0.smithi142.stdout:5/0: write - no filename 2024-03-29T04:44:50.587 INFO:tasks.workunit.client.0.smithi142.stdout:1/3: link c0 c1 0 2024-03-29T04:44:50.587 INFO:tasks.workunit.client.0.smithi142.stdout:1/4: truncate - no filename 2024-03-29T04:44:50.588 INFO:tasks.workunit.client.0.smithi142.stdout:2/5: symlink l0 0 2024-03-29T04:44:50.588 INFO:tasks.workunit.client.0.smithi142.stdout:2/6: write - no filename 2024-03-29T04:44:50.589 INFO:tasks.workunit.client.0.smithi142.stdout:2/7: readlink l0 0 2024-03-29T04:44:50.590 INFO:tasks.workunit.client.0.smithi142.stdout:0/12: unlink c1 0 2024-03-29T04:44:50.591 INFO:tasks.workunit.client.0.smithi142.stdout:3/4: symlink l1 0 2024-03-29T04:44:50.592 INFO:tasks.workunit.client.0.smithi142.stdout:4/6: mkdir d0 0 2024-03-29T04:44:50.595 INFO:tasks.workunit.client.0.smithi142.stdout:2/8: symlink l1 0 2024-03-29T04:44:50.595 INFO:tasks.workunit.client.0.smithi142.stdout:2/9: fsync - no filename 2024-03-29T04:44:50.597 INFO:tasks.workunit.client.0.smithi142.stdout:3/5: rename l1 to l2 0 2024-03-29T04:44:50.600 INFO:tasks.workunit.client.0.smithi142.stdout:1/5: unlink c0 0 2024-03-29T04:44:50.600 INFO:tasks.workunit.client.0.smithi142.stdout:1/6: dread - no filename 2024-03-29T04:44:50.602 INFO:tasks.workunit.client.0.smithi142.stdout:5/1: getdents . 0 2024-03-29T04:44:50.602 INFO:tasks.workunit.client.0.smithi142.stdout:5/2: fdatasync - no filename 2024-03-29T04:44:50.602 INFO:tasks.workunit.client.0.smithi142.stdout:5/3: fsync - no filename 2024-03-29T04:44:50.602 INFO:tasks.workunit.client.0.smithi142.stdout:5/4: chown . 477425950 1 2024-03-29T04:44:50.603 INFO:tasks.workunit.client.0.smithi142.stdout:5/5: dread - no filename 2024-03-29T04:44:50.603 INFO:tasks.workunit.client.0.smithi142.stdout:5/6: write - no filename 2024-03-29T04:44:50.603 INFO:tasks.workunit.client.0.smithi142.stdout:5/7: rmdir - no directory 2024-03-29T04:44:50.603 INFO:tasks.workunit.client.0.smithi142.stdout:5/8: write - no filename 2024-03-29T04:44:50.603 INFO:tasks.workunit.client.0.smithi142.stdout:5/9: chown . 6957 1 2024-03-29T04:44:50.603 INFO:tasks.workunit.client.0.smithi142.stdout:5/10: link - no file 2024-03-29T04:44:50.603 INFO:tasks.workunit.client.0.smithi142.stdout:5/11: chown . 6131 1 2024-03-29T04:44:50.603 INFO:tasks.workunit.client.0.smithi142.stdout:5/12: write - no filename 2024-03-29T04:44:50.603 INFO:tasks.workunit.client.0.smithi142.stdout:2/10: rename l0 to l2 0 2024-03-29T04:44:50.604 INFO:tasks.workunit.client.0.smithi142.stdout:2/11: dwrite - no filename 2024-03-29T04:44:50.605 INFO:tasks.workunit.client.0.smithi142.stdout:4/7: creat d0/f1 x:0 0 0 2024-03-29T04:44:50.605 INFO:tasks.workunit.client.0.smithi142.stdout:4/8: rename d0 to d0/d2 22 2024-03-29T04:44:50.606 INFO:tasks.workunit.client.0.smithi142.stdout:3/6: mkdir d3 0 2024-03-29T04:44:50.606 INFO:tasks.workunit.client.0.smithi142.stdout:3/7: dwrite - no filename 2024-03-29T04:44:50.609 INFO:tasks.workunit.client.0.smithi142.stdout:8/0: rename - no filename 2024-03-29T04:44:50.611 INFO:tasks.workunit.client.0.smithi142.stdout:6/0: creat f0 x:0 0 0 2024-03-29T04:44:50.612 INFO:tasks.workunit.client.0.smithi142.stdout:4/9: dwrite d0/f1 [0,4194304] 0 2024-03-29T04:44:50.612 INFO:tasks.workunit.client.0.smithi142.stdout:6/1: truncate f0 989670 0 2024-03-29T04:44:50.613 INFO:tasks.workunit.client.0.smithi142.stdout:4/10: chown d0/f1 119901192 1 2024-03-29T04:44:50.615 INFO:tasks.workunit.client.0.smithi142.stdout:6/2: dread f0 [0,4194304] 0 2024-03-29T04:44:50.616 INFO:tasks.workunit.client.0.smithi142.stdout:9/0: stat - no entries 2024-03-29T04:44:50.616 INFO:tasks.workunit.client.0.smithi142.stdout:9/1: truncate - no filename 2024-03-29T04:44:50.616 INFO:tasks.workunit.client.0.smithi142.stdout:9/2: dread - no filename 2024-03-29T04:44:50.616 INFO:tasks.workunit.client.0.smithi142.stdout:9/3: chown . 242 1 2024-03-29T04:44:50.617 INFO:tasks.workunit.client.0.smithi142.stdout:9/4: write - no filename 2024-03-29T04:44:50.617 INFO:tasks.workunit.client.0.smithi142.stdout:9/5: stat - no entries 2024-03-29T04:44:50.617 INFO:tasks.workunit.client.0.smithi142.stdout:7/0: symlink l0 0 2024-03-29T04:44:50.617 INFO:tasks.workunit.client.0.smithi142.stdout:7/1: dwrite - no filename 2024-03-29T04:44:50.617 INFO:tasks.workunit.client.0.smithi142.stdout:7/2: truncate - no filename 2024-03-29T04:44:50.618 INFO:tasks.workunit.client.0.smithi142.stdout:0/13: fsync f2 0 2024-03-29T04:44:50.622 INFO:tasks.workunit.client.0.smithi142.stdout:5/13: getdents . 0 2024-03-29T04:44:50.622 INFO:tasks.workunit.client.0.smithi142.stdout:5/14: truncate - no filename 2024-03-29T04:44:50.622 INFO:tasks.workunit.client.0.smithi142.stdout:5/15: dread - no filename 2024-03-29T04:44:50.622 INFO:tasks.workunit.client.0.smithi142.stdout:5/16: unlink - no file 2024-03-29T04:44:50.622 INFO:tasks.workunit.client.0.smithi142.stdout:5/17: dwrite - no filename 2024-03-29T04:44:50.622 INFO:tasks.workunit.client.0.smithi142.stdout:5/18: dwrite - no filename 2024-03-29T04:44:50.622 INFO:tasks.workunit.client.0.smithi142.stdout:5/19: fdatasync - no filename 2024-03-29T04:44:50.622 INFO:tasks.workunit.client.0.smithi142.stdout:5/20: truncate - no filename 2024-03-29T04:44:50.622 INFO:tasks.workunit.client.0.smithi142.stdout:5/21: rename - no filename 2024-03-29T04:44:50.622 INFO:tasks.workunit.client.0.smithi142.stdout:5/22: dwrite - no filename 2024-03-29T04:44:50.623 INFO:tasks.workunit.client.0.smithi142.stdout:6/3: dwrite f0 [0,4194304] 0 2024-03-29T04:44:50.624 INFO:tasks.workunit.client.0.smithi142.stdout:4/11: creat d0/f3 x:0 0 0 2024-03-29T04:44:50.624 INFO:tasks.workunit.client.0.smithi142.stdout:4/12: rename d0 to d0/d4 22 2024-03-29T04:44:50.625 INFO:tasks.workunit.client.0.smithi142.stdout:3/8: creat d3/f4 x:0 0 0 2024-03-29T04:44:50.628 INFO:tasks.workunit.client.0.smithi142.stdout:7/3: creat f1 x:0 0 0 2024-03-29T04:44:50.629 INFO:tasks.workunit.client.0.smithi142.stdout:0/14: creat f4 x:0 0 0 2024-03-29T04:44:50.632 INFO:tasks.workunit.client.0.smithi142.stdout:1/7: link c1 c2 0 2024-03-29T04:44:50.632 INFO:tasks.workunit.client.0.smithi142.stdout:1/8: dwrite - no filename 2024-03-29T04:44:50.632 INFO:tasks.workunit.client.0.smithi142.stdout:1/9: dread - no filename 2024-03-29T04:44:50.632 INFO:tasks.workunit.client.0.smithi142.stdout:8/1: creat f0 x:0 0 0 2024-03-29T04:44:50.632 INFO:tasks.workunit.client.0.smithi142.stdout:8/2: dread - f0 zero size 2024-03-29T04:44:50.632 INFO:tasks.workunit.client.0.smithi142.stdout:8/3: dread - f0 zero size 2024-03-29T04:44:50.633 INFO:tasks.workunit.client.0.smithi142.stdout:8/4: write f0 [291250,17022] 0 2024-03-29T04:44:50.634 INFO:tasks.workunit.client.0.smithi142.stdout:4/13: rename d0/f1 to d0/f5 0 2024-03-29T04:44:50.635 INFO:tasks.workunit.client.0.smithi142.stdout:9/6: mkdir d0 0 2024-03-29T04:44:50.635 INFO:tasks.workunit.client.0.smithi142.stdout:9/7: dwrite - no filename 2024-03-29T04:44:50.635 INFO:tasks.workunit.client.0.smithi142.stdout:9/8: dwrite - no filename 2024-03-29T04:44:50.635 INFO:tasks.workunit.client.0.smithi142.stdout:9/9: dwrite - no filename 2024-03-29T04:44:50.635 INFO:tasks.workunit.client.0.smithi142.stdout:9/10: write - no filename 2024-03-29T04:44:50.636 INFO:tasks.workunit.client.0.smithi142.stdout:3/9: mkdir d3/d5 0 2024-03-29T04:44:50.637 INFO:tasks.workunit.client.0.smithi142.stdout:0/15: mknod c5 0 2024-03-29T04:44:50.638 INFO:tasks.workunit.client.0.smithi142.stdout:5/23: symlink l0 0 2024-03-29T04:44:50.638 INFO:tasks.workunit.client.0.smithi142.stdout:5/24: write - no filename 2024-03-29T04:44:50.639 INFO:tasks.workunit.client.0.smithi142.stdout:5/25: stat l0 0 2024-03-29T04:44:50.639 INFO:tasks.workunit.client.0.smithi142.stdout:4/14: write d0/f5 [3996965,108530] 0 2024-03-29T04:44:50.640 INFO:tasks.workunit.client.0.smithi142.stdout:7/4: link f1 f2 0 2024-03-29T04:44:50.641 INFO:tasks.workunit.client.0.smithi142.stdout:0/16: mknod c6 0 2024-03-29T04:44:50.641 INFO:tasks.workunit.client.0.smithi142.stdout:1/10: chown c2 2679 1 2024-03-29T04:44:50.641 INFO:tasks.workunit.client.0.smithi142.stdout:0/17: dread - f4 zero size 2024-03-29T04:44:50.641 INFO:tasks.workunit.client.0.smithi142.stdout:1/11: fdatasync - no filename 2024-03-29T04:44:50.641 INFO:tasks.workunit.client.0.smithi142.stdout:1/12: dread - no filename 2024-03-29T04:44:50.642 INFO:tasks.workunit.client.0.smithi142.stdout:1/13: dwrite - no filename 2024-03-29T04:44:50.642 INFO:tasks.workunit.client.0.smithi142.stdout:0/18: write f4 [113989,30185] 0 2024-03-29T04:44:50.642 INFO:tasks.workunit.client.0.smithi142.stdout:9/11: creat d0/f1 x:0 0 0 2024-03-29T04:44:50.642 INFO:tasks.workunit.client.0.smithi142.stdout:9/12: readlink - no filename 2024-03-29T04:44:50.644 INFO:tasks.workunit.client.0.smithi142.stdout:4/15: mknod d0/c6 0 2024-03-29T04:44:50.644 INFO:tasks.workunit.client.0.smithi142.stdout:9/13: read - d0/f1 zero size 2024-03-29T04:44:50.644 INFO:tasks.workunit.client.0.smithi142.stdout:7/5: fdatasync f1 0 2024-03-29T04:44:50.644 INFO:tasks.workunit.client.0.smithi142.stdout:7/6: readlink l0 0 2024-03-29T04:44:50.644 INFO:tasks.workunit.client.0.smithi142.stdout:7/7: rmdir - no directory 2024-03-29T04:44:50.644 INFO:tasks.workunit.client.0.smithi142.stdout:1/14: creat f3 x:0 0 0 2024-03-29T04:44:50.645 INFO:tasks.workunit.client.0.smithi142.stdout:0/19: mkdir d7 0 2024-03-29T04:44:50.647 INFO:tasks.workunit.client.0.smithi142.stdout:0/20: read f4 [129805,31756] 0 2024-03-29T04:44:50.649 INFO:tasks.workunit.client.0.smithi142.stdout:1/15: rename c1 to c4 0 2024-03-29T04:44:50.650 INFO:tasks.workunit.client.0.smithi142.stdout:0/21: unlink c5 0 2024-03-29T04:44:50.651 INFO:tasks.workunit.client.0.smithi142.stdout:1/16: creat f5 x:0 0 0 2024-03-29T04:44:50.651 INFO:tasks.workunit.client.0.smithi142.stdout:0/22: stat f2 0 2024-03-29T04:44:50.652 INFO:tasks.workunit.client.0.smithi142.stdout:1/17: dread - f5 zero size 2024-03-29T04:44:50.654 INFO:tasks.workunit.client.0.smithi142.stdout:1/18: truncate f3 71363 0 2024-03-29T04:44:50.654 INFO:tasks.workunit.client.0.smithi142.stdout:4/16: dwrite d0/f3 [0,4194304] 0 2024-03-29T04:44:50.655 INFO:tasks.workunit.client.0.smithi142.stdout:9/14: dwrite d0/f1 [0,4194304] 0 2024-03-29T04:44:50.657 INFO:tasks.workunit.client.0.smithi142.stdout:1/19: creat f6 x:0 0 0 2024-03-29T04:44:50.657 INFO:tasks.workunit.client.0.smithi142.stdout:1/20: fdatasync f5 0 2024-03-29T04:44:50.657 INFO:tasks.workunit.client.0.smithi142.stdout:1/21: dread - f6 zero size 2024-03-29T04:44:50.658 INFO:tasks.workunit.client.0.smithi142.stdout:4/17: creat d0/f7 x:0 0 0 2024-03-29T04:44:50.660 INFO:tasks.workunit.client.0.smithi142.stdout:1/22: creat f7 x:0 0 0 2024-03-29T04:44:50.661 INFO:tasks.workunit.client.0.smithi142.stdout:4/18: dread d0/f5 [0,4194304] 0 2024-03-29T04:44:50.662 INFO:tasks.workunit.client.0.smithi142.stdout:4/19: chown d0/c6 2092 1 2024-03-29T04:44:50.662 INFO:tasks.workunit.client.0.smithi142.stdout:1/23: mkdir d8 0 2024-03-29T04:44:50.664 INFO:tasks.workunit.client.0.smithi142.stdout:8/5: read f0 [252232,114165] 0 2024-03-29T04:44:50.666 INFO:tasks.workunit.client.0.smithi142.stdout:1/24: rename f6 to d8/f9 0 2024-03-29T04:44:50.667 INFO:tasks.workunit.client.0.smithi142.stdout:1/25: chown f5 11 1 2024-03-29T04:44:50.688 INFO:tasks.workunit.client.0.smithi142.stdout:2/12: stat l2 0 2024-03-29T04:44:50.693 INFO:tasks.workunit.client.0.smithi142.stdout:8/6: dwrite f0 [0,4194304] 0 2024-03-29T04:44:50.700 INFO:tasks.workunit.client.0.smithi142.stdout:5/26: getdents . 0 2024-03-29T04:44:50.717 INFO:tasks.workunit.client.0.smithi142.stdout:9/15: truncate d0/f1 1803614 0 2024-03-29T04:44:50.932 INFO:tasks.workunit.client.0.smithi142.stdout:2/13: creat f3 x:0 0 0 2024-03-29T04:44:50.936 INFO:tasks.workunit.client.0.smithi142.stdout:8/7: symlink l1 0 2024-03-29T04:44:50.937 INFO:tasks.workunit.client.0.smithi142.stdout:8/8: write f0 [2777508,59875] 0 2024-03-29T04:44:50.938 INFO:tasks.workunit.client.0.smithi142.stdout:5/27: rename l0 to l1 0 2024-03-29T04:44:50.938 INFO:tasks.workunit.client.0.smithi142.stdout:5/28: rmdir - no directory 2024-03-29T04:44:50.939 INFO:tasks.workunit.client.0.smithi142.stdout:9/16: mkdir d0/d2 0 2024-03-29T04:44:50.941 INFO:tasks.workunit.client.0.smithi142.stdout:8/9: dread f0 [0,4194304] 0 2024-03-29T04:44:50.942 INFO:tasks.workunit.client.0.smithi142.stdout:5/29: mknod c2 0 2024-03-29T04:44:50.943 INFO:tasks.workunit.client.0.smithi142.stdout:9/17: write d0/f1 [1233863,127950] 0 2024-03-29T04:44:50.944 INFO:tasks.workunit.client.0.smithi142.stdout:8/10: fsync f0 0 2024-03-29T04:44:50.945 INFO:tasks.workunit.client.0.smithi142.stdout:5/30: unlink c2 0 2024-03-29T04:44:50.945 INFO:tasks.workunit.client.0.smithi142.stdout:5/31: dread - no filename 2024-03-29T04:44:50.945 INFO:tasks.workunit.client.0.smithi142.stdout:5/32: dwrite - no filename 2024-03-29T04:44:50.946 INFO:tasks.workunit.client.0.smithi142.stdout:9/18: mknod d0/c3 0 2024-03-29T04:44:50.947 INFO:tasks.workunit.client.0.smithi142.stdout:8/11: mknod c2 0 2024-03-29T04:44:50.948 INFO:tasks.workunit.client.0.smithi142.stdout:5/33: creat f3 x:0 0 0 2024-03-29T04:44:50.949 INFO:tasks.workunit.client.0.smithi142.stdout:5/34: dread - f3 zero size 2024-03-29T04:44:50.949 INFO:tasks.workunit.client.0.smithi142.stdout:9/19: mknod d0/d2/c4 0 2024-03-29T04:44:50.951 INFO:tasks.workunit.client.0.smithi142.stdout:8/12: dread f0 [0,4194304] 0 2024-03-29T04:44:50.953 INFO:tasks.workunit.client.0.smithi142.stdout:9/20: dread d0/f1 [0,4194304] 0 2024-03-29T04:44:50.953 INFO:tasks.workunit.client.0.smithi142.stdout:8/13: symlink l3 0 2024-03-29T04:44:50.954 INFO:tasks.workunit.client.0.smithi142.stdout:8/14: chown f0 2411 1 2024-03-29T04:44:50.956 INFO:tasks.workunit.client.0.smithi142.stdout:8/15: symlink l4 0 2024-03-29T04:44:50.956 INFO:tasks.workunit.client.0.smithi142.stdout:5/35: dwrite f3 [0,4194304] 0 2024-03-29T04:44:50.957 INFO:tasks.workunit.client.0.smithi142.stdout:9/21: dread d0/f1 [0,4194304] 0 2024-03-29T04:44:50.958 INFO:tasks.workunit.client.0.smithi142.stdout:8/16: mkdir d5 0 2024-03-29T04:44:50.961 INFO:tasks.workunit.client.0.smithi142.stdout:9/22: mknod d0/d2/c5 0 2024-03-29T04:44:50.961 INFO:tasks.workunit.client.0.smithi142.stdout:9/23: chown d0 10173395 1 2024-03-29T04:44:50.963 INFO:tasks.workunit.client.0.smithi142.stdout:8/17: dread f0 [0,4194304] 0 2024-03-29T04:44:50.963 INFO:tasks.workunit.client.0.smithi142.stdout:9/24: mkdir d0/d2/d6 0 2024-03-29T04:44:50.965 INFO:tasks.workunit.client.0.smithi142.stdout:9/25: symlink d0/d2/l7 0 2024-03-29T04:44:50.966 INFO:tasks.workunit.client.0.smithi142.stdout:8/18: symlink d5/l6 0 2024-03-29T04:44:50.968 INFO:tasks.workunit.client.0.smithi142.stdout:8/19: unlink l1 0 2024-03-29T04:44:50.969 INFO:tasks.workunit.client.0.smithi142.stdout:9/26: link d0/d2/c5 d0/c8 0 2024-03-29T04:44:50.970 INFO:tasks.workunit.client.0.smithi142.stdout:9/27: rename d0/d2 to d0/d2/d9 22 2024-03-29T04:44:50.971 INFO:tasks.workunit.client.0.smithi142.stdout:8/20: dread f0 [0,4194304] 0 2024-03-29T04:44:50.972 INFO:tasks.workunit.client.0.smithi142.stdout:8/21: write f0 [954209,73051] 0 2024-03-29T04:44:50.972 INFO:tasks.workunit.client.0.smithi142.stdout:9/28: dread d0/f1 [0,4194304] 0 2024-03-29T04:44:50.974 INFO:tasks.workunit.client.0.smithi142.stdout:8/22: creat d5/f7 x:0 0 0 2024-03-29T04:44:50.974 INFO:tasks.workunit.client.0.smithi142.stdout:9/29: fsync d0/f1 0 2024-03-29T04:44:50.974 INFO:tasks.workunit.client.0.smithi142.stdout:9/30: fdatasync d0/f1 0 2024-03-29T04:44:50.975 INFO:tasks.workunit.client.0.smithi142.stdout:8/23: symlink d5/l8 0 2024-03-29T04:44:50.975 INFO:tasks.workunit.client.0.smithi142.stdout:9/31: chown d0/d2/l7 17083 1 2024-03-29T04:44:50.981 INFO:tasks.workunit.client.0.smithi142.stdout:8/24: dwrite d5/f7 [0,4194304] 0 2024-03-29T04:44:50.982 INFO:tasks.workunit.client.0.smithi142.stdout:8/25: stat l3 0 2024-03-29T04:44:50.996 INFO:tasks.workunit.client.0.smithi142.stdout:5/36: fdatasync f3 0 2024-03-29T04:44:50.996 INFO:tasks.workunit.client.0.smithi142.stdout:5/37: write f3 [1106773,73884] 0 2024-03-29T04:44:50.997 INFO:tasks.workunit.client.0.smithi142.stdout:5/38: chown f3 756 1 2024-03-29T04:44:50.998 INFO:tasks.workunit.client.0.smithi142.stdout:5/39: mknod c4 0 2024-03-29T04:44:51.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:50 smithi152 ceph-mon[37177]: pgmap v261: 65 pgs: 65 active+clean; 249 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 350 op/s 2024-03-29T04:44:51.040 INFO:tasks.workunit.client.0.smithi142.stdout:5/40: rename l1 to l5 0 2024-03-29T04:44:51.043 INFO:tasks.workunit.client.0.smithi142.stdout:5/41: dread f3 [0,4194304] 0 2024-03-29T04:44:51.045 INFO:tasks.workunit.client.0.smithi142.stdout:5/42: creat f6 x:0 0 0 2024-03-29T04:44:51.048 INFO:tasks.workunit.client.0.smithi142.stdout:8/26: write d5/f7 [5222639,16777] 0 2024-03-29T04:44:51.050 INFO:tasks.workunit.client.0.smithi142.stdout:8/27: mknod d5/c9 0 2024-03-29T04:44:51.052 INFO:tasks.workunit.client.0.smithi142.stdout:8/28: unlink c2 0 2024-03-29T04:44:51.052 INFO:tasks.workunit.client.0.smithi142.stdout:5/43: truncate f3 4110485 0 2024-03-29T04:44:51.053 INFO:tasks.workunit.client.0.smithi142.stdout:5/44: unlink c4 0 2024-03-29T04:44:51.054 INFO:tasks.workunit.client.0.smithi142.stdout:5/45: write f6 [464741,113726] 0 2024-03-29T04:44:51.059 INFO:tasks.workunit.client.0.smithi142.stdout:8/29: truncate f0 1380216 0 2024-03-29T04:44:51.060 INFO:tasks.workunit.client.0.smithi142.stdout:5/46: truncate f6 332731 0 2024-03-29T04:44:51.062 INFO:tasks.workunit.client.0.smithi142.stdout:5/47: creat f7 x:0 0 0 2024-03-29T04:44:51.062 INFO:tasks.workunit.client.0.smithi142.stdout:8/30: fdatasync f0 0 2024-03-29T04:44:51.062 INFO:tasks.workunit.client.0.smithi142.stdout:5/48: chown f7 648416796 1 2024-03-29T04:44:51.065 INFO:tasks.workunit.client.0.smithi142.stdout:5/49: mknod c8 0 2024-03-29T04:44:51.068 INFO:tasks.workunit.client.0.smithi142.stdout:5/50: unlink f6 0 2024-03-29T04:44:51.069 INFO:tasks.workunit.client.0.smithi142.stdout:5/51: symlink l9 0 2024-03-29T04:44:51.071 INFO:tasks.workunit.client.0.smithi142.stdout:5/52: creat fa x:0 0 0 2024-03-29T04:44:51.071 INFO:tasks.workunit.client.0.smithi142.stdout:5/53: dread - fa zero size 2024-03-29T04:44:51.072 INFO:tasks.workunit.client.0.smithi142.stdout:5/54: stat f3 0 2024-03-29T04:44:51.074 INFO:tasks.workunit.client.0.smithi142.stdout:5/55: truncate f3 4842525 0 2024-03-29T04:44:51.075 INFO:tasks.workunit.client.0.smithi142.stdout:5/56: creat fb x:0 0 0 2024-03-29T04:44:51.076 INFO:tasks.workunit.client.0.smithi142.stdout:8/31: dwrite f0 [0,4194304] 0 2024-03-29T04:44:51.077 INFO:tasks.workunit.client.0.smithi142.stdout:8/32: stat d5/f7 0 2024-03-29T04:44:51.077 INFO:tasks.workunit.client.0.smithi142.stdout:8/33: readlink l3 0 2024-03-29T04:44:51.077 INFO:tasks.workunit.client.0.smithi142.stdout:5/57: mknod cc 0 2024-03-29T04:44:51.077 INFO:tasks.workunit.client.0.smithi142.stdout:5/58: chown c8 8203 1 2024-03-29T04:44:51.078 INFO:tasks.workunit.client.0.smithi142.stdout:8/34: unlink f0 0 2024-03-29T04:44:51.088 INFO:tasks.workunit.client.0.smithi142.stdout:5/59: stat f3 0 2024-03-29T04:44:51.099 INFO:tasks.workunit.client.0.smithi142.stdout:5/60: fdatasync f3 0 2024-03-29T04:44:51.099 INFO:tasks.workunit.client.0.smithi142.stdout:5/61: rmdir - no directory 2024-03-29T04:44:51.101 INFO:tasks.workunit.client.0.smithi142.stdout:5/62: readlink l5 0 2024-03-29T04:44:51.105 INFO:tasks.workunit.client.0.smithi142.stdout:5/63: creat fd x:0 0 0 2024-03-29T04:44:51.105 INFO:tasks.workunit.client.0.smithi142.stdout:5/64: write f7 [928367,57389] 0 2024-03-29T04:44:51.105 INFO:tasks.workunit.client.0.smithi142.stdout:5/65: readlink l5 0 2024-03-29T04:44:51.105 INFO:tasks.workunit.client.0.smithi142.stdout:5/66: stat c8 0 2024-03-29T04:44:51.105 INFO:tasks.workunit.client.0.smithi142.stdout:5/67: rmdir - no directory 2024-03-29T04:44:51.105 INFO:tasks.workunit.client.0.smithi142.stdout:5/68: chown fd 533099678 1 2024-03-29T04:44:51.105 INFO:tasks.workunit.client.0.smithi142.stdout:5/69: stat l5 0 2024-03-29T04:44:51.105 INFO:tasks.workunit.client.0.smithi142.stdout:8/35: truncate d5/f7 1443892 0 2024-03-29T04:44:51.105 INFO:tasks.workunit.client.0.smithi142.stdout:5/70: dread - fb zero size 2024-03-29T04:44:51.106 INFO:tasks.workunit.client.0.smithi142.stdout:5/71: fdatasync fa 0 2024-03-29T04:44:51.109 INFO:tasks.workunit.client.0.smithi142.stdout:8/36: dread d5/f7 [0,4194304] 0 2024-03-29T04:44:51.110 INFO:tasks.workunit.client.0.smithi142.stdout:8/37: symlink d5/la 0 2024-03-29T04:44:51.115 INFO:tasks.workunit.client.0.smithi142.stdout:8/38: dwrite d5/f7 [0,4194304] 0 2024-03-29T04:44:51.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:50 smithi142 ceph-mon[29027]: pgmap v261: 65 pgs: 65 active+clean; 249 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 350 op/s 2024-03-29T04:44:51.122 INFO:tasks.workunit.client.0.smithi142.stdout:8/39: dwrite d5/f7 [0,4194304] 0 2024-03-29T04:44:51.124 INFO:tasks.workunit.client.0.smithi142.stdout:8/40: mknod d5/cb 0 2024-03-29T04:44:51.124 INFO:tasks.workunit.client.0.smithi142.stdout:8/41: write d5/f7 [619070,44711] 0 2024-03-29T04:44:51.125 INFO:tasks.workunit.client.0.smithi142.stdout:8/42: write d5/f7 [868394,50273] 0 2024-03-29T04:44:51.126 INFO:tasks.workunit.client.0.smithi142.stdout:8/43: chown d5 18231656 1 2024-03-29T04:44:51.131 INFO:tasks.workunit.client.0.smithi142.stdout:8/44: dread d5/f7 [0,4194304] 0 2024-03-29T04:44:51.137 INFO:tasks.workunit.client.0.smithi142.stdout:8/45: dwrite d5/f7 [0,4194304] 0 2024-03-29T04:44:51.138 INFO:tasks.workunit.client.0.smithi142.stdout:8/46: write d5/f7 [1361022,35247] 0 2024-03-29T04:44:51.138 INFO:tasks.workunit.client.0.smithi142.stdout:8/47: read d5/f7 [615806,84770] 0 2024-03-29T04:44:51.140 INFO:tasks.workunit.client.0.smithi142.stdout:8/48: mkdir d5/dc 0 2024-03-29T04:44:51.142 INFO:tasks.workunit.client.0.smithi142.stdout:8/49: rename d5/dc to d5/dd 0 2024-03-29T04:44:51.145 INFO:tasks.workunit.client.0.smithi142.stdout:8/50: dread d5/f7 [0,4194304] 0 2024-03-29T04:44:51.146 INFO:tasks.workunit.client.0.smithi142.stdout:8/51: rename d5/dd to d5/dd/de 22 2024-03-29T04:44:51.148 INFO:tasks.workunit.client.0.smithi142.stdout:8/52: rename d5/la to d5/lf 0 2024-03-29T04:44:51.204 INFO:tasks.workunit.client.0.smithi142.stdout:8/53: rmdir d5/dd 0 2024-03-29T04:44:51.206 INFO:tasks.workunit.client.0.smithi142.stdout:8/54: mkdir d5/d10 0 2024-03-29T04:44:51.211 INFO:tasks.workunit.client.0.smithi142.stdout:8/55: dwrite d5/f7 [0,4194304] 0 2024-03-29T04:44:51.213 INFO:tasks.workunit.client.0.smithi142.stdout:8/56: mknod d5/d10/c11 0 2024-03-29T04:44:51.215 INFO:tasks.workunit.client.0.smithi142.stdout:8/57: creat d5/d10/f12 x:0 0 0 2024-03-29T04:44:51.217 INFO:tasks.workunit.client.0.smithi142.stdout:8/58: mknod d5/c13 0 2024-03-29T04:44:51.222 INFO:tasks.workunit.client.0.smithi142.stdout:8/59: rmdir d5/d10 39 2024-03-29T04:44:51.225 INFO:tasks.workunit.client.0.smithi142.stdout:8/60: link l3 d5/d10/l14 0 2024-03-29T04:44:51.230 INFO:tasks.workunit.client.0.smithi142.stdout:8/61: dwrite d5/d10/f12 [0,4194304] 0 2024-03-29T04:44:51.231 INFO:tasks.workunit.client.0.smithi142.stdout:8/62: chown d5 198 1 2024-03-29T04:44:51.232 INFO:tasks.workunit.client.0.smithi142.stdout:8/63: mknod d5/d10/c15 0 2024-03-29T04:44:51.234 INFO:tasks.workunit.client.0.smithi142.stdout:8/64: creat d5/d10/f16 x:0 0 0 2024-03-29T04:44:51.235 INFO:tasks.workunit.client.0.smithi142.stdout:8/65: write d5/d10/f16 [366913,61383] 0 2024-03-29T04:44:51.245 INFO:tasks.workunit.client.0.smithi142.stdout:8/66: truncate d5/d10/f12 516211 0 2024-03-29T04:44:51.247 INFO:tasks.workunit.client.0.smithi142.stdout:8/67: symlink d5/l17 0 2024-03-29T04:44:51.251 INFO:tasks.workunit.client.0.smithi142.stdout:8/68: dwrite d5/d10/f16 [0,4194304] 0 2024-03-29T04:44:51.259 INFO:tasks.workunit.client.0.smithi142.stdout:8/69: dwrite d5/d10/f16 [0,4194304] 0 2024-03-29T04:44:51.262 INFO:tasks.workunit.client.0.smithi142.stdout:8/70: rmdir d5/d10 39 2024-03-29T04:44:51.271 INFO:tasks.workunit.client.0.smithi142.stdout:8/71: dwrite d5/d10/f12 [0,4194304] 0 2024-03-29T04:44:51.272 INFO:tasks.workunit.client.0.smithi142.stdout:8/72: chown d5 100959082 1 2024-03-29T04:44:52.295 INFO:tasks.workunit.client.0.smithi142.stdout:9/32: sync 2024-03-29T04:44:52.295 INFO:tasks.workunit.client.0.smithi142.stdout:1/26: sync 2024-03-29T04:44:52.295 INFO:tasks.workunit.client.0.smithi142.stdout:3/10: sync 2024-03-29T04:44:52.296 INFO:tasks.workunit.client.0.smithi142.stdout:6/4: sync 2024-03-29T04:44:52.296 INFO:tasks.workunit.client.0.smithi142.stdout:2/14: sync 2024-03-29T04:44:52.296 INFO:tasks.workunit.client.0.smithi142.stdout:7/8: sync 2024-03-29T04:44:52.296 INFO:tasks.workunit.client.0.smithi142.stdout:4/20: sync 2024-03-29T04:44:52.296 INFO:tasks.workunit.client.0.smithi142.stdout:0/23: sync 2024-03-29T04:44:52.296 INFO:tasks.workunit.client.0.smithi142.stdout:4/21: chown d0 15 1 2024-03-29T04:44:52.297 INFO:tasks.workunit.client.0.smithi142.stdout:2/15: write f3 [858021,34652] 0 2024-03-29T04:44:52.297 INFO:tasks.workunit.client.0.smithi142.stdout:1/27: chown f5 2 1 2024-03-29T04:44:52.297 INFO:tasks.workunit.client.0.smithi142.stdout:2/16: stat l2 0 2024-03-29T04:44:52.298 INFO:tasks.workunit.client.0.smithi142.stdout:7/9: creat f3 x:0 0 0 2024-03-29T04:44:52.298 INFO:tasks.workunit.client.0.smithi142.stdout:1/28: dread - f5 zero size 2024-03-29T04:44:52.299 INFO:tasks.workunit.client.0.smithi142.stdout:7/10: dread - f3 zero size 2024-03-29T04:44:52.300 INFO:tasks.workunit.client.0.smithi142.stdout:7/11: dread - f3 zero size 2024-03-29T04:44:52.300 INFO:tasks.workunit.client.0.smithi142.stdout:6/5: rename f0 to f1 0 2024-03-29T04:44:52.300 INFO:tasks.workunit.client.0.smithi142.stdout:9/33: dread d0/f1 [0,4194304] 0 2024-03-29T04:44:52.304 INFO:tasks.workunit.client.0.smithi142.stdout:2/17: rename l1 to l4 0 2024-03-29T04:44:52.305 INFO:tasks.workunit.client.0.smithi142.stdout:2/18: stat f3 0 2024-03-29T04:44:52.306 INFO:tasks.workunit.client.0.smithi142.stdout:6/6: dread f1 [0,4194304] 0 2024-03-29T04:44:52.306 INFO:tasks.workunit.client.0.smithi142.stdout:3/11: mkdir d3/d5/d6 0 2024-03-29T04:44:52.308 INFO:tasks.workunit.client.0.smithi142.stdout:4/22: dwrite d0/f5 [0,4194304] 0 2024-03-29T04:44:52.310 INFO:tasks.workunit.client.0.smithi142.stdout:7/12: rename l0 to l4 0 2024-03-29T04:44:52.312 INFO:tasks.workunit.client.0.smithi142.stdout:1/29: creat d8/fa x:0 0 0 2024-03-29T04:44:52.315 INFO:tasks.workunit.client.0.smithi142.stdout:1/30: chown d8/f9 184369047 1 2024-03-29T04:44:52.316 INFO:tasks.workunit.client.0.smithi142.stdout:7/13: symlink l5 0 2024-03-29T04:44:52.317 INFO:tasks.workunit.client.0.smithi142.stdout:6/7: dread f1 [0,4194304] 0 2024-03-29T04:44:52.317 INFO:tasks.workunit.client.0.smithi142.stdout:7/14: fdatasync f2 0 2024-03-29T04:44:52.319 INFO:tasks.workunit.client.0.smithi142.stdout:7/15: fdatasync f1 0 2024-03-29T04:44:52.320 INFO:tasks.workunit.client.0.smithi142.stdout:9/34: link d0/d2/c5 d0/d2/d6/ca 0 2024-03-29T04:44:52.321 INFO:tasks.workunit.client.0.smithi142.stdout:3/12: dwrite d3/f4 [0,4194304] 0 2024-03-29T04:44:52.321 INFO:tasks.workunit.client.0.smithi142.stdout:6/8: chown f1 58286664 1 2024-03-29T04:44:52.322 INFO:tasks.workunit.client.0.smithi142.stdout:7/16: write f3 [785166,69693] 0 2024-03-29T04:44:52.322 INFO:tasks.workunit.client.0.smithi142.stdout:9/35: write d0/f1 [438723,98973] 0 2024-03-29T04:44:52.324 INFO:tasks.workunit.client.0.smithi142.stdout:9/36: stat d0/c3 0 2024-03-29T04:44:52.324 INFO:tasks.workunit.client.0.smithi142.stdout:6/9: symlink l2 0 2024-03-29T04:44:52.325 INFO:tasks.workunit.client.0.smithi142.stdout:4/23: dwrite d0/f5 [0,4194304] 0 2024-03-29T04:44:52.325 INFO:tasks.workunit.client.0.smithi142.stdout:6/10: stat f1 0 2024-03-29T04:44:52.329 INFO:tasks.workunit.client.0.smithi142.stdout:4/24: dread - d0/f7 zero size 2024-03-29T04:44:52.331 INFO:tasks.workunit.client.0.smithi142.stdout:6/11: creat f3 x:0 0 0 2024-03-29T04:44:52.335 INFO:tasks.workunit.client.0.smithi142.stdout:4/25: rename d0/f5 to d0/f8 0 2024-03-29T04:44:52.335 INFO:tasks.workunit.client.0.smithi142.stdout:6/12: truncate f3 867815 0 2024-03-29T04:44:52.336 INFO:tasks.workunit.client.0.smithi142.stdout:3/13: dread d3/f4 [0,4194304] 0 2024-03-29T04:44:52.337 INFO:tasks.workunit.client.0.smithi142.stdout:3/14: readlink l0 0 2024-03-29T04:44:52.338 INFO:tasks.workunit.client.0.smithi142.stdout:4/26: mkdir d0/d9 0 2024-03-29T04:44:52.340 INFO:tasks.workunit.client.0.smithi142.stdout:9/37: dwrite d0/f1 [0,4194304] 0 2024-03-29T04:44:52.341 INFO:tasks.workunit.client.0.smithi142.stdout:4/27: creat d0/fa x:0 0 0 2024-03-29T04:44:52.343 INFO:tasks.workunit.client.0.smithi142.stdout:6/13: dread f1 [0,4194304] 0 2024-03-29T04:44:52.344 INFO:tasks.workunit.client.0.smithi142.stdout:6/14: read f3 [160127,59957] 0 2024-03-29T04:44:52.345 INFO:tasks.workunit.client.0.smithi142.stdout:9/38: dread d0/f1 [0,4194304] 0 2024-03-29T04:44:52.345 INFO:tasks.workunit.client.0.smithi142.stdout:6/15: read f3 [846474,107370] 0 2024-03-29T04:44:52.350 INFO:tasks.workunit.client.0.smithi142.stdout:9/39: rename d0/d2/l7 to d0/lb 0 2024-03-29T04:44:52.351 INFO:tasks.workunit.client.0.smithi142.stdout:5/72: sync 2024-03-29T04:44:52.351 INFO:tasks.workunit.client.0.smithi142.stdout:8/73: sync 2024-03-29T04:44:52.352 INFO:tasks.workunit.client.0.smithi142.stdout:5/73: dread - fa zero size 2024-03-29T04:44:52.355 INFO:tasks.workunit.client.0.smithi142.stdout:6/16: dwrite f3 [0,4194304] 0 2024-03-29T04:44:52.355 INFO:tasks.workunit.client.0.smithi142.stdout:5/74: creat fe x:0 0 0 2024-03-29T04:44:52.358 INFO:tasks.workunit.client.0.smithi142.stdout:5/75: mkdir df 0 2024-03-29T04:44:52.359 INFO:tasks.workunit.client.0.smithi142.stdout:5/76: read f3 [3188533,10199] 0 2024-03-29T04:44:52.360 INFO:tasks.workunit.client.0.smithi142.stdout:5/77: truncate fe 953161 0 2024-03-29T04:44:52.360 INFO:tasks.workunit.client.0.smithi142.stdout:8/74: creat d5/f18 x:0 0 0 2024-03-29T04:44:52.361 INFO:tasks.workunit.client.0.smithi142.stdout:5/78: write fd [367181,99671] 0 2024-03-29T04:44:52.362 INFO:tasks.workunit.client.0.smithi142.stdout:5/79: dread - fb zero size 2024-03-29T04:44:52.365 INFO:tasks.workunit.client.0.smithi142.stdout:5/80: creat df/f10 x:0 0 0 2024-03-29T04:44:52.368 INFO:tasks.workunit.client.0.smithi142.stdout:8/75: dwrite d5/d10/f16 [0,4194304] 0 2024-03-29T04:44:52.373 INFO:tasks.workunit.client.0.smithi142.stdout:5/81: dwrite f7 [0,4194304] 0 2024-03-29T04:44:52.430 INFO:tasks.workunit.client.0.smithi142.stdout:0/24: sync 2024-03-29T04:44:52.434 INFO:tasks.workunit.client.0.smithi142.stdout:8/76: rename d5/f7 to d5/f19 0 2024-03-29T04:44:52.434 INFO:tasks.workunit.client.0.smithi142.stdout:5/82: mknod df/c11 0 2024-03-29T04:44:52.435 INFO:tasks.workunit.client.0.smithi142.stdout:5/83: write df/f10 [252660,117542] 0 2024-03-29T04:44:52.436 INFO:tasks.workunit.client.0.smithi142.stdout:0/25: rename l3 to d7/l8 0 2024-03-29T04:44:52.436 INFO:tasks.workunit.client.0.smithi142.stdout:5/84: write df/f10 [367658,82238] 0 2024-03-29T04:44:52.465 INFO:tasks.workunit.client.0.smithi142.stdout:1/31: sync 2024-03-29T04:44:52.465 INFO:tasks.workunit.client.0.smithi142.stdout:4/28: sync 2024-03-29T04:44:52.465 INFO:tasks.workunit.client.0.smithi142.stdout:7/17: sync 2024-03-29T04:44:52.532 INFO:tasks.workunit.client.0.smithi142.stdout:5/85: dread fe [0,4194304] 0 2024-03-29T04:44:52.534 INFO:tasks.workunit.client.0.smithi142.stdout:5/86: dread f3 [4194304,4194304] 0 2024-03-29T04:44:52.534 INFO:tasks.workunit.client.0.smithi142.stdout:7/18: read f3 [676589,93688] 0 2024-03-29T04:44:52.534 INFO:tasks.workunit.client.0.smithi142.stdout:5/87: write df/f10 [591964,109510] 0 2024-03-29T04:44:52.535 INFO:tasks.workunit.client.0.smithi142.stdout:7/19: chown f2 0 1 2024-03-29T04:44:52.535 INFO:tasks.workunit.client.0.smithi142.stdout:8/77: stat d5/d10/l14 0 2024-03-29T04:44:52.535 INFO:tasks.workunit.client.0.smithi142.stdout:7/20: dread - f2 zero size 2024-03-29T04:44:52.535 INFO:tasks.workunit.client.0.smithi142.stdout:5/88: chown f3 23086313 1 2024-03-29T04:44:52.536 INFO:tasks.workunit.client.0.smithi142.stdout:8/78: write d5/d10/f16 [1679267,49234] 0 2024-03-29T04:44:52.536 INFO:tasks.workunit.client.0.smithi142.stdout:5/89: chown df/f10 123 1 2024-03-29T04:44:52.538 INFO:tasks.workunit.client.0.smithi142.stdout:0/26: mkdir d7/d9 0 2024-03-29T04:44:52.539 INFO:tasks.workunit.client.0.smithi142.stdout:0/27: write f2 [721684,98965] 0 2024-03-29T04:44:52.541 INFO:tasks.workunit.client.0.smithi142.stdout:1/32: write f7 [180719,48656] 0 2024-03-29T04:44:52.544 INFO:tasks.workunit.client.0.smithi142.stdout:7/21: creat f6 x:0 0 0 2024-03-29T04:44:52.545 INFO:tasks.workunit.client.0.smithi142.stdout:5/90: symlink df/l12 0 2024-03-29T04:44:52.545 INFO:tasks.workunit.client.0.smithi142.stdout:5/91: stat df/c11 0 2024-03-29T04:44:52.548 INFO:tasks.workunit.client.0.smithi142.stdout:1/33: mkdir d8/db 0 2024-03-29T04:44:52.549 INFO:tasks.workunit.client.0.smithi142.stdout:4/29: creat d0/d9/fb x:0 0 0 2024-03-29T04:44:52.550 INFO:tasks.workunit.client.0.smithi142.stdout:7/22: mkdir d7 0 2024-03-29T04:44:52.551 INFO:tasks.workunit.client.0.smithi142.stdout:5/92: mknod df/c13 0 2024-03-29T04:44:52.553 INFO:tasks.workunit.client.0.smithi142.stdout:0/28: dwrite f4 [0,4194304] 0 2024-03-29T04:44:52.553 INFO:tasks.workunit.client.0.smithi142.stdout:5/93: write df/f10 [294570,8855] 0 2024-03-29T04:44:52.557 INFO:tasks.workunit.client.0.smithi142.stdout:8/79: link l4 d5/d10/l1a 0 2024-03-29T04:44:52.558 INFO:tasks.workunit.client.0.smithi142.stdout:1/34: mkdir d8/db/dc 0 2024-03-29T04:44:52.559 INFO:tasks.workunit.client.0.smithi142.stdout:1/35: stat d8/db/dc 0 2024-03-29T04:44:52.562 INFO:tasks.workunit.client.0.smithi142.stdout:7/23: mknod d7/c8 0 2024-03-29T04:44:52.563 INFO:tasks.workunit.client.0.smithi142.stdout:8/80: symlink d5/d10/l1b 0 2024-03-29T04:44:52.565 INFO:tasks.workunit.client.0.smithi142.stdout:0/29: creat d7/d9/fa x:0 0 0 2024-03-29T04:44:52.565 INFO:tasks.workunit.client.0.smithi142.stdout:1/36: dread f3 [0,4194304] 0 2024-03-29T04:44:52.567 INFO:tasks.workunit.client.0.smithi142.stdout:1/37: chown d8/f9 16 1 2024-03-29T04:44:52.569 INFO:tasks.workunit.client.0.smithi142.stdout:1/38: write f3 [499362,93459] 0 2024-03-29T04:44:52.571 INFO:tasks.workunit.client.0.smithi142.stdout:0/30: mknod d7/d9/cb 0 2024-03-29T04:44:52.573 INFO:tasks.workunit.client.0.smithi142.stdout:8/81: dwrite d5/d10/f12 [0,4194304] 0 2024-03-29T04:44:52.579 INFO:tasks.workunit.client.0.smithi142.stdout:8/82: mknod d5/d10/c1c 0 2024-03-29T04:44:52.579 INFO:tasks.workunit.client.0.smithi142.stdout:7/24: dwrite f3 [0,4194304] 0 2024-03-29T04:44:52.587 INFO:tasks.workunit.client.0.smithi142.stdout:1/39: dwrite f3 [0,4194304] 0 2024-03-29T04:44:52.587 INFO:tasks.workunit.client.0.smithi142.stdout:6/17: fsync f1 0 2024-03-29T04:44:52.588 INFO:tasks.workunit.client.0.smithi142.stdout:7/25: rename f2 to d7/f9 0 2024-03-29T04:44:52.588 INFO:tasks.workunit.client.0.smithi142.stdout:0/31: dwrite d7/d9/fa [0,4194304] 0 2024-03-29T04:44:52.589 INFO:tasks.workunit.client.0.smithi142.stdout:1/40: chown f3 99923430 1 2024-03-29T04:44:52.592 INFO:tasks.workunit.client.0.smithi142.stdout:1/41: dread - d8/fa zero size 2024-03-29T04:44:52.593 INFO:tasks.workunit.client.0.smithi142.stdout:2/19: chown l4 97298 1 2024-03-29T04:44:52.593 INFO:tasks.workunit.client.0.smithi142.stdout:8/83: dread d5/f19 [0,4194304] 0 2024-03-29T04:44:52.593 INFO:tasks.workunit.client.0.smithi142.stdout:3/15: rename d3/d5 to d3/d7 0 2024-03-29T04:44:52.595 INFO:tasks.workunit.client.0.smithi142.stdout:6/18: dread f3 [0,4194304] 0 2024-03-29T04:44:52.595 INFO:tasks.workunit.client.0.smithi142.stdout:6/19: chown f3 1027090934 1 2024-03-29T04:44:52.596 INFO:tasks.workunit.client.0.smithi142.stdout:6/20: chown l2 3580087 1 2024-03-29T04:44:52.598 INFO:tasks.workunit.client.0.smithi142.stdout:2/20: unlink l2 0 2024-03-29T04:44:52.600 INFO:tasks.workunit.client.0.smithi142.stdout:8/84: rmdir d5/d10 39 2024-03-29T04:44:52.601 INFO:tasks.workunit.client.0.smithi142.stdout:2/21: dread f3 [0,4194304] 0 2024-03-29T04:44:52.601 INFO:tasks.workunit.client.0.smithi142.stdout:3/16: mknod d3/c8 0 2024-03-29T04:44:52.604 INFO:tasks.workunit.client.0.smithi142.stdout:1/42: mknod d8/db/dc/cd 0 2024-03-29T04:44:52.605 INFO:tasks.workunit.client.0.smithi142.stdout:1/43: write f3 [591183,68105] 0 2024-03-29T04:44:52.605 INFO:tasks.workunit.client.0.smithi142.stdout:1/44: dread - f5 zero size 2024-03-29T04:44:52.608 INFO:tasks.workunit.client.0.smithi142.stdout:7/26: fdatasync f3 0 2024-03-29T04:44:52.611 INFO:tasks.workunit.client.0.smithi142.stdout:9/40: readlink d0/lb 0 2024-03-29T04:44:52.612 INFO:tasks.workunit.client.0.smithi142.stdout:6/21: write f3 [4640699,103710] 0 2024-03-29T04:44:52.613 INFO:tasks.workunit.client.0.smithi142.stdout:8/85: dread d5/d10/f16 [0,4194304] 0 2024-03-29T04:44:52.613 INFO:tasks.workunit.client.0.smithi142.stdout:9/41: stat d0/d2/c4 0 2024-03-29T04:44:52.614 INFO:tasks.workunit.client.0.smithi142.stdout:3/17: getdents d3/d7/d6 0 2024-03-29T04:44:52.615 INFO:tasks.workunit.client.0.smithi142.stdout:6/22: creat f4 x:0 0 0 2024-03-29T04:44:52.616 INFO:tasks.workunit.client.0.smithi142.stdout:1/45: dwrite f7 [0,4194304] 0 2024-03-29T04:44:52.618 INFO:tasks.workunit.client.0.smithi142.stdout:3/18: write d3/f4 [2038410,36138] 0 2024-03-29T04:44:52.619 INFO:tasks.workunit.client.0.smithi142.stdout:0/32: fdatasync d7/d9/fa 0 2024-03-29T04:44:52.621 INFO:tasks.workunit.client.0.smithi142.stdout:1/46: write d8/fa [1034719,99007] 0 2024-03-29T04:44:52.622 INFO:tasks.workunit.client.0.smithi142.stdout:1/47: stat f3 0 2024-03-29T04:44:52.623 INFO:tasks.workunit.client.0.smithi142.stdout:4/30: getdents d0/d9 0 2024-03-29T04:44:52.624 INFO:tasks.workunit.client.0.smithi142.stdout:3/19: mknod d3/c9 0 2024-03-29T04:44:52.624 INFO:tasks.workunit.client.0.smithi142.stdout:0/33: symlink d7/lc 0 2024-03-29T04:44:52.625 INFO:tasks.workunit.client.0.smithi142.stdout:4/31: stat d0 0 2024-03-29T04:44:52.625 INFO:tasks.workunit.client.0.smithi142.stdout:9/42: dread d0/f1 [0,4194304] 0 2024-03-29T04:44:52.626 INFO:tasks.workunit.client.0.smithi142.stdout:5/94: rmdir df 39 2024-03-29T04:44:52.627 INFO:tasks.workunit.client.0.smithi142.stdout:1/48: mkdir d8/db/dc/de 0 2024-03-29T04:44:52.627 INFO:tasks.workunit.client.0.smithi142.stdout:8/86: dwrite d5/d10/f16 [0,4194304] 0 2024-03-29T04:44:52.627 INFO:tasks.workunit.client.0.smithi142.stdout:6/23: dread f1 [0,4194304] 0 2024-03-29T04:44:52.628 INFO:tasks.workunit.client.0.smithi142.stdout:6/24: fsync f3 0 2024-03-29T04:44:52.629 INFO:tasks.workunit.client.0.smithi142.stdout:3/20: creat d3/fa x:0 0 0 2024-03-29T04:44:52.629 INFO:tasks.workunit.client.0.smithi142.stdout:0/34: mknod d7/cd 0 2024-03-29T04:44:52.630 INFO:tasks.workunit.client.0.smithi142.stdout:4/32: mkdir d0/dc 0 2024-03-29T04:44:52.630 INFO:tasks.workunit.client.0.smithi142.stdout:3/21: write d3/fa [581922,126036] 0 2024-03-29T04:44:52.634 INFO:tasks.workunit.client.0.smithi142.stdout:1/49: creat d8/db/ff x:0 0 0 2024-03-29T04:44:52.634 INFO:tasks.workunit.client.0.smithi142.stdout:1/50: readlink - no filename 2024-03-29T04:44:52.636 INFO:tasks.workunit.client.0.smithi142.stdout:6/25: mkdir d5 0 2024-03-29T04:44:52.637 INFO:tasks.workunit.client.0.smithi142.stdout:4/33: mkdir d0/dd 0 2024-03-29T04:44:52.638 INFO:tasks.workunit.client.0.smithi142.stdout:4/34: chown d0/f7 2 1 2024-03-29T04:44:52.638 INFO:tasks.workunit.client.0.smithi142.stdout:4/35: chown d0/dc 510 1 2024-03-29T04:44:52.642 INFO:tasks.workunit.client.0.smithi142.stdout:3/22: symlink d3/d7/d6/lb 0 2024-03-29T04:44:52.646 INFO:tasks.workunit.client.0.smithi142.stdout:7/27: write f3 [4915844,56702] 0 2024-03-29T04:44:52.646 INFO:tasks.workunit.client.0.smithi142.stdout:4/36: write d0/f8 [1277931,24303] 0 2024-03-29T04:44:52.646 INFO:tasks.workunit.client.0.smithi142.stdout:3/23: write d3/fa [1607234,17517] 0 2024-03-29T04:44:52.648 INFO:tasks.workunit.client.0.smithi142.stdout:1/51: dwrite d8/f9 [0,4194304] 0 2024-03-29T04:44:52.648 INFO:tasks.workunit.client.0.smithi142.stdout:7/28: dread - f1 zero size 2024-03-29T04:44:52.650 INFO:tasks.workunit.client.0.smithi142.stdout:6/26: rmdir d5 0 2024-03-29T04:44:52.651 INFO:tasks.workunit.client.0.smithi142.stdout:0/35: getdents d7/d9 0 2024-03-29T04:44:52.652 INFO:tasks.workunit.client.0.smithi142.stdout:6/27: chown f1 1032107 1 2024-03-29T04:44:52.652 INFO:tasks.workunit.client.0.smithi142.stdout:4/37: write d0/f7 [336821,47822] 0 2024-03-29T04:44:52.653 INFO:tasks.workunit.client.0.smithi142.stdout:2/22: truncate f3 311165 0 2024-03-29T04:44:52.654 INFO:tasks.workunit.client.0.smithi142.stdout:6/28: fsync f3 0 2024-03-29T04:44:52.654 INFO:tasks.workunit.client.0.smithi142.stdout:1/52: mknod d8/db/dc/c10 0 2024-03-29T04:44:52.655 INFO:tasks.workunit.client.0.smithi142.stdout:8/87: dwrite d5/f19 [4194304,4194304] 0 2024-03-29T04:44:52.655 INFO:tasks.workunit.client.0.smithi142.stdout:7/29: unlink l4 0 2024-03-29T04:44:52.656 INFO:tasks.workunit.client.0.smithi142.stdout:4/38: write d0/f3 [1820019,8013] 0 2024-03-29T04:44:52.660 INFO:tasks.workunit.client.0.smithi142.stdout:0/36: rename d7/cd to d7/ce 0 2024-03-29T04:44:52.662 INFO:tasks.workunit.client.0.smithi142.stdout:6/29: rename l2 to l6 0 2024-03-29T04:44:52.663 INFO:tasks.workunit.client.0.smithi142.stdout:5/95: write f7 [4628668,109995] 0 2024-03-29T04:44:52.665 INFO:tasks.workunit.client.0.smithi142.stdout:2/23: chown f3 122812 1 2024-03-29T04:44:52.666 INFO:tasks.workunit.client.0.smithi142.stdout:5/96: dread - fb zero size 2024-03-29T04:44:52.669 INFO:tasks.workunit.client.0.smithi142.stdout:1/53: creat d8/db/dc/de/f11 x:0 0 0 2024-03-29T04:44:52.670 INFO:tasks.workunit.client.0.smithi142.stdout:7/30: dwrite f6 [0,4194304] 0 2024-03-29T04:44:52.670 INFO:tasks.workunit.client.0.smithi142.stdout:5/97: write fd [1036488,31525] 0 2024-03-29T04:44:52.674 INFO:tasks.workunit.client.0.smithi142.stdout:0/37: dwrite f4 [4194304,4194304] 0 2024-03-29T04:44:52.675 INFO:tasks.workunit.client.0.smithi142.stdout:0/38: stat f2 0 2024-03-29T04:44:52.677 INFO:tasks.workunit.client.0.smithi142.stdout:1/54: creat d8/db/f12 x:0 0 0 2024-03-29T04:44:52.679 INFO:tasks.workunit.client.0.smithi142.stdout:7/31: creat d7/fa x:0 0 0 2024-03-29T04:44:52.679 INFO:tasks.workunit.client.0.smithi142.stdout:7/32: chown f6 19634462 1 2024-03-29T04:44:52.679 INFO:tasks.workunit.client.0.smithi142.stdout:7/33: dread - d7/fa zero size 2024-03-29T04:44:52.680 INFO:tasks.workunit.client.0.smithi142.stdout:7/34: write d7/fa [27461,64045] 0 2024-03-29T04:44:52.684 INFO:tasks.workunit.client.0.smithi142.stdout:5/98: creat df/f14 x:0 0 0 2024-03-29T04:44:52.684 INFO:tasks.workunit.client.0.smithi142.stdout:0/39: rmdir d7/d9 39 2024-03-29T04:44:52.685 INFO:tasks.workunit.client.0.smithi142.stdout:7/35: dread f6 [0,4194304] 0 2024-03-29T04:44:52.686 INFO:tasks.workunit.client.0.smithi142.stdout:1/55: creat d8/db/dc/de/f13 x:0 0 0 2024-03-29T04:44:52.686 INFO:tasks.workunit.client.0.smithi142.stdout:4/39: truncate d0/f8 376299 0 2024-03-29T04:44:52.687 INFO:tasks.workunit.client.0.smithi142.stdout:4/40: read d0/f3 [3076573,83688] 0 2024-03-29T04:44:52.687 INFO:tasks.workunit.client.0.smithi142.stdout:5/99: dread f7 [0,4194304] 0 2024-03-29T04:44:52.688 INFO:tasks.workunit.client.0.smithi142.stdout:6/30: getdents . 0 2024-03-29T04:44:52.688 INFO:tasks.workunit.client.0.smithi142.stdout:5/100: chown l5 61 1 2024-03-29T04:44:52.689 INFO:tasks.workunit.client.0.smithi142.stdout:4/41: truncate d0/fa 755010 0 2024-03-29T04:44:52.689 INFO:tasks.workunit.client.0.smithi142.stdout:6/31: readlink l6 0 2024-03-29T04:44:52.689 INFO:tasks.workunit.client.0.smithi142.stdout:6/32: write f1 [4041955,129270] 0 2024-03-29T04:44:52.692 INFO:tasks.workunit.client.0.smithi142.stdout:7/36: mkdir d7/db 0 2024-03-29T04:44:52.759 INFO:tasks.workunit.client.0.smithi142.stdout:5/101: mknod df/c15 0 2024-03-29T04:44:52.759 INFO:tasks.workunit.client.0.smithi142.stdout:4/42: symlink d0/le 0 2024-03-29T04:44:52.760 INFO:tasks.workunit.client.0.smithi142.stdout:6/33: creat f7 x:0 0 0 2024-03-29T04:44:52.760 INFO:tasks.workunit.client.0.smithi142.stdout:5/102: stat fd 0 2024-03-29T04:44:52.765 INFO:tasks.workunit.client.0.smithi142.stdout:2/24: dwrite f3 [0,4194304] 0 2024-03-29T04:44:52.767 INFO:tasks.workunit.client.0.smithi142.stdout:2/25: chown f3 24 1 2024-03-29T04:44:52.769 INFO:tasks.workunit.client.0.smithi142.stdout:0/40: dwrite d7/d9/fa [4194304,4194304] 0 2024-03-29T04:44:52.770 INFO:tasks.workunit.client.0.smithi142.stdout:0/41: stat d7 0 2024-03-29T04:44:52.772 INFO:tasks.workunit.client.0.smithi142.stdout:6/34: dwrite f4 [0,4194304] 0 2024-03-29T04:44:52.774 INFO:tasks.workunit.client.0.smithi142.stdout:6/35: truncate f7 731207 0 2024-03-29T04:44:52.775 INFO:tasks.workunit.client.0.smithi142.stdout:6/36: write f3 [756272,3386] 0 2024-03-29T04:44:52.775 INFO:tasks.workunit.client.0.smithi142.stdout:6/37: truncate f4 4437733 0 2024-03-29T04:44:52.785 INFO:tasks.workunit.client.0.smithi142.stdout:6/38: fsync f3 0 2024-03-29T04:44:52.785 INFO:tasks.workunit.client.0.smithi142.stdout:6/39: rmdir - no directory 2024-03-29T04:44:52.833 INFO:tasks.workunit.client.0.smithi142.stdout:9/43: sync 2024-03-29T04:44:52.834 INFO:tasks.workunit.client.0.smithi142.stdout:8/88: sync 2024-03-29T04:44:52.834 INFO:tasks.workunit.client.0.smithi142.stdout:3/24: sync 2024-03-29T04:44:52.837 INFO:tasks.workunit.client.0.smithi142.stdout:8/89: dwrite d5/f18 [0,4194304] 0 2024-03-29T04:44:52.891 INFO:tasks.workunit.client.0.smithi142.stdout:1/56: sync 2024-03-29T04:44:52.891 INFO:tasks.workunit.client.0.smithi142.stdout:7/37: sync 2024-03-29T04:44:52.932 INFO:tasks.workunit.client.0.smithi142.stdout:4/43: rmdir d0 39 2024-03-29T04:44:52.933 INFO:tasks.workunit.client.0.smithi142.stdout:5/103: mkdir df/d16 0 2024-03-29T04:44:52.935 INFO:tasks.workunit.client.0.smithi142.stdout:2/26: creat f5 x:0 0 0 2024-03-29T04:44:52.939 INFO:tasks.workunit.client.0.smithi142.stdout:3/25: creat d3/d7/fc x:0 0 0 2024-03-29T04:44:52.942 INFO:tasks.workunit.client.0.smithi142.stdout:8/90: unlink d5/d10/f12 0 2024-03-29T04:44:52.943 INFO:tasks.workunit.client.0.smithi142.stdout:7/38: mkdir d7/dc 0 2024-03-29T04:44:52.944 INFO:tasks.workunit.client.0.smithi142.stdout:7/39: chown f6 1981130 1 2024-03-29T04:44:52.944 INFO:tasks.workunit.client.0.smithi142.stdout:1/57: creat d8/db/f14 x:0 0 0 2024-03-29T04:44:52.944 INFO:tasks.workunit.client.0.smithi142.stdout:7/40: truncate d7/f9 245667 0 2024-03-29T04:44:52.945 INFO:tasks.workunit.client.0.smithi142.stdout:2/27: creat f6 x:0 0 0 2024-03-29T04:44:52.945 INFO:tasks.workunit.client.0.smithi142.stdout:0/42: getdents d7/d9 0 2024-03-29T04:44:52.946 INFO:tasks.workunit.client.0.smithi142.stdout:7/41: dread d7/fa [0,4194304] 0 2024-03-29T04:44:52.946 INFO:tasks.workunit.client.0.smithi142.stdout:2/28: truncate f6 109928 0 2024-03-29T04:44:52.947 INFO:tasks.workunit.client.0.smithi142.stdout:2/29: write f3 [1885998,54362] 0 2024-03-29T04:44:52.949 INFO:tasks.workunit.client.0.smithi142.stdout:3/26: symlink d3/d7/ld 0 2024-03-29T04:44:52.950 INFO:tasks.workunit.client.0.smithi142.stdout:4/44: fsync d0/f3 0 2024-03-29T04:44:52.952 INFO:tasks.workunit.client.0.smithi142.stdout:1/58: mkdir d8/db/dc/de/d15 0 2024-03-29T04:44:52.954 INFO:tasks.workunit.client.0.smithi142.stdout:0/43: dwrite d7/d9/fa [4194304,4194304] 0 2024-03-29T04:44:52.955 INFO:tasks.workunit.client.0.smithi142.stdout:6/40: link l6 l8 0 2024-03-29T04:44:52.955 INFO:tasks.workunit.client.0.smithi142.stdout:7/42: write f1 [465927,30379] 0 2024-03-29T04:44:52.955 INFO:tasks.workunit.client.0.smithi142.stdout:1/59: write d8/db/dc/de/f13 [876013,89881] 0 2024-03-29T04:44:52.956 INFO:tasks.workunit.client.0.smithi142.stdout:0/44: truncate f2 1729852 0 2024-03-29T04:44:52.956 INFO:tasks.workunit.client.0.smithi142.stdout:6/41: fsync f4 0 2024-03-29T04:44:52.957 INFO:tasks.workunit.client.0.smithi142.stdout:3/27: dread d3/fa [0,4194304] 0 2024-03-29T04:44:52.957 INFO:tasks.workunit.client.0.smithi142.stdout:2/30: creat f7 x:0 0 0 2024-03-29T04:44:52.957 INFO:tasks.workunit.client.0.smithi142.stdout:1/60: read - d8/db/f12 zero size 2024-03-29T04:44:52.958 INFO:tasks.workunit.client.0.smithi142.stdout:7/43: write d7/f9 [1442980,99461] 0 2024-03-29T04:44:52.958 INFO:tasks.workunit.client.0.smithi142.stdout:3/28: chown d3/c9 673 1 2024-03-29T04:44:52.959 INFO:tasks.workunit.client.0.smithi142.stdout:8/91: mkdir d5/d1d 0 2024-03-29T04:44:52.967 INFO:tasks.workunit.client.0.smithi142.stdout:0/45: creat d7/d9/ff x:0 0 0 2024-03-29T04:44:52.970 INFO:tasks.workunit.client.0.smithi142.stdout:3/29: readlink l2 0 2024-03-29T04:44:52.973 INFO:tasks.workunit.client.0.smithi142.stdout:7/44: truncate f6 3794969 0 2024-03-29T04:44:52.974 INFO:tasks.workunit.client.0.smithi142.stdout:8/92: unlink d5/c9 0 2024-03-29T04:44:52.975 INFO:tasks.workunit.client.0.smithi142.stdout:8/93: stat d5/d10/f16 0 2024-03-29T04:44:52.975 INFO:tasks.workunit.client.0.smithi142.stdout:4/45: creat d0/dd/ff x:0 0 0 2024-03-29T04:44:52.978 INFO:tasks.workunit.client.0.smithi142.stdout:0/46: rename f4 to d7/d9/f10 0 2024-03-29T04:44:52.980 INFO:tasks.workunit.client.0.smithi142.stdout:2/31: link f5 f8 0 2024-03-29T04:44:52.981 INFO:tasks.workunit.client.0.smithi142.stdout:3/30: creat d3/fe x:0 0 0 2024-03-29T04:44:52.982 INFO:tasks.workunit.client.0.smithi142.stdout:7/45: dwrite d7/f9 [0,4194304] 0 2024-03-29T04:44:52.984 INFO:tasks.workunit.client.0.smithi142.stdout:3/31: chown d3/f4 12 1 2024-03-29T04:44:52.984 INFO:tasks.workunit.client.0.smithi142.stdout:7/46: write d7/f9 [3307639,39447] 0 2024-03-29T04:44:52.988 INFO:tasks.workunit.client.0.smithi142.stdout:0/47: dwrite f2 [0,4194304] 0 2024-03-29T04:44:52.988 INFO:tasks.workunit.client.0.smithi142.stdout:8/94: mkdir d5/d1e 0 2024-03-29T04:44:52.989 INFO:tasks.workunit.client.0.smithi142.stdout:8/95: fdatasync d5/f19 0 2024-03-29T04:44:52.994 INFO:tasks.workunit.client.0.smithi142.stdout:3/32: mknod d3/d7/d6/cf 0 2024-03-29T04:44:52.997 INFO:tasks.workunit.client.0.smithi142.stdout:8/96: dwrite d5/d10/f16 [4194304,4194304] 0 2024-03-29T04:44:52.997 INFO:tasks.workunit.client.0.smithi142.stdout:4/46: link d0/dd/ff d0/f10 0 2024-03-29T04:44:52.998 INFO:tasks.workunit.client.0.smithi142.stdout:4/47: stat d0/c6 0 2024-03-29T04:44:52.998 INFO:tasks.workunit.client.0.smithi142.stdout:4/48: chown d0/f7 20 1 2024-03-29T04:44:53.000 INFO:tasks.workunit.client.0.smithi142.stdout:4/49: write d0/f7 [1068997,48232] 0 2024-03-29T04:44:53.000 INFO:tasks.workunit.client.0.smithi142.stdout:2/32: link f3 f9 0 2024-03-29T04:44:53.001 INFO:tasks.workunit.client.0.smithi142.stdout:2/33: truncate f8 483923 0 2024-03-29T04:44:53.001 INFO:tasks.workunit.client.0.smithi142.stdout:4/50: chown d0/le 11844 1 2024-03-29T04:44:53.002 INFO:tasks.workunit.client.0.smithi142.stdout:4/51: dread - d0/f10 zero size 2024-03-29T04:44:53.002 INFO:tasks.workunit.client.0.smithi142.stdout:2/34: truncate f5 566691 0 2024-03-29T04:44:53.022 INFO:tasks.workunit.client.0.smithi142.stdout:0/48: mknod d7/c11 0 2024-03-29T04:44:53.022 INFO:tasks.workunit.client.0.smithi142.stdout:2/35: creat fa x:0 0 0 2024-03-29T04:44:53.023 INFO:tasks.workunit.client.0.smithi142.stdout:0/49: write d7/d9/ff [960178,47609] 0 2024-03-29T04:44:53.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:52 smithi152 ceph-mon[37177]: pgmap v262: 65 pgs: 65 active+clean; 266 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 316 op/s 2024-03-29T04:44:53.079 INFO:tasks.workunit.client.0.smithi142.stdout:5/104: sync 2024-03-29T04:44:53.084 INFO:tasks.workunit.client.0.smithi142.stdout:2/36: dwrite fa [0,4194304] 0 2024-03-29T04:44:53.086 INFO:tasks.workunit.client.0.smithi142.stdout:0/50: dwrite d7/d9/fa [4194304,4194304] 0 2024-03-29T04:44:53.090 INFO:tasks.workunit.client.0.smithi142.stdout:0/51: dread d7/d9/ff [0,4194304] 0 2024-03-29T04:44:53.091 INFO:tasks.workunit.client.0.smithi142.stdout:0/52: fdatasync d7/d9/ff 0 2024-03-29T04:44:53.092 INFO:tasks.workunit.client.0.smithi142.stdout:6/42: getdents . 0 2024-03-29T04:44:53.093 INFO:tasks.workunit.client.0.smithi142.stdout:9/44: truncate d0/f1 3669652 0 2024-03-29T04:44:53.093 INFO:tasks.workunit.client.0.smithi142.stdout:1/61: rmdir d8/db/dc/de 39 2024-03-29T04:44:53.093 INFO:tasks.workunit.client.0.smithi142.stdout:6/43: write f1 [3227498,22861] 0 2024-03-29T04:44:53.094 INFO:tasks.workunit.client.0.smithi142.stdout:1/62: write f3 [40616,5176] 0 2024-03-29T04:44:53.095 INFO:tasks.workunit.client.0.smithi142.stdout:4/52: rename d0/dd to d0/d9/d11 0 2024-03-29T04:44:53.096 INFO:tasks.workunit.client.0.smithi142.stdout:1/63: stat d8/db/ff 0 2024-03-29T04:44:53.097 INFO:tasks.workunit.client.0.smithi142.stdout:8/97: creat d5/f1f x:0 0 0 2024-03-29T04:44:53.098 INFO:tasks.workunit.client.0.smithi142.stdout:6/44: readlink l6 0 2024-03-29T04:44:53.098 INFO:tasks.workunit.client.0.smithi142.stdout:4/53: mkdir d0/d12 0 2024-03-29T04:44:53.099 INFO:tasks.workunit.client.0.smithi142.stdout:6/45: write f1 [4112919,14110] 0 2024-03-29T04:44:53.099 INFO:tasks.workunit.client.0.smithi142.stdout:7/47: truncate f6 3326480 0 2024-03-29T04:44:53.101 INFO:tasks.workunit.client.0.smithi142.stdout:5/105: creat df/d16/f17 x:0 0 0 2024-03-29T04:44:53.102 INFO:tasks.workunit.client.0.smithi142.stdout:5/106: stat f3 0 2024-03-29T04:44:53.102 INFO:tasks.workunit.client.0.smithi142.stdout:7/48: write f3 [2353004,95049] 0 2024-03-29T04:44:53.103 INFO:tasks.workunit.client.0.smithi142.stdout:2/37: rename f5 to fb 0 2024-03-29T04:44:53.104 INFO:tasks.workunit.client.0.smithi142.stdout:7/49: write f3 [277471,94183] 0 2024-03-29T04:44:53.105 INFO:tasks.workunit.client.0.smithi142.stdout:2/38: read fb [171598,12659] 0 2024-03-29T04:44:53.105 INFO:tasks.workunit.client.0.smithi142.stdout:8/98: creat d5/d10/f20 x:0 0 0 2024-03-29T04:44:53.108 INFO:tasks.workunit.client.0.smithi142.stdout:4/54: dwrite d0/fa [0,4194304] 0 2024-03-29T04:44:53.110 INFO:tasks.workunit.client.0.smithi142.stdout:6/46: unlink f1 0 2024-03-29T04:44:53.111 INFO:tasks.workunit.client.0.smithi142.stdout:6/47: write f7 [853151,9774] 0 2024-03-29T04:44:53.112 INFO:tasks.workunit.client.0.smithi142.stdout:6/48: fdatasync f3 0 2024-03-29T04:44:53.113 INFO:tasks.workunit.client.0.smithi142.stdout:6/49: chown f7 1 1 2024-03-29T04:44:53.113 INFO:tasks.workunit.client.0.smithi142.stdout:8/99: dwrite d5/f19 [0,4194304] 0 2024-03-29T04:44:53.113 INFO:tasks.workunit.client.0.smithi142.stdout:6/50: chown f4 1391 1 2024-03-29T04:44:53.115 INFO:tasks.workunit.client.0.smithi142.stdout:5/107: creat df/d16/f18 x:0 0 0 2024-03-29T04:44:53.119 INFO:tasks.workunit.client.0.smithi142.stdout:8/100: dwrite d5/f19 [4194304,4194304] 0 2024-03-29T04:44:53.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:52 smithi142 ceph-mon[29027]: pgmap v262: 65 pgs: 65 active+clean; 266 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 316 op/s 2024-03-29T04:44:53.120 INFO:tasks.workunit.client.0.smithi142.stdout:8/101: dread - d5/d10/f20 zero size 2024-03-29T04:44:53.123 INFO:tasks.workunit.client.0.smithi142.stdout:4/55: getdents d0/d12 0 2024-03-29T04:44:53.126 INFO:tasks.workunit.client.0.smithi142.stdout:4/56: truncate d0/f8 651875 0 2024-03-29T04:44:53.166 INFO:tasks.workunit.client.0.smithi142.stdout:3/33: sync 2024-03-29T04:44:53.167 INFO:tasks.workunit.client.0.smithi142.stdout:3/34: fdatasync d3/d7/fc 0 2024-03-29T04:44:53.168 INFO:tasks.workunit.client.0.smithi142.stdout:3/35: write d3/f4 [282827,47768] 0 2024-03-29T04:44:53.169 INFO:tasks.workunit.client.0.smithi142.stdout:3/36: write d3/f4 [3997507,16297] 0 2024-03-29T04:44:53.170 INFO:tasks.workunit.client.0.smithi142.stdout:9/45: dread d0/f1 [0,4194304] 0 2024-03-29T04:44:53.172 INFO:tasks.workunit.client.0.smithi142.stdout:9/46: fdatasync d0/f1 0 2024-03-29T04:44:53.173 INFO:tasks.workunit.client.0.smithi142.stdout:9/47: chown d0/c3 5 1 2024-03-29T04:44:53.173 INFO:tasks.workunit.client.0.smithi142.stdout:0/53: truncate d7/d9/ff 664248 0 2024-03-29T04:44:53.177 INFO:tasks.workunit.client.0.smithi142.stdout:1/64: truncate f7 1548167 0 2024-03-29T04:44:53.178 INFO:tasks.workunit.client.0.smithi142.stdout:2/39: write f3 [3528186,8650] 0 2024-03-29T04:44:53.179 INFO:tasks.workunit.client.0.smithi142.stdout:1/65: creat d8/db/dc/de/d15/f16 x:0 0 0 2024-03-29T04:44:53.181 INFO:tasks.workunit.client.0.smithi142.stdout:5/108: write f3 [5839557,32081] 0 2024-03-29T04:44:53.181 INFO:tasks.workunit.client.0.smithi142.stdout:5/109: dread - fb zero size 2024-03-29T04:44:53.183 INFO:tasks.workunit.client.0.smithi142.stdout:5/110: fsync df/d16/f17 0 2024-03-29T04:44:53.183 INFO:tasks.workunit.client.0.smithi142.stdout:1/66: creat d8/db/dc/de/d15/f17 x:0 0 0 2024-03-29T04:44:53.183 INFO:tasks.workunit.client.0.smithi142.stdout:4/57: dread d0/f8 [0,4194304] 0 2024-03-29T04:44:53.183 INFO:tasks.workunit.client.0.smithi142.stdout:7/50: write f6 [442062,69927] 0 2024-03-29T04:44:53.201 INFO:tasks.workunit.client.0.smithi142.stdout:9/48: write d0/f1 [176250,95690] 0 2024-03-29T04:44:53.203 INFO:tasks.workunit.client.0.smithi142.stdout:1/67: dread - d8/db/dc/de/f11 zero size 2024-03-29T04:44:53.203 INFO:tasks.workunit.client.0.smithi142.stdout:2/40: truncate fa 3879744 0 2024-03-29T04:44:53.203 INFO:tasks.workunit.client.0.smithi142.stdout:2/41: rmdir - no directory 2024-03-29T04:44:53.205 INFO:tasks.workunit.client.0.smithi142.stdout:7/51: rename f3 to d7/fd 0 2024-03-29T04:44:53.207 INFO:tasks.workunit.client.0.smithi142.stdout:2/42: mknod cc 0 2024-03-29T04:44:53.208 INFO:tasks.workunit.client.0.smithi142.stdout:1/68: mknod d8/db/dc/de/d15/c18 0 2024-03-29T04:44:53.208 INFO:tasks.workunit.client.0.smithi142.stdout:3/37: dwrite d3/fa [0,4194304] 0 2024-03-29T04:44:53.209 INFO:tasks.workunit.client.0.smithi142.stdout:9/49: dread d0/f1 [0,4194304] 0 2024-03-29T04:44:53.210 INFO:tasks.workunit.client.0.smithi142.stdout:4/58: rename d0/fa to d0/d9/f13 0 2024-03-29T04:44:53.211 INFO:tasks.workunit.client.0.smithi142.stdout:4/59: dread d0/f8 [0,4194304] 0 2024-03-29T04:44:53.212 INFO:tasks.workunit.client.0.smithi142.stdout:2/43: creat fd x:0 0 0 2024-03-29T04:44:53.212 INFO:tasks.workunit.client.0.smithi142.stdout:7/52: truncate d7/fa 26634 0 2024-03-29T04:44:53.212 INFO:tasks.workunit.client.0.smithi142.stdout:2/44: stat f6 0 2024-03-29T04:44:53.213 INFO:tasks.workunit.client.0.smithi142.stdout:7/53: chown l5 119109 1 2024-03-29T04:44:53.213 INFO:tasks.workunit.client.0.smithi142.stdout:2/45: write fd [285491,120731] 0 2024-03-29T04:44:53.214 INFO:tasks.workunit.client.0.smithi142.stdout:9/50: dread d0/f1 [0,4194304] 0 2024-03-29T04:44:53.217 INFO:tasks.workunit.client.0.smithi142.stdout:3/38: rename d3/d7/ld to d3/d7/l10 0 2024-03-29T04:44:53.219 INFO:tasks.workunit.client.0.smithi142.stdout:5/111: getdents df/d16 0 2024-03-29T04:44:53.219 INFO:tasks.workunit.client.0.smithi142.stdout:5/112: readlink l5 0 2024-03-29T04:44:53.219 INFO:tasks.workunit.client.0.smithi142.stdout:5/113: readlink df/l12 0 2024-03-29T04:44:53.220 INFO:tasks.workunit.client.0.smithi142.stdout:4/60: truncate d0/f10 62413 0 2024-03-29T04:44:53.221 INFO:tasks.workunit.client.0.smithi142.stdout:7/54: fdatasync d7/fd 0 2024-03-29T04:44:53.222 INFO:tasks.workunit.client.0.smithi142.stdout:2/46: mkdir de 0 2024-03-29T04:44:53.223 INFO:tasks.workunit.client.0.smithi142.stdout:9/51: symlink d0/d2/lc 0 2024-03-29T04:44:53.224 INFO:tasks.workunit.client.0.smithi142.stdout:2/47: rename de to de/df 22 2024-03-29T04:44:53.224 INFO:tasks.workunit.client.0.smithi142.stdout:1/69: mkdir d8/db/dc/d19 0 2024-03-29T04:44:53.227 INFO:tasks.workunit.client.0.smithi142.stdout:7/55: mkdir d7/dc/de 0 2024-03-29T04:44:53.228 INFO:tasks.workunit.client.0.smithi142.stdout:3/39: dwrite d3/fe [0,4194304] 0 2024-03-29T04:44:53.230 INFO:tasks.workunit.client.0.smithi142.stdout:3/40: read d3/f4 [556123,46963] 0 2024-03-29T04:44:53.231 INFO:tasks.workunit.client.0.smithi142.stdout:1/70: truncate d8/db/dc/de/d15/f17 215496 0 2024-03-29T04:44:53.232 INFO:tasks.workunit.client.0.smithi142.stdout:9/52: creat d0/d2/d6/fd x:0 0 0 2024-03-29T04:44:53.232 INFO:tasks.workunit.client.0.smithi142.stdout:5/114: dwrite df/d16/f17 [0,4194304] 0 2024-03-29T04:44:53.233 INFO:tasks.workunit.client.0.smithi142.stdout:2/48: link f6 de/f10 0 2024-03-29T04:44:53.234 INFO:tasks.workunit.client.0.smithi142.stdout:2/49: write f9 [1844672,102105] 0 2024-03-29T04:44:53.235 INFO:tasks.workunit.client.0.smithi142.stdout:6/51: sync 2024-03-29T04:44:53.238 INFO:tasks.workunit.client.0.smithi142.stdout:3/41: creat d3/d7/f11 x:0 0 0 2024-03-29T04:44:53.241 INFO:tasks.workunit.client.0.smithi142.stdout:9/53: write d0/f1 [1095651,18021] 0 2024-03-29T04:44:53.241 INFO:tasks.workunit.client.0.smithi142.stdout:1/71: mkdir d8/db/dc/de/d15/d1a 0 2024-03-29T04:44:53.244 INFO:tasks.workunit.client.0.smithi142.stdout:2/50: fdatasync fb 0 2024-03-29T04:44:53.244 INFO:tasks.workunit.client.0.smithi142.stdout:2/51: readlink l4 0 2024-03-29T04:44:53.246 INFO:tasks.workunit.client.0.smithi142.stdout:3/42: creat d3/d7/f12 x:0 0 0 2024-03-29T04:44:53.248 INFO:tasks.workunit.client.0.smithi142.stdout:9/54: creat d0/d2/fe x:0 0 0 2024-03-29T04:44:53.248 INFO:tasks.workunit.client.0.smithi142.stdout:9/55: dread - d0/d2/d6/fd zero size 2024-03-29T04:44:53.250 INFO:tasks.workunit.client.0.smithi142.stdout:6/52: truncate f4 3687600 0 2024-03-29T04:44:53.251 INFO:tasks.workunit.client.0.smithi142.stdout:3/43: mknod d3/d7/c13 0 2024-03-29T04:44:53.252 INFO:tasks.workunit.client.0.smithi142.stdout:5/115: link l9 df/l19 0 2024-03-29T04:44:53.252 INFO:tasks.workunit.client.0.smithi142.stdout:1/72: creat d8/db/dc/d19/f1b x:0 0 0 2024-03-29T04:44:53.254 INFO:tasks.workunit.client.0.smithi142.stdout:3/44: readlink d3/d7/d6/lb 0 2024-03-29T04:44:53.257 INFO:tasks.workunit.client.0.smithi142.stdout:7/56: truncate f1 2496859 0 2024-03-29T04:44:53.257 INFO:tasks.workunit.client.0.smithi142.stdout:5/116: dread df/f10 [0,4194304] 0 2024-03-29T04:44:53.258 INFO:tasks.workunit.client.0.smithi142.stdout:1/73: write d8/db/dc/de/f11 [823392,96799] 0 2024-03-29T04:44:53.258 INFO:tasks.workunit.client.0.smithi142.stdout:3/45: creat d3/d7/d6/f14 x:0 0 0 2024-03-29T04:44:53.260 INFO:tasks.workunit.client.0.smithi142.stdout:9/56: dwrite d0/d2/d6/fd [0,4194304] 0 2024-03-29T04:44:53.263 INFO:tasks.workunit.client.0.smithi142.stdout:7/57: creat d7/ff x:0 0 0 2024-03-29T04:44:53.266 INFO:tasks.workunit.client.0.smithi142.stdout:6/53: dwrite f3 [4194304,4194304] 0 2024-03-29T04:44:53.267 INFO:tasks.workunit.client.0.smithi142.stdout:5/117: unlink df/d16/f18 0 2024-03-29T04:44:53.267 INFO:tasks.workunit.client.0.smithi142.stdout:9/57: rmdir d0/d2/d6 39 2024-03-29T04:44:53.268 INFO:tasks.workunit.client.0.smithi142.stdout:7/58: creat d7/dc/f10 x:0 0 0 2024-03-29T04:44:53.268 INFO:tasks.workunit.client.0.smithi142.stdout:9/58: read d0/f1 [925128,39621] 0 2024-03-29T04:44:53.269 INFO:tasks.workunit.client.0.smithi142.stdout:7/59: read - d7/ff zero size 2024-03-29T04:44:53.271 INFO:tasks.workunit.client.0.smithi142.stdout:6/54: mknod c9 0 2024-03-29T04:44:53.272 INFO:tasks.workunit.client.0.smithi142.stdout:5/118: mkdir df/d1a 0 2024-03-29T04:44:53.273 INFO:tasks.workunit.client.0.smithi142.stdout:7/60: rename l5 to d7/l11 0 2024-03-29T04:44:53.274 INFO:tasks.workunit.client.0.smithi142.stdout:5/119: stat df/c13 0 2024-03-29T04:44:53.274 INFO:tasks.workunit.client.0.smithi142.stdout:1/74: creat d8/db/dc/f1c x:0 0 0 2024-03-29T04:44:53.275 INFO:tasks.workunit.client.0.smithi142.stdout:6/55: rename l8 to la 0 2024-03-29T04:44:53.275 INFO:tasks.workunit.client.0.smithi142.stdout:9/59: mknod d0/d2/cf 0 2024-03-29T04:44:53.278 INFO:tasks.workunit.client.0.smithi142.stdout:3/46: dwrite d3/f4 [0,4194304] 0 2024-03-29T04:44:53.280 INFO:tasks.workunit.client.0.smithi142.stdout:1/75: unlink d8/db/ff 0 2024-03-29T04:44:53.284 INFO:tasks.workunit.client.0.smithi142.stdout:3/47: mkdir d3/d15 0 2024-03-29T04:44:53.285 INFO:tasks.workunit.client.0.smithi142.stdout:6/56: dwrite f3 [0,4194304] 0 2024-03-29T04:44:53.287 INFO:tasks.workunit.client.0.smithi142.stdout:1/76: rename d8/db/dc/de to d8/db/dc/d19/d1d 0 2024-03-29T04:44:53.288 INFO:tasks.workunit.client.0.smithi142.stdout:3/48: symlink d3/d7/d6/l16 0 2024-03-29T04:44:53.293 INFO:tasks.workunit.client.0.smithi142.stdout:6/57: truncate f4 904509 0 2024-03-29T04:44:53.294 INFO:tasks.workunit.client.0.smithi142.stdout:6/58: stat c9 0 2024-03-29T04:44:53.294 INFO:tasks.workunit.client.0.smithi142.stdout:3/49: chown d3/d7/d6/cf 11482945 1 2024-03-29T04:44:53.295 INFO:tasks.workunit.client.0.smithi142.stdout:5/120: write fe [327819,90190] 0 2024-03-29T04:44:53.296 INFO:tasks.workunit.client.0.smithi142.stdout:6/59: readlink la 0 2024-03-29T04:44:53.301 INFO:tasks.workunit.client.0.smithi142.stdout:9/60: dwrite d0/f1 [0,4194304] 0 2024-03-29T04:44:53.301 INFO:tasks.workunit.client.0.smithi142.stdout:6/60: write f3 [3951950,120998] 0 2024-03-29T04:44:53.302 INFO:tasks.workunit.client.0.smithi142.stdout:7/61: dwrite f1 [0,4194304] 0 2024-03-29T04:44:53.303 INFO:tasks.workunit.client.0.smithi142.stdout:1/77: dwrite f5 [0,4194304] 0 2024-03-29T04:44:53.307 INFO:tasks.workunit.client.0.smithi142.stdout:3/50: rename d3/d7 to d3/d15/d17 0 2024-03-29T04:44:53.307 INFO:tasks.workunit.client.0.smithi142.stdout:1/78: fdatasync f3 0 2024-03-29T04:44:53.309 INFO:tasks.workunit.client.0.smithi142.stdout:6/61: creat fb x:0 0 0 2024-03-29T04:44:53.311 INFO:tasks.workunit.client.0.smithi142.stdout:5/121: dwrite fa [0,4194304] 0 2024-03-29T04:44:53.316 INFO:tasks.workunit.client.0.smithi142.stdout:3/51: mknod d3/d15/d17/d6/c18 0 2024-03-29T04:44:53.317 INFO:tasks.workunit.client.0.smithi142.stdout:1/79: dread d8/db/dc/d19/d1d/f11 [0,4194304] 0 2024-03-29T04:44:53.321 INFO:tasks.workunit.client.0.smithi142.stdout:9/61: unlink d0/d2/d6/ca 0 2024-03-29T04:44:53.322 INFO:tasks.workunit.client.0.smithi142.stdout:7/62: rmdir d7/db 0 2024-03-29T04:44:53.323 INFO:tasks.workunit.client.0.smithi142.stdout:3/52: mknod d3/d15/d17/c19 0 2024-03-29T04:44:53.323 INFO:tasks.workunit.client.0.smithi142.stdout:8/102: sync 2024-03-29T04:44:53.324 INFO:tasks.workunit.client.0.smithi142.stdout:3/53: chown d3/d15/d17/f12 3 1 2024-03-29T04:44:53.325 INFO:tasks.workunit.client.0.smithi142.stdout:1/80: rmdir d8 39 2024-03-29T04:44:53.326 INFO:tasks.workunit.client.0.smithi142.stdout:7/63: dread d7/fd [0,4194304] 0 2024-03-29T04:44:53.327 INFO:tasks.workunit.client.0.smithi142.stdout:7/64: chown f1 146378 1 2024-03-29T04:44:53.327 INFO:tasks.workunit.client.0.smithi142.stdout:5/122: rename l9 to df/d16/l1b 0 2024-03-29T04:44:53.327 INFO:tasks.workunit.client.0.smithi142.stdout:5/123: dread - fb zero size 2024-03-29T04:44:53.328 INFO:tasks.workunit.client.0.smithi142.stdout:3/54: mknod d3/d15/d17/d6/c1a 0 2024-03-29T04:44:53.329 INFO:tasks.workunit.client.0.smithi142.stdout:3/55: dread - d3/d15/d17/f12 zero size 2024-03-29T04:44:53.331 INFO:tasks.workunit.client.0.smithi142.stdout:3/56: readlink l2 0 2024-03-29T04:44:53.331 INFO:tasks.workunit.client.0.smithi142.stdout:9/62: rename d0/d2/lc to d0/l10 0 2024-03-29T04:44:53.332 INFO:tasks.workunit.client.0.smithi142.stdout:3/57: write d3/d15/d17/f12 [591012,85147] 0 2024-03-29T04:44:53.333 INFO:tasks.workunit.client.0.smithi142.stdout:7/65: symlink d7/dc/de/l12 0 2024-03-29T04:44:53.333 INFO:tasks.workunit.client.0.smithi142.stdout:1/81: mknod d8/db/dc/c1e 0 2024-03-29T04:44:53.335 INFO:tasks.workunit.client.0.smithi142.stdout:3/58: write d3/f4 [2355421,510] 0 2024-03-29T04:44:53.335 INFO:tasks.workunit.client.0.smithi142.stdout:7/66: write d7/ff [530365,34178] 0 2024-03-29T04:44:53.338 INFO:tasks.workunit.client.0.smithi142.stdout:5/124: dwrite fd [0,4194304] 0 2024-03-29T04:44:53.340 INFO:tasks.workunit.client.0.smithi142.stdout:6/62: getdents . 0 2024-03-29T04:44:53.340 INFO:tasks.workunit.client.0.smithi142.stdout:3/59: chown d3/d15/d17/d6/c18 1521982 1 2024-03-29T04:44:53.340 INFO:tasks.workunit.client.0.smithi142.stdout:7/67: chown d7/dc/de/l12 937691 1 2024-03-29T04:44:53.341 INFO:tasks.workunit.client.0.smithi142.stdout:8/103: rename d5/d10/f16 to d5/f21 0 2024-03-29T04:44:53.343 INFO:tasks.workunit.client.0.smithi142.stdout:1/82: mkdir d8/db/dc/d19/d1d/d1f 0 2024-03-29T04:44:53.343 INFO:tasks.workunit.client.0.smithi142.stdout:0/54: sync 2024-03-29T04:44:53.343 INFO:tasks.workunit.client.0.smithi142.stdout:4/61: sync 2024-03-29T04:44:53.344 INFO:tasks.workunit.client.0.smithi142.stdout:2/52: sync 2024-03-29T04:44:53.344 INFO:tasks.workunit.client.0.smithi142.stdout:8/104: write d5/f18 [3773030,47067] 0 2024-03-29T04:44:53.346 INFO:tasks.workunit.client.0.smithi142.stdout:1/83: write d8/db/f12 [224795,4845] 0 2024-03-29T04:44:53.348 INFO:tasks.workunit.client.0.smithi142.stdout:1/84: chown d8/db/dc 46980 1 2024-03-29T04:44:53.349 INFO:tasks.workunit.client.0.smithi142.stdout:7/68: dread d7/fa [0,4194304] 0 2024-03-29T04:44:53.349 INFO:tasks.workunit.client.0.smithi142.stdout:3/60: rmdir d3/d15/d17/d6 39 2024-03-29T04:44:53.350 INFO:tasks.workunit.client.0.smithi142.stdout:6/63: dread f3 [0,4194304] 0 2024-03-29T04:44:53.352 INFO:tasks.workunit.client.0.smithi142.stdout:1/85: truncate d8/db/f12 669610 0 2024-03-29T04:44:53.354 INFO:tasks.workunit.client.0.smithi142.stdout:4/62: creat d0/f14 x:0 0 0 2024-03-29T04:44:53.355 INFO:tasks.workunit.client.0.smithi142.stdout:5/125: dwrite df/f10 [0,4194304] 0 2024-03-29T04:44:53.355 INFO:tasks.workunit.client.0.smithi142.stdout:4/63: dread - d0/f14 zero size 2024-03-29T04:44:53.356 INFO:tasks.workunit.client.0.smithi142.stdout:2/53: mkdir de/d11 0 2024-03-29T04:44:53.360 INFO:tasks.workunit.client.0.smithi142.stdout:6/64: creat fc x:0 0 0 2024-03-29T04:44:53.363 INFO:tasks.workunit.client.0.smithi142.stdout:1/86: creat d8/db/f20 x:0 0 0 2024-03-29T04:44:53.365 INFO:tasks.workunit.client.0.smithi142.stdout:9/63: rename d0/f1 to d0/d2/f11 0 2024-03-29T04:44:53.368 INFO:tasks.workunit.client.0.smithi142.stdout:3/61: symlink d3/d15/d17/d6/l1b 0 2024-03-29T04:44:53.370 INFO:tasks.workunit.client.0.smithi142.stdout:1/87: mkdir d8/db/dc/d19/d1d/d15/d21 0 2024-03-29T04:44:53.372 INFO:tasks.workunit.client.0.smithi142.stdout:8/105: rename d5/f21 to d5/f22 0 2024-03-29T04:44:53.372 INFO:tasks.workunit.client.0.smithi142.stdout:5/126: mknod df/d1a/c1c 0 2024-03-29T04:44:53.372 INFO:tasks.workunit.client.0.smithi142.stdout:1/88: write d8/f9 [1570806,110269] 0 2024-03-29T04:44:53.372 INFO:tasks.workunit.client.0.smithi142.stdout:9/64: mkdir d0/d12 0 2024-03-29T04:44:53.376 INFO:tasks.workunit.client.0.smithi142.stdout:0/55: rename d7/l8 to d7/l12 0 2024-03-29T04:44:53.380 INFO:tasks.workunit.client.0.smithi142.stdout:3/62: dwrite d3/d15/d17/f12 [0,4194304] 0 2024-03-29T04:44:53.381 INFO:tasks.workunit.client.0.smithi142.stdout:7/69: getdents d7/dc/de 0 2024-03-29T04:44:53.384 INFO:tasks.workunit.client.0.smithi142.stdout:1/89: mknod d8/db/dc/d19/d1d/d15/c22 0 2024-03-29T04:44:53.385 INFO:tasks.workunit.client.0.smithi142.stdout:8/106: dread d5/f18 [0,4194304] 0 2024-03-29T04:44:53.386 INFO:tasks.workunit.client.0.smithi142.stdout:4/64: write d0/d9/d11/ff [342720,20963] 0 2024-03-29T04:44:53.387 INFO:tasks.workunit.client.0.smithi142.stdout:5/127: dread df/d16/f17 [0,4194304] 0 2024-03-29T04:44:53.388 INFO:tasks.workunit.client.0.smithi142.stdout:0/56: write d7/d9/f10 [5696368,67988] 0 2024-03-29T04:44:53.388 INFO:tasks.workunit.client.0.smithi142.stdout:6/65: getdents . 0 2024-03-29T04:44:53.390 INFO:tasks.workunit.client.0.smithi142.stdout:4/65: chown d0/d12 11522064 1 2024-03-29T04:44:53.390 INFO:tasks.workunit.client.0.smithi142.stdout:7/70: rmdir d7/dc 39 2024-03-29T04:44:53.391 INFO:tasks.workunit.client.0.smithi142.stdout:2/54: dwrite de/f10 [0,4194304] 0 2024-03-29T04:44:53.391 INFO:tasks.workunit.client.0.smithi142.stdout:3/63: creat d3/d15/d17/d6/f1c x:0 0 0 2024-03-29T04:44:53.392 INFO:tasks.workunit.client.0.smithi142.stdout:9/65: truncate d0/d2/f11 674700 0 2024-03-29T04:44:53.392 INFO:tasks.workunit.client.0.smithi142.stdout:1/90: unlink d8/db/dc/d19/f1b 0 2024-03-29T04:44:53.392 INFO:tasks.workunit.client.0.smithi142.stdout:2/55: fdatasync f3 0 2024-03-29T04:44:53.393 INFO:tasks.workunit.client.0.smithi142.stdout:5/128: creat df/f1d x:0 0 0 2024-03-29T04:44:53.395 INFO:tasks.workunit.client.0.smithi142.stdout:6/66: unlink f7 0 2024-03-29T04:44:53.396 INFO:tasks.workunit.client.0.smithi142.stdout:2/56: write f9 [2455890,48102] 0 2024-03-29T04:44:53.400 INFO:tasks.workunit.client.0.smithi142.stdout:7/71: unlink d7/ff 0 2024-03-29T04:44:53.404 INFO:tasks.workunit.client.0.smithi142.stdout:3/64: rmdir d3 39 2024-03-29T04:44:53.406 INFO:tasks.workunit.client.0.smithi142.stdout:4/66: write d0/f8 [932719,75971] 0 2024-03-29T04:44:53.406 INFO:tasks.workunit.client.0.smithi142.stdout:4/67: readlink d0/le 0 2024-03-29T04:44:53.408 INFO:tasks.workunit.client.0.smithi142.stdout:4/68: write d0/f3 [353143,95424] 0 2024-03-29T04:44:53.408 INFO:tasks.workunit.client.0.smithi142.stdout:1/91: write d8/db/dc/d19/d1d/f11 [435102,12497] 0 2024-03-29T04:44:53.408 INFO:tasks.workunit.client.0.smithi142.stdout:6/67: mknod cd 0 2024-03-29T04:44:53.409 INFO:tasks.workunit.client.0.smithi142.stdout:5/129: creat df/d16/f1e x:0 0 0 2024-03-29T04:44:53.410 INFO:tasks.workunit.client.0.smithi142.stdout:1/92: truncate d8/db/dc/f1c 872273 0 2024-03-29T04:44:53.410 INFO:tasks.workunit.client.0.smithi142.stdout:2/57: rename cc to de/d11/c12 0 2024-03-29T04:44:53.411 INFO:tasks.workunit.client.0.smithi142.stdout:0/57: truncate d7/d9/ff 451255 0 2024-03-29T04:44:53.414 INFO:tasks.workunit.client.0.smithi142.stdout:0/58: write f2 [188009,60917] 0 2024-03-29T04:44:53.415 INFO:tasks.workunit.client.0.smithi142.stdout:6/68: symlink le 0 2024-03-29T04:44:53.416 INFO:tasks.workunit.client.0.smithi142.stdout:4/69: dwrite d0/f14 [0,4194304] 0 2024-03-29T04:44:53.417 INFO:tasks.workunit.client.0.smithi142.stdout:6/69: truncate fb 285850 0 2024-03-29T04:44:53.418 INFO:tasks.workunit.client.0.smithi142.stdout:6/70: truncate fb 1263351 0 2024-03-29T04:44:53.418 INFO:tasks.workunit.client.0.smithi142.stdout:5/130: rmdir df 39 2024-03-29T04:44:53.419 INFO:tasks.workunit.client.0.smithi142.stdout:3/65: dread d3/fa [0,4194304] 0 2024-03-29T04:44:53.420 INFO:tasks.workunit.client.0.smithi142.stdout:1/93: symlink d8/l23 0 2024-03-29T04:44:53.421 INFO:tasks.workunit.client.0.smithi142.stdout:3/66: chown d3/d15/d17/d6 22 1 2024-03-29T04:44:53.421 INFO:tasks.workunit.client.0.smithi142.stdout:2/58: truncate f8 1529646 0 2024-03-29T04:44:53.423 INFO:tasks.workunit.client.0.smithi142.stdout:1/94: write d8/db/dc/d19/d1d/d15/f16 [260200,123037] 0 2024-03-29T04:44:53.423 INFO:tasks.workunit.client.0.smithi142.stdout:8/107: truncate d5/f22 4971263 0 2024-03-29T04:44:53.424 INFO:tasks.workunit.client.0.smithi142.stdout:8/108: stat d5/c13 0 2024-03-29T04:44:53.425 INFO:tasks.workunit.client.0.smithi142.stdout:8/109: dread - d5/f1f zero size 2024-03-29T04:44:53.426 INFO:tasks.workunit.client.0.smithi142.stdout:7/72: unlink d7/dc/de/l12 0 2024-03-29T04:44:53.426 INFO:tasks.workunit.client.0.smithi142.stdout:7/73: read f1 [1452298,16336] 0 2024-03-29T04:44:53.427 INFO:tasks.workunit.client.0.smithi142.stdout:9/66: truncate d0/d2/d6/fd 1686794 0 2024-03-29T04:44:53.429 INFO:tasks.workunit.client.0.smithi142.stdout:3/67: dwrite d3/fa [0,4194304] 0 2024-03-29T04:44:53.430 INFO:tasks.workunit.client.0.smithi142.stdout:2/59: mknod de/d11/c13 0 2024-03-29T04:44:53.431 INFO:tasks.workunit.client.0.smithi142.stdout:2/60: chown de/f10 4 1 2024-03-29T04:44:53.431 INFO:tasks.workunit.client.0.smithi142.stdout:1/95: symlink d8/db/l24 0 2024-03-29T04:44:53.433 INFO:tasks.workunit.client.0.smithi142.stdout:7/74: dread d7/fd [0,4194304] 0 2024-03-29T04:44:53.433 INFO:tasks.workunit.client.0.smithi142.stdout:1/96: chown d8/db/dc/d19/d1d/d15/c18 11247 1 2024-03-29T04:44:53.436 INFO:tasks.workunit.client.0.smithi142.stdout:4/70: mkdir d0/dc/d15 0 2024-03-29T04:44:53.437 INFO:tasks.workunit.client.0.smithi142.stdout:0/59: mkdir d7/d13 0 2024-03-29T04:44:53.441 INFO:tasks.workunit.client.0.smithi142.stdout:9/67: dread d0/d2/f11 [0,4194304] 0 2024-03-29T04:44:53.441 INFO:tasks.workunit.client.0.smithi142.stdout:6/71: link fb ff 0 2024-03-29T04:44:53.442 INFO:tasks.workunit.client.0.smithi142.stdout:4/71: dwrite d0/f10 [0,4194304] 0 2024-03-29T04:44:53.443 INFO:tasks.workunit.client.0.smithi142.stdout:5/131: creat df/d1a/f1f x:0 0 0 2024-03-29T04:44:53.445 INFO:tasks.workunit.client.0.smithi142.stdout:5/132: dread - fb zero size 2024-03-29T04:44:53.445 INFO:tasks.workunit.client.0.smithi142.stdout:3/68: mknod d3/d15/c1d 0 2024-03-29T04:44:53.451 INFO:tasks.workunit.client.0.smithi142.stdout:3/69: dwrite d3/d15/d17/d6/f14 [0,4194304] 0 2024-03-29T04:44:53.452 INFO:tasks.workunit.client.0.smithi142.stdout:4/72: symlink d0/d9/d11/l16 0 2024-03-29T04:44:53.452 INFO:tasks.workunit.client.0.smithi142.stdout:5/133: mknod df/d1a/c20 0 2024-03-29T04:44:53.453 INFO:tasks.workunit.client.0.smithi142.stdout:3/70: stat d3/d15/d17/fc 0 2024-03-29T04:44:53.453 INFO:tasks.workunit.client.0.smithi142.stdout:2/61: truncate fa 3747286 0 2024-03-29T04:44:53.455 INFO:tasks.workunit.client.0.smithi142.stdout:5/134: chown df/l12 51 1 2024-03-29T04:44:53.455 INFO:tasks.workunit.client.0.smithi142.stdout:2/62: readlink l4 0 2024-03-29T04:44:53.456 INFO:tasks.workunit.client.0.smithi142.stdout:7/75: symlink d7/l13 0 2024-03-29T04:44:53.457 INFO:tasks.workunit.client.0.smithi142.stdout:3/71: write d3/d15/d17/f12 [1946716,100566] 0 2024-03-29T04:44:53.458 INFO:tasks.workunit.client.0.smithi142.stdout:0/60: symlink d7/l14 0 2024-03-29T04:44:53.459 INFO:tasks.workunit.client.0.smithi142.stdout:9/68: creat d0/d12/f13 x:0 0 0 2024-03-29T04:44:53.461 INFO:tasks.workunit.client.0.smithi142.stdout:6/72: dwrite f3 [0,4194304] 0 2024-03-29T04:44:53.462 INFO:tasks.workunit.client.0.smithi142.stdout:8/110: link l4 d5/d1e/l23 0 2024-03-29T04:44:53.464 INFO:tasks.workunit.client.0.smithi142.stdout:5/135: mkdir df/d1a/d21 0 2024-03-29T04:44:53.466 INFO:tasks.workunit.client.0.smithi142.stdout:8/111: fsync d5/f19 0 2024-03-29T04:44:53.466 INFO:tasks.workunit.client.0.smithi142.stdout:7/76: creat d7/dc/de/f14 x:0 0 0 2024-03-29T04:44:53.467 INFO:tasks.workunit.client.0.smithi142.stdout:5/136: fdatasync df/d16/f1e 0 2024-03-29T04:44:53.468 INFO:tasks.workunit.client.0.smithi142.stdout:6/73: dread ff [0,4194304] 0 2024-03-29T04:44:53.468 INFO:tasks.workunit.client.0.smithi142.stdout:2/63: dread fb [0,4194304] 0 2024-03-29T04:44:53.468 INFO:tasks.workunit.client.0.smithi142.stdout:0/61: mkdir d7/d9/d15 0 2024-03-29T04:44:53.470 INFO:tasks.workunit.client.0.smithi142.stdout:3/72: symlink d3/d15/d17/d6/l1e 0 2024-03-29T04:44:53.471 INFO:tasks.workunit.client.0.smithi142.stdout:1/97: write f7 [2267767,107158] 0 2024-03-29T04:44:53.472 INFO:tasks.workunit.client.0.smithi142.stdout:3/73: write d3/d15/d17/d6/f14 [312572,19314] 0 2024-03-29T04:44:53.472 INFO:tasks.workunit.client.0.smithi142.stdout:1/98: stat d8/db/dc/d19/d1d/f11 0 2024-03-29T04:44:53.477 INFO:tasks.workunit.client.0.smithi142.stdout:2/64: mknod de/c14 0 2024-03-29T04:44:53.478 INFO:tasks.workunit.client.0.smithi142.stdout:6/74: rename l6 to l10 0 2024-03-29T04:44:53.479 INFO:tasks.workunit.client.0.smithi142.stdout:8/112: dwrite d5/f18 [0,4194304] 0 2024-03-29T04:44:53.481 INFO:tasks.workunit.client.0.smithi142.stdout:5/137: creat df/d1a/f22 x:0 0 0 2024-03-29T04:44:53.483 INFO:tasks.workunit.client.0.smithi142.stdout:5/138: write df/d16/f1e [262454,105164] 0 2024-03-29T04:44:53.484 INFO:tasks.workunit.client.0.smithi142.stdout:5/139: dread - fb zero size 2024-03-29T04:44:53.484 INFO:tasks.workunit.client.0.smithi142.stdout:3/74: unlink d3/c9 0 2024-03-29T04:44:53.485 INFO:tasks.workunit.client.0.smithi142.stdout:5/140: chown df/l12 596 1 2024-03-29T04:44:53.486 INFO:tasks.workunit.client.0.smithi142.stdout:1/99: creat d8/db/f25 x:0 0 0 2024-03-29T04:44:53.487 INFO:tasks.workunit.client.0.smithi142.stdout:6/75: dwrite fc [0,4194304] 0 2024-03-29T04:44:53.488 INFO:tasks.workunit.client.0.smithi142.stdout:1/100: dread - d8/db/f14 zero size 2024-03-29T04:44:53.490 INFO:tasks.workunit.client.0.smithi142.stdout:8/113: rename d5/d1e to d5/d1d/d24 0 2024-03-29T04:44:53.490 INFO:tasks.workunit.client.0.smithi142.stdout:7/77: rmdir d7/dc/de 39 2024-03-29T04:44:53.491 INFO:tasks.workunit.client.0.smithi142.stdout:8/114: write d5/f19 [4923574,4507] 0 2024-03-29T04:44:53.492 INFO:tasks.workunit.client.0.smithi142.stdout:0/62: creat d7/d9/d15/f16 x:0 0 0 2024-03-29T04:44:53.494 INFO:tasks.workunit.client.0.smithi142.stdout:0/63: write d7/d9/fa [5449203,43813] 0 2024-03-29T04:44:53.494 INFO:tasks.workunit.client.0.smithi142.stdout:4/73: truncate d0/f3 3782145 0 2024-03-29T04:44:53.495 INFO:tasks.workunit.client.0.smithi142.stdout:3/75: mkdir d3/d1f 0 2024-03-29T04:44:53.496 INFO:tasks.workunit.client.0.smithi142.stdout:4/74: fsync d0/d9/fb 0 2024-03-29T04:44:53.497 INFO:tasks.workunit.client.0.smithi142.stdout:5/141: creat df/f23 x:0 0 0 2024-03-29T04:44:53.497 INFO:tasks.workunit.client.0.smithi142.stdout:3/76: read d3/f4 [2316808,84422] 0 2024-03-29T04:44:53.498 INFO:tasks.workunit.client.0.smithi142.stdout:6/76: creat f11 x:0 0 0 2024-03-29T04:44:53.500 INFO:tasks.workunit.client.0.smithi142.stdout:7/78: rename d7 to d7/dc/de/d15 22 2024-03-29T04:44:53.504 INFO:tasks.workunit.client.0.smithi142.stdout:4/75: creat d0/dc/f17 x:0 0 0 2024-03-29T04:44:53.505 INFO:tasks.workunit.client.0.smithi142.stdout:5/142: truncate f7 1576001 0 2024-03-29T04:44:53.506 INFO:tasks.workunit.client.0.smithi142.stdout:3/77: creat d3/d15/d17/d6/f20 x:0 0 0 2024-03-29T04:44:53.507 INFO:tasks.workunit.client.0.smithi142.stdout:1/101: creat d8/db/dc/d19/d1d/d15/d21/f26 x:0 0 0 2024-03-29T04:44:53.508 INFO:tasks.workunit.client.0.smithi142.stdout:7/79: dwrite d7/dc/de/f14 [0,4194304] 0 2024-03-29T04:44:53.509 INFO:tasks.workunit.client.0.smithi142.stdout:6/77: symlink l12 0 2024-03-29T04:44:53.509 INFO:tasks.workunit.client.0.smithi142.stdout:6/78: read ff [242993,51670] 0 2024-03-29T04:44:53.510 INFO:tasks.workunit.client.0.smithi142.stdout:6/79: stat c9 0 2024-03-29T04:44:53.510 INFO:tasks.workunit.client.0.smithi142.stdout:6/80: rmdir - no directory 2024-03-29T04:44:53.510 INFO:tasks.workunit.client.0.smithi142.stdout:1/102: dread - d8/db/dc/d19/d1d/d15/d21/f26 zero size 2024-03-29T04:44:53.514 INFO:tasks.workunit.client.0.smithi142.stdout:5/143: unlink df/d16/f17 0 2024-03-29T04:44:53.516 INFO:tasks.workunit.client.0.smithi142.stdout:6/81: dread f3 [4194304,4194304] 0 2024-03-29T04:44:53.517 INFO:tasks.workunit.client.0.smithi142.stdout:3/78: rename d3/f4 to d3/d15/f21 0 2024-03-29T04:44:53.518 INFO:tasks.workunit.client.0.smithi142.stdout:8/115: creat d5/f25 x:0 0 0 2024-03-29T04:44:53.519 INFO:tasks.workunit.client.0.smithi142.stdout:8/116: dread - d5/f1f zero size 2024-03-29T04:44:53.520 INFO:tasks.workunit.client.0.smithi142.stdout:1/103: creat d8/db/dc/d19/d1d/d15/f27 x:0 0 0 2024-03-29T04:44:53.521 INFO:tasks.workunit.client.0.smithi142.stdout:5/144: mkdir df/d16/d24 0 2024-03-29T04:44:53.522 INFO:tasks.workunit.client.0.smithi142.stdout:1/104: write d8/db/dc/d19/d1d/d15/f16 [499249,96253] 0 2024-03-29T04:44:53.526 INFO:tasks.workunit.client.0.smithi142.stdout:0/64: rename d7/ce to d7/c17 0 2024-03-29T04:44:53.530 INFO:tasks.workunit.client.0.smithi142.stdout:5/145: rmdir df/d1a 39 2024-03-29T04:44:53.531 INFO:tasks.workunit.client.0.smithi142.stdout:6/82: unlink f11 0 2024-03-29T04:44:53.532 INFO:tasks.workunit.client.0.smithi142.stdout:1/105: symlink d8/db/dc/d19/l28 0 2024-03-29T04:44:53.535 INFO:tasks.workunit.client.0.smithi142.stdout:4/76: getdents d0/d9 0 2024-03-29T04:44:53.535 INFO:tasks.workunit.client.0.smithi142.stdout:4/77: truncate d0/f7 2038624 0 2024-03-29T04:44:53.536 INFO:tasks.workunit.client.0.smithi142.stdout:5/146: mkdir df/d16/d25 0 2024-03-29T04:44:53.538 INFO:tasks.workunit.client.0.smithi142.stdout:4/78: read d0/f10 [3660760,129719] 0 2024-03-29T04:44:53.543 INFO:tasks.workunit.client.0.smithi142.stdout:5/147: creat df/d16/f26 x:0 0 0 2024-03-29T04:44:53.544 INFO:tasks.workunit.client.0.smithi142.stdout:5/148: fsync f3 0 2024-03-29T04:44:53.544 INFO:tasks.workunit.client.0.smithi142.stdout:1/106: creat d8/db/dc/d19/d1d/d1f/f29 x:0 0 0 2024-03-29T04:44:53.545 INFO:tasks.workunit.client.0.smithi142.stdout:6/83: write fb [51024,983] 0 2024-03-29T04:44:53.545 INFO:tasks.workunit.client.0.smithi142.stdout:3/79: link d3/d15/d17/c13 d3/d15/d17/c22 0 2024-03-29T04:44:53.546 INFO:tasks.workunit.client.0.smithi142.stdout:3/80: readlink d3/d15/d17/d6/l16 0 2024-03-29T04:44:53.550 INFO:tasks.workunit.client.0.smithi142.stdout:4/79: creat d0/dc/d15/f18 x:0 0 0 2024-03-29T04:44:53.551 INFO:tasks.workunit.client.0.smithi142.stdout:8/117: link l4 d5/l26 0 2024-03-29T04:44:53.554 INFO:tasks.workunit.client.0.smithi142.stdout:1/107: link d8/db/dc/d19/d1d/d15/f16 d8/f2a 0 2024-03-29T04:44:53.555 INFO:tasks.workunit.client.0.smithi142.stdout:0/65: dwrite d7/d9/ff [0,4194304] 0 2024-03-29T04:44:53.556 INFO:tasks.workunit.client.0.smithi142.stdout:3/81: dwrite d3/d15/d17/fc [0,4194304] 0 2024-03-29T04:44:53.556 INFO:tasks.workunit.client.0.smithi142.stdout:0/66: stat d7/d9/cb 0 2024-03-29T04:44:53.558 INFO:tasks.workunit.client.0.smithi142.stdout:6/84: rename l10 to l13 0 2024-03-29T04:44:53.559 INFO:tasks.workunit.client.0.smithi142.stdout:4/80: mkdir d0/d9/d19 0 2024-03-29T04:44:53.559 INFO:tasks.workunit.client.0.smithi142.stdout:8/118: symlink d5/d1d/l27 0 2024-03-29T04:44:53.560 INFO:tasks.workunit.client.0.smithi142.stdout:5/149: write f7 [1435969,41533] 0 2024-03-29T04:44:53.561 INFO:tasks.workunit.client.0.smithi142.stdout:0/67: stat d7/l12 0 2024-03-29T04:44:53.562 INFO:tasks.workunit.client.0.smithi142.stdout:8/119: read - d5/d10/f20 zero size 2024-03-29T04:44:53.563 INFO:tasks.workunit.client.0.smithi142.stdout:8/120: chown d5/d1d 8148 1 2024-03-29T04:44:53.567 INFO:tasks.workunit.client.0.smithi142.stdout:0/68: symlink d7/d13/l18 0 2024-03-29T04:44:53.569 INFO:tasks.workunit.client.0.smithi142.stdout:3/82: getdents d3/d15/d17/d6 0 2024-03-29T04:44:53.570 INFO:tasks.workunit.client.0.smithi142.stdout:5/150: mkdir df/d1a/d21/d27 0 2024-03-29T04:44:53.571 INFO:tasks.workunit.client.0.smithi142.stdout:4/81: dwrite d0/f8 [0,4194304] 0 2024-03-29T04:44:53.573 INFO:tasks.workunit.client.0.smithi142.stdout:3/83: write d3/fe [2255218,30114] 0 2024-03-29T04:44:53.574 INFO:tasks.workunit.client.0.smithi142.stdout:8/121: dread d5/f19 [0,4194304] 0 2024-03-29T04:44:53.574 INFO:tasks.workunit.client.0.smithi142.stdout:5/151: mkdir df/d1a/d28 0 2024-03-29T04:44:53.581 INFO:tasks.workunit.client.0.smithi142.stdout:8/122: truncate d5/d10/f20 657643 0 2024-03-29T04:44:53.582 INFO:tasks.workunit.client.0.smithi142.stdout:3/84: rename d3/d15/d17/fc to d3/d1f/f23 0 2024-03-29T04:44:53.584 INFO:tasks.workunit.client.0.smithi142.stdout:0/69: dwrite d7/d9/fa [4194304,4194304] 0 2024-03-29T04:44:53.587 INFO:tasks.workunit.client.0.smithi142.stdout:8/123: truncate d5/f1f 537756 0 2024-03-29T04:44:53.588 INFO:tasks.workunit.client.0.smithi142.stdout:0/70: fdatasync d7/d9/d15/f16 0 2024-03-29T04:44:53.590 INFO:tasks.workunit.client.0.smithi142.stdout:4/82: dwrite d0/dc/d15/f18 [0,4194304] 0 2024-03-29T04:44:53.590 INFO:tasks.workunit.client.0.smithi142.stdout:5/152: truncate fa 1139023 0 2024-03-29T04:44:53.590 INFO:tasks.workunit.client.0.smithi142.stdout:5/153: stat df/d1a 0 2024-03-29T04:44:53.592 INFO:tasks.workunit.client.0.smithi142.stdout:8/124: rename d5/d10/c11 to d5/d1d/c28 0 2024-03-29T04:44:53.592 INFO:tasks.workunit.client.0.smithi142.stdout:3/85: write d3/d15/f21 [2871613,26042] 0 2024-03-29T04:44:53.595 INFO:tasks.workunit.client.0.smithi142.stdout:0/71: symlink d7/l19 0 2024-03-29T04:44:53.596 INFO:tasks.workunit.client.0.smithi142.stdout:0/72: truncate d7/d9/f10 9386028 0 2024-03-29T04:44:53.596 INFO:tasks.workunit.client.0.smithi142.stdout:3/86: mkdir d3/d1f/d24 0 2024-03-29T04:44:53.599 INFO:tasks.workunit.client.0.smithi142.stdout:4/83: creat d0/d9/d19/f1a x:0 0 0 2024-03-29T04:44:53.599 INFO:tasks.workunit.client.0.smithi142.stdout:5/154: mkdir df/d29 0 2024-03-29T04:44:53.599 INFO:tasks.workunit.client.0.smithi142.stdout:4/84: write d0/f7 [532003,42050] 0 2024-03-29T04:44:53.601 INFO:tasks.workunit.client.0.smithi142.stdout:3/87: mknod d3/d15/d17/d6/c25 0 2024-03-29T04:44:53.604 INFO:tasks.workunit.client.0.smithi142.stdout:0/73: symlink d7/l1a 0 2024-03-29T04:44:53.604 INFO:tasks.workunit.client.0.smithi142.stdout:0/74: fdatasync d7/d9/f10 0 2024-03-29T04:44:53.605 INFO:tasks.workunit.client.0.smithi142.stdout:3/88: creat d3/d15/f26 x:0 0 0 2024-03-29T04:44:53.606 INFO:tasks.workunit.client.0.smithi142.stdout:5/155: creat df/d1a/d21/d27/f2a x:0 0 0 2024-03-29T04:44:53.611 INFO:tasks.workunit.client.0.smithi142.stdout:5/156: write df/f10 [2724685,122760] 0 2024-03-29T04:44:53.613 INFO:tasks.workunit.client.0.smithi142.stdout:3/89: rename d3/fa to d3/d15/d17/f27 0 2024-03-29T04:44:53.614 INFO:tasks.workunit.client.0.smithi142.stdout:4/85: dread d0/f7 [0,4194304] 0 2024-03-29T04:44:53.616 INFO:tasks.workunit.client.0.smithi142.stdout:0/75: dwrite d7/d9/d15/f16 [0,4194304] 0 2024-03-29T04:44:53.616 INFO:tasks.workunit.client.0.smithi142.stdout:3/90: write d3/d15/f21 [627353,90806] 0 2024-03-29T04:44:53.618 INFO:tasks.workunit.client.0.smithi142.stdout:4/86: chown d0/d9/fb 473354 1 2024-03-29T04:44:53.619 INFO:tasks.workunit.client.0.smithi142.stdout:8/125: write d5/f22 [5550781,88046] 0 2024-03-29T04:44:53.621 INFO:tasks.workunit.client.0.smithi142.stdout:4/87: readlink d0/d9/d11/l16 0 2024-03-29T04:44:53.621 INFO:tasks.workunit.client.0.smithi142.stdout:0/76: rmdir d7/d9 39 2024-03-29T04:44:53.623 INFO:tasks.workunit.client.0.smithi142.stdout:8/126: truncate d5/f1f 1179199 0 2024-03-29T04:44:53.623 INFO:tasks.workunit.client.0.smithi142.stdout:5/157: dwrite fe [0,4194304] 0 2024-03-29T04:44:53.624 INFO:tasks.workunit.client.0.smithi142.stdout:0/77: stat d7/d9/fa 0 2024-03-29T04:44:53.624 INFO:tasks.workunit.client.0.smithi142.stdout:0/78: fsync f2 0 2024-03-29T04:44:53.625 INFO:tasks.workunit.client.0.smithi142.stdout:0/79: stat d7/l1a 0 2024-03-29T04:44:53.626 INFO:tasks.workunit.client.0.smithi142.stdout:0/80: read d7/d9/f10 [8391070,51429] 0 2024-03-29T04:44:53.628 INFO:tasks.workunit.client.0.smithi142.stdout:0/81: write d7/d9/fa [5787627,61972] 0 2024-03-29T04:44:53.628 INFO:tasks.workunit.client.0.smithi142.stdout:4/88: creat d0/d9/d11/f1b x:0 0 0 2024-03-29T04:44:53.629 INFO:tasks.workunit.client.0.smithi142.stdout:5/158: chown df/l19 82360953 1 2024-03-29T04:44:53.631 INFO:tasks.workunit.client.0.smithi142.stdout:9/69: sync 2024-03-29T04:44:53.634 INFO:tasks.workunit.client.0.smithi142.stdout:4/89: symlink d0/d9/d19/l1c 0 2024-03-29T04:44:53.637 INFO:tasks.workunit.client.0.smithi142.stdout:8/127: mkdir d5/d29 0 2024-03-29T04:44:53.638 INFO:tasks.workunit.client.0.smithi142.stdout:3/91: rename d3/d15/d17/c13 to d3/d15/d17/c28 0 2024-03-29T04:44:53.640 INFO:tasks.workunit.client.0.smithi142.stdout:0/82: rename d7/d9/d15/f16 to d7/d9/d15/f1b 0 2024-03-29T04:44:53.640 INFO:tasks.workunit.client.0.smithi142.stdout:3/92: write d3/d15/d17/f11 [214051,88708] 0 2024-03-29T04:44:53.641 INFO:tasks.workunit.client.0.smithi142.stdout:0/83: fsync f2 0 2024-03-29T04:44:53.641 INFO:tasks.workunit.client.0.smithi142.stdout:5/159: mknod df/c2b 0 2024-03-29T04:44:53.642 INFO:tasks.workunit.client.0.smithi142.stdout:9/70: symlink d0/d2/l14 0 2024-03-29T04:44:53.643 INFO:tasks.workunit.client.0.smithi142.stdout:8/128: unlink d5/d1d/l27 0 2024-03-29T04:44:53.644 INFO:tasks.workunit.client.0.smithi142.stdout:4/90: mknod d0/d12/c1d 0 2024-03-29T04:44:53.644 INFO:tasks.workunit.client.0.smithi142.stdout:3/93: mkdir d3/d29 0 2024-03-29T04:44:53.646 INFO:tasks.workunit.client.0.smithi142.stdout:9/71: creat d0/f15 x:0 0 0 2024-03-29T04:44:53.647 INFO:tasks.workunit.client.0.smithi142.stdout:8/129: symlink d5/d10/l2a 0 2024-03-29T04:44:53.648 INFO:tasks.workunit.client.0.smithi142.stdout:4/91: rmdir d0/dc/d15 39 2024-03-29T04:44:53.650 INFO:tasks.workunit.client.0.smithi142.stdout:3/94: fsync d3/d1f/f23 0 2024-03-29T04:44:53.650 INFO:tasks.workunit.client.0.smithi142.stdout:5/160: mkdir df/d2c 0 2024-03-29T04:44:53.653 INFO:tasks.workunit.client.0.smithi142.stdout:0/84: link d7/d9/f10 d7/d9/f1c 0 2024-03-29T04:44:53.653 INFO:tasks.workunit.client.0.smithi142.stdout:9/72: mkdir d0/d2/d6/d16 0 2024-03-29T04:44:53.654 INFO:tasks.workunit.client.0.smithi142.stdout:8/130: fsync d5/f19 0 2024-03-29T04:44:53.654 INFO:tasks.workunit.client.0.smithi142.stdout:9/73: fsync d0/d12/f13 0 2024-03-29T04:44:53.655 INFO:tasks.workunit.client.0.smithi142.stdout:4/92: creat d0/d12/f1e x:0 0 0 2024-03-29T04:44:53.656 INFO:tasks.workunit.client.0.smithi142.stdout:3/95: fdatasync d3/d15/d17/f27 0 2024-03-29T04:44:53.657 INFO:tasks.workunit.client.0.smithi142.stdout:0/85: symlink d7/d9/d15/l1d 0 2024-03-29T04:44:53.659 INFO:tasks.workunit.client.0.smithi142.stdout:4/93: symlink d0/d12/l1f 0 2024-03-29T04:44:53.661 INFO:tasks.workunit.client.0.smithi142.stdout:3/96: mknod d3/d29/c2a 0 2024-03-29T04:44:53.661 INFO:tasks.workunit.client.0.smithi142.stdout:5/161: dwrite df/d16/f1e [0,4194304] 0 2024-03-29T04:44:53.664 INFO:tasks.workunit.client.0.smithi142.stdout:5/162: chown f3 0 1 2024-03-29T04:44:53.664 INFO:tasks.workunit.client.0.smithi142.stdout:9/74: dread d0/d2/d6/fd [0,4194304] 0 2024-03-29T04:44:53.665 INFO:tasks.workunit.client.0.smithi142.stdout:8/131: dwrite d5/f22 [0,4194304] 0 2024-03-29T04:44:53.668 INFO:tasks.workunit.client.0.smithi142.stdout:4/94: getdents d0/dc 0 2024-03-29T04:44:53.669 INFO:tasks.workunit.client.0.smithi142.stdout:3/97: rename d3/d15 to d3/d29/d2b 0 2024-03-29T04:44:53.673 INFO:tasks.workunit.client.0.smithi142.stdout:4/95: truncate d0/d9/d19/f1a 295517 0 2024-03-29T04:44:53.673 INFO:tasks.workunit.client.0.smithi142.stdout:3/98: write d3/d29/d2b/d17/f12 [2467951,105449] 0 2024-03-29T04:44:53.675 INFO:tasks.workunit.client.0.smithi142.stdout:5/163: dwrite df/f10 [0,4194304] 0 2024-03-29T04:44:53.676 INFO:tasks.workunit.client.0.smithi142.stdout:5/164: stat fb 0 2024-03-29T04:44:53.677 INFO:tasks.workunit.client.0.smithi142.stdout:8/132: dwrite d5/f18 [4194304,4194304] 0 2024-03-29T04:44:53.677 INFO:tasks.workunit.client.0.smithi142.stdout:4/96: dread - d0/d12/f1e zero size 2024-03-29T04:44:53.678 INFO:tasks.workunit.client.0.smithi142.stdout:5/165: chown l5 8586460 1 2024-03-29T04:44:53.678 INFO:tasks.workunit.client.0.smithi142.stdout:9/75: truncate d0/d2/d6/fd 861714 0 2024-03-29T04:44:53.679 INFO:tasks.workunit.client.0.smithi142.stdout:4/97: fdatasync d0/dc/d15/f18 0 2024-03-29T04:44:53.680 INFO:tasks.workunit.client.0.smithi142.stdout:9/76: dread - d0/d12/f13 zero size 2024-03-29T04:44:53.684 INFO:tasks.workunit.client.0.smithi142.stdout:9/77: write d0/d12/f13 [292695,114130] 0 2024-03-29T04:44:53.685 INFO:tasks.workunit.client.0.smithi142.stdout:3/99: mknod d3/d29/d2b/d17/c2c 0 2024-03-29T04:44:53.688 INFO:tasks.workunit.client.0.smithi142.stdout:0/86: dwrite d7/d9/f10 [8388608,4194304] 0 2024-03-29T04:44:53.689 INFO:tasks.workunit.client.0.smithi142.stdout:3/100: chown d3/d29/d2b/f26 868 1 2024-03-29T04:44:53.693 INFO:tasks.workunit.client.0.smithi142.stdout:8/133: dwrite d5/f1f [0,4194304] 0 2024-03-29T04:44:53.696 INFO:tasks.workunit.client.0.smithi142.stdout:3/101: chown d3/d29/d2b/d17/d6/c25 4 1 2024-03-29T04:44:53.696 INFO:tasks.workunit.client.0.smithi142.stdout:0/87: mkdir d7/d9/d1e 0 2024-03-29T04:44:53.696 INFO:tasks.workunit.client.0.smithi142.stdout:8/134: write d5/d10/f20 [1219870,59276] 0 2024-03-29T04:44:53.700 INFO:tasks.workunit.client.0.smithi142.stdout:3/102: chown d3/d29/d2b/d17/f27 1 1 2024-03-29T04:44:53.701 INFO:tasks.workunit.client.0.smithi142.stdout:9/78: dwrite d0/f15 [0,4194304] 0 2024-03-29T04:44:53.702 INFO:tasks.workunit.client.0.smithi142.stdout:3/103: write d3/d29/d2b/f21 [2788115,32879] 0 2024-03-29T04:44:53.704 INFO:tasks.workunit.client.0.smithi142.stdout:8/135: creat d5/d1d/d24/f2b x:0 0 0 2024-03-29T04:44:53.704 INFO:tasks.workunit.client.0.smithi142.stdout:9/79: truncate d0/d2/fe 279198 0 2024-03-29T04:44:53.705 INFO:tasks.workunit.client.0.smithi142.stdout:8/136: chown d5/d10/l14 475884244 1 2024-03-29T04:44:53.706 INFO:tasks.workunit.client.0.smithi142.stdout:0/88: dread d7/d9/f10 [8388608,4194304] 0 2024-03-29T04:44:53.706 INFO:tasks.workunit.client.0.smithi142.stdout:0/89: stat d7 0 2024-03-29T04:44:53.707 INFO:tasks.workunit.client.0.smithi142.stdout:3/104: readlink d3/d29/d2b/d17/l10 0 2024-03-29T04:44:53.709 INFO:tasks.workunit.client.0.smithi142.stdout:9/80: symlink d0/d2/d6/l17 0 2024-03-29T04:44:53.710 INFO:tasks.workunit.client.0.smithi142.stdout:8/137: symlink d5/d1d/d24/l2c 0 2024-03-29T04:44:53.711 INFO:tasks.workunit.client.0.smithi142.stdout:0/90: symlink d7/d9/l1f 0 2024-03-29T04:44:53.715 INFO:tasks.workunit.client.0.smithi142.stdout:9/81: mknod d0/c18 0 2024-03-29T04:44:53.718 INFO:tasks.workunit.client.0.smithi142.stdout:8/138: unlink d5/d10/l1b 0 2024-03-29T04:44:53.722 INFO:tasks.workunit.client.0.smithi142.stdout:8/139: mkdir d5/d10/d2d 0 2024-03-29T04:44:53.727 INFO:tasks.workunit.client.0.smithi142.stdout:8/140: mkdir d5/d2e 0 2024-03-29T04:44:53.727 INFO:tasks.workunit.client.0.smithi142.stdout:2/65: sync 2024-03-29T04:44:53.727 INFO:tasks.workunit.client.0.smithi142.stdout:0/91: getdents d7/d9/d15 0 2024-03-29T04:44:53.728 INFO:tasks.workunit.client.0.smithi142.stdout:2/66: rmdir de/d11 39 2024-03-29T04:44:53.728 INFO:tasks.workunit.client.0.smithi142.stdout:2/67: readlink l4 0 2024-03-29T04:44:53.728 INFO:tasks.workunit.client.0.smithi142.stdout:2/68: fdatasync fd 0 2024-03-29T04:44:53.729 INFO:tasks.workunit.client.0.smithi142.stdout:8/141: rename d5/d29 to d5/d1d/d2f 0 2024-03-29T04:44:53.730 INFO:tasks.workunit.client.0.smithi142.stdout:0/92: creat d7/d13/f20 x:0 0 0 2024-03-29T04:44:53.732 INFO:tasks.workunit.client.0.smithi142.stdout:3/105: rmdir d3 39 2024-03-29T04:44:53.733 INFO:tasks.workunit.client.0.smithi142.stdout:0/93: fdatasync d7/d9/f10 0 2024-03-29T04:44:53.735 INFO:tasks.workunit.client.0.smithi142.stdout:5/166: truncate f7 729355 0 2024-03-29T04:44:53.735 INFO:tasks.workunit.client.0.smithi142.stdout:4/98: write d0/f10 [2732132,98032] 0 2024-03-29T04:44:53.739 INFO:tasks.workunit.client.0.smithi142.stdout:4/99: mknod d0/d12/c20 0 2024-03-29T04:44:53.739 INFO:tasks.workunit.client.0.smithi142.stdout:8/142: dwrite d5/d10/f20 [0,4194304] 0 2024-03-29T04:44:53.740 INFO:tasks.workunit.client.0.smithi142.stdout:3/106: mkdir d3/d29/d2d 0 2024-03-29T04:44:53.740 INFO:tasks.workunit.client.0.smithi142.stdout:4/100: chown d0/d9/d11/f1b 749 1 2024-03-29T04:44:53.742 INFO:tasks.workunit.client.0.smithi142.stdout:0/94: dread f2 [0,4194304] 0 2024-03-29T04:44:53.744 INFO:tasks.workunit.client.0.smithi142.stdout:0/95: read f2 [3258046,92664] 0 2024-03-29T04:44:53.745 INFO:tasks.workunit.client.0.smithi142.stdout:5/167: creat df/d1a/d28/f2d x:0 0 0 2024-03-29T04:44:53.745 INFO:tasks.workunit.client.0.smithi142.stdout:4/101: creat d0/f21 x:0 0 0 2024-03-29T04:44:53.749 INFO:tasks.workunit.client.0.smithi142.stdout:8/143: dwrite d5/f1f [4194304,4194304] 0 2024-03-29T04:44:53.749 INFO:tasks.workunit.client.0.smithi142.stdout:4/102: dread - d0/d12/f1e zero size 2024-03-29T04:44:53.753 INFO:tasks.workunit.client.0.smithi142.stdout:0/96: creat d7/f21 x:0 0 0 2024-03-29T04:44:53.754 INFO:tasks.workunit.client.0.smithi142.stdout:5/168: mkdir df/d16/d24/d2e 0 2024-03-29T04:44:53.756 INFO:tasks.workunit.client.0.smithi142.stdout:2/69: write f8 [318454,94029] 0 2024-03-29T04:44:53.758 INFO:tasks.workunit.client.0.smithi142.stdout:3/107: dwrite d3/d29/d2b/d17/d6/f20 [0,4194304] 0 2024-03-29T04:44:53.775 INFO:tasks.workunit.client.0.smithi142.stdout:8/144: truncate d5/d1d/d24/f2b 944418 0 2024-03-29T04:44:53.776 INFO:tasks.workunit.client.0.smithi142.stdout:5/169: creat df/d1a/f2f x:0 0 0 2024-03-29T04:44:53.778 INFO:tasks.workunit.client.0.smithi142.stdout:8/145: read d5/f22 [4703455,12166] 0 2024-03-29T04:44:53.778 INFO:tasks.workunit.client.0.smithi142.stdout:6/85: sync 2024-03-29T04:44:53.778 INFO:tasks.workunit.client.0.smithi142.stdout:1/108: sync 2024-03-29T04:44:53.778 INFO:tasks.workunit.client.0.smithi142.stdout:7/80: sync 2024-03-29T04:44:53.779 INFO:tasks.workunit.client.0.smithi142.stdout:4/103: truncate d0/d9/f13 330855 0 2024-03-29T04:44:53.780 INFO:tasks.workunit.client.0.smithi142.stdout:9/82: dwrite d0/d2/d6/fd [0,4194304] 0 2024-03-29T04:44:53.781 INFO:tasks.workunit.client.0.smithi142.stdout:1/109: fdatasync d8/f9 0 2024-03-29T04:44:53.783 INFO:tasks.workunit.client.0.smithi142.stdout:4/104: rename d0/d9/d19 to d0/d9/d19/d22 22 2024-03-29T04:44:53.785 INFO:tasks.workunit.client.0.smithi142.stdout:0/97: dwrite d7/d9/fa [8388608,4194304] 0 2024-03-29T04:44:53.785 INFO:tasks.workunit.client.0.smithi142.stdout:4/105: chown d0/f14 0 1 2024-03-29T04:44:53.790 INFO:tasks.workunit.client.0.smithi142.stdout:7/81: dread f1 [0,4194304] 0 2024-03-29T04:44:53.791 INFO:tasks.workunit.client.0.smithi142.stdout:8/146: dwrite d5/f18 [4194304,4194304] 0 2024-03-29T04:44:53.799 INFO:tasks.workunit.client.0.smithi142.stdout:5/170: dread f3 [0,4194304] 0 2024-03-29T04:44:53.804 INFO:tasks.workunit.client.0.smithi142.stdout:5/171: dwrite fb [0,4194304] 0 2024-03-29T04:44:53.808 INFO:tasks.workunit.client.0.smithi142.stdout:5/172: dread fb [0,4194304] 0 2024-03-29T04:44:53.931 INFO:tasks.workunit.client.0.smithi142.stdout:3/108: unlink l2 0 2024-03-29T04:44:53.934 INFO:tasks.workunit.client.0.smithi142.stdout:6/86: creat f14 x:0 0 0 2024-03-29T04:44:53.934 INFO:tasks.workunit.client.0.smithi142.stdout:6/87: rmdir - no directory 2024-03-29T04:44:53.935 INFO:tasks.workunit.client.0.smithi142.stdout:3/109: dread d3/d29/d2b/f21 [0,4194304] 0 2024-03-29T04:44:53.935 INFO:tasks.workunit.client.0.smithi142.stdout:6/88: write fb [1021453,112258] 0 2024-03-29T04:44:53.936 INFO:tasks.workunit.client.0.smithi142.stdout:3/110: write d3/d29/d2b/d17/f11 [239552,55599] 0 2024-03-29T04:44:53.937 INFO:tasks.workunit.client.0.smithi142.stdout:1/110: creat d8/f2b x:0 0 0 2024-03-29T04:44:53.938 INFO:tasks.workunit.client.0.smithi142.stdout:4/106: creat d0/d9/d11/f23 x:0 0 0 2024-03-29T04:44:53.940 INFO:tasks.workunit.client.0.smithi142.stdout:7/82: write d7/f9 [2969002,86876] 0 2024-03-29T04:44:53.944 INFO:tasks.workunit.client.0.smithi142.stdout:5/173: rename df to df/d30 22 2024-03-29T04:44:53.944 INFO:tasks.workunit.client.0.smithi142.stdout:3/111: dwrite d3/d29/d2b/d17/d6/f14 [0,4194304] 0 2024-03-29T04:44:53.946 INFO:tasks.workunit.client.0.smithi142.stdout:1/111: dwrite d8/db/dc/f1c [0,4194304] 0 2024-03-29T04:44:53.946 INFO:tasks.workunit.client.0.smithi142.stdout:2/70: creat de/d11/f15 x:0 0 0 2024-03-29T04:44:53.947 INFO:tasks.workunit.client.0.smithi142.stdout:5/174: write df/d1a/d28/f2d [7723,25620] 0 2024-03-29T04:44:53.947 INFO:tasks.workunit.client.0.smithi142.stdout:1/112: fsync f7 0 2024-03-29T04:44:53.947 INFO:tasks.workunit.client.0.smithi142.stdout:3/112: write d3/fe [631698,11635] 0 2024-03-29T04:44:53.951 INFO:tasks.workunit.client.0.smithi142.stdout:6/89: fdatasync f4 0 2024-03-29T04:44:53.957 INFO:tasks.workunit.client.0.smithi142.stdout:1/113: dwrite d8/db/dc/d19/d1d/d15/d21/f26 [0,4194304] 0 2024-03-29T04:44:53.962 INFO:tasks.workunit.client.0.smithi142.stdout:4/107: symlink d0/d9/l24 0 2024-03-29T04:44:53.962 INFO:tasks.workunit.client.0.smithi142.stdout:4/108: stat d0/f7 0 2024-03-29T04:44:53.962 INFO:tasks.workunit.client.0.smithi142.stdout:4/109: chown d0/f8 2673 1 2024-03-29T04:44:53.963 INFO:tasks.workunit.client.0.smithi142.stdout:7/83: creat d7/dc/de/f16 x:0 0 0 2024-03-29T04:44:53.971 INFO:tasks.workunit.client.0.smithi142.stdout:5/175: creat df/d1a/f31 x:0 0 0 2024-03-29T04:44:53.972 INFO:tasks.workunit.client.0.smithi142.stdout:5/176: read - df/d1a/f1f zero size 2024-03-29T04:44:53.973 INFO:tasks.workunit.client.0.smithi142.stdout:5/177: dread - df/d1a/d21/d27/f2a zero size 2024-03-29T04:44:53.973 INFO:tasks.workunit.client.0.smithi142.stdout:3/113: rename d3/fe to d3/d29/f2e 0 2024-03-29T04:44:53.976 INFO:tasks.workunit.client.0.smithi142.stdout:6/90: chown la 84048393 1 2024-03-29T04:44:53.980 INFO:tasks.workunit.client.0.smithi142.stdout:1/114: unlink d8/db/dc/c1e 0 2024-03-29T04:44:53.981 INFO:tasks.workunit.client.0.smithi142.stdout:1/115: write d8/db/f20 [99079,67855] 0 2024-03-29T04:44:53.983 INFO:tasks.workunit.client.0.smithi142.stdout:8/147: creat d5/f30 x:0 0 0 2024-03-29T04:44:53.984 INFO:tasks.workunit.client.0.smithi142.stdout:4/110: creat d0/d9/d11/f25 x:0 0 0 2024-03-29T04:44:53.985 INFO:tasks.workunit.client.0.smithi142.stdout:4/111: stat d0/d12/c20 0 2024-03-29T04:44:53.986 INFO:tasks.workunit.client.0.smithi142.stdout:4/112: dread - d0/d9/d11/f1b zero size 2024-03-29T04:44:53.987 INFO:tasks.workunit.client.0.smithi142.stdout:4/113: fsync d0/dc/f17 0 2024-03-29T04:44:53.987 INFO:tasks.workunit.client.0.smithi142.stdout:1/116: dwrite d8/db/dc/d19/d1d/f13 [0,4194304] 0 2024-03-29T04:44:53.988 INFO:tasks.workunit.client.0.smithi142.stdout:4/114: dread - d0/d9/d11/f1b zero size 2024-03-29T04:44:53.989 INFO:tasks.workunit.client.0.smithi142.stdout:3/114: creat d3/d29/d2b/d17/d6/f2f x:0 0 0 2024-03-29T04:44:53.989 INFO:tasks.workunit.client.0.smithi142.stdout:3/115: chown d3/d29 6 1 2024-03-29T04:44:53.991 INFO:tasks.workunit.client.0.smithi142.stdout:6/91: symlink l15 0 2024-03-29T04:44:53.991 INFO:tasks.workunit.client.0.smithi142.stdout:9/83: link d0/c8 d0/c19 0 2024-03-29T04:44:53.991 INFO:tasks.workunit.client.0.smithi142.stdout:6/92: write f14 [805605,105683] 0 2024-03-29T04:44:53.993 INFO:tasks.workunit.client.0.smithi142.stdout:0/98: getdents d7/d13 0 2024-03-29T04:44:53.994 INFO:tasks.workunit.client.0.smithi142.stdout:6/93: dread fb [0,4194304] 0 2024-03-29T04:44:53.995 INFO:tasks.workunit.client.0.smithi142.stdout:3/116: dread d3/d29/d2b/d17/f27 [0,4194304] 0 2024-03-29T04:44:53.995 INFO:tasks.workunit.client.0.smithi142.stdout:6/94: chown fb 170954 1 2024-03-29T04:44:53.996 INFO:tasks.workunit.client.0.smithi142.stdout:8/148: stat d5/d1d/d24/l23 0 2024-03-29T04:44:53.996 INFO:tasks.workunit.client.0.smithi142.stdout:3/117: dread - d3/d29/d2b/d17/d6/f1c zero size 2024-03-29T04:44:53.997 INFO:tasks.workunit.client.0.smithi142.stdout:3/118: chown d3/d29/d2b/c1d 507074473 1 2024-03-29T04:44:53.997 INFO:tasks.workunit.client.0.smithi142.stdout:2/71: creat de/f16 x:0 0 0 2024-03-29T04:44:53.998 INFO:tasks.workunit.client.0.smithi142.stdout:3/119: read d3/d29/d2b/d17/f27 [2402728,31663] 0 2024-03-29T04:44:53.998 INFO:tasks.workunit.client.0.smithi142.stdout:2/72: dread - de/f16 zero size 2024-03-29T04:44:53.999 INFO:tasks.workunit.client.0.smithi142.stdout:1/117: write f3 [1176552,5213] 0 2024-03-29T04:44:54.000 INFO:tasks.workunit.client.0.smithi142.stdout:4/115: symlink d0/dc/l26 0 2024-03-29T04:44:54.002 INFO:tasks.workunit.client.0.smithi142.stdout:9/84: unlink d0/c3 0 2024-03-29T04:44:54.005 INFO:tasks.workunit.client.0.smithi142.stdout:6/95: symlink l16 0 2024-03-29T04:44:54.007 INFO:tasks.workunit.client.0.smithi142.stdout:3/120: symlink d3/l30 0 2024-03-29T04:44:54.008 INFO:tasks.workunit.client.0.smithi142.stdout:3/121: stat d3/d29/d2b/d17/d6/l1e 0 2024-03-29T04:44:54.009 INFO:tasks.workunit.client.0.smithi142.stdout:2/73: mkdir de/d11/d17 0 2024-03-29T04:44:54.012 INFO:tasks.workunit.client.0.smithi142.stdout:9/85: symlink d0/d2/d6/l1a 0 2024-03-29T04:44:54.013 INFO:tasks.workunit.client.0.smithi142.stdout:9/86: dread d0/d2/fe [0,4194304] 0 2024-03-29T04:44:54.014 INFO:tasks.workunit.client.0.smithi142.stdout:0/99: truncate d7/d9/d15/f1b 898859 0 2024-03-29T04:44:54.015 INFO:tasks.workunit.client.0.smithi142.stdout:6/96: fsync f4 0 2024-03-29T04:44:54.017 INFO:tasks.workunit.client.0.smithi142.stdout:0/100: dread d7/d9/fa [4194304,4194304] 0 2024-03-29T04:44:54.018 INFO:tasks.workunit.client.0.smithi142.stdout:7/84: rename d7/l11 to d7/dc/l17 0 2024-03-29T04:44:54.018 INFO:tasks.workunit.client.0.smithi142.stdout:0/101: read - d7/f21 zero size 2024-03-29T04:44:54.018 INFO:tasks.workunit.client.0.smithi142.stdout:7/85: chown d7/dc/de/f14 444501175 1 2024-03-29T04:44:54.019 INFO:tasks.workunit.client.0.smithi142.stdout:0/102: write d7/d9/ff [578998,103689] 0 2024-03-29T04:44:54.020 INFO:tasks.workunit.client.0.smithi142.stdout:0/103: write d7/d9/f10 [3495265,47722] 0 2024-03-29T04:44:54.021 INFO:tasks.workunit.client.0.smithi142.stdout:9/87: creat d0/d2/d6/f1b x:0 0 0 2024-03-29T04:44:54.024 INFO:tasks.workunit.client.0.smithi142.stdout:3/122: symlink d3/d1f/d24/l31 0 2024-03-29T04:44:54.026 INFO:tasks.workunit.client.0.smithi142.stdout:8/149: rename d5/l26 to d5/d1d/d2f/l31 0 2024-03-29T04:44:54.026 INFO:tasks.workunit.client.0.smithi142.stdout:2/74: mknod de/d11/d17/c18 0 2024-03-29T04:44:54.028 INFO:tasks.workunit.client.0.smithi142.stdout:1/118: creat d8/db/dc/f2c x:0 0 0 2024-03-29T04:44:54.028 INFO:tasks.workunit.client.0.smithi142.stdout:8/150: write d5/d1d/d24/f2b [1800183,99838] 0 2024-03-29T04:44:54.031 INFO:tasks.workunit.client.0.smithi142.stdout:3/123: dwrite d3/d29/d2b/d17/f11 [0,4194304] 0 2024-03-29T04:44:54.032 INFO:tasks.workunit.client.0.smithi142.stdout:9/88: unlink d0/c18 0 2024-03-29T04:44:54.033 INFO:tasks.workunit.client.0.smithi142.stdout:3/124: write d3/d29/d2b/d17/d6/f1c [973216,103323] 0 2024-03-29T04:44:54.043 INFO:tasks.workunit.client.0.smithi142.stdout:9/89: creat d0/d2/d6/d16/f1c x:0 0 0 2024-03-29T04:44:54.121 INFO:tasks.workunit.client.0.smithi142.stdout:5/178: sync 2024-03-29T04:44:54.122 INFO:tasks.workunit.client.0.smithi142.stdout:5/179: dread df/d1a/d28/f2d [0,4194304] 0 2024-03-29T04:44:54.125 INFO:tasks.workunit.client.0.smithi142.stdout:5/180: truncate f3 4154493 0 2024-03-29T04:44:54.126 INFO:tasks.workunit.client.0.smithi142.stdout:8/151: truncate d5/f22 5083229 0 2024-03-29T04:44:54.127 INFO:tasks.workunit.client.0.smithi142.stdout:5/181: getdents df/d29 0 2024-03-29T04:44:54.127 INFO:tasks.workunit.client.0.smithi142.stdout:8/152: truncate d5/d10/f20 4371976 0 2024-03-29T04:44:54.127 INFO:tasks.workunit.client.0.smithi142.stdout:8/153: dread - d5/f25 zero size 2024-03-29T04:44:54.129 INFO:tasks.workunit.client.0.smithi142.stdout:5/182: symlink df/d16/d24/d2e/l32 0 2024-03-29T04:44:54.130 INFO:tasks.workunit.client.0.smithi142.stdout:5/183: stat df/d16/d24/d2e 0 2024-03-29T04:44:54.131 INFO:tasks.workunit.client.0.smithi142.stdout:8/154: dread d5/d10/f20 [0,4194304] 0 2024-03-29T04:44:54.131 INFO:tasks.workunit.client.0.smithi142.stdout:5/184: chown df/d16/f26 224703583 1 2024-03-29T04:44:54.135 INFO:tasks.workunit.client.0.smithi142.stdout:8/155: truncate d5/f19 418730 0 2024-03-29T04:44:54.142 INFO:tasks.workunit.client.0.smithi142.stdout:4/116: creat d0/dc/f27 x:0 0 0 2024-03-29T04:44:54.143 INFO:tasks.workunit.client.0.smithi142.stdout:4/117: chown d0/le 180868 1 2024-03-29T04:44:54.143 INFO:tasks.workunit.client.0.smithi142.stdout:4/118: readlink d0/dc/l26 0 2024-03-29T04:44:54.144 INFO:tasks.workunit.client.0.smithi142.stdout:4/119: stat d0/d9/d11/f25 0 2024-03-29T04:44:54.144 INFO:tasks.workunit.client.0.smithi142.stdout:6/97: truncate ff 232362 0 2024-03-29T04:44:54.146 INFO:tasks.workunit.client.0.smithi142.stdout:0/104: rename d7/d9/d15 to d7/d9/d22 0 2024-03-29T04:44:54.147 INFO:tasks.workunit.client.0.smithi142.stdout:0/105: write d7/d9/f10 [4776453,480] 0 2024-03-29T04:44:54.149 INFO:tasks.workunit.client.0.smithi142.stdout:7/86: readlink d7/dc/l17 0 2024-03-29T04:44:54.149 INFO:tasks.workunit.client.0.smithi142.stdout:7/87: readlink d7/l13 0 2024-03-29T04:44:54.149 INFO:tasks.workunit.client.0.smithi142.stdout:6/98: dwrite fc [4194304,4194304] 0 2024-03-29T04:44:54.150 INFO:tasks.workunit.client.0.smithi142.stdout:7/88: fdatasync d7/f9 0 2024-03-29T04:44:54.151 INFO:tasks.workunit.client.0.smithi142.stdout:7/89: write d7/dc/de/f16 [794561,10567] 0 2024-03-29T04:44:54.151 INFO:tasks.workunit.client.0.smithi142.stdout:0/106: creat d7/d9/d22/f23 x:0 0 0 2024-03-29T04:44:54.154 INFO:tasks.workunit.client.0.smithi142.stdout:0/107: mknod d7/d9/c24 0 2024-03-29T04:44:54.155 INFO:tasks.workunit.client.0.smithi142.stdout:7/90: dread f1 [0,4194304] 0 2024-03-29T04:44:54.160 INFO:tasks.workunit.client.0.smithi142.stdout:7/91: mkdir d7/d18 0 2024-03-29T04:44:54.165 INFO:tasks.workunit.client.0.smithi142.stdout:2/75: truncate f9 542047 0 2024-03-29T04:44:54.165 INFO:tasks.workunit.client.0.smithi142.stdout:9/90: getdents d0/d2/d6/d16 0 2024-03-29T04:44:54.166 INFO:tasks.workunit.client.0.smithi142.stdout:3/125: truncate d3/d29/d2b/d17/f12 985533 0 2024-03-29T04:44:54.169 INFO:tasks.workunit.client.0.smithi142.stdout:9/91: dread d0/d2/f11 [0,4194304] 0 2024-03-29T04:44:54.171 INFO:tasks.workunit.client.0.smithi142.stdout:1/119: dwrite d8/db/dc/d19/d1d/d15/f16 [0,4194304] 0 2024-03-29T04:44:54.177 INFO:tasks.workunit.client.0.smithi142.stdout:8/156: truncate d5/f1f 3992523 0 2024-03-29T04:44:54.182 INFO:tasks.workunit.client.0.smithi142.stdout:1/120: mkdir d8/db/dc/d19/d1d/d2d 0 2024-03-29T04:44:54.185 INFO:tasks.workunit.client.0.smithi142.stdout:2/76: dwrite de/d11/f15 [0,4194304] 0 2024-03-29T04:44:54.186 INFO:tasks.workunit.client.0.smithi142.stdout:3/126: dwrite d3/d29/d2b/d17/f27 [0,4194304] 0 2024-03-29T04:44:54.186 INFO:tasks.workunit.client.0.smithi142.stdout:7/92: truncate f1 3101360 0 2024-03-29T04:44:54.188 INFO:tasks.workunit.client.0.smithi142.stdout:5/185: dwrite fb [0,4194304] 0 2024-03-29T04:44:54.190 INFO:tasks.workunit.client.0.smithi142.stdout:9/92: dread d0/d2/d6/fd [0,4194304] 0 2024-03-29T04:44:54.191 INFO:tasks.workunit.client.0.smithi142.stdout:5/186: dread - df/d16/f26 zero size 2024-03-29T04:44:54.194 INFO:tasks.workunit.client.0.smithi142.stdout:9/93: read d0/d2/f11 [211616,93744] 0 2024-03-29T04:44:54.194 INFO:tasks.workunit.client.0.smithi142.stdout:8/157: dread d5/f19 [0,4194304] 0 2024-03-29T04:44:54.194 INFO:tasks.workunit.client.0.smithi142.stdout:8/158: fsync d5/f30 0 2024-03-29T04:44:54.196 INFO:tasks.workunit.client.0.smithi142.stdout:1/121: mknod d8/db/dc/d19/d1d/d15/d21/c2e 0 2024-03-29T04:44:54.200 INFO:tasks.workunit.client.0.smithi142.stdout:5/187: mknod df/d16/c33 0 2024-03-29T04:44:54.201 INFO:tasks.workunit.client.0.smithi142.stdout:4/120: write d0/d9/f13 [534469,17872] 0 2024-03-29T04:44:54.205 INFO:tasks.workunit.client.0.smithi142.stdout:8/159: creat d5/d10/f32 x:0 0 0 2024-03-29T04:44:54.207 INFO:tasks.workunit.client.0.smithi142.stdout:1/122: mknod d8/db/dc/d19/d1d/c2f 0 2024-03-29T04:44:54.207 INFO:tasks.workunit.client.0.smithi142.stdout:6/99: link ff f17 0 2024-03-29T04:44:54.208 INFO:tasks.workunit.client.0.smithi142.stdout:3/127: mknod d3/d29/d2b/d17/c32 0 2024-03-29T04:44:54.209 INFO:tasks.workunit.client.0.smithi142.stdout:5/188: dwrite fd [0,4194304] 0 2024-03-29T04:44:54.209 INFO:tasks.workunit.client.0.smithi142.stdout:2/77: creat de/f19 x:0 0 0 2024-03-29T04:44:54.211 INFO:tasks.workunit.client.0.smithi142.stdout:1/123: dread d8/db/dc/d19/d1d/d15/f17 [0,4194304] 0 2024-03-29T04:44:54.215 INFO:tasks.workunit.client.0.smithi142.stdout:0/108: dread d7/d9/d22/f1b [0,4194304] 0 2024-03-29T04:44:54.215 INFO:tasks.workunit.client.0.smithi142.stdout:4/121: chown d0/f3 1070747 1 2024-03-29T04:44:54.215 INFO:tasks.workunit.client.0.smithi142.stdout:6/100: creat f18 x:0 0 0 2024-03-29T04:44:54.217 INFO:tasks.workunit.client.0.smithi142.stdout:6/101: stat la 0 2024-03-29T04:44:54.218 INFO:tasks.workunit.client.0.smithi142.stdout:2/78: creat de/d11/d17/f1a x:0 0 0 2024-03-29T04:44:54.222 INFO:tasks.workunit.client.0.smithi142.stdout:5/189: dwrite df/f10 [4194304,4194304] 0 2024-03-29T04:44:54.222 INFO:tasks.workunit.client.0.smithi142.stdout:3/128: dwrite d3/d29/d2b/d17/d6/f2f [0,4194304] 0 2024-03-29T04:44:54.224 INFO:tasks.workunit.client.0.smithi142.stdout:4/122: mkdir d0/d12/d28 0 2024-03-29T04:44:54.224 INFO:tasks.workunit.client.0.smithi142.stdout:3/129: fdatasync d3/d29/d2b/d17/f27 0 2024-03-29T04:44:54.224 INFO:tasks.workunit.client.0.smithi142.stdout:6/102: symlink l19 0 2024-03-29T04:44:54.225 INFO:tasks.workunit.client.0.smithi142.stdout:9/94: write d0/d2/fe [474901,118172] 0 2024-03-29T04:44:54.229 INFO:tasks.workunit.client.0.smithi142.stdout:3/130: mknod d3/d29/c33 0 2024-03-29T04:44:54.230 INFO:tasks.workunit.client.0.smithi142.stdout:3/131: fdatasync d3/d29/d2b/d17/f27 0 2024-03-29T04:44:54.232 INFO:tasks.workunit.client.0.smithi142.stdout:4/123: mknod d0/d9/d11/c29 0 2024-03-29T04:44:54.233 INFO:tasks.workunit.client.0.smithi142.stdout:3/132: dread - d3/d29/d2b/f26 zero size 2024-03-29T04:44:54.233 INFO:tasks.workunit.client.0.smithi142.stdout:7/93: write f1 [2551309,103181] 0 2024-03-29T04:44:54.236 INFO:tasks.workunit.client.0.smithi142.stdout:5/190: symlink df/l34 0 2024-03-29T04:44:54.236 INFO:tasks.workunit.client.0.smithi142.stdout:9/95: dwrite d0/d2/fe [0,4194304] 0 2024-03-29T04:44:54.236 INFO:tasks.workunit.client.0.smithi142.stdout:1/124: rmdir d8 39 2024-03-29T04:44:54.238 INFO:tasks.workunit.client.0.smithi142.stdout:5/191: dread - df/d1a/d21/d27/f2a zero size 2024-03-29T04:44:54.239 INFO:tasks.workunit.client.0.smithi142.stdout:8/160: truncate d5/f22 5635876 0 2024-03-29T04:44:54.241 INFO:tasks.workunit.client.0.smithi142.stdout:3/133: rmdir d3/d1f/d24 39 2024-03-29T04:44:54.242 INFO:tasks.workunit.client.0.smithi142.stdout:1/125: fdatasync f7 0 2024-03-29T04:44:54.243 INFO:tasks.workunit.client.0.smithi142.stdout:9/96: creat d0/f1d x:0 0 0 2024-03-29T04:44:54.245 INFO:tasks.workunit.client.0.smithi142.stdout:5/192: rename df/f23 to df/d1a/d28/f35 0 2024-03-29T04:44:54.246 INFO:tasks.workunit.client.0.smithi142.stdout:8/161: creat d5/d1d/d24/f33 x:0 0 0 2024-03-29T04:44:54.247 INFO:tasks.workunit.client.0.smithi142.stdout:4/124: symlink d0/dc/d15/l2a 0 2024-03-29T04:44:54.248 INFO:tasks.workunit.client.0.smithi142.stdout:4/125: write d0/dc/f27 [755648,79724] 0 2024-03-29T04:44:54.250 INFO:tasks.workunit.client.0.smithi142.stdout:7/94: truncate d7/fd 1776537 0 2024-03-29T04:44:54.252 INFO:tasks.workunit.client.0.smithi142.stdout:6/103: getdents . 0 2024-03-29T04:44:54.253 INFO:tasks.workunit.client.0.smithi142.stdout:6/104: write f18 [109278,21491] 0 2024-03-29T04:44:54.253 INFO:tasks.workunit.client.0.smithi142.stdout:7/95: fdatasync f1 0 2024-03-29T04:44:54.253 INFO:tasks.workunit.client.0.smithi142.stdout:9/97: unlink d0/d2/cf 0 2024-03-29T04:44:54.254 INFO:tasks.workunit.client.0.smithi142.stdout:9/98: stat d0/d2/d6/fd 0 2024-03-29T04:44:54.254 INFO:tasks.workunit.client.0.smithi142.stdout:5/193: fsync df/d1a/d28/f2d 0 2024-03-29T04:44:54.256 INFO:tasks.workunit.client.0.smithi142.stdout:2/79: truncate de/f10 1238687 0 2024-03-29T04:44:54.258 INFO:tasks.workunit.client.0.smithi142.stdout:2/80: read de/d11/f15 [436933,129641] 0 2024-03-29T04:44:54.259 INFO:tasks.workunit.client.0.smithi142.stdout:4/126: rename d0/d9/f13 to d0/d9/d11/f2b 0 2024-03-29T04:44:54.262 INFO:tasks.workunit.client.0.smithi142.stdout:3/134: truncate d3/d29/d2b/d17/f12 88779 0 2024-03-29T04:44:54.262 INFO:tasks.workunit.client.0.smithi142.stdout:5/194: dwrite df/d1a/f22 [0,4194304] 0 2024-03-29T04:44:54.262 INFO:tasks.workunit.client.0.smithi142.stdout:3/135: fsync d3/d29/d2b/d17/d6/f20 0 2024-03-29T04:44:54.262 INFO:tasks.workunit.client.0.smithi142.stdout:1/126: creat d8/db/dc/d19/d1d/d1f/f30 x:0 0 0 2024-03-29T04:44:54.263 INFO:tasks.workunit.client.0.smithi142.stdout:3/136: write d3/d29/d2b/d17/f11 [2789970,112705] 0 2024-03-29T04:44:54.264 INFO:tasks.workunit.client.0.smithi142.stdout:3/137: fdatasync d3/d29/d2b/d17/d6/f1c 0 2024-03-29T04:44:54.273 INFO:tasks.workunit.client.0.smithi142.stdout:4/127: mkdir d0/d9/d19/d2c 0 2024-03-29T04:44:54.273 INFO:tasks.workunit.client.0.smithi142.stdout:8/162: write d5/d10/f20 [1776840,122953] 0 2024-03-29T04:44:54.276 INFO:tasks.workunit.client.0.smithi142.stdout:6/105: truncate ff 72402 0 2024-03-29T04:44:54.277 INFO:tasks.workunit.client.0.smithi142.stdout:1/127: mkdir d8/db/dc/d19/d1d/d15/d21/d31 0 2024-03-29T04:44:54.278 INFO:tasks.workunit.client.0.smithi142.stdout:6/106: read f18 [52162,121397] 0 2024-03-29T04:44:54.278 INFO:tasks.workunit.client.0.smithi142.stdout:1/128: write d8/db/dc/f2c [882235,98103] 0 2024-03-29T04:44:54.279 INFO:tasks.workunit.client.0.smithi142.stdout:6/107: write f18 [973213,32226] 0 2024-03-29T04:44:54.279 INFO:tasks.workunit.client.0.smithi142.stdout:6/108: rmdir - no directory 2024-03-29T04:44:54.283 INFO:tasks.workunit.client.0.smithi142.stdout:1/129: dread d8/db/dc/d19/d1d/f13 [0,4194304] 0 2024-03-29T04:44:54.284 INFO:tasks.workunit.client.0.smithi142.stdout:1/130: chown d8/f2a 3498840 1 2024-03-29T04:44:54.286 INFO:tasks.workunit.client.0.smithi142.stdout:4/128: mknod d0/d9/c2d 0 2024-03-29T04:44:54.287 INFO:tasks.workunit.client.0.smithi142.stdout:5/195: truncate fa 621062 0 2024-03-29T04:44:54.288 INFO:tasks.workunit.client.0.smithi142.stdout:8/163: creat d5/d1d/d2f/f34 x:0 0 0 2024-03-29T04:44:54.289 INFO:tasks.workunit.client.0.smithi142.stdout:8/164: dread - d5/d1d/d24/f33 zero size 2024-03-29T04:44:54.290 INFO:tasks.workunit.client.0.smithi142.stdout:7/96: creat d7/f19 x:0 0 0 2024-03-29T04:44:54.292 INFO:tasks.workunit.client.0.smithi142.stdout:7/97: write d7/dc/f10 [398282,24052] 0 2024-03-29T04:44:54.295 INFO:tasks.workunit.client.0.smithi142.stdout:8/165: dwrite d5/f25 [0,4194304] 0 2024-03-29T04:44:54.296 INFO:tasks.workunit.client.0.smithi142.stdout:1/131: creat d8/db/dc/d19/d1d/d15/d21/f32 x:0 0 0 2024-03-29T04:44:54.297 INFO:tasks.workunit.client.0.smithi142.stdout:8/166: write d5/d1d/d2f/f34 [24799,84469] 0 2024-03-29T04:44:54.297 INFO:tasks.workunit.client.0.smithi142.stdout:8/167: stat d5/l8 0 2024-03-29T04:44:54.297 INFO:tasks.workunit.client.0.smithi142.stdout:4/129: symlink d0/dc/d15/l2e 0 2024-03-29T04:44:54.298 INFO:tasks.workunit.client.0.smithi142.stdout:9/99: truncate d0/d2/d6/fd 2589755 0 2024-03-29T04:44:54.299 INFO:tasks.workunit.client.0.smithi142.stdout:9/100: write d0/d2/fe [164099,125190] 0 2024-03-29T04:44:54.300 INFO:tasks.workunit.client.0.smithi142.stdout:5/196: creat df/d1a/d21/d27/f36 x:0 0 0 2024-03-29T04:44:54.303 INFO:tasks.workunit.client.0.smithi142.stdout:0/109: sync 2024-03-29T04:44:54.305 INFO:tasks.workunit.client.0.smithi142.stdout:4/130: dwrite d0/d9/d19/f1a [0,4194304] 0 2024-03-29T04:44:54.306 INFO:tasks.workunit.client.0.smithi142.stdout:3/138: rename d3/d29/d2b/d17 to d3/d1f/d34 0 2024-03-29T04:44:54.307 INFO:tasks.workunit.client.0.smithi142.stdout:4/131: write d0/d9/fb [419999,112602] 0 2024-03-29T04:44:54.312 INFO:tasks.workunit.client.0.smithi142.stdout:2/81: dread f6 [0,4194304] 0 2024-03-29T04:44:54.312 INFO:tasks.workunit.client.0.smithi142.stdout:8/168: rmdir d5/d1d 39 2024-03-29T04:44:54.314 INFO:tasks.workunit.client.0.smithi142.stdout:4/132: dread d0/f3 [0,4194304] 0 2024-03-29T04:44:54.320 INFO:tasks.workunit.client.0.smithi142.stdout:2/82: dwrite de/d11/d17/f1a [0,4194304] 0 2024-03-29T04:44:54.322 INFO:tasks.workunit.client.0.smithi142.stdout:2/83: fdatasync de/f16 0 2024-03-29T04:44:54.323 INFO:tasks.workunit.client.0.smithi142.stdout:1/132: rename d8/db/dc/d19/d1d/d15/d21/f26 to d8/db/dc/d19/d1d/d1f/f33 0 2024-03-29T04:44:54.324 INFO:tasks.workunit.client.0.smithi142.stdout:2/84: truncate f7 351317 0 2024-03-29T04:44:54.327 INFO:tasks.workunit.client.0.smithi142.stdout:6/109: dwrite ff [0,4194304] 0 2024-03-29T04:44:54.328 INFO:tasks.workunit.client.0.smithi142.stdout:7/98: creat d7/f1a x:0 0 0 2024-03-29T04:44:54.329 INFO:tasks.workunit.client.0.smithi142.stdout:1/133: truncate d8/db/f25 180421 0 2024-03-29T04:44:54.329 INFO:tasks.workunit.client.0.smithi142.stdout:1/134: fsync f7 0 2024-03-29T04:44:54.335 INFO:tasks.workunit.client.0.smithi142.stdout:6/110: dwrite f17 [0,4194304] 0 2024-03-29T04:44:54.336 INFO:tasks.workunit.client.0.smithi142.stdout:5/197: rename df/f1d to df/d16/d25/f37 0 2024-03-29T04:44:54.338 INFO:tasks.workunit.client.0.smithi142.stdout:2/85: mknod de/d11/d17/c1b 0 2024-03-29T04:44:54.339 INFO:tasks.workunit.client.0.smithi142.stdout:7/99: mknod d7/dc/de/c1b 0 2024-03-29T04:44:54.340 INFO:tasks.workunit.client.0.smithi142.stdout:1/135: dwrite d8/db/f20 [0,4194304] 0 2024-03-29T04:44:54.340 INFO:tasks.workunit.client.0.smithi142.stdout:0/110: write d7/d9/d22/f1b [384713,6051] 0 2024-03-29T04:44:54.340 INFO:tasks.workunit.client.0.smithi142.stdout:5/198: write df/d1a/f2f [526691,46307] 0 2024-03-29T04:44:54.341 INFO:tasks.workunit.client.0.smithi142.stdout:7/100: readlink d7/l13 0 2024-03-29T04:44:54.341 INFO:tasks.workunit.client.0.smithi142.stdout:0/111: dread - d7/f21 zero size 2024-03-29T04:44:54.342 INFO:tasks.workunit.client.0.smithi142.stdout:5/199: write df/d1a/f31 [802347,18567] 0 2024-03-29T04:44:54.348 INFO:tasks.workunit.client.0.smithi142.stdout:7/101: dwrite d7/dc/de/f16 [0,4194304] 0 2024-03-29T04:44:54.349 INFO:tasks.workunit.client.0.smithi142.stdout:1/136: creat d8/db/dc/d19/f34 x:0 0 0 2024-03-29T04:44:54.350 INFO:tasks.workunit.client.0.smithi142.stdout:6/111: link f17 f1a 0 2024-03-29T04:44:54.351 INFO:tasks.workunit.client.0.smithi142.stdout:9/101: rename d0/d2/d6/f1b to d0/d2/f1e 0 2024-03-29T04:44:54.353 INFO:tasks.workunit.client.0.smithi142.stdout:1/137: creat d8/db/dc/d19/d1d/d15/d21/f35 x:0 0 0 2024-03-29T04:44:54.355 INFO:tasks.workunit.client.0.smithi142.stdout:3/139: rename d3/d1f/f23 to d3/d1f/d34/f35 0 2024-03-29T04:44:54.357 INFO:tasks.workunit.client.0.smithi142.stdout:1/138: symlink d8/db/dc/d19/d1d/l36 0 2024-03-29T04:44:54.358 INFO:tasks.workunit.client.0.smithi142.stdout:9/102: creat d0/d2/f1f x:0 0 0 2024-03-29T04:44:54.358 INFO:tasks.workunit.client.0.smithi142.stdout:6/112: dwrite fc [8388608,4194304] 0 2024-03-29T04:44:54.359 INFO:tasks.workunit.client.0.smithi142.stdout:9/103: dread - d0/f1d zero size 2024-03-29T04:44:54.359 INFO:tasks.workunit.client.0.smithi142.stdout:0/112: rename c6 to d7/d9/c25 0 2024-03-29T04:44:54.360 INFO:tasks.workunit.client.0.smithi142.stdout:1/139: mkdir d8/db/d37 0 2024-03-29T04:44:54.360 INFO:tasks.workunit.client.0.smithi142.stdout:6/113: symlink l1b 0 2024-03-29T04:44:54.360 INFO:tasks.workunit.client.0.smithi142.stdout:0/113: write d7/d9/f1c [4695689,24655] 0 2024-03-29T04:44:54.374 INFO:tasks.workunit.client.0.smithi142.stdout:5/200: rename df/d16/c33 to df/d1a/c38 0 2024-03-29T04:44:54.374 INFO:tasks.workunit.client.0.smithi142.stdout:0/114: readlink d7/l1a 0 2024-03-29T04:44:54.374 INFO:tasks.workunit.client.0.smithi142.stdout:6/114: fdatasync f4 0 2024-03-29T04:44:54.374 INFO:tasks.workunit.client.0.smithi142.stdout:9/104: mknod d0/d2/c20 0 2024-03-29T04:44:54.375 INFO:tasks.workunit.client.0.smithi142.stdout:1/140: creat d8/db/d37/f38 x:0 0 0 2024-03-29T04:44:54.376 INFO:tasks.workunit.client.0.smithi142.stdout:7/102: rename d7/dc/de/c1b to d7/d18/c1c 0 2024-03-29T04:44:54.376 INFO:tasks.workunit.client.0.smithi142.stdout:9/105: write d0/d2/f1f [73894,109039] 0 2024-03-29T04:44:54.379 INFO:tasks.workunit.client.0.smithi142.stdout:0/115: rename d7/d13/f20 to d7/d9/d1e/f26 0 2024-03-29T04:44:54.381 INFO:tasks.workunit.client.0.smithi142.stdout:6/115: dread f3 [4194304,4194304] 0 2024-03-29T04:44:54.381 INFO:tasks.workunit.client.0.smithi142.stdout:9/106: truncate d0/d2/f1e 756503 0 2024-03-29T04:44:54.383 INFO:tasks.workunit.client.0.smithi142.stdout:5/201: dwrite df/d16/f26 [0,4194304] 0 2024-03-29T04:44:54.385 INFO:tasks.workunit.client.0.smithi142.stdout:1/141: dwrite d8/db/f20 [0,4194304] 0 2024-03-29T04:44:54.386 INFO:tasks.workunit.client.0.smithi142.stdout:1/142: read - d8/db/dc/d19/d1d/d15/d21/f32 zero size 2024-03-29T04:44:54.399 INFO:tasks.workunit.client.0.smithi142.stdout:0/116: mkdir d7/d13/d27 0 2024-03-29T04:44:54.400 INFO:tasks.workunit.client.0.smithi142.stdout:9/107: symlink d0/d2/d6/d16/l21 0 2024-03-29T04:44:54.401 INFO:tasks.workunit.client.0.smithi142.stdout:5/202: creat df/d16/f39 x:0 0 0 2024-03-29T04:44:54.402 INFO:tasks.workunit.client.0.smithi142.stdout:9/108: dread d0/d12/f13 [0,4194304] 0 2024-03-29T04:44:54.404 INFO:tasks.workunit.client.0.smithi142.stdout:0/117: creat d7/d13/f28 x:0 0 0 2024-03-29T04:44:54.407 INFO:tasks.workunit.client.0.smithi142.stdout:7/103: link d7/c8 d7/c1d 0 2024-03-29T04:44:54.408 INFO:tasks.workunit.client.0.smithi142.stdout:7/104: dread d7/fa [0,4194304] 0 2024-03-29T04:44:54.409 INFO:tasks.workunit.client.0.smithi142.stdout:4/133: truncate d0/d9/d19/f1a 3894149 0 2024-03-29T04:44:54.411 INFO:tasks.workunit.client.0.smithi142.stdout:0/118: dwrite d7/d9/ff [4194304,4194304] 0 2024-03-29T04:44:54.412 INFO:tasks.workunit.client.0.smithi142.stdout:0/119: write f2 [4185210,36220] 0 2024-03-29T04:44:54.417 INFO:tasks.workunit.client.0.smithi142.stdout:5/203: mknod df/c3a 0 2024-03-29T04:44:54.417 INFO:tasks.workunit.client.0.smithi142.stdout:5/204: chown df/d29 7 1 2024-03-29T04:44:54.417 INFO:tasks.workunit.client.0.smithi142.stdout:0/120: dread f2 [0,4194304] 0 2024-03-29T04:44:54.418 INFO:tasks.workunit.client.0.smithi142.stdout:5/205: dread - df/d1a/d21/d27/f2a zero size 2024-03-29T04:44:54.418 INFO:tasks.workunit.client.0.smithi142.stdout:8/169: truncate d5/d1d/d24/f2b 1306114 0 2024-03-29T04:44:54.418 INFO:tasks.workunit.client.0.smithi142.stdout:0/121: fsync d7/d9/f1c 0 2024-03-29T04:44:54.418 INFO:tasks.workunit.client.0.smithi142.stdout:5/206: stat df/d16/l1b 0 2024-03-29T04:44:54.419 INFO:tasks.workunit.client.0.smithi142.stdout:2/86: write f6 [1671846,77569] 0 2024-03-29T04:44:54.420 INFO:tasks.workunit.client.0.smithi142.stdout:4/134: symlink d0/d9/d19/l2f 0 2024-03-29T04:44:54.425 INFO:tasks.workunit.client.0.smithi142.stdout:9/109: rename d0/d2/f1e to d0/f22 0 2024-03-29T04:44:54.425 INFO:tasks.workunit.client.0.smithi142.stdout:2/87: dwrite f8 [0,4194304] 0 2024-03-29T04:44:54.429 INFO:tasks.workunit.client.0.smithi142.stdout:0/122: symlink d7/d9/d22/l29 0 2024-03-29T04:44:54.434 INFO:tasks.workunit.client.0.smithi142.stdout:1/143: getdents d8/db 0 2024-03-29T04:44:54.434 INFO:tasks.workunit.client.0.smithi142.stdout:1/144: write d8/f2a [4604426,61222] 0 2024-03-29T04:44:54.435 INFO:tasks.workunit.client.0.smithi142.stdout:7/105: creat d7/dc/f1e x:0 0 0 2024-03-29T04:44:54.436 INFO:tasks.workunit.client.0.smithi142.stdout:6/116: getdents . 0 2024-03-29T04:44:54.438 INFO:tasks.workunit.client.0.smithi142.stdout:2/88: mknod de/d11/c1c 0 2024-03-29T04:44:54.439 INFO:tasks.workunit.client.0.smithi142.stdout:5/207: mknod df/d29/c3b 0 2024-03-29T04:44:54.439 INFO:tasks.workunit.client.0.smithi142.stdout:5/208: readlink l5 0 2024-03-29T04:44:54.440 INFO:tasks.workunit.client.0.smithi142.stdout:0/123: truncate d7/d9/fa 5465819 0 2024-03-29T04:44:54.442 INFO:tasks.workunit.client.0.smithi142.stdout:1/145: dwrite d8/db/dc/d19/d1d/d1f/f30 [0,4194304] 0 2024-03-29T04:44:54.444 INFO:tasks.workunit.client.0.smithi142.stdout:6/117: mkdir d1c 0 2024-03-29T04:44:54.445 INFO:tasks.workunit.client.0.smithi142.stdout:3/140: sync 2024-03-29T04:44:54.446 INFO:tasks.workunit.client.0.smithi142.stdout:3/141: dread - d3/d29/d2b/f26 zero size 2024-03-29T04:44:54.446 INFO:tasks.workunit.client.0.smithi142.stdout:0/124: dread d7/d9/ff [4194304,4194304] 0 2024-03-29T04:44:54.446 INFO:tasks.workunit.client.0.smithi142.stdout:9/110: symlink d0/d2/l23 0 2024-03-29T04:44:54.449 INFO:tasks.workunit.client.0.smithi142.stdout:5/209: mknod df/d1a/c3c 0 2024-03-29T04:44:54.450 INFO:tasks.workunit.client.0.smithi142.stdout:2/89: fsync f8 0 2024-03-29T04:44:54.454 INFO:tasks.workunit.client.0.smithi142.stdout:0/125: symlink d7/d9/d1e/l2a 0 2024-03-29T04:44:54.455 INFO:tasks.workunit.client.0.smithi142.stdout:9/111: creat d0/d12/f24 x:0 0 0 2024-03-29T04:44:54.456 INFO:tasks.workunit.client.0.smithi142.stdout:4/135: getdents d0 0 2024-03-29T04:44:54.461 INFO:tasks.workunit.client.0.smithi142.stdout:6/118: getdents d1c 0 2024-03-29T04:44:54.466 INFO:tasks.workunit.client.0.smithi142.stdout:6/119: dread ff [0,4194304] 0 2024-03-29T04:44:54.466 INFO:tasks.workunit.client.0.smithi142.stdout:6/120: chown f17 13640 1 2024-03-29T04:44:54.466 INFO:tasks.workunit.client.0.smithi142.stdout:6/121: chown f1a 334 1 2024-03-29T04:44:54.467 INFO:tasks.workunit.client.0.smithi142.stdout:5/210: truncate f3 4919054 0 2024-03-29T04:44:54.468 INFO:tasks.workunit.client.0.smithi142.stdout:5/211: truncate df/d16/f39 756289 0 2024-03-29T04:44:54.472 INFO:tasks.workunit.client.0.smithi142.stdout:8/170: dwrite d5/f22 [4194304,4194304] 0 2024-03-29T04:44:54.472 INFO:tasks.workunit.client.0.smithi142.stdout:4/136: write d0/d9/d19/f1a [3608871,44736] 0 2024-03-29T04:44:54.474 INFO:tasks.workunit.client.0.smithi142.stdout:1/146: link c2 d8/db/dc/d19/d1d/d15/d1a/c39 0 2024-03-29T04:44:54.475 INFO:tasks.workunit.client.0.smithi142.stdout:1/147: readlink d8/db/l24 0 2024-03-29T04:44:54.476 INFO:tasks.workunit.client.0.smithi142.stdout:1/148: fsync d8/db/dc/f1c 0 2024-03-29T04:44:54.477 INFO:tasks.workunit.client.0.smithi142.stdout:1/149: readlink d8/db/l24 0 2024-03-29T04:44:54.477 INFO:tasks.workunit.client.0.smithi142.stdout:3/142: link d3/d1f/d34/c32 d3/d1f/d34/c36 0 2024-03-29T04:44:54.477 INFO:tasks.workunit.client.0.smithi142.stdout:8/171: dread d5/f22 [4194304,4194304] 0 2024-03-29T04:44:54.478 INFO:tasks.workunit.client.0.smithi142.stdout:8/172: readlink d5/d1d/d24/l2c 0 2024-03-29T04:44:54.479 INFO:tasks.workunit.client.0.smithi142.stdout:8/173: read d5/d10/f20 [61925,47539] 0 2024-03-29T04:44:54.479 INFO:tasks.workunit.client.0.smithi142.stdout:6/122: creat d1c/f1d x:0 0 0 2024-03-29T04:44:54.482 INFO:tasks.workunit.client.0.smithi142.stdout:4/137: mkdir d0/dc/d15/d30 0 2024-03-29T04:44:54.484 INFO:tasks.workunit.client.0.smithi142.stdout:0/126: getdents d7 0 2024-03-29T04:44:54.484 INFO:tasks.workunit.client.0.smithi142.stdout:4/138: read d0/f10 [4121386,62298] 0 2024-03-29T04:44:54.485 INFO:tasks.workunit.client.0.smithi142.stdout:9/112: getdents d0/d2 0 2024-03-29T04:44:54.487 INFO:tasks.workunit.client.0.smithi142.stdout:0/127: stat d7/d9/d1e/l2a 0 2024-03-29T04:44:54.489 INFO:tasks.workunit.client.0.smithi142.stdout:1/150: mknod d8/db/dc/d19/d1d/d15/c3a 0 2024-03-29T04:44:54.489 INFO:tasks.workunit.client.0.smithi142.stdout:7/106: dwrite d7/fd [0,4194304] 0 2024-03-29T04:44:54.496 INFO:tasks.workunit.client.0.smithi142.stdout:2/90: dwrite de/d11/f15 [0,4194304] 0 2024-03-29T04:44:54.498 INFO:tasks.workunit.client.0.smithi142.stdout:9/113: symlink d0/d12/l25 0 2024-03-29T04:44:54.499 INFO:tasks.workunit.client.0.smithi142.stdout:4/139: dwrite d0/d9/d11/f25 [0,4194304] 0 2024-03-29T04:44:54.502 INFO:tasks.workunit.client.0.smithi142.stdout:3/143: creat d3/d1f/d24/f37 x:0 0 0 2024-03-29T04:44:54.505 INFO:tasks.workunit.client.0.smithi142.stdout:6/123: write f4 [1593865,87009] 0 2024-03-29T04:44:54.508 INFO:tasks.workunit.client.0.smithi142.stdout:7/107: dwrite d7/dc/f10 [0,4194304] 0 2024-03-29T04:44:54.508 INFO:tasks.workunit.client.0.smithi142.stdout:8/174: fdatasync d5/f1f 0 2024-03-29T04:44:54.510 INFO:tasks.workunit.client.0.smithi142.stdout:1/151: symlink d8/db/dc/d19/d1d/d1f/l3b 0 2024-03-29T04:44:54.511 INFO:tasks.workunit.client.0.smithi142.stdout:7/108: dread - d7/f1a zero size 2024-03-29T04:44:54.515 INFO:tasks.workunit.client.0.smithi142.stdout:3/144: unlink d3/d29/d2b/f26 0 2024-03-29T04:44:54.515 INFO:tasks.workunit.client.0.smithi142.stdout:5/212: dwrite fa [0,4194304] 0 2024-03-29T04:44:54.516 INFO:tasks.workunit.client.0.smithi142.stdout:8/175: creat d5/d10/f35 x:0 0 0 2024-03-29T04:44:54.518 INFO:tasks.workunit.client.0.smithi142.stdout:6/124: dread f14 [0,4194304] 0 2024-03-29T04:44:54.520 INFO:tasks.workunit.client.0.smithi142.stdout:7/109: read f6 [1667721,61911] 0 2024-03-29T04:44:54.520 INFO:tasks.workunit.client.0.smithi142.stdout:9/114: getdents d0/d2 0 2024-03-29T04:44:54.521 INFO:tasks.workunit.client.0.smithi142.stdout:0/128: getdents d7/d9 0 2024-03-29T04:44:54.521 INFO:tasks.workunit.client.0.smithi142.stdout:9/115: chown d0 58 1 2024-03-29T04:44:54.524 INFO:tasks.workunit.client.0.smithi142.stdout:9/116: fsync d0/f1d 0 2024-03-29T04:44:54.524 INFO:tasks.workunit.client.0.smithi142.stdout:4/140: rmdir d0/dc/d15/d30 0 2024-03-29T04:44:54.525 INFO:tasks.workunit.client.0.smithi142.stdout:7/110: creat d7/d18/f1f x:0 0 0 2024-03-29T04:44:54.525 INFO:tasks.workunit.client.0.smithi142.stdout:6/125: fsync ff 0 2024-03-29T04:44:54.526 INFO:tasks.workunit.client.0.smithi142.stdout:8/176: mkdir d5/d36 0 2024-03-29T04:44:54.526 INFO:tasks.workunit.client.0.smithi142.stdout:6/126: stat fb 0 2024-03-29T04:44:54.527 INFO:tasks.workunit.client.0.smithi142.stdout:6/127: stat l15 0 2024-03-29T04:44:54.529 INFO:tasks.workunit.client.0.smithi142.stdout:7/111: fdatasync d7/f9 0 2024-03-29T04:44:54.529 INFO:tasks.workunit.client.0.smithi142.stdout:4/141: mknod d0/dc/d15/c31 0 2024-03-29T04:44:54.530 INFO:tasks.workunit.client.0.smithi142.stdout:5/213: dwrite df/f14 [0,4194304] 0 2024-03-29T04:44:54.530 INFO:tasks.workunit.client.0.smithi142.stdout:7/112: fdatasync d7/fd 0 2024-03-29T04:44:54.531 INFO:tasks.workunit.client.0.smithi142.stdout:1/152: link d8/db/dc/d19/d1d/d15/c22 d8/db/dc/d19/d1d/d15/d1a/c3c 0 2024-03-29T04:44:54.531 INFO:tasks.workunit.client.0.smithi142.stdout:8/177: fdatasync d5/f22 0 2024-03-29T04:44:54.532 INFO:tasks.workunit.client.0.smithi142.stdout:1/153: chown d8/f2b 27 1 2024-03-29T04:44:54.533 INFO:tasks.workunit.client.0.smithi142.stdout:3/145: dread d3/d29/d2b/f21 [0,4194304] 0 2024-03-29T04:44:54.533 INFO:tasks.workunit.client.0.smithi142.stdout:2/91: dread f3 [0,4194304] 0 2024-03-29T04:44:54.534 INFO:tasks.workunit.client.0.smithi142.stdout:6/128: truncate f18 1797045 0 2024-03-29T04:44:54.535 INFO:tasks.workunit.client.0.smithi142.stdout:1/154: chown d8/db/dc/f1c 140729 1 2024-03-29T04:44:54.536 INFO:tasks.workunit.client.0.smithi142.stdout:4/142: dread - d0/dc/f17 zero size 2024-03-29T04:44:54.537 INFO:tasks.workunit.client.0.smithi142.stdout:3/146: truncate d3/d1f/d34/d6/f1c 1720047 0 2024-03-29T04:44:54.537 INFO:tasks.workunit.client.0.smithi142.stdout:0/129: getdents d7/d9 0 2024-03-29T04:44:54.538 INFO:tasks.workunit.client.0.smithi142.stdout:4/143: write d0/d9/d11/f23 [85149,10716] 0 2024-03-29T04:44:54.539 INFO:tasks.workunit.client.0.smithi142.stdout:5/214: mkdir df/d16/d24/d3d 0 2024-03-29T04:44:54.540 INFO:tasks.workunit.client.0.smithi142.stdout:8/178: unlink d5/c13 0 2024-03-29T04:44:54.541 INFO:tasks.workunit.client.0.smithi142.stdout:9/117: link d0/d2/d6/d16/l21 d0/l26 0 2024-03-29T04:44:54.541 INFO:tasks.workunit.client.0.smithi142.stdout:4/144: read d0/f8 [2935892,35370] 0 2024-03-29T04:44:54.542 INFO:tasks.workunit.client.0.smithi142.stdout:6/129: dread f3 [4194304,4194304] 0 2024-03-29T04:44:54.542 INFO:tasks.workunit.client.0.smithi142.stdout:4/145: fsync d0/dc/f27 0 2024-03-29T04:44:54.543 INFO:tasks.workunit.client.0.smithi142.stdout:2/92: rename f8 to de/d11/f1d 0 2024-03-29T04:44:54.544 INFO:tasks.workunit.client.0.smithi142.stdout:1/155: mkdir d8/db/d3d 0 2024-03-29T04:44:54.544 INFO:tasks.workunit.client.0.smithi142.stdout:4/146: truncate d0/dc/f27 1805651 0 2024-03-29T04:44:54.548 INFO:tasks.workunit.client.0.smithi142.stdout:9/118: truncate d0/d12/f13 200734 0 2024-03-29T04:44:54.551 INFO:tasks.workunit.client.0.smithi142.stdout:8/179: creat d5/d2e/f37 x:0 0 0 2024-03-29T04:44:54.553 INFO:tasks.workunit.client.0.smithi142.stdout:4/147: fsync d0/d9/d11/ff 0 2024-03-29T04:44:54.553 INFO:tasks.workunit.client.0.smithi142.stdout:9/119: mkdir d0/d2/d6/d27 0 2024-03-29T04:44:54.554 INFO:tasks.workunit.client.0.smithi142.stdout:6/130: dwrite f14 [0,4194304] 0 2024-03-29T04:44:54.556 INFO:tasks.workunit.client.0.smithi142.stdout:8/180: symlink d5/d2e/l38 0 2024-03-29T04:44:54.556 INFO:tasks.workunit.client.0.smithi142.stdout:1/156: dread d8/db/dc/d19/d1d/f13 [0,4194304] 0 2024-03-29T04:44:54.558 INFO:tasks.workunit.client.0.smithi142.stdout:9/120: mkdir d0/d2/d28 0 2024-03-29T04:44:54.561 INFO:tasks.workunit.client.0.smithi142.stdout:1/157: dread d8/f2a [0,4194304] 0 2024-03-29T04:44:54.562 INFO:tasks.workunit.client.0.smithi142.stdout:1/158: write d8/db/f25 [755210,97575] 0 2024-03-29T04:44:54.563 INFO:tasks.workunit.client.0.smithi142.stdout:1/159: write d8/db/f25 [144482,19989] 0 2024-03-29T04:44:54.566 INFO:tasks.workunit.client.0.smithi142.stdout:2/93: write f9 [1080975,129403] 0 2024-03-29T04:44:54.567 INFO:tasks.workunit.client.0.smithi142.stdout:9/121: chown d0/c8 63703791 1 2024-03-29T04:44:54.567 INFO:tasks.workunit.client.0.smithi142.stdout:3/147: truncate d3/d1f/d34/f35 1144814 0 2024-03-29T04:44:54.569 INFO:tasks.workunit.client.0.smithi142.stdout:6/131: dread f18 [0,4194304] 0 2024-03-29T04:44:54.569 INFO:tasks.workunit.client.0.smithi142.stdout:6/132: chown c9 5176 1 2024-03-29T04:44:54.570 INFO:tasks.workunit.client.0.smithi142.stdout:2/94: mkdir de/d11/d1e 0 2024-03-29T04:44:54.571 INFO:tasks.workunit.client.0.smithi142.stdout:8/181: rmdir d5/d36 0 2024-03-29T04:44:54.573 INFO:tasks.workunit.client.0.smithi142.stdout:1/160: creat d8/db/dc/d19/d1d/d2d/f3e x:0 0 0 2024-03-29T04:44:54.574 INFO:tasks.workunit.client.0.smithi142.stdout:1/161: dread - d8/db/dc/d19/d1d/d15/d21/f32 zero size 2024-03-29T04:44:54.578 INFO:tasks.workunit.client.0.smithi142.stdout:3/148: mkdir d3/d1f/d38 0 2024-03-29T04:44:54.578 INFO:tasks.workunit.client.0.smithi142.stdout:3/149: readlink d3/d1f/d34/d6/lb 0 2024-03-29T04:44:54.579 INFO:tasks.workunit.client.0.smithi142.stdout:9/122: mknod d0/d2/d28/c29 0 2024-03-29T04:44:54.581 INFO:tasks.workunit.client.0.smithi142.stdout:2/95: mknod de/c1f 0 2024-03-29T04:44:54.584 INFO:tasks.workunit.client.0.smithi142.stdout:6/133: write f18 [11093,86023] 0 2024-03-29T04:44:54.584 INFO:tasks.workunit.client.0.smithi142.stdout:6/134: stat l13 0 2024-03-29T04:44:54.584 INFO:tasks.workunit.client.0.smithi142.stdout:9/123: rmdir d0/d2 39 2024-03-29T04:44:54.585 INFO:tasks.workunit.client.0.smithi142.stdout:9/124: write d0/d12/f24 [100541,19366] 0 2024-03-29T04:44:54.587 INFO:tasks.workunit.client.0.smithi142.stdout:8/182: creat d5/f39 x:0 0 0 2024-03-29T04:44:54.588 INFO:tasks.workunit.client.0.smithi142.stdout:6/135: mknod d1c/c1e 0 2024-03-29T04:44:54.589 INFO:tasks.workunit.client.0.smithi142.stdout:2/96: symlink de/d11/d1e/l20 0 2024-03-29T04:44:54.591 INFO:tasks.workunit.client.0.smithi142.stdout:6/136: truncate f3 9099144 0 2024-03-29T04:44:54.592 INFO:tasks.workunit.client.0.smithi142.stdout:9/125: mkdir d0/d2/d6/d2a 0 2024-03-29T04:44:54.596 INFO:tasks.workunit.client.0.smithi142.stdout:1/162: dwrite d8/db/dc/d19/d1d/d15/f17 [0,4194304] 0 2024-03-29T04:44:54.597 INFO:tasks.workunit.client.0.smithi142.stdout:8/183: dwrite d5/f22 [8388608,4194304] 0 2024-03-29T04:44:54.600 INFO:tasks.workunit.client.0.smithi142.stdout:9/126: dread d0/d2/fe [0,4194304] 0 2024-03-29T04:44:54.601 INFO:tasks.workunit.client.0.smithi142.stdout:2/97: link f9 de/d11/f21 0 2024-03-29T04:44:54.601 INFO:tasks.workunit.client.0.smithi142.stdout:2/98: chown de/d11/d17 203058 1 2024-03-29T04:44:54.605 INFO:tasks.workunit.client.0.smithi142.stdout:9/127: creat d0/d2/d6/d16/f2b x:0 0 0 2024-03-29T04:44:54.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.607+0000 7f0c87c6a700 1 -- 172.21.15.142:0/1292724063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c80103ff0 msgr2=0x7f0c80104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:54.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.607+0000 7f0c87c6a700 1 --2- 172.21.15.142:0/1292724063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c80103ff0 0x7f0c80104450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009910 tx=0x7f0c70009c20 comp rx=0 tx=0).stop 2024-03-29T04:44:54.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.608+0000 7f0c87c6a700 1 -- 172.21.15.142:0/1292724063 shutdown_connections 2024-03-29T04:44:54.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.608+0000 7f0c87c6a700 1 --2- 172.21.15.142:0/1292724063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c80103ff0 0x7f0c80104450 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:54.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.608+0000 7f0c87c6a700 1 --2- 172.21.15.142:0/1292724063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c80102df0 0x7f0c80103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:54.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.608+0000 7f0c87c6a700 1 -- 172.21.15.142:0/1292724063 >> 172.21.15.142:0/1292724063 conn(0x7f0c800fe390 msgr2=0x7f0c801007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:54.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.608+0000 7f0c87c6a700 1 -- 172.21.15.142:0/1292724063 shutdown_connections 2024-03-29T04:44:54.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.608+0000 7f0c87c6a700 1 -- 172.21.15.142:0/1292724063 wait complete. 2024-03-29T04:44:54.609 INFO:tasks.workunit.client.0.smithi142.stdout:2/99: mkdir de/d11/d1e/d22 0 2024-03-29T04:44:54.609 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.608+0000 7f0c87c6a700 1 Processor -- start 2024-03-29T04:44:54.609 INFO:tasks.workunit.client.0.smithi142.stdout:2/100: stat de/f10 0 2024-03-29T04:44:54.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.609+0000 7f0c87c6a700 1 -- start start 2024-03-29T04:44:54.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.609+0000 7f0c87c6a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c80102df0 0x7f0c801985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:54.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.609+0000 7f0c87c6a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c80103ff0 0x7f0c80198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:54.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.609+0000 7f0c87c6a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80199150 con 0x7f0c80102df0 2024-03-29T04:44:54.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.609+0000 7f0c87c6a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80199290 con 0x7f0c80103ff0 2024-03-29T04:44:54.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.609+0000 7f0c85a06700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c80102df0 0x7f0c801985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:54.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.609+0000 7f0c85a06700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c80102df0 0x7f0c801985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40086/0 (socket says 172.21.15.142:40086) 2024-03-29T04:44:54.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.609+0000 7f0c85a06700 1 -- 172.21.15.142:0/2956617072 learned_addr learned my addr 172.21.15.142:0/2956617072 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:44:54.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.609+0000 7f0c85205700 1 --2- 172.21.15.142:0/2956617072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c80103ff0 0x7f0c80198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:54.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.610+0000 7f0c85a06700 1 -- 172.21.15.142:0/2956617072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c80103ff0 msgr2=0x7f0c80198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:54.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.610+0000 7f0c85a06700 1 --2- 172.21.15.142:0/2956617072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c80103ff0 0x7f0c80198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:54.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.610+0000 7f0c85a06700 1 -- 172.21.15.142:0/2956617072 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c700095f0 con 0x7f0c80102df0 2024-03-29T04:44:54.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.610+0000 7f0c85a06700 1 --2- 172.21.15.142:0/2956617072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c80102df0 0x7f0c801985f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00b480 tx=0x7f0c7c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:54.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.610+0000 7f0c76ffd700 1 -- 172.21.15.142:0/2956617072 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c7c0118c0 con 0x7f0c80102df0 2024-03-29T04:44:54.613 INFO:tasks.workunit.client.0.smithi142.stdout:1/163: creat d8/db/dc/d19/d1d/d15/d21/d31/f3f x:0 0 0 2024-03-29T04:44:54.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.610+0000 7f0c87c6a700 1 -- 172.21.15.142:0/2956617072 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8019dd40 con 0x7f0c80102df0 2024-03-29T04:44:54.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.610+0000 7f0c76ffd700 1 -- 172.21.15.142:0/2956617072 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0c7c010380 con 0x7f0c80102df0 2024-03-29T04:44:54.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.610+0000 7f0c76ffd700 1 -- 172.21.15.142:0/2956617072 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c7c00f4f0 con 0x7f0c80102df0 2024-03-29T04:44:54.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.610+0000 7f0c87c6a700 1 -- 172.21.15.142:0/2956617072 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8019e2b0 con 0x7f0c80102df0 2024-03-29T04:44:54.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.611+0000 7f0c87c6a700 1 -- 172.21.15.142:0/2956617072 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8010ba60 con 0x7f0c80102df0 2024-03-29T04:44:54.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.612+0000 7f0c76ffd700 1 -- 172.21.15.142:0/2956617072 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0c7c0104f0 con 0x7f0c80102df0 2024-03-29T04:44:54.615 INFO:tasks.workunit.client.0.smithi142.stdout:9/128: creat d0/d2/f2c x:0 0 0 2024-03-29T04:44:54.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.613+0000 7f0c76ffd700 1 --2- 172.21.15.142:0/2956617072 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0c6c06c4e0 0x7f0c6c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:54.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.613+0000 7f0c76ffd700 1 -- 172.21.15.142:0/2956617072 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0c7c08b4f0 con 0x7f0c80102df0 2024-03-29T04:44:54.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.613+0000 7f0c85205700 1 --2- 172.21.15.142:0/2956617072 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0c6c06c4e0 0x7f0c6c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:54.616 INFO:tasks.workunit.client.0.smithi142.stdout:2/101: mkdir de/d11/d23 0 2024-03-29T04:44:54.616 INFO:tasks.workunit.client.0.smithi142.stdout:8/184: link d5/f1f d5/d1d/d24/f3a 0 2024-03-29T04:44:54.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.614+0000 7f0c85205700 1 --2- 172.21.15.142:0/2956617072 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0c6c06c4e0 0x7f0c6c06e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c700059d0 tx=0x7f0c70005920 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:54.617 INFO:tasks.workunit.client.0.smithi142.stdout:6/137: write f3 [1737709,115961] 0 2024-03-29T04:44:54.617 INFO:tasks.workunit.client.0.smithi142.stdout:2/102: symlink de/d11/d17/l24 0 2024-03-29T04:44:54.617 INFO:tasks.workunit.client.0.smithi142.stdout:8/185: creat d5/d10/d2d/f3b x:0 0 0 2024-03-29T04:44:54.617 INFO:tasks.workunit.client.0.smithi142.stdout:2/103: creat de/d11/d17/f25 x:0 0 0 2024-03-29T04:44:54.618 INFO:tasks.workunit.client.0.smithi142.stdout:6/138: link d1c/c1e d1c/c1f 0 2024-03-29T04:44:54.619 INFO:tasks.workunit.client.0.smithi142.stdout:8/186: mkdir d5/d10/d3c 0 2024-03-29T04:44:54.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.620+0000 7f0c76ffd700 1 -- 172.21.15.142:0/2956617072 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0c7c059700 con 0x7f0c80102df0 2024-03-29T04:44:54.619 INFO:tasks.workunit.client.0.smithi142.stdout:6/139: fsync f14 0 2024-03-29T04:44:54.622 INFO:tasks.workunit.client.0.smithi142.stdout:2/104: creat de/d11/d1e/d22/f26 x:0 0 0 2024-03-29T04:44:54.623 INFO:tasks.workunit.client.0.smithi142.stdout:3/150: dread d3/d1f/d34/f35 [0,4194304] 0 2024-03-29T04:44:54.625 INFO:tasks.workunit.client.0.smithi142.stdout:3/151: creat d3/d1f/d24/f39 x:0 0 0 2024-03-29T04:44:54.625 INFO:tasks.workunit.client.0.smithi142.stdout:2/105: dread f3 [0,4194304] 0 2024-03-29T04:44:54.625 INFO:tasks.workunit.client.0.smithi142.stdout:8/187: link d5/d1d/d24/l2c d5/d10/d3c/l3d 0 2024-03-29T04:44:54.628 INFO:tasks.workunit.client.0.smithi142.stdout:8/188: creat d5/d10/d3c/f3e x:0 0 0 2024-03-29T04:44:54.630 INFO:tasks.workunit.client.0.smithi142.stdout:8/189: write d5/d10/d2d/f3b [1048425,14793] 0 2024-03-29T04:44:54.633 INFO:tasks.workunit.client.0.smithi142.stdout:6/140: dread f4 [0,4194304] 0 2024-03-29T04:44:54.635 INFO:tasks.workunit.client.0.smithi142.stdout:8/190: unlink d5/d10/l14 0 2024-03-29T04:44:54.635 INFO:tasks.workunit.client.0.smithi142.stdout:3/152: dwrite d3/d29/f2e [0,4194304] 0 2024-03-29T04:44:54.635 INFO:tasks.workunit.client.0.smithi142.stdout:6/141: write f1a [2631348,39858] 0 2024-03-29T04:44:54.636 INFO:tasks.workunit.client.0.smithi142.stdout:8/191: chown d5/lf 188169568 1 2024-03-29T04:44:54.639 INFO:tasks.workunit.client.0.smithi142.stdout:8/192: unlink d5/d1d/d2f/f34 0 2024-03-29T04:44:54.642 INFO:tasks.workunit.client.0.smithi142.stdout:6/142: getdents d1c 0 2024-03-29T04:44:54.644 INFO:tasks.workunit.client.0.smithi142.stdout:3/153: rename d3/d1f/d34/d6/c25 to d3/d1f/d34/c3a 0 2024-03-29T04:44:54.645 INFO:tasks.workunit.client.0.smithi142.stdout:8/193: symlink d5/l3f 0 2024-03-29T04:44:54.647 INFO:tasks.workunit.client.0.smithi142.stdout:6/143: dread f1a [0,4194304] 0 2024-03-29T04:44:54.650 INFO:tasks.workunit.client.0.smithi142.stdout:3/154: creat d3/d29/f3b x:0 0 0 2024-03-29T04:44:54.650 INFO:tasks.workunit.client.0.smithi142.stdout:8/194: readlink d5/d1d/d24/l23 0 2024-03-29T04:44:54.651 INFO:tasks.workunit.client.0.smithi142.stdout:8/195: stat d5/cb 0 2024-03-29T04:44:54.651 INFO:tasks.workunit.client.0.smithi142.stdout:8/196: mknod d5/c40 0 2024-03-29T04:44:54.652 INFO:tasks.workunit.client.0.smithi142.stdout:6/144: dread f14 [0,4194304] 0 2024-03-29T04:44:54.652 INFO:tasks.workunit.client.0.smithi142.stdout:8/197: creat d5/d10/f41 x:0 0 0 2024-03-29T04:44:54.652 INFO:tasks.workunit.client.0.smithi142.stdout:8/198: fdatasync d5/f39 0 2024-03-29T04:44:54.653 INFO:tasks.workunit.client.0.smithi142.stdout:8/199: fdatasync d5/d10/d3c/f3e 0 2024-03-29T04:44:54.653 INFO:tasks.workunit.client.0.smithi142.stdout:6/145: rename c9 to d1c/c20 0 2024-03-29T04:44:54.656 INFO:tasks.workunit.client.0.smithi142.stdout:6/146: creat d1c/f21 x:0 0 0 2024-03-29T04:44:54.656 INFO:tasks.workunit.client.0.smithi142.stdout:8/200: dread d5/f22 [8388608,4194304] 0 2024-03-29T04:44:54.657 INFO:tasks.workunit.client.0.smithi142.stdout:3/155: truncate d3/d29/f2e 584005 0 2024-03-29T04:44:54.658 INFO:tasks.workunit.client.0.smithi142.stdout:6/147: symlink d1c/l22 0 2024-03-29T04:44:54.659 INFO:tasks.workunit.client.0.smithi142.stdout:3/156: read d3/d1f/d34/d6/f1c [1434880,128384] 0 2024-03-29T04:44:54.662 INFO:tasks.workunit.client.0.smithi142.stdout:6/148: dread f14 [0,4194304] 0 2024-03-29T04:44:54.662 INFO:tasks.workunit.client.0.smithi142.stdout:3/157: creat d3/f3c x:0 0 0 2024-03-29T04:44:54.666 INFO:tasks.workunit.client.0.smithi142.stdout:8/201: truncate d5/f22 8652921 0 2024-03-29T04:44:54.666 INFO:tasks.workunit.client.0.smithi142.stdout:8/202: readlink d5/d2e/l38 0 2024-03-29T04:44:54.666 INFO:tasks.workunit.client.0.smithi142.stdout:3/158: getdents d3/d29 0 2024-03-29T04:44:54.668 INFO:tasks.workunit.client.0.smithi142.stdout:8/203: mkdir d5/d1d/d2f/d42 0 2024-03-29T04:44:54.668 INFO:tasks.workunit.client.0.smithi142.stdout:3/159: rmdir d3/d1f/d34 39 2024-03-29T04:44:54.669 INFO:tasks.workunit.client.0.smithi142.stdout:6/149: dwrite d1c/f1d [0,4194304] 0 2024-03-29T04:44:54.669 INFO:tasks.workunit.client.0.smithi142.stdout:6/150: fsync f18 0 2024-03-29T04:44:54.674 INFO:tasks.workunit.client.0.smithi142.stdout:6/151: chown fc 23 1 2024-03-29T04:44:54.674 INFO:tasks.workunit.client.0.smithi142.stdout:6/152: chown d1c 127 1 2024-03-29T04:44:54.674 INFO:tasks.workunit.client.0.smithi142.stdout:3/160: getdents d3/d29/d2d 0 2024-03-29T04:44:54.675 INFO:tasks.workunit.client.0.smithi142.stdout:6/153: creat d1c/f23 x:0 0 0 2024-03-29T04:44:54.676 INFO:tasks.workunit.client.0.smithi142.stdout:6/154: readlink le 0 2024-03-29T04:44:54.677 INFO:tasks.workunit.client.0.smithi142.stdout:6/155: symlink d1c/l24 0 2024-03-29T04:44:54.678 INFO:tasks.workunit.client.0.smithi142.stdout:8/204: dread d5/f22 [4194304,4194304] 0 2024-03-29T04:44:54.707 INFO:tasks.workunit.client.0.smithi142.stdout:7/113: sync 2024-03-29T04:44:54.708 INFO:tasks.workunit.client.0.smithi142.stdout:6/156: creat d1c/f25 x:0 0 0 2024-03-29T04:44:54.709 INFO:tasks.workunit.client.0.smithi142.stdout:3/161: dread d3/d1f/d34/d6/f20 [0,4194304] 0 2024-03-29T04:44:54.709 INFO:tasks.workunit.client.0.smithi142.stdout:8/205: symlink d5/d2e/l43 0 2024-03-29T04:44:54.710 INFO:tasks.workunit.client.0.smithi142.stdout:7/114: mkdir d7/dc/de/d20 0 2024-03-29T04:44:54.710 INFO:tasks.workunit.client.0.smithi142.stdout:3/162: read d3/d1f/d34/f11 [3687518,93609] 0 2024-03-29T04:44:54.710 INFO:tasks.workunit.client.0.smithi142.stdout:6/157: dread f4 [0,4194304] 0 2024-03-29T04:44:54.711 INFO:tasks.workunit.client.0.smithi142.stdout:7/115: read - d7/f19 zero size 2024-03-29T04:44:54.712 INFO:tasks.workunit.client.0.smithi142.stdout:6/158: write fb [1185915,45126] 0 2024-03-29T04:44:54.715 INFO:tasks.workunit.client.0.smithi142.stdout:3/163: symlink d3/d1f/d34/l3d 0 2024-03-29T04:44:54.715 INFO:tasks.workunit.client.0.smithi142.stdout:3/164: readlink d3/d1f/d34/l3d 0 2024-03-29T04:44:54.716 INFO:tasks.workunit.client.0.smithi142.stdout:8/206: link d5/d10/d3c/l3d d5/d1d/d2f/d42/l44 0 2024-03-29T04:44:54.718 INFO:tasks.workunit.client.0.smithi142.stdout:3/165: fdatasync d3/d1f/d34/d6/f1c 0 2024-03-29T04:44:54.718 INFO:tasks.workunit.client.0.smithi142.stdout:7/116: getdents d7/d18 0 2024-03-29T04:44:54.723 INFO:tasks.workunit.client.0.smithi142.stdout:8/207: dwrite d5/f25 [0,4194304] 0 2024-03-29T04:44:54.725 INFO:tasks.workunit.client.0.smithi142.stdout:7/117: rmdir d7/dc 39 2024-03-29T04:44:54.731 INFO:tasks.workunit.client.0.smithi142.stdout:3/166: truncate d3/d1f/d34/d6/f14 2201737 0 2024-03-29T04:44:54.731 INFO:tasks.workunit.client.0.smithi142.stdout:7/118: dwrite d7/f9 [0,4194304] 0 2024-03-29T04:44:54.733 INFO:tasks.workunit.client.0.smithi142.stdout:3/167: dread - d3/d1f/d24/f39 zero size 2024-03-29T04:44:54.733 INFO:tasks.workunit.client.0.smithi142.stdout:7/119: dread - d7/d18/f1f zero size 2024-03-29T04:44:54.738 INFO:tasks.workunit.client.0.smithi142.stdout:8/208: dwrite d5/f19 [0,4194304] 0 2024-03-29T04:44:54.738 INFO:tasks.workunit.client.0.smithi142.stdout:8/209: chown d5/d10/d3c 1 1 2024-03-29T04:44:54.739 INFO:tasks.workunit.client.0.smithi142.stdout:8/210: chown d5/d10/c15 15600047 1 2024-03-29T04:44:54.740 INFO:tasks.workunit.client.0.smithi142.stdout:3/168: creat d3/d1f/d34/f3e x:0 0 0 2024-03-29T04:44:54.740 INFO:tasks.workunit.client.0.smithi142.stdout:7/120: read f6 [2224783,109616] 0 2024-03-29T04:44:54.740 INFO:tasks.workunit.client.0.smithi142.stdout:7/121: fsync d7/f19 0 2024-03-29T04:44:54.742 INFO:tasks.workunit.client.0.smithi142.stdout:8/211: unlink d5/f39 0 2024-03-29T04:44:54.743 INFO:tasks.workunit.client.0.smithi142.stdout:8/212: write d5/d10/d3c/f3e [935336,30851] 0 2024-03-29T04:44:54.746 INFO:tasks.workunit.client.0.smithi142.stdout:7/122: stat d7/dc/f1e 0 2024-03-29T04:44:54.748 INFO:tasks.workunit.client.0.smithi142.stdout:7/123: write d7/f1a [40832,16334] 0 2024-03-29T04:44:54.748 INFO:tasks.workunit.client.0.smithi142.stdout:8/213: chown d5/f1f 160036 1 2024-03-29T04:44:54.748 INFO:tasks.workunit.client.0.smithi142.stdout:3/169: creat d3/d29/d2d/f3f x:0 0 0 2024-03-29T04:44:54.748 INFO:tasks.workunit.client.0.smithi142.stdout:8/214: readlink d5/l3f 0 2024-03-29T04:44:54.749 INFO:tasks.workunit.client.0.smithi142.stdout:7/124: read d7/f9 [549542,107434] 0 2024-03-29T04:44:54.750 INFO:tasks.workunit.client.0.smithi142.stdout:7/125: write d7/fd [2535637,15916] 0 2024-03-29T04:44:54.751 INFO:tasks.workunit.client.0.smithi142.stdout:7/126: fdatasync d7/f19 0 2024-03-29T04:44:54.753 INFO:tasks.workunit.client.0.smithi142.stdout:7/127: getdents d7/dc/de/d20 0 2024-03-29T04:44:54.755 INFO:tasks.workunit.client.0.smithi142.stdout:7/128: rmdir d7 39 2024-03-29T04:44:54.757 INFO:tasks.workunit.client.0.smithi142.stdout:7/129: truncate d7/f1a 659768 0 2024-03-29T04:44:54.760 INFO:tasks.workunit.client.0.smithi142.stdout:7/130: dread d7/f1a [0,4194304] 0 2024-03-29T04:44:54.760 INFO:tasks.workunit.client.0.smithi142.stdout:7/131: dread - d7/d18/f1f zero size 2024-03-29T04:44:54.761 INFO:tasks.workunit.client.0.smithi142.stdout:7/132: chown d7/d18/f1f 260599 1 2024-03-29T04:44:54.767 INFO:tasks.workunit.client.0.smithi142.stdout:8/215: dwrite d5/d1d/d24/f3a [0,4194304] 0 2024-03-29T04:44:54.767 INFO:tasks.workunit.client.0.smithi142.stdout:3/170: dwrite d3/d1f/d34/f11 [0,4194304] 0 2024-03-29T04:44:54.770 INFO:tasks.workunit.client.0.smithi142.stdout:8/216: write d5/f18 [1587451,69597] 0 2024-03-29T04:44:54.771 INFO:tasks.workunit.client.0.smithi142.stdout:7/133: dwrite d7/dc/f1e [0,4194304] 0 2024-03-29T04:44:54.773 INFO:tasks.workunit.client.0.smithi142.stdout:7/134: chown f1 1361068 1 2024-03-29T04:44:54.780 INFO:tasks.workunit.client.0.smithi142.stdout:0/130: sync 2024-03-29T04:44:54.780 INFO:tasks.workunit.client.0.smithi142.stdout:5/215: sync 2024-03-29T04:44:54.789 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:54 smithi152 ceph-mon[37177]: pgmap v263: 65 pgs: 65 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 14 MiB/s wr, 280 op/s 2024-03-29T04:44:54.798 INFO:tasks.workunit.client.0.smithi142.stdout:8/217: mkdir d5/d1d/d2f/d45 0 2024-03-29T04:44:54.798 INFO:tasks.workunit.client.0.smithi142.stdout:3/171: truncate d3/d1f/d34/d6/f20 1622792 0 2024-03-29T04:44:54.799 INFO:tasks.workunit.client.0.smithi142.stdout:0/131: truncate d7/f21 222279 0 2024-03-29T04:44:54.800 INFO:tasks.workunit.client.0.smithi142.stdout:0/132: fdatasync d7/d9/f10 0 2024-03-29T04:44:54.801 INFO:tasks.workunit.client.0.smithi142.stdout:0/133: fdatasync d7/d9/d22/f1b 0 2024-03-29T04:44:54.803 INFO:tasks.workunit.client.0.smithi142.stdout:0/134: dread d7/d9/d22/f1b [0,4194304] 0 2024-03-29T04:44:54.804 INFO:tasks.workunit.client.0.smithi142.stdout:3/172: dread d3/d1f/d34/d6/f1c [0,4194304] 0 2024-03-29T04:44:54.846 INFO:tasks.workunit.client.0.smithi142.stdout:4/148: sync 2024-03-29T04:44:54.846 INFO:tasks.workunit.client.0.smithi142.stdout:4/149: chown d0/d12/c1d 1412790758 1 2024-03-29T04:44:54.847 INFO:tasks.workunit.client.0.smithi142.stdout:4/150: write d0/dc/d15/f18 [2260294,127497] 0 2024-03-29T04:44:54.870 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:54 smithi142 ceph-mon[29027]: pgmap v263: 65 pgs: 65 active+clean; 307 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 14 MiB/s wr, 280 op/s 2024-03-29T04:44:54.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.885+0000 7f0c87c6a700 1 -- 172.21.15.142:0/2956617072 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c80108940 con 0x7f0c6c06c4e0 2024-03-29T04:44:54.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.886+0000 7f0c76ffd700 1 -- 172.21.15.142:0/2956617072 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f0c80108940 con 0x7f0c6c06c4e0 2024-03-29T04:44:54.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.888+0000 7f0c87c6a700 1 -- 172.21.15.142:0/2956617072 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0c6c06c4e0 msgr2=0x7f0c6c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:54.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.888+0000 7f0c87c6a700 1 --2- 172.21.15.142:0/2956617072 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0c6c06c4e0 0x7f0c6c06e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c700059d0 tx=0x7f0c70005920 comp rx=0 tx=0).stop 2024-03-29T04:44:54.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.888+0000 7f0c87c6a700 1 -- 172.21.15.142:0/2956617072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c80102df0 msgr2=0x7f0c801985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:54.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.888+0000 7f0c87c6a700 1 --2- 172.21.15.142:0/2956617072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c80102df0 0x7f0c801985f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00b480 tx=0x7f0c7c00b840 comp rx=0 tx=0).stop 2024-03-29T04:44:54.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.888+0000 7f0c87c6a700 1 -- 172.21.15.142:0/2956617072 shutdown_connections 2024-03-29T04:44:54.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.888+0000 7f0c87c6a700 1 --2- 172.21.15.142:0/2956617072 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f0c6c06c4e0 0x7f0c6c06e9a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:54.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.888+0000 7f0c87c6a700 1 --2- 172.21.15.142:0/2956617072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c80102df0 0x7f0c801985f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:54.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.888+0000 7f0c87c6a700 1 --2- 172.21.15.142:0/2956617072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c80103ff0 0x7f0c80198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:54.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.888+0000 7f0c87c6a700 1 -- 172.21.15.142:0/2956617072 >> 172.21.15.142:0/2956617072 conn(0x7f0c800fe390 msgr2=0x7f0c80107220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:54.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.888+0000 7f0c87c6a700 1 -- 172.21.15.142:0/2956617072 shutdown_connections 2024-03-29T04:44:54.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:54.888+0000 7f0c87c6a700 1 -- 172.21.15.142:0/2956617072 wait complete. 2024-03-29T04:44:54.898 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:44:54.904 INFO:tasks.workunit.client.0.smithi142.stdout:1/164: sync 2024-03-29T04:44:54.904 INFO:tasks.workunit.client.0.smithi142.stdout:2/106: sync 2024-03-29T04:44:54.904 INFO:tasks.workunit.client.0.smithi142.stdout:6/159: sync 2024-03-29T04:44:54.904 INFO:tasks.workunit.client.0.smithi142.stdout:9/129: sync 2024-03-29T04:44:54.904 INFO:tasks.workunit.client.0.smithi142.stdout:6/160: dread - d1c/f25 zero size 2024-03-29T04:44:54.905 INFO:tasks.workunit.client.0.smithi142.stdout:1/165: readlink d8/db/dc/d19/l28 0 2024-03-29T04:44:54.906 INFO:tasks.workunit.client.0.smithi142.stdout:2/107: dread f7 [0,4194304] 0 2024-03-29T04:44:54.906 INFO:tasks.workunit.client.0.smithi142.stdout:9/130: dread d0/f22 [0,4194304] 0 2024-03-29T04:44:54.907 INFO:tasks.workunit.client.0.smithi142.stdout:9/131: truncate d0/d2/f1f 1187423 0 2024-03-29T04:44:54.907 INFO:tasks.workunit.client.0.smithi142.stdout:9/132: write d0/d2/d6/d16/f2b [527423,2563] 0 2024-03-29T04:44:54.948 INFO:tasks.workunit.client.0.smithi142.stdout:7/135: creat d7/dc/de/d20/f21 x:0 0 0 2024-03-29T04:44:54.948 INFO:tasks.workunit.client.0.smithi142.stdout:7/136: chown d7/dc/de 7672492 1 2024-03-29T04:44:54.948 INFO:tasks.workunit.client.0.smithi142.stdout:5/216: creat df/d1a/d21/f3e x:0 0 0 2024-03-29T04:44:54.950 INFO:tasks.workunit.client.0.smithi142.stdout:8/218: rename l4 to d5/d2e/l46 0 2024-03-29T04:44:54.950 INFO:tasks.workunit.client.0.smithi142.stdout:6/161: rename d1c to d1c/d26 22 2024-03-29T04:44:54.950 INFO:tasks.workunit.client.0.smithi142.stdout:8/219: chown d5/l17 49394274 1 2024-03-29T04:44:54.950 INFO:tasks.workunit.client.0.smithi142.stdout:6/162: chown f14 352591304 1 2024-03-29T04:44:54.951 INFO:tasks.workunit.client.0.smithi142.stdout:0/135: creat d7/f2b x:0 0 0 2024-03-29T04:44:54.951 INFO:tasks.workunit.client.0.smithi142.stdout:3/173: mkdir d3/d29/d2b/d40 0 2024-03-29T04:44:54.952 INFO:tasks.workunit.client.0.smithi142.stdout:4/151: mkdir d0/dc/d15/d32 0 2024-03-29T04:44:54.954 INFO:tasks.workunit.client.0.smithi142.stdout:1/166: mknod d8/db/dc/d19/d1d/d1f/c40 0 2024-03-29T04:44:54.956 INFO:tasks.workunit.client.0.smithi142.stdout:7/137: mknod d7/d18/c22 0 2024-03-29T04:44:54.956 INFO:tasks.workunit.client.0.smithi142.stdout:8/220: dwrite d5/d10/f41 [0,4194304] 0 2024-03-29T04:44:54.957 INFO:tasks.workunit.client.0.smithi142.stdout:6/163: creat d1c/f27 x:0 0 0 2024-03-29T04:44:54.957 INFO:tasks.workunit.client.0.smithi142.stdout:0/136: rmdir d7/d9 39 2024-03-29T04:44:54.957 INFO:tasks.workunit.client.0.smithi142.stdout:3/174: creat d3/d1f/d24/f41 x:0 0 0 2024-03-29T04:44:54.959 INFO:tasks.workunit.client.0.smithi142.stdout:0/137: write d7/d13/f28 [738111,49386] 0 2024-03-29T04:44:54.959 INFO:tasks.workunit.client.0.smithi142.stdout:2/108: fsync fa 0 2024-03-29T04:44:54.962 INFO:tasks.workunit.client.0.smithi142.stdout:9/133: creat d0/d2/d6/d2a/f2d x:0 0 0 2024-03-29T04:44:54.964 INFO:tasks.workunit.client.0.smithi142.stdout:7/138: fsync d7/fa 0 2024-03-29T04:44:54.964 INFO:tasks.workunit.client.0.smithi142.stdout:1/167: chown f3 105677 1 2024-03-29T04:44:54.965 INFO:tasks.workunit.client.0.smithi142.stdout:6/164: unlink l1b 0 2024-03-29T04:44:54.965 INFO:tasks.workunit.client.0.smithi142.stdout:5/217: dread f3 [0,4194304] 0 2024-03-29T04:44:54.965 INFO:tasks.workunit.client.0.smithi142.stdout:7/139: chown d7 49 1 2024-03-29T04:44:54.966 INFO:tasks.workunit.client.0.smithi142.stdout:6/165: stat l13 0 2024-03-29T04:44:54.967 INFO:tasks.workunit.client.0.smithi142.stdout:7/140: write d7/fd [4942211,82264] 0 2024-03-29T04:44:54.969 INFO:tasks.workunit.client.0.smithi142.stdout:1/168: rmdir d8 39 2024-03-29T04:44:54.970 INFO:tasks.workunit.client.0.smithi142.stdout:8/221: mknod d5/d1d/d2f/d42/c47 0 2024-03-29T04:44:54.973 INFO:tasks.workunit.client.0.smithi142.stdout:2/109: mkdir de/d11/d1e/d22/d27 0 2024-03-29T04:44:54.973 INFO:tasks.workunit.client.0.smithi142.stdout:6/166: dwrite fb [0,4194304] 0 2024-03-29T04:44:54.974 INFO:tasks.workunit.client.0.smithi142.stdout:6/167: write f14 [1369427,5121] 0 2024-03-29T04:44:54.975 INFO:tasks.workunit.client.0.smithi142.stdout:3/175: rename d3/d1f/d34/c32 to d3/d29/d2b/c42 0 2024-03-29T04:44:54.977 INFO:tasks.workunit.client.0.smithi142.stdout:5/218: creat df/d1a/f3f x:0 0 0 2024-03-29T04:44:54.992 INFO:tasks.workunit.client.0.smithi142.stdout:8/222: mkdir d5/d1d/d2f/d42/d48 0 2024-03-29T04:44:54.993 INFO:tasks.workunit.client.0.smithi142.stdout:5/219: chown df/d16/l1b 89 1 2024-03-29T04:44:54.994 INFO:tasks.workunit.client.0.smithi142.stdout:6/168: symlink d1c/l28 0 2024-03-29T04:44:54.995 INFO:tasks.workunit.client.0.smithi142.stdout:8/223: read d5/d10/d2d/f3b [799393,61630] 0 2024-03-29T04:44:54.995 INFO:tasks.workunit.client.0.smithi142.stdout:0/138: rename d7/l1a to d7/d13/l2c 0 2024-03-29T04:44:54.996 INFO:tasks.workunit.client.0.smithi142.stdout:3/176: mkdir d3/d1f/d34/d6/d43 0 2024-03-29T04:44:54.997 INFO:tasks.workunit.client.0.smithi142.stdout:5/220: truncate df/d16/d25/f37 490208 0 2024-03-29T04:44:54.998 INFO:tasks.workunit.client.0.smithi142.stdout:8/224: write d5/d10/d3c/f3e [1554789,93099] 0 2024-03-29T04:44:54.998 INFO:tasks.workunit.client.0.smithi142.stdout:9/134: link d0/d2/c5 d0/d12/c2e 0 2024-03-29T04:44:54.999 INFO:tasks.workunit.client.0.smithi142.stdout:2/110: creat de/d11/d23/f28 x:0 0 0 2024-03-29T04:44:55.000 INFO:tasks.workunit.client.0.smithi142.stdout:3/177: dread d3/d1f/d34/f35 [0,4194304] 0 2024-03-29T04:44:55.000 INFO:tasks.workunit.client.0.smithi142.stdout:3/178: chown d3/d1f/d34/c2c 0 1 2024-03-29T04:44:55.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.011+0000 7fdbff2da700 1 -- 172.21.15.142:0/3120832351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf80fee80 msgr2=0x7fdbf80ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:55.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.011+0000 7fdbff2da700 1 --2- 172.21.15.142:0/3120832351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf80fee80 0x7fdbf80ff2a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009910 tx=0x7fdbf4009c20 comp rx=0 tx=0).stop 2024-03-29T04:44:55.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.011+0000 7fdbff2da700 1 -- 172.21.15.142:0/3120832351 shutdown_connections 2024-03-29T04:44:55.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.011+0000 7fdbff2da700 1 --2- 172.21.15.142:0/3120832351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf8101f00 0x7fdbf8102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.011+0000 7fdbff2da700 1 --2- 172.21.15.142:0/3120832351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf80fee80 0x7fdbf80ff2a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.011+0000 7fdbff2da700 1 -- 172.21.15.142:0/3120832351 >> 172.21.15.142:0/3120832351 conn(0x7fdbf80fa7c0 msgr2=0x7fdbf80fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:55.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.012+0000 7fdbff2da700 1 -- 172.21.15.142:0/3120832351 shutdown_connections 2024-03-29T04:44:55.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.012+0000 7fdbff2da700 1 -- 172.21.15.142:0/3120832351 wait complete. 2024-03-29T04:44:55.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.012+0000 7fdbff2da700 1 Processor -- start 2024-03-29T04:44:55.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.012+0000 7fdbff2da700 1 -- start start 2024-03-29T04:44:55.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.012+0000 7fdbff2da700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf80fee80 0x7fdbf8101860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:55.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.012+0000 7fdbff2da700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf8101f00 0x7fdbf80ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:55.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.012+0000 7fdbff2da700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf81003f0 con 0x7fdbf80fee80 2024-03-29T04:44:55.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.012+0000 7fdbff2da700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf8100530 con 0x7fdbf8101f00 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.012+0000 7fdbfd076700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf80fee80 0x7fdbf8101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.012+0000 7fdbfd076700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf80fee80 0x7fdbf8101860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40114/0 (socket says 172.21.15.142:40114) 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.012+0000 7fdbfd076700 1 -- 172.21.15.142:0/2301279435 learned_addr learned my addr 172.21.15.142:0/2301279435 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.013+0000 7fdbfc875700 1 --2- 172.21.15.142:0/2301279435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf8101f00 0x7fdbf80ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.013+0000 7fdbfd076700 1 -- 172.21.15.142:0/2301279435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf8101f00 msgr2=0x7fdbf80ffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.013+0000 7fdbfd076700 1 --2- 172.21.15.142:0/2301279435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf8101f00 0x7fdbf80ffeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.013+0000 7fdbfd076700 1 -- 172.21.15.142:0/2301279435 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf40095f0 con 0x7fdbf80fee80 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.013+0000 7fdbfd076700 1 --2- 172.21.15.142:0/2301279435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf80fee80 0x7fdbf8101860 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400bda0 tx=0x7fdbf400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.013+0000 7fdbee7fc700 1 -- 172.21.15.142:0/2301279435 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbf401d070 con 0x7fdbf80fee80 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.013+0000 7fdbee7fc700 1 -- 172.21.15.142:0/2301279435 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdbf4004c50 con 0x7fdbf80fee80 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.013+0000 7fdbff2da700 1 -- 172.21.15.142:0/2301279435 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf81007b0 con 0x7fdbf80fee80 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.013+0000 7fdbee7fc700 1 -- 172.21.15.142:0/2301279435 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbf400f690 con 0x7fdbf80fee80 2024-03-29T04:44:55.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.013+0000 7fdbff2da700 1 -- 172.21.15.142:0/2301279435 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf8100ca0 con 0x7fdbf80fee80 2024-03-29T04:44:55.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.014+0000 7fdbff2da700 1 -- 172.21.15.142:0/2301279435 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf8109720 con 0x7fdbf80fee80 2024-03-29T04:44:55.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.015+0000 7fdbee7fc700 1 -- 172.21.15.142:0/2301279435 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fdbf40229b0 con 0x7fdbf80fee80 2024-03-29T04:44:55.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.015+0000 7fdbee7fc700 1 --2- 172.21.15.142:0/2301279435 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fdbe406c4e0 0x7fdbe406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:55.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.015+0000 7fdbfc875700 1 --2- 172.21.15.142:0/2301279435 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fdbe406c4e0 0x7fdbe406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:55.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.015+0000 7fdbee7fc700 1 -- 172.21.15.142:0/2301279435 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdbf408cfb0 con 0x7fdbf80fee80 2024-03-29T04:44:55.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.016+0000 7fdbfc875700 1 --2- 172.21.15.142:0/2301279435 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fdbe406c4e0 0x7fdbe406e9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8005fd0 tx=0x7fdbe800a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:55.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.020+0000 7fdbee7fc700 1 -- 172.21.15.142:0/2301279435 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdbf405b240 con 0x7fdbf80fee80 2024-03-29T04:44:55.054 INFO:tasks.workunit.client.0.smithi142.stdout:1/169: creat d8/db/d37/f41 x:0 0 0 2024-03-29T04:44:55.054 INFO:tasks.workunit.client.0.smithi142.stdout:6/169: mknod d1c/c29 0 2024-03-29T04:44:55.056 INFO:tasks.workunit.client.0.smithi142.stdout:0/139: creat d7/d13/f2d x:0 0 0 2024-03-29T04:44:55.060 INFO:tasks.workunit.client.0.smithi142.stdout:6/170: symlink d1c/l2a 0 2024-03-29T04:44:55.061 INFO:tasks.workunit.client.0.smithi142.stdout:1/170: symlink d8/db/dc/d19/d1d/d15/l42 0 2024-03-29T04:44:55.062 INFO:tasks.workunit.client.0.smithi142.stdout:0/140: dread - d7/d9/d22/f23 zero size 2024-03-29T04:44:55.063 INFO:tasks.workunit.client.0.smithi142.stdout:5/221: creat df/d2c/f40 x:0 0 0 2024-03-29T04:44:55.064 INFO:tasks.workunit.client.0.smithi142.stdout:5/222: write fe [3513471,98732] 0 2024-03-29T04:44:55.064 INFO:tasks.workunit.client.0.smithi142.stdout:5/223: stat df/l34 0 2024-03-29T04:44:55.064 INFO:tasks.workunit.client.0.smithi142.stdout:6/171: creat d1c/f2b x:0 0 0 2024-03-29T04:44:55.067 INFO:tasks.workunit.client.0.smithi142.stdout:0/141: creat d7/d9/f2e x:0 0 0 2024-03-29T04:44:55.067 INFO:tasks.workunit.client.0.smithi142.stdout:0/142: chown d7/c11 98011 1 2024-03-29T04:44:55.070 INFO:tasks.workunit.client.0.smithi142.stdout:2/111: creat de/f29 x:0 0 0 2024-03-29T04:44:55.071 INFO:tasks.workunit.client.0.smithi142.stdout:1/171: creat d8/db/d3d/f43 x:0 0 0 2024-03-29T04:44:55.072 INFO:tasks.workunit.client.0.smithi142.stdout:6/172: mknod d1c/c2c 0 2024-03-29T04:44:55.073 INFO:tasks.workunit.client.0.smithi142.stdout:9/135: getdents d0/d2/d6 0 2024-03-29T04:44:55.076 INFO:tasks.workunit.client.0.smithi142.stdout:1/172: mkdir d8/db/dc/d19/d1d/d2d/d44 0 2024-03-29T04:44:55.077 INFO:tasks.workunit.client.0.smithi142.stdout:6/173: chown d1c/c1f 6966903 1 2024-03-29T04:44:55.081 INFO:tasks.workunit.client.0.smithi142.stdout:0/143: mkdir d7/d13/d27/d2f 0 2024-03-29T04:44:55.083 INFO:tasks.workunit.client.0.smithi142.stdout:2/112: symlink de/l2a 0 2024-03-29T04:44:55.084 INFO:tasks.workunit.client.0.smithi142.stdout:1/173: mknod d8/db/dc/d19/d1d/d15/d21/c45 0 2024-03-29T04:44:55.085 INFO:tasks.workunit.client.0.smithi142.stdout:1/174: write d8/db/dc/d19/d1d/d15/f17 [3226259,38377] 0 2024-03-29T04:44:55.087 INFO:tasks.workunit.client.0.smithi142.stdout:9/136: rename d0/d12/f13 to d0/d2/f2f 0 2024-03-29T04:44:55.088 INFO:tasks.workunit.client.0.smithi142.stdout:2/113: mkdir de/d11/d2b 0 2024-03-29T04:44:55.091 INFO:tasks.workunit.client.0.smithi142.stdout:1/175: creat d8/db/dc/d19/d1d/d1f/f46 x:0 0 0 2024-03-29T04:44:55.092 INFO:tasks.workunit.client.0.smithi142.stdout:9/137: creat d0/d2/d6/f30 x:0 0 0 2024-03-29T04:44:55.095 INFO:tasks.workunit.client.0.smithi142.stdout:2/114: mknod de/d11/d2b/c2c 0 2024-03-29T04:44:55.097 INFO:tasks.workunit.client.0.smithi142.stdout:1/176: link d8/fa d8/db/dc/d19/d1d/d15/f47 0 2024-03-29T04:44:55.098 INFO:tasks.workunit.client.0.smithi142.stdout:9/138: dwrite d0/f1d [0,4194304] 0 2024-03-29T04:44:55.100 INFO:tasks.workunit.client.0.smithi142.stdout:2/115: mknod de/d11/d17/c2d 0 2024-03-29T04:44:55.103 INFO:tasks.workunit.client.0.smithi142.stdout:2/116: unlink fb 0 2024-03-29T04:44:55.107 INFO:tasks.workunit.client.0.smithi142.stdout:2/117: dread f7 [0,4194304] 0 2024-03-29T04:44:55.108 INFO:tasks.workunit.client.0.smithi142.stdout:7/141: sync 2024-03-29T04:44:55.117 INFO:tasks.workunit.client.0.smithi142.stdout:7/142: dwrite d7/f9 [0,4194304] 0 2024-03-29T04:44:55.118 INFO:tasks.workunit.client.0.smithi142.stdout:7/143: read d7/dc/f10 [574437,59190] 0 2024-03-29T04:44:55.119 INFO:tasks.workunit.client.0.smithi142.stdout:7/144: write d7/d18/f1f [973128,15517] 0 2024-03-29T04:44:55.125 INFO:tasks.workunit.client.0.smithi142.stdout:7/145: rename d7/d18/c22 to d7/c23 0 2024-03-29T04:44:55.128 INFO:tasks.workunit.client.0.smithi142.stdout:7/146: creat d7/d18/f24 x:0 0 0 2024-03-29T04:44:55.129 INFO:tasks.workunit.client.0.smithi142.stdout:7/147: truncate d7/d18/f1f 1162247 0 2024-03-29T04:44:55.135 INFO:tasks.workunit.client.0.smithi142.stdout:7/148: dwrite d7/d18/f1f [0,4194304] 0 2024-03-29T04:44:55.143 INFO:tasks.workunit.client.0.smithi142.stdout:7/149: dwrite d7/dc/de/f16 [0,4194304] 0 2024-03-29T04:44:55.149 INFO:tasks.workunit.client.0.smithi142.stdout:7/150: dread d7/dc/de/f16 [0,4194304] 0 2024-03-29T04:44:55.149 INFO:tasks.workunit.client.0.smithi142.stdout:7/151: dread - d7/d18/f24 zero size 2024-03-29T04:44:55.150 INFO:tasks.workunit.client.0.smithi142.stdout:7/152: dread - d7/d18/f24 zero size 2024-03-29T04:44:55.155 INFO:tasks.workunit.client.0.smithi142.stdout:7/153: dwrite d7/f9 [0,4194304] 0 2024-03-29T04:44:55.162 INFO:tasks.workunit.client.0.smithi142.stdout:7/154: link d7/c8 d7/d18/c25 0 2024-03-29T04:44:55.168 INFO:tasks.workunit.client.0.smithi142.stdout:7/155: dwrite d7/dc/de/f16 [0,4194304] 0 2024-03-29T04:44:55.169 INFO:tasks.workunit.client.0.smithi142.stdout:7/156: write d7/f9 [1587286,96871] 0 2024-03-29T04:44:55.173 INFO:tasks.workunit.client.0.smithi142.stdout:7/157: creat d7/dc/de/f26 x:0 0 0 2024-03-29T04:44:55.174 INFO:tasks.workunit.client.0.smithi142.stdout:7/158: chown d7/d18/f24 15 1 2024-03-29T04:44:55.175 INFO:tasks.workunit.client.0.smithi142.stdout:7/159: fsync d7/f9 0 2024-03-29T04:44:55.178 INFO:tasks.workunit.client.0.smithi142.stdout:7/160: chown d7/d18/c25 30525 1 2024-03-29T04:44:55.180 INFO:tasks.workunit.client.0.smithi142.stdout:7/161: mkdir d7/d27 0 2024-03-29T04:44:55.183 INFO:tasks.workunit.client.0.smithi142.stdout:7/162: unlink d7/dc/de/f14 0 2024-03-29T04:44:55.186 INFO:tasks.workunit.client.0.smithi142.stdout:7/163: dread d7/f9 [0,4194304] 0 2024-03-29T04:44:55.212 INFO:tasks.workunit.client.0.smithi142.stdout:4/152: truncate d0/dc/d15/f18 2254054 0 2024-03-29T04:44:55.213 INFO:tasks.workunit.client.0.smithi142.stdout:4/153: write d0/d9/d11/f23 [544415,35277] 0 2024-03-29T04:44:55.214 INFO:tasks.workunit.client.0.smithi142.stdout:4/154: chown d0/d9/d19/l2f 2690 1 2024-03-29T04:44:55.219 INFO:tasks.workunit.client.0.smithi142.stdout:4/155: dwrite d0/f21 [0,4194304] 0 2024-03-29T04:44:55.221 INFO:tasks.workunit.client.0.smithi142.stdout:8/225: sync 2024-03-29T04:44:55.225 INFO:tasks.workunit.client.0.smithi142.stdout:4/156: dwrite d0/d9/d19/f1a [0,4194304] 0 2024-03-29T04:44:55.228 INFO:tasks.workunit.client.0.smithi142.stdout:5/224: truncate df/d16/f1e 437818 0 2024-03-29T04:44:55.230 INFO:tasks.workunit.client.0.smithi142.stdout:8/226: dwrite d5/d10/f41 [0,4194304] 0 2024-03-29T04:44:55.231 INFO:tasks.workunit.client.0.smithi142.stdout:4/157: creat d0/dc/f33 x:0 0 0 2024-03-29T04:44:55.237 INFO:tasks.workunit.client.0.smithi142.stdout:9/139: truncate d0/d2/f1f 636190 0 2024-03-29T04:44:55.237 INFO:tasks.workunit.client.0.smithi142.stdout:5/225: rename df/c13 to df/d1a/d21/d27/c41 0 2024-03-29T04:44:55.239 INFO:tasks.workunit.client.0.smithi142.stdout:5/226: write fe [2514760,118098] 0 2024-03-29T04:44:55.240 INFO:tasks.workunit.client.0.smithi142.stdout:8/227: mknod d5/d10/d3c/c49 0 2024-03-29T04:44:55.244 INFO:tasks.workunit.client.0.smithi142.stdout:2/118: dwrite de/d11/f21 [0,4194304] 0 2024-03-29T04:44:55.246 INFO:tasks.workunit.client.0.smithi142.stdout:9/140: mkdir d0/d2/d6/d31 0 2024-03-29T04:44:55.247 INFO:tasks.workunit.client.0.smithi142.stdout:5/227: symlink df/d16/d24/d2e/l42 0 2024-03-29T04:44:55.250 INFO:tasks.workunit.client.0.smithi142.stdout:8/228: creat d5/d10/d2d/f4a x:0 0 0 2024-03-29T04:44:55.253 INFO:tasks.workunit.client.0.smithi142.stdout:5/228: symlink df/d1a/d28/l43 0 2024-03-29T04:44:55.254 INFO:tasks.workunit.client.0.smithi142.stdout:8/229: creat d5/d1d/d2f/d42/f4b x:0 0 0 2024-03-29T04:44:55.255 INFO:tasks.workunit.client.0.smithi142.stdout:5/229: read - df/d1a/d28/f35 zero size 2024-03-29T04:44:55.256 INFO:tasks.workunit.client.0.smithi142.stdout:5/230: stat fe 0 2024-03-29T04:44:55.256 INFO:tasks.workunit.client.0.smithi142.stdout:8/230: chown d5/f19 73800844 1 2024-03-29T04:44:55.257 INFO:tasks.workunit.client.0.smithi142.stdout:3/179: sync 2024-03-29T04:44:55.258 INFO:tasks.workunit.client.0.smithi142.stdout:9/141: dwrite d0/d2/d6/d16/f2b [0,4194304] 0 2024-03-29T04:44:55.261 INFO:tasks.workunit.client.0.smithi142.stdout:8/231: dread - d5/d10/d2d/f4a zero size 2024-03-29T04:44:55.264 INFO:tasks.workunit.client.0.smithi142.stdout:7/164: dwrite d7/d18/f1f [4194304,4194304] 0 2024-03-29T04:44:55.268 INFO:tasks.workunit.client.0.smithi142.stdout:8/232: mkdir d5/d10/d4c 0 2024-03-29T04:44:55.274 INFO:tasks.workunit.client.0.smithi142.stdout:9/142: creat d0/f32 x:0 0 0 2024-03-29T04:44:55.280 INFO:tasks.workunit.client.0.smithi142.stdout:2/119: symlink de/d11/d1e/d22/l2e 0 2024-03-29T04:44:55.282 INFO:tasks.workunit.client.0.smithi142.stdout:9/143: symlink d0/d2/d28/l33 0 2024-03-29T04:44:55.282 INFO:tasks.workunit.client.0.smithi142.stdout:3/180: dwrite d3/d29/d2d/f3f [0,4194304] 0 2024-03-29T04:44:55.286 INFO:tasks.workunit.client.0.smithi142.stdout:8/233: dread d5/d10/d3c/f3e [0,4194304] 0 2024-03-29T04:44:55.286 INFO:tasks.workunit.client.0.smithi142.stdout:4/158: dwrite d0/d9/d11/ff [0,4194304] 0 2024-03-29T04:44:55.286 INFO:tasks.workunit.client.0.smithi142.stdout:5/231: truncate fa 3302526 0 2024-03-29T04:44:55.290 INFO:tasks.workunit.client.0.smithi142.stdout:8/234: fsync d5/f22 0 2024-03-29T04:44:55.291 INFO:tasks.workunit.client.0.smithi142.stdout:9/144: write d0/d12/f24 [209355,95169] 0 2024-03-29T04:44:55.291 INFO:tasks.workunit.client.0.smithi142.stdout:7/165: dwrite d7/dc/de/d20/f21 [0,4194304] 0 2024-03-29T04:44:55.294 INFO:tasks.workunit.client.0.smithi142.stdout:3/181: read d3/d1f/d34/d6/f20 [492503,818] 0 2024-03-29T04:44:55.294 INFO:tasks.workunit.client.0.smithi142.stdout:4/159: write d0/d9/d11/f23 [185009,111779] 0 2024-03-29T04:44:55.297 INFO:tasks.workunit.client.0.smithi142.stdout:7/166: chown d7/f1a 7794649 1 2024-03-29T04:44:55.301 INFO:tasks.workunit.client.0.smithi142.stdout:5/232: fdatasync df/d16/f1e 0 2024-03-29T04:44:55.302 INFO:tasks.workunit.client.0.smithi142.stdout:4/160: unlink d0/dc/f27 0 2024-03-29T04:44:55.303 INFO:tasks.workunit.client.0.smithi142.stdout:2/120: dwrite de/f16 [0,4194304] 0 2024-03-29T04:44:55.304 INFO:tasks.workunit.client.0.smithi142.stdout:2/121: read f3 [343858,114080] 0 2024-03-29T04:44:55.311 INFO:tasks.workunit.client.0.smithi142.stdout:4/161: dwrite d0/d12/f1e [0,4194304] 0 2024-03-29T04:44:55.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.314+0000 7fdbff2da700 1 -- 172.21.15.142:0/2301279435 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbf8106850 con 0x7fdbe406c4e0 2024-03-29T04:44:55.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.315+0000 7fdbee7fc700 1 -- 172.21.15.142:0/2301279435 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fdbf8106850 con 0x7fdbe406c4e0 2024-03-29T04:44:55.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.317+0000 7fdbff2da700 1 -- 172.21.15.142:0/2301279435 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fdbe406c4e0 msgr2=0x7fdbe406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:55.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.317+0000 7fdbff2da700 1 --2- 172.21.15.142:0/2301279435 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fdbe406c4e0 0x7fdbe406e9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8005fd0 tx=0x7fdbe800a560 comp rx=0 tx=0).stop 2024-03-29T04:44:55.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.317+0000 7fdbff2da700 1 -- 172.21.15.142:0/2301279435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf80fee80 msgr2=0x7fdbf8101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:55.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.317+0000 7fdbff2da700 1 --2- 172.21.15.142:0/2301279435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf80fee80 0x7fdbf8101860 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400bda0 tx=0x7fdbf400be80 comp rx=0 tx=0).stop 2024-03-29T04:44:55.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.317+0000 7fdbff2da700 1 -- 172.21.15.142:0/2301279435 shutdown_connections 2024-03-29T04:44:55.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.317+0000 7fdbff2da700 1 --2- 172.21.15.142:0/2301279435 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fdbe406c4e0 0x7fdbe406e9a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.317+0000 7fdbff2da700 1 --2- 172.21.15.142:0/2301279435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbf80fee80 0x7fdbf8101860 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.317+0000 7fdbff2da700 1 --2- 172.21.15.142:0/2301279435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf8101f00 0x7fdbf80ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.317+0000 7fdbff2da700 1 -- 172.21.15.142:0/2301279435 >> 172.21.15.142:0/2301279435 conn(0x7fdbf80fa7c0 msgr2=0x7fdbf8105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:55.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.317+0000 7fdbff2da700 1 -- 172.21.15.142:0/2301279435 shutdown_connections 2024-03-29T04:44:55.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.317+0000 7fdbff2da700 1 -- 172.21.15.142:0/2301279435 wait complete. 2024-03-29T04:44:55.321 INFO:tasks.workunit.client.0.smithi142.stdout:1/177: sync 2024-03-29T04:44:55.321 INFO:tasks.workunit.client.0.smithi142.stdout:6/174: sync 2024-03-29T04:44:55.321 INFO:tasks.workunit.client.0.smithi142.stdout:0/144: sync 2024-03-29T04:44:55.321 INFO:tasks.workunit.client.0.smithi142.stdout:8/235: symlink d5/d1d/d2f/d45/l4d 0 2024-03-29T04:44:55.322 INFO:tasks.workunit.client.0.smithi142.stdout:6/175: fdatasync f1a 0 2024-03-29T04:44:55.324 INFO:tasks.workunit.client.0.smithi142.stdout:5/233: write df/d16/d25/f37 [923112,113067] 0 2024-03-29T04:44:55.325 INFO:tasks.workunit.client.0.smithi142.stdout:3/182: rmdir d3/d1f/d34/d6/d43 0 2024-03-29T04:44:55.328 INFO:tasks.workunit.client.0.smithi142.stdout:0/145: dwrite d7/d9/f1c [12582912,4194304] 0 2024-03-29T04:44:55.333 INFO:tasks.workunit.client.0.smithi142.stdout:3/183: dwrite d3/d1f/d24/f39 [0,4194304] 0 2024-03-29T04:44:55.352 INFO:tasks.workunit.client.0.smithi142.stdout:1/178: chown c4 422224236 1 2024-03-29T04:44:55.353 INFO:tasks.workunit.client.0.smithi142.stdout:7/167: link f6 d7/d18/f28 0 2024-03-29T04:44:55.353 INFO:tasks.workunit.client.0.smithi142.stdout:9/145: dwrite d0/d2/f11 [0,4194304] 0 2024-03-29T04:44:55.354 INFO:tasks.workunit.client.0.smithi142.stdout:8/236: mknod d5/d1d/d24/c4e 0 2024-03-29T04:44:55.355 INFO:tasks.workunit.client.0.smithi142.stdout:6/176: symlink d1c/l2d 0 2024-03-29T04:44:55.355 INFO:tasks.workunit.client.0.smithi142.stdout:6/177: chown d1c/l24 230 1 2024-03-29T04:44:55.356 INFO:tasks.workunit.client.0.smithi142.stdout:6/178: stat f14 0 2024-03-29T04:44:55.356 INFO:tasks.workunit.client.0.smithi142.stdout:6/179: chown d1c/c1e 1038 1 2024-03-29T04:44:55.365 INFO:tasks.workunit.client.0.smithi142.stdout:1/179: creat d8/db/dc/d19/d1d/f48 x:0 0 0 2024-03-29T04:44:55.367 INFO:tasks.workunit.client.0.smithi142.stdout:9/146: rename d0/d2/d6/d2a to d0/d2/d28/d34 0 2024-03-29T04:44:55.368 INFO:tasks.workunit.client.0.smithi142.stdout:6/180: dwrite d1c/f25 [0,4194304] 0 2024-03-29T04:44:55.368 INFO:tasks.workunit.client.0.smithi142.stdout:0/146: dwrite d7/d9/d1e/f26 [0,4194304] 0 2024-03-29T04:44:55.371 INFO:tasks.workunit.client.0.smithi142.stdout:2/122: dwrite fa [0,4194304] 0 2024-03-29T04:44:55.372 INFO:tasks.workunit.client.0.smithi142.stdout:7/168: creat d7/d27/f29 x:0 0 0 2024-03-29T04:44:55.374 INFO:tasks.workunit.client.0.smithi142.stdout:1/180: dread - d8/db/dc/d19/d1d/d2d/f3e zero size 2024-03-29T04:44:55.375 INFO:tasks.workunit.client.0.smithi142.stdout:5/234: write f3 [3810122,58722] 0 2024-03-29T04:44:55.375 INFO:tasks.workunit.client.0.smithi142.stdout:6/181: mknod d1c/c2e 0 2024-03-29T04:44:55.376 INFO:tasks.workunit.client.0.smithi142.stdout:6/182: fdatasync f17 0 2024-03-29T04:44:55.379 INFO:tasks.workunit.client.0.smithi142.stdout:6/183: truncate d1c/f27 1021203 0 2024-03-29T04:44:55.381 INFO:tasks.workunit.client.0.smithi142.stdout:1/181: write d8/db/dc/d19/d1d/f11 [579390,124865] 0 2024-03-29T04:44:55.381 INFO:tasks.workunit.client.0.smithi142.stdout:9/147: dread d0/f1d [0,4194304] 0 2024-03-29T04:44:55.386 INFO:tasks.workunit.client.0.smithi142.stdout:5/235: creat df/d1a/f44 x:0 0 0 2024-03-29T04:44:55.390 INFO:tasks.workunit.client.0.smithi142.stdout:7/169: dread d7/dc/f1e [0,4194304] 0 2024-03-29T04:44:55.391 INFO:tasks.workunit.client.0.smithi142.stdout:1/182: creat d8/db/f49 x:0 0 0 2024-03-29T04:44:55.392 INFO:tasks.workunit.client.0.smithi142.stdout:5/236: mkdir df/d1a/d21/d27/d45 0 2024-03-29T04:44:55.396 INFO:tasks.workunit.client.0.smithi142.stdout:8/237: dwrite d5/d10/f20 [0,4194304] 0 2024-03-29T04:44:55.396 INFO:tasks.workunit.client.0.smithi142.stdout:2/123: getdents de/d11/d23 0 2024-03-29T04:44:55.397 INFO:tasks.workunit.client.0.smithi142.stdout:7/170: chown d7/dc/de/d20 3 1 2024-03-29T04:44:55.399 INFO:tasks.workunit.client.0.smithi142.stdout:2/124: read f3 [1642940,4020] 0 2024-03-29T04:44:55.405 INFO:tasks.workunit.client.0.smithi142.stdout:8/238: write d5/d1d/d24/f33 [280512,10693] 0 2024-03-29T04:44:55.405 INFO:tasks.workunit.client.0.smithi142.stdout:6/184: dwrite f18 [0,4194304] 0 2024-03-29T04:44:55.406 INFO:tasks.workunit.client.0.smithi142.stdout:5/237: truncate df/d1a/d21/d27/f2a 722972 0 2024-03-29T04:44:55.408 INFO:tasks.workunit.client.0.smithi142.stdout:3/184: dwrite d3/d1f/d34/f12 [0,4194304] 0 2024-03-29T04:44:55.408 INFO:tasks.workunit.client.0.smithi142.stdout:1/183: mkdir d8/db/dc/d4a 0 2024-03-29T04:44:55.412 INFO:tasks.workunit.client.0.smithi142.stdout:1/184: fdatasync d8/db/f12 0 2024-03-29T04:44:55.413 INFO:tasks.workunit.client.0.smithi142.stdout:8/239: rename d5/d10/f20 to d5/d10/d2d/f4f 0 2024-03-29T04:44:55.413 INFO:tasks.workunit.client.0.smithi142.stdout:5/238: chown df/d16/f26 41 1 2024-03-29T04:44:55.413 INFO:tasks.workunit.client.0.smithi142.stdout:9/148: dwrite d0/d2/d6/d16/f2b [0,4194304] 0 2024-03-29T04:44:55.417 INFO:tasks.workunit.client.0.smithi142.stdout:1/185: readlink d8/db/dc/d19/d1d/d15/l42 0 2024-03-29T04:44:55.426 INFO:tasks.workunit.client.0.smithi142.stdout:8/240: creat d5/d1d/d2f/f50 x:0 0 0 2024-03-29T04:44:55.426 INFO:tasks.workunit.client.0.smithi142.stdout:9/149: rename d0/d2/d28 to d0/d2/d6/d35 0 2024-03-29T04:44:55.426 INFO:tasks.workunit.client.0.smithi142.stdout:7/171: dwrite d7/d27/f29 [0,4194304] 0 2024-03-29T04:44:55.427 INFO:tasks.workunit.client.0.smithi142.stdout:2/125: truncate fa 1476899 0 2024-03-29T04:44:55.428 INFO:tasks.workunit.client.0.smithi142.stdout:6/185: dwrite fb [0,4194304] 0 2024-03-29T04:44:55.429 INFO:tasks.workunit.client.0.smithi142.stdout:8/241: write d5/f18 [8151077,121413] 0 2024-03-29T04:44:55.430 INFO:tasks.workunit.client.0.smithi142.stdout:7/172: chown d7/f9 162261 1 2024-03-29T04:44:55.433 INFO:tasks.workunit.client.0.smithi142.stdout:6/186: readlink l19 0 2024-03-29T04:44:55.434 INFO:tasks.workunit.client.0.smithi142.stdout:5/239: getdents df/d16/d24/d2e 0 2024-03-29T04:44:55.437 INFO:tasks.workunit.client.0.smithi142.stdout:5/240: dread df/d1a/d21/d27/f2a [0,4194304] 0 2024-03-29T04:44:55.438 INFO:tasks.workunit.client.0.smithi142.stdout:6/187: symlink d1c/l2f 0 2024-03-29T04:44:55.440 INFO:tasks.workunit.client.0.smithi142.stdout:7/173: symlink d7/l2a 0 2024-03-29T04:44:55.443 INFO:tasks.workunit.client.0.smithi142.stdout:6/188: dread f17 [0,4194304] 0 2024-03-29T04:44:55.444 INFO:tasks.workunit.client.0.smithi142.stdout:7/174: stat d7/c23 0 2024-03-29T04:44:55.446 INFO:tasks.workunit.client.0.smithi142.stdout:5/241: mknod df/d16/d24/d3d/c46 0 2024-03-29T04:44:55.446 INFO:tasks.workunit.client.0.smithi142.stdout:1/186: getdents d8/db/dc/d19/d1d/d15/d1a 0 2024-03-29T04:44:55.447 INFO:tasks.workunit.client.0.smithi142.stdout:2/126: getdents de/d11/d1e/d22 0 2024-03-29T04:44:55.448 INFO:tasks.workunit.client.0.smithi142.stdout:7/175: creat d7/dc/de/d20/f2b x:0 0 0 2024-03-29T04:44:55.448 INFO:tasks.workunit.client.0.smithi142.stdout:8/242: getdents d5/d1d 0 2024-03-29T04:44:55.450 INFO:tasks.workunit.client.0.smithi142.stdout:7/176: creat d7/d18/f2c x:0 0 0 2024-03-29T04:44:55.452 INFO:tasks.workunit.client.0.smithi142.stdout:1/187: mkdir d8/db/dc/d19/d1d/d2d/d44/d4b 0 2024-03-29T04:44:55.453 INFO:tasks.workunit.client.0.smithi142.stdout:8/243: symlink d5/l51 0 2024-03-29T04:44:55.454 INFO:tasks.workunit.client.0.smithi142.stdout:9/150: getdents d0/d2 0 2024-03-29T04:44:55.455 INFO:tasks.workunit.client.0.smithi142.stdout:7/177: creat d7/dc/de/f2d x:0 0 0 2024-03-29T04:44:55.456 INFO:tasks.workunit.client.0.smithi142.stdout:7/178: fsync d7/d18/f1f 0 2024-03-29T04:44:55.457 INFO:tasks.workunit.client.0.smithi142.stdout:5/242: dread df/d16/f1e [0,4194304] 0 2024-03-29T04:44:55.458 INFO:tasks.workunit.client.0.smithi142.stdout:1/188: mkdir d8/db/dc/d19/d1d/d1f/d4c 0 2024-03-29T04:44:55.458 INFO:tasks.workunit.client.0.smithi142.stdout:1/189: dread - d8/db/d37/f41 zero size 2024-03-29T04:44:55.462 INFO:tasks.workunit.client.0.smithi142.stdout:6/189: truncate ff 2757015 0 2024-03-29T04:44:55.463 INFO:tasks.workunit.client.0.smithi142.stdout:5/243: dwrite df/d1a/f31 [0,4194304] 0 2024-03-29T04:44:55.463 INFO:tasks.workunit.client.0.smithi142.stdout:6/190: write d1c/f1d [4240607,43612] 0 2024-03-29T04:44:55.465 INFO:tasks.workunit.client.0.smithi142.stdout:8/244: fsync d5/d1d/d24/f2b 0 2024-03-29T04:44:55.469 INFO:tasks.workunit.client.0.smithi142.stdout:1/190: mkdir d8/db/dc/d19/d1d/d15/d21/d31/d4d 0 2024-03-29T04:44:55.471 INFO:tasks.workunit.client.0.smithi142.stdout:1/191: write d8/db/dc/d19/d1d/d1f/f46 [688078,44086] 0 2024-03-29T04:44:55.471 INFO:tasks.workunit.client.0.smithi142.stdout:5/244: mkdir df/d16/d24/d2e/d47 0 2024-03-29T04:44:55.472 INFO:tasks.workunit.client.0.smithi142.stdout:6/191: creat d1c/f30 x:0 0 0 2024-03-29T04:44:55.474 INFO:tasks.workunit.client.0.smithi142.stdout:7/179: write d7/dc/f10 [3858995,31312] 0 2024-03-29T04:44:55.477 INFO:tasks.workunit.client.0.smithi142.stdout:7/180: dread - d7/dc/de/f26 zero size 2024-03-29T04:44:55.477 INFO:tasks.workunit.client.0.smithi142.stdout:5/245: dread df/d1a/d28/f2d [0,4194304] 0 2024-03-29T04:44:55.477 INFO:tasks.workunit.client.0.smithi142.stdout:2/127: dwrite fa [0,4194304] 0 2024-03-29T04:44:55.478 INFO:tasks.workunit.client.0.smithi142.stdout:5/246: stat df/d16/d24/d3d/c46 0 2024-03-29T04:44:55.482 INFO:tasks.workunit.client.0.smithi142.stdout:4/162: sync 2024-03-29T04:44:55.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.489+0000 7fb5a4258700 1 -- 172.21.15.142:0/788362567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59c0ffae0 msgr2=0x7fb59c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:55.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.490+0000 7fb5a4258700 1 --2- 172.21.15.142:0/788362567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59c0ffae0 0x7fb59c0fff00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb598009910 tx=0x7fb598009c20 comp rx=0 tx=0).stop 2024-03-29T04:44:55.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.490+0000 7fb5a4258700 1 -- 172.21.15.142:0/788362567 shutdown_connections 2024-03-29T04:44:55.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.490+0000 7fb5a4258700 1 --2- 172.21.15.142:0/788362567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb59c100ce0 0x7fb59c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.490+0000 7fb5a4258700 1 --2- 172.21.15.142:0/788362567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59c0ffae0 0x7fb59c0fff00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.490+0000 7fb5a4258700 1 -- 172.21.15.142:0/788362567 >> 172.21.15.142:0/788362567 conn(0x7fb59c0fb080 msgr2=0x7fb59c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:55.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.490+0000 7fb5a4258700 1 -- 172.21.15.142:0/788362567 shutdown_connections 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.490+0000 7fb5a4258700 1 -- 172.21.15.142:0/788362567 wait complete. 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.490+0000 7fb5a4258700 1 Processor -- start 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.490+0000 7fb5a4258700 1 -- start start 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a4258700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb59c0ffae0 0x7fb59c069890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a4258700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59c100ce0 0x7fb59c19b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a4258700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c19b6e0 con 0x7fb59c100ce0 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a4258700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c19b820 con 0x7fb59c0ffae0 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a17f3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59c100ce0 0x7fb59c19b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a17f3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59c100ce0 0x7fb59c19b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40134/0 (socket says 172.21.15.142:40134) 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a17f3700 1 -- 172.21.15.142:0/2432841125 learned_addr learned my addr 172.21.15.142:0/2432841125 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a1ff4700 1 --2- 172.21.15.142:0/2432841125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb59c0ffae0 0x7fb59c069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a17f3700 1 -- 172.21.15.142:0/2432841125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb59c0ffae0 msgr2=0x7fb59c069890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:55.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a17f3700 1 --2- 172.21.15.142:0/2432841125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb59c0ffae0 0x7fb59c069890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a17f3700 1 -- 172.21.15.142:0/2432841125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5980095f0 con 0x7fb59c100ce0 2024-03-29T04:44:55.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a1ff4700 1 --2- 172.21.15.142:0/2432841125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb59c0ffae0 0x7fb59c069890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:44:55.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a17f3700 1 --2- 172.21.15.142:0/2432841125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59c100ce0 0x7fb59c19b110 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00e940 tx=0x7fb58c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:55.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb592ffd700 1 -- 172.21.15.142:0/2432841125 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58c00cba0 con 0x7fb59c100ce0 2024-03-29T04:44:55.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb5a4258700 1 -- 172.21.15.142:0/2432841125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c19bb00 con 0x7fb59c100ce0 2024-03-29T04:44:55.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.491+0000 7fb592ffd700 1 -- 172.21.15.142:0/2432841125 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb58c00ee90 con 0x7fb59c100ce0 2024-03-29T04:44:55.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.492+0000 7fb592ffd700 1 -- 172.21.15.142:0/2432841125 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58c005630 con 0x7fb59c100ce0 2024-03-29T04:44:55.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.492+0000 7fb5a4258700 1 -- 172.21.15.142:0/2432841125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c19c050 con 0x7fb59c100ce0 2024-03-29T04:44:55.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.492+0000 7fb5a4258700 1 -- 172.21.15.142:0/2432841125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59c108660 con 0x7fb59c100ce0 2024-03-29T04:44:55.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.493+0000 7fb592ffd700 1 -- 172.21.15.142:0/2432841125 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb58c004750 con 0x7fb59c100ce0 2024-03-29T04:44:55.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.493+0000 7fb592ffd700 1 --2- 172.21.15.142:0/2432841125 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb58806c4e0 0x7fb58806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:55.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.494+0000 7fb592ffd700 1 -- 172.21.15.142:0/2432841125 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb58c014070 con 0x7fb59c100ce0 2024-03-29T04:44:55.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.494+0000 7fb5a1ff4700 1 --2- 172.21.15.142:0/2432841125 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb58806c4e0 0x7fb58806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:55.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.494+0000 7fb5a1ff4700 1 --2- 172.21.15.142:0/2432841125 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb58806c4e0 0x7fb58806e9a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb59800b6f0 tx=0x7fb598005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:55.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.498+0000 7fb592ffd700 1 -- 172.21.15.142:0/2432841125 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb58c059ce0 con 0x7fb59c100ce0 2024-03-29T04:44:55.517 INFO:tasks.workunit.client.0.smithi142.stdout:8/245: mkdir d5/d1d/d2f/d42/d48/d52 0 2024-03-29T04:44:55.517 INFO:tasks.workunit.client.0.smithi142.stdout:8/246: chown d5/d10/f35 0 1 2024-03-29T04:44:55.519 INFO:tasks.workunit.client.0.smithi142.stdout:9/151: truncate d0/d2/f11 1666532 0 2024-03-29T04:44:55.520 INFO:tasks.workunit.client.0.smithi142.stdout:9/152: write d0/d12/f24 [37596,16203] 0 2024-03-29T04:44:55.520 INFO:tasks.workunit.client.0.smithi142.stdout:7/181: mknod d7/dc/de/d20/c2e 0 2024-03-29T04:44:55.520 INFO:tasks.workunit.client.0.smithi142.stdout:5/247: mknod df/d1a/d28/c48 0 2024-03-29T04:44:55.521 INFO:tasks.workunit.client.0.smithi142.stdout:9/153: truncate d0/f32 282438 0 2024-03-29T04:44:55.542 INFO:tasks.workunit.client.0.smithi142.stdout:3/185: sync 2024-03-29T04:44:55.543 INFO:tasks.workunit.client.0.smithi142.stdout:0/147: sync 2024-03-29T04:44:55.543 INFO:tasks.workunit.client.0.smithi142.stdout:3/186: read d3/d1f/d24/f39 [888535,1884] 0 2024-03-29T04:44:55.556 INFO:tasks.workunit.client.0.smithi142.stdout:8/247: symlink d5/d1d/d2f/d42/l53 0 2024-03-29T04:44:55.557 INFO:tasks.workunit.client.0.smithi142.stdout:8/248: fdatasync d5/d1d/d2f/d42/f4b 0 2024-03-29T04:44:55.558 INFO:tasks.workunit.client.0.smithi142.stdout:2/128: mknod de/d11/d17/c2f 0 2024-03-29T04:44:55.559 INFO:tasks.workunit.client.0.smithi142.stdout:5/248: fdatasync df/d1a/d28/f35 0 2024-03-29T04:44:55.560 INFO:tasks.workunit.client.0.smithi142.stdout:5/249: write df/d2c/f40 [259477,79576] 0 2024-03-29T04:44:55.563 INFO:tasks.workunit.client.0.smithi142.stdout:3/187: rename d3/d1f/d34/f12 to d3/d29/d2b/d40/f44 0 2024-03-29T04:44:55.564 INFO:tasks.workunit.client.0.smithi142.stdout:1/192: dwrite d8/f2a [0,4194304] 0 2024-03-29T04:44:55.564 INFO:tasks.workunit.client.0.smithi142.stdout:6/192: truncate f3 2583259 0 2024-03-29T04:44:55.565 INFO:tasks.workunit.client.0.smithi142.stdout:9/154: symlink d0/d2/d6/d27/l36 0 2024-03-29T04:44:55.568 INFO:tasks.workunit.client.0.smithi142.stdout:1/193: truncate d8/db/d37/f38 119041 0 2024-03-29T04:44:55.574 INFO:tasks.workunit.client.0.smithi142.stdout:6/193: dwrite d1c/f25 [0,4194304] 0 2024-03-29T04:44:55.575 INFO:tasks.workunit.client.0.smithi142.stdout:8/249: link d5/d1d/d2f/d42/f4b d5/d1d/d2f/f54 0 2024-03-29T04:44:55.575 INFO:tasks.workunit.client.0.smithi142.stdout:6/194: chown d1c/l2f 153 1 2024-03-29T04:44:55.577 INFO:tasks.workunit.client.0.smithi142.stdout:6/195: truncate d1c/f30 550708 0 2024-03-29T04:44:55.577 INFO:tasks.workunit.client.0.smithi142.stdout:1/194: dwrite d8/db/dc/d19/d1d/d15/f17 [0,4194304] 0 2024-03-29T04:44:55.579 INFO:tasks.workunit.client.0.smithi142.stdout:3/188: rename d3/d1f/d24/f41 to d3/d29/d2b/d40/f45 0 2024-03-29T04:44:55.582 INFO:tasks.workunit.client.0.smithi142.stdout:1/195: stat d8/db/dc/d19/d1d/d15/c3a 0 2024-03-29T04:44:55.582 INFO:tasks.workunit.client.0.smithi142.stdout:2/129: creat de/d11/d1e/d22/d27/f30 x:0 0 0 2024-03-29T04:44:55.583 INFO:tasks.workunit.client.0.smithi142.stdout:9/155: write d0/f1d [4786721,6394] 0 2024-03-29T04:44:55.584 INFO:tasks.workunit.client.0.smithi142.stdout:2/130: fdatasync de/d11/d17/f1a 0 2024-03-29T04:44:55.587 INFO:tasks.workunit.client.0.smithi142.stdout:4/163: getdents d0/d9/d19 0 2024-03-29T04:44:55.588 INFO:tasks.workunit.client.0.smithi142.stdout:7/182: getdents d7/dc 0 2024-03-29T04:44:55.590 INFO:tasks.workunit.client.0.smithi142.stdout:6/196: creat d1c/f31 x:0 0 0 2024-03-29T04:44:55.591 INFO:tasks.workunit.client.0.smithi142.stdout:7/183: dread - d7/dc/de/d20/f2b zero size 2024-03-29T04:44:55.593 INFO:tasks.workunit.client.0.smithi142.stdout:1/196: symlink d8/db/dc/d19/d1d/d15/l4e 0 2024-03-29T04:44:55.594 INFO:tasks.workunit.client.0.smithi142.stdout:4/164: mkdir d0/d9/d11/d34 0 2024-03-29T04:44:55.597 INFO:tasks.workunit.client.0.smithi142.stdout:7/184: dread d7/fa [0,4194304] 0 2024-03-29T04:44:55.597 INFO:tasks.workunit.client.0.smithi142.stdout:2/131: dwrite de/f19 [0,4194304] 0 2024-03-29T04:44:55.599 INFO:tasks.workunit.client.0.smithi142.stdout:1/197: chown d8/db/dc/d19/d1d/d1f/f46 141231414 1 2024-03-29T04:44:55.599 INFO:tasks.workunit.client.0.smithi142.stdout:4/165: symlink d0/d9/d11/l35 0 2024-03-29T04:44:55.599 INFO:tasks.workunit.client.0.smithi142.stdout:7/185: stat d7/dc/de/f2d 0 2024-03-29T04:44:55.600 INFO:tasks.workunit.client.0.smithi142.stdout:5/250: dread fb [0,4194304] 0 2024-03-29T04:44:55.601 INFO:tasks.workunit.client.0.smithi142.stdout:3/189: rename d3/d1f/d24/f39 to d3/d1f/f46 0 2024-03-29T04:44:55.606 INFO:tasks.workunit.client.0.smithi142.stdout:2/132: creat de/d11/d17/f31 x:0 0 0 2024-03-29T04:44:55.608 INFO:tasks.workunit.client.0.smithi142.stdout:7/186: mknod d7/d18/c2f 0 2024-03-29T04:44:55.609 INFO:tasks.workunit.client.0.smithi142.stdout:7/187: write d7/dc/f10 [2153462,98654] 0 2024-03-29T04:44:55.612 INFO:tasks.workunit.client.0.smithi142.stdout:3/190: rename d3/d1f/d38 to d3/d1f/d24/d47 0 2024-03-29T04:44:55.613 INFO:tasks.workunit.client.0.smithi142.stdout:8/250: truncate d5/f19 570128 0 2024-03-29T04:44:55.614 INFO:tasks.workunit.client.0.smithi142.stdout:6/197: link d1c/l2d d1c/l32 0 2024-03-29T04:44:55.614 INFO:tasks.workunit.client.0.smithi142.stdout:8/251: readlink d5/d1d/d2f/d42/l53 0 2024-03-29T04:44:55.614 INFO:tasks.workunit.client.0.smithi142.stdout:9/156: truncate d0/f22 677696 0 2024-03-29T04:44:55.615 INFO:tasks.workunit.client.0.smithi142.stdout:2/133: symlink de/d11/d23/l32 0 2024-03-29T04:44:55.617 INFO:tasks.workunit.client.0.smithi142.stdout:1/198: symlink d8/db/dc/d19/d1d/d2d/d44/d4b/l4f 0 2024-03-29T04:44:55.617 INFO:tasks.workunit.client.0.smithi142.stdout:1/199: stat d8 0 2024-03-29T04:44:55.618 INFO:tasks.workunit.client.0.smithi142.stdout:7/188: mkdir d7/d18/d30 0 2024-03-29T04:44:55.619 INFO:tasks.workunit.client.0.smithi142.stdout:5/251: link df/d1a/f1f df/d1a/d28/f49 0 2024-03-29T04:44:55.619 INFO:tasks.workunit.client.0.smithi142.stdout:2/134: dread de/f16 [0,4194304] 0 2024-03-29T04:44:55.620 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:55 smithi142 ceph-mon[29027]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:55.620 INFO:tasks.workunit.client.0.smithi142.stdout:6/198: mkdir d1c/d33 0 2024-03-29T04:44:55.621 INFO:tasks.workunit.client.0.smithi142.stdout:8/252: symlink d5/d1d/l55 0 2024-03-29T04:44:55.621 INFO:tasks.workunit.client.0.smithi142.stdout:1/200: read d8/db/dc/d19/d1d/d1f/f30 [2955783,85001] 0 2024-03-29T04:44:55.622 INFO:tasks.workunit.client.0.smithi142.stdout:9/157: creat d0/d2/d6/d16/f37 x:0 0 0 2024-03-29T04:44:55.622 INFO:tasks.workunit.client.0.smithi142.stdout:4/166: creat d0/dc/f36 x:0 0 0 2024-03-29T04:44:55.623 INFO:tasks.workunit.client.0.smithi142.stdout:8/253: chown d5/c40 21 1 2024-03-29T04:44:55.625 INFO:tasks.workunit.client.0.smithi142.stdout:2/135: chown de/d11/c13 1230487820 1 2024-03-29T04:44:55.627 INFO:tasks.workunit.client.0.smithi142.stdout:3/191: dwrite d3/d29/d2b/d40/f44 [0,4194304] 0 2024-03-29T04:44:55.629 INFO:tasks.workunit.client.0.smithi142.stdout:5/252: mkdir df/d4a 0 2024-03-29T04:44:55.631 INFO:tasks.workunit.client.0.smithi142.stdout:5/253: stat df/d1a/d21/d27/f2a 0 2024-03-29T04:44:55.633 INFO:tasks.workunit.client.0.smithi142.stdout:6/199: link d1c/f31 d1c/f34 0 2024-03-29T04:44:55.634 INFO:tasks.workunit.client.0.smithi142.stdout:8/254: creat d5/d10/d2d/f56 x:0 0 0 2024-03-29T04:44:55.635 INFO:tasks.workunit.client.0.smithi142.stdout:7/189: creat d7/f31 x:0 0 0 2024-03-29T04:44:55.636 INFO:tasks.workunit.client.0.smithi142.stdout:3/192: dwrite d3/d1f/d34/f11 [0,4194304] 0 2024-03-29T04:44:55.637 INFO:tasks.workunit.client.0.smithi142.stdout:8/255: write d5/f30 [1038051,69905] 0 2024-03-29T04:44:55.639 INFO:tasks.workunit.client.0.smithi142.stdout:7/190: fsync d7/dc/de/d20/f2b 0 2024-03-29T04:44:55.639 INFO:tasks.workunit.client.0.smithi142.stdout:1/201: mkdir d8/db/dc/d19/d1d/d1f/d50 0 2024-03-29T04:44:55.639 INFO:tasks.workunit.client.0.smithi142.stdout:1/202: dread - d8/f2b zero size 2024-03-29T04:44:55.641 INFO:tasks.workunit.client.0.smithi142.stdout:8/256: fdatasync d5/d1d/d24/f33 0 2024-03-29T04:44:55.642 INFO:tasks.workunit.client.0.smithi142.stdout:1/203: write d8/db/dc/d19/d1d/d15/d21/f35 [894867,78987] 0 2024-03-29T04:44:55.643 INFO:tasks.workunit.client.0.smithi142.stdout:6/200: dread f14 [0,4194304] 0 2024-03-29T04:44:55.643 INFO:tasks.workunit.client.0.smithi142.stdout:6/201: stat f4 0 2024-03-29T04:44:55.645 INFO:tasks.workunit.client.0.smithi142.stdout:5/254: creat df/d1a/d21/d27/f4b x:0 0 0 2024-03-29T04:44:55.646 INFO:tasks.workunit.client.0.smithi142.stdout:5/255: write f3 [1110954,107433] 0 2024-03-29T04:44:55.647 INFO:tasks.workunit.client.0.smithi142.stdout:5/256: write fe [506314,61757] 0 2024-03-29T04:44:55.649 INFO:tasks.workunit.client.0.smithi142.stdout:7/191: rmdir d7/d18 39 2024-03-29T04:44:55.651 INFO:tasks.workunit.client.0.smithi142.stdout:2/136: symlink de/d11/d2b/l33 0 2024-03-29T04:44:55.652 INFO:tasks.workunit.client.0.smithi142.stdout:2/137: read f3 [1876735,109295] 0 2024-03-29T04:44:55.652 INFO:tasks.workunit.client.0.smithi142.stdout:4/167: rmdir d0/dc/d15/d32 0 2024-03-29T04:44:55.654 INFO:tasks.workunit.client.0.smithi142.stdout:5/257: dwrite df/f14 [0,4194304] 0 2024-03-29T04:44:55.654 INFO:tasks.workunit.client.0.smithi142.stdout:1/204: creat d8/f51 x:0 0 0 2024-03-29T04:44:55.654 INFO:tasks.workunit.client.0.smithi142.stdout:4/168: chown d0/f14 3550 1 2024-03-29T04:44:55.654 INFO:tasks.workunit.client.0.smithi142.stdout:6/202: mknod d1c/c35 0 2024-03-29T04:44:55.655 INFO:tasks.workunit.client.0.smithi142.stdout:2/138: chown de/d11/d1e/d22/d27/f30 2588316 1 2024-03-29T04:44:55.656 INFO:tasks.workunit.client.0.smithi142.stdout:1/205: stat d8/db/dc/d19/d1d/d1f/c40 0 2024-03-29T04:44:55.657 INFO:tasks.workunit.client.0.smithi142.stdout:4/169: truncate d0/dc/f17 129325 0 2024-03-29T04:44:55.659 INFO:tasks.workunit.client.0.smithi142.stdout:9/158: write d0/d2/d6/fd [1509057,3768] 0 2024-03-29T04:44:55.661 INFO:tasks.workunit.client.0.smithi142.stdout:7/192: rmdir d7/dc 39 2024-03-29T04:44:55.662 INFO:tasks.workunit.client.0.smithi142.stdout:9/159: truncate d0/d2/d6/d16/f1c 1010367 0 2024-03-29T04:44:55.662 INFO:tasks.workunit.client.0.smithi142.stdout:5/258: symlink df/d29/l4c 0 2024-03-29T04:44:55.663 INFO:tasks.workunit.client.0.smithi142.stdout:1/206: dread d8/db/dc/d19/d1d/d15/f16 [0,4194304] 0 2024-03-29T04:44:55.664 INFO:tasks.workunit.client.0.smithi142.stdout:5/259: fsync df/d1a/d21/d27/f36 0 2024-03-29T04:44:55.664 INFO:tasks.workunit.client.0.smithi142.stdout:9/160: write d0/f15 [929439,23956] 0 2024-03-29T04:44:55.664 INFO:tasks.workunit.client.0.smithi142.stdout:6/203: rename d1c/f27 to d1c/d33/f36 0 2024-03-29T04:44:55.665 INFO:tasks.workunit.client.0.smithi142.stdout:2/139: truncate de/f16 3670421 0 2024-03-29T04:44:55.666 INFO:tasks.workunit.client.0.smithi142.stdout:9/161: read d0/f1d [3744874,13282] 0 2024-03-29T04:44:55.666 INFO:tasks.workunit.client.0.smithi142.stdout:1/207: read - d8/db/dc/d19/d1d/d1f/f29 zero size 2024-03-29T04:44:55.666 INFO:tasks.workunit.client.0.smithi142.stdout:6/204: rename d1c to d1c/d33/d37 22 2024-03-29T04:44:55.670 INFO:tasks.workunit.client.0.smithi142.stdout:2/140: write de/d11/d1e/d22/d27/f30 [586054,88792] 0 2024-03-29T04:44:55.670 INFO:tasks.workunit.client.0.smithi142.stdout:1/208: dread - d8/db/dc/d19/f34 zero size 2024-03-29T04:44:55.671 INFO:tasks.workunit.client.0.smithi142.stdout:7/193: write d7/dc/de/f26 [13777,50093] 0 2024-03-29T04:44:55.672 INFO:tasks.workunit.client.0.smithi142.stdout:5/260: write df/d1a/d21/f3e [594642,84873] 0 2024-03-29T04:44:55.672 INFO:tasks.workunit.client.0.smithi142.stdout:5/261: readlink df/l19 0 2024-03-29T04:44:55.675 INFO:tasks.workunit.client.0.smithi142.stdout:4/170: creat d0/d9/d19/d2c/f37 x:0 0 0 2024-03-29T04:44:55.675 INFO:tasks.workunit.client.0.smithi142.stdout:7/194: dread d7/dc/f1e [0,4194304] 0 2024-03-29T04:44:55.678 INFO:tasks.workunit.client.0.smithi142.stdout:6/205: write fb [1789799,76158] 0 2024-03-29T04:44:55.679 INFO:tasks.workunit.client.0.smithi142.stdout:2/141: mknod de/d11/d23/c34 0 2024-03-29T04:44:55.679 INFO:tasks.workunit.client.0.smithi142.stdout:2/142: fsync de/d11/d17/f31 0 2024-03-29T04:44:55.680 INFO:tasks.workunit.client.0.smithi142.stdout:1/209: fdatasync d8/db/dc/d19/d1d/d15/f16 0 2024-03-29T04:44:55.685 INFO:tasks.workunit.client.0.smithi142.stdout:3/193: dwrite d3/d1f/d34/d6/f14 [0,4194304] 0 2024-03-29T04:44:55.687 INFO:tasks.workunit.client.0.smithi142.stdout:6/206: symlink d1c/l38 0 2024-03-29T04:44:55.688 INFO:tasks.workunit.client.0.smithi142.stdout:2/143: mknod de/d11/d17/c35 0 2024-03-29T04:44:55.691 INFO:tasks.workunit.client.0.smithi142.stdout:7/195: symlink d7/d18/l32 0 2024-03-29T04:44:55.694 INFO:tasks.workunit.client.0.smithi142.stdout:1/210: symlink d8/db/dc/d19/d1d/d1f/d50/l52 0 2024-03-29T04:44:55.699 INFO:tasks.workunit.client.0.smithi142.stdout:3/194: link d3/f3c d3/d29/d2d/f48 0 2024-03-29T04:44:55.699 INFO:tasks.workunit.client.0.smithi142.stdout:6/207: dread f14 [0,4194304] 0 2024-03-29T04:44:55.699 INFO:tasks.workunit.client.0.smithi142.stdout:5/262: write df/d16/f1e [427105,33124] 0 2024-03-29T04:44:55.700 INFO:tasks.workunit.client.0.smithi142.stdout:4/171: link d0/dc/d15/f18 d0/d9/d11/f38 0 2024-03-29T04:44:55.702 INFO:tasks.workunit.client.0.smithi142.stdout:9/162: dwrite d0/d2/f1f [0,4194304] 0 2024-03-29T04:44:55.702 INFO:tasks.workunit.client.0.smithi142.stdout:2/144: creat de/f36 x:0 0 0 2024-03-29T04:44:55.703 INFO:tasks.workunit.client.0.smithi142.stdout:1/211: rename d8/db/dc/d4a to d8/db/dc/d19/d1d/d2d/d44/d53 0 2024-03-29T04:44:55.704 INFO:tasks.workunit.client.0.smithi142.stdout:6/208: creat d1c/f39 x:0 0 0 2024-03-29T04:44:55.705 INFO:tasks.workunit.client.0.smithi142.stdout:3/195: symlink d3/d1f/d24/d47/l49 0 2024-03-29T04:44:55.705 INFO:tasks.workunit.client.0.smithi142.stdout:6/209: write f1a [499409,9223] 0 2024-03-29T04:44:55.710 INFO:tasks.workunit.client.0.smithi142.stdout:3/196: dwrite d3/d29/d2b/d40/f44 [0,4194304] 0 2024-03-29T04:44:55.713 INFO:tasks.workunit.client.0.smithi142.stdout:1/212: rename d8/db/d3d/f43 to d8/f54 0 2024-03-29T04:44:55.717 INFO:tasks.workunit.client.0.smithi142.stdout:5/263: mkdir df/d16/d24/d2e/d47/d4d 0 2024-03-29T04:44:55.718 INFO:tasks.workunit.client.0.smithi142.stdout:5/264: readlink df/d29/l4c 0 2024-03-29T04:44:55.720 INFO:tasks.workunit.client.0.smithi142.stdout:5/265: write df/d1a/f2f [1329610,5932] 0 2024-03-29T04:44:55.720 INFO:tasks.workunit.client.0.smithi142.stdout:3/197: dwrite d3/d1f/d34/f27 [0,4194304] 0 2024-03-29T04:44:55.721 INFO:tasks.workunit.client.0.smithi142.stdout:5/266: stat df/d1a/d21 0 2024-03-29T04:44:55.723 INFO:tasks.workunit.client.0.smithi142.stdout:7/196: dread f6 [0,4194304] 0 2024-03-29T04:44:55.740 INFO:tasks.workunit.client.0.smithi142.stdout:0/148: sync 2024-03-29T04:44:55.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.775+0000 7fb5a4258700 1 -- 172.21.15.142:0/2432841125 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb59c105630 con 0x7fb58806c4e0 2024-03-29T04:44:55.784 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.785+0000 7fb592ffd700 1 -- 172.21.15.142:0/2432841125 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3456 (secure 0 0 0) 0x7fb59c105630 con 0x7fb58806c4e0 2024-03-29T04:44:55.785 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:44:55.785 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (6m) 2m ago 8m 25.1M - 0.25.0 c8568f914cd2 d4a3e12f4047 2024-03-29T04:44:55.785 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (8m) 2m ago 8m 7512k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:44:55.785 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (7m) 2m ago 7m 7432k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:44:55.786 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (8m) 2m ago 8m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:44:55.786 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (7m) 2m ago 7m 6660k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:44:55.786 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (6m) 2m ago 8m 85.8M - 9.4.7 954c08fa6188 aba58e552b7c 2024-03-29T04:44:55.786 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (2m) 2m ago 2m 17.7M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:44:55.786 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (2m) 2m ago 2m 12.8M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:44:55.786 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (2m) 2m ago 2m 17.3M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:44:55.786 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (2m) 2m ago 2m 17.4M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:44:55.786 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:9283,8765,8443 running (9m) 2m ago 9m 489M - 18.2.1 5be31c24972a a5f21a9cfe24 2024-03-29T04:44:55.786 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (7m) 2m ago 7m 441M - 18.2.1 5be31c24972a c6124d88151c 2024-03-29T04:44:55.786 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (9m) 2m ago 9m 46.5M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:44:55.786 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (7m) 2m ago 7m 35.2M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:44:55.786 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (8m) 2m ago 8m 16.2M - 1.5.0 0da6a335fe13 91b95c929db2 2024-03-29T04:44:55.787 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (7m) 2m ago 7m 15.0M - 1.5.0 0da6a335fe13 97e90e5b337b 2024-03-29T04:44:55.787 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (6m) 2m ago 6m 48.8M 3946M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:44:55.787 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (6m) 2m ago 5m 51.1M 3946M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:44:55.787 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (5m) 2m ago 5m 49.1M 3946M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:44:55.787 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (5m) 2m ago 5m 45.8M 4970M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:44:55.787 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (4m) 2m ago 4m 44.2M 4970M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:44:55.787 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (4m) 2m ago 4m 43.8M 4970M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:44:55.787 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (6m) 2m ago 7m 58.7M - 2.43.0 a07b618ecd1d c6c39d3c30f6 2024-03-29T04:44:55.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.787+0000 7fb5a4258700 1 -- 172.21.15.142:0/2432841125 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb58806c4e0 msgr2=0x7fb58806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:55.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.787+0000 7fb5a4258700 1 --2- 172.21.15.142:0/2432841125 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb58806c4e0 0x7fb58806e9a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb59800b6f0 tx=0x7fb598005940 comp rx=0 tx=0).stop 2024-03-29T04:44:55.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.787+0000 7fb5a4258700 1 -- 172.21.15.142:0/2432841125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59c100ce0 msgr2=0x7fb59c19b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:55.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.787+0000 7fb5a4258700 1 --2- 172.21.15.142:0/2432841125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59c100ce0 0x7fb59c19b110 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00e940 tx=0x7fb58c00ec50 comp rx=0 tx=0).stop 2024-03-29T04:44:55.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.787+0000 7fb5a4258700 1 -- 172.21.15.142:0/2432841125 shutdown_connections 2024-03-29T04:44:55.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.787+0000 7fb5a4258700 1 --2- 172.21.15.142:0/2432841125 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fb58806c4e0 0x7fb58806e9a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.787+0000 7fb5a4258700 1 --2- 172.21.15.142:0/2432841125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb59c0ffae0 0x7fb59c069890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.787+0000 7fb5a4258700 1 --2- 172.21.15.142:0/2432841125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb59c100ce0 0x7fb59c19b110 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.787+0000 7fb5a4258700 1 -- 172.21.15.142:0/2432841125 >> 172.21.15.142:0/2432841125 conn(0x7fb59c0fb080 msgr2=0x7fb59c103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:55.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.787+0000 7fb5a4258700 1 -- 172.21.15.142:0/2432841125 shutdown_connections 2024-03-29T04:44:55.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.787+0000 7fb5a4258700 1 -- 172.21.15.142:0/2432841125 wait complete. 2024-03-29T04:44:55.789 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:55 smithi152 ceph-mon[37177]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:55.806 INFO:tasks.workunit.client.0.smithi142.stdout:9/163: creat d0/f38 x:0 0 0 2024-03-29T04:44:55.806 INFO:tasks.workunit.client.0.smithi142.stdout:1/213: mknod d8/db/dc/d19/d1d/d15/c55 0 2024-03-29T04:44:55.807 INFO:tasks.workunit.client.0.smithi142.stdout:3/198: creat d3/d29/d2b/d40/f4a x:0 0 0 2024-03-29T04:44:55.807 INFO:tasks.workunit.client.0.smithi142.stdout:2/145: creat de/d11/f37 x:0 0 0 2024-03-29T04:44:55.808 INFO:tasks.workunit.client.0.smithi142.stdout:9/164: chown d0/d2/d6/d16/f37 12 1 2024-03-29T04:44:55.822 INFO:tasks.workunit.client.0.smithi142.stdout:5/267: dwrite df/d1a/d28/f49 [0,4194304] 0 2024-03-29T04:44:55.823 INFO:tasks.workunit.client.0.smithi142.stdout:5/268: fsync df/d1a/f31 0 2024-03-29T04:44:55.831 INFO:tasks.workunit.client.0.smithi142.stdout:5/269: dwrite df/d16/f26 [4194304,4194304] 0 2024-03-29T04:44:55.832 INFO:tasks.workunit.client.0.smithi142.stdout:5/270: write f3 [5742458,18768] 0 2024-03-29T04:44:55.842 INFO:tasks.workunit.client.0.smithi142.stdout:8/257: sync 2024-03-29T04:44:55.844 INFO:tasks.workunit.client.0.smithi142.stdout:7/197: mkdir d7/d18/d30/d33 0 2024-03-29T04:44:55.845 INFO:tasks.workunit.client.0.smithi142.stdout:7/198: readlink d7/l13 0 2024-03-29T04:44:55.846 INFO:tasks.workunit.client.0.smithi142.stdout:3/199: mknod d3/d29/d2b/c4b 0 2024-03-29T04:44:55.847 INFO:tasks.workunit.client.0.smithi142.stdout:9/165: stat d0/c8 0 2024-03-29T04:44:55.851 INFO:tasks.workunit.client.0.smithi142.stdout:9/166: dread d0/d2/d6/d16/f2b [0,4194304] 0 2024-03-29T04:44:55.853 INFO:tasks.workunit.client.0.smithi142.stdout:5/271: symlink df/d16/l4e 0 2024-03-29T04:44:55.853 INFO:tasks.workunit.client.0.smithi142.stdout:8/258: creat d5/d10/d2d/f57 x:0 0 0 2024-03-29T04:44:55.854 INFO:tasks.workunit.client.0.smithi142.stdout:6/210: link d1c/l32 d1c/d33/l3a 0 2024-03-29T04:44:55.854 INFO:tasks.workunit.client.0.smithi142.stdout:8/259: dread - d5/d10/d2d/f57 zero size 2024-03-29T04:44:55.855 INFO:tasks.workunit.client.0.smithi142.stdout:7/199: creat d7/dc/de/f34 x:0 0 0 2024-03-29T04:44:55.856 INFO:tasks.workunit.client.0.smithi142.stdout:3/200: creat d3/d29/d2b/d40/f4c x:0 0 0 2024-03-29T04:44:55.857 INFO:tasks.workunit.client.0.smithi142.stdout:1/214: mkdir d8/db/dc/d19/d1d/d15/d21/d31/d4d/d56 0 2024-03-29T04:44:55.858 INFO:tasks.workunit.client.0.smithi142.stdout:8/260: write d5/d10/d2d/f56 [724443,51536] 0 2024-03-29T04:44:55.859 INFO:tasks.workunit.client.0.smithi142.stdout:9/167: mkdir d0/d2/d39 0 2024-03-29T04:44:55.860 INFO:tasks.workunit.client.0.smithi142.stdout:6/211: symlink d1c/l3b 0 2024-03-29T04:44:55.860 INFO:tasks.workunit.client.0.smithi142.stdout:2/146: link de/c14 de/d11/d1e/d22/c38 0 2024-03-29T04:44:55.863 INFO:tasks.workunit.client.0.smithi142.stdout:2/147: chown f7 82 1 2024-03-29T04:44:55.864 INFO:tasks.workunit.client.0.smithi142.stdout:5/272: dwrite df/d1a/f3f [0,4194304] 0 2024-03-29T04:44:55.866 INFO:tasks.workunit.client.0.smithi142.stdout:6/212: creat d1c/f3c x:0 0 0 2024-03-29T04:44:55.867 INFO:tasks.workunit.client.0.smithi142.stdout:8/261: write d5/d10/f32 [920021,11862] 0 2024-03-29T04:44:55.867 INFO:tasks.workunit.client.0.smithi142.stdout:9/168: creat d0/d2/d6/d31/f3a x:0 0 0 2024-03-29T04:44:55.871 INFO:tasks.workunit.client.0.smithi142.stdout:7/200: dwrite f1 [4194304,4194304] 0 2024-03-29T04:44:55.872 INFO:tasks.workunit.client.0.smithi142.stdout:6/213: mknod d1c/c3d 0 2024-03-29T04:44:55.874 INFO:tasks.workunit.client.0.smithi142.stdout:8/262: mknod d5/d10/c58 0 2024-03-29T04:44:55.874 INFO:tasks.workunit.client.0.smithi142.stdout:6/214: readlink d1c/l2f 0 2024-03-29T04:44:55.876 INFO:tasks.workunit.client.0.smithi142.stdout:3/201: dwrite d3/d1f/d34/f11 [4194304,4194304] 0 2024-03-29T04:44:55.879 INFO:tasks.workunit.client.0.smithi142.stdout:2/148: dread fa [0,4194304] 0 2024-03-29T04:44:55.879 INFO:tasks.workunit.client.0.smithi142.stdout:3/202: chown d3/d1f/d34/l3d 35 1 2024-03-29T04:44:55.880 INFO:tasks.workunit.client.0.smithi142.stdout:5/273: dwrite df/d1a/d21/f3e [0,4194304] 0 2024-03-29T04:44:55.881 INFO:tasks.workunit.client.0.smithi142.stdout:2/149: rename de/d11/d17 to de/d11/d17/d39 22 2024-03-29T04:44:55.884 INFO:tasks.workunit.client.0.smithi142.stdout:9/169: link d0/d2/fe d0/d2/d6/d16/f3b 0 2024-03-29T04:44:55.884 INFO:tasks.workunit.client.0.smithi142.stdout:2/150: dread - de/d11/d17/f25 zero size 2024-03-29T04:44:55.885 INFO:tasks.workunit.client.0.smithi142.stdout:8/263: dwrite d5/f18 [8388608,4194304] 0 2024-03-29T04:44:55.887 INFO:tasks.workunit.client.0.smithi142.stdout:6/215: creat d1c/d33/f3e x:0 0 0 2024-03-29T04:44:55.889 INFO:tasks.workunit.client.0.smithi142.stdout:6/216: write d1c/f39 [775058,33575] 0 2024-03-29T04:44:55.890 INFO:tasks.workunit.client.0.smithi142.stdout:5/274: dread df/d1a/f31 [0,4194304] 0 2024-03-29T04:44:55.891 INFO:tasks.workunit.client.0.smithi142.stdout:5/275: chown df/d1a/f1f 32713963 1 2024-03-29T04:44:55.892 INFO:tasks.workunit.client.0.smithi142.stdout:5/276: fdatasync f3 0 2024-03-29T04:44:55.893 INFO:tasks.workunit.client.0.smithi142.stdout:9/170: mkdir d0/d2/d6/d27/d3c 0 2024-03-29T04:44:55.893 INFO:tasks.workunit.client.0.smithi142.stdout:8/264: dread d5/f19 [0,4194304] 0 2024-03-29T04:44:55.893 INFO:tasks.workunit.client.0.smithi142.stdout:3/203: dread d3/d1f/f46 [0,4194304] 0 2024-03-29T04:44:55.895 INFO:tasks.workunit.client.0.smithi142.stdout:3/204: fdatasync d3/d29/d2b/d40/f4c 0 2024-03-29T04:44:55.895 INFO:tasks.workunit.client.0.smithi142.stdout:8/265: write d5/d10/d2d/f4a [738553,66464] 0 2024-03-29T04:44:55.895 INFO:tasks.workunit.client.0.smithi142.stdout:2/151: unlink de/d11/f1d 0 2024-03-29T04:44:55.897 INFO:tasks.workunit.client.0.smithi142.stdout:2/152: chown de/d11/d2b/c2c 23720579 1 2024-03-29T04:44:55.897 INFO:tasks.workunit.client.0.smithi142.stdout:6/217: symlink d1c/d33/l3f 0 2024-03-29T04:44:55.902 INFO:tasks.workunit.client.0.smithi142.stdout:6/218: dread d1c/f30 [0,4194304] 0 2024-03-29T04:44:55.902 INFO:tasks.workunit.client.0.smithi142.stdout:8/266: dwrite d5/f1f [0,4194304] 0 2024-03-29T04:44:55.905 INFO:tasks.workunit.client.0.smithi142.stdout:8/267: read d5/d10/d2d/f56 [736335,100426] 0 2024-03-29T04:44:55.906 INFO:tasks.workunit.client.0.smithi142.stdout:8/268: fdatasync d5/d2e/f37 0 2024-03-29T04:44:55.911 INFO:tasks.workunit.client.0.smithi142.stdout:2/153: dwrite de/d11/f15 [4194304,4194304] 0 2024-03-29T04:44:55.913 INFO:tasks.workunit.client.0.smithi142.stdout:5/277: link df/d16/l1b df/d16/d24/l4f 0 2024-03-29T04:44:55.914 INFO:tasks.workunit.client.0.smithi142.stdout:8/269: mkdir d5/d1d/d59 0 2024-03-29T04:44:55.918 INFO:tasks.workunit.client.0.smithi142.stdout:5/278: dread df/d16/f26 [4194304,4194304] 0 2024-03-29T04:44:55.921 INFO:tasks.workunit.client.0.smithi142.stdout:5/279: truncate df/d16/f39 937352 0 2024-03-29T04:44:55.921 INFO:tasks.workunit.client.0.smithi142.stdout:2/154: write f9 [1917527,38532] 0 2024-03-29T04:44:55.921 INFO:tasks.workunit.client.0.smithi142.stdout:5/280: chown df/d16 255748707 1 2024-03-29T04:44:55.921 INFO:tasks.workunit.client.0.smithi142.stdout:8/270: rmdir d5/d1d/d2f/d42/d48 39 2024-03-29T04:44:55.921 INFO:tasks.workunit.client.0.smithi142.stdout:0/149: truncate d7/f21 45336 0 2024-03-29T04:44:55.922 INFO:tasks.workunit.client.0.smithi142.stdout:2/155: chown de/d11/d23/l32 29694 1 2024-03-29T04:44:55.922 INFO:tasks.workunit.client.0.smithi142.stdout:3/205: getdents d3/d1f/d24 0 2024-03-29T04:44:55.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.936+0000 7f4a1da1f700 1 -- 172.21.15.142:0/3138239751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18103f70 msgr2=0x7f4a18106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:55.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.936+0000 7f4a1da1f700 1 --2- 172.21.15.142:0/3138239751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18103f70 0x7f4a18106360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009910 tx=0x7f4a0c009c20 comp rx=0 tx=0).stop 2024-03-29T04:44:55.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.936+0000 7f4a1da1f700 1 -- 172.21.15.142:0/3138239751 shutdown_connections 2024-03-29T04:44:55.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.936+0000 7f4a1da1f700 1 --2- 172.21.15.142:0/3138239751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18103f70 0x7f4a18106360 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.936+0000 7f4a1da1f700 1 --2- 172.21.15.142:0/3138239751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a18101640 0x7f4a18103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.936+0000 7f4a1da1f700 1 -- 172.21.15.142:0/3138239751 >> 172.21.15.142:0/3138239751 conn(0x7f4a180fb030 msgr2=0x7f4a180fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:55.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.936+0000 7f4a1da1f700 1 -- 172.21.15.142:0/3138239751 shutdown_connections 2024-03-29T04:44:55.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.936+0000 7f4a1da1f700 1 -- 172.21.15.142:0/3138239751 wait complete. 2024-03-29T04:44:55.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a1da1f700 1 Processor -- start 2024-03-29T04:44:55.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a1da1f700 1 -- start start 2024-03-29T04:44:55.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a1da1f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18101640 0x7f4a18100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:55.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a1da1f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a18103f70 0x7f4a180ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:55.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a1da1f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180ffb30 con 0x7f4a18101640 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a1da1f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180ffc70 con 0x7f4a18103f70 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a16ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18101640 0x7f4a18100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a16ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18101640 0x7f4a18100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40142/0 (socket says 172.21.15.142:40142) 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a16ffd700 1 -- 172.21.15.142:0/1815774769 learned_addr learned my addr 172.21.15.142:0/1815774769 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a167fc700 1 --2- 172.21.15.142:0/1815774769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a18103f70 0x7f4a180ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a16ffd700 1 -- 172.21.15.142:0/1815774769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a18103f70 msgr2=0x7f4a180ff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a16ffd700 1 --2- 172.21.15.142:0/1815774769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a18103f70 0x7f4a180ff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.937+0000 7f4a16ffd700 1 -- 172.21.15.142:0/1815774769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c0095f0 con 0x7f4a18101640 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.938+0000 7f4a16ffd700 1 --2- 172.21.15.142:0/1815774769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18101640 0x7f4a18100fa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800d900 tx=0x7f4a0800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.938+0000 7f4a1ca1d700 1 -- 172.21.15.142:0/1815774769 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a080041f0 con 0x7f4a18101640 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.938+0000 7f4a1ca1d700 1 -- 172.21.15.142:0/1815774769 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4a08004ce0 con 0x7f4a18101640 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.938+0000 7f4a1da1f700 1 -- 172.21.15.142:0/1815774769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a180fff50 con 0x7f4a18101640 2024-03-29T04:44:55.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.938+0000 7f4a1ca1d700 1 -- 172.21.15.142:0/1815774769 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a0800b750 con 0x7f4a18101640 2024-03-29T04:44:55.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.938+0000 7f4a1da1f700 1 -- 172.21.15.142:0/1815774769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1819e390 con 0x7f4a18101640 2024-03-29T04:44:55.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.939+0000 7f4a1da1f700 1 -- 172.21.15.142:0/1815774769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a18190520 con 0x7f4a18101640 2024-03-29T04:44:55.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.939+0000 7f4a1ca1d700 1 -- 172.21.15.142:0/1815774769 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f4a08004e50 con 0x7f4a18101640 2024-03-29T04:44:55.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.940+0000 7f4a1ca1d700 1 --2- 172.21.15.142:0/1815774769 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4a0006c4e0 0x7f4a0006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:55.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.940+0000 7f4a167fc700 1 --2- 172.21.15.142:0/1815774769 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4a0006c4e0 0x7f4a0006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:55.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.940+0000 7f4a1ca1d700 1 -- 172.21.15.142:0/1815774769 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4a0808b810 con 0x7f4a18101640 2024-03-29T04:44:55.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.940+0000 7f4a167fc700 1 --2- 172.21.15.142:0/1815774769 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4a0006c4e0 0x7f4a0006e9a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00b5c0 tx=0x7f4a0c005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:55.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:55.944+0000 7f4a1ca1d700 1 -- 172.21.15.142:0/1815774769 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4a08059a20 con 0x7f4a18101640 2024-03-29T04:44:55.971 INFO:tasks.workunit.client.0.smithi142.stdout:4/172: sync 2024-03-29T04:44:55.983 INFO:tasks.workunit.client.0.smithi142.stdout:5/281: mkdir df/d16/d50 0 2024-03-29T04:44:55.984 INFO:tasks.workunit.client.0.smithi142.stdout:4/173: chown d0/d12 338 1 2024-03-29T04:44:55.984 INFO:tasks.workunit.client.0.smithi142.stdout:0/150: creat d7/d9/f30 x:0 0 0 2024-03-29T04:44:55.985 INFO:tasks.workunit.client.0.smithi142.stdout:8/271: truncate d5/d2e/f37 52221 0 2024-03-29T04:44:55.986 INFO:tasks.workunit.client.0.smithi142.stdout:0/151: write d7/d9/d1e/f26 [117799,104117] 0 2024-03-29T04:44:55.987 INFO:tasks.workunit.client.0.smithi142.stdout:4/174: write d0/dc/f33 [44115,80962] 0 2024-03-29T04:44:55.988 INFO:tasks.workunit.client.0.smithi142.stdout:0/152: chown d7/d9/d22 64737 1 2024-03-29T04:44:55.989 INFO:tasks.workunit.client.0.smithi142.stdout:9/171: rmdir d0/d2/d6/d16 39 2024-03-29T04:44:55.989 INFO:tasks.workunit.client.0.smithi142.stdout:0/153: read - d7/d9/d22/f23 zero size 2024-03-29T04:44:55.990 INFO:tasks.workunit.client.0.smithi142.stdout:6/219: rmdir d1c 39 2024-03-29T04:44:55.991 INFO:tasks.workunit.client.0.smithi142.stdout:2/156: mkdir de/d11/d1e/d22/d27/d3a 0 2024-03-29T04:44:55.991 INFO:tasks.workunit.client.0.smithi142.stdout:9/172: chown d0/d2/d6/f30 669696596 1 2024-03-29T04:44:55.991 INFO:tasks.workunit.client.0.smithi142.stdout:0/154: dread - d7/d9/d22/f23 zero size 2024-03-29T04:44:55.991 INFO:tasks.workunit.client.0.smithi142.stdout:3/206: fdatasync d3/d29/d2b/f21 0 2024-03-29T04:44:55.992 INFO:tasks.workunit.client.0.smithi142.stdout:5/282: symlink df/d16/l51 0 2024-03-29T04:44:55.992 INFO:tasks.workunit.client.0.smithi142.stdout:8/272: fdatasync d5/f19 0 2024-03-29T04:44:55.993 INFO:tasks.workunit.client.0.smithi142.stdout:2/157: chown de/d11/d17/c2f 107 1 2024-03-29T04:44:55.993 INFO:tasks.workunit.client.0.smithi142.stdout:3/207: readlink d3/d1f/d34/d6/l1b 0 2024-03-29T04:44:55.993 INFO:tasks.workunit.client.0.smithi142.stdout:9/173: rmdir d0/d12 39 2024-03-29T04:44:56.056 INFO:tasks.workunit.client.0.smithi142.stdout:0/155: rename c0 to d7/d9/c31 0 2024-03-29T04:44:56.057 INFO:tasks.workunit.client.0.smithi142.stdout:5/283: write df/d16/d25/f37 [15652,62666] 0 2024-03-29T04:44:56.057 INFO:tasks.workunit.client.0.smithi142.stdout:8/273: write d5/d1d/d24/f3a [1568002,2533] 0 2024-03-29T04:44:56.060 INFO:tasks.workunit.client.0.smithi142.stdout:4/175: link d0/d9/d11/f38 d0/d9/d19/f39 0 2024-03-29T04:44:56.060 INFO:tasks.workunit.client.0.smithi142.stdout:8/274: truncate d5/f1f 5034145 0 2024-03-29T04:44:56.062 INFO:tasks.workunit.client.0.smithi142.stdout:4/176: readlink d0/d9/d11/l16 0 2024-03-29T04:44:56.064 INFO:tasks.workunit.client.0.smithi142.stdout:3/208: dwrite d3/d1f/d34/f27 [0,4194304] 0 2024-03-29T04:44:56.065 INFO:tasks.workunit.client.0.smithi142.stdout:2/158: dread f6 [0,4194304] 0 2024-03-29T04:44:56.065 INFO:tasks.workunit.client.0.smithi142.stdout:3/209: readlink d3/l30 0 2024-03-29T04:44:56.066 INFO:tasks.workunit.client.0.smithi142.stdout:3/210: dread - d3/d29/d2d/f48 zero size 2024-03-29T04:44:56.072 INFO:tasks.workunit.client.0.smithi142.stdout:2/159: dwrite de/d11/f15 [8388608,4194304] 0 2024-03-29T04:44:56.104 INFO:tasks.workunit.client.0.smithi142.stdout:1/215: sync 2024-03-29T04:44:56.104 INFO:tasks.workunit.client.0.smithi142.stdout:1/216: stat d8/db/dc/f2c 0 2024-03-29T04:44:56.105 INFO:tasks.workunit.client.0.smithi142.stdout:1/217: chown d8/db/dc/d19/d1d/d15/c55 10067 1 2024-03-29T04:44:56.149 INFO:tasks.workunit.client.0.smithi142.stdout:7/201: sync 2024-03-29T04:44:56.149 INFO:tasks.workunit.client.0.smithi142.stdout:6/220: sync 2024-03-29T04:44:56.150 INFO:tasks.workunit.client.0.smithi142.stdout:5/284: rename df/l12 to df/d1a/d21/l52 0 2024-03-29T04:44:56.150 INFO:tasks.workunit.client.0.smithi142.stdout:0/156: rename d7/d13/d27 to d7/d13/d27/d2f/d32 22 2024-03-29T04:44:56.152 INFO:tasks.workunit.client.0.smithi142.stdout:0/157: write d7/f2b [750922,49097] 0 2024-03-29T04:44:56.197 INFO:tasks.workunit.client.0.smithi142.stdout:8/275: symlink d5/d2e/l5a 0 2024-03-29T04:44:56.211 INFO:tasks.workunit.client.0.smithi142.stdout:9/174: dwrite d0/d2/f11 [0,4194304] 0 2024-03-29T04:44:56.213 INFO:tasks.workunit.client.0.smithi142.stdout:1/218: fdatasync f5 0 2024-03-29T04:44:56.215 INFO:tasks.workunit.client.0.smithi142.stdout:1/219: dread d8/db/d37/f38 [0,4194304] 0 2024-03-29T04:44:56.218 INFO:tasks.workunit.client.0.smithi142.stdout:9/175: dwrite d0/d2/f2c [0,4194304] 0 2024-03-29T04:44:56.221 INFO:tasks.workunit.client.0.smithi142.stdout:7/202: creat d7/d18/f35 x:0 0 0 2024-03-29T04:44:56.225 INFO:tasks.workunit.client.0.smithi142.stdout:2/160: rename de/d11/d17/c35 to de/d11/d2b/c3b 0 2024-03-29T04:44:56.229 INFO:tasks.workunit.client.0.smithi142.stdout:0/158: mknod d7/d13/d27/c33 0 2024-03-29T04:44:56.231 INFO:tasks.workunit.client.0.smithi142.stdout:8/276: rmdir d5/d1d/d24 39 2024-03-29T04:44:56.232 INFO:tasks.workunit.client.0.smithi142.stdout:2/161: dwrite de/f29 [0,4194304] 0 2024-03-29T04:44:56.233 INFO:tasks.workunit.client.0.smithi142.stdout:4/177: creat d0/d12/d28/f3a x:0 0 0 2024-03-29T04:44:56.234 INFO:tasks.workunit.client.0.smithi142.stdout:2/162: stat de/c1f 0 2024-03-29T04:44:56.235 INFO:tasks.workunit.client.0.smithi142.stdout:2/163: read f9 [1713816,100530] 0 2024-03-29T04:44:56.235 INFO:tasks.workunit.client.0.smithi142.stdout:4/178: chown d0/d9/d19/d2c/f37 19539 1 2024-03-29T04:44:56.235 INFO:tasks.workunit.client.0.smithi142.stdout:2/164: stat de/d11/d17/l24 0 2024-03-29T04:44:56.242 INFO:tasks.workunit.client.0.smithi142.stdout:1/220: dread d8/db/dc/f1c [0,4194304] 0 2024-03-29T04:44:56.243 INFO:tasks.workunit.client.0.smithi142.stdout:1/221: write d8/db/dc/d19/d1d/d15/f16 [2486235,38110] 0 2024-03-29T04:44:56.244 INFO:tasks.workunit.client.0.smithi142.stdout:5/285: symlink df/d16/d50/l53 0 2024-03-29T04:44:56.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.249+0000 7f4a1da1f700 1 -- 172.21.15.142:0/1815774769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4a180619a0 con 0x7f4a18101640 2024-03-29T04:44:56.248 INFO:tasks.workunit.client.0.smithi142.stdout:8/277: unlink d5/d1d/c28 0 2024-03-29T04:44:56.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.249+0000 7f4a1ca1d700 1 -- 172.21.15.142:0/1815774769 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f4a080595b0 con 0x7f4a18101640 2024-03-29T04:44:56.249 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:44:56.249 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:44:56.249 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:44:56.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:44:56.249 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:44:56.249 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:44:56.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:44:56.249 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:44:56.249 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:44:56.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:44:56.250 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:44:56.250 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:44:56.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:44:56.250 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:44:56.250 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-03-29T04:44:56.250 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:44:56.250 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:44:56.250 INFO:tasks.workunit.client.0.smithi142.stdout:2/165: stat de/f16 0 2024-03-29T04:44:56.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.252+0000 7f4a1da1f700 1 -- 172.21.15.142:0/1815774769 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4a0006c4e0 msgr2=0x7f4a0006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:56.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.252+0000 7f4a1da1f700 1 --2- 172.21.15.142:0/1815774769 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4a0006c4e0 0x7f4a0006e9a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00b5c0 tx=0x7f4a0c005ee0 comp rx=0 tx=0).stop 2024-03-29T04:44:56.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.252+0000 7f4a1da1f700 1 -- 172.21.15.142:0/1815774769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18101640 msgr2=0x7f4a18100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:56.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.252+0000 7f4a1da1f700 1 --2- 172.21.15.142:0/1815774769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18101640 0x7f4a18100fa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800d900 tx=0x7f4a0800dc10 comp rx=0 tx=0).stop 2024-03-29T04:44:56.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.252+0000 7f4a1da1f700 1 -- 172.21.15.142:0/1815774769 shutdown_connections 2024-03-29T04:44:56.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.252+0000 7f4a1da1f700 1 --2- 172.21.15.142:0/1815774769 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f4a0006c4e0 0x7f4a0006e9a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:56.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.252+0000 7f4a1da1f700 1 --2- 172.21.15.142:0/1815774769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18101640 0x7f4a18100fa0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:56.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.252+0000 7f4a1da1f700 1 --2- 172.21.15.142:0/1815774769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a18103f70 0x7f4a180ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:56.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.252+0000 7f4a1da1f700 1 -- 172.21.15.142:0/1815774769 >> 172.21.15.142:0/1815774769 conn(0x7f4a180fb030 msgr2=0x7f4a180fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:56.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.252+0000 7f4a1da1f700 1 -- 172.21.15.142:0/1815774769 shutdown_connections 2024-03-29T04:44:56.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.252+0000 7f4a1da1f700 1 -- 172.21.15.142:0/1815774769 wait complete. 2024-03-29T04:44:56.253 INFO:tasks.workunit.client.0.smithi142.stdout:1/222: unlink c4 0 2024-03-29T04:44:56.255 INFO:tasks.workunit.client.0.smithi142.stdout:7/203: mknod d7/c36 0 2024-03-29T04:44:56.257 INFO:tasks.workunit.client.0.smithi142.stdout:6/221: rename f3 to d1c/f40 0 2024-03-29T04:44:56.258 INFO:tasks.workunit.client.0.smithi142.stdout:5/286: mkdir df/d16/d24/d2e/d47/d54 0 2024-03-29T04:44:56.259 INFO:tasks.workunit.client.0.smithi142.stdout:5/287: write df/f10 [5170867,86165] 0 2024-03-29T04:44:56.259 INFO:tasks.workunit.client.0.smithi142.stdout:0/159: mknod d7/d13/d27/d2f/c34 0 2024-03-29T04:44:56.262 INFO:tasks.workunit.client.0.smithi142.stdout:2/166: rmdir de/d11/d1e/d22/d27 39 2024-03-29T04:44:56.264 INFO:tasks.workunit.client.0.smithi142.stdout:1/223: dread - d8/db/dc/d19/d1d/d15/d21/f32 zero size 2024-03-29T04:44:56.266 INFO:tasks.workunit.client.0.smithi142.stdout:9/176: rename d0/d2/l23 to d0/d2/d39/l3d 0 2024-03-29T04:44:56.271 INFO:tasks.workunit.client.0.smithi142.stdout:7/204: mknod d7/d18/d30/d33/c37 0 2024-03-29T04:44:56.273 INFO:tasks.workunit.client.0.smithi142.stdout:1/224: fdatasync f7 0 2024-03-29T04:44:56.275 INFO:tasks.workunit.client.0.smithi142.stdout:4/179: rename d0/f10 to d0/d12/f3b 0 2024-03-29T04:44:56.276 INFO:tasks.workunit.client.0.smithi142.stdout:4/180: write d0/d12/d28/f3a [228555,108792] 0 2024-03-29T04:44:56.279 INFO:tasks.workunit.client.0.smithi142.stdout:5/288: dread fd [0,4194304] 0 2024-03-29T04:44:56.279 INFO:tasks.workunit.client.0.smithi142.stdout:9/177: dread d0/d2/d6/d16/f1c [0,4194304] 0 2024-03-29T04:44:56.282 INFO:tasks.workunit.client.0.smithi142.stdout:1/225: creat d8/db/dc/d19/f57 x:0 0 0 2024-03-29T04:44:56.283 INFO:tasks.workunit.client.0.smithi142.stdout:8/278: rename d5/l51 to d5/d10/d2d/l5b 0 2024-03-29T04:44:56.284 INFO:tasks.workunit.client.0.smithi142.stdout:1/226: read d8/db/dc/d19/d1d/d1f/f33 [319400,125115] 0 2024-03-29T04:44:56.286 INFO:tasks.workunit.client.0.smithi142.stdout:5/289: mkdir df/d1a/d21/d55 0 2024-03-29T04:44:56.288 INFO:tasks.workunit.client.0.smithi142.stdout:7/205: link d7/fd d7/d18/d30/d33/f38 0 2024-03-29T04:44:56.290 INFO:tasks.workunit.client.0.smithi142.stdout:7/206: chown d7/f19 13062 1 2024-03-29T04:44:56.290 INFO:tasks.workunit.client.0.smithi142.stdout:9/178: dread d0/d2/d6/fd [0,4194304] 0 2024-03-29T04:44:56.291 INFO:tasks.workunit.client.0.smithi142.stdout:3/211: truncate d3/d1f/d34/f11 8376147 0 2024-03-29T04:44:56.293 INFO:tasks.workunit.client.0.smithi142.stdout:8/279: symlink d5/d1d/d2f/d42/d48/l5c 0 2024-03-29T04:44:56.294 INFO:tasks.workunit.client.0.smithi142.stdout:1/227: dwrite d8/db/dc/d19/d1d/d1f/f29 [0,4194304] 0 2024-03-29T04:44:56.296 INFO:tasks.workunit.client.0.smithi142.stdout:7/207: unlink d7/d18/d30/d33/c37 0 2024-03-29T04:44:56.297 INFO:tasks.workunit.client.0.smithi142.stdout:4/181: getdents d0/dc/d15 0 2024-03-29T04:44:56.298 INFO:tasks.workunit.client.0.smithi142.stdout:5/290: dwrite f3 [0,4194304] 0 2024-03-29T04:44:56.299 INFO:tasks.workunit.client.0.smithi142.stdout:4/182: chown d0/d12/d28/f3a 0 1 2024-03-29T04:44:56.301 INFO:tasks.workunit.client.0.smithi142.stdout:7/208: mknod d7/d18/d30/c39 0 2024-03-29T04:44:56.303 INFO:tasks.workunit.client.0.smithi142.stdout:8/280: mkdir d5/d1d/d2f/d42/d5d 0 2024-03-29T04:44:56.307 INFO:tasks.workunit.client.0.smithi142.stdout:5/291: creat df/d16/d24/f56 x:0 0 0 2024-03-29T04:44:56.308 INFO:tasks.workunit.client.0.smithi142.stdout:5/292: chown df/d16/f39 551528 1 2024-03-29T04:44:56.310 INFO:tasks.workunit.client.0.smithi142.stdout:6/222: getdents d1c 0 2024-03-29T04:44:56.311 INFO:tasks.workunit.client.0.smithi142.stdout:4/183: creat d0/d12/f3c x:0 0 0 2024-03-29T04:44:56.311 INFO:tasks.workunit.client.0.smithi142.stdout:9/179: link d0/f1d d0/d2/d6/d16/f3e 0 2024-03-29T04:44:56.312 INFO:tasks.workunit.client.0.smithi142.stdout:7/209: mkdir d7/d27/d3a 0 2024-03-29T04:44:56.314 INFO:tasks.workunit.client.0.smithi142.stdout:8/281: creat d5/d10/d3c/f5e x:0 0 0 2024-03-29T04:44:56.314 INFO:tasks.workunit.client.0.smithi142.stdout:7/210: chown d7/dc/de/f2d 3741077 1 2024-03-29T04:44:56.315 INFO:tasks.workunit.client.0.smithi142.stdout:5/293: unlink df/d1a/d28/f35 0 2024-03-29T04:44:56.315 INFO:tasks.workunit.client.0.smithi142.stdout:4/184: dread d0/d12/f3b [0,4194304] 0 2024-03-29T04:44:56.317 INFO:tasks.workunit.client.0.smithi142.stdout:1/228: creat d8/db/dc/d19/d1d/d15/d21/d31/d4d/d56/f58 x:0 0 0 2024-03-29T04:44:56.318 INFO:tasks.workunit.client.0.smithi142.stdout:6/223: creat d1c/f41 x:0 0 0 2024-03-29T04:44:56.319 INFO:tasks.workunit.client.0.smithi142.stdout:9/180: symlink d0/d2/l3f 0 2024-03-29T04:44:56.320 INFO:tasks.workunit.client.0.smithi142.stdout:6/224: stat d1c/f2b 0 2024-03-29T04:44:56.321 INFO:tasks.workunit.client.0.smithi142.stdout:6/225: fsync f18 0 2024-03-29T04:44:56.321 INFO:tasks.workunit.client.0.smithi142.stdout:1/229: readlink d8/db/dc/d19/d1d/l36 0 2024-03-29T04:44:56.324 INFO:tasks.workunit.client.0.smithi142.stdout:6/226: write f18 [1289014,111706] 0 2024-03-29T04:44:56.324 INFO:tasks.workunit.client.0.smithi142.stdout:5/294: mknod df/d16/d24/d2e/c57 0 2024-03-29T04:44:56.324 INFO:tasks.workunit.client.0.smithi142.stdout:0/160: write d7/d9/fa [2759546,8] 0 2024-03-29T04:44:56.324 INFO:tasks.workunit.client.0.smithi142.stdout:8/282: dwrite d5/f1f [0,4194304] 0 2024-03-29T04:44:56.325 INFO:tasks.workunit.client.0.smithi142.stdout:7/211: dwrite d7/f9 [4194304,4194304] 0 2024-03-29T04:44:56.329 INFO:tasks.workunit.client.0.smithi142.stdout:8/283: fsync d5/f30 0 2024-03-29T04:44:56.330 INFO:tasks.workunit.client.0.smithi142.stdout:9/181: creat d0/d2/d6/d16/f40 x:0 0 0 2024-03-29T04:44:56.333 INFO:tasks.workunit.client.0.smithi142.stdout:6/227: mkdir d1c/d42 0 2024-03-29T04:44:56.336 INFO:tasks.workunit.client.0.smithi142.stdout:7/212: dwrite d7/d18/f24 [0,4194304] 0 2024-03-29T04:44:56.337 INFO:tasks.workunit.client.0.smithi142.stdout:0/161: rename f2 to d7/d13/f35 0 2024-03-29T04:44:56.338 INFO:tasks.workunit.client.0.smithi142.stdout:5/295: symlink df/d16/d25/l58 0 2024-03-29T04:44:56.342 INFO:tasks.workunit.client.0.smithi142.stdout:1/230: mknod d8/db/dc/d19/d1d/d15/d21/d31/d4d/c59 0 2024-03-29T04:44:56.351 INFO:tasks.workunit.client.0.smithi142.stdout:4/185: rmdir d0/d9/d11/d34 0 2024-03-29T04:44:56.351 INFO:tasks.workunit.client.0.smithi142.stdout:4/186: readlink d0/le 0 2024-03-29T04:44:56.351 INFO:tasks.workunit.client.0.smithi142.stdout:4/187: stat d0/d9/d19/l2f 0 2024-03-29T04:44:56.355 INFO:tasks.workunit.client.0.smithi142.stdout:7/213: fsync d7/f1a 0 2024-03-29T04:44:56.356 INFO:tasks.workunit.client.0.smithi142.stdout:8/284: rename d5/d1d/d2f/d42 to d5/d1d/d2f/d5f 0 2024-03-29T04:44:56.358 INFO:tasks.workunit.client.0.smithi142.stdout:3/212: truncate d3/d29/d2d/f3f 3042717 0 2024-03-29T04:44:56.359 INFO:tasks.workunit.client.0.smithi142.stdout:5/296: creat df/d16/d25/f59 x:0 0 0 2024-03-29T04:44:56.360 INFO:tasks.workunit.client.0.smithi142.stdout:8/285: dread d5/f1f [4194304,4194304] 0 2024-03-29T04:44:56.362 INFO:tasks.workunit.client.0.smithi142.stdout:1/231: rename d8/db/dc/d19/d1d/d15/d1a to d8/db/dc/d19/d1d/d15/d21/d5a 0 2024-03-29T04:44:56.363 INFO:tasks.workunit.client.0.smithi142.stdout:1/232: chown d8/db/d37/f41 24 1 2024-03-29T04:44:56.364 INFO:tasks.workunit.client.0.smithi142.stdout:1/233: write d8/f2a [5156175,31159] 0 2024-03-29T04:44:56.365 INFO:tasks.workunit.client.0.smithi142.stdout:4/188: write d0/d9/d11/ff [1341752,119181] 0 2024-03-29T04:44:56.366 INFO:tasks.workunit.client.0.smithi142.stdout:4/189: stat d0/c6 0 2024-03-29T04:44:56.372 INFO:tasks.workunit.client.0.smithi142.stdout:5/297: rename df/d1a/d28/l43 to df/d1a/d21/d55/l5a 0 2024-03-29T04:44:56.373 INFO:tasks.workunit.client.0.smithi142.stdout:2/167: dwrite de/d11/f21 [0,4194304] 0 2024-03-29T04:44:56.375 INFO:tasks.workunit.client.0.smithi142.stdout:4/190: mkdir d0/dc/d15/d3d 0 2024-03-29T04:44:56.375 INFO:tasks.workunit.client.0.smithi142.stdout:1/234: read d8/db/d37/f38 [116416,77236] 0 2024-03-29T04:44:56.378 INFO:tasks.workunit.client.0.smithi142.stdout:8/286: stat d5/d1d/d24/l2c 0 2024-03-29T04:44:56.381 INFO:tasks.workunit.client.0.smithi142.stdout:2/168: mknod de/d11/d17/c3c 0 2024-03-29T04:44:56.383 INFO:tasks.workunit.client.0.smithi142.stdout:4/191: creat d0/f3e x:0 0 0 2024-03-29T04:44:56.384 INFO:tasks.workunit.client.0.smithi142.stdout:0/162: getdents d7/d13 0 2024-03-29T04:44:56.385 INFO:tasks.workunit.client.0.smithi142.stdout:1/235: rename d8/l23 to d8/db/dc/d19/d1d/d15/d21/d31/l5b 0 2024-03-29T04:44:56.387 INFO:tasks.workunit.client.0.smithi142.stdout:0/163: write d7/f2b [253176,125128] 0 2024-03-29T04:44:56.392 INFO:tasks.workunit.client.0.smithi142.stdout:2/169: dwrite de/d11/d17/f25 [0,4194304] 0 2024-03-29T04:44:56.393 INFO:tasks.workunit.client.0.smithi142.stdout:6/228: dwrite d1c/f30 [0,4194304] 0 2024-03-29T04:44:56.394 INFO:tasks.workunit.client.0.smithi142.stdout:2/170: fdatasync de/d11/f21 0 2024-03-29T04:44:56.394 INFO:tasks.workunit.client.0.smithi142.stdout:6/229: chown d1c/f41 375675 1 2024-03-29T04:44:56.402 INFO:tasks.workunit.client.0.smithi142.stdout:6/230: dwrite d1c/f1d [4194304,4194304] 0 2024-03-29T04:44:56.403 INFO:tasks.workunit.client.0.smithi142.stdout:6/231: truncate d1c/f41 200809 0 2024-03-29T04:44:56.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.407+0000 7fdf00f68700 1 -- 172.21.15.142:0/3009148044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdefc0ffab0 msgr2=0x7fdefc0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:56.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.407+0000 7fdf00f68700 1 --2- 172.21.15.142:0/3009148044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdefc0ffab0 0x7fdefc0ffed0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009910 tx=0x7fdeec009c20 comp rx=0 tx=0).stop 2024-03-29T04:44:56.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.407+0000 7fdf00f68700 1 -- 172.21.15.142:0/3009148044 shutdown_connections 2024-03-29T04:44:56.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.407+0000 7fdf00f68700 1 --2- 172.21.15.142:0/3009148044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdefc100cb0 0x7fdefc101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:56.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.407+0000 7fdf00f68700 1 --2- 172.21.15.142:0/3009148044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdefc0ffab0 0x7fdefc0ffed0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:56.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.407+0000 7fdf00f68700 1 -- 172.21.15.142:0/3009148044 >> 172.21.15.142:0/3009148044 conn(0x7fdefc0fb050 msgr2=0x7fdefc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:56.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.407+0000 7fdf00f68700 1 -- 172.21.15.142:0/3009148044 shutdown_connections 2024-03-29T04:44:56.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.407+0000 7fdf00f68700 1 -- 172.21.15.142:0/3009148044 wait complete. 2024-03-29T04:44:56.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.408+0000 7fdf00f68700 1 Processor -- start 2024-03-29T04:44:56.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.408+0000 7fdf00f68700 1 -- start start 2024-03-29T04:44:56.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.408+0000 7fdf00f68700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdefc0ffab0 0x7fdefc069820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:56.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.408+0000 7fdf00f68700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdefc100cb0 0x7fdefc19b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:56.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.408+0000 7fdf00f68700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc19b6f0 con 0x7fdefc0ffab0 2024-03-29T04:44:56.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.408+0000 7fdf00f68700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc19b860 con 0x7fdefc100cb0 2024-03-29T04:44:56.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.408+0000 7fdefa59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdefc0ffab0 0x7fdefc069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:56.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.408+0000 7fdefa59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdefc0ffab0 0x7fdefc069820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40164/0 (socket says 172.21.15.142:40164) 2024-03-29T04:44:56.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.408+0000 7fdefa59c700 1 -- 172.21.15.142:0/2951939778 learned_addr learned my addr 172.21.15.142:0/2951939778 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:44:56.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.409+0000 7fdefa59c700 1 -- 172.21.15.142:0/2951939778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdefc100cb0 msgr2=0x7fdefc19b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:56.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.409+0000 7fdefa59c700 1 --2- 172.21.15.142:0/2951939778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdefc100cb0 0x7fdefc19b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:56.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.409+0000 7fdefa59c700 1 -- 172.21.15.142:0/2951939778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec0095f0 con 0x7fdefc0ffab0 2024-03-29T04:44:56.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.409+0000 7fdefa59c700 1 --2- 172.21.15.142:0/2951939778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdefc0ffab0 0x7fdefc069820 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00bda0 tx=0x7fdeec00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:56.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.409+0000 7fdeeb7fe700 1 -- 172.21.15.142:0/2951939778 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdeec01d070 con 0x7fdefc0ffab0 2024-03-29T04:44:56.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.409+0000 7fdeeb7fe700 1 -- 172.21.15.142:0/2951939778 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdeec004c50 con 0x7fdefc0ffab0 2024-03-29T04:44:56.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.409+0000 7fdf00f68700 1 -- 172.21.15.142:0/2951939778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc19bae0 con 0x7fdefc0ffab0 2024-03-29T04:44:56.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.409+0000 7fdeeb7fe700 1 -- 172.21.15.142:0/2951939778 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdeec00f690 con 0x7fdefc0ffab0 2024-03-29T04:44:56.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.410+0000 7fdf00f68700 1 -- 172.21.15.142:0/2951939778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc19c050 con 0x7fdefc0ffab0 2024-03-29T04:44:56.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.410+0000 7fdeeb7fe700 1 -- 172.21.15.142:0/2951939778 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fdeec00f7f0 con 0x7fdefc0ffab0 2024-03-29T04:44:56.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.411+0000 7fdeeb7fe700 1 --2- 172.21.15.142:0/2951939778 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fdee406c4e0 0x7fdee406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:44:56.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.411+0000 7fdf00f68700 1 -- 172.21.15.142:0/2951939778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc108690 con 0x7fdefc0ffab0 2024-03-29T04:44:56.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.411+0000 7fdeeb7fe700 1 -- 172.21.15.142:0/2951939778 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdeec091170 con 0x7fdefc0ffab0 2024-03-29T04:44:56.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.411+0000 7fdef9d9b700 1 --2- 172.21.15.142:0/2951939778 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fdee406c4e0 0x7fdee406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:44:56.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.412+0000 7fdef9d9b700 1 --2- 172.21.15.142:0/2951939778 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fdee406c4e0 0x7fdee406e9a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdefc069d80 tx=0x7fdef0006d20 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:44:56.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.416+0000 7fdeeb7fe700 1 -- 172.21.15.142:0/2951939778 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdeec092060 con 0x7fdefc0ffab0 2024-03-29T04:44:56.486 INFO:tasks.workunit.client.0.smithi142.stdout:3/213: sync 2024-03-29T04:44:56.534 INFO:tasks.workunit.client.0.smithi142.stdout:8/287: rename d5/d1d/d2f/d5f/d5d to d5/d10/d4c/d60 0 2024-03-29T04:44:56.538 INFO:tasks.workunit.client.0.smithi142.stdout:4/192: fdatasync d0/d9/d11/f38 0 2024-03-29T04:44:56.544 INFO:tasks.workunit.client.0.smithi142.stdout:9/182: dwrite d0/f22 [0,4194304] 0 2024-03-29T04:44:56.544 INFO:tasks.workunit.client.0.smithi142.stdout:6/232: unlink d1c/c29 0 2024-03-29T04:44:56.547 INFO:tasks.workunit.client.0.smithi142.stdout:7/214: dwrite d7/d18/d30/d33/f38 [4194304,4194304] 0 2024-03-29T04:44:56.547 INFO:tasks.workunit.client.0.smithi142.stdout:0/164: rename d7/d13/l2c to d7/d9/d1e/l36 0 2024-03-29T04:44:56.548 INFO:tasks.workunit.client.0.smithi142.stdout:8/288: creat d5/d1d/d2f/d5f/d48/f61 x:0 0 0 2024-03-29T04:44:56.552 INFO:tasks.workunit.client.0.smithi142.stdout:4/193: mkdir d0/d12/d3f 0 2024-03-29T04:44:56.552 INFO:tasks.workunit.client.0.smithi142.stdout:2/171: mkdir de/d11/d3d 0 2024-03-29T04:44:56.555 INFO:tasks.workunit.client.0.smithi142.stdout:6/233: write f4 [1617988,128305] 0 2024-03-29T04:44:56.556 INFO:tasks.workunit.client.0.smithi142.stdout:5/298: dwrite df/d1a/d21/f3e [4194304,4194304] 0 2024-03-29T04:44:56.558 INFO:tasks.workunit.client.0.smithi142.stdout:5/299: chown df/d16/d24/d2e/d47/d4d 21793 1 2024-03-29T04:44:56.558 INFO:tasks.workunit.client.0.smithi142.stdout:8/289: mknod d5/d2e/c62 0 2024-03-29T04:44:56.559 INFO:tasks.workunit.client.0.smithi142.stdout:5/300: fdatasync df/d1a/d28/f49 0 2024-03-29T04:44:56.560 INFO:tasks.workunit.client.0.smithi142.stdout:0/165: rename d7/d13/d27 to d7/d9/d22/d37 0 2024-03-29T04:44:56.560 INFO:tasks.workunit.client.0.smithi142.stdout:5/301: write df/d2c/f40 [988534,89473] 0 2024-03-29T04:44:56.561 INFO:tasks.workunit.client.0.smithi142.stdout:1/236: chown d8/db/dc/d19/d1d/d15/d21/d31/l5b 27448962 1 2024-03-29T04:44:56.562 INFO:tasks.workunit.client.0.smithi142.stdout:1/237: chown d8/db/f49 0 1 2024-03-29T04:44:56.565 INFO:tasks.workunit.client.0.smithi142.stdout:5/302: dread df/d1a/f1f [0,4194304] 0 2024-03-29T04:44:56.566 INFO:tasks.workunit.client.0.smithi142.stdout:4/194: symlink d0/d12/d28/l40 0 2024-03-29T04:44:56.568 INFO:tasks.workunit.client.0.smithi142.stdout:3/214: dread d3/d1f/d34/f11 [0,4194304] 0 2024-03-29T04:44:56.570 INFO:tasks.workunit.client.0.smithi142.stdout:6/234: unlink d1c/l22 0 2024-03-29T04:44:56.575 INFO:tasks.workunit.client.0.smithi142.stdout:6/235: dwrite fc [8388608,4194304] 0 2024-03-29T04:44:56.575 INFO:tasks.workunit.client.0.smithi142.stdout:0/166: readlink d7/d9/d1e/l36 0 2024-03-29T04:44:56.577 INFO:tasks.workunit.client.0.smithi142.stdout:4/195: mknod d0/d9/c41 0 2024-03-29T04:44:56.582 INFO:tasks.workunit.client.0.smithi142.stdout:1/238: link d8/db/f14 d8/db/dc/d19/f5c 0 2024-03-29T04:44:56.583 INFO:tasks.workunit.client.0.smithi142.stdout:7/215: write d7/fa [485544,114528] 0 2024-03-29T04:44:56.584 INFO:tasks.workunit.client.0.smithi142.stdout:6/236: dread f4 [0,4194304] 0 2024-03-29T04:44:56.585 INFO:tasks.workunit.client.0.smithi142.stdout:5/303: link df/d1a/f2f df/d4a/f5b 0 2024-03-29T04:44:56.586 INFO:tasks.workunit.client.0.smithi142.stdout:7/216: dread - d7/f19 zero size 2024-03-29T04:44:56.587 INFO:tasks.workunit.client.0.smithi142.stdout:6/237: stat d1c/d33/f36 0 2024-03-29T04:44:56.587 INFO:tasks.workunit.client.0.smithi142.stdout:2/172: write f7 [346739,14635] 0 2024-03-29T04:44:56.588 INFO:tasks.workunit.client.0.smithi142.stdout:3/215: mkdir d3/d1f/d34/d4d 0 2024-03-29T04:44:56.591 INFO:tasks.workunit.client.0.smithi142.stdout:6/238: dread - d1c/f34 zero size 2024-03-29T04:44:56.592 INFO:tasks.workunit.client.0.smithi142.stdout:4/196: dwrite d0/d12/f1e [0,4194304] 0 2024-03-29T04:44:56.592 INFO:tasks.workunit.client.0.smithi142.stdout:1/239: creat d8/db/dc/d19/d1d/f5d x:0 0 0 2024-03-29T04:44:56.592 INFO:tasks.workunit.client.0.smithi142.stdout:9/183: dwrite d0/d2/fe [0,4194304] 0 2024-03-29T04:44:56.592 INFO:tasks.workunit.client.0.smithi142.stdout:7/217: write d7/dc/de/d20/f21 [4118773,75038] 0 2024-03-29T04:44:56.594 INFO:tasks.workunit.client.0.smithi142.stdout:8/290: getdents d5/d10/d2d 0 2024-03-29T04:44:56.595 INFO:tasks.workunit.client.0.smithi142.stdout:5/304: mkdir df/d16/d24/d5c 0 2024-03-29T04:44:56.596 INFO:tasks.workunit.client.0.smithi142.stdout:6/239: read - d1c/f3c zero size 2024-03-29T04:44:56.600 INFO:tasks.workunit.client.0.smithi142.stdout:2/173: mkdir de/d11/d2b/d3e 0 2024-03-29T04:44:56.602 INFO:tasks.workunit.client.0.smithi142.stdout:5/305: stat df/d16/d24/d3d/c46 0 2024-03-29T04:44:56.603 INFO:tasks.workunit.client.0.smithi142.stdout:1/240: mkdir d8/db/d5e 0 2024-03-29T04:44:56.605 INFO:tasks.workunit.client.0.smithi142.stdout:2/174: fsync de/d11/d17/f31 0 2024-03-29T04:44:56.606 INFO:tasks.workunit.client.0.smithi142.stdout:1/241: read - d8/db/d37/f41 zero size 2024-03-29T04:44:56.607 INFO:tasks.workunit.client.0.smithi142.stdout:0/167: dread d7/d9/ff [0,4194304] 0 2024-03-29T04:44:56.609 INFO:tasks.workunit.client.0.smithi142.stdout:4/197: fdatasync d0/d9/d11/f38 0 2024-03-29T04:44:56.611 INFO:tasks.workunit.client.0.smithi142.stdout:6/240: mknod d1c/d42/c43 0 2024-03-29T04:44:56.611 INFO:tasks.workunit.client.0.smithi142.stdout:3/216: creat d3/d1f/d34/f4e x:0 0 0 2024-03-29T04:44:56.615 INFO:tasks.workunit.client.0.smithi142.stdout:8/291: dwrite d5/d10/d3c/f5e [0,4194304] 0 2024-03-29T04:44:56.615 INFO:tasks.workunit.client.0.smithi142.stdout:4/198: stat d0/f14 0 2024-03-29T04:44:56.617 INFO:tasks.workunit.client.0.smithi142.stdout:7/218: dwrite d7/fa [0,4194304] 0 2024-03-29T04:44:56.618 INFO:tasks.workunit.client.0.smithi142.stdout:0/168: truncate d7/d13/f2d 600147 0 2024-03-29T04:44:56.620 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:56 smithi142 ceph-mon[29027]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:56.620 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:56 smithi142 ceph-mon[29027]: pgmap v264: 65 pgs: 65 active+clean; 337 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 15 MiB/s wr, 238 op/s 2024-03-29T04:44:56.620 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:56 smithi142 ceph-mon[29027]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:56.620 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:56 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1815774769' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:44:56.622 INFO:tasks.workunit.client.0.smithi142.stdout:8/292: write d5/f18 [8190136,9929] 0 2024-03-29T04:44:56.622 INFO:tasks.workunit.client.0.smithi142.stdout:7/219: fsync d7/d27/f29 0 2024-03-29T04:44:56.633 INFO:tasks.workunit.client.0.smithi142.stdout:5/306: dwrite df/d2c/f40 [0,4194304] 0 2024-03-29T04:44:56.633 INFO:tasks.workunit.client.0.smithi142.stdout:1/242: dwrite d8/db/f25 [0,4194304] 0 2024-03-29T04:44:56.633 INFO:tasks.workunit.client.0.smithi142.stdout:6/241: link d1c/c1f d1c/c44 0 2024-03-29T04:44:56.637 INFO:tasks.workunit.client.0.smithi142.stdout:1/243: read - d8/db/f14 zero size 2024-03-29T04:44:56.638 INFO:tasks.workunit.client.0.smithi142.stdout:2/175: dread de/d11/f21 [0,4194304] 0 2024-03-29T04:44:56.639 INFO:tasks.workunit.client.0.smithi142.stdout:9/184: dwrite d0/d2/d6/d16/f2b [0,4194304] 0 2024-03-29T04:44:56.640 INFO:tasks.workunit.client.0.smithi142.stdout:8/293: mknod d5/d10/d3c/c63 0 2024-03-29T04:44:56.641 INFO:tasks.workunit.client.0.smithi142.stdout:0/169: rename d7/d13/f2d to d7/d9/d22/d37/f38 0 2024-03-29T04:44:56.647 INFO:tasks.workunit.client.0.smithi142.stdout:5/307: mkdir df/d29/d5d 0 2024-03-29T04:44:56.647 INFO:tasks.workunit.client.0.smithi142.stdout:4/199: dwrite d0/d9/d11/f23 [0,4194304] 0 2024-03-29T04:44:56.649 INFO:tasks.workunit.client.0.smithi142.stdout:9/185: rmdir d0/d2/d6 39 2024-03-29T04:44:56.650 INFO:tasks.workunit.client.0.smithi142.stdout:7/220: symlink d7/d27/d3a/l3b 0 2024-03-29T04:44:56.650 INFO:tasks.workunit.client.0.smithi142.stdout:6/242: write d1c/d33/f36 [675767,107093] 0 2024-03-29T04:44:56.651 INFO:tasks.workunit.client.0.smithi142.stdout:7/221: dread - d7/f19 zero size 2024-03-29T04:44:56.652 INFO:tasks.workunit.client.0.smithi142.stdout:8/294: symlink d5/d10/d2d/l64 0 2024-03-29T04:44:56.653 INFO:tasks.workunit.client.0.smithi142.stdout:0/170: creat d7/d9/d22/d37/f39 x:0 0 0 2024-03-29T04:44:56.653 INFO:tasks.workunit.client.0.smithi142.stdout:3/217: dwrite d3/d1f/d34/d6/f20 [0,4194304] 0 2024-03-29T04:44:56.654 INFO:tasks.workunit.client.0.smithi142.stdout:7/222: write f1 [2855520,126641] 0 2024-03-29T04:44:56.669 INFO:tasks.workunit.client.0.smithi142.stdout:6/243: mkdir d1c/d45 0 2024-03-29T04:44:56.669 INFO:tasks.workunit.client.0.smithi142.stdout:1/244: getdents d8/db/dc/d19/d1d/d2d/d44/d53 0 2024-03-29T04:44:56.670 INFO:tasks.workunit.client.0.smithi142.stdout:9/186: write d0/d2/d6/d35/d34/f2d [173436,88567] 0 2024-03-29T04:44:56.671 INFO:tasks.workunit.client.0.smithi142.stdout:7/223: fsync d7/dc/de/d20/f2b 0 2024-03-29T04:44:56.671 INFO:tasks.workunit.client.0.smithi142.stdout:8/295: rmdir d5 39 2024-03-29T04:44:56.673 INFO:tasks.workunit.client.0.smithi142.stdout:0/171: creat d7/d9/d22/f3a x:0 0 0 2024-03-29T04:44:56.673 INFO:tasks.workunit.client.0.smithi142.stdout:1/245: readlink d8/db/dc/d19/d1d/d1f/d50/l52 0 2024-03-29T04:44:56.673 INFO:tasks.workunit.client.0.smithi142.stdout:3/218: creat d3/d1f/d24/d47/f4f x:0 0 0 2024-03-29T04:44:56.674 INFO:tasks.workunit.client.0.smithi142.stdout:0/172: read - d7/d9/d22/f23 zero size 2024-03-29T04:44:56.675 INFO:tasks.workunit.client.0.smithi142.stdout:2/176: rename fa to de/d11/d1e/d22/f3f 0 2024-03-29T04:44:56.675 INFO:tasks.workunit.client.0.smithi142.stdout:7/224: mknod d7/d27/d3a/c3c 0 2024-03-29T04:44:56.677 INFO:tasks.workunit.client.0.smithi142.stdout:1/246: chown d8/db/dc/d19/d1d/d1f/f29 39 1 2024-03-29T04:44:56.680 INFO:tasks.workunit.client.0.smithi142.stdout:4/200: rename d0/d9/c41 to d0/d9/d19/d2c/c42 0 2024-03-29T04:44:56.681 INFO:tasks.workunit.client.0.smithi142.stdout:3/219: mkdir d3/d1f/d34/d4d/d50 0 2024-03-29T04:44:56.681 INFO:tasks.workunit.client.0.smithi142.stdout:9/187: dwrite d0/f15 [0,4194304] 0 2024-03-29T04:44:56.684 INFO:tasks.workunit.client.0.smithi142.stdout:2/177: dread f9 [0,4194304] 0 2024-03-29T04:44:56.685 INFO:tasks.workunit.client.0.smithi142.stdout:9/188: dread - d0/d2/d6/d16/f37 zero size 2024-03-29T04:44:56.685 INFO:tasks.workunit.client.0.smithi142.stdout:1/247: creat d8/db/dc/d19/d1d/d2d/d44/d53/f5f x:0 0 0 2024-03-29T04:44:56.686 INFO:tasks.workunit.client.0.smithi142.stdout:0/173: dwrite d7/d9/fa [0,4194304] 0 2024-03-29T04:44:56.686 INFO:tasks.workunit.client.0.smithi142.stdout:2/178: dread - de/f36 zero size 2024-03-29T04:44:56.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.687+0000 7fdf00f68700 1 -- 172.21.15.142:0/2951939778 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdefc105600 con 0x7fdee406c4e0 2024-03-29T04:44:56.688 INFO:tasks.workunit.client.0.smithi142.stdout:8/296: link d5/d1d/d2f/d5f/d48/f61 d5/d2e/f65 0 2024-03-29T04:44:56.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.689+0000 7fdeeb7fe700 1 -- 172.21.15.142:0/2951939778 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fdefc105600 con 0x7fdee406c4e0 2024-03-29T04:44:56.688 INFO:tasks.workunit.client.0.smithi142.stdout:6/244: truncate d1c/f41 149351 0 2024-03-29T04:44:56.689 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:44:56.689 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:44:56.689 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:44:56.689 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T04:44:56.689 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [], 2024-03-29T04:44:56.689 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "0/2 daemons upgraded", 2024-03-29T04:44:56.689 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 image on host smithi152", 2024-03-29T04:44:56.689 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:44:56.689 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:44:56.690 INFO:tasks.workunit.client.0.smithi142.stdout:4/201: truncate d0/f3 3121601 0 2024-03-29T04:44:56.690 INFO:tasks.workunit.client.0.smithi142.stdout:8/297: dread d5/f19 [0,4194304] 0 2024-03-29T04:44:56.690 INFO:tasks.workunit.client.0.smithi142.stdout:4/202: fdatasync d0/d9/fb 0 2024-03-29T04:44:56.691 INFO:tasks.workunit.client.0.smithi142.stdout:4/203: write d0/f21 [2809739,5108] 0 2024-03-29T04:44:56.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.693+0000 7fdf00f68700 1 -- 172.21.15.142:0/2951939778 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fdee406c4e0 msgr2=0x7fdee406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:56.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.693+0000 7fdf00f68700 1 --2- 172.21.15.142:0/2951939778 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fdee406c4e0 0x7fdee406e9a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdefc069d80 tx=0x7fdef0006d20 comp rx=0 tx=0).stop 2024-03-29T04:44:56.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.693+0000 7fdf00f68700 1 -- 172.21.15.142:0/2951939778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdefc0ffab0 msgr2=0x7fdefc069820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:44:56.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.693+0000 7fdf00f68700 1 --2- 172.21.15.142:0/2951939778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdefc0ffab0 0x7fdefc069820 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00bda0 tx=0x7fdeec00be80 comp rx=0 tx=0).stop 2024-03-29T04:44:56.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.693+0000 7fdf00f68700 1 -- 172.21.15.142:0/2951939778 shutdown_connections 2024-03-29T04:44:56.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.693+0000 7fdf00f68700 1 --2- 172.21.15.142:0/2951939778 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fdee406c4e0 0x7fdee406e9a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:56.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.693+0000 7fdf00f68700 1 --2- 172.21.15.142:0/2951939778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdefc0ffab0 0x7fdefc069820 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:56.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.693+0000 7fdf00f68700 1 --2- 172.21.15.142:0/2951939778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdefc100cb0 0x7fdefc19b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:44:56.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.693+0000 7fdf00f68700 1 -- 172.21.15.142:0/2951939778 >> 172.21.15.142:0/2951939778 conn(0x7fdefc0fb050 msgr2=0x7fdefc103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:44:56.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.693+0000 7fdf00f68700 1 -- 172.21.15.142:0/2951939778 shutdown_connections 2024-03-29T04:44:56.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:44:56.693+0000 7fdf00f68700 1 -- 172.21.15.142:0/2951939778 wait complete. 2024-03-29T04:44:56.694 INFO:tasks.workunit.client.0.smithi142.stdout:3/220: rename d3/d29/d2b/d40/f44 to d3/d1f/d34/d4d/d50/f51 0 2024-03-29T04:44:56.694 INFO:tasks.workunit.client.0.smithi142.stdout:9/189: mknod d0/d2/d6/d27/c41 0 2024-03-29T04:44:56.695 INFO:tasks.workunit.client.0.smithi142.stdout:0/174: rmdir d7/d9/d1e 39 2024-03-29T04:44:56.696 INFO:tasks.workunit.client.0.smithi142.stdout:2/179: mknod de/d11/d2b/c40 0 2024-03-29T04:44:56.697 INFO:tasks.workunit.client.0.smithi142.stdout:4/204: truncate d0/d12/d28/f3a 419848 0 2024-03-29T04:44:56.698 INFO:tasks.workunit.client.0.smithi142.stdout:1/248: creat d8/db/dc/d19/d1d/d15/f60 x:0 0 0 2024-03-29T04:44:56.699 INFO:tasks.workunit.client.0.smithi142.stdout:8/298: dread d5/d10/d3c/f5e [0,4194304] 0 2024-03-29T04:44:56.700 INFO:tasks.workunit.client.0.smithi142.stdout:6/245: unlink d1c/d42/c43 0 2024-03-29T04:44:56.704 INFO:tasks.workunit.client.0.smithi142.stdout:3/221: dread d3/d1f/d34/d6/f20 [0,4194304] 0 2024-03-29T04:44:56.705 INFO:tasks.workunit.client.0.smithi142.stdout:9/190: dwrite d0/f22 [0,4194304] 0 2024-03-29T04:44:56.755 INFO:tasks.workunit.client.0.smithi142.stdout:5/308: sync 2024-03-29T04:44:56.765 INFO:tasks.workunit.client.0.smithi142.stdout:7/225: truncate f6 3187495 0 2024-03-29T04:44:56.766 INFO:tasks.workunit.client.0.smithi142.stdout:2/180: symlink de/d11/d17/l41 0 2024-03-29T04:44:56.767 INFO:tasks.workunit.client.0.smithi142.stdout:6/246: mkdir d1c/d33/d46 0 2024-03-29T04:44:56.768 INFO:tasks.workunit.client.0.smithi142.stdout:8/299: mkdir d5/d1d/d24/d66 0 2024-03-29T04:44:56.771 INFO:tasks.workunit.client.0.smithi142.stdout:3/222: unlink d3/d29/f2e 0 2024-03-29T04:44:56.773 INFO:tasks.workunit.client.0.smithi142.stdout:2/181: dwrite fd [0,4194304] 0 2024-03-29T04:44:56.773 INFO:tasks.workunit.client.0.smithi142.stdout:8/300: dread d5/d10/f32 [0,4194304] 0 2024-03-29T04:44:56.773 INFO:tasks.workunit.client.0.smithi142.stdout:3/223: write d3/d1f/d34/f27 [1806562,117463] 0 2024-03-29T04:44:56.774 INFO:tasks.workunit.client.0.smithi142.stdout:1/249: rename d8/db/dc/d19/f34 to d8/db/dc/d19/d1d/d15/d21/d31/f61 0 2024-03-29T04:44:56.775 INFO:tasks.workunit.client.0.smithi142.stdout:1/250: truncate d8/f2b 309034 0 2024-03-29T04:44:56.776 INFO:tasks.workunit.client.0.smithi142.stdout:9/191: unlink d0/d2/c4 0 2024-03-29T04:44:56.781 INFO:tasks.workunit.client.0.smithi142.stdout:8/301: dwrite d5/d1d/d24/f33 [0,4194304] 0 2024-03-29T04:44:56.784 INFO:tasks.workunit.client.0.smithi142.stdout:4/205: creat d0/dc/d15/d3d/f43 x:0 0 0 2024-03-29T04:44:56.786 INFO:tasks.workunit.client.0.smithi142.stdout:4/206: dread d0/dc/f33 [0,4194304] 0 2024-03-29T04:44:56.789 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:56 smithi152 ceph-mon[37177]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:56.789 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:56 smithi152 ceph-mon[37177]: pgmap v264: 65 pgs: 65 active+clean; 337 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 15 MiB/s wr, 238 op/s 2024-03-29T04:44:56.789 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:56 smithi152 ceph-mon[37177]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:56.789 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:56 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1815774769' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:44:56.791 INFO:tasks.workunit.client.0.smithi142.stdout:5/309: rename f3 to df/d16/d24/d5c/f5e 0 2024-03-29T04:44:56.792 INFO:tasks.workunit.client.0.smithi142.stdout:1/251: mknod d8/db/c62 0 2024-03-29T04:44:56.793 INFO:tasks.workunit.client.0.smithi142.stdout:1/252: read - d8/db/f49 zero size 2024-03-29T04:44:56.796 INFO:tasks.workunit.client.0.smithi142.stdout:4/207: creat d0/d12/f44 x:0 0 0 2024-03-29T04:44:56.797 INFO:tasks.workunit.client.0.smithi142.stdout:4/208: write d0/d12/f3c [439567,14290] 0 2024-03-29T04:44:56.801 INFO:tasks.workunit.client.0.smithi142.stdout:5/310: symlink df/d16/d50/l5f 0 2024-03-29T04:44:56.802 INFO:tasks.workunit.client.0.smithi142.stdout:5/311: stat df/d1a/d28/f49 0 2024-03-29T04:44:56.802 INFO:tasks.workunit.client.0.smithi142.stdout:8/302: symlink d5/d1d/d24/d66/l67 0 2024-03-29T04:44:56.802 INFO:tasks.workunit.client.0.smithi142.stdout:0/175: truncate d7/d9/f10 850626 0 2024-03-29T04:44:56.803 INFO:tasks.workunit.client.0.smithi142.stdout:5/312: dread - df/d16/d25/f59 zero size 2024-03-29T04:44:56.803 INFO:tasks.workunit.client.0.smithi142.stdout:0/176: stat d7/d13/f28 0 2024-03-29T04:44:56.803 INFO:tasks.workunit.client.0.smithi142.stdout:5/313: fdatasync df/d2c/f40 0 2024-03-29T04:44:56.809 INFO:tasks.workunit.client.0.smithi142.stdout:1/253: creat d8/db/dc/d19/d1d/d15/d21/d5a/f63 x:0 0 0 2024-03-29T04:44:56.810 INFO:tasks.workunit.client.0.smithi142.stdout:4/209: mkdir d0/d45 0 2024-03-29T04:44:56.810 INFO:tasks.workunit.client.0.smithi142.stdout:8/303: creat d5/d1d/d2f/f68 x:0 0 0 2024-03-29T04:44:56.813 INFO:tasks.workunit.client.0.smithi142.stdout:0/177: symlink d7/d9/d22/d37/d2f/l3b 0 2024-03-29T04:44:56.814 INFO:tasks.workunit.client.0.smithi142.stdout:3/224: getdents d3/d29/d2b 0 2024-03-29T04:44:56.816 INFO:tasks.workunit.client.0.smithi142.stdout:2/182: rename f6 to de/f42 0 2024-03-29T04:44:56.816 INFO:tasks.workunit.client.0.smithi142.stdout:7/226: dwrite f6 [0,4194304] 0 2024-03-29T04:44:56.817 INFO:tasks.workunit.client.0.smithi142.stdout:4/210: symlink d0/l46 0 2024-03-29T04:44:56.818 INFO:tasks.workunit.client.0.smithi142.stdout:1/254: creat d8/db/dc/d19/d1d/d15/d21/d31/f64 x:0 0 0 2024-03-29T04:44:56.819 INFO:tasks.workunit.client.0.smithi142.stdout:7/227: write d7/dc/de/d20/f21 [4563960,70568] 0 2024-03-29T04:44:56.820 INFO:tasks.workunit.client.0.smithi142.stdout:5/314: dwrite fd [0,4194304] 0 2024-03-29T04:44:56.821 INFO:tasks.workunit.client.0.smithi142.stdout:4/211: dread d0/d12/d28/f3a [0,4194304] 0 2024-03-29T04:44:56.821 INFO:tasks.workunit.client.0.smithi142.stdout:7/228: read f6 [3702582,13218] 0 2024-03-29T04:44:56.822 INFO:tasks.workunit.client.0.smithi142.stdout:1/255: chown d8/f2a 44 1 2024-03-29T04:44:56.822 INFO:tasks.workunit.client.0.smithi142.stdout:3/225: unlink d3/d29/f3b 0 2024-03-29T04:44:56.823 INFO:tasks.workunit.client.0.smithi142.stdout:7/229: stat d7/dc/de/d20/f2b 0 2024-03-29T04:44:56.827 INFO:tasks.workunit.client.0.smithi142.stdout:3/226: readlink d3/d1f/d24/l31 0 2024-03-29T04:44:56.830 INFO:tasks.workunit.client.0.smithi142.stdout:2/183: unlink f3 0 2024-03-29T04:44:56.831 INFO:tasks.workunit.client.0.smithi142.stdout:1/256: write d8/db/dc/d19/d1d/d2d/f3e [834517,13257] 0 2024-03-29T04:44:56.832 INFO:tasks.workunit.client.0.smithi142.stdout:8/304: getdents d5/d10/d4c/d60 0 2024-03-29T04:44:56.833 INFO:tasks.workunit.client.0.smithi142.stdout:0/178: fsync d7/d9/f1c 0 2024-03-29T04:44:56.833 INFO:tasks.workunit.client.0.smithi142.stdout:2/184: write fd [3113708,98517] 0 2024-03-29T04:44:56.834 INFO:tasks.workunit.client.0.smithi142.stdout:4/212: mkdir d0/d12/d28/d47 0 2024-03-29T04:44:56.835 INFO:tasks.workunit.client.0.smithi142.stdout:3/227: mknod d3/d29/d2d/c52 0 2024-03-29T04:44:56.836 INFO:tasks.workunit.client.0.smithi142.stdout:5/315: dwrite df/d1a/f31 [0,4194304] 0 2024-03-29T04:44:56.841 INFO:tasks.workunit.client.0.smithi142.stdout:4/213: creat d0/f48 x:0 0 0 2024-03-29T04:44:56.841 INFO:tasks.workunit.client.0.smithi142.stdout:7/230: dwrite d7/dc/de/d20/f21 [0,4194304] 0 2024-03-29T04:44:56.841 INFO:tasks.workunit.client.0.smithi142.stdout:1/257: fdatasync d8/db/dc/d19/d1d/d15/f47 0 2024-03-29T04:44:56.843 INFO:tasks.workunit.client.0.smithi142.stdout:9/192: rename d0/d2/d6/d35/l33 to d0/l42 0 2024-03-29T04:44:56.843 INFO:tasks.workunit.client.0.smithi142.stdout:5/316: write df/d1a/d21/f3e [5162047,40813] 0 2024-03-29T04:44:56.850 INFO:tasks.workunit.client.0.smithi142.stdout:2/185: dread de/d11/d1e/d22/f3f [0,4194304] 0 2024-03-29T04:44:56.851 INFO:tasks.workunit.client.0.smithi142.stdout:8/305: rename d5/cb to d5/d1d/d24/d66/c69 0 2024-03-29T04:44:56.852 INFO:tasks.workunit.client.0.smithi142.stdout:7/231: mkdir d7/d18/d3d 0 2024-03-29T04:44:56.852 INFO:tasks.workunit.client.0.smithi142.stdout:2/186: write f7 [1393984,114490] 0 2024-03-29T04:44:56.853 INFO:tasks.workunit.client.0.smithi142.stdout:1/258: fsync d8/db/d37/f38 0 2024-03-29T04:44:56.857 INFO:tasks.workunit.client.0.smithi142.stdout:4/214: dwrite d0/dc/d15/d3d/f43 [0,4194304] 0 2024-03-29T04:44:56.859 INFO:tasks.workunit.client.0.smithi142.stdout:9/193: rename d0/f15 to d0/d12/f43 0 2024-03-29T04:44:56.859 INFO:tasks.workunit.client.0.smithi142.stdout:8/306: write d5/d10/d3c/f3e [1820714,15559] 0 2024-03-29T04:44:56.861 INFO:tasks.workunit.client.0.smithi142.stdout:5/317: symlink df/d16/d24/d2e/d47/d4d/l60 0 2024-03-29T04:44:56.861 INFO:tasks.workunit.client.0.smithi142.stdout:5/318: chown fd 462896 1 2024-03-29T04:44:56.862 INFO:tasks.workunit.client.0.smithi142.stdout:2/187: mknod de/d11/d17/c43 0 2024-03-29T04:44:56.862 INFO:tasks.workunit.client.0.smithi142.stdout:5/319: stat df/c15 0 2024-03-29T04:44:56.868 INFO:tasks.workunit.client.0.smithi142.stdout:1/259: mknod d8/db/dc/d19/d1d/d1f/d50/c65 0 2024-03-29T04:44:56.870 INFO:tasks.workunit.client.0.smithi142.stdout:3/228: dread d3/d1f/d34/d6/f14 [0,4194304] 0 2024-03-29T04:44:56.885 INFO:tasks.workunit.client.0.smithi142.stdout:1/260: read d8/db/f20 [379607,94744] 0 2024-03-29T04:44:56.885 INFO:tasks.workunit.client.0.smithi142.stdout:7/232: rename d7/dc/f1e to d7/d18/d30/f3e 0 2024-03-29T04:44:56.886 INFO:tasks.workunit.client.0.smithi142.stdout:0/179: dwrite d7/d9/ff [8388608,4194304] 0 2024-03-29T04:44:56.888 INFO:tasks.workunit.client.0.smithi142.stdout:1/261: write d8/f2a [3967741,61065] 0 2024-03-29T04:44:56.889 INFO:tasks.workunit.client.0.smithi142.stdout:0/180: write d7/d9/ff [5044572,113087] 0 2024-03-29T04:44:56.889 INFO:tasks.workunit.client.0.smithi142.stdout:3/229: mknod d3/d29/c53 0 2024-03-29T04:44:56.891 INFO:tasks.workunit.client.0.smithi142.stdout:9/194: write d0/f1d [4367575,128521] 0 2024-03-29T04:44:56.892 INFO:tasks.workunit.client.0.smithi142.stdout:0/181: write d7/d9/f2e [576880,123615] 0 2024-03-29T04:44:56.896 INFO:tasks.workunit.client.0.smithi142.stdout:3/230: mkdir d3/d54 0 2024-03-29T04:44:56.898 INFO:tasks.workunit.client.0.smithi142.stdout:4/215: getdents d0 0 2024-03-29T04:44:56.899 INFO:tasks.workunit.client.0.smithi142.stdout:1/262: mkdir d8/db/dc/d66 0 2024-03-29T04:44:56.901 INFO:tasks.workunit.client.0.smithi142.stdout:7/233: mkdir d7/dc/de/d20/d3f 0 2024-03-29T04:44:56.901 INFO:tasks.workunit.client.0.smithi142.stdout:2/188: chown de/d11/d1e/d22/d27/d3a 25401 1 2024-03-29T04:44:56.902 INFO:tasks.workunit.client.0.smithi142.stdout:9/195: dwrite d0/f38 [0,4194304] 0 2024-03-29T04:44:56.903 INFO:tasks.workunit.client.0.smithi142.stdout:0/182: creat d7/d9/f3c x:0 0 0 2024-03-29T04:44:56.903 INFO:tasks.workunit.client.0.smithi142.stdout:5/320: dread df/d1a/f22 [0,4194304] 0 2024-03-29T04:44:56.909 INFO:tasks.workunit.client.0.smithi142.stdout:0/183: readlink d7/d9/d1e/l2a 0 2024-03-29T04:44:56.910 INFO:tasks.workunit.client.0.smithi142.stdout:7/234: rename d7/f9 to d7/dc/de/d20/f40 0 2024-03-29T04:44:56.913 INFO:tasks.workunit.client.0.smithi142.stdout:5/321: rmdir df/d1a/d21/d27 39 2024-03-29T04:44:56.913 INFO:tasks.workunit.client.0.smithi142.stdout:4/216: dwrite d0/d9/d11/ff [0,4194304] 0 2024-03-29T04:44:56.914 INFO:tasks.workunit.client.0.smithi142.stdout:3/231: dread d3/d1f/d34/d6/f14 [0,4194304] 0 2024-03-29T04:44:56.917 INFO:tasks.workunit.client.0.smithi142.stdout:1/263: dwrite d8/f54 [0,4194304] 0 2024-03-29T04:44:56.918 INFO:tasks.workunit.client.0.smithi142.stdout:0/184: creat d7/d9/d22/d37/d2f/f3d x:0 0 0 2024-03-29T04:44:56.919 INFO:tasks.workunit.client.0.smithi142.stdout:2/189: dwrite de/f29 [0,4194304] 0 2024-03-29T04:44:56.919 INFO:tasks.workunit.client.0.smithi142.stdout:4/217: write d0/f14 [3431259,51501] 0 2024-03-29T04:44:56.921 INFO:tasks.workunit.client.0.smithi142.stdout:5/322: unlink df/d16/l51 0 2024-03-29T04:44:56.922 INFO:tasks.workunit.client.0.smithi142.stdout:5/323: write df/d16/d25/f59 [490649,91624] 0 2024-03-29T04:44:56.923 INFO:tasks.workunit.client.0.smithi142.stdout:5/324: readlink df/d16/d50/l5f 0 2024-03-29T04:44:56.923 INFO:tasks.workunit.client.0.smithi142.stdout:5/325: chown cc 53144856 1 2024-03-29T04:44:56.926 INFO:tasks.workunit.client.0.smithi142.stdout:3/232: symlink d3/d1f/d24/d47/l55 0 2024-03-29T04:44:56.926 INFO:tasks.workunit.client.0.smithi142.stdout:0/185: unlink d7/l19 0 2024-03-29T04:44:56.927 INFO:tasks.workunit.client.0.smithi142.stdout:0/186: chown d7/l12 12 1 2024-03-29T04:44:56.927 INFO:tasks.workunit.client.0.smithi142.stdout:1/264: symlink d8/db/l67 0 2024-03-29T04:44:56.928 INFO:tasks.workunit.client.0.smithi142.stdout:6/247: sync 2024-03-29T04:44:56.929 INFO:tasks.workunit.client.0.smithi142.stdout:0/187: truncate d7/d9/d22/d37/d2f/f3d 805166 0 2024-03-29T04:44:56.930 INFO:tasks.workunit.client.0.smithi142.stdout:9/196: getdents d0/d2/d6 0 2024-03-29T04:44:56.931 INFO:tasks.workunit.client.0.smithi142.stdout:7/235: creat d7/dc/de/d20/d3f/f41 x:0 0 0 2024-03-29T04:44:56.931 INFO:tasks.workunit.client.0.smithi142.stdout:7/236: stat d7/d18/d3d 0 2024-03-29T04:44:56.932 INFO:tasks.workunit.client.0.smithi142.stdout:5/326: readlink df/d16/d24/l4f 0 2024-03-29T04:44:56.933 INFO:tasks.workunit.client.0.smithi142.stdout:7/237: truncate d7/dc/de/f34 134565 0 2024-03-29T04:44:56.934 INFO:tasks.workunit.client.0.smithi142.stdout:2/190: fsync de/d11/f21 0 2024-03-29T04:44:56.935 INFO:tasks.workunit.client.0.smithi142.stdout:0/188: creat d7/d9/d22/f3e x:0 0 0 2024-03-29T04:44:56.936 INFO:tasks.workunit.client.0.smithi142.stdout:5/327: creat df/d16/d50/f61 x:0 0 0 2024-03-29T04:44:56.937 INFO:tasks.workunit.client.0.smithi142.stdout:7/238: stat d7/c8 0 2024-03-29T04:44:56.937 INFO:tasks.workunit.client.0.smithi142.stdout:5/328: chown df/d1a/d21/d55 79511 1 2024-03-29T04:44:56.938 INFO:tasks.workunit.client.0.smithi142.stdout:5/329: chown df/d16/d24/f56 679 1 2024-03-29T04:44:56.938 INFO:tasks.workunit.client.0.smithi142.stdout:4/218: rmdir d0/d12/d3f 0 2024-03-29T04:44:56.939 INFO:tasks.workunit.client.0.smithi142.stdout:6/248: symlink d1c/d45/l47 0 2024-03-29T04:44:56.940 INFO:tasks.workunit.client.0.smithi142.stdout:5/330: write df/f10 [631313,51123] 0 2024-03-29T04:44:56.941 INFO:tasks.workunit.client.0.smithi142.stdout:0/189: mknod d7/d9/d1e/c3f 0 2024-03-29T04:44:56.943 INFO:tasks.workunit.client.0.smithi142.stdout:7/239: dread d7/d18/d30/d33/f38 [4194304,4194304] 0 2024-03-29T04:44:56.944 INFO:tasks.workunit.client.0.smithi142.stdout:6/249: dread fc [8388608,4194304] 0 2024-03-29T04:44:56.945 INFO:tasks.workunit.client.0.smithi142.stdout:4/219: unlink d0/d9/l24 0 2024-03-29T04:44:56.946 INFO:tasks.workunit.client.0.smithi142.stdout:4/220: write d0/d9/d11/f23 [2530049,49735] 0 2024-03-29T04:44:56.947 INFO:tasks.workunit.client.0.smithi142.stdout:5/331: fdatasync df/f10 0 2024-03-29T04:44:56.947 INFO:tasks.workunit.client.0.smithi142.stdout:1/265: link d8/db/dc/d19/d1d/d15/d21/c45 d8/db/dc/d19/d1d/d1f/d4c/c68 0 2024-03-29T04:44:56.949 INFO:tasks.workunit.client.0.smithi142.stdout:3/233: getdents d3/d1f/d24/d47 0 2024-03-29T04:44:56.950 INFO:tasks.workunit.client.0.smithi142.stdout:0/190: symlink d7/d9/d22/d37/d2f/l40 0 2024-03-29T04:44:56.952 INFO:tasks.workunit.client.0.smithi142.stdout:7/240: write f1 [4938316,26460] 0 2024-03-29T04:44:56.953 INFO:tasks.workunit.client.0.smithi142.stdout:7/241: readlink d7/d27/d3a/l3b 0 2024-03-29T04:44:56.954 INFO:tasks.workunit.client.0.smithi142.stdout:4/221: creat d0/d9/f49 x:0 0 0 2024-03-29T04:44:56.955 INFO:tasks.workunit.client.0.smithi142.stdout:7/242: read d7/d18/f28 [2372821,76924] 0 2024-03-29T04:44:56.956 INFO:tasks.workunit.client.0.smithi142.stdout:5/332: rename df/d16/d50/l5f to df/d16/d25/l62 0 2024-03-29T04:44:56.957 INFO:tasks.workunit.client.0.smithi142.stdout:1/266: creat d8/db/dc/d19/d1d/d2d/d44/d53/f69 x:0 0 0 2024-03-29T04:44:56.958 INFO:tasks.workunit.client.0.smithi142.stdout:0/191: dwrite d7/f2b [0,4194304] 0 2024-03-29T04:44:56.961 INFO:tasks.workunit.client.0.smithi142.stdout:7/243: creat d7/d18/d30/d33/f42 x:0 0 0 2024-03-29T04:44:56.961 INFO:tasks.workunit.client.0.smithi142.stdout:6/250: link d1c/f40 d1c/d33/d46/f48 0 2024-03-29T04:44:56.962 INFO:tasks.workunit.client.0.smithi142.stdout:2/191: dwrite f9 [0,4194304] 0 2024-03-29T04:44:56.963 INFO:tasks.workunit.client.0.smithi142.stdout:2/192: chown l4 692336 1 2024-03-29T04:44:56.964 INFO:tasks.workunit.client.0.smithi142.stdout:0/192: write d7/d9/d22/d37/f38 [40264,75912] 0 2024-03-29T04:44:56.964 INFO:tasks.workunit.client.0.smithi142.stdout:4/222: link d0/d9/c2d d0/d12/d28/c4a 0 2024-03-29T04:44:56.966 INFO:tasks.workunit.client.0.smithi142.stdout:6/251: rename d1c/f2b to d1c/f49 0 2024-03-29T04:44:56.967 INFO:tasks.workunit.client.0.smithi142.stdout:2/193: creat de/d11/d1e/d22/d27/f44 x:0 0 0 2024-03-29T04:44:56.971 INFO:tasks.workunit.client.0.smithi142.stdout:6/252: readlink d1c/l32 0 2024-03-29T04:44:56.971 INFO:tasks.workunit.client.0.smithi142.stdout:7/244: creat d7/f43 x:0 0 0 2024-03-29T04:44:56.973 INFO:tasks.workunit.client.0.smithi142.stdout:2/194: rename de/d11/d1e/d22/d27/f44 to de/d11/d1e/d22/d27/f45 0 2024-03-29T04:44:56.976 INFO:tasks.workunit.client.0.smithi142.stdout:4/223: link d0/d9/d11/ff d0/d12/d28/d47/f4b 0 2024-03-29T04:44:56.976 INFO:tasks.workunit.client.0.smithi142.stdout:1/267: getdents d8/db/dc/d19/d1d/d1f/d4c 0 2024-03-29T04:44:56.979 INFO:tasks.workunit.client.0.smithi142.stdout:6/253: symlink d1c/l4a 0 2024-03-29T04:44:56.979 INFO:tasks.workunit.client.0.smithi142.stdout:7/245: mkdir d7/d27/d44 0 2024-03-29T04:44:56.979 INFO:tasks.workunit.client.0.smithi142.stdout:4/224: read d0/dc/d15/f18 [2049628,100187] 0 2024-03-29T04:44:56.980 INFO:tasks.workunit.client.0.smithi142.stdout:7/246: read f6 [430200,44019] 0 2024-03-29T04:44:56.980 INFO:tasks.workunit.client.0.smithi142.stdout:6/254: chown f4 3492 1 2024-03-29T04:44:56.981 INFO:tasks.workunit.client.0.smithi142.stdout:6/255: fdatasync ff 0 2024-03-29T04:44:56.981 INFO:tasks.workunit.client.0.smithi142.stdout:2/195: chown de/d11/c1c 94197 1 2024-03-29T04:44:56.982 INFO:tasks.workunit.client.0.smithi142.stdout:1/268: dread d8/db/f12 [0,4194304] 0 2024-03-29T04:44:56.983 INFO:tasks.workunit.client.0.smithi142.stdout:6/256: fsync f18 0 2024-03-29T04:44:56.984 INFO:tasks.workunit.client.0.smithi142.stdout:4/225: mkdir d0/d9/d11/d4c 0 2024-03-29T04:44:56.984 INFO:tasks.workunit.client.0.smithi142.stdout:6/257: truncate d1c/f39 1168213 0 2024-03-29T04:44:56.986 INFO:tasks.workunit.client.0.smithi142.stdout:2/196: chown de/d11/d23/f28 701203 1 2024-03-29T04:44:56.987 INFO:tasks.workunit.client.0.smithi142.stdout:7/247: symlink d7/dc/de/d20/l45 0 2024-03-29T04:44:56.988 INFO:tasks.workunit.client.0.smithi142.stdout:2/197: write de/f29 [3800111,101724] 0 2024-03-29T04:44:56.988 INFO:tasks.workunit.client.0.smithi142.stdout:1/269: creat d8/db/dc/d19/d1d/d2d/d44/d53/f6a x:0 0 0 2024-03-29T04:44:56.989 INFO:tasks.workunit.client.0.smithi142.stdout:6/258: truncate d1c/f34 689768 0 2024-03-29T04:44:56.989 INFO:tasks.workunit.client.0.smithi142.stdout:7/248: chown d7/dc/de/f26 74926545 1 2024-03-29T04:44:56.990 INFO:tasks.workunit.client.0.smithi142.stdout:7/249: dread - d7/d18/f2c zero size 2024-03-29T04:44:56.992 INFO:tasks.workunit.client.0.smithi142.stdout:1/270: chown d8/db/dc/d19/d1d/d15/d21/d31/d4d/c59 3 1 2024-03-29T04:44:56.996 INFO:tasks.workunit.client.0.smithi142.stdout:6/259: getdents d1c/d33 0 2024-03-29T04:44:56.997 INFO:tasks.workunit.client.0.smithi142.stdout:4/226: dread d0/d9/d11/f38 [0,4194304] 0 2024-03-29T04:44:56.998 INFO:tasks.workunit.client.0.smithi142.stdout:2/198: fsync de/d11/f21 0 2024-03-29T04:44:56.999 INFO:tasks.workunit.client.0.smithi142.stdout:1/271: symlink d8/db/dc/d19/d1d/d1f/d50/l6b 0 2024-03-29T04:44:57.000 INFO:tasks.workunit.client.0.smithi142.stdout:5/333: dwrite df/d1a/d21/d27/f2a [0,4194304] 0 2024-03-29T04:44:57.001 INFO:tasks.workunit.client.0.smithi142.stdout:6/260: symlink d1c/d42/l4b 0 2024-03-29T04:44:57.002 INFO:tasks.workunit.client.0.smithi142.stdout:2/199: chown de/d11/d1e/d22 3 1 2024-03-29T04:44:57.002 INFO:tasks.workunit.client.0.smithi142.stdout:4/227: creat d0/d9/f4d x:0 0 0 2024-03-29T04:44:57.002 INFO:tasks.workunit.client.0.smithi142.stdout:7/250: dread f6 [0,4194304] 0 2024-03-29T04:44:57.004 INFO:tasks.workunit.client.0.smithi142.stdout:3/234: dread d3/d29/d2b/f21 [0,4194304] 0 2024-03-29T04:44:57.004 INFO:tasks.workunit.client.0.smithi142.stdout:6/261: chown d1c/c35 1091 1 2024-03-29T04:44:57.004 INFO:tasks.workunit.client.0.smithi142.stdout:4/228: chown d0/d45 1 1 2024-03-29T04:44:57.006 INFO:tasks.workunit.client.0.smithi142.stdout:0/193: dwrite d7/f21 [0,4194304] 0 2024-03-29T04:44:57.008 INFO:tasks.workunit.client.0.smithi142.stdout:7/251: creat d7/d18/d30/f46 x:0 0 0 2024-03-29T04:44:57.010 INFO:tasks.workunit.client.0.smithi142.stdout:0/194: stat d7/d9/d22/d37/d2f/l40 0 2024-03-29T04:44:57.010 INFO:tasks.workunit.client.0.smithi142.stdout:8/307: sync 2024-03-29T04:44:57.011 INFO:tasks.workunit.client.0.smithi142.stdout:5/334: write df/d16/f26 [9191343,95528] 0 2024-03-29T04:44:57.013 INFO:tasks.workunit.client.0.smithi142.stdout:4/229: symlink d0/d45/l4e 0 2024-03-29T04:44:57.014 INFO:tasks.workunit.client.0.smithi142.stdout:0/195: stat d7/d9/d22/d37/d2f/c34 0 2024-03-29T04:44:57.016 INFO:tasks.workunit.client.0.smithi142.stdout:1/272: dwrite d8/f2b [0,4194304] 0 2024-03-29T04:44:57.019 INFO:tasks.workunit.client.0.smithi142.stdout:6/262: getdents d1c/d33/d46 0 2024-03-29T04:44:57.019 INFO:tasks.workunit.client.0.smithi142.stdout:0/196: fdatasync d7/d9/d22/f1b 0 2024-03-29T04:44:57.020 INFO:tasks.workunit.client.0.smithi142.stdout:5/335: mknod df/d1a/d21/d27/d45/c63 0 2024-03-29T04:44:57.022 INFO:tasks.workunit.client.0.smithi142.stdout:2/200: dwrite de/d11/d1e/d22/f26 [0,4194304] 0 2024-03-29T04:44:57.024 INFO:tasks.workunit.client.0.smithi142.stdout:0/197: chown d7/d9/cb 2780687 1 2024-03-29T04:44:57.027 INFO:tasks.workunit.client.0.smithi142.stdout:7/252: dwrite f6 [0,4194304] 0 2024-03-29T04:44:57.029 INFO:tasks.workunit.client.0.smithi142.stdout:2/201: chown de/d11/d1e/d22/c38 3912 1 2024-03-29T04:44:57.031 INFO:tasks.workunit.client.0.smithi142.stdout:0/198: creat d7/d13/f41 x:0 0 0 2024-03-29T04:44:57.031 INFO:tasks.workunit.client.0.smithi142.stdout:2/202: read de/f29 [2646666,81910] 0 2024-03-29T04:44:57.033 INFO:tasks.workunit.client.0.smithi142.stdout:0/199: stat d7/d9/d22/f3a 0 2024-03-29T04:44:57.034 INFO:tasks.workunit.client.0.smithi142.stdout:2/203: read f7 [922104,6607] 0 2024-03-29T04:44:57.035 INFO:tasks.workunit.client.0.smithi142.stdout:5/336: dread df/d1a/d21/d27/f2a [0,4194304] 0 2024-03-29T04:44:57.036 INFO:tasks.workunit.client.0.smithi142.stdout:3/235: dwrite d3/d29/d2d/f48 [0,4194304] 0 2024-03-29T04:44:57.039 INFO:tasks.workunit.client.0.smithi142.stdout:5/337: rename df/d16/d24/d2e/d47/d54 to df/d16/d24/d2e/d47/d54/d64 22 2024-03-29T04:44:57.045 INFO:tasks.workunit.client.0.smithi142.stdout:6/263: dread d1c/f40 [0,4194304] 0 2024-03-29T04:44:57.046 INFO:tasks.workunit.client.0.smithi142.stdout:9/197: sync 2024-03-29T04:44:57.046 INFO:tasks.workunit.client.0.smithi142.stdout:9/198: write d0/f38 [3549066,35888] 0 2024-03-29T04:44:57.047 INFO:tasks.workunit.client.0.smithi142.stdout:9/199: write d0/d2/d6/d31/f3a [861527,59370] 0 2024-03-29T04:44:57.084 INFO:tasks.workunit.client.0.smithi142.stdout:2/204: creat de/d11/d23/f46 x:0 0 0 2024-03-29T04:44:57.085 INFO:tasks.workunit.client.0.smithi142.stdout:3/236: mkdir d3/d29/d2d/d56 0 2024-03-29T04:44:57.086 INFO:tasks.workunit.client.0.smithi142.stdout:8/308: dwrite d5/d1d/d2f/d5f/f4b [0,4194304] 0 2024-03-29T04:44:57.087 INFO:tasks.workunit.client.0.smithi142.stdout:6/264: creat d1c/d42/f4c x:0 0 0 2024-03-29T04:44:57.091 INFO:tasks.workunit.client.0.smithi142.stdout:9/200: mknod d0/d2/d39/c44 0 2024-03-29T04:44:57.091 INFO:tasks.workunit.client.0.smithi142.stdout:8/309: write d5/d10/d2d/f4a [38264,2227] 0 2024-03-29T04:44:57.094 INFO:tasks.workunit.client.0.smithi142.stdout:4/230: dwrite d0/d12/d28/f3a [0,4194304] 0 2024-03-29T04:44:57.096 INFO:tasks.workunit.client.0.smithi142.stdout:2/205: dread fd [0,4194304] 0 2024-03-29T04:44:57.097 INFO:tasks.workunit.client.0.smithi142.stdout:5/338: rename df/d1a/d21/l52 to df/d16/l65 0 2024-03-29T04:44:57.099 INFO:tasks.workunit.client.0.smithi142.stdout:7/253: getdents d7/d27 0 2024-03-29T04:44:57.100 INFO:tasks.workunit.client.0.smithi142.stdout:5/339: readlink df/d16/l4e 0 2024-03-29T04:44:57.100 INFO:tasks.workunit.client.0.smithi142.stdout:3/237: write d3/d1f/d34/d6/f14 [1114509,121467] 0 2024-03-29T04:44:57.101 INFO:tasks.workunit.client.0.smithi142.stdout:1/273: truncate d8/db/dc/d19/d1d/d15/f16 181637 0 2024-03-29T04:44:57.104 INFO:tasks.workunit.client.0.smithi142.stdout:4/231: symlink d0/d12/l4f 0 2024-03-29T04:44:57.104 INFO:tasks.workunit.client.0.smithi142.stdout:9/201: dread d0/f38 [0,4194304] 0 2024-03-29T04:44:57.104 INFO:tasks.workunit.client.0.smithi142.stdout:2/206: fdatasync fd 0 2024-03-29T04:44:57.105 INFO:tasks.workunit.client.0.smithi142.stdout:9/202: readlink d0/lb 0 2024-03-29T04:44:57.105 INFO:tasks.workunit.client.0.smithi142.stdout:0/200: getdents d7/d9/d22/d37/d2f 0 2024-03-29T04:44:57.106 INFO:tasks.workunit.client.0.smithi142.stdout:7/254: creat d7/dc/de/d20/f47 x:0 0 0 2024-03-29T04:44:57.107 INFO:tasks.workunit.client.0.smithi142.stdout:7/255: fdatasync d7/d18/f2c 0 2024-03-29T04:44:57.107 INFO:tasks.workunit.client.0.smithi142.stdout:8/310: symlink d5/d1d/d2f/d5f/d48/d52/l6a 0 2024-03-29T04:44:57.107 INFO:tasks.workunit.client.0.smithi142.stdout:0/201: write d7/d9/d22/f23 [760592,119378] 0 2024-03-29T04:44:57.108 INFO:tasks.workunit.client.0.smithi142.stdout:5/340: dread df/d16/f1e [0,4194304] 0 2024-03-29T04:44:57.108 INFO:tasks.workunit.client.0.smithi142.stdout:3/238: creat d3/d1f/d24/d47/f57 x:0 0 0 2024-03-29T04:44:57.108 INFO:tasks.workunit.client.0.smithi142.stdout:7/256: chown d7/f19 7601 1 2024-03-29T04:44:57.109 INFO:tasks.workunit.client.0.smithi142.stdout:0/202: read d7/f2b [1275196,31374] 0 2024-03-29T04:44:57.110 INFO:tasks.workunit.client.0.smithi142.stdout:4/232: rename d0/f7 to d0/d12/f50 0 2024-03-29T04:44:57.110 INFO:tasks.workunit.client.0.smithi142.stdout:2/207: write de/d11/f37 [536559,34463] 0 2024-03-29T04:44:57.111 INFO:tasks.workunit.client.0.smithi142.stdout:1/274: fdatasync d8/db/dc/d19/d1d/d15/d21/d31/f61 0 2024-03-29T04:44:57.111 INFO:tasks.workunit.client.0.smithi142.stdout:6/265: link d1c/c2c d1c/d42/c4d 0 2024-03-29T04:44:57.112 INFO:tasks.workunit.client.0.smithi142.stdout:8/311: creat d5/d1d/d24/f6b x:0 0 0 2024-03-29T04:44:57.112 INFO:tasks.workunit.client.0.smithi142.stdout:6/266: readlink l13 0 2024-03-29T04:44:57.114 INFO:tasks.workunit.client.0.smithi142.stdout:9/203: mknod d0/c45 0 2024-03-29T04:44:57.115 INFO:tasks.workunit.client.0.smithi142.stdout:7/257: creat d7/d27/f48 x:0 0 0 2024-03-29T04:44:57.115 INFO:tasks.workunit.client.0.smithi142.stdout:5/341: mkdir df/d1a/d21/d55/d66 0 2024-03-29T04:44:57.116 INFO:tasks.workunit.client.0.smithi142.stdout:6/267: write d1c/f39 [715767,34796] 0 2024-03-29T04:44:57.116 INFO:tasks.workunit.client.0.smithi142.stdout:4/233: rmdir d0 39 2024-03-29T04:44:57.119 INFO:tasks.workunit.client.0.smithi142.stdout:3/239: truncate d3/d29/d2b/f21 2405179 0 2024-03-29T04:44:57.120 INFO:tasks.workunit.client.0.smithi142.stdout:2/208: symlink de/d11/d23/l47 0 2024-03-29T04:44:57.120 INFO:tasks.workunit.client.0.smithi142.stdout:6/268: read - d1c/f49 zero size 2024-03-29T04:44:57.120 INFO:tasks.workunit.client.0.smithi142.stdout:9/204: write d0/d2/fe [2922325,64741] 0 2024-03-29T04:44:57.123 INFO:tasks.workunit.client.0.smithi142.stdout:8/312: creat d5/d1d/d24/f6c x:0 0 0 2024-03-29T04:44:57.124 INFO:tasks.workunit.client.0.smithi142.stdout:0/203: dwrite d7/d9/d22/f23 [0,4194304] 0 2024-03-29T04:44:57.125 INFO:tasks.workunit.client.0.smithi142.stdout:8/313: readlink d5/d1d/d2f/d5f/d48/d52/l6a 0 2024-03-29T04:44:57.126 INFO:tasks.workunit.client.0.smithi142.stdout:4/234: chown d0/d9 66930291 1 2024-03-29T04:44:57.126 INFO:tasks.workunit.client.0.smithi142.stdout:8/314: write d5/f30 [706022,759] 0 2024-03-29T04:44:57.130 INFO:tasks.workunit.client.0.smithi142.stdout:6/269: creat d1c/d45/f4e x:0 0 0 2024-03-29T04:44:57.130 INFO:tasks.workunit.client.0.smithi142.stdout:6/270: write f1a [903554,41675] 0 2024-03-29T04:44:57.133 INFO:tasks.workunit.client.0.smithi142.stdout:1/275: rename d8/db/dc/d19/d1d/d1f/l3b to d8/db/dc/d19/d1d/d2d/d44/l6c 0 2024-03-29T04:44:57.136 INFO:tasks.workunit.client.0.smithi142.stdout:0/204: mkdir d7/d9/d22/d37/d2f/d42 0 2024-03-29T04:44:57.137 INFO:tasks.workunit.client.0.smithi142.stdout:0/205: read d7/d9/d22/d37/f38 [477493,75027] 0 2024-03-29T04:44:57.138 INFO:tasks.workunit.client.0.smithi142.stdout:2/209: symlink de/l48 0 2024-03-29T04:44:57.141 INFO:tasks.workunit.client.0.smithi142.stdout:6/271: unlink f17 0 2024-03-29T04:44:57.144 INFO:tasks.workunit.client.0.smithi142.stdout:7/258: creat d7/dc/f49 x:0 0 0 2024-03-29T04:44:57.144 INFO:tasks.workunit.client.0.smithi142.stdout:2/210: dread de/d11/f37 [0,4194304] 0 2024-03-29T04:44:57.146 INFO:tasks.workunit.client.0.smithi142.stdout:8/315: mknod d5/d1d/d2f/d5f/c6d 0 2024-03-29T04:44:57.146 INFO:tasks.workunit.client.0.smithi142.stdout:4/235: dread d0/d12/f50 [0,4194304] 0 2024-03-29T04:44:57.147 INFO:tasks.workunit.client.0.smithi142.stdout:3/240: write d3/d1f/d34/f35 [1006680,13753] 0 2024-03-29T04:44:57.149 INFO:tasks.workunit.client.0.smithi142.stdout:0/206: dwrite d7/d9/fa [0,4194304] 0 2024-03-29T04:44:57.149 INFO:tasks.workunit.client.0.smithi142.stdout:8/316: chown d5/d1d/l55 4782 1 2024-03-29T04:44:57.149 INFO:tasks.workunit.client.0.smithi142.stdout:1/276: link d8/db/f25 d8/db/dc/d19/d1d/d1f/f6d 0 2024-03-29T04:44:57.150 INFO:tasks.workunit.client.0.smithi142.stdout:9/205: dread d0/d2/f2c [0,4194304] 0 2024-03-29T04:44:57.151 INFO:tasks.workunit.client.0.smithi142.stdout:8/317: stat d5/d10/d2d/f56 0 2024-03-29T04:44:57.151 INFO:tasks.workunit.client.0.smithi142.stdout:0/207: fdatasync d7/d9/d1e/f26 0 2024-03-29T04:44:57.154 INFO:tasks.workunit.client.0.smithi142.stdout:4/236: symlink d0/l51 0 2024-03-29T04:44:57.155 INFO:tasks.workunit.client.0.smithi142.stdout:0/208: read d7/d9/d22/f1b [349216,37609] 0 2024-03-29T04:44:57.156 INFO:tasks.workunit.client.0.smithi142.stdout:0/209: stat d7/d9/d22 0 2024-03-29T04:44:57.160 INFO:tasks.workunit.client.0.smithi142.stdout:9/206: mknod d0/d2/d6/d35/d34/c46 0 2024-03-29T04:44:57.160 INFO:tasks.workunit.client.0.smithi142.stdout:7/259: dread d7/d27/f29 [0,4194304] 0 2024-03-29T04:44:57.163 INFO:tasks.workunit.client.0.smithi142.stdout:2/211: symlink de/d11/d1e/d22/l49 0 2024-03-29T04:44:57.164 INFO:tasks.workunit.client.0.smithi142.stdout:1/277: rename d8/f54 to d8/db/dc/d19/d1d/d15/d21/d31/d4d/f6e 0 2024-03-29T04:44:57.165 INFO:tasks.workunit.client.0.smithi142.stdout:4/237: mknod d0/d45/c52 0 2024-03-29T04:44:57.167 INFO:tasks.workunit.client.0.smithi142.stdout:0/210: creat d7/d9/d22/d37/f43 x:0 0 0 2024-03-29T04:44:57.168 INFO:tasks.workunit.client.0.smithi142.stdout:9/207: mknod d0/d2/d6/d35/c47 0 2024-03-29T04:44:57.170 INFO:tasks.workunit.client.0.smithi142.stdout:0/211: write d7/d9/f3c [114359,12703] 0 2024-03-29T04:44:57.171 INFO:tasks.workunit.client.0.smithi142.stdout:7/260: dwrite d7/dc/de/d20/f47 [0,4194304] 0 2024-03-29T04:44:57.174 INFO:tasks.workunit.client.0.smithi142.stdout:0/212: write d7/d9/f30 [58837,129665] 0 2024-03-29T04:44:57.175 INFO:tasks.workunit.client.0.smithi142.stdout:2/212: mkdir de/d11/d17/d4a 0 2024-03-29T04:44:57.176 INFO:tasks.workunit.client.0.smithi142.stdout:4/238: dwrite d0/f21 [0,4194304] 0 2024-03-29T04:44:57.177 INFO:tasks.workunit.client.0.smithi142.stdout:9/208: creat d0/d2/d6/d27/f48 x:0 0 0 2024-03-29T04:44:57.178 INFO:tasks.workunit.client.0.smithi142.stdout:6/272: truncate d1c/f25 2421548 0 2024-03-29T04:44:57.179 INFO:tasks.workunit.client.0.smithi142.stdout:7/261: rmdir d7/d18/d30 39 2024-03-29T04:44:57.180 INFO:tasks.workunit.client.0.smithi142.stdout:1/278: dwrite d8/db/dc/d19/d1d/d15/d21/f35 [0,4194304] 0 2024-03-29T04:44:57.181 INFO:tasks.workunit.client.0.smithi142.stdout:8/318: write d5/d2e/f65 [1006366,84132] 0 2024-03-29T04:44:57.182 INFO:tasks.workunit.client.0.smithi142.stdout:0/213: rename d7/d9/l1f to d7/d9/d1e/l44 0 2024-03-29T04:44:57.182 INFO:tasks.workunit.client.0.smithi142.stdout:6/273: write d1c/d42/f4c [91354,5799] 0 2024-03-29T04:44:57.182 INFO:tasks.workunit.client.0.smithi142.stdout:2/213: chown de/d11/d1e/d22/l2e 34 1 2024-03-29T04:44:57.182 INFO:tasks.workunit.client.0.smithi142.stdout:9/209: dread - d0/d2/d6/d16/f37 zero size 2024-03-29T04:44:57.184 INFO:tasks.workunit.client.0.smithi142.stdout:7/262: chown d7/d18/f35 794 1 2024-03-29T04:44:57.186 INFO:tasks.workunit.client.0.smithi142.stdout:1/279: write f7 [2576095,123904] 0 2024-03-29T04:44:57.188 INFO:tasks.workunit.client.0.smithi142.stdout:1/280: write f7 [1256977,44252] 0 2024-03-29T04:44:57.192 INFO:tasks.workunit.client.0.smithi142.stdout:0/214: mknod d7/c45 0 2024-03-29T04:44:57.193 INFO:tasks.workunit.client.0.smithi142.stdout:9/210: creat d0/d2/d6/d31/f49 x:0 0 0 2024-03-29T04:44:57.194 INFO:tasks.workunit.client.0.smithi142.stdout:8/319: mknod d5/d1d/d2f/d5f/c6e 0 2024-03-29T04:44:57.194 INFO:tasks.workunit.client.0.smithi142.stdout:0/215: chown d7/d9/d22/d37/c33 2 1 2024-03-29T04:44:57.195 INFO:tasks.workunit.client.0.smithi142.stdout:2/214: link de/f42 de/d11/d1e/d22/d27/f4b 0 2024-03-29T04:44:57.195 INFO:tasks.workunit.client.0.smithi142.stdout:7/263: dwrite d7/dc/de/d20/f21 [0,4194304] 0 2024-03-29T04:44:57.197 INFO:tasks.workunit.client.0.smithi142.stdout:8/320: symlink d5/d2e/l6f 0 2024-03-29T04:44:57.200 INFO:tasks.workunit.client.0.smithi142.stdout:7/264: rmdir d7/dc/de 39 2024-03-29T04:44:57.201 INFO:tasks.workunit.client.0.smithi142.stdout:1/281: link c2 d8/db/dc/d19/c6f 0 2024-03-29T04:44:57.204 INFO:tasks.workunit.client.0.smithi142.stdout:1/282: chown d8/db/dc/d19/d1d/d2d/d44/d53/f69 3 1 2024-03-29T04:44:57.204 INFO:tasks.workunit.client.0.smithi142.stdout:9/211: dwrite d0/d2/fe [0,4194304] 0 2024-03-29T04:44:57.204 INFO:tasks.workunit.client.0.smithi142.stdout:8/321: mkdir d5/d10/d3c/d70 0 2024-03-29T04:44:57.208 INFO:tasks.workunit.client.0.smithi142.stdout:0/216: rename d7/d9/d22/f23 to d7/d9/d22/d37/d2f/d42/f46 0 2024-03-29T04:44:57.208 INFO:tasks.workunit.client.0.smithi142.stdout:6/274: read d1c/f25 [270317,22979] 0 2024-03-29T04:44:57.209 INFO:tasks.workunit.client.0.smithi142.stdout:8/322: read d5/f22 [2171306,43538] 0 2024-03-29T04:44:57.210 INFO:tasks.workunit.client.0.smithi142.stdout:3/241: write d3/d29/d2b/f21 [2882620,31550] 0 2024-03-29T04:44:57.214 INFO:tasks.workunit.client.0.smithi142.stdout:7/265: creat d7/d18/d30/f4a x:0 0 0 2024-03-29T04:44:57.216 INFO:tasks.workunit.client.0.smithi142.stdout:3/242: chown d3/d29/d2b/d40/f4c 82906067 1 2024-03-29T04:44:57.217 INFO:tasks.workunit.client.0.smithi142.stdout:9/212: dwrite d0/d2/d6/d16/f37 [0,4194304] 0 2024-03-29T04:44:57.217 INFO:tasks.workunit.client.0.smithi142.stdout:6/275: truncate d1c/d33/d46/f48 2746509 0 2024-03-29T04:44:57.221 INFO:tasks.workunit.client.0.smithi142.stdout:0/217: write d7/d9/d22/d37/d2f/d42/f46 [2426150,119124] 0 2024-03-29T04:44:57.222 INFO:tasks.workunit.client.0.smithi142.stdout:9/213: symlink d0/d12/l4a 0 2024-03-29T04:44:57.223 INFO:tasks.workunit.client.0.smithi142.stdout:0/218: fdatasync d7/d9/f3c 0 2024-03-29T04:44:57.224 INFO:tasks.workunit.client.0.smithi142.stdout:8/323: dwrite d5/d10/d3c/f3e [0,4194304] 0 2024-03-29T04:44:57.224 INFO:tasks.workunit.client.0.smithi142.stdout:6/276: creat d1c/d33/f4f x:0 0 0 2024-03-29T04:44:57.226 INFO:tasks.workunit.client.0.smithi142.stdout:9/214: dread - d0/d2/d6/d31/f49 zero size 2024-03-29T04:44:57.226 INFO:tasks.workunit.client.0.smithi142.stdout:3/243: creat d3/d54/f58 x:0 0 0 2024-03-29T04:44:57.228 INFO:tasks.workunit.client.0.smithi142.stdout:7/266: rename d7/d18/d3d to d7/dc/d4b 0 2024-03-29T04:44:57.229 INFO:tasks.workunit.client.0.smithi142.stdout:0/219: write d7/d9/f10 [383462,77800] 0 2024-03-29T04:44:57.230 INFO:tasks.workunit.client.0.smithi142.stdout:6/277: mknod d1c/d45/c50 0 2024-03-29T04:44:57.234 INFO:tasks.workunit.client.0.smithi142.stdout:0/220: mknod d7/d9/d22/c47 0 2024-03-29T04:44:57.235 INFO:tasks.workunit.client.0.smithi142.stdout:7/267: chown d7/dc/de/d20/l45 0 1 2024-03-29T04:44:57.235 INFO:tasks.workunit.client.0.smithi142.stdout:6/278: mkdir d1c/d45/d51 0 2024-03-29T04:44:57.235 INFO:tasks.workunit.client.0.smithi142.stdout:1/283: dread d8/f9 [0,4194304] 0 2024-03-29T04:44:57.237 INFO:tasks.workunit.client.0.smithi142.stdout:0/221: chown d7/c17 2773 1 2024-03-29T04:44:57.237 INFO:tasks.workunit.client.0.smithi142.stdout:7/268: fdatasync d7/dc/de/f16 0 2024-03-29T04:44:57.238 INFO:tasks.workunit.client.0.smithi142.stdout:1/284: creat d8/f70 x:0 0 0 2024-03-29T04:44:57.238 INFO:tasks.workunit.client.0.smithi142.stdout:8/324: dwrite d5/d10/d2d/f57 [0,4194304] 0 2024-03-29T04:44:57.240 INFO:tasks.workunit.client.0.smithi142.stdout:8/325: fsync d5/f30 0 2024-03-29T04:44:57.241 INFO:tasks.workunit.client.0.smithi142.stdout:1/285: mknod d8/db/c71 0 2024-03-29T04:44:57.243 INFO:tasks.workunit.client.0.smithi142.stdout:7/269: mkdir d7/d27/d44/d4c 0 2024-03-29T04:44:57.243 INFO:tasks.workunit.client.0.smithi142.stdout:7/270: write d7/f43 [194252,79099] 0 2024-03-29T04:44:57.244 INFO:tasks.workunit.client.0.smithi142.stdout:0/222: rename d7/d13/f35 to d7/d9/d22/f48 0 2024-03-29T04:44:57.246 INFO:tasks.workunit.client.0.smithi142.stdout:7/271: fsync d7/d18/d30/f46 0 2024-03-29T04:44:57.246 INFO:tasks.workunit.client.0.smithi142.stdout:8/326: dread d5/f30 [0,4194304] 0 2024-03-29T04:44:57.248 INFO:tasks.workunit.client.0.smithi142.stdout:0/223: rmdir d7/d9/d22/d37/d2f/d42 39 2024-03-29T04:44:57.250 INFO:tasks.workunit.client.0.smithi142.stdout:8/327: write d5/d1d/d2f/f54 [3969206,75701] 0 2024-03-29T04:44:57.250 INFO:tasks.workunit.client.0.smithi142.stdout:9/215: truncate d0/d2/d6/d31/f3a 859914 0 2024-03-29T04:44:57.251 INFO:tasks.workunit.client.0.smithi142.stdout:7/272: creat d7/d18/d30/d33/f4d x:0 0 0 2024-03-29T04:44:57.251 INFO:tasks.workunit.client.0.smithi142.stdout:1/286: dwrite d8/db/dc/d19/d1d/d2d/d44/d53/f5f [0,4194304] 0 2024-03-29T04:44:57.252 INFO:tasks.workunit.client.0.smithi142.stdout:0/224: truncate d7/d13/f28 1059064 0 2024-03-29T04:44:57.254 INFO:tasks.workunit.client.0.smithi142.stdout:0/225: write d7/d9/d22/d37/d2f/f3d [1003403,3823] 0 2024-03-29T04:44:57.255 INFO:tasks.workunit.client.0.smithi142.stdout:1/287: write d8/db/dc/d19/d1d/d2d/d44/d53/f6a [74343,124600] 0 2024-03-29T04:44:57.255 INFO:tasks.workunit.client.0.smithi142.stdout:7/273: creat d7/dc/de/d20/f4e x:0 0 0 2024-03-29T04:44:57.256 INFO:tasks.workunit.client.0.smithi142.stdout:9/216: creat d0/f4b x:0 0 0 2024-03-29T04:44:57.257 INFO:tasks.workunit.client.0.smithi142.stdout:7/274: dread - d7/dc/de/d20/d3f/f41 zero size 2024-03-29T04:44:57.260 INFO:tasks.workunit.client.0.smithi142.stdout:7/275: creat d7/dc/d4b/f4f x:0 0 0 2024-03-29T04:44:57.260 INFO:tasks.workunit.client.0.smithi142.stdout:7/276: write f6 [3122893,110688] 0 2024-03-29T04:44:57.261 INFO:tasks.workunit.client.0.smithi142.stdout:1/288: truncate d8/db/dc/d19/d1d/d15/f47 1014720 0 2024-03-29T04:44:57.261 INFO:tasks.workunit.client.0.smithi142.stdout:0/226: rmdir d7/d9/d22/d37/d2f/d42 39 2024-03-29T04:44:57.262 INFO:tasks.workunit.client.0.smithi142.stdout:0/227: fsync d7/d9/d22/f1b 0 2024-03-29T04:44:57.264 INFO:tasks.workunit.client.0.smithi142.stdout:7/277: link d7/dc/de/d20/f2b d7/dc/de/d20/d3f/f50 0 2024-03-29T04:44:57.268 INFO:tasks.workunit.client.0.smithi142.stdout:0/228: dwrite d7/d9/d22/f3e [0,4194304] 0 2024-03-29T04:44:57.271 INFO:tasks.workunit.client.0.smithi142.stdout:0/229: creat d7/d9/f49 x:0 0 0 2024-03-29T04:44:57.294 INFO:tasks.workunit.client.0.smithi142.stdout:7/278: dread d7/fa [0,4194304] 0 2024-03-29T04:44:57.296 INFO:tasks.workunit.client.0.smithi142.stdout:7/279: creat d7/dc/de/d20/f51 x:0 0 0 2024-03-29T04:44:57.299 INFO:tasks.workunit.client.0.smithi142.stdout:7/280: dread d7/f1a [0,4194304] 0 2024-03-29T04:44:57.303 INFO:tasks.workunit.client.0.smithi142.stdout:8/328: dwrite d5/d10/f32 [0,4194304] 0 2024-03-29T04:44:57.304 INFO:tasks.workunit.client.0.smithi142.stdout:7/281: symlink d7/d18/l52 0 2024-03-29T04:44:57.306 INFO:tasks.workunit.client.0.smithi142.stdout:1/289: truncate d8/db/dc/d19/d1d/d15/f17 2726166 0 2024-03-29T04:44:57.306 INFO:tasks.workunit.client.0.smithi142.stdout:7/282: write d7/d18/f2c [444824,120291] 0 2024-03-29T04:44:57.345 INFO:tasks.workunit.client.0.smithi142.stdout:0/230: write d7/f2b [3185884,88083] 0 2024-03-29T04:44:57.346 INFO:tasks.workunit.client.0.smithi142.stdout:0/231: write d7/d9/ff [2678338,47562] 0 2024-03-29T04:44:57.348 INFO:tasks.workunit.client.0.smithi142.stdout:1/290: unlink d8/db/dc/d19/d1d/d1f/d50/l6b 0 2024-03-29T04:44:57.350 INFO:tasks.workunit.client.0.smithi142.stdout:1/291: unlink d8/db/f12 0 2024-03-29T04:44:57.353 INFO:tasks.workunit.client.0.smithi142.stdout:1/292: truncate d8/db/dc/d19/d1d/d15/d21/f32 659074 0 2024-03-29T04:44:57.357 INFO:tasks.workunit.client.0.smithi142.stdout:7/283: dread d7/d18/f1f [0,4194304] 0 2024-03-29T04:44:57.368 INFO:tasks.workunit.client.0.smithi142.stdout:1/293: symlink d8/db/dc/d19/d1d/d15/d21/l72 0 2024-03-29T04:44:57.369 INFO:tasks.workunit.client.0.smithi142.stdout:1/294: chown d8/db/dc/d19/d1d/d15/d21/d31/f61 33143433 1 2024-03-29T04:44:57.370 INFO:tasks.workunit.client.0.smithi142.stdout:1/295: dread - d8/db/dc/d19/f57 zero size 2024-03-29T04:44:57.370 INFO:tasks.workunit.client.0.smithi142.stdout:0/232: write d7/d9/d22/d37/f38 [125801,3246] 0 2024-03-29T04:44:57.371 INFO:tasks.workunit.client.0.smithi142.stdout:0/233: dread - d7/d13/f41 zero size 2024-03-29T04:44:57.372 INFO:tasks.workunit.client.0.smithi142.stdout:7/284: dread d7/d18/d30/d33/f38 [0,4194304] 0 2024-03-29T04:44:57.374 INFO:tasks.workunit.client.0.smithi142.stdout:0/234: mkdir d7/d13/d4a 0 2024-03-29T04:44:57.378 INFO:tasks.workunit.client.0.smithi142.stdout:1/296: dwrite d8/db/dc/d19/d1d/d15/f27 [0,4194304] 0 2024-03-29T04:44:57.378 INFO:tasks.workunit.client.0.smithi142.stdout:0/235: rename d7/c45 to d7/d9/d22/c4b 0 2024-03-29T04:44:57.381 INFO:tasks.workunit.client.0.smithi142.stdout:7/285: dwrite d7/d18/d30/d33/f38 [8388608,4194304] 0 2024-03-29T04:44:57.382 INFO:tasks.workunit.client.0.smithi142.stdout:1/297: creat d8/db/d5e/f73 x:0 0 0 2024-03-29T04:44:57.384 INFO:tasks.workunit.client.0.smithi142.stdout:1/298: dread - d8/db/dc/d19/d1d/f48 zero size 2024-03-29T04:44:57.388 INFO:tasks.workunit.client.0.smithi142.stdout:7/286: rename d7/dc/de/d20/l45 to d7/d27/d44/d4c/l53 0 2024-03-29T04:44:57.389 INFO:tasks.workunit.client.0.smithi142.stdout:7/287: readlink d7/l2a 0 2024-03-29T04:44:57.389 INFO:tasks.workunit.client.0.smithi142.stdout:7/288: readlink d7/l13 0 2024-03-29T04:44:57.395 INFO:tasks.workunit.client.0.smithi142.stdout:1/299: write d8/db/dc/d19/d1d/d1f/f6d [1188710,37730] 0 2024-03-29T04:44:57.400 INFO:tasks.workunit.client.0.smithi142.stdout:1/300: rename d8/db/dc/d19/d1d/d1f/f30 to d8/db/dc/d19/d1d/d15/d21/d31/f74 0 2024-03-29T04:44:57.401 INFO:tasks.workunit.client.0.smithi142.stdout:7/289: dwrite d7/f1a [0,4194304] 0 2024-03-29T04:44:57.402 INFO:tasks.workunit.client.0.smithi142.stdout:1/301: dread - d8/db/d5e/f73 zero size 2024-03-29T04:44:57.403 INFO:tasks.workunit.client.0.smithi142.stdout:7/290: chown d7/dc/de/d20/d3f/f41 221310953 1 2024-03-29T04:44:57.407 INFO:tasks.workunit.client.0.smithi142.stdout:1/302: dwrite d8/f2b [0,4194304] 0 2024-03-29T04:44:57.409 INFO:tasks.workunit.client.0.smithi142.stdout:1/303: fdatasync d8/db/dc/d19/d1d/d15/d21/d31/d4d/d56/f58 0 2024-03-29T04:44:57.411 INFO:tasks.workunit.client.0.smithi142.stdout:5/342: sync 2024-03-29T04:44:57.411 INFO:tasks.workunit.client.0.smithi142.stdout:5/343: readlink df/d16/l4e 0 2024-03-29T04:44:57.412 INFO:tasks.workunit.client.0.smithi142.stdout:1/304: creat d8/db/dc/d19/d1d/d15/d21/d5a/f75 x:0 0 0 2024-03-29T04:44:57.413 INFO:tasks.workunit.client.0.smithi142.stdout:1/305: fdatasync d8/db/f25 0 2024-03-29T04:44:57.414 INFO:tasks.workunit.client.0.smithi142.stdout:5/344: creat df/d29/f67 x:0 0 0 2024-03-29T04:44:57.416 INFO:tasks.workunit.client.0.smithi142.stdout:5/345: creat df/d16/d24/d3d/f68 x:0 0 0 2024-03-29T04:44:57.432 INFO:tasks.workunit.client.0.smithi142.stdout:7/291: dread f1 [0,4194304] 0 2024-03-29T04:44:57.432 INFO:tasks.workunit.client.0.smithi142.stdout:7/292: read d7/fa [1657456,72899] 0 2024-03-29T04:44:57.433 INFO:tasks.workunit.client.0.smithi142.stdout:5/346: dread df/d1a/f3f [0,4194304] 0 2024-03-29T04:44:57.436 INFO:tasks.workunit.client.0.smithi142.stdout:7/293: getdents d7/d27/d44/d4c 0 2024-03-29T04:44:57.437 INFO:tasks.workunit.client.0.smithi142.stdout:5/347: mknod df/d29/d5d/c69 0 2024-03-29T04:44:57.437 INFO:tasks.workunit.client.0.smithi142.stdout:7/294: write d7/dc/de/f34 [881561,39449] 0 2024-03-29T04:44:57.437 INFO:tasks.workunit.client.0.smithi142.stdout:7/295: read - d7/d18/f35 zero size 2024-03-29T04:44:57.439 INFO:tasks.workunit.client.0.smithi142.stdout:5/348: mknod df/d16/d24/d2e/c6a 0 2024-03-29T04:44:57.440 INFO:tasks.workunit.client.0.smithi142.stdout:7/296: mknod d7/d27/d3a/c54 0 2024-03-29T04:44:57.440 INFO:tasks.workunit.client.0.smithi142.stdout:5/349: chown df/d29/f67 2769 1 2024-03-29T04:44:57.440 INFO:tasks.workunit.client.0.smithi142.stdout:1/306: dread d8/db/dc/d19/d1d/d15/f17 [0,4194304] 0 2024-03-29T04:44:57.441 INFO:tasks.workunit.client.0.smithi142.stdout:7/297: dread - d7/dc/de/d20/f4e zero size 2024-03-29T04:44:57.441 INFO:tasks.workunit.client.0.smithi142.stdout:1/307: chown d8/db/dc/d19/d1d/d2d 495524 1 2024-03-29T04:44:57.441 INFO:tasks.workunit.client.0.smithi142.stdout:5/350: creat df/d29/f6b x:0 0 0 2024-03-29T04:44:57.441 INFO:tasks.workunit.client.0.smithi142.stdout:5/351: stat df/c3a 0 2024-03-29T04:44:57.442 INFO:tasks.workunit.client.0.smithi142.stdout:7/298: read d7/f43 [165131,109390] 0 2024-03-29T04:44:57.444 INFO:tasks.workunit.client.0.smithi142.stdout:7/299: mkdir d7/d18/d30/d55 0 2024-03-29T04:44:57.445 INFO:tasks.workunit.client.0.smithi142.stdout:7/300: chown f6 3 1 2024-03-29T04:44:57.446 INFO:tasks.workunit.client.0.smithi142.stdout:5/352: rmdir df/d16/d24/d2e/d47/d54 0 2024-03-29T04:44:57.447 INFO:tasks.workunit.client.0.smithi142.stdout:7/301: rename d7/d27/d44/d4c/l53 to d7/d27/d44/d4c/l56 0 2024-03-29T04:44:57.448 INFO:tasks.workunit.client.0.smithi142.stdout:5/353: rmdir df/d16/d24/d3d 39 2024-03-29T04:44:57.453 INFO:tasks.workunit.client.0.smithi142.stdout:7/302: creat d7/f57 x:0 0 0 2024-03-29T04:44:57.453 INFO:tasks.workunit.client.0.smithi142.stdout:5/354: dread df/d16/d25/f37 [0,4194304] 0 2024-03-29T04:44:57.455 INFO:tasks.workunit.client.0.smithi142.stdout:1/308: dwrite d8/db/dc/d19/f5c [0,4194304] 0 2024-03-29T04:44:57.455 INFO:tasks.workunit.client.0.smithi142.stdout:7/303: symlink d7/d27/d44/l58 0 2024-03-29T04:44:57.456 INFO:tasks.workunit.client.0.smithi142.stdout:5/355: creat df/d16/d24/d2e/f6c x:0 0 0 2024-03-29T04:44:57.457 INFO:tasks.workunit.client.0.smithi142.stdout:1/309: creat d8/db/d37/f76 x:0 0 0 2024-03-29T04:44:57.458 INFO:tasks.workunit.client.0.smithi142.stdout:7/304: symlink d7/d27/d3a/l59 0 2024-03-29T04:44:57.459 INFO:tasks.workunit.client.0.smithi142.stdout:5/356: truncate df/d16/f39 1839008 0 2024-03-29T04:44:57.459 INFO:tasks.workunit.client.0.smithi142.stdout:5/357: read df/d1a/f3f [2367557,83462] 0 2024-03-29T04:44:57.462 INFO:tasks.workunit.client.0.smithi142.stdout:7/305: rename d7/dc/de/d20/c2e to d7/dc/de/c5a 0 2024-03-29T04:44:57.463 INFO:tasks.workunit.client.0.smithi142.stdout:1/310: truncate d8/db/dc/d19/d1d/d15/f17 2386770 0 2024-03-29T04:44:57.463 INFO:tasks.workunit.client.0.smithi142.stdout:5/358: mknod df/c6d 0 2024-03-29T04:44:57.464 INFO:tasks.workunit.client.0.smithi142.stdout:5/359: chown df/d16/d50 0 1 2024-03-29T04:44:57.465 INFO:tasks.workunit.client.0.smithi142.stdout:7/306: dread d7/d18/f2c [0,4194304] 0 2024-03-29T04:44:57.466 INFO:tasks.workunit.client.0.smithi142.stdout:5/360: mkdir df/d29/d6e 0 2024-03-29T04:44:57.469 INFO:tasks.workunit.client.0.smithi142.stdout:1/311: rename d8/db/dc/d19/d1d/d15/d21 to d8/db/dc/d77 0 2024-03-29T04:44:57.471 INFO:tasks.workunit.client.0.smithi142.stdout:5/361: dwrite df/d16/f26 [4194304,4194304] 0 2024-03-29T04:44:57.473 INFO:tasks.workunit.client.0.smithi142.stdout:7/307: stat d7/dc/de/c5a 0 2024-03-29T04:44:57.474 INFO:tasks.workunit.client.0.smithi142.stdout:7/308: dread - d7/f57 zero size 2024-03-29T04:44:57.476 INFO:tasks.workunit.client.0.smithi142.stdout:5/362: mknod df/d1a/d21/d55/c6f 0 2024-03-29T04:44:57.476 INFO:tasks.workunit.client.0.smithi142.stdout:1/312: link d8/db/dc/cd d8/db/dc/d19/d1d/d1f/c78 0 2024-03-29T04:44:57.476 INFO:tasks.workunit.client.0.smithi142.stdout:5/363: write fe [295206,13345] 0 2024-03-29T04:44:57.477 INFO:tasks.workunit.client.0.smithi142.stdout:1/313: chown d8/db/dc/d77/l72 56110 1 2024-03-29T04:44:57.478 INFO:tasks.workunit.client.0.smithi142.stdout:7/309: creat d7/d27/d3a/f5b x:0 0 0 2024-03-29T04:44:57.481 INFO:tasks.workunit.client.0.smithi142.stdout:7/310: dread d7/dc/de/d20/f40 [4194304,4194304] 0 2024-03-29T04:44:57.482 INFO:tasks.workunit.client.0.smithi142.stdout:1/314: creat d8/db/d37/f79 x:0 0 0 2024-03-29T04:44:57.483 INFO:tasks.workunit.client.0.smithi142.stdout:5/364: mknod df/d16/d24/d3d/c70 0 2024-03-29T04:44:57.485 INFO:tasks.workunit.client.0.smithi142.stdout:7/311: dread d7/dc/de/d20/f40 [0,4194304] 0 2024-03-29T04:44:57.485 INFO:tasks.workunit.client.0.smithi142.stdout:1/315: rename d8/db/f49 to d8/db/dc/d77/d31/d4d/f7a 0 2024-03-29T04:44:57.486 INFO:tasks.workunit.client.0.smithi142.stdout:5/365: rmdir df/d16/d24/d2e/d47 39 2024-03-29T04:44:57.488 INFO:tasks.workunit.client.0.smithi142.stdout:7/312: creat d7/dc/de/f5c x:0 0 0 2024-03-29T04:44:57.493 INFO:tasks.workunit.client.0.smithi142.stdout:1/316: dwrite d8/db/dc/d77/d31/f64 [0,4194304] 0 2024-03-29T04:44:57.494 INFO:tasks.workunit.client.0.smithi142.stdout:1/317: fsync f7 0 2024-03-29T04:44:57.495 INFO:tasks.workunit.client.0.smithi142.stdout:1/318: chown d8/db 1688716 1 2024-03-29T04:44:57.495 INFO:tasks.workunit.client.0.smithi142.stdout:7/313: dwrite d7/d18/d30/f4a [0,4194304] 0 2024-03-29T04:44:57.498 INFO:tasks.workunit.client.0.smithi142.stdout:7/314: mknod d7/dc/de/c5d 0 2024-03-29T04:44:57.502 INFO:tasks.workunit.client.0.smithi142.stdout:7/315: creat d7/f5e x:0 0 0 2024-03-29T04:44:57.505 INFO:tasks.workunit.client.0.smithi142.stdout:7/316: getdents d7/dc/d4b 0 2024-03-29T04:44:57.509 INFO:tasks.workunit.client.0.smithi142.stdout:7/317: rename d7/d27/d3a/l3b to d7/d18/d30/d55/l5f 0 2024-03-29T04:44:57.511 INFO:tasks.workunit.client.0.smithi142.stdout:7/318: creat d7/d27/d44/f60 x:0 0 0 2024-03-29T04:44:57.517 INFO:tasks.workunit.client.0.smithi142.stdout:7/319: dwrite d7/d18/d30/d33/f38 [0,4194304] 0 2024-03-29T04:44:57.521 INFO:tasks.workunit.client.0.smithi142.stdout:7/320: creat d7/dc/de/d20/d3f/f61 x:0 0 0 2024-03-29T04:44:57.527 INFO:tasks.workunit.client.0.smithi142.stdout:7/321: getdents d7/d18/d30/d55 0 2024-03-29T04:44:57.528 INFO:tasks.workunit.client.0.smithi142.stdout:7/322: chown d7/dc/de/d20/f47 154660 1 2024-03-29T04:44:57.529 INFO:tasks.workunit.client.0.smithi142.stdout:1/319: dread d8/db/dc/d19/d1d/d1f/f6d [0,4194304] 0 2024-03-29T04:44:57.530 INFO:tasks.workunit.client.0.smithi142.stdout:7/323: mkdir d7/d62 0 2024-03-29T04:44:57.530 INFO:tasks.workunit.client.0.smithi142.stdout:1/320: write d8/db/dc/d77/d31/f3f [1015557,81726] 0 2024-03-29T04:44:57.532 INFO:tasks.workunit.client.0.smithi142.stdout:7/324: write d7/d18/d30/d33/f4d [904936,40858] 0 2024-03-29T04:44:57.532 INFO:tasks.workunit.client.0.smithi142.stdout:4/239: sync 2024-03-29T04:44:57.532 INFO:tasks.workunit.client.0.smithi142.stdout:2/215: sync 2024-03-29T04:44:57.532 INFO:tasks.workunit.client.0.smithi142.stdout:1/321: truncate d8/db/f25 4484279 0 2024-03-29T04:44:57.535 INFO:tasks.workunit.client.0.smithi142.stdout:4/240: rename d0/d9/fb to d0/dc/d15/f53 0 2024-03-29T04:44:57.538 INFO:tasks.workunit.client.0.smithi142.stdout:2/216: getdents de/d11/d1e 0 2024-03-29T04:44:57.540 INFO:tasks.workunit.client.0.smithi142.stdout:4/241: truncate d0/f3 1540495 0 2024-03-29T04:44:57.540 INFO:tasks.workunit.client.0.smithi142.stdout:1/322: dwrite d8/f51 [0,4194304] 0 2024-03-29T04:44:57.541 INFO:tasks.workunit.client.0.smithi142.stdout:2/217: stat de/d11/d2b/d3e 0 2024-03-29T04:44:57.541 INFO:tasks.workunit.client.0.smithi142.stdout:7/325: dwrite d7/dc/de/d20/f40 [4194304,4194304] 0 2024-03-29T04:44:57.543 INFO:tasks.workunit.client.0.smithi142.stdout:2/218: readlink de/d11/d23/l47 0 2024-03-29T04:44:57.543 INFO:tasks.workunit.client.0.smithi142.stdout:1/323: dread - d8/db/dc/d77/d31/d4d/f7a zero size 2024-03-29T04:44:57.544 INFO:tasks.workunit.client.0.smithi142.stdout:4/242: creat d0/d12/d28/f54 x:0 0 0 2024-03-29T04:44:57.545 INFO:tasks.workunit.client.0.smithi142.stdout:4/243: chown d0/d12/d28/d47/f4b 11 1 2024-03-29T04:44:57.550 INFO:tasks.workunit.client.0.smithi142.stdout:7/326: dread d7/d18/d30/d33/f38 [8388608,4194304] 0 2024-03-29T04:44:57.550 INFO:tasks.workunit.client.0.smithi142.stdout:1/324: truncate d8/db/dc/d19/d1d/d15/f47 1092381 0 2024-03-29T04:44:57.556 INFO:tasks.workunit.client.0.smithi142.stdout:7/327: dwrite d7/dc/de/f5c [0,4194304] 0 2024-03-29T04:44:57.558 INFO:tasks.workunit.client.0.smithi142.stdout:1/325: dread d8/f2a [0,4194304] 0 2024-03-29T04:44:57.561 INFO:tasks.workunit.client.0.smithi142.stdout:7/328: rename d7/l2a to d7/d27/d44/l63 0 2024-03-29T04:44:57.561 INFO:tasks.workunit.client.0.smithi142.stdout:1/326: stat d8/db/dc/d19/d1d/d2d/d44/d4b/l4f 0 2024-03-29T04:44:57.563 INFO:tasks.workunit.client.0.smithi142.stdout:7/329: creat d7/d62/f64 x:0 0 0 2024-03-29T04:44:57.564 INFO:tasks.workunit.client.0.smithi142.stdout:7/330: dread - d7/f57 zero size 2024-03-29T04:44:57.566 INFO:tasks.workunit.client.0.smithi142.stdout:4/244: write d0/d12/f50 [2934491,91764] 0 2024-03-29T04:44:57.567 INFO:tasks.workunit.client.0.smithi142.stdout:7/331: write d7/dc/de/d20/f21 [2391698,18785] 0 2024-03-29T04:44:57.569 INFO:tasks.workunit.client.0.smithi142.stdout:1/327: dwrite d8/db/d5e/f73 [0,4194304] 0 2024-03-29T04:44:57.571 INFO:tasks.workunit.client.0.smithi142.stdout:7/332: chown d7/dc/l17 1841225 1 2024-03-29T04:44:57.641 INFO:tasks.workunit.client.0.smithi142.stdout:3/244: sync 2024-03-29T04:44:57.715 INFO:tasks.workunit.client.0.smithi142.stdout:6/279: sync 2024-03-29T04:44:57.715 INFO:tasks.workunit.client.0.smithi142.stdout:9/217: sync 2024-03-29T04:44:57.715 INFO:tasks.workunit.client.0.smithi142.stdout:9/218: fsync d0/d2/d6/d31/f49 0 2024-03-29T04:44:57.764 INFO:tasks.workunit.client.0.smithi142.stdout:5/366: sync 2024-03-29T04:44:57.764 INFO:tasks.workunit.client.0.smithi142.stdout:8/329: sync 2024-03-29T04:44:57.764 INFO:tasks.workunit.client.0.smithi142.stdout:0/236: sync 2024-03-29T04:44:57.764 INFO:tasks.workunit.client.0.smithi142.stdout:2/219: sync 2024-03-29T04:44:57.765 INFO:tasks.workunit.client.0.smithi142.stdout:0/237: truncate d7/d9/f3c 709409 0 2024-03-29T04:44:57.766 INFO:tasks.workunit.client.0.smithi142.stdout:0/238: write d7/d9/f49 [573818,27588] 0 2024-03-29T04:44:57.769 INFO:tasks.workunit.client.0.smithi142.stdout:1/328: creat d8/db/dc/d19/d1d/d2d/d44/d53/f7b x:0 0 0 2024-03-29T04:44:57.771 INFO:tasks.workunit.client.0.smithi142.stdout:1/329: chown d8/db/dc/d19/d1d/f13 7 1 2024-03-29T04:44:57.772 INFO:tasks.workunit.client.0.smithi142.stdout:4/245: link d0/d9/f4d d0/d12/d28/d47/f55 0 2024-03-29T04:44:57.773 INFO:tasks.workunit.client.0.smithi142.stdout:0/239: dwrite d7/d9/d22/d37/d2f/f3d [0,4194304] 0 2024-03-29T04:44:57.774 INFO:tasks.workunit.client.0.smithi142.stdout:2/220: fsync de/f10 0 2024-03-29T04:44:57.774 INFO:tasks.workunit.client.0.smithi142.stdout:0/240: truncate d7/d9/f2e 977205 0 2024-03-29T04:44:57.778 INFO:tasks.workunit.client.0.smithi142.stdout:0/241: dread d7/d9/d22/f3e [0,4194304] 0 2024-03-29T04:44:57.778 INFO:tasks.workunit.client.0.smithi142.stdout:5/367: creat df/d2c/f71 x:0 0 0 2024-03-29T04:44:57.780 INFO:tasks.workunit.client.0.smithi142.stdout:1/330: creat d8/db/dc/d77/f7c x:0 0 0 2024-03-29T04:44:57.782 INFO:tasks.workunit.client.0.smithi142.stdout:3/245: mkdir d3/d1f/d59 0 2024-03-29T04:44:57.782 INFO:tasks.workunit.client.0.smithi142.stdout:3/246: chown d3/d1f/d34/d6/c18 46 1 2024-03-29T04:44:57.785 INFO:tasks.workunit.client.0.smithi142.stdout:9/219: creat d0/d2/d6/d27/d3c/f4c x:0 0 0 2024-03-29T04:44:57.786 INFO:tasks.workunit.client.0.smithi142.stdout:4/246: creat d0/d9/d19/d2c/f56 x:0 0 0 2024-03-29T04:44:57.786 INFO:tasks.workunit.client.0.smithi142.stdout:7/333: write d7/dc/de/d20/d3f/f50 [94476,115275] 0 2024-03-29T04:44:57.788 INFO:tasks.workunit.client.0.smithi142.stdout:7/334: dread - d7/d18/d30/d33/f42 zero size 2024-03-29T04:44:57.789 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:57 smithi152 ceph-mon[37177]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:57.790 INFO:tasks.workunit.client.0.smithi142.stdout:2/221: creat de/d11/d1e/d22/d27/f4c x:0 0 0 2024-03-29T04:44:57.791 INFO:tasks.workunit.client.0.smithi142.stdout:7/335: write d7/d18/f28 [3871313,39991] 0 2024-03-29T04:44:57.791 INFO:tasks.workunit.client.0.smithi142.stdout:3/247: dwrite d3/d29/d2b/d40/f4c [0,4194304] 0 2024-03-29T04:44:57.791 INFO:tasks.workunit.client.0.smithi142.stdout:0/242: mknod d7/d9/d1e/c4c 0 2024-03-29T04:44:57.793 INFO:tasks.workunit.client.0.smithi142.stdout:1/331: creat d8/db/dc/d19/d1d/d2d/f7d x:0 0 0 2024-03-29T04:44:57.793 INFO:tasks.workunit.client.0.smithi142.stdout:3/248: chown d3/d29/d2b/f21 2 1 2024-03-29T04:44:57.796 INFO:tasks.workunit.client.0.smithi142.stdout:8/330: creat d5/d1d/f71 x:0 0 0 2024-03-29T04:44:57.796 INFO:tasks.workunit.client.0.smithi142.stdout:4/247: mknod d0/d9/d11/c57 0 2024-03-29T04:44:57.797 INFO:tasks.workunit.client.0.smithi142.stdout:3/249: write d3/d29/d2b/d40/f4a [536424,110443] 0 2024-03-29T04:44:57.798 INFO:tasks.workunit.client.0.smithi142.stdout:8/331: chown d5/d1d/d24 25963 1 2024-03-29T04:44:57.799 INFO:tasks.workunit.client.0.smithi142.stdout:2/222: mkdir de/d11/d1e/d4d 0 2024-03-29T04:44:57.799 INFO:tasks.workunit.client.0.smithi142.stdout:5/368: dwrite df/d1a/f22 [4194304,4194304] 0 2024-03-29T04:44:57.803 INFO:tasks.workunit.client.0.smithi142.stdout:6/280: getdents d1c 0 2024-03-29T04:44:57.804 INFO:tasks.workunit.client.0.smithi142.stdout:9/220: symlink d0/d2/l4d 0 2024-03-29T04:44:57.805 INFO:tasks.workunit.client.0.smithi142.stdout:6/281: fdatasync f18 0 2024-03-29T04:44:57.808 INFO:tasks.workunit.client.0.smithi142.stdout:7/336: dwrite d7/d18/f2c [0,4194304] 0 2024-03-29T04:44:57.815 INFO:tasks.workunit.client.0.smithi142.stdout:8/332: creat d5/d1d/d2f/d5f/d48/d52/f72 x:0 0 0 2024-03-29T04:44:57.815 INFO:tasks.workunit.client.0.smithi142.stdout:7/337: write f1 [282701,29881] 0 2024-03-29T04:44:57.817 INFO:tasks.workunit.client.0.smithi142.stdout:6/282: chown ff 998829442 1 2024-03-29T04:44:57.818 INFO:tasks.workunit.client.0.smithi142.stdout:1/332: rename d8/db/dc/d19/d1d/d15/c22 to d8/db/dc/d19/d1d/d1f/d50/c7e 0 2024-03-29T04:44:57.819 INFO:tasks.workunit.client.0.smithi142.stdout:1/333: readlink d8/db/l24 0 2024-03-29T04:44:57.820 INFO:tasks.workunit.client.0.smithi142.stdout:1/334: truncate d8/f51 4204210 0 2024-03-29T04:44:57.820 INFO:tasks.workunit.client.0.smithi142.stdout:3/250: mknod d3/d1f/d59/c5a 0 2024-03-29T04:44:57.823 INFO:tasks.workunit.client.0.smithi142.stdout:3/251: write d3/d1f/d34/f3e [734843,77346] 0 2024-03-29T04:44:57.823 INFO:tasks.workunit.client.0.smithi142.stdout:6/283: dwrite f18 [0,4194304] 0 2024-03-29T04:44:57.823 INFO:tasks.workunit.client.0.smithi142.stdout:5/369: symlink df/d1a/d21/l72 0 2024-03-29T04:44:57.824 INFO:tasks.workunit.client.0.smithi142.stdout:6/284: dread - d1c/f3c zero size 2024-03-29T04:44:57.826 INFO:tasks.workunit.client.0.smithi142.stdout:9/221: symlink d0/l4e 0 2024-03-29T04:44:57.827 INFO:tasks.workunit.client.0.smithi142.stdout:6/285: stat d1c/d33/f36 0 2024-03-29T04:44:57.827 INFO:tasks.workunit.client.0.smithi142.stdout:9/222: stat d0/d2/f2c 0 2024-03-29T04:44:57.828 INFO:tasks.workunit.client.0.smithi142.stdout:1/335: rmdir d8/db/d37 39 2024-03-29T04:44:57.828 INFO:tasks.workunit.client.0.smithi142.stdout:9/223: chown d0/d2/d6/d16/f37 440 1 2024-03-29T04:44:57.830 INFO:tasks.workunit.client.0.smithi142.stdout:2/223: symlink de/d11/d3d/l4e 0 2024-03-29T04:44:57.832 INFO:tasks.workunit.client.0.smithi142.stdout:3/252: symlink d3/d1f/d34/d4d/d50/l5b 0 2024-03-29T04:44:57.834 INFO:tasks.workunit.client.0.smithi142.stdout:2/224: dread fd [0,4194304] 0 2024-03-29T04:44:57.834 INFO:tasks.workunit.client.0.smithi142.stdout:0/243: getdents d7/d9/d1e 0 2024-03-29T04:44:57.835 INFO:tasks.workunit.client.0.smithi142.stdout:7/338: link d7/d18/d30/d33/f4d d7/dc/de/d20/d3f/f65 0 2024-03-29T04:44:57.835 INFO:tasks.workunit.client.0.smithi142.stdout:7/339: chown d7 124379074 1 2024-03-29T04:44:57.836 INFO:tasks.workunit.client.0.smithi142.stdout:4/248: dread d0/d12/d28/d47/f4b [0,4194304] 0 2024-03-29T04:44:57.836 INFO:tasks.workunit.client.0.smithi142.stdout:6/286: mkdir d1c/d42/d52 0 2024-03-29T04:44:57.836 INFO:tasks.workunit.client.0.smithi142.stdout:6/287: fdatasync d1c/f23 0 2024-03-29T04:44:57.838 INFO:tasks.workunit.client.0.smithi142.stdout:1/336: fsync d8/f2a 0 2024-03-29T04:44:57.838 INFO:tasks.workunit.client.0.smithi142.stdout:6/288: dread - d1c/d33/f3e zero size 2024-03-29T04:44:57.839 INFO:tasks.workunit.client.0.smithi142.stdout:6/289: chown d1c/f21 0 1 2024-03-29T04:44:57.839 INFO:tasks.workunit.client.0.smithi142.stdout:9/224: rename d0/d2/d6/d35 to d0/d2/d6/d27/d3c/d4f 0 2024-03-29T04:44:57.839 INFO:tasks.workunit.client.0.smithi142.stdout:5/370: creat df/d29/d6e/f73 x:0 0 0 2024-03-29T04:44:57.840 INFO:tasks.workunit.client.0.smithi142.stdout:3/253: unlink d3/d1f/d34/d6/f1c 0 2024-03-29T04:44:57.843 INFO:tasks.workunit.client.0.smithi142.stdout:8/333: dwrite d5/d10/d2d/f3b [0,4194304] 0 2024-03-29T04:44:57.845 INFO:tasks.workunit.client.0.smithi142.stdout:1/337: mkdir d8/db/dc/d19/d1d/d2d/d44/d7f 0 2024-03-29T04:44:57.846 INFO:tasks.workunit.client.0.smithi142.stdout:1/338: truncate d8/db/dc/d77/d5a/f75 654514 0 2024-03-29T04:44:57.849 INFO:tasks.workunit.client.0.smithi142.stdout:6/290: creat d1c/d33/d46/f53 x:0 0 0 2024-03-29T04:44:57.851 INFO:tasks.workunit.client.0.smithi142.stdout:7/340: symlink d7/l66 0 2024-03-29T04:44:57.853 INFO:tasks.workunit.client.0.smithi142.stdout:4/249: link d0/d9/d19/d2c/f56 d0/d9/d19/f58 0 2024-03-29T04:44:57.854 INFO:tasks.workunit.client.0.smithi142.stdout:0/244: mknod d7/d9/d22/d37/d2f/d42/c4d 0 2024-03-29T04:44:57.856 INFO:tasks.workunit.client.0.smithi142.stdout:5/371: rename df/d1a/d21/d27/f2a to df/d1a/f74 0 2024-03-29T04:44:57.857 INFO:tasks.workunit.client.0.smithi142.stdout:7/341: dwrite f1 [4194304,4194304] 0 2024-03-29T04:44:57.858 INFO:tasks.workunit.client.0.smithi142.stdout:8/334: chown l3 347651120 1 2024-03-29T04:44:57.859 INFO:tasks.workunit.client.0.smithi142.stdout:8/335: write d5/d1d/f71 [178649,14967] 0 2024-03-29T04:44:57.860 INFO:tasks.workunit.client.0.smithi142.stdout:1/339: mkdir d8/db/dc/d19/d1d/d2d/d44/d80 0 2024-03-29T04:44:57.861 INFO:tasks.workunit.client.0.smithi142.stdout:2/225: creat de/d11/f4f x:0 0 0 2024-03-29T04:44:57.863 INFO:tasks.workunit.client.0.smithi142.stdout:6/291: rename ff to d1c/d42/d52/f54 0 2024-03-29T04:44:57.865 INFO:tasks.workunit.client.0.smithi142.stdout:5/372: mkdir df/d29/d5d/d75 0 2024-03-29T04:44:57.868 INFO:tasks.workunit.client.0.smithi142.stdout:2/226: dread de/d11/d1e/d22/d27/f4b [0,4194304] 0 2024-03-29T04:44:57.869 INFO:tasks.workunit.client.0.smithi142.stdout:5/373: write df/d29/f67 [491847,85078] 0 2024-03-29T04:44:57.869 INFO:tasks.workunit.client.0.smithi142.stdout:0/245: symlink d7/d9/d22/d37/d2f/l4e 0 2024-03-29T04:44:57.870 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:57 smithi142 ceph-mon[29027]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:44:57.873 INFO:tasks.workunit.client.0.smithi142.stdout:8/336: link d5/d10/f41 d5/d10/d2d/f73 0 2024-03-29T04:44:57.873 INFO:tasks.workunit.client.0.smithi142.stdout:6/292: write fc [5159822,67695] 0 2024-03-29T04:44:57.875 INFO:tasks.workunit.client.0.smithi142.stdout:5/374: stat df/d16/d24/d2e/c57 0 2024-03-29T04:44:57.875 INFO:tasks.workunit.client.0.smithi142.stdout:4/250: dwrite d0/dc/d15/f53 [0,4194304] 0 2024-03-29T04:44:57.876 INFO:tasks.workunit.client.0.smithi142.stdout:0/246: chown d7/d13/f28 1749144 1 2024-03-29T04:44:57.877 INFO:tasks.workunit.client.0.smithi142.stdout:7/342: link d7/d27/d44/l58 d7/l67 0 2024-03-29T04:44:57.877 INFO:tasks.workunit.client.0.smithi142.stdout:3/254: dread d3/d29/d2d/f3f [0,4194304] 0 2024-03-29T04:44:57.878 INFO:tasks.workunit.client.0.smithi142.stdout:9/225: dread d0/d2/f1f [0,4194304] 0 2024-03-29T04:44:57.880 INFO:tasks.workunit.client.0.smithi142.stdout:4/251: stat d0/d9/d11/f23 0 2024-03-29T04:44:57.885 INFO:tasks.workunit.client.0.smithi142.stdout:1/340: dwrite d8/db/d37/f41 [0,4194304] 0 2024-03-29T04:44:57.885 INFO:tasks.workunit.client.0.smithi142.stdout:5/375: truncate df/d16/d24/d3d/f68 477831 0 2024-03-29T04:44:57.886 INFO:tasks.workunit.client.0.smithi142.stdout:0/247: symlink d7/d9/d22/d37/d2f/l4f 0 2024-03-29T04:44:57.886 INFO:tasks.workunit.client.0.smithi142.stdout:4/252: read d0/f3 [369685,22424] 0 2024-03-29T04:44:57.889 INFO:tasks.workunit.client.0.smithi142.stdout:2/227: dwrite de/f19 [4194304,4194304] 0 2024-03-29T04:44:57.889 INFO:tasks.workunit.client.0.smithi142.stdout:9/226: chown d0/d2/d6/d27/d3c/d4f/d34 1180 1 2024-03-29T04:44:57.889 INFO:tasks.workunit.client.0.smithi142.stdout:5/376: symlink df/d16/l76 0 2024-03-29T04:44:57.889 INFO:tasks.workunit.client.0.smithi142.stdout:1/341: chown d8/db/dc/d77/f7c 1814361690 1 2024-03-29T04:44:57.889 INFO:tasks.workunit.client.0.smithi142.stdout:0/248: readlink d7/d9/d1e/l36 0 2024-03-29T04:44:57.890 INFO:tasks.workunit.client.0.smithi142.stdout:4/253: write d0/f48 [739565,66736] 0 2024-03-29T04:44:57.891 INFO:tasks.workunit.client.0.smithi142.stdout:7/343: link d7/dc/de/f26 d7/dc/f68 0 2024-03-29T04:44:57.891 INFO:tasks.workunit.client.0.smithi142.stdout:2/228: write f9 [2367235,112870] 0 2024-03-29T04:44:57.893 INFO:tasks.workunit.client.0.smithi142.stdout:4/254: fsync d0/d12/f3c 0 2024-03-29T04:44:57.897 INFO:tasks.workunit.client.0.smithi142.stdout:5/377: mknod df/c77 0 2024-03-29T04:44:57.897 INFO:tasks.workunit.client.0.smithi142.stdout:0/249: creat d7/d13/f50 x:0 0 0 2024-03-29T04:44:57.897 INFO:tasks.workunit.client.0.smithi142.stdout:5/378: stat df/d1a/f22 0 2024-03-29T04:44:57.898 INFO:tasks.workunit.client.0.smithi142.stdout:9/227: mkdir d0/d12/d50 0 2024-03-29T04:44:57.902 INFO:tasks.workunit.client.0.smithi142.stdout:7/344: truncate d7/d27/d44/f60 443141 0 2024-03-29T04:44:57.903 INFO:tasks.workunit.client.0.smithi142.stdout:0/250: read d7/d9/d1e/f26 [1126675,16531] 0 2024-03-29T04:44:57.903 INFO:tasks.workunit.client.0.smithi142.stdout:5/379: mknod df/d1a/d21/d27/d45/c78 0 2024-03-29T04:44:57.905 INFO:tasks.workunit.client.0.smithi142.stdout:7/345: stat d7/dc/de/c5d 0 2024-03-29T04:44:57.908 INFO:tasks.workunit.client.0.smithi142.stdout:0/251: symlink d7/d13/l51 0 2024-03-29T04:44:57.908 INFO:tasks.workunit.client.0.smithi142.stdout:4/255: dwrite d0/d9/d11/f25 [0,4194304] 0 2024-03-29T04:44:57.908 INFO:tasks.workunit.client.0.smithi142.stdout:2/229: dwrite de/f36 [0,4194304] 0 2024-03-29T04:44:57.909 INFO:tasks.workunit.client.0.smithi142.stdout:5/380: unlink df/d1a/d21/d55/c6f 0 2024-03-29T04:44:57.910 INFO:tasks.workunit.client.0.smithi142.stdout:0/252: read - d7/d9/d22/d37/f39 zero size 2024-03-29T04:44:57.911 INFO:tasks.workunit.client.0.smithi142.stdout:7/346: symlink d7/d62/l69 0 2024-03-29T04:44:57.913 INFO:tasks.workunit.client.0.smithi142.stdout:0/253: write d7/d9/d22/d37/f43 [743621,60588] 0 2024-03-29T04:44:57.916 INFO:tasks.workunit.client.0.smithi142.stdout:7/347: unlink d7/dc/f49 0 2024-03-29T04:44:57.917 INFO:tasks.workunit.client.0.smithi142.stdout:3/255: dread d3/d1f/f46 [0,4194304] 0 2024-03-29T04:44:57.917 INFO:tasks.workunit.client.0.smithi142.stdout:4/256: dread d0/d12/f3b [0,4194304] 0 2024-03-29T04:44:57.919 INFO:tasks.workunit.client.0.smithi142.stdout:2/230: mkdir de/d11/d1e/d22/d50 0 2024-03-29T04:44:57.921 INFO:tasks.workunit.client.0.smithi142.stdout:2/231: truncate de/d11/d23/f28 116030 0 2024-03-29T04:44:57.921 INFO:tasks.workunit.client.0.smithi142.stdout:0/254: fsync d7/d9/d22/f48 0 2024-03-29T04:44:57.925 INFO:tasks.workunit.client.0.smithi142.stdout:4/257: creat d0/dc/d15/f59 x:0 0 0 2024-03-29T04:44:57.927 INFO:tasks.workunit.client.0.smithi142.stdout:7/348: symlink d7/d18/l6a 0 2024-03-29T04:44:57.929 INFO:tasks.workunit.client.0.smithi142.stdout:1/342: dread f3 [0,4194304] 0 2024-03-29T04:44:57.930 INFO:tasks.workunit.client.0.smithi142.stdout:4/258: creat d0/f5a x:0 0 0 2024-03-29T04:44:57.930 INFO:tasks.workunit.client.0.smithi142.stdout:9/228: write d0/d2/d6/d31/f3a [1759669,4036] 0 2024-03-29T04:44:57.931 INFO:tasks.workunit.client.0.smithi142.stdout:9/229: fdatasync d0/d12/f24 0 2024-03-29T04:44:57.932 INFO:tasks.workunit.client.0.smithi142.stdout:1/343: truncate d8/db/dc/d19/d1d/d1f/f29 5170440 0 2024-03-29T04:44:57.932 INFO:tasks.workunit.client.0.smithi142.stdout:7/349: mknod d7/d18/d30/c6b 0 2024-03-29T04:44:57.932 INFO:tasks.workunit.client.0.smithi142.stdout:9/230: dread - d0/d2/d6/d27/d3c/f4c zero size 2024-03-29T04:44:57.934 INFO:tasks.workunit.client.0.smithi142.stdout:9/231: fsync d0/d2/d6/d27/d3c/d4f/d34/f2d 0 2024-03-29T04:44:57.934 INFO:tasks.workunit.client.0.smithi142.stdout:2/232: link de/c1f de/d11/d1e/c51 0 2024-03-29T04:44:57.934 INFO:tasks.workunit.client.0.smithi142.stdout:3/256: link d3/d1f/d34/c3a d3/d1f/d59/c5c 0 2024-03-29T04:44:57.936 INFO:tasks.workunit.client.0.smithi142.stdout:7/350: symlink d7/dc/de/l6c 0 2024-03-29T04:44:57.937 INFO:tasks.workunit.client.0.smithi142.stdout:9/232: unlink d0/d2/d6/d27/d3c/d4f/c47 0 2024-03-29T04:44:57.939 INFO:tasks.workunit.client.0.smithi142.stdout:3/257: unlink d3/d29/d2b/c4b 0 2024-03-29T04:44:57.940 INFO:tasks.workunit.client.0.smithi142.stdout:2/233: dwrite fd [0,4194304] 0 2024-03-29T04:44:57.940 INFO:tasks.workunit.client.0.smithi142.stdout:7/351: symlink d7/d18/d30/l6d 0 2024-03-29T04:44:57.940 INFO:tasks.workunit.client.0.smithi142.stdout:3/258: chown d3/d29/d2d 1 1 2024-03-29T04:44:57.942 INFO:tasks.workunit.client.0.smithi142.stdout:7/352: fdatasync d7/d18/f28 0 2024-03-29T04:44:57.943 INFO:tasks.workunit.client.0.smithi142.stdout:3/259: mkdir d3/d5d 0 2024-03-29T04:44:57.946 INFO:tasks.workunit.client.0.smithi142.stdout:3/260: chown d3/d29/d2d/f3f 3 1 2024-03-29T04:44:57.946 INFO:tasks.workunit.client.0.smithi142.stdout:7/353: creat d7/d27/f6e x:0 0 0 2024-03-29T04:44:57.949 INFO:tasks.workunit.client.0.smithi142.stdout:3/261: truncate d3/d1f/d34/f3e 1642971 0 2024-03-29T04:44:57.951 INFO:tasks.workunit.client.0.smithi142.stdout:2/234: dwrite de/d11/d17/f31 [0,4194304] 0 2024-03-29T04:44:57.952 INFO:tasks.workunit.client.0.smithi142.stdout:2/235: chown de/l2a 763861 1 2024-03-29T04:44:57.952 INFO:tasks.workunit.client.0.smithi142.stdout:0/255: dread d7/d9/ff [4194304,4194304] 0 2024-03-29T04:44:57.960 INFO:tasks.workunit.client.0.smithi142.stdout:4/259: dwrite d0/d9/f4d [0,4194304] 0 2024-03-29T04:44:57.961 INFO:tasks.workunit.client.0.smithi142.stdout:9/233: write d0/d2/f2f [1203808,2566] 0 2024-03-29T04:44:57.965 INFO:tasks.workunit.client.0.smithi142.stdout:4/260: mknod d0/dc/c5b 0 2024-03-29T04:44:57.966 INFO:tasks.workunit.client.0.smithi142.stdout:1/344: dwrite d8/fa [0,4194304] 0 2024-03-29T04:44:57.966 INFO:tasks.workunit.client.0.smithi142.stdout:2/236: write de/f10 [2561944,100634] 0 2024-03-29T04:44:57.969 INFO:tasks.workunit.client.0.smithi142.stdout:4/261: mkdir d0/d9/d11/d5c 0 2024-03-29T04:44:57.972 INFO:tasks.workunit.client.0.smithi142.stdout:1/345: creat d8/db/dc/d19/d1d/d1f/d4c/f81 x:0 0 0 2024-03-29T04:44:57.972 INFO:tasks.workunit.client.0.smithi142.stdout:0/256: dread d7/d9/d22/f1b [0,4194304] 0 2024-03-29T04:44:57.973 INFO:tasks.workunit.client.0.smithi142.stdout:2/237: rename de/d11/d17/c18 to de/d11/d2b/c52 0 2024-03-29T04:44:57.977 INFO:tasks.workunit.client.0.smithi142.stdout:4/262: rename d0/dc/d15/f18 to d0/d12/f5d 0 2024-03-29T04:44:57.977 INFO:tasks.workunit.client.0.smithi142.stdout:7/354: dread d7/dc/de/d20/f47 [0,4194304] 0 2024-03-29T04:44:57.979 INFO:tasks.workunit.client.0.smithi142.stdout:1/346: symlink d8/db/l82 0 2024-03-29T04:44:57.981 INFO:tasks.workunit.client.0.smithi142.stdout:0/257: dwrite d7/d9/d22/d37/f38 [0,4194304] 0 2024-03-29T04:44:57.981 INFO:tasks.workunit.client.0.smithi142.stdout:1/347: chown d8/db/dc/d19/d1d/d1f 21132 1 2024-03-29T04:44:57.982 INFO:tasks.workunit.client.0.smithi142.stdout:4/263: creat d0/d9/d19/d2c/f5e x:0 0 0 2024-03-29T04:44:57.982 INFO:tasks.workunit.client.0.smithi142.stdout:7/355: write d7/dc/de/d20/d3f/f50 [45427,124846] 0 2024-03-29T04:44:57.984 INFO:tasks.workunit.client.0.smithi142.stdout:1/348: dread - d8/db/d37/f79 zero size 2024-03-29T04:44:57.986 INFO:tasks.workunit.client.0.smithi142.stdout:4/264: mkdir d0/d9/d11/d4c/d5f 0 2024-03-29T04:44:57.986 INFO:tasks.workunit.client.0.smithi142.stdout:0/258: creat d7/f52 x:0 0 0 2024-03-29T04:44:57.992 INFO:tasks.workunit.client.0.smithi142.stdout:0/259: dwrite d7/d13/f41 [0,4194304] 0 2024-03-29T04:44:58.017 INFO:tasks.workunit.client.0.smithi142.stdout:8/337: sync 2024-03-29T04:44:58.029 INFO:tasks.workunit.client.0.smithi142.stdout:2/238: write de/f29 [4852227,26350] 0 2024-03-29T04:44:58.030 INFO:tasks.workunit.client.0.smithi142.stdout:0/260: getdents d7/d9/d22 0 2024-03-29T04:44:58.031 INFO:tasks.workunit.client.0.smithi142.stdout:0/261: chown d7/d9/f10 60429 1 2024-03-29T04:44:58.032 INFO:tasks.workunit.client.0.smithi142.stdout:2/239: mknod de/d11/d1e/c53 0 2024-03-29T04:44:58.033 INFO:tasks.workunit.client.0.smithi142.stdout:2/240: readlink de/d11/d1e/d22/l49 0 2024-03-29T04:44:58.033 INFO:tasks.workunit.client.0.smithi142.stdout:0/262: truncate d7/d9/d22/f1b 928987 0 2024-03-29T04:44:58.034 INFO:tasks.workunit.client.0.smithi142.stdout:2/241: stat de/d11/d17/c1b 0 2024-03-29T04:44:58.034 INFO:tasks.workunit.client.0.smithi142.stdout:8/338: rename d5/d1d/d2f/d5f/d48/l5c to d5/d1d/d2f/d5f/d48/l74 0 2024-03-29T04:44:58.035 INFO:tasks.workunit.client.0.smithi142.stdout:2/242: readlink de/d11/d1e/d22/l2e 0 2024-03-29T04:44:58.035 INFO:tasks.workunit.client.0.smithi142.stdout:2/243: chown de/d11/d1e/d22/f3f 3 1 2024-03-29T04:44:58.036 INFO:tasks.workunit.client.0.smithi142.stdout:0/263: creat d7/d9/d1e/f53 x:0 0 0 2024-03-29T04:44:58.037 INFO:tasks.workunit.client.0.smithi142.stdout:8/339: fsync d5/f19 0 2024-03-29T04:44:58.037 INFO:tasks.workunit.client.0.smithi142.stdout:0/264: truncate d7/d13/f50 729668 0 2024-03-29T04:44:58.038 INFO:tasks.workunit.client.0.smithi142.stdout:8/340: readlink d5/d10/d2d/l5b 0 2024-03-29T04:44:58.038 INFO:tasks.workunit.client.0.smithi142.stdout:2/244: chown de/d11/f15 679331 1 2024-03-29T04:44:58.038 INFO:tasks.workunit.client.0.smithi142.stdout:7/356: dread f6 [0,4194304] 0 2024-03-29T04:44:58.039 INFO:tasks.workunit.client.0.smithi142.stdout:0/265: read d7/d9/d22/d37/d2f/f3d [2813657,43315] 0 2024-03-29T04:44:58.041 INFO:tasks.workunit.client.0.smithi142.stdout:0/266: write d7/d9/d22/f48 [2028901,34803] 0 2024-03-29T04:44:58.042 INFO:tasks.workunit.client.0.smithi142.stdout:7/357: fsync d7/dc/f68 0 2024-03-29T04:44:58.043 INFO:tasks.workunit.client.0.smithi142.stdout:0/267: dread d7/d9/d22/f1b [0,4194304] 0 2024-03-29T04:44:58.047 INFO:tasks.workunit.client.0.smithi142.stdout:8/341: getdents d5/d1d/d24/d66 0 2024-03-29T04:44:58.049 INFO:tasks.workunit.client.0.smithi142.stdout:2/245: dwrite de/f16 [0,4194304] 0 2024-03-29T04:44:58.051 INFO:tasks.workunit.client.0.smithi142.stdout:7/358: dwrite d7/dc/de/d20/f4e [0,4194304] 0 2024-03-29T04:44:58.051 INFO:tasks.workunit.client.0.smithi142.stdout:0/268: truncate d7/d9/d22/d37/d2f/f3d 4208840 0 2024-03-29T04:44:58.055 INFO:tasks.workunit.client.0.smithi142.stdout:2/246: symlink de/d11/d1e/d22/d27/l54 0 2024-03-29T04:44:58.057 INFO:tasks.workunit.client.0.smithi142.stdout:7/359: mknod d7/dc/de/c6f 0 2024-03-29T04:44:58.057 INFO:tasks.workunit.client.0.smithi142.stdout:7/360: chown d7/d18/c1c 816576 1 2024-03-29T04:44:58.057 INFO:tasks.workunit.client.0.smithi142.stdout:0/269: mkdir d7/d9/d54 0 2024-03-29T04:44:58.058 INFO:tasks.workunit.client.0.smithi142.stdout:2/247: creat de/d11/d17/d4a/f55 x:0 0 0 2024-03-29T04:44:58.060 INFO:tasks.workunit.client.0.smithi142.stdout:2/248: stat de/d11/d1e/d22/c38 0 2024-03-29T04:44:58.064 INFO:tasks.workunit.client.0.smithi142.stdout:7/361: dwrite d7/d18/f35 [0,4194304] 0 2024-03-29T04:44:58.066 INFO:tasks.workunit.client.0.smithi142.stdout:2/249: dread fd [0,4194304] 0 2024-03-29T04:44:58.067 INFO:tasks.workunit.client.0.smithi142.stdout:2/250: dread - de/d11/d1e/d22/d27/f4c zero size 2024-03-29T04:44:58.069 INFO:tasks.workunit.client.0.smithi142.stdout:7/362: mknod d7/d18/d30/d33/c70 0 2024-03-29T04:44:58.070 INFO:tasks.workunit.client.0.smithi142.stdout:0/270: rename d7/d9/d22/d37/d2f to d7/d9/d55 0 2024-03-29T04:44:58.070 INFO:tasks.workunit.client.0.smithi142.stdout:2/251: truncate de/d11/d23/f46 269200 0 2024-03-29T04:44:58.073 INFO:tasks.workunit.client.0.smithi142.stdout:0/271: dread d7/d13/f50 [0,4194304] 0 2024-03-29T04:44:58.074 INFO:tasks.workunit.client.0.smithi142.stdout:2/252: fdatasync de/d11/f37 0 2024-03-29T04:44:58.076 INFO:tasks.workunit.client.0.smithi142.stdout:2/253: write de/d11/d1e/d22/d27/f45 [933494,102260] 0 2024-03-29T04:44:58.078 INFO:tasks.workunit.client.0.smithi142.stdout:2/254: mknod de/d11/d3d/c56 0 2024-03-29T04:44:58.081 INFO:tasks.workunit.client.0.smithi142.stdout:2/255: creat de/d11/d1e/d22/d50/f57 x:0 0 0 2024-03-29T04:44:58.085 INFO:tasks.workunit.client.0.smithi142.stdout:2/256: rename de/d11/f21 to de/d11/f58 0 2024-03-29T04:44:58.086 INFO:tasks.workunit.client.0.smithi142.stdout:7/363: rename d7/d18 to d7/dc/d4b/d71 0 2024-03-29T04:44:58.087 INFO:tasks.workunit.client.0.smithi142.stdout:7/364: write d7/d27/f6e [68460,60987] 0 2024-03-29T04:44:58.088 INFO:tasks.workunit.client.0.smithi142.stdout:2/257: rmdir de/d11/d23 39 2024-03-29T04:44:58.089 INFO:tasks.workunit.client.0.smithi142.stdout:2/258: write de/d11/f15 [9742612,50874] 0 2024-03-29T04:44:58.091 INFO:tasks.workunit.client.0.smithi142.stdout:0/272: rename d7/d13/f41 to d7/d9/d22/d37/f56 0 2024-03-29T04:44:58.092 INFO:tasks.workunit.client.0.smithi142.stdout:7/365: write d7/dc/d4b/d71/d30/f3e [336146,47187] 0 2024-03-29T04:44:58.095 INFO:tasks.workunit.client.0.smithi142.stdout:2/259: mknod de/d11/d1e/d22/d27/c59 0 2024-03-29T04:44:58.097 INFO:tasks.workunit.client.0.smithi142.stdout:7/366: write d7/f43 [1190932,34994] 0 2024-03-29T04:44:58.098 INFO:tasks.workunit.client.0.smithi142.stdout:7/367: read d7/dc/d4b/d71/f28 [1774110,94091] 0 2024-03-29T04:44:58.098 INFO:tasks.workunit.client.0.smithi142.stdout:7/368: dread - d7/f19 zero size 2024-03-29T04:44:58.099 INFO:tasks.workunit.client.0.smithi142.stdout:7/369: chown d7/dc/d4b/d71/d30/l6d 3440 1 2024-03-29T04:44:58.100 INFO:tasks.workunit.client.0.smithi142.stdout:0/273: creat d7/d9/d55/f57 x:0 0 0 2024-03-29T04:44:58.103 INFO:tasks.workunit.client.0.smithi142.stdout:0/274: mkdir d7/d9/d54/d58 0 2024-03-29T04:44:58.103 INFO:tasks.workunit.client.0.smithi142.stdout:2/260: dwrite de/d11/f58 [0,4194304] 0 2024-03-29T04:44:58.105 INFO:tasks.workunit.client.0.smithi142.stdout:2/261: fdatasync de/d11/d17/d4a/f55 0 2024-03-29T04:44:58.106 INFO:tasks.workunit.client.0.smithi142.stdout:2/262: readlink de/d11/d17/l41 0 2024-03-29T04:44:58.107 INFO:tasks.workunit.client.0.smithi142.stdout:0/275: unlink d7/d9/d22/f1b 0 2024-03-29T04:44:58.108 INFO:tasks.workunit.client.0.smithi142.stdout:2/263: fdatasync fd 0 2024-03-29T04:44:58.113 INFO:tasks.workunit.client.0.smithi142.stdout:0/276: getdents d7/d9/d22/d37 0 2024-03-29T04:44:58.115 INFO:tasks.workunit.client.0.smithi142.stdout:0/277: unlink d7/d9/c24 0 2024-03-29T04:44:58.116 INFO:tasks.workunit.client.0.smithi142.stdout:2/264: dwrite de/d11/f4f [0,4194304] 0 2024-03-29T04:44:58.117 INFO:tasks.workunit.client.0.smithi142.stdout:2/265: read f9 [2330906,38895] 0 2024-03-29T04:44:58.120 INFO:tasks.workunit.client.0.smithi142.stdout:2/266: dread de/d11/d23/f28 [0,4194304] 0 2024-03-29T04:44:58.121 INFO:tasks.workunit.client.0.smithi142.stdout:7/370: dread d7/dc/f10 [0,4194304] 0 2024-03-29T04:44:58.122 INFO:tasks.workunit.client.0.smithi142.stdout:0/278: read d7/d9/f1c [351242,88933] 0 2024-03-29T04:44:58.142 INFO:tasks.workunit.client.0.smithi142.stdout:6/293: sync 2024-03-29T04:44:58.145 INFO:tasks.workunit.client.0.smithi142.stdout:6/294: read d1c/f39 [741218,61963] 0 2024-03-29T04:44:58.207 INFO:tasks.workunit.client.0.smithi142.stdout:5/381: sync 2024-03-29T04:44:58.207 INFO:tasks.workunit.client.0.smithi142.stdout:3/262: sync 2024-03-29T04:44:58.207 INFO:tasks.workunit.client.0.smithi142.stdout:9/234: sync 2024-03-29T04:44:58.208 INFO:tasks.workunit.client.0.smithi142.stdout:5/382: fsync df/f14 0 2024-03-29T04:44:58.208 INFO:tasks.workunit.client.0.smithi142.stdout:3/263: chown d3/d1f/d24/d47/f4f 935 1 2024-03-29T04:44:58.211 INFO:tasks.workunit.client.0.smithi142.stdout:9/235: dread d0/d12/f24 [0,4194304] 0 2024-03-29T04:44:58.212 INFO:tasks.workunit.client.0.smithi142.stdout:9/236: fsync d0/f22 0 2024-03-29T04:44:58.213 INFO:tasks.workunit.client.0.smithi142.stdout:5/383: dwrite df/d1a/d21/d27/f36 [0,4194304] 0 2024-03-29T04:44:58.214 INFO:tasks.workunit.client.0.smithi142.stdout:5/384: truncate df/d16/f1e 1480234 0 2024-03-29T04:44:58.257 INFO:tasks.workunit.client.0.smithi142.stdout:8/342: sync 2024-03-29T04:44:58.257 INFO:tasks.workunit.client.0.smithi142.stdout:1/349: sync 2024-03-29T04:44:58.257 INFO:tasks.workunit.client.0.smithi142.stdout:4/265: sync 2024-03-29T04:44:58.283 INFO:tasks.workunit.client.0.smithi142.stdout:7/371: stat d7/d27/d44/d4c/l56 0 2024-03-29T04:44:58.284 INFO:tasks.workunit.client.0.smithi142.stdout:0/279: creat d7/d9/d54/f59 x:0 0 0 2024-03-29T04:44:58.285 INFO:tasks.workunit.client.0.smithi142.stdout:6/295: rmdir d1c/d42/d52 39 2024-03-29T04:44:58.289 INFO:tasks.workunit.client.0.smithi142.stdout:0/280: dread d7/d9/f2e [0,4194304] 0 2024-03-29T04:44:58.295 INFO:tasks.workunit.client.0.smithi142.stdout:3/264: symlink d3/d29/d2b/d40/l5e 0 2024-03-29T04:44:58.295 INFO:tasks.workunit.client.0.smithi142.stdout:3/265: dread - d3/d1f/d24/d47/f57 zero size 2024-03-29T04:44:58.299 INFO:tasks.workunit.client.0.smithi142.stdout:4/266: creat d0/d45/f60 x:0 0 0 2024-03-29T04:44:58.303 INFO:tasks.workunit.client.0.smithi142.stdout:2/267: truncate de/d11/d17/f1a 152685 0 2024-03-29T04:44:58.304 INFO:tasks.workunit.client.0.smithi142.stdout:2/268: readlink de/d11/d23/l47 0 2024-03-29T04:44:58.304 INFO:tasks.workunit.client.0.smithi142.stdout:2/269: stat fd 0 2024-03-29T04:44:58.304 INFO:tasks.workunit.client.0.smithi142.stdout:7/372: mkdir d7/dc/d4b/d71/d30/d72 0 2024-03-29T04:44:58.304 INFO:tasks.workunit.client.0.smithi142.stdout:2/270: chown de/f19 55 1 2024-03-29T04:44:58.306 INFO:tasks.workunit.client.0.smithi142.stdout:6/296: creat d1c/f55 x:0 0 0 2024-03-29T04:44:58.307 INFO:tasks.workunit.client.0.smithi142.stdout:0/281: symlink d7/d9/d22/d37/l5a 0 2024-03-29T04:44:58.308 INFO:tasks.workunit.client.0.smithi142.stdout:3/266: rmdir d3/d29 39 2024-03-29T04:44:58.310 INFO:tasks.workunit.client.0.smithi142.stdout:9/237: truncate d0/d2/d6/d16/f1c 608043 0 2024-03-29T04:44:58.312 INFO:tasks.workunit.client.0.smithi142.stdout:4/267: creat d0/d45/f61 x:0 0 0 2024-03-29T04:44:58.314 INFO:tasks.workunit.client.0.smithi142.stdout:1/350: mknod d8/db/dc/d19/d1d/d2d/d44/d80/c83 0 2024-03-29T04:44:58.315 INFO:tasks.workunit.client.0.smithi142.stdout:7/373: symlink d7/d27/d44/d4c/l73 0 2024-03-29T04:44:58.316 INFO:tasks.workunit.client.0.smithi142.stdout:2/271: mknod de/d11/d23/c5a 0 2024-03-29T04:44:58.317 INFO:tasks.workunit.client.0.smithi142.stdout:6/297: rmdir d1c/d42 39 2024-03-29T04:44:58.319 INFO:tasks.workunit.client.0.smithi142.stdout:3/267: readlink d3/d29/d2b/d40/l5e 0 2024-03-29T04:44:58.320 INFO:tasks.workunit.client.0.smithi142.stdout:3/268: dread - d3/d1f/d24/d47/f4f zero size 2024-03-29T04:44:58.320 INFO:tasks.workunit.client.0.smithi142.stdout:7/374: dwrite f1 [4194304,4194304] 0 2024-03-29T04:44:58.322 INFO:tasks.workunit.client.0.smithi142.stdout:5/385: rename df/d4a to df/d16/d24/d79 0 2024-03-29T04:44:58.323 INFO:tasks.workunit.client.0.smithi142.stdout:9/238: readlink d0/d2/d6/l17 0 2024-03-29T04:44:58.325 INFO:tasks.workunit.client.0.smithi142.stdout:9/239: read - d0/d2/d6/d27/d3c/f4c zero size 2024-03-29T04:44:58.326 INFO:tasks.workunit.client.0.smithi142.stdout:8/343: link d5/c40 d5/d1d/d24/c75 0 2024-03-29T04:44:58.327 INFO:tasks.workunit.client.0.smithi142.stdout:3/269: dwrite d3/d1f/d24/f37 [0,4194304] 0 2024-03-29T04:44:58.327 INFO:tasks.workunit.client.0.smithi142.stdout:8/344: truncate d5/d2e/f65 1941744 0 2024-03-29T04:44:58.328 INFO:tasks.workunit.client.0.smithi142.stdout:8/345: chown d5/d1d/d2f/d5f 2961287 1 2024-03-29T04:44:58.331 INFO:tasks.workunit.client.0.smithi142.stdout:2/272: creat de/d11/d1e/f5b x:0 0 0 2024-03-29T04:44:58.332 INFO:tasks.workunit.client.0.smithi142.stdout:2/273: chown de/d11/d1e/d22/d50 2825467 1 2024-03-29T04:44:58.334 INFO:tasks.workunit.client.0.smithi142.stdout:7/375: mknod d7/dc/d4b/d71/d30/d33/c74 0 2024-03-29T04:44:58.338 INFO:tasks.workunit.client.0.smithi142.stdout:3/270: unlink d3/d1f/d34/d4d/d50/l5b 0 2024-03-29T04:44:58.339 INFO:tasks.workunit.client.0.smithi142.stdout:8/346: rename d5/d10 to d5/d1d/d2f/d76 0 2024-03-29T04:44:58.340 INFO:tasks.workunit.client.0.smithi142.stdout:8/347: chown d5/d1d/d2f/d76/l2a 37428350 1 2024-03-29T04:44:58.340 INFO:tasks.workunit.client.0.smithi142.stdout:0/282: creat d7/d9/d55/f5b x:0 0 0 2024-03-29T04:44:58.341 INFO:tasks.workunit.client.0.smithi142.stdout:7/376: readlink d7/d27/d44/l58 0 2024-03-29T04:44:58.341 INFO:tasks.workunit.client.0.smithi142.stdout:0/283: chown d7/d9/d22/d37/f43 3705 1 2024-03-29T04:44:58.343 INFO:tasks.workunit.client.0.smithi142.stdout:7/377: chown d7/dc/d4b/d71/d30/f4a 933 1 2024-03-29T04:44:58.343 INFO:tasks.workunit.client.0.smithi142.stdout:7/378: dread - d7/f19 zero size 2024-03-29T04:44:58.344 INFO:tasks.workunit.client.0.smithi142.stdout:2/274: mknod de/d11/d1e/d4d/c5c 0 2024-03-29T04:44:58.344 INFO:tasks.workunit.client.0.smithi142.stdout:4/268: getdents d0/d9/d19 0 2024-03-29T04:44:58.346 INFO:tasks.workunit.client.0.smithi142.stdout:5/386: dread f7 [0,4194304] 0 2024-03-29T04:44:58.347 INFO:tasks.workunit.client.0.smithi142.stdout:1/351: getdents d8/db/dc/d19/d1d/d1f/d50 0 2024-03-29T04:44:58.347 INFO:tasks.workunit.client.0.smithi142.stdout:4/269: write d0/d9/d11/f23 [2435789,15351] 0 2024-03-29T04:44:58.348 INFO:tasks.workunit.client.0.smithi142.stdout:3/271: mkdir d3/d5d/d5f 0 2024-03-29T04:44:58.349 INFO:tasks.workunit.client.0.smithi142.stdout:5/387: write df/d2c/f71 [396774,38351] 0 2024-03-29T04:44:58.350 INFO:tasks.workunit.client.0.smithi142.stdout:8/348: creat d5/d1d/d59/f77 x:0 0 0 2024-03-29T04:44:58.350 INFO:tasks.workunit.client.0.smithi142.stdout:1/352: chown d8/db/d37/f76 0 1 2024-03-29T04:44:58.350 INFO:tasks.workunit.client.0.smithi142.stdout:6/298: write d1c/f25 [3153044,39280] 0 2024-03-29T04:44:58.350 INFO:tasks.workunit.client.0.smithi142.stdout:2/275: fsync de/d11/d23/f28 0 2024-03-29T04:44:58.351 INFO:tasks.workunit.client.0.smithi142.stdout:0/284: dwrite d7/d9/d55/d42/f46 [0,4194304] 0 2024-03-29T04:44:58.353 INFO:tasks.workunit.client.0.smithi142.stdout:7/379: creat d7/dc/de/d20/d3f/f75 x:0 0 0 2024-03-29T04:44:58.354 INFO:tasks.workunit.client.0.smithi142.stdout:6/299: dread - d1c/d45/f4e zero size 2024-03-29T04:44:58.354 INFO:tasks.workunit.client.0.smithi142.stdout:9/240: link d0/l42 d0/d12/d50/l51 0 2024-03-29T04:44:58.354 INFO:tasks.workunit.client.0.smithi142.stdout:1/353: chown d8/db/dc/d77/d31/f64 20093 1 2024-03-29T04:44:58.356 INFO:tasks.workunit.client.0.smithi142.stdout:9/241: fsync d0/d2/f2f 0 2024-03-29T04:44:58.358 INFO:tasks.workunit.client.0.smithi142.stdout:3/272: dread d3/d1f/d34/f35 [0,4194304] 0 2024-03-29T04:44:58.359 INFO:tasks.workunit.client.0.smithi142.stdout:9/242: chown d0/d2/d6/d31/f49 660474 1 2024-03-29T04:44:58.360 INFO:tasks.workunit.client.0.smithi142.stdout:1/354: write d8/db/dc/d19/d1d/d2d/d44/d53/f7b [339383,87615] 0 2024-03-29T04:44:58.361 INFO:tasks.workunit.client.0.smithi142.stdout:5/388: dwrite df/d16/f26 [4194304,4194304] 0 2024-03-29T04:44:58.362 INFO:tasks.workunit.client.0.smithi142.stdout:8/349: creat d5/d1d/d2f/d76/d4c/d60/f78 x:0 0 0 2024-03-29T04:44:58.364 INFO:tasks.workunit.client.0.smithi142.stdout:0/285: creat d7/d9/d55/d42/f5c x:0 0 0 2024-03-29T04:44:58.367 INFO:tasks.workunit.client.0.smithi142.stdout:3/273: creat d3/d1f/d34/d4d/f60 x:0 0 0 2024-03-29T04:44:58.368 INFO:tasks.workunit.client.0.smithi142.stdout:1/355: mknod d8/db/dc/d19/d1d/d1f/d50/c84 0 2024-03-29T04:44:58.369 INFO:tasks.workunit.client.0.smithi142.stdout:5/389: mkdir df/d29/d7a 0 2024-03-29T04:44:58.369 INFO:tasks.workunit.client.0.smithi142.stdout:3/274: truncate d3/d54/f58 968677 0 2024-03-29T04:44:58.372 INFO:tasks.workunit.client.0.smithi142.stdout:0/286: symlink d7/d9/d22/l5d 0 2024-03-29T04:44:58.372 INFO:tasks.workunit.client.0.smithi142.stdout:6/300: mknod d1c/d45/d51/c56 0 2024-03-29T04:44:58.373 INFO:tasks.workunit.client.0.smithi142.stdout:9/243: mkdir d0/d12/d50/d52 0 2024-03-29T04:44:58.375 INFO:tasks.workunit.client.0.smithi142.stdout:5/390: symlink df/d29/d6e/l7b 0 2024-03-29T04:44:58.376 INFO:tasks.workunit.client.0.smithi142.stdout:1/356: dwrite d8/db/dc/d77/d5a/f75 [0,4194304] 0 2024-03-29T04:44:58.379 INFO:tasks.workunit.client.0.smithi142.stdout:7/380: link d7/f31 d7/dc/d4b/d71/f76 0 2024-03-29T04:44:58.379 INFO:tasks.workunit.client.0.smithi142.stdout:6/301: rename d1c/l2d to d1c/d33/d46/l57 0 2024-03-29T04:44:58.379 INFO:tasks.workunit.client.0.smithi142.stdout:8/350: link d5/d1d/d2f/d5f/c47 d5/d1d/d2f/d76/d4c/d60/c79 0 2024-03-29T04:44:58.382 INFO:tasks.workunit.client.0.smithi142.stdout:0/287: creat d7/d13/d4a/f5e x:0 0 0 2024-03-29T04:44:58.383 INFO:tasks.workunit.client.0.smithi142.stdout:5/391: fdatasync df/d1a/f2f 0 2024-03-29T04:44:58.383 INFO:tasks.workunit.client.0.smithi142.stdout:0/288: chown d7/d13 13671 1 2024-03-29T04:44:58.385 INFO:tasks.workunit.client.0.smithi142.stdout:4/270: dread d0/d9/d11/f38 [0,4194304] 0 2024-03-29T04:44:58.386 INFO:tasks.workunit.client.0.smithi142.stdout:3/275: link d3/d1f/d34/d6/c1a d3/d1f/d34/d6/c61 0 2024-03-29T04:44:58.386 INFO:tasks.workunit.client.0.smithi142.stdout:9/244: dwrite d0/d2/d6/d16/f40 [0,4194304] 0 2024-03-29T04:44:58.386 INFO:tasks.workunit.client.0.smithi142.stdout:4/271: readlink d0/dc/d15/l2a 0 2024-03-29T04:44:58.388 INFO:tasks.workunit.client.0.smithi142.stdout:6/302: creat d1c/d45/d51/f58 x:0 0 0 2024-03-29T04:44:58.389 INFO:tasks.workunit.client.0.smithi142.stdout:7/381: creat d7/dc/d4b/d71/d30/d33/f77 x:0 0 0 2024-03-29T04:44:58.389 INFO:tasks.workunit.client.0.smithi142.stdout:6/303: fsync d1c/f3c 0 2024-03-29T04:44:58.392 INFO:tasks.workunit.client.0.smithi142.stdout:8/351: creat d5/d1d/d2f/d76/d4c/d60/f7a x:0 0 0 2024-03-29T04:44:58.394 INFO:tasks.workunit.client.0.smithi142.stdout:2/276: truncate de/d11/f58 488568 0 2024-03-29T04:44:58.394 INFO:tasks.workunit.client.0.smithi142.stdout:7/382: write d7/f19 [177739,30754] 0 2024-03-29T04:44:58.396 INFO:tasks.workunit.client.0.smithi142.stdout:5/392: read fa [802734,18315] 0 2024-03-29T04:44:58.397 INFO:tasks.workunit.client.0.smithi142.stdout:7/383: stat d7/d27/d3a 0 2024-03-29T04:44:58.398 INFO:tasks.workunit.client.0.smithi142.stdout:0/289: dwrite d7/d9/f30 [0,4194304] 0 2024-03-29T04:44:58.399 INFO:tasks.workunit.client.0.smithi142.stdout:1/357: rmdir d8/db/d3d 0 2024-03-29T04:44:58.401 INFO:tasks.workunit.client.0.smithi142.stdout:6/304: mknod d1c/d45/d51/c59 0 2024-03-29T04:44:58.403 INFO:tasks.workunit.client.0.smithi142.stdout:6/305: chown d1c/l28 2132836 1 2024-03-29T04:44:58.404 INFO:tasks.workunit.client.0.smithi142.stdout:2/277: mkdir de/d11/d17/d5d 0 2024-03-29T04:44:58.405 INFO:tasks.workunit.client.0.smithi142.stdout:2/278: chown de/d11/d23/l32 452203 1 2024-03-29T04:44:58.407 INFO:tasks.workunit.client.0.smithi142.stdout:8/352: rmdir d5/d1d/d2f/d76/d3c 39 2024-03-29T04:44:58.410 INFO:tasks.workunit.client.0.smithi142.stdout:0/290: read d7/d13/f50 [518942,71925] 0 2024-03-29T04:44:58.411 INFO:tasks.workunit.client.0.smithi142.stdout:1/358: symlink d8/db/d5e/l85 0 2024-03-29T04:44:58.413 INFO:tasks.workunit.client.0.smithi142.stdout:6/306: creat d1c/d33/f5a x:0 0 0 2024-03-29T04:44:58.413 INFO:tasks.workunit.client.0.smithi142.stdout:0/291: write d7/d9/d22/f3a [776336,6553] 0 2024-03-29T04:44:58.413 INFO:tasks.workunit.client.0.smithi142.stdout:8/353: dwrite d5/f30 [0,4194304] 0 2024-03-29T04:44:58.417 INFO:tasks.workunit.client.0.smithi142.stdout:5/393: dread df/d1a/f2f [0,4194304] 0 2024-03-29T04:44:58.417 INFO:tasks.workunit.client.0.smithi142.stdout:0/292: readlink d7/d9/d55/l3b 0 2024-03-29T04:44:58.417 INFO:tasks.workunit.client.0.smithi142.stdout:9/245: dread d0/f38 [0,4194304] 0 2024-03-29T04:44:58.420 INFO:tasks.workunit.client.0.smithi142.stdout:6/307: chown d1c/d33/f36 14 1 2024-03-29T04:44:58.420 INFO:tasks.workunit.client.0.smithi142.stdout:3/276: rename d3/d1f/d34/c3a to d3/c62 0 2024-03-29T04:44:58.422 INFO:tasks.workunit.client.0.smithi142.stdout:3/277: chown d3/d1f/d34/d4d/d50 700650 1 2024-03-29T04:44:58.422 INFO:tasks.workunit.client.0.smithi142.stdout:5/394: unlink df/d1a/f44 0 2024-03-29T04:44:58.423 INFO:tasks.workunit.client.0.smithi142.stdout:6/308: chown f18 171 1 2024-03-29T04:44:58.423 INFO:tasks.workunit.client.0.smithi142.stdout:4/272: write d0/f3 [1949543,84273] 0 2024-03-29T04:44:58.424 INFO:tasks.workunit.client.0.smithi142.stdout:1/359: dread d8/fa [0,4194304] 0 2024-03-29T04:44:58.426 INFO:tasks.workunit.client.0.smithi142.stdout:7/384: creat d7/dc/d4b/d71/f78 x:0 0 0 2024-03-29T04:44:58.427 INFO:tasks.workunit.client.0.smithi142.stdout:8/354: dwrite d5/d1d/d2f/d5f/d48/f61 [0,4194304] 0 2024-03-29T04:44:58.431 INFO:tasks.workunit.client.0.smithi142.stdout:9/246: unlink d0/d12/l4a 0 2024-03-29T04:44:58.434 INFO:tasks.workunit.client.0.smithi142.stdout:3/278: mknod d3/d5d/c63 0 2024-03-29T04:44:58.435 INFO:tasks.workunit.client.0.smithi142.stdout:3/279: write d3/d1f/d24/f37 [1967773,86264] 0 2024-03-29T04:44:58.438 INFO:tasks.workunit.client.0.smithi142.stdout:7/385: creat d7/d27/d44/d4c/f79 x:0 0 0 2024-03-29T04:44:58.439 INFO:tasks.workunit.client.0.smithi142.stdout:7/386: read - d7/dc/d4b/f4f zero size 2024-03-29T04:44:58.441 INFO:tasks.workunit.client.0.smithi142.stdout:8/355: symlink d5/d1d/d2f/d76/d4c/l7b 0 2024-03-29T04:44:58.443 INFO:tasks.workunit.client.0.smithi142.stdout:9/247: unlink d0/d2/fe 0 2024-03-29T04:44:58.444 INFO:tasks.workunit.client.0.smithi142.stdout:2/279: getdents de/d11/d1e 0 2024-03-29T04:44:58.447 INFO:tasks.workunit.client.0.smithi142.stdout:0/293: link d7/d9/d22/d37/f56 d7/d9/d55/d42/f5f 0 2024-03-29T04:44:58.449 INFO:tasks.workunit.client.0.smithi142.stdout:4/273: creat d0/d12/d28/d47/f62 x:0 0 0 2024-03-29T04:44:58.452 INFO:tasks.workunit.client.0.smithi142.stdout:8/356: creat d5/d1d/d59/f7c x:0 0 0 2024-03-29T04:44:58.452 INFO:tasks.workunit.client.0.smithi142.stdout:6/309: dwrite d1c/f39 [0,4194304] 0 2024-03-29T04:44:58.454 INFO:tasks.workunit.client.0.smithi142.stdout:9/248: fsync d0/d2/f2c 0 2024-03-29T04:44:58.455 INFO:tasks.workunit.client.0.smithi142.stdout:0/294: mknod d7/d9/d22/c60 0 2024-03-29T04:44:58.457 INFO:tasks.workunit.client.0.smithi142.stdout:1/360: link d8/db/dc/d77/d31/d4d/c59 d8/db/dc/d19/d1d/d15/c86 0 2024-03-29T04:44:58.462 INFO:tasks.workunit.client.0.smithi142.stdout:8/357: chown d5/d2e/l46 79668775 1 2024-03-29T04:44:58.463 INFO:tasks.workunit.client.0.smithi142.stdout:8/358: write d5/d1d/d24/f33 [299593,10] 0 2024-03-29T04:44:58.464 INFO:tasks.workunit.client.0.smithi142.stdout:6/310: mknod d1c/d45/d51/c5b 0 2024-03-29T04:44:58.464 INFO:tasks.workunit.client.0.smithi142.stdout:8/359: chown d5/d1d/d2f/f50 891269 1 2024-03-29T04:44:58.467 INFO:tasks.workunit.client.0.smithi142.stdout:5/395: getdents df/d1a 0 2024-03-29T04:44:58.468 INFO:tasks.workunit.client.0.smithi142.stdout:0/295: creat d7/d9/d22/f61 x:0 0 0 2024-03-29T04:44:58.469 INFO:tasks.workunit.client.0.smithi142.stdout:0/296: dread - d7/d9/d55/f57 zero size 2024-03-29T04:44:58.469 INFO:tasks.workunit.client.0.smithi142.stdout:3/280: truncate d3/d1f/d24/f37 3124459 0 2024-03-29T04:44:58.472 INFO:tasks.workunit.client.0.smithi142.stdout:7/387: truncate f1 2557643 0 2024-03-29T04:44:58.474 INFO:tasks.workunit.client.0.smithi142.stdout:6/311: mknod d1c/d33/d46/c5c 0 2024-03-29T04:44:58.475 INFO:tasks.workunit.client.0.smithi142.stdout:6/312: dread d1c/f41 [0,4194304] 0 2024-03-29T04:44:58.476 INFO:tasks.workunit.client.0.smithi142.stdout:8/360: mknod d5/d1d/d2f/d76/d4c/d60/c7d 0 2024-03-29T04:44:58.477 INFO:tasks.workunit.client.0.smithi142.stdout:2/280: link de/d11/d17/c2d de/d11/d1e/c5e 0 2024-03-29T04:44:58.478 INFO:tasks.workunit.client.0.smithi142.stdout:9/249: mknod d0/d12/d50/d52/c53 0 2024-03-29T04:44:58.480 INFO:tasks.workunit.client.0.smithi142.stdout:5/396: unlink df/d1a/c1c 0 2024-03-29T04:44:58.481 INFO:tasks.workunit.client.0.smithi142.stdout:1/361: write d8/db/f20 [4611140,79157] 0 2024-03-29T04:44:58.483 INFO:tasks.workunit.client.0.smithi142.stdout:4/274: link d0/d12/l1f d0/d9/d19/l63 0 2024-03-29T04:44:58.485 INFO:tasks.workunit.client.0.smithi142.stdout:7/388: symlink d7/d27/d44/l7a 0 2024-03-29T04:44:58.487 INFO:tasks.workunit.client.0.smithi142.stdout:8/361: creat d5/d1d/d2f/d5f/d48/d52/f7e x:0 0 0 2024-03-29T04:44:58.488 INFO:tasks.workunit.client.0.smithi142.stdout:5/397: dwrite df/d2c/f71 [0,4194304] 0 2024-03-29T04:44:58.489 INFO:tasks.workunit.client.0.smithi142.stdout:7/389: read - d7/dc/d4b/d71/d30/f46 zero size 2024-03-29T04:44:58.490 INFO:tasks.workunit.client.0.smithi142.stdout:9/250: creat d0/d12/f54 x:0 0 0 2024-03-29T04:44:58.491 INFO:tasks.workunit.client.0.smithi142.stdout:6/313: dread d1c/f34 [0,4194304] 0 2024-03-29T04:44:58.492 INFO:tasks.workunit.client.0.smithi142.stdout:1/362: dwrite d8/db/f20 [0,4194304] 0 2024-03-29T04:44:58.495 INFO:tasks.workunit.client.0.smithi142.stdout:2/281: dread f7 [0,4194304] 0 2024-03-29T04:44:58.496 INFO:tasks.workunit.client.0.smithi142.stdout:4/275: rmdir d0/d9/d11 39 2024-03-29T04:44:58.498 INFO:tasks.workunit.client.0.smithi142.stdout:7/390: dread d7/dc/d4b/d71/f28 [0,4194304] 0 2024-03-29T04:44:58.499 INFO:tasks.workunit.client.0.smithi142.stdout:8/362: mknod d5/d1d/d2f/d76/d4c/d60/c7f 0 2024-03-29T04:44:58.504 INFO:tasks.workunit.client.0.smithi142.stdout:9/251: creat d0/d2/d6/d27/d3c/d4f/d34/f55 x:0 0 0 2024-03-29T04:44:58.506 INFO:tasks.workunit.client.0.smithi142.stdout:1/363: creat d8/db/dc/d19/d1d/d1f/d50/f87 x:0 0 0 2024-03-29T04:44:58.508 INFO:tasks.workunit.client.0.smithi142.stdout:1/364: read - d8/db/dc/d19/d1d/d2d/f7d zero size 2024-03-29T04:44:58.509 INFO:tasks.workunit.client.0.smithi142.stdout:5/398: symlink df/d1a/d21/l7c 0 2024-03-29T04:44:58.509 INFO:tasks.workunit.client.0.smithi142.stdout:9/252: dwrite d0/f4b [0,4194304] 0 2024-03-29T04:44:58.511 INFO:tasks.workunit.client.0.smithi142.stdout:7/391: rename d7/f5e to d7/d27/d44/d4c/f7b 0 2024-03-29T04:44:58.511 INFO:tasks.workunit.client.0.smithi142.stdout:0/297: getdents d7/d13 0 2024-03-29T04:44:58.513 INFO:tasks.workunit.client.0.smithi142.stdout:2/282: truncate f9 419191 0 2024-03-29T04:44:58.513 INFO:tasks.workunit.client.0.smithi142.stdout:7/392: readlink d7/dc/d4b/d71/d30/l6d 0 2024-03-29T04:44:58.514 INFO:tasks.workunit.client.0.smithi142.stdout:4/276: mkdir d0/d9/d11/d4c/d64 0 2024-03-29T04:44:58.516 INFO:tasks.workunit.client.0.smithi142.stdout:9/253: creat d0/d2/d6/d27/d3c/d4f/f56 x:0 0 0 2024-03-29T04:44:58.517 INFO:tasks.workunit.client.0.smithi142.stdout:6/314: link d1c/d33/f3e d1c/f5d 0 2024-03-29T04:44:58.518 INFO:tasks.workunit.client.0.smithi142.stdout:8/363: rename d5/d1d/l55 to d5/d1d/d2f/d76/d3c/l80 0 2024-03-29T04:44:58.519 INFO:tasks.workunit.client.0.smithi142.stdout:0/298: fdatasync d7/d9/d55/d42/f5f 0 2024-03-29T04:44:58.520 INFO:tasks.workunit.client.0.smithi142.stdout:8/364: fsync d5/d1d/d2f/f50 0 2024-03-29T04:44:58.520 INFO:tasks.workunit.client.0.smithi142.stdout:4/277: mkdir d0/dc/d15/d3d/d65 0 2024-03-29T04:44:58.522 INFO:tasks.workunit.client.0.smithi142.stdout:9/254: chown d0/d2/d6/l1a 79077 1 2024-03-29T04:44:58.522 INFO:tasks.workunit.client.0.smithi142.stdout:5/399: dwrite df/d1a/f3f [0,4194304] 0 2024-03-29T04:44:58.522 INFO:tasks.workunit.client.0.smithi142.stdout:0/299: write d7/d9/d55/d42/f46 [4865706,73366] 0 2024-03-29T04:44:58.522 INFO:tasks.workunit.client.0.smithi142.stdout:4/278: read - d0/d9/d19/f58 zero size 2024-03-29T04:44:58.525 INFO:tasks.workunit.client.0.smithi142.stdout:4/279: chown d0/l46 1708293 1 2024-03-29T04:44:58.526 INFO:tasks.workunit.client.0.smithi142.stdout:4/280: chown d0/d9/d19/f58 14 1 2024-03-29T04:44:58.526 INFO:tasks.workunit.client.0.smithi142.stdout:4/281: chown d0/d12/d28 160980413 1 2024-03-29T04:44:58.528 INFO:tasks.workunit.client.0.smithi142.stdout:5/400: dread df/d1a/f3f [0,4194304] 0 2024-03-29T04:44:58.532 INFO:tasks.workunit.client.0.smithi142.stdout:5/401: dread df/d16/d25/f37 [0,4194304] 0 2024-03-29T04:44:58.536 INFO:tasks.workunit.client.0.smithi142.stdout:1/365: dread d8/db/dc/d77/d31/d4d/f6e [0,4194304] 0 2024-03-29T04:44:58.538 INFO:tasks.workunit.client.0.smithi142.stdout:5/402: dwrite df/d16/d24/d2e/f6c [0,4194304] 0 2024-03-29T04:44:58.543 INFO:tasks.workunit.client.0.smithi142.stdout:2/283: mknod de/d11/d17/d5d/c5f 0 2024-03-29T04:44:58.543 INFO:tasks.workunit.client.0.smithi142.stdout:5/403: dwrite df/d1a/d28/f49 [4194304,4194304] 0 2024-03-29T04:44:58.545 INFO:tasks.workunit.client.0.smithi142.stdout:5/404: truncate df/d1a/d28/f2d 84266 0 2024-03-29T04:44:58.546 INFO:tasks.workunit.client.0.smithi142.stdout:7/393: fsync d7/dc/de/d20/f40 0 2024-03-29T04:44:58.547 INFO:tasks.workunit.client.0.smithi142.stdout:7/394: fsync d7/f19 0 2024-03-29T04:44:58.548 INFO:tasks.workunit.client.0.smithi142.stdout:5/405: dread df/d16/d24/d79/f5b [0,4194304] 0 2024-03-29T04:44:58.551 INFO:tasks.workunit.client.0.smithi142.stdout:5/406: dread df/d16/f39 [0,4194304] 0 2024-03-29T04:44:58.577 INFO:tasks.workunit.client.0.smithi142.stdout:2/284: mkdir de/d11/d17/d60 0 2024-03-29T04:44:58.578 INFO:tasks.workunit.client.0.smithi142.stdout:0/300: dwrite d7/d9/ff [8388608,4194304] 0 2024-03-29T04:44:58.581 INFO:tasks.workunit.client.0.smithi142.stdout:6/315: truncate d1c/f25 1042902 0 2024-03-29T04:44:58.583 INFO:tasks.workunit.client.0.smithi142.stdout:8/365: mkdir d5/d1d/d81 0 2024-03-29T04:44:58.584 INFO:tasks.workunit.client.0.smithi142.stdout:3/281: sync 2024-03-29T04:44:58.585 INFO:tasks.workunit.client.0.smithi142.stdout:9/255: write d0/d2/d6/d16/f40 [2197501,60594] 0 2024-03-29T04:44:58.585 INFO:tasks.workunit.client.0.smithi142.stdout:8/366: readlink d5/d1d/d2f/d76/d4c/l7b 0 2024-03-29T04:44:58.585 INFO:tasks.workunit.client.0.smithi142.stdout:4/282: symlink d0/d9/l66 0 2024-03-29T04:44:58.586 INFO:tasks.workunit.client.0.smithi142.stdout:2/285: dwrite de/d11/d1e/d22/f26 [0,4194304] 0 2024-03-29T04:44:58.589 INFO:tasks.workunit.client.0.smithi142.stdout:1/366: unlink d8/db/dc/d77/c45 0 2024-03-29T04:44:58.590 INFO:tasks.workunit.client.0.smithi142.stdout:4/283: fsync d0/dc/d15/f59 0 2024-03-29T04:44:58.590 INFO:tasks.workunit.client.0.smithi142.stdout:0/301: dwrite d7/d9/d22/d37/f43 [0,4194304] 0 2024-03-29T04:44:58.592 INFO:tasks.workunit.client.0.smithi142.stdout:7/395: link d7/dc/d4b/d71/f2c d7/dc/de/d20/f7c 0 2024-03-29T04:44:58.593 INFO:tasks.workunit.client.0.smithi142.stdout:6/316: mkdir d1c/d45/d5e 0 2024-03-29T04:44:58.594 INFO:tasks.workunit.client.0.smithi142.stdout:5/407: symlink df/d1a/d21/d55/d66/l7d 0 2024-03-29T04:44:58.600 INFO:tasks.workunit.client.0.smithi142.stdout:2/286: mkdir de/d11/d1e/d4d/d61 0 2024-03-29T04:44:58.605 INFO:tasks.workunit.client.0.smithi142.stdout:1/367: creat d8/db/dc/d19/d1d/d2d/d44/d80/f88 x:0 0 0 2024-03-29T04:44:58.606 INFO:tasks.workunit.client.0.smithi142.stdout:7/396: mknod d7/dc/de/c7d 0 2024-03-29T04:44:58.606 INFO:tasks.workunit.client.0.smithi142.stdout:1/368: chown d8/db/f14 677066586 1 2024-03-29T04:44:58.606 INFO:tasks.workunit.client.0.smithi142.stdout:6/317: truncate f14 1728347 0 2024-03-29T04:44:58.608 INFO:tasks.workunit.client.0.smithi142.stdout:3/282: mkdir d3/d5d/d5f/d64 0 2024-03-29T04:44:58.609 INFO:tasks.workunit.client.0.smithi142.stdout:9/256: symlink d0/l57 0 2024-03-29T04:44:58.610 INFO:tasks.workunit.client.0.smithi142.stdout:8/367: creat d5/d1d/d2f/d76/d3c/d70/f82 x:0 0 0 2024-03-29T04:44:58.611 INFO:tasks.workunit.client.0.smithi142.stdout:2/287: creat de/d11/d1e/d22/d50/f62 x:0 0 0 2024-03-29T04:44:58.612 INFO:tasks.workunit.client.0.smithi142.stdout:2/288: read - de/d11/d1e/f5b zero size 2024-03-29T04:44:58.612 INFO:tasks.workunit.client.0.smithi142.stdout:4/284: link d0/d9/d11/f2b d0/f67 0 2024-03-29T04:44:58.614 INFO:tasks.workunit.client.0.smithi142.stdout:7/397: dwrite d7/dc/de/f34 [0,4194304] 0 2024-03-29T04:44:58.615 INFO:tasks.workunit.client.0.smithi142.stdout:7/398: chown d7/dc/d4b/d71/l6a 15 1 2024-03-29T04:44:58.619 INFO:tasks.workunit.client.0.smithi142.stdout:6/318: fsync d1c/f41 0 2024-03-29T04:44:58.624 INFO:tasks.workunit.client.0.smithi142.stdout:3/283: unlink d3/d5d/c63 0 2024-03-29T04:44:58.626 INFO:tasks.workunit.client.0.smithi142.stdout:0/302: creat d7/d9/d55/f62 x:0 0 0 2024-03-29T04:44:58.627 INFO:tasks.workunit.client.0.smithi142.stdout:8/368: unlink d5/d1d/d2f/d76/f32 0 2024-03-29T04:44:58.629 INFO:tasks.workunit.client.0.smithi142.stdout:8/369: write d5/d1d/d2f/d76/d2d/f57 [435430,85625] 0 2024-03-29T04:44:58.631 INFO:tasks.workunit.client.0.smithi142.stdout:9/257: dwrite d0/d2/f1f [4194304,4194304] 0 2024-03-29T04:44:58.631 INFO:tasks.workunit.client.0.smithi142.stdout:7/399: symlink d7/dc/de/d20/l7e 0 2024-03-29T04:44:58.632 INFO:tasks.workunit.client.0.smithi142.stdout:9/258: stat d0/f38 0 2024-03-29T04:44:58.634 INFO:tasks.workunit.client.0.smithi142.stdout:6/319: unlink l13 0 2024-03-29T04:44:58.634 INFO:tasks.workunit.client.0.smithi142.stdout:5/408: link cc df/d1a/d21/d55/c7e 0 2024-03-29T04:44:58.635 INFO:tasks.workunit.client.0.smithi142.stdout:6/320: fsync d1c/f55 0 2024-03-29T04:44:58.639 INFO:tasks.workunit.client.0.smithi142.stdout:5/409: dread df/d29/f67 [0,4194304] 0 2024-03-29T04:44:58.640 INFO:tasks.workunit.client.0.smithi142.stdout:9/259: dwrite d0/d2/d6/d31/f3a [0,4194304] 0 2024-03-29T04:44:58.644 INFO:tasks.workunit.client.0.smithi142.stdout:3/284: dread d3/d1f/d34/d4d/d50/f51 [0,4194304] 0 2024-03-29T04:44:58.650 INFO:tasks.workunit.client.0.smithi142.stdout:6/321: fdatasync d1c/f49 0 2024-03-29T04:44:58.654 INFO:tasks.workunit.client.0.smithi142.stdout:4/285: rename d0/dc/f17 to d0/dc/f68 0 2024-03-29T04:44:58.654 INFO:tasks.workunit.client.0.smithi142.stdout:1/369: truncate d8/db/f25 1154837 0 2024-03-29T04:44:58.655 INFO:tasks.workunit.client.0.smithi142.stdout:0/303: mknod d7/d9/c63 0 2024-03-29T04:44:58.657 INFO:tasks.workunit.client.0.smithi142.stdout:3/285: mknod d3/d29/d2d/c65 0 2024-03-29T04:44:58.659 INFO:tasks.workunit.client.0.smithi142.stdout:7/400: write d7/dc/d4b/d71/f76 [899819,80746] 0 2024-03-29T04:44:58.661 INFO:tasks.workunit.client.0.smithi142.stdout:5/410: unlink df/d1a/d21/d55/l5a 0 2024-03-29T04:44:58.661 INFO:tasks.workunit.client.0.smithi142.stdout:4/286: dwrite d0/f48 [0,4194304] 0 2024-03-29T04:44:58.662 INFO:tasks.workunit.client.0.smithi142.stdout:5/411: chown df/d1a/d21 276036 1 2024-03-29T04:44:58.663 INFO:tasks.workunit.client.0.smithi142.stdout:4/287: dread - d0/d45/f60 zero size 2024-03-29T04:44:58.666 INFO:tasks.workunit.client.0.smithi142.stdout:4/288: chown d0/f5a 336 1 2024-03-29T04:44:58.666 INFO:tasks.workunit.client.0.smithi142.stdout:1/370: symlink d8/db/dc/d19/d1d/d1f/d50/l89 0 2024-03-29T04:44:58.668 INFO:tasks.workunit.client.0.smithi142.stdout:6/322: chown fb 296 1 2024-03-29T04:44:58.669 INFO:tasks.workunit.client.0.smithi142.stdout:7/401: dwrite d7/d27/d44/f60 [0,4194304] 0 2024-03-29T04:44:58.670 INFO:tasks.workunit.client.0.smithi142.stdout:2/289: getdents de/d11/d1e 0 2024-03-29T04:44:58.671 INFO:tasks.workunit.client.0.smithi142.stdout:2/290: readlink l4 0 2024-03-29T04:44:58.671 INFO:tasks.workunit.client.0.smithi142.stdout:0/304: fdatasync d7/d9/d55/d42/f5f 0 2024-03-29T04:44:58.672 INFO:tasks.workunit.client.0.smithi142.stdout:0/305: readlink d7/l12 0 2024-03-29T04:44:58.673 INFO:tasks.workunit.client.0.smithi142.stdout:9/260: rename d0/d2/f2c to d0/d2/d6/d16/f58 0 2024-03-29T04:44:58.676 INFO:tasks.workunit.client.0.smithi142.stdout:4/289: unlink d0/f3e 0 2024-03-29T04:44:58.676 INFO:tasks.workunit.client.0.smithi142.stdout:4/290: dread - d0/d12/d28/f54 zero size 2024-03-29T04:44:58.677 INFO:tasks.workunit.client.0.smithi142.stdout:4/291: truncate d0/d12/f3c 598413 0 2024-03-29T04:44:58.683 INFO:tasks.workunit.client.0.smithi142.stdout:2/291: fdatasync de/d11/d1e/d22/f3f 0 2024-03-29T04:44:58.683 INFO:tasks.workunit.client.0.smithi142.stdout:0/306: creat d7/d9/d54/f64 x:0 0 0 2024-03-29T04:44:58.685 INFO:tasks.workunit.client.0.smithi142.stdout:4/292: dwrite d0/f21 [0,4194304] 0 2024-03-29T04:44:58.686 INFO:tasks.workunit.client.0.smithi142.stdout:1/371: dwrite d8/db/dc/d77/d31/d4d/f6e [0,4194304] 0 2024-03-29T04:44:58.686 INFO:tasks.workunit.client.0.smithi142.stdout:2/292: write de/d11/d1e/d22/d27/f45 [1973160,128475] 0 2024-03-29T04:44:58.687 INFO:tasks.workunit.client.0.smithi142.stdout:4/293: write d0/dc/d15/f53 [2076848,16102] 0 2024-03-29T04:44:58.688 INFO:tasks.workunit.client.0.smithi142.stdout:3/286: rename d3/d1f/d24/l31 to d3/d29/d2b/d40/l66 0 2024-03-29T04:44:58.688 INFO:tasks.workunit.client.0.smithi142.stdout:2/293: write de/f36 [3179029,74869] 0 2024-03-29T04:44:58.690 INFO:tasks.workunit.client.0.smithi142.stdout:3/287: fsync d3/d1f/d24/d47/f57 0 2024-03-29T04:44:58.690 INFO:tasks.workunit.client.0.smithi142.stdout:3/288: stat d3/d1f 0 2024-03-29T04:44:58.693 INFO:tasks.workunit.client.0.smithi142.stdout:7/402: mknod d7/dc/d4b/d71/c7f 0 2024-03-29T04:44:58.694 INFO:tasks.workunit.client.0.smithi142.stdout:0/307: creat d7/d9/d54/f65 x:0 0 0 2024-03-29T04:44:58.695 INFO:tasks.workunit.client.0.smithi142.stdout:0/308: chown d7/d9/d22/d37/c33 22982 1 2024-03-29T04:44:58.696 INFO:tasks.workunit.client.0.smithi142.stdout:0/309: stat d7/d9/d55/d42/c4d 0 2024-03-29T04:44:58.698 INFO:tasks.workunit.client.0.smithi142.stdout:1/372: write d8/db/dc/d77/d31/d4d/f7a [35536,118198] 0 2024-03-29T04:44:58.699 INFO:tasks.workunit.client.0.smithi142.stdout:6/323: mkdir d1c/d45/d5e/d5f 0 2024-03-29T04:44:58.700 INFO:tasks.workunit.client.0.smithi142.stdout:4/294: creat d0/dc/d15/d3d/f69 x:0 0 0 2024-03-29T04:44:58.701 INFO:tasks.workunit.client.0.smithi142.stdout:3/289: write d3/d1f/d34/f35 [2054031,59732] 0 2024-03-29T04:44:58.701 INFO:tasks.workunit.client.0.smithi142.stdout:0/310: dread d7/d9/f30 [0,4194304] 0 2024-03-29T04:44:58.704 INFO:tasks.workunit.client.0.smithi142.stdout:2/294: symlink de/d11/d17/d60/l63 0 2024-03-29T04:44:58.704 INFO:tasks.workunit.client.0.smithi142.stdout:2/295: readlink de/l2a 0 2024-03-29T04:44:58.705 INFO:tasks.workunit.client.0.smithi142.stdout:1/373: creat d8/db/d37/f8a x:0 0 0 2024-03-29T04:44:58.707 INFO:tasks.workunit.client.0.smithi142.stdout:7/403: link d7/f31 d7/d27/d44/d4c/f80 0 2024-03-29T04:44:58.708 INFO:tasks.workunit.client.0.smithi142.stdout:3/290: read d3/d1f/d24/f37 [2207496,66657] 0 2024-03-29T04:44:58.709 INFO:tasks.workunit.client.0.smithi142.stdout:6/324: truncate f14 2423863 0 2024-03-29T04:44:58.711 INFO:tasks.workunit.client.0.smithi142.stdout:4/295: dwrite d0/d12/d28/d47/f62 [0,4194304] 0 2024-03-29T04:44:58.712 INFO:tasks.workunit.client.0.smithi142.stdout:2/296: creat de/d11/d1e/d22/d50/f64 x:0 0 0 2024-03-29T04:44:58.714 INFO:tasks.workunit.client.0.smithi142.stdout:0/311: dwrite d7/d9/d22/f61 [0,4194304] 0 2024-03-29T04:44:58.714 INFO:tasks.workunit.client.0.smithi142.stdout:7/404: creat d7/d27/f81 x:0 0 0 2024-03-29T04:44:58.716 INFO:tasks.workunit.client.0.smithi142.stdout:4/296: write d0/f21 [3494996,68849] 0 2024-03-29T04:44:58.721 INFO:tasks.workunit.client.0.smithi142.stdout:2/297: unlink de/f42 0 2024-03-29T04:44:58.722 INFO:tasks.workunit.client.0.smithi142.stdout:4/297: fdatasync d0/d12/d28/f3a 0 2024-03-29T04:44:58.724 INFO:tasks.workunit.client.0.smithi142.stdout:6/325: creat d1c/d45/d5e/d5f/f60 x:0 0 0 2024-03-29T04:44:58.726 INFO:tasks.workunit.client.0.smithi142.stdout:3/291: dwrite d3/d1f/d34/f3e [0,4194304] 0 2024-03-29T04:44:58.726 INFO:tasks.workunit.client.0.smithi142.stdout:0/312: rename d7/d9/d22/c47 to d7/d9/d54/d58/c66 0 2024-03-29T04:44:58.728 INFO:tasks.workunit.client.0.smithi142.stdout:4/298: dread d0/f21 [0,4194304] 0 2024-03-29T04:44:58.729 INFO:tasks.workunit.client.0.smithi142.stdout:2/298: rmdir de/d11/d1e/d22/d50 39 2024-03-29T04:44:58.730 INFO:tasks.workunit.client.0.smithi142.stdout:4/299: write d0/d9/d11/f1b [497489,56853] 0 2024-03-29T04:44:58.730 INFO:tasks.workunit.client.0.smithi142.stdout:7/405: creat d7/dc/d4b/f82 x:0 0 0 2024-03-29T04:44:58.731 INFO:tasks.workunit.client.0.smithi142.stdout:7/406: chown d7/dc/de/f26 78 1 2024-03-29T04:44:58.732 INFO:tasks.workunit.client.0.smithi142.stdout:7/407: chown d7/dc/de/d20/f51 5287 1 2024-03-29T04:44:58.733 INFO:tasks.workunit.client.0.smithi142.stdout:0/313: mkdir d7/d13/d67 0 2024-03-29T04:44:58.735 INFO:tasks.workunit.client.0.smithi142.stdout:3/292: unlink d3/d1f/d34/d6/f14 0 2024-03-29T04:44:58.737 INFO:tasks.workunit.client.0.smithi142.stdout:2/299: creat de/d11/d3d/f65 x:0 0 0 2024-03-29T04:44:58.739 INFO:tasks.workunit.client.0.smithi142.stdout:4/300: rename d0/d9/d19/f39 to d0/d45/f6a 0 2024-03-29T04:44:58.740 INFO:tasks.workunit.client.0.smithi142.stdout:7/408: mknod d7/dc/de/d20/c83 0 2024-03-29T04:44:58.741 INFO:tasks.workunit.client.0.smithi142.stdout:7/409: truncate d7/d27/f6e 136244 0 2024-03-29T04:44:58.742 INFO:tasks.workunit.client.0.smithi142.stdout:1/374: truncate d8/db/dc/d19/f5c 1534596 0 2024-03-29T04:44:58.742 INFO:tasks.workunit.client.0.smithi142.stdout:0/314: rmdir d7/d9/d54/d58 39 2024-03-29T04:44:58.742 INFO:tasks.workunit.client.0.smithi142.stdout:4/301: dread d0/d12/f3b [0,4194304] 0 2024-03-29T04:44:58.744 INFO:tasks.workunit.client.0.smithi142.stdout:4/302: read d0/d9/d19/f1a [2188620,81746] 0 2024-03-29T04:44:58.745 INFO:tasks.workunit.client.0.smithi142.stdout:3/293: mkdir d3/d29/d2b/d40/d67 0 2024-03-29T04:44:58.746 INFO:tasks.workunit.client.0.smithi142.stdout:2/300: creat de/d11/d1e/d22/d27/f66 x:0 0 0 2024-03-29T04:44:58.747 INFO:tasks.workunit.client.0.smithi142.stdout:3/294: read - d3/d1f/d34/d4d/f60 zero size 2024-03-29T04:44:58.789 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:44:58 smithi152 ceph-mon[37177]: pgmap v265: 65 pgs: 65 active+clean; 476 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 38 MiB/s wr, 244 op/s 2024-03-29T04:44:58.802 INFO:tasks.workunit.client.0.smithi142.stdout:6/326: write fb [2261691,20824] 0 2024-03-29T04:44:58.803 INFO:tasks.workunit.client.0.smithi142.stdout:7/410: creat d7/dc/d4b/d71/f84 x:0 0 0 2024-03-29T04:44:58.803 INFO:tasks.workunit.client.0.smithi142.stdout:0/315: dwrite d7/f52 [0,4194304] 0 2024-03-29T04:44:58.804 INFO:tasks.workunit.client.0.smithi142.stdout:1/375: dwrite d8/db/dc/d19/d1d/d15/f60 [0,4194304] 0 2024-03-29T04:44:58.805 INFO:tasks.workunit.client.0.smithi142.stdout:0/316: chown d7/c17 421886 1 2024-03-29T04:44:58.807 INFO:tasks.workunit.client.0.smithi142.stdout:6/327: mkdir d1c/d33/d46/d61 0 2024-03-29T04:44:58.810 INFO:tasks.workunit.client.0.smithi142.stdout:0/317: dread d7/d9/d22/d37/f43 [0,4194304] 0 2024-03-29T04:44:58.810 INFO:tasks.workunit.client.0.smithi142.stdout:3/295: link d3/d1f/d34/d4d/d50/f51 d3/d1f/d34/d6/f68 0 2024-03-29T04:44:58.811 INFO:tasks.workunit.client.0.smithi142.stdout:6/328: creat d1c/f62 x:0 0 0 2024-03-29T04:44:58.816 INFO:tasks.workunit.client.0.smithi142.stdout:3/296: unlink d3/d1f/d34/f11 0 2024-03-29T04:44:58.817 INFO:tasks.workunit.client.0.smithi142.stdout:3/297: chown d3/d1f/f46 4874003 1 2024-03-29T04:44:58.818 INFO:tasks.workunit.client.0.smithi142.stdout:6/329: symlink d1c/l63 0 2024-03-29T04:44:58.822 INFO:tasks.workunit.client.0.smithi142.stdout:3/298: getdents d3/d1f 0 2024-03-29T04:44:58.822 INFO:tasks.workunit.client.0.smithi142.stdout:6/330: read d1c/d42/f4c [71122,80144] 0 2024-03-29T04:44:58.824 INFO:tasks.workunit.client.0.smithi142.stdout:3/299: creat d3/d1f/d24/f69 x:0 0 0 2024-03-29T04:44:58.824 INFO:tasks.workunit.client.0.smithi142.stdout:6/331: mknod d1c/d45/d5e/d5f/c64 0 2024-03-29T04:44:58.826 INFO:tasks.workunit.client.0.smithi142.stdout:3/300: unlink d3/d1f/d34/f27 0 2024-03-29T04:44:58.870 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:44:58 smithi142 ceph-mon[29027]: pgmap v265: 65 pgs: 65 active+clean; 476 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 38 MiB/s wr, 244 op/s 2024-03-29T04:44:58.883 INFO:tasks.workunit.client.0.smithi142.stdout:0/318: dread d7/d9/d22/f3e [0,4194304] 0 2024-03-29T04:44:58.885 INFO:tasks.workunit.client.0.smithi142.stdout:0/319: creat d7/d9/f68 x:0 0 0 2024-03-29T04:44:58.888 INFO:tasks.workunit.client.0.smithi142.stdout:0/320: link d7/d9/d55/c34 d7/d9/d22/c69 0 2024-03-29T04:44:58.889 INFO:tasks.workunit.client.0.smithi142.stdout:0/321: truncate d7/d9/f30 2996851 0 2024-03-29T04:44:58.892 INFO:tasks.workunit.client.0.smithi142.stdout:0/322: getdents d7/d9/d1e 0 2024-03-29T04:44:58.895 INFO:tasks.workunit.client.0.smithi142.stdout:0/323: rename d7/d9/d55/c34 to d7/d13/d67/c6a 0 2024-03-29T04:44:58.897 INFO:tasks.workunit.client.0.smithi142.stdout:0/324: truncate d7/d13/f50 1571912 0 2024-03-29T04:44:58.897 INFO:tasks.workunit.client.0.smithi142.stdout:0/325: stat d7/d9/d1e 0 2024-03-29T04:44:58.899 INFO:tasks.workunit.client.0.smithi142.stdout:0/326: rmdir d7/d13/d4a 39 2024-03-29T04:44:58.900 INFO:tasks.workunit.client.0.smithi142.stdout:0/327: write d7/d13/f28 [2065464,73496] 0 2024-03-29T04:44:58.901 INFO:tasks.workunit.client.0.smithi142.stdout:0/328: dread - d7/d9/d54/f65 zero size 2024-03-29T04:44:58.905 INFO:tasks.workunit.client.0.smithi142.stdout:0/329: rename d7/d9/d1e/f26 to d7/d9/d54/d58/f6b 0 2024-03-29T04:44:58.907 INFO:tasks.workunit.client.0.smithi142.stdout:0/330: creat d7/d9/d1e/f6c x:0 0 0 2024-03-29T04:44:58.909 INFO:tasks.workunit.client.0.smithi142.stdout:0/331: read d7/d9/f1c [435414,94439] 0 2024-03-29T04:44:58.914 INFO:tasks.workunit.client.0.smithi142.stdout:0/332: dwrite d7/d9/d55/d42/f46 [0,4194304] 0 2024-03-29T04:44:58.914 INFO:tasks.workunit.client.0.smithi142.stdout:0/333: chown d7/d9/d55/f5b 0 1 2024-03-29T04:44:58.918 INFO:tasks.workunit.client.0.smithi142.stdout:0/334: getdents d7 0 2024-03-29T04:44:58.918 INFO:tasks.workunit.client.0.smithi142.stdout:0/335: dread - d7/d9/d55/f5b zero size 2024-03-29T04:44:58.921 INFO:tasks.workunit.client.0.smithi142.stdout:8/370: sync 2024-03-29T04:44:58.921 INFO:tasks.workunit.client.0.smithi142.stdout:5/412: sync 2024-03-29T04:44:58.921 INFO:tasks.workunit.client.0.smithi142.stdout:9/261: sync 2024-03-29T04:44:58.922 INFO:tasks.workunit.client.0.smithi142.stdout:0/336: rename d7/d9/d55/l3b to d7/d9/d54/d58/l6d 0 2024-03-29T04:44:58.923 INFO:tasks.workunit.client.0.smithi142.stdout:9/262: write d0/f1d [2670981,104889] 0 2024-03-29T04:44:58.923 INFO:tasks.workunit.client.0.smithi142.stdout:0/337: stat d7/d9/d55/d42/f46 0 2024-03-29T04:44:58.924 INFO:tasks.workunit.client.0.smithi142.stdout:0/338: write d7/f52 [1620165,63526] 0 2024-03-29T04:44:58.926 INFO:tasks.workunit.client.0.smithi142.stdout:5/413: mknod df/d16/d24/d2e/d47/d4d/c7f 0 2024-03-29T04:44:58.930 INFO:tasks.workunit.client.0.smithi142.stdout:9/263: rename d0/d2/d6/fd to d0/d12/f59 0 2024-03-29T04:44:58.931 INFO:tasks.workunit.client.0.smithi142.stdout:9/264: write d0/d2/d6/d31/f49 [232235,75265] 0 2024-03-29T04:44:58.932 INFO:tasks.workunit.client.0.smithi142.stdout:5/414: dwrite df/f10 [0,4194304] 0 2024-03-29T04:44:58.932 INFO:tasks.workunit.client.0.smithi142.stdout:9/265: chown d0/d2/d6/d27 516230367 1 2024-03-29T04:44:58.937 INFO:tasks.workunit.client.0.smithi142.stdout:9/266: symlink d0/d12/d50/d52/l5a 0 2024-03-29T04:44:58.937 INFO:tasks.workunit.client.0.smithi142.stdout:0/339: truncate d7/d9/d22/d37/f38 2379342 0 2024-03-29T04:44:58.938 INFO:tasks.workunit.client.0.smithi142.stdout:5/415: symlink df/d29/d7a/l80 0 2024-03-29T04:44:58.940 INFO:tasks.workunit.client.0.smithi142.stdout:8/371: dwrite d5/d1d/d2f/d76/d2d/f56 [0,4194304] 0 2024-03-29T04:44:58.942 INFO:tasks.workunit.client.0.smithi142.stdout:8/372: chown d5/l8 118 1 2024-03-29T04:44:58.949 INFO:tasks.workunit.client.0.smithi142.stdout:8/373: chown d5/d1d/d2f/d76/d3c/c49 4024369 1 2024-03-29T04:44:58.950 INFO:tasks.workunit.client.0.smithi142.stdout:0/340: dwrite d7/d9/f68 [0,4194304] 0 2024-03-29T04:44:58.953 INFO:tasks.workunit.client.0.smithi142.stdout:5/416: dwrite df/d16/f26 [0,4194304] 0 2024-03-29T04:44:58.953 INFO:tasks.workunit.client.0.smithi142.stdout:9/267: dwrite d0/d12/f43 [0,4194304] 0 2024-03-29T04:44:58.954 INFO:tasks.workunit.client.0.smithi142.stdout:8/374: chown d5/d1d/d2f/d76/d3c/l3d 10079766 1 2024-03-29T04:44:58.957 INFO:tasks.workunit.client.0.smithi142.stdout:0/341: unlink d7/d9/d1e/f6c 0 2024-03-29T04:44:58.960 INFO:tasks.workunit.client.0.smithi142.stdout:5/417: creat df/d1a/d21/d55/f81 x:0 0 0 2024-03-29T04:44:58.961 INFO:tasks.workunit.client.0.smithi142.stdout:9/268: fsync d0/d2/d6/d16/f37 0 2024-03-29T04:44:58.962 INFO:tasks.workunit.client.0.smithi142.stdout:8/375: symlink d5/l83 0 2024-03-29T04:44:58.968 INFO:tasks.workunit.client.0.smithi142.stdout:9/269: link d0/d12/d50/d52/l5a d0/d2/d6/d27/d3c/d4f/d34/l5b 0 2024-03-29T04:44:58.968 INFO:tasks.workunit.client.0.smithi142.stdout:4/303: sync 2024-03-29T04:44:58.969 INFO:tasks.workunit.client.0.smithi142.stdout:4/304: truncate d0/dc/d15/f59 894944 0 2024-03-29T04:44:58.971 INFO:tasks.workunit.client.0.smithi142.stdout:4/305: dread d0/dc/d15/f59 [0,4194304] 0 2024-03-29T04:44:58.973 INFO:tasks.workunit.client.0.smithi142.stdout:9/270: rename d0/d12/d50/l51 to d0/d2/d6/d27/d3c/d4f/l5c 0 2024-03-29T04:44:58.973 INFO:tasks.workunit.client.0.smithi142.stdout:9/271: read d0/d2/d6/d16/f40 [4059270,48086] 0 2024-03-29T04:44:58.975 INFO:tasks.workunit.client.0.smithi142.stdout:9/272: fsync d0/d2/d6/d16/f2b 0 2024-03-29T04:44:58.977 INFO:tasks.workunit.client.0.smithi142.stdout:9/273: symlink d0/d2/d6/d27/l5d 0 2024-03-29T04:44:58.982 INFO:tasks.workunit.client.0.smithi142.stdout:8/376: write d5/f19 [1508489,62606] 0 2024-03-29T04:44:58.986 INFO:tasks.workunit.client.0.smithi142.stdout:9/274: dwrite d0/d2/d6/d27/d3c/d4f/f56 [0,4194304] 0 2024-03-29T04:44:58.988 INFO:tasks.workunit.client.0.smithi142.stdout:9/275: write d0/d2/d6/d27/f48 [718807,1182] 0 2024-03-29T04:44:58.990 INFO:tasks.workunit.client.0.smithi142.stdout:5/418: dread fa [0,4194304] 0 2024-03-29T04:44:58.991 INFO:tasks.workunit.client.0.smithi142.stdout:8/377: dread d5/d1d/d2f/d76/d2d/f56 [0,4194304] 0 2024-03-29T04:44:58.991 INFO:tasks.workunit.client.0.smithi142.stdout:5/419: write df/d2c/f40 [1269315,50745] 0 2024-03-29T04:44:58.991 INFO:tasks.workunit.client.0.smithi142.stdout:9/276: mknod d0/c5e 0 2024-03-29T04:44:58.992 INFO:tasks.workunit.client.0.smithi142.stdout:9/277: dread - d0/d12/f54 zero size 2024-03-29T04:44:58.994 INFO:tasks.workunit.client.0.smithi142.stdout:8/378: unlink d5/d2e/f37 0 2024-03-29T04:44:58.994 INFO:tasks.workunit.client.0.smithi142.stdout:8/379: dread - d5/d1d/d2f/f68 zero size 2024-03-29T04:44:58.997 INFO:tasks.workunit.client.0.smithi142.stdout:9/278: write d0/f38 [2929496,36202] 0 2024-03-29T04:44:58.998 INFO:tasks.workunit.client.0.smithi142.stdout:8/380: unlink d5/d1d/d2f/d76/d4c/d60/c7f 0 2024-03-29T04:44:58.998 INFO:tasks.workunit.client.0.smithi142.stdout:9/279: truncate d0/f1d 5395904 0 2024-03-29T04:44:59.000 INFO:tasks.workunit.client.0.smithi142.stdout:9/280: unlink d0/f38 0 2024-03-29T04:44:59.000 INFO:tasks.workunit.client.0.smithi142.stdout:5/420: link fa df/d29/f82 0 2024-03-29T04:44:59.001 INFO:tasks.workunit.client.0.smithi142.stdout:8/381: creat d5/d1d/d2f/d76/d4c/d60/f84 x:0 0 0 2024-03-29T04:44:59.002 INFO:tasks.workunit.client.0.smithi142.stdout:8/382: chown d5/d2e/l46 5673 1 2024-03-29T04:44:59.003 INFO:tasks.workunit.client.0.smithi142.stdout:5/421: symlink df/d16/l83 0 2024-03-29T04:44:59.004 INFO:tasks.workunit.client.0.smithi142.stdout:9/281: unlink d0/d12/d50/d52/l5a 0 2024-03-29T04:44:59.005 INFO:tasks.workunit.client.0.smithi142.stdout:5/422: symlink df/d1a/d21/d55/d66/l84 0 2024-03-29T04:44:59.008 INFO:tasks.workunit.client.0.smithi142.stdout:5/423: write df/d1a/f3f [2054048,26406] 0 2024-03-29T04:44:59.008 INFO:tasks.workunit.client.0.smithi142.stdout:8/383: creat d5/d1d/f85 x:0 0 0 2024-03-29T04:44:59.010 INFO:tasks.workunit.client.0.smithi142.stdout:8/384: unlink d5/l83 0 2024-03-29T04:44:59.012 INFO:tasks.workunit.client.0.smithi142.stdout:8/385: write d5/d1d/d2f/f54 [237206,26320] 0 2024-03-29T04:44:59.012 INFO:tasks.workunit.client.0.smithi142.stdout:9/282: dwrite d0/d2/d6/d16/f58 [0,4194304] 0 2024-03-29T04:44:59.013 INFO:tasks.workunit.client.0.smithi142.stdout:9/283: fsync d0/f32 0 2024-03-29T04:44:59.014 INFO:tasks.workunit.client.0.smithi142.stdout:8/386: rmdir d5/d1d/d2f/d5f 39 2024-03-29T04:44:59.017 INFO:tasks.workunit.client.0.smithi142.stdout:9/284: dread d0/f4b [0,4194304] 0 2024-03-29T04:44:59.017 INFO:tasks.workunit.client.0.smithi142.stdout:9/285: fdatasync d0/f22 0 2024-03-29T04:44:59.018 INFO:tasks.workunit.client.0.smithi142.stdout:8/387: unlink d5/d1d/d2f/d76/d3c/l3d 0 2024-03-29T04:44:59.018 INFO:tasks.workunit.client.0.smithi142.stdout:8/388: chown d5/d1d/d2f/d76/d3c/f5e 69617229 1 2024-03-29T04:44:59.021 INFO:tasks.workunit.client.0.smithi142.stdout:5/424: rmdir df/d1a/d21 39 2024-03-29T04:44:59.022 INFO:tasks.workunit.client.0.smithi142.stdout:9/286: symlink d0/d2/d6/l5f 0 2024-03-29T04:44:59.025 INFO:tasks.workunit.client.0.smithi142.stdout:5/425: mknod df/d29/d6e/c85 0 2024-03-29T04:44:59.026 INFO:tasks.workunit.client.0.smithi142.stdout:9/287: creat d0/d12/f60 x:0 0 0 2024-03-29T04:44:59.030 INFO:tasks.workunit.client.0.smithi142.stdout:9/288: unlink d0/c8 0 2024-03-29T04:44:59.031 INFO:tasks.workunit.client.0.smithi142.stdout:8/389: getdents d5/d1d/d24 0 2024-03-29T04:44:59.032 INFO:tasks.workunit.client.0.smithi142.stdout:5/426: symlink df/d1a/d21/d27/l86 0 2024-03-29T04:44:59.033 INFO:tasks.workunit.client.0.smithi142.stdout:5/427: stat df/d16/d25/f59 0 2024-03-29T04:44:59.036 INFO:tasks.workunit.client.0.smithi142.stdout:9/289: dwrite d0/f1d [4194304,4194304] 0 2024-03-29T04:44:59.036 INFO:tasks.workunit.client.0.smithi142.stdout:8/390: getdents d5/d1d/d2f/d76/d3c/d70 0 2024-03-29T04:44:59.037 INFO:tasks.workunit.client.0.smithi142.stdout:8/391: chown d5/d1d/d2f/d76/d4c/d60/f84 0 1 2024-03-29T04:44:59.039 INFO:tasks.workunit.client.0.smithi142.stdout:5/428: symlink df/d16/d24/d2e/l87 0 2024-03-29T04:44:59.040 INFO:tasks.workunit.client.0.smithi142.stdout:9/290: readlink d0/l42 0 2024-03-29T04:44:59.041 INFO:tasks.workunit.client.0.smithi142.stdout:9/291: truncate d0/d2/f11 4759736 0 2024-03-29T04:44:59.042 INFO:tasks.workunit.client.0.smithi142.stdout:8/392: readlink d5/d1d/d2f/d5f/l44 0 2024-03-29T04:44:59.046 INFO:tasks.workunit.client.0.smithi142.stdout:9/292: read - d0/d2/d6/d27/d3c/d4f/d34/f55 zero size 2024-03-29T04:44:59.046 INFO:tasks.workunit.client.0.smithi142.stdout:5/429: link f7 df/d16/d24/f88 0 2024-03-29T04:44:59.048 INFO:tasks.workunit.client.0.smithi142.stdout:8/393: dwrite d5/d1d/d2f/d76/d3c/f3e [0,4194304] 0 2024-03-29T04:44:59.048 INFO:tasks.workunit.client.0.smithi142.stdout:9/293: chown d0/d2/d6/d27/d3c/d4f/d34/f2d 1 1 2024-03-29T04:44:59.053 INFO:tasks.workunit.client.0.smithi142.stdout:8/394: rename d5/d1d/d2f/d76/d4c/d60/f78 to d5/d1d/d2f/d76/d4c/f86 0 2024-03-29T04:44:59.054 INFO:tasks.workunit.client.0.smithi142.stdout:8/395: readlink d5/d1d/d24/d66/l67 0 2024-03-29T04:44:59.068 INFO:tasks.workunit.client.0.smithi142.stdout:9/294: dread d0/d2/d6/d16/f37 [0,4194304] 0 2024-03-29T04:44:59.069 INFO:tasks.workunit.client.0.smithi142.stdout:8/396: write d5/d1d/d2f/d76/f41 [3673232,119344] 0 2024-03-29T04:44:59.071 INFO:tasks.workunit.client.0.smithi142.stdout:5/430: dread df/d1a/f22 [4194304,4194304] 0 2024-03-29T04:44:59.071 INFO:tasks.workunit.client.0.smithi142.stdout:8/397: mknod d5/d1d/d59/c87 0 2024-03-29T04:44:59.072 INFO:tasks.workunit.client.0.smithi142.stdout:9/295: truncate d0/d2/d6/d16/f1c 128146 0 2024-03-29T04:44:59.073 INFO:tasks.workunit.client.0.smithi142.stdout:5/431: mknod df/d16/d24/d5c/c89 0 2024-03-29T04:44:59.075 INFO:tasks.workunit.client.0.smithi142.stdout:5/432: unlink df/d16/d24/d3d/f68 0 2024-03-29T04:44:59.076 INFO:tasks.workunit.client.0.smithi142.stdout:5/433: readlink df/d1a/d21/d55/d66/l84 0 2024-03-29T04:44:59.077 INFO:tasks.workunit.client.0.smithi142.stdout:5/434: write df/d1a/d21/d27/f36 [2054211,41807] 0 2024-03-29T04:44:59.077 INFO:tasks.workunit.client.0.smithi142.stdout:9/296: rename d0/d2/d6/d27/d3c/d4f/d34 to d0/d2/d61 0 2024-03-29T04:44:59.080 INFO:tasks.workunit.client.0.smithi142.stdout:5/435: unlink df/f10 0 2024-03-29T04:44:59.082 INFO:tasks.workunit.client.0.smithi142.stdout:5/436: read fb [84855,105259] 0 2024-03-29T04:44:59.084 INFO:tasks.workunit.client.0.smithi142.stdout:9/297: dwrite d0/d2/d61/f2d [0,4194304] 0 2024-03-29T04:44:59.084 INFO:tasks.workunit.client.0.smithi142.stdout:5/437: mkdir df/d29/d6e/d8a 0 2024-03-29T04:44:59.085 INFO:tasks.workunit.client.0.smithi142.stdout:9/298: write d0/d2/d6/d27/f48 [663845,5190] 0 2024-03-29T04:44:59.086 INFO:tasks.workunit.client.0.smithi142.stdout:7/411: sync 2024-03-29T04:44:59.086 INFO:tasks.workunit.client.0.smithi142.stdout:6/332: sync 2024-03-29T04:44:59.086 INFO:tasks.workunit.client.0.smithi142.stdout:2/301: sync 2024-03-29T04:44:59.086 INFO:tasks.workunit.client.0.smithi142.stdout:1/376: sync 2024-03-29T04:44:59.086 INFO:tasks.workunit.client.0.smithi142.stdout:3/301: sync 2024-03-29T04:44:59.088 INFO:tasks.workunit.client.0.smithi142.stdout:3/302: write d3/f3c [3123735,9984] 0 2024-03-29T04:44:59.093 INFO:tasks.workunit.client.0.smithi142.stdout:9/299: dwrite d0/d2/f11 [4194304,4194304] 0 2024-03-29T04:44:59.093 INFO:tasks.workunit.client.0.smithi142.stdout:2/302: read de/f10 [869625,86194] 0 2024-03-29T04:44:59.093 INFO:tasks.workunit.client.0.smithi142.stdout:7/412: creat d7/dc/de/d20/f85 x:0 0 0 2024-03-29T04:44:59.094 INFO:tasks.workunit.client.0.smithi142.stdout:2/303: chown de/f36 6258 1 2024-03-29T04:44:59.094 INFO:tasks.workunit.client.0.smithi142.stdout:6/333: creat d1c/d33/d46/f65 x:0 0 0 2024-03-29T04:44:59.095 INFO:tasks.workunit.client.0.smithi142.stdout:2/304: readlink de/d11/d1e/l20 0 2024-03-29T04:44:59.095 INFO:tasks.workunit.client.0.smithi142.stdout:9/300: truncate d0/d2/d61/f55 517987 0 2024-03-29T04:44:59.096 INFO:tasks.workunit.client.0.smithi142.stdout:9/301: read d0/d2/f1f [4463695,94958] 0 2024-03-29T04:44:59.096 INFO:tasks.workunit.client.0.smithi142.stdout:7/413: truncate d7/dc/de/d20/d3f/f65 1326396 0 2024-03-29T04:44:59.097 INFO:tasks.workunit.client.0.smithi142.stdout:1/377: mkdir d8/db/dc/d19/d1d/d2d/d8b 0 2024-03-29T04:44:59.097 INFO:tasks.workunit.client.0.smithi142.stdout:9/302: write d0/d12/f60 [8222,34512] 0 2024-03-29T04:44:59.098 INFO:tasks.workunit.client.0.smithi142.stdout:3/303: mkdir d3/d29/d2b/d6a 0 2024-03-29T04:44:59.100 INFO:tasks.workunit.client.0.smithi142.stdout:5/438: creat df/d16/d24/d2e/f8b x:0 0 0 2024-03-29T04:44:59.101 INFO:tasks.workunit.client.0.smithi142.stdout:6/334: mkdir d1c/d33/d46/d66 0 2024-03-29T04:44:59.103 INFO:tasks.workunit.client.0.smithi142.stdout:7/414: dwrite d7/dc/d4b/d71/f78 [0,4194304] 0 2024-03-29T04:44:59.107 INFO:tasks.workunit.client.0.smithi142.stdout:5/439: unlink df/d2c/f40 0 2024-03-29T04:44:59.109 INFO:tasks.workunit.client.0.smithi142.stdout:6/335: unlink d1c/c2e 0 2024-03-29T04:44:59.110 INFO:tasks.workunit.client.0.smithi142.stdout:6/336: truncate d1c/d33/f5a 1012251 0 2024-03-29T04:44:59.112 INFO:tasks.workunit.client.0.smithi142.stdout:3/304: mkdir d3/d5d/d5f/d64/d6b 0 2024-03-29T04:44:59.113 INFO:tasks.workunit.client.0.smithi142.stdout:7/415: fsync d7/dc/d4b/d71/f76 0 2024-03-29T04:44:59.115 INFO:tasks.workunit.client.0.smithi142.stdout:6/337: creat d1c/d33/f67 x:0 0 0 2024-03-29T04:44:59.115 INFO:tasks.workunit.client.0.smithi142.stdout:9/303: creat d0/f62 x:0 0 0 2024-03-29T04:44:59.115 INFO:tasks.workunit.client.0.smithi142.stdout:6/338: dread - d1c/f49 zero size 2024-03-29T04:44:59.116 INFO:tasks.workunit.client.0.smithi142.stdout:3/305: mkdir d3/d29/d2d/d6c 0 2024-03-29T04:44:59.117 INFO:tasks.workunit.client.0.smithi142.stdout:5/440: fdatasync f7 0 2024-03-29T04:44:59.119 INFO:tasks.workunit.client.0.smithi142.stdout:7/416: rename d7/dc/d4b/d71/d30/d55 to d7/dc/de/d20/d86 0 2024-03-29T04:44:59.120 INFO:tasks.workunit.client.0.smithi142.stdout:6/339: creat d1c/d45/d5e/d5f/f68 x:0 0 0 2024-03-29T04:44:59.122 INFO:tasks.workunit.client.0.smithi142.stdout:1/378: dread d8/db/f25 [0,4194304] 0 2024-03-29T04:44:59.124 INFO:tasks.workunit.client.0.smithi142.stdout:5/441: mkdir df/d16/d24/d2e/d47/d8c 0 2024-03-29T04:44:59.124 INFO:tasks.workunit.client.0.smithi142.stdout:7/417: dwrite d7/dc/de/f34 [0,4194304] 0 2024-03-29T04:44:59.125 INFO:tasks.workunit.client.0.smithi142.stdout:3/306: mknod d3/c6d 0 2024-03-29T04:44:59.130 INFO:tasks.workunit.client.0.smithi142.stdout:6/340: creat d1c/d33/d46/f69 x:0 0 0 2024-03-29T04:44:59.133 INFO:tasks.workunit.client.0.smithi142.stdout:9/304: truncate d0/d2/d6/d16/f2b 2929253 0 2024-03-29T04:44:59.133 INFO:tasks.workunit.client.0.smithi142.stdout:7/418: dwrite d7/d27/d44/d4c/f79 [0,4194304] 0 2024-03-29T04:44:59.135 INFO:tasks.workunit.client.0.smithi142.stdout:1/379: unlink d8/db/d37/f8a 0 2024-03-29T04:44:59.138 INFO:tasks.workunit.client.0.smithi142.stdout:3/307: creat d3/d1f/d59/f6e x:0 0 0 2024-03-29T04:44:59.139 INFO:tasks.workunit.client.0.smithi142.stdout:2/305: truncate de/d11/d1e/d22/f26 578253 0 2024-03-29T04:44:59.140 INFO:tasks.workunit.client.0.smithi142.stdout:2/306: fsync de/d11/d1e/f5b 0 2024-03-29T04:44:59.140 INFO:tasks.workunit.client.0.smithi142.stdout:5/442: dread fd [0,4194304] 0 2024-03-29T04:44:59.141 INFO:tasks.workunit.client.0.smithi142.stdout:6/341: unlink d1c/d45/d51/c59 0 2024-03-29T04:44:59.145 INFO:tasks.workunit.client.0.smithi142.stdout:7/419: unlink d7/dc/de/c5d 0 2024-03-29T04:44:59.146 INFO:tasks.workunit.client.0.smithi142.stdout:1/380: mkdir d8/db/dc/d77/d31/d4d/d8c 0 2024-03-29T04:44:59.147 INFO:tasks.workunit.client.0.smithi142.stdout:3/308: rmdir d3/d29/d2b 39 2024-03-29T04:44:59.147 INFO:tasks.workunit.client.0.smithi142.stdout:3/309: readlink l0 0 2024-03-29T04:44:59.148 INFO:tasks.workunit.client.0.smithi142.stdout:3/310: readlink d3/d1f/d34/l10 0 2024-03-29T04:44:59.148 INFO:tasks.workunit.client.0.smithi142.stdout:3/311: write d3/d1f/d34/f35 [220728,80222] 0 2024-03-29T04:44:59.149 INFO:tasks.workunit.client.0.smithi142.stdout:5/443: creat df/d1a/d28/f8d x:0 0 0 2024-03-29T04:44:59.151 INFO:tasks.workunit.client.0.smithi142.stdout:6/342: symlink d1c/d33/d46/l6a 0 2024-03-29T04:44:59.151 INFO:tasks.workunit.client.0.smithi142.stdout:6/343: fsync d1c/f23 0 2024-03-29T04:44:59.157 INFO:tasks.workunit.client.0.smithi142.stdout:2/307: write de/d11/d1e/d22/d27/f4c [55063,28424] 0 2024-03-29T04:44:59.158 INFO:tasks.workunit.client.0.smithi142.stdout:2/308: fdatasync de/d11/d1e/d22/d27/f66 0 2024-03-29T04:44:59.162 INFO:tasks.workunit.client.0.smithi142.stdout:2/309: symlink de/d11/d1e/d4d/l67 0 2024-03-29T04:44:59.166 INFO:tasks.workunit.client.0.smithi142.stdout:6/344: rename f14 to d1c/d33/d46/d66/f6b 0 2024-03-29T04:44:59.167 INFO:tasks.workunit.client.0.smithi142.stdout:7/420: creat d7/f87 x:0 0 0 2024-03-29T04:44:59.167 INFO:tasks.workunit.client.0.smithi142.stdout:2/310: mknod de/d11/d3d/c68 0 2024-03-29T04:44:59.168 INFO:tasks.workunit.client.0.smithi142.stdout:1/381: rmdir d8/db 39 2024-03-29T04:44:59.170 INFO:tasks.workunit.client.0.smithi142.stdout:6/345: symlink d1c/d45/d5e/d5f/l6c 0 2024-03-29T04:44:59.171 INFO:tasks.workunit.client.0.smithi142.stdout:7/421: creat d7/d27/f88 x:0 0 0 2024-03-29T04:44:59.171 INFO:tasks.workunit.client.0.smithi142.stdout:9/305: write d0/d2/d6/d16/f1c [1044611,37333] 0 2024-03-29T04:44:59.172 INFO:tasks.workunit.client.0.smithi142.stdout:3/312: write d3/d1f/d34/d6/f20 [1463211,93155] 0 2024-03-29T04:44:59.175 INFO:tasks.workunit.client.0.smithi142.stdout:9/306: creat d0/d12/d50/f63 x:0 0 0 2024-03-29T04:44:59.176 INFO:tasks.workunit.client.0.smithi142.stdout:1/382: creat d8/db/d5e/f8d x:0 0 0 2024-03-29T04:44:59.177 INFO:tasks.workunit.client.0.smithi142.stdout:7/422: dwrite d7/dc/de/f16 [0,4194304] 0 2024-03-29T04:44:59.181 INFO:tasks.workunit.client.0.smithi142.stdout:6/346: rename d1c/l63 to d1c/d42/l6d 0 2024-03-29T04:44:59.181 INFO:tasks.workunit.client.0.smithi142.stdout:2/311: getdents de/d11/d17/d5d 0 2024-03-29T04:44:59.182 INFO:tasks.workunit.client.0.smithi142.stdout:1/383: chown d8/db/dc/d19/d1d/d15/c86 57 1 2024-03-29T04:44:59.183 INFO:tasks.workunit.client.0.smithi142.stdout:3/313: rename d3/d29/c2a to d3/d5d/c6f 0 2024-03-29T04:44:59.186 INFO:tasks.workunit.client.0.smithi142.stdout:6/347: creat d1c/f6e x:0 0 0 2024-03-29T04:44:59.187 INFO:tasks.workunit.client.0.smithi142.stdout:9/307: dwrite d0/d2/d61/f2d [0,4194304] 0 2024-03-29T04:44:59.187 INFO:tasks.workunit.client.0.smithi142.stdout:2/312: link fd de/d11/d17/f69 0 2024-03-29T04:44:59.190 INFO:tasks.workunit.client.0.smithi142.stdout:1/384: truncate d8/db/dc/d19/d1d/f48 47039 0 2024-03-29T04:44:59.190 INFO:tasks.workunit.client.0.smithi142.stdout:7/423: dwrite d7/dc/d4b/d71/d30/d33/f77 [0,4194304] 0 2024-03-29T04:44:59.190 INFO:tasks.workunit.client.0.smithi142.stdout:3/314: rename d3/c8 to d3/d5d/d5f/d64/c70 0 2024-03-29T04:44:59.191 INFO:tasks.workunit.client.0.smithi142.stdout:6/348: rmdir d1c/d42/d52 39 2024-03-29T04:44:59.194 INFO:tasks.workunit.client.0.smithi142.stdout:3/315: stat d3/d29/d2b/d40/f4a 0 2024-03-29T04:44:59.197 INFO:tasks.workunit.client.0.smithi142.stdout:9/308: dread d0/d2/d6/d16/f58 [0,4194304] 0 2024-03-29T04:44:59.197 INFO:tasks.workunit.client.0.smithi142.stdout:2/313: mknod de/d11/c6a 0 2024-03-29T04:44:59.201 INFO:tasks.workunit.client.0.smithi142.stdout:7/424: dwrite d7/dc/de/d20/d3f/f50 [0,4194304] 0 2024-03-29T04:44:59.201 INFO:tasks.workunit.client.0.smithi142.stdout:1/385: dread d8/db/dc/d77/d31/f3f [0,4194304] 0 2024-03-29T04:44:59.204 INFO:tasks.workunit.client.0.smithi142.stdout:6/349: symlink d1c/d33/d46/d61/l6f 0 2024-03-29T04:44:59.206 INFO:tasks.workunit.client.0.smithi142.stdout:7/425: mkdir d7/dc/d89 0 2024-03-29T04:44:59.208 INFO:tasks.workunit.client.0.smithi142.stdout:6/350: mknod d1c/d33/d46/d66/c70 0 2024-03-29T04:44:59.208 INFO:tasks.workunit.client.0.smithi142.stdout:6/351: chown d1c/d33/f36 972186 1 2024-03-29T04:44:59.208 INFO:tasks.workunit.client.0.smithi142.stdout:1/386: read d8/f9 [2829884,36674] 0 2024-03-29T04:44:59.209 INFO:tasks.workunit.client.0.smithi142.stdout:7/426: unlink d7/dc/d4b/d71/c2f 0 2024-03-29T04:44:59.210 INFO:tasks.workunit.client.0.smithi142.stdout:7/427: chown d7/d27/f88 1523 1 2024-03-29T04:44:59.212 INFO:tasks.workunit.client.0.smithi142.stdout:1/387: dread d8/db/dc/d19/d1d/d2d/f3e [0,4194304] 0 2024-03-29T04:44:59.213 INFO:tasks.workunit.client.0.smithi142.stdout:7/428: symlink d7/dc/d4b/d71/d30/d72/l8a 0 2024-03-29T04:44:59.215 INFO:tasks.workunit.client.0.smithi142.stdout:1/388: write d8/db/dc/d19/d1d/d2d/f3e [1787183,99194] 0 2024-03-29T04:44:59.217 INFO:tasks.workunit.client.0.smithi142.stdout:1/389: write d8/db/d5e/f8d [1021421,98877] 0 2024-03-29T04:44:59.217 INFO:tasks.workunit.client.0.smithi142.stdout:7/429: getdents d7/dc/de 0 2024-03-29T04:44:59.218 INFO:tasks.workunit.client.0.smithi142.stdout:0/342: sync 2024-03-29T04:44:59.219 INFO:tasks.workunit.client.0.smithi142.stdout:0/343: dread - d7/d9/d55/f5b zero size 2024-03-29T04:44:59.219 INFO:tasks.workunit.client.0.smithi142.stdout:7/430: mknod d7/d27/c8b 0 2024-03-29T04:44:59.220 INFO:tasks.workunit.client.0.smithi142.stdout:1/390: stat d8/db/dc/d19/d1d/d1f/c78 0 2024-03-29T04:44:59.222 INFO:tasks.workunit.client.0.smithi142.stdout:3/316: write d3/d1f/f46 [3716091,33500] 0 2024-03-29T04:44:59.222 INFO:tasks.workunit.client.0.smithi142.stdout:2/314: write de/f10 [2660802,129317] 0 2024-03-29T04:44:59.226 INFO:tasks.workunit.client.0.smithi142.stdout:3/317: mknod d3/d1f/d24/c71 0 2024-03-29T04:44:59.227 INFO:tasks.workunit.client.0.smithi142.stdout:9/309: truncate d0/d2/d6/d31/f3a 3631387 0 2024-03-29T04:44:59.228 INFO:tasks.workunit.client.0.smithi142.stdout:1/391: dwrite d8/db/dc/d77/f32 [0,4194304] 0 2024-03-29T04:44:59.231 INFO:tasks.workunit.client.0.smithi142.stdout:0/344: write d7/d9/f2e [322587,119536] 0 2024-03-29T04:44:59.234 INFO:tasks.workunit.client.0.smithi142.stdout:3/318: creat d3/d1f/d34/f72 x:0 0 0 2024-03-29T04:44:59.235 INFO:tasks.workunit.client.0.smithi142.stdout:3/319: dread - d3/d1f/d34/f4e zero size 2024-03-29T04:44:59.236 INFO:tasks.workunit.client.0.smithi142.stdout:9/310: dwrite d0/d2/d6/d16/f37 [0,4194304] 0 2024-03-29T04:44:59.240 INFO:tasks.workunit.client.0.smithi142.stdout:3/320: symlink d3/d29/d2d/d56/l73 0 2024-03-29T04:44:59.247 INFO:tasks.workunit.client.0.smithi142.stdout:0/345: write d7/d9/d22/f3e [1289600,80334] 0 2024-03-29T04:44:59.247 INFO:tasks.workunit.client.0.smithi142.stdout:0/346: dread - d7/d9/d55/f62 zero size 2024-03-29T04:44:59.249 INFO:tasks.workunit.client.0.smithi142.stdout:0/347: stat d7/d9/d54/d58/c66 0 2024-03-29T04:44:59.253 INFO:tasks.workunit.client.0.smithi142.stdout:0/348: getdents d7/d9 0 2024-03-29T04:44:59.254 INFO:tasks.workunit.client.0.smithi142.stdout:2/315: dread de/f29 [0,4194304] 0 2024-03-29T04:44:59.255 INFO:tasks.workunit.client.0.smithi142.stdout:0/349: mkdir d7/d9/d6e 0 2024-03-29T04:44:59.255 INFO:tasks.workunit.client.0.smithi142.stdout:1/392: truncate d8/db/dc/d77/d5a/f75 4097585 0 2024-03-29T04:44:59.258 INFO:tasks.workunit.client.0.smithi142.stdout:3/321: truncate d3/d29/d2d/f48 3445724 0 2024-03-29T04:44:59.261 INFO:tasks.workunit.client.0.smithi142.stdout:1/393: mkdir d8/db/d8e 0 2024-03-29T04:44:59.262 INFO:tasks.workunit.client.0.smithi142.stdout:1/394: dread - d8/f70 zero size 2024-03-29T04:44:59.265 INFO:tasks.workunit.client.0.smithi142.stdout:3/322: dwrite d3/d29/d2b/f21 [0,4194304] 0 2024-03-29T04:44:59.265 INFO:tasks.workunit.client.0.smithi142.stdout:9/311: dread d0/f22 [0,4194304] 0 2024-03-29T04:44:59.267 INFO:tasks.workunit.client.0.smithi142.stdout:2/316: write f9 [686340,46414] 0 2024-03-29T04:44:59.270 INFO:tasks.workunit.client.0.smithi142.stdout:1/395: creat d8/db/dc/d77/d31/d4d/f8f x:0 0 0 2024-03-29T04:44:59.270 INFO:tasks.workunit.client.0.smithi142.stdout:0/350: dread d7/d9/f30 [0,4194304] 0 2024-03-29T04:44:59.270 INFO:tasks.workunit.client.0.smithi142.stdout:9/312: rmdir d0/d2/d6 39 2024-03-29T04:44:59.272 INFO:tasks.workunit.client.0.smithi142.stdout:2/317: truncate de/d11/d1e/d22/f3f 4367329 0 2024-03-29T04:44:59.272 INFO:tasks.workunit.client.0.smithi142.stdout:1/396: dread - d8/db/dc/d19/f57 zero size 2024-03-29T04:44:59.273 INFO:tasks.workunit.client.0.smithi142.stdout:9/313: mkdir d0/d2/d61/d64 0 2024-03-29T04:44:59.275 INFO:tasks.workunit.client.0.smithi142.stdout:3/323: link d3/d1f/d34/c36 d3/d1f/d24/c74 0 2024-03-29T04:44:59.276 INFO:tasks.workunit.client.0.smithi142.stdout:2/318: truncate de/d11/d1e/d22/d27/f4c 779917 0 2024-03-29T04:44:59.277 INFO:tasks.workunit.client.0.smithi142.stdout:1/397: creat d8/db/dc/d19/d1d/d2d/f90 x:0 0 0 2024-03-29T04:44:59.278 INFO:tasks.workunit.client.0.smithi142.stdout:1/398: readlink d8/db/d5e/l85 0 2024-03-29T04:44:59.279 INFO:tasks.workunit.client.0.smithi142.stdout:3/324: creat d3/d1f/d24/d47/f75 x:0 0 0 2024-03-29T04:44:59.279 INFO:tasks.workunit.client.0.smithi142.stdout:9/314: truncate d0/d2/d6/d31/f49 541141 0 2024-03-29T04:44:59.280 INFO:tasks.workunit.client.0.smithi142.stdout:2/319: unlink de/d11/d3d/l4e 0 2024-03-29T04:44:59.280 INFO:tasks.workunit.client.0.smithi142.stdout:0/351: dwrite d7/d9/d54/f65 [0,4194304] 0 2024-03-29T04:44:59.281 INFO:tasks.workunit.client.0.smithi142.stdout:9/315: chown d0/d2/d6/d27/d3c 833845 1 2024-03-29T04:44:59.285 INFO:tasks.workunit.client.0.smithi142.stdout:1/399: symlink d8/db/dc/d77/d5a/l91 0 2024-03-29T04:44:59.286 INFO:tasks.workunit.client.0.smithi142.stdout:0/352: truncate d7/d9/d1e/f53 657944 0 2024-03-29T04:44:59.289 INFO:tasks.workunit.client.0.smithi142.stdout:2/320: write fd [3832318,99386] 0 2024-03-29T04:44:59.290 INFO:tasks.workunit.client.0.smithi142.stdout:1/400: fsync d8/db/dc/d19/d1d/d15/f17 0 2024-03-29T04:44:59.293 INFO:tasks.workunit.client.0.smithi142.stdout:1/401: stat d8/db/dc/d19/d1d/d2d/d44/d53 0 2024-03-29T04:44:59.294 INFO:tasks.workunit.client.0.smithi142.stdout:3/325: getdents d3/d29/d2d/d56 0 2024-03-29T04:44:59.295 INFO:tasks.workunit.client.0.smithi142.stdout:2/321: truncate de/d11/d1e/d22/d50/f57 799436 0 2024-03-29T04:44:59.296 INFO:tasks.workunit.client.0.smithi142.stdout:3/326: read d3/d1f/d34/f3e [3956402,58483] 0 2024-03-29T04:44:59.296 INFO:tasks.workunit.client.0.smithi142.stdout:0/353: dwrite d7/d9/f49 [0,4194304] 0 2024-03-29T04:44:59.297 INFO:tasks.workunit.client.0.smithi142.stdout:1/402: symlink d8/db/dc/d19/d1d/d1f/l92 0 2024-03-29T04:44:59.298 INFO:tasks.workunit.client.0.smithi142.stdout:2/322: dread de/d11/d23/f28 [0,4194304] 0 2024-03-29T04:44:59.301 INFO:tasks.workunit.client.0.smithi142.stdout:0/354: creat d7/d9/d6e/f6f x:0 0 0 2024-03-29T04:44:59.301 INFO:tasks.workunit.client.0.smithi142.stdout:3/327: dread d3/d54/f58 [0,4194304] 0 2024-03-29T04:44:59.302 INFO:tasks.workunit.client.0.smithi142.stdout:2/323: unlink de/d11/d1e/l20 0 2024-03-29T04:44:59.303 INFO:tasks.workunit.client.0.smithi142.stdout:1/403: creat d8/db/dc/d77/d5a/f93 x:0 0 0 2024-03-29T04:44:59.304 INFO:tasks.workunit.client.0.smithi142.stdout:3/328: creat d3/d5d/d5f/d64/f76 x:0 0 0 2024-03-29T04:44:59.305 INFO:tasks.workunit.client.0.smithi142.stdout:2/324: truncate f7 103020 0 2024-03-29T04:44:59.305 INFO:tasks.workunit.client.0.smithi142.stdout:3/329: read d3/d1f/d34/d4d/d50/f51 [115715,59211] 0 2024-03-29T04:44:59.306 INFO:tasks.workunit.client.0.smithi142.stdout:0/355: mkdir d7/d13/d4a/d70 0 2024-03-29T04:44:59.307 INFO:tasks.workunit.client.0.smithi142.stdout:2/325: truncate de/d11/d1e/d22/d50/f64 934615 0 2024-03-29T04:44:59.309 INFO:tasks.workunit.client.0.smithi142.stdout:0/356: unlink d7/l14 0 2024-03-29T04:44:59.310 INFO:tasks.workunit.client.0.smithi142.stdout:2/326: write de/d11/d1e/d22/d27/f66 [339878,18799] 0 2024-03-29T04:44:59.313 INFO:tasks.workunit.client.0.smithi142.stdout:3/330: dwrite d3/d1f/d34/f4e [0,4194304] 0 2024-03-29T04:44:59.313 INFO:tasks.workunit.client.0.smithi142.stdout:2/327: write de/f16 [3520430,100875] 0 2024-03-29T04:44:59.319 INFO:tasks.workunit.client.0.smithi142.stdout:0/357: dwrite d7/d9/f49 [0,4194304] 0 2024-03-29T04:44:59.320 INFO:tasks.workunit.client.0.smithi142.stdout:2/328: creat de/d11/d1e/d4d/f6b x:0 0 0 2024-03-29T04:44:59.325 INFO:tasks.workunit.client.0.smithi142.stdout:0/358: rmdir d7/d9/d22 39 2024-03-29T04:44:59.327 INFO:tasks.workunit.client.0.smithi142.stdout:3/331: rename d3/d1f/d34/f35 to d3/d29/f77 0 2024-03-29T04:44:59.328 INFO:tasks.workunit.client.0.smithi142.stdout:3/332: read - d3/d1f/d24/f69 zero size 2024-03-29T04:44:59.328 INFO:tasks.workunit.client.0.smithi142.stdout:1/404: dread d8/db/d37/f41 [0,4194304] 0 2024-03-29T04:44:59.329 INFO:tasks.workunit.client.0.smithi142.stdout:2/329: link de/f29 de/d11/d1e/d22/f6c 0 2024-03-29T04:44:59.330 INFO:tasks.workunit.client.0.smithi142.stdout:2/330: write de/f19 [7857247,89576] 0 2024-03-29T04:44:59.332 INFO:tasks.workunit.client.0.smithi142.stdout:1/405: creat d8/db/d37/f94 x:0 0 0 2024-03-29T04:44:59.333 INFO:tasks.workunit.client.0.smithi142.stdout:2/331: rmdir de/d11/d1e/d22 39 2024-03-29T04:44:59.334 INFO:tasks.workunit.client.0.smithi142.stdout:0/359: getdents d7 0 2024-03-29T04:44:59.335 INFO:tasks.workunit.client.0.smithi142.stdout:1/406: write f3 [1097855,43877] 0 2024-03-29T04:44:59.335 INFO:tasks.workunit.client.0.smithi142.stdout:2/332: chown de/d11/c13 50 1 2024-03-29T04:44:59.337 INFO:tasks.workunit.client.0.smithi142.stdout:0/360: creat d7/d9/d1e/f71 x:0 0 0 2024-03-29T04:44:59.337 INFO:tasks.workunit.client.0.smithi142.stdout:2/333: fsync de/d11/f37 0 2024-03-29T04:44:59.341 INFO:tasks.workunit.client.0.smithi142.stdout:1/407: rename d8/db/dc/d77/d31/f74 to d8/db/dc/d77/d31/d4d/d8c/f95 0 2024-03-29T04:44:59.345 INFO:tasks.workunit.client.0.smithi142.stdout:0/361: rename d7/d9/d22 to d7/d13/d67/d72 0 2024-03-29T04:44:59.347 INFO:tasks.workunit.client.0.smithi142.stdout:0/362: creat d7/d13/d67/d72/d37/f73 x:0 0 0 2024-03-29T04:44:59.350 INFO:tasks.workunit.client.0.smithi142.stdout:2/334: truncate de/f19 266274 0 2024-03-29T04:44:59.351 INFO:tasks.workunit.client.0.smithi142.stdout:3/333: dwrite d3/d1f/d34/d4d/d50/f51 [0,4194304] 0 2024-03-29T04:44:59.353 INFO:tasks.workunit.client.0.smithi142.stdout:2/335: creat de/d11/d17/d60/f6d x:0 0 0 2024-03-29T04:44:59.354 INFO:tasks.workunit.client.0.smithi142.stdout:3/334: creat d3/d54/f78 x:0 0 0 2024-03-29T04:44:59.354 INFO:tasks.workunit.client.0.smithi142.stdout:0/363: creat d7/d9/d55/d42/f74 x:0 0 0 2024-03-29T04:44:59.356 INFO:tasks.workunit.client.0.smithi142.stdout:2/336: mknod de/d11/d2b/d3e/c6e 0 2024-03-29T04:44:59.357 INFO:tasks.workunit.client.0.smithi142.stdout:3/335: mkdir d3/d29/d2d/d79 0 2024-03-29T04:44:59.357 INFO:tasks.workunit.client.0.smithi142.stdout:0/364: mkdir d7/d13/d67/d72/d75 0 2024-03-29T04:44:59.358 INFO:tasks.workunit.client.0.smithi142.stdout:2/337: mkdir de/d11/d17/d60/d6f 0 2024-03-29T04:44:59.358 INFO:tasks.workunit.client.0.smithi142.stdout:0/365: dread - d7/d9/d55/f5b zero size 2024-03-29T04:44:59.358 INFO:tasks.workunit.client.0.smithi142.stdout:4/306: sync 2024-03-29T04:44:59.359 INFO:tasks.workunit.client.0.smithi142.stdout:8/398: sync 2024-03-29T04:44:59.360 INFO:tasks.workunit.client.0.smithi142.stdout:0/366: fsync d7/d9/d55/f57 0 2024-03-29T04:44:59.364 INFO:tasks.workunit.client.0.smithi142.stdout:8/399: creat d5/d1d/d2f/d76/f88 x:0 0 0 2024-03-29T04:44:59.365 INFO:tasks.workunit.client.0.smithi142.stdout:4/307: link d0/f3 d0/d12/d28/d47/f6b 0 2024-03-29T04:44:59.366 INFO:tasks.workunit.client.0.smithi142.stdout:3/336: dwrite d3/d1f/d24/d47/f57 [0,4194304] 0 2024-03-29T04:44:59.369 INFO:tasks.workunit.client.0.smithi142.stdout:4/308: chown d0/dc/d15/l2e 29066 1 2024-03-29T04:44:59.370 INFO:tasks.workunit.client.0.smithi142.stdout:2/338: dwrite de/d11/d1e/d22/d50/f64 [0,4194304] 0 2024-03-29T04:44:59.372 INFO:tasks.workunit.client.0.smithi142.stdout:2/339: fsync de/d11/d17/f69 0 2024-03-29T04:44:59.372 INFO:tasks.workunit.client.0.smithi142.stdout:3/337: symlink d3/l7a 0 2024-03-29T04:44:59.374 INFO:tasks.workunit.client.0.smithi142.stdout:0/367: link d7/d9/d55/d42/c4d d7/d9/d55/c76 0 2024-03-29T04:44:59.376 INFO:tasks.workunit.client.0.smithi142.stdout:0/368: rename d7/d13/d4a to d7/d13/d4a/d77 22 2024-03-29T04:44:59.378 INFO:tasks.workunit.client.0.smithi142.stdout:4/309: truncate d0/d9/d11/f38 1966943 0 2024-03-29T04:44:59.380 INFO:tasks.workunit.client.0.smithi142.stdout:0/369: dread d7/d13/d67/d72/d37/f43 [0,4194304] 0 2024-03-29T04:44:59.380 INFO:tasks.workunit.client.0.smithi142.stdout:3/338: creat d3/d29/d2b/d6a/f7b x:0 0 0 2024-03-29T04:44:59.382 INFO:tasks.workunit.client.0.smithi142.stdout:2/340: getdents de/d11/d1e 0 2024-03-29T04:44:59.385 INFO:tasks.workunit.client.0.smithi142.stdout:4/310: creat d0/d9/d11/d4c/d5f/f6c x:0 0 0 2024-03-29T04:44:59.386 INFO:tasks.workunit.client.0.smithi142.stdout:3/339: creat d3/d29/d2b/d40/f7c x:0 0 0 2024-03-29T04:44:59.386 INFO:tasks.workunit.client.0.smithi142.stdout:3/340: stat d3/d5d/d5f/d64 0 2024-03-29T04:44:59.387 INFO:tasks.workunit.client.0.smithi142.stdout:0/370: fdatasync d7/d9/f30 0 2024-03-29T04:44:59.389 INFO:tasks.workunit.client.0.smithi142.stdout:2/341: symlink de/d11/d1e/d4d/d61/l70 0 2024-03-29T04:44:59.389 INFO:tasks.workunit.client.0.smithi142.stdout:4/311: dwrite d0/dc/d15/f53 [0,4194304] 0 2024-03-29T04:44:59.394 INFO:tasks.workunit.client.0.smithi142.stdout:0/371: truncate d7/d9/d54/f64 246433 0 2024-03-29T04:44:59.394 INFO:tasks.workunit.client.0.smithi142.stdout:0/372: readlink d7/l12 0 2024-03-29T04:44:59.394 INFO:tasks.workunit.client.0.smithi142.stdout:2/342: mkdir de/d11/d17/d5d/d71 0 2024-03-29T04:44:59.396 INFO:tasks.workunit.client.0.smithi142.stdout:4/312: mkdir d0/d9/d11/d5c/d6d 0 2024-03-29T04:44:59.397 INFO:tasks.workunit.client.0.smithi142.stdout:0/373: write d7/d9/d1e/f71 [614172,59915] 0 2024-03-29T04:44:59.398 INFO:tasks.workunit.client.0.smithi142.stdout:4/313: mkdir d0/d9/d11/d5c/d6d/d6e 0 2024-03-29T04:44:59.400 INFO:tasks.workunit.client.0.smithi142.stdout:0/374: rmdir d7/d9/d55/d42 39 2024-03-29T04:44:59.401 INFO:tasks.workunit.client.0.smithi142.stdout:4/314: read d0/dc/f33 [53989,62881] 0 2024-03-29T04:44:59.405 INFO:tasks.workunit.client.0.smithi142.stdout:4/315: creat d0/d9/d11/d4c/d5f/f6f x:0 0 0 2024-03-29T04:44:59.406 INFO:tasks.workunit.client.0.smithi142.stdout:0/375: creat d7/d13/f78 x:0 0 0 2024-03-29T04:44:59.407 INFO:tasks.workunit.client.0.smithi142.stdout:0/376: write d7/d13/f28 [31567,97473] 0 2024-03-29T04:44:59.408 INFO:tasks.workunit.client.0.smithi142.stdout:0/377: write d7/d9/d55/f57 [58753,104558] 0 2024-03-29T04:44:59.412 INFO:tasks.workunit.client.0.smithi142.stdout:4/316: dwrite d0/d9/d11/f1b [0,4194304] 0 2024-03-29T04:44:59.429 INFO:tasks.workunit.client.0.smithi142.stdout:4/317: dread d0/d9/d11/f23 [0,4194304] 0 2024-03-29T04:44:59.431 INFO:tasks.workunit.client.0.smithi142.stdout:0/378: dread d7/d13/d67/d72/d37/f56 [0,4194304] 0 2024-03-29T04:44:59.431 INFO:tasks.workunit.client.0.smithi142.stdout:0/379: fsync d7/d9/d1e/f53 0 2024-03-29T04:44:59.433 INFO:tasks.workunit.client.0.smithi142.stdout:4/318: dread d0/d9/d11/f2b [0,4194304] 0 2024-03-29T04:44:59.433 INFO:tasks.workunit.client.0.smithi142.stdout:0/380: truncate d7/d9/f3c 778725 0 2024-03-29T04:44:59.436 INFO:tasks.workunit.client.0.smithi142.stdout:4/319: getdents d0/dc/d15 0 2024-03-29T04:44:59.437 INFO:tasks.workunit.client.0.smithi142.stdout:4/320: readlink d0/l46 0 2024-03-29T04:44:59.437 INFO:tasks.workunit.client.0.smithi142.stdout:4/321: chown d0/f8 3729 1 2024-03-29T04:44:59.439 INFO:tasks.workunit.client.0.smithi142.stdout:4/322: dread - d0/d9/d11/d4c/d5f/f6c zero size 2024-03-29T04:44:59.440 INFO:tasks.workunit.client.0.smithi142.stdout:0/381: dwrite d7/d9/d54/d58/f6b [4194304,4194304] 0 2024-03-29T04:44:59.441 INFO:tasks.workunit.client.0.smithi142.stdout:0/382: write d7/d13/d67/d72/f3a [944550,70416] 0 2024-03-29T04:44:59.445 INFO:tasks.workunit.client.0.smithi142.stdout:0/383: mknod d7/d9/d54/c79 0 2024-03-29T04:44:59.448 INFO:tasks.workunit.client.0.smithi142.stdout:4/323: write d0/d9/d19/f1a [2113142,22562] 0 2024-03-29T04:44:59.450 INFO:tasks.workunit.client.0.smithi142.stdout:4/324: read d0/f3 [1422966,102639] 0 2024-03-29T04:44:59.451 INFO:tasks.workunit.client.0.smithi142.stdout:0/384: dwrite d7/d13/d4a/f5e [0,4194304] 0 2024-03-29T04:44:59.453 INFO:tasks.workunit.client.0.smithi142.stdout:0/385: fsync d7/d13/d67/d72/d37/f43 0 2024-03-29T04:44:59.455 INFO:tasks.workunit.client.0.smithi142.stdout:5/444: sync 2024-03-29T04:44:59.455 INFO:tasks.workunit.client.0.smithi142.stdout:6/352: sync 2024-03-29T04:44:59.456 INFO:tasks.workunit.client.0.smithi142.stdout:4/325: dread d0/d12/f3b [0,4194304] 0 2024-03-29T04:44:59.461 INFO:tasks.workunit.client.0.smithi142.stdout:6/353: rename d1c/d45/d5e/d5f to d1c/d33/d46/d71 0 2024-03-29T04:44:59.461 INFO:tasks.workunit.client.0.smithi142.stdout:0/386: dwrite d7/d13/d67/d72/d37/f43 [4194304,4194304] 0 2024-03-29T04:44:59.463 INFO:tasks.workunit.client.0.smithi142.stdout:6/354: truncate d1c/d33/d46/d71/f60 516215 0 2024-03-29T04:44:59.463 INFO:tasks.workunit.client.0.smithi142.stdout:4/326: truncate d0/d12/f5d 2119525 0 2024-03-29T04:44:59.464 INFO:tasks.workunit.client.0.smithi142.stdout:6/355: chown d1c/f5d 1682778001 1 2024-03-29T04:44:59.464 INFO:tasks.workunit.client.0.smithi142.stdout:0/387: symlink d7/d9/d1e/l7a 0 2024-03-29T04:44:59.465 INFO:tasks.workunit.client.0.smithi142.stdout:6/356: chown d1c/f23 1691868 1 2024-03-29T04:44:59.466 INFO:tasks.workunit.client.0.smithi142.stdout:0/388: read d7/d9/d54/d58/f6b [5722364,63851] 0 2024-03-29T04:44:59.466 INFO:tasks.workunit.client.0.smithi142.stdout:6/357: chown d1c/f49 452269921 1 2024-03-29T04:44:59.467 INFO:tasks.workunit.client.0.smithi142.stdout:4/327: getdents d0/d9/d11/d5c/d6d/d6e 0 2024-03-29T04:44:59.468 INFO:tasks.workunit.client.0.smithi142.stdout:6/358: write d1c/f23 [187609,31942] 0 2024-03-29T04:44:59.468 INFO:tasks.workunit.client.0.smithi142.stdout:0/389: write d7/d13/d67/d72/f61 [3859748,129515] 0 2024-03-29T04:44:59.472 INFO:tasks.workunit.client.0.smithi142.stdout:4/328: rename d0/dc/d15/l2e to d0/d12/d28/l70 0 2024-03-29T04:44:59.474 INFO:tasks.workunit.client.0.smithi142.stdout:6/359: rename d1c/c20 to d1c/d33/d46/d71/c72 0 2024-03-29T04:44:59.476 INFO:tasks.workunit.client.0.smithi142.stdout:0/390: truncate d7/d13/d67/d72/d37/f38 1670358 0 2024-03-29T04:44:59.478 INFO:tasks.workunit.client.0.smithi142.stdout:6/360: mkdir d1c/d33/d46/d61/d73 0 2024-03-29T04:44:59.480 INFO:tasks.workunit.client.0.smithi142.stdout:0/391: creat d7/d9/d55/d42/f7b x:0 0 0 2024-03-29T04:44:59.480 INFO:tasks.workunit.client.0.smithi142.stdout:6/361: dread d1c/d33/d46/d71/f60 [0,4194304] 0 2024-03-29T04:44:59.481 INFO:tasks.workunit.client.0.smithi142.stdout:5/445: dread df/d1a/f31 [0,4194304] 0 2024-03-29T04:44:59.481 INFO:tasks.workunit.client.0.smithi142.stdout:0/392: chown d7/d13/d67 28 1 2024-03-29T04:44:59.481 INFO:tasks.workunit.client.0.smithi142.stdout:5/446: fsync df/d29/f6b 0 2024-03-29T04:44:59.486 INFO:tasks.workunit.client.0.smithi142.stdout:6/362: mknod d1c/c74 0 2024-03-29T04:44:59.487 INFO:tasks.workunit.client.0.smithi142.stdout:5/447: mknod df/d1a/c8e 0 2024-03-29T04:44:59.493 INFO:tasks.workunit.client.0.smithi142.stdout:6/363: truncate d1c/f25 1909121 0 2024-03-29T04:44:59.494 INFO:tasks.workunit.client.0.smithi142.stdout:5/448: creat df/d16/d24/d2e/f8f x:0 0 0 2024-03-29T04:44:59.495 INFO:tasks.workunit.client.0.smithi142.stdout:0/393: truncate d7/d9/d1e/f71 483507 0 2024-03-29T04:44:59.495 INFO:tasks.workunit.client.0.smithi142.stdout:5/449: chown df/d1a/d28/f49 700111 1 2024-03-29T04:44:59.496 INFO:tasks.workunit.client.0.smithi142.stdout:4/329: dwrite d0/d45/f6a [0,4194304] 0 2024-03-29T04:44:59.496 INFO:tasks.workunit.client.0.smithi142.stdout:6/364: creat d1c/d33/d46/d71/f75 x:0 0 0 2024-03-29T04:44:59.497 INFO:tasks.workunit.client.0.smithi142.stdout:0/394: unlink d7/d9/d1e/l2a 0 2024-03-29T04:44:59.498 INFO:tasks.workunit.client.0.smithi142.stdout:4/330: creat d0/dc/d15/d3d/f71 x:0 0 0 2024-03-29T04:44:59.499 INFO:tasks.workunit.client.0.smithi142.stdout:5/450: mkdir df/d29/d5d/d75/d90 0 2024-03-29T04:44:59.499 INFO:tasks.workunit.client.0.smithi142.stdout:4/331: chown d0/d9/d19/f58 119 1 2024-03-29T04:44:59.501 INFO:tasks.workunit.client.0.smithi142.stdout:0/395: rename d7/d9/d55/f62 to d7/d9/d55/d42/f7c 0 2024-03-29T04:44:59.504 INFO:tasks.workunit.client.0.smithi142.stdout:4/332: rename d0/f5a to d0/dc/d15/f72 0 2024-03-29T04:44:59.504 INFO:tasks.workunit.client.0.smithi142.stdout:4/333: chown d0/d12/f1e 6 1 2024-03-29T04:44:59.506 INFO:tasks.workunit.client.0.smithi142.stdout:4/334: creat d0/d9/d11/d4c/d5f/f73 x:0 0 0 2024-03-29T04:44:59.510 INFO:tasks.workunit.client.0.smithi142.stdout:5/451: link df/d1a/d21/l72 df/d1a/l91 0 2024-03-29T04:44:59.511 INFO:tasks.workunit.client.0.smithi142.stdout:6/365: dread d1c/d33/d46/d66/f6b [0,4194304] 0 2024-03-29T04:44:59.511 INFO:tasks.workunit.client.0.smithi142.stdout:5/452: stat df/d1a/d21/f3e 0 2024-03-29T04:44:59.512 INFO:tasks.workunit.client.0.smithi142.stdout:6/366: truncate d1c/d33/f4f 444434 0 2024-03-29T04:44:59.514 INFO:tasks.workunit.client.0.smithi142.stdout:5/453: creat df/d1a/d21/d55/d66/f92 x:0 0 0 2024-03-29T04:44:59.514 INFO:tasks.workunit.client.0.smithi142.stdout:6/367: creat d1c/d45/f76 x:0 0 0 2024-03-29T04:44:59.515 INFO:tasks.workunit.client.0.smithi142.stdout:6/368: write d1c/f55 [102777,54742] 0 2024-03-29T04:44:59.516 INFO:tasks.workunit.client.0.smithi142.stdout:5/454: mknod df/d16/d24/d5c/c93 0 2024-03-29T04:44:59.516 INFO:tasks.workunit.client.0.smithi142.stdout:5/455: fsync df/d16/d50/f61 0 2024-03-29T04:44:59.518 INFO:tasks.workunit.client.0.smithi142.stdout:6/369: symlink d1c/d45/l77 0 2024-03-29T04:44:59.519 INFO:tasks.workunit.client.0.smithi142.stdout:6/370: truncate d1c/f49 962505 0 2024-03-29T04:44:59.519 INFO:tasks.workunit.client.0.smithi142.stdout:5/456: mknod df/d29/d5d/c94 0 2024-03-29T04:44:59.520 INFO:tasks.workunit.client.0.smithi142.stdout:6/371: truncate d1c/d33/f67 389552 0 2024-03-29T04:44:59.523 INFO:tasks.workunit.client.0.smithi142.stdout:4/335: rmdir d0/dc 39 2024-03-29T04:44:59.526 INFO:tasks.workunit.client.0.smithi142.stdout:5/457: dwrite df/d1a/f3f [0,4194304] 0 2024-03-29T04:44:59.527 INFO:tasks.workunit.client.0.smithi142.stdout:6/372: dwrite d1c/f55 [0,4194304] 0 2024-03-29T04:44:59.530 INFO:tasks.workunit.client.0.smithi142.stdout:4/336: getdents d0/dc/d15/d3d 0 2024-03-29T04:44:59.533 INFO:tasks.workunit.client.0.smithi142.stdout:6/373: read d1c/f40 [376536,39793] 0 2024-03-29T04:44:59.534 INFO:tasks.workunit.client.0.smithi142.stdout:0/396: dwrite d7/d9/d55/f3d [0,4194304] 0 2024-03-29T04:44:59.535 INFO:tasks.workunit.client.0.smithi142.stdout:4/337: readlink d0/d9/d19/l1c 0 2024-03-29T04:44:59.535 INFO:tasks.workunit.client.0.smithi142.stdout:6/374: write d1c/d33/d46/d71/f75 [632036,92169] 0 2024-03-29T04:44:59.536 INFO:tasks.workunit.client.0.smithi142.stdout:7/431: sync 2024-03-29T04:44:59.536 INFO:tasks.workunit.client.0.smithi142.stdout:5/458: symlink df/d29/d6e/d8a/l95 0 2024-03-29T04:44:59.538 INFO:tasks.workunit.client.0.smithi142.stdout:5/459: chown df/d1a/d21/d55/d66/l84 493037 1 2024-03-29T04:44:59.540 INFO:tasks.workunit.client.0.smithi142.stdout:4/338: dread d0/d9/d11/f1b [0,4194304] 0 2024-03-29T04:44:59.540 INFO:tasks.workunit.client.0.smithi142.stdout:6/375: symlink d1c/d45/d51/l78 0 2024-03-29T04:44:59.547 INFO:tasks.workunit.client.0.smithi142.stdout:5/460: link df/d16/d50/l53 df/d2c/l96 0 2024-03-29T04:44:59.547 INFO:tasks.workunit.client.0.smithi142.stdout:6/376: dwrite d1c/f49 [0,4194304] 0 2024-03-29T04:44:59.549 INFO:tasks.workunit.client.0.smithi142.stdout:6/377: fsync d1c/d33/f36 0 2024-03-29T04:44:59.551 INFO:tasks.workunit.client.0.smithi142.stdout:6/378: dread - d1c/d33/d46/f53 zero size 2024-03-29T04:44:59.553 INFO:tasks.workunit.client.0.smithi142.stdout:4/339: dwrite d0/d9/d11/ff [0,4194304] 0 2024-03-29T04:44:59.557 INFO:tasks.workunit.client.0.smithi142.stdout:5/461: dwrite fe [0,4194304] 0 2024-03-29T04:44:59.560 INFO:tasks.workunit.client.0.smithi142.stdout:5/462: fsync df/d1a/d28/f49 0 2024-03-29T04:44:59.565 INFO:tasks.workunit.client.0.smithi142.stdout:7/432: write d7/f31 [775241,115886] 0 2024-03-29T04:44:59.566 INFO:tasks.workunit.client.0.smithi142.stdout:6/379: dwrite f4 [0,4194304] 0 2024-03-29T04:44:59.567 INFO:tasks.workunit.client.0.smithi142.stdout:4/340: dwrite d0/d12/d28/f54 [0,4194304] 0 2024-03-29T04:44:59.575 INFO:tasks.workunit.client.0.smithi142.stdout:7/433: fdatasync d7/dc/d4b/d71/d30/d33/f38 0 2024-03-29T04:44:59.575 INFO:tasks.workunit.client.0.smithi142.stdout:5/463: dwrite df/d29/d6e/f73 [0,4194304] 0 2024-03-29T04:44:59.578 INFO:tasks.workunit.client.0.smithi142.stdout:6/380: rmdir d1c/d45/d5e 0 2024-03-29T04:44:59.580 INFO:tasks.workunit.client.0.smithi142.stdout:7/434: creat d7/d27/d44/d4c/f8c x:0 0 0 2024-03-29T04:44:59.585 INFO:tasks.workunit.client.0.smithi142.stdout:7/435: mkdir d7/dc/de/d8d 0 2024-03-29T04:44:59.585 INFO:tasks.workunit.client.0.smithi142.stdout:6/381: symlink d1c/d42/d52/l79 0 2024-03-29T04:44:59.586 INFO:tasks.workunit.client.0.smithi142.stdout:7/436: getdents d7/dc/de/d8d 0 2024-03-29T04:44:59.588 INFO:tasks.workunit.client.0.smithi142.stdout:7/437: write d7/dc/d4b/d71/f1f [9426814,122628] 0 2024-03-29T04:44:59.591 INFO:tasks.workunit.client.0.smithi142.stdout:7/438: truncate d7/dc/f68 1094442 0 2024-03-29T04:44:59.593 INFO:tasks.workunit.client.0.smithi142.stdout:6/382: dwrite d1c/d45/d51/f58 [0,4194304] 0 2024-03-29T04:44:59.593 INFO:tasks.workunit.client.0.smithi142.stdout:7/439: write d7/dc/de/d20/d3f/f41 [500530,125990] 0 2024-03-29T04:44:59.597 INFO:tasks.workunit.client.0.smithi142.stdout:7/440: mknod d7/dc/de/d8d/c8e 0 2024-03-29T04:44:59.603 INFO:tasks.workunit.client.0.smithi142.stdout:6/383: dwrite d1c/d45/f4e [0,4194304] 0 2024-03-29T04:44:59.606 INFO:tasks.workunit.client.0.smithi142.stdout:7/441: dwrite d7/f19 [0,4194304] 0 2024-03-29T04:44:59.607 INFO:tasks.workunit.client.0.smithi142.stdout:7/442: read - d7/d27/f81 zero size 2024-03-29T04:44:59.608 INFO:tasks.workunit.client.0.smithi142.stdout:6/384: truncate d1c/d33/d46/d71/f68 1022229 0 2024-03-29T04:44:59.610 INFO:tasks.workunit.client.0.smithi142.stdout:6/385: stat l15 0 2024-03-29T04:44:59.611 INFO:tasks.workunit.client.0.smithi142.stdout:6/386: dread - d1c/d33/d46/f69 zero size 2024-03-29T04:44:59.618 INFO:tasks.workunit.client.0.smithi142.stdout:6/387: rmdir d1c/d33/d46/d61/d73 0 2024-03-29T04:44:59.619 INFO:tasks.workunit.client.0.smithi142.stdout:5/464: write df/d2c/f71 [4172408,69502] 0 2024-03-29T04:44:59.620 INFO:tasks.workunit.client.0.smithi142.stdout:6/388: write d1c/d45/d51/f58 [3790118,76781] 0 2024-03-29T04:44:59.621 INFO:tasks.workunit.client.0.smithi142.stdout:4/341: dwrite d0/f48 [4194304,4194304] 0 2024-03-29T04:44:59.621 INFO:tasks.workunit.client.0.smithi142.stdout:5/465: readlink df/d16/l83 0 2024-03-29T04:44:59.621 INFO:tasks.workunit.client.0.smithi142.stdout:6/389: chown d1c/d33/f5a 1643 1 2024-03-29T04:44:59.625 INFO:tasks.workunit.client.0.smithi142.stdout:6/390: chown d1c/d33/d46/l57 110800 1 2024-03-29T04:44:59.625 INFO:tasks.workunit.client.0.smithi142.stdout:4/342: rmdir d0/dc/d15/d3d 39 2024-03-29T04:44:59.626 INFO:tasks.workunit.client.0.smithi142.stdout:1/408: sync 2024-03-29T04:44:59.626 INFO:tasks.workunit.client.0.smithi142.stdout:2/343: sync 2024-03-29T04:44:59.626 INFO:tasks.workunit.client.0.smithi142.stdout:3/341: sync 2024-03-29T04:44:59.627 INFO:tasks.workunit.client.0.smithi142.stdout:8/400: sync 2024-03-29T04:44:59.627 INFO:tasks.workunit.client.0.smithi142.stdout:6/391: write d1c/d45/f4e [2549443,27577] 0 2024-03-29T04:44:59.628 INFO:tasks.workunit.client.0.smithi142.stdout:9/316: sync 2024-03-29T04:44:59.630 INFO:tasks.workunit.client.0.smithi142.stdout:4/343: fdatasync d0/dc/d15/f59 0 2024-03-29T04:44:59.631 INFO:tasks.workunit.client.0.smithi142.stdout:2/344: creat de/d11/d1e/f72 x:0 0 0 2024-03-29T04:44:59.634 INFO:tasks.workunit.client.0.smithi142.stdout:8/401: symlink d5/d1d/d2f/l89 0 2024-03-29T04:44:59.635 INFO:tasks.workunit.client.0.smithi142.stdout:4/344: dread - d0/d9/d11/d4c/d5f/f6c zero size 2024-03-29T04:44:59.635 INFO:tasks.workunit.client.0.smithi142.stdout:5/466: dwrite df/d1a/d21/f3e [4194304,4194304] 0 2024-03-29T04:44:59.636 INFO:tasks.workunit.client.0.smithi142.stdout:1/409: dread d8/db/dc/d19/d1d/f11 [0,4194304] 0 2024-03-29T04:44:59.639 INFO:tasks.workunit.client.0.smithi142.stdout:7/443: write d7/dc/de/f26 [923331,100228] 0 2024-03-29T04:44:59.641 INFO:tasks.workunit.client.0.smithi142.stdout:8/402: chown d5/d1d/d2f/d5f/d48/d52/f7e 291477163 1 2024-03-29T04:44:59.641 INFO:tasks.workunit.client.0.smithi142.stdout:1/410: chown d8/db/dc/d77 3 1 2024-03-29T04:44:59.641 INFO:tasks.workunit.client.0.smithi142.stdout:4/345: chown d0/d9/d19/f1a 1740457001 1 2024-03-29T04:44:59.642 INFO:tasks.workunit.client.0.smithi142.stdout:5/467: mkdir df/d29/d7a/d97 0 2024-03-29T04:44:59.643 INFO:tasks.workunit.client.0.smithi142.stdout:6/392: getdents d1c/d33/d46/d71 0 2024-03-29T04:44:59.644 INFO:tasks.workunit.client.0.smithi142.stdout:9/317: dwrite d0/d2/d6/d31/f49 [0,4194304] 0 2024-03-29T04:44:59.646 INFO:tasks.workunit.client.0.smithi142.stdout:8/403: rmdir d5/d1d/d2f/d45 39 2024-03-29T04:44:59.646 INFO:tasks.workunit.client.0.smithi142.stdout:5/468: dread - df/d16/d24/f56 zero size 2024-03-29T04:44:59.647 INFO:tasks.workunit.client.0.smithi142.stdout:2/345: dwrite f9 [0,4194304] 0 2024-03-29T04:44:59.651 INFO:tasks.workunit.client.0.smithi142.stdout:4/346: creat d0/d12/d28/d47/f74 x:0 0 0 2024-03-29T04:44:59.651 INFO:tasks.workunit.client.0.smithi142.stdout:2/346: read de/f16 [4091207,127194] 0 2024-03-29T04:44:59.651 INFO:tasks.workunit.client.0.smithi142.stdout:6/393: rmdir d1c/d42/d52 39 2024-03-29T04:44:59.652 INFO:tasks.workunit.client.0.smithi142.stdout:1/411: mkdir d8/db/dc/d19/d1d/d1f/d50/d96 0 2024-03-29T04:44:59.652 INFO:tasks.workunit.client.0.smithi142.stdout:8/404: fsync d5/f22 0 2024-03-29T04:44:59.657 INFO:tasks.workunit.client.0.smithi142.stdout:7/444: link d7/dc/d4b/d71/l52 d7/dc/l8f 0 2024-03-29T04:44:59.658 INFO:tasks.workunit.client.0.smithi142.stdout:8/405: write d5/d1d/d2f/d76/d2d/f57 [3118310,4538] 0 2024-03-29T04:44:59.658 INFO:tasks.workunit.client.0.smithi142.stdout:1/412: mkdir d8/db/dc/d19/d1d/d1f/d97 0 2024-03-29T04:44:59.661 INFO:tasks.workunit.client.0.smithi142.stdout:7/445: creat d7/f90 x:0 0 0 2024-03-29T04:44:59.662 INFO:tasks.workunit.client.0.smithi142.stdout:8/406: mknod d5/d1d/d2f/d76/d2d/c8a 0 2024-03-29T04:44:59.662 INFO:tasks.workunit.client.0.smithi142.stdout:5/469: dwrite df/f14 [0,4194304] 0 2024-03-29T04:44:59.663 INFO:tasks.workunit.client.0.smithi142.stdout:2/347: rename de/d11/d2b/c52 to de/d11/c73 0 2024-03-29T04:44:59.664 INFO:tasks.workunit.client.0.smithi142.stdout:5/470: dread - df/d1a/d21/d55/f81 zero size 2024-03-29T04:44:59.666 INFO:tasks.workunit.client.0.smithi142.stdout:1/413: dread d8/db/dc/d19/d1d/d2d/d44/d53/f6a [0,4194304] 0 2024-03-29T04:44:59.666 INFO:tasks.workunit.client.0.smithi142.stdout:7/446: rename d7/dc/de/d20/d3f/f50 to d7/d27/f91 0 2024-03-29T04:44:59.668 INFO:tasks.workunit.client.0.smithi142.stdout:5/471: creat df/d29/d6e/d8a/f98 x:0 0 0 2024-03-29T04:44:59.670 INFO:tasks.workunit.client.0.smithi142.stdout:2/348: rename de/f10 to de/d11/d17/d5d/f74 0 2024-03-29T04:44:59.670 INFO:tasks.workunit.client.0.smithi142.stdout:7/447: mkdir d7/d27/d3a/d92 0 2024-03-29T04:44:59.674 INFO:tasks.workunit.client.0.smithi142.stdout:8/407: rename d5/d1d/d2f/d76/d3c/f3e to d5/d1d/d2f/d76/d4c/f8b 0 2024-03-29T04:44:59.675 INFO:tasks.workunit.client.0.smithi142.stdout:8/408: chown d5/d1d/d2f/d5f/d48/d52/f72 497027 1 2024-03-29T04:44:59.676 INFO:tasks.workunit.client.0.smithi142.stdout:5/472: mknod df/d16/d24/d2e/c99 0 2024-03-29T04:44:59.681 INFO:tasks.workunit.client.0.smithi142.stdout:6/394: write d1c/d33/d46/f48 [3714422,28950] 0 2024-03-29T04:44:59.681 INFO:tasks.workunit.client.0.smithi142.stdout:5/473: unlink df/d1a/d28/f49 0 2024-03-29T04:44:59.683 INFO:tasks.workunit.client.0.smithi142.stdout:8/409: dwrite d5/d1d/d2f/d76/d2d/f4a [0,4194304] 0 2024-03-29T04:44:59.684 INFO:tasks.workunit.client.0.smithi142.stdout:8/410: chown d5/d1d/d2f/d76 5813463 1 2024-03-29T04:44:59.684 INFO:tasks.workunit.client.0.smithi142.stdout:2/349: mknod de/d11/d1e/d22/c75 0 2024-03-29T04:44:59.686 INFO:tasks.workunit.client.0.smithi142.stdout:5/474: readlink df/d16/d25/l62 0 2024-03-29T04:44:59.688 INFO:tasks.workunit.client.0.smithi142.stdout:6/395: rename d1c/l28 to d1c/d33/d46/d71/l7a 0 2024-03-29T04:44:59.691 INFO:tasks.workunit.client.0.smithi142.stdout:5/475: rmdir df/d1a/d21/d27 39 2024-03-29T04:44:59.695 INFO:tasks.workunit.client.0.smithi142.stdout:2/350: rename de/f36 to de/d11/d2b/d3e/f76 0 2024-03-29T04:44:59.697 INFO:tasks.workunit.client.0.smithi142.stdout:5/476: dwrite df/d1a/d21/f3e [4194304,4194304] 0 2024-03-29T04:44:59.698 INFO:tasks.workunit.client.0.smithi142.stdout:5/477: write df/d16/f26 [9865038,38008] 0 2024-03-29T04:44:59.700 INFO:tasks.workunit.client.0.smithi142.stdout:6/396: creat d1c/d42/d52/f7b x:0 0 0 2024-03-29T04:44:59.700 INFO:tasks.workunit.client.0.smithi142.stdout:5/478: read df/d16/d25/f37 [540067,124978] 0 2024-03-29T04:44:59.700 INFO:tasks.workunit.client.0.smithi142.stdout:5/479: readlink df/d16/d25/l62 0 2024-03-29T04:44:59.701 INFO:tasks.workunit.client.0.smithi142.stdout:5/480: dread - df/d16/d24/f56 zero size 2024-03-29T04:44:59.701 INFO:tasks.workunit.client.0.smithi142.stdout:5/481: chown df/d16/d24/d2e/c6a 0 1 2024-03-29T04:44:59.702 INFO:tasks.workunit.client.0.smithi142.stdout:2/351: rmdir de/d11/d17/d5d 39 2024-03-29T04:44:59.705 INFO:tasks.workunit.client.0.smithi142.stdout:1/414: truncate d8/db/dc/d19/d1d/d2d/d44/d53/f7b 70121 0 2024-03-29T04:44:59.707 INFO:tasks.workunit.client.0.smithi142.stdout:6/397: stat d1c/c1f 0 2024-03-29T04:44:59.713 INFO:tasks.workunit.client.0.smithi142.stdout:5/482: creat df/d1a/d21/d27/d45/f9a x:0 0 0 2024-03-29T04:44:59.713 INFO:tasks.workunit.client.0.smithi142.stdout:5/483: dread - df/d1a/d28/f8d zero size 2024-03-29T04:44:59.714 INFO:tasks.workunit.client.0.smithi142.stdout:8/411: truncate d5/d1d/d2f/d76/d2d/f57 3717794 0 2024-03-29T04:44:59.715 INFO:tasks.workunit.client.0.smithi142.stdout:5/484: truncate df/d16/d25/f37 1440976 0 2024-03-29T04:44:59.716 INFO:tasks.workunit.client.0.smithi142.stdout:8/412: creat d5/d1d/d24/f8c x:0 0 0 2024-03-29T04:44:59.717 INFO:tasks.workunit.client.0.smithi142.stdout:5/485: mknod df/d29/d5d/d75/c9b 0 2024-03-29T04:44:59.718 INFO:tasks.workunit.client.0.smithi142.stdout:8/413: rename d5 to d5/d1d/d2f/d76/d2d/d8d 22 2024-03-29T04:44:59.718 INFO:tasks.workunit.client.0.smithi142.stdout:6/398: getdents d1c/d33/d46/d61 0 2024-03-29T04:44:59.719 INFO:tasks.workunit.client.0.smithi142.stdout:2/352: dread de/d11/d17/f25 [0,4194304] 0 2024-03-29T04:44:59.719 INFO:tasks.workunit.client.0.smithi142.stdout:8/414: fsync d5/d1d/d2f/d5f/d48/f61 0 2024-03-29T04:44:59.721 INFO:tasks.workunit.client.0.smithi142.stdout:6/399: chown le 381544348 1 2024-03-29T04:44:59.723 INFO:tasks.workunit.client.0.smithi142.stdout:5/486: creat df/d29/d7a/d97/f9c x:0 0 0 2024-03-29T04:44:59.728 INFO:tasks.workunit.client.0.smithi142.stdout:1/415: dwrite d8/db/dc/d77/d31/d4d/d8c/f95 [0,4194304] 0 2024-03-29T04:44:59.729 INFO:tasks.workunit.client.0.smithi142.stdout:2/353: rename de/d11/d1e/c53 to de/d11/d2b/d3e/c77 0 2024-03-29T04:44:59.730 INFO:tasks.workunit.client.0.smithi142.stdout:5/487: mkdir df/d1a/d21/d9d 0 2024-03-29T04:44:59.736 INFO:tasks.workunit.client.0.smithi142.stdout:8/415: getdents d5/d1d/d24 0 2024-03-29T04:44:59.741 INFO:tasks.workunit.client.0.smithi142.stdout:8/416: creat d5/d1d/d2f/d5f/d48/f8e x:0 0 0 2024-03-29T04:44:59.743 INFO:tasks.workunit.client.0.smithi142.stdout:5/488: creat df/d1a/f9e x:0 0 0 2024-03-29T04:44:59.744 INFO:tasks.workunit.client.0.smithi142.stdout:2/354: write de/d11/d1e/d22/d27/f66 [95362,50904] 0 2024-03-29T04:44:59.745 INFO:tasks.workunit.client.0.smithi142.stdout:8/417: chown d5/d1d/d2f/d5f/c47 3385 1 2024-03-29T04:44:59.746 INFO:tasks.workunit.client.0.smithi142.stdout:5/489: creat df/d16/d50/f9f x:0 0 0 2024-03-29T04:44:59.748 INFO:tasks.workunit.client.0.smithi142.stdout:5/490: dread - df/d1a/d28/f8d zero size 2024-03-29T04:44:59.750 INFO:tasks.workunit.client.0.smithi142.stdout:8/418: creat d5/d1d/d2f/d76/d4c/d60/f8f x:0 0 0 2024-03-29T04:44:59.750 INFO:tasks.workunit.client.0.smithi142.stdout:5/491: readlink df/d1a/d21/d55/d66/l7d 0 2024-03-29T04:44:59.750 INFO:tasks.workunit.client.0.smithi142.stdout:2/355: creat de/d11/d1e/d4d/d61/f78 x:0 0 0 2024-03-29T04:44:59.751 INFO:tasks.workunit.client.0.smithi142.stdout:1/416: dwrite d8/db/dc/d19/d1d/d15/f16 [0,4194304] 0 2024-03-29T04:44:59.751 INFO:tasks.workunit.client.0.smithi142.stdout:0/397: sync 2024-03-29T04:44:59.752 INFO:tasks.workunit.client.0.smithi142.stdout:6/400: dread fc [4194304,4194304] 0 2024-03-29T04:44:59.753 INFO:tasks.workunit.client.0.smithi142.stdout:2/356: readlink de/d11/d2b/l33 0 2024-03-29T04:44:59.754 INFO:tasks.workunit.client.0.smithi142.stdout:8/419: unlink d5/d1d/d2f/d5f/c6e 0 2024-03-29T04:44:59.756 INFO:tasks.workunit.client.0.smithi142.stdout:2/357: chown de/f19 266095055 1 2024-03-29T04:44:59.757 INFO:tasks.workunit.client.0.smithi142.stdout:5/492: truncate df/d16/d24/f88 1654397 0 2024-03-29T04:44:59.757 INFO:tasks.workunit.client.0.smithi142.stdout:0/398: chown d7/d9/d1e/f71 456603 1 2024-03-29T04:44:59.757 INFO:tasks.workunit.client.0.smithi142.stdout:8/420: dread - d5/d1d/f85 zero size 2024-03-29T04:44:59.757 INFO:tasks.workunit.client.0.smithi142.stdout:6/401: symlink d1c/d33/d46/d61/l7c 0 2024-03-29T04:44:59.760 INFO:tasks.workunit.client.0.smithi142.stdout:0/399: write d7/d9/f10 [1256966,17930] 0 2024-03-29T04:44:59.764 INFO:tasks.workunit.client.0.smithi142.stdout:2/358: dwrite de/d11/d17/d4a/f55 [0,4194304] 0 2024-03-29T04:44:59.767 INFO:tasks.workunit.client.0.smithi142.stdout:8/421: link d5/d1d/d2f/d76/d4c/d60/f8f d5/d1d/d2f/f90 0 2024-03-29T04:44:59.769 INFO:tasks.workunit.client.0.smithi142.stdout:0/400: creat d7/d13/f7d x:0 0 0 2024-03-29T04:44:59.775 INFO:tasks.workunit.client.0.smithi142.stdout:5/493: dwrite df/d16/d24/d79/f5b [0,4194304] 0 2024-03-29T04:44:59.779 INFO:tasks.workunit.client.0.smithi142.stdout:5/494: stat df/d29/d5d/d75 0 2024-03-29T04:44:59.780 INFO:tasks.workunit.client.0.smithi142.stdout:2/359: dwrite f9 [0,4194304] 0 2024-03-29T04:44:59.787 INFO:tasks.workunit.client.0.smithi142.stdout:8/422: dwrite d5/d1d/d2f/d76/f41 [4194304,4194304] 0 2024-03-29T04:44:59.788 INFO:tasks.workunit.client.0.smithi142.stdout:1/417: dread d8/db/dc/d19/d1d/d15/f47 [0,4194304] 0 2024-03-29T04:44:59.789 INFO:tasks.workunit.client.0.smithi142.stdout:8/423: write d5/f18 [9903867,97576] 0 2024-03-29T04:44:59.793 INFO:tasks.workunit.client.0.smithi142.stdout:0/401: dwrite d7/d9/d54/f64 [0,4194304] 0 2024-03-29T04:44:59.793 INFO:tasks.workunit.client.0.smithi142.stdout:8/424: fdatasync d5/d1d/d2f/d76/d4c/d60/f8f 0 2024-03-29T04:44:59.794 INFO:tasks.workunit.client.0.smithi142.stdout:1/418: creat d8/db/dc/d77/d5a/f98 x:0 0 0 2024-03-29T04:44:59.800 INFO:tasks.workunit.client.0.smithi142.stdout:0/402: symlink d7/d13/l7e 0 2024-03-29T04:44:59.801 INFO:tasks.workunit.client.0.smithi142.stdout:8/425: creat d5/d1d/d81/f91 x:0 0 0 2024-03-29T04:44:59.801 INFO:tasks.workunit.client.0.smithi142.stdout:0/403: readlink d7/d13/d67/d72/l5d 0 2024-03-29T04:44:59.803 INFO:tasks.workunit.client.0.smithi142.stdout:8/426: symlink d5/d1d/d2f/d76/d3c/d70/l92 0 2024-03-29T04:44:59.804 INFO:tasks.workunit.client.0.smithi142.stdout:8/427: read - d5/d1d/d59/f77 zero size 2024-03-29T04:44:59.806 INFO:tasks.workunit.client.0.smithi142.stdout:8/428: mkdir d5/d93 0 2024-03-29T04:44:59.809 INFO:tasks.workunit.client.0.smithi142.stdout:8/429: creat d5/f94 x:0 0 0 2024-03-29T04:44:59.810 INFO:tasks.workunit.client.0.smithi142.stdout:5/495: dwrite df/d1a/f1f [4194304,4194304] 0 2024-03-29T04:44:59.811 INFO:tasks.workunit.client.0.smithi142.stdout:8/430: write d5/d1d/d2f/d76/d2d/f4f [172106,121869] 0 2024-03-29T04:44:59.812 INFO:tasks.workunit.client.0.smithi142.stdout:8/431: readlink d5/d1d/d2f/d76/d2d/l64 0 2024-03-29T04:44:59.814 INFO:tasks.workunit.client.0.smithi142.stdout:8/432: rmdir d5/d2e 39 2024-03-29T04:44:59.815 INFO:tasks.workunit.client.0.smithi142.stdout:5/496: creat df/d16/d24/fa0 x:0 0 0 2024-03-29T04:44:59.823 INFO:tasks.workunit.client.0.smithi142.stdout:1/419: truncate d8/fa 2418839 0 2024-03-29T04:44:59.825 INFO:tasks.workunit.client.0.smithi142.stdout:8/433: rename d5/d1d/d2f/d5f/d48/d52/f72 to d5/d1d/d2f/d45/f95 0 2024-03-29T04:44:59.826 INFO:tasks.workunit.client.0.smithi142.stdout:5/497: dwrite df/d1a/f3f [0,4194304] 0 2024-03-29T04:44:59.826 INFO:tasks.workunit.client.0.smithi142.stdout:1/420: unlink d8/db/dc/d19/d1d/d1f/d50/c65 0 2024-03-29T04:44:59.827 INFO:tasks.workunit.client.0.smithi142.stdout:5/498: dread - df/d16/d24/d2e/f8f zero size 2024-03-29T04:44:59.828 INFO:tasks.workunit.client.0.smithi142.stdout:8/434: mknod d5/c96 0 2024-03-29T04:44:59.829 INFO:tasks.workunit.client.0.smithi142.stdout:5/499: read df/d1a/f2f [2693177,31078] 0 2024-03-29T04:44:59.829 INFO:tasks.workunit.client.0.smithi142.stdout:8/435: dread - d5/d1d/d2f/f50 zero size 2024-03-29T04:44:59.829 INFO:tasks.workunit.client.0.smithi142.stdout:1/421: symlink d8/l99 0 2024-03-29T04:44:59.830 INFO:tasks.workunit.client.0.smithi142.stdout:3/342: sync 2024-03-29T04:44:59.831 INFO:tasks.workunit.client.0.smithi142.stdout:9/318: sync 2024-03-29T04:44:59.831 INFO:tasks.workunit.client.0.smithi142.stdout:7/448: sync 2024-03-29T04:44:59.831 INFO:tasks.workunit.client.0.smithi142.stdout:4/347: sync 2024-03-29T04:44:59.831 INFO:tasks.workunit.client.0.smithi142.stdout:1/422: chown d8/db/dc/d77/d31 9 1 2024-03-29T04:44:59.834 INFO:tasks.workunit.client.0.smithi142.stdout:3/343: creat d3/d29/d2b/f7d x:0 0 0 2024-03-29T04:44:59.835 INFO:tasks.workunit.client.0.smithi142.stdout:4/348: rmdir d0/d9/d19 39 2024-03-29T04:44:59.837 INFO:tasks.workunit.client.0.smithi142.stdout:1/423: symlink d8/db/dc/d19/d1d/d2d/d8b/l9a 0 2024-03-29T04:44:59.840 INFO:tasks.workunit.client.0.smithi142.stdout:3/344: creat d3/d1f/f7e x:0 0 0 2024-03-29T04:44:59.841 INFO:tasks.workunit.client.0.smithi142.stdout:8/436: dwrite d5/d1d/d2f/f68 [0,4194304] 0 2024-03-29T04:44:59.843 INFO:tasks.workunit.client.0.smithi142.stdout:9/319: getdents d0/d2/d6/d27/d3c/d4f 0 2024-03-29T04:44:59.843 INFO:tasks.workunit.client.0.smithi142.stdout:5/500: dwrite df/f14 [0,4194304] 0 2024-03-29T04:44:59.845 INFO:tasks.workunit.client.0.smithi142.stdout:9/320: write d0/d12/f54 [767942,42232] 0 2024-03-29T04:44:59.845 INFO:tasks.workunit.client.0.smithi142.stdout:1/424: unlink d8/db/l82 0 2024-03-29T04:44:59.849 INFO:tasks.workunit.client.0.smithi142.stdout:3/345: chown d3/d29/d2b/c42 254 1 2024-03-29T04:44:59.851 INFO:tasks.workunit.client.0.smithi142.stdout:7/449: link d7/dc/de/f5c d7/dc/d4b/d71/d30/d33/f93 0 2024-03-29T04:44:59.851 INFO:tasks.workunit.client.0.smithi142.stdout:5/501: fsync df/d1a/f22 0 2024-03-29T04:44:59.857 INFO:tasks.workunit.client.0.smithi142.stdout:4/349: dread d0/d12/d28/f3a [0,4194304] 0 2024-03-29T04:44:59.859 INFO:tasks.workunit.client.0.smithi142.stdout:5/502: mknod df/d16/d24/d79/ca1 0 2024-03-29T04:44:59.860 INFO:tasks.workunit.client.0.smithi142.stdout:1/425: dwrite d8/db/dc/d19/d1d/d2d/f3e [0,4194304] 0 2024-03-29T04:44:59.863 INFO:tasks.workunit.client.0.smithi142.stdout:4/350: symlink d0/dc/d15/l75 0 2024-03-29T04:44:59.866 INFO:tasks.workunit.client.0.smithi142.stdout:5/503: dread df/d1a/d28/f2d [0,4194304] 0 2024-03-29T04:44:59.868 INFO:tasks.workunit.client.0.smithi142.stdout:7/450: dwrite d7/fa [0,4194304] 0 2024-03-29T04:44:59.870 INFO:tasks.workunit.client.0.smithi142.stdout:1/426: creat d8/db/dc/d19/d1d/d1f/f9b x:0 0 0 2024-03-29T04:44:59.872 INFO:tasks.workunit.client.0.smithi142.stdout:4/351: symlink d0/dc/l76 0 2024-03-29T04:44:59.872 INFO:tasks.workunit.client.0.smithi142.stdout:5/504: readlink df/d16/l1b 0 2024-03-29T04:44:59.877 INFO:tasks.workunit.client.0.smithi142.stdout:8/437: dwrite d5/f25 [0,4194304] 0 2024-03-29T04:44:59.877 INFO:tasks.workunit.client.0.smithi142.stdout:5/505: creat df/fa2 x:0 0 0 2024-03-29T04:44:59.878 INFO:tasks.workunit.client.0.smithi142.stdout:3/346: dwrite d3/d1f/d34/d4d/d50/f51 [0,4194304] 0 2024-03-29T04:44:59.879 INFO:tasks.workunit.client.0.smithi142.stdout:1/427: rmdir d8/db/d8e 0 2024-03-29T04:44:59.880 INFO:tasks.workunit.client.0.smithi142.stdout:1/428: read - d8/db/dc/d19/d1d/f5d zero size 2024-03-29T04:44:59.880 INFO:tasks.workunit.client.0.smithi142.stdout:4/352: creat d0/d9/d11/d4c/d64/f77 x:0 0 0 2024-03-29T04:44:59.882 INFO:tasks.workunit.client.0.smithi142.stdout:7/451: link d7/dc/d4b/d71/f76 d7/d27/d44/f94 0 2024-03-29T04:44:59.884 INFO:tasks.workunit.client.0.smithi142.stdout:8/438: creat d5/d1d/d2f/d76/d3c/d70/f97 x:0 0 0 2024-03-29T04:44:59.884 INFO:tasks.workunit.client.0.smithi142.stdout:7/452: chown d7/dc/d4b/d71/f24 17 1 2024-03-29T04:44:59.885 INFO:tasks.workunit.client.0.smithi142.stdout:4/353: read - d0/d45/f61 zero size 2024-03-29T04:44:59.887 INFO:tasks.workunit.client.0.smithi142.stdout:9/321: dread d0/d2/d6/d16/f37 [0,4194304] 0 2024-03-29T04:44:59.887 INFO:tasks.workunit.client.0.smithi142.stdout:7/453: dread - d7/d27/d44/d4c/f8c zero size 2024-03-29T04:44:59.888 INFO:tasks.workunit.client.0.smithi142.stdout:4/354: chown d0/d45/f61 0 1 2024-03-29T04:44:59.888 INFO:tasks.workunit.client.0.smithi142.stdout:1/429: read f5 [2179494,69380] 0 2024-03-29T04:44:59.888 INFO:tasks.workunit.client.0.smithi142.stdout:9/322: readlink d0/d12/l25 0 2024-03-29T04:44:59.889 INFO:tasks.workunit.client.0.smithi142.stdout:3/347: getdents d3/d1f/d34/d6 0 2024-03-29T04:44:59.889 INFO:tasks.workunit.client.0.smithi142.stdout:8/439: mknod d5/d1d/d2f/d5f/d48/d52/c98 0 2024-03-29T04:44:59.891 INFO:tasks.workunit.client.0.smithi142.stdout:4/355: mkdir d0/d9/d11/d78 0 2024-03-29T04:44:59.892 INFO:tasks.workunit.client.0.smithi142.stdout:8/440: chown d5/d1d/d2f/d76/d3c 112780460 1 2024-03-29T04:44:59.892 INFO:tasks.workunit.client.0.smithi142.stdout:3/348: creat d3/d29/d2b/d40/d67/f7f x:0 0 0 2024-03-29T04:44:59.893 INFO:tasks.workunit.client.0.smithi142.stdout:1/430: chown d8/db/dc/d77/d31/f61 536 1 2024-03-29T04:44:59.893 INFO:tasks.workunit.client.0.smithi142.stdout:9/323: symlink d0/d2/d6/l65 0 2024-03-29T04:44:59.895 INFO:tasks.workunit.client.0.smithi142.stdout:8/441: chown d5/d1d/d2f/d76/d2d/l5b 459 1 2024-03-29T04:44:59.896 INFO:tasks.workunit.client.0.smithi142.stdout:4/356: rename d0/f67 to d0/d9/d11/d78/f79 0 2024-03-29T04:44:59.897 INFO:tasks.workunit.client.0.smithi142.stdout:1/431: mkdir d8/db/dc/d19/d1d/d15/d9c 0 2024-03-29T04:44:59.897 INFO:tasks.workunit.client.0.smithi142.stdout:8/442: read - d5/f94 zero size 2024-03-29T04:44:59.897 INFO:tasks.workunit.client.0.smithi142.stdout:4/357: chown d0/l51 3 1 2024-03-29T04:44:59.898 INFO:tasks.workunit.client.0.smithi142.stdout:9/324: getdents d0/d2/d6/d27/d3c/d4f 0 2024-03-29T04:44:59.898 INFO:tasks.workunit.client.0.smithi142.stdout:7/454: dwrite d7/dc/d4b/d71/d30/f3e [4194304,4194304] 0 2024-03-29T04:44:59.900 INFO:tasks.workunit.client.0.smithi142.stdout:8/443: dread - d5/d1d/d2f/d76/d4c/d60/f84 zero size 2024-03-29T04:44:59.902 INFO:tasks.workunit.client.0.smithi142.stdout:4/358: truncate d0/d9/d11/d4c/d5f/f6c 698435 0 2024-03-29T04:44:59.903 INFO:tasks.workunit.client.0.smithi142.stdout:1/432: mknod d8/db/dc/d19/d1d/d2d/d44/d4b/c9d 0 2024-03-29T04:44:59.905 INFO:tasks.workunit.client.0.smithi142.stdout:1/433: fdatasync d8/db/dc/d19/f57 0 2024-03-29T04:44:59.905 INFO:tasks.workunit.client.0.smithi142.stdout:7/455: dread d7/d27/d44/d4c/f80 [0,4194304] 0 2024-03-29T04:44:59.906 INFO:tasks.workunit.client.0.smithi142.stdout:4/359: mkdir d0/d12/d28/d47/d7a 0 2024-03-29T04:44:59.910 INFO:tasks.workunit.client.0.smithi142.stdout:8/444: dwrite d5/d1d/d2f/d76/f88 [0,4194304] 0 2024-03-29T04:44:59.913 INFO:tasks.workunit.client.0.smithi142.stdout:1/434: mkdir d8/db/dc/d19/d1d/d1f/d97/d9e 0 2024-03-29T04:44:59.914 INFO:tasks.workunit.client.0.smithi142.stdout:8/445: symlink d5/d1d/d2f/d76/d3c/l99 0 2024-03-29T04:44:59.915 INFO:tasks.workunit.client.0.smithi142.stdout:1/435: stat d8/db/dc/d77/d31/d4d 0 2024-03-29T04:44:59.916 INFO:tasks.workunit.client.0.smithi142.stdout:7/456: fsync d7/dc/d4b/d71/f76 0 2024-03-29T04:44:59.944 INFO:tasks.workunit.client.0.smithi142.stdout:6/402: sync 2024-03-29T04:44:59.956 INFO:tasks.workunit.client.0.smithi142.stdout:4/360: dwrite d0/d12/f3c [0,4194304] 0 2024-03-29T04:44:59.957 INFO:tasks.workunit.client.0.smithi142.stdout:8/446: creat d5/d1d/d2f/d45/f9a x:0 0 0 2024-03-29T04:44:59.958 INFO:tasks.workunit.client.0.smithi142.stdout:4/361: chown d0/d12/d28/d47 77 1 2024-03-29T04:44:59.958 INFO:tasks.workunit.client.0.smithi142.stdout:1/436: truncate d8/db/dc/d77/d5a/f98 80716 0 2024-03-29T04:44:59.959 INFO:tasks.workunit.client.0.smithi142.stdout:7/457: chown d7/dc/l8f 28559445 1 2024-03-29T04:44:59.960 INFO:tasks.workunit.client.0.smithi142.stdout:8/447: symlink d5/d1d/d2f/d76/d4c/l9b 0 2024-03-29T04:44:59.961 INFO:tasks.workunit.client.0.smithi142.stdout:4/362: creat d0/d12/f7b x:0 0 0 2024-03-29T04:44:59.961 INFO:tasks.workunit.client.0.smithi142.stdout:8/448: chown d5/d1d/d2f/d5f/l44 1998383446 1 2024-03-29T04:44:59.962 INFO:tasks.workunit.client.0.smithi142.stdout:7/458: unlink d7/dc/f68 0 2024-03-29T04:44:59.964 INFO:tasks.workunit.client.0.smithi142.stdout:7/459: write d7/dc/d4b/d71/d30/d33/f38 [11620788,88752] 0 2024-03-29T04:44:59.964 INFO:tasks.workunit.client.0.smithi142.stdout:7/460: dread - d7/d27/f88 zero size 2024-03-29T04:44:59.965 INFO:tasks.workunit.client.0.smithi142.stdout:1/437: fdatasync d8/db/dc/d19/d1d/d15/f17 0 2024-03-29T04:44:59.966 INFO:tasks.workunit.client.0.smithi142.stdout:6/403: write d1c/f34 [337438,51885] 0 2024-03-29T04:44:59.969 INFO:tasks.workunit.client.0.smithi142.stdout:7/461: symlink d7/d27/d3a/l95 0 2024-03-29T04:44:59.970 INFO:tasks.workunit.client.0.smithi142.stdout:6/404: mknod d1c/d33/d46/d71/c7d 0 2024-03-29T04:44:59.970 INFO:tasks.workunit.client.0.smithi142.stdout:9/325: dread d0/d2/f1f [4194304,4194304] 0 2024-03-29T04:44:59.971 INFO:tasks.workunit.client.0.smithi142.stdout:7/462: dread - d7/dc/d4b/d71/d30/d33/f42 zero size 2024-03-29T04:44:59.973 INFO:tasks.workunit.client.0.smithi142.stdout:6/405: write f1a [2127112,2482] 0 2024-03-29T04:44:59.975 INFO:tasks.workunit.client.0.smithi142.stdout:8/449: rename d5/d1d/d2f/d5f/d48/d52 to d5/d1d/d2f/d76/d9c 0 2024-03-29T04:44:59.980 INFO:tasks.workunit.client.0.smithi142.stdout:8/450: mknod d5/d1d/d81/c9d 0 2024-03-29T04:44:59.981 INFO:tasks.workunit.client.0.smithi142.stdout:4/363: dread d0/d12/d28/d47/f55 [0,4194304] 0 2024-03-29T04:44:59.981 INFO:tasks.workunit.client.0.smithi142.stdout:1/438: getdents d8/db/dc/d77/d31/d4d 0 2024-03-29T04:44:59.981 INFO:tasks.workunit.client.0.smithi142.stdout:7/463: dwrite d7/f87 [0,4194304] 0 2024-03-29T04:44:59.983 INFO:tasks.workunit.client.0.smithi142.stdout:4/364: stat d0/d45 0 2024-03-29T04:44:59.987 INFO:tasks.workunit.client.0.smithi142.stdout:7/464: mknod d7/d27/d44/c96 0 2024-03-29T04:44:59.989 INFO:tasks.workunit.client.0.smithi142.stdout:4/365: truncate d0/d9/d11/d4c/d5f/f6c 1310091 0 2024-03-29T04:44:59.990 INFO:tasks.workunit.client.0.smithi142.stdout:6/406: dwrite f18 [0,4194304] 0 2024-03-29T04:44:59.991 INFO:tasks.workunit.client.0.smithi142.stdout:6/407: stat d1c/f39 0 2024-03-29T04:44:59.991 INFO:tasks.workunit.client.0.smithi142.stdout:9/326: rename d0/d12/f43 to d0/d2/d6/d16/f66 0 2024-03-29T04:44:59.993 INFO:tasks.workunit.client.0.smithi142.stdout:8/451: link d5/d1d/f85 d5/d93/f9e 0 2024-03-29T04:44:59.997 INFO:tasks.workunit.client.0.smithi142.stdout:9/327: mknod d0/d2/d6/d31/c67 0 2024-03-29T04:44:59.998 INFO:tasks.workunit.client.0.smithi142.stdout:4/366: dwrite d0/dc/d15/f53 [0,4194304] 0 2024-03-29T04:44:59.998 INFO:tasks.workunit.client.0.smithi142.stdout:7/465: mkdir d7/d27/d3a/d92/d97 0 2024-03-29T04:45:00.000 INFO:tasks.workunit.client.0.smithi142.stdout:9/328: write d0/d2/d6/d16/f37 [1845222,71067] 0 2024-03-29T04:45:00.004 INFO:tasks.workunit.client.0.smithi142.stdout:1/439: rename d8/db/dc/d19/d1d/d2d/d44/d4b/c9d to d8/db/c9f 0 2024-03-29T04:45:00.006 INFO:tasks.workunit.client.0.smithi142.stdout:4/367: rmdir d0/d12/d28 39 2024-03-29T04:45:00.006 INFO:tasks.workunit.client.0.smithi142.stdout:8/452: dwrite d5/f19 [0,4194304] 0 2024-03-29T04:45:00.006 INFO:tasks.workunit.client.0.smithi142.stdout:7/466: fdatasync d7/d27/f91 0 2024-03-29T04:45:00.007 INFO:tasks.workunit.client.0.smithi142.stdout:8/453: fdatasync d5/d1d/d2f/d76/d2d/f4a 0 2024-03-29T04:45:00.008 INFO:tasks.workunit.client.0.smithi142.stdout:7/467: truncate d7/f90 134177 0 2024-03-29T04:45:00.039 INFO:tasks.workunit.client.0.smithi142.stdout:2/360: sync 2024-03-29T04:45:00.040 INFO:tasks.workunit.client.0.smithi142.stdout:9/329: dwrite d0/d2/d61/f55 [0,4194304] 0 2024-03-29T04:45:00.043 INFO:tasks.workunit.client.0.smithi142.stdout:9/330: chown d0/f62 0 1 2024-03-29T04:45:00.047 INFO:tasks.workunit.client.0.smithi142.stdout:4/368: dwrite d0/d9/d11/f38 [0,4194304] 0 2024-03-29T04:45:00.051 INFO:tasks.workunit.client.0.smithi142.stdout:7/468: dwrite d7/dc/d4b/d71/f78 [0,4194304] 0 2024-03-29T04:45:00.051 INFO:tasks.workunit.client.0.smithi142.stdout:4/369: write d0/d9/f4d [1406019,29527] 0 2024-03-29T04:45:00.052 INFO:tasks.workunit.client.0.smithi142.stdout:8/454: dwrite d5/d1d/d2f/d76/d3c/f5e [0,4194304] 0 2024-03-29T04:45:00.053 INFO:tasks.workunit.client.0.smithi142.stdout:8/455: dread - d5/f94 zero size 2024-03-29T04:45:00.054 INFO:tasks.workunit.client.0.smithi142.stdout:7/469: dread d7/f31 [0,4194304] 0 2024-03-29T04:45:00.057 INFO:tasks.workunit.client.0.smithi142.stdout:8/456: creat d5/d1d/d2f/d45/f9f x:0 0 0 2024-03-29T04:45:00.058 INFO:tasks.workunit.client.0.smithi142.stdout:9/331: truncate d0/d2/f11 3005510 0 2024-03-29T04:45:00.058 INFO:tasks.workunit.client.0.smithi142.stdout:4/370: write d0/d9/d11/d78/f79 [1432790,111976] 0 2024-03-29T04:45:00.060 INFO:tasks.workunit.client.0.smithi142.stdout:9/332: chown d0/d2/c20 912 1 2024-03-29T04:45:00.061 INFO:tasks.workunit.client.0.smithi142.stdout:9/333: chown d0/d2/d6/d31/c67 188392786 1 2024-03-29T04:45:00.062 INFO:tasks.workunit.client.0.smithi142.stdout:7/470: creat d7/dc/de/d20/f98 x:0 0 0 2024-03-29T04:45:00.062 INFO:tasks.workunit.client.0.smithi142.stdout:9/334: fsync d0/d12/d50/f63 0 2024-03-29T04:45:00.063 INFO:tasks.workunit.client.0.smithi142.stdout:9/335: stat d0/d12/d50 0 2024-03-29T04:45:00.064 INFO:tasks.workunit.client.0.smithi142.stdout:7/471: chown d7/d27/d44/d4c/f8c 0 1 2024-03-29T04:45:00.066 INFO:tasks.workunit.client.0.smithi142.stdout:8/457: rename d5/d1d/d2f/d76/c58 to d5/d1d/d2f/d76/ca0 0 2024-03-29T04:45:00.067 INFO:tasks.workunit.client.0.smithi142.stdout:4/371: creat d0/dc/d15/d3d/d65/f7c x:0 0 0 2024-03-29T04:45:00.067 INFO:tasks.workunit.client.0.smithi142.stdout:8/458: fdatasync d5/d1d/d2f/f50 0 2024-03-29T04:45:00.069 INFO:tasks.workunit.client.0.smithi142.stdout:8/459: chown d5/d1d/d2f/d5f/c6d 307033157 1 2024-03-29T04:45:00.070 INFO:tasks.workunit.client.0.smithi142.stdout:4/372: symlink d0/dc/d15/d3d/d65/l7d 0 2024-03-29T04:45:00.071 INFO:tasks.workunit.client.0.smithi142.stdout:7/472: dwrite d7/dc/d4b/d71/d30/d33/f42 [0,4194304] 0 2024-03-29T04:45:00.073 INFO:tasks.workunit.client.0.smithi142.stdout:4/373: mknod d0/dc/d15/d3d/d65/c7e 0 2024-03-29T04:45:00.075 INFO:tasks.workunit.client.0.smithi142.stdout:8/460: creat d5/d1d/d2f/d5f/fa1 x:0 0 0 2024-03-29T04:45:00.076 INFO:tasks.workunit.client.0.smithi142.stdout:4/374: dread - d0/d12/d28/d47/f74 zero size 2024-03-29T04:45:00.078 INFO:tasks.workunit.client.0.smithi142.stdout:8/461: mkdir d5/d2e/da2 0 2024-03-29T04:45:00.079 INFO:tasks.workunit.client.0.smithi142.stdout:8/462: read - d5/d1d/d2f/d45/f9a zero size 2024-03-29T04:45:00.082 INFO:tasks.workunit.client.0.smithi142.stdout:8/463: mknod d5/d1d/d24/ca3 0 2024-03-29T04:45:00.085 INFO:tasks.workunit.client.0.smithi142.stdout:4/375: getdents d0/d12 0 2024-03-29T04:45:00.090 INFO:tasks.workunit.client.0.smithi142.stdout:8/464: dwrite d5/d1d/d2f/d45/f9a [0,4194304] 0 2024-03-29T04:45:00.093 INFO:tasks.workunit.client.0.smithi142.stdout:4/376: dwrite d0/d9/d11/f25 [0,4194304] 0 2024-03-29T04:45:00.096 INFO:tasks.workunit.client.0.smithi142.stdout:4/377: write d0/d12/d28/f54 [3690653,63804] 0 2024-03-29T04:45:00.101 INFO:tasks.workunit.client.0.smithi142.stdout:8/465: dwrite d5/d1d/d59/f77 [0,4194304] 0 2024-03-29T04:45:00.106 INFO:tasks.workunit.client.0.smithi142.stdout:8/466: dread d5/f19 [0,4194304] 0 2024-03-29T04:45:00.107 INFO:tasks.workunit.client.0.smithi142.stdout:7/473: dread d7/d27/f91 [0,4194304] 0 2024-03-29T04:45:00.107 INFO:tasks.workunit.client.0.smithi142.stdout:4/378: truncate d0/d9/d11/f23 3209484 0 2024-03-29T04:45:00.109 INFO:tasks.workunit.client.0.smithi142.stdout:4/379: write d0/d12/f5d [2520929,114949] 0 2024-03-29T04:45:00.110 INFO:tasks.workunit.client.0.smithi142.stdout:4/380: read d0/dc/f33 [82464,16547] 0 2024-03-29T04:45:00.111 INFO:tasks.workunit.client.0.smithi142.stdout:8/467: rename d5/d1d/d2f/d76/d3c/d70/f97 to d5/d1d/d81/fa4 0 2024-03-29T04:45:00.112 INFO:tasks.workunit.client.0.smithi142.stdout:8/468: chown d5/d1d/d24/d66 0 1 2024-03-29T04:45:00.113 INFO:tasks.workunit.client.0.smithi142.stdout:7/474: creat d7/dc/d4b/d71/d30/f99 x:0 0 0 2024-03-29T04:45:00.117 INFO:tasks.workunit.client.0.smithi142.stdout:4/381: link d0/c6 d0/dc/d15/d3d/d65/c7f 0 2024-03-29T04:45:00.118 INFO:tasks.workunit.client.0.smithi142.stdout:8/469: dwrite d5/d1d/d24/f33 [0,4194304] 0 2024-03-29T04:45:00.120 INFO:tasks.workunit.client.0.smithi142.stdout:8/470: stat d5/d1d/d2f/d76/ca0 0 2024-03-29T04:45:00.122 INFO:tasks.workunit.client.0.smithi142.stdout:4/382: rename d0/dc/l26 to d0/dc/d15/d3d/l80 0 2024-03-29T04:45:00.124 INFO:tasks.workunit.client.0.smithi142.stdout:8/471: mknod d5/d1d/d24/ca5 0 2024-03-29T04:45:00.126 INFO:tasks.workunit.client.0.smithi142.stdout:4/383: mkdir d0/d9/d11/d4c/d5f/d81 0 2024-03-29T04:45:00.126 INFO:tasks.workunit.client.0.smithi142.stdout:4/384: stat d0/d12/f7b 0 2024-03-29T04:45:00.129 INFO:tasks.workunit.client.0.smithi142.stdout:4/385: truncate d0/dc/d15/f59 855423 0 2024-03-29T04:45:00.130 INFO:tasks.workunit.client.0.smithi142.stdout:8/472: getdents d5/d1d/d2f/d76/d3c/d70 0 2024-03-29T04:45:00.132 INFO:tasks.workunit.client.0.smithi142.stdout:8/473: write d5/d1d/d2f/d76/d2d/f4f [2789358,54460] 0 2024-03-29T04:45:00.133 INFO:tasks.workunit.client.0.smithi142.stdout:8/474: read - d5/d1d/d59/f7c zero size 2024-03-29T04:45:00.134 INFO:tasks.workunit.client.0.smithi142.stdout:4/386: dread d0/d12/d28/d47/f55 [0,4194304] 0 2024-03-29T04:45:00.137 INFO:tasks.workunit.client.0.smithi142.stdout:4/387: chown d0/d9/d19/d2c/c42 277487663 1 2024-03-29T04:45:00.138 INFO:tasks.workunit.client.0.smithi142.stdout:8/475: creat d5/d1d/d2f/d5f/fa6 x:0 0 0 2024-03-29T04:45:00.138 INFO:tasks.workunit.client.0.smithi142.stdout:7/475: dread d7/d27/d44/d4c/f79 [0,4194304] 0 2024-03-29T04:45:00.138 INFO:tasks.workunit.client.0.smithi142.stdout:4/388: dread - d0/d45/f60 zero size 2024-03-29T04:45:00.139 INFO:tasks.workunit.client.0.smithi142.stdout:7/476: read - d7/dc/d4b/f4f zero size 2024-03-29T04:45:00.140 INFO:tasks.workunit.client.0.smithi142.stdout:4/389: fdatasync d0/d9/d19/f58 0 2024-03-29T04:45:00.143 INFO:tasks.workunit.client.0.smithi142.stdout:8/476: rename d5/d1d/d24/f3a to d5/d1d/d2f/fa7 0 2024-03-29T04:45:00.144 INFO:tasks.workunit.client.0.smithi142.stdout:4/390: creat d0/d9/d11/d4c/d64/f82 x:0 0 0 2024-03-29T04:45:00.146 INFO:tasks.workunit.client.0.smithi142.stdout:7/477: rename d7/d27/f29 to d7/dc/d4b/f9a 0 2024-03-29T04:45:00.147 INFO:tasks.workunit.client.0.smithi142.stdout:4/391: mknod d0/d9/d11/d4c/d5f/d81/c83 0 2024-03-29T04:45:00.149 INFO:tasks.workunit.client.0.smithi142.stdout:7/478: truncate d7/d27/d44/d4c/f8c 423660 0 2024-03-29T04:45:00.149 INFO:tasks.workunit.client.0.smithi142.stdout:8/477: dwrite d5/d1d/d2f/d76/d3c/d70/f82 [0,4194304] 0 2024-03-29T04:45:00.150 INFO:tasks.workunit.client.0.smithi142.stdout:4/392: write d0/dc/d15/d3d/d65/f7c [998974,26576] 0 2024-03-29T04:45:00.150 INFO:tasks.workunit.client.0.smithi142.stdout:4/393: readlink d0/d9/d19/l63 0 2024-03-29T04:45:00.153 INFO:tasks.workunit.client.0.smithi142.stdout:5/506: sync 2024-03-29T04:45:00.153 INFO:tasks.workunit.client.0.smithi142.stdout:0/404: sync 2024-03-29T04:45:00.155 INFO:tasks.workunit.client.0.smithi142.stdout:4/394: mkdir d0/d9/d11/d84 0 2024-03-29T04:45:00.156 INFO:tasks.workunit.client.0.smithi142.stdout:4/395: fsync d0/d45/f61 0 2024-03-29T04:45:00.158 INFO:tasks.workunit.client.0.smithi142.stdout:0/405: creat d7/d13/d4a/d70/f7f x:0 0 0 2024-03-29T04:45:00.159 INFO:tasks.workunit.client.0.smithi142.stdout:5/507: link df/d16/d24/d2e/c99 df/d1a/d21/d9d/ca3 0 2024-03-29T04:45:00.161 INFO:tasks.workunit.client.0.smithi142.stdout:5/508: write df/d29/f6b [108381,50016] 0 2024-03-29T04:45:00.163 INFO:tasks.workunit.client.0.smithi142.stdout:4/396: dwrite d0/f14 [0,4194304] 0 2024-03-29T04:45:00.168 INFO:tasks.workunit.client.0.smithi142.stdout:4/397: rename d0/f3 to d0/dc/d15/d3d/d65/f85 0 2024-03-29T04:45:00.170 INFO:tasks.workunit.client.0.smithi142.stdout:4/398: stat d0/d9/d11/d4c/d64/f77 0 2024-03-29T04:45:00.172 INFO:tasks.workunit.client.0.smithi142.stdout:5/509: dwrite df/d16/d24/fa0 [0,4194304] 0 2024-03-29T04:45:00.173 INFO:tasks.workunit.client.0.smithi142.stdout:4/399: readlink d0/d9/d19/l63 0 2024-03-29T04:45:00.175 INFO:tasks.workunit.client.0.smithi142.stdout:8/478: dread d5/f30 [0,4194304] 0 2024-03-29T04:45:00.176 INFO:tasks.workunit.client.0.smithi142.stdout:5/510: fsync df/d16/d24/d79/f5b 0 2024-03-29T04:45:00.177 INFO:tasks.workunit.client.0.smithi142.stdout:4/400: creat d0/dc/d15/d3d/f86 x:0 0 0 2024-03-29T04:45:00.177 INFO:tasks.workunit.client.0.smithi142.stdout:0/406: dwrite d7/d9/d55/d42/f5f [0,4194304] 0 2024-03-29T04:45:00.178 INFO:tasks.workunit.client.0.smithi142.stdout:8/479: mknod d5/d1d/d24/ca8 0 2024-03-29T04:45:00.179 INFO:tasks.workunit.client.0.smithi142.stdout:4/401: readlink d0/d9/d19/l1c 0 2024-03-29T04:45:00.187 INFO:tasks.workunit.client.0.smithi142.stdout:8/480: creat d5/d1d/d2f/fa9 x:0 0 0 2024-03-29T04:45:00.189 INFO:tasks.workunit.client.0.smithi142.stdout:5/511: getdents df/d1a 0 2024-03-29T04:45:00.190 INFO:tasks.workunit.client.0.smithi142.stdout:5/512: dread - df/d16/d50/f61 zero size 2024-03-29T04:45:00.196 INFO:tasks.workunit.client.0.smithi142.stdout:5/513: dwrite df/d1a/d28/f8d [0,4194304] 0 2024-03-29T04:45:00.199 INFO:tasks.workunit.client.0.smithi142.stdout:8/481: truncate d5/f30 1460470 0 2024-03-29T04:45:00.201 INFO:tasks.workunit.client.0.smithi142.stdout:0/407: dread d7/d13/d4a/f5e [0,4194304] 0 2024-03-29T04:45:00.201 INFO:tasks.workunit.client.0.smithi142.stdout:5/514: creat df/d1a/d21/fa4 x:0 0 0 2024-03-29T04:45:00.202 INFO:tasks.workunit.client.0.smithi142.stdout:8/482: symlink d5/d93/laa 0 2024-03-29T04:45:00.205 INFO:tasks.workunit.client.0.smithi142.stdout:0/408: rename d7/d9/d55/l40 to d7/d13/d67/d72/l80 0 2024-03-29T04:45:00.207 INFO:tasks.workunit.client.0.smithi142.stdout:5/515: rename df/d1a/d21/d9d to df/d1a/d21/da5 0 2024-03-29T04:45:00.207 INFO:tasks.workunit.client.0.smithi142.stdout:6/408: sync 2024-03-29T04:45:00.207 INFO:tasks.workunit.client.0.smithi142.stdout:1/440: sync 2024-03-29T04:45:00.207 INFO:tasks.workunit.client.0.smithi142.stdout:3/349: sync 2024-03-29T04:45:00.208 INFO:tasks.workunit.client.0.smithi142.stdout:4/402: dread d0/d9/d11/ff [0,4194304] 0 2024-03-29T04:45:00.208 INFO:tasks.workunit.client.0.smithi142.stdout:8/483: rename d5/d1d/d2f/d76/d3c/d70/l92 to d5/d93/lab 0 2024-03-29T04:45:00.209 INFO:tasks.workunit.client.0.smithi142.stdout:1/441: stat d8/db/dc/d77/d31/d4d 0 2024-03-29T04:45:00.209 INFO:tasks.workunit.client.0.smithi142.stdout:6/409: mkdir d1c/d33/d46/d61/d7e 0 2024-03-29T04:45:00.210 INFO:tasks.workunit.client.0.smithi142.stdout:3/350: truncate d3/d1f/f7e 534848 0 2024-03-29T04:45:00.211 INFO:tasks.workunit.client.0.smithi142.stdout:5/516: creat df/d16/d24/d3d/fa6 x:0 0 0 2024-03-29T04:45:00.212 INFO:tasks.workunit.client.0.smithi142.stdout:4/403: symlink d0/d9/d11/d78/l87 0 2024-03-29T04:45:00.212 INFO:tasks.workunit.client.0.smithi142.stdout:1/442: truncate d8/db/dc/d77/f7c 193152 0 2024-03-29T04:45:00.213 INFO:tasks.workunit.client.0.smithi142.stdout:4/404: dread - d0/dc/d15/d3d/f86 zero size 2024-03-29T04:45:00.214 INFO:tasks.workunit.client.0.smithi142.stdout:3/351: readlink d3/d1f/d24/d47/l55 0 2024-03-29T04:45:00.215 INFO:tasks.workunit.client.0.smithi142.stdout:8/484: dread d5/d1d/d2f/f68 [0,4194304] 0 2024-03-29T04:45:00.218 INFO:tasks.workunit.client.0.smithi142.stdout:1/443: creat d8/db/dc/d19/d1d/d15/fa0 x:0 0 0 2024-03-29T04:45:00.221 INFO:tasks.workunit.client.0.smithi142.stdout:0/409: truncate d7/d9/d54/f64 393911 0 2024-03-29T04:45:00.221 INFO:tasks.workunit.client.0.smithi142.stdout:4/405: dread d0/d9/d11/f2b [0,4194304] 0 2024-03-29T04:45:00.221 INFO:tasks.workunit.client.0.smithi142.stdout:6/410: truncate d1c/d33/f36 368084 0 2024-03-29T04:45:00.223 INFO:tasks.workunit.client.0.smithi142.stdout:3/352: rename d3/d29/c33 to d3/d29/d2b/d40/c80 0 2024-03-29T04:45:00.223 INFO:tasks.workunit.client.0.smithi142.stdout:5/517: dwrite df/d16/d24/d5c/f5e [4194304,4194304] 0 2024-03-29T04:45:00.223 INFO:tasks.workunit.client.0.smithi142.stdout:8/485: mknod d5/d1d/d2f/d76/d4c/cac 0 2024-03-29T04:45:00.225 INFO:tasks.workunit.client.0.smithi142.stdout:1/444: mknod d8/db/dc/d77/d31/ca1 0 2024-03-29T04:45:00.228 INFO:tasks.workunit.client.0.smithi142.stdout:5/518: unlink df/d16/d24/d2e/l87 0 2024-03-29T04:45:00.230 INFO:tasks.workunit.client.0.smithi142.stdout:3/353: truncate d3/d1f/d24/f37 3761064 0 2024-03-29T04:45:00.230 INFO:tasks.workunit.client.0.smithi142.stdout:1/445: mknod d8/db/dc/d77/ca2 0 2024-03-29T04:45:00.231 INFO:tasks.workunit.client.0.smithi142.stdout:0/410: dwrite d7/d13/d67/d72/f3e [0,4194304] 0 2024-03-29T04:45:00.232 INFO:tasks.workunit.client.0.smithi142.stdout:4/406: creat d0/dc/f88 x:0 0 0 2024-03-29T04:45:00.232 INFO:tasks.workunit.client.0.smithi142.stdout:1/446: fsync f5 0 2024-03-29T04:45:00.232 INFO:tasks.workunit.client.0.smithi142.stdout:5/519: symlink df/d16/la7 0 2024-03-29T04:45:00.233 INFO:tasks.workunit.client.0.smithi142.stdout:3/354: write d3/d1f/d24/d47/f75 [480551,113450] 0 2024-03-29T04:45:00.234 INFO:tasks.workunit.client.0.smithi142.stdout:4/407: chown d0/dc/f36 426033 1 2024-03-29T04:45:00.235 INFO:tasks.workunit.client.0.smithi142.stdout:0/411: mkdir d7/d9/d1e/d81 0 2024-03-29T04:45:00.235 INFO:tasks.workunit.client.0.smithi142.stdout:1/447: mknod d8/db/dc/d77/ca3 0 2024-03-29T04:45:00.236 INFO:tasks.workunit.client.0.smithi142.stdout:0/412: fdatasync d7/d13/d4a/d70/f7f 0 2024-03-29T04:45:00.236 INFO:tasks.workunit.client.0.smithi142.stdout:3/355: symlink d3/d54/l81 0 2024-03-29T04:45:00.238 INFO:tasks.workunit.client.0.smithi142.stdout:4/408: creat d0/d9/d11/d4c/f89 x:0 0 0 2024-03-29T04:45:00.241 INFO:tasks.workunit.client.0.smithi142.stdout:5/520: creat df/fa8 x:0 0 0 2024-03-29T04:45:00.242 INFO:tasks.workunit.client.0.smithi142.stdout:1/448: dwrite f7 [0,4194304] 0 2024-03-29T04:45:00.247 INFO:tasks.workunit.client.0.smithi142.stdout:3/356: creat d3/d29/d2b/d40/f82 x:0 0 0 2024-03-29T04:45:00.247 INFO:tasks.workunit.client.0.smithi142.stdout:4/409: creat d0/d9/d11/d4c/d5f/f8a x:0 0 0 2024-03-29T04:45:00.250 INFO:tasks.workunit.client.0.smithi142.stdout:5/521: creat df/d1a/d21/d27/d45/fa9 x:0 0 0 2024-03-29T04:45:00.252 INFO:tasks.workunit.client.0.smithi142.stdout:1/449: mkdir d8/db/dc/d19/d1d/d2d/d8b/da4 0 2024-03-29T04:45:00.254 INFO:tasks.workunit.client.0.smithi142.stdout:6/411: truncate d1c/f30 2367103 0 2024-03-29T04:45:00.256 INFO:tasks.workunit.client.0.smithi142.stdout:8/486: dread d5/d1d/d2f/d76/d2d/f4a [0,4194304] 0 2024-03-29T04:45:00.256 INFO:tasks.workunit.client.0.smithi142.stdout:5/522: mknod df/d1a/d21/d55/caa 0 2024-03-29T04:45:00.257 INFO:tasks.workunit.client.0.smithi142.stdout:4/410: creat d0/d12/d28/d47/d7a/f8b x:0 0 0 2024-03-29T04:45:00.257 INFO:tasks.workunit.client.0.smithi142.stdout:5/523: fdatasync df/d1a/d21/d27/d45/fa9 0 2024-03-29T04:45:00.258 INFO:tasks.workunit.client.0.smithi142.stdout:8/487: truncate d5/d1d/d2f/d76/d2d/f4a 4922108 0 2024-03-29T04:45:00.259 INFO:tasks.workunit.client.0.smithi142.stdout:1/450: rename d8/db/dc/d77/ca3 to d8/db/dc/d77/d31/d4d/d56/ca5 0 2024-03-29T04:45:00.259 INFO:tasks.workunit.client.0.smithi142.stdout:5/524: dread - df/d1a/d21/d27/d45/fa9 zero size 2024-03-29T04:45:00.259 INFO:tasks.workunit.client.0.smithi142.stdout:0/413: getdents d7/d13/d67/d72 0 2024-03-29T04:45:00.261 INFO:tasks.workunit.client.0.smithi142.stdout:4/411: unlink d0/d9/d19/f1a 0 2024-03-29T04:45:00.261 INFO:tasks.workunit.client.0.smithi142.stdout:1/451: chown d8/db/dc/d77/d31/d4d/c59 119487544 1 2024-03-29T04:45:00.262 INFO:tasks.workunit.client.0.smithi142.stdout:9/336: sync 2024-03-29T04:45:00.262 INFO:tasks.workunit.client.0.smithi142.stdout:2/361: sync 2024-03-29T04:45:00.263 INFO:tasks.workunit.client.0.smithi142.stdout:4/412: write d0/dc/d15/d3d/f71 [209117,4317] 0 2024-03-29T04:45:00.263 INFO:tasks.workunit.client.0.smithi142.stdout:2/362: fsync de/d11/f58 0 2024-03-29T04:45:00.270 INFO:tasks.workunit.client.0.smithi142.stdout:8/488: truncate d5/d1d/d2f/d5f/d48/f61 1317455 0 2024-03-29T04:45:00.272 INFO:tasks.workunit.client.0.smithi142.stdout:5/525: creat df/d16/d24/d2e/fab x:0 0 0 2024-03-29T04:45:00.274 INFO:tasks.workunit.client.0.smithi142.stdout:4/413: rename d0/dc/d15/d3d/f86 to d0/d9/d11/f8c 0 2024-03-29T04:45:00.275 INFO:tasks.workunit.client.0.smithi142.stdout:4/414: chown d0/d9/d11/d84 80 1 2024-03-29T04:45:00.276 INFO:tasks.workunit.client.0.smithi142.stdout:0/414: dread d7/d9/f49 [0,4194304] 0 2024-03-29T04:45:00.277 INFO:tasks.workunit.client.0.smithi142.stdout:6/412: dread fb [0,4194304] 0 2024-03-29T04:45:00.279 INFO:tasks.workunit.client.0.smithi142.stdout:8/489: mknod d5/d2e/cad 0 2024-03-29T04:45:00.282 INFO:tasks.workunit.client.0.smithi142.stdout:4/415: mkdir d0/d9/d11/d5c/d8d 0 2024-03-29T04:45:00.283 INFO:tasks.workunit.client.0.smithi142.stdout:3/357: dwrite d3/d29/d2d/f3f [0,4194304] 0 2024-03-29T04:45:00.284 INFO:tasks.workunit.client.0.smithi142.stdout:0/415: fdatasync d7/d9/f68 0 2024-03-29T04:45:00.287 INFO:tasks.workunit.client.0.smithi142.stdout:1/452: dread d8/db/dc/d77/f32 [0,4194304] 0 2024-03-29T04:45:00.288 INFO:tasks.workunit.client.0.smithi142.stdout:7/479: sync 2024-03-29T04:45:00.288 INFO:tasks.workunit.client.0.smithi142.stdout:6/413: dwrite d1c/d33/d46/d71/f68 [0,4194304] 0 2024-03-29T04:45:00.292 INFO:tasks.workunit.client.0.smithi142.stdout:4/416: symlink d0/dc/d15/d3d/l8e 0 2024-03-29T04:45:00.299 INFO:tasks.workunit.client.0.smithi142.stdout:7/480: rmdir d7/d27/d44/d4c 39 2024-03-29T04:45:00.299 INFO:tasks.workunit.client.0.smithi142.stdout:3/358: dread d3/d54/f58 [0,4194304] 0 2024-03-29T04:45:00.300 INFO:tasks.workunit.client.0.smithi142.stdout:5/526: getdents df/d16/d50 0 2024-03-29T04:45:00.303 INFO:tasks.workunit.client.0.smithi142.stdout:4/417: mknod d0/d12/d28/c8f 0 2024-03-29T04:45:00.303 INFO:tasks.workunit.client.0.smithi142.stdout:2/363: dwrite de/f29 [0,4194304] 0 2024-03-29T04:45:00.303 INFO:tasks.workunit.client.0.smithi142.stdout:6/414: unlink d1c/d33/d46/d71/l7a 0 2024-03-29T04:45:00.305 INFO:tasks.workunit.client.0.smithi142.stdout:7/481: mkdir d7/dc/d4b/d9b 0 2024-03-29T04:45:00.306 INFO:tasks.workunit.client.0.smithi142.stdout:3/359: mknod d3/d5d/c83 0 2024-03-29T04:45:00.306 INFO:tasks.workunit.client.0.smithi142.stdout:8/490: dwrite d5/d1d/d2f/d76/d2d/f56 [0,4194304] 0 2024-03-29T04:45:00.308 INFO:tasks.workunit.client.0.smithi142.stdout:1/453: truncate d8/db/dc/d19/f5c 849142 0 2024-03-29T04:45:00.309 INFO:tasks.workunit.client.0.smithi142.stdout:9/337: dwrite d0/d2/d6/d16/f3b [4194304,4194304] 0 2024-03-29T04:45:00.309 INFO:tasks.workunit.client.0.smithi142.stdout:5/527: creat df/d1a/d21/d27/d45/fac x:0 0 0 2024-03-29T04:45:00.309 INFO:tasks.workunit.client.0.smithi142.stdout:6/415: creat d1c/d45/d51/f7f x:0 0 0 2024-03-29T04:45:00.313 INFO:tasks.workunit.client.0.smithi142.stdout:2/364: mkdir de/d11/d17/d4a/d79 0 2024-03-29T04:45:00.314 INFO:tasks.workunit.client.0.smithi142.stdout:3/360: chown d3/d29/d2d/f48 0 1 2024-03-29T04:45:00.317 INFO:tasks.workunit.client.0.smithi142.stdout:5/528: creat df/d29/d5d/d75/fad x:0 0 0 2024-03-29T04:45:00.318 INFO:tasks.workunit.client.0.smithi142.stdout:8/491: mknod d5/d1d/d24/d66/cae 0 2024-03-29T04:45:00.319 INFO:tasks.workunit.client.0.smithi142.stdout:7/482: mknod d7/dc/d4b/d71/d30/d33/c9c 0 2024-03-29T04:45:00.321 INFO:tasks.workunit.client.0.smithi142.stdout:6/416: read d1c/f25 [10180,127800] 0 2024-03-29T04:45:00.323 INFO:tasks.workunit.client.0.smithi142.stdout:5/529: chown df/d16/d25/l62 377634 1 2024-03-29T04:45:00.323 INFO:tasks.workunit.client.0.smithi142.stdout:0/416: dread d7/f21 [0,4194304] 0 2024-03-29T04:45:00.326 INFO:tasks.workunit.client.0.smithi142.stdout:5/530: stat df/c11 0 2024-03-29T04:45:00.326 INFO:tasks.workunit.client.0.smithi142.stdout:1/454: getdents d8/db/dc/d19 0 2024-03-29T04:45:00.326 INFO:tasks.workunit.client.0.smithi142.stdout:2/365: write de/d11/d1e/d22/d27/f45 [1468042,15026] 0 2024-03-29T04:45:00.328 INFO:tasks.workunit.client.0.smithi142.stdout:4/418: dwrite d0/dc/d15/d3d/d65/f7c [0,4194304] 0 2024-03-29T04:45:00.329 INFO:tasks.workunit.client.0.smithi142.stdout:9/338: rename d0/d2/d6/d27/d3c/d4f/c29 to d0/d12/c68 0 2024-03-29T04:45:00.330 INFO:tasks.workunit.client.0.smithi142.stdout:6/417: read d1c/f55 [1664538,104548] 0 2024-03-29T04:45:00.331 INFO:tasks.workunit.client.0.smithi142.stdout:5/531: creat df/d1a/d28/fae x:0 0 0 2024-03-29T04:45:00.333 INFO:tasks.workunit.client.0.smithi142.stdout:9/339: write d0/d12/f54 [1145648,54105] 0 2024-03-29T04:45:00.333 INFO:tasks.workunit.client.0.smithi142.stdout:7/483: creat d7/dc/d4b/d71/f9d x:0 0 0 2024-03-29T04:45:00.334 INFO:tasks.workunit.client.0.smithi142.stdout:7/484: fsync d7/dc/d4b/d71/d30/f3e 0 2024-03-29T04:45:00.335 INFO:tasks.workunit.client.0.smithi142.stdout:2/366: mkdir de/d11/d17/d60/d7a 0 2024-03-29T04:45:00.336 INFO:tasks.workunit.client.0.smithi142.stdout:7/485: fsync d7/dc/de/d20/d3f/f75 0 2024-03-29T04:45:00.340 INFO:tasks.workunit.client.0.smithi142.stdout:4/419: creat d0/dc/d15/d3d/d65/f90 x:0 0 0 2024-03-29T04:45:00.341 INFO:tasks.workunit.client.0.smithi142.stdout:2/367: dread de/f19 [0,4194304] 0 2024-03-29T04:45:00.345 INFO:tasks.workunit.client.0.smithi142.stdout:9/340: truncate d0/d2/d6/f30 574278 0 2024-03-29T04:45:00.347 INFO:tasks.workunit.client.0.smithi142.stdout:9/341: write d0/d12/f24 [1155637,20239] 0 2024-03-29T04:45:00.347 INFO:tasks.workunit.client.0.smithi142.stdout:1/455: creat d8/db/dc/d19/d1d/d1f/d97/d9e/fa6 x:0 0 0 2024-03-29T04:45:00.348 INFO:tasks.workunit.client.0.smithi142.stdout:1/456: chown d8/f9 1931747 1 2024-03-29T04:45:00.349 INFO:tasks.workunit.client.0.smithi142.stdout:2/368: mkdir de/d11/d2b/d7b 0 2024-03-29T04:45:00.349 INFO:tasks.workunit.client.0.smithi142.stdout:1/457: read d8/db/d37/f38 [34311,95162] 0 2024-03-29T04:45:00.350 INFO:tasks.workunit.client.0.smithi142.stdout:2/369: write de/d11/f15 [7268931,39220] 0 2024-03-29T04:45:00.351 INFO:tasks.workunit.client.0.smithi142.stdout:5/532: unlink df/d2c/l96 0 2024-03-29T04:45:00.353 INFO:tasks.workunit.client.0.smithi142.stdout:2/370: write de/f29 [5774477,120441] 0 2024-03-29T04:45:00.353 INFO:tasks.workunit.client.0.smithi142.stdout:5/533: chown df/d16/d24/d5c 5064433 1 2024-03-29T04:45:00.354 INFO:tasks.workunit.client.0.smithi142.stdout:4/420: mkdir d0/dc/d15/d91 0 2024-03-29T04:45:00.355 INFO:tasks.workunit.client.0.smithi142.stdout:8/492: dread d5/f22 [0,4194304] 0 2024-03-29T04:45:00.355 INFO:tasks.workunit.client.0.smithi142.stdout:2/371: chown de/d11/d17/c43 656783 1 2024-03-29T04:45:00.356 INFO:tasks.workunit.client.0.smithi142.stdout:5/534: chown df/d16/d24/d5c/f5e 9 1 2024-03-29T04:45:00.356 INFO:tasks.workunit.client.0.smithi142.stdout:8/493: dread - d5/f94 zero size 2024-03-29T04:45:00.357 INFO:tasks.workunit.client.0.smithi142.stdout:3/361: dread d3/d1f/d34/f4e [0,4194304] 0 2024-03-29T04:45:00.358 INFO:tasks.workunit.client.0.smithi142.stdout:1/458: creat d8/db/dc/d19/d1d/d1f/d97/d9e/fa7 x:0 0 0 2024-03-29T04:45:00.359 INFO:tasks.workunit.client.0.smithi142.stdout:3/362: chown d3/d29/d2b/d6a 56847 1 2024-03-29T04:45:00.359 INFO:tasks.workunit.client.0.smithi142.stdout:4/421: creat d0/d9/d11/d5c/f92 x:0 0 0 2024-03-29T04:45:00.361 INFO:tasks.workunit.client.0.smithi142.stdout:2/372: symlink de/d11/d2b/d3e/l7c 0 2024-03-29T04:45:00.365 INFO:tasks.workunit.client.0.smithi142.stdout:1/459: dread d8/db/dc/d19/d1d/d2d/d44/d53/f6a [0,4194304] 0 2024-03-29T04:45:00.365 INFO:tasks.workunit.client.0.smithi142.stdout:7/486: write d7/dc/d4b/d71/d30/d33/f4d [281634,33645] 0 2024-03-29T04:45:00.365 INFO:tasks.workunit.client.0.smithi142.stdout:5/535: getdents df/d16/d24/d2e/d47/d8c 0 2024-03-29T04:45:00.367 INFO:tasks.workunit.client.0.smithi142.stdout:8/494: rmdir d5/d2e/da2 0 2024-03-29T04:45:00.370 INFO:tasks.workunit.client.0.smithi142.stdout:4/422: dread d0/d12/f3c [0,4194304] 0 2024-03-29T04:45:00.370 INFO:tasks.workunit.client.0.smithi142.stdout:2/373: rename de/d11/d17/d60/d7a to de/d11/d1e/d4d/d61/d7d 0 2024-03-29T04:45:00.370 INFO:tasks.workunit.client.0.smithi142.stdout:9/342: dread d0/d2/d6/d16/f2b [0,4194304] 0 2024-03-29T04:45:00.371 INFO:tasks.workunit.client.0.smithi142.stdout:0/417: dread d7/d9/d55/f3d [0,4194304] 0 2024-03-29T04:45:00.372 INFO:tasks.workunit.client.0.smithi142.stdout:7/487: rename d7/d27 to d7/d27/d44/d4c/d9e 22 2024-03-29T04:45:00.373 INFO:tasks.workunit.client.0.smithi142.stdout:5/536: chown df/d1a/d21/d27/d45/fa9 22 1 2024-03-29T04:45:00.374 INFO:tasks.workunit.client.0.smithi142.stdout:4/423: read - d0/d12/f44 zero size 2024-03-29T04:45:00.375 INFO:tasks.workunit.client.0.smithi142.stdout:3/363: truncate d3/d29/d2b/f21 1508731 0 2024-03-29T04:45:00.375 INFO:tasks.workunit.client.0.smithi142.stdout:5/537: stat df/c6d 0 2024-03-29T04:45:00.376 INFO:tasks.workunit.client.0.smithi142.stdout:6/418: dwrite d1c/d33/f3e [0,4194304] 0 2024-03-29T04:45:00.377 INFO:tasks.workunit.client.0.smithi142.stdout:7/488: write d7/dc/de/f16 [411210,492] 0 2024-03-29T04:45:00.380 INFO:tasks.workunit.client.0.smithi142.stdout:6/419: chown d1c/l38 24055807 1 2024-03-29T04:45:00.386 INFO:tasks.workunit.client.0.smithi142.stdout:8/495: dread d5/f19 [0,4194304] 0 2024-03-29T04:45:00.387 INFO:tasks.workunit.client.0.smithi142.stdout:3/364: write d3/f3c [582505,112261] 0 2024-03-29T04:45:00.388 INFO:tasks.workunit.client.0.smithi142.stdout:4/424: creat d0/dc/f93 x:0 0 0 2024-03-29T04:45:00.389 INFO:tasks.workunit.client.0.smithi142.stdout:7/489: rename d7/d27/f6e to d7/d27/f9f 0 2024-03-29T04:45:00.392 INFO:tasks.workunit.client.0.smithi142.stdout:7/490: chown d7/l66 0 1 2024-03-29T04:45:00.392 INFO:tasks.workunit.client.0.smithi142.stdout:0/418: dread d7/d9/d54/f64 [0,4194304] 0 2024-03-29T04:45:00.392 INFO:tasks.workunit.client.0.smithi142.stdout:5/538: dwrite df/d29/d5d/d75/fad [0,4194304] 0 2024-03-29T04:45:00.395 INFO:tasks.workunit.client.0.smithi142.stdout:7/491: write d7/fa [1654382,69316] 0 2024-03-29T04:45:00.395 INFO:tasks.workunit.client.0.smithi142.stdout:8/496: unlink d5/d1d/d2f/d76/d4c/d60/c7d 0 2024-03-29T04:45:00.396 INFO:tasks.workunit.client.0.smithi142.stdout:3/365: symlink d3/d5d/d5f/d64/d6b/l84 0 2024-03-29T04:45:00.398 INFO:tasks.workunit.client.0.smithi142.stdout:1/460: truncate d8/db/dc/d77/d31/d4d/f7a 43283 0 2024-03-29T04:45:00.405 INFO:tasks.workunit.client.0.smithi142.stdout:9/343: dread d0/d12/f59 [0,4194304] 0 2024-03-29T04:45:00.405 INFO:tasks.workunit.client.0.smithi142.stdout:1/461: dread - d8/db/d37/f76 zero size 2024-03-29T04:45:00.405 INFO:tasks.workunit.client.0.smithi142.stdout:2/374: dread de/d11/d2b/d3e/f76 [0,4194304] 0 2024-03-29T04:45:00.406 INFO:tasks.workunit.client.0.smithi142.stdout:0/419: rename d7/d13/d67/d72/f61 to d7/d9/d54/d58/f82 0 2024-03-29T04:45:00.406 INFO:tasks.workunit.client.0.smithi142.stdout:6/420: dread d1c/d33/d46/f48 [0,4194304] 0 2024-03-29T04:45:00.406 INFO:tasks.workunit.client.0.smithi142.stdout:2/375: fdatasync de/d11/d1e/f72 0 2024-03-29T04:45:00.406 INFO:tasks.workunit.client.0.smithi142.stdout:3/366: rmdir d3/d29/d2b 39 2024-03-29T04:45:00.406 INFO:tasks.workunit.client.0.smithi142.stdout:1/462: dread - d8/db/d37/f94 zero size 2024-03-29T04:45:00.406 INFO:tasks.workunit.client.0.smithi142.stdout:9/344: chown d0/d2/d6/d27/l5d 331 1 2024-03-29T04:45:00.407 INFO:tasks.workunit.client.0.smithi142.stdout:6/421: fdatasync d1c/f5d 0 2024-03-29T04:45:00.407 INFO:tasks.workunit.client.0.smithi142.stdout:8/497: mknod d5/d1d/d2f/d5f/caf 0 2024-03-29T04:45:00.408 INFO:tasks.workunit.client.0.smithi142.stdout:4/425: getdents d0/d9/d11/d5c/d6d 0 2024-03-29T04:45:00.409 INFO:tasks.workunit.client.0.smithi142.stdout:0/420: unlink d7/d9/d55/d42/f7b 0 2024-03-29T04:45:00.409 INFO:tasks.workunit.client.0.smithi142.stdout:4/426: read d0/d9/d11/d78/f79 [56597,20367] 0 2024-03-29T04:45:00.410 INFO:tasks.workunit.client.0.smithi142.stdout:4/427: dread - d0/dc/f36 zero size 2024-03-29T04:45:00.411 INFO:tasks.workunit.client.0.smithi142.stdout:9/345: read d0/f1d [5727829,115947] 0 2024-03-29T04:45:00.412 INFO:tasks.workunit.client.0.smithi142.stdout:3/367: dread - d3/d29/d2b/d40/d67/f7f zero size 2024-03-29T04:45:00.412 INFO:tasks.workunit.client.0.smithi142.stdout:4/428: write d0/d9/d11/d4c/d64/f77 [400903,30606] 0 2024-03-29T04:45:00.413 INFO:tasks.workunit.client.0.smithi142.stdout:2/376: creat de/d11/d1e/d4d/d61/d7d/f7e x:0 0 0 2024-03-29T04:45:00.414 INFO:tasks.workunit.client.0.smithi142.stdout:8/498: creat d5/d93/fb0 x:0 0 0 2024-03-29T04:45:00.414 INFO:tasks.workunit.client.0.smithi142.stdout:1/463: symlink d8/db/dc/d19/d1d/d2d/d44/la8 0 2024-03-29T04:45:00.416 INFO:tasks.workunit.client.0.smithi142.stdout:7/492: getdents d7/d27/d44/d4c 0 2024-03-29T04:45:00.417 INFO:tasks.workunit.client.0.smithi142.stdout:3/368: truncate d3/d29/d2b/d40/f4a 1340331 0 2024-03-29T04:45:00.417 INFO:tasks.workunit.client.0.smithi142.stdout:8/499: chown d5/d1d/d24/d66/c69 8 1 2024-03-29T04:45:00.418 INFO:tasks.workunit.client.0.smithi142.stdout:1/464: dread - d8/db/d37/f94 zero size 2024-03-29T04:45:00.419 INFO:tasks.workunit.client.0.smithi142.stdout:6/422: mknod d1c/c80 0 2024-03-29T04:45:00.420 INFO:tasks.workunit.client.0.smithi142.stdout:2/377: fsync de/d11/f37 0 2024-03-29T04:45:00.420 INFO:tasks.workunit.client.0.smithi142.stdout:0/421: dread d7/d13/f50 [0,4194304] 0 2024-03-29T04:45:00.423 INFO:tasks.workunit.client.0.smithi142.stdout:7/493: creat d7/dc/de/d20/d3f/fa0 x:0 0 0 2024-03-29T04:45:00.424 INFO:tasks.workunit.client.0.smithi142.stdout:7/494: fdatasync d7/f19 0 2024-03-29T04:45:00.426 INFO:tasks.workunit.client.0.smithi142.stdout:5/539: dread df/d1a/f3f [0,4194304] 0 2024-03-29T04:45:00.427 INFO:tasks.workunit.client.0.smithi142.stdout:1/465: dwrite d8/db/dc/d19/d1d/d15/f27 [4194304,4194304] 0 2024-03-29T04:45:00.432 INFO:tasks.workunit.client.0.smithi142.stdout:2/378: read de/d11/d1e/d22/f26 [418421,97152] 0 2024-03-29T04:45:00.432 INFO:tasks.workunit.client.0.smithi142.stdout:3/369: symlink d3/d29/l85 0 2024-03-29T04:45:00.432 INFO:tasks.workunit.client.0.smithi142.stdout:2/379: chown de/d11/d1e 6294512 1 2024-03-29T04:45:00.433 INFO:tasks.workunit.client.0.smithi142.stdout:2/380: write de/d11/d1e/d22/d50/f62 [1024604,110569] 0 2024-03-29T04:45:00.436 INFO:tasks.workunit.client.0.smithi142.stdout:4/429: write d0/d9/d19/f58 [585938,128134] 0 2024-03-29T04:45:00.438 INFO:tasks.workunit.client.0.smithi142.stdout:1/466: write d8/db/dc/d19/d1d/d2d/d44/d53/f6a [643872,25233] 0 2024-03-29T04:45:00.440 INFO:tasks.workunit.client.0.smithi142.stdout:0/422: link d7/d9/f10 d7/d13/d67/d72/d37/f83 0 2024-03-29T04:45:00.440 INFO:tasks.workunit.client.0.smithi142.stdout:9/346: truncate d0/d2/d6/d27/f48 185441 0 2024-03-29T04:45:00.442 INFO:tasks.workunit.client.0.smithi142.stdout:3/370: creat d3/d5d/d5f/f86 x:0 0 0 2024-03-29T04:45:00.443 INFO:tasks.workunit.client.0.smithi142.stdout:8/500: creat d5/d1d/d2f/d5f/fb1 x:0 0 0 2024-03-29T04:45:00.444 INFO:tasks.workunit.client.0.smithi142.stdout:7/495: write f1 [2175463,8737] 0 2024-03-29T04:45:00.445 INFO:tasks.workunit.client.0.smithi142.stdout:6/423: creat d1c/d42/f81 x:0 0 0 2024-03-29T04:45:00.445 INFO:tasks.workunit.client.0.smithi142.stdout:5/540: mknod df/d1a/d21/caf 0 2024-03-29T04:45:00.446 INFO:tasks.workunit.client.0.smithi142.stdout:4/430: mkdir d0/d9/d11/d4c/d64/d94 0 2024-03-29T04:45:00.448 INFO:tasks.workunit.client.0.smithi142.stdout:3/371: mkdir d3/d1f/d24/d47/d87 0 2024-03-29T04:45:00.450 INFO:tasks.workunit.client.0.smithi142.stdout:2/381: mkdir de/d7f 0 2024-03-29T04:45:00.452 INFO:tasks.workunit.client.0.smithi142.stdout:0/423: dwrite d7/d13/d67/d72/d37/f39 [0,4194304] 0 2024-03-29T04:45:00.452 INFO:tasks.workunit.client.0.smithi142.stdout:5/541: write df/d1a/d21/fa4 [994688,41037] 0 2024-03-29T04:45:00.457 INFO:tasks.workunit.client.0.smithi142.stdout:4/431: creat d0/dc/d15/d3d/d65/f95 x:0 0 0 2024-03-29T04:45:00.460 INFO:tasks.workunit.client.0.smithi142.stdout:0/424: write d7/d9/d55/d42/f7c [209841,28115] 0 2024-03-29T04:45:00.463 INFO:tasks.workunit.client.0.smithi142.stdout:3/372: fsync d3/d1f/d24/f37 0 2024-03-29T04:45:00.463 INFO:tasks.workunit.client.0.smithi142.stdout:8/501: dwrite d5/d1d/d2f/d76/d2d/f4f [0,4194304] 0 2024-03-29T04:45:00.466 INFO:tasks.workunit.client.0.smithi142.stdout:5/542: fdatasync fa 0 2024-03-29T04:45:00.467 INFO:tasks.workunit.client.0.smithi142.stdout:7/496: rename d7/d27/d3a/l59 to d7/dc/la1 0 2024-03-29T04:45:00.467 INFO:tasks.workunit.client.0.smithi142.stdout:8/502: fsync d5/f18 0 2024-03-29T04:45:00.468 INFO:tasks.workunit.client.0.smithi142.stdout:1/467: dread d8/db/dc/d19/d1d/d15/f27 [0,4194304] 0 2024-03-29T04:45:00.469 INFO:tasks.workunit.client.0.smithi142.stdout:2/382: creat de/d11/d1e/f80 x:0 0 0 2024-03-29T04:45:00.470 INFO:tasks.workunit.client.0.smithi142.stdout:9/347: dread d0/f4b [0,4194304] 0 2024-03-29T04:45:00.474 INFO:tasks.workunit.client.0.smithi142.stdout:4/432: dread d0/dc/d15/d3d/d65/f7c [0,4194304] 0 2024-03-29T04:45:00.474 INFO:tasks.workunit.client.0.smithi142.stdout:1/468: write d8/db/dc/d77/d31/d4d/d56/f58 [474174,57519] 0 2024-03-29T04:45:00.474 INFO:tasks.workunit.client.0.smithi142.stdout:9/348: write d0/d12/d50/f63 [141882,21526] 0 2024-03-29T04:45:00.477 INFO:tasks.workunit.client.0.smithi142.stdout:5/543: readlink df/d1a/l91 0 2024-03-29T04:45:00.478 INFO:tasks.workunit.client.0.smithi142.stdout:4/433: write d0/d12/f1e [553072,67898] 0 2024-03-29T04:45:00.479 INFO:tasks.workunit.client.0.smithi142.stdout:8/503: rmdir d5/d1d 39 2024-03-29T04:45:00.480 INFO:tasks.workunit.client.0.smithi142.stdout:6/424: dread f18 [0,4194304] 0 2024-03-29T04:45:00.480 INFO:tasks.workunit.client.0.smithi142.stdout:7/497: dwrite f6 [0,4194304] 0 2024-03-29T04:45:00.482 INFO:tasks.workunit.client.0.smithi142.stdout:6/425: readlink d1c/d45/d51/l78 0 2024-03-29T04:45:00.482 INFO:tasks.workunit.client.0.smithi142.stdout:7/498: dread - d7/dc/de/d20/f98 zero size 2024-03-29T04:45:00.484 INFO:tasks.workunit.client.0.smithi142.stdout:9/349: creat d0/d2/d6/d27/d3c/f69 x:0 0 0 2024-03-29T04:45:00.485 INFO:tasks.workunit.client.0.smithi142.stdout:1/469: symlink d8/db/dc/d77/d31/d4d/d56/la9 0 2024-03-29T04:45:00.486 INFO:tasks.workunit.client.0.smithi142.stdout:0/425: dread d7/d13/d67/d72/d37/f83 [0,4194304] 0 2024-03-29T04:45:00.486 INFO:tasks.workunit.client.0.smithi142.stdout:9/350: read d0/d2/d6/d31/f49 [3714605,82363] 0 2024-03-29T04:45:00.487 INFO:tasks.workunit.client.0.smithi142.stdout:7/499: dread d7/dc/d4b/d71/f28 [0,4194304] 0 2024-03-29T04:45:00.489 INFO:tasks.workunit.client.0.smithi142.stdout:0/426: readlink d7/d13/d67/d72/l1d 0 2024-03-29T04:45:00.489 INFO:tasks.workunit.client.0.smithi142.stdout:2/383: symlink de/d11/d1e/d4d/d61/l81 0 2024-03-29T04:45:00.492 INFO:tasks.workunit.client.0.smithi142.stdout:0/427: write d7/d13/d67/d72/d37/f56 [1253039,17345] 0 2024-03-29T04:45:00.493 INFO:tasks.workunit.client.0.smithi142.stdout:3/373: dwrite d3/d1f/d34/f3e [0,4194304] 0 2024-03-29T04:45:00.494 INFO:tasks.workunit.client.0.smithi142.stdout:4/434: fsync d0/dc/d15/d3d/d65/f85 0 2024-03-29T04:45:00.495 INFO:tasks.workunit.client.0.smithi142.stdout:8/504: dread d5/d1d/d2f/d76/d2d/f4a [0,4194304] 0 2024-03-29T04:45:00.497 INFO:tasks.workunit.client.0.smithi142.stdout:6/426: symlink d1c/l82 0 2024-03-29T04:45:00.497 INFO:tasks.workunit.client.0.smithi142.stdout:4/435: chown d0/d12/d28/l40 64149892 1 2024-03-29T04:45:00.498 INFO:tasks.workunit.client.0.smithi142.stdout:6/427: write d1c/d33/f3e [743072,32188] 0 2024-03-29T04:45:00.499 INFO:tasks.workunit.client.0.smithi142.stdout:6/428: fdatasync d1c/f62 0 2024-03-29T04:45:00.501 INFO:tasks.workunit.client.0.smithi142.stdout:3/374: truncate d3/d1f/d59/f6e 395682 0 2024-03-29T04:45:00.502 INFO:tasks.workunit.client.0.smithi142.stdout:5/544: link df/d16/d25/l58 df/d16/d25/lb0 0 2024-03-29T04:45:00.503 INFO:tasks.workunit.client.0.smithi142.stdout:5/545: write df/d1a/d21/d27/f36 [3770976,43626] 0 2024-03-29T04:45:00.504 INFO:tasks.workunit.client.0.smithi142.stdout:9/351: mkdir d0/d2/d6/d16/d6a 0 2024-03-29T04:45:00.505 INFO:tasks.workunit.client.0.smithi142.stdout:8/505: creat d5/d93/fb2 x:0 0 0 2024-03-29T04:45:00.507 INFO:tasks.workunit.client.0.smithi142.stdout:6/429: creat d1c/d45/d51/f83 x:0 0 0 2024-03-29T04:45:00.509 INFO:tasks.workunit.client.0.smithi142.stdout:6/430: chown d1c/d42/d52/f54 2853687 1 2024-03-29T04:45:00.510 INFO:tasks.workunit.client.0.smithi142.stdout:9/352: unlink d0/d12/l25 0 2024-03-29T04:45:00.511 INFO:tasks.workunit.client.0.smithi142.stdout:7/500: rename d7/d27/d44/l7a to d7/la2 0 2024-03-29T04:45:00.512 INFO:tasks.workunit.client.0.smithi142.stdout:5/546: read df/d1a/d21/d27/f36 [1474982,125590] 0 2024-03-29T04:45:00.514 INFO:tasks.workunit.client.0.smithi142.stdout:1/470: dwrite d8/db/dc/d19/f5c [0,4194304] 0 2024-03-29T04:45:00.516 INFO:tasks.workunit.client.0.smithi142.stdout:5/547: chown df/d1a/d28/fae 6 1 2024-03-29T04:45:00.519 INFO:tasks.workunit.client.0.smithi142.stdout:5/548: write df/d1a/f22 [3716148,54167] 0 2024-03-29T04:45:00.519 INFO:tasks.workunit.client.0.smithi142.stdout:2/384: rename f9 to de/d11/d3d/f82 0 2024-03-29T04:45:00.523 INFO:tasks.workunit.client.0.smithi142.stdout:6/431: dwrite f4 [0,4194304] 0 2024-03-29T04:45:00.524 INFO:tasks.workunit.client.0.smithi142.stdout:0/428: truncate d7/d13/d67/d72/d37/f43 1925235 0 2024-03-29T04:45:00.524 INFO:tasks.workunit.client.0.smithi142.stdout:7/501: dread d7/d27/d44/d4c/f80 [0,4194304] 0 2024-03-29T04:45:00.525 INFO:tasks.workunit.client.0.smithi142.stdout:5/549: mkdir df/d16/d24/d5c/db1 0 2024-03-29T04:45:00.527 INFO:tasks.workunit.client.0.smithi142.stdout:4/436: rename d0/d9/d11/d78/l87 to d0/d12/d28/d47/l96 0 2024-03-29T04:45:00.527 INFO:tasks.workunit.client.0.smithi142.stdout:5/550: stat l5 0 2024-03-29T04:45:00.529 INFO:tasks.workunit.client.0.smithi142.stdout:3/375: dwrite d3/d1f/d34/d6/f2f [0,4194304] 0 2024-03-29T04:45:00.531 INFO:tasks.workunit.client.0.smithi142.stdout:7/502: mkdir d7/d27/d44/d4c/da3 0 2024-03-29T04:45:00.532 INFO:tasks.workunit.client.0.smithi142.stdout:6/432: write d1c/f55 [2728991,86939] 0 2024-03-29T04:45:00.533 INFO:tasks.workunit.client.0.smithi142.stdout:3/376: readlink d3/d5d/d5f/d64/d6b/l84 0 2024-03-29T04:45:00.537 INFO:tasks.workunit.client.0.smithi142.stdout:8/506: dwrite d5/d1d/d2f/f90 [0,4194304] 0 2024-03-29T04:45:00.538 INFO:tasks.workunit.client.0.smithi142.stdout:5/551: write df/d16/d25/f37 [1464830,20141] 0 2024-03-29T04:45:00.538 INFO:tasks.workunit.client.0.smithi142.stdout:3/377: symlink d3/d5d/d5f/d64/l88 0 2024-03-29T04:45:00.538 INFO:tasks.workunit.client.0.smithi142.stdout:6/433: stat d1c/d33/d46/d71/f60 0 2024-03-29T04:45:00.538 INFO:tasks.workunit.client.0.smithi142.stdout:2/385: truncate de/d11/d1e/d22/d27/f4b 3519073 0 2024-03-29T04:45:00.540 INFO:tasks.workunit.client.0.smithi142.stdout:0/429: rename d7/d9/d55/d42/c4d to d7/d13/d4a/d70/c84 0 2024-03-29T04:45:00.540 INFO:tasks.workunit.client.0.smithi142.stdout:5/552: fdatasync df/d16/d24/d3d/fa6 0 2024-03-29T04:45:00.541 INFO:tasks.workunit.client.0.smithi142.stdout:8/507: write d5/d93/fb0 [974848,12201] 0 2024-03-29T04:45:00.543 INFO:tasks.workunit.client.0.smithi142.stdout:6/434: mknod d1c/c84 0 2024-03-29T04:45:00.545 INFO:tasks.workunit.client.0.smithi142.stdout:0/430: write d7/d13/f28 [3167441,21046] 0 2024-03-29T04:45:00.547 INFO:tasks.workunit.client.0.smithi142.stdout:8/508: readlink d5/d1d/d2f/d76/d4c/l9b 0 2024-03-29T04:45:00.549 INFO:tasks.workunit.client.0.smithi142.stdout:7/503: dwrite d7/dc/de/d20/f85 [0,4194304] 0 2024-03-29T04:45:00.549 INFO:tasks.workunit.client.0.smithi142.stdout:5/553: read f7 [1209057,11286] 0 2024-03-29T04:45:00.550 INFO:tasks.workunit.client.0.smithi142.stdout:7/504: write d7/dc/de/d20/d3f/f61 [832974,96504] 0 2024-03-29T04:45:00.551 INFO:tasks.workunit.client.0.smithi142.stdout:7/505: read - d7/f57 zero size 2024-03-29T04:45:00.552 INFO:tasks.workunit.client.0.smithi142.stdout:7/506: chown d7/d27/d44/d4c 298783 1 2024-03-29T04:45:00.553 INFO:tasks.workunit.client.0.smithi142.stdout:0/431: symlink d7/d13/d4a/l85 0 2024-03-29T04:45:00.554 INFO:tasks.workunit.client.0.smithi142.stdout:4/437: write d0/d12/d28/d47/f6b [12835,106298] 0 2024-03-29T04:45:00.555 INFO:tasks.workunit.client.0.smithi142.stdout:0/432: write d7/d13/f7d [35201,120983] 0 2024-03-29T04:45:00.559 INFO:tasks.workunit.client.0.smithi142.stdout:2/386: creat de/d11/d1e/f83 x:0 0 0 2024-03-29T04:45:00.559 INFO:tasks.workunit.client.0.smithi142.stdout:5/554: mknod df/d29/d5d/cb2 0 2024-03-29T04:45:00.559 INFO:tasks.workunit.client.0.smithi142.stdout:0/433: dread d7/d9/f49 [0,4194304] 0 2024-03-29T04:45:00.564 INFO:tasks.workunit.client.0.smithi142.stdout:6/435: link d1c/d42/d52/f7b d1c/d33/d46/d61/f85 0 2024-03-29T04:45:00.565 INFO:tasks.workunit.client.0.smithi142.stdout:4/438: mkdir d0/d9/d11/d97 0 2024-03-29T04:45:00.567 INFO:tasks.workunit.client.0.smithi142.stdout:0/434: rmdir d7/d9/d6e 39 2024-03-29T04:45:00.569 INFO:tasks.workunit.client.0.smithi142.stdout:5/555: mkdir df/d29/d7a/d97/db3 0 2024-03-29T04:45:00.579 INFO:tasks.workunit.client.0.smithi142.stdout:2/387: unlink de/d11/d1e/d22/d50/f64 0 2024-03-29T04:45:00.579 INFO:tasks.workunit.client.0.smithi142.stdout:5/556: chown df/d16/d24/d2e/l32 685 1 2024-03-29T04:45:00.579 INFO:tasks.workunit.client.0.smithi142.stdout:7/507: symlink d7/dc/la4 0 2024-03-29T04:45:00.579 INFO:tasks.workunit.client.0.smithi142.stdout:6/436: mkdir d1c/d33/d46/d86 0 2024-03-29T04:45:00.580 INFO:tasks.workunit.client.0.smithi142.stdout:8/509: write d5/d2e/f65 [467803,37931] 0 2024-03-29T04:45:00.580 INFO:tasks.workunit.client.0.smithi142.stdout:4/439: creat d0/d9/d11/d4c/d64/f98 x:0 0 0 2024-03-29T04:45:00.582 INFO:tasks.workunit.client.0.smithi142.stdout:2/388: write de/d11/d1e/d22/f3f [1631938,79398] 0 2024-03-29T04:45:00.583 INFO:tasks.workunit.client.0.smithi142.stdout:8/510: rmdir d5/d2e 39 2024-03-29T04:45:00.583 INFO:tasks.workunit.client.0.smithi142.stdout:5/557: read df/d16/d25/f37 [619755,83476] 0 2024-03-29T04:45:00.584 INFO:tasks.workunit.client.0.smithi142.stdout:8/511: read - d5/d1d/d24/f8c zero size 2024-03-29T04:45:00.586 INFO:tasks.workunit.client.0.smithi142.stdout:6/437: creat d1c/d42/d52/f87 x:0 0 0 2024-03-29T04:45:00.588 INFO:tasks.workunit.client.0.smithi142.stdout:6/438: write d1c/d33/f3e [1341059,59618] 0 2024-03-29T04:45:00.588 INFO:tasks.workunit.client.0.smithi142.stdout:7/508: dwrite d7/dc/de/f16 [0,4194304] 0 2024-03-29T04:45:00.590 INFO:tasks.workunit.client.0.smithi142.stdout:6/439: chown d1c/d33/d46/d71/c72 1853121295 1 2024-03-29T04:45:00.590 INFO:tasks.workunit.client.0.smithi142.stdout:7/509: stat d7/dc/de/d20/f2b 0 2024-03-29T04:45:00.592 INFO:tasks.workunit.client.0.smithi142.stdout:0/435: chown d7/d9/d54/d58/l6d 52276283 1 2024-03-29T04:45:00.594 INFO:tasks.workunit.client.0.smithi142.stdout:0/436: chown d7/d13/d67/d72/l29 5347 1 2024-03-29T04:45:00.598 INFO:tasks.workunit.client.0.smithi142.stdout:2/389: dwrite de/d11/d1e/d22/d27/f4b [0,4194304] 0 2024-03-29T04:45:00.598 INFO:tasks.workunit.client.0.smithi142.stdout:6/440: unlink d1c/f30 0 2024-03-29T04:45:00.599 INFO:tasks.workunit.client.0.smithi142.stdout:5/558: write df/d1a/f2f [4308798,50777] 0 2024-03-29T04:45:00.600 INFO:tasks.workunit.client.0.smithi142.stdout:4/440: getdents d0/d9/d11/d78 0 2024-03-29T04:45:00.601 INFO:tasks.workunit.client.0.smithi142.stdout:5/559: stat df/d1a/d21/d27/d45/fa9 0 2024-03-29T04:45:00.602 INFO:tasks.workunit.client.0.smithi142.stdout:5/560: chown df/d1a/c38 7029 1 2024-03-29T04:45:00.603 INFO:tasks.workunit.client.0.smithi142.stdout:8/512: getdents d5/d1d/d2f/d76/d3c 0 2024-03-29T04:45:00.605 INFO:tasks.workunit.client.0.smithi142.stdout:4/441: symlink d0/dc/d15/d3d/d65/l99 0 2024-03-29T04:45:00.605 INFO:tasks.workunit.client.0.smithi142.stdout:6/441: symlink d1c/d33/d46/d61/d7e/l88 0 2024-03-29T04:45:00.609 INFO:tasks.workunit.client.0.smithi142.stdout:8/513: fsync d5/d2e/f65 0 2024-03-29T04:45:00.610 INFO:tasks.workunit.client.0.smithi142.stdout:6/442: fdatasync d1c/f21 0 2024-03-29T04:45:00.610 INFO:tasks.workunit.client.0.smithi142.stdout:6/443: dread - d1c/f3c zero size 2024-03-29T04:45:00.611 INFO:tasks.workunit.client.0.smithi142.stdout:7/510: truncate d7/dc/d4b/d71/f35 2949343 0 2024-03-29T04:45:00.611 INFO:tasks.workunit.client.0.smithi142.stdout:6/444: write d1c/f55 [2460110,84311] 0 2024-03-29T04:45:00.611 INFO:tasks.workunit.client.0.smithi142.stdout:5/561: dwrite df/d29/d6e/f73 [0,4194304] 0 2024-03-29T04:45:00.612 INFO:tasks.workunit.client.0.smithi142.stdout:0/437: write d7/d9/d1e/f71 [662789,67266] 0 2024-03-29T04:45:00.612 INFO:tasks.workunit.client.0.smithi142.stdout:8/514: write d5/d1d/d2f/d76/d4c/d60/f84 [688551,92214] 0 2024-03-29T04:45:00.634 INFO:tasks.workunit.client.0.smithi142.stdout:5/562: write df/d1a/f9e [729622,92076] 0 2024-03-29T04:45:00.636 INFO:tasks.workunit.client.0.smithi142.stdout:0/438: write d7/d9/d54/f59 [92780,77603] 0 2024-03-29T04:45:00.637 INFO:tasks.workunit.client.0.smithi142.stdout:4/442: dread d0/d12/d28/f3a [0,4194304] 0 2024-03-29T04:45:00.638 INFO:tasks.workunit.client.0.smithi142.stdout:8/515: creat d5/d1d/d2f/d76/d4c/d60/fb3 x:0 0 0 2024-03-29T04:45:00.640 INFO:tasks.workunit.client.0.smithi142.stdout:4/443: truncate d0/d45/f6a 5105756 0 2024-03-29T04:45:00.640 INFO:tasks.workunit.client.0.smithi142.stdout:8/516: chown d5/d1d/d2f/d5f/c47 149082767 1 2024-03-29T04:45:00.641 INFO:tasks.workunit.client.0.smithi142.stdout:4/444: stat d0/d9/d11/d4c/d64/d94 0 2024-03-29T04:45:00.641 INFO:tasks.workunit.client.0.smithi142.stdout:6/445: rename d1c/c2c to d1c/d42/c89 0 2024-03-29T04:45:00.642 INFO:tasks.workunit.client.0.smithi142.stdout:4/445: fdatasync d0/dc/f36 0 2024-03-29T04:45:00.643 INFO:tasks.workunit.client.0.smithi142.stdout:6/446: write d1c/d45/f76 [431200,56277] 0 2024-03-29T04:45:00.643 INFO:tasks.workunit.client.0.smithi142.stdout:8/517: mkdir d5/d1d/d24/db4 0 2024-03-29T04:45:00.643 INFO:tasks.workunit.client.0.smithi142.stdout:6/447: fdatasync f4 0 2024-03-29T04:45:00.646 INFO:tasks.workunit.client.0.smithi142.stdout:8/518: truncate d5/d1d/d2f/d76/d4c/d60/f7a 914758 0 2024-03-29T04:45:00.648 INFO:tasks.workunit.client.0.smithi142.stdout:4/446: mknod d0/d9/d11/d5c/d6d/c9a 0 2024-03-29T04:45:00.648 INFO:tasks.workunit.client.0.smithi142.stdout:7/511: truncate d7/dc/de/d20/f4e 1122473 0 2024-03-29T04:45:00.648 INFO:tasks.workunit.client.0.smithi142.stdout:0/439: dread d7/d13/d67/d72/d37/f38 [0,4194304] 0 2024-03-29T04:45:00.649 INFO:tasks.workunit.client.0.smithi142.stdout:7/512: dread - d7/dc/d4b/f82 zero size 2024-03-29T04:45:00.651 INFO:tasks.workunit.client.0.smithi142.stdout:8/519: unlink d5/d1d/d2f/d76/d3c/d70/f82 0 2024-03-29T04:45:00.652 INFO:tasks.workunit.client.0.smithi142.stdout:5/563: truncate df/d16/d24/d5c/f5e 509359 0 2024-03-29T04:45:00.653 INFO:tasks.workunit.client.0.smithi142.stdout:0/440: symlink d7/d13/d67/d72/l86 0 2024-03-29T04:45:00.656 INFO:tasks.workunit.client.0.smithi142.stdout:7/513: creat d7/dc/de/d8d/fa5 x:0 0 0 2024-03-29T04:45:00.656 INFO:tasks.workunit.client.0.smithi142.stdout:4/447: dwrite d0/d9/d11/d4c/d5f/f73 [0,4194304] 0 2024-03-29T04:45:00.656 INFO:tasks.workunit.client.0.smithi142.stdout:0/441: write d7/d13/d67/d72/d37/f73 [946414,76353] 0 2024-03-29T04:45:00.658 INFO:tasks.workunit.client.0.smithi142.stdout:8/520: unlink d5/d1d/d2f/d5f/f4b 0 2024-03-29T04:45:00.658 INFO:tasks.workunit.client.0.smithi142.stdout:4/448: write d0/d9/d11/d5c/f92 [1004584,103992] 0 2024-03-29T04:45:00.659 INFO:tasks.workunit.client.0.smithi142.stdout:0/442: dread d7/d9/d54/f64 [0,4194304] 0 2024-03-29T04:45:00.660 INFO:tasks.workunit.client.0.smithi142.stdout:8/521: chown d5/f22 54722 1 2024-03-29T04:45:00.662 INFO:tasks.workunit.client.0.smithi142.stdout:5/564: symlink df/d1a/d21/d27/lb4 0 2024-03-29T04:45:00.663 INFO:tasks.workunit.client.0.smithi142.stdout:6/448: dread d1c/f39 [0,4194304] 0 2024-03-29T04:45:00.665 INFO:tasks.workunit.client.0.smithi142.stdout:7/514: mknod d7/d27/d3a/d92/ca6 0 2024-03-29T04:45:00.671 INFO:tasks.workunit.client.0.smithi142.stdout:5/565: mkdir df/d16/d24/d5c/db5 0 2024-03-29T04:45:00.671 INFO:tasks.workunit.client.0.smithi142.stdout:7/515: dread d7/dc/de/f16 [0,4194304] 0 2024-03-29T04:45:00.672 INFO:tasks.workunit.client.0.smithi142.stdout:6/449: fdatasync d1c/d42/d52/f7b 0 2024-03-29T04:45:00.674 INFO:tasks.workunit.client.0.smithi142.stdout:7/516: stat d7/dc/de/d8d/c8e 0 2024-03-29T04:45:00.674 INFO:tasks.workunit.client.0.smithi142.stdout:4/449: dwrite d0/d12/f3b [0,4194304] 0 2024-03-29T04:45:00.674 INFO:tasks.workunit.client.0.smithi142.stdout:8/522: symlink d5/d1d/lb5 0 2024-03-29T04:45:00.676 INFO:tasks.workunit.client.0.smithi142.stdout:5/566: write fd [1032597,18599] 0 2024-03-29T04:45:00.676 INFO:tasks.workunit.client.0.smithi142.stdout:0/443: read d7/d9/f2e [303896,55098] 0 2024-03-29T04:45:00.677 INFO:tasks.workunit.client.0.smithi142.stdout:4/450: chown d0/dc/d15/f72 117693 1 2024-03-29T04:45:00.677 INFO:tasks.workunit.client.0.smithi142.stdout:4/451: stat d0/d12/d28 0 2024-03-29T04:45:00.678 INFO:tasks.workunit.client.0.smithi142.stdout:6/450: dread d1c/d42/d52/f54 [0,4194304] 0 2024-03-29T04:45:00.683 INFO:tasks.workunit.client.0.smithi142.stdout:6/451: dread d1c/f40 [0,4194304] 0 2024-03-29T04:45:00.684 INFO:tasks.workunit.client.0.smithi142.stdout:4/452: rename d0/d9/d19/d2c/f5e to d0/d9/d11/d5c/d8d/f9b 0 2024-03-29T04:45:00.688 INFO:tasks.workunit.client.0.smithi142.stdout:6/452: mkdir d1c/d45/d8a 0 2024-03-29T04:45:00.689 INFO:tasks.workunit.client.0.smithi142.stdout:6/453: chown d1c/d45/l47 294373 1 2024-03-29T04:45:00.689 INFO:tasks.workunit.client.0.smithi142.stdout:5/567: creat df/d16/d24/fb6 x:0 0 0 2024-03-29T04:45:00.690 INFO:tasks.workunit.client.0.smithi142.stdout:5/568: chown df/d1a/d28/f8d 2 1 2024-03-29T04:45:00.691 INFO:tasks.workunit.client.0.smithi142.stdout:7/517: dread d7/dc/de/f5c [0,4194304] 0 2024-03-29T04:45:00.692 INFO:tasks.workunit.client.0.smithi142.stdout:7/518: write d7/f90 [377966,105663] 0 2024-03-29T04:45:00.692 INFO:tasks.workunit.client.0.smithi142.stdout:4/453: rename d0/d9/d11/d4c/d64/f82 to d0/d9/d11/d4c/d5f/f9c 0 2024-03-29T04:45:00.693 INFO:tasks.workunit.client.0.smithi142.stdout:0/444: creat d7/d9/d55/f87 x:0 0 0 2024-03-29T04:45:00.696 INFO:tasks.workunit.client.0.smithi142.stdout:5/569: rmdir df/d1a/d21/da5 39 2024-03-29T04:45:00.696 INFO:tasks.workunit.client.0.smithi142.stdout:5/570: chown df/d16/f39 8505138 1 2024-03-29T04:45:00.698 INFO:tasks.workunit.client.0.smithi142.stdout:6/454: rename d1c/d42 to d1c/d33/d46/d8b 0 2024-03-29T04:45:00.699 INFO:tasks.workunit.client.0.smithi142.stdout:7/519: creat d7/dc/de/d8d/fa7 x:0 0 0 2024-03-29T04:45:00.701 INFO:tasks.workunit.client.0.smithi142.stdout:7/520: write d7/dc/d4b/d71/d30/f46 [417062,32478] 0 2024-03-29T04:45:00.701 INFO:tasks.workunit.client.0.smithi142.stdout:4/454: symlink d0/d9/d11/d5c/d6d/l9d 0 2024-03-29T04:45:00.703 INFO:tasks.workunit.client.0.smithi142.stdout:7/521: dread d7/d27/f9f [0,4194304] 0 2024-03-29T04:45:00.704 INFO:tasks.workunit.client.0.smithi142.stdout:0/445: rename d7/d9/d54/f59 to d7/d9/d1e/f88 0 2024-03-29T04:45:00.705 INFO:tasks.workunit.client.0.smithi142.stdout:6/455: creat d1c/d33/d46/f8c x:0 0 0 2024-03-29T04:45:00.706 INFO:tasks.workunit.client.0.smithi142.stdout:8/523: symlink d5/d1d/d2f/d76/d3c/d70/lb6 0 2024-03-29T04:45:00.706 INFO:tasks.workunit.client.0.smithi142.stdout:6/456: write d1c/f23 [918663,51403] 0 2024-03-29T04:45:00.706 INFO:tasks.workunit.client.0.smithi142.stdout:5/571: symlink df/d29/d7a/d97/db3/lb7 0 2024-03-29T04:45:00.708 INFO:tasks.workunit.client.0.smithi142.stdout:7/522: rename d7/f43 to d7/d27/fa8 0 2024-03-29T04:45:00.710 INFO:tasks.workunit.client.0.smithi142.stdout:4/455: mkdir d0/d9/d11/d97/d9e 0 2024-03-29T04:45:00.713 INFO:tasks.workunit.client.0.smithi142.stdout:7/523: mknod d7/dc/de/d20/d3f/ca9 0 2024-03-29T04:45:00.714 INFO:tasks.workunit.client.0.smithi142.stdout:4/456: fdatasync d0/d9/d11/f2b 0 2024-03-29T04:45:00.714 INFO:tasks.workunit.client.0.smithi142.stdout:0/446: link d7/c11 d7/d13/d67/d72/d37/c89 0 2024-03-29T04:45:00.716 INFO:tasks.workunit.client.0.smithi142.stdout:8/524: rename d5/l6 to d5/d1d/d2f/d45/lb7 0 2024-03-29T04:45:00.716 INFO:tasks.workunit.client.0.smithi142.stdout:5/572: link df/d29/c3b df/d29/cb8 0 2024-03-29T04:45:00.717 INFO:tasks.workunit.client.0.smithi142.stdout:0/447: fdatasync d7/d9/d55/f57 0 2024-03-29T04:45:00.717 INFO:tasks.workunit.client.0.smithi142.stdout:7/524: write d7/dc/d4b/d71/d30/f46 [414555,105686] 0 2024-03-29T04:45:00.717 INFO:tasks.workunit.client.0.smithi142.stdout:6/457: dwrite fb [0,4194304] 0 2024-03-29T04:45:00.719 INFO:tasks.workunit.client.0.smithi142.stdout:4/457: mkdir d0/d9/d11/d84/d9f 0 2024-03-29T04:45:00.720 INFO:tasks.workunit.client.0.smithi142.stdout:7/525: write d7/dc/de/d20/d3f/f75 [932204,70971] 0 2024-03-29T04:45:00.720 INFO:tasks.workunit.client.0.smithi142.stdout:5/573: creat df/d2c/fb9 x:0 0 0 2024-03-29T04:45:00.724 INFO:tasks.workunit.client.0.smithi142.stdout:4/458: mknod d0/d9/d11/d78/ca0 0 2024-03-29T04:45:00.726 INFO:tasks.workunit.client.0.smithi142.stdout:5/574: unlink df/d16/d24/f56 0 2024-03-29T04:45:00.727 INFO:tasks.workunit.client.0.smithi142.stdout:7/526: mknod d7/dc/de/d8d/caa 0 2024-03-29T04:45:00.729 INFO:tasks.workunit.client.0.smithi142.stdout:8/525: dwrite d5/d1d/d2f/d76/d4c/d60/fb3 [0,4194304] 0 2024-03-29T04:45:00.731 INFO:tasks.workunit.client.0.smithi142.stdout:9/353: sync 2024-03-29T04:45:00.732 INFO:tasks.workunit.client.0.smithi142.stdout:0/448: dwrite d7/d9/d55/f87 [0,4194304] 0 2024-03-29T04:45:00.732 INFO:tasks.workunit.client.0.smithi142.stdout:6/458: link d1c/d33/d46/d8b/d52/f87 d1c/d33/d46/d8b/d52/f8d 0 2024-03-29T04:45:00.735 INFO:tasks.workunit.client.0.smithi142.stdout:5/575: creat df/d16/d24/d2e/d47/d8c/fba x:0 0 0 2024-03-29T04:45:00.737 INFO:tasks.workunit.client.0.smithi142.stdout:7/527: mkdir d7/d27/d44/dab 0 2024-03-29T04:45:00.738 INFO:tasks.workunit.client.0.smithi142.stdout:6/459: write fc [4706545,117703] 0 2024-03-29T04:45:00.738 INFO:tasks.workunit.client.0.smithi142.stdout:8/526: rmdir d5/d1d/d2f/d45 39 2024-03-29T04:45:00.741 INFO:tasks.workunit.client.0.smithi142.stdout:4/459: dwrite d0/dc/d15/d3d/f43 [0,4194304] 0 2024-03-29T04:45:00.743 INFO:tasks.workunit.client.0.smithi142.stdout:8/527: write d5/d1d/d2f/d76/d4c/d60/f8f [4278663,19675] 0 2024-03-29T04:45:00.747 INFO:tasks.workunit.client.0.smithi142.stdout:0/449: dwrite d7/d9/d55/d42/f46 [4194304,4194304] 0 2024-03-29T04:45:00.750 INFO:tasks.workunit.client.0.smithi142.stdout:7/528: creat d7/dc/d4b/d71/d30/d72/fac x:0 0 0 2024-03-29T04:45:00.755 INFO:tasks.workunit.client.0.smithi142.stdout:0/450: dwrite d7/d13/d67/d72/f48 [0,4194304] 0 2024-03-29T04:45:00.756 INFO:tasks.workunit.client.0.smithi142.stdout:4/460: rename d0/f8 to d0/d12/d28/d47/d7a/fa1 0 2024-03-29T04:45:00.758 INFO:tasks.workunit.client.0.smithi142.stdout:5/576: link fe df/d29/d5d/d75/fbb 0 2024-03-29T04:45:00.759 INFO:tasks.workunit.client.0.smithi142.stdout:7/529: fsync d7/d27/d44/d4c/f79 0 2024-03-29T04:45:00.762 INFO:tasks.workunit.client.0.smithi142.stdout:0/451: symlink d7/l8a 0 2024-03-29T04:45:00.766 INFO:tasks.workunit.client.0.smithi142.stdout:5/577: readlink df/d16/l65 0 2024-03-29T04:45:00.767 INFO:tasks.workunit.client.0.smithi142.stdout:7/530: mknod d7/dc/de/d20/cad 0 2024-03-29T04:45:00.767 INFO:tasks.workunit.client.0.smithi142.stdout:7/531: stat d7 0 2024-03-29T04:45:00.769 INFO:tasks.workunit.client.0.smithi142.stdout:8/528: rename d5/f30 to d5/d93/fb8 0 2024-03-29T04:45:00.771 INFO:tasks.workunit.client.0.smithi142.stdout:5/578: truncate df/d16/f39 55272 0 2024-03-29T04:45:00.772 INFO:tasks.workunit.client.0.smithi142.stdout:7/532: creat d7/dc/de/d20/d3f/fae x:0 0 0 2024-03-29T04:45:00.774 INFO:tasks.workunit.client.0.smithi142.stdout:5/579: mkdir df/d29/dbc 0 2024-03-29T04:45:00.776 INFO:tasks.workunit.client.0.smithi142.stdout:8/529: dwrite d5/d1d/d24/f8c [0,4194304] 0 2024-03-29T04:45:00.777 INFO:tasks.workunit.client.0.smithi142.stdout:4/461: getdents d0/dc/d15/d3d 0 2024-03-29T04:45:00.778 INFO:tasks.workunit.client.0.smithi142.stdout:4/462: stat d0/d9/d11/d4c/d5f/f9c 0 2024-03-29T04:45:00.780 INFO:tasks.workunit.client.0.smithi142.stdout:3/378: sync 2024-03-29T04:45:00.780 INFO:tasks.workunit.client.0.smithi142.stdout:1/471: sync 2024-03-29T04:45:00.780 INFO:tasks.workunit.client.0.smithi142.stdout:2/390: sync 2024-03-29T04:45:00.780 INFO:tasks.workunit.client.0.smithi142.stdout:7/533: rename d7/d62/f64 to d7/d27/d3a/d92/d97/faf 0 2024-03-29T04:45:00.780 INFO:tasks.workunit.client.0.smithi142.stdout:0/452: link d7/d9/f30 d7/d13/d4a/d70/f8b 0 2024-03-29T04:45:00.781 INFO:tasks.workunit.client.0.smithi142.stdout:3/379: dread - d3/d54/f78 zero size 2024-03-29T04:45:00.783 INFO:tasks.workunit.client.0.smithi142.stdout:5/580: symlink df/d1a/d21/d55/d66/lbd 0 2024-03-29T04:45:00.785 INFO:tasks.workunit.client.0.smithi142.stdout:8/530: creat d5/d1d/d2f/d76/d9c/fb9 x:0 0 0 2024-03-29T04:45:00.786 INFO:tasks.workunit.client.0.smithi142.stdout:2/391: read de/d11/d1e/d22/d27/f4c [694352,31611] 0 2024-03-29T04:45:00.787 INFO:tasks.workunit.client.0.smithi142.stdout:6/460: write d1c/d33/f36 [61393,50004] 0 2024-03-29T04:45:00.790 INFO:tasks.workunit.client.0.smithi142.stdout:2/392: fsync de/d11/f4f 0 2024-03-29T04:45:00.794 INFO:tasks.workunit.client.0.smithi142.stdout:9/354: dwrite d0/d2/d6/d16/f2b [0,4194304] 0 2024-03-29T04:45:00.796 INFO:tasks.workunit.client.0.smithi142.stdout:5/581: creat df/d1a/d21/d55/d66/fbe x:0 0 0 2024-03-29T04:45:00.798 INFO:tasks.workunit.client.0.smithi142.stdout:1/472: rename d8/db/dc/d77/d5a/f93 to d8/db/dc/d19/d1d/d2d/faa 0 2024-03-29T04:45:00.799 INFO:tasks.workunit.client.0.smithi142.stdout:3/380: dread d3/d1f/d34/d6/f2f [0,4194304] 0 2024-03-29T04:45:00.802 INFO:tasks.workunit.client.0.smithi142.stdout:4/463: getdents d0/d12 0 2024-03-29T04:45:00.803 INFO:tasks.workunit.client.0.smithi142.stdout:8/531: truncate d5/d1d/d2f/fa7 3240198 0 2024-03-29T04:45:00.805 INFO:tasks.workunit.client.0.smithi142.stdout:6/461: dwrite fc [4194304,4194304] 0 2024-03-29T04:45:00.805 INFO:tasks.workunit.client.0.smithi142.stdout:1/473: write d8/db/dc/d19/d1d/d2d/f3e [727869,11464] 0 2024-03-29T04:45:00.805 INFO:tasks.workunit.client.0.smithi142.stdout:9/355: mkdir d0/d2/d6b 0 2024-03-29T04:45:00.807 INFO:tasks.workunit.client.0.smithi142.stdout:1/474: fdatasync d8/f51 0 2024-03-29T04:45:00.809 INFO:tasks.workunit.client.0.smithi142.stdout:9/356: chown d0/d2/d61/f2d 15881883 1 2024-03-29T04:45:00.810 INFO:tasks.workunit.client.0.smithi142.stdout:1/475: chown d8/db/dc/d77/d31/d4d/d56 1296 1 2024-03-29T04:45:00.810 INFO:tasks.workunit.client.0.smithi142.stdout:0/453: dread d7/d13/d67/d72/d37/f56 [0,4194304] 0 2024-03-29T04:45:00.811 INFO:tasks.workunit.client.0.smithi142.stdout:9/357: fdatasync d0/d2/d6/d27/d3c/f4c 0 2024-03-29T04:45:00.811 INFO:tasks.workunit.client.0.smithi142.stdout:8/532: mknod d5/d1d/d2f/d76/d2d/cba 0 2024-03-29T04:45:00.812 INFO:tasks.workunit.client.0.smithi142.stdout:8/533: chown d5/d1d/d2f/d76/c15 0 1 2024-03-29T04:45:00.813 INFO:tasks.workunit.client.0.smithi142.stdout:3/381: dread d3/d1f/d24/d47/f75 [0,4194304] 0 2024-03-29T04:45:00.814 INFO:tasks.workunit.client.0.smithi142.stdout:2/393: rename de/d7f to de/d11/d1e/d84 0 2024-03-29T04:45:00.815 INFO:tasks.workunit.client.0.smithi142.stdout:8/534: stat d5/d1d/d24/d66 0 2024-03-29T04:45:00.820 INFO:tasks.workunit.client.0.smithi142.stdout:7/534: dread d7/dc/d4b/f9a [0,4194304] 0 2024-03-29T04:45:00.821 INFO:tasks.workunit.client.0.smithi142.stdout:9/358: mkdir d0/d6c 0 2024-03-29T04:45:00.826 INFO:tasks.workunit.client.0.smithi142.stdout:5/582: rename df/d16/d50 to df/d1a/d21/d27/d45/dbf 0 2024-03-29T04:45:00.827 INFO:tasks.workunit.client.0.smithi142.stdout:0/454: rmdir d7/d9/d6e 39 2024-03-29T04:45:00.831 INFO:tasks.workunit.client.0.smithi142.stdout:5/583: fsync df/d1a/d21/d27/d45/fa9 0 2024-03-29T04:45:00.831 INFO:tasks.workunit.client.0.smithi142.stdout:4/464: rename d0/d9/d11/ff to d0/d9/d11/d97/d9e/fa2 0 2024-03-29T04:45:00.832 INFO:tasks.workunit.client.0.smithi142.stdout:1/476: dwrite d8/db/dc/d77/d31/f64 [0,4194304] 0 2024-03-29T04:45:00.833 INFO:tasks.workunit.client.0.smithi142.stdout:3/382: link d3/d29/d2b/d40/l5e d3/d29/d2d/d6c/l89 0 2024-03-29T04:45:00.835 INFO:tasks.workunit.client.0.smithi142.stdout:2/394: dwrite de/d11/d1e/d22/d50/f62 [0,4194304] 0 2024-03-29T04:45:00.835 INFO:tasks.workunit.client.0.smithi142.stdout:4/465: chown d0/d9/d11/f1b 426546 1 2024-03-29T04:45:00.838 INFO:tasks.workunit.client.0.smithi142.stdout:0/455: mknod d7/d9/d54/c8c 0 2024-03-29T04:45:00.838 INFO:tasks.workunit.client.0.smithi142.stdout:7/535: link d7/d27/d44/d4c/l73 d7/dc/d4b/lb0 0 2024-03-29T04:45:00.843 INFO:tasks.workunit.client.0.smithi142.stdout:8/535: rename d5/d1d/lb5 to d5/d1d/d2f/d76/lbb 0 2024-03-29T04:45:00.843 INFO:tasks.workunit.client.0.smithi142.stdout:6/462: dread d1c/f55 [0,4194304] 0 2024-03-29T04:45:00.843 INFO:tasks.workunit.client.0.smithi142.stdout:1/477: mknod d8/db/dc/d19/d1d/d2d/d8b/cab 0 2024-03-29T04:45:00.845 INFO:tasks.workunit.client.0.smithi142.stdout:2/395: mkdir de/d11/d2b/d3e/d85 0 2024-03-29T04:45:00.846 INFO:tasks.workunit.client.0.smithi142.stdout:0/456: mknod d7/d9/d54/d58/c8d 0 2024-03-29T04:45:00.847 INFO:tasks.workunit.client.0.smithi142.stdout:4/466: mknod d0/d9/d11/d97/ca3 0 2024-03-29T04:45:00.847 INFO:tasks.workunit.client.0.smithi142.stdout:0/457: chown d7 7638 1 2024-03-29T04:45:00.849 INFO:tasks.workunit.client.0.smithi142.stdout:6/463: dread - d1c/d33/d46/d8b/d52/f8d zero size 2024-03-29T04:45:00.849 INFO:tasks.workunit.client.0.smithi142.stdout:9/359: dwrite d0/d12/f59 [0,4194304] 0 2024-03-29T04:45:00.850 INFO:tasks.workunit.client.0.smithi142.stdout:8/536: mknod d5/d1d/d59/cbc 0 2024-03-29T04:45:00.853 INFO:tasks.workunit.client.0.smithi142.stdout:6/464: dread - d1c/f21 zero size 2024-03-29T04:45:00.854 INFO:tasks.workunit.client.0.smithi142.stdout:2/396: write de/d11/d1e/d22/d50/f57 [1383093,51436] 0 2024-03-29T04:45:00.854 INFO:tasks.workunit.client.0.smithi142.stdout:1/478: mknod d8/db/dc/d19/d1d/d2d/d8b/cac 0 2024-03-29T04:45:00.860 INFO:tasks.workunit.client.0.smithi142.stdout:3/383: dwrite d3/d1f/d24/d47/f57 [0,4194304] 0 2024-03-29T04:45:00.860 INFO:tasks.workunit.client.0.smithi142.stdout:9/360: chown d0/d2/d6/d27/d3c/f4c 7 1 2024-03-29T04:45:00.860 INFO:tasks.workunit.client.0.smithi142.stdout:5/584: dwrite df/d1a/d21/f3e [4194304,4194304] 0 2024-03-29T04:45:00.862 INFO:tasks.workunit.client.0.smithi142.stdout:8/537: symlink d5/d1d/d2f/d76/d4c/d60/lbd 0 2024-03-29T04:45:00.867 INFO:tasks.workunit.client.0.smithi142.stdout:1/479: write d8/db/dc/d19/d1d/d15/f60 [3737260,120889] 0 2024-03-29T04:45:00.867 INFO:tasks.workunit.client.0.smithi142.stdout:2/397: creat de/d11/d17/d5d/f86 x:0 0 0 2024-03-29T04:45:00.870 INFO:tasks.workunit.client.0.smithi142.stdout:3/384: creat d3/d29/d2d/f8a x:0 0 0 2024-03-29T04:45:00.871 INFO:tasks.workunit.client.0.smithi142.stdout:5/585: symlink df/d1a/lc0 0 2024-03-29T04:45:00.871 INFO:tasks.workunit.client.0.smithi142.stdout:8/538: creat d5/d1d/d59/fbe x:0 0 0 2024-03-29T04:45:00.872 INFO:tasks.workunit.client.0.smithi142.stdout:7/536: dwrite d7/dc/d4b/d71/f28 [4194304,4194304] 0 2024-03-29T04:45:00.874 INFO:tasks.workunit.client.0.smithi142.stdout:3/385: chown d3/d5d/d5f 78 1 2024-03-29T04:45:00.875 INFO:tasks.workunit.client.0.smithi142.stdout:1/480: stat d8/db/dc/d19/d1d/d1f/f33 0 2024-03-29T04:45:00.876 INFO:tasks.workunit.client.0.smithi142.stdout:4/467: dwrite d0/d12/f3b [0,4194304] 0 2024-03-29T04:45:00.876 INFO:tasks.workunit.client.0.smithi142.stdout:5/586: read fb [3005068,73850] 0 2024-03-29T04:45:00.877 INFO:tasks.workunit.client.0.smithi142.stdout:9/361: fsync d0/d12/f59 0 2024-03-29T04:45:00.878 INFO:tasks.workunit.client.0.smithi142.stdout:2/398: creat de/d11/f87 x:0 0 0 2024-03-29T04:45:00.882 INFO:tasks.workunit.client.0.smithi142.stdout:4/468: write d0/d9/d11/f2b [497312,120422] 0 2024-03-29T04:45:00.883 INFO:tasks.workunit.client.0.smithi142.stdout:7/537: readlink d7/d27/d44/d4c/l73 0 2024-03-29T04:45:00.887 INFO:tasks.workunit.client.0.smithi142.stdout:1/481: truncate d8/f9 2949601 0 2024-03-29T04:45:00.887 INFO:tasks.workunit.client.0.smithi142.stdout:6/465: dwrite d1c/d33/d46/d8b/f4c [0,4194304] 0 2024-03-29T04:45:00.887 INFO:tasks.workunit.client.0.smithi142.stdout:2/399: symlink de/d11/d1e/d22/d50/l88 0 2024-03-29T04:45:00.890 INFO:tasks.workunit.client.0.smithi142.stdout:9/362: symlink d0/d2/d6/d27/d3c/d4f/l6d 0 2024-03-29T04:45:00.896 INFO:tasks.workunit.client.0.smithi142.stdout:6/466: rmdir d1c/d45/d51 39 2024-03-29T04:45:00.896 INFO:tasks.workunit.client.0.smithi142.stdout:2/400: mknod de/d11/d23/c89 0 2024-03-29T04:45:00.898 INFO:tasks.workunit.client.0.smithi142.stdout:3/386: dwrite d3/d29/d2d/f48 [0,4194304] 0 2024-03-29T04:45:00.898 INFO:tasks.workunit.client.0.smithi142.stdout:1/482: write d8/db/dc/d19/d1d/d1f/d97/d9e/fa6 [445348,4183] 0 2024-03-29T04:45:00.901 INFO:tasks.workunit.client.0.smithi142.stdout:8/539: dwrite d5/d1d/d2f/d76/d4c/f86 [0,4194304] 0 2024-03-29T04:45:00.902 INFO:tasks.workunit.client.0.smithi142.stdout:5/587: dread f7 [0,4194304] 0 2024-03-29T04:45:00.904 INFO:tasks.workunit.client.0.smithi142.stdout:2/401: dread - de/d11/d1e/d4d/d61/d7d/f7e zero size 2024-03-29T04:45:00.908 INFO:tasks.workunit.client.0.smithi142.stdout:6/467: write d1c/d33/d46/f48 [1573347,32117] 0 2024-03-29T04:45:00.908 INFO:tasks.workunit.client.0.smithi142.stdout:5/588: fsync df/d16/d25/f59 0 2024-03-29T04:45:00.909 INFO:tasks.workunit.client.0.smithi142.stdout:3/387: rename d3/d1f/d34 to d3/d29/d2b/d40/d67/d8b 0 2024-03-29T04:45:00.912 INFO:tasks.workunit.client.0.smithi142.stdout:7/538: getdents d7/dc/d4b/d71 0 2024-03-29T04:45:00.917 INFO:tasks.workunit.client.0.smithi142.stdout:6/468: mknod d1c/d33/d46/d71/c8e 0 2024-03-29T04:45:00.919 INFO:tasks.workunit.client.0.smithi142.stdout:8/540: dwrite d5/d1d/d2f/d76/d2d/f4f [4194304,4194304] 0 2024-03-29T04:45:00.922 INFO:tasks.workunit.client.0.smithi142.stdout:6/469: chown d1c/f21 51561393 1 2024-03-29T04:45:00.923 INFO:tasks.workunit.client.0.smithi142.stdout:7/539: creat d7/dc/d4b/d71/d30/fb1 x:0 0 0 2024-03-29T04:45:00.924 INFO:tasks.workunit.client.0.smithi142.stdout:4/469: dwrite d0/dc/f33 [0,4194304] 0 2024-03-29T04:45:00.924 INFO:tasks.workunit.client.0.smithi142.stdout:1/483: getdents d8/db/dc/d19/d1d/d1f/d97/d9e 0 2024-03-29T04:45:00.926 INFO:tasks.workunit.client.0.smithi142.stdout:3/388: getdents d3/d29/d2b/d40/d67/d8b/d4d 0 2024-03-29T04:45:00.930 INFO:tasks.workunit.client.0.smithi142.stdout:5/589: dwrite df/d29/d6e/d8a/f98 [0,4194304] 0 2024-03-29T04:45:00.931 INFO:tasks.workunit.client.0.smithi142.stdout:9/363: dwrite d0/d2/d6/d16/f40 [4194304,4194304] 0 2024-03-29T04:45:00.931 INFO:tasks.workunit.client.0.smithi142.stdout:4/470: mkdir d0/d9/d11/da4 0 2024-03-29T04:45:00.933 INFO:tasks.workunit.client.0.smithi142.stdout:5/590: fsync df/d1a/d21/d55/f81 0 2024-03-29T04:45:00.934 INFO:tasks.workunit.client.0.smithi142.stdout:7/540: read d7/f1a [222263,45099] 0 2024-03-29T04:45:00.939 INFO:tasks.workunit.client.0.smithi142.stdout:7/541: fsync d7/f19 0 2024-03-29T04:45:00.940 INFO:tasks.workunit.client.0.smithi142.stdout:5/591: dread - df/d1a/d28/fae zero size 2024-03-29T04:45:00.940 INFO:tasks.workunit.client.0.smithi142.stdout:8/541: link d5/d1d/d24/c4e d5/d2e/cbf 0 2024-03-29T04:45:00.942 INFO:tasks.workunit.client.0.smithi142.stdout:2/402: write de/d11/f37 [1077242,77827] 0 2024-03-29T04:45:00.942 INFO:tasks.workunit.client.0.smithi142.stdout:9/364: read - d0/d2/d6/d27/d3c/f4c zero size 2024-03-29T04:45:00.943 INFO:tasks.workunit.client.0.smithi142.stdout:7/542: readlink d7/d27/d44/d4c/l73 0 2024-03-29T04:45:00.944 INFO:tasks.workunit.client.0.smithi142.stdout:3/389: dread d3/d29/d2b/d40/f4a [0,4194304] 0 2024-03-29T04:45:00.945 INFO:tasks.workunit.client.0.smithi142.stdout:1/484: link d8/db/dc/d19/d1d/d1f/d97/d9e/fa7 d8/db/dc/d19/d1d/d2d/fad 0 2024-03-29T04:45:00.946 INFO:tasks.workunit.client.0.smithi142.stdout:4/471: dread d0/d12/d28/f3a [0,4194304] 0 2024-03-29T04:45:00.947 INFO:tasks.workunit.client.0.smithi142.stdout:6/470: rename d1c/d45/d51/c5b to d1c/c8f 0 2024-03-29T04:45:00.949 INFO:tasks.workunit.client.0.smithi142.stdout:5/592: unlink df/d29/d7a/d97/f9c 0 2024-03-29T04:45:00.952 INFO:tasks.workunit.client.0.smithi142.stdout:9/365: creat d0/d2/d6/d27/d3c/d4f/f6e x:0 0 0 2024-03-29T04:45:00.953 INFO:tasks.workunit.client.0.smithi142.stdout:8/542: symlink d5/d1d/d2f/d76/d3c/d70/lc0 0 2024-03-29T04:45:00.953 INFO:tasks.workunit.client.0.smithi142.stdout:4/472: mkdir d0/d9/d11/d5c/d8d/da5 0 2024-03-29T04:45:00.954 INFO:tasks.workunit.client.0.smithi142.stdout:8/543: chown d5/d1d/d2f/d76/d2d/l5b 16151758 1 2024-03-29T04:45:00.955 INFO:tasks.workunit.client.0.smithi142.stdout:5/593: dwrite df/d1a/f1f [4194304,4194304] 0 2024-03-29T04:45:00.957 INFO:tasks.workunit.client.0.smithi142.stdout:6/471: mkdir d1c/d90 0 2024-03-29T04:45:00.957 INFO:tasks.workunit.client.0.smithi142.stdout:8/544: write d5/d1d/d2f/d76/d4c/d60/fb3 [2741228,124850] 0 2024-03-29T04:45:00.959 INFO:tasks.workunit.client.0.smithi142.stdout:5/594: dread df/d29/f6b [0,4194304] 0 2024-03-29T04:45:00.960 INFO:tasks.workunit.client.0.smithi142.stdout:2/403: getdents de/d11/d1e 0 2024-03-29T04:45:00.964 INFO:tasks.workunit.client.0.smithi142.stdout:2/404: write de/d11/d1e/d4d/d61/f78 [34132,122830] 0 2024-03-29T04:45:00.968 INFO:tasks.workunit.client.0.smithi142.stdout:5/595: dwrite df/fa2 [0,4194304] 0 2024-03-29T04:45:00.972 INFO:tasks.workunit.client.0.smithi142.stdout:3/390: dread d3/d1f/d24/f37 [0,4194304] 0 2024-03-29T04:45:00.974 INFO:tasks.workunit.client.0.smithi142.stdout:3/391: fsync d3/d29/d2b/d40/d67/d8b/f72 0 2024-03-29T04:45:00.974 INFO:tasks.workunit.client.0.smithi142.stdout:2/405: rename de/d11/c12 to de/d11/d1e/d4d/c8a 0 2024-03-29T04:45:00.974 INFO:tasks.workunit.client.0.smithi142.stdout:2/406: readlink de/d11/d1e/d4d/l67 0 2024-03-29T04:45:00.979 INFO:tasks.workunit.client.0.smithi142.stdout:3/392: mknod d3/d29/d2b/d40/d67/c8c 0 2024-03-29T04:45:00.981 INFO:tasks.workunit.client.0.smithi142.stdout:5/596: link df/c2b df/d29/d5d/d75/cc1 0 2024-03-29T04:45:00.981 INFO:tasks.workunit.client.0.smithi142.stdout:3/393: write d3/d29/d2b/d40/d67/f7f [882003,64312] 0 2024-03-29T04:45:00.982 INFO:tasks.workunit.client.0.smithi142.stdout:5/597: write df/d2c/fb9 [725752,34263] 0 2024-03-29T04:45:00.982 INFO:tasks.workunit.client.0.smithi142.stdout:4/473: truncate d0/d12/f3b 4147851 0 2024-03-29T04:45:00.983 INFO:tasks.workunit.client.0.smithi142.stdout:2/407: creat de/f8b x:0 0 0 2024-03-29T04:45:00.984 INFO:tasks.workunit.client.0.smithi142.stdout:4/474: chown d0/d12/d28/l40 69093 1 2024-03-29T04:45:00.986 INFO:tasks.workunit.client.0.smithi142.stdout:3/394: rename d3/d29/d2b/d40/f45 to d3/d54/f8d 0 2024-03-29T04:45:00.986 INFO:tasks.workunit.client.0.smithi142.stdout:2/408: chown de/d11/d1e/d22/d50/f57 230 1 2024-03-29T04:45:00.987 INFO:tasks.workunit.client.0.smithi142.stdout:5/598: creat df/d29/d5d/d75/d90/fc2 x:0 0 0 2024-03-29T04:45:00.988 INFO:tasks.workunit.client.0.smithi142.stdout:4/475: creat d0/d9/d11/d4c/d64/fa6 x:0 0 0 2024-03-29T04:45:00.988 INFO:tasks.workunit.client.0.smithi142.stdout:2/409: chown de/d11/d2b/l33 105287358 1 2024-03-29T04:45:00.990 INFO:tasks.workunit.client.0.smithi142.stdout:8/545: dwrite d5/d1d/f85 [0,4194304] 0 2024-03-29T04:45:00.991 INFO:tasks.workunit.client.0.smithi142.stdout:4/476: chown d0/d12/f7b 58 1 2024-03-29T04:45:00.991 INFO:tasks.workunit.client.0.smithi142.stdout:2/410: dread - de/d11/d1e/d4d/f6b zero size 2024-03-29T04:45:00.997 INFO:tasks.workunit.client.0.smithi142.stdout:3/395: dwrite d3/d29/d2b/d40/d67/d8b/f3e [4194304,4194304] 0 2024-03-29T04:45:00.997 INFO:tasks.workunit.client.0.smithi142.stdout:8/546: mknod d5/d1d/d81/cc1 0 2024-03-29T04:45:01.000 INFO:tasks.workunit.client.0.smithi142.stdout:5/599: creat df/d16/d24/d5c/db1/fc3 x:0 0 0 2024-03-29T04:45:01.003 INFO:tasks.workunit.client.0.smithi142.stdout:2/411: rmdir de/d11/d17/d5d 39 2024-03-29T04:45:01.004 INFO:tasks.workunit.client.0.smithi142.stdout:3/396: read d3/d29/d2b/d40/d67/d8b/d6/f20 [2615139,113138] 0 2024-03-29T04:45:01.005 INFO:tasks.workunit.client.0.smithi142.stdout:4/477: truncate d0/f21 4901494 0 2024-03-29T04:45:01.010 INFO:tasks.workunit.client.0.smithi142.stdout:3/397: dread d3/f3c [0,4194304] 0 2024-03-29T04:45:01.013 INFO:tasks.workunit.client.0.smithi142.stdout:2/412: creat de/d11/d2b/d7b/f8c x:0 0 0 2024-03-29T04:45:01.014 INFO:tasks.workunit.client.0.smithi142.stdout:8/547: dwrite d5/d1d/d2f/d5f/d48/f61 [0,4194304] 0 2024-03-29T04:45:01.015 INFO:tasks.workunit.client.0.smithi142.stdout:5/600: dread df/d16/d25/f59 [0,4194304] 0 2024-03-29T04:45:01.021 INFO:tasks.workunit.client.0.smithi142.stdout:8/548: write d5/d1d/d2f/d45/f9a [124608,62779] 0 2024-03-29T04:45:01.022 INFO:tasks.workunit.client.0.smithi142.stdout:5/601: rmdir df/d16/d24/d2e 39 2024-03-29T04:45:01.023 INFO:tasks.workunit.client.0.smithi142.stdout:4/478: link d0/d45/l4e d0/d9/d11/d78/la7 0 2024-03-29T04:45:01.025 INFO:tasks.workunit.client.0.smithi142.stdout:5/602: rmdir df/d29/d5d 39 2024-03-29T04:45:01.026 INFO:tasks.workunit.client.0.smithi142.stdout:4/479: rmdir d0/d9/d11/d5c/d6d 39 2024-03-29T04:45:01.027 INFO:tasks.workunit.client.0.smithi142.stdout:8/549: mknod d5/d1d/d2f/d5f/cc2 0 2024-03-29T04:45:01.028 INFO:tasks.workunit.client.0.smithi142.stdout:5/603: unlink df/d2c/f71 0 2024-03-29T04:45:01.029 INFO:tasks.workunit.client.0.smithi142.stdout:4/480: mknod d0/d45/ca8 0 2024-03-29T04:45:01.031 INFO:tasks.workunit.client.0.smithi142.stdout:4/481: symlink d0/d9/d19/la9 0 2024-03-29T04:45:01.033 INFO:tasks.workunit.client.0.smithi142.stdout:5/604: fsync df/d29/d5d/d75/fbb 0 2024-03-29T04:45:01.037 INFO:tasks.workunit.client.0.smithi142.stdout:8/550: getdents d5/d1d/d59 0 2024-03-29T04:45:01.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:00 smithi152 ceph-mon[37177]: pgmap v266: 65 pgs: 65 active+clean; 828 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 9.6 MiB/s rd, 89 MiB/s wr, 258 op/s 2024-03-29T04:45:01.040 INFO:tasks.workunit.client.0.smithi142.stdout:3/398: write d3/d29/d2b/d40/f4a [887540,13318] 0 2024-03-29T04:45:01.040 INFO:tasks.workunit.client.0.smithi142.stdout:8/551: mkdir d5/d1d/d2f/d45/dc3 0 2024-03-29T04:45:01.046 INFO:tasks.workunit.client.0.smithi142.stdout:4/482: truncate d0/d12/d28/d47/f6b 808284 0 2024-03-29T04:45:01.048 INFO:tasks.workunit.client.0.smithi142.stdout:4/483: dread - d0/d12/f44 zero size 2024-03-29T04:45:01.048 INFO:tasks.workunit.client.0.smithi142.stdout:3/399: dwrite d3/d29/d2b/d40/f4c [4194304,4194304] 0 2024-03-29T04:45:01.050 INFO:tasks.workunit.client.0.smithi142.stdout:2/413: dwrite de/d11/d1e/d22/f26 [0,4194304] 0 2024-03-29T04:45:01.052 INFO:tasks.workunit.client.0.smithi142.stdout:4/484: stat d0/d9/d11/d5c/d8d/da5 0 2024-03-29T04:45:01.053 INFO:tasks.workunit.client.0.smithi142.stdout:3/400: read - d3/d5d/d5f/d64/f76 zero size 2024-03-29T04:45:01.053 INFO:tasks.workunit.client.0.smithi142.stdout:3/401: stat d3 0 2024-03-29T04:45:01.055 INFO:tasks.workunit.client.0.smithi142.stdout:2/414: mknod de/d11/d17/d60/c8d 0 2024-03-29T04:45:01.055 INFO:tasks.workunit.client.0.smithi142.stdout:8/552: dwrite d5/d1d/d2f/fa7 [0,4194304] 0 2024-03-29T04:45:01.056 INFO:tasks.workunit.client.0.smithi142.stdout:3/402: dread - d3/d29/d2b/d40/d67/d8b/f72 zero size 2024-03-29T04:45:01.057 INFO:tasks.workunit.client.0.smithi142.stdout:4/485: creat d0/d12/d28/faa x:0 0 0 2024-03-29T04:45:01.060 INFO:tasks.workunit.client.0.smithi142.stdout:8/553: symlink d5/d1d/d2f/d45/dc3/lc4 0 2024-03-29T04:45:01.065 INFO:tasks.workunit.client.0.smithi142.stdout:2/415: link de/d11/d23/f46 de/d11/d1e/d22/f8e 0 2024-03-29T04:45:01.066 INFO:tasks.workunit.client.0.smithi142.stdout:4/486: creat d0/d9/d11/d4c/d64/d94/fab x:0 0 0 2024-03-29T04:45:01.067 INFO:tasks.workunit.client.0.smithi142.stdout:8/554: creat d5/fc5 x:0 0 0 2024-03-29T04:45:01.068 INFO:tasks.workunit.client.0.smithi142.stdout:2/416: creat de/d11/d1e/d4d/d61/f8f x:0 0 0 2024-03-29T04:45:01.069 INFO:tasks.workunit.client.0.smithi142.stdout:4/487: mknod d0/d9/d11/d4c/d5f/d81/cac 0 2024-03-29T04:45:01.071 INFO:tasks.workunit.client.0.smithi142.stdout:8/555: mknod d5/d1d/d59/cc6 0 2024-03-29T04:45:01.071 INFO:tasks.workunit.client.0.smithi142.stdout:4/488: write d0/d9/d11/d4c/d5f/f73 [4204848,85448] 0 2024-03-29T04:45:01.072 INFO:tasks.workunit.client.0.smithi142.stdout:8/556: stat d5/d1d/d2f/d76/d4c/cac 0 2024-03-29T04:45:01.073 INFO:tasks.workunit.client.0.smithi142.stdout:2/417: rename de/d11/d1e/f5b to de/d11/d23/f90 0 2024-03-29T04:45:01.075 INFO:tasks.workunit.client.0.smithi142.stdout:4/489: creat d0/d45/fad x:0 0 0 2024-03-29T04:45:01.076 INFO:tasks.workunit.client.0.smithi142.stdout:8/557: mknod d5/d1d/d24/db4/cc7 0 2024-03-29T04:45:01.078 INFO:tasks.workunit.client.0.smithi142.stdout:2/418: getdents de/d11/d17/d4a 0 2024-03-29T04:45:01.080 INFO:tasks.workunit.client.0.smithi142.stdout:3/403: dwrite d3/d29/d2b/d40/d67/d8b/d6/f20 [0,4194304] 0 2024-03-29T04:45:01.085 INFO:tasks.workunit.client.0.smithi142.stdout:8/558: dwrite d5/f94 [0,4194304] 0 2024-03-29T04:45:01.087 INFO:tasks.workunit.client.0.smithi142.stdout:3/404: mknod d3/d54/c8e 0 2024-03-29T04:45:01.091 INFO:tasks.workunit.client.0.smithi142.stdout:2/419: truncate de/d11/d1e/f83 763870 0 2024-03-29T04:45:01.091 INFO:tasks.workunit.client.0.smithi142.stdout:8/559: creat d5/d1d/d2f/fc8 x:0 0 0 2024-03-29T04:45:01.092 INFO:tasks.workunit.client.0.smithi142.stdout:2/420: chown de/d11/c6a 145053 1 2024-03-29T04:45:01.093 INFO:tasks.workunit.client.0.smithi142.stdout:8/560: write d5/d1d/d2f/d5f/d48/f8e [203639,105768] 0 2024-03-29T04:45:01.094 INFO:tasks.workunit.client.0.smithi142.stdout:3/405: getdents d3/d5d/d5f/d64 0 2024-03-29T04:45:01.096 INFO:tasks.workunit.client.0.smithi142.stdout:2/421: symlink de/d11/d1e/d22/d27/d3a/l91 0 2024-03-29T04:45:01.098 INFO:tasks.workunit.client.0.smithi142.stdout:3/406: dread d3/d29/d2b/d40/d67/f7f [0,4194304] 0 2024-03-29T04:45:01.101 INFO:tasks.workunit.client.0.smithi142.stdout:3/407: rename d3/d5d to d3/d29/d8f 0 2024-03-29T04:45:01.101 INFO:tasks.workunit.client.0.smithi142.stdout:3/408: chown d3/d54/l81 814894719 1 2024-03-29T04:45:01.103 INFO:tasks.workunit.client.0.smithi142.stdout:3/409: fdatasync d3/d29/d2b/d40/d67/d8b/d6/f20 0 2024-03-29T04:45:01.103 INFO:tasks.workunit.client.0.smithi142.stdout:3/410: fdatasync d3/d1f/d24/d47/f4f 0 2024-03-29T04:45:01.105 INFO:tasks.workunit.client.0.smithi142.stdout:3/411: creat d3/d1f/d24/d47/f90 x:0 0 0 2024-03-29T04:45:01.107 INFO:tasks.workunit.client.0.smithi142.stdout:3/412: truncate d3/d1f/d59/f6e 960666 0 2024-03-29T04:45:01.110 INFO:tasks.workunit.client.0.smithi142.stdout:2/422: dwrite f7 [0,4194304] 0 2024-03-29T04:45:01.111 INFO:tasks.workunit.client.0.smithi142.stdout:2/423: truncate de/d11/d1e/f72 830985 0 2024-03-29T04:45:01.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:00 smithi142 ceph-mon[29027]: pgmap v266: 65 pgs: 65 active+clean; 828 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 9.6 MiB/s rd, 89 MiB/s wr, 258 op/s 2024-03-29T04:45:01.122 INFO:tasks.workunit.client.0.smithi142.stdout:2/424: dread de/d11/d2b/d3e/f76 [0,4194304] 0 2024-03-29T04:45:01.123 INFO:tasks.workunit.client.0.smithi142.stdout:2/425: readlink de/d11/d17/d60/l63 0 2024-03-29T04:45:01.124 INFO:tasks.workunit.client.0.smithi142.stdout:2/426: rmdir de/d11/d23 39 2024-03-29T04:45:01.126 INFO:tasks.workunit.client.0.smithi142.stdout:3/413: dread d3/d1f/f46 [0,4194304] 0 2024-03-29T04:45:01.128 INFO:tasks.workunit.client.0.smithi142.stdout:2/427: link de/d11/d1e/c5e de/d11/d17/d5d/d71/c92 0 2024-03-29T04:45:01.129 INFO:tasks.workunit.client.0.smithi142.stdout:0/458: sync 2024-03-29T04:45:01.130 INFO:tasks.workunit.client.0.smithi142.stdout:2/428: mkdir de/d11/d17/d60/d6f/d93 0 2024-03-29T04:45:01.130 INFO:tasks.workunit.client.0.smithi142.stdout:0/459: write d7/d9/d55/d42/f74 [250218,51500] 0 2024-03-29T04:45:01.133 INFO:tasks.workunit.client.0.smithi142.stdout:2/429: rmdir de/d11/d1e/d4d/d61/d7d 39 2024-03-29T04:45:01.134 INFO:tasks.workunit.client.0.smithi142.stdout:0/460: unlink d7/d9/f30 0 2024-03-29T04:45:01.136 INFO:tasks.workunit.client.0.smithi142.stdout:0/461: unlink d7/d9/d1e/l44 0 2024-03-29T04:45:01.137 INFO:tasks.workunit.client.0.smithi142.stdout:0/462: stat d7/d9/d55/d42 0 2024-03-29T04:45:01.139 INFO:tasks.workunit.client.0.smithi142.stdout:2/430: dwrite de/d11/d17/f25 [4194304,4194304] 0 2024-03-29T04:45:01.146 INFO:tasks.workunit.client.0.smithi142.stdout:2/431: write de/d11/d1e/d22/f8e [419983,5095] 0 2024-03-29T04:45:01.149 INFO:tasks.workunit.client.0.smithi142.stdout:0/463: truncate d7/d9/f49 3376790 0 2024-03-29T04:45:01.150 INFO:tasks.workunit.client.0.smithi142.stdout:0/464: truncate d7/d9/d55/f57 425419 0 2024-03-29T04:45:01.153 INFO:tasks.workunit.client.0.smithi142.stdout:2/432: dread de/d11/d1e/d22/d27/f66 [0,4194304] 0 2024-03-29T04:45:01.181 INFO:tasks.workunit.client.0.smithi142.stdout:2/433: symlink de/d11/d2b/l94 0 2024-03-29T04:45:01.184 INFO:tasks.workunit.client.0.smithi142.stdout:2/434: creat de/d11/d17/d60/d6f/f95 x:0 0 0 2024-03-29T04:45:01.189 INFO:tasks.workunit.client.0.smithi142.stdout:2/435: dread - de/d11/d1e/d4d/d61/d7d/f7e zero size 2024-03-29T04:45:01.191 INFO:tasks.workunit.client.0.smithi142.stdout:2/436: rmdir de/d11/d17/d5d 39 2024-03-29T04:45:01.198 INFO:tasks.workunit.client.0.smithi142.stdout:2/437: dwrite de/f8b [0,4194304] 0 2024-03-29T04:45:01.200 INFO:tasks.workunit.client.0.smithi142.stdout:2/438: truncate de/d11/d1e/d4d/d61/f8f 436806 0 2024-03-29T04:45:01.201 INFO:tasks.workunit.client.0.smithi142.stdout:2/439: read de/d11/f58 [2177392,114002] 0 2024-03-29T04:45:01.221 INFO:tasks.workunit.client.0.smithi142.stdout:0/465: truncate d7/d9/d54/f65 3207484 0 2024-03-29T04:45:01.222 INFO:tasks.workunit.client.0.smithi142.stdout:7/543: sync 2024-03-29T04:45:01.222 INFO:tasks.workunit.client.0.smithi142.stdout:0/466: creat d7/d13/d67/f8e x:0 0 0 2024-03-29T04:45:01.225 INFO:tasks.workunit.client.0.smithi142.stdout:7/544: mkdir d7/d27/d44/db2 0 2024-03-29T04:45:01.228 INFO:tasks.workunit.client.0.smithi142.stdout:7/545: mkdir d7/dc/d4b/d71/d30/d72/db3 0 2024-03-29T04:45:01.228 INFO:tasks.workunit.client.0.smithi142.stdout:0/467: dwrite d7/d13/f78 [0,4194304] 0 2024-03-29T04:45:01.269 INFO:tasks.workunit.client.0.smithi142.stdout:7/546: symlink d7/dc/d4b/lb4 0 2024-03-29T04:45:01.269 INFO:tasks.workunit.client.0.smithi142.stdout:2/440: mkdir de/d11/d96 0 2024-03-29T04:45:01.272 INFO:tasks.workunit.client.0.smithi142.stdout:2/441: symlink de/d11/d17/d60/d6f/l97 0 2024-03-29T04:45:01.272 INFO:tasks.workunit.client.0.smithi142.stdout:0/468: creat d7/d9/d55/f8f x:0 0 0 2024-03-29T04:45:01.276 INFO:tasks.workunit.client.0.smithi142.stdout:0/469: mkdir d7/d13/d4a/d70/d90 0 2024-03-29T04:45:01.277 INFO:tasks.workunit.client.0.smithi142.stdout:7/547: dwrite d7/dc/d4b/d71/d30/d33/f38 [0,4194304] 0 2024-03-29T04:45:01.278 INFO:tasks.workunit.client.0.smithi142.stdout:0/470: chown d7/d13/d67 8355676 1 2024-03-29T04:45:01.280 INFO:tasks.workunit.client.0.smithi142.stdout:2/442: dwrite de/d11/d1e/d22/d27/f4b [4194304,4194304] 0 2024-03-29T04:45:01.280 INFO:tasks.workunit.client.0.smithi142.stdout:7/548: rename d7/d27/c8b to d7/dc/d4b/d9b/cb5 0 2024-03-29T04:45:01.282 INFO:tasks.workunit.client.0.smithi142.stdout:0/471: symlink d7/d13/d67/d72/d75/l91 0 2024-03-29T04:45:01.286 INFO:tasks.workunit.client.0.smithi142.stdout:2/443: creat de/d11/d1e/d84/f98 x:0 0 0 2024-03-29T04:45:01.288 INFO:tasks.workunit.client.0.smithi142.stdout:0/472: truncate d7/d13/d67/d72/d37/f43 1592861 0 2024-03-29T04:45:01.293 INFO:tasks.workunit.client.0.smithi142.stdout:0/473: rmdir d7/d9 39 2024-03-29T04:45:01.294 INFO:tasks.workunit.client.0.smithi142.stdout:7/549: chown d7/dc/d4b/d9b/cb5 49312173 1 2024-03-29T04:45:01.295 INFO:tasks.workunit.client.0.smithi142.stdout:0/474: creat d7/d13/d67/f92 x:0 0 0 2024-03-29T04:45:01.295 INFO:tasks.workunit.client.0.smithi142.stdout:2/444: getdents de/d11/d1e 0 2024-03-29T04:45:01.296 INFO:tasks.workunit.client.0.smithi142.stdout:7/550: mkdir d7/d27/db6 0 2024-03-29T04:45:01.297 INFO:tasks.workunit.client.0.smithi142.stdout:0/475: mknod d7/d13/d67/d72/d75/c93 0 2024-03-29T04:45:01.297 INFO:tasks.workunit.client.0.smithi142.stdout:2/445: symlink de/d11/d1e/d4d/d61/d7d/l99 0 2024-03-29T04:45:01.299 INFO:tasks.workunit.client.0.smithi142.stdout:2/446: creat de/d11/d2b/d3e/d85/f9a x:0 0 0 2024-03-29T04:45:01.300 INFO:tasks.workunit.client.0.smithi142.stdout:0/476: creat d7/d9/f94 x:0 0 0 2024-03-29T04:45:01.301 INFO:tasks.workunit.client.0.smithi142.stdout:2/447: chown de/d11/d23/c34 6264 1 2024-03-29T04:45:01.301 INFO:tasks.workunit.client.0.smithi142.stdout:2/448: dread - de/d11/d1e/d84/f98 zero size 2024-03-29T04:45:01.304 INFO:tasks.workunit.client.0.smithi142.stdout:2/449: write de/d11/d2b/d3e/d85/f9a [381231,26330] 0 2024-03-29T04:45:01.306 INFO:tasks.workunit.client.0.smithi142.stdout:0/477: dwrite d7/d13/d67/d72/f3e [4194304,4194304] 0 2024-03-29T04:45:01.307 INFO:tasks.workunit.client.0.smithi142.stdout:2/450: unlink de/d11/d2b/c40 0 2024-03-29T04:45:01.308 INFO:tasks.workunit.client.0.smithi142.stdout:7/551: dread d7/d27/fa8 [0,4194304] 0 2024-03-29T04:45:01.308 INFO:tasks.workunit.client.0.smithi142.stdout:7/552: fsync d7/d27/f91 0 2024-03-29T04:45:01.309 INFO:tasks.workunit.client.0.smithi142.stdout:7/553: readlink d7/dc/la4 0 2024-03-29T04:45:01.311 INFO:tasks.workunit.client.0.smithi142.stdout:7/554: mknod d7/dc/d4b/d9b/cb7 0 2024-03-29T04:45:01.313 INFO:tasks.workunit.client.0.smithi142.stdout:0/478: creat d7/d9/d6e/f95 x:0 0 0 2024-03-29T04:45:01.317 INFO:tasks.workunit.client.0.smithi142.stdout:9/366: sync 2024-03-29T04:45:01.318 INFO:tasks.workunit.client.0.smithi142.stdout:1/485: sync 2024-03-29T04:45:01.319 INFO:tasks.workunit.client.0.smithi142.stdout:2/451: rmdir de/d11/d17/d60/d6f/d93 0 2024-03-29T04:45:01.321 INFO:tasks.workunit.client.0.smithi142.stdout:0/479: mkdir d7/d9/d55/d96 0 2024-03-29T04:45:01.321 INFO:tasks.workunit.client.0.smithi142.stdout:0/480: dread - d7/d9/f94 zero size 2024-03-29T04:45:01.322 INFO:tasks.workunit.client.0.smithi142.stdout:9/367: mkdir d0/d2/d6/d27/d3c/d6f 0 2024-03-29T04:45:01.323 INFO:tasks.workunit.client.0.smithi142.stdout:1/486: mknod d8/db/dc/d19/d1d/d2d/d44/d53/cae 0 2024-03-29T04:45:01.324 INFO:tasks.workunit.client.0.smithi142.stdout:1/487: fdatasync d8/db/d37/f41 0 2024-03-29T04:45:01.325 INFO:tasks.workunit.client.0.smithi142.stdout:0/481: creat d7/d9/d55/d96/f97 x:0 0 0 2024-03-29T04:45:01.326 INFO:tasks.workunit.client.0.smithi142.stdout:7/555: rename d7/dc to d7/d27/d44/db2/db8 0 2024-03-29T04:45:01.328 INFO:tasks.workunit.client.0.smithi142.stdout:0/482: read d7/d13/f28 [2903833,127474] 0 2024-03-29T04:45:01.328 INFO:tasks.workunit.client.0.smithi142.stdout:1/488: creat d8/db/dc/d77/d31/d4d/d56/faf x:0 0 0 2024-03-29T04:45:01.330 INFO:tasks.workunit.client.0.smithi142.stdout:2/452: rename de/d11/d1e/d4d/d61/d7d/l99 to de/d11/d1e/l9b 0 2024-03-29T04:45:01.331 INFO:tasks.workunit.client.0.smithi142.stdout:0/483: stat d7/d13/d4a/d70/d90 0 2024-03-29T04:45:01.332 INFO:tasks.workunit.client.0.smithi142.stdout:9/368: getdents d0/d2 0 2024-03-29T04:45:01.333 INFO:tasks.workunit.client.0.smithi142.stdout:2/453: chown de/d11/d17/d5d 5444755 1 2024-03-29T04:45:01.334 INFO:tasks.workunit.client.0.smithi142.stdout:1/489: creat d8/db/dc/d19/d1d/d2d/d44/d7f/fb0 x:0 0 0 2024-03-29T04:45:01.336 INFO:tasks.workunit.client.0.smithi142.stdout:7/556: getdents d7/d27/d44/db2/db8/d4b 0 2024-03-29T04:45:01.336 INFO:tasks.workunit.client.0.smithi142.stdout:1/490: dread - d8/db/dc/d19/d1d/d1f/d97/d9e/fa7 zero size 2024-03-29T04:45:01.339 INFO:tasks.workunit.client.0.smithi142.stdout:0/484: rename d7/d9/f49 to d7/d9/d55/f98 0 2024-03-29T04:45:01.341 INFO:tasks.workunit.client.0.smithi142.stdout:7/557: creat d7/d27/d44/db2/db8/de/d8d/fb9 x:0 0 0 2024-03-29T04:45:01.343 INFO:tasks.workunit.client.0.smithi142.stdout:1/491: dwrite d8/db/dc/d77/d31/d4d/d56/faf [0,4194304] 0 2024-03-29T04:45:01.344 INFO:tasks.workunit.client.0.smithi142.stdout:0/485: creat d7/d9/d54/f99 x:0 0 0 2024-03-29T04:45:01.349 INFO:tasks.workunit.client.0.smithi142.stdout:0/486: mkdir d7/d13/d4a/d9a 0 2024-03-29T04:45:01.349 INFO:tasks.workunit.client.0.smithi142.stdout:2/454: symlink de/d11/d1e/d22/d50/l9c 0 2024-03-29T04:45:01.350 INFO:tasks.workunit.client.0.smithi142.stdout:7/558: link d7/d27/d44/db2/db8/de/d8d/caa d7/d27/d44/db2/db8/d4b/d71/d30/d72/db3/cba 0 2024-03-29T04:45:01.352 INFO:tasks.workunit.client.0.smithi142.stdout:2/455: write de/d11/d23/f46 [1034284,92222] 0 2024-03-29T04:45:01.352 INFO:tasks.workunit.client.0.smithi142.stdout:0/487: symlink d7/d9/d1e/l9b 0 2024-03-29T04:45:01.353 INFO:tasks.workunit.client.0.smithi142.stdout:1/492: dwrite d8/db/dc/d19/d1d/d15/f16 [4194304,4194304] 0 2024-03-29T04:45:01.354 INFO:tasks.workunit.client.0.smithi142.stdout:2/456: dread - de/d11/d23/f90 zero size 2024-03-29T04:45:01.354 INFO:tasks.workunit.client.0.smithi142.stdout:0/488: dread - d7/d9/d55/d96/f97 zero size 2024-03-29T04:45:01.359 INFO:tasks.workunit.client.0.smithi142.stdout:0/489: creat d7/d13/f9c x:0 0 0 2024-03-29T04:45:01.360 INFO:tasks.workunit.client.0.smithi142.stdout:1/493: creat d8/db/dc/d19/d1d/d1f/d4c/fb1 x:0 0 0 2024-03-29T04:45:01.362 INFO:tasks.workunit.client.0.smithi142.stdout:1/494: chown d8/db/dc/d19/d1d/d2d/d8b/cab 7227937 1 2024-03-29T04:45:01.363 INFO:tasks.workunit.client.0.smithi142.stdout:1/495: write d8/db/dc/d19/d1d/d2d/d44/d80/f88 [892509,36134] 0 2024-03-29T04:45:01.368 INFO:tasks.workunit.client.0.smithi142.stdout:1/496: creat d8/db/dc/d19/d1d/d2d/d44/d4b/fb2 x:0 0 0 2024-03-29T04:45:01.369 INFO:tasks.workunit.client.0.smithi142.stdout:1/497: write d8/db/d37/f41 [3441778,89050] 0 2024-03-29T04:45:01.371 INFO:tasks.workunit.client.0.smithi142.stdout:1/498: write f3 [194195,36499] 0 2024-03-29T04:45:01.375 INFO:tasks.workunit.client.0.smithi142.stdout:1/499: rename f7 to d8/db/fb3 0 2024-03-29T04:45:01.376 INFO:tasks.workunit.client.0.smithi142.stdout:0/490: dwrite d7/d13/d67/d72/d37/f43 [0,4194304] 0 2024-03-29T04:45:01.377 INFO:tasks.workunit.client.0.smithi142.stdout:2/457: dread de/f16 [0,4194304] 0 2024-03-29T04:45:01.377 INFO:tasks.workunit.client.0.smithi142.stdout:1/500: chown d8/db/dc/d77/c2e 2649 1 2024-03-29T04:45:01.379 INFO:tasks.workunit.client.0.smithi142.stdout:0/491: mknod d7/d13/d4a/d70/c9d 0 2024-03-29T04:45:01.380 INFO:tasks.workunit.client.0.smithi142.stdout:0/492: fsync d7/d13/f9c 0 2024-03-29T04:45:01.381 INFO:tasks.workunit.client.0.smithi142.stdout:7/559: dread d7/fa [0,4194304] 0 2024-03-29T04:45:01.381 INFO:tasks.workunit.client.0.smithi142.stdout:1/501: mkdir d8/db/dc/d19/d1d/d2d/d44/d4b/db4 0 2024-03-29T04:45:01.383 INFO:tasks.workunit.client.0.smithi142.stdout:1/502: dread d8/db/dc/d77/f7c [0,4194304] 0 2024-03-29T04:45:01.385 INFO:tasks.workunit.client.0.smithi142.stdout:7/560: mknod d7/d27/d44/db2/db8/d4b/d71/cbb 0 2024-03-29T04:45:01.387 INFO:tasks.workunit.client.0.smithi142.stdout:2/458: rename de/d11/f58 to de/f9d 0 2024-03-29T04:45:01.387 INFO:tasks.workunit.client.0.smithi142.stdout:0/493: dwrite d7/d13/d67/d72/f3a [0,4194304] 0 2024-03-29T04:45:01.387 INFO:tasks.workunit.client.0.smithi142.stdout:7/561: write d7/d27/d44/db2/db8/d4b/f4f [299548,46147] 0 2024-03-29T04:45:01.389 INFO:tasks.workunit.client.0.smithi142.stdout:1/503: mknod d8/db/dc/d19/d1d/d2d/d44/d53/cb5 0 2024-03-29T04:45:01.390 INFO:tasks.workunit.client.0.smithi142.stdout:5/605: sync 2024-03-29T04:45:01.390 INFO:tasks.workunit.client.0.smithi142.stdout:4/490: sync 2024-03-29T04:45:01.390 INFO:tasks.workunit.client.0.smithi142.stdout:6/472: sync 2024-03-29T04:45:01.393 INFO:tasks.workunit.client.0.smithi142.stdout:0/494: write d7/d9/d55/d42/f5f [938849,35865] 0 2024-03-29T04:45:01.394 INFO:tasks.workunit.client.0.smithi142.stdout:6/473: dread d1c/f41 [0,4194304] 0 2024-03-29T04:45:01.394 INFO:tasks.workunit.client.0.smithi142.stdout:4/491: write d0/dc/f88 [938324,36413] 0 2024-03-29T04:45:01.395 INFO:tasks.workunit.client.0.smithi142.stdout:7/562: dread - d7/d27/d44/db2/db8/de/f2d zero size 2024-03-29T04:45:01.398 INFO:tasks.workunit.client.0.smithi142.stdout:6/474: symlink d1c/l91 0 2024-03-29T04:45:01.398 INFO:tasks.workunit.client.0.smithi142.stdout:7/563: fdatasync d7/d27/d44/db2/db8/de/d8d/fa7 0 2024-03-29T04:45:01.399 INFO:tasks.workunit.client.0.smithi142.stdout:4/492: symlink d0/lae 0 2024-03-29T04:45:01.401 INFO:tasks.workunit.client.0.smithi142.stdout:5/606: getdents df/d16/d24/d2e/d47 0 2024-03-29T04:45:01.401 INFO:tasks.workunit.client.0.smithi142.stdout:2/459: dwrite de/d11/d1e/d4d/d61/f8f [0,4194304] 0 2024-03-29T04:45:01.402 INFO:tasks.workunit.client.0.smithi142.stdout:7/564: chown d7/d27/d44/f60 72999976 1 2024-03-29T04:45:01.406 INFO:tasks.workunit.client.0.smithi142.stdout:6/475: truncate d1c/d33/d46/d71/f60 572267 0 2024-03-29T04:45:01.407 INFO:tasks.workunit.client.0.smithi142.stdout:7/565: read d7/d27/d44/f60 [739745,112631] 0 2024-03-29T04:45:01.407 INFO:tasks.workunit.client.0.smithi142.stdout:5/607: truncate df/d16/d24/d2e/f8b 700740 0 2024-03-29T04:45:01.410 INFO:tasks.workunit.client.0.smithi142.stdout:0/495: chown d7/d13/d4a/d70/c84 506687851 1 2024-03-29T04:45:01.411 INFO:tasks.workunit.client.0.smithi142.stdout:5/608: fsync df/d1a/d21/d27/d45/fa9 0 2024-03-29T04:45:01.412 INFO:tasks.workunit.client.0.smithi142.stdout:4/493: dwrite d0/d12/f44 [0,4194304] 0 2024-03-29T04:45:01.415 INFO:tasks.workunit.client.0.smithi142.stdout:7/566: symlink d7/lbc 0 2024-03-29T04:45:01.418 INFO:tasks.workunit.client.0.smithi142.stdout:2/460: dwrite de/d11/d1e/d22/f3f [0,4194304] 0 2024-03-29T04:45:01.418 INFO:tasks.workunit.client.0.smithi142.stdout:5/609: creat df/d29/d7a/fc4 x:0 0 0 2024-03-29T04:45:01.418 INFO:tasks.workunit.client.0.smithi142.stdout:1/504: dread f3 [0,4194304] 0 2024-03-29T04:45:01.421 INFO:tasks.workunit.client.0.smithi142.stdout:5/610: readlink df/d16/d24/l4f 0 2024-03-29T04:45:01.422 INFO:tasks.workunit.client.0.smithi142.stdout:7/567: write d7/d27/d44/db2/db8/de/d20/d3f/f65 [816560,25994] 0 2024-03-29T04:45:01.425 INFO:tasks.workunit.client.0.smithi142.stdout:0/496: rename d7/d13/d67/d72/c69 to d7/d9/d6e/c9e 0 2024-03-29T04:45:01.426 INFO:tasks.workunit.client.0.smithi142.stdout:3/414: sync 2024-03-29T04:45:01.426 INFO:tasks.workunit.client.0.smithi142.stdout:8/561: sync 2024-03-29T04:45:01.426 INFO:tasks.workunit.client.0.smithi142.stdout:9/369: sync 2024-03-29T04:45:01.429 INFO:tasks.workunit.client.0.smithi142.stdout:7/568: truncate d7/d27/d44/db2/db8/d4b/d71/d30/fb1 720103 0 2024-03-29T04:45:01.430 INFO:tasks.workunit.client.0.smithi142.stdout:1/505: dread d8/db/dc/d77/f7c [0,4194304] 0 2024-03-29T04:45:01.430 INFO:tasks.workunit.client.0.smithi142.stdout:0/497: truncate d7/d9/f94 368068 0 2024-03-29T04:45:01.430 INFO:tasks.workunit.client.0.smithi142.stdout:2/461: dwrite de/d11/d1e/d22/d27/f4b [4194304,4194304] 0 2024-03-29T04:45:01.430 INFO:tasks.workunit.client.0.smithi142.stdout:8/562: creat d5/d93/fc9 x:0 0 0 2024-03-29T04:45:01.432 INFO:tasks.workunit.client.0.smithi142.stdout:6/476: truncate fc 3163349 0 2024-03-29T04:45:01.433 INFO:tasks.workunit.client.0.smithi142.stdout:3/415: write d3/d54/f8d [7428,119785] 0 2024-03-29T04:45:01.434 INFO:tasks.workunit.client.0.smithi142.stdout:5/611: mknod df/d16/d24/d2e/cc5 0 2024-03-29T04:45:01.434 INFO:tasks.workunit.client.0.smithi142.stdout:2/462: write de/d11/d3d/f65 [588312,21374] 0 2024-03-29T04:45:01.434 INFO:tasks.workunit.client.0.smithi142.stdout:0/498: write d7/d9/d55/f87 [2964651,10194] 0 2024-03-29T04:45:01.435 INFO:tasks.workunit.client.0.smithi142.stdout:4/494: truncate d0/d12/f5d 3240681 0 2024-03-29T04:45:01.437 INFO:tasks.workunit.client.0.smithi142.stdout:7/569: creat d7/d27/d44/db2/db8/d4b/d9b/fbd x:0 0 0 2024-03-29T04:45:01.438 INFO:tasks.workunit.client.0.smithi142.stdout:1/506: symlink d8/db/dc/d19/d1d/d2d/d44/d53/lb6 0 2024-03-29T04:45:01.440 INFO:tasks.workunit.client.0.smithi142.stdout:3/416: readlink d3/d29/d2b/d40/d67/d8b/d6/lb 0 2024-03-29T04:45:01.440 INFO:tasks.workunit.client.0.smithi142.stdout:8/563: chown d5/d1d/d2f/l31 27856598 1 2024-03-29T04:45:01.443 INFO:tasks.workunit.client.0.smithi142.stdout:3/417: fdatasync d3/d1f/d59/f6e 0 2024-03-29T04:45:01.443 INFO:tasks.workunit.client.0.smithi142.stdout:4/495: mkdir d0/d9/d11/d5c/daf 0 2024-03-29T04:45:01.445 INFO:tasks.workunit.client.0.smithi142.stdout:0/499: write d7/d9/d54/d58/f82 [1238213,16176] 0 2024-03-29T04:45:01.446 INFO:tasks.workunit.client.0.smithi142.stdout:5/612: mknod df/d29/dbc/cc6 0 2024-03-29T04:45:01.447 INFO:tasks.workunit.client.0.smithi142.stdout:6/477: dwrite f1a [0,4194304] 0 2024-03-29T04:45:01.447 INFO:tasks.workunit.client.0.smithi142.stdout:3/418: dread - d3/d29/d2b/d40/d67/d8b/f72 zero size 2024-03-29T04:45:01.450 INFO:tasks.workunit.client.0.smithi142.stdout:1/507: stat d8/db/dc/d19/d1d/d1f/d4c/f81 0 2024-03-29T04:45:01.454 INFO:tasks.workunit.client.0.smithi142.stdout:7/570: rename d7/d27/d44/db2/db8/de/d20/d3f/ca9 to d7/d27/d44/db2/db8/d89/cbe 0 2024-03-29T04:45:01.455 INFO:tasks.workunit.client.0.smithi142.stdout:1/508: write d8/db/dc/d19/d1d/d2d/f3e [997019,7972] 0 2024-03-29T04:45:01.457 INFO:tasks.workunit.client.0.smithi142.stdout:4/496: rmdir d0/d9/d11/d97/d9e 39 2024-03-29T04:45:01.459 INFO:tasks.workunit.client.0.smithi142.stdout:8/564: dwrite d5/d1d/d2f/d76/d2d/f4f [0,4194304] 0 2024-03-29T04:45:01.460 INFO:tasks.workunit.client.0.smithi142.stdout:4/497: write d0/dc/d15/d3d/f69 [959,42117] 0 2024-03-29T04:45:01.462 INFO:tasks.workunit.client.0.smithi142.stdout:5/613: symlink df/d1a/d21/d27/d45/dbf/lc7 0 2024-03-29T04:45:01.463 INFO:tasks.workunit.client.0.smithi142.stdout:4/498: write d0/d12/d28/d47/f62 [3166540,80882] 0 2024-03-29T04:45:01.466 INFO:tasks.workunit.client.0.smithi142.stdout:7/571: fsync d7/d27/d44/db2/db8/f10 0 2024-03-29T04:45:01.467 INFO:tasks.workunit.client.0.smithi142.stdout:1/509: truncate d8/db/f25 118546 0 2024-03-29T04:45:01.468 INFO:tasks.workunit.client.0.smithi142.stdout:5/614: write df/d16/d24/fa0 [3813374,33290] 0 2024-03-29T04:45:01.471 INFO:tasks.workunit.client.0.smithi142.stdout:2/463: truncate de/d11/d1e/d22/f26 3149438 0 2024-03-29T04:45:01.473 INFO:tasks.workunit.client.0.smithi142.stdout:9/370: dwrite d0/d2/d6/d31/f3a [4194304,4194304] 0 2024-03-29T04:45:01.473 INFO:tasks.workunit.client.0.smithi142.stdout:7/572: stat d7/d27/d44/db2/db8/d4b/d71/d30/d33/f4d 0 2024-03-29T04:45:01.474 INFO:tasks.workunit.client.0.smithi142.stdout:0/500: dwrite d7/d9/d55/f3d [0,4194304] 0 2024-03-29T04:45:01.476 INFO:tasks.workunit.client.0.smithi142.stdout:1/510: creat d8/db/dc/d19/d1d/d2d/d44/d53/fb7 x:0 0 0 2024-03-29T04:45:01.480 INFO:tasks.workunit.client.0.smithi142.stdout:3/419: rename d3/d29/d2b/d40/f7c to d3/d29/d2b/d40/d67/d8b/f91 0 2024-03-29T04:45:01.485 INFO:tasks.workunit.client.0.smithi142.stdout:1/511: chown d8/db/dc/d19/d1d/l36 2712492 1 2024-03-29T04:45:01.487 INFO:tasks.workunit.client.0.smithi142.stdout:6/478: dwrite d1c/f25 [0,4194304] 0 2024-03-29T04:45:01.489 INFO:tasks.workunit.client.0.smithi142.stdout:4/499: dwrite d0/d9/f4d [0,4194304] 0 2024-03-29T04:45:01.490 INFO:tasks.workunit.client.0.smithi142.stdout:7/573: truncate d7/d27/d3a/d92/d97/faf 545695 0 2024-03-29T04:45:01.491 INFO:tasks.workunit.client.0.smithi142.stdout:5/615: getdents df/d16 0 2024-03-29T04:45:01.492 INFO:tasks.workunit.client.0.smithi142.stdout:3/420: symlink d3/d1f/d24/d47/l92 0 2024-03-29T04:45:01.498 INFO:tasks.workunit.client.0.smithi142.stdout:5/616: write df/d1a/d21/d27/d45/dbf/f9f [529654,116543] 0 2024-03-29T04:45:01.498 INFO:tasks.workunit.client.0.smithi142.stdout:6/479: read d1c/d45/d51/f58 [2702266,114877] 0 2024-03-29T04:45:01.500 INFO:tasks.workunit.client.0.smithi142.stdout:2/464: getdents de/d11/d1e/d22/d27 0 2024-03-29T04:45:01.501 INFO:tasks.workunit.client.0.smithi142.stdout:6/480: write d1c/f3c [835895,104658] 0 2024-03-29T04:45:01.502 INFO:tasks.workunit.client.0.smithi142.stdout:0/501: dwrite d7/d13/d4a/f5e [0,4194304] 0 2024-03-29T04:45:01.502 INFO:tasks.workunit.client.0.smithi142.stdout:4/500: mknod d0/d12/d28/cb0 0 2024-03-29T04:45:01.503 INFO:tasks.workunit.client.0.smithi142.stdout:1/512: dread d8/db/dc/d77/d31/f3f [0,4194304] 0 2024-03-29T04:45:01.504 INFO:tasks.workunit.client.0.smithi142.stdout:7/574: symlink d7/d27/d44/db2/db8/d4b/d9b/lbf 0 2024-03-29T04:45:01.504 INFO:tasks.workunit.client.0.smithi142.stdout:6/481: dread - d1c/f62 zero size 2024-03-29T04:45:01.505 INFO:tasks.workunit.client.0.smithi142.stdout:8/565: dread d5/f18 [4194304,4194304] 0 2024-03-29T04:45:01.506 INFO:tasks.workunit.client.0.smithi142.stdout:9/371: getdents d0/d2/d6/d27/d3c/d4f 0 2024-03-29T04:45:01.510 INFO:tasks.workunit.client.0.smithi142.stdout:0/502: write d7/d13/f9c [444138,24778] 0 2024-03-29T04:45:01.511 INFO:tasks.workunit.client.0.smithi142.stdout:3/421: write d3/d29/d2b/d40/d67/d8b/f91 [139153,88775] 0 2024-03-29T04:45:01.513 INFO:tasks.workunit.client.0.smithi142.stdout:2/465: chown de/d11/c73 29198582 1 2024-03-29T04:45:01.516 INFO:tasks.workunit.client.0.smithi142.stdout:6/482: write d1c/d45/d51/f83 [737749,42284] 0 2024-03-29T04:45:01.519 INFO:tasks.workunit.client.0.smithi142.stdout:1/513: mkdir d8/db/dc/d19/d1d/d2d/d44/d4b/db4/db8 0 2024-03-29T04:45:01.519 INFO:tasks.workunit.client.0.smithi142.stdout:4/501: dread d0/dc/f88 [0,4194304] 0 2024-03-29T04:45:01.521 INFO:tasks.workunit.client.0.smithi142.stdout:5/617: dwrite df/d1a/d21/d55/f81 [0,4194304] 0 2024-03-29T04:45:01.521 INFO:tasks.workunit.client.0.smithi142.stdout:0/503: dread d7/d9/f94 [0,4194304] 0 2024-03-29T04:45:01.523 INFO:tasks.workunit.client.0.smithi142.stdout:6/483: write d1c/f40 [257542,112498] 0 2024-03-29T04:45:01.523 INFO:tasks.workunit.client.0.smithi142.stdout:7/575: truncate d7/d27/d44/db2/db8/d4b/d9b/fbd 310574 0 2024-03-29T04:45:01.525 INFO:tasks.workunit.client.0.smithi142.stdout:2/466: write de/d11/d1e/d22/d27/f66 [124713,56039] 0 2024-03-29T04:45:01.525 INFO:tasks.workunit.client.0.smithi142.stdout:9/372: dwrite d0/d12/f54 [0,4194304] 0 2024-03-29T04:45:01.526 INFO:tasks.workunit.client.0.smithi142.stdout:3/422: mkdir d3/d29/d2b/d40/d67/d8b/d6/d93 0 2024-03-29T04:45:01.527 INFO:tasks.workunit.client.0.smithi142.stdout:0/504: unlink d7/d9/d1e/l7a 0 2024-03-29T04:45:01.531 INFO:tasks.workunit.client.0.smithi142.stdout:3/423: chown d3/d29/d2b/d40/d67/f7f 8736759 1 2024-03-29T04:45:01.532 INFO:tasks.workunit.client.0.smithi142.stdout:4/502: rename d0/dc/d15/d3d/f71 to d0/d9/d11/d78/fb1 0 2024-03-29T04:45:01.532 INFO:tasks.workunit.client.0.smithi142.stdout:6/484: rename d1c/d33/d46/d61 to d1c/d33/d46/d61/d92 22 2024-03-29T04:45:01.534 INFO:tasks.workunit.client.0.smithi142.stdout:2/467: unlink de/d11/d1e/d22/f3f 0 2024-03-29T04:45:01.534 INFO:tasks.workunit.client.0.smithi142.stdout:4/503: write d0/d9/d11/d4c/d5f/f6f [69215,16731] 0 2024-03-29T04:45:01.536 INFO:tasks.workunit.client.0.smithi142.stdout:0/505: write d7/d13/d67/d72/d37/f83 [5071,97140] 0 2024-03-29T04:45:01.536 INFO:tasks.workunit.client.0.smithi142.stdout:2/468: chown de/d11/d17/f69 627986 1 2024-03-29T04:45:01.537 INFO:tasks.workunit.client.0.smithi142.stdout:9/373: mknod d0/d12/d50/d52/c70 0 2024-03-29T04:45:01.537 INFO:tasks.workunit.client.0.smithi142.stdout:2/469: dread - de/d11/d2b/d7b/f8c zero size 2024-03-29T04:45:01.540 INFO:tasks.workunit.client.0.smithi142.stdout:1/514: unlink d8/db/dc/d19/d1d/d2d/d44/l6c 0 2024-03-29T04:45:01.541 INFO:tasks.workunit.client.0.smithi142.stdout:6/485: dread d1c/f34 [0,4194304] 0 2024-03-29T04:45:01.542 INFO:tasks.workunit.client.0.smithi142.stdout:7/576: mkdir d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0 0 2024-03-29T04:45:01.542 INFO:tasks.workunit.client.0.smithi142.stdout:7/577: read - d7/d27/d44/d4c/f7b zero size 2024-03-29T04:45:01.544 INFO:tasks.workunit.client.0.smithi142.stdout:5/618: link df/d29/d5d/d75/d90/fc2 df/d16/d24/d3d/fc8 0 2024-03-29T04:45:01.547 INFO:tasks.workunit.client.0.smithi142.stdout:1/515: mkdir d8/db/dc/d77/db9 0 2024-03-29T04:45:01.549 INFO:tasks.workunit.client.0.smithi142.stdout:6/486: rmdir d1c/d33/d46 39 2024-03-29T04:45:01.550 INFO:tasks.workunit.client.0.smithi142.stdout:7/578: mkdir d7/d27/d44/db2/db8/de/dc1 0 2024-03-29T04:45:01.550 INFO:tasks.workunit.client.0.smithi142.stdout:5/619: dwrite df/d1a/f2f [0,4194304] 0 2024-03-29T04:45:01.551 INFO:tasks.workunit.client.0.smithi142.stdout:4/504: mkdir d0/d9/db2 0 2024-03-29T04:45:01.551 INFO:tasks.workunit.client.0.smithi142.stdout:5/620: chown df/d29/d7a/d97 391495 1 2024-03-29T04:45:01.555 INFO:tasks.workunit.client.0.smithi142.stdout:0/506: unlink d7/d9/cb 0 2024-03-29T04:45:01.557 INFO:tasks.workunit.client.0.smithi142.stdout:0/507: stat d7/d13/d67/d72/c4b 0 2024-03-29T04:45:01.557 INFO:tasks.workunit.client.0.smithi142.stdout:7/579: dwrite d7/d27/d44/db2/db8/d4b/d9b/fbd [0,4194304] 0 2024-03-29T04:45:01.560 INFO:tasks.workunit.client.0.smithi142.stdout:7/580: fdatasync d7/d27/d44/db2/db8/d4b/d71/f1f 0 2024-03-29T04:45:01.561 INFO:tasks.workunit.client.0.smithi142.stdout:7/581: readlink d7/d27/d44/db2/db8/de/d20/l7e 0 2024-03-29T04:45:01.568 INFO:tasks.workunit.client.0.smithi142.stdout:0/508: creat d7/d9/d55/d96/f9f x:0 0 0 2024-03-29T04:45:01.569 INFO:tasks.workunit.client.0.smithi142.stdout:4/505: dwrite d0/d9/d11/d5c/d8d/f9b [0,4194304] 0 2024-03-29T04:45:01.569 INFO:tasks.workunit.client.0.smithi142.stdout:6/487: mknod d1c/d33/d46/c93 0 2024-03-29T04:45:01.569 INFO:tasks.workunit.client.0.smithi142.stdout:1/516: creat d8/db/dc/d19/d1d/d2d/d8b/da4/fba x:0 0 0 2024-03-29T04:45:01.571 INFO:tasks.workunit.client.0.smithi142.stdout:1/517: chown d8/db/dc/d19/d1d/d15/f17 1006685 1 2024-03-29T04:45:01.571 INFO:tasks.workunit.client.0.smithi142.stdout:4/506: chown d0/d9/d11/d97/ca3 7527638 1 2024-03-29T04:45:01.572 INFO:tasks.workunit.client.0.smithi142.stdout:9/374: getdents d0/d2/d6/d31 0 2024-03-29T04:45:01.574 INFO:tasks.workunit.client.0.smithi142.stdout:2/470: rename de/d11/d17/d5d/d71/c92 to de/d11/d1e/c9e 0 2024-03-29T04:45:01.576 INFO:tasks.workunit.client.0.smithi142.stdout:7/582: mkdir d7/d27/d44/d4c/da3/dc2 0 2024-03-29T04:45:01.576 INFO:tasks.workunit.client.0.smithi142.stdout:2/471: read de/d11/d17/f69 [4014579,57680] 0 2024-03-29T04:45:01.577 INFO:tasks.workunit.client.0.smithi142.stdout:1/518: rmdir d8/db/dc/d19/d1d/d1f/d4c 39 2024-03-29T04:45:01.577 INFO:tasks.workunit.client.0.smithi142.stdout:2/472: write de/d11/f4f [2456954,93707] 0 2024-03-29T04:45:01.578 INFO:tasks.workunit.client.0.smithi142.stdout:7/583: rename d7/d27/d44/db2/db8/d89 to d7/d27/d44/db2/db8/d89/dc3 22 2024-03-29T04:45:01.579 INFO:tasks.workunit.client.0.smithi142.stdout:4/507: creat d0/d9/db2/fb3 x:0 0 0 2024-03-29T04:45:01.580 INFO:tasks.workunit.client.0.smithi142.stdout:4/508: read - d0/d45/f61 zero size 2024-03-29T04:45:01.581 INFO:tasks.workunit.client.0.smithi142.stdout:3/424: truncate d3/d29/d2b/d40/f4c 6311284 0 2024-03-29T04:45:01.582 INFO:tasks.workunit.client.0.smithi142.stdout:7/584: mkdir d7/d27/d44/db2/dc4 0 2024-03-29T04:45:01.583 INFO:tasks.workunit.client.0.smithi142.stdout:0/509: link d7/d9/d54/f65 d7/d13/d67/d72/fa0 0 2024-03-29T04:45:01.584 INFO:tasks.workunit.client.0.smithi142.stdout:4/509: write d0/d9/d19/d2c/f56 [1355618,43416] 0 2024-03-29T04:45:01.584 INFO:tasks.workunit.client.0.smithi142.stdout:2/473: dread de/d11/d23/f46 [0,4194304] 0 2024-03-29T04:45:01.586 INFO:tasks.workunit.client.0.smithi142.stdout:5/621: write df/d29/d5d/d75/d90/fc2 [3743,38631] 0 2024-03-29T04:45:01.586 INFO:tasks.workunit.client.0.smithi142.stdout:3/425: symlink d3/d29/d2b/d40/l94 0 2024-03-29T04:45:01.587 INFO:tasks.workunit.client.0.smithi142.stdout:4/510: dread - d0/dc/f93 zero size 2024-03-29T04:45:01.588 INFO:tasks.workunit.client.0.smithi142.stdout:1/519: creat d8/db/dc/d19/d1d/d2d/d44/d4b/db4/db8/fbb x:0 0 0 2024-03-29T04:45:01.588 INFO:tasks.workunit.client.0.smithi142.stdout:0/510: symlink d7/d13/d4a/la1 0 2024-03-29T04:45:01.590 INFO:tasks.workunit.client.0.smithi142.stdout:2/474: rmdir de/d11/d17/d5d 39 2024-03-29T04:45:01.591 INFO:tasks.workunit.client.0.smithi142.stdout:3/426: mkdir d3/d29/d8f/d95 0 2024-03-29T04:45:01.592 INFO:tasks.workunit.client.0.smithi142.stdout:5/622: symlink df/d16/d25/lc9 0 2024-03-29T04:45:01.593 INFO:tasks.workunit.client.0.smithi142.stdout:6/488: dread d1c/d33/d46/d66/f6b [0,4194304] 0 2024-03-29T04:45:01.594 INFO:tasks.workunit.client.0.smithi142.stdout:3/427: dread - d3/d54/f78 zero size 2024-03-29T04:45:01.595 INFO:tasks.workunit.client.0.smithi142.stdout:4/511: truncate d0/d9/d11/d4c/d64/f77 1230605 0 2024-03-29T04:45:01.596 INFO:tasks.workunit.client.0.smithi142.stdout:7/585: truncate d7/d27/d44/db2/db8/d4b/d71/d30/f3e 5209752 0 2024-03-29T04:45:01.599 INFO:tasks.workunit.client.0.smithi142.stdout:3/428: readlink d3/d29/d2d/d56/l73 0 2024-03-29T04:45:01.606 INFO:tasks.workunit.client.0.smithi142.stdout:9/375: dread d0/d2/d6/d16/f58 [0,4194304] 0 2024-03-29T04:45:01.606 INFO:tasks.workunit.client.0.smithi142.stdout:4/512: mknod d0/d9/d11/d4c/d5f/cb4 0 2024-03-29T04:45:01.606 INFO:tasks.workunit.client.0.smithi142.stdout:0/511: dwrite d7/d13/d67/d72/d37/f56 [0,4194304] 0 2024-03-29T04:45:01.607 INFO:tasks.workunit.client.0.smithi142.stdout:7/586: chown d7/d27/d44/db2/db8/d4b/d71/d30/d33/f38 1 1 2024-03-29T04:45:01.610 INFO:tasks.workunit.client.0.smithi142.stdout:1/520: dwrite d8/db/dc/d19/d1d/f48 [0,4194304] 0 2024-03-29T04:45:01.614 INFO:tasks.workunit.client.0.smithi142.stdout:3/429: truncate d3/d29/d8f/d5f/f86 224773 0 2024-03-29T04:45:01.614 INFO:tasks.workunit.client.0.smithi142.stdout:6/489: dwrite d1c/f6e [0,4194304] 0 2024-03-29T04:45:01.615 INFO:tasks.workunit.client.0.smithi142.stdout:6/490: stat d1c/d45/l47 0 2024-03-29T04:45:01.615 INFO:tasks.workunit.client.0.smithi142.stdout:1/521: write d8/db/dc/d77/d31/d4d/f6e [4042493,93853] 0 2024-03-29T04:45:01.616 INFO:tasks.workunit.client.0.smithi142.stdout:4/513: mknod d0/d9/d11/d4c/d5f/d81/cb5 0 2024-03-29T04:45:01.621 INFO:tasks.workunit.client.0.smithi142.stdout:2/475: dread de/d11/f4f [0,4194304] 0 2024-03-29T04:45:01.622 INFO:tasks.workunit.client.0.smithi142.stdout:3/430: symlink d3/d54/l96 0 2024-03-29T04:45:01.625 INFO:tasks.workunit.client.0.smithi142.stdout:7/587: dwrite d7/d27/d44/db2/db8/d4b/d71/f24 [0,4194304] 0 2024-03-29T04:45:01.628 INFO:tasks.workunit.client.0.smithi142.stdout:0/512: mknod d7/d13/d4a/d70/ca2 0 2024-03-29T04:45:01.629 INFO:tasks.workunit.client.0.smithi142.stdout:4/514: symlink d0/d9/d11/d84/lb6 0 2024-03-29T04:45:01.633 INFO:tasks.workunit.client.0.smithi142.stdout:2/476: write de/d11/d1e/d4d/d61/f78 [649837,32836] 0 2024-03-29T04:45:01.634 INFO:tasks.workunit.client.0.smithi142.stdout:1/522: dwrite d8/db/dc/d77/d31/f61 [0,4194304] 0 2024-03-29T04:45:01.635 INFO:tasks.workunit.client.0.smithi142.stdout:4/515: truncate d0/d45/fad 568494 0 2024-03-29T04:45:01.637 INFO:tasks.workunit.client.0.smithi142.stdout:6/491: dwrite d1c/f39 [4194304,4194304] 0 2024-03-29T04:45:01.637 INFO:tasks.workunit.client.0.smithi142.stdout:0/513: symlink d7/d13/d67/d72/d75/la3 0 2024-03-29T04:45:01.641 INFO:tasks.workunit.client.0.smithi142.stdout:7/588: fdatasync d7/d27/d44/db2/db8/de/f26 0 2024-03-29T04:45:01.646 INFO:tasks.workunit.client.0.smithi142.stdout:1/523: fdatasync d8/db/dc/d77/d31/f3f 0 2024-03-29T04:45:01.651 INFO:tasks.workunit.client.0.smithi142.stdout:3/431: dwrite d3/d29/d2b/d40/d67/d8b/d6/f2f [0,4194304] 0 2024-03-29T04:45:01.651 INFO:tasks.workunit.client.0.smithi142.stdout:5/623: dread df/d1a/d21/f3e [0,4194304] 0 2024-03-29T04:45:01.651 INFO:tasks.workunit.client.0.smithi142.stdout:0/514: mkdir d7/d13/d67/d72/d37/da4 0 2024-03-29T04:45:01.654 INFO:tasks.workunit.client.0.smithi142.stdout:7/589: dread d7/d27/d44/d4c/f8c [0,4194304] 0 2024-03-29T04:45:01.655 INFO:tasks.workunit.client.0.smithi142.stdout:1/524: read - d8/db/dc/d19/d1d/d1f/f9b zero size 2024-03-29T04:45:01.657 INFO:tasks.workunit.client.0.smithi142.stdout:2/477: dwrite de/d11/d17/f31 [0,4194304] 0 2024-03-29T04:45:01.659 INFO:tasks.workunit.client.0.smithi142.stdout:3/432: write d3/d29/d2b/d40/f4a [1013396,106621] 0 2024-03-29T04:45:01.661 INFO:tasks.workunit.client.0.smithi142.stdout:5/624: mknod df/d16/d24/d2e/d47/d8c/cca 0 2024-03-29T04:45:01.661 INFO:tasks.workunit.client.0.smithi142.stdout:0/515: write d7/d9/f1c [1456669,76128] 0 2024-03-29T04:45:01.663 INFO:tasks.workunit.client.0.smithi142.stdout:6/492: dwrite d1c/d45/d51/f7f [0,4194304] 0 2024-03-29T04:45:01.666 INFO:tasks.workunit.client.0.smithi142.stdout:7/590: mknod d7/d27/d44/db2/db8/de/d8d/cc5 0 2024-03-29T04:45:01.668 INFO:tasks.workunit.client.0.smithi142.stdout:2/478: rmdir de/d11/d17/d4a 39 2024-03-29T04:45:01.669 INFO:tasks.workunit.client.0.smithi142.stdout:0/516: write d7/d13/d67/d72/d37/f56 [3248239,55561] 0 2024-03-29T04:45:01.670 INFO:tasks.workunit.client.0.smithi142.stdout:6/493: rmdir d1c/d33/d46/d66 39 2024-03-29T04:45:01.672 INFO:tasks.workunit.client.0.smithi142.stdout:5/625: creat df/d1a/d21/fcb x:0 0 0 2024-03-29T04:45:01.675 INFO:tasks.workunit.client.0.smithi142.stdout:4/516: dwrite d0/d12/d28/d47/f4b [0,4194304] 0 2024-03-29T04:45:01.675 INFO:tasks.workunit.client.0.smithi142.stdout:2/479: dread de/d11/f4f [0,4194304] 0 2024-03-29T04:45:01.677 INFO:tasks.workunit.client.0.smithi142.stdout:2/480: write de/f8b [2780195,107658] 0 2024-03-29T04:45:01.678 INFO:tasks.workunit.client.0.smithi142.stdout:6/494: mkdir d1c/d90/d94 0 2024-03-29T04:45:01.680 INFO:tasks.workunit.client.0.smithi142.stdout:5/626: rename df/d29/d5d/d75/fad to df/d16/d24/d2e/d47/d4d/fcc 0 2024-03-29T04:45:01.687 INFO:tasks.workunit.client.0.smithi142.stdout:6/495: creat d1c/d33/d46/d8b/d52/f95 x:0 0 0 2024-03-29T04:45:01.688 INFO:tasks.workunit.client.0.smithi142.stdout:7/591: getdents d7/d27/d44/db2/db8/d4b 0 2024-03-29T04:45:01.688 INFO:tasks.workunit.client.0.smithi142.stdout:6/496: dread d1c/f41 [0,4194304] 0 2024-03-29T04:45:01.688 INFO:tasks.workunit.client.0.smithi142.stdout:4/517: symlink d0/d9/d11/d5c/d6d/lb7 0 2024-03-29T04:45:01.689 INFO:tasks.workunit.client.0.smithi142.stdout:2/481: creat de/d11/d17/d5d/d71/f9f x:0 0 0 2024-03-29T04:45:01.690 INFO:tasks.workunit.client.0.smithi142.stdout:4/518: fdatasync d0/d12/d28/d47/d7a/f8b 0 2024-03-29T04:45:01.690 INFO:tasks.workunit.client.0.smithi142.stdout:0/517: getdents d7/d13/d4a/d70 0 2024-03-29T04:45:01.692 INFO:tasks.workunit.client.0.smithi142.stdout:4/519: rmdir d0/d9 39 2024-03-29T04:45:01.693 INFO:tasks.workunit.client.0.smithi142.stdout:6/497: chown la 3 1 2024-03-29T04:45:01.695 INFO:tasks.workunit.client.0.smithi142.stdout:6/498: chown d1c/d33/d46/l6a 20974137 1 2024-03-29T04:45:01.695 INFO:tasks.workunit.client.0.smithi142.stdout:2/482: creat de/d11/d1e/d4d/d61/fa0 x:0 0 0 2024-03-29T04:45:01.696 INFO:tasks.workunit.client.0.smithi142.stdout:7/592: dwrite d7/d27/d44/db2/db8/de/d8d/fa5 [0,4194304] 0 2024-03-29T04:45:01.697 INFO:tasks.workunit.client.0.smithi142.stdout:7/593: readlink d7/d27/d44/db2/db8/d4b/d71/l6a 0 2024-03-29T04:45:01.698 INFO:tasks.workunit.client.0.smithi142.stdout:0/518: rename d7/d9/d54/d58/c8d to d7/d9/d55/d96/ca5 0 2024-03-29T04:45:01.699 INFO:tasks.workunit.client.0.smithi142.stdout:4/520: mknod d0/d9/cb8 0 2024-03-29T04:45:01.700 INFO:tasks.workunit.client.0.smithi142.stdout:7/594: chown d7/d27/d44/db2/db8/de/d20/cad 7 1 2024-03-29T04:45:01.701 INFO:tasks.workunit.client.0.smithi142.stdout:2/483: fsync de/d11/d1e/d22/d50/f62 0 2024-03-29T04:45:01.701 INFO:tasks.workunit.client.0.smithi142.stdout:3/433: truncate d3/d1f/d24/d47/f57 3047360 0 2024-03-29T04:45:01.702 INFO:tasks.workunit.client.0.smithi142.stdout:0/519: creat d7/d13/d4a/fa6 x:0 0 0 2024-03-29T04:45:01.704 INFO:tasks.workunit.client.0.smithi142.stdout:7/595: stat d7/d27/d44/db2/db8/de/d20/d3f/f65 0 2024-03-29T04:45:01.706 INFO:tasks.workunit.client.0.smithi142.stdout:3/434: truncate d3/d29/d2b/d40/d67/d8b/f72 560537 0 2024-03-29T04:45:01.707 INFO:tasks.workunit.client.0.smithi142.stdout:2/484: mkdir de/d11/da1 0 2024-03-29T04:45:01.709 INFO:tasks.workunit.client.0.smithi142.stdout:5/627: truncate df/d16/f26 5526969 0 2024-03-29T04:45:01.710 INFO:tasks.workunit.client.0.smithi142.stdout:0/520: dwrite d7/f52 [0,4194304] 0 2024-03-29T04:45:01.712 INFO:tasks.workunit.client.0.smithi142.stdout:2/485: rmdir de/d11/d1e/d4d/d61 39 2024-03-29T04:45:01.717 INFO:tasks.workunit.client.0.smithi142.stdout:2/486: write fd [4512406,69250] 0 2024-03-29T04:45:01.719 INFO:tasks.workunit.client.0.smithi142.stdout:5/628: creat df/d1a/d21/d55/d66/fcd x:0 0 0 2024-03-29T04:45:01.720 INFO:tasks.workunit.client.0.smithi142.stdout:0/521: mkdir d7/d13/d4a/d9a/da7 0 2024-03-29T04:45:01.721 INFO:tasks.workunit.client.0.smithi142.stdout:3/435: dwrite d3/d29/d2b/d40/f82 [0,4194304] 0 2024-03-29T04:45:01.724 INFO:tasks.workunit.client.0.smithi142.stdout:5/629: chown df/d1a/d21/d55/d66 23 1 2024-03-29T04:45:01.727 INFO:tasks.workunit.client.0.smithi142.stdout:5/630: fdatasync df/d29/d7a/fc4 0 2024-03-29T04:45:01.727 INFO:tasks.workunit.client.0.smithi142.stdout:7/596: dread d7/d27/d44/db2/db8/de/d20/f21 [0,4194304] 0 2024-03-29T04:45:01.728 INFO:tasks.workunit.client.0.smithi142.stdout:2/487: mknod de/d11/d17/d5d/d71/ca2 0 2024-03-29T04:45:01.729 INFO:tasks.workunit.client.0.smithi142.stdout:3/436: truncate d3/d29/d8f/d5f/d64/f76 415803 0 2024-03-29T04:45:01.730 INFO:tasks.workunit.client.0.smithi142.stdout:5/631: chown df/d29/cb8 27840575 1 2024-03-29T04:45:01.732 INFO:tasks.workunit.client.0.smithi142.stdout:2/488: mknod de/d11/d2b/d3e/d85/ca3 0 2024-03-29T04:45:01.734 INFO:tasks.workunit.client.0.smithi142.stdout:8/566: sync 2024-03-29T04:45:01.735 INFO:tasks.workunit.client.0.smithi142.stdout:3/437: rename d3/d29/d2b/d40/d67/f7f to d3/d54/f97 0 2024-03-29T04:45:01.739 INFO:tasks.workunit.client.0.smithi142.stdout:8/567: chown d5/d1d/d24/d66/l67 6098501 1 2024-03-29T04:45:01.741 INFO:tasks.workunit.client.0.smithi142.stdout:6/499: dread f18 [0,4194304] 0 2024-03-29T04:45:01.742 INFO:tasks.workunit.client.0.smithi142.stdout:7/597: dwrite d7/d27/d44/db2/db8/de/d20/f40 [0,4194304] 0 2024-03-29T04:45:01.743 INFO:tasks.workunit.client.0.smithi142.stdout:4/521: dread d0/d9/d11/f25 [0,4194304] 0 2024-03-29T04:45:01.743 INFO:tasks.workunit.client.0.smithi142.stdout:5/632: dwrite df/d1a/f9e [0,4194304] 0 2024-03-29T04:45:01.746 INFO:tasks.workunit.client.0.smithi142.stdout:7/598: truncate d7/d27/d44/db2/db8/de/d20/d3f/fae 351526 0 2024-03-29T04:45:01.748 INFO:tasks.workunit.client.0.smithi142.stdout:4/522: read d0/d12/d28/f54 [61392,98652] 0 2024-03-29T04:45:01.756 INFO:tasks.workunit.client.0.smithi142.stdout:0/522: dread d7/d9/f68 [0,4194304] 0 2024-03-29T04:45:01.783 INFO:tasks.workunit.client.0.smithi142.stdout:9/376: sync 2024-03-29T04:45:01.783 INFO:tasks.workunit.client.0.smithi142.stdout:1/525: sync 2024-03-29T04:45:01.942 INFO:tasks.workunit.client.0.smithi142.stdout:2/489: truncate de/d11/d1e/d22/d27/f4c 1458818 0 2024-03-29T04:45:01.943 INFO:tasks.workunit.client.0.smithi142.stdout:6/500: chown l19 0 1 2024-03-29T04:45:01.944 INFO:tasks.workunit.client.0.smithi142.stdout:6/501: fsync d1c/d33/d46/d71/f68 0 2024-03-29T04:45:01.944 INFO:tasks.workunit.client.0.smithi142.stdout:6/502: fdatasync d1c/d33/d46/f69 0 2024-03-29T04:45:01.945 INFO:tasks.workunit.client.0.smithi142.stdout:8/568: creat d5/d1d/d2f/d76/d2d/fca x:0 0 0 2024-03-29T04:45:01.945 INFO:tasks.workunit.client.0.smithi142.stdout:8/569: dread - d5/fc5 zero size 2024-03-29T04:45:01.945 INFO:tasks.workunit.client.0.smithi142.stdout:6/503: fdatasync d1c/d33/f3e 0 2024-03-29T04:45:01.947 INFO:tasks.workunit.client.0.smithi142.stdout:5/633: creat df/d16/d24/d2e/d47/fce x:0 0 0 2024-03-29T04:45:01.947 INFO:tasks.workunit.client.0.smithi142.stdout:3/438: write d3/d29/d2b/d40/f4c [2425989,23735] 0 2024-03-29T04:45:01.949 INFO:tasks.workunit.client.0.smithi142.stdout:6/504: dread d1c/f3c [0,4194304] 0 2024-03-29T04:45:01.950 INFO:tasks.workunit.client.0.smithi142.stdout:4/523: rename d0/d9/d11/d78/f79 to d0/d9/d19/d2c/fb9 0 2024-03-29T04:45:01.950 INFO:tasks.workunit.client.0.smithi142.stdout:8/570: rename d5/d1d/d2f to d5/d1d/d2f/d76/d4c/dcb 22 2024-03-29T04:45:01.954 INFO:tasks.workunit.client.0.smithi142.stdout:6/505: dwrite d1c/d33/d46/f8c [0,4194304] 0 2024-03-29T04:45:01.955 INFO:tasks.workunit.client.0.smithi142.stdout:6/506: dread - d1c/f21 zero size 2024-03-29T04:45:01.955 INFO:tasks.workunit.client.0.smithi142.stdout:6/507: write d1c/f40 [1517821,49415] 0 2024-03-29T04:45:01.990 INFO:tasks.workunit.client.0.smithi142.stdout:9/377: creat d0/d2/d6/d31/f71 x:0 0 0 2024-03-29T04:45:01.993 INFO:tasks.workunit.client.0.smithi142.stdout:2/490: fdatasync de/d11/d1e/f80 0 2024-03-29T04:45:01.996 INFO:tasks.workunit.client.0.smithi142.stdout:7/599: creat d7/d27/d44/db2/dc4/fc6 x:0 0 0 2024-03-29T04:45:01.999 INFO:tasks.workunit.client.0.smithi142.stdout:2/491: dwrite de/d11/d17/f69 [0,4194304] 0 2024-03-29T04:45:01.999 INFO:tasks.workunit.client.0.smithi142.stdout:4/524: mkdir d0/d9/db2/dba 0 2024-03-29T04:45:02.007 INFO:tasks.workunit.client.0.smithi142.stdout:4/525: dwrite d0/d9/d11/d4c/d5f/f6c [0,4194304] 0 2024-03-29T04:45:02.008 INFO:tasks.workunit.client.0.smithi142.stdout:4/526: chown d0/lae 552326 1 2024-03-29T04:45:02.189 INFO:tasks.workunit.client.0.smithi142.stdout:9/378: creat d0/d2/d6/d27/d3c/f72 x:0 0 0 2024-03-29T04:45:02.189 INFO:tasks.workunit.client.0.smithi142.stdout:9/379: dread - d0/d2/d6/d27/d3c/f72 zero size 2024-03-29T04:45:02.192 INFO:tasks.workunit.client.0.smithi142.stdout:7/600: creat d7/d27/d44/db2/db8/de/d20/d3f/fc7 x:0 0 0 2024-03-29T04:45:02.193 INFO:tasks.workunit.client.0.smithi142.stdout:1/526: truncate d8/db/dc/d77/d31/d4d/d8c/f95 2859234 0 2024-03-29T04:45:02.316 INFO:tasks.workunit.client.0.smithi142.stdout:8/571: link d5/d1d/d2f/d45/f9f d5/d2e/fcc 0 2024-03-29T04:45:02.318 INFO:tasks.workunit.client.0.smithi142.stdout:6/508: symlink d1c/d33/d46/l96 0 2024-03-29T04:45:02.318 INFO:tasks.workunit.client.0.smithi142.stdout:3/439: truncate d3/d29/d8f/d5f/d64/f76 143650 0 2024-03-29T04:45:02.320 INFO:tasks.workunit.client.0.smithi142.stdout:0/523: link d7/d9/d55/c76 d7/d9/d55/d96/ca8 0 2024-03-29T04:45:02.322 INFO:tasks.workunit.client.0.smithi142.stdout:0/524: dread - d7/d9/d6e/f95 zero size 2024-03-29T04:45:02.322 INFO:tasks.workunit.client.0.smithi142.stdout:5/634: rename df/d1a/f31 to df/d1a/d21/fcf 0 2024-03-29T04:45:02.323 INFO:tasks.workunit.client.0.smithi142.stdout:8/572: dwrite d5/d1d/d2f/d76/d4c/d60/f8f [4194304,4194304] 0 2024-03-29T04:45:02.325 INFO:tasks.workunit.client.0.smithi142.stdout:8/573: chown d5/d1d/d59 0 1 2024-03-29T04:45:02.325 INFO:tasks.workunit.client.0.smithi142.stdout:1/527: symlink d8/db/dc/d19/d1d/d2d/lbc 0 2024-03-29T04:45:02.329 INFO:tasks.workunit.client.0.smithi142.stdout:6/509: unlink d1c/d33/f36 0 2024-03-29T04:45:02.330 INFO:tasks.workunit.client.0.smithi142.stdout:3/440: creat d3/d1f/d24/d47/f98 x:0 0 0 2024-03-29T04:45:02.331 INFO:tasks.workunit.client.0.smithi142.stdout:3/441: write d3/d1f/d24/d47/f90 [863912,8852] 0 2024-03-29T04:45:02.335 INFO:tasks.workunit.client.0.smithi142.stdout:9/380: rename d0/c5e to d0/d2/c73 0 2024-03-29T04:45:02.337 INFO:tasks.workunit.client.0.smithi142.stdout:2/492: symlink de/d11/d17/d4a/d79/la4 0 2024-03-29T04:45:02.338 INFO:tasks.workunit.client.0.smithi142.stdout:8/574: symlink d5/d1d/d2f/d5f/d48/lcd 0 2024-03-29T04:45:02.339 INFO:tasks.workunit.client.0.smithi142.stdout:2/493: truncate de/d11/d17/f25 8570907 0 2024-03-29T04:45:02.340 INFO:tasks.workunit.client.0.smithi142.stdout:1/528: chown d8/db/dc/d77/d31/d4d/f7a 1659058 1 2024-03-29T04:45:02.344 INFO:tasks.workunit.client.0.smithi142.stdout:6/510: creat d1c/d45/d51/f97 x:0 0 0 2024-03-29T04:45:02.347 INFO:tasks.workunit.client.0.smithi142.stdout:3/442: creat d3/d29/d2b/d6a/f99 x:0 0 0 2024-03-29T04:45:02.348 INFO:tasks.workunit.client.0.smithi142.stdout:7/601: link d7/d27/d44/db2/db8/de/l6c d7/d27/d44/db2/db8/de/d20/d3f/lc8 0 2024-03-29T04:45:02.349 INFO:tasks.workunit.client.0.smithi142.stdout:7/602: stat d7/d27/d44/db2/db8/d4b/d71/d30/d33 0 2024-03-29T04:45:02.352 INFO:tasks.workunit.client.0.smithi142.stdout:8/575: mkdir d5/d1d/d24/d66/dce 0 2024-03-29T04:45:02.353 INFO:tasks.workunit.client.0.smithi142.stdout:2/494: mknod de/d11/d17/d60/d6f/ca5 0 2024-03-29T04:45:02.354 INFO:tasks.workunit.client.0.smithi142.stdout:2/495: readlink de/d11/d1e/d22/d50/l9c 0 2024-03-29T04:45:02.355 INFO:tasks.workunit.client.0.smithi142.stdout:2/496: stat de/d11/c1c 0 2024-03-29T04:45:02.355 INFO:tasks.workunit.client.0.smithi142.stdout:5/635: link df/d1a/d21/d55/d66/fcd df/d29/d5d/fd0 0 2024-03-29T04:45:02.359 INFO:tasks.workunit.client.0.smithi142.stdout:7/603: unlink d7/d27/d44/db2/db8/de/d20/f85 0 2024-03-29T04:45:02.360 INFO:tasks.workunit.client.0.smithi142.stdout:8/576: creat d5/d1d/d2f/d45/fcf x:0 0 0 2024-03-29T04:45:02.361 INFO:tasks.workunit.client.0.smithi142.stdout:1/529: mkdir d8/db/dc/d77/db9/dbd 0 2024-03-29T04:45:02.361 INFO:tasks.workunit.client.0.smithi142.stdout:4/527: write d0/f21 [5825916,39892] 0 2024-03-29T04:45:02.363 INFO:tasks.workunit.client.0.smithi142.stdout:6/511: creat d1c/d33/d46/d66/f98 x:0 0 0 2024-03-29T04:45:02.364 INFO:tasks.workunit.client.0.smithi142.stdout:5/636: dwrite df/d16/d24/d79/f5b [4194304,4194304] 0 2024-03-29T04:45:02.364 INFO:tasks.workunit.client.0.smithi142.stdout:2/497: truncate de/d11/d1e/d22/d27/f30 1179872 0 2024-03-29T04:45:02.365 INFO:tasks.workunit.client.0.smithi142.stdout:6/512: read d1c/d33/d46/d66/f6b [35371,92297] 0 2024-03-29T04:45:02.366 INFO:tasks.workunit.client.0.smithi142.stdout:7/604: mknod d7/d27/d44/db2/db8/d4b/cc9 0 2024-03-29T04:45:02.367 INFO:tasks.workunit.client.0.smithi142.stdout:4/528: mknod d0/d9/d11/d84/cbb 0 2024-03-29T04:45:02.368 INFO:tasks.workunit.client.0.smithi142.stdout:1/530: dread - d8/db/dc/d77/d5a/f63 zero size 2024-03-29T04:45:02.371 INFO:tasks.workunit.client.0.smithi142.stdout:5/637: rmdir df/d16/d24/d2e/d47 39 2024-03-29T04:45:02.374 INFO:tasks.workunit.client.0.smithi142.stdout:0/525: dwrite d7/d9/d54/f64 [0,4194304] 0 2024-03-29T04:45:02.374 INFO:tasks.workunit.client.0.smithi142.stdout:3/443: getdents d3/d29/d8f/d5f/d64/d6b 0 2024-03-29T04:45:02.375 INFO:tasks.workunit.client.0.smithi142.stdout:0/526: write d7/d9/d55/f5b [235951,46541] 0 2024-03-29T04:45:02.377 INFO:tasks.workunit.client.0.smithi142.stdout:0/527: chown d7/d9/d1e/c4c 1366555 1 2024-03-29T04:45:02.378 INFO:tasks.workunit.client.0.smithi142.stdout:0/528: chown d7/l8a 153053 1 2024-03-29T04:45:02.379 INFO:tasks.workunit.client.0.smithi142.stdout:7/605: unlink d7/d27/d44/db2/db8/d4b/d71/f24 0 2024-03-29T04:45:02.382 INFO:tasks.workunit.client.0.smithi142.stdout:4/529: rename d0/d9/d11/d5c to d0/d9/db2/dba/dbc 0 2024-03-29T04:45:02.385 INFO:tasks.workunit.client.0.smithi142.stdout:2/498: fsync de/d11/d1e/d22/f26 0 2024-03-29T04:45:02.389 INFO:tasks.workunit.client.0.smithi142.stdout:9/381: mknod d0/d2/d61/c74 0 2024-03-29T04:45:02.391 INFO:tasks.workunit.client.0.smithi142.stdout:0/529: mknod d7/d13/d67/d72/d75/ca9 0 2024-03-29T04:45:02.393 INFO:tasks.workunit.client.0.smithi142.stdout:0/530: write d7/d9/d55/f57 [1237700,31615] 0 2024-03-29T04:45:02.396 INFO:tasks.workunit.client.0.smithi142.stdout:1/531: rename d8/db/dc/d19/d1d to d8/db/dc/d77/db9/dbe 0 2024-03-29T04:45:02.397 INFO:tasks.workunit.client.0.smithi142.stdout:2/499: rename de/d11/d17/d4a to de/d11/d17/d4a/d79/da6 22 2024-03-29T04:45:02.397 INFO:tasks.workunit.client.0.smithi142.stdout:4/530: mknod d0/d45/cbd 0 2024-03-29T04:45:02.398 INFO:tasks.workunit.client.0.smithi142.stdout:3/444: dread d3/d29/d2b/f21 [0,4194304] 0 2024-03-29T04:45:02.400 INFO:tasks.workunit.client.0.smithi142.stdout:7/606: dread d7/d27/d44/db2/db8/d4b/d71/d30/d33/f93 [0,4194304] 0 2024-03-29T04:45:02.400 INFO:tasks.workunit.client.0.smithi142.stdout:4/531: read - d0/d9/d11/d4c/d5f/f8a zero size 2024-03-29T04:45:02.400 INFO:tasks.workunit.client.0.smithi142.stdout:6/513: creat d1c/d33/f99 x:0 0 0 2024-03-29T04:45:02.401 INFO:tasks.workunit.client.0.smithi142.stdout:1/532: read d8/db/dc/d77/d31/d4d/f6e [2432520,81878] 0 2024-03-29T04:45:02.403 INFO:tasks.workunit.client.0.smithi142.stdout:0/531: creat d7/d13/d67/d72/d37/faa x:0 0 0 2024-03-29T04:45:02.403 INFO:tasks.workunit.client.0.smithi142.stdout:9/382: chown d0/d2/d6/d27/d3c/f69 38500 1 2024-03-29T04:45:02.405 INFO:tasks.workunit.client.0.smithi142.stdout:4/532: write d0/d9/d11/d4c/d5f/f6f [913520,89415] 0 2024-03-29T04:45:02.405 INFO:tasks.workunit.client.0.smithi142.stdout:2/500: creat de/d11/d2b/d7b/fa7 x:0 0 0 2024-03-29T04:45:02.406 INFO:tasks.workunit.client.0.smithi142.stdout:3/445: fdatasync d3/d1f/d24/f37 0 2024-03-29T04:45:02.406 INFO:tasks.workunit.client.0.smithi142.stdout:7/607: chown d7/d27/d44/db2/db8/d4b/d71/f9d 9776 1 2024-03-29T04:45:02.409 INFO:tasks.workunit.client.0.smithi142.stdout:0/532: rmdir d7/d9 39 2024-03-29T04:45:02.411 INFO:tasks.workunit.client.0.smithi142.stdout:5/638: dwrite df/d16/d24/d2e/d47/d4d/fcc [0,4194304] 0 2024-03-29T04:45:02.412 INFO:tasks.workunit.client.0.smithi142.stdout:1/533: write d8/db/dc/d77/db9/dbe/d2d/d44/d53/f5f [534276,117886] 0 2024-03-29T04:45:02.420 INFO:tasks.workunit.client.0.smithi142.stdout:9/383: rename d0/d2/d6/d16/f1c to d0/d2/d6/d31/f75 0 2024-03-29T04:45:02.423 INFO:tasks.workunit.client.0.smithi142.stdout:5/639: creat df/d16/d24/d2e/d47/fd1 x:0 0 0 2024-03-29T04:45:02.424 INFO:tasks.workunit.client.0.smithi142.stdout:3/446: write d3/d29/d2b/f21 [1498699,82955] 0 2024-03-29T04:45:02.425 INFO:tasks.workunit.client.0.smithi142.stdout:0/533: mkdir d7/d9/d55/d96/dab 0 2024-03-29T04:45:02.425 INFO:tasks.workunit.client.0.smithi142.stdout:6/514: link d1c/d33/d46/l57 d1c/d33/d46/d8b/d52/l9a 0 2024-03-29T04:45:02.427 INFO:tasks.workunit.client.0.smithi142.stdout:7/608: symlink d7/d27/d44/db2/db8/de/dc1/lca 0 2024-03-29T04:45:02.430 INFO:tasks.workunit.client.0.smithi142.stdout:5/640: write df/d1a/d21/f3e [5890241,43833] 0 2024-03-29T04:45:02.431 INFO:tasks.workunit.client.0.smithi142.stdout:2/501: dread de/d11/d1e/d22/d27/f45 [0,4194304] 0 2024-03-29T04:45:02.431 INFO:tasks.workunit.client.0.smithi142.stdout:3/447: creat d3/d29/d2d/d56/f9a x:0 0 0 2024-03-29T04:45:02.432 INFO:tasks.workunit.client.0.smithi142.stdout:6/515: mkdir d1c/d33/d46/d61/d9b 0 2024-03-29T04:45:02.433 INFO:tasks.workunit.client.0.smithi142.stdout:0/534: symlink d7/d9/lac 0 2024-03-29T04:45:02.434 INFO:tasks.workunit.client.0.smithi142.stdout:1/534: dwrite f3 [0,4194304] 0 2024-03-29T04:45:02.435 INFO:tasks.workunit.client.0.smithi142.stdout:9/384: dwrite d0/d2/d6/d16/f40 [4194304,4194304] 0 2024-03-29T04:45:02.436 INFO:tasks.workunit.client.0.smithi142.stdout:7/609: readlink d7/d27/d44/l63 0 2024-03-29T04:45:02.439 INFO:tasks.workunit.client.0.smithi142.stdout:3/448: symlink d3/d29/d2b/d6a/l9b 0 2024-03-29T04:45:02.443 INFO:tasks.workunit.client.0.smithi142.stdout:9/385: fdatasync d0/f62 0 2024-03-29T04:45:02.445 INFO:tasks.workunit.client.0.smithi142.stdout:4/533: dread d0/dc/f33 [0,4194304] 0 2024-03-29T04:45:02.446 INFO:tasks.workunit.client.0.smithi142.stdout:5/641: write df/d1a/d21/d27/d45/dbf/f9f [1067191,95209] 0 2024-03-29T04:45:02.447 INFO:tasks.workunit.client.0.smithi142.stdout:9/386: write d0/f32 [909811,55384] 0 2024-03-29T04:45:02.450 INFO:tasks.workunit.client.0.smithi142.stdout:6/516: getdents d1c/d33/d46/d8b 0 2024-03-29T04:45:02.450 INFO:tasks.workunit.client.0.smithi142.stdout:1/535: fsync d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/fbb 0 2024-03-29T04:45:02.452 INFO:tasks.workunit.client.0.smithi142.stdout:7/610: truncate d7/d27/d44/f94 2002478 0 2024-03-29T04:45:02.453 INFO:tasks.workunit.client.0.smithi142.stdout:2/502: dwrite de/d11/f4f [0,4194304] 0 2024-03-29T04:45:02.454 INFO:tasks.workunit.client.0.smithi142.stdout:0/535: link d7/d13/d67/d72/d37/f56 d7/d13/d4a/d9a/da7/fad 0 2024-03-29T04:45:02.457 INFO:tasks.workunit.client.0.smithi142.stdout:6/517: truncate d1c/f62 627893 0 2024-03-29T04:45:02.458 INFO:tasks.workunit.client.0.smithi142.stdout:1/536: symlink d8/db/dc/d77/d31/d4d/d8c/lbf 0 2024-03-29T04:45:02.461 INFO:tasks.workunit.client.0.smithi142.stdout:7/611: fdatasync d7/d27/d44/db2/db8/de/d20/f2b 0 2024-03-29T04:45:02.461 INFO:tasks.workunit.client.0.smithi142.stdout:5/642: dread df/d1a/d21/f3e [0,4194304] 0 2024-03-29T04:45:02.464 INFO:tasks.workunit.client.0.smithi142.stdout:1/537: symlink d8/lc0 0 2024-03-29T04:45:02.466 INFO:tasks.workunit.client.0.smithi142.stdout:0/536: dwrite d7/d9/d54/d58/f82 [0,4194304] 0 2024-03-29T04:45:02.468 INFO:tasks.workunit.client.0.smithi142.stdout:5/643: mknod df/d29/d5d/d75/cd2 0 2024-03-29T04:45:02.468 INFO:tasks.workunit.client.0.smithi142.stdout:7/612: symlink d7/d27/d44/db2/db8/d4b/lcb 0 2024-03-29T04:45:02.470 INFO:tasks.workunit.client.0.smithi142.stdout:5/644: write df/d1a/d28/f8d [1072069,121565] 0 2024-03-29T04:45:02.473 INFO:tasks.workunit.client.0.smithi142.stdout:7/613: dread - d7/d27/d44/db2/db8/de/d20/d3f/fa0 zero size 2024-03-29T04:45:02.475 INFO:tasks.workunit.client.0.smithi142.stdout:7/614: write d7/d27/d44/db2/db8/de/d20/f40 [4262574,61615] 0 2024-03-29T04:45:02.476 INFO:tasks.workunit.client.0.smithi142.stdout:5/645: mkdir df/d1a/d21/dd3 0 2024-03-29T04:45:02.478 INFO:tasks.workunit.client.0.smithi142.stdout:5/646: write df/d16/d25/f59 [1585806,96286] 0 2024-03-29T04:45:02.479 INFO:tasks.workunit.client.0.smithi142.stdout:3/449: truncate d3/d29/d2b/d40/d67/d8b/f3e 4205898 0 2024-03-29T04:45:02.480 INFO:tasks.workunit.client.0.smithi142.stdout:7/615: write d7/d27/d44/db2/db8/d4b/d71/d30/f4a [340216,77404] 0 2024-03-29T04:45:02.483 INFO:tasks.workunit.client.0.smithi142.stdout:2/503: truncate de/f29 561980 0 2024-03-29T04:45:02.485 INFO:tasks.workunit.client.0.smithi142.stdout:5/647: mknod df/d16/d24/d3d/cd4 0 2024-03-29T04:45:02.485 INFO:tasks.workunit.client.0.smithi142.stdout:2/504: chown de/d11 0 1 2024-03-29T04:45:02.488 INFO:tasks.workunit.client.0.smithi142.stdout:4/534: dwrite d0/d9/d11/f23 [0,4194304] 0 2024-03-29T04:45:02.489 INFO:tasks.workunit.client.0.smithi142.stdout:3/450: mknod d3/d1f/d24/d47/c9c 0 2024-03-29T04:45:02.491 INFO:tasks.workunit.client.0.smithi142.stdout:7/616: mkdir d7/d27/d44/dab/dcc 0 2024-03-29T04:45:02.491 INFO:tasks.workunit.client.0.smithi142.stdout:9/387: dread d0/d2/d6/d27/d3c/d4f/f56 [0,4194304] 0 2024-03-29T04:45:02.497 INFO:tasks.workunit.client.0.smithi142.stdout:3/451: creat d3/d29/d2b/d40/d67/d8b/d4d/f9d x:0 0 0 2024-03-29T04:45:02.501 INFO:tasks.workunit.client.0.smithi142.stdout:9/388: write d0/d2/d6/d27/d3c/d4f/f6e [740166,102986] 0 2024-03-29T04:45:02.501 INFO:tasks.workunit.client.0.smithi142.stdout:3/452: symlink d3/d29/d8f/l9e 0 2024-03-29T04:45:02.504 INFO:tasks.workunit.client.0.smithi142.stdout:5/648: write df/d1a/d21/d55/d66/fcd [141605,125155] 0 2024-03-29T04:45:02.506 INFO:tasks.workunit.client.0.smithi142.stdout:7/617: write d7/d27/d44/f94 [2918303,16715] 0 2024-03-29T04:45:02.508 INFO:tasks.workunit.client.0.smithi142.stdout:2/505: dwrite de/f9d [0,4194304] 0 2024-03-29T04:45:02.509 INFO:tasks.workunit.client.0.smithi142.stdout:6/518: dread d1c/f55 [0,4194304] 0 2024-03-29T04:45:02.509 INFO:tasks.workunit.client.0.smithi142.stdout:4/535: dwrite d0/d9/d11/d4c/d5f/f6f [0,4194304] 0 2024-03-29T04:45:02.511 INFO:tasks.workunit.client.0.smithi142.stdout:0/537: dread d7/d13/d4a/d70/f8b [0,4194304] 0 2024-03-29T04:45:02.511 INFO:tasks.workunit.client.0.smithi142.stdout:9/389: write d0/d2/d6/d16/f2b [1813231,32605] 0 2024-03-29T04:45:02.512 INFO:tasks.workunit.client.0.smithi142.stdout:2/506: fdatasync de/d11/f15 0 2024-03-29T04:45:02.516 INFO:tasks.workunit.client.0.smithi142.stdout:3/453: write d3/d1f/d24/f69 [36955,107678] 0 2024-03-29T04:45:02.519 INFO:tasks.workunit.client.0.smithi142.stdout:5/649: mkdir df/d29/dbc/dd5 0 2024-03-29T04:45:02.525 INFO:tasks.workunit.client.0.smithi142.stdout:4/536: rename d0/dc/d15/d3d/l8e to d0/d9/d19/lbe 0 2024-03-29T04:45:02.525 INFO:tasks.workunit.client.0.smithi142.stdout:3/454: symlink d3/d29/d2b/l9f 0 2024-03-29T04:45:02.529 INFO:tasks.workunit.client.0.smithi142.stdout:7/618: symlink d7/d27/d44/db2/db8/de/d20/d3f/lcd 0 2024-03-29T04:45:02.530 INFO:tasks.workunit.client.0.smithi142.stdout:9/390: rename d0/d2/d6/d27/d3c/d4f/f6e to d0/d12/d50/f76 0 2024-03-29T04:45:02.533 INFO:tasks.workunit.client.0.smithi142.stdout:4/537: unlink d0/d9/d11/d4c/d5f/f9c 0 2024-03-29T04:45:02.535 INFO:tasks.workunit.client.0.smithi142.stdout:3/455: unlink d3/d1f/d24/d47/f98 0 2024-03-29T04:45:02.540 INFO:tasks.workunit.client.0.smithi142.stdout:7/619: mknod d7/d27/d3a/d92/d97/cce 0 2024-03-29T04:45:02.543 INFO:tasks.workunit.client.0.smithi142.stdout:0/538: dwrite d7/d13/d67/d72/f3a [0,4194304] 0 2024-03-29T04:45:02.543 INFO:tasks.workunit.client.0.smithi142.stdout:6/519: dwrite d1c/f40 [4194304,4194304] 0 2024-03-29T04:45:02.547 INFO:tasks.workunit.client.0.smithi142.stdout:2/507: dwrite de/d11/d2b/d3e/d85/f9a [0,4194304] 0 2024-03-29T04:45:02.548 INFO:tasks.workunit.client.0.smithi142.stdout:5/650: dwrite df/d16/d24/d5c/db1/fc3 [0,4194304] 0 2024-03-29T04:45:02.553 INFO:tasks.workunit.client.0.smithi142.stdout:7/620: readlink d7/d27/d44/db2/db8/la1 0 2024-03-29T04:45:02.554 INFO:tasks.workunit.client.0.smithi142.stdout:6/520: unlink d1c/d33/d46/d61/f85 0 2024-03-29T04:45:02.560 INFO:tasks.workunit.client.0.smithi142.stdout:0/539: write d7/d9/d1e/f88 [926650,35149] 0 2024-03-29T04:45:02.561 INFO:tasks.workunit.client.0.smithi142.stdout:9/391: link d0/d2/c20 d0/d12/d50/d52/c77 0 2024-03-29T04:45:02.562 INFO:tasks.workunit.client.0.smithi142.stdout:4/538: dwrite d0/d9/d11/d4c/d5f/f6f [0,4194304] 0 2024-03-29T04:45:02.562 INFO:tasks.workunit.client.0.smithi142.stdout:5/651: getdents df/d16/d24/d2e/d47/d8c 0 2024-03-29T04:45:02.567 INFO:tasks.workunit.client.0.smithi142.stdout:2/508: rename de/d11/d2b/d3e/d85 to de/d11/d17/d4a/d79/da8 0 2024-03-29T04:45:02.567 INFO:tasks.workunit.client.0.smithi142.stdout:4/539: chown d0/d12/f44 97609876 1 2024-03-29T04:45:02.570 INFO:tasks.workunit.client.0.smithi142.stdout:7/621: rmdir d7/d27/db6 0 2024-03-29T04:45:02.570 INFO:tasks.workunit.client.0.smithi142.stdout:2/509: read de/d11/d3d/f82 [2504669,33632] 0 2024-03-29T04:45:02.572 INFO:tasks.workunit.client.0.smithi142.stdout:5/652: read df/d16/d24/fa0 [1210034,79076] 0 2024-03-29T04:45:02.572 INFO:tasks.workunit.client.0.smithi142.stdout:9/392: creat d0/d12/d50/d52/f78 x:0 0 0 2024-03-29T04:45:02.574 INFO:tasks.workunit.client.0.smithi142.stdout:6/521: dwrite d1c/f49 [0,4194304] 0 2024-03-29T04:45:02.574 INFO:tasks.workunit.client.0.smithi142.stdout:4/540: readlink d0/d9/d19/lbe 0 2024-03-29T04:45:02.574 INFO:tasks.workunit.client.0.smithi142.stdout:0/540: symlink d7/d9/d55/lae 0 2024-03-29T04:45:02.575 INFO:tasks.workunit.client.0.smithi142.stdout:7/622: chown d7/la2 340 1 2024-03-29T04:45:02.577 INFO:tasks.workunit.client.0.smithi142.stdout:7/623: truncate d7/f57 39486 0 2024-03-29T04:45:02.578 INFO:tasks.workunit.client.0.smithi142.stdout:2/510: creat de/d11/d1e/d22/d27/fa9 x:0 0 0 2024-03-29T04:45:02.579 INFO:tasks.workunit.client.0.smithi142.stdout:5/653: read df/d29/d6e/f73 [708572,73612] 0 2024-03-29T04:45:02.580 INFO:tasks.workunit.client.0.smithi142.stdout:4/541: truncate d0/d9/d19/d2c/fb9 2381101 0 2024-03-29T04:45:02.581 INFO:tasks.workunit.client.0.smithi142.stdout:0/541: write d7/d9/d55/d42/f5c [209354,81076] 0 2024-03-29T04:45:02.581 INFO:tasks.workunit.client.0.smithi142.stdout:6/522: creat d1c/d33/d46/d71/f9c x:0 0 0 2024-03-29T04:45:02.585 INFO:tasks.workunit.client.0.smithi142.stdout:2/511: symlink de/d11/d17/d5d/laa 0 2024-03-29T04:45:02.588 INFO:tasks.workunit.client.0.smithi142.stdout:0/542: mkdir d7/d13/d4a/daf 0 2024-03-29T04:45:02.590 INFO:tasks.workunit.client.0.smithi142.stdout:5/654: symlink df/d29/d5d/d75/d90/ld6 0 2024-03-29T04:45:02.591 INFO:tasks.workunit.client.0.smithi142.stdout:7/624: dwrite d7/d27/d44/db2/db8/d4b/f82 [0,4194304] 0 2024-03-29T04:45:02.593 INFO:tasks.workunit.client.0.smithi142.stdout:2/512: rename de/l48 to de/d11/d17/d4a/d79/lab 0 2024-03-29T04:45:02.593 INFO:tasks.workunit.client.0.smithi142.stdout:3/456: dread d3/d29/d2b/d40/d67/d8b/f3e [0,4194304] 0 2024-03-29T04:45:02.595 INFO:tasks.workunit.client.0.smithi142.stdout:4/542: creat d0/d9/db2/dba/dbc/d8d/da5/fbf x:0 0 0 2024-03-29T04:45:02.595 INFO:tasks.workunit.client.0.smithi142.stdout:6/523: link d1c/d33/d46/d8b/d52/f95 d1c/d33/d46/d86/f9d 0 2024-03-29T04:45:02.596 INFO:tasks.workunit.client.0.smithi142.stdout:6/524: readlink d1c/l2a 0 2024-03-29T04:45:02.600 INFO:tasks.workunit.client.0.smithi142.stdout:7/625: mknod d7/d27/d44/db2/db8/de/d20/d3f/ccf 0 2024-03-29T04:45:02.601 INFO:tasks.workunit.client.0.smithi142.stdout:4/543: unlink d0/d9/d11/d4c/d5f/d81/cb5 0 2024-03-29T04:45:02.602 INFO:tasks.workunit.client.0.smithi142.stdout:4/544: chown d0/l51 905807 1 2024-03-29T04:45:02.602 INFO:tasks.workunit.client.0.smithi142.stdout:5/655: dread df/d16/d24/d5c/f5e [0,4194304] 0 2024-03-29T04:45:02.603 INFO:tasks.workunit.client.0.smithi142.stdout:7/626: stat d7/d27/d44/db2/db8/d4b/d9b/lbf 0 2024-03-29T04:45:02.605 INFO:tasks.workunit.client.0.smithi142.stdout:7/627: write f6 [6085473,113237] 0 2024-03-29T04:45:02.606 INFO:tasks.workunit.client.0.smithi142.stdout:4/545: chown d0/dc/d15/d3d/f43 1473365723 1 2024-03-29T04:45:02.608 INFO:tasks.workunit.client.0.smithi142.stdout:2/513: dwrite de/d11/d17/d5d/f74 [0,4194304] 0 2024-03-29T04:45:02.609 INFO:tasks.workunit.client.0.smithi142.stdout:5/656: getdents df/d16 0 2024-03-29T04:45:02.611 INFO:tasks.workunit.client.0.smithi142.stdout:4/546: creat d0/d9/db2/dba/fc0 x:0 0 0 2024-03-29T04:45:02.613 INFO:tasks.workunit.client.0.smithi142.stdout:7/628: creat d7/d27/d44/db2/db8/d4b/fd0 x:0 0 0 2024-03-29T04:45:02.616 INFO:tasks.workunit.client.0.smithi142.stdout:2/514: read - de/d11/d1e/d4d/d61/fa0 zero size 2024-03-29T04:45:02.617 INFO:tasks.workunit.client.0.smithi142.stdout:6/525: dread f4 [0,4194304] 0 2024-03-29T04:45:02.617 INFO:tasks.workunit.client.0.smithi142.stdout:5/657: symlink df/d16/d24/d2e/d47/d4d/ld7 0 2024-03-29T04:45:02.618 INFO:tasks.workunit.client.0.smithi142.stdout:0/543: write d7/d9/d54/f65 [410999,63036] 0 2024-03-29T04:45:02.619 INFO:tasks.workunit.client.0.smithi142.stdout:7/629: write d7/d27/d44/db2/db8/d4b/d71/f28 [1663087,51018] 0 2024-03-29T04:45:02.620 INFO:tasks.workunit.client.0.smithi142.stdout:6/526: chown l15 37139485 1 2024-03-29T04:45:02.624 INFO:tasks.workunit.client.0.smithi142.stdout:4/547: dwrite d0/d9/d11/d4c/f89 [0,4194304] 0 2024-03-29T04:45:02.624 INFO:tasks.workunit.client.0.smithi142.stdout:5/658: write df/d16/d24/d2e/f6c [1917402,64173] 0 2024-03-29T04:45:02.626 INFO:tasks.workunit.client.0.smithi142.stdout:5/659: readlink df/l19 0 2024-03-29T04:45:02.628 INFO:tasks.workunit.client.0.smithi142.stdout:0/544: rename d7/d13/d67/d72/l80 to d7/d9/d1e/d81/lb0 0 2024-03-29T04:45:02.630 INFO:tasks.workunit.client.0.smithi142.stdout:2/515: link de/c14 de/d11/d2b/d7b/cac 0 2024-03-29T04:45:02.632 INFO:tasks.workunit.client.0.smithi142.stdout:3/457: dread d3/d29/d2b/d40/d67/d8b/f4e [0,4194304] 0 2024-03-29T04:45:02.634 INFO:tasks.workunit.client.0.smithi142.stdout:6/527: dread d1c/d45/d51/f7f [0,4194304] 0 2024-03-29T04:45:02.634 INFO:tasks.workunit.client.0.smithi142.stdout:5/660: fsync df/d29/f67 0 2024-03-29T04:45:02.635 INFO:tasks.workunit.client.0.smithi142.stdout:3/458: readlink d3/d29/d8f/d5f/d64/d6b/l84 0 2024-03-29T04:45:02.636 INFO:tasks.workunit.client.0.smithi142.stdout:0/545: creat d7/d13/d67/d72/fb1 x:0 0 0 2024-03-29T04:45:02.639 INFO:tasks.workunit.client.0.smithi142.stdout:6/528: truncate d1c/f31 1500373 0 2024-03-29T04:45:02.641 INFO:tasks.workunit.client.0.smithi142.stdout:2/516: symlink de/d11/d1e/lad 0 2024-03-29T04:45:02.641 INFO:tasks.workunit.client.0.smithi142.stdout:7/630: truncate d7/f31 2167400 0 2024-03-29T04:45:02.643 INFO:tasks.workunit.client.0.smithi142.stdout:0/546: rename d7/d9/d55/lae to d7/d9/d54/d58/lb2 0 2024-03-29T04:45:02.644 INFO:tasks.workunit.client.0.smithi142.stdout:6/529: mkdir d1c/d33/d46/d61/d9e 0 2024-03-29T04:45:02.644 INFO:tasks.workunit.client.0.smithi142.stdout:6/530: readlink le 0 2024-03-29T04:45:02.645 INFO:tasks.workunit.client.0.smithi142.stdout:4/548: getdents d0/d9/db2/dba/dbc/d6d 0 2024-03-29T04:45:02.647 INFO:tasks.workunit.client.0.smithi142.stdout:0/547: creat d7/d13/d4a/d9a/fb3 x:0 0 0 2024-03-29T04:45:02.648 INFO:tasks.workunit.client.0.smithi142.stdout:2/517: mknod de/d11/d1e/d4d/cae 0 2024-03-29T04:45:02.650 INFO:tasks.workunit.client.0.smithi142.stdout:0/548: stat d7/d9/d54/d58 0 2024-03-29T04:45:02.650 INFO:tasks.workunit.client.0.smithi142.stdout:2/518: chown de/d11/d17/c3c 8 1 2024-03-29T04:45:02.651 INFO:tasks.workunit.client.0.smithi142.stdout:6/531: creat d1c/d33/d46/d71/f9f x:0 0 0 2024-03-29T04:45:02.655 INFO:tasks.workunit.client.0.smithi142.stdout:5/661: truncate df/d1a/d21/d27/d45/dbf/f9f 906144 0 2024-03-29T04:45:02.658 INFO:tasks.workunit.client.0.smithi142.stdout:4/549: dwrite d0/d12/d28/d47/f4b [4194304,4194304] 0 2024-03-29T04:45:02.665 INFO:tasks.workunit.client.0.smithi142.stdout:4/550: read d0/d9/f4d [2831466,44301] 0 2024-03-29T04:45:02.671 INFO:tasks.workunit.client.0.smithi142.stdout:2/519: dwrite de/f8b [0,4194304] 0 2024-03-29T04:45:02.674 INFO:tasks.workunit.client.0.smithi142.stdout:7/631: dread d7/f31 [0,4194304] 0 2024-03-29T04:45:02.676 INFO:tasks.workunit.client.0.smithi142.stdout:0/549: dwrite d7/d13/d4a/d9a/fb3 [0,4194304] 0 2024-03-29T04:45:02.682 INFO:tasks.workunit.client.0.smithi142.stdout:6/532: dread d1c/d45/d51/f7f [0,4194304] 0 2024-03-29T04:45:02.688 INFO:tasks.workunit.client.0.smithi142.stdout:2/520: dread de/d11/d1e/d22/f8e [0,4194304] 0 2024-03-29T04:45:02.689 INFO:tasks.workunit.client.0.smithi142.stdout:6/533: chown d1c/f34 516069 1 2024-03-29T04:45:02.691 INFO:tasks.workunit.client.0.smithi142.stdout:4/551: dwrite d0/d9/db2/dba/dbc/f92 [0,4194304] 0 2024-03-29T04:45:02.695 INFO:tasks.workunit.client.0.smithi142.stdout:7/632: dwrite d7/d27/d44/db2/db8/de/d8d/fa5 [0,4194304] 0 2024-03-29T04:45:02.698 INFO:tasks.workunit.client.0.smithi142.stdout:5/662: dread df/d16/d24/fa0 [0,4194304] 0 2024-03-29T04:45:02.698 INFO:tasks.workunit.client.0.smithi142.stdout:4/552: write d0/d9/db2/dba/dbc/f92 [2528122,49561] 0 2024-03-29T04:45:02.700 INFO:tasks.workunit.client.0.smithi142.stdout:6/534: creat d1c/d45/d8a/fa0 x:0 0 0 2024-03-29T04:45:02.702 INFO:tasks.workunit.client.0.smithi142.stdout:5/663: read df/d16/d25/f37 [1062207,96707] 0 2024-03-29T04:45:02.702 INFO:tasks.workunit.client.0.smithi142.stdout:6/535: stat d1c/d33/d46/d8b/d52/f87 0 2024-03-29T04:45:02.703 INFO:tasks.workunit.client.0.smithi142.stdout:6/536: dread - d1c/d33/f99 zero size 2024-03-29T04:45:02.704 INFO:tasks.workunit.client.0.smithi142.stdout:0/550: truncate d7/d13/d4a/f5e 1613556 0 2024-03-29T04:45:02.705 INFO:tasks.workunit.client.0.smithi142.stdout:0/551: fdatasync d7/d9/f68 0 2024-03-29T04:45:02.706 INFO:tasks.workunit.client.0.smithi142.stdout:7/633: mknod d7/d27/d3a/d92/cd1 0 2024-03-29T04:45:02.709 INFO:tasks.workunit.client.0.smithi142.stdout:4/553: rmdir d0/d9/db2/dba 39 2024-03-29T04:45:02.711 INFO:tasks.workunit.client.0.smithi142.stdout:2/521: creat de/d11/d1e/d22/faf x:0 0 0 2024-03-29T04:45:02.711 INFO:tasks.workunit.client.0.smithi142.stdout:5/664: symlink df/d29/d6e/d8a/ld8 0 2024-03-29T04:45:02.713 INFO:tasks.workunit.client.0.smithi142.stdout:6/537: creat d1c/d33/d46/d61/fa1 x:0 0 0 2024-03-29T04:45:02.713 INFO:tasks.workunit.client.0.smithi142.stdout:5/665: write df/d1a/d21/d55/d66/f92 [1034626,21798] 0 2024-03-29T04:45:02.714 INFO:tasks.workunit.client.0.smithi142.stdout:7/634: mkdir d7/d27/d44/db2/dd2 0 2024-03-29T04:45:02.716 INFO:tasks.workunit.client.0.smithi142.stdout:5/666: write df/d16/d24/d2e/d47/d8c/fba [7361,63258] 0 2024-03-29T04:45:02.717 INFO:tasks.workunit.client.0.smithi142.stdout:4/554: symlink d0/d9/d19/lc1 0 2024-03-29T04:45:02.721 INFO:tasks.workunit.client.0.smithi142.stdout:0/552: link d7/d9/d1e/f71 d7/d9/d1e/d81/fb4 0 2024-03-29T04:45:02.723 INFO:tasks.workunit.client.0.smithi142.stdout:5/667: readlink df/d16/d25/l58 0 2024-03-29T04:45:02.724 INFO:tasks.workunit.client.0.smithi142.stdout:7/635: creat d7/d27/d44/db2/db8/de/d20/fd3 x:0 0 0 2024-03-29T04:45:02.725 INFO:tasks.workunit.client.0.smithi142.stdout:2/522: readlink de/d11/d1e/d22/d27/d3a/l91 0 2024-03-29T04:45:02.728 INFO:tasks.workunit.client.0.smithi142.stdout:6/538: creat d1c/d90/d94/fa2 x:0 0 0 2024-03-29T04:45:02.729 INFO:tasks.workunit.client.0.smithi142.stdout:6/539: stat d1c/d33/d46/d61/fa1 0 2024-03-29T04:45:02.729 INFO:tasks.workunit.client.0.smithi142.stdout:0/553: creat d7/d13/d67/d72/d75/fb5 x:0 0 0 2024-03-29T04:45:02.731 INFO:tasks.workunit.client.0.smithi142.stdout:5/668: rmdir df/d16/d25 39 2024-03-29T04:45:02.734 INFO:tasks.workunit.client.0.smithi142.stdout:7/636: mkdir d7/d27/d44/db2/db8/de/d20/d86/dd4 0 2024-03-29T04:45:02.736 INFO:tasks.workunit.client.0.smithi142.stdout:6/540: write d1c/d45/d51/f7f [5187209,129580] 0 2024-03-29T04:45:02.736 INFO:tasks.workunit.client.0.smithi142.stdout:7/637: read d7/d27/d44/d4c/f79 [1791227,76124] 0 2024-03-29T04:45:02.738 INFO:tasks.workunit.client.0.smithi142.stdout:5/669: mknod df/d29/d5d/d75/d90/cd9 0 2024-03-29T04:45:02.739 INFO:tasks.workunit.client.0.smithi142.stdout:4/555: rmdir d0/d9/d11/da4 0 2024-03-29T04:45:02.740 INFO:tasks.workunit.client.0.smithi142.stdout:2/523: symlink de/d11/d17/lb0 0 2024-03-29T04:45:02.743 INFO:tasks.workunit.client.0.smithi142.stdout:6/541: rename d1c/d33/d46/f53 to d1c/d90/fa3 0 2024-03-29T04:45:02.744 INFO:tasks.workunit.client.0.smithi142.stdout:5/670: symlink df/d1a/d28/lda 0 2024-03-29T04:45:02.745 INFO:tasks.workunit.client.0.smithi142.stdout:5/671: chown df/d1a/d21/d55/d66/l7d 5484253 1 2024-03-29T04:45:02.745 INFO:tasks.workunit.client.0.smithi142.stdout:0/554: link d7/d13/d67/d72/l5d d7/d13/d67/d72/d37/da4/lb6 0 2024-03-29T04:45:02.746 INFO:tasks.workunit.client.0.smithi142.stdout:6/542: mkdir d1c/d33/d46/d8b/da4 0 2024-03-29T04:45:02.750 INFO:tasks.workunit.client.0.smithi142.stdout:5/672: stat df/d16/d25 0 2024-03-29T04:45:02.752 INFO:tasks.workunit.client.0.smithi142.stdout:5/673: dread - df/d1a/d28/fae zero size 2024-03-29T04:45:02.752 INFO:tasks.workunit.client.0.smithi142.stdout:2/524: dwrite de/d11/d1e/d22/f8e [0,4194304] 0 2024-03-29T04:45:02.753 INFO:tasks.workunit.client.0.smithi142.stdout:5/674: chown df/d16/d24/d3d/fc8 72 1 2024-03-29T04:45:02.754 INFO:tasks.workunit.client.0.smithi142.stdout:6/543: creat d1c/d45/fa5 x:0 0 0 2024-03-29T04:45:02.756 INFO:tasks.workunit.client.0.smithi142.stdout:5/675: write df/d16/d24/d2e/f8f [665163,122711] 0 2024-03-29T04:45:02.758 INFO:tasks.workunit.client.0.smithi142.stdout:2/525: truncate de/d11/d1e/d22/f26 1070868 0 2024-03-29T04:45:02.759 INFO:tasks.workunit.client.0.smithi142.stdout:5/676: write df/d16/d24/d3d/fc8 [256951,72146] 0 2024-03-29T04:45:02.762 INFO:tasks.workunit.client.0.smithi142.stdout:0/555: truncate d7/d9/fa 1389481 0 2024-03-29T04:45:02.766 INFO:tasks.workunit.client.0.smithi142.stdout:6/544: dwrite d1c/d45/d51/f97 [0,4194304] 0 2024-03-29T04:45:02.772 INFO:tasks.workunit.client.0.smithi142.stdout:5/677: rename df/d1a/d21/d55/d66/l84 to df/d16/d24/d5c/db5/ldb 0 2024-03-29T04:45:02.773 INFO:tasks.workunit.client.0.smithi142.stdout:0/556: mknod d7/d13/d4a/d70/d90/cb7 0 2024-03-29T04:45:02.774 INFO:tasks.workunit.client.0.smithi142.stdout:4/556: dwrite d0/dc/d15/d3d/d65/f85 [0,4194304] 0 2024-03-29T04:45:02.774 INFO:tasks.workunit.client.0.smithi142.stdout:4/557: chown d0 69 1 2024-03-29T04:45:02.776 INFO:tasks.workunit.client.0.smithi142.stdout:5/678: dread - df/d16/d24/d2e/d47/fd1 zero size 2024-03-29T04:45:02.776 INFO:tasks.workunit.client.0.smithi142.stdout:6/545: mknod d1c/d45/ca6 0 2024-03-29T04:45:02.776 INFO:tasks.workunit.client.0.smithi142.stdout:2/526: link de/f29 de/d11/d23/fb1 0 2024-03-29T04:45:02.783 INFO:tasks.workunit.client.0.smithi142.stdout:6/546: symlink d1c/d90/d94/la7 0 2024-03-29T04:45:02.784 INFO:tasks.workunit.client.0.smithi142.stdout:2/527: fdatasync de/d11/d23/f90 0 2024-03-29T04:45:02.786 INFO:tasks.workunit.client.0.smithi142.stdout:4/558: rename d0/d9/db2/dba/dbc/d6d/l9d to d0/d9/d11/lc2 0 2024-03-29T04:45:02.788 INFO:tasks.workunit.client.0.smithi142.stdout:0/557: getdents d7/d9/d1e 0 2024-03-29T04:45:02.789 INFO:tasks.workunit.client.0.smithi142.stdout:4/559: truncate d0/d9/db2/fb3 416727 0 2024-03-29T04:45:02.790 INFO:tasks.workunit.client.0.smithi142.stdout:6/547: rename d1c/f25 to d1c/d33/d46/d61/d9e/fa8 0 2024-03-29T04:45:02.791 INFO:tasks.workunit.client.0.smithi142.stdout:2/528: symlink de/d11/d1e/d4d/d61/lb2 0 2024-03-29T04:45:02.792 INFO:tasks.workunit.client.0.smithi142.stdout:0/558: creat d7/d13/d67/d72/fb8 x:0 0 0 2024-03-29T04:45:02.793 INFO:tasks.workunit.client.0.smithi142.stdout:4/560: getdents d0/dc/d15/d91 0 2024-03-29T04:45:02.793 INFO:tasks.workunit.client.0.smithi142.stdout:2/529: write de/d11/d1e/d4d/d61/d7d/f7e [181854,32862] 0 2024-03-29T04:45:02.797 INFO:tasks.workunit.client.0.smithi142.stdout:0/559: creat d7/d9/fb9 x:0 0 0 2024-03-29T04:45:02.799 INFO:tasks.workunit.client.0.smithi142.stdout:2/530: unlink de/d11/f15 0 2024-03-29T04:45:02.800 INFO:tasks.workunit.client.0.smithi142.stdout:6/548: link d1c/d33/d46/l96 d1c/d33/la9 0 2024-03-29T04:45:02.800 INFO:tasks.workunit.client.0.smithi142.stdout:2/531: chown de/d11/d23 844 1 2024-03-29T04:45:02.801 INFO:tasks.workunit.client.0.smithi142.stdout:5/679: dread df/d16/f26 [0,4194304] 0 2024-03-29T04:45:02.801 INFO:tasks.workunit.client.0.smithi142.stdout:2/532: chown de/d11/d1e/d22/d27/fa9 174039 1 2024-03-29T04:45:02.805 INFO:tasks.workunit.client.0.smithi142.stdout:2/533: mknod de/d11/d17/d5d/cb3 0 2024-03-29T04:45:02.826 INFO:tasks.workunit.client.0.smithi142.stdout:8/577: sync 2024-03-29T04:45:02.826 INFO:tasks.workunit.client.0.smithi142.stdout:6/549: mknod d1c/d33/d46/caa 0 2024-03-29T04:45:02.828 INFO:tasks.workunit.client.0.smithi142.stdout:0/560: creat d7/d9/d54/fba x:0 0 0 2024-03-29T04:45:02.829 INFO:tasks.workunit.client.0.smithi142.stdout:4/561: truncate d0/f48 480875 0 2024-03-29T04:45:02.835 INFO:tasks.workunit.client.0.smithi142.stdout:2/534: mkdir de/d11/d1e/d4d/d61/d7d/db4 0 2024-03-29T04:45:02.835 INFO:tasks.workunit.client.0.smithi142.stdout:5/680: dwrite df/d16/d24/d2e/fab [0,4194304] 0 2024-03-29T04:45:02.837 INFO:tasks.workunit.client.0.smithi142.stdout:4/562: chown d0/dc/f33 461362796 1 2024-03-29T04:45:02.839 INFO:tasks.workunit.client.0.smithi142.stdout:6/550: dwrite d1c/f5d [0,4194304] 0 2024-03-29T04:45:02.841 INFO:tasks.workunit.client.0.smithi142.stdout:0/561: write d7/d13/d67/d72/f3a [2724244,20156] 0 2024-03-29T04:45:02.841 INFO:tasks.workunit.client.0.smithi142.stdout:2/535: mknod de/d11/d1e/d22/d27/d3a/cb5 0 2024-03-29T04:45:02.843 INFO:tasks.workunit.client.0.smithi142.stdout:2/536: write de/d11/d2b/d7b/fa7 [816032,20587] 0 2024-03-29T04:45:02.843 INFO:tasks.workunit.client.0.smithi142.stdout:8/578: creat d5/d1d/d2f/fd0 x:0 0 0 2024-03-29T04:45:02.848 INFO:tasks.workunit.client.0.smithi142.stdout:5/681: mknod df/d16/d24/cdc 0 2024-03-29T04:45:02.851 INFO:tasks.workunit.client.0.smithi142.stdout:0/562: creat d7/d9/fbb x:0 0 0 2024-03-29T04:45:02.852 INFO:tasks.workunit.client.0.smithi142.stdout:0/563: chown d7/d9/fa 16 1 2024-03-29T04:45:02.854 INFO:tasks.workunit.client.0.smithi142.stdout:0/564: stat d7/d9/f3c 0 2024-03-29T04:45:02.854 INFO:tasks.workunit.client.0.smithi142.stdout:0/565: chown d7/d9/d54 4753042 1 2024-03-29T04:45:02.854 INFO:tasks.workunit.client.0.smithi142.stdout:0/566: chown d7/d9/d1e/c3f 67324778 1 2024-03-29T04:45:02.855 INFO:tasks.workunit.client.0.smithi142.stdout:0/567: chown d7/d9/d1e/l9b 1 1 2024-03-29T04:45:02.986 INFO:tasks.workunit.client.0.smithi142.stdout:0/568: dwrite d7/d13/f9c [0,4194304] 0 2024-03-29T04:45:02.987 INFO:tasks.workunit.client.0.smithi142.stdout:4/563: truncate d0/d45/fad 299203 0 2024-03-29T04:45:02.989 INFO:tasks.workunit.client.0.smithi142.stdout:0/569: write d7/d9/d54/f99 [783928,74534] 0 2024-03-29T04:45:02.989 INFO:tasks.workunit.client.0.smithi142.stdout:2/537: truncate de/d11/d1e/d22/d50/f57 1203216 0 2024-03-29T04:45:03.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:02 smithi152 ceph-mon[37177]: pgmap v267: 65 pgs: 65 active+clean; 1.0 GiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 21 MiB/s rd, 121 MiB/s wr, 262 op/s 2024-03-29T04:45:03.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:02 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:45:03.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:02 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:45:03.085 INFO:tasks.workunit.client.0.smithi142.stdout:1/538: sync 2024-03-29T04:45:03.090 INFO:tasks.workunit.client.0.smithi142.stdout:2/538: readlink de/d11/d1e/d4d/d61/l81 0 2024-03-29T04:45:03.093 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:02 smithi142 ceph-mon[29027]: pgmap v267: 65 pgs: 65 active+clean; 1.0 GiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 21 MiB/s rd, 121 MiB/s wr, 262 op/s 2024-03-29T04:45:03.093 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:02 smithi142 ceph-mon[29027]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:45:03.093 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:02 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:45:03.093 INFO:tasks.workunit.client.0.smithi142.stdout:8/579: dwrite d5/d2e/fcc [0,4194304] 0 2024-03-29T04:45:03.110 INFO:tasks.workunit.client.0.smithi142.stdout:2/539: read de/d11/d17/d4a/f55 [3275482,4338] 0 2024-03-29T04:45:03.134 INFO:tasks.workunit.client.0.smithi142.stdout:6/551: dread d1c/d45/f4e [0,4194304] 0 2024-03-29T04:45:03.269 INFO:tasks.workunit.client.0.smithi142.stdout:9/393: sync 2024-03-29T04:45:03.282 INFO:tasks.workunit.client.0.smithi142.stdout:9/394: dwrite d0/d2/d6/d31/f3a [0,4194304] 0 2024-03-29T04:45:03.312 INFO:tasks.workunit.client.0.smithi142.stdout:3/459: sync 2024-03-29T04:45:03.313 INFO:tasks.workunit.client.0.smithi142.stdout:3/460: fdatasync d3/d1f/d24/d47/f4f 0 2024-03-29T04:45:03.376 INFO:tasks.workunit.client.0.smithi142.stdout:5/682: sync 2024-03-29T04:45:03.376 INFO:tasks.workunit.client.0.smithi142.stdout:7/638: sync 2024-03-29T04:45:03.376 INFO:tasks.workunit.client.0.smithi142.stdout:1/539: sync 2024-03-29T04:45:03.379 INFO:tasks.workunit.client.0.smithi142.stdout:7/639: chown d7/d27/d44/db2/db8/d4b/d71/d30/d33/c70 1 1 2024-03-29T04:45:03.778 INFO:tasks.workunit.client.0.smithi142.stdout:0/570: stat d7/d9/d55/f98 0 2024-03-29T04:45:03.778 INFO:tasks.workunit.client.0.smithi142.stdout:6/552: mknod d1c/d90/d94/cab 0 2024-03-29T04:45:03.779 INFO:tasks.workunit.client.0.smithi142.stdout:5/683: fsync df/d16/d24/d5c/f5e 0 2024-03-29T04:45:03.780 INFO:tasks.workunit.client.0.smithi142.stdout:0/571: write d7/d13/d67/d72/d37/f83 [2491984,105410] 0 2024-03-29T04:45:03.781 INFO:tasks.workunit.client.0.smithi142.stdout:3/461: dread d3/d29/d8f/d5f/d64/f76 [0,4194304] 0 2024-03-29T04:45:03.782 INFO:tasks.workunit.client.0.smithi142.stdout:1/540: write d8/db/dc/f2c [1456024,38552] 0 2024-03-29T04:45:03.783 INFO:tasks.workunit.client.0.smithi142.stdout:8/580: link d5/d1d/d2f/d76/d4c/d60/f84 d5/d1d/d24/d66/dce/fd1 0 2024-03-29T04:45:03.784 INFO:tasks.workunit.client.0.smithi142.stdout:9/395: creat d0/d6c/f79 x:0 0 0 2024-03-29T04:45:03.784 INFO:tasks.workunit.client.0.smithi142.stdout:7/640: mknod d7/d27/d44/db2/db8/de/d20/d86/dd4/cd5 0 2024-03-29T04:45:03.785 INFO:tasks.workunit.client.0.smithi142.stdout:0/572: write d7/d13/d67/d72/d37/f73 [1331821,26575] 0 2024-03-29T04:45:03.786 INFO:tasks.workunit.client.0.smithi142.stdout:2/540: read de/d11/d1e/d22/d27/f30 [1157224,95023] 0 2024-03-29T04:45:03.788 INFO:tasks.workunit.client.0.smithi142.stdout:6/553: rename d1c/d33/d46/f8c to d1c/d33/d46/d61/d7e/fac 0 2024-03-29T04:45:03.791 INFO:tasks.workunit.client.0.smithi142.stdout:1/541: mknod d8/db/dc/d77/db9/dbe/d2d/d44/d53/cc1 0 2024-03-29T04:45:03.791 INFO:tasks.workunit.client.0.smithi142.stdout:9/396: chown d0/d12/f59 1003 1 2024-03-29T04:45:03.793 INFO:tasks.workunit.client.0.smithi142.stdout:5/684: creat df/d1a/d21/da5/fdd x:0 0 0 2024-03-29T04:45:03.793 INFO:tasks.workunit.client.0.smithi142.stdout:2/541: read - de/d11/d17/d60/d6f/f95 zero size 2024-03-29T04:45:03.797 INFO:tasks.workunit.client.0.smithi142.stdout:8/581: rename d5/d1d/d2f/d76/d4c to d5/d2e/dd2 0 2024-03-29T04:45:03.798 INFO:tasks.workunit.client.0.smithi142.stdout:0/573: dread d7/d13/f50 [0,4194304] 0 2024-03-29T04:45:03.799 INFO:tasks.workunit.client.0.smithi142.stdout:5/685: chown df/d1a/d21/fcf 1354445 1 2024-03-29T04:45:03.802 INFO:tasks.workunit.client.0.smithi142.stdout:3/462: dread d3/d1f/d24/f37 [0,4194304] 0 2024-03-29T04:45:03.802 INFO:tasks.workunit.client.0.smithi142.stdout:8/582: chown d5/f19 5991 1 2024-03-29T04:45:03.803 INFO:tasks.workunit.client.0.smithi142.stdout:1/542: rmdir d8/db/dc/d77/d5a 39 2024-03-29T04:45:03.804 INFO:tasks.workunit.client.0.smithi142.stdout:5/686: chown df/d16/l65 150139 1 2024-03-29T04:45:03.804 INFO:tasks.workunit.client.0.smithi142.stdout:1/543: fsync f3 0 2024-03-29T04:45:03.805 INFO:tasks.workunit.client.0.smithi142.stdout:7/641: dwrite d7/fd [8388608,4194304] 0 2024-03-29T04:45:03.809 INFO:tasks.workunit.client.0.smithi142.stdout:4/564: dread d0/d9/d11/d97/d9e/fa2 [0,4194304] 0 2024-03-29T04:45:03.810 INFO:tasks.workunit.client.0.smithi142.stdout:0/574: fdatasync d7/d9/ff 0 2024-03-29T04:45:03.811 INFO:tasks.workunit.client.0.smithi142.stdout:6/554: dwrite d1c/d33/d46/d71/f9c [0,4194304] 0 2024-03-29T04:45:03.812 INFO:tasks.workunit.client.0.smithi142.stdout:8/583: fdatasync d5/d2e/dd2/d60/f84 0 2024-03-29T04:45:03.814 INFO:tasks.workunit.client.0.smithi142.stdout:3/463: mknod d3/d29/d2b/d40/ca0 0 2024-03-29T04:45:03.815 INFO:tasks.workunit.client.0.smithi142.stdout:5/687: chown df/d1a/d21/d27/d45/dbf/f9f 6 1 2024-03-29T04:45:03.816 INFO:tasks.workunit.client.0.smithi142.stdout:3/464: chown d3/d29/d2b/d40/d67/d8b/f91 499 1 2024-03-29T04:45:03.818 INFO:tasks.workunit.client.0.smithi142.stdout:9/397: getdents d0 0 2024-03-29T04:45:03.820 INFO:tasks.workunit.client.0.smithi142.stdout:4/565: fdatasync d0/d9/f49 0 2024-03-29T04:45:03.824 INFO:tasks.workunit.client.0.smithi142.stdout:6/555: symlink d1c/d90/lad 0 2024-03-29T04:45:03.826 INFO:tasks.workunit.client.0.smithi142.stdout:7/642: symlink d7/d27/d44/db2/db8/d4b/d71/d30/ld6 0 2024-03-29T04:45:03.828 INFO:tasks.workunit.client.0.smithi142.stdout:2/542: dwrite de/d11/d1e/d22/d27/f30 [0,4194304] 0 2024-03-29T04:45:03.830 INFO:tasks.workunit.client.0.smithi142.stdout:7/643: stat d7/d27/d44/db2/db8/d4b/d71/d30/d33/c9c 0 2024-03-29T04:45:03.832 INFO:tasks.workunit.client.0.smithi142.stdout:3/465: creat d3/d54/fa1 x:0 0 0 2024-03-29T04:45:03.833 INFO:tasks.workunit.client.0.smithi142.stdout:4/566: symlink d0/d9/d11/d84/lc3 0 2024-03-29T04:45:03.839 INFO:tasks.workunit.client.0.smithi142.stdout:9/398: dwrite d0/d12/d50/f63 [0,4194304] 0 2024-03-29T04:45:03.839 INFO:tasks.workunit.client.0.smithi142.stdout:6/556: creat d1c/d33/d46/d61/fae x:0 0 0 2024-03-29T04:45:03.841 INFO:tasks.workunit.client.0.smithi142.stdout:3/466: dread d3/d1f/d24/f69 [0,4194304] 0 2024-03-29T04:45:03.844 INFO:tasks.workunit.client.0.smithi142.stdout:2/543: unlink de/d11/d1e/f80 0 2024-03-29T04:45:03.853 INFO:tasks.workunit.client.0.smithi142.stdout:0/575: dread d7/d13/d67/d72/f3e [0,4194304] 0 2024-03-29T04:45:03.864 INFO:tasks.workunit.client.0.smithi142.stdout:9/399: mknod d0/d2/d6/d27/d3c/c7a 0 2024-03-29T04:45:03.865 INFO:tasks.workunit.client.0.smithi142.stdout:8/584: link d5/d2e/l6f d5/d1d/d2f/d45/dc3/ld3 0 2024-03-29T04:45:03.866 INFO:tasks.workunit.client.0.smithi142.stdout:5/688: dread fe [0,4194304] 0 2024-03-29T04:45:03.867 INFO:tasks.workunit.client.0.smithi142.stdout:8/585: write d5/d2e/dd2/d60/f8f [5273168,44388] 0 2024-03-29T04:45:03.869 INFO:tasks.workunit.client.0.smithi142.stdout:3/467: rmdir d3/d29/d2d/d56 39 2024-03-29T04:45:03.870 INFO:tasks.workunit.client.0.smithi142.stdout:0/576: rmdir d7/d9/d6e 39 2024-03-29T04:45:03.871 INFO:tasks.workunit.client.0.smithi142.stdout:0/577: stat d7/d9/d55/d42/f5f 0 2024-03-29T04:45:03.872 INFO:tasks.workunit.client.0.smithi142.stdout:0/578: chown d7/d9/d54/f99 1662959887 1 2024-03-29T04:45:03.873 INFO:tasks.workunit.client.0.smithi142.stdout:0/579: fdatasync d7/d9/d54/d58/f82 0 2024-03-29T04:45:03.873 INFO:tasks.workunit.client.0.smithi142.stdout:2/544: dread de/d11/d17/f1a [0,4194304] 0 2024-03-29T04:45:03.875 INFO:tasks.workunit.client.0.smithi142.stdout:9/400: chown d0/d2/l3f 77258 1 2024-03-29T04:45:03.878 INFO:tasks.workunit.client.0.smithi142.stdout:4/567: creat d0/d9/db2/dba/dbc/d6d/d6e/fc4 x:0 0 0 2024-03-29T04:45:03.881 INFO:tasks.workunit.client.0.smithi142.stdout:6/557: rename d1c/d33/d46/d61/d9e/fa8 to d1c/d33/d46/d8b/da4/faf 0 2024-03-29T04:45:03.881 INFO:tasks.workunit.client.0.smithi142.stdout:7/644: creat d7/d27/d44/fd7 x:0 0 0 2024-03-29T04:45:03.883 INFO:tasks.workunit.client.0.smithi142.stdout:8/586: creat d5/d2e/dd2/fd4 x:0 0 0 2024-03-29T04:45:03.884 INFO:tasks.workunit.client.0.smithi142.stdout:1/544: getdents d8/db/dc/d77/db9/dbe/d1f/d4c 0 2024-03-29T04:45:03.886 INFO:tasks.workunit.client.0.smithi142.stdout:2/545: mknod de/d11/d17/d60/d6f/cb6 0 2024-03-29T04:45:03.887 INFO:tasks.workunit.client.0.smithi142.stdout:9/401: fdatasync d0/d2/d6/d16/f58 0 2024-03-29T04:45:03.888 INFO:tasks.workunit.client.0.smithi142.stdout:4/568: chown d0/c6 1128 1 2024-03-29T04:45:03.888 INFO:tasks.workunit.client.0.smithi142.stdout:5/689: rename df/d1a/d21/caf to df/d1a/d21/d27/cde 0 2024-03-29T04:45:03.889 INFO:tasks.workunit.client.0.smithi142.stdout:6/558: readlink d1c/d33/la9 0 2024-03-29T04:45:03.890 INFO:tasks.workunit.client.0.smithi142.stdout:7/645: symlink d7/d27/d44/db2/db8/de/ld8 0 2024-03-29T04:45:03.891 INFO:tasks.workunit.client.0.smithi142.stdout:8/587: creat d5/d2e/fd5 x:0 0 0 2024-03-29T04:45:03.891 INFO:tasks.workunit.client.0.smithi142.stdout:5/690: chown df/d29/d5d/d75/d90/fc2 15168 1 2024-03-29T04:45:03.892 INFO:tasks.workunit.client.0.smithi142.stdout:3/468: link d3/d29/d2b/d40/d67/d8b/f91 d3/d1f/d24/d47/d87/fa2 0 2024-03-29T04:45:03.892 INFO:tasks.workunit.client.0.smithi142.stdout:7/646: dread - d7/d27/d44/db2/db8/de/f2d zero size 2024-03-29T04:45:03.893 INFO:tasks.workunit.client.0.smithi142.stdout:1/545: read - d8/db/d37/f79 zero size 2024-03-29T04:45:03.893 INFO:tasks.workunit.client.0.smithi142.stdout:0/580: mkdir d7/dbc 0 2024-03-29T04:45:03.893 INFO:tasks.workunit.client.0.smithi142.stdout:8/588: chown d5/d1d/d2f/fd0 147 1 2024-03-29T04:45:03.894 INFO:tasks.workunit.client.0.smithi142.stdout:9/402: dread d0/d2/d6/d27/d3c/d4f/f56 [0,4194304] 0 2024-03-29T04:45:03.894 INFO:tasks.workunit.client.0.smithi142.stdout:2/546: creat de/d11/d23/fb7 x:0 0 0 2024-03-29T04:45:03.895 INFO:tasks.workunit.client.0.smithi142.stdout:1/546: chown d8/db/d5e 15665231 1 2024-03-29T04:45:03.897 INFO:tasks.workunit.client.0.smithi142.stdout:8/589: chown d5/d1d/d24/ca8 308617 1 2024-03-29T04:45:03.898 INFO:tasks.workunit.client.0.smithi142.stdout:4/569: rename d0/d12/c1d to d0/d9/db2/cc5 0 2024-03-29T04:45:03.898 INFO:tasks.workunit.client.0.smithi142.stdout:0/581: truncate d7/d9/fbb 1005015 0 2024-03-29T04:45:03.898 INFO:tasks.workunit.client.0.smithi142.stdout:7/647: write d7/d27/d44/db2/db8/de/d20/fd3 [742133,57761] 0 2024-03-29T04:45:03.901 INFO:tasks.workunit.client.0.smithi142.stdout:9/403: mkdir d0/d2/d6/d27/d3c/d7b 0 2024-03-29T04:45:03.903 INFO:tasks.workunit.client.0.smithi142.stdout:2/547: mknod de/d11/d1e/d22/d27/d3a/cb8 0 2024-03-29T04:45:03.904 INFO:tasks.workunit.client.0.smithi142.stdout:1/547: chown d8/db/dc/d77/db9/dbe/d1f/d50/f87 1140718 1 2024-03-29T04:45:03.906 INFO:tasks.workunit.client.0.smithi142.stdout:4/570: mkdir d0/d9/db2/dc6 0 2024-03-29T04:45:03.907 INFO:tasks.workunit.client.0.smithi142.stdout:5/691: dwrite df/d16/d24/d3d/fa6 [0,4194304] 0 2024-03-29T04:45:03.909 INFO:tasks.workunit.client.0.smithi142.stdout:1/548: write d8/db/d37/f76 [112606,34644] 0 2024-03-29T04:45:03.909 INFO:tasks.workunit.client.0.smithi142.stdout:7/648: mkdir d7/d27/d44/db2/db8/d4b/d71/d30/dd9 0 2024-03-29T04:45:03.913 INFO:tasks.workunit.client.0.smithi142.stdout:8/590: dwrite d5/d1d/d2f/d76/d2d/f4f [4194304,4194304] 0 2024-03-29T04:45:03.915 INFO:tasks.workunit.client.0.smithi142.stdout:8/591: write d5/d1d/d2f/d76/f41 [7611681,109213] 0 2024-03-29T04:45:03.918 INFO:tasks.workunit.client.0.smithi142.stdout:9/404: rename d0/d2/l3f to d0/d2/d6/d27/d3c/d7b/l7c 0 2024-03-29T04:45:03.920 INFO:tasks.workunit.client.0.smithi142.stdout:4/571: mknod d0/d9/d11/cc7 0 2024-03-29T04:45:03.921 INFO:tasks.workunit.client.0.smithi142.stdout:4/572: chown d0/c6 219082 1 2024-03-29T04:45:03.924 INFO:tasks.workunit.client.0.smithi142.stdout:8/592: dwrite d5/f1f [0,4194304] 0 2024-03-29T04:45:03.925 INFO:tasks.workunit.client.0.smithi142.stdout:2/548: truncate de/d11/d3d/f82 2272221 0 2024-03-29T04:45:03.926 INFO:tasks.workunit.client.0.smithi142.stdout:5/692: readlink df/d1a/d21/d27/d45/dbf/l53 0 2024-03-29T04:45:03.932 INFO:tasks.workunit.client.0.smithi142.stdout:2/549: rmdir de/d11/d3d 39 2024-03-29T04:45:03.934 INFO:tasks.workunit.client.0.smithi142.stdout:1/549: creat d8/db/dc/d77/db9/dbe/d15/d9c/fc2 x:0 0 0 2024-03-29T04:45:03.934 INFO:tasks.workunit.client.0.smithi142.stdout:8/593: dwrite d5/d1d/d2f/d5f/fa6 [0,4194304] 0 2024-03-29T04:45:03.936 INFO:tasks.workunit.client.0.smithi142.stdout:7/649: link d7/d27/d44/fd7 d7/d27/d44/dab/fda 0 2024-03-29T04:45:03.936 INFO:tasks.workunit.client.0.smithi142.stdout:8/594: truncate d5/fc5 411940 0 2024-03-29T04:45:03.937 INFO:tasks.workunit.client.0.smithi142.stdout:5/693: symlink df/d16/d24/d5c/ldf 0 2024-03-29T04:45:03.938 INFO:tasks.workunit.client.0.smithi142.stdout:7/650: write d7/f90 [843865,34196] 0 2024-03-29T04:45:03.940 INFO:tasks.workunit.client.0.smithi142.stdout:5/694: mkdir df/d16/d24/d2e/d47/d8c/de0 0 2024-03-29T04:45:03.942 INFO:tasks.workunit.client.0.smithi142.stdout:5/695: write df/d1a/d28/f8d [134739,107693] 0 2024-03-29T04:45:03.946 INFO:tasks.workunit.client.0.smithi142.stdout:1/550: truncate d8/db/dc/d77/db9/dbe/f13 3049000 0 2024-03-29T04:45:03.948 INFO:tasks.workunit.client.0.smithi142.stdout:7/651: unlink d7/d27/d44/db2/db8/d4b/d71/d30/f46 0 2024-03-29T04:45:03.951 INFO:tasks.workunit.client.0.smithi142.stdout:1/551: truncate d8/db/dc/d77/db9/dbe/d2d/d44/d7f/fb0 66285 0 2024-03-29T04:45:03.952 INFO:tasks.workunit.client.0.smithi142.stdout:2/550: creat de/d11/d1e/fb9 x:0 0 0 2024-03-29T04:45:03.954 INFO:tasks.workunit.client.0.smithi142.stdout:4/573: dread d0/dc/d15/f53 [0,4194304] 0 2024-03-29T04:45:03.955 INFO:tasks.workunit.client.0.smithi142.stdout:0/582: truncate d7/d13/d67/d72/d37/f39 284984 0 2024-03-29T04:45:03.957 INFO:tasks.workunit.client.0.smithi142.stdout:4/574: write d0/d45/f61 [46492,67794] 0 2024-03-29T04:45:03.957 INFO:tasks.workunit.client.0.smithi142.stdout:5/696: symlink df/d29/d7a/d97/le1 0 2024-03-29T04:45:03.961 INFO:tasks.workunit.client.0.smithi142.stdout:5/697: write df/d1a/f2f [2081217,31162] 0 2024-03-29T04:45:03.961 INFO:tasks.workunit.client.0.smithi142.stdout:1/552: dwrite d8/db/dc/d19/f5c [0,4194304] 0 2024-03-29T04:45:03.963 INFO:tasks.workunit.client.0.smithi142.stdout:7/652: mkdir d7/d27/d44/db2/db8/de/d20/d86/dd4/ddb 0 2024-03-29T04:45:03.966 INFO:tasks.workunit.client.0.smithi142.stdout:8/595: write d5/d1d/d2f/d76/d9c/f7e [36633,54017] 0 2024-03-29T04:45:03.970 INFO:tasks.workunit.client.0.smithi142.stdout:0/583: dwrite d7/d9/d54/f65 [0,4194304] 0 2024-03-29T04:45:03.972 INFO:tasks.workunit.client.0.smithi142.stdout:2/551: rmdir de/d11/d17 39 2024-03-29T04:45:03.972 INFO:tasks.workunit.client.0.smithi142.stdout:0/584: chown d7/d13/d67/d72/c4b 50432 1 2024-03-29T04:45:03.976 INFO:tasks.workunit.client.0.smithi142.stdout:4/575: unlink d0/d45/c52 0 2024-03-29T04:45:03.982 INFO:tasks.workunit.client.0.smithi142.stdout:5/698: mkdir df/d29/d5d/d75/de2 0 2024-03-29T04:45:03.987 INFO:tasks.workunit.client.0.smithi142.stdout:1/553: creat d8/db/dc/d77/db9/dbe/d1f/d50/fc3 x:0 0 0 2024-03-29T04:45:03.988 INFO:tasks.workunit.client.0.smithi142.stdout:8/596: creat d5/d2e/dd2/d60/fd6 x:0 0 0 2024-03-29T04:45:03.990 INFO:tasks.workunit.client.0.smithi142.stdout:2/552: dread - de/d11/d17/d5d/f86 zero size 2024-03-29T04:45:03.991 INFO:tasks.workunit.client.0.smithi142.stdout:4/576: creat d0/d12/d28/d47/d7a/fc8 x:0 0 0 2024-03-29T04:45:03.991 INFO:tasks.workunit.client.0.smithi142.stdout:9/405: symlink d0/d2/d6/d16/d6a/l7d 0 2024-03-29T04:45:03.993 INFO:tasks.workunit.client.0.smithi142.stdout:5/699: dwrite df/d16/d24/d2e/d47/d8c/fba [0,4194304] 0 2024-03-29T04:45:03.996 INFO:tasks.workunit.client.0.smithi142.stdout:1/554: creat d8/db/dc/d77/db9/dbe/d1f/d4c/fc4 x:0 0 0 2024-03-29T04:45:03.997 INFO:tasks.workunit.client.0.smithi142.stdout:9/406: write d0/d2/d6/d27/d3c/f4c [366521,98693] 0 2024-03-29T04:45:03.997 INFO:tasks.workunit.client.0.smithi142.stdout:2/553: write de/d11/d1e/d4d/d61/f78 [1665158,53644] 0 2024-03-29T04:45:03.999 INFO:tasks.workunit.client.0.smithi142.stdout:5/700: readlink df/d1a/d21/l72 0 2024-03-29T04:45:04.000 INFO:tasks.workunit.client.0.smithi142.stdout:9/407: chown d0/d2/d6/d31 4867128 1 2024-03-29T04:45:04.001 INFO:tasks.workunit.client.0.smithi142.stdout:2/554: chown de/d11/d1e 43125 1 2024-03-29T04:45:04.002 INFO:tasks.workunit.client.0.smithi142.stdout:0/585: rename d7/d9/d55/l4e to d7/d13/lbd 0 2024-03-29T04:45:04.005 INFO:tasks.workunit.client.0.smithi142.stdout:1/555: write d8/db/dc/d77/db9/dbe/d15/f60 [3154351,53949] 0 2024-03-29T04:45:04.005 INFO:tasks.workunit.client.0.smithi142.stdout:9/408: mknod d0/d2/d6/d27/d3c/d7b/c7e 0 2024-03-29T04:45:04.008 INFO:tasks.workunit.client.0.smithi142.stdout:4/577: rename d0/d12/d28 to d0/d9/d11/d4c/d5f/d81/dc9 0 2024-03-29T04:45:04.010 INFO:tasks.workunit.client.0.smithi142.stdout:1/556: unlink d8/db/dc/d19/f5c 0 2024-03-29T04:45:04.016 INFO:tasks.workunit.client.0.smithi142.stdout:1/557: chown d8/db/c62 2135256704 1 2024-03-29T04:45:04.016 INFO:tasks.workunit.client.0.smithi142.stdout:7/653: dwrite d7/d27/d44/db2/db8/de/d20/f4e [0,4194304] 0 2024-03-29T04:45:04.016 INFO:tasks.workunit.client.0.smithi142.stdout:4/578: read d0/d9/d19/d2c/fb9 [2089478,128408] 0 2024-03-29T04:45:04.021 INFO:tasks.workunit.client.0.smithi142.stdout:0/586: dwrite d7/d9/d55/f87 [0,4194304] 0 2024-03-29T04:45:04.022 INFO:tasks.workunit.client.0.smithi142.stdout:4/579: dread - d0/dc/d15/f72 zero size 2024-03-29T04:45:04.023 INFO:tasks.workunit.client.0.smithi142.stdout:8/597: dread d5/d1d/d2f/d76/d2d/f73 [0,4194304] 0 2024-03-29T04:45:04.026 INFO:tasks.workunit.client.0.smithi142.stdout:2/555: rename de/d11/d1e/d4d to de/d11/d1e/d22/d27/dba 0 2024-03-29T04:45:04.026 INFO:tasks.workunit.client.0.smithi142.stdout:9/409: dwrite d0/d12/f59 [0,4194304] 0 2024-03-29T04:45:04.026 INFO:tasks.workunit.client.0.smithi142.stdout:4/580: chown d0/d9/d19/d2c/fb9 21165122 1 2024-03-29T04:45:04.033 INFO:tasks.workunit.client.0.smithi142.stdout:1/558: dread d8/db/dc/d77/db9/dbe/d2d/d44/d7f/fb0 [0,4194304] 0 2024-03-29T04:45:04.034 INFO:tasks.workunit.client.0.smithi142.stdout:7/654: creat d7/d27/d44/db2/db8/d89/fdc x:0 0 0 2024-03-29T04:45:04.035 INFO:tasks.workunit.client.0.smithi142.stdout:4/581: creat d0/d9/d11/d84/fca x:0 0 0 2024-03-29T04:45:04.037 INFO:tasks.workunit.client.0.smithi142.stdout:2/556: dread - de/d11/d17/d60/f6d zero size 2024-03-29T04:45:04.041 INFO:tasks.workunit.client.0.smithi142.stdout:8/598: dread d5/d1d/d24/f2b [0,4194304] 0 2024-03-29T04:45:04.042 INFO:tasks.workunit.client.0.smithi142.stdout:1/559: rmdir d8/db/dc/d77/db9/dbe 39 2024-03-29T04:45:04.043 INFO:tasks.workunit.client.0.smithi142.stdout:9/410: rename d0/d2/c73 to d0/d2/d6/d27/d3c/c7f 0 2024-03-29T04:45:04.044 INFO:tasks.workunit.client.0.smithi142.stdout:5/701: dread df/d16/d24/d2e/f6c [0,4194304] 0 2024-03-29T04:45:04.047 INFO:tasks.workunit.client.0.smithi142.stdout:0/587: dwrite d7/d9/d55/d96/f97 [0,4194304] 0 2024-03-29T04:45:04.050 INFO:tasks.workunit.client.0.smithi142.stdout:7/655: dread d7/d27/d44/db2/db8/d4b/f4f [0,4194304] 0 2024-03-29T04:45:04.050 INFO:tasks.workunit.client.0.smithi142.stdout:8/599: read d5/d1d/d59/f77 [3711853,123343] 0 2024-03-29T04:45:04.054 INFO:tasks.workunit.client.0.smithi142.stdout:0/588: unlink d7/d9/d54/fba 0 2024-03-29T04:45:04.055 INFO:tasks.workunit.client.0.smithi142.stdout:5/702: link df/d29/f6b df/d16/d24/d2e/d47/d8c/de0/fe3 0 2024-03-29T04:45:04.056 INFO:tasks.workunit.client.0.smithi142.stdout:2/557: rmdir de/d11/d1e/d22 39 2024-03-29T04:45:04.058 INFO:tasks.workunit.client.0.smithi142.stdout:1/560: dwrite d8/db/d5e/f8d [0,4194304] 0 2024-03-29T04:45:04.058 INFO:tasks.workunit.client.0.smithi142.stdout:5/703: fsync df/d1a/d21/d27/d45/fac 0 2024-03-29T04:45:04.060 INFO:tasks.workunit.client.0.smithi142.stdout:7/656: unlink d7/d27/d44/d4c/l56 0 2024-03-29T04:45:04.061 INFO:tasks.workunit.client.0.smithi142.stdout:8/600: creat d5/d2e/dd2/fd7 x:0 0 0 2024-03-29T04:45:04.063 INFO:tasks.workunit.client.0.smithi142.stdout:8/601: chown d5/d1d/d24/d66/dce 9715431 1 2024-03-29T04:45:04.065 INFO:tasks.workunit.client.0.smithi142.stdout:0/589: symlink d7/d9/d55/d96/lbe 0 2024-03-29T04:45:04.069 INFO:tasks.workunit.client.0.smithi142.stdout:7/657: dwrite d7/d27/d44/db2/db8/de/d20/f4e [0,4194304] 0 2024-03-29T04:45:04.073 INFO:tasks.workunit.client.0.smithi142.stdout:5/704: creat df/d16/d24/d2e/d47/d4d/fe4 x:0 0 0 2024-03-29T04:45:04.074 INFO:tasks.workunit.client.0.smithi142.stdout:4/582: dread d0/f21 [0,4194304] 0 2024-03-29T04:45:04.076 INFO:tasks.workunit.client.0.smithi142.stdout:5/705: write df/d1a/d21/d55/f81 [1800087,49265] 0 2024-03-29T04:45:04.080 INFO:tasks.workunit.client.0.smithi142.stdout:7/658: dread d7/d27/d44/db2/db8/de/d20/f4e [0,4194304] 0 2024-03-29T04:45:04.081 INFO:tasks.workunit.client.0.smithi142.stdout:1/561: read - d8/db/dc/d77/db9/dbe/d1f/d4c/fc4 zero size 2024-03-29T04:45:04.081 INFO:tasks.workunit.client.0.smithi142.stdout:5/706: fdatasync df/d1a/d21/d55/d66/fcd 0 2024-03-29T04:45:04.081 INFO:tasks.workunit.client.0.smithi142.stdout:9/411: truncate d0/d2/d6/d16/f66 5164530 0 2024-03-29T04:45:04.082 INFO:tasks.workunit.client.0.smithi142.stdout:0/590: read d7/f2b [3226971,88182] 0 2024-03-29T04:45:04.087 INFO:tasks.workunit.client.0.smithi142.stdout:8/602: dread d5/d2e/dd2/f8b [0,4194304] 0 2024-03-29T04:45:04.088 INFO:tasks.workunit.client.0.smithi142.stdout:4/583: write d0/dc/f68 [441224,115926] 0 2024-03-29T04:45:04.091 INFO:tasks.workunit.client.0.smithi142.stdout:9/412: dwrite d0/f22 [4194304,4194304] 0 2024-03-29T04:45:04.092 INFO:tasks.workunit.client.0.smithi142.stdout:0/591: write d7/f21 [1758914,95078] 0 2024-03-29T04:45:04.093 INFO:tasks.workunit.client.0.smithi142.stdout:1/562: creat d8/db/dc/d77/db9/dbe/fc5 x:0 0 0 2024-03-29T04:45:04.094 INFO:tasks.workunit.client.0.smithi142.stdout:7/659: write d7/d27/d3a/f5b [1015098,97800] 0 2024-03-29T04:45:04.096 INFO:tasks.workunit.client.0.smithi142.stdout:4/584: symlink d0/d9/db2/dba/lcb 0 2024-03-29T04:45:04.101 INFO:tasks.workunit.client.0.smithi142.stdout:5/707: link df/d29/d7a/fc4 df/d2c/fe5 0 2024-03-29T04:45:04.101 INFO:tasks.workunit.client.0.smithi142.stdout:1/563: dwrite f5 [0,4194304] 0 2024-03-29T04:45:04.102 INFO:tasks.workunit.client.0.smithi142.stdout:5/708: write df/d1a/d21/d27/d45/dbf/f61 [103195,98819] 0 2024-03-29T04:45:04.105 INFO:tasks.workunit.client.0.smithi142.stdout:0/592: rmdir d7/d13 39 2024-03-29T04:45:04.106 INFO:tasks.workunit.client.0.smithi142.stdout:2/558: getdents de/d11/d1e/d22 0 2024-03-29T04:45:04.107 INFO:tasks.workunit.client.0.smithi142.stdout:7/660: symlink d7/d62/ldd 0 2024-03-29T04:45:04.109 INFO:tasks.workunit.client.0.smithi142.stdout:7/661: chown d7/d27/d44/db2/db8/de/d20/l7e 957457406 1 2024-03-29T04:45:04.111 INFO:tasks.workunit.client.0.smithi142.stdout:5/709: creat df/d2c/fe6 x:0 0 0 2024-03-29T04:45:04.115 INFO:tasks.workunit.client.0.smithi142.stdout:2/559: mknod de/d11/d17/d60/d6f/cbb 0 2024-03-29T04:45:04.117 INFO:tasks.workunit.client.0.smithi142.stdout:7/662: mknod d7/d27/d3a/d92/cde 0 2024-03-29T04:45:04.118 INFO:tasks.workunit.client.0.smithi142.stdout:9/413: creat d0/d2/d61/d64/f80 x:0 0 0 2024-03-29T04:45:04.120 INFO:tasks.workunit.client.0.smithi142.stdout:0/593: dwrite d7/d13/d67/f8e [0,4194304] 0 2024-03-29T04:45:04.121 INFO:tasks.workunit.client.0.smithi142.stdout:5/710: fdatasync df/d2c/fe5 0 2024-03-29T04:45:04.122 INFO:tasks.workunit.client.0.smithi142.stdout:8/603: getdents d5/d1d/d24/db4 0 2024-03-29T04:45:04.124 INFO:tasks.workunit.client.0.smithi142.stdout:4/585: write d0/d9/d11/f38 [1607221,9067] 0 2024-03-29T04:45:04.126 INFO:tasks.workunit.client.0.smithi142.stdout:7/663: mkdir d7/d27/d44/db2/db8/de/d20/ddf 0 2024-03-29T04:45:04.127 INFO:tasks.workunit.client.0.smithi142.stdout:7/664: read - d7/d27/d44/db2/db8/d4b/d71/f9d zero size 2024-03-29T04:45:04.129 INFO:tasks.workunit.client.0.smithi142.stdout:4/586: dread d0/dc/f68 [0,4194304] 0 2024-03-29T04:45:04.129 INFO:tasks.workunit.client.0.smithi142.stdout:1/564: rename d8/db/dc/d77/d5a/f75 to d8/db/dc/d77/d5a/fc6 0 2024-03-29T04:45:04.130 INFO:tasks.workunit.client.0.smithi142.stdout:9/414: write d0/d2/d6/d31/f49 [4213294,127096] 0 2024-03-29T04:45:04.132 INFO:tasks.workunit.client.0.smithi142.stdout:5/711: rename df/d2c/fb9 to df/d1a/d21/d27/fe7 0 2024-03-29T04:45:04.133 INFO:tasks.workunit.client.0.smithi142.stdout:2/560: creat de/d11/d3d/fbc x:0 0 0 2024-03-29T04:45:04.135 INFO:tasks.workunit.client.0.smithi142.stdout:7/665: mkdir d7/d27/d44/db2/db8/de/dc1/de0 0 2024-03-29T04:45:04.137 INFO:tasks.workunit.client.0.smithi142.stdout:1/565: symlink d8/db/dc/d77/db9/dbe/d2d/d44/d7f/lc7 0 2024-03-29T04:45:04.137 INFO:tasks.workunit.client.0.smithi142.stdout:9/415: dread d0/f32 [0,4194304] 0 2024-03-29T04:45:04.139 INFO:tasks.workunit.client.0.smithi142.stdout:9/416: fsync d0/d12/f54 0 2024-03-29T04:45:04.139 INFO:tasks.workunit.client.0.smithi142.stdout:5/712: symlink df/d16/le8 0 2024-03-29T04:45:04.141 INFO:tasks.workunit.client.0.smithi142.stdout:0/594: rename d7/d13/d67 to d7/d13/dbf 0 2024-03-29T04:45:04.141 INFO:tasks.workunit.client.0.smithi142.stdout:8/604: creat d5/d1d/fd8 x:0 0 0 2024-03-29T04:45:04.143 INFO:tasks.workunit.client.0.smithi142.stdout:5/713: fsync df/d1a/d28/f2d 0 2024-03-29T04:45:04.144 INFO:tasks.workunit.client.0.smithi142.stdout:0/595: mknod d7/d9/d55/d96/cc0 0 2024-03-29T04:45:04.145 INFO:tasks.workunit.client.0.smithi142.stdout:8/605: rmdir d5/d1d/d24/d66 39 2024-03-29T04:45:04.145 INFO:tasks.workunit.client.0.smithi142.stdout:5/714: truncate df/d1a/d28/fae 285512 0 2024-03-29T04:45:04.146 INFO:tasks.workunit.client.0.smithi142.stdout:5/715: readlink df/d16/le8 0 2024-03-29T04:45:04.149 INFO:tasks.workunit.client.0.smithi142.stdout:0/596: fsync d7/f2b 0 2024-03-29T04:45:04.150 INFO:tasks.workunit.client.0.smithi142.stdout:5/716: dread df/d16/d24/fa0 [0,4194304] 0 2024-03-29T04:45:04.155 INFO:tasks.workunit.client.0.smithi142.stdout:8/606: dread d5/d93/fb8 [0,4194304] 0 2024-03-29T04:45:04.157 INFO:tasks.workunit.client.0.smithi142.stdout:7/666: dread d7/d27/d44/db2/db8/de/f34 [0,4194304] 0 2024-03-29T04:45:04.157 INFO:tasks.workunit.client.0.smithi142.stdout:1/566: truncate d8/db/dc/d77/d5a/fc6 3625702 0 2024-03-29T04:45:04.159 INFO:tasks.workunit.client.0.smithi142.stdout:1/567: dread - d8/db/dc/d77/db9/dbe/d15/fa0 zero size 2024-03-29T04:45:04.159 INFO:tasks.workunit.client.0.smithi142.stdout:0/597: mkdir d7/d9/d55/d96/dab/dc1 0 2024-03-29T04:45:04.161 INFO:tasks.workunit.client.0.smithi142.stdout:8/607: rmdir d5/d1d/d2f/d45/dc3 39 2024-03-29T04:45:04.164 INFO:tasks.workunit.client.0.smithi142.stdout:5/717: getdents df/d29/d6e 0 2024-03-29T04:45:04.167 INFO:tasks.workunit.client.0.smithi142.stdout:7/667: dwrite d7/d27/d44/db2/db8/de/d20/f98 [0,4194304] 0 2024-03-29T04:45:04.169 INFO:tasks.workunit.client.0.smithi142.stdout:8/608: mkdir d5/d1d/dd9 0 2024-03-29T04:45:04.171 INFO:tasks.workunit.client.0.smithi142.stdout:5/718: write df/d1a/d21/d27/fe7 [613569,82799] 0 2024-03-29T04:45:04.172 INFO:tasks.workunit.client.0.smithi142.stdout:0/598: link d7/d13/dbf/d72/d37/f56 d7/d9/d1e/d81/fc2 0 2024-03-29T04:45:04.173 INFO:tasks.workunit.client.0.smithi142.stdout:1/568: dwrite d8/db/dc/d77/d31/f3f [0,4194304] 0 2024-03-29T04:45:04.176 INFO:tasks.workunit.client.0.smithi142.stdout:5/719: mkdir df/d2c/de9 0 2024-03-29T04:45:04.180 INFO:tasks.workunit.client.0.smithi142.stdout:7/668: creat d7/d27/d44/db2/db8/de/d20/ddf/fe1 x:0 0 0 2024-03-29T04:45:04.182 INFO:tasks.workunit.client.0.smithi142.stdout:1/569: mknod d8/db/dc/d77/db9/dbe/d2d/d44/d80/cc8 0 2024-03-29T04:45:04.189 INFO:tasks.workunit.client.0.smithi142.stdout:0/599: dwrite d7/d13/dbf/f92 [0,4194304] 0 2024-03-29T04:45:04.191 INFO:tasks.workunit.client.0.smithi142.stdout:7/669: fdatasync d7/d27/d44/db2/db8/d4b/d71/d30/d33/f77 0 2024-03-29T04:45:04.194 INFO:tasks.workunit.client.0.smithi142.stdout:8/609: dwrite d5/d93/fb8 [0,4194304] 0 2024-03-29T04:45:04.196 INFO:tasks.workunit.client.0.smithi142.stdout:1/570: write d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/fbb [10465,42722] 0 2024-03-29T04:45:04.201 INFO:tasks.workunit.client.0.smithi142.stdout:5/720: dwrite df/d16/d24/d5c/db1/fc3 [4194304,4194304] 0 2024-03-29T04:45:04.206 INFO:tasks.workunit.client.0.smithi142.stdout:7/670: rename d7/l13 to d7/d27/d44/db2/dd2/le2 0 2024-03-29T04:45:04.206 INFO:tasks.workunit.client.0.smithi142.stdout:5/721: mkdir df/d29/d7a/d97/db3/dea 0 2024-03-29T04:45:04.208 INFO:tasks.workunit.client.0.smithi142.stdout:1/571: mknod d8/db/dc/d77/db9/dbe/d1f/d97/cc9 0 2024-03-29T04:45:04.209 INFO:tasks.workunit.client.0.smithi142.stdout:7/671: creat d7/d27/d44/db2/db8/d4b/d71/fe3 x:0 0 0 2024-03-29T04:45:04.211 INFO:tasks.workunit.client.0.smithi142.stdout:5/722: creat df/d29/d5d/d75/d90/feb x:0 0 0 2024-03-29T04:45:04.212 INFO:tasks.workunit.client.0.smithi142.stdout:0/600: dread d7/d9/d55/f57 [0,4194304] 0 2024-03-29T04:45:04.214 INFO:tasks.workunit.client.0.smithi142.stdout:7/672: dread d7/d27/d44/d4c/f8c [0,4194304] 0 2024-03-29T04:45:04.218 INFO:tasks.workunit.client.0.smithi142.stdout:1/572: dwrite d8/db/dc/d77/db9/dbe/d2d/d44/d53/f6a [0,4194304] 0 2024-03-29T04:45:04.220 INFO:tasks.workunit.client.0.smithi142.stdout:1/573: fsync d8/db/d37/f94 0 2024-03-29T04:45:04.220 INFO:tasks.workunit.client.0.smithi142.stdout:8/610: write d5/f22 [9598952,31300] 0 2024-03-29T04:45:04.226 INFO:tasks.workunit.client.0.smithi142.stdout:7/673: dwrite d7/d27/d44/db2/db8/de/f5c [0,4194304] 0 2024-03-29T04:45:04.229 INFO:tasks.workunit.client.0.smithi142.stdout:5/723: write df/d2c/fe5 [660211,98558] 0 2024-03-29T04:45:04.232 INFO:tasks.workunit.client.0.smithi142.stdout:8/611: read d5/d1d/d2f/d76/d2d/f3b [350496,103367] 0 2024-03-29T04:45:04.233 INFO:tasks.workunit.client.0.smithi142.stdout:1/574: dwrite d8/db/d37/f41 [0,4194304] 0 2024-03-29T04:45:04.235 INFO:tasks.workunit.client.0.smithi142.stdout:7/674: symlink d7/d27/d44/db2/db8/d4b/le4 0 2024-03-29T04:45:04.236 INFO:tasks.workunit.client.0.smithi142.stdout:5/724: symlink df/d29/d5d/d75/d90/lec 0 2024-03-29T04:45:04.240 INFO:tasks.workunit.client.0.smithi142.stdout:7/675: fdatasync d7/d27/d44/d4c/f8c 0 2024-03-29T04:45:04.241 INFO:tasks.workunit.client.0.smithi142.stdout:8/612: creat d5/d1d/d2f/d76/d3c/d70/fda x:0 0 0 2024-03-29T04:45:04.242 INFO:tasks.workunit.client.0.smithi142.stdout:1/575: creat d8/db/dc/d77/db9/dbe/d2d/d44/d53/fca x:0 0 0 2024-03-29T04:45:04.242 INFO:tasks.workunit.client.0.smithi142.stdout:8/613: fdatasync d5/d1d/d2f/d45/fcf 0 2024-03-29T04:45:04.244 INFO:tasks.workunit.client.0.smithi142.stdout:7/676: readlink d7/d27/d44/db2/db8/de/d20/d86/l5f 0 2024-03-29T04:45:04.244 INFO:tasks.workunit.client.0.smithi142.stdout:5/725: rename df/d1a/c20 to df/d1a/d21/d27/d45/ced 0 2024-03-29T04:45:04.246 INFO:tasks.workunit.client.0.smithi142.stdout:1/576: symlink d8/lcb 0 2024-03-29T04:45:04.246 INFO:tasks.workunit.client.0.smithi142.stdout:5/726: write df/d16/d25/f59 [1916832,123652] 0 2024-03-29T04:45:04.248 INFO:tasks.workunit.client.0.smithi142.stdout:0/601: dread d7/d13/d4a/d9a/fb3 [0,4194304] 0 2024-03-29T04:45:04.248 INFO:tasks.workunit.client.0.smithi142.stdout:7/677: truncate d7/d27/d44/db2/db8/d4b/fd0 281059 0 2024-03-29T04:45:04.249 INFO:tasks.workunit.client.0.smithi142.stdout:8/614: link d5/d1d/d59/cc6 d5/d2e/cdb 0 2024-03-29T04:45:04.249 INFO:tasks.workunit.client.0.smithi142.stdout:5/727: write df/d1a/d21/d55/f81 [517067,7413] 0 2024-03-29T04:45:04.250 INFO:tasks.workunit.client.0.smithi142.stdout:8/615: fdatasync d5/d1d/d59/f7c 0 2024-03-29T04:45:04.251 INFO:tasks.workunit.client.0.smithi142.stdout:8/616: read - d5/d93/fb2 zero size 2024-03-29T04:45:04.251 INFO:tasks.workunit.client.0.smithi142.stdout:0/602: creat d7/d9/d55/d96/fc3 x:0 0 0 2024-03-29T04:45:04.255 INFO:tasks.workunit.client.0.smithi142.stdout:0/603: creat d7/d9/d1e/d81/fc4 x:0 0 0 2024-03-29T04:45:04.257 INFO:tasks.workunit.client.0.smithi142.stdout:5/728: mknod df/d1a/d21/cee 0 2024-03-29T04:45:04.257 INFO:tasks.workunit.client.0.smithi142.stdout:8/617: dread d5/d93/fb0 [0,4194304] 0 2024-03-29T04:45:04.257 INFO:tasks.workunit.client.0.smithi142.stdout:5/729: fdatasync df/d1a/d21/d27/fe7 0 2024-03-29T04:45:04.258 INFO:tasks.workunit.client.0.smithi142.stdout:8/618: write d5/d1d/d24/f33 [1273314,47215] 0 2024-03-29T04:45:04.262 INFO:tasks.workunit.client.0.smithi142.stdout:7/678: write d7/d27/d44/db2/db8/d4b/d71/f2c [5078756,67379] 0 2024-03-29T04:45:04.262 INFO:tasks.workunit.client.0.smithi142.stdout:5/730: mkdir df/d29/d7a/d97/def 0 2024-03-29T04:45:04.264 INFO:tasks.workunit.client.0.smithi142.stdout:7/679: mkdir d7/d27/d3a/d92/de5 0 2024-03-29T04:45:04.265 INFO:tasks.workunit.client.0.smithi142.stdout:8/619: truncate d5/d1d/d2f/f50 497882 0 2024-03-29T04:45:04.265 INFO:tasks.workunit.client.0.smithi142.stdout:0/604: creat d7/d13/fc5 x:0 0 0 2024-03-29T04:45:04.265 INFO:tasks.workunit.client.0.smithi142.stdout:5/731: chown df/c2b 41993 1 2024-03-29T04:45:04.267 INFO:tasks.workunit.client.0.smithi142.stdout:7/680: write d7/d27/d44/db2/db8/de/d20/d3f/f41 [1531621,34709] 0 2024-03-29T04:45:04.268 INFO:tasks.workunit.client.0.smithi142.stdout:8/620: symlink d5/d1d/d2f/ldc 0 2024-03-29T04:45:04.268 INFO:tasks.workunit.client.0.smithi142.stdout:0/605: symlink d7/d9/d1e/lc6 0 2024-03-29T04:45:04.269 INFO:tasks.workunit.client.0.smithi142.stdout:8/621: write d5/d1d/d81/f91 [768851,28227] 0 2024-03-29T04:45:04.270 INFO:tasks.workunit.client.0.smithi142.stdout:5/732: rmdir df/d1a/d21/d55/d66 39 2024-03-29T04:45:04.271 INFO:tasks.workunit.client.0.smithi142.stdout:8/622: write d5/f18 [12136676,43462] 0 2024-03-29T04:45:04.272 INFO:tasks.workunit.client.0.smithi142.stdout:5/733: rmdir df/d1a/d28 39 2024-03-29T04:45:04.274 INFO:tasks.workunit.client.0.smithi142.stdout:7/681: truncate d7/d27/d44/db2/db8/d4b/d71/d30/f99 839940 0 2024-03-29T04:45:04.274 INFO:tasks.workunit.client.0.smithi142.stdout:8/623: fdatasync d5/d1d/d2f/f90 0 2024-03-29T04:45:04.277 INFO:tasks.workunit.client.0.smithi142.stdout:5/734: dread df/d1a/d28/fae [0,4194304] 0 2024-03-29T04:45:04.281 INFO:tasks.workunit.client.0.smithi142.stdout:7/682: creat d7/d27/d44/db2/db8/fe6 x:0 0 0 2024-03-29T04:45:04.281 INFO:tasks.workunit.client.0.smithi142.stdout:5/735: rmdir df 39 2024-03-29T04:45:04.283 INFO:tasks.workunit.client.0.smithi142.stdout:7/683: read d7/d27/f91 [1479140,116375] 0 2024-03-29T04:45:04.284 INFO:tasks.workunit.client.0.smithi142.stdout:7/684: chown d7/d62 327971 1 2024-03-29T04:45:04.285 INFO:tasks.workunit.client.0.smithi142.stdout:5/736: dread - df/d1a/d21/d27/f4b zero size 2024-03-29T04:45:04.288 INFO:tasks.workunit.client.0.smithi142.stdout:5/737: creat df/d1a/d21/d27/d45/dbf/ff0 x:0 0 0 2024-03-29T04:45:04.289 INFO:tasks.workunit.client.0.smithi142.stdout:0/606: dread d7/d13/dbf/d72/f48 [0,4194304] 0 2024-03-29T04:45:04.290 INFO:tasks.workunit.client.0.smithi142.stdout:5/738: truncate df/d29/d5d/d75/d90/fc2 1234228 0 2024-03-29T04:45:04.291 INFO:tasks.workunit.client.0.smithi142.stdout:0/607: write d7/d13/dbf/f92 [4624921,93871] 0 2024-03-29T04:45:04.292 INFO:tasks.workunit.client.0.smithi142.stdout:0/608: read d7/d13/dbf/f92 [1955318,128485] 0 2024-03-29T04:45:04.294 INFO:tasks.workunit.client.0.smithi142.stdout:8/624: dread d5/d1d/d2f/d76/d2d/f57 [0,4194304] 0 2024-03-29T04:45:04.294 INFO:tasks.workunit.client.0.smithi142.stdout:5/739: mkdir df/d16/d24/d2e/d47/d8c/df1 0 2024-03-29T04:45:04.294 INFO:tasks.workunit.client.0.smithi142.stdout:8/625: chown d5/d1d/d2f 846181938 1 2024-03-29T04:45:04.295 INFO:tasks.workunit.client.0.smithi142.stdout:5/740: readlink df/d29/d5d/d75/d90/lec 0 2024-03-29T04:45:04.296 INFO:tasks.workunit.client.0.smithi142.stdout:0/609: rmdir d7/d13/dbf 39 2024-03-29T04:45:04.297 INFO:tasks.workunit.client.0.smithi142.stdout:5/741: creat df/d16/d24/d5c/db1/ff2 x:0 0 0 2024-03-29T04:45:04.298 INFO:tasks.workunit.client.0.smithi142.stdout:8/626: dread d5/d2e/dd2/d60/f84 [0,4194304] 0 2024-03-29T04:45:04.299 INFO:tasks.workunit.client.0.smithi142.stdout:0/610: creat d7/d13/dbf/d72/fc7 x:0 0 0 2024-03-29T04:45:04.301 INFO:tasks.workunit.client.0.smithi142.stdout:8/627: creat d5/d2e/dd2/fdd x:0 0 0 2024-03-29T04:45:04.301 INFO:tasks.workunit.client.0.smithi142.stdout:0/611: creat d7/d13/dbf/fc8 x:0 0 0 2024-03-29T04:45:04.304 INFO:tasks.workunit.client.0.smithi142.stdout:0/612: symlink d7/d9/lc9 0 2024-03-29T04:45:04.305 INFO:tasks.workunit.client.0.smithi142.stdout:8/628: dwrite d5/f25 [4194304,4194304] 0 2024-03-29T04:45:04.306 INFO:tasks.workunit.client.0.smithi142.stdout:0/613: truncate d7/d9/d55/f5b 1306655 0 2024-03-29T04:45:04.307 INFO:tasks.workunit.client.0.smithi142.stdout:0/614: chown d7/d9/d55/l4f 677215754 1 2024-03-29T04:45:04.310 INFO:tasks.workunit.client.0.smithi142.stdout:8/629: rename d5/d1d/d24/f6c to d5/d1d/d2f/d76/d3c/fde 0 2024-03-29T04:45:04.313 INFO:tasks.workunit.client.0.smithi142.stdout:0/615: dread d7/d9/d54/f65 [0,4194304] 0 2024-03-29T04:45:04.315 INFO:tasks.workunit.client.0.smithi142.stdout:8/630: mknod d5/d1d/d2f/d45/dc3/cdf 0 2024-03-29T04:45:04.317 INFO:tasks.workunit.client.0.smithi142.stdout:0/616: rename d7/d9/d55/d96 to d7/d13/dbf/dca 0 2024-03-29T04:45:04.319 INFO:tasks.workunit.client.0.smithi142.stdout:8/631: chown d5/d1d/d24/d66/c69 7697 1 2024-03-29T04:45:04.320 INFO:tasks.workunit.client.0.smithi142.stdout:8/632: chown d5/f19 223 1 2024-03-29T04:45:04.321 INFO:tasks.workunit.client.0.smithi142.stdout:8/633: write d5/d1d/d24/f8c [1677083,105512] 0 2024-03-29T04:45:04.326 INFO:tasks.workunit.client.0.smithi142.stdout:0/617: dwrite d7/d13/dbf/d72/d37/f73 [0,4194304] 0 2024-03-29T04:45:04.327 INFO:tasks.workunit.client.0.smithi142.stdout:8/634: creat d5/d1d/d2f/d5f/fe0 x:0 0 0 2024-03-29T04:45:04.331 INFO:tasks.workunit.client.0.smithi142.stdout:0/618: rename d7/d13/dbf/d72/f48 to d7/d9/fcb 0 2024-03-29T04:45:04.331 INFO:tasks.workunit.client.0.smithi142.stdout:8/635: creat d5/d1d/d2f/fe1 x:0 0 0 2024-03-29T04:45:04.332 INFO:tasks.workunit.client.0.smithi142.stdout:8/636: fdatasync d5/d1d/d2f/fc8 0 2024-03-29T04:45:04.334 INFO:tasks.workunit.client.0.smithi142.stdout:8/637: mkdir d5/d1d/d2f/d76/d2d/de2 0 2024-03-29T04:45:04.336 INFO:tasks.workunit.client.0.smithi142.stdout:0/619: link d7/d13/dbf/d72/f3e d7/d13/d4a/d9a/fcc 0 2024-03-29T04:45:04.336 INFO:tasks.workunit.client.0.smithi142.stdout:8/638: creat d5/d1d/d2f/d76/d2d/de2/fe3 x:0 0 0 2024-03-29T04:45:04.337 INFO:tasks.workunit.client.0.smithi142.stdout:0/620: read d7/d9/d55/f5b [76878,84652] 0 2024-03-29T04:45:04.338 INFO:tasks.workunit.client.0.smithi142.stdout:8/639: chown d5/d2e/dd2/fd7 1266699215 1 2024-03-29T04:45:04.341 INFO:tasks.workunit.client.0.smithi142.stdout:8/640: creat d5/d1d/d2f/d45/fe4 x:0 0 0 2024-03-29T04:45:04.341 INFO:tasks.workunit.client.0.smithi142.stdout:0/621: dread d7/d9/d55/d42/f74 [0,4194304] 0 2024-03-29T04:45:04.348 INFO:tasks.workunit.client.0.smithi142.stdout:8/641: write d5/d1d/d24/f6b [302425,36384] 0 2024-03-29T04:45:04.348 INFO:tasks.workunit.client.0.smithi142.stdout:0/622: write d7/d13/f50 [474932,38896] 0 2024-03-29T04:45:04.350 INFO:tasks.workunit.client.0.smithi142.stdout:8/642: write d5/d1d/d24/f8c [4388815,108891] 0 2024-03-29T04:45:04.355 INFO:tasks.workunit.client.0.smithi142.stdout:8/643: dread d5/d93/fb8 [0,4194304] 0 2024-03-29T04:45:04.358 INFO:tasks.workunit.client.0.smithi142.stdout:0/623: dwrite d7/d13/d4a/d70/f8b [0,4194304] 0 2024-03-29T04:45:04.359 INFO:tasks.workunit.client.0.smithi142.stdout:8/644: creat d5/d1d/d2f/d76/d3c/fe5 x:0 0 0 2024-03-29T04:45:04.359 INFO:tasks.workunit.client.0.smithi142.stdout:0/624: chown d7/d9/d55/c76 989 1 2024-03-29T04:45:04.364 INFO:tasks.workunit.client.0.smithi142.stdout:0/625: rename d7/d9/f68 to d7/d9/fcd 0 2024-03-29T04:45:04.366 INFO:tasks.workunit.client.0.smithi142.stdout:8/645: dwrite d5/d2e/dd2/d60/fd6 [0,4194304] 0 2024-03-29T04:45:04.369 INFO:tasks.workunit.client.0.smithi142.stdout:8/646: mkdir d5/d2e/dd2/de6 0 2024-03-29T04:45:04.377 INFO:tasks.workunit.client.0.smithi142.stdout:8/647: dwrite d5/d1d/d59/fbe [0,4194304] 0 2024-03-29T04:45:04.383 INFO:tasks.workunit.client.0.smithi142.stdout:8/648: rename d5/d1d/d2f/d76/d3c/fde to d5/d1d/d24/db4/fe7 0 2024-03-29T04:45:04.385 INFO:tasks.workunit.client.0.smithi142.stdout:8/649: read - d5/d1d/d2f/d76/f35 zero size 2024-03-29T04:45:04.387 INFO:tasks.workunit.client.0.smithi142.stdout:8/650: write d5/f19 [175534,16105] 0 2024-03-29T04:45:04.388 INFO:tasks.workunit.client.0.smithi142.stdout:8/651: chown d5/d2e/c62 74 1 2024-03-29T04:45:04.388 INFO:tasks.workunit.client.0.smithi142.stdout:8/652: fdatasync d5/f18 0 2024-03-29T04:45:04.392 INFO:tasks.workunit.client.0.smithi142.stdout:8/653: link d5/d1d/d24/ca3 d5/d1d/d2f/ce8 0 2024-03-29T04:45:04.394 INFO:tasks.workunit.client.0.smithi142.stdout:8/654: mkdir d5/d1d/d2f/d76/d3c/d70/de9 0 2024-03-29T04:45:04.398 INFO:tasks.workunit.client.0.smithi142.stdout:8/655: dread d5/d1d/d2f/d5f/d48/f8e [0,4194304] 0 2024-03-29T04:45:04.398 INFO:tasks.workunit.client.0.smithi142.stdout:8/656: fdatasync d5/d2e/dd2/d60/fd6 0 2024-03-29T04:45:04.403 INFO:tasks.workunit.client.0.smithi142.stdout:8/657: link d5/d1d/d24/c4e d5/d1d/d24/d66/dce/cea 0 2024-03-29T04:45:04.405 INFO:tasks.workunit.client.0.smithi142.stdout:8/658: truncate d5/d1d/d2f/fa9 713047 0 2024-03-29T04:45:04.407 INFO:tasks.workunit.client.0.smithi142.stdout:8/659: unlink d5/d1d/d24/d66/dce/fd1 0 2024-03-29T04:45:04.409 INFO:tasks.workunit.client.0.smithi142.stdout:8/660: creat d5/d1d/d24/d66/dce/feb x:0 0 0 2024-03-29T04:45:04.411 INFO:tasks.workunit.client.0.smithi142.stdout:8/661: dread d5/d1d/d2f/d76/d9c/f7e [0,4194304] 0 2024-03-29T04:45:04.417 INFO:tasks.workunit.client.0.smithi142.stdout:8/662: dwrite d5/fc5 [0,4194304] 0 2024-03-29T04:45:04.421 INFO:tasks.workunit.client.0.smithi142.stdout:8/663: getdents d5/d2e/dd2/d60 0 2024-03-29T04:45:04.423 INFO:tasks.workunit.client.0.smithi142.stdout:8/664: symlink d5/d2e/dd2/d60/lec 0 2024-03-29T04:45:04.432 INFO:tasks.workunit.client.0.smithi142.stdout:8/665: truncate d5/f22 8797242 0 2024-03-29T04:45:04.436 INFO:tasks.workunit.client.0.smithi142.stdout:8/666: link l3 d5/d1d/d2f/led 0 2024-03-29T04:45:04.437 INFO:tasks.workunit.client.0.smithi142.stdout:8/667: symlink d5/d1d/d81/lee 0 2024-03-29T04:45:04.443 INFO:tasks.workunit.client.0.smithi142.stdout:8/668: dwrite d5/d1d/d2f/d45/fe4 [0,4194304] 0 2024-03-29T04:45:04.449 INFO:tasks.workunit.client.0.smithi142.stdout:8/669: write d5/d2e/dd2/d60/f84 [30596,130104] 0 2024-03-29T04:45:04.450 INFO:tasks.workunit.client.0.smithi142.stdout:8/670: stat d5/d1d/d24/d66/dce 0 2024-03-29T04:45:04.452 INFO:tasks.workunit.client.0.smithi142.stdout:8/671: write d5/d2e/dd2/d60/f8f [8782787,77275] 0 2024-03-29T04:45:04.454 INFO:tasks.workunit.client.0.smithi142.stdout:8/672: truncate d5/d1d/d2f/d45/fcf 783983 0 2024-03-29T04:45:04.455 INFO:tasks.workunit.client.0.smithi142.stdout:8/673: dread - d5/d1d/d2f/d5f/fb1 zero size 2024-03-29T04:45:04.458 INFO:tasks.workunit.client.0.smithi142.stdout:8/674: rename d5/d2e/dd2/d60/f84 to d5/d1d/d59/fef 0 2024-03-29T04:45:04.462 INFO:tasks.workunit.client.0.smithi142.stdout:8/675: dread d5/d1d/d2f/d76/d2d/f73 [4194304,4194304] 0 2024-03-29T04:45:04.474 INFO:tasks.workunit.client.0.smithi142.stdout:8/676: dread d5/d2e/fcc [0,4194304] 0 2024-03-29T04:45:04.480 INFO:tasks.workunit.client.0.smithi142.stdout:8/677: dwrite d5/d93/f9e [0,4194304] 0 2024-03-29T04:45:04.484 INFO:tasks.workunit.client.0.smithi142.stdout:8/678: mknod d5/d1d/d24/d66/dce/cf0 0 2024-03-29T04:45:04.495 INFO:tasks.workunit.client.0.smithi142.stdout:8/679: dread d5/d1d/d2f/d76/d2d/f56 [0,4194304] 0 2024-03-29T04:45:04.496 INFO:tasks.workunit.client.0.smithi142.stdout:8/680: chown d5/d1d/d24/f8c 656870185 1 2024-03-29T04:45:04.497 INFO:tasks.workunit.client.0.smithi142.stdout:8/681: truncate d5/d1d/d2f/d76/d3c/fe5 339383 0 2024-03-29T04:45:04.500 INFO:tasks.workunit.client.0.smithi142.stdout:8/682: truncate d5/d1d/d24/db4/fe7 374021 0 2024-03-29T04:45:04.501 INFO:tasks.workunit.client.0.smithi142.stdout:8/683: write d5/d1d/d2f/d45/f9a [1969775,11788] 0 2024-03-29T04:45:04.507 INFO:tasks.workunit.client.0.smithi142.stdout:8/684: dwrite d5/d1d/d2f/d76/d3c/d70/fda [0,4194304] 0 2024-03-29T04:45:04.513 INFO:tasks.workunit.client.0.smithi142.stdout:8/685: dwrite d5/d2e/dd2/fd7 [0,4194304] 0 2024-03-29T04:45:04.526 INFO:tasks.workunit.client.0.smithi142.stdout:8/686: truncate d5/d1d/d2f/d5f/fa6 4119481 0 2024-03-29T04:45:04.546 INFO:tasks.workunit.client.0.smithi142.stdout:8/687: dread d5/d1d/d24/f8c [0,4194304] 0 2024-03-29T04:45:04.547 INFO:tasks.workunit.client.0.smithi142.stdout:8/688: creat d5/d2e/dd2/ff1 x:0 0 0 2024-03-29T04:45:04.549 INFO:tasks.workunit.client.0.smithi142.stdout:8/689: symlink d5/lf2 0 2024-03-29T04:45:04.555 INFO:tasks.workunit.client.0.smithi142.stdout:8/690: dwrite d5/d1d/d2f/d5f/d48/f8e [0,4194304] 0 2024-03-29T04:45:04.789 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:04 smithi152 ceph-mon[37177]: pgmap v268: 65 pgs: 65 active+clean; 1.2 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 34 MiB/s rd, 157 MiB/s wr, 326 op/s 2024-03-29T04:45:04.870 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:04 smithi142 ceph-mon[29027]: pgmap v268: 65 pgs: 65 active+clean; 1.2 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 34 MiB/s rd, 157 MiB/s wr, 326 op/s 2024-03-29T04:45:05.135 INFO:tasks.workunit.client.0.smithi142.stdout:5/742: sync 2024-03-29T04:45:05.135 INFO:tasks.workunit.client.0.smithi142.stdout:1/577: sync 2024-03-29T04:45:05.135 INFO:tasks.workunit.client.0.smithi142.stdout:4/587: sync 2024-03-29T04:45:05.135 INFO:tasks.workunit.client.0.smithi142.stdout:2/561: sync 2024-03-29T04:45:05.135 INFO:tasks.workunit.client.0.smithi142.stdout:7/685: sync 2024-03-29T04:45:05.135 INFO:tasks.workunit.client.0.smithi142.stdout:0/626: sync 2024-03-29T04:45:05.136 INFO:tasks.workunit.client.0.smithi142.stdout:6/559: sync 2024-03-29T04:45:05.136 INFO:tasks.workunit.client.0.smithi142.stdout:9/417: sync 2024-03-29T04:45:05.136 INFO:tasks.workunit.client.0.smithi142.stdout:3/469: sync 2024-03-29T04:45:05.136 INFO:tasks.workunit.client.0.smithi142.stdout:8/691: sync 2024-03-29T04:45:05.136 INFO:tasks.workunit.client.0.smithi142.stdout:4/588: stat d0/dc/d15/d3d/d65/l7d 0 2024-03-29T04:45:05.136 INFO:tasks.workunit.client.0.smithi142.stdout:5/743: write df/d1a/f2f [147192,25593] 0 2024-03-29T04:45:05.136 INFO:tasks.workunit.client.0.smithi142.stdout:3/470: write d3/d29/d2d/f8a [109432,101845] 0 2024-03-29T04:45:05.136 INFO:tasks.workunit.client.0.smithi142.stdout:0/627: write d7/d13/dbf/dca/f9f [263462,434] 0 2024-03-29T04:45:05.138 INFO:tasks.workunit.client.0.smithi142.stdout:6/560: symlink d1c/d33/d46/d61/d7e/lb0 0 2024-03-29T04:45:05.140 INFO:tasks.workunit.client.0.smithi142.stdout:9/418: unlink d0/d2/d6/d27/d3c/d7b/l7c 0 2024-03-29T04:45:05.140 INFO:tasks.workunit.client.0.smithi142.stdout:5/744: write df/d1a/d21/da5/fdd [515343,35419] 0 2024-03-29T04:45:05.141 INFO:tasks.workunit.client.0.smithi142.stdout:1/578: symlink d8/db/dc/d77/db9/dbe/d2d/d44/d53/lcc 0 2024-03-29T04:45:05.141 INFO:tasks.workunit.client.0.smithi142.stdout:4/589: dread - d0/d9/d11/d4c/d5f/d81/dc9/d47/d7a/f8b zero size 2024-03-29T04:45:05.144 INFO:tasks.workunit.client.0.smithi142.stdout:6/561: symlink d1c/d45/d8a/lb1 0 2024-03-29T04:45:05.151 INFO:tasks.workunit.client.0.smithi142.stdout:8/692: dwrite d5/d1d/d2f/d76/d2d/fca [0,4194304] 0 2024-03-29T04:45:05.153 INFO:tasks.workunit.client.0.smithi142.stdout:5/745: creat df/d1a/ff3 x:0 0 0 2024-03-29T04:45:05.155 INFO:tasks.workunit.client.0.smithi142.stdout:9/419: creat d0/d2/d6/f81 x:0 0 0 2024-03-29T04:45:05.157 INFO:tasks.workunit.client.0.smithi142.stdout:0/628: dwrite d7/d13/dbf/dca/f97 [4194304,4194304] 0 2024-03-29T04:45:05.160 INFO:tasks.workunit.client.0.smithi142.stdout:3/471: dwrite d3/d29/d2d/d56/f9a [0,4194304] 0 2024-03-29T04:45:05.160 INFO:tasks.workunit.client.0.smithi142.stdout:9/420: fdatasync d0/d2/d6/d16/f37 0 2024-03-29T04:45:05.163 INFO:tasks.workunit.client.0.smithi142.stdout:4/590: getdents d0/d9/db2 0 2024-03-29T04:45:05.163 INFO:tasks.workunit.client.0.smithi142.stdout:7/686: write d7/d27/d44/dab/fda [711525,36200] 0 2024-03-29T04:45:05.166 INFO:tasks.workunit.client.0.smithi142.stdout:1/579: dwrite d8/db/dc/d77/db9/dbe/d15/f60 [0,4194304] 0 2024-03-29T04:45:05.172 INFO:tasks.workunit.client.0.smithi142.stdout:0/629: chown d7/d13/dbf/d72/d37/da4/lb6 1 1 2024-03-29T04:45:05.174 INFO:tasks.workunit.client.0.smithi142.stdout:3/472: dread d3/d29/d8f/d5f/f86 [0,4194304] 0 2024-03-29T04:45:05.174 INFO:tasks.workunit.client.0.smithi142.stdout:6/562: getdents d1c/d33/d46/d8b 0 2024-03-29T04:45:05.174 INFO:tasks.workunit.client.0.smithi142.stdout:6/563: dread - d1c/d33/d46/d86/f9d zero size 2024-03-29T04:45:05.175 INFO:tasks.workunit.client.0.smithi142.stdout:8/693: dwrite d5/d1d/d24/f6b [0,4194304] 0 2024-03-29T04:45:05.178 INFO:tasks.workunit.client.0.smithi142.stdout:8/694: write d5/d1d/d2f/fa7 [4346264,88257] 0 2024-03-29T04:45:05.179 INFO:tasks.workunit.client.0.smithi142.stdout:7/687: symlink d7/d27/d44/dab/le7 0 2024-03-29T04:45:05.183 INFO:tasks.workunit.client.0.smithi142.stdout:4/591: fdatasync d0/d9/d11/d4c/d5f/d81/dc9/d47/f55 0 2024-03-29T04:45:05.185 INFO:tasks.workunit.client.0.smithi142.stdout:7/688: dread - d7/d27/d44/db2/db8/d89/fdc zero size 2024-03-29T04:45:05.186 INFO:tasks.workunit.client.0.smithi142.stdout:8/695: truncate d5/d1d/d2f/d5f/fe0 714245 0 2024-03-29T04:45:05.187 INFO:tasks.workunit.client.0.smithi142.stdout:6/564: creat d1c/d33/d46/d61/d7e/fb2 x:0 0 0 2024-03-29T04:45:05.188 INFO:tasks.workunit.client.0.smithi142.stdout:9/421: dwrite d0/d2/d6/d27/d3c/d4f/f56 [0,4194304] 0 2024-03-29T04:45:05.188 INFO:tasks.workunit.client.0.smithi142.stdout:0/630: write d7/d13/dbf/d72/fa0 [2443204,64456] 0 2024-03-29T04:45:05.189 INFO:tasks.workunit.client.0.smithi142.stdout:5/746: getdents df/d16/d25 0 2024-03-29T04:45:05.191 INFO:tasks.workunit.client.0.smithi142.stdout:6/565: readlink d1c/d33/d46/d61/d7e/lb0 0 2024-03-29T04:45:05.191 INFO:tasks.workunit.client.0.smithi142.stdout:3/473: link d3/d29/d2d/f3f d3/d1f/d24/d47/d87/fa3 0 2024-03-29T04:45:05.192 INFO:tasks.workunit.client.0.smithi142.stdout:5/747: dread - df/d1a/d21/d27/f4b zero size 2024-03-29T04:45:05.192 INFO:tasks.workunit.client.0.smithi142.stdout:8/696: stat d5/d1d/d2f/f68 0 2024-03-29T04:45:05.194 INFO:tasks.workunit.client.0.smithi142.stdout:8/697: fsync d5/d2e/dd2/d60/fb3 0 2024-03-29T04:45:05.197 INFO:tasks.workunit.client.0.smithi142.stdout:7/689: symlink d7/d27/d44/db2/db8/d4b/d71/d30/d72/db3/le8 0 2024-03-29T04:45:05.199 INFO:tasks.workunit.client.0.smithi142.stdout:6/566: creat d1c/d33/d46/d71/fb3 x:0 0 0 2024-03-29T04:45:05.199 INFO:tasks.workunit.client.0.smithi142.stdout:9/422: dwrite d0/d2/d6/d16/f3b [0,4194304] 0 2024-03-29T04:45:05.200 INFO:tasks.workunit.client.0.smithi142.stdout:5/748: fdatasync df/d16/f26 0 2024-03-29T04:45:05.200 INFO:tasks.workunit.client.0.smithi142.stdout:6/567: stat d1c/d33/d46/d61/l6f 0 2024-03-29T04:45:05.201 INFO:tasks.workunit.client.0.smithi142.stdout:0/631: creat d7/d13/dbf/dca/dab/dc1/fce x:0 0 0 2024-03-29T04:45:05.203 INFO:tasks.workunit.client.0.smithi142.stdout:4/592: getdents d0/d9/d11/d97/d9e 0 2024-03-29T04:45:05.205 INFO:tasks.workunit.client.0.smithi142.stdout:4/593: fsync d0/d9/d11/d4c/f89 0 2024-03-29T04:45:05.205 INFO:tasks.workunit.client.0.smithi142.stdout:8/698: unlink d5/d1d/d59/cbc 0 2024-03-29T04:45:05.206 INFO:tasks.workunit.client.0.smithi142.stdout:3/474: truncate d3/d29/d2b/d40/d67/d8b/f3e 669999 0 2024-03-29T04:45:05.207 INFO:tasks.workunit.client.0.smithi142.stdout:4/594: readlink d0/d12/l1f 0 2024-03-29T04:45:05.209 INFO:tasks.workunit.client.0.smithi142.stdout:1/580: write d8/f70 [433067,68792] 0 2024-03-29T04:45:05.209 INFO:tasks.workunit.client.0.smithi142.stdout:4/595: readlink d0/d9/d19/l1c 0 2024-03-29T04:45:05.212 INFO:tasks.workunit.client.0.smithi142.stdout:6/568: creat d1c/d45/d51/fb4 x:0 0 0 2024-03-29T04:45:05.213 INFO:tasks.workunit.client.0.smithi142.stdout:0/632: truncate d7/d9/d55/f5b 1646239 0 2024-03-29T04:45:05.214 INFO:tasks.workunit.client.0.smithi142.stdout:9/423: dwrite d0/d2/d6/f81 [0,4194304] 0 2024-03-29T04:45:05.215 INFO:tasks.workunit.client.0.smithi142.stdout:7/690: mknod d7/d27/d3a/d92/de5/ce9 0 2024-03-29T04:45:05.216 INFO:tasks.workunit.client.0.smithi142.stdout:1/581: write d8/db/dc/d77/db9/dbe/d1f/f9b [300684,111626] 0 2024-03-29T04:45:05.220 INFO:tasks.workunit.client.0.smithi142.stdout:7/691: readlink d7/d27/d44/db2/db8/de/d20/d3f/lcd 0 2024-03-29T04:45:05.220 INFO:tasks.workunit.client.0.smithi142.stdout:9/424: dread d0/d2/d6/f30 [0,4194304] 0 2024-03-29T04:45:05.222 INFO:tasks.workunit.client.0.smithi142.stdout:5/749: dwrite df/d1a/d21/d55/d66/fbe [0,4194304] 0 2024-03-29T04:45:05.224 INFO:tasks.workunit.client.0.smithi142.stdout:9/425: truncate d0/d2/d6/d27/d3c/f72 422272 0 2024-03-29T04:45:05.228 INFO:tasks.workunit.client.0.smithi142.stdout:0/633: symlink d7/d13/dbf/dca/dab/dc1/lcf 0 2024-03-29T04:45:05.232 INFO:tasks.workunit.client.0.smithi142.stdout:3/475: creat d3/d1f/fa4 x:0 0 0 2024-03-29T04:45:05.235 INFO:tasks.workunit.client.0.smithi142.stdout:1/582: symlink d8/db/dc/d77/d31/d4d/lcd 0 2024-03-29T04:45:05.237 INFO:tasks.workunit.client.0.smithi142.stdout:7/692: mkdir d7/d27/d3a/d92/d97/dea 0 2024-03-29T04:45:05.238 INFO:tasks.workunit.client.0.smithi142.stdout:2/562: sync 2024-03-29T04:45:05.239 INFO:tasks.workunit.client.0.smithi142.stdout:5/750: mknod df/d29/d5d/d75/cf4 0 2024-03-29T04:45:05.240 INFO:tasks.workunit.client.0.smithi142.stdout:2/563: truncate de/f19 576872 0 2024-03-29T04:45:05.240 INFO:tasks.workunit.client.0.smithi142.stdout:9/426: creat d0/d2/d6/d27/f82 x:0 0 0 2024-03-29T04:45:05.241 INFO:tasks.workunit.client.0.smithi142.stdout:8/699: link d5/d1d/d24/db4/cc7 d5/d1d/d2f/d76/d3c/d70/cf3 0 2024-03-29T04:45:05.244 INFO:tasks.workunit.client.0.smithi142.stdout:4/596: write d0/d9/d11/d4c/d5f/d81/dc9/d47/f55 [3449947,65521] 0 2024-03-29T04:45:05.245 INFO:tasks.workunit.client.0.smithi142.stdout:3/476: creat d3/d1f/d24/fa5 x:0 0 0 2024-03-29T04:45:05.246 INFO:tasks.workunit.client.0.smithi142.stdout:4/597: truncate d0/dc/d15/f72 542772 0 2024-03-29T04:45:05.247 INFO:tasks.workunit.client.0.smithi142.stdout:1/583: creat d8/db/dc/d77/d5a/fce x:0 0 0 2024-03-29T04:45:05.248 INFO:tasks.workunit.client.0.smithi142.stdout:3/477: read d3/d1f/d24/d47/d87/fa3 [3882102,47679] 0 2024-03-29T04:45:05.249 INFO:tasks.workunit.client.0.smithi142.stdout:7/693: write d7/d27/f9f [539115,57433] 0 2024-03-29T04:45:05.250 INFO:tasks.workunit.client.0.smithi142.stdout:5/751: rename df/d29/d5d/d75/d90/feb to df/d16/d25/ff5 0 2024-03-29T04:45:05.251 INFO:tasks.workunit.client.0.smithi142.stdout:3/478: readlink d3/d29/d8f/d5f/d64/l88 0 2024-03-29T04:45:05.251 INFO:tasks.workunit.client.0.smithi142.stdout:6/569: dwrite d1c/d33/d46/d66/f6b [0,4194304] 0 2024-03-29T04:45:05.252 INFO:tasks.workunit.client.0.smithi142.stdout:3/479: fdatasync d3/d1f/d24/fa5 0 2024-03-29T04:45:05.253 INFO:tasks.workunit.client.0.smithi142.stdout:2/564: mknod de/d11/d1e/d22/d50/cbd 0 2024-03-29T04:45:05.254 INFO:tasks.workunit.client.0.smithi142.stdout:3/480: chown d3/d29/d2b/d40/d67/d8b/d4d/f9d 69178 1 2024-03-29T04:45:05.272 INFO:tasks.workunit.client.0.smithi142.stdout:8/700: fsync d5/d93/fb8 0 2024-03-29T04:45:05.273 INFO:tasks.workunit.client.0.smithi142.stdout:1/584: mknod d8/db/dc/d77/db9/dbe/d2d/d44/ccf 0 2024-03-29T04:45:05.273 INFO:tasks.workunit.client.0.smithi142.stdout:5/752: mkdir df/d29/d5d/d75/d90/df6 0 2024-03-29T04:45:05.274 INFO:tasks.workunit.client.0.smithi142.stdout:0/634: write d7/f2b [3120464,77321] 0 2024-03-29T04:45:05.277 INFO:tasks.workunit.client.0.smithi142.stdout:8/701: chown d5/d1d/d2f/d76/d3c/c63 100574090 1 2024-03-29T04:45:05.277 INFO:tasks.workunit.client.0.smithi142.stdout:6/570: rename d1c/d33/d46/d8b/d52/f54 to d1c/d33/d46/d61/d9e/fb5 0 2024-03-29T04:45:05.278 INFO:tasks.workunit.client.0.smithi142.stdout:3/481: truncate d3/d1f/d24/f69 489110 0 2024-03-29T04:45:05.279 INFO:tasks.workunit.client.0.smithi142.stdout:9/427: getdents d0/d2/d6/d27/d3c/d7b 0 2024-03-29T04:45:05.280 INFO:tasks.workunit.client.0.smithi142.stdout:8/702: dread - d5/d2e/dd2/fdd zero size 2024-03-29T04:45:05.281 INFO:tasks.workunit.client.0.smithi142.stdout:2/565: dwrite de/d11/d23/fb7 [0,4194304] 0 2024-03-29T04:45:05.281 INFO:tasks.workunit.client.0.smithi142.stdout:6/571: dread - d1c/d33/d46/d8b/d52/f87 zero size 2024-03-29T04:45:05.281 INFO:tasks.workunit.client.0.smithi142.stdout:4/598: dread d0/d12/f3c [0,4194304] 0 2024-03-29T04:45:05.282 INFO:tasks.workunit.client.0.smithi142.stdout:0/635: write d7/d9/fcd [3562483,62024] 0 2024-03-29T04:45:05.283 INFO:tasks.workunit.client.0.smithi142.stdout:3/482: write d3/d29/d2b/d40/d67/d8b/d4d/f9d [238806,33875] 0 2024-03-29T04:45:05.283 INFO:tasks.workunit.client.0.smithi142.stdout:1/585: creat d8/db/dc/d77/db9/dbe/d2d/d44/d4b/fd0 x:0 0 0 2024-03-29T04:45:05.285 INFO:tasks.workunit.client.0.smithi142.stdout:5/753: creat df/d16/d24/ff7 x:0 0 0 2024-03-29T04:45:05.286 INFO:tasks.workunit.client.0.smithi142.stdout:9/428: mkdir d0/d2/d6/d31/d83 0 2024-03-29T04:45:05.287 INFO:tasks.workunit.client.0.smithi142.stdout:4/599: mknod d0/d9/d11/d4c/ccc 0 2024-03-29T04:45:05.291 INFO:tasks.workunit.client.0.smithi142.stdout:9/429: write d0/f32 [1486627,11496] 0 2024-03-29T04:45:05.292 INFO:tasks.workunit.client.0.smithi142.stdout:8/703: dwrite d5/fc5 [0,4194304] 0 2024-03-29T04:45:05.294 INFO:tasks.workunit.client.0.smithi142.stdout:9/430: stat d0/d2/c5 0 2024-03-29T04:45:05.294 INFO:tasks.workunit.client.0.smithi142.stdout:8/704: chown d5/d2e/dd2/fdd 209826 1 2024-03-29T04:45:05.294 INFO:tasks.workunit.client.0.smithi142.stdout:1/586: dread d8/db/dc/d77/db9/dbe/d2d/d44/d7f/fb0 [0,4194304] 0 2024-03-29T04:45:05.295 INFO:tasks.workunit.client.0.smithi142.stdout:8/705: chown d5/d1d/d2f/fd0 44 1 2024-03-29T04:45:05.295 INFO:tasks.workunit.client.0.smithi142.stdout:6/572: rename d1c/d33/d46/d8b to d1c/d33/db6 0 2024-03-29T04:45:05.297 INFO:tasks.workunit.client.0.smithi142.stdout:9/431: readlink d0/d2/d6/d16/l21 0 2024-03-29T04:45:05.300 INFO:tasks.workunit.client.0.smithi142.stdout:0/636: rename d7/d13/d4a/daf to d7/d13/dbf/dca/dab/dc1/dd0 0 2024-03-29T04:45:05.300 INFO:tasks.workunit.client.0.smithi142.stdout:0/637: stat d7/d9/fbb 0 2024-03-29T04:45:05.301 INFO:tasks.workunit.client.0.smithi142.stdout:1/587: fdatasync d8/db/dc/d77/db9/dbe/d1f/d97/d9e/fa7 0 2024-03-29T04:45:05.302 INFO:tasks.workunit.client.0.smithi142.stdout:9/432: mknod d0/d2/d61/d64/c84 0 2024-03-29T04:45:05.304 INFO:tasks.workunit.client.0.smithi142.stdout:0/638: creat d7/d13/d4a/d70/fd1 x:0 0 0 2024-03-29T04:45:05.305 INFO:tasks.workunit.client.0.smithi142.stdout:5/754: write df/d1a/d28/fae [1026077,36184] 0 2024-03-29T04:45:05.306 INFO:tasks.workunit.client.0.smithi142.stdout:0/639: chown d7/d9/d1e/lc6 5 1 2024-03-29T04:45:05.307 INFO:tasks.workunit.client.0.smithi142.stdout:4/600: truncate d0/d9/db2/dba/dbc/f92 3677611 0 2024-03-29T04:45:05.308 INFO:tasks.workunit.client.0.smithi142.stdout:1/588: mkdir d8/db/dc/d77/db9/dbe/d2d/d8b/da4/dd1 0 2024-03-29T04:45:05.311 INFO:tasks.workunit.client.0.smithi142.stdout:5/755: mknod df/d16/d24/d5c/db5/cf8 0 2024-03-29T04:45:05.312 INFO:tasks.workunit.client.0.smithi142.stdout:6/573: link d1c/d45/d51/f58 d1c/d45/fb7 0 2024-03-29T04:45:05.313 INFO:tasks.workunit.client.0.smithi142.stdout:6/574: stat d1c/l2a 0 2024-03-29T04:45:05.314 INFO:tasks.workunit.client.0.smithi142.stdout:6/575: dread - d1c/d33/d46/f69 zero size 2024-03-29T04:45:05.314 INFO:tasks.workunit.client.0.smithi142.stdout:5/756: truncate df/d1a/d21/d27/d45/f9a 872993 0 2024-03-29T04:45:05.316 INFO:tasks.workunit.client.0.smithi142.stdout:4/601: link d0/d9/d11/d97/d9e/fa2 d0/dc/d15/d3d/fcd 0 2024-03-29T04:45:05.317 INFO:tasks.workunit.client.0.smithi142.stdout:4/602: readlink d0/d9/d11/d4c/d5f/d81/dc9/l40 0 2024-03-29T04:45:05.317 INFO:tasks.workunit.client.0.smithi142.stdout:4/603: readlink d0/dc/d15/l2a 0 2024-03-29T04:45:05.318 INFO:tasks.workunit.client.0.smithi142.stdout:6/576: rmdir d1c/d33 39 2024-03-29T04:45:05.324 INFO:tasks.workunit.client.0.smithi142.stdout:4/604: rename d0/d9/d19/lc1 to d0/d9/d11/d4c/d64/d94/lce 0 2024-03-29T04:45:05.324 INFO:tasks.workunit.client.0.smithi142.stdout:9/433: dread d0/d2/f11 [0,4194304] 0 2024-03-29T04:45:05.325 INFO:tasks.workunit.client.0.smithi142.stdout:8/706: truncate d5/d1d/d2f/d5f/fe0 56619 0 2024-03-29T04:45:05.325 INFO:tasks.workunit.client.0.smithi142.stdout:9/434: read d0/f32 [402809,89715] 0 2024-03-29T04:45:05.326 INFO:tasks.workunit.client.0.smithi142.stdout:6/577: mknod d1c/d33/db6/d52/cb8 0 2024-03-29T04:45:05.328 INFO:tasks.workunit.client.0.smithi142.stdout:8/707: unlink d5/d1d/d2f/d5f/caf 0 2024-03-29T04:45:05.329 INFO:tasks.workunit.client.0.smithi142.stdout:4/605: rename d0/d9/d11/d4c/d64/f98 to d0/dc/d15/d3d/d65/fcf 0 2024-03-29T04:45:05.330 INFO:tasks.workunit.client.0.smithi142.stdout:1/589: dread d8/db/dc/d77/db9/dbe/d1f/f33 [0,4194304] 0 2024-03-29T04:45:05.331 INFO:tasks.workunit.client.0.smithi142.stdout:6/578: rmdir d1c/d33/db6 39 2024-03-29T04:45:05.331 INFO:tasks.workunit.client.0.smithi142.stdout:4/606: write d0/dc/f36 [266857,106748] 0 2024-03-29T04:45:05.332 INFO:tasks.workunit.client.0.smithi142.stdout:9/435: dwrite d0/d2/d6/d16/f37 [0,4194304] 0 2024-03-29T04:45:05.333 INFO:tasks.workunit.client.0.smithi142.stdout:8/708: rmdir d5 39 2024-03-29T04:45:05.338 INFO:tasks.workunit.client.0.smithi142.stdout:1/590: mknod d8/db/dc/d77/db9/dbe/cd2 0 2024-03-29T04:45:05.339 INFO:tasks.workunit.client.0.smithi142.stdout:9/436: mkdir d0/d2/d6/d31/d85 0 2024-03-29T04:45:05.340 INFO:tasks.workunit.client.0.smithi142.stdout:8/709: creat d5/d2e/ff4 x:0 0 0 2024-03-29T04:45:05.344 INFO:tasks.workunit.client.0.smithi142.stdout:9/437: chown d0/f1d 477 1 2024-03-29T04:45:05.348 INFO:tasks.workunit.client.0.smithi142.stdout:5/757: dwrite df/d16/f39 [0,4194304] 0 2024-03-29T04:45:05.348 INFO:tasks.workunit.client.0.smithi142.stdout:6/579: dwrite d1c/f5d [0,4194304] 0 2024-03-29T04:45:05.350 INFO:tasks.workunit.client.0.smithi142.stdout:1/591: dread - d8/db/dc/d77/db9/dbe/d2d/d44/d4b/fb2 zero size 2024-03-29T04:45:05.353 INFO:tasks.workunit.client.0.smithi142.stdout:8/710: symlink d5/d1d/d2f/d76/d2d/de2/lf5 0 2024-03-29T04:45:05.353 INFO:tasks.workunit.client.0.smithi142.stdout:4/607: dwrite d0/dc/f88 [0,4194304] 0 2024-03-29T04:45:05.354 INFO:tasks.workunit.client.0.smithi142.stdout:5/758: fdatasync df/d16/d24/d2e/d47/d8c/fba 0 2024-03-29T04:45:05.358 INFO:tasks.workunit.client.0.smithi142.stdout:8/711: write d5/d1d/fd8 [207086,107488] 0 2024-03-29T04:45:05.358 INFO:tasks.workunit.client.0.smithi142.stdout:1/592: fdatasync d8/db/dc/d77/db9/dbe/d15/f60 0 2024-03-29T04:45:05.360 INFO:tasks.workunit.client.0.smithi142.stdout:1/593: fsync d8/db/d5e/f8d 0 2024-03-29T04:45:05.361 INFO:tasks.workunit.client.0.smithi142.stdout:4/608: creat d0/d9/d11/d4c/d5f/fd0 x:0 0 0 2024-03-29T04:45:05.364 INFO:tasks.workunit.client.0.smithi142.stdout:9/438: dwrite d0/d2/d6/d27/d3c/f4c [0,4194304] 0 2024-03-29T04:45:05.367 INFO:tasks.workunit.client.0.smithi142.stdout:8/712: unlink d5/d93/lab 0 2024-03-29T04:45:05.368 INFO:tasks.workunit.client.0.smithi142.stdout:8/713: chown d5/c40 555681 1 2024-03-29T04:45:05.370 INFO:tasks.workunit.client.0.smithi142.stdout:9/439: creat d0/d12/f86 x:0 0 0 2024-03-29T04:45:05.370 INFO:tasks.workunit.client.0.smithi142.stdout:5/759: getdents df/d1a/d28 0 2024-03-29T04:45:05.372 INFO:tasks.workunit.client.0.smithi142.stdout:4/609: creat d0/fd1 x:0 0 0 2024-03-29T04:45:05.373 INFO:tasks.workunit.client.0.smithi142.stdout:8/714: unlink d5/d2e/cdb 0 2024-03-29T04:45:05.375 INFO:tasks.workunit.client.0.smithi142.stdout:5/760: creat df/d29/d7a/d97/ff9 x:0 0 0 2024-03-29T04:45:05.377 INFO:tasks.workunit.client.0.smithi142.stdout:8/715: mkdir d5/d2e/dd2/df6 0 2024-03-29T04:45:05.378 INFO:tasks.workunit.client.0.smithi142.stdout:4/610: dread d0/dc/d15/f72 [0,4194304] 0 2024-03-29T04:45:05.380 INFO:tasks.workunit.client.0.smithi142.stdout:5/761: symlink df/d29/lfa 0 2024-03-29T04:45:05.381 INFO:tasks.workunit.client.0.smithi142.stdout:8/716: creat d5/d2e/dd2/d60/ff7 x:0 0 0 2024-03-29T04:45:05.381 INFO:tasks.workunit.client.0.smithi142.stdout:6/580: dread fc [0,4194304] 0 2024-03-29T04:45:05.384 INFO:tasks.workunit.client.0.smithi142.stdout:7/694: sync 2024-03-29T04:45:05.385 INFO:tasks.workunit.client.0.smithi142.stdout:6/581: chown d1c/d33/db6/c4d 4804013 1 2024-03-29T04:45:05.389 INFO:tasks.workunit.client.0.smithi142.stdout:5/762: dwrite df/d29/d5d/d75/d90/fc2 [0,4194304] 0 2024-03-29T04:45:05.392 INFO:tasks.workunit.client.0.smithi142.stdout:1/594: dread d8/db/dc/d77/db9/dbe/d15/f16 [0,4194304] 0 2024-03-29T04:45:05.393 INFO:tasks.workunit.client.0.smithi142.stdout:8/717: dwrite d5/d2e/dd2/fdd [0,4194304] 0 2024-03-29T04:45:05.393 INFO:tasks.workunit.client.0.smithi142.stdout:6/582: link d1c/l82 d1c/d33/d46/d86/lb9 0 2024-03-29T04:45:05.395 INFO:tasks.workunit.client.0.smithi142.stdout:8/718: truncate d5/d1d/d2f/d76/d3c/f5e 4534170 0 2024-03-29T04:45:05.399 INFO:tasks.workunit.client.0.smithi142.stdout:7/695: link d7/f57 d7/d27/d44/db2/db8/d4b/d71/d30/d72/feb 0 2024-03-29T04:45:05.401 INFO:tasks.workunit.client.0.smithi142.stdout:9/440: dread d0/d2/d6/d16/f66 [0,4194304] 0 2024-03-29T04:45:05.401 INFO:tasks.workunit.client.0.smithi142.stdout:5/763: symlink df/d16/d24/d5c/db5/lfb 0 2024-03-29T04:45:05.402 INFO:tasks.workunit.client.0.smithi142.stdout:6/583: rename d1c to d1c/d33/dba 22 2024-03-29T04:45:05.405 INFO:tasks.workunit.client.0.smithi142.stdout:8/719: rename d5/c96 to d5/d1d/d59/cf8 0 2024-03-29T04:45:05.406 INFO:tasks.workunit.client.0.smithi142.stdout:6/584: symlink d1c/d90/d94/lbb 0 2024-03-29T04:45:05.407 INFO:tasks.workunit.client.0.smithi142.stdout:1/595: getdents d8/db 0 2024-03-29T04:45:05.408 INFO:tasks.workunit.client.0.smithi142.stdout:5/764: mkdir df/d2c/de9/dfc 0 2024-03-29T04:45:05.410 INFO:tasks.workunit.client.0.smithi142.stdout:6/585: write d1c/d33/d46/f48 [7444923,83879] 0 2024-03-29T04:45:05.411 INFO:tasks.workunit.client.0.smithi142.stdout:9/441: getdents d0/d2/d61 0 2024-03-29T04:45:05.412 INFO:tasks.workunit.client.0.smithi142.stdout:7/696: dwrite d7/d27/d44/db2/db8/d4b/d71/d30/d72/feb [0,4194304] 0 2024-03-29T04:45:05.413 INFO:tasks.workunit.client.0.smithi142.stdout:8/720: rename d5/d1d/d2f/d76/c15 to d5/d1d/d2f/d76/d9c/cf9 0 2024-03-29T04:45:05.414 INFO:tasks.workunit.client.0.smithi142.stdout:5/765: creat df/d29/d7a/d97/db3/ffd x:0 0 0 2024-03-29T04:45:05.417 INFO:tasks.workunit.client.0.smithi142.stdout:8/721: write d5/d1d/d2f/d76/f88 [2406890,69719] 0 2024-03-29T04:45:05.418 INFO:tasks.workunit.client.0.smithi142.stdout:6/586: write d1c/d33/db6/da4/faf [2718564,112601] 0 2024-03-29T04:45:05.419 INFO:tasks.workunit.client.0.smithi142.stdout:8/722: write d5/d1d/d59/fbe [4228920,38393] 0 2024-03-29T04:45:05.421 INFO:tasks.workunit.client.0.smithi142.stdout:5/766: write df/d16/d24/d2e/f8b [1537600,18339] 0 2024-03-29T04:45:05.421 INFO:tasks.workunit.client.0.smithi142.stdout:7/697: rename d7/d27/d44/db2/db8/d4b/d9b to d7/d27/d44/db2/db8/d4b/d71/dec 0 2024-03-29T04:45:05.421 INFO:tasks.workunit.client.0.smithi142.stdout:5/767: chown df/l34 3527 1 2024-03-29T04:45:05.423 INFO:tasks.workunit.client.0.smithi142.stdout:9/442: creat d0/d2/d6/d31/d85/f87 x:0 0 0 2024-03-29T04:45:05.425 INFO:tasks.workunit.client.0.smithi142.stdout:1/596: write d8/db/dc/d77/db9/dbe/d1f/d50/f87 [619382,567] 0 2024-03-29T04:45:05.427 INFO:tasks.workunit.client.0.smithi142.stdout:5/768: chown df/d1a/d21/da5/ca3 1752687 1 2024-03-29T04:45:05.427 INFO:tasks.workunit.client.0.smithi142.stdout:1/597: dread d8/db/dc/d77/db9/dbe/d2d/d44/d7f/fb0 [0,4194304] 0 2024-03-29T04:45:05.428 INFO:tasks.workunit.client.0.smithi142.stdout:1/598: fsync d8/db/dc/f2c 0 2024-03-29T04:45:05.429 INFO:tasks.workunit.client.0.smithi142.stdout:7/698: mknod d7/d27/d44/db2/db8/d4b/d71/dec/ced 0 2024-03-29T04:45:05.430 INFO:tasks.workunit.client.0.smithi142.stdout:9/443: write d0/d12/d50/f76 [1019721,29554] 0 2024-03-29T04:45:05.432 INFO:tasks.workunit.client.0.smithi142.stdout:6/587: unlink l16 0 2024-03-29T04:45:05.432 INFO:tasks.workunit.client.0.smithi142.stdout:8/723: mknod d5/d2e/dd2/df6/cfa 0 2024-03-29T04:45:05.433 INFO:tasks.workunit.client.0.smithi142.stdout:6/588: truncate d1c/d90/fa3 750859 0 2024-03-29T04:45:05.435 INFO:tasks.workunit.client.0.smithi142.stdout:1/599: chown d8/db/dc/d77/d5a/c3c 423615133 1 2024-03-29T04:45:05.437 INFO:tasks.workunit.client.0.smithi142.stdout:7/699: creat d7/d27/d44/db2/db8/de/dc1/fee x:0 0 0 2024-03-29T04:45:05.440 INFO:tasks.workunit.client.0.smithi142.stdout:8/724: rename d5/d1d/d2f/d76/d2d/l64 to d5/d1d/d2f/d76/d3c/d70/lfb 0 2024-03-29T04:45:05.441 INFO:tasks.workunit.client.0.smithi142.stdout:6/589: dwrite d1c/d33/d46/d71/fb3 [0,4194304] 0 2024-03-29T04:45:05.443 INFO:tasks.workunit.client.0.smithi142.stdout:6/590: chown d1c/d45/c50 485 1 2024-03-29T04:45:05.443 INFO:tasks.workunit.client.0.smithi142.stdout:5/769: link df/d29/d7a/fc4 df/d1a/ffe 0 2024-03-29T04:45:05.444 INFO:tasks.workunit.client.0.smithi142.stdout:8/725: write d5/d1d/d2f/d76/f88 [638180,68252] 0 2024-03-29T04:45:05.445 INFO:tasks.workunit.client.0.smithi142.stdout:1/600: rename d8/db/dc/d77/db9/dbe/d15/c55 to d8/db/cd3 0 2024-03-29T04:45:05.446 INFO:tasks.workunit.client.0.smithi142.stdout:5/770: write df/f14 [767722,77472] 0 2024-03-29T04:45:05.449 INFO:tasks.workunit.client.1.smithi152.stdout:6/40: sync 2024-03-29T04:45:05.450 INFO:tasks.workunit.client.1.smithi152.stdout:5/2: sync 2024-03-29T04:45:05.450 INFO:tasks.workunit.client.1.smithi152.stdout:8/5: sync 2024-03-29T04:45:05.450 INFO:tasks.workunit.client.1.smithi152.stdout:4/0: sync 2024-03-29T04:45:05.450 INFO:tasks.workunit.client.1.smithi152.stdout:9/8: sync 2024-03-29T04:45:05.450 INFO:tasks.workunit.client.1.smithi152.stdout:1/36: sync 2024-03-29T04:45:05.450 INFO:tasks.workunit.client.1.smithi152.stdout:3/14: sync 2024-03-29T04:45:05.450 INFO:tasks.workunit.client.1.smithi152.stdout:2/3: sync 2024-03-29T04:45:05.450 INFO:tasks.workunit.client.1.smithi152.stdout:2/4: dread - no filename 2024-03-29T04:45:05.450 INFO:tasks.workunit.client.1.smithi152.stdout:7/9: sync 2024-03-29T04:45:05.450 INFO:tasks.workunit.client.1.smithi152.stdout:0/77: sync 2024-03-29T04:45:05.450 INFO:tasks.workunit.client.1.smithi152.stdout:4/1: chown . 8427795 1 2024-03-29T04:45:05.450 INFO:tasks.workunit.client.1.smithi152.stdout:4/2: unlink - no file 2024-03-29T04:45:05.451 INFO:tasks.workunit.client.1.smithi152.stdout:4/3: link - no file 2024-03-29T04:45:05.451 INFO:tasks.workunit.client.1.smithi152.stdout:4/4: readlink - no filename 2024-03-29T04:45:05.451 INFO:tasks.workunit.client.1.smithi152.stdout:4/5: rename - no filename 2024-03-29T04:45:05.451 INFO:tasks.workunit.client.0.smithi142.stdout:9/444: dread d0/d2/d6/d16/f40 [0,4194304] 0 2024-03-29T04:45:05.452 INFO:tasks.workunit.client.1.smithi152.stdout:2/5: stat d0 0 2024-03-29T04:45:05.452 INFO:tasks.workunit.client.1.smithi152.stdout:2/6: write - no filename 2024-03-29T04:45:05.452 INFO:tasks.workunit.client.1.smithi152.stdout:2/7: dread - no filename 2024-03-29T04:45:05.453 INFO:tasks.workunit.client.1.smithi152.stdout:2/8: write - no filename 2024-03-29T04:45:05.453 INFO:tasks.workunit.client.1.smithi152.stdout:2/9: dread - no filename 2024-03-29T04:45:05.453 INFO:tasks.workunit.client.1.smithi152.stdout:2/10: write - no filename 2024-03-29T04:45:05.453 INFO:tasks.workunit.client.1.smithi152.stdout:5/3: chown f0 933 1 2024-03-29T04:45:05.453 INFO:tasks.workunit.client.1.smithi152.stdout:2/11: rename d0 to d0/d1 22 2024-03-29T04:45:05.453 INFO:tasks.workunit.client.1.smithi152.stdout:2/12: write - no filename 2024-03-29T04:45:05.453 INFO:tasks.workunit.client.1.smithi152.stdout:9/9: mknod c2 0 2024-03-29T04:45:05.454 INFO:tasks.workunit.client.1.smithi152.stdout:9/10: chown c2 9206617 1 2024-03-29T04:45:05.454 INFO:tasks.workunit.client.0.smithi142.stdout:9/445: stat d0/d2 0 2024-03-29T04:45:05.454 INFO:tasks.workunit.client.0.smithi142.stdout:7/700: dwrite d7/d27/d44/db2/db8/de/d20/d3f/f41 [0,4194304] 0 2024-03-29T04:45:05.454 INFO:tasks.workunit.client.0.smithi142.stdout:5/771: creat df/d16/d24/d5c/fff x:0 0 0 2024-03-29T04:45:05.455 INFO:tasks.workunit.client.1.smithi152.stdout:6/41: fdatasync d1/fe 0 2024-03-29T04:45:05.457 INFO:tasks.workunit.client.0.smithi142.stdout:8/726: dwrite d5/d1d/d2f/f50 [0,4194304] 0 2024-03-29T04:45:05.457 INFO:tasks.workunit.client.1.smithi152.stdout:5/4: fdatasync f0 0 2024-03-29T04:45:05.458 INFO:tasks.workunit.client.0.smithi142.stdout:6/591: rename d1c/d45/d8a/lb1 to d1c/d45/lbc 0 2024-03-29T04:45:05.460 INFO:tasks.workunit.client.1.smithi152.stdout:1/37: creat d5/d7/d9/f11 x:0 0 0 2024-03-29T04:45:05.460 INFO:tasks.workunit.client.0.smithi142.stdout:6/592: fsync d1c/d33/d46/d71/f68 0 2024-03-29T04:45:05.461 INFO:tasks.workunit.client.1.smithi152.stdout:3/15: creat f0 x:0 0 0 2024-03-29T04:45:05.462 INFO:tasks.workunit.client.0.smithi142.stdout:6/593: stat d1c/f62 0 2024-03-29T04:45:05.462 INFO:tasks.workunit.client.0.smithi142.stdout:6/594: fsync f4 0 2024-03-29T04:45:05.463 INFO:tasks.workunit.client.1.smithi152.stdout:4/6: mkdir d0 0 2024-03-29T04:45:05.466 INFO:tasks.workunit.client.1.smithi152.stdout:3/16: write f0 [105682,95905] 0 2024-03-29T04:45:05.466 INFO:tasks.workunit.client.1.smithi152.stdout:8/6: getdents d1/d2/d3 0 2024-03-29T04:45:05.467 INFO:tasks.workunit.client.1.smithi152.stdout:6/42: mkdir d1/d3/d9/d13 0 2024-03-29T04:45:05.467 INFO:tasks.workunit.client.1.smithi152.stdout:9/11: creat f3 x:0 0 0 2024-03-29T04:45:05.469 INFO:tasks.workunit.client.0.smithi142.stdout:2/566: sync 2024-03-29T04:45:05.470 INFO:tasks.workunit.client.1.smithi152.stdout:5/5: creat f1 x:0 0 0 2024-03-29T04:45:05.470 INFO:tasks.workunit.client.1.smithi152.stdout:5/6: rmdir - no directory 2024-03-29T04:45:05.471 INFO:tasks.workunit.client.1.smithi152.stdout:5/7: dread - f1 zero size 2024-03-29T04:45:05.471 INFO:tasks.workunit.client.1.smithi152.stdout:5/8: chown f1 13 1 2024-03-29T04:45:05.472 INFO:tasks.workunit.client.1.smithi152.stdout:2/13: creat d0/f2 x:0 0 0 2024-03-29T04:45:05.473 INFO:tasks.workunit.client.1.smithi152.stdout:7/10: rmdir d3 0 2024-03-29T04:45:05.473 INFO:tasks.workunit.client.1.smithi152.stdout:7/11: dwrite - no filename 2024-03-29T04:45:05.473 INFO:tasks.workunit.client.1.smithi152.stdout:7/12: dwrite - no filename 2024-03-29T04:45:05.474 INFO:tasks.workunit.client.1.smithi152.stdout:6/43: dread - d1/d3/d9/fb zero size 2024-03-29T04:45:05.476 INFO:tasks.workunit.client.1.smithi152.stdout:8/7: mknod d1/d2/d3/c4 0 2024-03-29T04:45:05.478 INFO:tasks.workunit.client.1.smithi152.stdout:7/13: creat f4 x:0 0 0 2024-03-29T04:45:05.479 INFO:tasks.workunit.client.1.smithi152.stdout:0/78: getdents d2 0 2024-03-29T04:45:05.479 INFO:tasks.workunit.client.1.smithi152.stdout:5/9: dwrite f1 [0,4194304] 0 2024-03-29T04:45:05.480 INFO:tasks.workunit.client.1.smithi152.stdout:6/44: dread - d1/d3/df/f10 zero size 2024-03-29T04:45:05.480 INFO:tasks.workunit.client.1.smithi152.stdout:5/10: dread - f0 zero size 2024-03-29T04:45:05.481 INFO:tasks.workunit.client.1.smithi152.stdout:9/12: link c2 c4 0 2024-03-29T04:45:05.482 INFO:tasks.workunit.client.1.smithi152.stdout:6/45: chown d1/d3/df/f10 8 1 2024-03-29T04:45:05.483 INFO:tasks.workunit.client.1.smithi152.stdout:8/8: creat d1/d2/d3/f5 x:0 0 0 2024-03-29T04:45:05.484 INFO:tasks.workunit.client.1.smithi152.stdout:7/14: mknod c5 0 2024-03-29T04:45:05.485 INFO:tasks.workunit.client.0.smithi142.stdout:8/727: creat d5/d1d/d59/ffc x:0 0 0 2024-03-29T04:45:05.485 INFO:tasks.workunit.client.1.smithi152.stdout:7/15: write f4 [771937,35833] 0 2024-03-29T04:45:05.486 INFO:tasks.workunit.client.1.smithi152.stdout:3/17: getdents . 0 2024-03-29T04:45:05.486 INFO:tasks.workunit.client.0.smithi142.stdout:9/446: rename d0/d2/d6/d27/d3c/d4f/l6d to d0/d2/d6/d27/l88 0 2024-03-29T04:45:05.487 INFO:tasks.workunit.client.0.smithi142.stdout:6/595: chown d1c/d90/d94/fa2 9023690 1 2024-03-29T04:45:05.488 INFO:tasks.workunit.client.1.smithi152.stdout:3/18: dread f0 [0,4194304] 0 2024-03-29T04:45:05.488 INFO:tasks.workunit.client.1.smithi152.stdout:3/19: rmdir - no directory 2024-03-29T04:45:05.489 INFO:tasks.workunit.client.0.smithi142.stdout:2/567: write de/d11/d17/d5d/d71/f9f [511097,115031] 0 2024-03-29T04:45:05.489 INFO:tasks.workunit.client.1.smithi152.stdout:0/79: symlink d2/da/d17/l1e 0 2024-03-29T04:45:05.489 INFO:tasks.workunit.client.1.smithi152.stdout:3/20: chown f0 156990 1 2024-03-29T04:45:05.491 INFO:tasks.workunit.client.1.smithi152.stdout:5/11: mknod c2 0 2024-03-29T04:45:05.492 INFO:tasks.workunit.client.1.smithi152.stdout:9/13: mkdir d5 0 2024-03-29T04:45:05.492 INFO:tasks.workunit.client.0.smithi142.stdout:6/596: creat d1c/d33/d46/d61/d7e/fbd x:0 0 0 2024-03-29T04:45:05.494 INFO:tasks.workunit.client.0.smithi142.stdout:7/701: link d7/d27/d44/db2/dd2/le2 d7/d27/d44/db2/db8/d4b/d71/d30/lef 0 2024-03-29T04:45:05.494 INFO:tasks.workunit.client.0.smithi142.stdout:7/702: write d7/f19 [1438623,7904] 0 2024-03-29T04:45:05.496 INFO:tasks.workunit.client.1.smithi152.stdout:9/14: chown f3 16769 1 2024-03-29T04:45:05.496 INFO:tasks.workunit.client.1.smithi152.stdout:7/16: dwrite f4 [0,4194304] 0 2024-03-29T04:45:05.496 INFO:tasks.workunit.client.1.smithi152.stdout:2/14: rmdir d0 39 2024-03-29T04:45:05.497 INFO:tasks.workunit.client.1.smithi152.stdout:2/15: readlink - no filename 2024-03-29T04:45:05.498 INFO:tasks.workunit.client.0.smithi142.stdout:2/568: creat de/d11/d1e/d22/d27/dba/d61/d7d/fbe x:0 0 0 2024-03-29T04:45:05.498 INFO:tasks.workunit.client.1.smithi152.stdout:9/15: chown d5 1285424 1 2024-03-29T04:45:05.499 INFO:tasks.workunit.client.1.smithi152.stdout:7/17: chown f4 37998 1 2024-03-29T04:45:05.499 INFO:tasks.workunit.client.1.smithi152.stdout:9/16: fdatasync f3 0 2024-03-29T04:45:05.500 INFO:tasks.workunit.client.1.smithi152.stdout:0/80: creat d2/f1f x:0 0 0 2024-03-29T04:45:05.500 INFO:tasks.workunit.client.0.smithi142.stdout:6/597: creat d1c/d33/d46/d61/d9e/fbe x:0 0 0 2024-03-29T04:45:05.502 INFO:tasks.workunit.client.1.smithi152.stdout:3/21: creat f1 x:0 0 0 2024-03-29T04:45:05.502 INFO:tasks.workunit.client.1.smithi152.stdout:7/18: write f4 [603991,71449] 0 2024-03-29T04:45:05.503 INFO:tasks.workunit.client.0.smithi142.stdout:9/447: rmdir d0/d2/d6b 0 2024-03-29T04:45:05.504 INFO:tasks.workunit.client.1.smithi152.stdout:1/38: dwrite d5/d7/fe [0,4194304] 0 2024-03-29T04:45:05.504 INFO:tasks.workunit.client.1.smithi152.stdout:5/12: creat f3 x:0 0 0 2024-03-29T04:45:05.505 INFO:tasks.workunit.client.0.smithi142.stdout:2/569: mkdir de/d11/d1e/d22/dbf 0 2024-03-29T04:45:05.505 INFO:tasks.workunit.client.1.smithi152.stdout:2/16: stat d0 0 2024-03-29T04:45:05.507 INFO:tasks.workunit.client.0.smithi142.stdout:2/570: chown de/d11/d1e/f72 0 1 2024-03-29T04:45:05.510 INFO:tasks.workunit.client.0.smithi142.stdout:1/601: write d8/db/dc/d77/db9/dbe/d2d/f7d [654292,65882] 0 2024-03-29T04:45:05.511 INFO:tasks.workunit.client.0.smithi142.stdout:9/448: rmdir d0/d12/d50 39 2024-03-29T04:45:05.512 INFO:tasks.workunit.client.1.smithi152.stdout:9/17: fdatasync f1 0 2024-03-29T04:45:05.512 INFO:tasks.workunit.client.1.smithi152.stdout:3/22: rename f0 to f2 0 2024-03-29T04:45:05.514 INFO:tasks.workunit.client.0.smithi142.stdout:1/602: write d8/db/dc/d77/db9/dbe/d15/fa0 [115563,115089] 0 2024-03-29T04:45:05.514 INFO:tasks.workunit.client.1.smithi152.stdout:3/23: truncate f1 233739 0 2024-03-29T04:45:05.516 INFO:tasks.workunit.client.1.smithi152.stdout:0/81: truncate d2/f7 1398381 0 2024-03-29T04:45:05.517 INFO:tasks.workunit.client.1.smithi152.stdout:1/39: mkdir d5/d7/d9/d12 0 2024-03-29T04:45:05.517 INFO:tasks.workunit.client.0.smithi142.stdout:1/603: mknod d8/db/dc/d77/db9/dbe/d2d/d8b/cd4 0 2024-03-29T04:45:05.518 INFO:tasks.workunit.client.1.smithi152.stdout:6/46: dwrite d1/fe [0,4194304] 0 2024-03-29T04:45:05.518 INFO:tasks.workunit.client.1.smithi152.stdout:2/17: mkdir d0/d3 0 2024-03-29T04:45:05.518 INFO:tasks.workunit.client.0.smithi142.stdout:9/449: symlink d0/d12/l89 0 2024-03-29T04:45:05.519 INFO:tasks.workunit.client.1.smithi152.stdout:8/9: getdents d1/d2/d3 0 2024-03-29T04:45:05.522 INFO:tasks.workunit.client.0.smithi142.stdout:9/450: chown d0/d2/d6/d16/f66 55 1 2024-03-29T04:45:05.522 INFO:tasks.workunit.client.0.smithi142.stdout:7/703: dwrite d7/d27/d44/db2/db8/de/f26 [0,4194304] 0 2024-03-29T04:45:05.523 INFO:tasks.workunit.client.1.smithi152.stdout:3/24: write f2 [87821,63303] 0 2024-03-29T04:45:05.525 INFO:tasks.workunit.client.1.smithi152.stdout:0/82: fdatasync d2/f8 0 2024-03-29T04:45:05.527 INFO:tasks.workunit.client.0.smithi142.stdout:9/451: symlink d0/d2/d6/d27/d3c/d4f/l8a 0 2024-03-29T04:45:05.529 INFO:tasks.workunit.client.1.smithi152.stdout:6/47: dread - d1/fd zero size 2024-03-29T04:45:05.531 INFO:tasks.workunit.client.0.smithi142.stdout:7/704: rename d7/d27/d44/db2/db8/de/c6f to d7/d27/d44/db2/db8/de/d20/d86/dd4/cf0 0 2024-03-29T04:45:05.531 INFO:tasks.workunit.client.1.smithi152.stdout:2/18: dwrite d0/f2 [0,4194304] 0 2024-03-29T04:45:05.532 INFO:tasks.workunit.client.1.smithi152.stdout:8/10: creat d1/f6 x:0 0 0 2024-03-29T04:45:05.534 INFO:tasks.workunit.client.1.smithi152.stdout:2/19: chown d0 64 1 2024-03-29T04:45:05.534 INFO:tasks.workunit.client.1.smithi152.stdout:8/11: chown l0 14998 1 2024-03-29T04:45:05.534 INFO:tasks.workunit.client.1.smithi152.stdout:2/20: stat d0/d3 0 2024-03-29T04:45:05.535 INFO:tasks.workunit.client.0.smithi142.stdout:9/452: truncate d0/f32 199926 0 2024-03-29T04:45:05.535 INFO:tasks.workunit.client.0.smithi142.stdout:6/598: write d1c/d33/db6/d52/f7b [938409,6481] 0 2024-03-29T04:45:05.535 INFO:tasks.workunit.client.1.smithi152.stdout:2/21: write d0/f2 [934875,91601] 0 2024-03-29T04:45:05.536 INFO:tasks.workunit.client.0.smithi142.stdout:9/453: truncate d0/d6c/f79 238752 0 2024-03-29T04:45:05.538 INFO:tasks.workunit.client.0.smithi142.stdout:2/571: stat de/d11/d1e/d22/d27 0 2024-03-29T04:45:05.539 INFO:tasks.workunit.client.0.smithi142.stdout:7/705: dread d7/d27/d44/db2/db8/de/d20/d3f/f41 [0,4194304] 0 2024-03-29T04:45:05.542 INFO:tasks.workunit.client.0.smithi142.stdout:6/599: unlink d1c/d45/d51/l78 0 2024-03-29T04:45:05.542 INFO:tasks.workunit.client.1.smithi152.stdout:0/83: creat d2/f20 x:0 0 0 2024-03-29T04:45:05.542 INFO:tasks.workunit.client.1.smithi152.stdout:7/19: write f4 [4904289,106408] 0 2024-03-29T04:45:05.542 INFO:tasks.workunit.client.1.smithi152.stdout:6/48: fdatasync d1/d3/d9/f11 0 2024-03-29T04:45:05.544 INFO:tasks.workunit.client.1.smithi152.stdout:7/20: stat c2 0 2024-03-29T04:45:05.544 INFO:tasks.workunit.client.0.smithi142.stdout:7/706: unlink d7/c8 0 2024-03-29T04:45:05.547 INFO:tasks.workunit.client.0.smithi142.stdout:2/572: mknod de/d11/d1e/d22/d50/cc0 0 2024-03-29T04:45:05.548 INFO:tasks.workunit.client.1.smithi152.stdout:3/25: unlink f1 0 2024-03-29T04:45:05.548 INFO:tasks.workunit.client.1.smithi152.stdout:2/22: dwrite d0/f2 [0,4194304] 0 2024-03-29T04:45:05.549 INFO:tasks.workunit.client.0.smithi142.stdout:7/707: rename d7/f90 to d7/d62/ff1 0 2024-03-29T04:45:05.550 INFO:tasks.workunit.client.0.smithi142.stdout:7/708: stat d7/d27/d44/db2/db8/de/d20/d3f/ccf 0 2024-03-29T04:45:05.551 INFO:tasks.workunit.client.0.smithi142.stdout:9/454: unlink d0/d2/d61/l5b 0 2024-03-29T04:45:05.552 INFO:tasks.workunit.client.0.smithi142.stdout:7/709: read - d7/d27/d44/db2/db8/d4b/d71/f9d zero size 2024-03-29T04:45:05.557 INFO:tasks.workunit.client.0.smithi142.stdout:2/573: symlink de/d11/d1e/d22/d27/dba/lc1 0 2024-03-29T04:45:05.558 INFO:tasks.workunit.client.1.smithi152.stdout:5/13: dwrite f0 [0,4194304] 0 2024-03-29T04:45:05.558 INFO:tasks.workunit.client.1.smithi152.stdout:9/18: write f0 [227408,40638] 0 2024-03-29T04:45:05.558 INFO:tasks.workunit.client.0.smithi142.stdout:9/455: rmdir d0/d2/d6 39 2024-03-29T04:45:05.559 INFO:tasks.workunit.client.0.smithi142.stdout:1/604: dread d8/db/dc/d77/d31/f3f [0,4194304] 0 2024-03-29T04:45:05.561 INFO:tasks.workunit.client.1.smithi152.stdout:0/84: dread d2/da/fb [0,4194304] 0 2024-03-29T04:45:05.561 INFO:tasks.workunit.client.1.smithi152.stdout:8/12: rmdir d1 39 2024-03-29T04:45:05.563 INFO:tasks.workunit.client.0.smithi142.stdout:7/710: link d7/d27/d44/db2/db8/d4b/d71/f28 d7/d27/d3a/d92/d97/ff2 0 2024-03-29T04:45:05.563 INFO:tasks.workunit.client.0.smithi142.stdout:7/711: stat d7/l66 0 2024-03-29T04:45:05.566 INFO:tasks.workunit.client.1.smithi152.stdout:2/23: mknod d0/d3/c4 0 2024-03-29T04:45:05.566 INFO:tasks.workunit.client.1.smithi152.stdout:7/21: dread f4 [4194304,4194304] 0 2024-03-29T04:45:05.568 INFO:tasks.workunit.client.1.smithi152.stdout:5/14: write f0 [1541221,29568] 0 2024-03-29T04:45:05.569 INFO:tasks.workunit.client.1.smithi152.stdout:0/85: stat d2/da/d17/l1e 0 2024-03-29T04:45:05.569 INFO:tasks.workunit.client.1.smithi152.stdout:7/22: chown l1 414561480 1 2024-03-29T04:45:05.571 INFO:tasks.workunit.client.0.smithi142.stdout:7/712: fsync d7/d27/d44/db2/db8/de/d20/f51 0 2024-03-29T04:45:05.572 INFO:tasks.workunit.client.0.smithi142.stdout:6/600: write d1c/d33/db6/d52/f95 [921319,128282] 0 2024-03-29T04:45:05.572 INFO:tasks.workunit.client.1.smithi152.stdout:9/19: creat d5/f6 x:0 0 0 2024-03-29T04:45:05.573 INFO:tasks.workunit.client.0.smithi142.stdout:2/574: mknod de/d11/d1e/d84/cc2 0 2024-03-29T04:45:05.575 INFO:tasks.workunit.client.1.smithi152.stdout:8/13: fsync d1/d2/d3/f5 0 2024-03-29T04:45:05.576 INFO:tasks.workunit.client.1.smithi152.stdout:2/24: creat d0/d3/f5 x:0 0 0 2024-03-29T04:45:05.576 INFO:tasks.workunit.client.0.smithi142.stdout:9/456: symlink d0/d2/d6/d27/d3c/d7b/l8b 0 2024-03-29T04:45:05.576 INFO:tasks.workunit.client.1.smithi152.stdout:5/15: symlink l4 0 2024-03-29T04:45:05.576 INFO:tasks.workunit.client.1.smithi152.stdout:0/86: chown d2/da/c10 693247 1 2024-03-29T04:45:05.577 INFO:tasks.workunit.client.0.smithi142.stdout:9/457: readlink d0/d2/d6/d27/d3c/d7b/l8b 0 2024-03-29T04:45:05.578 INFO:tasks.workunit.client.1.smithi152.stdout:1/40: dwrite d5/d7/d8/fb [0,4194304] 0 2024-03-29T04:45:05.582 INFO:tasks.workunit.client.1.smithi152.stdout:9/20: unlink f0 0 2024-03-29T04:45:05.583 INFO:tasks.workunit.client.1.smithi152.stdout:3/26: dwrite f2 [0,4194304] 0 2024-03-29T04:45:05.584 INFO:tasks.workunit.client.0.smithi142.stdout:6/601: creat d1c/d33/d46/d66/fbf x:0 0 0 2024-03-29T04:45:05.586 INFO:tasks.workunit.client.0.smithi142.stdout:7/713: mknod d7/d27/d44/db2/db8/d4b/d71/d30/cf3 0 2024-03-29T04:45:05.587 INFO:tasks.workunit.client.1.smithi152.stdout:1/41: write d5/d7/fe [2192038,25313] 0 2024-03-29T04:45:05.593 INFO:tasks.workunit.client.1.smithi152.stdout:9/21: creat d5/f7 x:0 0 0 2024-03-29T04:45:05.594 INFO:tasks.workunit.client.0.smithi142.stdout:6/602: dwrite d1c/d90/d94/fa2 [0,4194304] 0 2024-03-29T04:45:05.596 INFO:tasks.workunit.client.0.smithi142.stdout:9/458: creat d0/d2/d6/d27/d3c/d4f/f8c x:0 0 0 2024-03-29T04:45:05.596 INFO:tasks.workunit.client.1.smithi152.stdout:6/49: dwrite d1/f8 [0,4194304] 0 2024-03-29T04:45:05.597 INFO:tasks.workunit.client.0.smithi142.stdout:6/603: fsync d1c/f39 0 2024-03-29T04:45:05.599 INFO:tasks.workunit.client.1.smithi152.stdout:9/22: dread - f3 zero size 2024-03-29T04:45:05.599 INFO:tasks.workunit.client.1.smithi152.stdout:1/42: stat d5/d7/d8 0 2024-03-29T04:45:05.600 INFO:tasks.workunit.client.0.smithi142.stdout:9/459: fdatasync d0/d2/d6/d16/f3b 0 2024-03-29T04:45:05.601 INFO:tasks.workunit.client.0.smithi142.stdout:2/575: dwrite de/d11/d1e/d22/f8e [0,4194304] 0 2024-03-29T04:45:05.602 INFO:tasks.workunit.client.0.smithi142.stdout:1/605: write d8/f9 [2454572,108340] 0 2024-03-29T04:45:05.602 INFO:tasks.workunit.client.1.smithi152.stdout:6/50: rename d1 to d1/d14 22 2024-03-29T04:45:05.604 INFO:tasks.workunit.client.0.smithi142.stdout:2/576: fsync de/d11/d17/d5d/f86 0 2024-03-29T04:45:05.605 INFO:tasks.workunit.client.0.smithi142.stdout:1/606: write d8/db/dc/d77/db9/dbe/d15/f60 [1218262,114798] 0 2024-03-29T04:45:05.606 INFO:tasks.workunit.client.0.smithi142.stdout:9/460: symlink d0/d2/d61/l8d 0 2024-03-29T04:45:05.606 INFO:tasks.workunit.client.0.smithi142.stdout:1/607: fdatasync f5 0 2024-03-29T04:45:05.607 INFO:tasks.workunit.client.1.smithi152.stdout:9/23: fdatasync d5/f6 0 2024-03-29T04:45:05.611 INFO:tasks.workunit.client.1.smithi152.stdout:1/43: mkdir d5/d7/d8/d13 0 2024-03-29T04:45:05.612 INFO:tasks.workunit.client.1.smithi152.stdout:7/23: dwrite f4 [0,4194304] 0 2024-03-29T04:45:05.613 INFO:tasks.workunit.client.1.smithi152.stdout:5/16: dread f1 [0,4194304] 0 2024-03-29T04:45:05.617 INFO:tasks.workunit.client.1.smithi152.stdout:2/25: dwrite d0/f2 [0,4194304] 0 2024-03-29T04:45:05.617 INFO:tasks.workunit.client.1.smithi152.stdout:6/51: creat d1/d3/f15 x:0 0 0 2024-03-29T04:45:05.619 INFO:tasks.workunit.client.0.smithi142.stdout:0/640: sync 2024-03-29T04:45:05.619 INFO:tasks.workunit.client.0.smithi142.stdout:4/611: sync 2024-03-29T04:45:05.619 INFO:tasks.workunit.client.0.smithi142.stdout:3/483: sync 2024-03-29T04:45:05.620 INFO:tasks.workunit.client.0.smithi142.stdout:0/641: stat d7/d9/d54/d58 0 2024-03-29T04:45:05.620 INFO:tasks.workunit.client.0.smithi142.stdout:1/608: rename d8/db/dc/d19/f57 to d8/db/dc/d77/db9/dbe/d1f/d97/fd5 0 2024-03-29T04:45:05.620 INFO:tasks.workunit.client.1.smithi152.stdout:8/14: dwrite d1/d2/d3/f5 [0,4194304] 0 2024-03-29T04:45:05.621 INFO:tasks.workunit.client.0.smithi142.stdout:3/484: chown d3/d1f/d24/d47/d87/fa3 0 1 2024-03-29T04:45:05.621 INFO:tasks.workunit.client.0.smithi142.stdout:3/485: readlink d3/d29/d8f/l9e 0 2024-03-29T04:45:05.622 INFO:tasks.workunit.client.0.smithi142.stdout:9/461: mkdir d0/d12/d50/d8e 0 2024-03-29T04:45:05.623 INFO:tasks.workunit.client.1.smithi152.stdout:3/27: dwrite f2 [0,4194304] 0 2024-03-29T04:45:05.623 INFO:tasks.workunit.client.0.smithi142.stdout:9/462: chown d0/d6c 16942 1 2024-03-29T04:45:05.624 INFO:tasks.workunit.client.0.smithi142.stdout:1/609: truncate d8/db/dc/d77/db9/dbe/d2d/d8b/da4/fba 628689 0 2024-03-29T04:45:05.627 INFO:tasks.workunit.client.0.smithi142.stdout:2/577: link de/d11/d1e/d22/d27/dba/d61/fa0 de/d11/d1e/d22/d50/fc3 0 2024-03-29T04:45:05.628 INFO:tasks.workunit.client.0.smithi142.stdout:4/612: fdatasync d0/d12/f50 0 2024-03-29T04:45:05.628 INFO:tasks.workunit.client.0.smithi142.stdout:2/578: readlink de/d11/d17/l41 0 2024-03-29T04:45:05.630 INFO:tasks.workunit.client.1.smithi152.stdout:0/87: dwrite d2/f8 [4194304,4194304] 0 2024-03-29T04:45:05.634 INFO:tasks.workunit.client.1.smithi152.stdout:5/17: truncate f1 4292392 0 2024-03-29T04:45:05.635 INFO:tasks.workunit.client.0.smithi142.stdout:0/642: dread d7/d9/f94 [0,4194304] 0 2024-03-29T04:45:05.635 INFO:tasks.workunit.client.1.smithi152.stdout:2/26: mknod d0/d3/c6 0 2024-03-29T04:45:05.636 INFO:tasks.workunit.client.1.smithi152.stdout:3/28: mknod c3 0 2024-03-29T04:45:05.637 INFO:tasks.workunit.client.0.smithi142.stdout:0/643: truncate d7/d13/d4a/d70/fd1 753954 0 2024-03-29T04:45:05.641 INFO:tasks.workunit.client.1.smithi152.stdout:5/18: creat f5 x:0 0 0 2024-03-29T04:45:05.641 INFO:tasks.workunit.client.1.smithi152.stdout:5/19: rmdir - no directory 2024-03-29T04:45:05.643 INFO:tasks.workunit.client.0.smithi142.stdout:9/463: creat d0/d2/d6/d27/d3c/d7b/f8f x:0 0 0 2024-03-29T04:45:05.643 INFO:tasks.workunit.client.1.smithi152.stdout:2/27: symlink d0/d3/l7 0 2024-03-29T04:45:05.644 INFO:tasks.workunit.client.1.smithi152.stdout:2/28: chown d0 70312169 1 2024-03-29T04:45:05.644 INFO:tasks.workunit.client.0.smithi142.stdout:1/610: truncate d8/db/fb3 4477050 0 2024-03-29T04:45:05.645 INFO:tasks.workunit.client.1.smithi152.stdout:5/20: mkdir d6 0 2024-03-29T04:45:05.645 INFO:tasks.workunit.client.0.smithi142.stdout:2/579: chown de/d11/d1e/d22/l2e 4113476 1 2024-03-29T04:45:05.646 INFO:tasks.workunit.client.1.smithi152.stdout:1/44: link c3 d5/d7/d9/c14 0 2024-03-29T04:45:05.647 INFO:tasks.workunit.client.1.smithi152.stdout:3/29: dwrite f2 [4194304,4194304] 0 2024-03-29T04:45:05.648 INFO:tasks.workunit.client.0.smithi142.stdout:9/464: creat d0/d12/d50/f90 x:0 0 0 2024-03-29T04:45:05.649 INFO:tasks.workunit.client.0.smithi142.stdout:4/613: symlink d0/d9/db2/dba/dbc/daf/ld2 0 2024-03-29T04:45:05.650 INFO:tasks.workunit.client.1.smithi152.stdout:2/29: truncate d0/d3/f5 980424 0 2024-03-29T04:45:05.651 INFO:tasks.workunit.client.0.smithi142.stdout:4/614: chown d0/fd1 408883428 1 2024-03-29T04:45:05.651 INFO:tasks.workunit.client.1.smithi152.stdout:5/21: mkdir d6/d7 0 2024-03-29T04:45:05.652 INFO:tasks.workunit.client.0.smithi142.stdout:1/611: mkdir d8/db/dd6 0 2024-03-29T04:45:05.652 INFO:tasks.workunit.client.0.smithi142.stdout:2/580: dwrite fd [4194304,4194304] 0 2024-03-29T04:45:05.653 INFO:tasks.workunit.client.0.smithi142.stdout:4/615: fsync d0/d9/d19/d2c/f56 0 2024-03-29T04:45:05.655 INFO:tasks.workunit.client.1.smithi152.stdout:1/45: link c3 d5/c15 0 2024-03-29T04:45:05.657 INFO:tasks.workunit.client.0.smithi142.stdout:2/581: mkdir de/d11/d17/d60/dc4 0 2024-03-29T04:45:05.657 INFO:tasks.workunit.client.1.smithi152.stdout:9/24: dwrite f1 [0,4194304] 0 2024-03-29T04:45:05.658 INFO:tasks.workunit.client.1.smithi152.stdout:2/30: dread d0/d3/f5 [0,4194304] 0 2024-03-29T04:45:05.661 INFO:tasks.workunit.client.1.smithi152.stdout:5/22: creat d6/d7/f8 x:0 0 0 2024-03-29T04:45:05.661 INFO:tasks.workunit.client.0.smithi142.stdout:4/616: rename d0/d9/d19/lbe to d0/d9/db2/dba/dbc/d8d/da5/ld3 0 2024-03-29T04:45:05.662 INFO:tasks.workunit.client.1.smithi152.stdout:2/31: mkdir d0/d8 0 2024-03-29T04:45:05.663 INFO:tasks.workunit.client.1.smithi152.stdout:9/25: mkdir d5/d8 0 2024-03-29T04:45:05.664 INFO:tasks.workunit.client.0.smithi142.stdout:4/617: write d0/dc/f93 [506179,45982] 0 2024-03-29T04:45:05.665 INFO:tasks.workunit.client.0.smithi142.stdout:9/465: getdents d0/d2/d6/d31/d85 0 2024-03-29T04:45:05.667 INFO:tasks.workunit.client.0.smithi142.stdout:9/466: write d0/d12/d50/f63 [1663573,118536] 0 2024-03-29T04:45:05.667 INFO:tasks.workunit.client.1.smithi152.stdout:9/26: truncate d5/f6 1009316 0 2024-03-29T04:45:05.667 INFO:tasks.workunit.client.0.smithi142.stdout:2/582: creat de/d11/d1e/d22/d27/dba/d61/d7d/db4/fc5 x:0 0 0 2024-03-29T04:45:05.671 INFO:tasks.workunit.client.1.smithi152.stdout:9/27: creat d5/d8/f9 x:0 0 0 2024-03-29T04:45:05.672 INFO:tasks.workunit.client.0.smithi142.stdout:2/583: chown de/d11/d1e/d22/d27/dba/d61/d7d 12070 1 2024-03-29T04:45:05.673 INFO:tasks.workunit.client.1.smithi152.stdout:2/32: dwrite d0/d3/f5 [0,4194304] 0 2024-03-29T04:45:05.674 INFO:tasks.workunit.client.0.smithi142.stdout:4/618: dwrite d0/d9/d11/d4c/d5f/f6f [0,4194304] 0 2024-03-29T04:45:05.676 INFO:tasks.workunit.client.0.smithi142.stdout:9/467: dread d0/d2/f11 [0,4194304] 0 2024-03-29T04:45:05.678 INFO:tasks.workunit.client.1.smithi152.stdout:9/28: rename c2 to d5/d8/ca 0 2024-03-29T04:45:05.679 INFO:tasks.workunit.client.0.smithi142.stdout:2/584: creat de/d11/d17/d60/fc6 x:0 0 0 2024-03-29T04:45:05.679 INFO:tasks.workunit.client.0.smithi142.stdout:4/619: symlink d0/ld4 0 2024-03-29T04:45:05.682 INFO:tasks.workunit.client.1.smithi152.stdout:2/33: rmdir d0/d8 0 2024-03-29T04:45:05.683 INFO:tasks.workunit.client.1.smithi152.stdout:7/24: fsync f4 0 2024-03-29T04:45:05.684 INFO:tasks.workunit.client.0.smithi142.stdout:4/620: rmdir d0/d9/db2/dba/dbc/d8d/da5 39 2024-03-29T04:45:05.686 INFO:tasks.workunit.client.0.smithi142.stdout:4/621: write d0/d9/d11/d4c/d5f/d81/dc9/d47/f74 [378797,16687] 0 2024-03-29T04:45:05.687 INFO:tasks.workunit.client.0.smithi142.stdout:9/468: rename d0/d2/d6/d27/l5d to d0/d2/d6/l91 0 2024-03-29T04:45:05.688 INFO:tasks.workunit.client.1.smithi152.stdout:2/34: creat d0/f9 x:0 0 0 2024-03-29T04:45:05.690 INFO:tasks.workunit.client.0.smithi142.stdout:7/714: truncate d7/d27/d44/db2/db8/de/f5c 2586972 0 2024-03-29T04:45:05.691 INFO:tasks.workunit.client.1.smithi152.stdout:7/25: unlink c5 0 2024-03-29T04:45:05.691 INFO:tasks.workunit.client.1.smithi152.stdout:9/29: dwrite f3 [0,4194304] 0 2024-03-29T04:45:05.692 INFO:tasks.workunit.client.0.smithi142.stdout:7/715: chown d7/d27/d44/db2/db8/de/dc1/de0 1586821 1 2024-03-29T04:45:05.692 INFO:tasks.workunit.client.0.smithi142.stdout:9/469: rmdir d0/d2/d6/d31/d85 39 2024-03-29T04:45:05.694 INFO:tasks.workunit.client.1.smithi152.stdout:7/26: unlink f4 0 2024-03-29T04:45:05.694 INFO:tasks.workunit.client.1.smithi152.stdout:6/52: dwrite f0 [0,4194304] 0 2024-03-29T04:45:05.695 INFO:tasks.workunit.client.0.smithi142.stdout:4/622: rename d0/d9/d11/f2b to d0/d9/d11/d4c/d64/fd5 0 2024-03-29T04:45:05.696 INFO:tasks.workunit.client.0.smithi142.stdout:4/623: stat d0/d9/d11/cc7 0 2024-03-29T04:45:05.697 INFO:tasks.workunit.client.0.smithi142.stdout:0/644: dread d7/f2b [0,4194304] 0 2024-03-29T04:45:05.698 INFO:tasks.workunit.client.0.smithi142.stdout:9/470: creat d0/d2/d6/d27/f92 x:0 0 0 2024-03-29T04:45:05.698 INFO:tasks.workunit.client.1.smithi152.stdout:6/53: creat d1/f16 x:0 0 0 2024-03-29T04:45:05.699 INFO:tasks.workunit.client.1.smithi152.stdout:6/54: chown f0 10960 1 2024-03-29T04:45:05.699 INFO:tasks.workunit.client.1.smithi152.stdout:5/23: fsync f5 0 2024-03-29T04:45:05.699 INFO:tasks.workunit.client.0.smithi142.stdout:4/624: read d0/dc/d15/f72 [243676,16334] 0 2024-03-29T04:45:05.700 INFO:tasks.workunit.client.0.smithi142.stdout:9/471: chown d0/d2/d6/d16 533912335 1 2024-03-29T04:45:05.701 INFO:tasks.workunit.client.1.smithi152.stdout:6/55: creat d1/f17 x:0 0 0 2024-03-29T04:45:05.701 INFO:tasks.workunit.client.0.smithi142.stdout:4/625: symlink d0/d9/d19/d2c/ld6 0 2024-03-29T04:45:05.702 INFO:tasks.workunit.client.1.smithi152.stdout:6/56: chown d1/d3/d9/la 23494 1 2024-03-29T04:45:05.703 INFO:tasks.workunit.client.0.smithi142.stdout:0/645: mknod d7/d9/d55/cd2 0 2024-03-29T04:45:05.703 INFO:tasks.workunit.client.1.smithi152.stdout:5/24: mknod d6/c9 0 2024-03-29T04:45:05.704 INFO:tasks.workunit.client.0.smithi142.stdout:9/472: creat d0/d12/d50/f93 x:0 0 0 2024-03-29T04:45:05.705 INFO:tasks.workunit.client.1.smithi152.stdout:0/88: dwrite d2/f5 [0,4194304] 0 2024-03-29T04:45:05.705 INFO:tasks.workunit.client.1.smithi152.stdout:5/25: stat c2 0 2024-03-29T04:45:05.705 INFO:tasks.workunit.client.1.smithi152.stdout:0/89: chown d2/d1b 251131169 1 2024-03-29T04:45:05.706 INFO:tasks.workunit.client.0.smithi142.stdout:4/626: mknod d0/d9/d11/cd7 0 2024-03-29T04:45:05.707 INFO:tasks.workunit.client.1.smithi152.stdout:5/26: write f5 [686275,30088] 0 2024-03-29T04:45:05.707 INFO:tasks.workunit.client.1.smithi152.stdout:2/35: rmdir d0/d3 39 2024-03-29T04:45:05.707 INFO:tasks.workunit.client.1.smithi152.stdout:8/15: fsync d1/d2/d3/f5 0 2024-03-29T04:45:05.709 INFO:tasks.workunit.client.0.smithi142.stdout:0/646: rmdir d7/d9/d1e 39 2024-03-29T04:45:05.710 INFO:tasks.workunit.client.1.smithi152.stdout:5/27: fdatasync f0 0 2024-03-29T04:45:05.710 INFO:tasks.workunit.client.1.smithi152.stdout:0/90: rename d2 to d2/da/d17/d21 22 2024-03-29T04:45:05.710 INFO:tasks.workunit.client.1.smithi152.stdout:2/36: fdatasync d0/f9 0 2024-03-29T04:45:05.711 INFO:tasks.workunit.client.1.smithi152.stdout:6/57: link d1/f16 d1/d3/f18 0 2024-03-29T04:45:05.712 INFO:tasks.workunit.client.0.smithi142.stdout:9/473: link d0/d12/d50/f90 d0/f94 0 2024-03-29T04:45:05.713 INFO:tasks.workunit.client.0.smithi142.stdout:4/627: mknod d0/cd8 0 2024-03-29T04:45:05.715 INFO:tasks.workunit.client.0.smithi142.stdout:0/647: rename d7/d13/d4a/d70 to d7/d13/dbf/dca/dab/dc1/dd3 0 2024-03-29T04:45:05.717 INFO:tasks.workunit.client.0.smithi142.stdout:9/474: mkdir d0/d2/d6/d31/d85/d95 0 2024-03-29T04:45:05.718 INFO:tasks.workunit.client.0.smithi142.stdout:4/628: mkdir d0/d9/d11/d97/dd9 0 2024-03-29T04:45:05.719 INFO:tasks.workunit.client.0.smithi142.stdout:4/629: chown d0/d9/d11 40208972 1 2024-03-29T04:45:05.720 INFO:tasks.workunit.client.0.smithi142.stdout:9/475: write d0/f94 [112289,99015] 0 2024-03-29T04:45:05.720 INFO:tasks.workunit.client.0.smithi142.stdout:0/648: link d7/d13/dbf/dca/dab/dc1/fce d7/d13/d4a/d9a/da7/fd4 0 2024-03-29T04:45:05.721 INFO:tasks.workunit.client.0.smithi142.stdout:4/630: dread d0/d45/f61 [0,4194304] 0 2024-03-29T04:45:05.724 INFO:tasks.workunit.client.0.smithi142.stdout:4/631: write d0/d12/f1e [2214735,95784] 0 2024-03-29T04:45:05.735 INFO:tasks.workunit.client.0.smithi142.stdout:0/649: dread d7/d9/d54/f65 [0,4194304] 0 2024-03-29T04:45:05.737 INFO:tasks.workunit.client.0.smithi142.stdout:0/650: unlink d7/d13/dbf/d72/f3e 0 2024-03-29T04:45:05.738 INFO:tasks.workunit.client.0.smithi142.stdout:9/476: dread d0/f1d [4194304,4194304] 0 2024-03-29T04:45:05.739 INFO:tasks.workunit.client.0.smithi142.stdout:9/477: readlink d0/d12/l89 0 2024-03-29T04:45:05.742 INFO:tasks.workunit.client.0.smithi142.stdout:9/478: dread d0/d2/d6/d27/f48 [0,4194304] 0 2024-03-29T04:45:05.742 INFO:tasks.workunit.client.0.smithi142.stdout:9/479: write d0/d12/d50/d52/f78 [548861,68294] 0 2024-03-29T04:45:05.745 INFO:tasks.workunit.client.0.smithi142.stdout:9/480: chown d0/d2/d6/d16/f3b 7367 1 2024-03-29T04:45:05.745 INFO:tasks.workunit.client.0.smithi142.stdout:9/481: unlink d0/l57 0 2024-03-29T04:45:05.747 INFO:tasks.workunit.client.0.smithi142.stdout:9/482: link d0/d2/d61/c74 d0/d2/d6/d31/d85/d95/c96 0 2024-03-29T04:45:05.748 INFO:tasks.workunit.client.0.smithi142.stdout:9/483: dread - d0/d12/d50/f93 zero size 2024-03-29T04:45:05.748 INFO:tasks.workunit.client.0.smithi142.stdout:9/484: dread - d0/d12/d50/f93 zero size 2024-03-29T04:45:05.754 INFO:tasks.workunit.client.0.smithi142.stdout:9/485: dwrite d0/f94 [0,4194304] 0 2024-03-29T04:45:05.754 INFO:tasks.workunit.client.0.smithi142.stdout:5/772: sync 2024-03-29T04:45:05.754 INFO:tasks.workunit.client.0.smithi142.stdout:5/773: chown f7 11210216 1 2024-03-29T04:45:05.767 INFO:tasks.workunit.client.1.smithi152.stdout:8/16: unlink d1/d2/d3/f5 0 2024-03-29T04:45:05.768 INFO:tasks.workunit.client.1.smithi152.stdout:0/91: mknod d2/da/df/c22 0 2024-03-29T04:45:05.770 INFO:tasks.workunit.client.1.smithi152.stdout:1/46: write d5/d7/d9/f10 [50592,32870] 0 2024-03-29T04:45:05.770 INFO:tasks.workunit.client.1.smithi152.stdout:6/58: symlink d1/l19 0 2024-03-29T04:45:05.772 INFO:tasks.workunit.client.1.smithi152.stdout:9/30: rmdir d5/d8 39 2024-03-29T04:45:05.772 INFO:tasks.workunit.client.0.smithi142.stdout:9/486: getdents d0/d12/d50/d8e 0 2024-03-29T04:45:05.773 INFO:tasks.workunit.client.0.smithi142.stdout:9/487: chown d0/l42 436033 1 2024-03-29T04:45:05.773 INFO:tasks.workunit.client.1.smithi152.stdout:8/17: rename d1/f6 to d1/f7 0 2024-03-29T04:45:05.774 INFO:tasks.workunit.client.0.smithi142.stdout:5/774: creat df/d16/d24/d2e/d47/d8c/de0/f100 x:0 0 0 2024-03-29T04:45:05.776 INFO:tasks.workunit.client.1.smithi152.stdout:5/28: dwrite f5 [0,4194304] 0 2024-03-29T04:45:05.777 INFO:tasks.workunit.client.0.smithi142.stdout:3/486: dwrite d3/d29/d2b/d40/d67/d8b/f91 [0,4194304] 0 2024-03-29T04:45:05.779 INFO:tasks.workunit.client.0.smithi142.stdout:0/651: dread d7/d13/dbf/d72/d37/f39 [0,4194304] 0 2024-03-29T04:45:05.780 INFO:tasks.workunit.client.0.smithi142.stdout:9/488: dread d0/d2/d6/d16/f40 [4194304,4194304] 0 2024-03-29T04:45:05.780 INFO:tasks.workunit.client.0.smithi142.stdout:9/489: read d0/f94 [899959,126274] 0 2024-03-29T04:45:05.781 INFO:tasks.workunit.client.0.smithi142.stdout:5/775: unlink df/d1a/f1f 0 2024-03-29T04:45:05.781 INFO:tasks.workunit.client.1.smithi152.stdout:6/59: mkdir d1/d3/d1a 0 2024-03-29T04:45:05.782 INFO:tasks.workunit.client.0.smithi142.stdout:5/776: truncate df/d1a/d21/d27/fe7 1805082 0 2024-03-29T04:45:05.783 INFO:tasks.workunit.client.0.smithi142.stdout:3/487: dread d3/d54/f8d [0,4194304] 0 2024-03-29T04:45:05.785 INFO:tasks.workunit.client.1.smithi152.stdout:2/37: dread d0/d3/f5 [0,4194304] 0 2024-03-29T04:45:05.788 INFO:tasks.workunit.client.0.smithi142.stdout:3/488: write d3/d29/d2b/d40/f4c [3884548,16274] 0 2024-03-29T04:45:05.789 INFO:tasks.workunit.client.0.smithi142.stdout:0/652: unlink d7/d9/d54/f65 0 2024-03-29T04:45:05.792 INFO:tasks.workunit.client.1.smithi152.stdout:1/47: dread d5/d7/fe [0,4194304] 0 2024-03-29T04:45:05.793 INFO:tasks.workunit.client.0.smithi142.stdout:1/612: dwrite d8/db/dc/d77/d31/d4d/f8f [0,4194304] 0 2024-03-29T04:45:05.794 INFO:tasks.workunit.client.0.smithi142.stdout:9/490: creat d0/d2/d6/d27/f97 x:0 0 0 2024-03-29T04:45:05.796 INFO:tasks.workunit.client.1.smithi152.stdout:9/31: dwrite d5/d8/f9 [0,4194304] 0 2024-03-29T04:45:05.796 INFO:tasks.workunit.client.0.smithi142.stdout:1/613: fdatasync d8/f9 0 2024-03-29T04:45:05.796 INFO:tasks.workunit.client.0.smithi142.stdout:9/491: chown d0/f1d 30 1 2024-03-29T04:45:05.796 INFO:tasks.workunit.client.0.smithi142.stdout:8/728: sync 2024-03-29T04:45:05.796 INFO:tasks.workunit.client.0.smithi142.stdout:6/604: sync 2024-03-29T04:45:05.797 INFO:tasks.workunit.client.0.smithi142.stdout:2/585: truncate de/d11/d1e/d22/d27/dba/d61/f8f 593889 0 2024-03-29T04:45:05.798 INFO:tasks.workunit.client.0.smithi142.stdout:1/614: readlink d8/db/dc/d77/db9/dbe/d2d/d8b/l9a 0 2024-03-29T04:45:05.799 INFO:tasks.workunit.client.1.smithi152.stdout:6/60: mkdir d1/d3/d1a/d1b 0 2024-03-29T04:45:05.800 INFO:tasks.workunit.client.0.smithi142.stdout:6/605: read d1c/f34 [1223004,58500] 0 2024-03-29T04:45:05.802 INFO:tasks.workunit.client.0.smithi142.stdout:6/606: write d1c/f62 [1335603,31381] 0 2024-03-29T04:45:05.803 INFO:tasks.workunit.client.1.smithi152.stdout:8/18: dwrite d1/f7 [0,4194304] 0 2024-03-29T04:45:05.807 INFO:tasks.workunit.client.1.smithi152.stdout:5/29: truncate f1 1713489 0 2024-03-29T04:45:05.809 INFO:tasks.workunit.client.1.smithi152.stdout:1/48: rename d5/d7/d8/lc to d5/d7/d9/l16 0 2024-03-29T04:45:05.810 INFO:tasks.workunit.client.1.smithi152.stdout:2/38: dwrite d0/f9 [0,4194304] 0 2024-03-29T04:45:05.810 INFO:tasks.workunit.client.0.smithi142.stdout:7/716: dwrite d7/d27/d44/db2/db8/de/d20/f47 [4194304,4194304] 0 2024-03-29T04:45:05.811 INFO:tasks.workunit.client.1.smithi152.stdout:5/30: chown d6/c9 317431 1 2024-03-29T04:45:05.812 INFO:tasks.workunit.client.0.smithi142.stdout:7/717: fdatasync d7/d27/d44/db2/db8/de/d20/f7c 0 2024-03-29T04:45:05.818 INFO:tasks.workunit.client.1.smithi152.stdout:8/19: unlink d1/d2/d3/c4 0 2024-03-29T04:45:05.823 INFO:tasks.workunit.client.0.smithi142.stdout:7/718: creat d7/d27/d44/db2/dc4/ff4 x:0 0 0 2024-03-29T04:45:05.826 INFO:tasks.workunit.client.0.smithi142.stdout:3/489: dread d3/d1f/d24/d47/d87/fa3 [0,4194304] 0 2024-03-29T04:45:05.826 INFO:tasks.workunit.client.1.smithi152.stdout:0/92: dwrite d2/da/fb [0,4194304] 0 2024-03-29T04:45:05.827 INFO:tasks.workunit.client.0.smithi142.stdout:2/586: mkdir de/d11/dc7 0 2024-03-29T04:45:05.827 INFO:tasks.workunit.client.0.smithi142.stdout:9/492: link d0/d2/d6/d16/f3b d0/d2/d6/d27/d3c/d4f/f98 0 2024-03-29T04:45:05.828 INFO:tasks.workunit.client.0.smithi142.stdout:2/587: chown de/d11/d17/d5d/laa 3275 1 2024-03-29T04:45:05.828 INFO:tasks.workunit.client.0.smithi142.stdout:9/493: chown d0/d2/d6/d16/f3b 27615 1 2024-03-29T04:45:05.829 INFO:tasks.workunit.client.1.smithi152.stdout:2/39: mknod d0/ca 0 2024-03-29T04:45:05.829 INFO:tasks.workunit.client.0.smithi142.stdout:2/588: readlink de/d11/d17/d4a/d79/la4 0 2024-03-29T04:45:05.831 INFO:tasks.workunit.client.0.smithi142.stdout:6/607: mkdir d1c/d33/d46/d86/dc0 0 2024-03-29T04:45:05.832 INFO:tasks.workunit.client.1.smithi152.stdout:9/32: dwrite d5/f7 [0,4194304] 0 2024-03-29T04:45:05.835 INFO:tasks.workunit.client.1.smithi152.stdout:9/33: chown f3 242 1 2024-03-29T04:45:05.835 INFO:tasks.workunit.client.1.smithi152.stdout:5/31: getdents d6 0 2024-03-29T04:45:05.836 INFO:tasks.workunit.client.0.smithi142.stdout:3/490: stat d3/d1f/d59/c5c 0 2024-03-29T04:45:05.838 INFO:tasks.workunit.client.0.smithi142.stdout:0/653: getdents d7/d13/dbf/d72/d75 0 2024-03-29T04:45:05.838 INFO:tasks.workunit.client.0.smithi142.stdout:3/491: read d3/d1f/d24/f37 [3204383,111170] 0 2024-03-29T04:45:05.840 INFO:tasks.workunit.client.1.smithi152.stdout:9/34: chown d5/f6 1444720 1 2024-03-29T04:45:05.840 INFO:tasks.workunit.client.0.smithi142.stdout:3/492: rename d3 to d3/d1f/d59/da6 22 2024-03-29T04:45:05.841 INFO:tasks.workunit.client.0.smithi142.stdout:9/494: symlink d0/d2/d6/d31/d85/l99 0 2024-03-29T04:45:05.841 INFO:tasks.workunit.client.1.smithi152.stdout:9/35: chown d5/d8 24466 1 2024-03-29T04:45:05.841 INFO:tasks.workunit.client.1.smithi152.stdout:8/20: dwrite d1/f7 [0,4194304] 0 2024-03-29T04:45:05.842 INFO:tasks.workunit.client.1.smithi152.stdout:1/49: dread d5/d7/fe [0,4194304] 0 2024-03-29T04:45:05.842 INFO:tasks.workunit.client.1.smithi152.stdout:2/40: creat d0/fb x:0 0 0 2024-03-29T04:45:05.844 INFO:tasks.workunit.client.1.smithi152.stdout:0/93: dread d2/f8 [4194304,4194304] 0 2024-03-29T04:45:05.849 INFO:tasks.workunit.client.1.smithi152.stdout:5/32: rename f1 to d6/d7/fa 0 2024-03-29T04:45:05.849 INFO:tasks.workunit.client.1.smithi152.stdout:2/41: rename d0 to d0/d3/dc 22 2024-03-29T04:45:05.852 INFO:tasks.workunit.client.0.smithi142.stdout:2/589: fdatasync de/d11/d17/d4a/f55 0 2024-03-29T04:45:05.853 INFO:tasks.workunit.client.0.smithi142.stdout:4/632: dwrite d0/d45/fad [0,4194304] 0 2024-03-29T04:45:05.854 INFO:tasks.workunit.client.0.smithi142.stdout:6/608: mkdir d1c/d33/d46/d66/dc1 0 2024-03-29T04:45:05.855 INFO:tasks.workunit.client.1.smithi152.stdout:0/94: rmdir d2 39 2024-03-29T04:45:05.855 INFO:tasks.workunit.client.0.smithi142.stdout:1/615: dread d8/f51 [0,4194304] 0 2024-03-29T04:45:05.857 INFO:tasks.workunit.client.0.smithi142.stdout:1/616: chown d8/db/dc/d77/db9/dbe/d2d/d8b 4 1 2024-03-29T04:45:05.859 INFO:tasks.workunit.client.1.smithi152.stdout:1/50: unlink d5/d7/d9/c14 0 2024-03-29T04:45:05.859 INFO:tasks.workunit.client.0.smithi142.stdout:0/654: dread d7/d13/dbf/dca/f97 [4194304,4194304] 0 2024-03-29T04:45:05.860 INFO:tasks.workunit.client.1.smithi152.stdout:5/33: symlink d6/lb 0 2024-03-29T04:45:05.861 INFO:tasks.workunit.client.0.smithi142.stdout:0/655: stat d7/d9/f94 0 2024-03-29T04:45:05.861 INFO:tasks.workunit.client.1.smithi152.stdout:2/42: symlink d0/d3/ld 0 2024-03-29T04:45:05.862 INFO:tasks.workunit.client.0.smithi142.stdout:5/777: truncate df/d16/d24/d2e/d47/d4d/fcc 1213743 0 2024-03-29T04:45:05.864 INFO:tasks.workunit.client.0.smithi142.stdout:5/778: write df/d1a/d21/da5/fdd [1229105,107509] 0 2024-03-29T04:45:05.864 INFO:tasks.workunit.client.1.smithi152.stdout:6/61: dwrite d1/d3/d9/fb [0,4194304] 0 2024-03-29T04:45:05.865 INFO:tasks.workunit.client.1.smithi152.stdout:6/62: stat d1/d3/d9/la 0 2024-03-29T04:45:05.867 INFO:tasks.workunit.client.0.smithi142.stdout:5/779: chown df/d16/d24/d2e/fab 32172 1 2024-03-29T04:45:05.869 INFO:tasks.workunit.client.1.smithi152.stdout:5/34: symlink d6/d7/lc 0 2024-03-29T04:45:05.870 INFO:tasks.workunit.client.0.smithi142.stdout:0/656: dwrite d7/d13/dbf/dca/dab/dc1/dd3/fd1 [0,4194304] 0 2024-03-29T04:45:05.871 INFO:tasks.workunit.client.0.smithi142.stdout:0/657: dread - d7/d13/d4a/fa6 zero size 2024-03-29T04:45:05.873 INFO:tasks.workunit.client.1.smithi152.stdout:5/35: rmdir d6 39 2024-03-29T04:45:05.876 INFO:tasks.workunit.client.1.smithi152.stdout:2/43: symlink d0/le 0 2024-03-29T04:45:05.876 INFO:tasks.workunit.client.1.smithi152.stdout:6/63: dwrite d1/f17 [0,4194304] 0 2024-03-29T04:45:05.882 INFO:tasks.workunit.client.1.smithi152.stdout:8/21: truncate d1/f7 864424 0 2024-03-29T04:45:05.884 INFO:tasks.workunit.client.1.smithi152.stdout:9/36: truncate d5/d8/f9 849649 0 2024-03-29T04:45:05.885 INFO:tasks.workunit.client.1.smithi152.stdout:2/44: mkdir d0/d3/df 0 2024-03-29T04:45:05.887 INFO:tasks.workunit.client.0.smithi142.stdout:6/609: dread d1c/d45/fb7 [0,4194304] 0 2024-03-29T04:45:05.887 INFO:tasks.workunit.client.1.smithi152.stdout:6/64: rename d1/d3/l6 to d1/l1c 0 2024-03-29T04:45:05.888 INFO:tasks.workunit.client.1.smithi152.stdout:0/95: link d2/cd d2/da/df/c23 0 2024-03-29T04:45:05.889 INFO:tasks.workunit.client.1.smithi152.stdout:1/51: link c4 d5/c17 0 2024-03-29T04:45:05.889 INFO:tasks.workunit.client.1.smithi152.stdout:2/45: dread d0/d3/f5 [0,4194304] 0 2024-03-29T04:45:05.891 INFO:tasks.workunit.client.1.smithi152.stdout:1/52: write d5/d7/d9/f10 [545136,7298] 0 2024-03-29T04:45:05.892 INFO:tasks.workunit.client.1.smithi152.stdout:5/36: rmdir d6/d7 39 2024-03-29T04:45:05.894 INFO:tasks.workunit.client.1.smithi152.stdout:2/46: dread d0/f9 [0,4194304] 0 2024-03-29T04:45:05.897 INFO:tasks.workunit.client.1.smithi152.stdout:9/37: mkdir d5/db 0 2024-03-29T04:45:05.899 INFO:tasks.workunit.client.1.smithi152.stdout:1/53: rename d5/d7/d9/l16 to d5/d7/d8/d13/l18 0 2024-03-29T04:45:05.900 INFO:tasks.workunit.client.1.smithi152.stdout:0/96: dwrite d2/f4 [0,4194304] 0 2024-03-29T04:45:05.902 INFO:tasks.workunit.client.1.smithi152.stdout:8/22: write d1/f7 [220589,23675] 0 2024-03-29T04:45:05.902 INFO:tasks.workunit.client.0.smithi142.stdout:9/495: symlink d0/d2/d61/l9a 0 2024-03-29T04:45:05.904 INFO:tasks.workunit.client.0.smithi142.stdout:3/493: symlink d3/d29/d8f/d5f/d64/d6b/la7 0 2024-03-29T04:45:05.905 INFO:tasks.workunit.client.0.smithi142.stdout:2/590: rmdir de/d11/d2b 39 2024-03-29T04:45:05.908 INFO:tasks.workunit.client.1.smithi152.stdout:9/38: dwrite f1 [0,4194304] 0 2024-03-29T04:45:05.910 INFO:tasks.workunit.client.0.smithi142.stdout:4/633: write d0/dc/d15/d3d/d65/f90 [966553,120112] 0 2024-03-29T04:45:05.910 INFO:tasks.workunit.client.1.smithi152.stdout:0/97: dread d2/f5 [0,4194304] 0 2024-03-29T04:45:05.913 INFO:tasks.workunit.client.1.smithi152.stdout:2/47: rename d0/fb to d0/d3/f10 0 2024-03-29T04:45:05.915 INFO:tasks.workunit.client.1.smithi152.stdout:6/65: getdents d1 0 2024-03-29T04:45:05.915 INFO:tasks.workunit.client.1.smithi152.stdout:0/98: write d2/f20 [548238,58329] 0 2024-03-29T04:45:05.918 INFO:tasks.workunit.client.1.smithi152.stdout:8/23: symlink d1/d2/d3/l8 0 2024-03-29T04:45:05.919 INFO:tasks.workunit.client.1.smithi152.stdout:8/24: write d1/f7 [822384,101860] 0 2024-03-29T04:45:05.920 INFO:tasks.workunit.client.1.smithi152.stdout:9/39: dread f1 [0,4194304] 0 2024-03-29T04:45:05.921 INFO:tasks.workunit.client.1.smithi152.stdout:2/48: symlink d0/d3/l11 0 2024-03-29T04:45:05.921 INFO:tasks.workunit.client.0.smithi142.stdout:7/719: mkdir d7/d27/d44/db2/db8/d4b/d71/d30/d33/df5 0 2024-03-29T04:45:05.922 INFO:tasks.workunit.client.1.smithi152.stdout:6/66: creat d1/d3/d1a/f1d x:0 0 0 2024-03-29T04:45:05.923 INFO:tasks.workunit.client.1.smithi152.stdout:2/49: chown d0/f9 1079543825 1 2024-03-29T04:45:05.924 INFO:tasks.workunit.client.1.smithi152.stdout:1/54: dwrite d5/d7/fe [4194304,4194304] 0 2024-03-29T04:45:05.925 INFO:tasks.workunit.client.1.smithi152.stdout:8/25: creat d1/d2/d3/f9 x:0 0 0 2024-03-29T04:45:05.925 INFO:tasks.workunit.client.0.smithi142.stdout:5/780: creat df/d29/d6e/f101 x:0 0 0 2024-03-29T04:45:05.925 INFO:tasks.workunit.client.1.smithi152.stdout:9/40: creat d5/d8/fc x:0 0 0 2024-03-29T04:45:05.927 INFO:tasks.workunit.client.0.smithi142.stdout:0/658: creat d7/d13/dbf/dca/dab/fd5 x:0 0 0 2024-03-29T04:45:05.927 INFO:tasks.workunit.client.1.smithi152.stdout:0/99: rmdir d2/d1b 0 2024-03-29T04:45:05.929 INFO:tasks.workunit.client.1.smithi152.stdout:5/37: write d6/d7/fa [275272,31275] 0 2024-03-29T04:45:05.931 INFO:tasks.workunit.client.1.smithi152.stdout:5/38: fdatasync d6/d7/f8 0 2024-03-29T04:45:05.931 INFO:tasks.workunit.client.0.smithi142.stdout:6/610: mknod d1c/d90/cc2 0 2024-03-29T04:45:05.933 INFO:tasks.workunit.client.1.smithi152.stdout:6/67: creat d1/d3/d1a/d1b/f1e x:0 0 0 2024-03-29T04:45:05.936 INFO:tasks.workunit.client.1.smithi152.stdout:9/41: unlink d5/d8/ca 0 2024-03-29T04:45:05.937 INFO:tasks.workunit.client.1.smithi152.stdout:0/100: rmdir d2/da/df 39 2024-03-29T04:45:05.938 INFO:tasks.workunit.client.0.smithi142.stdout:5/781: rmdir df/d16/d24/d2e/d47/d8c/de0 39 2024-03-29T04:45:05.939 INFO:tasks.workunit.client.0.smithi142.stdout:8/729: sync 2024-03-29T04:45:05.940 INFO:tasks.workunit.client.1.smithi152.stdout:0/101: fdatasync d2/f20 0 2024-03-29T04:45:05.940 INFO:tasks.workunit.client.1.smithi152.stdout:2/50: creat d0/f12 x:0 0 0 2024-03-29T04:45:05.941 INFO:tasks.workunit.client.0.smithi142.stdout:6/611: truncate d1c/f41 812451 0 2024-03-29T04:45:05.942 INFO:tasks.workunit.client.1.smithi152.stdout:1/55: dwrite d5/d7/d9/f10 [0,4194304] 0 2024-03-29T04:45:05.942 INFO:tasks.workunit.client.1.smithi152.stdout:8/26: dwrite d1/d2/d3/f9 [0,4194304] 0 2024-03-29T04:45:05.942 INFO:tasks.workunit.client.0.smithi142.stdout:9/496: symlink d0/d2/d6/d31/l9b 0 2024-03-29T04:45:05.943 INFO:tasks.workunit.client.1.smithi152.stdout:9/42: symlink d5/d8/ld 0 2024-03-29T04:45:05.944 INFO:tasks.workunit.client.0.smithi142.stdout:2/591: rename de/d11/d17/c2d to de/d11/d23/cc8 0 2024-03-29T04:45:05.946 INFO:tasks.workunit.client.1.smithi152.stdout:6/68: creat d1/d3/d9/d13/f1f x:0 0 0 2024-03-29T04:45:05.946 INFO:tasks.workunit.client.0.smithi142.stdout:9/497: write d0/d12/d50/f76 [1845242,41949] 0 2024-03-29T04:45:05.950 INFO:tasks.workunit.client.0.smithi142.stdout:6/612: creat d1c/d33/d46/d61/d7e/fc3 x:0 0 0 2024-03-29T04:45:05.950 INFO:tasks.workunit.client.0.smithi142.stdout:8/730: dwrite d5/d93/fc9 [0,4194304] 0 2024-03-29T04:45:05.950 INFO:tasks.workunit.client.1.smithi152.stdout:8/27: symlink d1/la 0 2024-03-29T04:45:05.952 INFO:tasks.workunit.client.1.smithi152.stdout:8/28: write d1/d2/d3/f9 [2182158,118096] 0 2024-03-29T04:45:05.953 INFO:tasks.workunit.client.1.smithi152.stdout:1/56: mkdir d5/d7/d8/d13/d19 0 2024-03-29T04:45:05.956 INFO:tasks.workunit.client.0.smithi142.stdout:5/782: rename df/d29/d5d to df/d16/d24/d2e/d47/d4d/d102 0 2024-03-29T04:45:05.957 INFO:tasks.workunit.client.1.smithi152.stdout:6/69: creat d1/d3/d1a/f20 x:0 0 0 2024-03-29T04:45:05.957 INFO:tasks.workunit.client.0.smithi142.stdout:3/494: dread d3/d1f/d24/d47/f57 [0,4194304] 0 2024-03-29T04:45:05.959 INFO:tasks.workunit.client.0.smithi142.stdout:2/592: symlink de/d11/d3d/lc9 0 2024-03-29T04:45:05.961 INFO:tasks.workunit.client.0.smithi142.stdout:3/495: dread d3/d29/d8f/d5f/d64/f76 [0,4194304] 0 2024-03-29T04:45:05.961 INFO:tasks.workunit.client.1.smithi152.stdout:9/43: mknod d5/ce 0 2024-03-29T04:45:05.962 INFO:tasks.workunit.client.0.smithi142.stdout:8/731: symlink d5/d93/lfd 0 2024-03-29T04:45:05.962 INFO:tasks.workunit.client.0.smithi142.stdout:6/613: rmdir d1c/d33 39 2024-03-29T04:45:05.964 INFO:tasks.workunit.client.0.smithi142.stdout:4/634: dread d0/d9/d11/f1b [0,4194304] 0 2024-03-29T04:45:05.965 INFO:tasks.workunit.client.1.smithi152.stdout:8/29: unlink d1/f7 0 2024-03-29T04:45:05.965 INFO:tasks.workunit.client.1.smithi152.stdout:2/51: write d0/f9 [318811,62656] 0 2024-03-29T04:45:05.966 INFO:tasks.workunit.client.0.smithi142.stdout:2/593: mknod de/d11/d17/d4a/d79/cca 0 2024-03-29T04:45:05.967 INFO:tasks.workunit.client.1.smithi152.stdout:8/30: fdatasync d1/d2/d3/f9 0 2024-03-29T04:45:05.968 INFO:tasks.workunit.client.1.smithi152.stdout:0/102: creat d2/da/f24 x:0 0 0 2024-03-29T04:45:05.969 INFO:tasks.workunit.client.0.smithi142.stdout:9/498: creat d0/d2/d6/f9c x:0 0 0 2024-03-29T04:45:05.970 INFO:tasks.workunit.client.0.smithi142.stdout:4/635: dread d0/d45/fad [0,4194304] 0 2024-03-29T04:45:05.970 INFO:tasks.workunit.client.1.smithi152.stdout:2/52: dread d0/f2 [0,4194304] 0 2024-03-29T04:45:05.972 INFO:tasks.workunit.client.1.smithi152.stdout:2/53: write d0/f2 [456340,4874] 0 2024-03-29T04:45:05.972 INFO:tasks.workunit.client.1.smithi152.stdout:1/57: getdents d5/d7/d8/d13/d19 0 2024-03-29T04:45:05.973 INFO:tasks.workunit.client.0.smithi142.stdout:5/783: rename df/d16/d24/d5c/db5/ldb to df/d16/d24/d3d/l103 0 2024-03-29T04:45:05.975 INFO:tasks.workunit.client.1.smithi152.stdout:8/31: symlink d1/d2/lb 0 2024-03-29T04:45:05.976 INFO:tasks.workunit.client.0.smithi142.stdout:3/496: creat d3/d29/d2d/d6c/fa8 x:0 0 0 2024-03-29T04:45:05.977 INFO:tasks.workunit.client.1.smithi152.stdout:9/44: rename d5/d8/ld to d5/d8/lf 0 2024-03-29T04:45:05.978 INFO:tasks.workunit.client.0.smithi142.stdout:3/497: write d3/d1f/fa4 [475273,114799] 0 2024-03-29T04:45:05.979 INFO:tasks.workunit.client.1.smithi152.stdout:9/45: chown d5/d8/f9 1287347 1 2024-03-29T04:45:05.980 INFO:tasks.workunit.client.0.smithi142.stdout:8/732: dread d5/d2e/fcc [0,4194304] 0 2024-03-29T04:45:05.980 INFO:tasks.workunit.client.1.smithi152.stdout:0/103: symlink d2/da/df/l25 0 2024-03-29T04:45:05.981 INFO:tasks.workunit.client.1.smithi152.stdout:2/54: creat d0/d3/df/f13 x:0 0 0 2024-03-29T04:45:05.983 INFO:tasks.workunit.client.1.smithi152.stdout:9/46: getdents d5/db 0 2024-03-29T04:45:05.983 INFO:tasks.workunit.client.0.smithi142.stdout:9/499: rename d0/d2/d61/c46 to d0/d2/d6/d27/c9d 0 2024-03-29T04:45:05.984 INFO:tasks.workunit.client.1.smithi152.stdout:8/32: dwrite d1/d2/d3/f9 [0,4194304] 0 2024-03-29T04:45:05.984 INFO:tasks.workunit.client.1.smithi152.stdout:8/33: chown d1 108 1 2024-03-29T04:45:05.986 INFO:tasks.workunit.client.0.smithi142.stdout:8/733: dwrite d5/f1f [0,4194304] 0 2024-03-29T04:45:05.986 INFO:tasks.workunit.client.1.smithi152.stdout:8/34: write d1/d2/d3/f9 [2056825,26551] 0 2024-03-29T04:45:05.988 INFO:tasks.workunit.client.0.smithi142.stdout:2/594: symlink de/d11/d2b/lcb 0 2024-03-29T04:45:05.988 INFO:tasks.workunit.client.1.smithi152.stdout:6/70: write d1/d3/f18 [525247,44410] 0 2024-03-29T04:45:05.990 INFO:tasks.workunit.client.0.smithi142.stdout:2/595: write de/d11/d1e/d84/f98 [32572,122293] 0 2024-03-29T04:45:05.990 INFO:tasks.workunit.client.1.smithi152.stdout:6/71: write d1/d3/d9/d13/f1f [494654,129903] 0 2024-03-29T04:45:05.992 INFO:tasks.workunit.client.1.smithi152.stdout:2/55: mkdir d0/d14 0 2024-03-29T04:45:05.993 INFO:tasks.workunit.client.1.smithi152.stdout:9/47: symlink d5/l10 0 2024-03-29T04:45:05.995 INFO:tasks.workunit.client.1.smithi152.stdout:8/35: dread d1/d2/d3/f9 [0,4194304] 0 2024-03-29T04:45:05.996 INFO:tasks.workunit.client.1.smithi152.stdout:1/58: link d5/d7/d8/d13/l18 d5/d7/d8/d13/d19/l1a 0 2024-03-29T04:45:05.997 INFO:tasks.workunit.client.1.smithi152.stdout:6/72: creat d1/d3/d9/f21 x:0 0 0 2024-03-29T04:45:05.998 INFO:tasks.workunit.client.0.smithi142.stdout:0/659: dread d7/d13/dbf/d72/fa0 [0,4194304] 0 2024-03-29T04:45:05.998 INFO:tasks.workunit.client.1.smithi152.stdout:0/104: link d2/da/c19 d2/da/df/c26 0 2024-03-29T04:45:05.999 INFO:tasks.workunit.client.1.smithi152.stdout:9/48: truncate d5/f6 1207221 0 2024-03-29T04:45:06.001 INFO:tasks.workunit.client.1.smithi152.stdout:2/56: dread d0/d3/f5 [0,4194304] 0 2024-03-29T04:45:06.002 INFO:tasks.workunit.client.0.smithi142.stdout:5/784: symlink df/d16/d24/l104 0 2024-03-29T04:45:06.003 INFO:tasks.workunit.client.0.smithi142.stdout:9/500: mknod d0/c9e 0 2024-03-29T04:45:06.005 INFO:tasks.workunit.client.0.smithi142.stdout:8/734: creat d5/d2e/dd2/d60/ffe x:0 0 0 2024-03-29T04:45:06.005 INFO:tasks.workunit.client.0.smithi142.stdout:1/617: truncate d8/db/fb3 3974480 0 2024-03-29T04:45:06.007 INFO:tasks.workunit.client.0.smithi142.stdout:7/720: truncate d7/d27/d44/db2/db8/de/d20/f21 4085394 0 2024-03-29T04:45:06.007 INFO:tasks.workunit.client.1.smithi152.stdout:2/57: write d0/f12 [979490,119162] 0 2024-03-29T04:45:06.008 INFO:tasks.workunit.client.0.smithi142.stdout:3/498: creat d3/d1f/d24/d47/d87/fa9 x:0 0 0 2024-03-29T04:45:06.011 INFO:tasks.workunit.client.1.smithi152.stdout:6/73: dwrite d1/d3/d9/f21 [0,4194304] 0 2024-03-29T04:45:06.011 INFO:tasks.workunit.client.1.smithi152.stdout:8/36: dread d1/d2/d3/f9 [0,4194304] 0 2024-03-29T04:45:06.012 INFO:tasks.workunit.client.1.smithi152.stdout:1/59: dread d5/d7/d9/f10 [0,4194304] 0 2024-03-29T04:45:06.016 INFO:tasks.workunit.client.1.smithi152.stdout:0/105: write d2/f5 [5025048,14386] 0 2024-03-29T04:45:06.017 INFO:tasks.workunit.client.1.smithi152.stdout:1/60: rmdir d5/d7/d8/d13/d19 39 2024-03-29T04:45:06.018 INFO:tasks.workunit.client.1.smithi152.stdout:6/74: mknod d1/d3/df/c22 0 2024-03-29T04:45:06.019 INFO:tasks.workunit.client.0.smithi142.stdout:1/618: truncate d8/db/dc/d77/d5a/f63 117553 0 2024-03-29T04:45:06.021 INFO:tasks.workunit.client.1.smithi152.stdout:1/61: write d5/d7/d8/fb [2989322,52286] 0 2024-03-29T04:45:06.023 INFO:tasks.workunit.client.1.smithi152.stdout:9/49: dwrite f1 [0,4194304] 0 2024-03-29T04:45:06.023 INFO:tasks.workunit.client.0.smithi142.stdout:5/785: mkdir df/d29/d105 0 2024-03-29T04:45:06.025 INFO:tasks.workunit.client.1.smithi152.stdout:6/75: creat d1/d3/df/f23 x:0 0 0 2024-03-29T04:45:06.025 INFO:tasks.workunit.client.0.smithi142.stdout:9/501: symlink d0/d2/d39/l9f 0 2024-03-29T04:45:06.025 INFO:tasks.workunit.client.1.smithi152.stdout:9/50: fsync d5/f7 0 2024-03-29T04:45:06.026 INFO:tasks.workunit.client.0.smithi142.stdout:6/614: truncate f1a 3895543 0 2024-03-29T04:45:06.028 INFO:tasks.workunit.client.0.smithi142.stdout:1/619: dwrite d8/db/dc/d77/db9/dbe/d2d/d44/d4b/fd0 [0,4194304] 0 2024-03-29T04:45:06.029 INFO:tasks.workunit.client.1.smithi152.stdout:2/58: dwrite d0/d3/f5 [0,4194304] 0 2024-03-29T04:45:06.030 INFO:tasks.workunit.client.1.smithi152.stdout:6/76: rmdir d1/d3/d9 39 2024-03-29T04:45:06.031 INFO:tasks.workunit.client.0.smithi142.stdout:2/596: creat de/d11/d17/fcc x:0 0 0 2024-03-29T04:45:06.033 INFO:tasks.workunit.client.1.smithi152.stdout:9/51: readlink d5/d8/lf 0 2024-03-29T04:45:06.034 INFO:tasks.workunit.client.0.smithi142.stdout:7/721: unlink d7/d27/d44/db2/db8/de/d20/d3f/lc8 0 2024-03-29T04:45:06.035 INFO:tasks.workunit.client.1.smithi152.stdout:8/37: dwrite d1/d2/d3/f9 [0,4194304] 0 2024-03-29T04:45:06.035 INFO:tasks.workunit.client.0.smithi142.stdout:7/722: stat d7/f31 0 2024-03-29T04:45:06.037 INFO:tasks.workunit.client.1.smithi152.stdout:1/62: dread d5/d7/d8/fb [0,4194304] 0 2024-03-29T04:45:06.037 INFO:tasks.workunit.client.0.smithi142.stdout:4/636: truncate d0/d9/d11/d4c/d64/f77 170033 0 2024-03-29T04:45:06.038 INFO:tasks.workunit.client.0.smithi142.stdout:2/597: truncate de/d11/d1e/d22/d50/fc3 139473 0 2024-03-29T04:45:06.039 INFO:tasks.workunit.client.1.smithi152.stdout:8/38: chown d1/d2/d3/l8 1157 1 2024-03-29T04:45:06.039 INFO:tasks.workunit.client.1.smithi152.stdout:6/77: fdatasync d1/d3/d9/f21 0 2024-03-29T04:45:06.040 INFO:tasks.workunit.client.1.smithi152.stdout:6/78: dread - d1/d3/df/f23 zero size 2024-03-29T04:45:06.040 INFO:tasks.workunit.client.0.smithi142.stdout:1/620: dwrite d8/db/dc/d77/db9/dbe/d15/fa0 [0,4194304] 0 2024-03-29T04:45:06.041 INFO:tasks.workunit.client.1.smithi152.stdout:2/59: getdents d0/d14 0 2024-03-29T04:45:06.043 INFO:tasks.workunit.client.1.smithi152.stdout:9/52: write d5/d8/f9 [1421252,7533] 0 2024-03-29T04:45:06.043 INFO:tasks.workunit.client.0.smithi142.stdout:4/637: chown d0/d9/d11/f38 6 1 2024-03-29T04:45:06.043 INFO:tasks.workunit.client.0.smithi142.stdout:7/723: write d7/d27/d44/db2/db8/d4b/d71/d30/d33/f4d [2011202,23196] 0 2024-03-29T04:45:06.048 INFO:tasks.workunit.client.0.smithi142.stdout:6/615: creat d1c/d45/d51/fc4 x:0 0 0 2024-03-29T04:45:06.048 INFO:tasks.workunit.client.1.smithi152.stdout:8/39: link d1/d2/d3/f9 d1/d2/fc 0 2024-03-29T04:45:06.049 INFO:tasks.workunit.client.0.smithi142.stdout:8/735: creat d5/d1d/d24/fff x:0 0 0 2024-03-29T04:45:06.052 INFO:tasks.workunit.client.1.smithi152.stdout:1/63: rmdir d5/d7/d8/d13/d19 39 2024-03-29T04:45:06.052 INFO:tasks.workunit.client.1.smithi152.stdout:9/53: truncate f3 5144581 0 2024-03-29T04:45:06.054 INFO:tasks.workunit.client.1.smithi152.stdout:2/60: mkdir d0/d14/d15 0 2024-03-29T04:45:06.054 INFO:tasks.workunit.client.0.smithi142.stdout:1/621: creat d8/db/dc/d77/d31/d4d/d8c/fd7 x:0 0 0 2024-03-29T04:45:06.055 INFO:tasks.workunit.client.0.smithi142.stdout:8/736: dread d5/fc5 [0,4194304] 0 2024-03-29T04:45:06.056 INFO:tasks.workunit.client.1.smithi152.stdout:9/54: creat d5/d8/f11 x:0 0 0 2024-03-29T04:45:06.057 INFO:tasks.workunit.client.1.smithi152.stdout:6/79: dwrite d1/f17 [0,4194304] 0 2024-03-29T04:45:06.057 INFO:tasks.workunit.client.1.smithi152.stdout:8/40: link d1/d2/d3/l8 d1/ld 0 2024-03-29T04:45:06.058 INFO:tasks.workunit.client.1.smithi152.stdout:6/80: readlink d1/l1c 0 2024-03-29T04:45:06.059 INFO:tasks.workunit.client.0.smithi142.stdout:3/499: rename d3/d1f/d24/f69 to d3/d29/d2d/faa 0 2024-03-29T04:45:06.060 INFO:tasks.workunit.client.1.smithi152.stdout:9/55: fdatasync d5/d8/f9 0 2024-03-29T04:45:06.060 INFO:tasks.workunit.client.0.smithi142.stdout:7/724: mkdir d7/d27/d44/db2/db8/de/d20/d86/dd4/ddb/df6 0 2024-03-29T04:45:06.061 INFO:tasks.workunit.client.0.smithi142.stdout:4/638: dwrite d0/dc/d15/f53 [4194304,4194304] 0 2024-03-29T04:45:06.062 INFO:tasks.workunit.client.1.smithi152.stdout:6/81: dread - d1/d3/df/f10 zero size 2024-03-29T04:45:06.062 INFO:tasks.workunit.client.1.smithi152.stdout:2/61: creat d0/f16 x:0 0 0 2024-03-29T04:45:06.064 INFO:tasks.workunit.client.1.smithi152.stdout:1/64: dread d5/d7/d8/fb [0,4194304] 0 2024-03-29T04:45:06.068 INFO:tasks.workunit.client.1.smithi152.stdout:6/82: symlink d1/d3/d9/l24 0 2024-03-29T04:45:06.070 INFO:tasks.workunit.client.1.smithi152.stdout:9/56: dread d5/d8/f9 [0,4194304] 0 2024-03-29T04:45:06.071 INFO:tasks.workunit.client.0.smithi142.stdout:7/725: creat d7/d62/ff7 x:0 0 0 2024-03-29T04:45:06.072 INFO:tasks.workunit.client.1.smithi152.stdout:1/65: unlink c3 0 2024-03-29T04:45:06.076 INFO:tasks.workunit.client.0.smithi142.stdout:1/622: creat d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/fd8 x:0 0 0 2024-03-29T04:45:06.076 INFO:tasks.workunit.client.1.smithi152.stdout:6/83: link d1/d3/d1a/d1b/f1e d1/d3/d1a/f25 0 2024-03-29T04:45:06.078 INFO:tasks.workunit.client.1.smithi152.stdout:1/66: mkdir d5/d7/d1b 0 2024-03-29T04:45:06.079 INFO:tasks.workunit.client.1.smithi152.stdout:6/84: write d1/f8 [1848669,66994] 0 2024-03-29T04:45:06.079 INFO:tasks.workunit.client.0.smithi142.stdout:4/639: read d0/dc/d15/f59 [703414,79474] 0 2024-03-29T04:45:06.079 INFO:tasks.workunit.client.0.smithi142.stdout:6/616: dread d1c/d33/d46/f48 [0,4194304] 0 2024-03-29T04:45:06.079 INFO:tasks.workunit.client.1.smithi152.stdout:1/67: fsync d5/d7/d9/f11 0 2024-03-29T04:45:06.084 INFO:tasks.workunit.client.1.smithi152.stdout:9/57: mkdir d5/d12 0 2024-03-29T04:45:06.084 INFO:tasks.workunit.client.1.smithi152.stdout:8/41: getdents d1 0 2024-03-29T04:45:06.086 INFO:tasks.workunit.client.1.smithi152.stdout:6/85: creat d1/d3/d9/d13/f26 x:0 0 0 2024-03-29T04:45:06.086 INFO:tasks.workunit.client.0.smithi142.stdout:0/660: rename d7/d9/fa to d7/d13/dbf/d72/fd6 0 2024-03-29T04:45:06.086 INFO:tasks.workunit.client.0.smithi142.stdout:1/623: chown d8/db/dc/d77/db9/dbe/d2d/d44/d80/c83 122906889 1 2024-03-29T04:45:06.087 INFO:tasks.workunit.client.1.smithi152.stdout:8/42: symlink d1/d2/d3/le 0 2024-03-29T04:45:06.088 INFO:tasks.workunit.client.1.smithi152.stdout:8/43: read d1/d2/d3/f9 [1307285,115774] 0 2024-03-29T04:45:06.089 INFO:tasks.workunit.client.0.smithi142.stdout:9/502: write d0/d2/d6/d16/f3e [5689260,111631] 0 2024-03-29T04:45:06.089 INFO:tasks.workunit.client.1.smithi152.stdout:6/86: creat d1/d3/d9/f27 x:0 0 0 2024-03-29T04:45:06.090 INFO:tasks.workunit.client.1.smithi152.stdout:6/87: write d1/d3/d9/d13/f1f [1360545,42161] 0 2024-03-29T04:45:06.091 INFO:tasks.workunit.client.1.smithi152.stdout:9/58: link f3 d5/db/f13 0 2024-03-29T04:45:06.093 INFO:tasks.workunit.client.1.smithi152.stdout:8/44: write d1/d2/fc [4085058,64641] 0 2024-03-29T04:45:06.094 INFO:tasks.workunit.client.0.smithi142.stdout:7/726: dread d7/d27/d44/db2/db8/de/d20/d3f/f61 [0,4194304] 0 2024-03-29T04:45:06.094 INFO:tasks.workunit.client.0.smithi142.stdout:4/640: dwrite d0/dc/d15/d3d/d65/f85 [0,4194304] 0 2024-03-29T04:45:06.094 INFO:tasks.workunit.client.1.smithi152.stdout:6/88: mkdir d1/d3/d9/d13/d28 0 2024-03-29T04:45:06.095 INFO:tasks.workunit.client.1.smithi152.stdout:9/59: unlink d5/d8/lf 0 2024-03-29T04:45:06.096 INFO:tasks.workunit.client.0.smithi142.stdout:5/786: dwrite df/d1a/d21/d27/d45/dbf/f9f [0,4194304] 0 2024-03-29T04:45:06.097 INFO:tasks.workunit.client.0.smithi142.stdout:2/598: rename de/d11/d96 to de/d11/d1e/d22/d27/dba/d61/dcd 0 2024-03-29T04:45:06.098 INFO:tasks.workunit.client.0.smithi142.stdout:0/661: rmdir d7/d13/dbf/dca/dab/dc1/dd3/d90 39 2024-03-29T04:45:06.098 INFO:tasks.workunit.client.0.smithi142.stdout:1/624: write d8/db/dc/d77/db9/dbe/d2d/d44/d4b/fd0 [4744332,20293] 0 2024-03-29T04:45:06.099 INFO:tasks.workunit.client.1.smithi152.stdout:9/60: symlink d5/d8/l14 0 2024-03-29T04:45:06.103 INFO:tasks.workunit.client.0.smithi142.stdout:2/599: chown de/d11/d23/l32 546357 1 2024-03-29T04:45:06.103 INFO:tasks.workunit.client.1.smithi152.stdout:8/45: dwrite d1/d2/d3/f9 [4194304,4194304] 0 2024-03-29T04:45:06.104 INFO:tasks.workunit.client.0.smithi142.stdout:3/500: dwrite d3/d29/d2d/f3f [0,4194304] 0 2024-03-29T04:45:06.104 INFO:tasks.workunit.client.0.smithi142.stdout:1/625: chown d8/db/dc/d77/db9/dbe/d1f/d97/fd5 48304 1 2024-03-29T04:45:06.104 INFO:tasks.workunit.client.1.smithi152.stdout:8/46: read d1/d2/fc [3187963,61175] 0 2024-03-29T04:45:06.113 INFO:tasks.workunit.client.1.smithi152.stdout:9/61: mknod d5/d12/c15 0 2024-03-29T04:45:06.115 INFO:tasks.workunit.client.0.smithi142.stdout:5/787: dwrite df/d16/d24/d2e/f6c [4194304,4194304] 0 2024-03-29T04:45:06.122 INFO:tasks.workunit.client.1.smithi152.stdout:9/62: symlink d5/l16 0 2024-03-29T04:45:06.123 INFO:tasks.workunit.client.1.smithi152.stdout:9/63: write d5/f6 [1877071,43534] 0 2024-03-29T04:45:06.123 INFO:tasks.workunit.client.0.smithi142.stdout:5/788: dwrite df/d16/d24/d79/f5b [4194304,4194304] 0 2024-03-29T04:45:06.125 INFO:tasks.workunit.client.0.smithi142.stdout:5/789: dread - df/d1a/d21/d27/d45/fac zero size 2024-03-29T04:45:06.157 INFO:tasks.workunit.client.1.smithi152.stdout:4/7: sync 2024-03-29T04:45:06.158 INFO:tasks.workunit.client.1.smithi152.stdout:7/27: sync 2024-03-29T04:45:06.158 INFO:tasks.workunit.client.1.smithi152.stdout:3/30: sync 2024-03-29T04:45:06.158 INFO:tasks.workunit.client.1.smithi152.stdout:4/8: dread - no filename 2024-03-29T04:45:06.159 INFO:tasks.workunit.client.0.smithi142.stdout:6/617: rename d1c/d33/d46/d71/c8e to d1c/d33/d46/d61/d9b/cc5 0 2024-03-29T04:45:06.159 INFO:tasks.workunit.client.1.smithi152.stdout:8/47: getdents d1/d2/d3 0 2024-03-29T04:45:06.160 INFO:tasks.workunit.client.1.smithi152.stdout:6/89: write d1/d3/d9/f11 [632365,119339] 0 2024-03-29T04:45:06.161 INFO:tasks.workunit.client.1.smithi152.stdout:7/28: symlink l6 0 2024-03-29T04:45:06.161 INFO:tasks.workunit.client.1.smithi152.stdout:7/29: dwrite - no filename 2024-03-29T04:45:06.161 INFO:tasks.workunit.client.1.smithi152.stdout:7/30: dwrite - no filename 2024-03-29T04:45:06.162 INFO:tasks.workunit.client.1.smithi152.stdout:7/31: readlink l1 0 2024-03-29T04:45:06.163 INFO:tasks.workunit.client.1.smithi152.stdout:7/32: dwrite - no filename 2024-03-29T04:45:06.163 INFO:tasks.workunit.client.1.smithi152.stdout:7/33: write - no filename 2024-03-29T04:45:06.164 INFO:tasks.workunit.client.1.smithi152.stdout:6/90: mkdir d1/d3/d1a/d1b/d29 0 2024-03-29T04:45:06.165 INFO:tasks.workunit.client.1.smithi152.stdout:8/48: write d1/d2/d3/f9 [6811234,80861] 0 2024-03-29T04:45:06.169 INFO:tasks.workunit.client.1.smithi152.stdout:4/9: creat d0/f1 x:0 0 0 2024-03-29T04:45:06.170 INFO:tasks.workunit.client.1.smithi152.stdout:7/34: mkdir d7 0 2024-03-29T04:45:06.170 INFO:tasks.workunit.client.1.smithi152.stdout:7/35: dread - no filename 2024-03-29T04:45:06.172 INFO:tasks.workunit.client.1.smithi152.stdout:6/91: unlink d1/f16 0 2024-03-29T04:45:06.172 INFO:tasks.workunit.client.1.smithi152.stdout:9/64: dread d5/f6 [0,4194304] 0 2024-03-29T04:45:06.174 INFO:tasks.workunit.client.1.smithi152.stdout:9/65: chown d5/db 1011061 1 2024-03-29T04:45:06.175 INFO:tasks.workunit.client.1.smithi152.stdout:9/66: chown d5/d12 4 1 2024-03-29T04:45:06.176 INFO:tasks.workunit.client.1.smithi152.stdout:6/92: symlink d1/d3/d9/d13/l2a 0 2024-03-29T04:45:06.177 INFO:tasks.workunit.client.1.smithi152.stdout:7/36: mkdir d7/d8 0 2024-03-29T04:45:06.177 INFO:tasks.workunit.client.1.smithi152.stdout:7/37: read - no filename 2024-03-29T04:45:06.179 INFO:tasks.workunit.client.1.smithi152.stdout:8/49: link d1/d2/lb d1/lf 0 2024-03-29T04:45:06.180 INFO:tasks.workunit.client.1.smithi152.stdout:3/31: dwrite f2 [0,4194304] 0 2024-03-29T04:45:06.180 INFO:tasks.workunit.client.1.smithi152.stdout:6/93: dread - d1/d3/d1a/f1d zero size 2024-03-29T04:45:06.181 INFO:tasks.workunit.client.0.smithi142.stdout:8/737: dread d5/d2e/dd2/d60/f8f [4194304,4194304] 0 2024-03-29T04:45:06.182 INFO:tasks.workunit.client.1.smithi152.stdout:7/38: unlink l6 0 2024-03-29T04:45:06.182 INFO:tasks.workunit.client.1.smithi152.stdout:7/39: truncate - no filename 2024-03-29T04:45:06.187 INFO:tasks.workunit.client.1.smithi152.stdout:9/67: dwrite f1 [0,4194304] 0 2024-03-29T04:45:06.187 INFO:tasks.workunit.client.1.smithi152.stdout:6/94: truncate d1/d3/d9/fb 4277651 0 2024-03-29T04:45:06.190 INFO:tasks.workunit.client.1.smithi152.stdout:9/68: write d5/f7 [2090239,60204] 0 2024-03-29T04:45:06.191 INFO:tasks.workunit.client.1.smithi152.stdout:6/95: chown d1/d3/d9/f11 0 1 2024-03-29T04:45:06.192 INFO:tasks.workunit.client.1.smithi152.stdout:6/96: chown d1/d3/d1a/f25 922 1 2024-03-29T04:45:06.192 INFO:tasks.workunit.client.1.smithi152.stdout:7/40: creat d7/f9 x:0 0 0 2024-03-29T04:45:06.193 INFO:tasks.workunit.client.1.smithi152.stdout:7/41: dread - d7/f9 zero size 2024-03-29T04:45:06.194 INFO:tasks.workunit.client.1.smithi152.stdout:9/69: write d5/f7 [4130499,51325] 0 2024-03-29T04:45:06.195 INFO:tasks.workunit.client.1.smithi152.stdout:9/70: chown f3 15823497 1 2024-03-29T04:45:06.198 INFO:tasks.workunit.client.1.smithi152.stdout:8/50: fdatasync d1/d2/fc 0 2024-03-29T04:45:06.199 INFO:tasks.workunit.client.1.smithi152.stdout:7/42: creat d7/d8/fa x:0 0 0 2024-03-29T04:45:06.200 INFO:tasks.workunit.client.1.smithi152.stdout:8/51: read d1/d2/fc [8366562,101694] 0 2024-03-29T04:45:06.201 INFO:tasks.workunit.client.1.smithi152.stdout:7/43: dread - d7/d8/fa zero size 2024-03-29T04:45:06.201 INFO:tasks.workunit.client.1.smithi152.stdout:9/71: rename d5/d8/fc to d5/f17 0 2024-03-29T04:45:06.202 INFO:tasks.workunit.client.1.smithi152.stdout:4/10: fsync d0/f1 0 2024-03-29T04:45:06.203 INFO:tasks.workunit.client.1.smithi152.stdout:6/97: dread d1/d3/d9/f21 [0,4194304] 0 2024-03-29T04:45:06.204 INFO:tasks.workunit.client.1.smithi152.stdout:7/44: mkdir d7/d8/db 0 2024-03-29T04:45:06.205 INFO:tasks.workunit.client.1.smithi152.stdout:6/98: truncate d1/d3/f15 19480 0 2024-03-29T04:45:06.206 INFO:tasks.workunit.client.1.smithi152.stdout:7/45: write d7/d8/fa [925178,11142] 0 2024-03-29T04:45:06.208 INFO:tasks.workunit.client.1.smithi152.stdout:9/72: mknod d5/db/c18 0 2024-03-29T04:45:06.211 INFO:tasks.workunit.client.1.smithi152.stdout:9/73: write d5/f7 [3386898,111207] 0 2024-03-29T04:45:06.215 INFO:tasks.workunit.client.1.smithi152.stdout:7/46: symlink d7/d8/lc 0 2024-03-29T04:45:06.216 INFO:tasks.workunit.client.1.smithi152.stdout:3/32: truncate f2 8359158 0 2024-03-29T04:45:06.221 INFO:tasks.workunit.client.1.smithi152.stdout:4/11: dwrite d0/f1 [0,4194304] 0 2024-03-29T04:45:06.221 INFO:tasks.workunit.client.1.smithi152.stdout:3/33: mknod c4 0 2024-03-29T04:45:06.227 INFO:tasks.workunit.client.1.smithi152.stdout:4/12: chown d0 124 1 2024-03-29T04:45:06.229 INFO:tasks.workunit.client.0.smithi142.stdout:1/626: fsync d8/db/dc/d77/db9/dbe/f11 0 2024-03-29T04:45:06.230 INFO:tasks.workunit.client.1.smithi152.stdout:3/34: rename f2 to f5 0 2024-03-29T04:45:06.230 INFO:tasks.workunit.client.1.smithi152.stdout:6/99: dwrite d1/d3/d9/f11 [0,4194304] 0 2024-03-29T04:45:06.230 INFO:tasks.workunit.client.0.smithi142.stdout:4/641: getdents d0/d9/d11/d97/dd9 0 2024-03-29T04:45:06.231 INFO:tasks.workunit.client.0.smithi142.stdout:1/627: write d8/db/dc/d77/d31/d4d/d56/faf [432388,12447] 0 2024-03-29T04:45:06.232 INFO:tasks.workunit.client.0.smithi142.stdout:4/642: write d0/d9/d11/d84/fca [467873,57336] 0 2024-03-29T04:45:06.236 INFO:tasks.workunit.client.1.smithi152.stdout:4/13: creat d0/f2 x:0 0 0 2024-03-29T04:45:06.238 INFO:tasks.workunit.client.0.smithi142.stdout:3/501: truncate d3/d29/d2b/f7d 75549 0 2024-03-29T04:45:06.239 INFO:tasks.workunit.client.1.smithi152.stdout:9/74: dwrite d5/d8/f11 [0,4194304] 0 2024-03-29T04:45:06.240 INFO:tasks.workunit.client.1.smithi152.stdout:3/35: write f5 [400681,117116] 0 2024-03-29T04:45:06.241 INFO:tasks.workunit.client.1.smithi152.stdout:7/47: dwrite d7/f9 [0,4194304] 0 2024-03-29T04:45:06.242 INFO:tasks.workunit.client.1.smithi152.stdout:4/14: symlink d0/l3 0 2024-03-29T04:45:06.244 INFO:tasks.workunit.client.1.smithi152.stdout:9/75: write f1 [1725592,26455] 0 2024-03-29T04:45:06.247 INFO:tasks.workunit.client.1.smithi152.stdout:6/100: write d1/d3/d1a/f20 [868061,44333] 0 2024-03-29T04:45:06.248 INFO:tasks.workunit.client.0.smithi142.stdout:5/790: unlink df/d1a/f74 0 2024-03-29T04:45:06.250 INFO:tasks.workunit.client.0.smithi142.stdout:5/791: stat df/d1a/d21/fa4 0 2024-03-29T04:45:06.251 INFO:tasks.workunit.client.0.smithi142.stdout:5/792: dread - df/d16/d24/d5c/db1/ff2 zero size 2024-03-29T04:45:06.251 INFO:tasks.workunit.client.1.smithi152.stdout:9/76: creat d5/d12/f19 x:0 0 0 2024-03-29T04:45:06.251 INFO:tasks.workunit.client.0.smithi142.stdout:5/793: readlink df/d29/d6e/l7b 0 2024-03-29T04:45:06.252 INFO:tasks.workunit.client.1.smithi152.stdout:7/48: creat d7/d8/db/fd x:0 0 0 2024-03-29T04:45:06.252 INFO:tasks.workunit.client.0.smithi142.stdout:5/794: fdatasync df/d16/d24/d5c/fff 0 2024-03-29T04:45:06.253 INFO:tasks.workunit.client.1.smithi152.stdout:6/101: dread - d1/d3/d1a/f1d zero size 2024-03-29T04:45:06.253 INFO:tasks.workunit.client.1.smithi152.stdout:7/49: readlink l1 0 2024-03-29T04:45:06.254 INFO:tasks.workunit.client.0.smithi142.stdout:5/795: truncate df/d1a/d21/d55/d66/f92 1767013 0 2024-03-29T04:45:06.255 INFO:tasks.workunit.client.1.smithi152.stdout:3/36: getdents . 0 2024-03-29T04:45:06.255 INFO:tasks.workunit.client.1.smithi152.stdout:9/77: creat d5/d8/f1a x:0 0 0 2024-03-29T04:45:06.257 INFO:tasks.workunit.client.0.smithi142.stdout:8/738: symlink d5/d2e/l100 0 2024-03-29T04:45:06.258 INFO:tasks.workunit.client.1.smithi152.stdout:3/37: chown f5 13765 1 2024-03-29T04:45:06.259 INFO:tasks.workunit.client.1.smithi152.stdout:6/102: dread - d1/d3/df/f23 zero size 2024-03-29T04:45:06.260 INFO:tasks.workunit.client.1.smithi152.stdout:3/38: fdatasync f5 0 2024-03-29T04:45:06.260 INFO:tasks.workunit.client.0.smithi142.stdout:0/662: creat d7/d13/dbf/d72/d75/fd7 x:0 0 0 2024-03-29T04:45:06.261 INFO:tasks.workunit.client.1.smithi152.stdout:9/78: write d5/f6 [469220,63779] 0 2024-03-29T04:45:06.262 INFO:tasks.workunit.client.1.smithi152.stdout:3/39: chown c4 1144 1 2024-03-29T04:45:06.264 INFO:tasks.workunit.client.0.smithi142.stdout:4/643: write d0/dc/f33 [4469241,96186] 0 2024-03-29T04:45:06.266 INFO:tasks.workunit.client.1.smithi152.stdout:4/15: dwrite d0/f1 [0,4194304] 0 2024-03-29T04:45:06.267 INFO:tasks.workunit.client.1.smithi152.stdout:9/79: chown d5/l16 1956621024 1 2024-03-29T04:45:06.268 INFO:tasks.workunit.client.0.smithi142.stdout:3/502: fdatasync d3/d29/d2d/f48 0 2024-03-29T04:45:06.271 INFO:tasks.workunit.client.1.smithi152.stdout:6/103: mknod d1/d3/d1a/d1b/d29/c2b 0 2024-03-29T04:45:06.272 INFO:tasks.workunit.client.0.smithi142.stdout:2/600: rename de/c14 to de/d11/dc7/cce 0 2024-03-29T04:45:06.272 INFO:tasks.workunit.client.1.smithi152.stdout:9/80: mkdir d5/db/d1b 0 2024-03-29T04:45:06.274 INFO:tasks.workunit.client.1.smithi152.stdout:7/50: dwrite d7/d8/fa [0,4194304] 0 2024-03-29T04:45:06.274 INFO:tasks.workunit.client.1.smithi152.stdout:4/16: creat d0/f4 x:0 0 0 2024-03-29T04:45:06.277 INFO:tasks.workunit.client.1.smithi152.stdout:6/104: creat d1/d3/d1a/f2c x:0 0 0 2024-03-29T04:45:06.278 INFO:tasks.workunit.client.0.smithi142.stdout:6/618: unlink l12 0 2024-03-29T04:45:06.280 INFO:tasks.workunit.client.0.smithi142.stdout:6/619: write d1c/d33/d46/f69 [447320,16306] 0 2024-03-29T04:45:06.283 INFO:tasks.workunit.client.1.smithi152.stdout:9/81: dread d5/db/f13 [0,4194304] 0 2024-03-29T04:45:06.285 INFO:tasks.workunit.client.1.smithi152.stdout:3/40: truncate f5 7965806 0 2024-03-29T04:45:06.286 INFO:tasks.workunit.client.0.smithi142.stdout:4/644: creat d0/d9/d11/d4c/d5f/d81/dc9/d47/fda x:0 0 0 2024-03-29T04:45:06.286 INFO:tasks.workunit.client.1.smithi152.stdout:7/51: rename d7/d8/db/fd to d7/d8/db/fe 0 2024-03-29T04:45:06.289 INFO:tasks.workunit.client.1.smithi152.stdout:6/105: truncate d1/d3/d9/f27 748234 0 2024-03-29T04:45:06.289 INFO:tasks.workunit.client.0.smithi142.stdout:3/503: dread d3/d29/d2b/d40/d67/d8b/f4e [0,4194304] 0 2024-03-29T04:45:06.289 INFO:tasks.workunit.client.0.smithi142.stdout:8/739: dwrite d5/d1d/d2f/d76/d9c/f7e [0,4194304] 0 2024-03-29T04:45:06.290 INFO:tasks.workunit.client.0.smithi142.stdout:8/740: chown d5/d2e/dd2/d60/f8f 56400 1 2024-03-29T04:45:06.291 INFO:tasks.workunit.client.1.smithi152.stdout:9/82: dread f1 [0,4194304] 0 2024-03-29T04:45:06.291 INFO:tasks.workunit.client.1.smithi152.stdout:7/52: mkdir d7/d8/db/df 0 2024-03-29T04:45:06.291 INFO:tasks.workunit.client.0.smithi142.stdout:7/727: rename d7/d27/d44/db2/db8/de/d8d/c8e to d7/d27/d44/db2/db8/d4b/d71/dec/cf8 0 2024-03-29T04:45:06.292 INFO:tasks.workunit.client.0.smithi142.stdout:2/601: creat de/d11/d2b/d3e/fcf x:0 0 0 2024-03-29T04:45:06.293 INFO:tasks.workunit.client.0.smithi142.stdout:2/602: dread - de/d11/d1e/d22/d27/dba/f6b zero size 2024-03-29T04:45:06.299 INFO:tasks.workunit.client.1.smithi152.stdout:9/83: symlink d5/db/d1b/l1c 0 2024-03-29T04:45:06.302 INFO:tasks.workunit.client.1.smithi152.stdout:7/53: getdents d7/d8/db 0 2024-03-29T04:45:06.303 INFO:tasks.workunit.client.1.smithi152.stdout:6/106: dwrite d1/fe [0,4194304] 0 2024-03-29T04:45:06.303 INFO:tasks.workunit.client.1.smithi152.stdout:9/84: readlink d5/db/d1b/l1c 0 2024-03-29T04:45:06.306 INFO:tasks.workunit.client.1.smithi152.stdout:9/85: dread - d5/d12/f19 zero size 2024-03-29T04:45:06.307 INFO:tasks.workunit.client.0.smithi142.stdout:4/645: dread - d0/d9/f49 zero size 2024-03-29T04:45:06.307 INFO:tasks.workunit.client.1.smithi152.stdout:6/107: unlink d1/d3/df/f10 0 2024-03-29T04:45:06.308 INFO:tasks.workunit.client.0.smithi142.stdout:4/646: readlink d0/d9/d11/d84/lc3 0 2024-03-29T04:45:06.310 INFO:tasks.workunit.client.0.smithi142.stdout:4/647: truncate d0/d9/d11/d4c/d64/fa6 833929 0 2024-03-29T04:45:06.310 INFO:tasks.workunit.client.1.smithi152.stdout:6/108: readlink d1/lc 0 2024-03-29T04:45:06.311 INFO:tasks.workunit.client.1.smithi152.stdout:3/41: dwrite f5 [4194304,4194304] 0 2024-03-29T04:45:06.311 INFO:tasks.workunit.client.0.smithi142.stdout:8/741: write d5/d93/fb0 [305027,21807] 0 2024-03-29T04:45:06.311 INFO:tasks.workunit.client.1.smithi152.stdout:7/54: write d7/d8/db/fe [579719,36889] 0 2024-03-29T04:45:06.313 INFO:tasks.workunit.client.0.smithi142.stdout:3/504: dread d3/d1f/f7e [0,4194304] 0 2024-03-29T04:45:06.314 INFO:tasks.workunit.client.1.smithi152.stdout:9/86: truncate d5/d12/f19 217389 0 2024-03-29T04:45:06.314 INFO:tasks.workunit.client.1.smithi152.stdout:9/87: chown d5/db/d1b/l1c 61 1 2024-03-29T04:45:06.318 INFO:tasks.workunit.client.1.smithi152.stdout:7/55: creat d7/f10 x:0 0 0 2024-03-29T04:45:06.319 INFO:tasks.workunit.client.0.smithi142.stdout:2/603: symlink de/d11/d17/d60/dc4/ld0 0 2024-03-29T04:45:06.320 INFO:tasks.workunit.client.1.smithi152.stdout:3/42: rename c3 to c6 0 2024-03-29T04:45:06.320 INFO:tasks.workunit.client.1.smithi152.stdout:9/88: symlink d5/d12/l1d 0 2024-03-29T04:45:06.320 INFO:tasks.workunit.client.0.smithi142.stdout:3/505: symlink d3/d29/d8f/d5f/d64/d6b/lab 0 2024-03-29T04:45:06.320 INFO:tasks.workunit.client.0.smithi142.stdout:8/742: dwrite d5/d2e/dd2/d60/ffe [0,4194304] 0 2024-03-29T04:45:06.321 INFO:tasks.workunit.client.1.smithi152.stdout:7/56: truncate d7/d8/db/fe 1323689 0 2024-03-29T04:45:06.321 INFO:tasks.workunit.client.1.smithi152.stdout:6/109: creat d1/d3/d9/d13/d28/f2d x:0 0 0 2024-03-29T04:45:06.322 INFO:tasks.workunit.client.0.smithi142.stdout:8/743: write d5/d93/f9e [1840083,33686] 0 2024-03-29T04:45:06.325 INFO:tasks.workunit.client.0.smithi142.stdout:6/620: dread f1a [0,4194304] 0 2024-03-29T04:45:06.326 INFO:tasks.workunit.client.0.smithi142.stdout:2/604: mkdir de/d11/d23/dd1 0 2024-03-29T04:45:06.326 INFO:tasks.workunit.client.1.smithi152.stdout:7/57: unlink c2 0 2024-03-29T04:45:06.329 INFO:tasks.workunit.client.1.smithi152.stdout:9/89: dread d5/f7 [0,4194304] 0 2024-03-29T04:45:06.330 INFO:tasks.workunit.client.0.smithi142.stdout:4/648: symlink d0/d9/d11/d4c/d64/ldb 0 2024-03-29T04:45:06.331 INFO:tasks.workunit.client.1.smithi152.stdout:3/43: dwrite f5 [4194304,4194304] 0 2024-03-29T04:45:06.339 INFO:tasks.workunit.client.1.smithi152.stdout:6/110: getdents d1/d3/d1a 0 2024-03-29T04:45:06.341 INFO:tasks.workunit.client.1.smithi152.stdout:9/90: mknod d5/c1e 0 2024-03-29T04:45:06.341 INFO:tasks.workunit.client.1.smithi152.stdout:7/58: dread d7/d8/db/fe [0,4194304] 0 2024-03-29T04:45:06.342 INFO:tasks.workunit.client.0.smithi142.stdout:8/744: getdents d5/d2e/dd2/d60 0 2024-03-29T04:45:06.345 INFO:tasks.workunit.client.1.smithi152.stdout:7/59: fdatasync d7/d8/fa 0 2024-03-29T04:45:06.345 INFO:tasks.workunit.client.1.smithi152.stdout:3/44: dwrite f5 [4194304,4194304] 0 2024-03-29T04:45:06.346 INFO:tasks.workunit.client.0.smithi142.stdout:4/649: unlink d0/d9/d11/d4c/d5f/d81/dc9/f54 0 2024-03-29T04:45:06.347 INFO:tasks.workunit.client.0.smithi142.stdout:8/745: fsync d5/d1d/d2f/d76/d2d/f56 0 2024-03-29T04:45:06.347 INFO:tasks.workunit.client.0.smithi142.stdout:4/650: chown d0/dc/d15/d3d/d65/f95 3 1 2024-03-29T04:45:06.347 INFO:tasks.workunit.client.0.smithi142.stdout:4/651: stat d0/l46 0 2024-03-29T04:45:06.358 INFO:tasks.workunit.client.0.smithi142.stdout:1/628: write d8/db/dc/d77/d31/d4d/d56/f58 [947385,51105] 0 2024-03-29T04:45:06.359 INFO:tasks.workunit.client.0.smithi142.stdout:8/746: mkdir d5/d1d/d2f/d5f/d101 0 2024-03-29T04:45:06.359 INFO:tasks.workunit.client.0.smithi142.stdout:8/747: chown d5/d1d/d81/c9d 4989802 1 2024-03-29T04:45:06.359 INFO:tasks.workunit.client.1.smithi152.stdout:6/111: dwrite d1/d3/d9/d13/f1f [0,4194304] 0 2024-03-29T04:45:06.359 INFO:tasks.workunit.client.1.smithi152.stdout:3/45: dread f5 [0,4194304] 0 2024-03-29T04:45:06.359 INFO:tasks.workunit.client.1.smithi152.stdout:9/91: truncate d5/d8/f11 662827 0 2024-03-29T04:45:06.359 INFO:tasks.workunit.client.1.smithi152.stdout:6/112: write d1/d3/d1a/f20 [478960,91399] 0 2024-03-29T04:45:06.360 INFO:tasks.workunit.client.1.smithi152.stdout:6/113: fsync d1/d3/d1a/f2c 0 2024-03-29T04:45:06.360 INFO:tasks.workunit.client.1.smithi152.stdout:7/60: truncate d7/f9 1880427 0 2024-03-29T04:45:06.361 INFO:tasks.workunit.client.0.smithi142.stdout:1/629: symlink d8/db/dc/ld9 0 2024-03-29T04:45:06.362 INFO:tasks.workunit.client.1.smithi152.stdout:6/114: stat d1/d3/d9/f21 0 2024-03-29T04:45:06.363 INFO:tasks.workunit.client.0.smithi142.stdout:8/748: mknod d5/d1d/d2f/d76/d3c/d70/c102 0 2024-03-29T04:45:06.366 INFO:tasks.workunit.client.0.smithi142.stdout:3/506: mknod d3/d29/d2b/d40/cac 0 2024-03-29T04:45:06.367 INFO:tasks.workunit.client.1.smithi152.stdout:3/46: dwrite f5 [0,4194304] 0 2024-03-29T04:45:06.367 INFO:tasks.workunit.client.1.smithi152.stdout:6/115: unlink d1/d3/df/f23 0 2024-03-29T04:45:06.368 INFO:tasks.workunit.client.0.smithi142.stdout:3/507: fsync d3/d29/d2d/d56/f9a 0 2024-03-29T04:45:06.368 INFO:tasks.workunit.client.0.smithi142.stdout:8/749: symlink d5/d1d/d2f/d5f/d101/l103 0 2024-03-29T04:45:06.370 INFO:tasks.workunit.client.0.smithi142.stdout:8/750: fdatasync d5/d93/fb0 0 2024-03-29T04:45:06.371 INFO:tasks.workunit.client.0.smithi142.stdout:1/630: dwrite d8/db/dc/d77/db9/dbe/f48 [4194304,4194304] 0 2024-03-29T04:45:06.371 INFO:tasks.workunit.client.0.smithi142.stdout:5/796: truncate fb 153806 0 2024-03-29T04:45:06.372 INFO:tasks.workunit.client.0.smithi142.stdout:2/605: fsync de/d11/d2b/d3e/fcf 0 2024-03-29T04:45:06.374 INFO:tasks.workunit.client.1.smithi152.stdout:7/61: dwrite d7/d8/fa [0,4194304] 0 2024-03-29T04:45:06.375 INFO:tasks.workunit.client.0.smithi142.stdout:8/751: truncate d5/d1d/d81/f91 844796 0 2024-03-29T04:45:06.376 INFO:tasks.workunit.client.1.smithi152.stdout:3/47: mkdir d7 0 2024-03-29T04:45:06.378 INFO:tasks.workunit.client.1.smithi152.stdout:9/92: truncate d5/f6 1130088 0 2024-03-29T04:45:06.378 INFO:tasks.workunit.client.0.smithi142.stdout:0/663: truncate d7/f2b 2178946 0 2024-03-29T04:45:06.379 INFO:tasks.workunit.client.1.smithi152.stdout:7/62: rename d7 to d7/d8/db/d11 22 2024-03-29T04:45:06.379 INFO:tasks.workunit.client.0.smithi142.stdout:0/664: write d7/d9/d55/f3d [5240099,14112] 0 2024-03-29T04:45:06.381 INFO:tasks.workunit.client.1.smithi152.stdout:3/48: mknod d7/c8 0 2024-03-29T04:45:06.382 INFO:tasks.workunit.client.1.smithi152.stdout:9/93: unlink d5/f17 0 2024-03-29T04:45:06.385 INFO:tasks.workunit.client.1.smithi152.stdout:6/116: dwrite d1/d3/d9/d13/f26 [0,4194304] 0 2024-03-29T04:45:06.386 INFO:tasks.workunit.client.0.smithi142.stdout:5/797: mknod df/d29/d7a/d97/c106 0 2024-03-29T04:45:06.386 INFO:tasks.workunit.client.0.smithi142.stdout:2/606: symlink de/d11/d17/d4a/d79/da8/ld2 0 2024-03-29T04:45:06.387 INFO:tasks.workunit.client.1.smithi152.stdout:9/94: fsync f1 0 2024-03-29T04:45:06.387 INFO:tasks.workunit.client.1.smithi152.stdout:3/49: chown c6 706530792 1 2024-03-29T04:45:06.389 INFO:tasks.workunit.client.0.smithi142.stdout:5/798: truncate df/d1a/ffe 1010929 0 2024-03-29T04:45:06.390 INFO:tasks.workunit.client.0.smithi142.stdout:8/752: rename d5/d2e/c62 to d5/d1d/d2f/d76/d2d/c104 0 2024-03-29T04:45:06.395 INFO:tasks.workunit.client.0.smithi142.stdout:9/503: creat d0/d12/fa0 x:0 0 0 2024-03-29T04:45:06.395 INFO:tasks.workunit.client.1.smithi152.stdout:3/50: fdatasync f5 0 2024-03-29T04:45:06.396 INFO:tasks.workunit.client.0.smithi142.stdout:5/799: creat df/d16/d24/d2e/d47/d4d/d102/f107 x:0 0 0 2024-03-29T04:45:06.397 INFO:tasks.workunit.client.0.smithi142.stdout:8/753: creat d5/d1d/d24/d66/dce/f105 x:0 0 0 2024-03-29T04:45:06.398 INFO:tasks.workunit.client.1.smithi152.stdout:3/51: rename d7/c8 to d7/c9 0 2024-03-29T04:45:06.398 INFO:tasks.workunit.client.0.smithi142.stdout:7/728: symlink d7/d27/d3a/lf9 0 2024-03-29T04:45:06.399 INFO:tasks.workunit.client.1.smithi152.stdout:3/52: write f5 [1627420,77557] 0 2024-03-29T04:45:06.399 INFO:tasks.workunit.client.1.smithi152.stdout:3/53: readlink - no filename 2024-03-29T04:45:06.401 INFO:tasks.workunit.client.1.smithi152.stdout:3/54: mknod d7/ca 0 2024-03-29T04:45:06.405 INFO:tasks.workunit.client.0.smithi142.stdout:5/800: creat df/d1a/d21/d55/d66/f108 x:0 0 0 2024-03-29T04:45:06.406 INFO:tasks.workunit.client.0.smithi142.stdout:7/729: readlink d7/d27/d44/db2/dd2/le2 0 2024-03-29T04:45:06.408 INFO:tasks.workunit.client.0.smithi142.stdout:2/607: link de/d11/d1e/d22/d27/dba/d61/fa0 de/fd3 0 2024-03-29T04:45:06.409 INFO:tasks.workunit.client.0.smithi142.stdout:5/801: read df/d16/d24/d2e/d47/d4d/d102/d75/d90/fc2 [2590741,40751] 0 2024-03-29T04:45:06.410 INFO:tasks.workunit.client.0.smithi142.stdout:5/802: chown df/d1a/d21/d27/l86 115399 1 2024-03-29T04:45:06.411 INFO:tasks.workunit.client.0.smithi142.stdout:5/803: chown df/d16/d24 0 1 2024-03-29T04:45:06.411 INFO:tasks.workunit.client.0.smithi142.stdout:7/730: mkdir d7/d27/d44/db2/db8/d89/dfa 0 2024-03-29T04:45:06.413 INFO:tasks.workunit.client.0.smithi142.stdout:8/754: creat d5/d1d/d2f/d76/f106 x:0 0 0 2024-03-29T04:45:06.415 INFO:tasks.workunit.client.1.smithi152.stdout:5/39: sync 2024-03-29T04:45:06.416 INFO:tasks.workunit.client.0.smithi142.stdout:6/621: dwrite fb [0,4194304] 0 2024-03-29T04:45:06.416 INFO:tasks.workunit.client.0.smithi142.stdout:0/665: dread d7/d13/dbf/d72/d37/f83 [0,4194304] 0 2024-03-29T04:45:06.417 INFO:tasks.workunit.client.0.smithi142.stdout:2/608: chown de/d11/d3d/f82 2015505 1 2024-03-29T04:45:06.417 INFO:tasks.workunit.client.1.smithi152.stdout:7/63: dwrite d7/d8/db/fe [0,4194304] 0 2024-03-29T04:45:06.419 INFO:tasks.workunit.client.1.smithi152.stdout:5/40: write d6/d7/fa [2048535,54518] 0 2024-03-29T04:45:06.419 INFO:tasks.workunit.client.0.smithi142.stdout:4/652: write d0/d9/d11/f1b [651090,92419] 0 2024-03-29T04:45:06.419 INFO:tasks.workunit.client.0.smithi142.stdout:1/631: dread d8/db/dc/d77/d5a/fc6 [0,4194304] 0 2024-03-29T04:45:06.420 INFO:tasks.workunit.client.1.smithi152.stdout:6/117: dwrite d1/d3/d1a/d1b/f1e [0,4194304] 0 2024-03-29T04:45:06.421 INFO:tasks.workunit.client.1.smithi152.stdout:5/41: fsync f5 0 2024-03-29T04:45:06.422 INFO:tasks.workunit.client.1.smithi152.stdout:7/64: write d7/d8/fa [2178262,95448] 0 2024-03-29T04:45:06.423 INFO:tasks.workunit.client.0.smithi142.stdout:2/609: write de/d11/d17/d4a/d79/da8/f9a [661212,26129] 0 2024-03-29T04:45:06.424 INFO:tasks.workunit.client.1.smithi152.stdout:7/65: read d7/d8/db/fe [3213925,65104] 0 2024-03-29T04:45:06.425 INFO:tasks.workunit.client.1.smithi152.stdout:3/55: truncate f5 4229233 0 2024-03-29T04:45:06.425 INFO:tasks.workunit.client.0.smithi142.stdout:2/610: dread - de/d11/d3d/fbc zero size 2024-03-29T04:45:06.426 INFO:tasks.workunit.client.0.smithi142.stdout:6/622: rename d1c/l38 to d1c/d33/d46/d66/lc6 0 2024-03-29T04:45:06.427 INFO:tasks.workunit.client.1.smithi152.stdout:9/95: write d5/f6 [1105573,13717] 0 2024-03-29T04:45:06.427 INFO:tasks.workunit.client.0.smithi142.stdout:5/804: link df/d29/l4c df/d16/d24/d5c/l109 0 2024-03-29T04:45:06.428 INFO:tasks.workunit.client.1.smithi152.stdout:5/42: rmdir d6 39 2024-03-29T04:45:06.429 INFO:tasks.workunit.client.0.smithi142.stdout:2/611: write de/d11/d1e/d22/d27/dba/d61/d7d/f7e [745162,21015] 0 2024-03-29T04:45:06.429 INFO:tasks.workunit.client.1.smithi152.stdout:6/118: read d1/d3/f15 [1683,115252] 0 2024-03-29T04:45:06.433 INFO:tasks.workunit.client.0.smithi142.stdout:8/755: creat d5/d1d/d24/f107 x:0 0 0 2024-03-29T04:45:06.433 INFO:tasks.workunit.client.1.smithi152.stdout:9/96: mknod d5/d8/c1f 0 2024-03-29T04:45:06.437 INFO:tasks.workunit.client.0.smithi142.stdout:0/666: rename d7/d13/dbf/dca/f9f to d7/d13/dbf/dca/dab/fd8 0 2024-03-29T04:45:06.440 INFO:tasks.workunit.client.1.smithi152.stdout:5/43: dwrite d6/d7/fa [0,4194304] 0 2024-03-29T04:45:06.442 INFO:tasks.workunit.client.1.smithi152.stdout:9/97: creat d5/f20 x:0 0 0 2024-03-29T04:45:06.443 INFO:tasks.workunit.client.0.smithi142.stdout:5/805: creat df/d1a/d21/da5/f10a x:0 0 0 2024-03-29T04:45:06.444 INFO:tasks.workunit.client.0.smithi142.stdout:4/653: symlink d0/dc/d15/ldc 0 2024-03-29T04:45:06.445 INFO:tasks.workunit.client.1.smithi152.stdout:5/44: dread f5 [0,4194304] 0 2024-03-29T04:45:06.446 INFO:tasks.workunit.client.0.smithi142.stdout:2/612: mknod de/d11/d17/d60/d6f/cd4 0 2024-03-29T04:45:06.447 INFO:tasks.workunit.client.1.smithi152.stdout:3/56: write f5 [2272246,58044] 0 2024-03-29T04:45:06.448 INFO:tasks.workunit.client.0.smithi142.stdout:8/756: mknod d5/d2e/dd2/df6/c108 0 2024-03-29T04:45:06.449 INFO:tasks.workunit.client.0.smithi142.stdout:2/613: chown de/d11/d23/l32 0 1 2024-03-29T04:45:06.454 INFO:tasks.workunit.client.1.smithi152.stdout:5/45: mkdir d6/dd 0 2024-03-29T04:45:06.454 INFO:tasks.workunit.client.1.smithi152.stdout:3/57: dwrite f5 [4194304,4194304] 0 2024-03-29T04:45:06.455 INFO:tasks.workunit.client.0.smithi142.stdout:3/508: dwrite d3/d29/d2b/d40/d67/d8b/f3e [0,4194304] 0 2024-03-29T04:45:06.455 INFO:tasks.workunit.client.0.smithi142.stdout:8/757: write d5/d1d/d24/d66/dce/f105 [340517,1746] 0 2024-03-29T04:45:06.455 INFO:tasks.workunit.client.0.smithi142.stdout:5/806: mkdir df/d16/d24/d2e/d47/d10b 0 2024-03-29T04:45:06.456 INFO:tasks.workunit.client.1.smithi152.stdout:3/58: write f5 [4023573,27733] 0 2024-03-29T04:45:06.456 INFO:tasks.workunit.client.0.smithi142.stdout:9/504: write d0/d2/f11 [2811532,75423] 0 2024-03-29T04:45:06.457 INFO:tasks.workunit.client.0.smithi142.stdout:7/731: dread d7/d27/d44/db2/db8/de/d20/f2b [0,4194304] 0 2024-03-29T04:45:06.459 INFO:tasks.workunit.client.0.smithi142.stdout:8/758: write d5/d1d/d2f/d5f/fb1 [993722,20570] 0 2024-03-29T04:45:06.459 INFO:tasks.workunit.client.1.smithi152.stdout:5/46: creat d6/dd/fe x:0 0 0 2024-03-29T04:45:06.461 INFO:tasks.workunit.client.0.smithi142.stdout:9/505: write d0/d2/d6/d27/d3c/d4f/f56 [3937697,30012] 0 2024-03-29T04:45:06.464 INFO:tasks.workunit.client.1.smithi152.stdout:5/47: mkdir d6/dd/df 0 2024-03-29T04:45:06.464 INFO:tasks.workunit.client.0.smithi142.stdout:2/614: symlink de/d11/d17/d60/d6f/ld5 0 2024-03-29T04:45:06.465 INFO:tasks.workunit.client.1.smithi152.stdout:5/48: fsync f5 0 2024-03-29T04:45:06.465 INFO:tasks.workunit.client.0.smithi142.stdout:8/759: dread d5/d1d/d2f/f50 [0,4194304] 0 2024-03-29T04:45:06.466 INFO:tasks.workunit.client.0.smithi142.stdout:3/509: symlink d3/d29/d8f/lad 0 2024-03-29T04:45:06.469 INFO:tasks.workunit.client.0.smithi142.stdout:5/807: creat df/d1a/d21/d27/f10c x:0 0 0 2024-03-29T04:45:06.469 INFO:tasks.workunit.client.0.smithi142.stdout:3/510: readlink d3/d29/d8f/d5f/d64/d6b/la7 0 2024-03-29T04:45:06.470 INFO:tasks.workunit.client.1.smithi152.stdout:5/49: rmdir d6/dd/df 0 2024-03-29T04:45:06.470 INFO:tasks.workunit.client.0.smithi142.stdout:3/511: write d3/d29/d2d/d6c/fa8 [629219,59781] 0 2024-03-29T04:45:06.475 INFO:tasks.workunit.client.0.smithi142.stdout:9/506: creat d0/d6c/fa1 x:0 0 0 2024-03-29T04:45:06.476 INFO:tasks.workunit.client.0.smithi142.stdout:1/632: write d8/db/dc/d77/db9/dbe/d1f/f46 [319673,42143] 0 2024-03-29T04:45:06.477 INFO:tasks.workunit.client.1.smithi152.stdout:5/50: dwrite d6/d7/fa [0,4194304] 0 2024-03-29T04:45:06.480 INFO:tasks.workunit.client.0.smithi142.stdout:0/667: symlink d7/d13/d4a/d9a/da7/ld9 0 2024-03-29T04:45:06.481 INFO:tasks.workunit.client.0.smithi142.stdout:7/732: creat d7/d27/d44/db2/db8/de/dc1/de0/ffb x:0 0 0 2024-03-29T04:45:06.482 INFO:tasks.workunit.client.1.smithi152.stdout:5/51: getdents d6/d7 0 2024-03-29T04:45:06.484 INFO:tasks.workunit.client.1.smithi152.stdout:9/98: write d5/d8/f11 [92473,28405] 0 2024-03-29T04:45:06.485 INFO:tasks.workunit.client.0.smithi142.stdout:5/808: creat df/d2c/f10d x:0 0 0 2024-03-29T04:45:06.485 INFO:tasks.workunit.client.0.smithi142.stdout:6/623: write d1c/d33/d46/d71/f75 [177671,75940] 0 2024-03-29T04:45:06.486 INFO:tasks.workunit.client.1.smithi152.stdout:9/99: stat d5 0 2024-03-29T04:45:06.486 INFO:tasks.workunit.client.0.smithi142.stdout:3/512: fdatasync d3/f3c 0 2024-03-29T04:45:06.488 INFO:tasks.workunit.client.1.smithi152.stdout:9/100: creat d5/db/f21 x:0 0 0 2024-03-29T04:45:06.488 INFO:tasks.workunit.client.0.smithi142.stdout:9/507: unlink d0/d2/d6/d16/f37 0 2024-03-29T04:45:06.488 INFO:tasks.workunit.client.1.smithi152.stdout:9/101: stat d5/db/d1b/l1c 0 2024-03-29T04:45:06.489 INFO:tasks.workunit.client.1.smithi152.stdout:5/52: fsync d6/dd/fe 0 2024-03-29T04:45:06.489 INFO:tasks.workunit.client.0.smithi142.stdout:1/633: rmdir d8/db/dc 39 2024-03-29T04:45:06.490 INFO:tasks.workunit.client.0.smithi142.stdout:7/733: truncate d7/d27/d44/db2/db8/de/d20/d3f/fae 598972 0 2024-03-29T04:45:06.491 INFO:tasks.workunit.client.0.smithi142.stdout:9/508: chown d0/c45 0 1 2024-03-29T04:45:06.491 INFO:tasks.workunit.client.1.smithi152.stdout:5/53: truncate d6/dd/fe 34337 0 2024-03-29T04:45:06.491 INFO:tasks.workunit.client.1.smithi152.stdout:5/54: chown c2 178783 1 2024-03-29T04:45:06.492 INFO:tasks.workunit.client.0.smithi142.stdout:7/734: readlink d7/l66 0 2024-03-29T04:45:06.492 INFO:tasks.workunit.client.1.smithi152.stdout:9/102: mknod d5/c22 0 2024-03-29T04:45:06.493 INFO:tasks.workunit.client.0.smithi142.stdout:0/668: creat d7/d9/d54/d58/fda x:0 0 0 2024-03-29T04:45:06.494 INFO:tasks.workunit.client.0.smithi142.stdout:2/615: dread de/f8b [0,4194304] 0 2024-03-29T04:45:06.496 INFO:tasks.workunit.client.0.smithi142.stdout:2/616: write de/d11/d1e/d22/d27/dba/d61/d7d/f7e [336426,33495] 0 2024-03-29T04:45:06.496 INFO:tasks.workunit.client.0.smithi142.stdout:7/735: read d7/f1a [1817106,111099] 0 2024-03-29T04:45:06.497 INFO:tasks.workunit.client.0.smithi142.stdout:4/654: write d0/f48 [653474,124619] 0 2024-03-29T04:45:06.498 INFO:tasks.workunit.client.1.smithi152.stdout:9/103: dread f3 [0,4194304] 0 2024-03-29T04:45:06.498 INFO:tasks.workunit.client.1.smithi152.stdout:5/55: dwrite f0 [0,4194304] 0 2024-03-29T04:45:06.499 INFO:tasks.workunit.client.0.smithi142.stdout:3/513: symlink d3/d29/d2d/d6c/lae 0 2024-03-29T04:45:06.499 INFO:tasks.workunit.client.1.smithi152.stdout:5/56: write d6/d7/f8 [507831,45754] 0 2024-03-29T04:45:06.500 INFO:tasks.workunit.client.0.smithi142.stdout:2/617: truncate de/d11/d23/f90 918606 0 2024-03-29T04:45:06.501 INFO:tasks.workunit.client.0.smithi142.stdout:8/760: dread d5/d2e/f65 [0,4194304] 0 2024-03-29T04:45:06.502 INFO:tasks.workunit.client.0.smithi142.stdout:5/809: truncate df/d1a/f3f 1529874 0 2024-03-29T04:45:06.503 INFO:tasks.workunit.client.1.smithi152.stdout:5/57: mkdir d6/dd/d10 0 2024-03-29T04:45:06.504 INFO:tasks.workunit.client.0.smithi142.stdout:9/509: fdatasync d0/d12/f24 0 2024-03-29T04:45:06.505 INFO:tasks.workunit.client.1.smithi152.stdout:5/58: dread - f3 zero size 2024-03-29T04:45:06.505 INFO:tasks.workunit.client.1.smithi152.stdout:9/104: link d5/db/f13 d5/d8/f23 0 2024-03-29T04:45:06.505 INFO:tasks.workunit.client.0.smithi142.stdout:6/624: mkdir d1c/dc7 0 2024-03-29T04:45:06.507 INFO:tasks.workunit.client.0.smithi142.stdout:6/625: write f1a [2942979,57277] 0 2024-03-29T04:45:06.509 INFO:tasks.workunit.client.0.smithi142.stdout:4/655: mkdir d0/d9/d11/d4c/d64/d94/ddd 0 2024-03-29T04:45:06.509 INFO:tasks.workunit.client.1.smithi152.stdout:9/105: dread d5/f7 [0,4194304] 0 2024-03-29T04:45:06.510 INFO:tasks.workunit.client.0.smithi142.stdout:6/626: dread - d1c/d45/fa5 zero size 2024-03-29T04:45:06.512 INFO:tasks.workunit.client.1.smithi152.stdout:9/106: read d5/d8/f23 [511705,33674] 0 2024-03-29T04:45:06.513 INFO:tasks.workunit.client.0.smithi142.stdout:8/761: dwrite d5/d1d/d2f/d45/fe4 [0,4194304] 0 2024-03-29T04:45:06.513 INFO:tasks.workunit.client.0.smithi142.stdout:5/810: dwrite df/d16/d24/d2e/d47/fce [0,4194304] 0 2024-03-29T04:45:06.515 INFO:tasks.workunit.client.0.smithi142.stdout:4/656: write d0/d9/db2/dba/dbc/d6d/d6e/fc4 [502658,112868] 0 2024-03-29T04:45:06.516 INFO:tasks.workunit.client.1.smithi152.stdout:9/107: mknod d5/db/d1b/c24 0 2024-03-29T04:45:06.517 INFO:tasks.workunit.client.0.smithi142.stdout:1/634: mkdir d8/db/dc/d77/db9/dbe/d2d/d8b/dda 0 2024-03-29T04:45:06.518 INFO:tasks.workunit.client.0.smithi142.stdout:8/762: write d5/d1d/d2f/d76/d2d/de2/fe3 [258809,77919] 0 2024-03-29T04:45:06.519 INFO:tasks.workunit.client.1.smithi152.stdout:9/108: write d5/f6 [494485,69023] 0 2024-03-29T04:45:06.521 INFO:tasks.workunit.client.1.smithi152.stdout:9/109: creat d5/db/f25 x:0 0 0 2024-03-29T04:45:06.522 INFO:tasks.workunit.client.0.smithi142.stdout:5/811: truncate df/d1a/d21/d27/d45/fac 428834 0 2024-03-29T04:45:06.523 INFO:tasks.workunit.client.0.smithi142.stdout:6/627: mknod d1c/d33/d46/d71/cc8 0 2024-03-29T04:45:06.523 INFO:tasks.workunit.client.1.smithi152.stdout:9/110: read d5/d12/f19 [20602,4949] 0 2024-03-29T04:45:06.526 INFO:tasks.workunit.client.1.smithi152.stdout:9/111: chown d5/f6 1023297 1 2024-03-29T04:45:06.526 INFO:tasks.workunit.client.1.smithi152.stdout:9/112: fsync d5/f6 0 2024-03-29T04:45:06.527 INFO:tasks.workunit.client.0.smithi142.stdout:2/618: symlink de/d11/d1e/d22/d27/dba/d61/ld6 0 2024-03-29T04:45:06.530 INFO:tasks.workunit.client.0.smithi142.stdout:1/635: mknod d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/cdb 0 2024-03-29T04:45:06.531 INFO:tasks.workunit.client.0.smithi142.stdout:7/736: dread d7/d27/d44/db2/db8/de/d20/d3f/f75 [0,4194304] 0 2024-03-29T04:45:06.532 INFO:tasks.workunit.client.0.smithi142.stdout:4/657: dwrite d0/d9/d11/d4c/d5f/f6c [0,4194304] 0 2024-03-29T04:45:06.534 INFO:tasks.workunit.client.0.smithi142.stdout:8/763: dwrite d5/d2e/dd2/d60/fb3 [0,4194304] 0 2024-03-29T04:45:06.536 INFO:tasks.workunit.client.0.smithi142.stdout:5/812: mknod df/d16/d24/d5c/db1/c10e 0 2024-03-29T04:45:06.539 INFO:tasks.workunit.client.0.smithi142.stdout:0/669: getdents d7/d13/d4a/d9a 0 2024-03-29T04:45:06.544 INFO:tasks.workunit.client.0.smithi142.stdout:9/510: creat d0/d2/fa2 x:0 0 0 2024-03-29T04:45:06.545 INFO:tasks.workunit.client.0.smithi142.stdout:6/628: mkdir d1c/dc9 0 2024-03-29T04:45:06.546 INFO:tasks.workunit.client.0.smithi142.stdout:3/514: link d3/d29/d2b/f7d d3/d1f/d24/faf 0 2024-03-29T04:45:06.547 INFO:tasks.workunit.client.0.smithi142.stdout:8/764: mknod d5/d1d/d81/c109 0 2024-03-29T04:45:06.547 INFO:tasks.workunit.client.0.smithi142.stdout:1/636: rename d8/db/dc/d77/db9/dbe/d1f/f9b to d8/db/dc/d77/db9/dbe/d2d/d8b/da4/fdc 0 2024-03-29T04:45:06.548 INFO:tasks.workunit.client.0.smithi142.stdout:1/637: fdatasync f3 0 2024-03-29T04:45:06.549 INFO:tasks.workunit.client.1.smithi152.stdout:9/113: truncate d5/f7 1796551 0 2024-03-29T04:45:06.549 INFO:tasks.workunit.client.0.smithi142.stdout:8/765: write d5/d1d/d2f/d76/d2d/f56 [1878910,58155] 0 2024-03-29T04:45:06.550 INFO:tasks.workunit.client.1.smithi152.stdout:9/114: dread d5/f6 [0,4194304] 0 2024-03-29T04:45:06.553 INFO:tasks.workunit.client.1.smithi152.stdout:9/115: symlink d5/db/l26 0 2024-03-29T04:45:06.554 INFO:tasks.workunit.client.0.smithi142.stdout:6/629: dread - d1c/d33/db6/f81 zero size 2024-03-29T04:45:06.556 INFO:tasks.workunit.client.0.smithi142.stdout:6/630: stat d1c/d45/d51/f58 0 2024-03-29T04:45:06.558 INFO:tasks.workunit.client.1.smithi152.stdout:9/116: symlink d5/db/d1b/l27 0 2024-03-29T04:45:06.559 INFO:tasks.workunit.client.0.smithi142.stdout:8/766: read d5/fc5 [149510,33332] 0 2024-03-29T04:45:06.559 INFO:tasks.workunit.client.0.smithi142.stdout:0/670: creat d7/d9/d55/fdb x:0 0 0 2024-03-29T04:45:06.559 INFO:tasks.workunit.client.1.smithi152.stdout:9/117: symlink d5/db/l28 0 2024-03-29T04:45:06.559 INFO:tasks.workunit.client.1.smithi152.stdout:9/118: read d5/f6 [97345,114808] 0 2024-03-29T04:45:06.562 INFO:tasks.workunit.client.0.smithi142.stdout:3/515: dwrite d3/d29/d2b/d40/d67/d8b/d4d/f9d [0,4194304] 0 2024-03-29T04:45:06.562 INFO:tasks.workunit.client.0.smithi142.stdout:6/631: write d1c/d33/d46/d61/fa1 [165720,85223] 0 2024-03-29T04:45:06.562 INFO:tasks.workunit.client.1.smithi152.stdout:9/119: creat d5/f29 x:0 0 0 2024-03-29T04:45:06.563 INFO:tasks.workunit.client.1.smithi152.stdout:9/120: chown d5/d8/l14 28610519 1 2024-03-29T04:45:06.564 INFO:tasks.workunit.client.0.smithi142.stdout:8/767: mknod d5/d1d/d24/c10a 0 2024-03-29T04:45:06.565 INFO:tasks.workunit.client.1.smithi152.stdout:9/121: dread d5/d8/f9 [0,4194304] 0 2024-03-29T04:45:06.567 INFO:tasks.workunit.client.0.smithi142.stdout:2/619: fsync de/f19 0 2024-03-29T04:45:06.568 INFO:tasks.workunit.client.1.smithi152.stdout:9/122: mknod d5/db/c2a 0 2024-03-29T04:45:06.568 INFO:tasks.workunit.client.0.smithi142.stdout:0/671: truncate d7/f2b 1350062 0 2024-03-29T04:45:06.569 INFO:tasks.workunit.client.1.smithi152.stdout:9/123: fsync d5/db/f25 0 2024-03-29T04:45:06.569 INFO:tasks.workunit.client.0.smithi142.stdout:7/737: dread d7/d27/d44/db2/db8/f10 [0,4194304] 0 2024-03-29T04:45:06.570 INFO:tasks.workunit.client.0.smithi142.stdout:6/632: symlink d1c/dc7/lca 0 2024-03-29T04:45:06.571 INFO:tasks.workunit.client.0.smithi142.stdout:9/511: write d0/d2/d6/d16/f3b [2255696,90518] 0 2024-03-29T04:45:06.572 INFO:tasks.workunit.client.1.smithi152.stdout:9/124: unlink f1 0 2024-03-29T04:45:06.572 INFO:tasks.workunit.client.0.smithi142.stdout:4/658: dwrite d0/d9/d19/d2c/f37 [0,4194304] 0 2024-03-29T04:45:06.573 INFO:tasks.workunit.client.1.smithi152.stdout:9/125: fdatasync d5/d8/f11 0 2024-03-29T04:45:06.574 INFO:tasks.workunit.client.1.smithi152.stdout:9/126: dread d5/d8/f11 [0,4194304] 0 2024-03-29T04:45:06.576 INFO:tasks.workunit.client.0.smithi142.stdout:9/512: stat d0/d2/d6/d27/f48 0 2024-03-29T04:45:06.578 INFO:tasks.workunit.client.1.smithi152.stdout:9/127: creat d5/db/d1b/f2b x:0 0 0 2024-03-29T04:45:06.580 INFO:tasks.workunit.client.1.smithi152.stdout:9/128: dread d5/f6 [0,4194304] 0 2024-03-29T04:45:06.580 INFO:tasks.workunit.client.0.smithi142.stdout:2/620: chown de/d11/d2b/d3e/c77 63797 1 2024-03-29T04:45:06.580 INFO:tasks.workunit.client.0.smithi142.stdout:4/659: unlink d0/d45/f61 0 2024-03-29T04:45:06.582 INFO:tasks.workunit.client.0.smithi142.stdout:5/813: dwrite df/d16/d24/d2e/d47/d4d/d102/d75/d90/fc2 [0,4194304] 0 2024-03-29T04:45:06.584 INFO:tasks.workunit.client.1.smithi152.stdout:9/129: dread d5/d8/f23 [4194304,4194304] 0 2024-03-29T04:45:06.585 INFO:tasks.workunit.client.0.smithi142.stdout:9/513: mkdir d0/d12/d50/d52/da3 0 2024-03-29T04:45:06.587 INFO:tasks.workunit.client.1.smithi152.stdout:9/130: mkdir d5/d12/d2c 0 2024-03-29T04:45:06.587 INFO:tasks.workunit.client.0.smithi142.stdout:5/814: read df/d16/f26 [3707459,76130] 0 2024-03-29T04:45:06.591 INFO:tasks.workunit.client.0.smithi142.stdout:1/638: dread d8/db/dc/d77/d31/d4d/d8c/f95 [0,4194304] 0 2024-03-29T04:45:06.592 INFO:tasks.workunit.client.0.smithi142.stdout:9/514: write d0/d2/d6/d16/f58 [3335180,118071] 0 2024-03-29T04:45:06.592 INFO:tasks.workunit.client.0.smithi142.stdout:7/738: rename d7/d27/d44/db2/db8/d4b/d71/f76 to d7/d27/d44/db2/db8/d4b/d71/d30/d33/df5/ffc 0 2024-03-29T04:45:06.595 INFO:tasks.workunit.client.0.smithi142.stdout:9/515: stat d0/d12 0 2024-03-29T04:45:06.596 INFO:tasks.workunit.client.0.smithi142.stdout:3/516: dwrite d3/d54/f8d [0,4194304] 0 2024-03-29T04:45:06.597 INFO:tasks.workunit.client.0.smithi142.stdout:9/516: chown d0/f1d 859103 1 2024-03-29T04:45:06.600 INFO:tasks.workunit.client.0.smithi142.stdout:4/660: dwrite d0/d9/d11/d4c/d5f/d81/dc9/d47/f6b [0,4194304] 0 2024-03-29T04:45:06.602 INFO:tasks.workunit.client.0.smithi142.stdout:9/517: write d0/d2/d6/d31/f3a [6856963,124224] 0 2024-03-29T04:45:06.603 INFO:tasks.workunit.client.0.smithi142.stdout:7/739: mknod d7/d27/d44/cfd 0 2024-03-29T04:45:06.607 INFO:tasks.workunit.client.0.smithi142.stdout:5/815: dwrite df/d1a/d21/d55/f81 [0,4194304] 0 2024-03-29T04:45:06.609 INFO:tasks.workunit.client.0.smithi142.stdout:5/816: chown df/d16/d24/d2e/d47/d4d/d102/d75 1724258 1 2024-03-29T04:45:06.609 INFO:tasks.workunit.client.0.smithi142.stdout:6/633: dread d1c/f5d [0,4194304] 0 2024-03-29T04:45:06.610 INFO:tasks.workunit.client.0.smithi142.stdout:4/661: creat d0/d9/d11/fde x:0 0 0 2024-03-29T04:45:06.610 INFO:tasks.workunit.client.0.smithi142.stdout:1/639: write d8/db/dc/d77/db9/dbe/f11 [636712,45022] 0 2024-03-29T04:45:06.613 INFO:tasks.workunit.client.0.smithi142.stdout:3/517: write d3/d29/d2b/d6a/f7b [964463,74264] 0 2024-03-29T04:45:06.613 INFO:tasks.workunit.client.0.smithi142.stdout:9/518: creat d0/d12/d50/d8e/fa4 x:0 0 0 2024-03-29T04:45:06.614 INFO:tasks.workunit.client.0.smithi142.stdout:2/621: dread f7 [0,4194304] 0 2024-03-29T04:45:06.616 INFO:tasks.workunit.client.0.smithi142.stdout:9/519: write d0/d12/d50/f63 [4564256,58544] 0 2024-03-29T04:45:06.617 INFO:tasks.workunit.client.0.smithi142.stdout:4/662: dread d0/dc/d15/f72 [0,4194304] 0 2024-03-29T04:45:06.618 INFO:tasks.workunit.client.0.smithi142.stdout:6/634: rename d1c/c3d to d1c/d33/d46/d61/d9b/ccb 0 2024-03-29T04:45:06.619 INFO:tasks.workunit.client.0.smithi142.stdout:3/518: mknod d3/d1f/d59/cb0 0 2024-03-29T04:45:06.621 INFO:tasks.workunit.client.0.smithi142.stdout:3/519: chown d3/d29/d2b/d40/d67/d8b/c19 28170 1 2024-03-29T04:45:06.621 INFO:tasks.workunit.client.0.smithi142.stdout:9/520: symlink d0/d2/d6/d16/d6a/la5 0 2024-03-29T04:45:06.622 INFO:tasks.workunit.client.0.smithi142.stdout:3/520: read d3/d29/d8f/d5f/d64/f76 [37641,78282] 0 2024-03-29T04:45:06.622 INFO:tasks.workunit.client.0.smithi142.stdout:2/622: rename de/d11/d17/d4a/d79/da8/ld2 to de/d11/d17/d60/ld7 0 2024-03-29T04:45:06.623 INFO:tasks.workunit.client.0.smithi142.stdout:6/635: mknod d1c/d33/db6/da4/ccc 0 2024-03-29T04:45:06.623 INFO:tasks.workunit.client.0.smithi142.stdout:5/817: link df/d1a/d21/d27/f4b df/d29/d6e/f10f 0 2024-03-29T04:45:06.626 INFO:tasks.workunit.client.0.smithi142.stdout:5/818: write df/d1a/d21/d55/d66/fcd [967342,106440] 0 2024-03-29T04:45:06.626 INFO:tasks.workunit.client.0.smithi142.stdout:2/623: write de/d11/d1e/d22/d27/dba/d61/d7d/db4/fc5 [614542,105755] 0 2024-03-29T04:45:06.626 INFO:tasks.workunit.client.0.smithi142.stdout:4/663: rename d0/d9/d11/d4c/d5f/d81/dc9/l40 to d0/d9/d11/d4c/d5f/d81/ldf 0 2024-03-29T04:45:06.627 INFO:tasks.workunit.client.0.smithi142.stdout:3/521: mkdir d3/d29/d2b/d40/db1 0 2024-03-29T04:45:06.630 INFO:tasks.workunit.client.0.smithi142.stdout:5/819: write df/d1a/d28/fae [1237990,2195] 0 2024-03-29T04:45:06.632 INFO:tasks.workunit.client.0.smithi142.stdout:3/522: creat d3/d1f/d59/fb2 x:0 0 0 2024-03-29T04:45:06.635 INFO:tasks.workunit.client.0.smithi142.stdout:6/636: dwrite d1c/d33/d46/d71/f9f [0,4194304] 0 2024-03-29T04:45:06.636 INFO:tasks.workunit.client.0.smithi142.stdout:3/523: chown d3/d29/d2b/d40/d67/d8b/d6/cf 119836459 1 2024-03-29T04:45:06.638 INFO:tasks.workunit.client.0.smithi142.stdout:2/624: dwrite de/d11/d1e/fb9 [0,4194304] 0 2024-03-29T04:45:06.640 INFO:tasks.workunit.client.0.smithi142.stdout:3/524: read d3/d29/d2b/d40/d67/d8b/f4e [3128421,77700] 0 2024-03-29T04:45:06.641 INFO:tasks.workunit.client.0.smithi142.stdout:7/740: dread d7/d27/d44/db2/db8/de/f5c [0,4194304] 0 2024-03-29T04:45:06.677 INFO:tasks.workunit.client.0.smithi142.stdout:4/664: symlink d0/d9/d19/d2c/le0 0 2024-03-29T04:45:06.678 INFO:tasks.workunit.client.0.smithi142.stdout:4/665: readlink d0/dc/d15/l2a 0 2024-03-29T04:45:06.678 INFO:tasks.workunit.client.0.smithi142.stdout:9/521: write d0/d12/f60 [128211,40481] 0 2024-03-29T04:45:06.680 INFO:tasks.workunit.client.0.smithi142.stdout:6/637: mkdir d1c/d33/d46/d66/dcd 0 2024-03-29T04:45:06.682 INFO:tasks.workunit.client.0.smithi142.stdout:3/525: creat d3/d1f/d59/fb3 x:0 0 0 2024-03-29T04:45:06.684 INFO:tasks.workunit.client.0.smithi142.stdout:5/820: mknod df/d1a/d21/c110 0 2024-03-29T04:45:06.688 INFO:tasks.workunit.client.0.smithi142.stdout:7/741: mkdir d7/d27/d44/d4c/da3/dc2/dfe 0 2024-03-29T04:45:06.690 INFO:tasks.workunit.client.0.smithi142.stdout:3/526: fsync d3/d29/d2b/d40/d67/d8b/d6/f68 0 2024-03-29T04:45:06.691 INFO:tasks.workunit.client.0.smithi142.stdout:3/527: dread d3/d29/d8f/d5f/d64/f76 [0,4194304] 0 2024-03-29T04:45:06.692 INFO:tasks.workunit.client.0.smithi142.stdout:5/821: rename df/d1a/d21/d27/d45/dbf/ff0 to df/d1a/d21/d27/d45/f111 0 2024-03-29T04:45:06.695 INFO:tasks.workunit.client.0.smithi142.stdout:6/638: mkdir d1c/dc9/dce 0 2024-03-29T04:45:06.697 INFO:tasks.workunit.client.0.smithi142.stdout:4/666: dread d0/d9/d11/d4c/d5f/f73 [0,4194304] 0 2024-03-29T04:45:06.699 INFO:tasks.workunit.client.0.smithi142.stdout:5/822: dwrite df/d29/d7a/d97/ff9 [0,4194304] 0 2024-03-29T04:45:06.701 INFO:tasks.workunit.client.0.smithi142.stdout:3/528: rename d3/d54 to d3/d29/d8f/d5f/db4 0 2024-03-29T04:45:06.701 INFO:tasks.workunit.client.0.smithi142.stdout:9/522: link d0/d2/d61/l9a d0/d2/d6/d27/la6 0 2024-03-29T04:45:06.703 INFO:tasks.workunit.client.0.smithi142.stdout:2/625: getdents de/d11/d17/d5d/d71 0 2024-03-29T04:45:06.705 INFO:tasks.workunit.client.0.smithi142.stdout:7/742: dwrite d7/d27/d44/db2/db8/de/d20/f4e [0,4194304] 0 2024-03-29T04:45:06.706 INFO:tasks.workunit.client.0.smithi142.stdout:6/639: creat d1c/d33/d46/d86/fcf x:0 0 0 2024-03-29T04:45:06.706 INFO:tasks.workunit.client.0.smithi142.stdout:3/529: truncate d3/d1f/fa4 1609244 0 2024-03-29T04:45:06.708 INFO:tasks.workunit.client.0.smithi142.stdout:5/823: mkdir df/d2c/d112 0 2024-03-29T04:45:06.709 INFO:tasks.workunit.client.0.smithi142.stdout:5/824: fsync df/f14 0 2024-03-29T04:45:06.710 INFO:tasks.workunit.client.0.smithi142.stdout:2/626: creat de/d11/d3d/fd8 x:0 0 0 2024-03-29T04:45:06.711 INFO:tasks.workunit.client.0.smithi142.stdout:3/530: fdatasync d3/d29/d8f/d5f/d64/f76 0 2024-03-29T04:45:06.714 INFO:tasks.workunit.client.0.smithi142.stdout:6/640: rename d1c/d33/d46/c93 to d1c/d33/d46/d66/cd0 0 2024-03-29T04:45:06.714 INFO:tasks.workunit.client.0.smithi142.stdout:7/743: dwrite d7/d27/d44/db2/db8/de/d20/f7c [0,4194304] 0 2024-03-29T04:45:06.716 INFO:tasks.workunit.client.0.smithi142.stdout:9/523: mknod d0/d2/d6/d27/d3c/d6f/ca7 0 2024-03-29T04:45:06.719 INFO:tasks.workunit.client.0.smithi142.stdout:5/825: fdatasync df/d29/f82 0 2024-03-29T04:45:06.720 INFO:tasks.workunit.client.0.smithi142.stdout:4/667: creat d0/d9/d11/d97/fe1 x:0 0 0 2024-03-29T04:45:06.721 INFO:tasks.workunit.client.0.smithi142.stdout:2/627: creat de/d11/d3d/fd9 x:0 0 0 2024-03-29T04:45:06.723 INFO:tasks.workunit.client.0.smithi142.stdout:6/641: rename d1c/d33/d46/d61/d9e/fbe to d1c/d33/d46/d61/d9e/fd1 0 2024-03-29T04:45:06.724 INFO:tasks.workunit.client.0.smithi142.stdout:6/642: fdatasync d1c/d45/d51/fc4 0 2024-03-29T04:45:06.725 INFO:tasks.workunit.client.0.smithi142.stdout:2/628: rmdir de/d11/d17/d60 39 2024-03-29T04:45:06.727 INFO:tasks.workunit.client.0.smithi142.stdout:9/524: dread d0/d12/d50/f76 [0,4194304] 0 2024-03-29T04:45:06.727 INFO:tasks.workunit.client.0.smithi142.stdout:6/643: creat d1c/d90/d94/fd2 x:0 0 0 2024-03-29T04:45:06.728 INFO:tasks.workunit.client.0.smithi142.stdout:4/668: symlink d0/dc/d15/d91/le2 0 2024-03-29T04:45:06.729 INFO:tasks.workunit.client.0.smithi142.stdout:5/826: truncate df/d16/d24/d2e/d47/d8c/fba 3181534 0 2024-03-29T04:45:06.731 INFO:tasks.workunit.client.0.smithi142.stdout:4/669: read d0/d45/fad [1470875,2214] 0 2024-03-29T04:45:06.734 INFO:tasks.workunit.client.0.smithi142.stdout:9/525: link d0/f62 d0/d2/d6/d27/fa8 0 2024-03-29T04:45:06.736 INFO:tasks.workunit.client.0.smithi142.stdout:4/670: stat d0/d9/d11/d4c/d5f/d81/dc9/d47/l96 0 2024-03-29T04:45:06.736 INFO:tasks.workunit.client.0.smithi142.stdout:6/644: dwrite d1c/d33/d46/d71/f75 [0,4194304] 0 2024-03-29T04:45:06.736 INFO:tasks.workunit.client.0.smithi142.stdout:5/827: dread df/d29/f67 [0,4194304] 0 2024-03-29T04:45:06.741 INFO:tasks.workunit.client.0.smithi142.stdout:5/828: chown df/d16/d25/l62 403 1 2024-03-29T04:45:06.741 INFO:tasks.workunit.client.0.smithi142.stdout:6/645: fsync f18 0 2024-03-29T04:45:06.744 INFO:tasks.workunit.client.0.smithi142.stdout:4/671: dread d0/d9/d11/d84/fca [0,4194304] 0 2024-03-29T04:45:06.745 INFO:tasks.workunit.client.0.smithi142.stdout:9/526: link d0/lb d0/d6c/la9 0 2024-03-29T04:45:06.749 INFO:tasks.workunit.client.0.smithi142.stdout:3/531: dwrite d3/d29/d2b/d40/d67/d8b/d4d/f60 [0,4194304] 0 2024-03-29T04:45:06.749 INFO:tasks.workunit.client.0.smithi142.stdout:7/744: dread d7/d27/d44/db2/db8/de/d20/f98 [0,4194304] 0 2024-03-29T04:45:06.751 INFO:tasks.workunit.client.0.smithi142.stdout:6/646: write d1c/d33/d46/d61/d9e/fd1 [824449,125178] 0 2024-03-29T04:45:06.751 INFO:tasks.workunit.client.0.smithi142.stdout:9/527: read d0/d12/f60 [88230,61382] 0 2024-03-29T04:45:06.753 INFO:tasks.workunit.client.0.smithi142.stdout:2/629: dwrite de/f29 [0,4194304] 0 2024-03-29T04:45:06.754 INFO:tasks.workunit.client.0.smithi142.stdout:9/528: fdatasync d0/d2/d6/d31/f49 0 2024-03-29T04:45:06.755 INFO:tasks.workunit.client.0.smithi142.stdout:4/672: rename d0/d9/d11/d4c/d64 to d0/d9/d11/d4c/d64/d94/ddd/de3 22 2024-03-29T04:45:06.758 INFO:tasks.workunit.client.0.smithi142.stdout:7/745: write d7/d27/d44/db2/db8/de/d20/ddf/fe1 [5048,38324] 0 2024-03-29T04:45:06.759 INFO:tasks.workunit.client.0.smithi142.stdout:3/532: symlink d3/d29/d2d/d56/lb5 0 2024-03-29T04:45:06.761 INFO:tasks.workunit.client.0.smithi142.stdout:6/647: dwrite d1c/f62 [0,4194304] 0 2024-03-29T04:45:06.761 INFO:tasks.workunit.client.0.smithi142.stdout:2/630: symlink de/d11/d1e/d84/lda 0 2024-03-29T04:45:06.764 INFO:tasks.workunit.client.0.smithi142.stdout:5/829: dread df/d1a/d21/d55/d66/f92 [0,4194304] 0 2024-03-29T04:45:06.766 INFO:tasks.workunit.client.0.smithi142.stdout:4/673: mkdir d0/d9/d11/d4c/d5f/d81/dc9/d47/d7a/de4 0 2024-03-29T04:45:06.771 INFO:tasks.workunit.client.0.smithi142.stdout:2/631: creat de/d11/d1e/d22/d50/fdb x:0 0 0 2024-03-29T04:45:06.773 INFO:tasks.workunit.client.0.smithi142.stdout:4/674: mkdir d0/d9/d11/d4c/d5f/d81/dc9/de5 0 2024-03-29T04:45:06.775 INFO:tasks.workunit.client.0.smithi142.stdout:9/529: rename d0/d2/d6/d31/d85 to d0/d2/d6/d16/daa 0 2024-03-29T04:45:06.775 INFO:tasks.workunit.client.0.smithi142.stdout:2/632: mknod de/d11/d17/d4a/d79/da8/cdc 0 2024-03-29T04:45:06.777 INFO:tasks.workunit.client.0.smithi142.stdout:9/530: mknod d0/d2/d6/d27/d3c/d4f/cab 0 2024-03-29T04:45:06.778 INFO:tasks.workunit.client.0.smithi142.stdout:2/633: symlink de/d11/d2b/d7b/ldd 0 2024-03-29T04:45:06.779 INFO:tasks.workunit.client.0.smithi142.stdout:4/675: link d0/f14 d0/d9/db2/dba/fe6 0 2024-03-29T04:45:06.782 INFO:tasks.workunit.client.0.smithi142.stdout:4/676: creat d0/d9/d11/d4c/d5f/d81/dc9/fe7 x:0 0 0 2024-03-29T04:45:06.784 INFO:tasks.workunit.client.0.smithi142.stdout:4/677: fsync d0/d9/f4d 0 2024-03-29T04:45:06.785 INFO:tasks.workunit.client.0.smithi142.stdout:2/634: getdents de/d11/d1e/d22/d27/d3a 0 2024-03-29T04:45:06.786 INFO:tasks.workunit.client.0.smithi142.stdout:4/678: creat d0/dc/d15/d3d/fe8 x:0 0 0 2024-03-29T04:45:06.789 INFO:tasks.workunit.client.0.smithi142.stdout:6/648: dread d1c/d45/d51/f7f [0,4194304] 0 2024-03-29T04:45:06.790 INFO:tasks.workunit.client.0.smithi142.stdout:9/531: dread d0/d2/d6/d16/f66 [0,4194304] 0 2024-03-29T04:45:06.796 INFO:tasks.workunit.client.0.smithi142.stdout:4/679: unlink d0/d9/d11/d78/la7 0 2024-03-29T04:45:06.796 INFO:tasks.workunit.client.0.smithi142.stdout:7/746: dread d7/d27/d44/db2/db8/d4b/f9a [0,4194304] 0 2024-03-29T04:45:06.796 INFO:tasks.workunit.client.0.smithi142.stdout:3/533: dwrite d3/f3c [0,4194304] 0 2024-03-29T04:45:06.797 INFO:tasks.workunit.client.0.smithi142.stdout:6/649: truncate d1c/d33/d46/d86/fcf 422561 0 2024-03-29T04:45:06.798 INFO:tasks.workunit.client.0.smithi142.stdout:5/830: dwrite f7 [0,4194304] 0 2024-03-29T04:45:06.799 INFO:tasks.workunit.client.0.smithi142.stdout:6/650: readlink d1c/d90/lad 0 2024-03-29T04:45:06.799 INFO:tasks.workunit.client.0.smithi142.stdout:2/635: dwrite de/f19 [0,4194304] 0 2024-03-29T04:45:06.802 INFO:tasks.workunit.client.0.smithi142.stdout:5/831: fdatasync df/d1a/ff3 0 2024-03-29T04:45:06.831 INFO:tasks.workunit.client.0.smithi142.stdout:6/651: dread d1c/d33/d46/d71/f9c [0,4194304] 0 2024-03-29T04:45:06.859 INFO:tasks.workunit.client.0.smithi142.stdout:9/532: creat d0/d2/d61/d64/fac x:0 0 0 2024-03-29T04:45:06.859 INFO:tasks.workunit.client.0.smithi142.stdout:4/680: write d0/f21 [786857,83550] 0 2024-03-29T04:45:06.860 INFO:tasks.workunit.client.0.smithi142.stdout:9/533: fdatasync d0/d2/d6/d31/f71 0 2024-03-29T04:45:06.860 INFO:tasks.workunit.client.0.smithi142.stdout:3/534: creat d3/d29/d2b/d40/d67/fb6 x:0 0 0 2024-03-29T04:45:06.861 INFO:tasks.workunit.client.0.smithi142.stdout:3/535: stat d3/d29/d8f/d5f/db4/f58 0 2024-03-29T04:45:06.861 INFO:tasks.workunit.client.0.smithi142.stdout:2/636: unlink de/d11/d1e/d22/d50/l88 0 2024-03-29T04:45:06.863 INFO:tasks.workunit.client.0.smithi142.stdout:2/637: write de/d11/d3d/fd9 [517851,76394] 0 2024-03-29T04:45:06.863 INFO:tasks.workunit.client.0.smithi142.stdout:2/638: fsync de/d11/d1e/fb9 0 2024-03-29T04:45:06.966 INFO:tasks.workunit.client.1.smithi152.stdout:0/106: sync 2024-03-29T04:45:06.973 INFO:tasks.workunit.client.1.smithi152.stdout:0/107: dwrite d2/da/f24 [0,4194304] 0 2024-03-29T04:45:06.989 INFO:tasks.workunit.client.0.smithi142.stdout:5/832: dwrite df/d1a/ffe [0,4194304] 0 2024-03-29T04:45:07.005 INFO:tasks.workunit.client.0.smithi142.stdout:4/681: mkdir d0/d9/db2/dba/de9 0 2024-03-29T04:45:07.006 INFO:tasks.workunit.client.0.smithi142.stdout:3/536: creat d3/d29/d2b/d40/d67/fb7 x:0 0 0 2024-03-29T04:45:07.007 INFO:tasks.workunit.client.0.smithi142.stdout:2/639: symlink de/d11/d1e/d22/d27/dba/d61/d7d/db4/lde 0 2024-03-29T04:45:07.008 INFO:tasks.workunit.client.0.smithi142.stdout:2/640: read - de/d11/d17/fcc zero size 2024-03-29T04:45:07.009 INFO:tasks.workunit.client.0.smithi142.stdout:7/747: rename d7/d27/d44/db2/db8/d89/cbe to d7/d27/d44/db2/db8/de/d20/d86/dd4/ddb/df6/cff 0 2024-03-29T04:45:07.012 INFO:tasks.workunit.client.0.smithi142.stdout:5/833: symlink df/d16/d24/d5c/db1/l113 0 2024-03-29T04:45:07.014 INFO:tasks.workunit.client.0.smithi142.stdout:4/682: fsync d0/d9/d11/d4c/d5f/d81/dc9/d47/f62 0 2024-03-29T04:45:07.016 INFO:tasks.workunit.client.0.smithi142.stdout:3/537: unlink d3/d29/d2b/d40/d67/d8b/d6/c18 0 2024-03-29T04:45:07.019 INFO:tasks.workunit.client.0.smithi142.stdout:7/748: truncate d7/d27/d44/db2/db8/de/d20/d3f/f61 1288008 0 2024-03-29T04:45:07.020 INFO:tasks.workunit.client.0.smithi142.stdout:5/834: fsync df/fa2 0 2024-03-29T04:45:07.021 INFO:tasks.workunit.client.0.smithi142.stdout:4/683: truncate d0/d9/f49 528461 0 2024-03-29T04:45:07.022 INFO:tasks.workunit.client.0.smithi142.stdout:3/538: creat d3/d29/d8f/d5f/d64/fb8 x:0 0 0 2024-03-29T04:45:07.023 INFO:tasks.workunit.client.0.smithi142.stdout:6/652: getdents d1c/d45/d51 0 2024-03-29T04:45:07.026 INFO:tasks.workunit.client.0.smithi142.stdout:5/835: unlink df/d29/d7a/d97/ff9 0 2024-03-29T04:45:07.027 INFO:tasks.workunit.client.0.smithi142.stdout:4/684: mkdir d0/d9/d11/d4c/d5f/d81/dc9/d47/dea 0 2024-03-29T04:45:07.029 INFO:tasks.workunit.client.0.smithi142.stdout:7/749: dwrite d7/d27/d44/db2/db8/d4b/d71/f9d [0,4194304] 0 2024-03-29T04:45:07.031 INFO:tasks.workunit.client.1.smithi152.stdout:2/62: sync 2024-03-29T04:45:07.033 INFO:tasks.workunit.client.1.smithi152.stdout:2/63: write d0/d3/f5 [2833569,64245] 0 2024-03-29T04:45:07.035 INFO:tasks.workunit.client.0.smithi142.stdout:3/539: rename d3/d29/d2b/d40/d67/d8b/d6/d93 to d3/db9 0 2024-03-29T04:45:07.038 INFO:tasks.workunit.client.1.smithi152.stdout:2/64: getdents d0/d3 0 2024-03-29T04:45:07.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:06 smithi152 ceph-mon[37177]: pgmap v269: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 41 MiB/s rd, 165 MiB/s wr, 318 op/s 2024-03-29T04:45:07.040 INFO:tasks.workunit.client.0.smithi142.stdout:4/685: dwrite d0/d9/d19/d2c/fb9 [0,4194304] 0 2024-03-29T04:45:07.042 INFO:tasks.workunit.client.1.smithi152.stdout:2/65: dread d0/d3/f5 [0,4194304] 0 2024-03-29T04:45:07.043 INFO:tasks.workunit.client.0.smithi142.stdout:4/686: mknod d0/d9/d11/d4c/d64/ceb 0 2024-03-29T04:45:07.050 INFO:tasks.workunit.client.0.smithi142.stdout:4/687: getdents d0/d9/d11/d4c 0 2024-03-29T04:45:07.050 INFO:tasks.workunit.client.1.smithi152.stdout:2/66: truncate d0/d3/f5 3443744 0 2024-03-29T04:45:07.051 INFO:tasks.workunit.client.1.smithi152.stdout:2/67: chown d0/f2 0 1 2024-03-29T04:45:07.051 INFO:tasks.workunit.client.1.smithi152.stdout:2/68: readlink d0/d3/l7 0 2024-03-29T04:45:07.051 INFO:tasks.workunit.client.1.smithi152.stdout:2/69: rename d0 to d0/d14/d17 22 2024-03-29T04:45:07.052 INFO:tasks.workunit.client.1.smithi152.stdout:2/70: truncate d0/f16 800328 0 2024-03-29T04:45:07.053 INFO:tasks.workunit.client.0.smithi142.stdout:9/534: truncate d0/d2/d6/d16/f2b 2203015 0 2024-03-29T04:45:07.055 INFO:tasks.workunit.client.0.smithi142.stdout:9/535: dread - d0/d12/d50/d8e/fa4 zero size 2024-03-29T04:45:07.055 INFO:tasks.workunit.client.0.smithi142.stdout:2/641: write de/d11/f37 [1460429,124829] 0 2024-03-29T04:45:07.055 INFO:tasks.workunit.client.1.smithi152.stdout:2/71: creat d0/d3/f18 x:0 0 0 2024-03-29T04:45:07.056 INFO:tasks.workunit.client.0.smithi142.stdout:7/750: dread d7/f87 [0,4194304] 0 2024-03-29T04:45:07.058 INFO:tasks.workunit.client.0.smithi142.stdout:5/836: getdents df/d29/d7a/d97 0 2024-03-29T04:45:07.059 INFO:tasks.workunit.client.0.smithi142.stdout:9/536: fdatasync d0/f94 0 2024-03-29T04:45:07.059 INFO:tasks.workunit.client.0.smithi142.stdout:6/653: write d1c/d45/f76 [559044,19663] 0 2024-03-29T04:45:07.062 INFO:tasks.workunit.client.1.smithi152.stdout:2/72: dwrite d0/f2 [0,4194304] 0 2024-03-29T04:45:07.063 INFO:tasks.workunit.client.0.smithi142.stdout:3/540: dread d3/d29/d2b/d40/d67/d8b/d6/f20 [0,4194304] 0 2024-03-29T04:45:07.065 INFO:tasks.workunit.client.0.smithi142.stdout:3/541: read d3/d29/d2d/f48 [41411,39563] 0 2024-03-29T04:45:07.065 INFO:tasks.workunit.client.0.smithi142.stdout:3/542: chown d3/d1f/d24/d47/f75 633 1 2024-03-29T04:45:07.068 INFO:tasks.workunit.client.0.smithi142.stdout:9/537: mknod d0/d12/d50/cad 0 2024-03-29T04:45:07.070 INFO:tasks.workunit.client.0.smithi142.stdout:1/640: sync 2024-03-29T04:45:07.070 INFO:tasks.workunit.client.0.smithi142.stdout:0/672: sync 2024-03-29T04:45:07.070 INFO:tasks.workunit.client.0.smithi142.stdout:8/768: sync 2024-03-29T04:45:07.073 INFO:tasks.workunit.client.0.smithi142.stdout:3/543: symlink d3/d29/d2d/d79/lba 0 2024-03-29T04:45:07.074 INFO:tasks.workunit.client.0.smithi142.stdout:4/688: dread d0/d9/f4d [0,4194304] 0 2024-03-29T04:45:07.075 INFO:tasks.workunit.client.0.smithi142.stdout:8/769: creat d5/d2e/dd2/d60/f10b x:0 0 0 2024-03-29T04:45:07.077 INFO:tasks.workunit.client.0.smithi142.stdout:0/673: mknod d7/d9/d54/cdc 0 2024-03-29T04:45:07.078 INFO:tasks.workunit.client.0.smithi142.stdout:1/641: mknod d8/db/dc/d77/db9/dbe/d15/d9c/cdd 0 2024-03-29T04:45:07.078 INFO:tasks.workunit.client.0.smithi142.stdout:9/538: dwrite d0/d2/fa2 [0,4194304] 0 2024-03-29T04:45:07.079 INFO:tasks.workunit.client.0.smithi142.stdout:2/642: truncate fd 4863293 0 2024-03-29T04:45:07.079 INFO:tasks.workunit.client.0.smithi142.stdout:3/544: creat d3/d1f/d59/fbb x:0 0 0 2024-03-29T04:45:07.081 INFO:tasks.workunit.client.0.smithi142.stdout:1/642: fsync d8/f51 0 2024-03-29T04:45:07.082 INFO:tasks.workunit.client.0.smithi142.stdout:4/689: creat d0/d9/db2/dba/dbc/d6d/d6e/fec x:0 0 0 2024-03-29T04:45:07.083 INFO:tasks.workunit.client.0.smithi142.stdout:9/539: readlink d0/d2/d6/d27/d3c/d4f/l8a 0 2024-03-29T04:45:07.084 INFO:tasks.workunit.client.0.smithi142.stdout:8/770: rename d5/d1d/d24/l23 to d5/d2e/dd2/df6/l10c 0 2024-03-29T04:45:07.084 INFO:tasks.workunit.client.0.smithi142.stdout:6/654: truncate d1c/f40 7445889 0 2024-03-29T04:45:07.085 INFO:tasks.workunit.client.0.smithi142.stdout:0/674: truncate d7/d13/dbf/dca/dab/fd5 792 0 2024-03-29T04:45:07.086 INFO:tasks.workunit.client.0.smithi142.stdout:8/771: readlink d5/d93/laa 0 2024-03-29T04:45:07.088 INFO:tasks.workunit.client.0.smithi142.stdout:0/675: stat d7/d9/d55/c76 0 2024-03-29T04:45:07.088 INFO:tasks.workunit.client.0.smithi142.stdout:7/751: dread d7/d27/d44/f60 [0,4194304] 0 2024-03-29T04:45:07.088 INFO:tasks.workunit.client.0.smithi142.stdout:9/540: write d0/d2/d6/d27/d3c/f4c [250607,14868] 0 2024-03-29T04:45:07.090 INFO:tasks.workunit.client.0.smithi142.stdout:3/545: rename d3/d29/d2b/d40/d67/d8b/d6/l1b to d3/d29/d8f/d5f/d64/lbc 0 2024-03-29T04:45:07.093 INFO:tasks.workunit.client.0.smithi142.stdout:2/643: getdents de/d11/d2b 0 2024-03-29T04:45:07.093 INFO:tasks.workunit.client.0.smithi142.stdout:5/837: dread df/d16/d24/d5c/db1/fc3 [0,4194304] 0 2024-03-29T04:45:07.095 INFO:tasks.workunit.client.0.smithi142.stdout:0/676: unlink d7/d9/f94 0 2024-03-29T04:45:07.096 INFO:tasks.workunit.client.0.smithi142.stdout:5/838: fsync df/d29/d7a/fc4 0 2024-03-29T04:45:07.096 INFO:tasks.workunit.client.0.smithi142.stdout:1/643: creat d8/db/dc/d77/db9/dbe/d2d/d8b/dda/fde x:0 0 0 2024-03-29T04:45:07.098 INFO:tasks.workunit.client.0.smithi142.stdout:2/644: chown de/d11/d23/fb7 696688 1 2024-03-29T04:45:07.098 INFO:tasks.workunit.client.0.smithi142.stdout:8/772: dwrite d5/d1d/d2f/fd0 [0,4194304] 0 2024-03-29T04:45:07.099 INFO:tasks.workunit.client.0.smithi142.stdout:9/541: fdatasync d0/d12/d50/f76 0 2024-03-29T04:45:07.100 INFO:tasks.workunit.client.0.smithi142.stdout:7/752: mknod d7/d27/d44/db2/db8/de/d20/d3f/c100 0 2024-03-29T04:45:07.101 INFO:tasks.workunit.client.0.smithi142.stdout:2/645: chown de/d11/d23 2252 1 2024-03-29T04:45:07.102 INFO:tasks.workunit.client.0.smithi142.stdout:9/542: fdatasync d0/d12/fa0 0 2024-03-29T04:45:07.104 INFO:tasks.workunit.client.0.smithi142.stdout:7/753: stat d7/d27/d3a/d92/d97/dea 0 2024-03-29T04:45:07.105 INFO:tasks.workunit.client.0.smithi142.stdout:5/839: mknod df/d29/d7a/c114 0 2024-03-29T04:45:07.106 INFO:tasks.workunit.client.0.smithi142.stdout:9/543: write d0/d2/d6/f81 [1045278,19715] 0 2024-03-29T04:45:07.107 INFO:tasks.workunit.client.0.smithi142.stdout:7/754: write d7/d62/ff7 [869393,79227] 0 2024-03-29T04:45:07.109 INFO:tasks.workunit.client.0.smithi142.stdout:9/544: write d0/d2/d6/d16/f3b [2831596,76906] 0 2024-03-29T04:45:07.109 INFO:tasks.workunit.client.0.smithi142.stdout:0/677: dread d7/d9/d1e/f71 [0,4194304] 0 2024-03-29T04:45:07.110 INFO:tasks.workunit.client.0.smithi142.stdout:7/755: chown d7/d27/d44/db2/db8/de/d8d/fa5 20425 1 2024-03-29T04:45:07.111 INFO:tasks.workunit.client.0.smithi142.stdout:1/644: truncate d8/db/dc/d77/d31/f61 2113387 0 2024-03-29T04:45:07.112 INFO:tasks.workunit.client.0.smithi142.stdout:9/545: chown d0/d2/d61/d64/f80 58934401 1 2024-03-29T04:45:07.112 INFO:tasks.workunit.client.0.smithi142.stdout:5/840: dread df/d1a/ffe [0,4194304] 0 2024-03-29T04:45:07.112 INFO:tasks.workunit.client.0.smithi142.stdout:0/678: write d7/d9/d55/d42/f5c [299230,74182] 0 2024-03-29T04:45:07.114 INFO:tasks.workunit.client.0.smithi142.stdout:8/773: symlink d5/d1d/dd9/l10d 0 2024-03-29T04:45:07.114 INFO:tasks.workunit.client.0.smithi142.stdout:5/841: fsync df/d1a/f2f 0 2024-03-29T04:45:07.115 INFO:tasks.workunit.client.0.smithi142.stdout:6/655: truncate d1c/d33/db6/da4/faf 1110053 0 2024-03-29T04:45:07.118 INFO:tasks.workunit.client.0.smithi142.stdout:7/756: read - d7/d27/f81 zero size 2024-03-29T04:45:07.119 INFO:tasks.workunit.client.0.smithi142.stdout:8/774: dwrite d5/d1d/d24/f6b [0,4194304] 0 2024-03-29T04:45:07.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:06 smithi142 ceph-mon[29027]: pgmap v269: 65 pgs: 65 active+clean; 1.3 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 41 MiB/s rd, 165 MiB/s wr, 318 op/s 2024-03-29T04:45:07.121 INFO:tasks.workunit.client.0.smithi142.stdout:0/679: creat d7/d13/dbf/dca/dab/dc1/fdd x:0 0 0 2024-03-29T04:45:07.124 INFO:tasks.workunit.client.0.smithi142.stdout:6/656: chown d1c/d33/db6/d52/l9a 16540689 1 2024-03-29T04:45:07.125 INFO:tasks.workunit.client.0.smithi142.stdout:2/646: creat de/d11/d1e/d22/d27/dba/d61/fdf x:0 0 0 2024-03-29T04:45:07.127 INFO:tasks.workunit.client.0.smithi142.stdout:7/757: symlink d7/d27/d44/db2/db8/d4b/d71/d30/d33/df5/l101 0 2024-03-29T04:45:07.129 INFO:tasks.workunit.client.0.smithi142.stdout:8/775: dread d5/d2e/f65 [0,4194304] 0 2024-03-29T04:45:07.130 INFO:tasks.workunit.client.0.smithi142.stdout:0/680: creat d7/d13/dbf/dca/dab/dc1/fde x:0 0 0 2024-03-29T04:45:07.131 INFO:tasks.workunit.client.0.smithi142.stdout:0/681: chown d7/d13/dbf/d72/d75/fd7 232111942 1 2024-03-29T04:45:07.131 INFO:tasks.workunit.client.0.smithi142.stdout:1/645: creat d8/db/dc/d77/fdf x:0 0 0 2024-03-29T04:45:07.132 INFO:tasks.workunit.client.0.smithi142.stdout:5/842: dread df/d16/d24/d5c/f5e [0,4194304] 0 2024-03-29T04:45:07.134 INFO:tasks.workunit.client.0.smithi142.stdout:7/758: creat d7/d27/d44/db2/db8/de/d20/d86/dd4/ddb/df6/f102 x:0 0 0 2024-03-29T04:45:07.135 INFO:tasks.workunit.client.0.smithi142.stdout:8/776: mknod d5/d1d/d2f/d5f/d48/c10e 0 2024-03-29T04:45:07.135 INFO:tasks.workunit.client.0.smithi142.stdout:7/759: write d7/fd [6932009,52117] 0 2024-03-29T04:45:07.136 INFO:tasks.workunit.client.0.smithi142.stdout:1/646: chown d8/db/d37/f38 32312732 1 2024-03-29T04:45:07.137 INFO:tasks.workunit.client.0.smithi142.stdout:7/760: chown d7/d27/d3a/l95 1 1 2024-03-29T04:45:07.139 INFO:tasks.workunit.client.0.smithi142.stdout:9/546: write d0/d2/f2f [324080,16810] 0 2024-03-29T04:45:07.141 INFO:tasks.workunit.client.0.smithi142.stdout:9/547: chown d0/d12/d50/f93 1 1 2024-03-29T04:45:07.142 INFO:tasks.workunit.client.0.smithi142.stdout:8/777: symlink d5/d1d/d2f/d5f/d48/l10f 0 2024-03-29T04:45:07.143 INFO:tasks.workunit.client.0.smithi142.stdout:6/657: link d1c/d33/d46/d71/c72 d1c/d45/d51/cd3 0 2024-03-29T04:45:07.144 INFO:tasks.workunit.client.0.smithi142.stdout:2/647: creat de/d11/d1e/d22/d27/fe0 x:0 0 0 2024-03-29T04:45:07.145 INFO:tasks.workunit.client.0.smithi142.stdout:7/761: creat d7/d27/d44/db2/db8/de/d8d/f103 x:0 0 0 2024-03-29T04:45:07.145 INFO:tasks.workunit.client.0.smithi142.stdout:0/682: unlink d7/d13/dbf/dca/dab/dc1/dd3/d90/cb7 0 2024-03-29T04:45:07.147 INFO:tasks.workunit.client.0.smithi142.stdout:6/658: write d1c/d33/f99 [738027,96053] 0 2024-03-29T04:45:07.148 INFO:tasks.workunit.client.0.smithi142.stdout:1/647: dwrite d8/db/dc/d77/d31/d4d/d56/f58 [0,4194304] 0 2024-03-29T04:45:07.150 INFO:tasks.workunit.client.0.smithi142.stdout:5/843: dwrite df/d1a/d21/d27/d45/f111 [0,4194304] 0 2024-03-29T04:45:07.155 INFO:tasks.workunit.client.0.smithi142.stdout:0/683: mkdir d7/d13/dbf/dca/dab/ddf 0 2024-03-29T04:45:07.156 INFO:tasks.workunit.client.0.smithi142.stdout:7/762: creat d7/d27/d44/db2/db8/de/d20/d86/f104 x:0 0 0 2024-03-29T04:45:07.157 INFO:tasks.workunit.client.0.smithi142.stdout:9/548: truncate d0/d2/f1f 1647530 0 2024-03-29T04:45:07.158 INFO:tasks.workunit.client.0.smithi142.stdout:6/659: mknod d1c/d33/d46/d66/dc1/cd4 0 2024-03-29T04:45:07.159 INFO:tasks.workunit.client.0.smithi142.stdout:5/844: chown df/d16/d24/d2e/d47/d4d/d102/d75/de2 50 1 2024-03-29T04:45:07.161 INFO:tasks.workunit.client.0.smithi142.stdout:8/778: dwrite d5/d1d/d2f/d5f/fb1 [0,4194304] 0 2024-03-29T04:45:07.161 INFO:tasks.workunit.client.0.smithi142.stdout:0/684: unlink d7/d9/d1e/lc6 0 2024-03-29T04:45:07.162 INFO:tasks.workunit.client.0.smithi142.stdout:1/648: mknod d8/db/dc/d66/ce0 0 2024-03-29T04:45:07.163 INFO:tasks.workunit.client.0.smithi142.stdout:9/549: write d0/d2/d6/d27/d3c/d4f/f98 [1405061,37530] 0 2024-03-29T04:45:07.163 INFO:tasks.workunit.client.0.smithi142.stdout:5/845: fsync df/d29/d6e/f101 0 2024-03-29T04:45:07.169 INFO:tasks.workunit.client.0.smithi142.stdout:2/648: dread de/d11/d1e/d22/d27/dba/d61/f78 [0,4194304] 0 2024-03-29T04:45:07.172 INFO:tasks.workunit.client.0.smithi142.stdout:7/763: dread d7/d27/d44/db2/db8/de/d20/f4e [0,4194304] 0 2024-03-29T04:45:07.173 INFO:tasks.workunit.client.0.smithi142.stdout:0/685: dwrite d7/d9/d1e/f88 [0,4194304] 0 2024-03-29T04:45:07.178 INFO:tasks.workunit.client.1.smithi152.stdout:1/68: sync 2024-03-29T04:45:07.178 INFO:tasks.workunit.client.1.smithi152.stdout:4/17: sync 2024-03-29T04:45:07.178 INFO:tasks.workunit.client.1.smithi152.stdout:8/52: sync 2024-03-29T04:45:07.179 INFO:tasks.workunit.client.0.smithi142.stdout:0/686: dwrite d7/d13/dbf/dca/dab/dc1/dd3/f8b [0,4194304] 0 2024-03-29T04:45:07.182 INFO:tasks.workunit.client.1.smithi152.stdout:8/53: creat d1/f10 x:0 0 0 2024-03-29T04:45:07.185 INFO:tasks.workunit.client.1.smithi152.stdout:4/18: dread d0/f1 [0,4194304] 0 2024-03-29T04:45:07.188 INFO:tasks.workunit.client.1.smithi152.stdout:4/19: truncate d0/f4 211050 0 2024-03-29T04:45:07.189 INFO:tasks.workunit.client.0.smithi142.stdout:9/550: creat d0/d2/d6/d16/d6a/fae x:0 0 0 2024-03-29T04:45:07.189 INFO:tasks.workunit.client.0.smithi142.stdout:7/764: chown d7/c1d 9 1 2024-03-29T04:45:07.190 INFO:tasks.workunit.client.1.smithi152.stdout:8/54: dread d1/d2/fc [4194304,4194304] 0 2024-03-29T04:45:07.191 INFO:tasks.workunit.client.0.smithi142.stdout:7/765: stat d7/d27/d44/db2/db8/de/d20/d86/dd4/ddb/df6 0 2024-03-29T04:45:07.191 INFO:tasks.workunit.client.0.smithi142.stdout:0/687: chown d7/d9/d55/d42/f5f 3 1 2024-03-29T04:45:07.192 INFO:tasks.workunit.client.0.smithi142.stdout:6/660: unlink d1c/d33/d46/d61/d9b/cc5 0 2024-03-29T04:45:07.194 INFO:tasks.workunit.client.0.smithi142.stdout:4/690: sync 2024-03-29T04:45:07.198 INFO:tasks.workunit.client.0.smithi142.stdout:8/779: getdents d5/d1d/d81 0 2024-03-29T04:45:07.199 INFO:tasks.workunit.client.0.smithi142.stdout:0/688: rename d7/d13/dbf/d72/l5d to d7/d13/dbf/dca/dab/dc1/le0 0 2024-03-29T04:45:07.200 INFO:tasks.workunit.client.0.smithi142.stdout:6/661: unlink d1c/d33/d46/d66/cd0 0 2024-03-29T04:45:07.201 INFO:tasks.workunit.client.0.smithi142.stdout:5/846: link df/d1a/lc0 df/l115 0 2024-03-29T04:45:07.202 INFO:tasks.workunit.client.0.smithi142.stdout:5/847: stat df/d16/d24/d2e/d47/d8c/df1 0 2024-03-29T04:45:07.202 INFO:tasks.workunit.client.0.smithi142.stdout:2/649: getdents de/d11/d1e/d22/d27/dba 0 2024-03-29T04:45:07.204 INFO:tasks.workunit.client.0.smithi142.stdout:9/551: link d0/d12/f54 d0/d2/d6/d16/faf 0 2024-03-29T04:45:07.205 INFO:tasks.workunit.client.0.smithi142.stdout:8/780: creat d5/d2e/dd2/d60/f110 x:0 0 0 2024-03-29T04:45:07.210 INFO:tasks.workunit.client.0.smithi142.stdout:0/689: mkdir d7/d13/dbf/dca/dab/dc1/dd3/de1 0 2024-03-29T04:45:07.211 INFO:tasks.workunit.client.0.smithi142.stdout:5/848: mkdir df/d16/d24/d2e/d47/d8c/df1/d116 0 2024-03-29T04:45:07.211 INFO:tasks.workunit.client.0.smithi142.stdout:1/649: write d8/db/dc/d77/f35 [746401,53721] 0 2024-03-29T04:45:07.211 INFO:tasks.workunit.client.0.smithi142.stdout:2/650: getdents de/d11/d1e/d22/dbf 0 2024-03-29T04:45:07.212 INFO:tasks.workunit.client.0.smithi142.stdout:9/552: mknod d0/d2/d6/d31/cb0 0 2024-03-29T04:45:07.213 INFO:tasks.workunit.client.0.smithi142.stdout:2/651: stat de/d11/d1e/d22/d50/fdb 0 2024-03-29T04:45:07.214 INFO:tasks.workunit.client.0.smithi142.stdout:0/690: mknod d7/d13/dbf/d72/d75/ce2 0 2024-03-29T04:45:07.215 INFO:tasks.workunit.client.0.smithi142.stdout:0/691: chown d7/d9/d1e/f88 30190 1 2024-03-29T04:45:07.216 INFO:tasks.workunit.client.0.smithi142.stdout:0/692: chown d7/f21 4555255 1 2024-03-29T04:45:07.216 INFO:tasks.workunit.client.0.smithi142.stdout:0/693: chown d7/dbc 0 1 2024-03-29T04:45:07.217 INFO:tasks.workunit.client.0.smithi142.stdout:5/849: symlink df/d29/d7a/d97/db3/dea/l117 0 2024-03-29T04:45:07.219 INFO:tasks.workunit.client.0.smithi142.stdout:1/650: truncate d8/db/f14 2724586 0 2024-03-29T04:45:07.220 INFO:tasks.workunit.client.0.smithi142.stdout:4/691: dread d0/d9/db2/dba/dbc/d8d/f9b [0,4194304] 0 2024-03-29T04:45:07.241 INFO:tasks.workunit.client.1.smithi152.stdout:4/20: truncate d0/f1 4461015 0 2024-03-29T04:45:07.243 INFO:tasks.workunit.client.1.smithi152.stdout:1/69: dwrite d5/d7/d9/f10 [0,4194304] 0 2024-03-29T04:45:07.244 INFO:tasks.workunit.client.1.smithi152.stdout:4/21: unlink d0/f2 0 2024-03-29T04:45:07.246 INFO:tasks.workunit.client.1.smithi152.stdout:1/70: symlink d5/d7/d8/d13/l1c 0 2024-03-29T04:45:07.247 INFO:tasks.workunit.client.1.smithi152.stdout:4/22: rename d0/l3 to d0/l5 0 2024-03-29T04:45:07.248 INFO:tasks.workunit.client.1.smithi152.stdout:1/71: fdatasync d5/d7/d9/f11 0 2024-03-29T04:45:07.248 INFO:tasks.workunit.client.1.smithi152.stdout:8/55: dwrite d1/d2/d3/f9 [4194304,4194304] 0 2024-03-29T04:45:07.249 INFO:tasks.workunit.client.1.smithi152.stdout:4/23: mkdir d0/d6 0 2024-03-29T04:45:07.250 INFO:tasks.workunit.client.1.smithi152.stdout:8/56: chown d1/d2/d3/l8 31708024 1 2024-03-29T04:45:07.251 INFO:tasks.workunit.client.1.smithi152.stdout:1/72: symlink d5/l1d 0 2024-03-29T04:45:07.252 INFO:tasks.workunit.client.1.smithi152.stdout:4/24: mkdir d0/d7 0 2024-03-29T04:45:07.253 INFO:tasks.workunit.client.1.smithi152.stdout:4/25: read d0/f4 [168717,126934] 0 2024-03-29T04:45:07.253 INFO:tasks.workunit.client.1.smithi152.stdout:8/57: rename l0 to d1/d2/d3/l11 0 2024-03-29T04:45:07.254 INFO:tasks.workunit.client.1.smithi152.stdout:1/73: creat d5/d7/d8/f1e x:0 0 0 2024-03-29T04:45:07.254 INFO:tasks.workunit.client.1.smithi152.stdout:1/74: chown d5/d7/d1b 719 1 2024-03-29T04:45:07.255 INFO:tasks.workunit.client.1.smithi152.stdout:1/75: chown d5/d7/d8/f1e 49 1 2024-03-29T04:45:07.256 INFO:tasks.workunit.client.1.smithi152.stdout:1/76: write d5/d7/d9/f11 [345882,1580] 0 2024-03-29T04:45:07.261 INFO:tasks.workunit.client.0.smithi142.stdout:9/553: creat d0/d12/d50/d52/da3/fb1 x:0 0 0 2024-03-29T04:45:07.262 INFO:tasks.workunit.client.0.smithi142.stdout:0/694: rename d7/d13/dbf/d72/d37/l5a to d7/d9/d1e/d81/le3 0 2024-03-29T04:45:07.263 INFO:tasks.workunit.client.0.smithi142.stdout:5/850: creat df/d16/d24/d2e/d47/d4d/f118 x:0 0 0 2024-03-29T04:45:07.265 INFO:tasks.workunit.client.1.smithi152.stdout:4/26: readlink d0/l5 0 2024-03-29T04:45:07.265 INFO:tasks.workunit.client.0.smithi142.stdout:8/781: dread d5/d2e/dd2/d60/fd6 [0,4194304] 0 2024-03-29T04:45:07.266 INFO:tasks.workunit.client.1.smithi152.stdout:1/77: dwrite d5/d7/d9/f10 [4194304,4194304] 0 2024-03-29T04:45:07.267 INFO:tasks.workunit.client.0.smithi142.stdout:0/695: write d7/d9/d54/d58/fda [481651,92860] 0 2024-03-29T04:45:07.267 INFO:tasks.workunit.client.0.smithi142.stdout:1/651: symlink d8/le1 0 2024-03-29T04:45:07.269 INFO:tasks.workunit.client.0.smithi142.stdout:5/851: write df/d16/d24/d2e/d47/d4d/fe4 [364011,21687] 0 2024-03-29T04:45:07.270 INFO:tasks.workunit.client.0.smithi142.stdout:8/782: fdatasync d5/d2e/fcc 0 2024-03-29T04:45:07.271 INFO:tasks.workunit.client.0.smithi142.stdout:2/652: dread de/d11/d1e/d22/d27/dba/d61/f8f [0,4194304] 0 2024-03-29T04:45:07.272 INFO:tasks.workunit.client.1.smithi152.stdout:4/27: dread d0/f1 [0,4194304] 0 2024-03-29T04:45:07.273 INFO:tasks.workunit.client.1.smithi152.stdout:1/78: dread d5/d7/d9/f10 [0,4194304] 0 2024-03-29T04:45:07.273 INFO:tasks.workunit.client.0.smithi142.stdout:5/852: write df/d16/d24/d3d/fc8 [1535999,95303] 0 2024-03-29T04:45:07.274 INFO:tasks.workunit.client.0.smithi142.stdout:2/653: chown de/d11/d17/c1b 2454 1 2024-03-29T04:45:07.276 INFO:tasks.workunit.client.1.smithi152.stdout:1/79: creat d5/d7/f1f x:0 0 0 2024-03-29T04:45:07.277 INFO:tasks.workunit.client.0.smithi142.stdout:8/783: symlink d5/d1d/dd9/l111 0 2024-03-29T04:45:07.278 INFO:tasks.workunit.client.1.smithi152.stdout:4/28: rmdir d0/d6 0 2024-03-29T04:45:07.279 INFO:tasks.workunit.client.0.smithi142.stdout:4/692: dwrite d0/d9/d11/d78/fb1 [0,4194304] 0 2024-03-29T04:45:07.283 INFO:tasks.workunit.client.0.smithi142.stdout:5/853: creat df/d16/f119 x:0 0 0 2024-03-29T04:45:07.285 INFO:tasks.workunit.client.0.smithi142.stdout:5/854: mknod df/d2c/c11a 0 2024-03-29T04:45:07.285 INFO:tasks.workunit.client.1.smithi152.stdout:1/80: creat d5/f20 x:0 0 0 2024-03-29T04:45:07.287 INFO:tasks.workunit.client.0.smithi142.stdout:3/546: sync 2024-03-29T04:45:07.288 INFO:tasks.workunit.client.1.smithi152.stdout:7/66: sync 2024-03-29T04:45:07.288 INFO:tasks.workunit.client.0.smithi142.stdout:2/654: link de/d11/d2b/d3e/l7c de/d11/d1e/d22/d27/dba/d61/le1 0 2024-03-29T04:45:07.289 INFO:tasks.workunit.client.1.smithi152.stdout:7/67: chown d7 1058328985 1 2024-03-29T04:45:07.289 INFO:tasks.workunit.client.0.smithi142.stdout:5/855: mkdir df/d2c/d112/d11b 0 2024-03-29T04:45:07.291 INFO:tasks.workunit.client.0.smithi142.stdout:3/547: rmdir d3/d29 39 2024-03-29T04:45:07.293 INFO:tasks.workunit.client.0.smithi142.stdout:2/655: mkdir de/d11/da1/de2 0 2024-03-29T04:45:07.293 INFO:tasks.workunit.client.0.smithi142.stdout:8/784: rmdir d5/d2e/dd2/d60 39 2024-03-29T04:45:07.295 INFO:tasks.workunit.client.1.smithi152.stdout:1/81: mkdir d5/d7/d8/d13/d19/d21 0 2024-03-29T04:45:07.296 INFO:tasks.workunit.client.0.smithi142.stdout:0/696: dread d7/d13/dbf/d72/f3a [0,4194304] 0 2024-03-29T04:45:07.296 INFO:tasks.workunit.client.1.smithi152.stdout:8/58: truncate d1/d2/fc 7279766 0 2024-03-29T04:45:07.297 INFO:tasks.workunit.client.0.smithi142.stdout:2/656: write de/d11/d17/d4a/f55 [1503263,42555] 0 2024-03-29T04:45:07.297 INFO:tasks.workunit.client.0.smithi142.stdout:0/697: fdatasync d7/d13/f50 0 2024-03-29T04:45:07.297 INFO:tasks.workunit.client.1.smithi152.stdout:7/68: dread d7/f9 [0,4194304] 0 2024-03-29T04:45:07.298 INFO:tasks.workunit.client.0.smithi142.stdout:3/548: unlink d3/d29/d2b/d6a/f99 0 2024-03-29T04:45:07.298 INFO:tasks.workunit.client.1.smithi152.stdout:7/69: fdatasync d7/f10 0 2024-03-29T04:45:07.299 INFO:tasks.workunit.client.1.smithi152.stdout:1/82: symlink d5/d7/d8/d13/d19/l22 0 2024-03-29T04:45:07.302 INFO:tasks.workunit.client.0.smithi142.stdout:5/856: creat df/d29/f11c x:0 0 0 2024-03-29T04:45:07.303 INFO:tasks.workunit.client.1.smithi152.stdout:7/70: mkdir d7/d8/d12 0 2024-03-29T04:45:07.305 INFO:tasks.workunit.client.1.smithi152.stdout:4/29: dwrite d0/f4 [0,4194304] 0 2024-03-29T04:45:07.305 INFO:tasks.workunit.client.1.smithi152.stdout:7/71: read d7/d8/fa [298027,88825] 0 2024-03-29T04:45:07.306 INFO:tasks.workunit.client.0.smithi142.stdout:9/554: dread d0/d2/d6/d27/d3c/d4f/f56 [0,4194304] 0 2024-03-29T04:45:07.307 INFO:tasks.workunit.client.0.smithi142.stdout:2/657: chown de/d11/d17/d4a/d79/lab 803 1 2024-03-29T04:45:07.309 INFO:tasks.workunit.client.0.smithi142.stdout:2/658: fdatasync de/d11/d23/f46 0 2024-03-29T04:45:07.311 INFO:tasks.workunit.client.0.smithi142.stdout:5/857: rmdir df/d16/d24/d2e 39 2024-03-29T04:45:07.312 INFO:tasks.workunit.client.0.smithi142.stdout:5/858: chown fa 57762105 1 2024-03-29T04:45:07.312 INFO:tasks.workunit.client.1.smithi152.stdout:1/83: dwrite d5/f20 [0,4194304] 0 2024-03-29T04:45:07.312 INFO:tasks.workunit.client.1.smithi152.stdout:7/72: creat d7/d8/db/df/f13 x:0 0 0 2024-03-29T04:45:07.315 INFO:tasks.workunit.client.1.smithi152.stdout:9/131: sync 2024-03-29T04:45:07.315 INFO:tasks.workunit.client.1.smithi152.stdout:6/119: sync 2024-03-29T04:45:07.315 INFO:tasks.workunit.client.1.smithi152.stdout:7/73: write d7/f10 [873788,64902] 0 2024-03-29T04:45:07.315 INFO:tasks.workunit.client.1.smithi152.stdout:3/59: sync 2024-03-29T04:45:07.315 INFO:tasks.workunit.client.1.smithi152.stdout:4/30: dread d0/f1 [0,4194304] 0 2024-03-29T04:45:07.316 INFO:tasks.workunit.client.0.smithi142.stdout:7/766: sync 2024-03-29T04:45:07.316 INFO:tasks.workunit.client.0.smithi142.stdout:9/555: symlink d0/d2/d61/d64/lb2 0 2024-03-29T04:45:07.317 INFO:tasks.workunit.client.0.smithi142.stdout:1/652: truncate d8/db/d37/f41 2161427 0 2024-03-29T04:45:07.317 INFO:tasks.workunit.client.1.smithi152.stdout:6/120: chown d1/d3/d9 3 1 2024-03-29T04:45:07.317 INFO:tasks.workunit.client.0.smithi142.stdout:9/556: stat d0/d2/d6/d27/c41 0 2024-03-29T04:45:07.319 INFO:tasks.workunit.client.1.smithi152.stdout:9/132: rename d5/ce to d5/db/c2d 0 2024-03-29T04:45:07.319 INFO:tasks.workunit.client.0.smithi142.stdout:7/767: chown d7/c1d 896585 1 2024-03-29T04:45:07.319 INFO:tasks.workunit.client.0.smithi142.stdout:1/653: chown d8/f9 2 1 2024-03-29T04:45:07.320 INFO:tasks.workunit.client.0.smithi142.stdout:4/693: write d0/dc/d15/d3d/d65/f95 [153132,117600] 0 2024-03-29T04:45:07.320 INFO:tasks.workunit.client.1.smithi152.stdout:7/74: dread d7/d8/db/fe [0,4194304] 0 2024-03-29T04:45:07.321 INFO:tasks.workunit.client.0.smithi142.stdout:3/549: creat d3/d1f/fbd x:0 0 0 2024-03-29T04:45:07.323 INFO:tasks.workunit.client.1.smithi152.stdout:1/84: unlink d5/d7/d8/d13/l18 0 2024-03-29T04:45:07.326 INFO:tasks.workunit.client.0.smithi142.stdout:8/785: dread d5/d1d/d2f/d45/f9a [0,4194304] 0 2024-03-29T04:45:07.326 INFO:tasks.workunit.client.0.smithi142.stdout:1/654: dread - d8/db/dc/d77/db9/dbe/fc5 zero size 2024-03-29T04:45:07.326 INFO:tasks.workunit.client.1.smithi152.stdout:7/75: creat d7/f14 x:0 0 0 2024-03-29T04:45:07.326 INFO:tasks.workunit.client.1.smithi152.stdout:9/133: dread d5/f7 [0,4194304] 0 2024-03-29T04:45:07.327 INFO:tasks.workunit.client.0.smithi142.stdout:2/659: symlink de/d11/d17/d60/le3 0 2024-03-29T04:45:07.327 INFO:tasks.workunit.client.1.smithi152.stdout:9/134: write d5/f6 [1215699,112042] 0 2024-03-29T04:45:07.327 INFO:tasks.workunit.client.1.smithi152.stdout:4/31: mknod d0/d7/c8 0 2024-03-29T04:45:07.330 INFO:tasks.workunit.client.1.smithi152.stdout:8/59: truncate d1/d2/d3/f9 5297960 0 2024-03-29T04:45:07.331 INFO:tasks.workunit.client.1.smithi152.stdout:3/60: getdents d7 0 2024-03-29T04:45:07.331 INFO:tasks.workunit.client.1.smithi152.stdout:3/61: chown c4 11 1 2024-03-29T04:45:07.332 INFO:tasks.workunit.client.1.smithi152.stdout:4/32: symlink d0/l9 0 2024-03-29T04:45:07.332 INFO:tasks.workunit.client.1.smithi152.stdout:3/62: write f5 [9032768,109939] 0 2024-03-29T04:45:07.332 INFO:tasks.workunit.client.1.smithi152.stdout:9/135: fsync d5/d12/f19 0 2024-03-29T04:45:07.333 INFO:tasks.workunit.client.1.smithi152.stdout:9/136: readlink d5/l16 0 2024-03-29T04:45:07.334 INFO:tasks.workunit.client.1.smithi152.stdout:1/85: mknod d5/d7/d8/d13/c23 0 2024-03-29T04:45:07.335 INFO:tasks.workunit.client.1.smithi152.stdout:7/76: mkdir d7/d8/d12/d15 0 2024-03-29T04:45:07.339 INFO:tasks.workunit.client.1.smithi152.stdout:4/33: dread d0/f1 [0,4194304] 0 2024-03-29T04:45:07.339 INFO:tasks.workunit.client.1.smithi152.stdout:6/121: write d1/d3/f18 [349951,399] 0 2024-03-29T04:45:07.342 INFO:tasks.workunit.client.1.smithi152.stdout:3/63: dwrite f5 [4194304,4194304] 0 2024-03-29T04:45:07.346 INFO:tasks.workunit.client.1.smithi152.stdout:6/122: dread d1/d3/f18 [0,4194304] 0 2024-03-29T04:45:07.347 INFO:tasks.workunit.client.1.smithi152.stdout:9/137: rename c4 to d5/d12/c2e 0 2024-03-29T04:45:07.347 INFO:tasks.workunit.client.1.smithi152.stdout:9/138: chown d5 35620054 1 2024-03-29T04:45:07.350 INFO:tasks.workunit.client.1.smithi152.stdout:9/139: truncate d5/f20 378966 0 2024-03-29T04:45:07.354 INFO:tasks.workunit.client.1.smithi152.stdout:3/64: dwrite f5 [0,4194304] 0 2024-03-29T04:45:07.359 INFO:tasks.workunit.client.1.smithi152.stdout:6/123: dwrite d1/fe [0,4194304] 0 2024-03-29T04:45:07.360 INFO:tasks.workunit.client.1.smithi152.stdout:7/77: dwrite d7/f9 [0,4194304] 0 2024-03-29T04:45:07.362 INFO:tasks.workunit.client.1.smithi152.stdout:8/60: dread d1/d2/fc [4194304,4194304] 0 2024-03-29T04:45:07.362 INFO:tasks.workunit.client.1.smithi152.stdout:3/65: rmdir d7 39 2024-03-29T04:45:07.363 INFO:tasks.workunit.client.1.smithi152.stdout:0/108: sync 2024-03-29T04:45:07.363 INFO:tasks.workunit.client.1.smithi152.stdout:2/73: sync 2024-03-29T04:45:07.364 INFO:tasks.workunit.client.1.smithi152.stdout:5/59: sync 2024-03-29T04:45:07.366 INFO:tasks.workunit.client.1.smithi152.stdout:1/86: getdents d5/d7 0 2024-03-29T04:45:07.366 INFO:tasks.workunit.client.1.smithi152.stdout:6/124: unlink d1/d3/d9/fb 0 2024-03-29T04:45:07.368 INFO:tasks.workunit.client.1.smithi152.stdout:1/87: dread - d5/d7/f1f zero size 2024-03-29T04:45:07.370 INFO:tasks.workunit.client.1.smithi152.stdout:6/125: write d1/d3/d9/d13/f1f [2670280,92191] 0 2024-03-29T04:45:07.371 INFO:tasks.workunit.client.1.smithi152.stdout:8/61: mknod d1/d2/c12 0 2024-03-29T04:45:07.375 INFO:tasks.workunit.client.1.smithi152.stdout:9/140: creat d5/f2f x:0 0 0 2024-03-29T04:45:07.376 INFO:tasks.workunit.client.1.smithi152.stdout:3/66: creat d7/fb x:0 0 0 2024-03-29T04:45:07.376 INFO:tasks.workunit.client.1.smithi152.stdout:6/126: chown d1/d3/d9/d13/d28 10833 1 2024-03-29T04:45:07.378 INFO:tasks.workunit.client.1.smithi152.stdout:8/62: dread d1/d2/fc [4194304,4194304] 0 2024-03-29T04:45:07.382 INFO:tasks.workunit.client.1.smithi152.stdout:9/141: fdatasync d5/d12/f19 0 2024-03-29T04:45:07.382 INFO:tasks.workunit.client.1.smithi152.stdout:8/63: dread d1/d2/d3/f9 [4194304,4194304] 0 2024-03-29T04:45:07.382 INFO:tasks.workunit.client.1.smithi152.stdout:3/67: mknod d7/cc 0 2024-03-29T04:45:07.382 INFO:tasks.workunit.client.1.smithi152.stdout:6/127: creat d1/d3/df/f2e x:0 0 0 2024-03-29T04:45:07.383 INFO:tasks.workunit.client.1.smithi152.stdout:4/34: truncate d0/f4 1228694 0 2024-03-29T04:45:07.384 INFO:tasks.workunit.client.0.smithi142.stdout:4/694: symlink d0/d9/d19/d2c/led 0 2024-03-29T04:45:07.384 INFO:tasks.workunit.client.1.smithi152.stdout:4/35: chown d0/d7 1861761 1 2024-03-29T04:45:07.384 INFO:tasks.workunit.client.1.smithi152.stdout:9/142: symlink d5/d12/l30 0 2024-03-29T04:45:07.385 INFO:tasks.workunit.client.0.smithi142.stdout:0/698: rename d7/d13/dbf/d72/d37/da4/lb6 to d7/d9/le4 0 2024-03-29T04:45:07.385 INFO:tasks.workunit.client.1.smithi152.stdout:0/109: link d2/da/df/c26 d2/c27 0 2024-03-29T04:45:07.385 INFO:tasks.workunit.client.0.smithi142.stdout:1/655: creat d8/db/d37/fe2 x:0 0 0 2024-03-29T04:45:07.386 INFO:tasks.workunit.client.1.smithi152.stdout:9/143: stat d5/db/c18 0 2024-03-29T04:45:07.387 INFO:tasks.workunit.client.0.smithi142.stdout:8/786: creat d5/d1d/d2f/d76/d3c/f112 x:0 0 0 2024-03-29T04:45:07.387 INFO:tasks.workunit.client.0.smithi142.stdout:3/550: symlink d3/d29/d8f/d95/lbe 0 2024-03-29T04:45:07.387 INFO:tasks.workunit.client.0.smithi142.stdout:2/660: dread - de/d11/d1e/d22/d27/dba/f6b zero size 2024-03-29T04:45:07.388 INFO:tasks.workunit.client.1.smithi152.stdout:3/68: creat d7/fd x:0 0 0 2024-03-29T04:45:07.388 INFO:tasks.workunit.client.1.smithi152.stdout:7/78: write d7/d8/fa [2864678,68649] 0 2024-03-29T04:45:07.388 INFO:tasks.workunit.client.0.smithi142.stdout:9/557: getdents d0/d6c 0 2024-03-29T04:45:07.389 INFO:tasks.workunit.client.0.smithi142.stdout:4/695: chown d0/d9/d11/d4c/d5f/d81/dc9/d47/f62 31 1 2024-03-29T04:45:07.391 INFO:tasks.workunit.client.0.smithi142.stdout:9/558: stat d0/d2/d6/f30 0 2024-03-29T04:45:07.391 INFO:tasks.workunit.client.0.smithi142.stdout:3/551: write d3/d29/d2b/f21 [1912887,110305] 0 2024-03-29T04:45:07.392 INFO:tasks.workunit.client.1.smithi152.stdout:6/128: creat d1/d3/d1a/f2f x:0 0 0 2024-03-29T04:45:07.392 INFO:tasks.workunit.client.1.smithi152.stdout:4/36: mknod d0/ca 0 2024-03-29T04:45:07.393 INFO:tasks.workunit.client.0.smithi142.stdout:9/559: chown d0/d12/f86 1340030 1 2024-03-29T04:45:07.395 INFO:tasks.workunit.client.1.smithi152.stdout:0/110: dread d2/f9 [0,4194304] 0 2024-03-29T04:45:07.395 INFO:tasks.workunit.client.0.smithi142.stdout:0/699: creat d7/d13/dbf/dca/dab/dc1/dd3/d90/fe5 x:0 0 0 2024-03-29T04:45:07.396 INFO:tasks.workunit.client.1.smithi152.stdout:7/79: mknod d7/d8/db/df/c16 0 2024-03-29T04:45:07.396 INFO:tasks.workunit.client.1.smithi152.stdout:5/60: truncate d6/d7/fa 1825207 0 2024-03-29T04:45:07.396 INFO:tasks.workunit.client.1.smithi152.stdout:7/80: chown d7/d8 419 1 2024-03-29T04:45:07.397 INFO:tasks.workunit.client.1.smithi152.stdout:1/88: truncate d5/f20 1918992 0 2024-03-29T04:45:07.398 INFO:tasks.workunit.client.0.smithi142.stdout:5/859: rename df/d16/d24/l104 to df/d16/d24/d2e/d47/d8c/l11d 0 2024-03-29T04:45:07.398 INFO:tasks.workunit.client.1.smithi152.stdout:7/81: stat d7/d8/db/fe 0 2024-03-29T04:45:07.399 INFO:tasks.workunit.client.1.smithi152.stdout:9/144: dread d5/db/f13 [0,4194304] 0 2024-03-29T04:45:07.399 INFO:tasks.workunit.client.1.smithi152.stdout:6/129: mknod d1/d3/d9/d13/d28/c30 0 2024-03-29T04:45:07.400 INFO:tasks.workunit.client.1.smithi152.stdout:7/82: readlink d7/d8/lc 0 2024-03-29T04:45:07.401 INFO:tasks.workunit.client.1.smithi152.stdout:3/69: link d7/c9 d7/ce 0 2024-03-29T04:45:07.401 INFO:tasks.workunit.client.0.smithi142.stdout:2/661: symlink de/d11/d1e/d22/d27/dba/le4 0 2024-03-29T04:45:07.401 INFO:tasks.workunit.client.1.smithi152.stdout:5/61: creat d6/dd/f11 x:0 0 0 2024-03-29T04:45:07.404 INFO:tasks.workunit.client.1.smithi152.stdout:1/89: chown c1 4837 1 2024-03-29T04:45:07.406 INFO:tasks.workunit.client.0.smithi142.stdout:4/696: truncate d0/f14 5089048 0 2024-03-29T04:45:07.407 INFO:tasks.workunit.client.0.smithi142.stdout:8/787: dread d5/d1d/d2f/d76/d2d/f3b [0,4194304] 0 2024-03-29T04:45:07.409 INFO:tasks.workunit.client.0.smithi142.stdout:9/560: symlink d0/d12/d50/d8e/lb3 0 2024-03-29T04:45:07.410 INFO:tasks.workunit.client.1.smithi152.stdout:1/90: chown d5/d7/f1f 152503 1 2024-03-29T04:45:07.412 INFO:tasks.workunit.client.1.smithi152.stdout:7/83: dwrite d7/f14 [0,4194304] 0 2024-03-29T04:45:07.413 INFO:tasks.workunit.client.0.smithi142.stdout:0/700: fsync d7/d13/f78 0 2024-03-29T04:45:07.416 INFO:tasks.workunit.client.0.smithi142.stdout:5/860: creat df/d2c/d112/f11e x:0 0 0 2024-03-29T04:45:07.416 INFO:tasks.workunit.client.1.smithi152.stdout:5/62: link d6/d7/lc d6/d7/l12 0 2024-03-29T04:45:07.418 INFO:tasks.workunit.client.1.smithi152.stdout:5/63: dread - d6/dd/f11 zero size 2024-03-29T04:45:07.419 INFO:tasks.workunit.client.0.smithi142.stdout:2/662: creat de/d11/d17/d5d/fe5 x:0 0 0 2024-03-29T04:45:07.420 INFO:tasks.workunit.client.1.smithi152.stdout:0/111: rename d2/da/df/c26 to d2/da/c28 0 2024-03-29T04:45:07.421 INFO:tasks.workunit.client.1.smithi152.stdout:5/64: read - d6/dd/f11 zero size 2024-03-29T04:45:07.423 INFO:tasks.workunit.client.0.smithi142.stdout:8/788: readlink d5/d1d/d2f/l31 0 2024-03-29T04:45:07.424 INFO:tasks.workunit.client.1.smithi152.stdout:0/112: fsync d2/da/f24 0 2024-03-29T04:45:07.424 INFO:tasks.workunit.client.1.smithi152.stdout:8/64: dwrite d1/d2/d3/f9 [0,4194304] 0 2024-03-29T04:45:07.425 INFO:tasks.workunit.client.1.smithi152.stdout:3/70: rename f5 to d7/ff 0 2024-03-29T04:45:07.425 INFO:tasks.workunit.client.0.smithi142.stdout:8/789: write d5/d1d/d2f/d76/d2d/f56 [2229861,121921] 0 2024-03-29T04:45:07.426 INFO:tasks.workunit.client.0.smithi142.stdout:9/561: unlink d0/f22 0 2024-03-29T04:45:07.428 INFO:tasks.workunit.client.0.smithi142.stdout:9/562: dread - d0/d2/d6/d27/d3c/d4f/f8c zero size 2024-03-29T04:45:07.428 INFO:tasks.workunit.client.1.smithi152.stdout:5/65: chown d6/d7/f8 0 1 2024-03-29T04:45:07.431 INFO:tasks.workunit.client.1.smithi152.stdout:6/130: truncate d1/d3/d9/f27 274046 0 2024-03-29T04:45:07.434 INFO:tasks.workunit.client.0.smithi142.stdout:5/861: mkdir df/d1a/d21/d27/d45/dbf/d11f 0 2024-03-29T04:45:07.434 INFO:tasks.workunit.client.1.smithi152.stdout:8/65: symlink d1/l13 0 2024-03-29T04:45:07.434 INFO:tasks.workunit.client.1.smithi152.stdout:0/113: mkdir d2/d29 0 2024-03-29T04:45:07.435 INFO:tasks.workunit.client.1.smithi152.stdout:7/84: dwrite d7/d8/db/df/f13 [0,4194304] 0 2024-03-29T04:45:07.436 INFO:tasks.workunit.client.1.smithi152.stdout:3/71: creat d7/f10 x:0 0 0 2024-03-29T04:45:07.436 INFO:tasks.workunit.client.0.smithi142.stdout:2/663: unlink de/d11/d2b/l94 0 2024-03-29T04:45:07.438 INFO:tasks.workunit.client.1.smithi152.stdout:9/145: dwrite d5/db/f13 [4194304,4194304] 0 2024-03-29T04:45:07.441 INFO:tasks.workunit.client.1.smithi152.stdout:7/85: write d7/f10 [160848,18169] 0 2024-03-29T04:45:07.441 INFO:tasks.workunit.client.1.smithi152.stdout:6/131: creat d1/d3/d9/d13/f31 x:0 0 0 2024-03-29T04:45:07.442 INFO:tasks.workunit.client.1.smithi152.stdout:0/114: write d2/f20 [1581712,119239] 0 2024-03-29T04:45:07.444 INFO:tasks.workunit.client.1.smithi152.stdout:5/66: symlink d6/d7/l13 0 2024-03-29T04:45:07.444 INFO:tasks.workunit.client.1.smithi152.stdout:8/66: write d1/d2/fc [2806124,60489] 0 2024-03-29T04:45:07.445 INFO:tasks.workunit.client.1.smithi152.stdout:0/115: write d2/da/fb [666696,26580] 0 2024-03-29T04:45:07.445 INFO:tasks.workunit.client.1.smithi152.stdout:1/91: link d5/l1d d5/d7/d8/d13/d19/d21/l24 0 2024-03-29T04:45:07.447 INFO:tasks.workunit.client.1.smithi152.stdout:6/132: symlink d1/l32 0 2024-03-29T04:45:07.449 INFO:tasks.workunit.client.0.smithi142.stdout:8/790: creat d5/d2e/f113 x:0 0 0 2024-03-29T04:45:07.449 INFO:tasks.workunit.client.1.smithi152.stdout:7/86: creat d7/d8/db/df/f17 x:0 0 0 2024-03-29T04:45:07.450 INFO:tasks.workunit.client.0.smithi142.stdout:3/552: link d3/d29/d8f/l9e d3/d29/d2b/d40/d67/lbf 0 2024-03-29T04:45:07.451 INFO:tasks.workunit.client.1.smithi152.stdout:6/133: chown d1/d3/d9/d13/l2a 0 1 2024-03-29T04:45:07.451 INFO:tasks.workunit.client.1.smithi152.stdout:8/67: chown d1/d2/d3/le 283785 1 2024-03-29T04:45:07.451 INFO:tasks.workunit.client.0.smithi142.stdout:1/656: dwrite d8/db/d5e/f73 [0,4194304] 0 2024-03-29T04:45:07.452 INFO:tasks.workunit.client.0.smithi142.stdout:3/553: dread - d3/d29/d8f/d5f/d64/fb8 zero size 2024-03-29T04:45:07.453 INFO:tasks.workunit.client.1.smithi152.stdout:4/37: truncate d0/f4 2072959 0 2024-03-29T04:45:07.453 INFO:tasks.workunit.client.1.smithi152.stdout:3/72: dread d7/ff [8388608,4194304] 0 2024-03-29T04:45:07.455 INFO:tasks.workunit.client.1.smithi152.stdout:0/116: dread d2/da/fb [0,4194304] 0 2024-03-29T04:45:07.455 INFO:tasks.workunit.client.0.smithi142.stdout:6/662: sync 2024-03-29T04:45:07.456 INFO:tasks.workunit.client.1.smithi152.stdout:9/146: mkdir d5/d8/d31 0 2024-03-29T04:45:07.456 INFO:tasks.workunit.client.1.smithi152.stdout:0/117: write d2/da/fb [525121,11455] 0 2024-03-29T04:45:07.458 INFO:tasks.workunit.client.0.smithi142.stdout:3/554: read d3/d29/d2d/f8a [178622,117705] 0 2024-03-29T04:45:07.458 INFO:tasks.workunit.client.1.smithi152.stdout:7/87: mkdir d7/d8/d18 0 2024-03-29T04:45:07.461 INFO:tasks.workunit.client.0.smithi142.stdout:4/697: dwrite d0/dc/d15/d3d/d65/f7c [0,4194304] 0 2024-03-29T04:45:07.462 INFO:tasks.workunit.client.1.smithi152.stdout:8/68: symlink d1/d2/d3/l14 0 2024-03-29T04:45:07.463 INFO:tasks.workunit.client.1.smithi152.stdout:3/73: mknod d7/c11 0 2024-03-29T04:45:07.464 INFO:tasks.workunit.client.0.smithi142.stdout:9/563: link d0/d2/d6/d27/f92 d0/d12/d50/d52/fb4 0 2024-03-29T04:45:07.465 INFO:tasks.workunit.client.1.smithi152.stdout:4/38: unlink d0/f1 0 2024-03-29T04:45:07.465 INFO:tasks.workunit.client.0.smithi142.stdout:6/663: dwrite f4 [4194304,4194304] 0 2024-03-29T04:45:07.466 INFO:tasks.workunit.client.0.smithi142.stdout:1/657: rename d8/db/d37/f79 to d8/db/d37/fe3 0 2024-03-29T04:45:07.469 INFO:tasks.workunit.client.0.smithi142.stdout:3/555: mkdir d3/d29/d2b/dc0 0 2024-03-29T04:45:07.469 INFO:tasks.workunit.client.0.smithi142.stdout:9/564: write d0/d2/d6/d31/f3a [2643253,125364] 0 2024-03-29T04:45:07.469 INFO:tasks.workunit.client.1.smithi152.stdout:9/147: dwrite f3 [0,4194304] 0 2024-03-29T04:45:07.472 INFO:tasks.workunit.client.1.smithi152.stdout:0/118: symlink d2/da/df/l2a 0 2024-03-29T04:45:07.476 INFO:tasks.workunit.client.0.smithi142.stdout:1/658: truncate d8/db/dc/d77/db9/dbe/d2d/faa 887347 0 2024-03-29T04:45:07.476 INFO:tasks.workunit.client.1.smithi152.stdout:8/69: mkdir d1/d2/d15 0 2024-03-29T04:45:07.478 INFO:tasks.workunit.client.1.smithi152.stdout:3/74: symlink d7/l12 0 2024-03-29T04:45:07.479 INFO:tasks.workunit.client.0.smithi142.stdout:2/664: dwrite de/d11/d1e/d22/d27/dba/d61/d7d/f7e [0,4194304] 0 2024-03-29T04:45:07.479 INFO:tasks.workunit.client.1.smithi152.stdout:0/119: creat d2/da/d17/f2b x:0 0 0 2024-03-29T04:45:07.480 INFO:tasks.workunit.client.0.smithi142.stdout:6/664: rename d1c/d33/d46/d66/fbf to d1c/d90/fd5 0 2024-03-29T04:45:07.482 INFO:tasks.workunit.client.1.smithi152.stdout:5/67: link d6/dd/f11 d6/f14 0 2024-03-29T04:45:07.482 INFO:tasks.workunit.client.1.smithi152.stdout:0/120: write d2/da/d17/f2b [356715,67604] 0 2024-03-29T04:45:07.484 INFO:tasks.workunit.client.0.smithi142.stdout:5/862: getdents df/d29/d7a/d97 0 2024-03-29T04:45:07.485 INFO:tasks.workunit.client.1.smithi152.stdout:5/68: write f5 [5014719,18191] 0 2024-03-29T04:45:07.485 INFO:tasks.workunit.client.1.smithi152.stdout:9/148: rename f3 to d5/db/d1b/f32 0 2024-03-29T04:45:07.485 INFO:tasks.workunit.client.0.smithi142.stdout:3/556: dread d3/d29/d2b/d40/d67/d8b/d6/f20 [0,4194304] 0 2024-03-29T04:45:07.487 INFO:tasks.workunit.client.1.smithi152.stdout:8/70: dwrite d1/d2/d3/f9 [0,4194304] 0 2024-03-29T04:45:07.490 INFO:tasks.workunit.client.1.smithi152.stdout:3/75: symlink d7/l13 0 2024-03-29T04:45:07.491 INFO:tasks.workunit.client.0.smithi142.stdout:0/701: truncate d7/d13/dbf/f92 2098935 0 2024-03-29T04:45:07.491 INFO:tasks.workunit.client.1.smithi152.stdout:3/76: dread - d7/fd zero size 2024-03-29T04:45:07.492 INFO:tasks.workunit.client.0.smithi142.stdout:1/659: dread - d8/db/dc/d77/db9/dbe/d2d/d44/d53/f69 zero size 2024-03-29T04:45:07.496 INFO:tasks.workunit.client.0.smithi142.stdout:8/791: write d5/d1d/d2f/d76/f35 [611130,117845] 0 2024-03-29T04:45:07.496 INFO:tasks.workunit.client.1.smithi152.stdout:9/149: symlink d5/db/l33 0 2024-03-29T04:45:07.496 INFO:tasks.workunit.client.1.smithi152.stdout:9/150: stat d5/d8/f23 0 2024-03-29T04:45:07.497 INFO:tasks.workunit.client.0.smithi142.stdout:0/702: chown d7/d9/d55/f87 653485754 1 2024-03-29T04:45:07.497 INFO:tasks.workunit.client.0.smithi142.stdout:2/665: getdents de/d11/d1e/d84 0 2024-03-29T04:45:07.497 INFO:tasks.workunit.client.1.smithi152.stdout:5/69: rename d6/dd/d10 to d6/dd/d15 0 2024-03-29T04:45:07.498 INFO:tasks.workunit.client.1.smithi152.stdout:8/71: readlink d1/d2/lb 0 2024-03-29T04:45:07.501 INFO:tasks.workunit.client.0.smithi142.stdout:0/703: mknod d7/d9/d1e/d81/ce6 0 2024-03-29T04:45:07.502 INFO:tasks.workunit.client.0.smithi142.stdout:5/863: dwrite df/d1a/d21/d27/d45/dbf/f61 [0,4194304] 0 2024-03-29T04:45:07.504 INFO:tasks.workunit.client.0.smithi142.stdout:1/660: truncate d8/db/dc/d77/d31/d4d/f7a 861198 0 2024-03-29T04:45:07.506 INFO:tasks.workunit.client.1.smithi152.stdout:1/92: write d5/f20 [1750669,41840] 0 2024-03-29T04:45:07.507 INFO:tasks.workunit.client.0.smithi142.stdout:6/665: dread d1c/d33/db6/d52/f95 [0,4194304] 0 2024-03-29T04:45:07.507 INFO:tasks.workunit.client.1.smithi152.stdout:3/77: dwrite d7/ff [4194304,4194304] 0 2024-03-29T04:45:07.508 INFO:tasks.workunit.client.1.smithi152.stdout:8/72: unlink d1/d2/lb 0 2024-03-29T04:45:07.509 INFO:tasks.workunit.client.1.smithi152.stdout:8/73: chown d1/f10 415340 1 2024-03-29T04:45:07.510 INFO:tasks.workunit.client.0.smithi142.stdout:0/704: read d7/d9/d55/d42/f46 [5846885,91855] 0 2024-03-29T04:45:07.511 INFO:tasks.workunit.client.1.smithi152.stdout:5/70: mkdir d6/d16 0 2024-03-29T04:45:07.512 INFO:tasks.workunit.client.1.smithi152.stdout:9/151: rename d5/db/c2d to d5/db/c34 0 2024-03-29T04:45:07.516 INFO:tasks.workunit.client.0.smithi142.stdout:8/792: dread d5/d2e/dd2/d60/fd6 [0,4194304] 0 2024-03-29T04:45:07.516 INFO:tasks.workunit.client.0.smithi142.stdout:2/666: dwrite de/d11/d1e/fb9 [0,4194304] 0 2024-03-29T04:45:07.517 INFO:tasks.workunit.client.1.smithi152.stdout:1/93: stat d5/c17 0 2024-03-29T04:45:07.517 INFO:tasks.workunit.client.1.smithi152.stdout:3/78: rename d7/ca to d7/c14 0 2024-03-29T04:45:07.517 INFO:tasks.workunit.client.1.smithi152.stdout:8/74: dread d1/d2/fc [0,4194304] 0 2024-03-29T04:45:07.518 INFO:tasks.workunit.client.0.smithi142.stdout:9/565: write d0/d2/d6/f30 [1166422,44792] 0 2024-03-29T04:45:07.518 INFO:tasks.workunit.client.1.smithi152.stdout:8/75: stat d1/ld 0 2024-03-29T04:45:07.518 INFO:tasks.workunit.client.1.smithi152.stdout:5/71: dread f0 [0,4194304] 0 2024-03-29T04:45:07.521 INFO:tasks.workunit.client.0.smithi142.stdout:8/793: symlink d5/d1d/d2f/d76/d2d/de2/l114 0 2024-03-29T04:45:07.523 INFO:tasks.workunit.client.0.smithi142.stdout:6/666: getdents d1c/d33/db6 0 2024-03-29T04:45:07.525 INFO:tasks.workunit.client.0.smithi142.stdout:9/566: rename d0 to d0/d2/d6/d16/daa/db5 22 2024-03-29T04:45:07.527 INFO:tasks.workunit.client.0.smithi142.stdout:2/667: mkdir de/d11/d17/d60/de6 0 2024-03-29T04:45:07.527 INFO:tasks.workunit.client.0.smithi142.stdout:0/705: truncate d7/d9/d1e/f71 228117 0 2024-03-29T04:45:07.527 INFO:tasks.workunit.client.0.smithi142.stdout:6/667: stat d1c/d33/db6 0 2024-03-29T04:45:07.528 INFO:tasks.workunit.client.0.smithi142.stdout:5/864: truncate df/d16/d24/d2e/d47/d4d/fcc 1891129 0 2024-03-29T04:45:07.529 INFO:tasks.workunit.client.0.smithi142.stdout:8/794: creat d5/d2e/dd2/df6/f115 x:0 0 0 2024-03-29T04:45:07.529 INFO:tasks.workunit.client.0.smithi142.stdout:3/557: dwrite d3/d29/d2b/d40/d67/d8b/d4d/f9d [4194304,4194304] 0 2024-03-29T04:45:07.530 INFO:tasks.workunit.client.0.smithi142.stdout:1/661: getdents d8/db/dc/d77/db9/dbe/d2d/d44/d53 0 2024-03-29T04:45:07.531 INFO:tasks.workunit.client.1.smithi152.stdout:8/76: rename d1/l13 to d1/d2/d3/l16 0 2024-03-29T04:45:07.533 INFO:tasks.workunit.client.0.smithi142.stdout:6/668: creat d1c/d33/d46/d66/dc1/fd6 x:0 0 0 2024-03-29T04:45:07.534 INFO:tasks.workunit.client.0.smithi142.stdout:2/668: read de/d11/d1e/d22/d50/f57 [1057310,100510] 0 2024-03-29T04:45:07.534 INFO:tasks.workunit.client.0.smithi142.stdout:8/795: creat d5/d2e/dd2/d60/f116 x:0 0 0 2024-03-29T04:45:07.536 INFO:tasks.workunit.client.0.smithi142.stdout:3/558: symlink d3/db9/lc1 0 2024-03-29T04:45:07.537 INFO:tasks.workunit.client.0.smithi142.stdout:5/865: rmdir df/d16/d24/d2e/d47/d4d/d102/d75 39 2024-03-29T04:45:07.537 INFO:tasks.workunit.client.1.smithi152.stdout:7/88: truncate d7/d8/db/df/f13 889567 0 2024-03-29T04:45:07.539 INFO:tasks.workunit.client.0.smithi142.stdout:8/796: dread - d5/d1d/d2f/d5f/fa1 zero size 2024-03-29T04:45:07.540 INFO:tasks.workunit.client.0.smithi142.stdout:3/559: write d3/d1f/d24/fa5 [339200,112873] 0 2024-03-29T04:45:07.542 INFO:tasks.workunit.client.1.smithi152.stdout:9/152: creat d5/d12/d2c/f35 x:0 0 0 2024-03-29T04:45:07.542 INFO:tasks.workunit.client.1.smithi152.stdout:1/94: unlink d5/d7/d8/d13/d19/d21/l24 0 2024-03-29T04:45:07.544 INFO:tasks.workunit.client.0.smithi142.stdout:0/706: dwrite d7/d13/dbf/dca/dab/dc1/dd3/d90/fe5 [0,4194304] 0 2024-03-29T04:45:07.545 INFO:tasks.workunit.client.1.smithi152.stdout:8/77: dwrite d1/f10 [0,4194304] 0 2024-03-29T04:45:07.550 INFO:tasks.workunit.client.1.smithi152.stdout:9/153: mknod d5/db/c36 0 2024-03-29T04:45:07.551 INFO:tasks.workunit.client.0.smithi142.stdout:1/662: rename d8/db/dc/d77/d31/d4d/lcd to d8/db/dc/d77/db9/le4 0 2024-03-29T04:45:07.553 INFO:tasks.workunit.client.1.smithi152.stdout:1/95: creat d5/d7/f25 x:0 0 0 2024-03-29T04:45:07.555 INFO:tasks.workunit.client.0.smithi142.stdout:3/560: dread - d3/d29/d8f/d5f/db4/f78 zero size 2024-03-29T04:45:07.557 INFO:tasks.workunit.client.0.smithi142.stdout:3/561: stat d3/d29/d2b/d40/d67/d8b/c36 0 2024-03-29T04:45:07.559 INFO:tasks.workunit.client.1.smithi152.stdout:9/154: stat d5/db/l33 0 2024-03-29T04:45:07.559 INFO:tasks.workunit.client.0.smithi142.stdout:2/669: rename de/d11/d1e/d22/d27/d3a/cb8 to de/d11/da1/ce7 0 2024-03-29T04:45:07.559 INFO:tasks.workunit.client.1.smithi152.stdout:8/78: fdatasync d1/d2/d3/f9 0 2024-03-29T04:45:07.561 INFO:tasks.workunit.client.1.smithi152.stdout:4/39: dwrite d0/f4 [0,4194304] 0 2024-03-29T04:45:07.561 INFO:tasks.workunit.client.1.smithi152.stdout:1/96: chown d5/c17 462996779 1 2024-03-29T04:45:07.563 INFO:tasks.workunit.client.1.smithi152.stdout:0/121: dwrite d2/f7 [0,4194304] 0 2024-03-29T04:45:07.564 INFO:tasks.workunit.client.1.smithi152.stdout:1/97: stat d5/d7/d9/f10 0 2024-03-29T04:45:07.564 INFO:tasks.workunit.client.1.smithi152.stdout:7/89: dwrite d7/d8/db/df/f17 [0,4194304] 0 2024-03-29T04:45:07.568 INFO:tasks.workunit.client.1.smithi152.stdout:1/98: write d5/d7/f1f [156111,126744] 0 2024-03-29T04:45:07.569 INFO:tasks.workunit.client.0.smithi142.stdout:8/797: dread d5/d2e/dd2/d60/f8f [0,4194304] 0 2024-03-29T04:45:07.571 INFO:tasks.workunit.client.0.smithi142.stdout:1/663: rename d8/db/d37/f41 to d8/db/dc/d77/db9/dbe/d2d/d44/d7f/fe5 0 2024-03-29T04:45:07.572 INFO:tasks.workunit.client.1.smithi152.stdout:3/79: truncate d7/ff 5561112 0 2024-03-29T04:45:07.572 INFO:tasks.workunit.client.1.smithi152.stdout:9/155: chown d5/db/c34 0 1 2024-03-29T04:45:07.575 INFO:tasks.workunit.client.1.smithi152.stdout:3/80: readlink d7/l12 0 2024-03-29T04:45:07.576 INFO:tasks.workunit.client.0.smithi142.stdout:0/707: getdents d7 0 2024-03-29T04:45:07.579 INFO:tasks.workunit.client.0.smithi142.stdout:5/866: dread df/d16/d24/d2e/d47/d4d/d102/d75/d90/fc2 [0,4194304] 0 2024-03-29T04:45:07.579 INFO:tasks.workunit.client.1.smithi152.stdout:5/72: dwrite d6/dd/f11 [0,4194304] 0 2024-03-29T04:45:07.581 INFO:tasks.workunit.client.0.smithi142.stdout:1/664: getdents d8/db/dc/d19 0 2024-03-29T04:45:07.582 INFO:tasks.workunit.client.0.smithi142.stdout:7/768: sync 2024-03-29T04:45:07.582 INFO:tasks.workunit.client.0.smithi142.stdout:1/665: fsync d8/db/dc/d77/db9/dbe/d15/f60 0 2024-03-29T04:45:07.583 INFO:tasks.workunit.client.0.smithi142.stdout:5/867: mkdir df/d16/d25/d120 0 2024-03-29T04:45:07.584 INFO:tasks.workunit.client.1.smithi152.stdout:4/40: dwrite d0/f4 [0,4194304] 0 2024-03-29T04:45:07.585 INFO:tasks.workunit.client.1.smithi152.stdout:7/90: creat d7/d8/db/f19 x:0 0 0 2024-03-29T04:45:07.585 INFO:tasks.workunit.client.1.smithi152.stdout:0/122: dread d2/f8 [0,4194304] 0 2024-03-29T04:45:07.586 INFO:tasks.workunit.client.1.smithi152.stdout:0/123: readlink d2/da/df/l2a 0 2024-03-29T04:45:07.588 INFO:tasks.workunit.client.0.smithi142.stdout:1/666: rename d8/db/dc/d77/d31/f64 to d8/db/dc/d77/d5a/fe6 0 2024-03-29T04:45:07.588 INFO:tasks.workunit.client.1.smithi152.stdout:9/156: creat d5/db/d1b/f37 x:0 0 0 2024-03-29T04:45:07.592 INFO:tasks.workunit.client.0.smithi142.stdout:7/769: dwrite d7/d27/d44/db2/db8/de/d20/d86/f104 [0,4194304] 0 2024-03-29T04:45:07.593 INFO:tasks.workunit.client.0.smithi142.stdout:7/770: chown d7/d27/d44/db2/dc4/ff4 9 1 2024-03-29T04:45:07.594 INFO:tasks.workunit.client.0.smithi142.stdout:5/868: rename df/d16/d24/fb6 to df/d16/d24/f121 0 2024-03-29T04:45:07.598 INFO:tasks.workunit.client.1.smithi152.stdout:4/41: chown d0/l5 995 1 2024-03-29T04:45:07.599 INFO:tasks.workunit.client.1.smithi152.stdout:0/124: write d2/f8 [3713234,83485] 0 2024-03-29T04:45:07.601 INFO:tasks.workunit.client.1.smithi152.stdout:3/81: dwrite d7/fd [0,4194304] 0 2024-03-29T04:45:07.602 INFO:tasks.workunit.client.0.smithi142.stdout:1/667: link d8/db/dc/d77/db9/dbe/d2d/f3e d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/fe7 0 2024-03-29T04:45:07.602 INFO:tasks.workunit.client.1.smithi152.stdout:7/91: creat d7/d8/d18/f1a x:0 0 0 2024-03-29T04:45:07.605 INFO:tasks.workunit.client.1.smithi152.stdout:7/92: write d7/d8/db/f19 [127207,120312] 0 2024-03-29T04:45:07.606 INFO:tasks.workunit.client.0.smithi142.stdout:1/668: truncate d8/db/dc/d77/d31/d4d/f8f 4844768 0 2024-03-29T04:45:07.606 INFO:tasks.workunit.client.1.smithi152.stdout:9/157: dwrite d5/d8/f11 [0,4194304] 0 2024-03-29T04:45:07.608 INFO:tasks.workunit.client.0.smithi142.stdout:5/869: dread df/d16/d24/d2e/f8f [0,4194304] 0 2024-03-29T04:45:07.609 INFO:tasks.workunit.client.0.smithi142.stdout:0/708: dread d7/d9/d55/f98 [0,4194304] 0 2024-03-29T04:45:07.610 INFO:tasks.workunit.client.0.smithi142.stdout:7/771: getdents d7/d27/d44/db2 0 2024-03-29T04:45:07.612 INFO:tasks.workunit.client.1.smithi152.stdout:8/79: creat d1/d2/d3/f17 x:0 0 0 2024-03-29T04:45:07.613 INFO:tasks.workunit.client.0.smithi142.stdout:1/669: mknod d8/db/dc/d77/ce8 0 2024-03-29T04:45:07.614 INFO:tasks.workunit.client.1.smithi152.stdout:4/42: link d0/d7/c8 d0/cb 0 2024-03-29T04:45:07.614 INFO:tasks.workunit.client.0.smithi142.stdout:3/562: dread d3/d1f/d24/d47/d87/fa2 [0,4194304] 0 2024-03-29T04:45:07.615 INFO:tasks.workunit.client.1.smithi152.stdout:4/43: chown d0/l5 11479 1 2024-03-29T04:45:07.617 INFO:tasks.workunit.client.0.smithi142.stdout:5/870: creat df/d1a/d21/d55/d66/f122 x:0 0 0 2024-03-29T04:45:07.617 INFO:tasks.workunit.client.0.smithi142.stdout:7/772: chown d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0 1 1 2024-03-29T04:45:07.617 INFO:tasks.workunit.client.0.smithi142.stdout:3/563: stat d3/d29/d2b/d40/d67/d8b 0 2024-03-29T04:45:07.618 INFO:tasks.workunit.client.1.smithi152.stdout:9/158: dwrite d5/f29 [0,4194304] 0 2024-03-29T04:45:07.620 INFO:tasks.workunit.client.0.smithi142.stdout:3/564: readlink d3/d29/d8f/d5f/d64/l88 0 2024-03-29T04:45:07.620 INFO:tasks.workunit.client.0.smithi142.stdout:1/670: dread - d8/db/dc/d77/db9/dbe/d1f/d97/d9e/fa7 zero size 2024-03-29T04:45:07.621 INFO:tasks.workunit.client.0.smithi142.stdout:2/670: truncate de/d11/d17/d5d/f74 5187372 0 2024-03-29T04:45:07.622 INFO:tasks.workunit.client.1.smithi152.stdout:9/159: fdatasync d5/db/d1b/f37 0 2024-03-29T04:45:07.624 INFO:tasks.workunit.client.1.smithi152.stdout:8/80: rmdir d1/d2/d15 0 2024-03-29T04:45:07.627 INFO:tasks.workunit.client.0.smithi142.stdout:1/671: unlink d8/db/dc/d77/d31/f3f 0 2024-03-29T04:45:07.627 INFO:tasks.workunit.client.0.smithi142.stdout:5/871: dread df/d16/d24/d2e/d47/d4d/fe4 [0,4194304] 0 2024-03-29T04:45:07.629 INFO:tasks.workunit.client.1.smithi152.stdout:9/160: mknod d5/db/c38 0 2024-03-29T04:45:07.630 INFO:tasks.workunit.client.1.smithi152.stdout:5/73: write f0 [3226190,124329] 0 2024-03-29T04:45:07.631 INFO:tasks.workunit.client.1.smithi152.stdout:8/81: write d1/d2/fc [5197926,20661] 0 2024-03-29T04:45:07.633 INFO:tasks.workunit.client.0.smithi142.stdout:2/671: creat de/d11/d17/d5d/d71/fe8 x:0 0 0 2024-03-29T04:45:07.633 INFO:tasks.workunit.client.0.smithi142.stdout:8/798: dread d5/d1d/d2f/d45/fe4 [0,4194304] 0 2024-03-29T04:45:07.633 INFO:tasks.workunit.client.1.smithi152.stdout:4/44: dwrite d0/f4 [4194304,4194304] 0 2024-03-29T04:45:07.634 INFO:tasks.workunit.client.0.smithi142.stdout:7/773: dwrite d7/d27/d44/db2/db8/de/d20/d86/dd4/ddb/df6/f102 [0,4194304] 0 2024-03-29T04:45:07.635 INFO:tasks.workunit.client.1.smithi152.stdout:9/161: creat d5/d12/d2c/f39 x:0 0 0 2024-03-29T04:45:07.636 INFO:tasks.workunit.client.1.smithi152.stdout:4/45: symlink d0/lc 0 2024-03-29T04:45:07.636 INFO:tasks.workunit.client.1.smithi152.stdout:4/46: readlink d0/l9 0 2024-03-29T04:45:07.637 INFO:tasks.workunit.client.0.smithi142.stdout:2/672: chown de/d11/d1e/d22/faf 1336460 1 2024-03-29T04:45:07.638 INFO:tasks.workunit.client.0.smithi142.stdout:8/799: creat d5/d1d/dd9/f117 x:0 0 0 2024-03-29T04:45:07.642 INFO:tasks.workunit.client.1.smithi152.stdout:9/162: rmdir d5/d8 39 2024-03-29T04:45:07.644 INFO:tasks.workunit.client.1.smithi152.stdout:4/47: unlink d0/l5 0 2024-03-29T04:45:07.644 INFO:tasks.workunit.client.0.smithi142.stdout:2/673: unlink de/d11/d1e/d22/d50/cbd 0 2024-03-29T04:45:07.645 INFO:tasks.workunit.client.0.smithi142.stdout:8/800: symlink d5/d1d/d2f/d76/d3c/l118 0 2024-03-29T04:45:07.645 INFO:tasks.workunit.client.1.smithi152.stdout:3/82: fdatasync d7/fd 0 2024-03-29T04:45:07.647 INFO:tasks.workunit.client.0.smithi142.stdout:7/774: truncate d7/d27/d44/db2/db8/d4b/d71/d30/d72/fac 507583 0 2024-03-29T04:45:07.650 INFO:tasks.workunit.client.0.smithi142.stdout:8/801: mknod d5/d1d/d24/db4/c119 0 2024-03-29T04:45:07.651 INFO:tasks.workunit.client.1.smithi152.stdout:5/74: link d6/d7/fa d6/dd/f17 0 2024-03-29T04:45:07.651 INFO:tasks.workunit.client.1.smithi152.stdout:9/163: rename d5/db/c34 to d5/d12/d2c/c3a 0 2024-03-29T04:45:07.654 INFO:tasks.workunit.client.1.smithi152.stdout:8/82: link d1/d2/d3/l8 d1/d2/d3/l18 0 2024-03-29T04:45:07.654 INFO:tasks.workunit.client.1.smithi152.stdout:9/164: dread - d5/f2f zero size 2024-03-29T04:45:07.654 INFO:tasks.workunit.client.0.smithi142.stdout:5/872: dread df/d16/d25/f59 [0,4194304] 0 2024-03-29T04:45:07.654 INFO:tasks.workunit.client.0.smithi142.stdout:8/802: chown d5/d2e/f65 155 1 2024-03-29T04:45:07.654 INFO:tasks.workunit.client.0.smithi142.stdout:2/674: mknod de/d11/d23/dd1/ce9 0 2024-03-29T04:45:07.655 INFO:tasks.workunit.client.1.smithi152.stdout:8/83: write d1/d2/d3/f17 [955848,198] 0 2024-03-29T04:45:07.660 INFO:tasks.workunit.client.1.smithi152.stdout:4/48: dread d0/f4 [0,4194304] 0 2024-03-29T04:45:07.660 INFO:tasks.workunit.client.0.smithi142.stdout:3/565: dread d3/d29/d2b/d40/d67/d8b/f3e [0,4194304] 0 2024-03-29T04:45:07.662 INFO:tasks.workunit.client.0.smithi142.stdout:5/873: truncate df/d16/d24/d2e/d47/d4d/fcc 1091680 0 2024-03-29T04:45:07.662 INFO:tasks.workunit.client.1.smithi152.stdout:8/84: mknod d1/c19 0 2024-03-29T04:45:07.664 INFO:tasks.workunit.client.1.smithi152.stdout:0/125: dwrite d2/f9 [4194304,4194304] 0 2024-03-29T04:45:07.666 INFO:tasks.workunit.client.1.smithi152.stdout:9/165: link d5/f6 d5/d8/d31/f3b 0 2024-03-29T04:45:07.667 INFO:tasks.workunit.client.1.smithi152.stdout:4/49: fsync d0/f4 0 2024-03-29T04:45:07.667 INFO:tasks.workunit.client.1.smithi152.stdout:4/50: fsync d0/f4 0 2024-03-29T04:45:07.668 INFO:tasks.workunit.client.1.smithi152.stdout:0/126: symlink d2/da/d17/l2c 0 2024-03-29T04:45:07.668 INFO:tasks.workunit.client.0.smithi142.stdout:5/874: getdents df/d1a/d21 0 2024-03-29T04:45:07.669 INFO:tasks.workunit.client.0.smithi142.stdout:7/775: dread d7/d27/d44/db2/db8/de/f34 [0,4194304] 0 2024-03-29T04:45:07.670 INFO:tasks.workunit.client.0.smithi142.stdout:2/675: rmdir de/d11/d17/d5d 39 2024-03-29T04:45:07.670 INFO:tasks.workunit.client.1.smithi152.stdout:8/85: dread d1/d2/d3/f17 [0,4194304] 0 2024-03-29T04:45:07.670 INFO:tasks.workunit.client.1.smithi152.stdout:4/51: truncate d0/f4 8731056 0 2024-03-29T04:45:07.671 INFO:tasks.workunit.client.1.smithi152.stdout:4/52: chown d0/l9 53 1 2024-03-29T04:45:07.671 INFO:tasks.workunit.client.0.smithi142.stdout:5/875: write df/d1a/d21/d55/d66/fbe [1409018,2218] 0 2024-03-29T04:45:07.672 INFO:tasks.workunit.client.1.smithi152.stdout:0/127: symlink d2/da/d17/l2d 0 2024-03-29T04:45:07.675 INFO:tasks.workunit.client.0.smithi142.stdout:2/676: symlink de/d11/d2b/lea 0 2024-03-29T04:45:07.676 INFO:tasks.workunit.client.1.smithi152.stdout:9/166: mknod d5/d8/d31/c3c 0 2024-03-29T04:45:07.678 INFO:tasks.workunit.client.0.smithi142.stdout:5/876: dread - df/d16/d24/f121 zero size 2024-03-29T04:45:07.678 INFO:tasks.workunit.client.1.smithi152.stdout:8/86: creat d1/d2/f1a x:0 0 0 2024-03-29T04:45:07.678 INFO:tasks.workunit.client.0.smithi142.stdout:7/776: dwrite d7/d27/d44/db2/db8/de/f26 [0,4194304] 0 2024-03-29T04:45:07.681 INFO:tasks.workunit.client.1.smithi152.stdout:4/53: mknod d0/cd 0 2024-03-29T04:45:07.682 INFO:tasks.workunit.client.1.smithi152.stdout:4/54: write d0/f4 [2952080,83432] 0 2024-03-29T04:45:07.682 INFO:tasks.workunit.client.0.smithi142.stdout:8/803: truncate d5/d1d/d2f/d5f/d48/f61 824745 0 2024-03-29T04:45:07.683 INFO:tasks.workunit.client.1.smithi152.stdout:0/128: creat d2/f2e x:0 0 0 2024-03-29T04:45:07.684 INFO:tasks.workunit.client.0.smithi142.stdout:5/877: unlink df/d16/d24/d2e/d47/fd1 0 2024-03-29T04:45:07.686 INFO:tasks.workunit.client.1.smithi152.stdout:9/167: symlink d5/db/l3d 0 2024-03-29T04:45:07.687 INFO:tasks.workunit.client.1.smithi152.stdout:8/87: fsync d1/f10 0 2024-03-29T04:45:07.687 INFO:tasks.workunit.client.1.smithi152.stdout:5/75: write d6/dd/f17 [790245,82917] 0 2024-03-29T04:45:07.689 INFO:tasks.workunit.client.1.smithi152.stdout:5/76: write d6/dd/f11 [2609857,97228] 0 2024-03-29T04:45:07.689 INFO:tasks.workunit.client.0.smithi142.stdout:2/677: dwrite de/d11/d17/d60/fc6 [0,4194304] 0 2024-03-29T04:45:07.689 INFO:tasks.workunit.client.0.smithi142.stdout:8/804: fsync d5/d2e/dd2/d60/f8f 0 2024-03-29T04:45:07.690 INFO:tasks.workunit.client.1.smithi152.stdout:0/129: unlink d2/da/d17/l2d 0 2024-03-29T04:45:07.694 INFO:tasks.workunit.client.0.smithi142.stdout:2/678: stat de/d11/d1e/c51 0 2024-03-29T04:45:07.694 INFO:tasks.workunit.client.1.smithi152.stdout:8/88: mknod d1/d2/d3/c1b 0 2024-03-29T04:45:07.695 INFO:tasks.workunit.client.1.smithi152.stdout:9/168: creat d5/db/d1b/f3e x:0 0 0 2024-03-29T04:45:07.697 INFO:tasks.workunit.client.0.smithi142.stdout:2/679: chown de/d11/d17/l24 1262424 1 2024-03-29T04:45:07.699 INFO:tasks.workunit.client.1.smithi152.stdout:8/89: write d1/d2/fc [1183475,89157] 0 2024-03-29T04:45:07.699 INFO:tasks.workunit.client.1.smithi152.stdout:5/77: getdents d6/d16 0 2024-03-29T04:45:07.701 INFO:tasks.workunit.client.1.smithi152.stdout:9/169: creat d5/d12/d2c/f3f x:0 0 0 2024-03-29T04:45:07.703 INFO:tasks.workunit.client.0.smithi142.stdout:5/878: dwrite df/d16/d24/d2e/d47/d4d/fe4 [0,4194304] 0 2024-03-29T04:45:07.704 INFO:tasks.workunit.client.1.smithi152.stdout:0/130: dwrite d2/da/f24 [0,4194304] 0 2024-03-29T04:45:07.706 INFO:tasks.workunit.client.1.smithi152.stdout:9/170: creat d5/db/d1b/f40 x:0 0 0 2024-03-29T04:45:07.707 INFO:tasks.workunit.client.0.smithi142.stdout:5/879: mknod df/d1a/d28/c123 0 2024-03-29T04:45:07.709 INFO:tasks.workunit.client.0.smithi142.stdout:5/880: chown df/d16/l76 1731406614 1 2024-03-29T04:45:07.709 INFO:tasks.workunit.client.1.smithi152.stdout:4/55: getdents d0 0 2024-03-29T04:45:07.711 INFO:tasks.workunit.client.1.smithi152.stdout:8/90: dread d1/d2/d3/f9 [0,4194304] 0 2024-03-29T04:45:07.711 INFO:tasks.workunit.client.1.smithi152.stdout:9/171: mknod d5/d12/d2c/c41 0 2024-03-29T04:45:07.712 INFO:tasks.workunit.client.0.smithi142.stdout:2/680: rename de/d11/d1e/l9b to de/d11/d1e/d22/d50/leb 0 2024-03-29T04:45:07.713 INFO:tasks.workunit.client.1.smithi152.stdout:0/131: creat d2/d29/f2f x:0 0 0 2024-03-29T04:45:07.714 INFO:tasks.workunit.client.1.smithi152.stdout:9/172: chown d5/db/d1b/f3e 11722169 1 2024-03-29T04:45:07.715 INFO:tasks.workunit.client.0.smithi142.stdout:5/881: creat df/d16/d25/f124 x:0 0 0 2024-03-29T04:45:07.715 INFO:tasks.workunit.client.0.smithi142.stdout:8/805: write d5/d1d/d2f/d76/d2d/f57 [1501667,128955] 0 2024-03-29T04:45:07.719 INFO:tasks.workunit.client.1.smithi152.stdout:0/132: rmdir d2/da 39 2024-03-29T04:45:07.720 INFO:tasks.workunit.client.0.smithi142.stdout:7/777: dwrite d7/d27/d44/db2/db8/de/d20/f21 [0,4194304] 0 2024-03-29T04:45:07.721 INFO:tasks.workunit.client.1.smithi152.stdout:4/56: dwrite d0/f4 [4194304,4194304] 0 2024-03-29T04:45:07.721 INFO:tasks.workunit.client.1.smithi152.stdout:8/91: creat d1/f1c x:0 0 0 2024-03-29T04:45:07.724 INFO:tasks.workunit.client.0.smithi142.stdout:2/681: dwrite de/d11/d1e/d22/f8e [4194304,4194304] 0 2024-03-29T04:45:07.725 INFO:tasks.workunit.client.1.smithi152.stdout:9/173: dwrite d5/db/d1b/f37 [0,4194304] 0 2024-03-29T04:45:07.725 INFO:tasks.workunit.client.0.smithi142.stdout:5/882: dread fb [0,4194304] 0 2024-03-29T04:45:07.726 INFO:tasks.workunit.client.1.smithi152.stdout:9/174: fdatasync d5/d12/d2c/f39 0 2024-03-29T04:45:07.727 INFO:tasks.workunit.client.0.smithi142.stdout:7/778: mkdir d7/d27/d44/db2/db8/d4b/d71/d30/d72/d105 0 2024-03-29T04:45:07.728 INFO:tasks.workunit.client.1.smithi152.stdout:4/57: mkdir d0/d7/de 0 2024-03-29T04:45:07.728 INFO:tasks.workunit.client.0.smithi142.stdout:2/682: rmdir de/d11/d17/d60/dc4 39 2024-03-29T04:45:07.729 INFO:tasks.workunit.client.1.smithi152.stdout:0/133: rename d2/da/df/c16 to d2/d29/c30 0 2024-03-29T04:45:07.739 INFO:tasks.workunit.client.0.smithi142.stdout:7/779: getdents d7/d27/d44 0 2024-03-29T04:45:07.740 INFO:tasks.workunit.client.1.smithi152.stdout:8/92: getdents d1/d2/d3 0 2024-03-29T04:45:07.740 INFO:tasks.workunit.client.1.smithi152.stdout:0/134: dwrite d2/f7 [0,4194304] 0 2024-03-29T04:45:07.742 INFO:tasks.workunit.client.1.smithi152.stdout:9/175: getdents d5/d12/d2c 0 2024-03-29T04:45:07.743 INFO:tasks.workunit.client.1.smithi152.stdout:8/93: fdatasync d1/d2/d3/f17 0 2024-03-29T04:45:07.744 INFO:tasks.workunit.client.1.smithi152.stdout:9/176: truncate d5/db/f25 661590 0 2024-03-29T04:45:07.745 INFO:tasks.workunit.client.0.smithi142.stdout:7/780: mknod d7/c106 0 2024-03-29T04:45:07.746 INFO:tasks.workunit.client.1.smithi152.stdout:8/94: chown d1/ld 3 1 2024-03-29T04:45:07.748 INFO:tasks.workunit.client.1.smithi152.stdout:0/135: mkdir d2/d31 0 2024-03-29T04:45:07.748 INFO:tasks.workunit.client.1.smithi152.stdout:0/136: chown d2/f9 2818339 1 2024-03-29T04:45:07.749 INFO:tasks.workunit.client.1.smithi152.stdout:0/137: write d2/f8 [2214565,109789] 0 2024-03-29T04:45:07.750 INFO:tasks.workunit.client.1.smithi152.stdout:9/177: link d5/db/d1b/f2b d5/d8/f42 0 2024-03-29T04:45:07.751 INFO:tasks.workunit.client.0.smithi142.stdout:8/806: truncate d5/d2e/f65 765043 0 2024-03-29T04:45:07.752 INFO:tasks.workunit.client.1.smithi152.stdout:0/138: write d2/da/fb [1251514,80097] 0 2024-03-29T04:45:07.752 INFO:tasks.workunit.client.0.smithi142.stdout:7/781: creat d7/f107 x:0 0 0 2024-03-29T04:45:07.753 INFO:tasks.workunit.client.0.smithi142.stdout:7/782: chown d7/d27/d44/db2/db8/de/d20/c83 14 1 2024-03-29T04:45:07.760 INFO:tasks.workunit.client.0.smithi142.stdout:7/783: dwrite d7/d27/d44/db2/db8/de/d20/f98 [4194304,4194304] 0 2024-03-29T04:45:07.771 INFO:tasks.workunit.client.1.smithi152.stdout:9/178: write d5/db/d1b/f37 [4465423,111154] 0 2024-03-29T04:45:07.772 INFO:tasks.workunit.client.0.smithi142.stdout:4/698: sync 2024-03-29T04:45:07.773 INFO:tasks.workunit.client.1.smithi152.stdout:9/179: unlink d5/f6 0 2024-03-29T04:45:07.774 INFO:tasks.workunit.client.1.smithi152.stdout:9/180: chown d5/d12/d2c/f39 15 1 2024-03-29T04:45:07.778 INFO:tasks.workunit.client.1.smithi152.stdout:9/181: rmdir d5/db/d1b 39 2024-03-29T04:45:07.784 INFO:tasks.workunit.client.0.smithi142.stdout:8/807: dread d5/d1d/d59/f77 [0,4194304] 0 2024-03-29T04:45:07.789 INFO:tasks.workunit.client.0.smithi142.stdout:8/808: dwrite d5/d2e/ff4 [0,4194304] 0 2024-03-29T04:45:07.789 INFO:tasks.workunit.client.0.smithi142.stdout:8/809: readlink d5/d1d/d2f/d76/l1a 0 2024-03-29T04:45:07.809 INFO:tasks.workunit.client.0.smithi142.stdout:8/810: write d5/d1d/d2f/f90 [2695639,117444] 0 2024-03-29T04:45:07.812 INFO:tasks.workunit.client.1.smithi152.stdout:9/182: creat d5/db/d1b/f43 x:0 0 0 2024-03-29T04:45:07.814 INFO:tasks.workunit.client.0.smithi142.stdout:8/811: link d5/d1d/d2f/d76/d3c/d70/c102 d5/d2e/dd2/c11a 0 2024-03-29T04:45:07.815 INFO:tasks.workunit.client.0.smithi142.stdout:8/812: read d5/d1d/fd8 [47725,71218] 0 2024-03-29T04:45:07.818 INFO:tasks.workunit.client.1.smithi152.stdout:9/183: write d5/d8/d31/f3b [789622,102503] 0 2024-03-29T04:45:07.819 INFO:tasks.workunit.client.0.smithi142.stdout:8/813: creat d5/d2e/dd2/f11b x:0 0 0 2024-03-29T04:45:07.819 INFO:tasks.workunit.client.1.smithi152.stdout:9/184: chown d5/f29 139104 1 2024-03-29T04:45:07.820 INFO:tasks.workunit.client.0.smithi142.stdout:8/814: dread d5/d1d/d2f/fa9 [0,4194304] 0 2024-03-29T04:45:07.823 INFO:tasks.workunit.client.0.smithi142.stdout:8/815: rename d5/d2e/dd2/cac to d5/d1d/d2f/d76/d3c/c11c 0 2024-03-29T04:45:07.824 INFO:tasks.workunit.client.1.smithi152.stdout:9/185: dwrite d5/db/f21 [0,4194304] 0 2024-03-29T04:45:07.825 INFO:tasks.workunit.client.0.smithi142.stdout:8/816: write d5/d1d/d2f/d76/d9c/f7e [1967795,89090] 0 2024-03-29T04:45:07.827 INFO:tasks.workunit.client.1.smithi152.stdout:9/186: symlink d5/d8/l44 0 2024-03-29T04:45:07.828 INFO:tasks.workunit.client.1.smithi152.stdout:9/187: chown d5/db/f13 5 1 2024-03-29T04:45:07.828 INFO:tasks.workunit.client.1.smithi152.stdout:9/188: write d5/f2f [73016,46525] 0 2024-03-29T04:45:07.829 INFO:tasks.workunit.client.1.smithi152.stdout:9/189: read d5/db/f25 [634578,49130] 0 2024-03-29T04:45:07.831 INFO:tasks.workunit.client.0.smithi142.stdout:8/817: dwrite d5/d1d/d2f/d76/f88 [0,4194304] 0 2024-03-29T04:45:07.832 INFO:tasks.workunit.client.0.smithi142.stdout:8/818: fdatasync d5/d2e/dd2/f11b 0 2024-03-29T04:45:07.833 INFO:tasks.workunit.client.1.smithi152.stdout:9/190: creat d5/d12/d2c/f45 x:0 0 0 2024-03-29T04:45:07.836 INFO:tasks.workunit.client.0.smithi142.stdout:8/819: rename d5/d1d/d2f/d5f/d48/c10e to d5/d1d/d2f/d76/d2d/c11d 0 2024-03-29T04:45:07.838 INFO:tasks.workunit.client.1.smithi152.stdout:9/191: dwrite d5/d8/f9 [0,4194304] 0 2024-03-29T04:45:07.841 INFO:tasks.workunit.client.1.smithi152.stdout:9/192: creat d5/db/d1b/f46 x:0 0 0 2024-03-29T04:45:07.841 INFO:tasks.workunit.client.1.smithi152.stdout:9/193: dread - d5/d12/d2c/f45 zero size 2024-03-29T04:45:07.847 INFO:tasks.workunit.client.1.smithi152.stdout:9/194: dwrite d5/d12/d2c/f3f [0,4194304] 0 2024-03-29T04:45:07.848 INFO:tasks.workunit.client.1.smithi152.stdout:9/195: dread - d5/d8/f1a zero size 2024-03-29T04:45:07.850 INFO:tasks.workunit.client.1.smithi152.stdout:9/196: mknod d5/d12/d2c/c47 0 2024-03-29T04:45:07.854 INFO:tasks.workunit.client.1.smithi152.stdout:9/197: link d5/c22 d5/d12/c48 0 2024-03-29T04:45:07.874 INFO:tasks.workunit.client.1.smithi152.stdout:2/74: sync 2024-03-29T04:45:07.874 INFO:tasks.workunit.client.1.smithi152.stdout:6/134: sync 2024-03-29T04:45:07.877 INFO:tasks.workunit.client.1.smithi152.stdout:2/75: creat d0/d14/f19 x:0 0 0 2024-03-29T04:45:07.877 INFO:tasks.workunit.client.1.smithi152.stdout:6/135: creat d1/d3/d9/d13/f33 x:0 0 0 2024-03-29T04:45:07.878 INFO:tasks.workunit.client.1.smithi152.stdout:2/76: fsync d0/f16 0 2024-03-29T04:45:07.879 INFO:tasks.workunit.client.1.smithi152.stdout:9/198: dread d5/f2f [0,4194304] 0 2024-03-29T04:45:07.880 INFO:tasks.workunit.client.1.smithi152.stdout:6/136: fdatasync d1/d3/f15 0 2024-03-29T04:45:07.880 INFO:tasks.workunit.client.1.smithi152.stdout:9/199: readlink d5/d12/l30 0 2024-03-29T04:45:07.880 INFO:tasks.workunit.client.1.smithi152.stdout:6/137: readlink d1/l32 0 2024-03-29T04:45:07.881 INFO:tasks.workunit.client.1.smithi152.stdout:2/77: creat d0/d14/f1a x:0 0 0 2024-03-29T04:45:07.881 INFO:tasks.workunit.client.1.smithi152.stdout:6/138: stat d1/d3/d1a 0 2024-03-29T04:45:07.885 INFO:tasks.workunit.client.1.smithi152.stdout:2/78: rmdir d0/d3 39 2024-03-29T04:45:07.888 INFO:tasks.workunit.client.0.smithi142.stdout:5/883: truncate df/d16/d24/d2e/f6c 7629706 0 2024-03-29T04:45:07.889 INFO:tasks.workunit.client.1.smithi152.stdout:9/200: getdents d5/db 0 2024-03-29T04:45:07.889 INFO:tasks.workunit.client.0.smithi142.stdout:2/683: truncate de/d11/d1e/d22/d27/dba/d61/d7d/f7e 3249689 0 2024-03-29T04:45:07.890 INFO:tasks.workunit.client.1.smithi152.stdout:2/79: creat d0/d14/f1b x:0 0 0 2024-03-29T04:45:07.891 INFO:tasks.workunit.client.1.smithi152.stdout:4/58: fdatasync d0/f4 0 2024-03-29T04:45:07.892 INFO:tasks.workunit.client.1.smithi152.stdout:9/201: write d5/f29 [2712268,75120] 0 2024-03-29T04:45:07.893 INFO:tasks.workunit.client.1.smithi152.stdout:2/80: mknod d0/d14/c1c 0 2024-03-29T04:45:07.895 INFO:tasks.workunit.client.1.smithi152.stdout:6/139: dwrite d1/d3/d9/d13/f26 [0,4194304] 0 2024-03-29T04:45:07.896 INFO:tasks.workunit.client.1.smithi152.stdout:4/59: creat d0/d7/de/ff x:0 0 0 2024-03-29T04:45:07.897 INFO:tasks.workunit.client.0.smithi142.stdout:4/699: write d0/d9/db2/dba/dbc/f92 [1413020,6041] 0 2024-03-29T04:45:07.900 INFO:tasks.workunit.client.1.smithi152.stdout:6/140: creat d1/d3/d9/d13/d28/f34 x:0 0 0 2024-03-29T04:45:07.901 INFO:tasks.workunit.client.1.smithi152.stdout:2/81: creat d0/d3/f1d x:0 0 0 2024-03-29T04:45:07.904 INFO:tasks.workunit.client.1.smithi152.stdout:6/141: mknod d1/d3/d9/c35 0 2024-03-29T04:45:07.905 INFO:tasks.workunit.client.1.smithi152.stdout:2/82: write d0/d3/df/f13 [938319,79945] 0 2024-03-29T04:45:07.905 INFO:tasks.workunit.client.1.smithi152.stdout:6/142: readlink d1/l32 0 2024-03-29T04:45:07.908 INFO:tasks.workunit.client.1.smithi152.stdout:4/60: dwrite d0/d7/de/ff [0,4194304] 0 2024-03-29T04:45:07.911 INFO:tasks.workunit.client.1.smithi152.stdout:2/83: creat d0/d3/df/f1e x:0 0 0 2024-03-29T04:45:07.914 INFO:tasks.workunit.client.0.smithi142.stdout:5/884: mkdir df/d16/d24/d2e/d47/d4d/d125 0 2024-03-29T04:45:07.916 INFO:tasks.workunit.client.0.smithi142.stdout:2/684: symlink de/d11/d17/d4a/d79/lec 0 2024-03-29T04:45:07.917 INFO:tasks.workunit.client.1.smithi152.stdout:2/84: truncate d0/d3/df/f13 1213752 0 2024-03-29T04:45:07.917 INFO:tasks.workunit.client.1.smithi152.stdout:6/143: link d1/d3/d9/l12 d1/d3/d9/d13/l36 0 2024-03-29T04:45:07.918 INFO:tasks.workunit.client.1.smithi152.stdout:9/202: dwrite d5/d8/f42 [0,4194304] 0 2024-03-29T04:45:07.921 INFO:tasks.workunit.client.0.smithi142.stdout:5/885: unlink df/d1a/c3c 0 2024-03-29T04:45:07.923 INFO:tasks.workunit.client.1.smithi152.stdout:9/203: write d5/db/d1b/f40 [396700,126813] 0 2024-03-29T04:45:07.923 INFO:tasks.workunit.client.1.smithi152.stdout:9/204: dread - d5/db/d1b/f46 zero size 2024-03-29T04:45:07.924 INFO:tasks.workunit.client.1.smithi152.stdout:6/144: dread d1/d3/d9/d13/f26 [0,4194304] 0 2024-03-29T04:45:07.925 INFO:tasks.workunit.client.1.smithi152.stdout:2/85: truncate d0/f16 761794 0 2024-03-29T04:45:07.926 INFO:tasks.workunit.client.1.smithi152.stdout:9/205: truncate d5/db/d1b/f2b 4875069 0 2024-03-29T04:45:07.927 INFO:tasks.workunit.client.1.smithi152.stdout:6/145: read d1/d3/d9/f11 [1158140,89100] 0 2024-03-29T04:45:07.927 INFO:tasks.workunit.client.1.smithi152.stdout:2/86: chown d0/d3/f10 89 1 2024-03-29T04:45:07.928 INFO:tasks.workunit.client.1.smithi152.stdout:9/206: write d5/d8/f1a [77215,112628] 0 2024-03-29T04:45:07.928 INFO:tasks.workunit.client.0.smithi142.stdout:5/886: link df/d16/le8 df/d16/d24/d3d/l126 0 2024-03-29T04:45:07.929 INFO:tasks.workunit.client.1.smithi152.stdout:9/207: dread - d5/d12/d2c/f35 zero size 2024-03-29T04:45:07.930 INFO:tasks.workunit.client.1.smithi152.stdout:6/146: mkdir d1/d3/d1a/d37 0 2024-03-29T04:45:07.931 INFO:tasks.workunit.client.0.smithi142.stdout:5/887: creat df/d2c/f127 x:0 0 0 2024-03-29T04:45:07.932 INFO:tasks.workunit.client.0.smithi142.stdout:5/888: chown df/d1a/d21/d55/f81 20777913 1 2024-03-29T04:45:07.932 INFO:tasks.workunit.client.1.smithi152.stdout:6/147: write d1/d3/d1a/f2c [455972,29104] 0 2024-03-29T04:45:07.933 INFO:tasks.workunit.client.0.smithi142.stdout:5/889: fdatasync df/d29/d6e/f101 0 2024-03-29T04:45:07.933 INFO:tasks.workunit.client.1.smithi152.stdout:9/208: stat d5/d8/d31/f3b 0 2024-03-29T04:45:07.938 INFO:tasks.workunit.client.1.smithi152.stdout:9/209: creat d5/db/d1b/f49 x:0 0 0 2024-03-29T04:45:07.940 INFO:tasks.workunit.client.0.smithi142.stdout:4/700: dread d0/dc/d15/d3d/d65/f85 [0,4194304] 0 2024-03-29T04:45:07.940 INFO:tasks.workunit.client.1.smithi152.stdout:4/61: fdatasync d0/d7/de/ff 0 2024-03-29T04:45:07.940 INFO:tasks.workunit.client.1.smithi152.stdout:2/87: dwrite d0/d3/df/f1e [0,4194304] 0 2024-03-29T04:45:07.941 INFO:tasks.workunit.client.1.smithi152.stdout:6/148: rename d1/l1c to d1/d3/d1a/d1b/d29/l38 0 2024-03-29T04:45:07.941 INFO:tasks.workunit.client.1.smithi152.stdout:2/88: stat d0/d3/l7 0 2024-03-29T04:45:07.943 INFO:tasks.workunit.client.0.smithi142.stdout:5/890: dread df/d16/d24/d2e/f8b [0,4194304] 0 2024-03-29T04:45:07.943 INFO:tasks.workunit.client.0.smithi142.stdout:4/701: unlink d0/d9/d11/f1b 0 2024-03-29T04:45:07.944 INFO:tasks.workunit.client.1.smithi152.stdout:6/149: write d1/d3/d9/f21 [4518211,105015] 0 2024-03-29T04:45:07.945 INFO:tasks.workunit.client.1.smithi152.stdout:4/62: truncate d0/f4 8564000 0 2024-03-29T04:45:07.946 INFO:tasks.workunit.client.0.smithi142.stdout:9/567: sync 2024-03-29T04:45:07.947 INFO:tasks.workunit.client.0.smithi142.stdout:9/568: readlink d0/d2/d6/l1a 0 2024-03-29T04:45:07.947 INFO:tasks.workunit.client.0.smithi142.stdout:4/702: rmdir d0/d9/db2/dba/dbc 39 2024-03-29T04:45:07.949 INFO:tasks.workunit.client.0.smithi142.stdout:5/891: mknod df/d16/d24/d2e/d47/d8c/de0/c128 0 2024-03-29T04:45:07.949 INFO:tasks.workunit.client.1.smithi152.stdout:2/89: mknod d0/d3/df/c1f 0 2024-03-29T04:45:07.952 INFO:tasks.workunit.client.1.smithi152.stdout:2/90: readlink d0/d3/l11 0 2024-03-29T04:45:07.952 INFO:tasks.workunit.client.1.smithi152.stdout:9/210: dwrite d5/db/d1b/f40 [0,4194304] 0 2024-03-29T04:45:07.953 INFO:tasks.workunit.client.0.smithi142.stdout:5/892: creat df/d16/d24/d2e/d47/d4d/d102/d75/f129 x:0 0 0 2024-03-29T04:45:07.955 INFO:tasks.workunit.client.0.smithi142.stdout:9/569: dwrite d0/d12/d50/f63 [0,4194304] 0 2024-03-29T04:45:07.955 INFO:tasks.workunit.client.0.smithi142.stdout:4/703: mknod d0/d9/db2/dba/dbc/cee 0 2024-03-29T04:45:07.956 INFO:tasks.workunit.client.0.smithi142.stdout:2/685: dread de/f29 [0,4194304] 0 2024-03-29T04:45:07.957 INFO:tasks.workunit.client.0.smithi142.stdout:5/893: fsync df/d16/d24/d3d/fa6 0 2024-03-29T04:45:07.958 INFO:tasks.workunit.client.1.smithi152.stdout:2/91: unlink d0/f2 0 2024-03-29T04:45:07.958 INFO:tasks.workunit.client.1.smithi152.stdout:9/211: truncate d5/db/d1b/f46 218932 0 2024-03-29T04:45:07.958 INFO:tasks.workunit.client.0.smithi142.stdout:2/686: chown de/d11/d17/d60 17 1 2024-03-29T04:45:07.959 INFO:tasks.workunit.client.1.smithi152.stdout:6/150: dwrite d1/d3/d1a/f1d [0,4194304] 0 2024-03-29T04:45:07.960 INFO:tasks.workunit.client.1.smithi152.stdout:4/63: dread d0/d7/de/ff [0,4194304] 0 2024-03-29T04:45:07.961 INFO:tasks.workunit.client.0.smithi142.stdout:9/570: mknod d0/d6c/cb6 0 2024-03-29T04:45:07.964 INFO:tasks.workunit.client.0.smithi142.stdout:4/704: creat d0/d9/d11/d4c/d5f/d81/dc9/d47/fef x:0 0 0 2024-03-29T04:45:07.965 INFO:tasks.workunit.client.0.smithi142.stdout:4/705: fdatasync d0/d12/f1e 0 2024-03-29T04:45:07.965 INFO:tasks.workunit.client.1.smithi152.stdout:6/151: truncate d1/d3/f18 174904 0 2024-03-29T04:45:07.966 INFO:tasks.workunit.client.0.smithi142.stdout:5/894: symlink df/d16/d25/l12a 0 2024-03-29T04:45:07.966 INFO:tasks.workunit.client.1.smithi152.stdout:4/64: rmdir d0/d7 39 2024-03-29T04:45:07.967 INFO:tasks.workunit.client.0.smithi142.stdout:2/687: rmdir de/d11/d2b/d7b 39 2024-03-29T04:45:07.969 INFO:tasks.workunit.client.1.smithi152.stdout:2/92: link d0/f12 d0/d14/d15/f20 0 2024-03-29T04:45:07.971 INFO:tasks.workunit.client.0.smithi142.stdout:4/706: rmdir d0/d9 39 2024-03-29T04:45:07.971 INFO:tasks.workunit.client.1.smithi152.stdout:2/93: read d0/d3/df/f13 [68578,114372] 0 2024-03-29T04:45:07.973 INFO:tasks.workunit.client.0.smithi142.stdout:5/895: symlink df/d16/d24/d2e/d47/l12b 0 2024-03-29T04:45:07.973 INFO:tasks.workunit.client.0.smithi142.stdout:2/688: stat de/d11/d1e/d22/d27/dba/c5c 0 2024-03-29T04:45:07.976 INFO:tasks.workunit.client.1.smithi152.stdout:4/65: creat d0/d7/f10 x:0 0 0 2024-03-29T04:45:07.977 INFO:tasks.workunit.client.1.smithi152.stdout:2/94: dread d0/d3/df/f1e [0,4194304] 0 2024-03-29T04:45:07.977 INFO:tasks.workunit.client.1.smithi152.stdout:4/66: fdatasync d0/d7/f10 0 2024-03-29T04:45:07.977 INFO:tasks.workunit.client.1.smithi152.stdout:2/95: readlink d0/le 0 2024-03-29T04:45:07.978 INFO:tasks.workunit.client.1.smithi152.stdout:6/152: getdents d1/d3/d1a 0 2024-03-29T04:45:07.979 INFO:tasks.workunit.client.1.smithi152.stdout:4/67: read - d0/d7/f10 zero size 2024-03-29T04:45:07.979 INFO:tasks.workunit.client.1.smithi152.stdout:2/96: write d0/d14/f1b [960288,81413] 0 2024-03-29T04:45:07.979 INFO:tasks.workunit.client.0.smithi142.stdout:2/689: symlink de/d11/d1e/d22/d27/led 0 2024-03-29T04:45:07.981 INFO:tasks.workunit.client.1.smithi152.stdout:4/68: write d0/d7/de/ff [3438696,4221] 0 2024-03-29T04:45:07.981 INFO:tasks.workunit.client.1.smithi152.stdout:6/153: symlink d1/d3/df/l39 0 2024-03-29T04:45:07.983 INFO:tasks.workunit.client.1.smithi152.stdout:2/97: symlink d0/d14/l21 0 2024-03-29T04:45:07.983 INFO:tasks.workunit.client.0.smithi142.stdout:2/690: mknod de/d11/d2b/d3e/cee 0 2024-03-29T04:45:07.986 INFO:tasks.workunit.client.0.smithi142.stdout:2/691: link de/d11/d2b/d3e/fcf de/d11/d17/d60/de6/fef 0 2024-03-29T04:45:07.987 INFO:tasks.workunit.client.1.smithi152.stdout:9/212: truncate d5/d8/f23 2579037 0 2024-03-29T04:45:07.988 INFO:tasks.workunit.client.0.smithi142.stdout:2/692: write de/d11/d3d/fd8 [513857,94036] 0 2024-03-29T04:45:07.989 INFO:tasks.workunit.client.1.smithi152.stdout:2/98: mkdir d0/d14/d15/d22 0 2024-03-29T04:45:07.989 INFO:tasks.workunit.client.1.smithi152.stdout:2/99: write d0/d3/f18 [234938,2132] 0 2024-03-29T04:45:07.990 INFO:tasks.workunit.client.1.smithi152.stdout:6/154: rmdir d1/d3/d1a/d37 0 2024-03-29T04:45:07.994 INFO:tasks.workunit.client.0.smithi142.stdout:2/693: unlink de/d11/d1e/d22/d27/dba/cae 0 2024-03-29T04:45:07.994 INFO:tasks.workunit.client.1.smithi152.stdout:2/100: write d0/d14/f1a [971495,59941] 0 2024-03-29T04:45:07.994 INFO:tasks.workunit.client.1.smithi152.stdout:9/213: mknod d5/d8/c4a 0 2024-03-29T04:45:07.995 INFO:tasks.workunit.client.1.smithi152.stdout:4/69: fsync d0/d7/f10 0 2024-03-29T04:45:07.995 INFO:tasks.workunit.client.1.smithi152.stdout:6/155: rename d1/d3/d9/d13/d28/f2d to d1/d3/d9/f3a 0 2024-03-29T04:45:07.995 INFO:tasks.workunit.client.1.smithi152.stdout:4/70: truncate d0/d7/f10 394847 0 2024-03-29T04:45:07.995 INFO:tasks.workunit.client.0.smithi142.stdout:9/571: dwrite d0/f94 [0,4194304] 0 2024-03-29T04:45:07.996 INFO:tasks.workunit.client.0.smithi142.stdout:2/694: write de/d11/d17/d5d/f86 [266717,16453] 0 2024-03-29T04:45:07.997 INFO:tasks.workunit.client.1.smithi152.stdout:2/101: dwrite d0/f9 [0,4194304] 0 2024-03-29T04:45:08.000 INFO:tasks.workunit.client.1.smithi152.stdout:2/102: write d0/d3/f18 [304782,104073] 0 2024-03-29T04:45:08.000 INFO:tasks.workunit.client.1.smithi152.stdout:6/156: mkdir d1/d3/d9/d3b 0 2024-03-29T04:45:08.000 INFO:tasks.workunit.client.0.smithi142.stdout:5/896: dwrite df/d1a/d21/fcb [0,4194304] 0 2024-03-29T04:45:08.001 INFO:tasks.workunit.client.1.smithi152.stdout:2/103: readlink d0/d3/l7 0 2024-03-29T04:45:08.007 INFO:tasks.workunit.client.0.smithi142.stdout:9/572: dread d0/d2/d6/f30 [0,4194304] 0 2024-03-29T04:45:08.008 INFO:tasks.workunit.client.1.smithi152.stdout:9/214: link d5/db/c18 d5/d12/c4b 0 2024-03-29T04:45:08.008 INFO:tasks.workunit.client.1.smithi152.stdout:2/104: creat d0/d3/df/f23 x:0 0 0 2024-03-29T04:45:08.008 INFO:tasks.workunit.client.0.smithi142.stdout:5/897: creat df/d16/d24/d2e/d47/d4d/d102/d75/d90/f12c x:0 0 0 2024-03-29T04:45:08.008 INFO:tasks.workunit.client.0.smithi142.stdout:9/573: dread - d0/f62 zero size 2024-03-29T04:45:08.009 INFO:tasks.workunit.client.1.smithi152.stdout:2/105: read - d0/d14/f19 zero size 2024-03-29T04:45:08.009 INFO:tasks.workunit.client.0.smithi142.stdout:5/898: dread - df/d16/d25/ff5 zero size 2024-03-29T04:45:08.010 INFO:tasks.workunit.client.1.smithi152.stdout:4/71: getdents d0/d7 0 2024-03-29T04:45:08.016 INFO:tasks.workunit.client.1.smithi152.stdout:9/215: creat d5/d8/d31/f4c x:0 0 0 2024-03-29T04:45:08.018 INFO:tasks.workunit.client.0.smithi142.stdout:5/899: dread df/d1a/d21/d27/fe7 [0,4194304] 0 2024-03-29T04:45:08.019 INFO:tasks.workunit.client.0.smithi142.stdout:5/900: write df/d2c/f10d [360893,74328] 0 2024-03-29T04:45:08.019 INFO:tasks.workunit.client.1.smithi152.stdout:4/72: creat d0/f11 x:0 0 0 2024-03-29T04:45:08.019 INFO:tasks.workunit.client.1.smithi152.stdout:4/73: stat d0/ca 0 2024-03-29T04:45:08.020 INFO:tasks.workunit.client.1.smithi152.stdout:4/74: creat d0/f12 x:0 0 0 2024-03-29T04:45:08.022 INFO:tasks.workunit.client.1.smithi152.stdout:4/75: write d0/d7/f10 [49449,19589] 0 2024-03-29T04:45:08.023 INFO:tasks.workunit.client.1.smithi152.stdout:4/76: truncate d0/f12 513712 0 2024-03-29T04:45:08.024 INFO:tasks.workunit.client.1.smithi152.stdout:4/77: fsync d0/f4 0 2024-03-29T04:45:08.025 INFO:tasks.workunit.client.1.smithi152.stdout:4/78: write d0/f12 [1145009,79166] 0 2024-03-29T04:45:08.027 INFO:tasks.workunit.client.0.smithi142.stdout:2/695: write de/d11/d1e/d22/d27/dba/d61/fa0 [481506,83234] 0 2024-03-29T04:45:08.028 INFO:tasks.workunit.client.0.smithi142.stdout:9/574: write d0/f32 [386306,21831] 0 2024-03-29T04:45:08.028 INFO:tasks.workunit.client.0.smithi142.stdout:5/901: write df/d16/d24/d5c/f5e [237372,77350] 0 2024-03-29T04:45:08.029 INFO:tasks.workunit.client.1.smithi152.stdout:4/79: unlink d0/d7/f10 0 2024-03-29T04:45:08.032 INFO:tasks.workunit.client.0.smithi142.stdout:5/902: dread df/d29/f6b [0,4194304] 0 2024-03-29T04:45:08.035 INFO:tasks.workunit.client.1.smithi152.stdout:4/80: creat d0/f13 x:0 0 0 2024-03-29T04:45:08.035 INFO:tasks.workunit.client.1.smithi152.stdout:4/81: stat d0/d7/de 0 2024-03-29T04:45:08.037 INFO:tasks.workunit.client.1.smithi152.stdout:4/82: creat d0/d7/f14 x:0 0 0 2024-03-29T04:45:08.039 INFO:tasks.workunit.client.1.smithi152.stdout:6/157: write d1/d3/d9/f27 [140675,58965] 0 2024-03-29T04:45:08.040 INFO:tasks.workunit.client.0.smithi142.stdout:9/575: dread d0/d2/f1f [0,4194304] 0 2024-03-29T04:45:08.042 INFO:tasks.workunit.client.0.smithi142.stdout:9/576: write d0/d2/d61/d64/f80 [90699,104181] 0 2024-03-29T04:45:08.042 INFO:tasks.workunit.client.1.smithi152.stdout:4/83: dread d0/f12 [0,4194304] 0 2024-03-29T04:45:08.044 INFO:tasks.workunit.client.1.smithi152.stdout:4/84: chown d0/ca 1 1 2024-03-29T04:45:08.044 INFO:tasks.workunit.client.1.smithi152.stdout:6/158: write d1/d3/d9/f3a [842719,9708] 0 2024-03-29T04:45:08.046 INFO:tasks.workunit.client.1.smithi152.stdout:2/106: dwrite d0/d3/df/f13 [0,4194304] 0 2024-03-29T04:45:08.047 INFO:tasks.workunit.client.1.smithi152.stdout:4/85: creat d0/f15 x:0 0 0 2024-03-29T04:45:08.048 INFO:tasks.workunit.client.1.smithi152.stdout:6/159: creat d1/d3/d9/d3b/f3c x:0 0 0 2024-03-29T04:45:08.051 INFO:tasks.workunit.client.1.smithi152.stdout:6/160: write d1/d3/d9/d13/d28/f34 [502241,8623] 0 2024-03-29T04:45:08.055 INFO:tasks.workunit.client.1.smithi152.stdout:6/161: unlink d1/d3/d9/l24 0 2024-03-29T04:45:08.055 INFO:tasks.workunit.client.1.smithi152.stdout:4/86: dwrite d0/d7/f14 [0,4194304] 0 2024-03-29T04:45:08.056 INFO:tasks.workunit.client.1.smithi152.stdout:4/87: dread - d0/f13 zero size 2024-03-29T04:45:08.056 INFO:tasks.workunit.client.1.smithi152.stdout:4/88: readlink d0/lc 0 2024-03-29T04:45:08.057 INFO:tasks.workunit.client.1.smithi152.stdout:6/162: truncate d1/d3/d9/f27 1290731 0 2024-03-29T04:45:08.057 INFO:tasks.workunit.client.1.smithi152.stdout:4/89: dread - d0/f13 zero size 2024-03-29T04:45:08.060 INFO:tasks.workunit.client.1.smithi152.stdout:6/163: creat d1/d3/d9/d13/f3d x:0 0 0 2024-03-29T04:45:08.061 INFO:tasks.workunit.client.1.smithi152.stdout:4/90: mkdir d0/d16 0 2024-03-29T04:45:08.061 INFO:tasks.workunit.client.1.smithi152.stdout:6/164: chown d1/d3/d1a/f2c 2181 1 2024-03-29T04:45:08.062 INFO:tasks.workunit.client.1.smithi152.stdout:4/91: write d0/f11 [71886,41424] 0 2024-03-29T04:45:08.062 INFO:tasks.workunit.client.1.smithi152.stdout:6/165: chown d1/d3/f15 33257153 1 2024-03-29T04:45:08.065 INFO:tasks.workunit.client.1.smithi152.stdout:6/166: mkdir d1/d3/d9/d13/d3e 0 2024-03-29T04:45:08.065 INFO:tasks.workunit.client.1.smithi152.stdout:6/167: stat d1/fd 0 2024-03-29T04:45:08.065 INFO:tasks.workunit.client.1.smithi152.stdout:6/168: read d1/d3/d9/d13/f26 [1015521,112915] 0 2024-03-29T04:45:08.069 INFO:tasks.workunit.client.1.smithi152.stdout:6/169: rename d1/d3/d1a/d1b/d29/l38 to d1/d3/d1a/d1b/d29/l3f 0 2024-03-29T04:45:08.069 INFO:tasks.workunit.client.1.smithi152.stdout:4/92: read d0/f11 [55189,24037] 0 2024-03-29T04:45:08.070 INFO:tasks.workunit.client.1.smithi152.stdout:6/170: write d1/d3/d9/f3a [1148636,24275] 0 2024-03-29T04:45:08.070 INFO:tasks.workunit.client.1.smithi152.stdout:4/93: chown d0/f15 13 1 2024-03-29T04:45:08.071 INFO:tasks.workunit.client.1.smithi152.stdout:6/171: write d1/d3/d9/d13/f31 [421720,125929] 0 2024-03-29T04:45:08.073 INFO:tasks.workunit.client.1.smithi152.stdout:6/172: write d1/d3/d1a/f2f [751499,49728] 0 2024-03-29T04:45:08.077 INFO:tasks.workunit.client.1.smithi152.stdout:1/99: sync 2024-03-29T04:45:08.078 INFO:tasks.workunit.client.1.smithi152.stdout:4/94: dwrite d0/f12 [0,4194304] 0 2024-03-29T04:45:08.078 INFO:tasks.workunit.client.1.smithi152.stdout:4/95: chown d0/ca 11822093 1 2024-03-29T04:45:08.080 INFO:tasks.workunit.client.1.smithi152.stdout:6/173: rmdir d1/d3/d9/d13 39 2024-03-29T04:45:08.082 INFO:tasks.workunit.client.1.smithi152.stdout:6/174: write f0 [984641,106977] 0 2024-03-29T04:45:08.083 INFO:tasks.workunit.client.0.smithi142.stdout:0/709: sync 2024-03-29T04:45:08.083 INFO:tasks.workunit.client.0.smithi142.stdout:6/669: sync 2024-03-29T04:45:08.083 INFO:tasks.workunit.client.1.smithi152.stdout:4/96: dread d0/f12 [0,4194304] 0 2024-03-29T04:45:08.084 INFO:tasks.workunit.client.1.smithi152.stdout:1/100: getdents d5/d7 0 2024-03-29T04:45:08.085 INFO:tasks.workunit.client.1.smithi152.stdout:1/101: dread - d5/d7/f25 zero size 2024-03-29T04:45:08.086 INFO:tasks.workunit.client.1.smithi152.stdout:6/175: symlink d1/d3/d9/l40 0 2024-03-29T04:45:08.090 INFO:tasks.workunit.client.1.smithi152.stdout:4/97: dwrite d0/d7/de/ff [0,4194304] 0 2024-03-29T04:45:08.092 INFO:tasks.workunit.client.1.smithi152.stdout:6/176: dread d1/d3/d9/f21 [0,4194304] 0 2024-03-29T04:45:08.094 INFO:tasks.workunit.client.1.smithi152.stdout:6/177: stat d1/d3/d1a/f1d 0 2024-03-29T04:45:08.096 INFO:tasks.workunit.client.1.smithi152.stdout:4/98: getdents d0/d7/de 0 2024-03-29T04:45:08.097 INFO:tasks.workunit.client.1.smithi152.stdout:6/178: fsync d1/d3/d9/d13/f3d 0 2024-03-29T04:45:08.098 INFO:tasks.workunit.client.1.smithi152.stdout:4/99: mkdir d0/d16/d17 0 2024-03-29T04:45:08.099 INFO:tasks.workunit.client.1.smithi152.stdout:6/179: creat d1/d3/d9/f41 x:0 0 0 2024-03-29T04:45:08.099 INFO:tasks.workunit.client.1.smithi152.stdout:4/100: symlink d0/d7/de/l18 0 2024-03-29T04:45:08.101 INFO:tasks.workunit.client.1.smithi152.stdout:4/101: write d0/d7/de/ff [3431770,12524] 0 2024-03-29T04:45:08.101 INFO:tasks.workunit.client.1.smithi152.stdout:6/180: truncate d1/d3/d9/d13/d28/f34 605376 0 2024-03-29T04:45:08.101 INFO:tasks.workunit.client.1.smithi152.stdout:4/102: chown d0/d16/d17 364 1 2024-03-29T04:45:08.102 INFO:tasks.workunit.client.1.smithi152.stdout:4/103: fdatasync d0/d7/f14 0 2024-03-29T04:45:08.103 INFO:tasks.workunit.client.1.smithi152.stdout:4/104: read d0/f12 [1406885,116466] 0 2024-03-29T04:45:08.104 INFO:tasks.workunit.client.1.smithi152.stdout:4/105: fsync d0/f12 0 2024-03-29T04:45:08.106 INFO:tasks.workunit.client.1.smithi152.stdout:4/106: mknod d0/d7/c19 0 2024-03-29T04:45:08.107 INFO:tasks.workunit.client.1.smithi152.stdout:6/181: truncate d1/d3/d9/d13/f31 423453 0 2024-03-29T04:45:08.108 INFO:tasks.workunit.client.1.smithi152.stdout:4/107: creat d0/d16/f1a x:0 0 0 2024-03-29T04:45:08.110 INFO:tasks.workunit.client.1.smithi152.stdout:6/182: getdents d1/d3/df 0 2024-03-29T04:45:08.112 INFO:tasks.workunit.client.1.smithi152.stdout:6/183: dread - d1/d3/d9/d3b/f3c zero size 2024-03-29T04:45:08.114 INFO:tasks.workunit.client.0.smithi142.stdout:2/696: write de/d11/f4f [1066839,22] 0 2024-03-29T04:45:08.115 INFO:tasks.workunit.client.1.smithi152.stdout:4/108: dwrite d0/d7/f14 [0,4194304] 0 2024-03-29T04:45:08.117 INFO:tasks.workunit.client.1.smithi152.stdout:6/184: creat d1/d3/d9/d13/d3e/f42 x:0 0 0 2024-03-29T04:45:08.118 INFO:tasks.workunit.client.0.smithi142.stdout:7/784: sync 2024-03-29T04:45:08.118 INFO:tasks.workunit.client.0.smithi142.stdout:1/672: sync 2024-03-29T04:45:08.118 INFO:tasks.workunit.client.0.smithi142.stdout:8/820: sync 2024-03-29T04:45:08.118 INFO:tasks.workunit.client.0.smithi142.stdout:4/707: sync 2024-03-29T04:45:08.118 INFO:tasks.workunit.client.0.smithi142.stdout:3/566: sync 2024-03-29T04:45:08.118 INFO:tasks.workunit.client.0.smithi142.stdout:9/577: creat d0/d12/d50/d52/fb7 x:0 0 0 2024-03-29T04:45:08.119 INFO:tasks.workunit.client.1.smithi152.stdout:4/109: symlink d0/d7/de/l1b 0 2024-03-29T04:45:08.119 INFO:tasks.workunit.client.0.smithi142.stdout:7/785: chown d7/d27/d44/dab/dcc 89 1 2024-03-29T04:45:08.121 INFO:tasks.workunit.client.0.smithi142.stdout:0/710: truncate d7/d13/dbf/d72/d75/fb5 313235 0 2024-03-29T04:45:08.121 INFO:tasks.workunit.client.0.smithi142.stdout:9/578: write d0/d2/f11 [880608,76266] 0 2024-03-29T04:45:08.121 INFO:tasks.workunit.client.1.smithi152.stdout:4/110: chown d0/f13 1 1 2024-03-29T04:45:08.123 INFO:tasks.workunit.client.1.smithi152.stdout:4/111: truncate d0/f4 7959352 0 2024-03-29T04:45:08.124 INFO:tasks.workunit.client.0.smithi142.stdout:4/708: write d0/d9/d19/d2c/f56 [328857,88220] 0 2024-03-29T04:45:08.125 INFO:tasks.workunit.client.0.smithi142.stdout:7/786: unlink d7/f19 0 2024-03-29T04:45:08.126 INFO:tasks.workunit.client.1.smithi152.stdout:4/112: write d0/d16/f1a [988325,85577] 0 2024-03-29T04:45:08.126 INFO:tasks.workunit.client.1.smithi152.stdout:6/185: dwrite d1/d3/d9/d13/f1f [0,4194304] 0 2024-03-29T04:45:08.126 INFO:tasks.workunit.client.0.smithi142.stdout:7/787: dread - d7/d27/d44/db2/db8/de/f2d zero size 2024-03-29T04:45:08.127 INFO:tasks.workunit.client.0.smithi142.stdout:3/567: mknod d3/d1f/d24/d47/cc2 0 2024-03-29T04:45:08.127 INFO:tasks.workunit.client.1.smithi152.stdout:4/113: fdatasync d0/f15 0 2024-03-29T04:45:08.128 INFO:tasks.workunit.client.1.smithi152.stdout:4/114: rename d0/d7 to d0/d7/d1c 22 2024-03-29T04:45:08.129 INFO:tasks.workunit.client.1.smithi152.stdout:4/115: stat d0/d7 0 2024-03-29T04:45:08.129 INFO:tasks.workunit.client.1.smithi152.stdout:4/116: fdatasync d0/f12 0 2024-03-29T04:45:08.130 INFO:tasks.workunit.client.0.smithi142.stdout:9/579: mknod d0/d6c/cb8 0 2024-03-29T04:45:08.130 INFO:tasks.workunit.client.1.smithi152.stdout:6/186: fdatasync d1/d3/f18 0 2024-03-29T04:45:08.131 INFO:tasks.workunit.client.1.smithi152.stdout:4/117: chown d0/d7/de/ff 11481815 1 2024-03-29T04:45:08.131 INFO:tasks.workunit.client.0.smithi142.stdout:0/711: mkdir d7/d13/d4a/de7 0 2024-03-29T04:45:08.131 INFO:tasks.workunit.client.0.smithi142.stdout:9/580: stat d0/d2/d61/f55 0 2024-03-29T04:45:08.131 INFO:tasks.workunit.client.1.smithi152.stdout:4/118: chown d0/cd 112150 1 2024-03-29T04:45:08.132 INFO:tasks.workunit.client.1.smithi152.stdout:4/119: fsync d0/f15 0 2024-03-29T04:45:08.134 INFO:tasks.workunit.client.0.smithi142.stdout:4/709: dread d0/d9/db2/dba/dbc/d6d/d6e/fc4 [0,4194304] 0 2024-03-29T04:45:08.135 INFO:tasks.workunit.client.1.smithi152.stdout:6/187: creat d1/f43 x:0 0 0 2024-03-29T04:45:08.136 INFO:tasks.workunit.client.0.smithi142.stdout:5/903: write df/d16/d24/d2e/d47/d4d/fcc [266334,87608] 0 2024-03-29T04:45:08.139 INFO:tasks.workunit.client.0.smithi142.stdout:3/568: dwrite d3/d1f/d24/d47/d87/fa3 [0,4194304] 0 2024-03-29T04:45:08.141 INFO:tasks.workunit.client.0.smithi142.stdout:4/710: dread d0/dc/d15/d3d/d65/f7c [0,4194304] 0 2024-03-29T04:45:08.143 INFO:tasks.workunit.client.1.smithi152.stdout:6/188: getdents d1/d3/d9/d13/d3e 0 2024-03-29T04:45:08.145 INFO:tasks.workunit.client.0.smithi142.stdout:7/788: rmdir d7 39 2024-03-29T04:45:08.147 INFO:tasks.workunit.client.0.smithi142.stdout:6/670: getdents d1c/d33/d46/d66 0 2024-03-29T04:45:08.147 INFO:tasks.workunit.client.0.smithi142.stdout:2/697: write de/d11/d17/d5d/f74 [3236261,89219] 0 2024-03-29T04:45:08.148 INFO:tasks.workunit.client.0.smithi142.stdout:0/712: rename d7/f2b to d7/fe8 0 2024-03-29T04:45:08.151 INFO:tasks.workunit.client.1.smithi152.stdout:4/120: truncate d0/f11 64009 0 2024-03-29T04:45:08.155 INFO:tasks.workunit.client.0.smithi142.stdout:5/904: chown df/d16/d24/d2e/f6c 1960931 1 2024-03-29T04:45:08.155 INFO:tasks.workunit.client.1.smithi152.stdout:6/189: truncate d1/f17 3352736 0 2024-03-29T04:45:08.156 INFO:tasks.workunit.client.0.smithi142.stdout:4/711: creat d0/d9/d11/d4c/d5f/d81/ff0 x:0 0 0 2024-03-29T04:45:08.158 INFO:tasks.workunit.client.0.smithi142.stdout:2/698: creat de/d11/d23/dd1/ff0 x:0 0 0 2024-03-29T04:45:08.158 INFO:tasks.workunit.client.1.smithi152.stdout:6/190: rename d1/l19 to d1/d3/d1a/d1b/l44 0 2024-03-29T04:45:08.159 INFO:tasks.workunit.client.1.smithi152.stdout:6/191: chown d1/d3/d9/d3b/f3c 2942791 1 2024-03-29T04:45:08.161 INFO:tasks.workunit.client.0.smithi142.stdout:0/713: write d7/d13/dbf/f8e [417158,47502] 0 2024-03-29T04:45:08.161 INFO:tasks.workunit.client.0.smithi142.stdout:1/673: getdents d8/db/dc/d77/d31 0 2024-03-29T04:45:08.163 INFO:tasks.workunit.client.0.smithi142.stdout:9/581: write d0/f62 [910643,28858] 0 2024-03-29T04:45:08.163 INFO:tasks.workunit.client.0.smithi142.stdout:5/905: mknod df/d29/d7a/d97/c12d 0 2024-03-29T04:45:08.165 INFO:tasks.workunit.client.0.smithi142.stdout:4/712: creat d0/d9/d11/d4c/d5f/d81/ff1 x:0 0 0 2024-03-29T04:45:08.167 INFO:tasks.workunit.client.1.smithi152.stdout:4/121: write d0/f11 [458234,115611] 0 2024-03-29T04:45:08.167 INFO:tasks.workunit.client.1.smithi152.stdout:6/192: write d1/d3/d9/f11 [2868094,9133] 0 2024-03-29T04:45:08.168 INFO:tasks.workunit.client.1.smithi152.stdout:4/122: fdatasync d0/d7/f14 0 2024-03-29T04:45:08.170 INFO:tasks.workunit.client.0.smithi142.stdout:5/906: read - df/fa8 zero size 2024-03-29T04:45:08.171 INFO:tasks.workunit.client.0.smithi142.stdout:9/582: dread d0/f32 [0,4194304] 0 2024-03-29T04:45:08.172 INFO:tasks.workunit.client.0.smithi142.stdout:4/713: readlink d0/d45/l4e 0 2024-03-29T04:45:08.172 INFO:tasks.workunit.client.0.smithi142.stdout:7/789: dread d7/d27/d44/db2/db8/d4b/d71/d30/d33/f4d [0,4194304] 0 2024-03-29T04:45:08.173 INFO:tasks.workunit.client.0.smithi142.stdout:2/699: link de/d11/d3d/fd9 de/d11/d23/dd1/ff1 0 2024-03-29T04:45:08.174 INFO:tasks.workunit.client.0.smithi142.stdout:6/671: dread d1c/f49 [0,4194304] 0 2024-03-29T04:45:08.176 INFO:tasks.workunit.client.0.smithi142.stdout:3/569: rename d3/d29/d2d/d6c/l89 to d3/d29/d2b/d40/d67/d8b/lc3 0 2024-03-29T04:45:08.178 INFO:tasks.workunit.client.1.smithi152.stdout:6/193: dwrite d1/fd [0,4194304] 0 2024-03-29T04:45:08.179 INFO:tasks.workunit.client.1.smithi152.stdout:6/194: fsync d1/d3/d9/d3b/f3c 0 2024-03-29T04:45:08.180 INFO:tasks.workunit.client.0.smithi142.stdout:4/714: dread - d0/d45/f60 zero size 2024-03-29T04:45:08.180 INFO:tasks.workunit.client.1.smithi152.stdout:6/195: dread - d1/d3/d9/d13/f3d zero size 2024-03-29T04:45:08.182 INFO:tasks.workunit.client.0.smithi142.stdout:2/700: fsync de/f16 0 2024-03-29T04:45:08.186 INFO:tasks.workunit.client.0.smithi142.stdout:6/672: mknod d1c/d33/d46/d86/cd7 0 2024-03-29T04:45:08.187 INFO:tasks.workunit.client.0.smithi142.stdout:6/673: dread - d1c/d90/d94/fd2 zero size 2024-03-29T04:45:08.187 INFO:tasks.workunit.client.0.smithi142.stdout:6/674: chown d1c/d33/d46/f65 7 1 2024-03-29T04:45:08.188 INFO:tasks.workunit.client.0.smithi142.stdout:0/714: write d7/d9/d54/f64 [4643932,25644] 0 2024-03-29T04:45:08.189 INFO:tasks.workunit.client.0.smithi142.stdout:1/674: write d8/db/dc/d77/d31/d4d/f7a [61712,30390] 0 2024-03-29T04:45:08.190 INFO:tasks.workunit.client.1.smithi152.stdout:6/196: dwrite d1/d3/d9/d13/d28/f34 [0,4194304] 0 2024-03-29T04:45:08.198 INFO:tasks.workunit.client.1.smithi152.stdout:6/197: dwrite d1/d3/d9/f11 [0,4194304] 0 2024-03-29T04:45:08.209 INFO:tasks.workunit.client.1.smithi152.stdout:6/198: dwrite d1/d3/f15 [0,4194304] 0 2024-03-29T04:45:08.210 INFO:tasks.workunit.client.1.smithi152.stdout:6/199: write d1/d3/d9/d3b/f3c [112443,44059] 0 2024-03-29T04:45:08.211 INFO:tasks.workunit.client.1.smithi152.stdout:6/200: dread - d1/d3/d9/d13/f3d zero size 2024-03-29T04:45:08.212 INFO:tasks.workunit.client.1.smithi152.stdout:6/201: read - d1/d3/d9/d13/f33 zero size 2024-03-29T04:45:08.213 INFO:tasks.workunit.client.1.smithi152.stdout:6/202: chown d1/fe 22588960 1 2024-03-29T04:45:08.215 INFO:tasks.workunit.client.1.smithi152.stdout:6/203: mkdir d1/d3/d9/d13/d28/d45 0 2024-03-29T04:45:08.217 INFO:tasks.workunit.client.1.smithi152.stdout:6/204: write d1/fd [4701716,13571] 0 2024-03-29T04:45:08.225 INFO:tasks.workunit.client.1.smithi152.stdout:6/205: write d1/f17 [2702633,41481] 0 2024-03-29T04:45:08.227 INFO:tasks.workunit.client.1.smithi152.stdout:6/206: fsync d1/f17 0 2024-03-29T04:45:08.233 INFO:tasks.workunit.client.1.smithi152.stdout:6/207: dwrite d1/d3/d9/d3b/f3c [0,4194304] 0 2024-03-29T04:45:08.238 INFO:tasks.workunit.client.1.smithi152.stdout:6/208: dread d1/d3/d1a/f1d [0,4194304] 0 2024-03-29T04:45:08.240 INFO:tasks.workunit.client.1.smithi152.stdout:6/209: write d1/d3/d9/f21 [2562769,34678] 0 2024-03-29T04:45:08.242 INFO:tasks.workunit.client.1.smithi152.stdout:6/210: creat d1/d3/d9/d13/f46 x:0 0 0 2024-03-29T04:45:08.244 INFO:tasks.workunit.client.1.smithi152.stdout:6/211: symlink d1/d3/d9/l47 0 2024-03-29T04:45:08.245 INFO:tasks.workunit.client.1.smithi152.stdout:6/212: read d1/d3/d9/d13/f1f [1295013,53786] 0 2024-03-29T04:45:08.251 INFO:tasks.workunit.client.1.smithi152.stdout:6/213: dwrite d1/f17 [0,4194304] 0 2024-03-29T04:45:08.253 INFO:tasks.workunit.client.1.smithi152.stdout:6/214: creat d1/d3/df/f48 x:0 0 0 2024-03-29T04:45:08.281 INFO:tasks.workunit.client.0.smithi142.stdout:0/715: symlink d7/d13/dbf/d72/d37/le9 0 2024-03-29T04:45:08.282 INFO:tasks.workunit.client.0.smithi142.stdout:7/790: creat d7/d27/d44/db2/db8/d4b/d71/f108 x:0 0 0 2024-03-29T04:45:08.285 INFO:tasks.workunit.client.0.smithi142.stdout:5/907: getdents df/d16/d24/d2e 0 2024-03-29T04:45:08.285 INFO:tasks.workunit.client.0.smithi142.stdout:9/583: dread d0/d2/d6/d27/d3c/f4c [0,4194304] 0 2024-03-29T04:45:08.285 INFO:tasks.workunit.client.0.smithi142.stdout:6/675: dwrite d1c/d33/d46/d71/f75 [0,4194304] 0 2024-03-29T04:45:08.287 INFO:tasks.workunit.client.0.smithi142.stdout:1/675: write d8/db/dc/d77/db9/dbe/d2d/d44/d53/fb7 [458897,10673] 0 2024-03-29T04:45:08.291 INFO:tasks.workunit.client.0.smithi142.stdout:0/716: write d7/fe8 [1715939,7812] 0 2024-03-29T04:45:08.293 INFO:tasks.workunit.client.0.smithi142.stdout:6/676: read d1c/f40 [4007819,61294] 0 2024-03-29T04:45:08.294 INFO:tasks.workunit.client.0.smithi142.stdout:3/570: dwrite d3/d29/d8f/d5f/db4/f58 [0,4194304] 0 2024-03-29T04:45:08.296 INFO:tasks.workunit.client.0.smithi142.stdout:4/715: dwrite d0/d9/f49 [0,4194304] 0 2024-03-29T04:45:08.296 INFO:tasks.workunit.client.0.smithi142.stdout:3/571: fdatasync d3/d1f/d59/fb2 0 2024-03-29T04:45:08.297 INFO:tasks.workunit.client.1.smithi152.stdout:3/83: sync 2024-03-29T04:45:08.297 INFO:tasks.workunit.client.1.smithi152.stdout:7/93: sync 2024-03-29T04:45:08.300 INFO:tasks.workunit.client.0.smithi142.stdout:5/908: mkdir df/d16/d24/d3d/d12e 0 2024-03-29T04:45:08.301 INFO:tasks.workunit.client.0.smithi142.stdout:9/584: mknod d0/d2/d6/d27/d3c/d4f/cb9 0 2024-03-29T04:45:08.302 INFO:tasks.workunit.client.0.smithi142.stdout:9/585: chown d0/f4b 382869 1 2024-03-29T04:45:08.302 INFO:tasks.workunit.client.1.smithi152.stdout:3/84: unlink d7/fd 0 2024-03-29T04:45:08.303 INFO:tasks.workunit.client.0.smithi142.stdout:6/677: symlink d1c/d33/d46/d66/ld8 0 2024-03-29T04:45:08.304 INFO:tasks.workunit.client.0.smithi142.stdout:7/791: truncate d7/d27/d44/db2/db8/de/d20/d3f/fa0 231578 0 2024-03-29T04:45:08.304 INFO:tasks.workunit.client.0.smithi142.stdout:4/716: truncate d0/d9/d11/d4c/d5f/f8a 490729 0 2024-03-29T04:45:08.305 INFO:tasks.workunit.client.0.smithi142.stdout:3/572: truncate d3/d29/d8f/d5f/d64/f76 12804 0 2024-03-29T04:45:08.305 INFO:tasks.workunit.client.1.smithi152.stdout:7/94: dwrite d7/d8/db/f19 [0,4194304] 0 2024-03-29T04:45:08.307 INFO:tasks.workunit.client.0.smithi142.stdout:1/676: mkdir d8/db/dc/d77/de9 0 2024-03-29T04:45:08.308 INFO:tasks.workunit.client.1.smithi152.stdout:7/95: write d7/d8/db/df/f17 [263528,35373] 0 2024-03-29T04:45:08.308 INFO:tasks.workunit.client.0.smithi142.stdout:0/717: truncate d7/d9/d6e/f6f 56092 0 2024-03-29T04:45:08.308 INFO:tasks.workunit.client.1.smithi152.stdout:3/85: creat d7/f15 x:0 0 0 2024-03-29T04:45:08.309 INFO:tasks.workunit.client.1.smithi152.stdout:7/96: readlink d7/d8/lc 0 2024-03-29T04:45:08.311 INFO:tasks.workunit.client.0.smithi142.stdout:7/792: mkdir d7/d27/d44/d109 0 2024-03-29T04:45:08.311 INFO:tasks.workunit.client.1.smithi152.stdout:3/86: unlink d7/fb 0 2024-03-29T04:45:08.314 INFO:tasks.workunit.client.0.smithi142.stdout:3/573: rmdir d3/d29/d2b/d40/d67/d8b 39 2024-03-29T04:45:08.314 INFO:tasks.workunit.client.1.smithi152.stdout:7/97: write d7/d8/db/fe [748416,104334] 0 2024-03-29T04:45:08.315 INFO:tasks.workunit.client.0.smithi142.stdout:8/821: sync 2024-03-29T04:45:08.316 INFO:tasks.workunit.client.1.smithi152.stdout:3/87: mknod d7/c16 0 2024-03-29T04:45:08.316 INFO:tasks.workunit.client.1.smithi152.stdout:3/88: stat d7/ff 0 2024-03-29T04:45:08.319 INFO:tasks.workunit.client.1.smithi152.stdout:7/98: dread d7/f14 [0,4194304] 0 2024-03-29T04:45:08.321 INFO:tasks.workunit.client.0.smithi142.stdout:4/717: dwrite d0/d9/d11/d4c/d5f/d81/dc9/d47/fda [0,4194304] 0 2024-03-29T04:45:08.322 INFO:tasks.workunit.client.0.smithi142.stdout:6/678: creat d1c/d45/fd9 x:0 0 0 2024-03-29T04:45:08.323 INFO:tasks.workunit.client.1.smithi152.stdout:7/99: mknod d7/d8/c1b 0 2024-03-29T04:45:08.324 INFO:tasks.workunit.client.0.smithi142.stdout:5/909: link df/d16/d24/d2e/d47/d4d/c7f df/d16/d24/d2e/d47/d4d/d102/d75/c12f 0 2024-03-29T04:45:08.325 INFO:tasks.workunit.client.0.smithi142.stdout:8/822: chown d5/d2e/l38 4309461 1 2024-03-29T04:45:08.325 INFO:tasks.workunit.client.0.smithi142.stdout:0/718: getdents d7/d13/dbf/dca/dab/ddf 0 2024-03-29T04:45:08.325 INFO:tasks.workunit.client.0.smithi142.stdout:9/586: dwrite d0/d2/d6/d31/f75 [0,4194304] 0 2024-03-29T04:45:08.325 INFO:tasks.workunit.client.1.smithi152.stdout:3/89: dwrite d7/ff [0,4194304] 0 2024-03-29T04:45:08.326 INFO:tasks.workunit.client.1.smithi152.stdout:3/90: chown c6 2822693 1 2024-03-29T04:45:08.327 INFO:tasks.workunit.client.0.smithi142.stdout:4/718: readlink d0/l46 0 2024-03-29T04:45:08.328 INFO:tasks.workunit.client.0.smithi142.stdout:1/677: dwrite d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/fbb [0,4194304] 0 2024-03-29T04:45:08.330 INFO:tasks.workunit.client.0.smithi142.stdout:9/587: fdatasync d0/d12/d50/d8e/fa4 0 2024-03-29T04:45:08.330 INFO:tasks.workunit.client.1.smithi152.stdout:7/100: rmdir d7/d8/db/df 39 2024-03-29T04:45:08.331 INFO:tasks.workunit.client.0.smithi142.stdout:5/910: truncate df/d1a/f2f 8684143 0 2024-03-29T04:45:08.333 INFO:tasks.workunit.client.0.smithi142.stdout:6/679: chown d1c/d33/f67 75 1 2024-03-29T04:45:08.334 INFO:tasks.workunit.client.1.smithi152.stdout:3/91: symlink d7/l17 0 2024-03-29T04:45:08.337 INFO:tasks.workunit.client.1.smithi152.stdout:3/92: creat d7/f18 x:0 0 0 2024-03-29T04:45:08.338 INFO:tasks.workunit.client.0.smithi142.stdout:1/678: unlink d8/db/dc/d77/db9/dbe/d2d/faa 0 2024-03-29T04:45:08.340 INFO:tasks.workunit.client.1.smithi152.stdout:7/101: link d7/f10 d7/d8/d12/f1c 0 2024-03-29T04:45:08.341 INFO:tasks.workunit.client.0.smithi142.stdout:3/574: creat d3/d1f/fc4 x:0 0 0 2024-03-29T04:45:08.342 INFO:tasks.workunit.client.1.smithi152.stdout:3/93: chown d7/c14 55438 1 2024-03-29T04:45:08.342 INFO:tasks.workunit.client.0.smithi142.stdout:2/701: sync 2024-03-29T04:45:08.344 INFO:tasks.workunit.client.1.smithi152.stdout:3/94: mkdir d7/d19 0 2024-03-29T04:45:08.345 INFO:tasks.workunit.client.0.smithi142.stdout:4/719: write d0/d9/d11/d4c/d5f/d81/dc9/d47/f6b [2422373,51233] 0 2024-03-29T04:45:08.346 INFO:tasks.workunit.client.0.smithi142.stdout:9/588: symlink d0/d2/lba 0 2024-03-29T04:45:08.347 INFO:tasks.workunit.client.0.smithi142.stdout:6/680: mkdir d1c/d33/d46/d61/d9e/dda 0 2024-03-29T04:45:08.348 INFO:tasks.workunit.client.1.smithi152.stdout:3/95: creat d7/f1a x:0 0 0 2024-03-29T04:45:08.348 INFO:tasks.workunit.client.1.smithi152.stdout:3/96: write d7/f10 [623527,24547] 0 2024-03-29T04:45:08.350 INFO:tasks.workunit.client.0.smithi142.stdout:5/911: symlink df/d16/d24/d5c/db5/l130 0 2024-03-29T04:45:08.351 INFO:tasks.workunit.client.0.smithi142.stdout:4/720: dread d0/dc/d15/d3d/d65/f95 [0,4194304] 0 2024-03-29T04:45:08.352 INFO:tasks.workunit.client.0.smithi142.stdout:5/912: stat df/d16/l4e 0 2024-03-29T04:45:08.353 INFO:tasks.workunit.client.0.smithi142.stdout:1/679: fsync d8/db/dc/d77/db9/dbe/d2d/d44/d7f/fb0 0 2024-03-29T04:45:08.357 INFO:tasks.workunit.client.0.smithi142.stdout:8/823: getdents d5/d1d/d24/d66 0 2024-03-29T04:45:08.357 INFO:tasks.workunit.client.1.smithi152.stdout:3/97: dwrite d7/f1a [0,4194304] 0 2024-03-29T04:45:08.358 INFO:tasks.workunit.client.1.smithi152.stdout:3/98: truncate d7/f18 610432 0 2024-03-29T04:45:08.359 INFO:tasks.workunit.client.1.smithi152.stdout:3/99: dread - d7/f15 zero size 2024-03-29T04:45:08.359 INFO:tasks.workunit.client.0.smithi142.stdout:6/681: dwrite d1c/d33/d46/d61/d7e/fbd [0,4194304] 0 2024-03-29T04:45:08.359 INFO:tasks.workunit.client.0.smithi142.stdout:9/589: rmdir d0/d2/d6/d27/d3c/d4f 39 2024-03-29T04:45:08.361 INFO:tasks.workunit.client.1.smithi152.stdout:3/100: read d7/f10 [111327,66285] 0 2024-03-29T04:45:08.364 INFO:tasks.workunit.client.1.smithi152.stdout:3/101: write d7/f15 [669582,70512] 0 2024-03-29T04:45:08.367 INFO:tasks.workunit.client.1.smithi152.stdout:7/102: write d7/d8/db/f19 [4706665,7922] 0 2024-03-29T04:45:08.367 INFO:tasks.workunit.client.1.smithi152.stdout:7/103: chown d7/d8/fa 32332 1 2024-03-29T04:45:08.368 INFO:tasks.workunit.client.0.smithi142.stdout:0/719: rmdir d7/dbc 0 2024-03-29T04:45:08.368 INFO:tasks.workunit.client.1.smithi152.stdout:3/102: mkdir d7/d19/d1b 0 2024-03-29T04:45:08.369 INFO:tasks.workunit.client.1.smithi152.stdout:3/103: chown d7/f1a 239263879 1 2024-03-29T04:45:08.369 INFO:tasks.workunit.client.1.smithi152.stdout:7/104: write d7/d8/d18/f1a [65522,124240] 0 2024-03-29T04:45:08.372 INFO:tasks.workunit.client.1.smithi152.stdout:3/104: mkdir d7/d1c 0 2024-03-29T04:45:08.372 INFO:tasks.workunit.client.0.smithi142.stdout:7/793: dread d7/d27/d44/db2/db8/de/d20/ddf/fe1 [0,4194304] 0 2024-03-29T04:45:08.374 INFO:tasks.workunit.client.0.smithi142.stdout:5/913: mknod df/d1a/d28/c131 0 2024-03-29T04:45:08.375 INFO:tasks.workunit.client.0.smithi142.stdout:1/680: fsync d8/db/dc/d77/d31/f61 0 2024-03-29T04:45:08.376 INFO:tasks.workunit.client.1.smithi152.stdout:7/105: dread d7/d8/d12/f1c [0,4194304] 0 2024-03-29T04:45:08.377 INFO:tasks.workunit.client.0.smithi142.stdout:8/824: mkdir d5/d1d/d24/d66/d11e 0 2024-03-29T04:45:08.378 INFO:tasks.workunit.client.1.smithi152.stdout:3/105: dread d7/f10 [0,4194304] 0 2024-03-29T04:45:08.379 INFO:tasks.workunit.client.1.smithi152.stdout:7/106: mkdir d7/d8/d12/d1d 0 2024-03-29T04:45:08.379 INFO:tasks.workunit.client.1.smithi152.stdout:3/106: read d7/f10 [602368,37832] 0 2024-03-29T04:45:08.382 INFO:tasks.workunit.client.0.smithi142.stdout:3/575: dwrite d3/d1f/d24/d47/f75 [0,4194304] 0 2024-03-29T04:45:08.383 INFO:tasks.workunit.client.1.smithi152.stdout:7/107: creat d7/d8/d12/d1d/f1e x:0 0 0 2024-03-29T04:45:08.383 INFO:tasks.workunit.client.0.smithi142.stdout:2/702: dread de/d11/d23/fb7 [0,4194304] 0 2024-03-29T04:45:08.383 INFO:tasks.workunit.client.1.smithi152.stdout:3/107: creat d7/d19/d1b/f1d x:0 0 0 2024-03-29T04:45:08.384 INFO:tasks.workunit.client.0.smithi142.stdout:7/794: creat d7/d27/d44/db2/db8/de/dc1/f10a x:0 0 0 2024-03-29T04:45:08.385 INFO:tasks.workunit.client.0.smithi142.stdout:5/914: rmdir df/d16/d24/d2e 39 2024-03-29T04:45:08.386 INFO:tasks.workunit.client.1.smithi152.stdout:7/108: mknod d7/d8/d12/d1d/c1f 0 2024-03-29T04:45:08.386 INFO:tasks.workunit.client.1.smithi152.stdout:7/109: stat d7/d8/d18 0 2024-03-29T04:45:08.388 INFO:tasks.workunit.client.1.smithi152.stdout:3/108: mkdir d7/d19/d1b/d1e 0 2024-03-29T04:45:08.390 INFO:tasks.workunit.client.1.smithi152.stdout:7/110: read d7/d8/db/df/f17 [1933360,41428] 0 2024-03-29T04:45:08.394 INFO:tasks.workunit.client.0.smithi142.stdout:4/721: dwrite d0/dc/d15/d3d/d65/f7c [4194304,4194304] 0 2024-03-29T04:45:08.395 INFO:tasks.workunit.client.1.smithi152.stdout:7/111: rename d7/f9 to d7/d8/d12/f20 0 2024-03-29T04:45:08.396 INFO:tasks.workunit.client.1.smithi152.stdout:3/109: dwrite d7/d19/d1b/f1d [0,4194304] 0 2024-03-29T04:45:08.396 INFO:tasks.workunit.client.0.smithi142.stdout:4/722: readlink d0/d9/d11/l35 0 2024-03-29T04:45:08.397 INFO:tasks.workunit.client.0.smithi142.stdout:0/720: dread d7/d13/dbf/dca/dab/dc1/dd3/f8b [0,4194304] 0 2024-03-29T04:45:08.399 INFO:tasks.workunit.client.1.smithi152.stdout:7/112: mkdir d7/d8/db/df/d21 0 2024-03-29T04:45:08.401 INFO:tasks.workunit.client.1.smithi152.stdout:3/110: getdents d7/d19/d1b/d1e 0 2024-03-29T04:45:08.402 INFO:tasks.workunit.client.0.smithi142.stdout:8/825: mknod d5/d2e/dd2/de6/c11f 0 2024-03-29T04:45:08.402 INFO:tasks.workunit.client.0.smithi142.stdout:2/703: symlink de/d11/d1e/d22/d27/dba/d61/dcd/lf2 0 2024-03-29T04:45:08.403 INFO:tasks.workunit.client.1.smithi152.stdout:3/111: dread d7/f10 [0,4194304] 0 2024-03-29T04:45:08.403 INFO:tasks.workunit.client.0.smithi142.stdout:4/723: write d0/dc/f88 [3424304,33547] 0 2024-03-29T04:45:08.404 INFO:tasks.workunit.client.1.smithi152.stdout:3/112: chown d7/cc 141 1 2024-03-29T04:45:08.404 INFO:tasks.workunit.client.1.smithi152.stdout:7/113: unlink d7/d8/d12/f1c 0 2024-03-29T04:45:08.404 INFO:tasks.workunit.client.1.smithi152.stdout:3/113: chown d7/c14 0 1 2024-03-29T04:45:08.405 INFO:tasks.workunit.client.0.smithi142.stdout:5/915: rename df/d16/l1b to df/d16/d24/d5c/db1/l132 0 2024-03-29T04:45:08.405 INFO:tasks.workunit.client.1.smithi152.stdout:7/114: chown d7/d8/d12/f20 2411629 1 2024-03-29T04:45:08.407 INFO:tasks.workunit.client.0.smithi142.stdout:9/590: link d0/c19 d0/d2/d6/d27/d3c/d4f/cbb 0 2024-03-29T04:45:08.407 INFO:tasks.workunit.client.1.smithi152.stdout:7/115: truncate d7/d8/d12/d1d/f1e 32870 0 2024-03-29T04:45:08.407 INFO:tasks.workunit.client.1.smithi152.stdout:3/114: mknod d7/d1c/c1f 0 2024-03-29T04:45:08.408 INFO:tasks.workunit.client.0.smithi142.stdout:2/704: dread - de/d11/f87 zero size 2024-03-29T04:45:08.409 INFO:tasks.workunit.client.0.smithi142.stdout:1/681: dread d8/db/dc/d77/d5a/fc6 [0,4194304] 0 2024-03-29T04:45:08.409 INFO:tasks.workunit.client.0.smithi142.stdout:6/682: dread d1c/f6e [0,4194304] 0 2024-03-29T04:45:08.409 INFO:tasks.workunit.client.1.smithi152.stdout:7/116: fsync d7/d8/d12/f20 0 2024-03-29T04:45:08.411 INFO:tasks.workunit.client.1.smithi152.stdout:3/115: symlink d7/d19/l20 0 2024-03-29T04:45:08.412 INFO:tasks.workunit.client.0.smithi142.stdout:9/591: stat d0/d2/d6/d31/f75 0 2024-03-29T04:45:08.412 INFO:tasks.workunit.client.0.smithi142.stdout:7/795: dwrite d7/d27/d44/fd7 [0,4194304] 0 2024-03-29T04:45:08.413 INFO:tasks.workunit.client.0.smithi142.stdout:5/916: fsync df/d16/d24/d2e/d47/d4d/d102/d75/f129 0 2024-03-29T04:45:08.414 INFO:tasks.workunit.client.0.smithi142.stdout:0/721: creat d7/d9/d54/fea x:0 0 0 2024-03-29T04:45:08.414 INFO:tasks.workunit.client.0.smithi142.stdout:4/724: truncate d0/d45/fad 1685259 0 2024-03-29T04:45:08.416 INFO:tasks.workunit.client.0.smithi142.stdout:6/683: write d1c/d33/db6/d52/f7b [563608,36995] 0 2024-03-29T04:45:08.417 INFO:tasks.workunit.client.0.smithi142.stdout:3/576: write d3/d1f/d59/f6e [807832,128330] 0 2024-03-29T04:45:08.420 INFO:tasks.workunit.client.0.smithi142.stdout:8/826: dwrite d5/d1d/d59/f77 [0,4194304] 0 2024-03-29T04:45:08.423 INFO:tasks.workunit.client.0.smithi142.stdout:8/827: write d5/d1d/dd9/f117 [554143,68498] 0 2024-03-29T04:45:08.423 INFO:tasks.workunit.client.1.smithi152.stdout:3/116: dwrite d7/f15 [0,4194304] 0 2024-03-29T04:45:08.423 INFO:tasks.workunit.client.1.smithi152.stdout:7/117: dwrite d7/f14 [0,4194304] 0 2024-03-29T04:45:08.427 INFO:tasks.workunit.client.0.smithi142.stdout:3/577: dwrite d3/d1f/d24/d47/d87/fa9 [0,4194304] 0 2024-03-29T04:45:08.429 INFO:tasks.workunit.client.0.smithi142.stdout:5/917: rmdir df/d16/d24 39 2024-03-29T04:45:08.430 INFO:tasks.workunit.client.1.smithi152.stdout:7/118: creat d7/d8/d12/f22 x:0 0 0 2024-03-29T04:45:08.431 INFO:tasks.workunit.client.1.smithi152.stdout:7/119: readlink d7/d8/lc 0 2024-03-29T04:45:08.432 INFO:tasks.workunit.client.0.smithi142.stdout:3/578: fsync d3/d29/d8f/d5f/db4/f58 0 2024-03-29T04:45:08.432 INFO:tasks.workunit.client.0.smithi142.stdout:1/682: mkdir d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/dea 0 2024-03-29T04:45:08.433 INFO:tasks.workunit.client.1.smithi152.stdout:3/117: dwrite d7/f15 [0,4194304] 0 2024-03-29T04:45:08.433 INFO:tasks.workunit.client.0.smithi142.stdout:7/796: mkdir d7/d27/d44/db2/db8/d4b/d71/d30/d33/d10b 0 2024-03-29T04:45:08.435 INFO:tasks.workunit.client.1.smithi152.stdout:7/120: symlink d7/d8/d18/l23 0 2024-03-29T04:45:08.435 INFO:tasks.workunit.client.0.smithi142.stdout:6/684: chown d1c/l82 114987979 1 2024-03-29T04:45:08.436 INFO:tasks.workunit.client.0.smithi142.stdout:8/828: dwrite d5/d93/fc9 [0,4194304] 0 2024-03-29T04:45:08.441 INFO:tasks.workunit.client.0.smithi142.stdout:2/705: creat de/d11/d1e/d22/ff3 x:0 0 0 2024-03-29T04:45:08.441 INFO:tasks.workunit.client.0.smithi142.stdout:2/706: fdatasync de/d11/d17/d5d/fe5 0 2024-03-29T04:45:08.443 INFO:tasks.workunit.client.0.smithi142.stdout:3/579: mkdir d3/d29/d8f/d5f/dc5 0 2024-03-29T04:45:08.443 INFO:tasks.workunit.client.1.smithi152.stdout:3/118: dwrite d7/ff [4194304,4194304] 0 2024-03-29T04:45:08.445 INFO:tasks.workunit.client.0.smithi142.stdout:6/685: creat d1c/d33/d46/d86/fdb x:0 0 0 2024-03-29T04:45:08.445 INFO:tasks.workunit.client.0.smithi142.stdout:3/580: readlink d3/d1f/d24/d47/l55 0 2024-03-29T04:45:08.446 INFO:tasks.workunit.client.0.smithi142.stdout:7/797: mkdir d7/d27/d44/db2/db8/d4b/d71/d30/d33/d10c 0 2024-03-29T04:45:08.446 INFO:tasks.workunit.client.1.smithi152.stdout:7/121: dwrite d7/d8/db/fe [0,4194304] 0 2024-03-29T04:45:08.447 INFO:tasks.workunit.client.0.smithi142.stdout:3/581: write d3/d29/d2d/d56/f9a [2853212,118658] 0 2024-03-29T04:45:08.447 INFO:tasks.workunit.client.1.smithi152.stdout:3/119: creat d7/d19/d1b/f21 x:0 0 0 2024-03-29T04:45:08.448 INFO:tasks.workunit.client.0.smithi142.stdout:2/707: read de/d11/d23/f28 [36711,76153] 0 2024-03-29T04:45:08.448 INFO:tasks.workunit.client.1.smithi152.stdout:5/78: sync 2024-03-29T04:45:08.449 INFO:tasks.workunit.client.1.smithi152.stdout:7/122: write d7/d8/d12/f22 [267321,129578] 0 2024-03-29T04:45:08.449 INFO:tasks.workunit.client.1.smithi152.stdout:5/79: rename d6 to d6/dd/d18 22 2024-03-29T04:45:08.450 INFO:tasks.workunit.client.1.smithi152.stdout:7/123: chown d7/d8/db 1237088 1 2024-03-29T04:45:08.452 INFO:tasks.workunit.client.1.smithi152.stdout:3/120: mkdir d7/d19/d1b/d1e/d22 0 2024-03-29T04:45:08.453 INFO:tasks.workunit.client.0.smithi142.stdout:3/582: symlink d3/d29/d8f/d95/lc6 0 2024-03-29T04:45:08.454 INFO:tasks.workunit.client.1.smithi152.stdout:3/121: fdatasync d7/d19/d1b/f21 0 2024-03-29T04:45:08.454 INFO:tasks.workunit.client.1.smithi152.stdout:3/122: chown c6 675 1 2024-03-29T04:45:08.455 INFO:tasks.workunit.client.1.smithi152.stdout:3/123: read d7/f15 [1723557,109146] 0 2024-03-29T04:45:08.456 INFO:tasks.workunit.client.0.smithi142.stdout:5/918: getdents df/d1a/d21/d27 0 2024-03-29T04:45:08.456 INFO:tasks.workunit.client.1.smithi152.stdout:3/124: chown d7/l13 26247 1 2024-03-29T04:45:08.456 INFO:tasks.workunit.client.1.smithi152.stdout:5/80: rename d6/d7/l12 to d6/dd/l19 0 2024-03-29T04:45:08.456 INFO:tasks.workunit.client.1.smithi152.stdout:5/81: readlink l4 0 2024-03-29T04:45:08.456 INFO:tasks.workunit.client.1.smithi152.stdout:3/125: dread - d7/d19/d1b/f21 zero size 2024-03-29T04:45:08.457 INFO:tasks.workunit.client.0.smithi142.stdout:3/583: truncate d3/d1f/f7e 717797 0 2024-03-29T04:45:08.458 INFO:tasks.workunit.client.0.smithi142.stdout:3/584: chown d3/d1f/d59/cb0 945844 1 2024-03-29T04:45:08.459 INFO:tasks.workunit.client.1.smithi152.stdout:5/82: write d6/d7/fa [2483864,74654] 0 2024-03-29T04:45:08.460 INFO:tasks.workunit.client.0.smithi142.stdout:7/798: getdents d7/d27/d44/db2/db8/de/d20/d86/dd4/ddb 0 2024-03-29T04:45:08.463 INFO:tasks.workunit.client.0.smithi142.stdout:5/919: dwrite df/d1a/ffe [0,4194304] 0 2024-03-29T04:45:08.466 INFO:tasks.workunit.client.0.smithi142.stdout:5/920: stat df/d1a/l91 0 2024-03-29T04:45:08.466 INFO:tasks.workunit.client.1.smithi152.stdout:5/83: getdents d6 0 2024-03-29T04:45:08.468 INFO:tasks.workunit.client.1.smithi152.stdout:5/84: stat f0 0 2024-03-29T04:45:08.468 INFO:tasks.workunit.client.1.smithi152.stdout:5/85: stat d6/d7 0 2024-03-29T04:45:08.469 INFO:tasks.workunit.client.1.smithi152.stdout:3/126: dwrite d7/d19/d1b/f1d [0,4194304] 0 2024-03-29T04:45:08.474 INFO:tasks.workunit.client.1.smithi152.stdout:3/127: creat d7/d19/d1b/f23 x:0 0 0 2024-03-29T04:45:08.479 INFO:tasks.workunit.client.1.smithi152.stdout:3/128: mknod d7/c24 0 2024-03-29T04:45:08.479 INFO:tasks.workunit.client.0.smithi142.stdout:5/921: dwrite df/d16/d24/d2e/d47/d4d/d102/d75/f129 [0,4194304] 0 2024-03-29T04:45:08.481 INFO:tasks.workunit.client.0.smithi142.stdout:5/922: dread - df/d16/d24/d5c/db1/ff2 zero size 2024-03-29T04:45:08.482 INFO:tasks.workunit.client.1.smithi152.stdout:3/129: creat d7/d19/f25 x:0 0 0 2024-03-29T04:45:08.485 INFO:tasks.workunit.client.0.smithi142.stdout:9/592: write d0/d2/d6/d27/d3c/f69 [317221,39466] 0 2024-03-29T04:45:08.487 INFO:tasks.workunit.client.0.smithi142.stdout:4/725: write d0/d9/db2/dba/fe6 [3558716,73804] 0 2024-03-29T04:45:08.489 INFO:tasks.workunit.client.1.smithi152.stdout:7/124: truncate d7/d8/db/fe 2465257 0 2024-03-29T04:45:08.490 INFO:tasks.workunit.client.0.smithi142.stdout:4/726: readlink d0/dc/l76 0 2024-03-29T04:45:08.494 INFO:tasks.workunit.client.0.smithi142.stdout:4/727: fdatasync d0/dc/f33 0 2024-03-29T04:45:08.496 INFO:tasks.workunit.client.0.smithi142.stdout:0/722: dwrite d7/d9/d55/d42/f46 [0,4194304] 0 2024-03-29T04:45:08.499 INFO:tasks.workunit.client.0.smithi142.stdout:9/593: dwrite d0/d2/d6/f9c [0,4194304] 0 2024-03-29T04:45:08.499 INFO:tasks.workunit.client.0.smithi142.stdout:4/728: write d0/d9/d11/d84/fca [391090,85074] 0 2024-03-29T04:45:08.501 INFO:tasks.workunit.client.0.smithi142.stdout:9/594: write d0/d2/d6/d31/f75 [1448528,43378] 0 2024-03-29T04:45:08.507 INFO:tasks.workunit.client.1.smithi152.stdout:3/130: fsync d7/d19/d1b/f1d 0 2024-03-29T04:45:08.507 INFO:tasks.workunit.client.0.smithi142.stdout:0/723: rename d7/d9/d54/d58/f82 to d7/d13/dbf/feb 0 2024-03-29T04:45:08.511 INFO:tasks.workunit.client.0.smithi142.stdout:0/724: creat d7/d9/d6e/fec x:0 0 0 2024-03-29T04:45:08.511 INFO:tasks.workunit.client.0.smithi142.stdout:9/595: dwrite d0/d2/d6/f9c [0,4194304] 0 2024-03-29T04:45:08.512 INFO:tasks.workunit.client.0.smithi142.stdout:1/683: write d8/db/dc/d77/db9/dbe/d2d/d44/d7f/fb0 [248519,2845] 0 2024-03-29T04:45:08.513 INFO:tasks.workunit.client.1.smithi152.stdout:3/131: dread d7/d19/d1b/f1d [0,4194304] 0 2024-03-29T04:45:08.517 INFO:tasks.workunit.client.1.smithi152.stdout:3/132: unlink d7/l17 0 2024-03-29T04:45:08.517 INFO:tasks.workunit.client.1.smithi152.stdout:7/125: dwrite d7/d8/db/fe [0,4194304] 0 2024-03-29T04:45:08.518 INFO:tasks.workunit.client.0.smithi142.stdout:6/686: write d1c/d33/d46/d71/f60 [1204303,37146] 0 2024-03-29T04:45:08.519 INFO:tasks.workunit.client.1.smithi152.stdout:3/133: write d7/d19/d1b/f1d [4288537,111307] 0 2024-03-29T04:45:08.520 INFO:tasks.workunit.client.1.smithi152.stdout:5/86: dread d6/d7/fa [0,4194304] 0 2024-03-29T04:45:08.520 INFO:tasks.workunit.client.1.smithi152.stdout:3/134: truncate d7/d19/f25 326804 0 2024-03-29T04:45:08.523 INFO:tasks.workunit.client.1.smithi152.stdout:3/135: chown d7/l13 1509426 1 2024-03-29T04:45:08.523 INFO:tasks.workunit.client.1.smithi152.stdout:5/87: write f0 [1654097,122001] 0 2024-03-29T04:45:08.523 INFO:tasks.workunit.client.1.smithi152.stdout:3/136: stat d7/f15 0 2024-03-29T04:45:08.523 INFO:tasks.workunit.client.0.smithi142.stdout:1/684: rename d8/db/dc/d77/db9/dbe/d2d/d44/d80/f88 to d8/db/dc/d77/db9/dbe/d2d/d8b/da4/dd1/feb 0 2024-03-29T04:45:08.524 INFO:tasks.workunit.client.1.smithi152.stdout:3/137: write d7/f18 [1187134,75824] 0 2024-03-29T04:45:08.526 INFO:tasks.workunit.client.1.smithi152.stdout:3/138: write d7/ff [2427452,75378] 0 2024-03-29T04:45:08.527 INFO:tasks.workunit.client.0.smithi142.stdout:9/596: creat d0/d12/fbc x:0 0 0 2024-03-29T04:45:08.530 INFO:tasks.workunit.client.0.smithi142.stdout:0/725: creat d7/d9/d54/fed x:0 0 0 2024-03-29T04:45:08.531 INFO:tasks.workunit.client.1.smithi152.stdout:5/88: link d6/d7/l13 d6/dd/d15/l1a 0 2024-03-29T04:45:08.535 INFO:tasks.workunit.client.1.smithi152.stdout:3/139: dwrite d7/d19/d1b/f23 [0,4194304] 0 2024-03-29T04:45:08.535 INFO:tasks.workunit.client.0.smithi142.stdout:6/687: mkdir d1c/d90/ddc 0 2024-03-29T04:45:08.539 INFO:tasks.workunit.client.0.smithi142.stdout:9/597: fdatasync d0/d2/d6/d16/daa/f87 0 2024-03-29T04:45:08.539 INFO:tasks.workunit.client.0.smithi142.stdout:3/585: dwrite d3/d29/d2b/d40/d67/d8b/d6/f2f [0,4194304] 0 2024-03-29T04:45:08.539 INFO:tasks.workunit.client.0.smithi142.stdout:2/708: dwrite de/d11/d17/d60/de6/fef [0,4194304] 0 2024-03-29T04:45:08.540 INFO:tasks.workunit.client.0.smithi142.stdout:8/829: dwrite d5/f22 [4194304,4194304] 0 2024-03-29T04:45:08.541 INFO:tasks.workunit.client.0.smithi142.stdout:7/799: dwrite f1 [0,4194304] 0 2024-03-29T04:45:08.541 INFO:tasks.workunit.client.0.smithi142.stdout:5/923: truncate df/d1a/d21/fcb 545377 0 2024-03-29T04:45:08.542 INFO:tasks.workunit.client.1.smithi152.stdout:3/140: creat d7/d19/f26 x:0 0 0 2024-03-29T04:45:08.543 INFO:tasks.workunit.client.0.smithi142.stdout:0/726: write d7/d9/d55/f87 [986342,88346] 0 2024-03-29T04:45:08.545 INFO:tasks.workunit.client.0.smithi142.stdout:1/685: mknod d8/db/dc/d77/db9/dbe/d15/cec 0 2024-03-29T04:45:08.547 INFO:tasks.workunit.client.1.smithi152.stdout:5/89: rename d6/dd/d15 to d6/d1b 0 2024-03-29T04:45:08.548 INFO:tasks.workunit.client.1.smithi152.stdout:3/141: rmdir d7/d19/d1b/d1e 39 2024-03-29T04:45:08.550 INFO:tasks.workunit.client.0.smithi142.stdout:4/729: write d0/d12/f3b [2555477,52311] 0 2024-03-29T04:45:08.554 INFO:tasks.workunit.client.1.smithi152.stdout:5/90: dread f0 [0,4194304] 0 2024-03-29T04:45:08.556 INFO:tasks.workunit.client.0.smithi142.stdout:4/730: write d0/d9/d11/d4c/d5f/d81/dc9/d47/f4b [8262476,34156] 0 2024-03-29T04:45:08.557 INFO:tasks.workunit.client.0.smithi142.stdout:1/686: mkdir d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/ded 0 2024-03-29T04:45:08.557 INFO:tasks.workunit.client.1.smithi152.stdout:5/91: write f0 [2670967,124710] 0 2024-03-29T04:45:08.560 INFO:tasks.workunit.client.0.smithi142.stdout:9/598: rename d0/d2/d6/d27/d3c/d4f/l8a to d0/d2/d6/d16/d6a/lbd 0 2024-03-29T04:45:08.561 INFO:tasks.workunit.client.0.smithi142.stdout:0/727: dwrite d7/d9/d55/fdb [0,4194304] 0 2024-03-29T04:45:08.561 INFO:tasks.workunit.client.0.smithi142.stdout:4/731: chown d0/dc/d15/d3d/d65/f95 42666101 1 2024-03-29T04:45:08.561 INFO:tasks.workunit.client.0.smithi142.stdout:5/924: symlink df/d29/d7a/d97/def/l133 0 2024-03-29T04:45:08.562 INFO:tasks.workunit.client.0.smithi142.stdout:7/800: symlink d7/d27/d44/db2/db8/d4b/d71/d30/d72/l10d 0 2024-03-29T04:45:08.567 INFO:tasks.workunit.client.0.smithi142.stdout:0/728: unlink d7/d13/dbf/fc8 0 2024-03-29T04:45:08.567 INFO:tasks.workunit.client.0.smithi142.stdout:5/925: rmdir df/d29/dbc 39 2024-03-29T04:45:08.568 INFO:tasks.workunit.client.0.smithi142.stdout:5/926: read fb [60743,111088] 0 2024-03-29T04:45:08.571 INFO:tasks.workunit.client.0.smithi142.stdout:9/599: rename d0/d12/d50/d52/c70 to d0/d2/d61/cbe 0 2024-03-29T04:45:08.573 INFO:tasks.workunit.client.0.smithi142.stdout:4/732: dwrite d0/d9/d11/d97/fe1 [0,4194304] 0 2024-03-29T04:45:08.574 INFO:tasks.workunit.client.0.smithi142.stdout:7/801: mkdir d7/d27/d44/db2/db8/d4b/d71/d30/dd9/d10e 0 2024-03-29T04:45:08.575 INFO:tasks.workunit.client.0.smithi142.stdout:0/729: mknod d7/d13/cee 0 2024-03-29T04:45:08.576 INFO:tasks.workunit.client.0.smithi142.stdout:9/600: write d0/d2/d6/d27/d3c/f72 [524969,48588] 0 2024-03-29T04:45:08.576 INFO:tasks.workunit.client.0.smithi142.stdout:5/927: rename df/d29/d7a/d97/le1 to df/d16/d24/d2e/d47/d4d/d102/d75/l134 0 2024-03-29T04:45:08.580 INFO:tasks.workunit.client.0.smithi142.stdout:9/601: creat d0/d2/d39/fbf x:0 0 0 2024-03-29T04:45:08.581 INFO:tasks.workunit.client.0.smithi142.stdout:8/830: dread d5/f1f [0,4194304] 0 2024-03-29T04:45:08.581 INFO:tasks.workunit.client.0.smithi142.stdout:0/730: mknod d7/d9/d55/d42/cef 0 2024-03-29T04:45:08.582 INFO:tasks.workunit.client.0.smithi142.stdout:5/928: symlink df/d16/d24/d2e/d47/d8c/de0/l135 0 2024-03-29T04:45:08.583 INFO:tasks.workunit.client.0.smithi142.stdout:0/731: write d7/fe8 [1352064,126614] 0 2024-03-29T04:45:08.584 INFO:tasks.workunit.client.0.smithi142.stdout:7/802: truncate d7/d27/d44/db2/db8/d4b/d71/d30/d33/df5/ffc 2610953 0 2024-03-29T04:45:08.584 INFO:tasks.workunit.client.0.smithi142.stdout:0/732: chown d7/d13/dbf/d72/d37/f43 3 1 2024-03-29T04:45:08.586 INFO:tasks.workunit.client.0.smithi142.stdout:8/831: unlink d5/d2e/dd2/l7b 0 2024-03-29T04:45:08.588 INFO:tasks.workunit.client.0.smithi142.stdout:9/602: creat d0/d2/d6/d16/fc0 x:0 0 0 2024-03-29T04:45:08.589 INFO:tasks.workunit.client.0.smithi142.stdout:9/603: write d0/f62 [1230866,108973] 0 2024-03-29T04:45:08.589 INFO:tasks.workunit.client.0.smithi142.stdout:7/803: truncate d7/d27/d44/db2/db8/d4b/f9a 4996472 0 2024-03-29T04:45:08.590 INFO:tasks.workunit.client.0.smithi142.stdout:8/832: mkdir d5/d1d/dd9/d120 0 2024-03-29T04:45:08.591 INFO:tasks.workunit.client.1.smithi152.stdout:3/142: rmdir d7/d19 39 2024-03-29T04:45:08.594 INFO:tasks.workunit.client.1.smithi152.stdout:5/92: symlink d6/d7/l1c 0 2024-03-29T04:45:08.594 INFO:tasks.workunit.client.0.smithi142.stdout:7/804: dread - d7/d27/d44/db2/db8/de/d20/d3f/fc7 zero size 2024-03-29T04:45:08.595 INFO:tasks.workunit.client.1.smithi152.stdout:5/93: dread - f3 zero size 2024-03-29T04:45:08.595 INFO:tasks.workunit.client.0.smithi142.stdout:8/833: mkdir d5/d2e/dd2/d60/d121 0 2024-03-29T04:45:08.596 INFO:tasks.workunit.client.0.smithi142.stdout:9/604: dwrite d0/d12/d50/f90 [0,4194304] 0 2024-03-29T04:45:08.596 INFO:tasks.workunit.client.0.smithi142.stdout:0/733: getdents d7/d13/dbf/dca/dab/dc1/dd3 0 2024-03-29T04:45:08.597 INFO:tasks.workunit.client.1.smithi152.stdout:5/94: mkdir d6/d1d 0 2024-03-29T04:45:08.598 INFO:tasks.workunit.client.0.smithi142.stdout:9/605: fdatasync d0/d12/d50/d52/da3/fb1 0 2024-03-29T04:45:08.598 INFO:tasks.workunit.client.0.smithi142.stdout:7/805: fsync d7/d27/fa8 0 2024-03-29T04:45:08.599 INFO:tasks.workunit.client.1.smithi152.stdout:5/95: write d6/dd/fe [854959,86068] 0 2024-03-29T04:45:08.602 INFO:tasks.workunit.client.0.smithi142.stdout:8/834: mknod d5/d2e/dd2/c122 0 2024-03-29T04:45:08.603 INFO:tasks.workunit.client.1.smithi152.stdout:5/96: rename f3 to d6/d1d/f1e 0 2024-03-29T04:45:08.605 INFO:tasks.workunit.client.1.smithi152.stdout:5/97: truncate d6/d7/f8 1058132 0 2024-03-29T04:45:08.606 INFO:tasks.workunit.client.0.smithi142.stdout:7/806: mkdir d7/d27/d44/d4c/da3/d10f 0 2024-03-29T04:45:08.606 INFO:tasks.workunit.client.0.smithi142.stdout:0/734: read d7/d13/dbf/d72/fd6 [765326,84220] 0 2024-03-29T04:45:08.607 INFO:tasks.workunit.client.0.smithi142.stdout:7/807: chown d7/d27/d44/db2/db8/d4b/fd0 204 1 2024-03-29T04:45:08.607 INFO:tasks.workunit.client.1.smithi152.stdout:5/98: write d6/d1d/f1e [607289,77348] 0 2024-03-29T04:45:08.608 INFO:tasks.workunit.client.1.smithi152.stdout:5/99: readlink l4 0 2024-03-29T04:45:08.608 INFO:tasks.workunit.client.1.smithi152.stdout:5/100: chown f0 843481 1 2024-03-29T04:45:08.609 INFO:tasks.workunit.client.0.smithi142.stdout:6/688: write d1c/d45/fb7 [3936212,128458] 0 2024-03-29T04:45:08.611 INFO:tasks.workunit.client.1.smithi152.stdout:5/101: mknod d6/dd/c1f 0 2024-03-29T04:45:08.613 INFO:tasks.workunit.client.0.smithi142.stdout:7/808: creat d7/d27/d44/f110 x:0 0 0 2024-03-29T04:45:08.613 INFO:tasks.workunit.client.0.smithi142.stdout:8/835: dread d5/d1d/d59/fef [0,4194304] 0 2024-03-29T04:45:08.614 INFO:tasks.workunit.client.1.smithi152.stdout:5/102: dread d6/dd/f17 [0,4194304] 0 2024-03-29T04:45:08.614 INFO:tasks.workunit.client.0.smithi142.stdout:6/689: creat d1c/d33/d46/d71/fdd x:0 0 0 2024-03-29T04:45:08.616 INFO:tasks.workunit.client.0.smithi142.stdout:6/690: chown d1c/d33/d46/f65 364 1 2024-03-29T04:45:08.618 INFO:tasks.workunit.client.0.smithi142.stdout:8/836: symlink d5/d1d/d2f/d45/dc3/l123 0 2024-03-29T04:45:08.618 INFO:tasks.workunit.client.0.smithi142.stdout:2/709: dwrite de/d11/d1e/d22/faf [0,4194304] 0 2024-03-29T04:45:08.618 INFO:tasks.workunit.client.1.smithi152.stdout:5/103: dread d6/d1d/f1e [0,4194304] 0 2024-03-29T04:45:08.618 INFO:tasks.workunit.client.1.smithi152.stdout:5/104: read f0 [471072,35386] 0 2024-03-29T04:45:08.621 INFO:tasks.workunit.client.0.smithi142.stdout:1/687: write d8/db/dc/d77/db9/dbe/d2d/f3e [2048720,55483] 0 2024-03-29T04:45:08.625 INFO:tasks.workunit.client.0.smithi142.stdout:8/837: creat d5/d2e/dd2/d60/d121/f124 x:0 0 0 2024-03-29T04:45:08.626 INFO:tasks.workunit.client.0.smithi142.stdout:8/838: chown d5/d1d/d81 0 1 2024-03-29T04:45:08.627 INFO:tasks.workunit.client.1.smithi152.stdout:5/105: dwrite d6/d1d/f1e [0,4194304] 0 2024-03-29T04:45:08.628 INFO:tasks.workunit.client.0.smithi142.stdout:7/809: dwrite d7/d27/d44/db2/db8/de/f34 [0,4194304] 0 2024-03-29T04:45:08.629 INFO:tasks.workunit.client.0.smithi142.stdout:8/839: write d5/d2e/dd2/d60/ff7 [104783,8286] 0 2024-03-29T04:45:08.630 INFO:tasks.workunit.client.0.smithi142.stdout:1/688: rename d8/db/dc/d77/db9/dbe/d1f/d50/c7e to d8/db/dc/d77/db9/dbe/d1f/d97/cee 0 2024-03-29T04:45:08.631 INFO:tasks.workunit.client.1.smithi152.stdout:5/106: rename c2 to d6/d16/c20 0 2024-03-29T04:45:08.634 INFO:tasks.workunit.client.0.smithi142.stdout:1/689: chown d8/db/dc/d77/db9/dbe/d1f/c40 81643 1 2024-03-29T04:45:08.634 INFO:tasks.workunit.client.0.smithi142.stdout:8/840: unlink d5/l8 0 2024-03-29T04:45:08.635 INFO:tasks.workunit.client.0.smithi142.stdout:1/690: chown d8/db/dc/d77/db9/dbe/d2d/d8b 57912 1 2024-03-29T04:45:08.636 INFO:tasks.workunit.client.0.smithi142.stdout:8/841: write d5/d2e/dd2/d60/ffe [445387,27813] 0 2024-03-29T04:45:08.642 INFO:tasks.workunit.client.1.smithi152.stdout:5/107: dwrite d6/f14 [0,4194304] 0 2024-03-29T04:45:08.643 INFO:tasks.workunit.client.0.smithi142.stdout:2/710: dread de/d11/d23/f46 [0,4194304] 0 2024-03-29T04:45:08.644 INFO:tasks.workunit.client.0.smithi142.stdout:8/842: creat d5/d2e/dd2/f125 x:0 0 0 2024-03-29T04:45:08.645 INFO:tasks.workunit.client.0.smithi142.stdout:2/711: read de/d11/d1e/d22/faf [24840,70546] 0 2024-03-29T04:45:08.652 INFO:tasks.workunit.client.0.smithi142.stdout:8/843: creat d5/d1d/d2f/d76/d3c/d70/de9/f126 x:0 0 0 2024-03-29T04:45:08.653 INFO:tasks.workunit.client.0.smithi142.stdout:9/606: dwrite d0/f4b [0,4194304] 0 2024-03-29T04:45:08.657 INFO:tasks.workunit.client.0.smithi142.stdout:0/735: dwrite d7/d13/f78 [0,4194304] 0 2024-03-29T04:45:08.662 INFO:tasks.workunit.client.0.smithi142.stdout:0/736: chown d7/d9/d55 3 1 2024-03-29T04:45:08.664 INFO:tasks.workunit.client.0.smithi142.stdout:2/712: getdents de/d11/d1e/d22/d27/dba 0 2024-03-29T04:45:08.665 INFO:tasks.workunit.client.0.smithi142.stdout:8/844: creat d5/d1d/d2f/d76/d3c/d70/de9/f127 x:0 0 0 2024-03-29T04:45:08.666 INFO:tasks.workunit.client.0.smithi142.stdout:7/810: dread d7/d27/d44/db2/db8/d4b/d71/f9d [0,4194304] 0 2024-03-29T04:45:08.669 INFO:tasks.workunit.client.0.smithi142.stdout:6/691: dwrite d1c/f3c [0,4194304] 0 2024-03-29T04:45:08.670 INFO:tasks.workunit.client.0.smithi142.stdout:0/737: truncate d7/d13/dbf/dca/dab/dc1/fde 30554 0 2024-03-29T04:45:08.673 INFO:tasks.workunit.client.0.smithi142.stdout:8/845: creat d5/f128 x:0 0 0 2024-03-29T04:45:08.674 INFO:tasks.workunit.client.0.smithi142.stdout:6/692: mknod d1c/d33/d46/d61/d9e/cde 0 2024-03-29T04:45:08.675 INFO:tasks.workunit.client.0.smithi142.stdout:8/846: write d5/d1d/f85 [1993926,34869] 0 2024-03-29T04:45:08.676 INFO:tasks.workunit.client.0.smithi142.stdout:0/738: creat d7/d13/dbf/dca/ff0 x:0 0 0 2024-03-29T04:45:08.680 INFO:tasks.workunit.client.0.smithi142.stdout:8/847: fdatasync d5/d1d/d2f/d76/d2d/de2/fe3 0 2024-03-29T04:45:08.681 INFO:tasks.workunit.client.0.smithi142.stdout:6/693: write d1c/d45/f76 [714396,54847] 0 2024-03-29T04:45:08.682 INFO:tasks.workunit.client.1.smithi152.stdout:5/108: dread f5 [0,4194304] 0 2024-03-29T04:45:08.682 INFO:tasks.workunit.client.0.smithi142.stdout:7/811: dwrite d7/d27/d44/f110 [0,4194304] 0 2024-03-29T04:45:08.682 INFO:tasks.workunit.client.0.smithi142.stdout:8/848: write d5/f25 [1604249,74669] 0 2024-03-29T04:45:08.684 INFO:tasks.workunit.client.1.smithi152.stdout:8/95: sync 2024-03-29T04:45:08.684 INFO:tasks.workunit.client.1.smithi152.stdout:0/139: sync 2024-03-29T04:45:08.686 INFO:tasks.workunit.client.1.smithi152.stdout:8/96: write d1/d2/d3/f17 [50628,122044] 0 2024-03-29T04:45:08.687 INFO:tasks.workunit.client.1.smithi152.stdout:0/140: mknod d2/d29/c32 0 2024-03-29T04:45:08.689 INFO:tasks.workunit.client.1.smithi152.stdout:8/97: mkdir d1/d2/d1d 0 2024-03-29T04:45:08.690 INFO:tasks.workunit.client.1.smithi152.stdout:0/141: mknod d2/da/df/c33 0 2024-03-29T04:45:08.691 INFO:tasks.workunit.client.0.smithi142.stdout:6/694: dread d1c/f31 [0,4194304] 0 2024-03-29T04:45:08.691 INFO:tasks.workunit.client.0.smithi142.stdout:8/849: rename d5/d93/fc9 to d5/d1d/d2f/d45/f129 0 2024-03-29T04:45:08.692 INFO:tasks.workunit.client.0.smithi142.stdout:8/850: write d5/d2e/dd2/d60/f116 [551736,81158] 0 2024-03-29T04:45:08.693 INFO:tasks.workunit.client.1.smithi152.stdout:8/98: mknod d1/d2/c1e 0 2024-03-29T04:45:08.694 INFO:tasks.workunit.client.1.smithi152.stdout:5/109: dwrite d6/dd/f17 [0,4194304] 0 2024-03-29T04:45:08.699 INFO:tasks.workunit.client.1.smithi152.stdout:0/142: rename d2/da/df/l25 to d2/l34 0 2024-03-29T04:45:08.699 INFO:tasks.workunit.client.1.smithi152.stdout:8/99: fdatasync d1/d2/fc 0 2024-03-29T04:45:08.700 INFO:tasks.workunit.client.0.smithi142.stdout:8/851: creat d5/d1d/d24/db4/f12a x:0 0 0 2024-03-29T04:45:08.702 INFO:tasks.workunit.client.0.smithi142.stdout:8/852: readlink d5/l3f 0 2024-03-29T04:45:08.702 INFO:tasks.workunit.client.0.smithi142.stdout:2/713: dwrite de/d11/d2b/d3e/f76 [0,4194304] 0 2024-03-29T04:45:08.706 INFO:tasks.workunit.client.0.smithi142.stdout:6/695: truncate d1c/f39 603411 0 2024-03-29T04:45:08.708 INFO:tasks.workunit.client.1.smithi152.stdout:0/143: dwrite d2/f9 [4194304,4194304] 0 2024-03-29T04:45:08.708 INFO:tasks.workunit.client.0.smithi142.stdout:8/853: mkdir d5/d1d/d2f/d76/d2d/de2/d12b 0 2024-03-29T04:45:08.709 INFO:tasks.workunit.client.1.smithi152.stdout:0/144: fdatasync d2/da/fb 0 2024-03-29T04:45:08.710 INFO:tasks.workunit.client.1.smithi152.stdout:5/110: link d6/f14 d6/f21 0 2024-03-29T04:45:08.710 INFO:tasks.workunit.client.0.smithi142.stdout:2/714: mkdir de/d11/d1e/df4 0 2024-03-29T04:45:08.711 INFO:tasks.workunit.client.1.smithi152.stdout:8/100: creat d1/d2/d1d/f1f x:0 0 0 2024-03-29T04:45:08.711 INFO:tasks.workunit.client.1.smithi152.stdout:0/145: read d2/f8 [7770633,107958] 0 2024-03-29T04:45:08.712 INFO:tasks.workunit.client.0.smithi142.stdout:2/715: chown de/d11/d23/dd1/ff0 4137985 1 2024-03-29T04:45:08.714 INFO:tasks.workunit.client.1.smithi152.stdout:8/101: mknod d1/d2/c20 0 2024-03-29T04:45:08.718 INFO:tasks.workunit.client.0.smithi142.stdout:2/716: chown de/d11/d17/f69 365 1 2024-03-29T04:45:08.718 INFO:tasks.workunit.client.1.smithi152.stdout:5/111: creat d6/f22 x:0 0 0 2024-03-29T04:45:08.718 INFO:tasks.workunit.client.0.smithi142.stdout:6/696: link d1c/d33/d46/d61/l7c d1c/d33/db6/ldf 0 2024-03-29T04:45:08.720 INFO:tasks.workunit.client.0.smithi142.stdout:2/717: chown de/d11/d17/d60/le3 818 1 2024-03-29T04:45:08.720 INFO:tasks.workunit.client.0.smithi142.stdout:7/812: truncate d7/d27/d44/db2/db8/de/d20/d86/dd4/ddb/df6/f102 2387115 0 2024-03-29T04:45:08.722 INFO:tasks.workunit.client.1.smithi152.stdout:5/112: creat d6/f23 x:0 0 0 2024-03-29T04:45:08.739 INFO:tasks.workunit.client.0.smithi142.stdout:8/854: rename d5/d1d/d2f/d76/d2d/de2 to d5/d1d/d2f/d5f/d12c 0 2024-03-29T04:45:08.742 INFO:tasks.workunit.client.1.smithi152.stdout:0/146: dread d2/da/fb [0,4194304] 0 2024-03-29T04:45:08.743 INFO:tasks.workunit.client.0.smithi142.stdout:2/718: rename de/d11/d1e/f72 to de/d11/d17/d60/dc4/ff5 0 2024-03-29T04:45:08.744 INFO:tasks.workunit.client.1.smithi152.stdout:0/147: creat d2/d31/f35 x:0 0 0 2024-03-29T04:45:08.744 INFO:tasks.workunit.client.0.smithi142.stdout:8/855: mkdir d5/d2e/dd2/d60/d12d 0 2024-03-29T04:45:08.745 INFO:tasks.workunit.client.1.smithi152.stdout:0/148: write d2/f4 [2927176,26514] 0 2024-03-29T04:45:08.745 INFO:tasks.workunit.client.0.smithi142.stdout:8/856: dread - d5/d2e/f113 zero size 2024-03-29T04:45:08.746 INFO:tasks.workunit.client.0.smithi142.stdout:2/719: truncate de/d11/d1e/d22/d27/fa9 739297 0 2024-03-29T04:45:08.747 INFO:tasks.workunit.client.1.smithi152.stdout:5/113: rmdir d6 39 2024-03-29T04:45:08.751 INFO:tasks.workunit.client.1.smithi152.stdout:0/149: dwrite d2/f1f [0,4194304] 0 2024-03-29T04:45:08.751 INFO:tasks.workunit.client.1.smithi152.stdout:5/114: truncate d6/dd/f11 1829565 0 2024-03-29T04:45:08.751 INFO:tasks.workunit.client.1.smithi152.stdout:0/150: fdatasync d2/f7 0 2024-03-29T04:45:08.755 INFO:tasks.workunit.client.1.smithi152.stdout:5/115: rename d6/d7/l1c to d6/d1b/l24 0 2024-03-29T04:45:08.756 INFO:tasks.workunit.client.1.smithi152.stdout:5/116: rmdir d6/d1b 39 2024-03-29T04:45:08.758 INFO:tasks.workunit.client.1.smithi152.stdout:5/117: mkdir d6/d16/d25 0 2024-03-29T04:45:08.760 INFO:tasks.workunit.client.1.smithi152.stdout:5/118: creat d6/f26 x:0 0 0 2024-03-29T04:45:08.774 INFO:tasks.workunit.client.1.smithi152.stdout:5/119: write d6/f14 [1342457,68427] 0 2024-03-29T04:45:08.776 INFO:tasks.workunit.client.1.smithi152.stdout:5/120: creat d6/dd/f27 x:0 0 0 2024-03-29T04:45:08.777 INFO:tasks.workunit.client.1.smithi152.stdout:5/121: write d6/f23 [757165,13280] 0 2024-03-29T04:45:08.778 INFO:tasks.workunit.client.1.smithi152.stdout:5/122: rmdir d6/d1b 39 2024-03-29T04:45:08.780 INFO:tasks.workunit.client.1.smithi152.stdout:5/123: mknod d6/d7/c28 0 2024-03-29T04:45:08.784 INFO:tasks.workunit.client.1.smithi152.stdout:5/124: dwrite d6/d7/f8 [0,4194304] 0 2024-03-29T04:45:08.788 INFO:tasks.workunit.client.1.smithi152.stdout:5/125: symlink d6/d1b/l29 0 2024-03-29T04:45:08.793 INFO:tasks.workunit.client.1.smithi152.stdout:5/126: link d6/d7/l13 d6/d16/d25/l2a 0 2024-03-29T04:45:08.798 INFO:tasks.workunit.client.1.smithi152.stdout:5/127: rmdir d6/dd 39 2024-03-29T04:45:08.799 INFO:tasks.workunit.client.1.smithi152.stdout:5/128: read d6/dd/f17 [469131,109603] 0 2024-03-29T04:45:08.803 INFO:tasks.workunit.client.1.smithi152.stdout:5/129: mknod d6/d16/d25/c2b 0 2024-03-29T04:45:08.803 INFO:tasks.workunit.client.1.smithi152.stdout:5/130: write d6/f23 [1038166,110101] 0 2024-03-29T04:45:08.805 INFO:tasks.workunit.client.1.smithi152.stdout:5/131: rmdir d6/d16/d25 39 2024-03-29T04:45:08.805 INFO:tasks.workunit.client.1.smithi152.stdout:5/132: chown d6/d1b/l29 4090 1 2024-03-29T04:45:08.806 INFO:tasks.workunit.client.1.smithi152.stdout:5/133: stat d6/dd/f27 0 2024-03-29T04:45:08.809 INFO:tasks.workunit.client.1.smithi152.stdout:5/134: unlink d6/d1d/f1e 0 2024-03-29T04:45:08.812 INFO:tasks.workunit.client.1.smithi152.stdout:5/135: rename f0 to d6/d1d/f2c 0 2024-03-29T04:45:08.814 INFO:tasks.workunit.client.1.smithi152.stdout:5/136: creat d6/d7/f2d x:0 0 0 2024-03-29T04:45:08.814 INFO:tasks.workunit.client.1.smithi152.stdout:5/137: stat d6/d7/fa 0 2024-03-29T04:45:08.816 INFO:tasks.workunit.client.1.smithi152.stdout:5/138: mkdir d6/d16/d25/d2e 0 2024-03-29T04:45:08.817 INFO:tasks.workunit.client.1.smithi152.stdout:5/139: readlink d6/d1b/l29 0 2024-03-29T04:45:08.821 INFO:tasks.workunit.client.1.smithi152.stdout:5/140: creat d6/d1b/f2f x:0 0 0 2024-03-29T04:45:08.822 INFO:tasks.workunit.client.0.smithi142.stdout:8/857: stat d5/d1d/d59/c87 0 2024-03-29T04:45:08.822 INFO:tasks.workunit.client.0.smithi142.stdout:6/697: dread d1c/d45/f4e [0,4194304] 0 2024-03-29T04:45:08.822 INFO:tasks.workunit.client.0.smithi142.stdout:6/698: chown d1c/dc7 11218 1 2024-03-29T04:45:08.822 INFO:tasks.workunit.client.0.smithi142.stdout:8/858: chown d5/d2e/dd2/d60/f7a 77 1 2024-03-29T04:45:08.822 INFO:tasks.workunit.client.0.smithi142.stdout:6/699: readlink l15 0 2024-03-29T04:45:08.823 INFO:tasks.workunit.client.1.smithi152.stdout:5/141: dread d6/dd/f17 [0,4194304] 0 2024-03-29T04:45:08.823 INFO:tasks.workunit.client.0.smithi142.stdout:8/859: symlink d5/d1d/d2f/d76/l12e 0 2024-03-29T04:45:08.823 INFO:tasks.workunit.client.1.smithi152.stdout:5/142: mkdir d6/d1d/d30 0 2024-03-29T04:45:08.826 INFO:tasks.workunit.client.0.smithi142.stdout:6/700: getdents d1c/d33/d46/d71 0 2024-03-29T04:45:08.828 INFO:tasks.workunit.client.1.smithi152.stdout:5/143: link d6/dd/fe d6/d1b/f31 0 2024-03-29T04:45:08.831 INFO:tasks.workunit.client.0.smithi142.stdout:8/860: dwrite d5/d1d/d2f/d5f/d12c/fe3 [0,4194304] 0 2024-03-29T04:45:08.834 INFO:tasks.workunit.client.1.smithi152.stdout:5/144: dread d6/dd/fe [0,4194304] 0 2024-03-29T04:45:08.834 INFO:tasks.workunit.client.0.smithi142.stdout:6/701: write d1c/d33/db6/f81 [851348,80764] 0 2024-03-29T04:45:08.835 INFO:tasks.workunit.client.0.smithi142.stdout:6/702: dread - d1c/f21 zero size 2024-03-29T04:45:08.838 INFO:tasks.workunit.client.1.smithi152.stdout:5/145: mknod d6/d1d/d30/c32 0 2024-03-29T04:45:08.839 INFO:tasks.workunit.client.0.smithi142.stdout:2/720: dread de/d11/d17/d4a/d79/da8/f9a [0,4194304] 0 2024-03-29T04:45:08.839 INFO:tasks.workunit.client.0.smithi142.stdout:2/721: dread - de/d11/f87 zero size 2024-03-29T04:45:08.841 INFO:tasks.workunit.client.1.smithi152.stdout:5/146: symlink d6/d1d/d30/l33 0 2024-03-29T04:45:08.841 INFO:tasks.workunit.client.1.smithi152.stdout:5/147: chown d6/d1d 1786152442 1 2024-03-29T04:45:08.842 INFO:tasks.workunit.client.0.smithi142.stdout:2/722: write de/d11/d1e/d22/d50/fdb [481519,25616] 0 2024-03-29T04:45:08.847 INFO:tasks.workunit.client.1.smithi152.stdout:5/148: dwrite d6/f23 [0,4194304] 0 2024-03-29T04:45:08.848 INFO:tasks.workunit.client.0.smithi142.stdout:2/723: dwrite de/d11/d17/d4a/f55 [0,4194304] 0 2024-03-29T04:45:08.851 INFO:tasks.workunit.client.1.smithi152.stdout:5/149: write d6/dd/f17 [4801231,81493] 0 2024-03-29T04:45:08.852 INFO:tasks.workunit.client.0.smithi142.stdout:2/724: dread de/d11/d23/fb1 [0,4194304] 0 2024-03-29T04:45:08.853 INFO:tasks.workunit.client.1.smithi152.stdout:5/150: dread d6/dd/fe [0,4194304] 0 2024-03-29T04:45:08.856 INFO:tasks.workunit.client.1.smithi152.stdout:5/151: chown d6/d1d/f2c 295 1 2024-03-29T04:45:08.857 INFO:tasks.workunit.client.0.smithi142.stdout:2/725: mkdir de/d11/d2b/d3e/df6 0 2024-03-29T04:45:08.859 INFO:tasks.workunit.client.0.smithi142.stdout:2/726: symlink de/d11/d1e/d84/lf7 0 2024-03-29T04:45:08.861 INFO:tasks.workunit.client.1.smithi152.stdout:5/152: rename d6/c9 to d6/c34 0 2024-03-29T04:45:08.865 INFO:tasks.workunit.client.0.smithi142.stdout:8/861: dread d5/d2e/dd2/d60/fb3 [0,4194304] 0 2024-03-29T04:45:08.865 INFO:tasks.workunit.client.1.smithi152.stdout:5/153: dread d6/dd/f17 [0,4194304] 0 2024-03-29T04:45:08.866 INFO:tasks.workunit.client.1.smithi152.stdout:5/154: dread - d6/dd/f27 zero size 2024-03-29T04:45:08.868 INFO:tasks.workunit.client.1.smithi152.stdout:5/155: creat d6/dd/f35 x:0 0 0 2024-03-29T04:45:08.870 INFO:tasks.workunit.client.1.smithi152.stdout:5/156: dread d6/d1b/f31 [0,4194304] 0 2024-03-29T04:45:08.872 INFO:tasks.workunit.client.1.smithi152.stdout:5/157: mkdir d6/d1b/d36 0 2024-03-29T04:45:08.875 INFO:tasks.workunit.client.1.smithi152.stdout:5/158: unlink d6/d1b/f2f 0 2024-03-29T04:45:08.877 INFO:tasks.workunit.client.0.smithi142.stdout:2/727: dwrite de/d11/d1e/d22/d50/f62 [0,4194304] 0 2024-03-29T04:45:08.880 INFO:tasks.workunit.client.1.smithi152.stdout:9/216: sync 2024-03-29T04:45:08.881 INFO:tasks.workunit.client.1.smithi152.stdout:5/159: dwrite d6/d7/f2d [0,4194304] 0 2024-03-29T04:45:08.881 INFO:tasks.workunit.client.1.smithi152.stdout:9/217: fdatasync d5/d8/f42 0 2024-03-29T04:45:08.884 INFO:tasks.workunit.client.1.smithi152.stdout:9/218: dread d5/db/f25 [0,4194304] 0 2024-03-29T04:45:08.885 INFO:tasks.workunit.client.1.smithi152.stdout:9/219: fdatasync d5/db/f21 0 2024-03-29T04:45:08.885 INFO:tasks.workunit.client.0.smithi142.stdout:2/728: dwrite de/f9d [0,4194304] 0 2024-03-29T04:45:08.887 INFO:tasks.workunit.client.1.smithi152.stdout:5/160: dread d6/f23 [0,4194304] 0 2024-03-29T04:45:08.887 INFO:tasks.workunit.client.1.smithi152.stdout:9/220: creat d5/db/d1b/f4d x:0 0 0 2024-03-29T04:45:08.890 INFO:tasks.workunit.client.1.smithi152.stdout:9/221: creat d5/db/f4e x:0 0 0 2024-03-29T04:45:08.890 INFO:tasks.workunit.client.1.smithi152.stdout:9/222: readlink d5/d8/l14 0 2024-03-29T04:45:08.892 INFO:tasks.workunit.client.0.smithi142.stdout:3/586: sync 2024-03-29T04:45:08.893 INFO:tasks.workunit.client.1.smithi152.stdout:5/161: symlink d6/d16/d25/d2e/l37 0 2024-03-29T04:45:08.895 INFO:tasks.workunit.client.0.smithi142.stdout:2/729: creat de/d11/d17/d60/ff8 x:0 0 0 2024-03-29T04:45:08.897 INFO:tasks.workunit.client.1.smithi152.stdout:5/162: write f5 [2947149,111017] 0 2024-03-29T04:45:08.900 INFO:tasks.workunit.client.1.smithi152.stdout:5/163: creat d6/d1d/d30/f38 x:0 0 0 2024-03-29T04:45:08.901 INFO:tasks.workunit.client.0.smithi142.stdout:3/587: dwrite d3/d29/d2d/f3f [0,4194304] 0 2024-03-29T04:45:08.904 INFO:tasks.workunit.client.0.smithi142.stdout:3/588: mkdir d3/d1f/d59/dc7 0 2024-03-29T04:45:08.904 INFO:tasks.workunit.client.1.smithi152.stdout:5/164: rename d6/d7/fa to d6/d1b/d36/f39 0 2024-03-29T04:45:08.905 INFO:tasks.workunit.client.1.smithi152.stdout:5/165: dread - d6/dd/f27 zero size 2024-03-29T04:45:08.907 INFO:tasks.workunit.client.0.smithi142.stdout:3/589: creat d3/d29/d8f/d5f/d64/fc8 x:0 0 0 2024-03-29T04:45:08.987 INFO:tasks.workunit.client.0.smithi142.stdout:3/590: mknod d3/d1f/d24/d47/d87/cc9 0 2024-03-29T04:45:08.987 INFO:tasks.workunit.client.0.smithi142.stdout:2/730: dread de/d11/d1e/d22/d50/fc3 [0,4194304] 0 2024-03-29T04:45:08.988 INFO:tasks.workunit.client.0.smithi142.stdout:3/591: mknod d3/d1f/d24/d47/cca 0 2024-03-29T04:45:08.990 INFO:tasks.workunit.client.1.smithi152.stdout:5/166: creat d6/d16/d25/f3a x:0 0 0 2024-03-29T04:45:08.990 INFO:tasks.workunit.client.1.smithi152.stdout:5/167: fsync f5 0 2024-03-29T04:45:08.992 INFO:tasks.workunit.client.0.smithi142.stdout:2/731: rename de/d11/d1e/d22/d27/l54 to de/d11/d2b/d7b/lf9 0 2024-03-29T04:45:08.992 INFO:tasks.workunit.client.1.smithi152.stdout:9/223: truncate d5/db/f4e 270588 0 2024-03-29T04:45:08.994 INFO:tasks.workunit.client.0.smithi142.stdout:2/732: creat de/d11/d2b/d3e/ffa x:0 0 0 2024-03-29T04:45:08.994 INFO:tasks.workunit.client.0.smithi142.stdout:2/733: chown de 1 1 2024-03-29T04:45:08.995 INFO:tasks.workunit.client.1.smithi152.stdout:9/224: creat d5/db/d1b/f4f x:0 0 0 2024-03-29T04:45:08.996 INFO:tasks.workunit.client.1.smithi152.stdout:5/168: rename d6/d1b/d36 to d6/d1d/d3b 0 2024-03-29T04:45:08.997 INFO:tasks.workunit.client.1.smithi152.stdout:9/225: rename d5/d12/d2c/f45 to d5/d12/d2c/f50 0 2024-03-29T04:45:08.999 INFO:tasks.workunit.client.1.smithi152.stdout:9/226: creat d5/db/d1b/f51 x:0 0 0 2024-03-29T04:45:09.000 INFO:tasks.workunit.client.0.smithi142.stdout:3/592: truncate d3/d29/d2b/d40/d67/d8b/f4e 2138622 0 2024-03-29T04:45:09.001 INFO:tasks.workunit.client.1.smithi152.stdout:5/169: getdents d6/d1d/d3b 0 2024-03-29T04:45:09.002 INFO:tasks.workunit.client.0.smithi142.stdout:2/734: write de/d11/d1e/d22/d27/dba/d61/d7d/f7e [4034987,84758] 0 2024-03-29T04:45:09.004 INFO:tasks.workunit.client.0.smithi142.stdout:2/735: write de/d11/d17/d5d/f74 [605877,100542] 0 2024-03-29T04:45:09.004 INFO:tasks.workunit.client.1.smithi152.stdout:5/170: fsync d6/d1d/f2c 0 2024-03-29T04:45:09.006 INFO:tasks.workunit.client.0.smithi142.stdout:2/736: getdents de/d11/da1/de2 0 2024-03-29T04:45:09.011 INFO:tasks.workunit.client.0.smithi142.stdout:2/737: dread de/d11/d1e/d22/d50/fdb [0,4194304] 0 2024-03-29T04:45:09.011 INFO:tasks.workunit.client.1.smithi152.stdout:5/171: truncate d6/dd/f17 3294199 0 2024-03-29T04:45:09.015 INFO:tasks.workunit.client.1.smithi152.stdout:9/227: dwrite d5/db/d1b/f32 [0,4194304] 0 2024-03-29T04:45:09.019 INFO:tasks.workunit.client.1.smithi152.stdout:9/228: mknod d5/d8/d31/c52 0 2024-03-29T04:45:09.020 INFO:tasks.workunit.client.1.smithi152.stdout:5/172: dwrite d6/dd/f11 [0,4194304] 0 2024-03-29T04:45:09.022 INFO:tasks.workunit.client.0.smithi142.stdout:3/593: dread d3/d29/d2b/d40/d67/d8b/d4d/f60 [0,4194304] 0 2024-03-29T04:45:09.023 INFO:tasks.workunit.client.1.smithi152.stdout:9/229: rename d5/db/d1b/f32 to d5/d8/f53 0 2024-03-29T04:45:09.026 INFO:tasks.workunit.client.0.smithi142.stdout:3/594: dread d3/d29/d8f/d5f/d64/f76 [0,4194304] 0 2024-03-29T04:45:09.028 INFO:tasks.workunit.client.1.smithi152.stdout:5/173: rename d6/d7/l13 to d6/d1d/d30/l3c 0 2024-03-29T04:45:09.029 INFO:tasks.workunit.client.0.smithi142.stdout:3/595: symlink d3/d29/d8f/d5f/d64/lcb 0 2024-03-29T04:45:09.036 INFO:tasks.workunit.client.0.smithi142.stdout:3/596: dwrite d3/d29/d8f/d5f/db4/f8d [0,4194304] 0 2024-03-29T04:45:09.036 INFO:tasks.workunit.client.1.smithi152.stdout:9/230: rename d5/d8/f42 to d5/d12/d2c/f54 0 2024-03-29T04:45:09.037 INFO:tasks.workunit.client.1.smithi152.stdout:9/231: chown d5/db/d1b/f37 394212 1 2024-03-29T04:45:09.039 INFO:tasks.workunit.client.0.smithi142.stdout:9/607: sync 2024-03-29T04:45:09.039 INFO:tasks.workunit.client.0.smithi142.stdout:5/929: sync 2024-03-29T04:45:09.039 INFO:tasks.workunit.client.0.smithi142.stdout:4/733: sync 2024-03-29T04:45:09.039 INFO:tasks.workunit.client.0.smithi142.stdout:1/691: sync 2024-03-29T04:45:09.039 INFO:tasks.workunit.client.0.smithi142.stdout:0/739: sync 2024-03-29T04:45:09.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:08 smithi152 ceph-mon[37177]: pgmap v270: 65 pgs: 65 active+clean; 1.4 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 49 MiB/s rd, 186 MiB/s wr, 330 op/s 2024-03-29T04:45:09.040 INFO:tasks.workunit.client.1.smithi152.stdout:5/174: link d6/c34 d6/d1d/d30/c3d 0 2024-03-29T04:45:09.041 INFO:tasks.workunit.client.0.smithi142.stdout:3/597: write d3/d29/f77 [1803847,70176] 0 2024-03-29T04:45:09.041 INFO:tasks.workunit.client.0.smithi142.stdout:0/740: fsync d7/d9/d54/d58/fda 0 2024-03-29T04:45:09.041 INFO:tasks.workunit.client.0.smithi142.stdout:1/692: rmdir d8/db/dc/d77/db9 39 2024-03-29T04:45:09.042 INFO:tasks.workunit.client.1.smithi152.stdout:9/232: truncate d5/db/f25 952918 0 2024-03-29T04:45:09.043 INFO:tasks.workunit.client.0.smithi142.stdout:0/741: readlink d7/d13/dbf/dca/lbe 0 2024-03-29T04:45:09.044 INFO:tasks.workunit.client.0.smithi142.stdout:4/734: mknod d0/d9/db2/cf2 0 2024-03-29T04:45:09.045 INFO:tasks.workunit.client.1.smithi152.stdout:5/175: creat d6/d1d/d30/f3e x:0 0 0 2024-03-29T04:45:09.045 INFO:tasks.workunit.client.1.smithi152.stdout:9/233: write d5/db/d1b/f46 [1124931,115092] 0 2024-03-29T04:45:09.045 INFO:tasks.workunit.client.1.smithi152.stdout:5/176: chown d6/d1b 3 1 2024-03-29T04:45:09.046 INFO:tasks.workunit.client.0.smithi142.stdout:3/598: chown d3/d29/d2b/d40/d67/d8b/c28 1 1 2024-03-29T04:45:09.047 INFO:tasks.workunit.client.1.smithi152.stdout:5/177: read d6/d7/f2d [3772232,104369] 0 2024-03-29T04:45:09.047 INFO:tasks.workunit.client.0.smithi142.stdout:1/693: creat d8/db/dc/d77/d31/d4d/fef x:0 0 0 2024-03-29T04:45:09.048 INFO:tasks.workunit.client.0.smithi142.stdout:4/735: readlink d0/d9/d11/d4c/d5f/d81/dc9/d47/l96 0 2024-03-29T04:45:09.050 INFO:tasks.workunit.client.0.smithi142.stdout:5/930: link df/d1a/c8e df/d1a/c136 0 2024-03-29T04:45:09.050 INFO:tasks.workunit.client.0.smithi142.stdout:1/694: stat d8/f51 0 2024-03-29T04:45:09.051 INFO:tasks.workunit.client.1.smithi152.stdout:5/178: rename d6/f22 to d6/d1d/d3b/f3f 0 2024-03-29T04:45:09.051 INFO:tasks.workunit.client.0.smithi142.stdout:9/608: dwrite d0/d2/d6/d27/f82 [0,4194304] 0 2024-03-29T04:45:09.053 INFO:tasks.workunit.client.1.smithi152.stdout:9/234: link d5/db/d1b/f37 d5/db/d1b/f55 0 2024-03-29T04:45:09.054 INFO:tasks.workunit.client.0.smithi142.stdout:4/736: write d0/d9/d11/d84/fca [862199,20871] 0 2024-03-29T04:45:09.055 INFO:tasks.workunit.client.0.smithi142.stdout:9/609: readlink d0/lb 0 2024-03-29T04:45:09.056 INFO:tasks.workunit.client.0.smithi142.stdout:4/737: stat d0/dc/f88 0 2024-03-29T04:45:09.057 INFO:tasks.workunit.client.1.smithi152.stdout:2/107: sync 2024-03-29T04:45:09.057 INFO:tasks.workunit.client.1.smithi152.stdout:1/102: sync 2024-03-29T04:45:09.057 INFO:tasks.workunit.client.1.smithi152.stdout:5/179: creat d6/d1d/f40 x:0 0 0 2024-03-29T04:45:09.058 INFO:tasks.workunit.client.1.smithi152.stdout:9/235: mknod d5/d12/d2c/c56 0 2024-03-29T04:45:09.059 INFO:tasks.workunit.client.0.smithi142.stdout:4/738: write d0/f48 [722177,129654] 0 2024-03-29T04:45:09.060 INFO:tasks.workunit.client.1.smithi152.stdout:2/108: rmdir d0/d3/df 39 2024-03-29T04:45:09.064 INFO:tasks.workunit.client.1.smithi152.stdout:1/103: mknod d5/d7/d8/d13/d19/d21/c26 0 2024-03-29T04:45:09.065 INFO:tasks.workunit.client.1.smithi152.stdout:5/180: dread d6/d7/f2d [0,4194304] 0 2024-03-29T04:45:09.066 INFO:tasks.workunit.client.1.smithi152.stdout:9/236: mknod d5/d12/d2c/c57 0 2024-03-29T04:45:09.069 INFO:tasks.workunit.client.1.smithi152.stdout:9/237: read d5/db/f21 [3113992,59954] 0 2024-03-29T04:45:09.071 INFO:tasks.workunit.client.1.smithi152.stdout:5/181: creat d6/f41 x:0 0 0 2024-03-29T04:45:09.073 INFO:tasks.workunit.client.1.smithi152.stdout:1/104: rename d5/d7/d8/d13/l1c to d5/d7/d8/d13/d19/l27 0 2024-03-29T04:45:09.074 INFO:tasks.workunit.client.1.smithi152.stdout:1/105: read - d5/d7/f25 zero size 2024-03-29T04:45:09.077 INFO:tasks.workunit.client.1.smithi152.stdout:9/238: rename d5/db/d1b/f3e to d5/db/f58 0 2024-03-29T04:45:09.077 INFO:tasks.workunit.client.1.smithi152.stdout:1/106: fdatasync d5/d7/d8/fb 0 2024-03-29T04:45:09.078 INFO:tasks.workunit.client.1.smithi152.stdout:2/109: dread d0/d3/f5 [0,4194304] 0 2024-03-29T04:45:09.079 INFO:tasks.workunit.client.1.smithi152.stdout:2/110: chown d0/d14/f1a 185587 1 2024-03-29T04:45:09.084 INFO:tasks.workunit.client.1.smithi152.stdout:2/111: creat d0/d3/df/f24 x:0 0 0 2024-03-29T04:45:09.085 INFO:tasks.workunit.client.1.smithi152.stdout:1/107: dwrite d5/d7/d8/fb [0,4194304] 0 2024-03-29T04:45:09.086 INFO:tasks.workunit.client.1.smithi152.stdout:9/239: dwrite d5/db/d1b/f46 [0,4194304] 0 2024-03-29T04:45:09.087 INFO:tasks.workunit.client.1.smithi152.stdout:1/108: truncate d5/f20 2409245 0 2024-03-29T04:45:09.089 INFO:tasks.workunit.client.1.smithi152.stdout:9/240: fsync d5/d12/d2c/f54 0 2024-03-29T04:45:09.091 INFO:tasks.workunit.client.1.smithi152.stdout:1/109: write d5/d7/d9/f10 [5020639,27339] 0 2024-03-29T04:45:09.095 INFO:tasks.workunit.client.1.smithi152.stdout:1/110: dread - d5/d7/d8/f1e zero size 2024-03-29T04:45:09.095 INFO:tasks.workunit.client.1.smithi152.stdout:2/112: dwrite d0/d3/f1d [0,4194304] 0 2024-03-29T04:45:09.097 INFO:tasks.workunit.client.1.smithi152.stdout:2/113: fsync d0/d14/f19 0 2024-03-29T04:45:09.098 INFO:tasks.workunit.client.1.smithi152.stdout:9/241: dread d5/d8/f53 [0,4194304] 0 2024-03-29T04:45:09.102 INFO:tasks.workunit.client.1.smithi152.stdout:9/242: dread d5/db/f4e [0,4194304] 0 2024-03-29T04:45:09.103 INFO:tasks.workunit.client.1.smithi152.stdout:1/111: rename d5/d7/d8/d13/d19/l22 to d5/d7/d1b/l28 0 2024-03-29T04:45:09.107 INFO:tasks.workunit.client.1.smithi152.stdout:2/114: dwrite d0/d14/f19 [0,4194304] 0 2024-03-29T04:45:09.108 INFO:tasks.workunit.client.1.smithi152.stdout:2/115: readlink d0/d14/l21 0 2024-03-29T04:45:09.112 INFO:tasks.workunit.client.1.smithi152.stdout:9/243: dwrite d5/db/d1b/f4f [0,4194304] 0 2024-03-29T04:45:09.112 INFO:tasks.workunit.client.1.smithi152.stdout:2/116: unlink d0/f9 0 2024-03-29T04:45:09.117 INFO:tasks.workunit.client.1.smithi152.stdout:9/244: mkdir d5/d8/d31/d59 0 2024-03-29T04:45:09.117 INFO:tasks.workunit.client.1.smithi152.stdout:1/112: rmdir d5 39 2024-03-29T04:45:09.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:08 smithi142 ceph-mon[29027]: pgmap v270: 65 pgs: 65 active+clean; 1.4 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 49 MiB/s rd, 186 MiB/s wr, 330 op/s 2024-03-29T04:45:09.120 INFO:tasks.workunit.client.1.smithi152.stdout:1/113: write d5/d7/f1f [283489,127023] 0 2024-03-29T04:45:09.122 INFO:tasks.workunit.client.1.smithi152.stdout:9/245: creat d5/db/f5a x:0 0 0 2024-03-29T04:45:09.123 INFO:tasks.workunit.client.0.smithi142.stdout:5/931: rename df/d16/d24/d5c/c89 to df/d16/d24/d2e/d47/d8c/df1/c137 0 2024-03-29T04:45:09.123 INFO:tasks.workunit.client.1.smithi152.stdout:9/246: stat d5/f7 0 2024-03-29T04:45:09.124 INFO:tasks.workunit.client.0.smithi142.stdout:5/932: stat fe 0 2024-03-29T04:45:09.125 INFO:tasks.workunit.client.1.smithi152.stdout:9/247: truncate d5/d8/d31/f4c 436569 0 2024-03-29T04:45:09.125 INFO:tasks.workunit.client.0.smithi142.stdout:3/599: dwrite d3/d29/d8f/d5f/d64/fc8 [0,4194304] 0 2024-03-29T04:45:09.126 INFO:tasks.workunit.client.0.smithi142.stdout:0/742: dwrite d7/d9/fb9 [0,4194304] 0 2024-03-29T04:45:09.126 INFO:tasks.workunit.client.1.smithi152.stdout:9/248: write d5/d8/f9 [4844132,30705] 0 2024-03-29T04:45:09.127 INFO:tasks.workunit.client.1.smithi152.stdout:1/114: dwrite d5/f20 [0,4194304] 0 2024-03-29T04:45:09.132 INFO:tasks.workunit.client.0.smithi142.stdout:5/933: fsync df/d1a/f9e 0 2024-03-29T04:45:09.133 INFO:tasks.workunit.client.0.smithi142.stdout:3/600: mkdir d3/d29/d2d/d79/dcc 0 2024-03-29T04:45:09.134 INFO:tasks.workunit.client.1.smithi152.stdout:1/115: mknod d5/d7/d9/d12/c29 0 2024-03-29T04:45:09.135 INFO:tasks.workunit.client.1.smithi152.stdout:1/116: write d5/d7/d8/f1e [577959,100244] 0 2024-03-29T04:45:09.136 INFO:tasks.workunit.client.0.smithi142.stdout:4/739: symlink d0/d9/d11/d4c/d5f/d81/dc9/d47/d7a/de4/lf3 0 2024-03-29T04:45:09.137 INFO:tasks.workunit.client.0.smithi142.stdout:4/740: fsync d0/d9/d11/d4c/d5f/d81/dc9/d47/f62 0 2024-03-29T04:45:09.138 INFO:tasks.workunit.client.0.smithi142.stdout:4/741: chown d0/d9/d11/d4c/d5f/cb4 436166 1 2024-03-29T04:45:09.139 INFO:tasks.workunit.client.1.smithi152.stdout:1/117: write d5/d7/d8/f1e [325686,43253] 0 2024-03-29T04:45:09.142 INFO:tasks.workunit.client.1.smithi152.stdout:1/118: mkdir d5/d7/d8/d13/d19/d21/d2a 0 2024-03-29T04:45:09.142 INFO:tasks.workunit.client.0.smithi142.stdout:1/695: dread d8/f2a [0,4194304] 0 2024-03-29T04:45:09.144 INFO:tasks.workunit.client.0.smithi142.stdout:1/696: write d8/db/dc/f2c [1664699,129530] 0 2024-03-29T04:45:09.149 INFO:tasks.workunit.client.1.smithi152.stdout:1/119: dwrite d5/d7/f25 [0,4194304] 0 2024-03-29T04:45:09.150 INFO:tasks.workunit.client.1.smithi152.stdout:1/120: write d5/d7/d8/f1e [169261,96678] 0 2024-03-29T04:45:09.153 INFO:tasks.workunit.client.1.smithi152.stdout:1/121: mkdir d5/d7/d9/d2b 0 2024-03-29T04:45:09.154 INFO:tasks.workunit.client.1.smithi152.stdout:1/122: read d5/d7/d8/fb [4126428,109905] 0 2024-03-29T04:45:09.156 INFO:tasks.workunit.client.1.smithi152.stdout:1/123: getdents d5/d7/d9/d2b 0 2024-03-29T04:45:09.160 INFO:tasks.workunit.client.0.smithi142.stdout:7/813: sync 2024-03-29T04:45:09.160 INFO:tasks.workunit.client.0.smithi142.stdout:6/703: sync 2024-03-29T04:45:09.162 INFO:tasks.workunit.client.1.smithi152.stdout:1/124: dwrite d5/d7/d8/f1e [0,4194304] 0 2024-03-29T04:45:09.164 INFO:tasks.workunit.client.1.smithi152.stdout:1/125: mknod d5/d7/d1b/c2c 0 2024-03-29T04:45:09.166 INFO:tasks.workunit.client.1.smithi152.stdout:1/126: creat d5/d7/d8/f2d x:0 0 0 2024-03-29T04:45:09.169 INFO:tasks.workunit.client.1.smithi152.stdout:1/127: rename d5/d7/d8/d13/d19/d21/d2a to d5/d7/d1b/d2e 0 2024-03-29T04:45:09.171 INFO:tasks.workunit.client.1.smithi152.stdout:1/128: rmdir d5/d7/d8/d13 39 2024-03-29T04:45:09.173 INFO:tasks.workunit.client.1.smithi152.stdout:1/129: creat d5/d7/d9/d2b/f2f x:0 0 0 2024-03-29T04:45:09.174 INFO:tasks.workunit.client.1.smithi152.stdout:1/130: fdatasync d5/d7/f25 0 2024-03-29T04:45:09.174 INFO:tasks.workunit.client.1.smithi152.stdout:1/131: chown c1 2352 1 2024-03-29T04:45:09.178 INFO:tasks.workunit.client.1.smithi152.stdout:1/132: creat d5/d7/d8/d13/d19/f30 x:0 0 0 2024-03-29T04:45:09.180 INFO:tasks.workunit.client.0.smithi142.stdout:3/601: creat d3/d29/d2d/d56/fcd x:0 0 0 2024-03-29T04:45:09.182 INFO:tasks.workunit.client.1.smithi152.stdout:1/133: dread d5/d7/f25 [0,4194304] 0 2024-03-29T04:45:09.182 INFO:tasks.workunit.client.0.smithi142.stdout:5/934: mkdir df/d16/d25/d120/d138 0 2024-03-29T04:45:09.183 INFO:tasks.workunit.client.1.smithi152.stdout:1/134: stat d5/ca 0 2024-03-29T04:45:09.183 INFO:tasks.workunit.client.0.smithi142.stdout:0/743: write d7/d9/d1e/d81/fb4 [594984,48618] 0 2024-03-29T04:45:09.184 INFO:tasks.workunit.client.0.smithi142.stdout:4/742: truncate d0/d9/d11/f23 34449 0 2024-03-29T04:45:09.184 INFO:tasks.workunit.client.0.smithi142.stdout:0/744: chown d7/d13/l7e 74 1 2024-03-29T04:45:09.185 INFO:tasks.workunit.client.0.smithi142.stdout:0/745: chown d7/d13/dbf/d72/d37/f56 49878342 1 2024-03-29T04:45:09.186 INFO:tasks.workunit.client.0.smithi142.stdout:1/697: creat d8/db/dc/d19/ff0 x:0 0 0 2024-03-29T04:45:09.186 INFO:tasks.workunit.client.0.smithi142.stdout:0/746: chown d7/d13/d4a/d9a/da7/fd4 19848792 1 2024-03-29T04:45:09.187 INFO:tasks.workunit.client.0.smithi142.stdout:6/704: creat d1c/d45/d8a/fe0 x:0 0 0 2024-03-29T04:45:09.187 INFO:tasks.workunit.client.1.smithi152.stdout:1/135: rename d5/d7/d1b/l28 to d5/d7/d1b/d2e/l31 0 2024-03-29T04:45:09.187 INFO:tasks.workunit.client.0.smithi142.stdout:7/814: chown d7/d27/d44/db2/db8/de/d8d/caa 35866 1 2024-03-29T04:45:09.188 INFO:tasks.workunit.client.0.smithi142.stdout:3/602: creat d3/d29/d2d/d79/fce x:0 0 0 2024-03-29T04:45:09.189 INFO:tasks.workunit.client.0.smithi142.stdout:5/935: unlink df/d1a/d21/d27/lb4 0 2024-03-29T04:45:09.190 INFO:tasks.workunit.client.0.smithi142.stdout:0/747: read d7/d13/dbf/dca/dab/dc1/dd3/fd1 [3717763,4853] 0 2024-03-29T04:45:09.190 INFO:tasks.workunit.client.0.smithi142.stdout:1/698: symlink d8/db/d37/lf1 0 2024-03-29T04:45:09.191 INFO:tasks.workunit.client.1.smithi152.stdout:1/136: unlink d5/d7/d8/d13/d19/l27 0 2024-03-29T04:45:09.192 INFO:tasks.workunit.client.0.smithi142.stdout:3/603: readlink d3/d29/d2b/d40/l66 0 2024-03-29T04:45:09.193 INFO:tasks.workunit.client.1.smithi152.stdout:1/137: symlink d5/d7/d8/d13/l32 0 2024-03-29T04:45:09.194 INFO:tasks.workunit.client.0.smithi142.stdout:4/743: creat d0/d9/d11/d4c/d64/d94/ddd/ff4 x:0 0 0 2024-03-29T04:45:09.195 INFO:tasks.workunit.client.1.smithi152.stdout:1/138: symlink d5/d7/d9/l33 0 2024-03-29T04:45:09.196 INFO:tasks.workunit.client.0.smithi142.stdout:4/744: readlink d0/dc/d15/d91/le2 0 2024-03-29T04:45:09.196 INFO:tasks.workunit.client.0.smithi142.stdout:7/815: dwrite d7/d27/d44/db2/db8/fe6 [0,4194304] 0 2024-03-29T04:45:09.196 INFO:tasks.workunit.client.0.smithi142.stdout:1/699: mknod d8/db/dc/d77/d5a/cf2 0 2024-03-29T04:45:09.197 INFO:tasks.workunit.client.1.smithi152.stdout:1/139: readlink d5/d7/d8/d13/d19/l1a 0 2024-03-29T04:45:09.199 INFO:tasks.workunit.client.1.smithi152.stdout:1/140: write d5/d7/d9/d2b/f2f [449959,18923] 0 2024-03-29T04:45:09.200 INFO:tasks.workunit.client.0.smithi142.stdout:5/936: rename df/d16/d24/d2e/d47/d8c to df/d16/d25/d120/d139 0 2024-03-29T04:45:09.209 INFO:tasks.workunit.client.0.smithi142.stdout:6/705: write d1c/d33/d46/d66/f98 [509108,24418] 0 2024-03-29T04:45:09.209 INFO:tasks.workunit.client.0.smithi142.stdout:7/816: fsync d7/d27/d44/fd7 0 2024-03-29T04:45:09.210 INFO:tasks.workunit.client.1.smithi152.stdout:1/141: write d5/d7/d8/fb [538417,20843] 0 2024-03-29T04:45:09.211 INFO:tasks.workunit.client.1.smithi152.stdout:1/142: chown d5/d7/d9 716565 1 2024-03-29T04:45:09.211 INFO:tasks.workunit.client.0.smithi142.stdout:6/706: stat d1c/d33/db6/da4/ccc 0 2024-03-29T04:45:09.211 INFO:tasks.workunit.client.0.smithi142.stdout:7/817: chown d7/d27/d44/db2/db8/d4b/d71/d30/d33/df5 414159 1 2024-03-29T04:45:09.212 INFO:tasks.workunit.client.0.smithi142.stdout:3/604: dread d3/d29/d2b/d40/d67/d8b/f4e [0,4194304] 0 2024-03-29T04:45:09.215 INFO:tasks.workunit.client.0.smithi142.stdout:5/937: creat df/d16/d25/d120/d139/df1/d116/f13a x:0 0 0 2024-03-29T04:45:09.217 INFO:tasks.workunit.client.0.smithi142.stdout:5/938: read - df/d1a/d21/d55/d66/f108 zero size 2024-03-29T04:45:09.217 INFO:tasks.workunit.client.0.smithi142.stdout:4/745: mknod d0/d9/db2/dba/dbc/d6d/d6e/cf5 0 2024-03-29T04:45:09.218 INFO:tasks.workunit.client.1.smithi152.stdout:1/143: link d5/d7/d1b/d2e/l31 d5/d7/d8/d13/d19/d21/l34 0 2024-03-29T04:45:09.219 INFO:tasks.workunit.client.0.smithi142.stdout:3/605: dwrite d3/d1f/d24/d47/f75 [0,4194304] 0 2024-03-29T04:45:09.220 INFO:tasks.workunit.client.0.smithi142.stdout:8/862: sync 2024-03-29T04:45:09.220 INFO:tasks.workunit.client.0.smithi142.stdout:2/738: sync 2024-03-29T04:45:09.220 INFO:tasks.workunit.client.0.smithi142.stdout:9/610: sync 2024-03-29T04:45:09.220 INFO:tasks.workunit.client.0.smithi142.stdout:1/700: write d8/db/dc/d77/db9/dbe/d2d/d44/d53/f5f [2400336,70317] 0 2024-03-29T04:45:09.221 INFO:tasks.workunit.client.0.smithi142.stdout:6/707: mkdir d1c/de1 0 2024-03-29T04:45:09.223 INFO:tasks.workunit.client.1.smithi152.stdout:1/144: dwrite d5/d7/fe [0,4194304] 0 2024-03-29T04:45:09.225 INFO:tasks.workunit.client.1.smithi152.stdout:7/126: sync 2024-03-29T04:45:09.225 INFO:tasks.workunit.client.1.smithi152.stdout:8/102: sync 2024-03-29T04:45:09.225 INFO:tasks.workunit.client.1.smithi152.stdout:6/215: sync 2024-03-29T04:45:09.225 INFO:tasks.workunit.client.1.smithi152.stdout:4/123: sync 2024-03-29T04:45:09.225 INFO:tasks.workunit.client.1.smithi152.stdout:3/143: sync 2024-03-29T04:45:09.226 INFO:tasks.workunit.client.1.smithi152.stdout:0/151: sync 2024-03-29T04:45:09.226 INFO:tasks.workunit.client.0.smithi142.stdout:4/746: dread d0/d9/d11/d84/fca [0,4194304] 0 2024-03-29T04:45:09.229 INFO:tasks.workunit.client.1.smithi152.stdout:7/127: fsync d7/d8/d12/f22 0 2024-03-29T04:45:09.229 INFO:tasks.workunit.client.0.smithi142.stdout:7/818: write d7/d27/d3a/d92/d97/ff2 [8693651,88330] 0 2024-03-29T04:45:09.230 INFO:tasks.workunit.client.1.smithi152.stdout:6/216: rmdir d1/d3/df 39 2024-03-29T04:45:09.231 INFO:tasks.workunit.client.1.smithi152.stdout:7/128: chown d7/d8/d12/d1d/f1e 0 1 2024-03-29T04:45:09.231 INFO:tasks.workunit.client.1.smithi152.stdout:8/103: readlink d1/d2/d3/l11 0 2024-03-29T04:45:09.231 INFO:tasks.workunit.client.0.smithi142.stdout:7/819: chown d7/d27/d44/db2/db8/de/d20/f98 276298885 1 2024-03-29T04:45:09.231 INFO:tasks.workunit.client.0.smithi142.stdout:5/939: rename df/d29/f67 to df/d1a/d21/d55/d66/f13b 0 2024-03-29T04:45:09.233 INFO:tasks.workunit.client.1.smithi152.stdout:8/104: chown d1/f10 5 1 2024-03-29T04:45:09.233 INFO:tasks.workunit.client.1.smithi152.stdout:1/145: creat d5/d7/d8/d13/f35 x:0 0 0 2024-03-29T04:45:09.233 INFO:tasks.workunit.client.0.smithi142.stdout:9/611: mknod d0/d2/d61/d64/cc1 0 2024-03-29T04:45:09.234 INFO:tasks.workunit.client.1.smithi152.stdout:0/152: creat d2/d31/f36 x:0 0 0 2024-03-29T04:45:09.235 INFO:tasks.workunit.client.0.smithi142.stdout:8/863: symlink d5/d1d/d2f/d76/d9c/l12f 0 2024-03-29T04:45:09.235 INFO:tasks.workunit.client.0.smithi142.stdout:9/612: dread - d0/d2/d6/d31/f71 zero size 2024-03-29T04:45:09.236 INFO:tasks.workunit.client.1.smithi152.stdout:6/217: mkdir d1/d3/d1a/d49 0 2024-03-29T04:45:09.236 INFO:tasks.workunit.client.1.smithi152.stdout:3/144: dread - d7/d19/d1b/f21 zero size 2024-03-29T04:45:09.236 INFO:tasks.workunit.client.0.smithi142.stdout:8/864: fsync d5/d1d/d59/fbe 0 2024-03-29T04:45:09.237 INFO:tasks.workunit.client.0.smithi142.stdout:3/606: creat d3/d29/d2b/d40/db1/fcf x:0 0 0 2024-03-29T04:45:09.237 INFO:tasks.workunit.client.0.smithi142.stdout:9/613: stat d0/d6c/la9 0 2024-03-29T04:45:09.239 INFO:tasks.workunit.client.0.smithi142.stdout:8/865: write d5/d1d/d59/f7c [124874,88278] 0 2024-03-29T04:45:09.240 INFO:tasks.workunit.client.0.smithi142.stdout:1/701: rename d8/db/dc/d77/db9/dbe/d15/d9c/cdd to d8/db/dc/d77/db9/dbe/d2d/d8b/dda/cf3 0 2024-03-29T04:45:09.241 INFO:tasks.workunit.client.1.smithi152.stdout:1/146: rmdir d5/d7/d8/d13/d19 39 2024-03-29T04:45:09.241 INFO:tasks.workunit.client.1.smithi152.stdout:4/124: dwrite d0/d7/de/ff [0,4194304] 0 2024-03-29T04:45:09.242 INFO:tasks.workunit.client.1.smithi152.stdout:8/105: mkdir d1/d21 0 2024-03-29T04:45:09.243 INFO:tasks.workunit.client.0.smithi142.stdout:4/747: dread d0/dc/f68 [0,4194304] 0 2024-03-29T04:45:09.245 INFO:tasks.workunit.client.0.smithi142.stdout:4/748: fsync d0/f21 0 2024-03-29T04:45:09.248 INFO:tasks.workunit.client.1.smithi152.stdout:4/125: mknod d0/c1d 0 2024-03-29T04:45:09.248 INFO:tasks.workunit.client.1.smithi152.stdout:7/129: dread d7/f14 [0,4194304] 0 2024-03-29T04:45:09.249 INFO:tasks.workunit.client.0.smithi142.stdout:4/749: dread - d0/d9/d11/d4c/d5f/d81/ff1 zero size 2024-03-29T04:45:09.250 INFO:tasks.workunit.client.0.smithi142.stdout:3/607: creat d3/fd0 x:0 0 0 2024-03-29T04:45:09.250 INFO:tasks.workunit.client.0.smithi142.stdout:9/614: dwrite d0/d2/d6/d16/fc0 [0,4194304] 0 2024-03-29T04:45:09.251 INFO:tasks.workunit.client.1.smithi152.stdout:0/153: dwrite d2/f4 [0,4194304] 0 2024-03-29T04:45:09.251 INFO:tasks.workunit.client.1.smithi152.stdout:8/106: truncate d1/d2/d1d/f1f 246367 0 2024-03-29T04:45:09.253 INFO:tasks.workunit.client.1.smithi152.stdout:1/147: rmdir d5/d7/d8/d13 39 2024-03-29T04:45:09.253 INFO:tasks.workunit.client.0.smithi142.stdout:6/708: dread d1c/d33/d46/d71/fb3 [0,4194304] 0 2024-03-29T04:45:09.254 INFO:tasks.workunit.client.0.smithi142.stdout:4/750: read d0/dc/d15/d3d/fcd [7856540,89736] 0 2024-03-29T04:45:09.254 INFO:tasks.workunit.client.0.smithi142.stdout:3/608: truncate d3/d29/d2b/f21 2455306 0 2024-03-29T04:45:09.255 INFO:tasks.workunit.client.1.smithi152.stdout:6/218: rename d1/d3/d9/l12 to d1/d3/d1a/d1b/l4a 0 2024-03-29T04:45:09.257 INFO:tasks.workunit.client.0.smithi142.stdout:4/751: chown d0/d9/d11/fde 1259 1 2024-03-29T04:45:09.258 INFO:tasks.workunit.client.0.smithi142.stdout:3/609: read - d3/d29/d2b/d40/d67/fb6 zero size 2024-03-29T04:45:09.258 INFO:tasks.workunit.client.0.smithi142.stdout:6/709: symlink d1c/d45/d8a/le2 0 2024-03-29T04:45:09.260 INFO:tasks.workunit.client.0.smithi142.stdout:9/615: truncate d0/d2/d6/d27/f97 737888 0 2024-03-29T04:45:09.261 INFO:tasks.workunit.client.1.smithi152.stdout:7/130: dread d7/d8/db/f19 [0,4194304] 0 2024-03-29T04:45:09.261 INFO:tasks.workunit.client.1.smithi152.stdout:0/154: dwrite d2/f2e [0,4194304] 0 2024-03-29T04:45:09.263 INFO:tasks.workunit.client.1.smithi152.stdout:8/107: unlink d1/d2/d3/f9 0 2024-03-29T04:45:09.267 INFO:tasks.workunit.client.0.smithi142.stdout:5/940: dwrite df/fa8 [0,4194304] 0 2024-03-29T04:45:09.267 INFO:tasks.workunit.client.1.smithi152.stdout:8/108: write d1/d2/d3/f17 [559762,106005] 0 2024-03-29T04:45:09.268 INFO:tasks.workunit.client.1.smithi152.stdout:3/145: truncate d7/d19/d1b/f1d 151626 0 2024-03-29T04:45:09.268 INFO:tasks.workunit.client.0.smithi142.stdout:3/610: creat d3/d29/d2d/d56/fd1 x:0 0 0 2024-03-29T04:45:09.270 INFO:tasks.workunit.client.1.smithi152.stdout:8/109: dread - d1/d2/f1a zero size 2024-03-29T04:45:09.271 INFO:tasks.workunit.client.0.smithi142.stdout:5/941: write df/d1a/ffe [636884,85792] 0 2024-03-29T04:45:09.272 INFO:tasks.workunit.client.1.smithi152.stdout:7/131: dwrite d7/d8/db/fe [0,4194304] 0 2024-03-29T04:45:09.273 INFO:tasks.workunit.client.1.smithi152.stdout:0/155: creat d2/d29/f37 x:0 0 0 2024-03-29T04:45:09.276 INFO:tasks.workunit.client.1.smithi152.stdout:3/146: read d7/f18 [844643,83558] 0 2024-03-29T04:45:09.276 INFO:tasks.workunit.client.1.smithi152.stdout:8/110: read - d1/f1c zero size 2024-03-29T04:45:09.276 INFO:tasks.workunit.client.1.smithi152.stdout:1/148: symlink d5/d7/d8/l36 0 2024-03-29T04:45:09.277 INFO:tasks.workunit.client.0.smithi142.stdout:1/702: dwrite d8/db/dc/d77/db9/dbe/d2d/d44/d53/f7b [0,4194304] 0 2024-03-29T04:45:09.279 INFO:tasks.workunit.client.0.smithi142.stdout:4/752: write d0/d9/d11/d4c/d5f/f73 [236297,4743] 0 2024-03-29T04:45:09.283 INFO:tasks.workunit.client.1.smithi152.stdout:6/219: dwrite d1/d3/df/f2e [0,4194304] 0 2024-03-29T04:45:09.283 INFO:tasks.workunit.client.0.smithi142.stdout:5/942: creat df/d29/d7a/d97/db3/f13c x:0 0 0 2024-03-29T04:45:09.284 INFO:tasks.workunit.client.0.smithi142.stdout:8/866: dread d5/d1d/d59/fbe [0,4194304] 0 2024-03-29T04:45:09.285 INFO:tasks.workunit.client.0.smithi142.stdout:1/703: stat d8/db/dc/d77/d5a/cf2 0 2024-03-29T04:45:09.286 INFO:tasks.workunit.client.1.smithi152.stdout:7/132: chown d7/d8/d12/d1d/c1f 30 1 2024-03-29T04:45:09.286 INFO:tasks.workunit.client.1.smithi152.stdout:4/126: link d0/f4 d0/d7/f1e 0 2024-03-29T04:45:09.287 INFO:tasks.workunit.client.1.smithi152.stdout:3/147: rmdir d7/d19/d1b 39 2024-03-29T04:45:09.288 INFO:tasks.workunit.client.1.smithi152.stdout:1/149: stat d5/d7/d8/d13/d19/f30 0 2024-03-29T04:45:09.289 INFO:tasks.workunit.client.0.smithi142.stdout:7/820: dwrite d7/d27/d44/db2/db8/d4b/f82 [0,4194304] 0 2024-03-29T04:45:09.292 INFO:tasks.workunit.client.1.smithi152.stdout:6/220: symlink d1/d3/d9/d13/d3e/l4b 0 2024-03-29T04:45:09.292 INFO:tasks.workunit.client.1.smithi152.stdout:6/221: readlink d1/d3/df/l39 0 2024-03-29T04:45:09.293 INFO:tasks.workunit.client.0.smithi142.stdout:5/943: creat df/d1a/d21/d55/d66/f13d x:0 0 0 2024-03-29T04:45:09.293 INFO:tasks.workunit.client.0.smithi142.stdout:6/710: dwrite d1c/d45/d51/f7f [4194304,4194304] 0 2024-03-29T04:45:09.293 INFO:tasks.workunit.client.1.smithi152.stdout:0/156: creat d2/f38 x:0 0 0 2024-03-29T04:45:09.294 INFO:tasks.workunit.client.0.smithi142.stdout:6/711: readlink le 0 2024-03-29T04:45:09.295 INFO:tasks.workunit.client.0.smithi142.stdout:8/867: write d5/d1d/d2f/fc8 [444498,23659] 0 2024-03-29T04:45:09.296 INFO:tasks.workunit.client.1.smithi152.stdout:7/133: rename d7/d8/db/f19 to d7/d8/d18/f24 0 2024-03-29T04:45:09.297 INFO:tasks.workunit.client.0.smithi142.stdout:1/704: stat d8/db/dc/d77/db9/dbe/d2d/d8b/da4/dd1/feb 0 2024-03-29T04:45:09.297 INFO:tasks.workunit.client.0.smithi142.stdout:3/611: link d3/d29/d2d/d56/lb5 d3/d29/d2b/d40/d67/d8b/d4d/ld2 0 2024-03-29T04:45:09.301 INFO:tasks.workunit.client.0.smithi142.stdout:7/821: creat d7/d27/d3a/d92/d97/f111 x:0 0 0 2024-03-29T04:45:09.303 INFO:tasks.workunit.client.1.smithi152.stdout:7/134: write d7/d8/d18/f1a [262571,24514] 0 2024-03-29T04:45:09.304 INFO:tasks.workunit.client.1.smithi152.stdout:6/222: dwrite d1/d3/d9/d13/f33 [0,4194304] 0 2024-03-29T04:45:09.305 INFO:tasks.workunit.client.1.smithi152.stdout:4/127: read d0/d7/f1e [7715203,385] 0 2024-03-29T04:45:09.305 INFO:tasks.workunit.client.0.smithi142.stdout:3/612: readlink d3/d29/d2b/d6a/l9b 0 2024-03-29T04:45:09.307 INFO:tasks.workunit.client.0.smithi142.stdout:8/868: unlink d5/d1d/d24/c4e 0 2024-03-29T04:45:09.307 INFO:tasks.workunit.client.1.smithi152.stdout:4/128: rename d0/d7 to d0/d7/d1f 22 2024-03-29T04:45:09.309 INFO:tasks.workunit.client.1.smithi152.stdout:4/129: dread - d0/f15 zero size 2024-03-29T04:45:09.309 INFO:tasks.workunit.client.0.smithi142.stdout:6/712: read d1c/d33/d46/d61/d9e/fb5 [1963232,66899] 0 2024-03-29T04:45:09.310 INFO:tasks.workunit.client.0.smithi142.stdout:7/822: creat d7/d27/d44/d109/f112 x:0 0 0 2024-03-29T04:45:09.310 INFO:tasks.workunit.client.1.smithi152.stdout:4/130: fsync d0/f12 0 2024-03-29T04:45:09.311 INFO:tasks.workunit.client.1.smithi152.stdout:1/150: unlink c2 0 2024-03-29T04:45:09.315 INFO:tasks.workunit.client.1.smithi152.stdout:0/157: mknod d2/c39 0 2024-03-29T04:45:09.316 INFO:tasks.workunit.client.1.smithi152.stdout:8/111: read d1/d2/fc [769594,1741] 0 2024-03-29T04:45:09.316 INFO:tasks.workunit.client.1.smithi152.stdout:6/223: dwrite d1/d3/d9/f41 [0,4194304] 0 2024-03-29T04:45:09.316 INFO:tasks.workunit.client.1.smithi152.stdout:0/158: readlink d2/l6 0 2024-03-29T04:45:09.317 INFO:tasks.workunit.client.0.smithi142.stdout:3/613: chown d3/d29/d2b/d40/d67/d8b/l3d 5001 1 2024-03-29T04:45:09.318 INFO:tasks.workunit.client.0.smithi142.stdout:1/705: dwrite d8/db/dc/d77/d31/d4d/f7a [0,4194304] 0 2024-03-29T04:45:09.318 INFO:tasks.workunit.client.1.smithi152.stdout:0/159: dread - d2/d29/f2f zero size 2024-03-29T04:45:09.319 INFO:tasks.workunit.client.1.smithi152.stdout:0/160: chown d2/da/d17/l13 155950 1 2024-03-29T04:45:09.320 INFO:tasks.workunit.client.0.smithi142.stdout:6/713: chown d1c/d33/d46/d66/f98 3 1 2024-03-29T04:45:09.320 INFO:tasks.workunit.client.1.smithi152.stdout:0/161: write d2/da/d17/f2b [414901,119622] 0 2024-03-29T04:45:09.322 INFO:tasks.workunit.client.1.smithi152.stdout:0/162: dread - d2/d29/f37 zero size 2024-03-29T04:45:09.323 INFO:tasks.workunit.client.0.smithi142.stdout:5/944: dwrite df/d1a/d21/d55/f81 [4194304,4194304] 0 2024-03-29T04:45:09.327 INFO:tasks.workunit.client.1.smithi152.stdout:4/131: mknod d0/d7/de/c20 0 2024-03-29T04:45:09.328 INFO:tasks.workunit.client.0.smithi142.stdout:1/706: mkdir d8/db/dc/d77/db9/dbe/d2d/d44/d7f/df4 0 2024-03-29T04:45:09.330 INFO:tasks.workunit.client.0.smithi142.stdout:7/823: fsync d7/d27/d44/db2/db8/d4b/d71/dec/fbd 0 2024-03-29T04:45:09.331 INFO:tasks.workunit.client.0.smithi142.stdout:8/869: dwrite d5/d1d/d24/fff [0,4194304] 0 2024-03-29T04:45:09.331 INFO:tasks.workunit.client.0.smithi142.stdout:9/616: dread d0/d2/d61/f55 [0,4194304] 0 2024-03-29T04:45:09.332 INFO:tasks.workunit.client.1.smithi152.stdout:7/135: getdents d7/d8/d12/d15 0 2024-03-29T04:45:09.333 INFO:tasks.workunit.client.0.smithi142.stdout:3/614: mknod d3/d29/d2b/d40/d67/d8b/d4d/d50/cd3 0 2024-03-29T04:45:09.334 INFO:tasks.workunit.client.1.smithi152.stdout:0/163: dwrite d2/d31/f35 [0,4194304] 0 2024-03-29T04:45:09.335 INFO:tasks.workunit.client.0.smithi142.stdout:5/945: symlink df/d1a/d21/da5/l13e 0 2024-03-29T04:45:09.336 INFO:tasks.workunit.client.0.smithi142.stdout:7/824: stat d7/d27/d44/db2/db8/d4b/d71/d30/dd9/d10e 0 2024-03-29T04:45:09.336 INFO:tasks.workunit.client.0.smithi142.stdout:6/714: mknod d1c/ce3 0 2024-03-29T04:45:09.337 INFO:tasks.workunit.client.1.smithi152.stdout:9/249: sync 2024-03-29T04:45:09.337 INFO:tasks.workunit.client.1.smithi152.stdout:2/117: sync 2024-03-29T04:45:09.337 INFO:tasks.workunit.client.1.smithi152.stdout:4/132: creat d0/d16/f21 x:0 0 0 2024-03-29T04:45:09.338 INFO:tasks.workunit.client.0.smithi142.stdout:1/707: mkdir d8/db/dc/d77/db9/dbe/d2d/d44/d53/df5 0 2024-03-29T04:45:09.339 INFO:tasks.workunit.client.0.smithi142.stdout:0/748: sync 2024-03-29T04:45:09.340 INFO:tasks.workunit.client.0.smithi142.stdout:2/739: sync 2024-03-29T04:45:09.340 INFO:tasks.workunit.client.0.smithi142.stdout:8/870: rmdir d5/d2e 39 2024-03-29T04:45:09.340 INFO:tasks.workunit.client.0.smithi142.stdout:5/946: readlink df/d1a/d28/lda 0 2024-03-29T04:45:09.340 INFO:tasks.workunit.client.1.smithi152.stdout:1/151: unlink d5/d7/d8/d13/d19/d21/l34 0 2024-03-29T04:45:09.340 INFO:tasks.workunit.client.0.smithi142.stdout:9/617: write d0/d12/d50/f76 [111577,115454] 0 2024-03-29T04:45:09.342 INFO:tasks.workunit.client.0.smithi142.stdout:6/715: unlink d1c/d33/d46/d71/c64 0 2024-03-29T04:45:09.343 INFO:tasks.workunit.client.1.smithi152.stdout:1/152: write d5/f20 [1949100,83205] 0 2024-03-29T04:45:09.343 INFO:tasks.workunit.client.1.smithi152.stdout:7/136: symlink d7/d8/d12/d1d/l25 0 2024-03-29T04:45:09.343 INFO:tasks.workunit.client.1.smithi152.stdout:5/182: sync 2024-03-29T04:45:09.345 INFO:tasks.workunit.client.1.smithi152.stdout:7/137: chown d7/f14 0 1 2024-03-29T04:45:09.347 INFO:tasks.workunit.client.0.smithi142.stdout:9/618: fsync d0/d12/f86 0 2024-03-29T04:45:09.347 INFO:tasks.workunit.client.1.smithi152.stdout:4/133: dwrite d0/f13 [0,4194304] 0 2024-03-29T04:45:09.348 INFO:tasks.workunit.client.0.smithi142.stdout:0/749: chown d7/d13/f7d 60361709 1 2024-03-29T04:45:09.351 INFO:tasks.workunit.client.1.smithi152.stdout:4/134: write d0/d7/f14 [3927810,36811] 0 2024-03-29T04:45:09.351 INFO:tasks.workunit.client.0.smithi142.stdout:7/825: creat d7/d27/d44/db2/f113 x:0 0 0 2024-03-29T04:45:09.353 INFO:tasks.workunit.client.0.smithi142.stdout:7/826: write d7/d27/d44/db2/db8/de/d20/f21 [1427857,123698] 0 2024-03-29T04:45:09.354 INFO:tasks.workunit.client.1.smithi152.stdout:2/118: rename d0/f12 to d0/d14/d15/f25 0 2024-03-29T04:45:09.356 INFO:tasks.workunit.client.1.smithi152.stdout:2/119: chown d0/d14/f1b 4026 1 2024-03-29T04:45:09.357 INFO:tasks.workunit.client.0.smithi142.stdout:4/753: truncate d0/dc/d15/f72 531767 0 2024-03-29T04:45:09.357 INFO:tasks.workunit.client.1.smithi152.stdout:4/135: write d0/d7/f14 [4488924,76009] 0 2024-03-29T04:45:09.357 INFO:tasks.workunit.client.1.smithi152.stdout:5/183: creat d6/d1b/f42 x:0 0 0 2024-03-29T04:45:09.357 INFO:tasks.workunit.client.0.smithi142.stdout:4/754: readlink d0/d9/d19/l2f 0 2024-03-29T04:45:09.358 INFO:tasks.workunit.client.1.smithi152.stdout:1/153: read d5/d7/d9/f10 [7834,124628] 0 2024-03-29T04:45:09.358 INFO:tasks.workunit.client.1.smithi152.stdout:7/138: symlink d7/d8/d12/d1d/l26 0 2024-03-29T04:45:09.359 INFO:tasks.workunit.client.0.smithi142.stdout:5/947: mknod df/d29/d6e/d8a/c13f 0 2024-03-29T04:45:09.360 INFO:tasks.workunit.client.1.smithi152.stdout:3/148: truncate d7/d19/d1b/f23 3974965 0 2024-03-29T04:45:09.360 INFO:tasks.workunit.client.1.smithi152.stdout:2/120: read d0/d3/f1d [2995015,49107] 0 2024-03-29T04:45:09.361 INFO:tasks.workunit.client.1.smithi152.stdout:6/224: link d1/d3/d9/l47 d1/d3/l4c 0 2024-03-29T04:45:09.362 INFO:tasks.workunit.client.1.smithi152.stdout:1/154: write d5/d7/f25 [5125025,63809] 0 2024-03-29T04:45:09.362 INFO:tasks.workunit.client.1.smithi152.stdout:3/149: read - d7/d19/f26 zero size 2024-03-29T04:45:09.363 INFO:tasks.workunit.client.0.smithi142.stdout:2/740: unlink de/d11/d1e/d22/d27/f4b 0 2024-03-29T04:45:09.365 INFO:tasks.workunit.client.1.smithi152.stdout:2/121: rmdir d0/d3 39 2024-03-29T04:45:09.365 INFO:tasks.workunit.client.1.smithi152.stdout:7/139: symlink d7/d8/d12/l27 0 2024-03-29T04:45:09.366 INFO:tasks.workunit.client.0.smithi142.stdout:6/716: rename d1c/d33/d46/d71/f75 to d1c/d90/d94/fe4 0 2024-03-29T04:45:09.368 INFO:tasks.workunit.client.0.smithi142.stdout:9/619: mkdir d0/d6c/dc2 0 2024-03-29T04:45:09.369 INFO:tasks.workunit.client.0.smithi142.stdout:9/620: readlink d0/d2/d6/d16/d6a/la5 0 2024-03-29T04:45:09.371 INFO:tasks.workunit.client.1.smithi152.stdout:1/155: dread d5/f20 [0,4194304] 0 2024-03-29T04:45:09.373 INFO:tasks.workunit.client.1.smithi152.stdout:1/156: readlink d5/d7/d9/l33 0 2024-03-29T04:45:09.374 INFO:tasks.workunit.client.0.smithi142.stdout:3/615: link d3/d29/c53 d3/d29/d8f/d5f/d64/d6b/cd4 0 2024-03-29T04:45:09.378 INFO:tasks.workunit.client.1.smithi152.stdout:6/225: creat d1/d3/d9/d13/d3e/f4d x:0 0 0 2024-03-29T04:45:09.380 INFO:tasks.workunit.client.1.smithi152.stdout:2/122: creat d0/d14/d15/f26 x:0 0 0 2024-03-29T04:45:09.380 INFO:tasks.workunit.client.0.smithi142.stdout:4/755: dread d0/d12/f1e [0,4194304] 0 2024-03-29T04:45:09.381 INFO:tasks.workunit.client.0.smithi142.stdout:5/948: truncate df/d1a/d21/d27/fe7 1164489 0 2024-03-29T04:45:09.383 INFO:tasks.workunit.client.1.smithi152.stdout:9/250: dwrite d5/f7 [0,4194304] 0 2024-03-29T04:45:09.384 INFO:tasks.workunit.client.1.smithi152.stdout:8/112: fdatasync d1/d2/f1a 0 2024-03-29T04:45:09.384 INFO:tasks.workunit.client.0.smithi142.stdout:2/741: mknod de/d11/d3d/cfb 0 2024-03-29T04:45:09.386 INFO:tasks.workunit.client.1.smithi152.stdout:9/251: readlink d5/l10 0 2024-03-29T04:45:09.386 INFO:tasks.workunit.client.1.smithi152.stdout:1/157: creat d5/d7/d9/d2b/f37 x:0 0 0 2024-03-29T04:45:09.387 INFO:tasks.workunit.client.0.smithi142.stdout:1/708: dwrite d8/db/f14 [0,4194304] 0 2024-03-29T04:45:09.391 INFO:tasks.workunit.client.1.smithi152.stdout:7/140: dread d7/d8/fa [0,4194304] 0 2024-03-29T04:45:09.395 INFO:tasks.workunit.client.1.smithi152.stdout:3/150: rename d7/l12 to d7/d19/d1b/l27 0 2024-03-29T04:45:09.396 INFO:tasks.workunit.client.0.smithi142.stdout:8/871: rename d5/d2e/dd2/df6/l10c to d5/d1d/d2f/d76/d3c/d70/de9/l130 0 2024-03-29T04:45:09.398 INFO:tasks.workunit.client.0.smithi142.stdout:9/621: creat d0/d2/d61/fc3 x:0 0 0 2024-03-29T04:45:09.398 INFO:tasks.workunit.client.0.smithi142.stdout:0/750: dwrite d7/d13/f28 [0,4194304] 0 2024-03-29T04:45:09.398 INFO:tasks.workunit.client.1.smithi152.stdout:4/136: write d0/f4 [2704757,52167] 0 2024-03-29T04:45:09.399 INFO:tasks.workunit.client.1.smithi152.stdout:9/252: mknod d5/d12/c5b 0 2024-03-29T04:45:09.399 INFO:tasks.workunit.client.1.smithi152.stdout:4/137: write d0/f11 [596954,9596] 0 2024-03-29T04:45:09.401 INFO:tasks.workunit.client.0.smithi142.stdout:4/756: dread d0/d9/f49 [0,4194304] 0 2024-03-29T04:45:09.401 INFO:tasks.workunit.client.0.smithi142.stdout:0/751: chown d7/d13/dbf/d72/d75/ca9 2 1 2024-03-29T04:45:09.401 INFO:tasks.workunit.client.0.smithi142.stdout:7/827: truncate d7/d27/d44/db2/db8/d4b/f9a 4944709 0 2024-03-29T04:45:09.402 INFO:tasks.workunit.client.1.smithi152.stdout:5/184: truncate d6/f14 1315358 0 2024-03-29T04:45:09.402 INFO:tasks.workunit.client.0.smithi142.stdout:3/616: symlink d3/d29/d8f/d5f/ld5 0 2024-03-29T04:45:09.403 INFO:tasks.workunit.client.1.smithi152.stdout:5/185: fdatasync d6/dd/f27 0 2024-03-29T04:45:09.403 INFO:tasks.workunit.client.1.smithi152.stdout:7/141: creat d7/d8/d12/f28 x:0 0 0 2024-03-29T04:45:09.404 INFO:tasks.workunit.client.0.smithi142.stdout:7/828: read d7/d27/d44/db2/db8/de/d20/d3f/f65 [1566892,21413] 0 2024-03-29T04:45:09.408 INFO:tasks.workunit.client.1.smithi152.stdout:5/186: truncate d6/d1d/d30/f3e 924658 0 2024-03-29T04:45:09.409 INFO:tasks.workunit.client.1.smithi152.stdout:3/151: symlink d7/d1c/l28 0 2024-03-29T04:45:09.409 INFO:tasks.workunit.client.1.smithi152.stdout:6/226: dread d1/d3/d9/f21 [0,4194304] 0 2024-03-29T04:45:09.410 INFO:tasks.workunit.client.1.smithi152.stdout:2/123: creat d0/d14/d15/d22/f27 x:0 0 0 2024-03-29T04:45:09.412 INFO:tasks.workunit.client.0.smithi142.stdout:2/742: dwrite de/d11/d23/f46 [4194304,4194304] 0 2024-03-29T04:45:09.414 INFO:tasks.workunit.client.1.smithi152.stdout:8/113: dwrite d1/d2/f1a [0,4194304] 0 2024-03-29T04:45:09.414 INFO:tasks.workunit.client.0.smithi142.stdout:2/743: truncate de/f29 4417049 0 2024-03-29T04:45:09.414 INFO:tasks.workunit.client.0.smithi142.stdout:2/744: read - de/d11/d17/d60/f6d zero size 2024-03-29T04:45:09.417 INFO:tasks.workunit.client.0.smithi142.stdout:5/949: write df/d1a/d21/fcf [1093978,13687] 0 2024-03-29T04:45:09.422 INFO:tasks.workunit.client.0.smithi142.stdout:6/717: rename d1c/de1 to d1c/d33/db6/da4/de5 0 2024-03-29T04:45:09.422 INFO:tasks.workunit.client.0.smithi142.stdout:2/745: dwrite de/f9d [0,4194304] 0 2024-03-29T04:45:09.423 INFO:tasks.workunit.client.1.smithi152.stdout:2/124: rename d0/d14/l21 to d0/d14/d15/l28 0 2024-03-29T04:45:09.423 INFO:tasks.workunit.client.1.smithi152.stdout:9/253: creat d5/f5c x:0 0 0 2024-03-29T04:45:09.424 INFO:tasks.workunit.client.1.smithi152.stdout:2/125: chown d0/d14/d15 977 1 2024-03-29T04:45:09.424 INFO:tasks.workunit.client.1.smithi152.stdout:8/114: getdents d1/d21 0 2024-03-29T04:45:09.427 INFO:tasks.workunit.client.0.smithi142.stdout:3/617: symlink d3/db9/ld6 0 2024-03-29T04:45:09.427 INFO:tasks.workunit.client.1.smithi152.stdout:1/158: getdents d5/d7 0 2024-03-29T04:45:09.427 INFO:tasks.workunit.client.1.smithi152.stdout:3/152: creat d7/d19/f29 x:0 0 0 2024-03-29T04:45:09.428 INFO:tasks.workunit.client.1.smithi152.stdout:3/153: chown d7/d19/l20 1690 1 2024-03-29T04:45:09.429 INFO:tasks.workunit.client.0.smithi142.stdout:7/829: unlink d7/d27/d44/db2/db8/d4b/d71/d30/cf3 0 2024-03-29T04:45:09.431 INFO:tasks.workunit.client.0.smithi142.stdout:1/709: mknod d8/db/dd6/cf6 0 2024-03-29T04:45:09.432 INFO:tasks.workunit.client.1.smithi152.stdout:6/227: getdents d1/d3/d9/d13/d28 0 2024-03-29T04:45:09.435 INFO:tasks.workunit.client.0.smithi142.stdout:8/872: rename d5/d1d/d2f/d76/d3c/d70/de9/f127 to d5/d1d/d24/d66/f131 0 2024-03-29T04:45:09.435 INFO:tasks.workunit.client.1.smithi152.stdout:6/228: write d1/d3/d9/d13/f3d [446980,55399] 0 2024-03-29T04:45:09.436 INFO:tasks.workunit.client.0.smithi142.stdout:8/873: chown d5/d1d/d2f/d76/d3c/d70 99302256 1 2024-03-29T04:45:09.437 INFO:tasks.workunit.client.1.smithi152.stdout:1/159: mkdir d5/d7/d38 0 2024-03-29T04:45:09.437 INFO:tasks.workunit.client.0.smithi142.stdout:9/622: unlink d0/d2/d61/cbe 0 2024-03-29T04:45:09.438 INFO:tasks.workunit.client.1.smithi152.stdout:8/115: dwrite d1/d2/d3/f17 [0,4194304] 0 2024-03-29T04:45:09.438 INFO:tasks.workunit.client.0.smithi142.stdout:9/623: chown d0/d2/d6/d16/daa 62 1 2024-03-29T04:45:09.439 INFO:tasks.workunit.client.0.smithi142.stdout:9/624: write d0/d12/d50/f93 [287843,106059] 0 2024-03-29T04:45:09.440 INFO:tasks.workunit.client.1.smithi152.stdout:2/126: rename d0/d14/d15/d22/f27 to d0/d14/d15/f29 0 2024-03-29T04:45:09.443 INFO:tasks.workunit.client.0.smithi142.stdout:8/874: dwrite d5/d1d/d59/f7c [0,4194304] 0 2024-03-29T04:45:09.443 INFO:tasks.workunit.client.0.smithi142.stdout:0/752: write d7/d9/d55/d42/f7c [1233579,32489] 0 2024-03-29T04:45:09.444 INFO:tasks.workunit.client.0.smithi142.stdout:4/757: write d0/d9/d11/d4c/d64/d94/fab [340826,49035] 0 2024-03-29T04:45:09.444 INFO:tasks.workunit.client.1.smithi152.stdout:8/116: symlink d1/d2/d3/l22 0 2024-03-29T04:45:09.444 INFO:tasks.workunit.client.1.smithi152.stdout:8/117: fdatasync d1/f1c 0 2024-03-29T04:45:09.444 INFO:tasks.workunit.client.1.smithi152.stdout:1/160: symlink d5/d7/d8/d13/d19/d21/l39 0 2024-03-29T04:45:09.447 INFO:tasks.workunit.client.0.smithi142.stdout:4/758: chown d0/d9/d11/d4c/d5f/d81/dc9/d47/f74 251 1 2024-03-29T04:45:09.448 INFO:tasks.workunit.client.1.smithi152.stdout:3/154: creat d7/d19/d1b/d1e/f2a x:0 0 0 2024-03-29T04:45:09.448 INFO:tasks.workunit.client.0.smithi142.stdout:7/830: creat d7/d62/f114 x:0 0 0 2024-03-29T04:45:09.450 INFO:tasks.workunit.client.0.smithi142.stdout:5/950: mkdir df/d29/d140 0 2024-03-29T04:45:09.451 INFO:tasks.workunit.client.1.smithi152.stdout:6/229: creat d1/d3/f4e x:0 0 0 2024-03-29T04:45:09.452 INFO:tasks.workunit.client.0.smithi142.stdout:5/951: write df/d16/d24/d2e/d47/d4d/d102/f107 [105499,236] 0 2024-03-29T04:45:09.453 INFO:tasks.workunit.client.1.smithi152.stdout:1/161: symlink d5/d7/d8/l3a 0 2024-03-29T04:45:09.453 INFO:tasks.workunit.client.0.smithi142.stdout:6/718: rename d1c/d33/db6/d52/f7b to d1c/d33/d46/d61/fe6 0 2024-03-29T04:45:09.454 INFO:tasks.workunit.client.0.smithi142.stdout:5/952: fdatasync df/d16/d24/ff7 0 2024-03-29T04:45:09.455 INFO:tasks.workunit.client.0.smithi142.stdout:2/746: creat de/d11/d1e/d22/dbf/ffc x:0 0 0 2024-03-29T04:45:09.457 INFO:tasks.workunit.client.0.smithi142.stdout:9/625: chown d0/d12/c68 9 1 2024-03-29T04:45:09.457 INFO:tasks.workunit.client.0.smithi142.stdout:3/618: mknod d3/d29/cd7 0 2024-03-29T04:45:09.459 INFO:tasks.workunit.client.0.smithi142.stdout:9/626: chown d0/d2/d6/d16/d6a/fae 48994 1 2024-03-29T04:45:09.461 INFO:tasks.workunit.client.0.smithi142.stdout:7/831: fsync d7/d27/d44/db2/db8/d4b/d71/dec/fbd 0 2024-03-29T04:45:09.461 INFO:tasks.workunit.client.1.smithi152.stdout:2/127: rename d0/d3/c4 to d0/c2a 0 2024-03-29T04:45:09.462 INFO:tasks.workunit.client.1.smithi152.stdout:7/142: dwrite d7/d8/db/df/f13 [0,4194304] 0 2024-03-29T04:45:09.463 INFO:tasks.workunit.client.1.smithi152.stdout:2/128: chown d0/d14/c1c 0 1 2024-03-29T04:45:09.464 INFO:tasks.workunit.client.1.smithi152.stdout:8/118: link d1/d2/d1d/f1f d1/d2/d1d/f23 0 2024-03-29T04:45:09.466 INFO:tasks.workunit.client.0.smithi142.stdout:2/747: chown de/d11/d1e/d22/d27/dba/d61/le1 1001 1 2024-03-29T04:45:09.467 INFO:tasks.workunit.client.1.smithi152.stdout:1/162: dwrite d5/d7/fe [0,4194304] 0 2024-03-29T04:45:09.467 INFO:tasks.workunit.client.1.smithi152.stdout:9/254: dread d5/d12/d2c/f3f [0,4194304] 0 2024-03-29T04:45:09.469 INFO:tasks.workunit.client.0.smithi142.stdout:8/875: write d5/d1d/d24/d66/dce/feb [273761,1882] 0 2024-03-29T04:45:09.470 INFO:tasks.workunit.client.1.smithi152.stdout:7/143: chown d7/d8/db/fe 183258331 1 2024-03-29T04:45:09.470 INFO:tasks.workunit.client.1.smithi152.stdout:6/230: getdents d1 0 2024-03-29T04:45:09.471 INFO:tasks.workunit.client.1.smithi152.stdout:3/155: getdents d7 0 2024-03-29T04:45:09.473 INFO:tasks.workunit.client.1.smithi152.stdout:6/231: fsync d1/f17 0 2024-03-29T04:45:09.473 INFO:tasks.workunit.client.1.smithi152.stdout:3/156: write d7/d19/f29 [950300,74306] 0 2024-03-29T04:45:09.476 INFO:tasks.workunit.client.1.smithi152.stdout:3/157: stat d7/d19 0 2024-03-29T04:45:09.476 INFO:tasks.workunit.client.1.smithi152.stdout:8/119: rmdir d1/d2/d3 39 2024-03-29T04:45:09.477 INFO:tasks.workunit.client.0.smithi142.stdout:9/627: rmdir d0/d2/d6/d27 39 2024-03-29T04:45:09.477 INFO:tasks.workunit.client.0.smithi142.stdout:1/710: dread d8/db/dc/d77/db9/dbe/d1f/f29 [0,4194304] 0 2024-03-29T04:45:09.480 INFO:tasks.workunit.client.0.smithi142.stdout:7/832: dread - d7/d27/d44/db2/db8/d4b/d71/fe3 zero size 2024-03-29T04:45:09.481 INFO:tasks.workunit.client.1.smithi152.stdout:7/144: creat d7/d8/d12/d1d/f29 x:0 0 0 2024-03-29T04:45:09.482 INFO:tasks.workunit.client.0.smithi142.stdout:4/759: truncate d0/d9/db2/dba/fe6 5026470 0 2024-03-29T04:45:09.483 INFO:tasks.workunit.client.1.smithi152.stdout:3/158: chown d7/d19/d1b/l27 13198202 1 2024-03-29T04:45:09.483 INFO:tasks.workunit.client.1.smithi152.stdout:2/129: truncate d0/d14/f1b 314638 0 2024-03-29T04:45:09.484 INFO:tasks.workunit.client.0.smithi142.stdout:5/953: fsync df/d29/d6e/f10f 0 2024-03-29T04:45:09.484 INFO:tasks.workunit.client.0.smithi142.stdout:4/760: dread - d0/d9/d11/d4c/d5f/d81/dc9/d47/fef zero size 2024-03-29T04:45:09.485 INFO:tasks.workunit.client.1.smithi152.stdout:2/130: stat d0/d14/d15/f26 0 2024-03-29T04:45:09.485 INFO:tasks.workunit.client.1.smithi152.stdout:8/120: creat d1/d21/f24 x:0 0 0 2024-03-29T04:45:09.487 INFO:tasks.workunit.client.1.smithi152.stdout:1/163: rename d5/d7/d8/l3a to d5/d7/d8/d13/d19/l3b 0 2024-03-29T04:45:09.488 INFO:tasks.workunit.client.1.smithi152.stdout:9/255: dwrite d5/db/d1b/f37 [0,4194304] 0 2024-03-29T04:45:09.488 INFO:tasks.workunit.client.0.smithi142.stdout:6/719: rename cd to d1c/d33/d46/d86/dc0/ce7 0 2024-03-29T04:45:09.488 INFO:tasks.workunit.client.0.smithi142.stdout:6/720: fsync d1c/d45/f76 0 2024-03-29T04:45:09.488 INFO:tasks.workunit.client.0.smithi142.stdout:2/748: unlink de/d11/d1e/fb9 0 2024-03-29T04:45:09.490 INFO:tasks.workunit.client.0.smithi142.stdout:8/876: chown d5/d1d/d2f/d76/d3c/c11c 3308553 1 2024-03-29T04:45:09.490 INFO:tasks.workunit.client.1.smithi152.stdout:9/256: fsync d5/f7 0 2024-03-29T04:45:09.490 INFO:tasks.workunit.client.1.smithi152.stdout:7/145: unlink d7/d8/d12/d1d/l25 0 2024-03-29T04:45:09.492 INFO:tasks.workunit.client.1.smithi152.stdout:2/131: creat d0/d14/d15/f2b x:0 0 0 2024-03-29T04:45:09.492 INFO:tasks.workunit.client.1.smithi152.stdout:6/232: rename d1/d3/f15 to d1/d3/df/f4f 0 2024-03-29T04:45:09.492 INFO:tasks.workunit.client.1.smithi152.stdout:3/159: read d7/d19/d1b/f23 [1568622,10235] 0 2024-03-29T04:45:09.494 INFO:tasks.workunit.client.1.smithi152.stdout:9/257: symlink d5/d12/l5d 0 2024-03-29T04:45:09.496 INFO:tasks.workunit.client.0.smithi142.stdout:8/877: dwrite d5/d1d/d24/f107 [0,4194304] 0 2024-03-29T04:45:09.496 INFO:tasks.workunit.client.1.smithi152.stdout:9/258: fsync d5/db/d1b/f51 0 2024-03-29T04:45:09.498 INFO:tasks.workunit.client.1.smithi152.stdout:8/121: rename d1/f1c to d1/d21/f25 0 2024-03-29T04:45:09.498 INFO:tasks.workunit.client.1.smithi152.stdout:9/259: read d5/d8/f9 [2880004,37096] 0 2024-03-29T04:45:09.499 INFO:tasks.workunit.client.0.smithi142.stdout:0/753: link d7/d9/d6e/f95 d7/d9/ff1 0 2024-03-29T04:45:09.502 INFO:tasks.workunit.client.0.smithi142.stdout:3/619: write d3/d29/d2d/faa [633928,33219] 0 2024-03-29T04:45:09.505 INFO:tasks.workunit.client.0.smithi142.stdout:5/954: dwrite df/d16/d25/f59 [0,4194304] 0 2024-03-29T04:45:09.505 INFO:tasks.workunit.client.0.smithi142.stdout:1/711: creat d8/ff7 x:0 0 0 2024-03-29T04:45:09.506 INFO:tasks.workunit.client.1.smithi152.stdout:1/164: link d5/d7/d8/d13/f35 d5/f3c 0 2024-03-29T04:45:09.506 INFO:tasks.workunit.client.0.smithi142.stdout:2/749: write de/d11/d17/d5d/d71/fe8 [410344,16662] 0 2024-03-29T04:45:09.507 INFO:tasks.workunit.client.1.smithi152.stdout:3/160: dread d7/d19/d1b/f1d [0,4194304] 0 2024-03-29T04:45:09.507 INFO:tasks.workunit.client.0.smithi142.stdout:3/620: chown d3/d29/d8f/d5f/d64/fc8 880505 1 2024-03-29T04:45:09.509 INFO:tasks.workunit.client.1.smithi152.stdout:9/260: dwrite d5/f5c [0,4194304] 0 2024-03-29T04:45:09.509 INFO:tasks.workunit.client.0.smithi142.stdout:0/754: unlink d7/d9/d54/f99 0 2024-03-29T04:45:09.510 INFO:tasks.workunit.client.0.smithi142.stdout:6/721: dwrite d1c/d33/d46/d61/d7e/fac [0,4194304] 0 2024-03-29T04:45:09.510 INFO:tasks.workunit.client.1.smithi152.stdout:8/122: rmdir d1/d2 39 2024-03-29T04:45:09.511 INFO:tasks.workunit.client.0.smithi142.stdout:8/878: creat d5/d1d/d2f/d76/d3c/d70/de9/f132 x:0 0 0 2024-03-29T04:45:09.511 INFO:tasks.workunit.client.0.smithi142.stdout:5/955: chown df/d16/d24/d2e/d47/d4d/d102/c69 16043 1 2024-03-29T04:45:09.514 INFO:tasks.workunit.client.0.smithi142.stdout:2/750: dread de/d11/d1e/d22/d50/fdb [0,4194304] 0 2024-03-29T04:45:09.514 INFO:tasks.workunit.client.1.smithi152.stdout:6/233: rename d1/d3/d9/d13/f3d to d1/f50 0 2024-03-29T04:45:09.515 INFO:tasks.workunit.client.0.smithi142.stdout:7/833: rename d7/la2 to d7/d27/d44/db2/db8/l115 0 2024-03-29T04:45:09.517 INFO:tasks.workunit.client.1.smithi152.stdout:7/146: getdents d7/d8/d12/d15 0 2024-03-29T04:45:09.518 INFO:tasks.workunit.client.0.smithi142.stdout:0/755: mkdir d7/d9/d6e/df2 0 2024-03-29T04:45:09.518 INFO:tasks.workunit.client.0.smithi142.stdout:5/956: write df/d29/d7a/d97/db3/f13c [325126,72003] 0 2024-03-29T04:45:09.519 INFO:tasks.workunit.client.1.smithi152.stdout:7/147: readlink d7/d8/d12/l27 0 2024-03-29T04:45:09.524 INFO:tasks.workunit.client.1.smithi152.stdout:3/161: fdatasync d7/f18 0 2024-03-29T04:45:09.526 INFO:tasks.workunit.client.1.smithi152.stdout:9/261: creat d5/d12/d2c/f5e x:0 0 0 2024-03-29T04:45:09.527 INFO:tasks.workunit.client.1.smithi152.stdout:6/234: mknod d1/d3/d1a/d1b/c51 0 2024-03-29T04:45:09.527 INFO:tasks.workunit.client.0.smithi142.stdout:3/621: dwrite d3/d1f/d24/d47/d87/fa3 [0,4194304] 0 2024-03-29T04:45:09.528 INFO:tasks.workunit.client.1.smithi152.stdout:7/148: creat d7/d8/d18/f2a x:0 0 0 2024-03-29T04:45:09.530 INFO:tasks.workunit.client.0.smithi142.stdout:4/761: rename d0/d9/db2/dba/lcb to d0/d9/db2/dba/dbc/lf6 0 2024-03-29T04:45:09.530 INFO:tasks.workunit.client.0.smithi142.stdout:5/957: mknod df/d16/d25/d120/d139/c141 0 2024-03-29T04:45:09.531 INFO:tasks.workunit.client.0.smithi142.stdout:7/834: creat d7/d27/d44/d4c/da3/d10f/f116 x:0 0 0 2024-03-29T04:45:09.531 INFO:tasks.workunit.client.1.smithi152.stdout:6/235: dread d1/f17 [0,4194304] 0 2024-03-29T04:45:09.532 INFO:tasks.workunit.client.1.smithi152.stdout:8/123: unlink d1/d2/d1d/f1f 0 2024-03-29T04:45:09.532 INFO:tasks.workunit.client.0.smithi142.stdout:2/751: dwrite de/d11/d1e/d22/d27/dba/d61/fdf [0,4194304] 0 2024-03-29T04:45:09.533 INFO:tasks.workunit.client.0.smithi142.stdout:4/762: fsync d0/d9/d11/d97/fe1 0 2024-03-29T04:45:09.534 INFO:tasks.workunit.client.0.smithi142.stdout:6/722: rename d1c/d45/l77 to d1c/d33/d46/d66/dcd/le8 0 2024-03-29T04:45:09.535 INFO:tasks.workunit.client.0.smithi142.stdout:3/622: read d3/d29/f77 [189685,90791] 0 2024-03-29T04:45:09.536 INFO:tasks.workunit.client.1.smithi152.stdout:2/132: getdents d0/d14/d15 0 2024-03-29T04:45:09.536 INFO:tasks.workunit.client.1.smithi152.stdout:3/162: symlink d7/l2b 0 2024-03-29T04:45:09.537 INFO:tasks.workunit.client.1.smithi152.stdout:2/133: stat d0/d3 0 2024-03-29T04:45:09.537 INFO:tasks.workunit.client.0.smithi142.stdout:7/835: fsync d7/d27/d44/d109/f112 0 2024-03-29T04:45:09.538 INFO:tasks.workunit.client.1.smithi152.stdout:9/262: creat d5/d12/d2c/f5f x:0 0 0 2024-03-29T04:45:09.539 INFO:tasks.workunit.client.0.smithi142.stdout:8/879: rename d5/d1d/dd9/f117 to d5/d1d/d81/f133 0 2024-03-29T04:45:09.542 INFO:tasks.workunit.client.1.smithi152.stdout:7/149: rename d7/d8/d18/f24 to d7/d8/db/df/f2b 0 2024-03-29T04:45:09.546 INFO:tasks.workunit.client.1.smithi152.stdout:1/165: link d5/d7/d9/f10 d5/d7/d8/d13/d19/f3d 0 2024-03-29T04:45:09.546 INFO:tasks.workunit.client.0.smithi142.stdout:6/723: dwrite d1c/d33/d46/d61/fe6 [0,4194304] 0 2024-03-29T04:45:09.548 INFO:tasks.workunit.client.0.smithi142.stdout:2/752: truncate de/d11/d1e/d22/d27/f66 199212 0 2024-03-29T04:45:09.548 INFO:tasks.workunit.client.0.smithi142.stdout:1/712: truncate d8/db/dc/d77/db9/dbe/d2d/d44/d53/f5f 2844037 0 2024-03-29T04:45:09.549 INFO:tasks.workunit.client.1.smithi152.stdout:2/134: dwrite d0/d3/df/f23 [0,4194304] 0 2024-03-29T04:45:09.550 INFO:tasks.workunit.client.0.smithi142.stdout:2/753: fsync de/d11/d23/dd1/ff0 0 2024-03-29T04:45:09.552 INFO:tasks.workunit.client.1.smithi152.stdout:2/135: chown d0/d3/df/f23 213 1 2024-03-29T04:45:09.554 INFO:tasks.workunit.client.1.smithi152.stdout:6/236: symlink d1/d3/d9/d13/d28/d45/l52 0 2024-03-29T04:45:09.555 INFO:tasks.workunit.client.0.smithi142.stdout:7/836: creat d7/d27/d44/db2/db8/d4b/d71/dec/f117 x:0 0 0 2024-03-29T04:45:09.555 INFO:tasks.workunit.client.1.smithi152.stdout:7/150: dwrite d7/d8/d12/f20 [0,4194304] 0 2024-03-29T04:45:09.555 INFO:tasks.workunit.client.0.smithi142.stdout:7/837: dread - d7/d27/d44/d109/f112 zero size 2024-03-29T04:45:09.558 INFO:tasks.workunit.client.1.smithi152.stdout:8/124: creat d1/f26 x:0 0 0 2024-03-29T04:45:09.558 INFO:tasks.workunit.client.1.smithi152.stdout:2/136: chown d0/d14/d15 29 1 2024-03-29T04:45:09.562 INFO:tasks.workunit.client.1.smithi152.stdout:9/263: creat d5/db/f60 x:0 0 0 2024-03-29T04:45:09.563 INFO:tasks.workunit.client.0.smithi142.stdout:6/724: symlink d1c/d33/d46/d61/d9b/le9 0 2024-03-29T04:45:09.564 INFO:tasks.workunit.client.1.smithi152.stdout:1/166: creat d5/f3e x:0 0 0 2024-03-29T04:45:09.565 INFO:tasks.workunit.client.0.smithi142.stdout:1/713: creat d8/db/dc/d77/db9/dbe/d2d/ff8 x:0 0 0 2024-03-29T04:45:09.565 INFO:tasks.workunit.client.0.smithi142.stdout:6/725: chown d1c/d33/d46/d71/f60 19359527 1 2024-03-29T04:45:09.565 INFO:tasks.workunit.client.1.smithi152.stdout:9/264: write d5/db/d1b/f55 [1203672,40274] 0 2024-03-29T04:45:09.569 INFO:tasks.workunit.client.1.smithi152.stdout:6/237: dwrite d1/fe [0,4194304] 0 2024-03-29T04:45:09.570 INFO:tasks.workunit.client.1.smithi152.stdout:1/167: dread - d5/d7/d8/d13/f35 zero size 2024-03-29T04:45:09.570 INFO:tasks.workunit.client.0.smithi142.stdout:5/958: dread df/d16/d24/d2e/d47/d4d/fe4 [0,4194304] 0 2024-03-29T04:45:09.571 INFO:tasks.workunit.client.1.smithi152.stdout:8/125: unlink d1/d2/fc 0 2024-03-29T04:45:09.573 INFO:tasks.workunit.client.0.smithi142.stdout:6/726: creat d1c/d45/d8a/fea x:0 0 0 2024-03-29T04:45:09.574 INFO:tasks.workunit.client.0.smithi142.stdout:1/714: dwrite d8/db/dc/d77/db9/dbe/d15/d9c/fc2 [0,4194304] 0 2024-03-29T04:45:09.574 INFO:tasks.workunit.client.1.smithi152.stdout:7/151: link d7/d8/d18/f2a d7/d8/db/df/d21/f2c 0 2024-03-29T04:45:09.575 INFO:tasks.workunit.client.0.smithi142.stdout:1/715: dread - d8/db/dc/d77/fdf zero size 2024-03-29T04:45:09.575 INFO:tasks.workunit.client.1.smithi152.stdout:2/137: dwrite d0/d14/f19 [4194304,4194304] 0 2024-03-29T04:45:09.576 INFO:tasks.workunit.client.0.smithi142.stdout:0/756: rename d7/d9/d54/d58/c66 to d7/d9/d54/cf3 0 2024-03-29T04:45:09.576 INFO:tasks.workunit.client.0.smithi142.stdout:3/623: link d3/d29/d2b/d40/d67/lbf d3/d1f/d24/ld8 0 2024-03-29T04:45:09.578 INFO:tasks.workunit.client.0.smithi142.stdout:0/757: chown d7/d9/f2e 76784 1 2024-03-29T04:45:09.579 INFO:tasks.workunit.client.0.smithi142.stdout:3/624: fdatasync d3/d29/d2d/f3f 0 2024-03-29T04:45:09.584 INFO:tasks.workunit.client.0.smithi142.stdout:6/727: fsync d1c/d33/d46/d71/f9c 0 2024-03-29T04:45:09.587 INFO:tasks.workunit.client.0.smithi142.stdout:4/763: rename d0/d9/d11/d4c/d5f/d81/dc9/fe7 to d0/d9/d11/d4c/d5f/d81/dc9/d47/d7a/de4/ff7 0 2024-03-29T04:45:09.589 INFO:tasks.workunit.client.1.smithi152.stdout:6/238: creat d1/d3/d1a/f53 x:0 0 0 2024-03-29T04:45:09.590 INFO:tasks.workunit.client.0.smithi142.stdout:1/716: mkdir d8/db/dc/d77/d31/d4d/d8c/df9 0 2024-03-29T04:45:09.594 INFO:tasks.workunit.client.0.smithi142.stdout:3/625: dwrite d3/d29/d2b/d40/d67/fb7 [0,4194304] 0 2024-03-29T04:45:09.595 INFO:tasks.workunit.client.1.smithi152.stdout:7/152: creat d7/d8/f2d x:0 0 0 2024-03-29T04:45:09.596 INFO:tasks.workunit.client.0.smithi142.stdout:6/728: symlink d1c/d33/db6/da4/de5/leb 0 2024-03-29T04:45:09.596 INFO:tasks.workunit.client.1.smithi152.stdout:7/153: readlink d7/d8/d12/l27 0 2024-03-29T04:45:09.596 INFO:tasks.workunit.client.1.smithi152.stdout:2/138: mkdir d0/d3/df/d2c 0 2024-03-29T04:45:09.597 INFO:tasks.workunit.client.1.smithi152.stdout:7/154: fsync d7/d8/d12/f22 0 2024-03-29T04:45:09.598 INFO:tasks.workunit.client.0.smithi142.stdout:2/754: truncate de/f29 3358406 0 2024-03-29T04:45:09.599 INFO:tasks.workunit.client.0.smithi142.stdout:0/758: fdatasync d7/d9/d6e/f95 0 2024-03-29T04:45:09.600 INFO:tasks.workunit.client.0.smithi142.stdout:8/880: dwrite d5/d1d/d2f/d76/d2d/f4a [0,4194304] 0 2024-03-29T04:45:09.602 INFO:tasks.workunit.client.1.smithi152.stdout:9/265: creat d5/f61 x:0 0 0 2024-03-29T04:45:09.603 INFO:tasks.workunit.client.1.smithi152.stdout:6/239: mkdir d1/d3/d9/d13/d3e/d54 0 2024-03-29T04:45:09.606 INFO:tasks.workunit.client.1.smithi152.stdout:2/139: creat d0/d3/df/d2c/f2d x:0 0 0 2024-03-29T04:45:09.607 INFO:tasks.workunit.client.1.smithi152.stdout:9/266: truncate d5/d12/d2c/f39 771771 0 2024-03-29T04:45:09.609 INFO:tasks.workunit.client.1.smithi152.stdout:2/140: dread - d0/d14/d15/f2b zero size 2024-03-29T04:45:09.609 INFO:tasks.workunit.client.1.smithi152.stdout:7/155: dread d7/d8/fa [0,4194304] 0 2024-03-29T04:45:09.610 INFO:tasks.workunit.client.1.smithi152.stdout:6/240: readlink d1/d3/d1a/d1b/l4a 0 2024-03-29T04:45:09.610 INFO:tasks.workunit.client.0.smithi142.stdout:7/838: dwrite d7/d27/d44/db2/db8/d4b/d71/f1f [4194304,4194304] 0 2024-03-29T04:45:09.610 INFO:tasks.workunit.client.0.smithi142.stdout:4/764: creat d0/d9/db2/dba/de9/ff8 x:0 0 0 2024-03-29T04:45:09.612 INFO:tasks.workunit.client.0.smithi142.stdout:6/729: dwrite d1c/d33/d46/d71/f9c [0,4194304] 0 2024-03-29T04:45:09.613 INFO:tasks.workunit.client.1.smithi152.stdout:8/126: getdents d1/d2 0 2024-03-29T04:45:09.613 INFO:tasks.workunit.client.1.smithi152.stdout:6/241: fdatasync d1/d3/d1a/f2c 0 2024-03-29T04:45:09.615 INFO:tasks.workunit.client.1.smithi152.stdout:1/168: dread d5/d7/fe [4194304,4194304] 0 2024-03-29T04:45:09.616 INFO:tasks.workunit.client.0.smithi142.stdout:5/959: dwrite df/d1a/d21/fa4 [0,4194304] 0 2024-03-29T04:45:09.617 INFO:tasks.workunit.client.1.smithi152.stdout:7/156: symlink d7/d8/l2e 0 2024-03-29T04:45:09.617 INFO:tasks.workunit.client.1.smithi152.stdout:8/127: dread - d1/d21/f24 zero size 2024-03-29T04:45:09.617 INFO:tasks.workunit.client.0.smithi142.stdout:1/717: getdents d8/db/dc/d77/db9/dbe/d1f/d50/d96 0 2024-03-29T04:45:09.617 INFO:tasks.workunit.client.0.smithi142.stdout:0/759: creat d7/d13/d4a/de7/ff4 x:0 0 0 2024-03-29T04:45:09.619 INFO:tasks.workunit.client.1.smithi152.stdout:7/157: write d7/d8/d12/f22 [806593,6198] 0 2024-03-29T04:45:09.619 INFO:tasks.workunit.client.1.smithi152.stdout:8/128: write d1/d2/f1a [3032652,63596] 0 2024-03-29T04:45:09.620 INFO:tasks.workunit.client.0.smithi142.stdout:7/839: chown d7/d27/d44/db2/db8/la4 68464 1 2024-03-29T04:45:09.620 INFO:tasks.workunit.client.1.smithi152.stdout:7/158: chown d7/d8/d12/d15 1862897628 1 2024-03-29T04:45:09.622 INFO:tasks.workunit.client.0.smithi142.stdout:6/730: mknod d1c/d33/db6/cec 0 2024-03-29T04:45:09.622 INFO:tasks.workunit.client.0.smithi142.stdout:1/718: chown d8/db/dc/d77/db9/dbe/d2d/fad 76 1 2024-03-29T04:45:09.628 INFO:tasks.workunit.client.0.smithi142.stdout:6/731: read - d1c/d33/d46/d66/dc1/fd6 zero size 2024-03-29T04:45:09.629 INFO:tasks.workunit.client.1.smithi152.stdout:0/164: sync 2024-03-29T04:45:09.630 INFO:tasks.workunit.client.1.smithi152.stdout:5/187: sync 2024-03-29T04:45:09.630 INFO:tasks.workunit.client.1.smithi152.stdout:4/138: sync 2024-03-29T04:45:09.631 INFO:tasks.workunit.client.1.smithi152.stdout:4/139: write d0/f12 [545034,26215] 0 2024-03-29T04:45:09.633 INFO:tasks.workunit.client.0.smithi142.stdout:8/881: getdents d5/d1d/d59 0 2024-03-29T04:45:09.633 INFO:tasks.workunit.client.0.smithi142.stdout:7/840: dread d7/d27/d44/db2/db8/fe6 [0,4194304] 0 2024-03-29T04:45:09.633 INFO:tasks.workunit.client.1.smithi152.stdout:8/129: dwrite d1/d2/d3/f17 [0,4194304] 0 2024-03-29T04:45:09.634 INFO:tasks.workunit.client.0.smithi142.stdout:1/719: readlink d8/db/l24 0 2024-03-29T04:45:09.636 INFO:tasks.workunit.client.0.smithi142.stdout:2/755: dread de/f16 [0,4194304] 0 2024-03-29T04:45:09.637 INFO:tasks.workunit.client.0.smithi142.stdout:6/732: mknod d1c/d33/db6/da4/de5/ced 0 2024-03-29T04:45:09.637 INFO:tasks.workunit.client.0.smithi142.stdout:4/765: getdents d0/d9/d11/d84/d9f 0 2024-03-29T04:45:09.637 INFO:tasks.workunit.client.0.smithi142.stdout:5/960: creat df/d16/d25/d120/d139/f142 x:0 0 0 2024-03-29T04:45:09.638 INFO:tasks.workunit.client.1.smithi152.stdout:6/242: rmdir d1/d3 39 2024-03-29T04:45:09.638 INFO:tasks.workunit.client.1.smithi152.stdout:4/140: chown d0 988 1 2024-03-29T04:45:09.638 INFO:tasks.workunit.client.1.smithi152.stdout:1/169: readlink d5/l1d 0 2024-03-29T04:45:09.639 INFO:tasks.workunit.client.0.smithi142.stdout:8/882: symlink d5/d1d/dd9/l134 0 2024-03-29T04:45:09.640 INFO:tasks.workunit.client.0.smithi142.stdout:8/883: chown d5/d2e/dd2/d60/f116 2040 1 2024-03-29T04:45:09.641 INFO:tasks.workunit.client.1.smithi152.stdout:2/141: creat d0/d3/f2e x:0 0 0 2024-03-29T04:45:09.641 INFO:tasks.workunit.client.0.smithi142.stdout:5/961: dread df/d1a/d28/f2d [0,4194304] 0 2024-03-29T04:45:09.641 INFO:tasks.workunit.client.0.smithi142.stdout:1/720: creat d8/db/dc/d66/ffa x:0 0 0 2024-03-29T04:45:09.642 INFO:tasks.workunit.client.1.smithi152.stdout:5/188: creat d6/d16/d25/f43 x:0 0 0 2024-03-29T04:45:09.642 INFO:tasks.workunit.client.0.smithi142.stdout:1/721: readlink d8/l99 0 2024-03-29T04:45:09.643 INFO:tasks.workunit.client.0.smithi142.stdout:7/841: write d7/d27/d44/db2/db8/de/d20/d3f/f41 [4850933,50775] 0 2024-03-29T04:45:09.643 INFO:tasks.workunit.client.0.smithi142.stdout:5/962: chown df/d29/d6e/d8a/l95 38102760 1 2024-03-29T04:45:09.644 INFO:tasks.workunit.client.1.smithi152.stdout:6/243: chown d1/d3/d1a/d1b 7 1 2024-03-29T04:45:09.644 INFO:tasks.workunit.client.0.smithi142.stdout:7/842: write d7/d27/d44/db2/db8/d4b/f82 [3246503,97839] 0 2024-03-29T04:45:09.645 INFO:tasks.workunit.client.0.smithi142.stdout:5/963: truncate df/d29/d6e/f101 950131 0 2024-03-29T04:45:09.646 INFO:tasks.workunit.client.0.smithi142.stdout:7/843: chown d7/d27/f88 38699 1 2024-03-29T04:45:09.646 INFO:tasks.workunit.client.0.smithi142.stdout:2/756: creat de/d11/dc7/ffd x:0 0 0 2024-03-29T04:45:09.647 INFO:tasks.workunit.client.1.smithi152.stdout:0/165: mkdir d2/da/d3a 0 2024-03-29T04:45:09.650 INFO:tasks.workunit.client.1.smithi152.stdout:3/163: dwrite d7/d19/d1b/f23 [0,4194304] 0 2024-03-29T04:45:09.652 INFO:tasks.workunit.client.1.smithi152.stdout:2/142: creat d0/d14/d15/d22/f2f x:0 0 0 2024-03-29T04:45:09.652 INFO:tasks.workunit.client.0.smithi142.stdout:3/626: dread d3/d29/d2b/d40/d67/d8b/d6/f20 [0,4194304] 0 2024-03-29T04:45:09.653 INFO:tasks.workunit.client.1.smithi152.stdout:0/166: readlink d2/da/lc 0 2024-03-29T04:45:09.653 INFO:tasks.workunit.client.0.smithi142.stdout:3/627: chown d3/d29/d2b/d40/d67/d8b 93 1 2024-03-29T04:45:09.654 INFO:tasks.workunit.client.0.smithi142.stdout:6/733: creat d1c/d33/d46/d61/fee x:0 0 0 2024-03-29T04:45:09.654 INFO:tasks.workunit.client.0.smithi142.stdout:4/766: creat d0/d9/d11/ff9 x:0 0 0 2024-03-29T04:45:09.654 INFO:tasks.workunit.client.0.smithi142.stdout:2/757: dread de/d11/d17/f1a [0,4194304] 0 2024-03-29T04:45:09.655 INFO:tasks.workunit.client.1.smithi152.stdout:8/130: getdents d1/d2/d3 0 2024-03-29T04:45:09.655 INFO:tasks.workunit.client.1.smithi152.stdout:1/170: creat d5/f3f x:0 0 0 2024-03-29T04:45:09.656 INFO:tasks.workunit.client.1.smithi152.stdout:4/141: dwrite d0/f12 [0,4194304] 0 2024-03-29T04:45:09.656 INFO:tasks.workunit.client.1.smithi152.stdout:6/244: mkdir d1/d3/d9/d13/d28/d45/d55 0 2024-03-29T04:45:09.658 INFO:tasks.workunit.client.0.smithi142.stdout:5/964: mknod df/d16/d25/c143 0 2024-03-29T04:45:09.659 INFO:tasks.workunit.client.0.smithi142.stdout:7/844: creat d7/d62/f118 x:0 0 0 2024-03-29T04:45:09.660 INFO:tasks.workunit.client.0.smithi142.stdout:5/965: chown df/d16/d25/d120/d138 6 1 2024-03-29T04:45:09.661 INFO:tasks.workunit.client.0.smithi142.stdout:8/884: mknod d5/d1d/d2f/d5f/c135 0 2024-03-29T04:45:09.662 INFO:tasks.workunit.client.0.smithi142.stdout:4/767: rmdir d0/d9/d11/d97/d9e 39 2024-03-29T04:45:09.665 INFO:tasks.workunit.client.0.smithi142.stdout:1/722: getdents d8/db/dc/d77/db9/dbd 0 2024-03-29T04:45:09.665 INFO:tasks.workunit.client.0.smithi142.stdout:2/758: creat de/d11/d1e/d22/d27/dba/d61/d7d/db4/ffe x:0 0 0 2024-03-29T04:45:09.666 INFO:tasks.workunit.client.0.smithi142.stdout:6/734: dwrite d1c/d33/d46/d61/d9e/fd1 [0,4194304] 0 2024-03-29T04:45:09.668 INFO:tasks.workunit.client.0.smithi142.stdout:6/735: stat l15 0 2024-03-29T04:45:09.668 INFO:tasks.workunit.client.1.smithi152.stdout:2/143: dread - d0/d14/d15/f26 zero size 2024-03-29T04:45:09.669 INFO:tasks.workunit.client.1.smithi152.stdout:1/171: rmdir d5/d7/d8/d13/d19 39 2024-03-29T04:45:09.671 INFO:tasks.workunit.client.1.smithi152.stdout:4/142: fdatasync d0/f11 0 2024-03-29T04:45:09.671 INFO:tasks.workunit.client.0.smithi142.stdout:0/760: dread d7/d9/d55/fdb [0,4194304] 0 2024-03-29T04:45:09.671 INFO:tasks.workunit.client.0.smithi142.stdout:8/885: mkdir d5/d2e/dd2/d60/d12d/d136 0 2024-03-29T04:45:09.671 INFO:tasks.workunit.client.0.smithi142.stdout:1/723: chown d8/db/dc/d77/db9/dbe/d2d/d44/d7f 0 1 2024-03-29T04:45:09.673 INFO:tasks.workunit.client.1.smithi152.stdout:0/167: truncate d2/da/fb 2052803 0 2024-03-29T04:45:09.676 INFO:tasks.workunit.client.0.smithi142.stdout:5/966: getdents df/d2c 0 2024-03-29T04:45:09.676 INFO:tasks.workunit.client.1.smithi152.stdout:9/267: dwrite d5/db/f4e [0,4194304] 0 2024-03-29T04:45:09.677 INFO:tasks.workunit.client.1.smithi152.stdout:3/164: link d7/f10 d7/d1c/f2c 0 2024-03-29T04:45:09.677 INFO:tasks.workunit.client.1.smithi152.stdout:6/245: dread d1/d3/d1a/f20 [0,4194304] 0 2024-03-29T04:45:09.678 INFO:tasks.workunit.client.1.smithi152.stdout:6/246: stat d1/d3 0 2024-03-29T04:45:09.678 INFO:tasks.workunit.client.0.smithi142.stdout:5/967: read - df/d16/d24/d2e/d47/d4d/f118 zero size 2024-03-29T04:45:09.680 INFO:tasks.workunit.client.0.smithi142.stdout:1/724: dwrite d8/db/dc/d77/d31/d4d/f7a [0,4194304] 0 2024-03-29T04:45:09.682 INFO:tasks.workunit.client.1.smithi152.stdout:6/247: chown d1/d3/d9 28442047 1 2024-03-29T04:45:09.686 INFO:tasks.workunit.client.0.smithi142.stdout:0/761: dread d7/d9/d55/fdb [0,4194304] 0 2024-03-29T04:45:09.687 INFO:tasks.workunit.client.1.smithi152.stdout:9/268: stat d5/db/d1b/f2b 0 2024-03-29T04:45:09.687 INFO:tasks.workunit.client.0.smithi142.stdout:8/886: link d5/d2e/dd2/df6/f115 d5/d1d/d2f/d76/d3c/d70/f137 0 2024-03-29T04:45:09.688 INFO:tasks.workunit.client.1.smithi152.stdout:7/159: dwrite d7/d8/d18/f2a [0,4194304] 0 2024-03-29T04:45:09.689 INFO:tasks.workunit.client.1.smithi152.stdout:2/144: rename d0/d14 to d0/d3/d30 0 2024-03-29T04:45:09.691 INFO:tasks.workunit.client.0.smithi142.stdout:5/968: unlink df/d16/d24/d5c/db5/cf8 0 2024-03-29T04:45:09.691 INFO:tasks.workunit.client.1.smithi152.stdout:1/172: dread d5/d7/d8/fb [0,4194304] 0 2024-03-29T04:45:09.693 INFO:tasks.workunit.client.0.smithi142.stdout:6/736: link fb d1c/d33/d46/d61/d9e/fef 0 2024-03-29T04:45:09.693 INFO:tasks.workunit.client.1.smithi152.stdout:6/248: mknod d1/d3/d9/d3b/c56 0 2024-03-29T04:45:09.693 INFO:tasks.workunit.client.0.smithi142.stdout:3/628: write d3/d1f/f46 [1165910,94438] 0 2024-03-29T04:45:09.695 INFO:tasks.workunit.client.1.smithi152.stdout:2/145: rmdir d0/d3/df 39 2024-03-29T04:45:09.696 INFO:tasks.workunit.client.1.smithi152.stdout:8/131: write d1/d2/d1d/f23 [520043,5930] 0 2024-03-29T04:45:09.697 INFO:tasks.workunit.client.1.smithi152.stdout:9/269: write d5/db/f13 [77676,105370] 0 2024-03-29T04:45:09.698 INFO:tasks.workunit.client.1.smithi152.stdout:1/173: mknod d5/d7/d1b/d2e/c40 0 2024-03-29T04:45:09.700 INFO:tasks.workunit.client.0.smithi142.stdout:0/762: write d7/d13/dbf/d72/f3a [3502723,53886] 0 2024-03-29T04:45:09.700 INFO:tasks.workunit.client.1.smithi152.stdout:0/168: dread d2/f20 [0,4194304] 0 2024-03-29T04:45:09.700 INFO:tasks.workunit.client.0.smithi142.stdout:7/845: write d7/d27/d44/db2/db8/d4b/d71/d30/d33/f42 [4100972,4210] 0 2024-03-29T04:45:09.702 INFO:tasks.workunit.client.0.smithi142.stdout:8/887: symlink d5/d1d/d2f/d76/d3c/l138 0 2024-03-29T04:45:09.703 INFO:tasks.workunit.client.0.smithi142.stdout:5/969: symlink df/d29/d7a/l144 0 2024-03-29T04:45:09.704 INFO:tasks.workunit.client.1.smithi152.stdout:3/165: getdents d7/d19/d1b 0 2024-03-29T04:45:09.704 INFO:tasks.workunit.client.1.smithi152.stdout:6/249: dread d1/d3/d1a/f2f [0,4194304] 0 2024-03-29T04:45:09.705 INFO:tasks.workunit.client.1.smithi152.stdout:2/146: chown d0/d3/df/c1f 119 1 2024-03-29T04:45:09.705 INFO:tasks.workunit.client.0.smithi142.stdout:3/629: mknod d3/d29/d8f/cd9 0 2024-03-29T04:45:09.707 INFO:tasks.workunit.client.0.smithi142.stdout:6/737: mkdir d1c/d33/db6/d52/df0 0 2024-03-29T04:45:09.707 INFO:tasks.workunit.client.1.smithi152.stdout:1/174: mknod d5/d7/d8/d13/c41 0 2024-03-29T04:45:09.708 INFO:tasks.workunit.client.0.smithi142.stdout:1/725: mkdir d8/db/dc/d77/db9/dbe/d2d/d44/d7f/df4/dfb 0 2024-03-29T04:45:09.710 INFO:tasks.workunit.client.1.smithi152.stdout:8/132: dwrite d1/d2/d3/f17 [0,4194304] 0 2024-03-29T04:45:09.710 INFO:tasks.workunit.client.1.smithi152.stdout:2/147: readlink d0/d3/d30/d15/l28 0 2024-03-29T04:45:09.710 INFO:tasks.workunit.client.0.smithi142.stdout:8/888: symlink d5/d1d/dd9/l139 0 2024-03-29T04:45:09.711 INFO:tasks.workunit.client.1.smithi152.stdout:6/250: readlink d1/d3/l4c 0 2024-03-29T04:45:09.711 INFO:tasks.workunit.client.0.smithi142.stdout:7/846: mknod d7/d27/d44/db2/db8/d4b/d71/d30/d33/df5/c119 0 2024-03-29T04:45:09.713 INFO:tasks.workunit.client.0.smithi142.stdout:2/759: dwrite de/d11/d17/f1a [0,4194304] 0 2024-03-29T04:45:09.713 INFO:tasks.workunit.client.0.smithi142.stdout:0/763: dwrite d7/d9/d55/d42/f5c [0,4194304] 0 2024-03-29T04:45:09.714 INFO:tasks.workunit.client.1.smithi152.stdout:0/169: getdents d2/da/df 0 2024-03-29T04:45:09.714 INFO:tasks.workunit.client.1.smithi152.stdout:6/251: readlink d1/d3/d9/l40 0 2024-03-29T04:45:09.716 INFO:tasks.workunit.client.0.smithi142.stdout:5/970: truncate df/d16/d25/f37 511207 0 2024-03-29T04:45:09.717 INFO:tasks.workunit.client.0.smithi142.stdout:5/971: chown df/d1a/d21/d55 1438 1 2024-03-29T04:45:09.718 INFO:tasks.workunit.client.1.smithi152.stdout:1/175: chown d5/d7/d9/d12/c29 13086808 1 2024-03-29T04:45:09.718 INFO:tasks.workunit.client.1.smithi152.stdout:4/143: dread d0/d7/f1e [4194304,4194304] 0 2024-03-29T04:45:09.719 INFO:tasks.workunit.client.1.smithi152.stdout:4/144: write d0/f12 [3296004,56916] 0 2024-03-29T04:45:09.720 INFO:tasks.workunit.client.1.smithi152.stdout:4/145: stat d0/d7 0 2024-03-29T04:45:09.720 INFO:tasks.workunit.client.1.smithi152.stdout:8/133: mknod d1/d2/d3/c27 0 2024-03-29T04:45:09.722 INFO:tasks.workunit.client.0.smithi142.stdout:7/847: mknod d7/d27/d44/d4c/c11a 0 2024-03-29T04:45:09.725 INFO:tasks.workunit.client.1.smithi152.stdout:3/166: dwrite d7/d19/f26 [0,4194304] 0 2024-03-29T04:45:09.726 INFO:tasks.workunit.client.1.smithi152.stdout:0/170: chown d2/da/d17/l13 7199834 1 2024-03-29T04:45:09.726 INFO:tasks.workunit.client.0.smithi142.stdout:1/726: mknod d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/cfc 0 2024-03-29T04:45:09.726 INFO:tasks.workunit.client.0.smithi142.stdout:8/889: read d5/d1d/d2f/f54 [2333596,721] 0 2024-03-29T04:45:09.729 INFO:tasks.workunit.client.0.smithi142.stdout:5/972: creat df/d29/d7a/d97/f145 x:0 0 0 2024-03-29T04:45:09.729 INFO:tasks.workunit.client.0.smithi142.stdout:8/890: chown d5/d1d/d2f/d5f/fb1 1 1 2024-03-29T04:45:09.730 INFO:tasks.workunit.client.1.smithi152.stdout:2/148: creat d0/d3/f31 x:0 0 0 2024-03-29T04:45:09.733 INFO:tasks.workunit.client.0.smithi142.stdout:6/738: link d1c/d90/d94/la7 d1c/d33/db6/da4/lf1 0 2024-03-29T04:45:09.734 INFO:tasks.workunit.client.1.smithi152.stdout:6/252: creat d1/d3/d1a/d49/f57 x:0 0 0 2024-03-29T04:45:09.736 INFO:tasks.workunit.client.1.smithi152.stdout:0/171: rmdir d2/da/df 39 2024-03-29T04:45:09.737 INFO:tasks.workunit.client.1.smithi152.stdout:3/167: dread d7/d19/d1b/f23 [0,4194304] 0 2024-03-29T04:45:09.738 INFO:tasks.workunit.client.0.smithi142.stdout:0/764: rename d7/c17 to d7/d13/dbf/cf5 0 2024-03-29T04:45:09.738 INFO:tasks.workunit.client.0.smithi142.stdout:7/848: dread d7/d62/ff1 [0,4194304] 0 2024-03-29T04:45:09.739 INFO:tasks.workunit.client.1.smithi152.stdout:1/176: dread d5/d7/d8/f1e [0,4194304] 0 2024-03-29T04:45:09.739 INFO:tasks.workunit.client.0.smithi142.stdout:5/973: dwrite df/d16/d25/d120/d139/df1/d116/f13a [0,4194304] 0 2024-03-29T04:45:09.740 INFO:tasks.workunit.client.1.smithi152.stdout:1/177: stat d5/d7/d1b 0 2024-03-29T04:45:09.740 INFO:tasks.workunit.client.1.smithi152.stdout:8/134: dwrite d1/f10 [0,4194304] 0 2024-03-29T04:45:09.740 INFO:tasks.workunit.client.0.smithi142.stdout:0/765: stat d7/d9/d54/d58 0 2024-03-29T04:45:09.742 INFO:tasks.workunit.client.0.smithi142.stdout:1/727: unlink d8/db/dc/d77/db9/dbe/f48 0 2024-03-29T04:45:09.743 INFO:tasks.workunit.client.0.smithi142.stdout:5/974: fdatasync df/d16/d25/d120/d139/de0/f100 0 2024-03-29T04:45:09.744 INFO:tasks.workunit.client.0.smithi142.stdout:8/891: mkdir d5/d1d/d2f/d76/d2d/d13a 0 2024-03-29T04:45:09.745 INFO:tasks.workunit.client.1.smithi152.stdout:6/253: creat d1/d3/d1a/d1b/f58 x:0 0 0 2024-03-29T04:45:09.745 INFO:tasks.workunit.client.1.smithi152.stdout:0/172: mknod d2/d29/c3b 0 2024-03-29T04:45:09.746 INFO:tasks.workunit.client.0.smithi142.stdout:1/728: chown d8/f2b 1 1 2024-03-29T04:45:09.749 INFO:tasks.workunit.client.1.smithi152.stdout:1/178: mknod d5/d7/d1b/c42 0 2024-03-29T04:45:09.749 INFO:tasks.workunit.client.1.smithi152.stdout:8/135: write d1/d21/f25 [786034,15808] 0 2024-03-29T04:45:09.752 INFO:tasks.workunit.client.1.smithi152.stdout:6/254: mknod d1/d3/d1a/d1b/c59 0 2024-03-29T04:45:09.753 INFO:tasks.workunit.client.1.smithi152.stdout:0/173: creat d2/d31/f3c x:0 0 0 2024-03-29T04:45:09.754 INFO:tasks.workunit.client.0.smithi142.stdout:0/766: mkdir d7/df6 0 2024-03-29T04:45:09.755 INFO:tasks.workunit.client.0.smithi142.stdout:7/849: mknod d7/d27/d44/d4c/da3/dc2/dfe/c11b 0 2024-03-29T04:45:09.756 INFO:tasks.workunit.client.0.smithi142.stdout:1/729: read d8/db/fb3 [79423,24241] 0 2024-03-29T04:45:09.756 INFO:tasks.workunit.client.0.smithi142.stdout:6/739: dread d1c/d33/f5a [0,4194304] 0 2024-03-29T04:45:09.757 INFO:tasks.workunit.client.1.smithi152.stdout:3/168: dwrite d7/f1a [0,4194304] 0 2024-03-29T04:45:09.758 INFO:tasks.workunit.client.0.smithi142.stdout:8/892: dwrite d5/d1d/d2f/fd0 [0,4194304] 0 2024-03-29T04:45:09.759 INFO:tasks.workunit.client.0.smithi142.stdout:5/975: rename df/d1a/d21/fcf to df/d16/d24/f146 0 2024-03-29T04:45:09.760 INFO:tasks.workunit.client.1.smithi152.stdout:8/136: dwrite d1/f10 [0,4194304] 0 2024-03-29T04:45:09.760 INFO:tasks.workunit.client.1.smithi152.stdout:0/174: creat d2/da/d17/f3d x:0 0 0 2024-03-29T04:45:09.763 INFO:tasks.workunit.client.0.smithi142.stdout:7/850: chown d7/d27/d44/db2/db8/d4b/d71/d30/d33/c70 130255471 1 2024-03-29T04:45:09.764 INFO:tasks.workunit.client.1.smithi152.stdout:2/149: getdents d0 0 2024-03-29T04:45:09.764 INFO:tasks.workunit.client.0.smithi142.stdout:9/628: sync 2024-03-29T04:45:09.764 INFO:tasks.workunit.client.0.smithi142.stdout:1/730: dread - d8/db/dc/d77/db9/dbe/d2d/d44/d53/fca zero size 2024-03-29T04:45:09.764 INFO:tasks.workunit.client.1.smithi152.stdout:6/255: mknod d1/d3/d1a/c5a 0 2024-03-29T04:45:09.765 INFO:tasks.workunit.client.0.smithi142.stdout:6/740: dread d1c/d33/d46/d86/fcf [0,4194304] 0 2024-03-29T04:45:09.767 INFO:tasks.workunit.client.0.smithi142.stdout:2/760: dread de/d11/d1e/d22/d27/dba/d61/d7d/f7e [0,4194304] 0 2024-03-29T04:45:09.769 INFO:tasks.workunit.client.0.smithi142.stdout:0/767: mkdir d7/d9/d55/df7 0 2024-03-29T04:45:09.770 INFO:tasks.workunit.client.1.smithi152.stdout:3/169: write d7/ff [4076624,36855] 0 2024-03-29T04:45:09.771 INFO:tasks.workunit.client.0.smithi142.stdout:3/630: dread d3/d29/d2d/f48 [0,4194304] 0 2024-03-29T04:45:09.771 INFO:tasks.workunit.client.1.smithi152.stdout:4/146: dread d0/d7/f14 [0,4194304] 0 2024-03-29T04:45:09.773 INFO:tasks.workunit.client.0.smithi142.stdout:5/976: fdatasync df/d29/d6e/f10f 0 2024-03-29T04:45:09.775 INFO:tasks.workunit.client.1.smithi152.stdout:4/147: fdatasync d0/d7/de/ff 0 2024-03-29T04:45:09.775 INFO:tasks.workunit.client.1.smithi152.stdout:6/256: mknod d1/d3/d9/d13/d28/d45/c5b 0 2024-03-29T04:45:09.775 INFO:tasks.workunit.client.0.smithi142.stdout:2/761: write de/d11/d1e/d84/f98 [58403,20511] 0 2024-03-29T04:45:09.776 INFO:tasks.workunit.client.1.smithi152.stdout:0/175: mknod d2/c3e 0 2024-03-29T04:45:09.777 INFO:tasks.workunit.client.0.smithi142.stdout:6/741: mkdir d1c/dc7/df2 0 2024-03-29T04:45:09.778 INFO:tasks.workunit.client.0.smithi142.stdout:7/851: truncate d7/d27/d44/db2/db8/de/d8d/fb9 1000021 0 2024-03-29T04:45:09.780 INFO:tasks.workunit.client.1.smithi152.stdout:6/257: creat d1/d3/d1a/d49/f5c x:0 0 0 2024-03-29T04:45:09.781 INFO:tasks.workunit.client.1.smithi152.stdout:0/176: chown d2/da/d17/f3d 16 1 2024-03-29T04:45:09.781 INFO:tasks.workunit.client.1.smithi152.stdout:4/148: mkdir d0/d16/d22 0 2024-03-29T04:45:09.782 INFO:tasks.workunit.client.0.smithi142.stdout:0/768: rename d7/d9/d1e/c4c to d7/d13/dbf/dca/dab/ddf/cf8 0 2024-03-29T04:45:09.783 INFO:tasks.workunit.client.1.smithi152.stdout:8/137: dwrite d1/d2/f1a [0,4194304] 0 2024-03-29T04:45:09.784 INFO:tasks.workunit.client.1.smithi152.stdout:4/149: read d0/d7/f1e [6595119,81507] 0 2024-03-29T04:45:09.785 INFO:tasks.workunit.client.0.smithi142.stdout:6/742: write d1c/d45/d51/f7f [3097789,100569] 0 2024-03-29T04:45:09.787 INFO:tasks.workunit.client.0.smithi142.stdout:5/977: dwrite df/d16/d24/ff7 [0,4194304] 0 2024-03-29T04:45:09.787 INFO:tasks.workunit.client.0.smithi142.stdout:6/743: stat d1c/c80 0 2024-03-29T04:45:09.787 INFO:tasks.workunit.client.1.smithi152.stdout:3/170: dread d7/f1a [0,4194304] 0 2024-03-29T04:45:09.788 INFO:tasks.workunit.client.1.smithi152.stdout:2/150: dwrite d0/d3/d30/d15/d22/f2f [0,4194304] 0 2024-03-29T04:45:09.789 INFO:tasks.workunit.client.0.smithi142.stdout:7/852: dread - d7/d27/d44/d4c/f7b zero size 2024-03-29T04:45:09.790 INFO:tasks.workunit.client.0.smithi142.stdout:9/629: mknod d0/d2/d6/d27/d3c/d7b/cc4 0 2024-03-29T04:45:09.792 INFO:tasks.workunit.client.1.smithi152.stdout:0/177: mknod d2/da/d17/c3f 0 2024-03-29T04:45:09.792 INFO:tasks.workunit.client.1.smithi152.stdout:3/171: stat d7/d19/f25 0 2024-03-29T04:45:09.792 INFO:tasks.workunit.client.0.smithi142.stdout:6/744: write d1c/d45/d51/f58 [633080,21117] 0 2024-03-29T04:45:09.795 INFO:tasks.workunit.client.1.smithi152.stdout:0/178: fdatasync d2/f8 0 2024-03-29T04:45:09.796 INFO:tasks.workunit.client.1.smithi152.stdout:0/179: read d2/f2e [1902754,30551] 0 2024-03-29T04:45:09.797 INFO:tasks.workunit.client.0.smithi142.stdout:6/745: dread - d1c/d45/d51/fb4 zero size 2024-03-29T04:45:09.799 INFO:tasks.workunit.client.1.smithi152.stdout:6/258: dread d1/d3/d9/d13/f33 [0,4194304] 0 2024-03-29T04:45:09.800 INFO:tasks.workunit.client.1.smithi152.stdout:0/180: fdatasync d2/d31/f36 0 2024-03-29T04:45:09.802 INFO:tasks.workunit.client.1.smithi152.stdout:0/181: fdatasync d2/da/d17/f3d 0 2024-03-29T04:45:09.804 INFO:tasks.workunit.client.0.smithi142.stdout:7/853: dread d7/d27/d44/db2/db8/d4b/d71/d30/f99 [0,4194304] 0 2024-03-29T04:45:09.804 INFO:tasks.workunit.client.1.smithi152.stdout:3/172: creat d7/d19/d1b/d1e/d22/f2d x:0 0 0 2024-03-29T04:45:09.805 INFO:tasks.workunit.client.1.smithi152.stdout:4/150: dwrite d0/d7/f14 [0,4194304] 0 2024-03-29T04:45:09.805 INFO:tasks.workunit.client.0.smithi142.stdout:7/854: fdatasync d7/d27/d44/db2/db8/de/dc1/f10a 0 2024-03-29T04:45:09.806 INFO:tasks.workunit.client.1.smithi152.stdout:2/151: symlink d0/d3/l32 0 2024-03-29T04:45:09.808 INFO:tasks.workunit.client.1.smithi152.stdout:0/182: dread d2/f20 [0,4194304] 0 2024-03-29T04:45:09.808 INFO:tasks.workunit.client.1.smithi152.stdout:6/259: symlink d1/d3/d1a/d1b/l5d 0 2024-03-29T04:45:09.809 INFO:tasks.workunit.client.0.smithi142.stdout:8/893: dread d5/f25 [0,4194304] 0 2024-03-29T04:45:09.809 INFO:tasks.workunit.client.1.smithi152.stdout:2/152: chown d0/d3/f5 1 1 2024-03-29T04:45:09.810 INFO:tasks.workunit.client.0.smithi142.stdout:6/746: creat d1c/d33/d46/d86/dc0/ff3 x:0 0 0 2024-03-29T04:45:09.813 INFO:tasks.workunit.client.0.smithi142.stdout:6/747: read fc [2371885,108268] 0 2024-03-29T04:45:09.815 INFO:tasks.workunit.client.0.smithi142.stdout:6/748: write f18 [3532187,21556] 0 2024-03-29T04:45:09.815 INFO:tasks.workunit.client.0.smithi142.stdout:7/855: dread d7/d27/d44/db2/db8/de/d20/d3f/f65 [0,4194304] 0 2024-03-29T04:45:09.816 INFO:tasks.workunit.client.0.smithi142.stdout:5/978: creat df/d29/d140/f147 x:0 0 0 2024-03-29T04:45:09.818 INFO:tasks.workunit.client.0.smithi142.stdout:7/856: truncate d7/f1a 5036371 0 2024-03-29T04:45:09.820 INFO:tasks.workunit.client.1.smithi152.stdout:2/153: fdatasync d0/d3/f2e 0 2024-03-29T04:45:09.822 INFO:tasks.workunit.client.0.smithi142.stdout:4/768: sync 2024-03-29T04:45:09.823 INFO:tasks.workunit.client.1.smithi152.stdout:1/179: write d5/f3c [4006,66391] 0 2024-03-29T04:45:09.825 INFO:tasks.workunit.client.1.smithi152.stdout:2/154: chown d0/c2a 305202615 1 2024-03-29T04:45:09.826 INFO:tasks.workunit.client.0.smithi142.stdout:1/731: rename d8/db/dc/d77/db9/dbe/d1f/d4c/c68 to d8/db/dc/d77/db9/dbe/d1f/d4c/cfd 0 2024-03-29T04:45:09.829 INFO:tasks.workunit.client.0.smithi142.stdout:3/631: rename d3/d29/d8f/d5f/d64/d6b/l84 to d3/d29/d2b/d40/d67/d8b/lda 0 2024-03-29T04:45:09.829 INFO:tasks.workunit.client.1.smithi152.stdout:3/173: dwrite d7/d19/d1b/f21 [0,4194304] 0 2024-03-29T04:45:09.829 INFO:tasks.workunit.client.1.smithi152.stdout:4/151: dwrite d0/f4 [0,4194304] 0 2024-03-29T04:45:09.829 INFO:tasks.workunit.client.0.smithi142.stdout:0/769: link d7/d13/l18 d7/d13/dbf/dca/dab/dc1/dd3/de1/lf9 0 2024-03-29T04:45:09.830 INFO:tasks.workunit.client.0.smithi142.stdout:3/632: chown d3/d29/d2d/d79/lba 3 1 2024-03-29T04:45:09.834 INFO:tasks.workunit.client.0.smithi142.stdout:0/770: chown d7/d13/dbf/dca/dab/dc1/dd3/c9d 8 1 2024-03-29T04:45:09.836 INFO:tasks.workunit.client.0.smithi142.stdout:0/771: write d7/d13/dbf/d72/d75/fd7 [538689,30805] 0 2024-03-29T04:45:09.837 INFO:tasks.workunit.client.0.smithi142.stdout:4/769: creat d0/dc/d15/d3d/d65/ffa x:0 0 0 2024-03-29T04:45:09.837 INFO:tasks.workunit.client.1.smithi152.stdout:1/180: dread - d5/d7/d8/f2d zero size 2024-03-29T04:45:09.837 INFO:tasks.workunit.client.1.smithi152.stdout:6/260: dwrite d1/d3/d9/f41 [0,4194304] 0 2024-03-29T04:45:09.837 INFO:tasks.workunit.client.1.smithi152.stdout:0/183: dwrite d2/f20 [0,4194304] 0 2024-03-29T04:45:09.838 INFO:tasks.workunit.client.1.smithi152.stdout:0/184: readlink d2/da/d17/l1e 0 2024-03-29T04:45:09.839 INFO:tasks.workunit.client.1.smithi152.stdout:6/261: read d1/d3/d9/f21 [1008997,115463] 0 2024-03-29T04:45:09.840 INFO:tasks.workunit.client.0.smithi142.stdout:2/762: dread de/d11/d1e/d22/d27/f30 [0,4194304] 0 2024-03-29T04:45:09.841 INFO:tasks.workunit.client.0.smithi142.stdout:8/894: dread d5/d2e/f65 [0,4194304] 0 2024-03-29T04:45:09.843 INFO:tasks.workunit.client.1.smithi152.stdout:2/155: creat d0/d3/d30/f33 x:0 0 0 2024-03-29T04:45:09.844 INFO:tasks.workunit.client.0.smithi142.stdout:0/772: dwrite d7/d9/d1e/d81/fc4 [0,4194304] 0 2024-03-29T04:45:09.846 INFO:tasks.workunit.client.1.smithi152.stdout:1/181: mknod d5/d7/d9/c43 0 2024-03-29T04:45:09.847 INFO:tasks.workunit.client.0.smithi142.stdout:4/770: mknod d0/d9/d11/d4c/d64/cfb 0 2024-03-29T04:45:09.850 INFO:tasks.workunit.client.0.smithi142.stdout:9/630: truncate d0/d12/f60 17059 0 2024-03-29T04:45:09.850 INFO:tasks.workunit.client.1.smithi152.stdout:2/156: write d0/d3/f5 [1232000,66780] 0 2024-03-29T04:45:09.851 INFO:tasks.workunit.client.0.smithi142.stdout:9/631: stat d0/d2/d6/d27/f82 0 2024-03-29T04:45:09.851 INFO:tasks.workunit.client.1.smithi152.stdout:0/185: dread d2/f4 [0,4194304] 0 2024-03-29T04:45:09.851 INFO:tasks.workunit.client.0.smithi142.stdout:1/732: creat d8/db/dc/d77/db9/ffe x:0 0 0 2024-03-29T04:45:09.852 INFO:tasks.workunit.client.1.smithi152.stdout:4/152: dread d0/d7/de/ff [0,4194304] 0 2024-03-29T04:45:09.853 INFO:tasks.workunit.client.0.smithi142.stdout:1/733: write d8/db/dc/d77/fdf [750834,7704] 0 2024-03-29T04:45:09.855 INFO:tasks.workunit.client.1.smithi152.stdout:6/262: creat d1/d3/d1a/d49/f5e x:0 0 0 2024-03-29T04:45:09.855 INFO:tasks.workunit.client.1.smithi152.stdout:3/174: truncate d7/f18 897627 0 2024-03-29T04:45:09.855 INFO:tasks.workunit.client.0.smithi142.stdout:3/633: creat d3/d29/d2b/d40/d67/fdb x:0 0 0 2024-03-29T04:45:09.858 INFO:tasks.workunit.client.1.smithi152.stdout:2/157: mknod d0/d3/df/d2c/c34 0 2024-03-29T04:45:09.858 INFO:tasks.workunit.client.0.smithi142.stdout:2/763: unlink de/d11/d1e/d84/lda 0 2024-03-29T04:45:09.859 INFO:tasks.workunit.client.0.smithi142.stdout:5/979: getdents df/d1a/d21/d55 0 2024-03-29T04:45:09.860 INFO:tasks.workunit.client.1.smithi152.stdout:1/182: dread d5/d7/d9/f11 [0,4194304] 0 2024-03-29T04:45:09.860 INFO:tasks.workunit.client.1.smithi152.stdout:2/158: truncate d0/d3/f31 260358 0 2024-03-29T04:45:09.863 INFO:tasks.workunit.client.1.smithi152.stdout:3/175: dwrite d7/d19/f26 [0,4194304] 0 2024-03-29T04:45:09.863 INFO:tasks.workunit.client.1.smithi152.stdout:4/153: getdents d0/d7 0 2024-03-29T04:45:09.864 INFO:tasks.workunit.client.0.smithi142.stdout:0/773: rename d7/d13/dbf/d72/f3a to d7/d13/dbf/d72/d75/ffa 0 2024-03-29T04:45:09.864 INFO:tasks.workunit.client.1.smithi152.stdout:2/159: write d0/d3/d30/d15/f2b [799523,20233] 0 2024-03-29T04:45:09.866 INFO:tasks.workunit.client.1.smithi152.stdout:4/154: rmdir d0/d7/de 39 2024-03-29T04:45:09.867 INFO:tasks.workunit.client.1.smithi152.stdout:4/155: chown d0/d7 300 1 2024-03-29T04:45:09.869 INFO:tasks.workunit.client.1.smithi152.stdout:2/160: creat d0/d3/df/f35 x:0 0 0 2024-03-29T04:45:09.874 INFO:tasks.workunit.client.0.smithi142.stdout:7/857: getdents d7/d27/d44/db2/dc4 0 2024-03-29T04:45:09.874 INFO:tasks.workunit.client.0.smithi142.stdout:9/632: creat d0/d2/d39/fc5 x:0 0 0 2024-03-29T04:45:09.874 INFO:tasks.workunit.client.1.smithi152.stdout:4/156: mkdir d0/d7/de/d23 0 2024-03-29T04:45:09.874 INFO:tasks.workunit.client.0.smithi142.stdout:0/774: chown d7/d13/d4a 82634 1 2024-03-29T04:45:09.876 INFO:tasks.workunit.client.0.smithi142.stdout:7/858: stat d7/d27/d44/db2/dd2 0 2024-03-29T04:45:09.877 INFO:tasks.workunit.client.0.smithi142.stdout:1/734: dwrite d8/db/dc/d77/db9/dbe/d15/d9c/fc2 [0,4194304] 0 2024-03-29T04:45:09.877 INFO:tasks.workunit.client.0.smithi142.stdout:3/634: mknod d3/d29/d8f/d95/cdc 0 2024-03-29T04:45:09.879 INFO:tasks.workunit.client.1.smithi152.stdout:2/161: dread d0/d3/f18 [0,4194304] 0 2024-03-29T04:45:09.881 INFO:tasks.workunit.client.0.smithi142.stdout:6/749: dwrite d1c/f62 [4194304,4194304] 0 2024-03-29T04:45:09.885 INFO:tasks.workunit.client.1.smithi152.stdout:4/157: dwrite d0/d7/f14 [4194304,4194304] 0 2024-03-29T04:45:09.885 INFO:tasks.workunit.client.1.smithi152.stdout:2/162: creat d0/d3/df/f36 x:0 0 0 2024-03-29T04:45:09.885 INFO:tasks.workunit.client.0.smithi142.stdout:2/764: dread de/d11/d17/d5d/d71/f9f [0,4194304] 0 2024-03-29T04:45:09.886 INFO:tasks.workunit.client.0.smithi142.stdout:7/859: dread d7/d27/d44/d4c/f8c [0,4194304] 0 2024-03-29T04:45:09.886 INFO:tasks.workunit.client.0.smithi142.stdout:9/633: dread - d0/d2/d6/d16/daa/f87 zero size 2024-03-29T04:45:09.886 INFO:tasks.workunit.client.0.smithi142.stdout:8/895: dread d5/d1d/d2f/d76/d2d/f4f [4194304,4194304] 0 2024-03-29T04:45:09.887 INFO:tasks.workunit.client.1.smithi152.stdout:4/158: write d0/f15 [58751,56738] 0 2024-03-29T04:45:09.887 INFO:tasks.workunit.client.0.smithi142.stdout:3/635: fdatasync d3/d1f/d24/fa5 0 2024-03-29T04:45:09.890 INFO:tasks.workunit.client.1.smithi152.stdout:2/163: write d0/d3/df/f24 [814589,118413] 0 2024-03-29T04:45:09.890 INFO:tasks.workunit.client.0.smithi142.stdout:1/735: stat d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/fd8 0 2024-03-29T04:45:09.891 INFO:tasks.workunit.client.0.smithi142.stdout:5/980: dwrite df/d1a/d21/d55/d66/f13d [0,4194304] 0 2024-03-29T04:45:09.894 INFO:tasks.workunit.client.1.smithi152.stdout:0/186: dread d2/f5 [0,4194304] 0 2024-03-29T04:45:09.896 INFO:tasks.workunit.client.0.smithi142.stdout:5/981: chown df/d16/d24/d2e/f8f 6564345 1 2024-03-29T04:45:09.897 INFO:tasks.workunit.client.1.smithi152.stdout:4/159: write d0/f13 [1604778,88801] 0 2024-03-29T04:45:09.899 INFO:tasks.workunit.client.0.smithi142.stdout:5/982: dread - df/d1a/d21/d55/d66/f108 zero size 2024-03-29T04:45:09.899 INFO:tasks.workunit.client.0.smithi142.stdout:9/634: dwrite d0/f1d [4194304,4194304] 0 2024-03-29T04:45:09.901 INFO:tasks.workunit.client.1.smithi152.stdout:0/187: rename d2/da/c10 to d2/d29/c40 0 2024-03-29T04:45:09.901 INFO:tasks.workunit.client.1.smithi152.stdout:4/160: rename d0/d7/de/d23 to d0/d7/de/d23/d24 22 2024-03-29T04:45:09.902 INFO:tasks.workunit.client.0.smithi142.stdout:2/765: dread de/d11/d1e/d22/d27/dba/d61/d7d/f7e [0,4194304] 0 2024-03-29T04:45:09.904 INFO:tasks.workunit.client.0.smithi142.stdout:2/766: chown de/d11/d1e/d22/d27/dba/c8a 1 1 2024-03-29T04:45:09.910 INFO:tasks.workunit.client.1.smithi152.stdout:2/164: dwrite d0/d3/df/f35 [0,4194304] 0 2024-03-29T04:45:09.910 INFO:tasks.workunit.client.1.smithi152.stdout:0/188: dread - d2/d31/f36 zero size 2024-03-29T04:45:09.911 INFO:tasks.workunit.client.1.smithi152.stdout:1/183: dread d5/d7/d9/f10 [4194304,4194304] 0 2024-03-29T04:45:09.915 INFO:tasks.workunit.client.1.smithi152.stdout:3/176: dread d7/f15 [0,4194304] 0 2024-03-29T04:45:09.916 INFO:tasks.workunit.client.1.smithi152.stdout:4/161: rename d0/d16 to d0/d7/d25 0 2024-03-29T04:45:09.916 INFO:tasks.workunit.client.1.smithi152.stdout:1/184: symlink d5/d7/d9/l44 0 2024-03-29T04:45:09.918 INFO:tasks.workunit.client.1.smithi152.stdout:0/189: creat d2/da/d17/f41 x:0 0 0 2024-03-29T04:45:09.918 INFO:tasks.workunit.client.1.smithi152.stdout:1/185: chown d5/f3c 1639 1 2024-03-29T04:45:09.919 INFO:tasks.workunit.client.1.smithi152.stdout:3/177: write d7/f1a [1832037,26614] 0 2024-03-29T04:45:09.919 INFO:tasks.workunit.client.1.smithi152.stdout:4/162: creat d0/d7/d25/f26 x:0 0 0 2024-03-29T04:45:09.920 INFO:tasks.workunit.client.1.smithi152.stdout:0/190: write d2/d31/f36 [437661,15108] 0 2024-03-29T04:45:09.920 INFO:tasks.workunit.client.1.smithi152.stdout:2/165: link d0/d3/d30/f1b d0/d3/d30/d15/d22/f37 0 2024-03-29T04:45:09.921 INFO:tasks.workunit.client.0.smithi142.stdout:4/771: rename d0/f21 to d0/dc/d15/d91/ffc 0 2024-03-29T04:45:09.923 INFO:tasks.workunit.client.1.smithi152.stdout:2/166: chown d0/d3/d30/f33 13494 1 2024-03-29T04:45:09.923 INFO:tasks.workunit.client.1.smithi152.stdout:3/178: chown d7/d19/f26 1373 1 2024-03-29T04:45:09.924 INFO:tasks.workunit.client.0.smithi142.stdout:0/775: rmdir d7/d13/d4a/d9a/da7 39 2024-03-29T04:45:09.925 INFO:tasks.workunit.client.1.smithi152.stdout:0/191: symlink d2/l42 0 2024-03-29T04:45:09.926 INFO:tasks.workunit.client.0.smithi142.stdout:6/750: mkdir d1c/d33/db6/d52/df4 0 2024-03-29T04:45:09.927 INFO:tasks.workunit.client.1.smithi152.stdout:2/167: creat d0/d3/d30/d15/d22/f38 x:0 0 0 2024-03-29T04:45:09.927 INFO:tasks.workunit.client.0.smithi142.stdout:6/751: dread - d1c/d45/fa5 zero size 2024-03-29T04:45:09.927 INFO:tasks.workunit.client.1.smithi152.stdout:1/186: creat d5/d7/d8/d13/f45 x:0 0 0 2024-03-29T04:45:09.928 INFO:tasks.workunit.client.0.smithi142.stdout:3/636: fdatasync d3/d29/d8f/d5f/db4/fa1 0 2024-03-29T04:45:09.928 INFO:tasks.workunit.client.1.smithi152.stdout:3/179: truncate d7/d19/d1b/f1d 1179851 0 2024-03-29T04:45:09.930 INFO:tasks.workunit.client.1.smithi152.stdout:0/192: mknod d2/da/d3a/c43 0 2024-03-29T04:45:09.930 INFO:tasks.workunit.client.0.smithi142.stdout:8/896: mkdir d5/d1d/d2f/d76/d3c/d70/d13b 0 2024-03-29T04:45:09.930 INFO:tasks.workunit.client.1.smithi152.stdout:4/163: dwrite d0/f12 [4194304,4194304] 0 2024-03-29T04:45:09.933 INFO:tasks.workunit.client.0.smithi142.stdout:2/767: creat de/d11/da1/fff x:0 0 0 2024-03-29T04:45:09.936 INFO:tasks.workunit.client.1.smithi152.stdout:0/193: fsync d2/f5 0 2024-03-29T04:45:09.936 INFO:tasks.workunit.client.0.smithi142.stdout:1/736: rename d8/db/dc/d77/db9/dbe/d2d/d44/d53/df5 to d8/db/dc/d77/db9/dbe/d2d/d44/dff 0 2024-03-29T04:45:09.939 INFO:tasks.workunit.client.0.smithi142.stdout:4/772: dread d0/d9/d11/f23 [0,4194304] 0 2024-03-29T04:45:09.940 INFO:tasks.workunit.client.0.smithi142.stdout:0/776: creat d7/d13/dbf/dca/dab/ddf/ffb x:0 0 0 2024-03-29T04:45:09.942 INFO:tasks.workunit.client.0.smithi142.stdout:7/860: link d7/fd d7/d27/d44/db2/db8/d4b/d71/d30/d33/f11c 0 2024-03-29T04:45:09.943 INFO:tasks.workunit.client.0.smithi142.stdout:7/861: readlink d7/d27/d44/db2/db8/d4b/lcb 0 2024-03-29T04:45:09.943 INFO:tasks.workunit.client.1.smithi152.stdout:3/180: dwrite d7/f1a [0,4194304] 0 2024-03-29T04:45:09.944 INFO:tasks.workunit.client.1.smithi152.stdout:1/187: dwrite d5/d7/d9/d2b/f37 [0,4194304] 0 2024-03-29T04:45:09.945 INFO:tasks.workunit.client.0.smithi142.stdout:8/897: fdatasync d5/d1d/d2f/d45/f95 0 2024-03-29T04:45:09.948 INFO:tasks.workunit.client.0.smithi142.stdout:5/983: mkdir df/d16/d24/d3d/d12e/d148 0 2024-03-29T04:45:09.948 INFO:tasks.workunit.client.1.smithi152.stdout:4/164: rename d0/d7/d25/d22 to d0/d27 0 2024-03-29T04:45:09.952 INFO:tasks.workunit.client.1.smithi152.stdout:1/188: dread d5/f20 [0,4194304] 0 2024-03-29T04:45:09.953 INFO:tasks.workunit.client.0.smithi142.stdout:2/768: creat de/d11/d17/d60/d6f/f100 x:0 0 0 2024-03-29T04:45:09.953 INFO:tasks.workunit.client.1.smithi152.stdout:3/181: mknod d7/d19/d1b/c2e 0 2024-03-29T04:45:09.955 INFO:tasks.workunit.client.0.smithi142.stdout:1/737: fsync d8/db/dc/d77/d31/d4d/d8c/f95 0 2024-03-29T04:45:09.956 INFO:tasks.workunit.client.1.smithi152.stdout:1/189: fdatasync d5/d7/d9/f10 0 2024-03-29T04:45:09.957 INFO:tasks.workunit.client.0.smithi142.stdout:0/777: mknod d7/d9/d54/d58/cfc 0 2024-03-29T04:45:09.958 INFO:tasks.workunit.client.1.smithi152.stdout:0/194: getdents d2 0 2024-03-29T04:45:09.959 INFO:tasks.workunit.client.0.smithi142.stdout:6/752: mknod d1c/dc7/df2/cf5 0 2024-03-29T04:45:09.961 INFO:tasks.workunit.client.0.smithi142.stdout:4/773: dread d0/d9/d11/d4c/d5f/f6c [0,4194304] 0 2024-03-29T04:45:09.962 INFO:tasks.workunit.client.1.smithi152.stdout:4/165: rename d0/d7/f1e to d0/d7/de/d23/f28 0 2024-03-29T04:45:09.963 INFO:tasks.workunit.client.1.smithi152.stdout:0/195: dread d2/d31/f36 [0,4194304] 0 2024-03-29T04:45:09.963 INFO:tasks.workunit.client.0.smithi142.stdout:8/898: dread d5/d1d/d24/f6b [0,4194304] 0 2024-03-29T04:45:09.964 INFO:tasks.workunit.client.1.smithi152.stdout:2/168: dwrite d0/d3/d30/d15/d22/f37 [0,4194304] 0 2024-03-29T04:45:09.965 INFO:tasks.workunit.client.1.smithi152.stdout:1/190: unlink d5/d7/d8/d13/c23 0 2024-03-29T04:45:09.966 INFO:tasks.workunit.client.0.smithi142.stdout:2/769: unlink de/d11/d1e/d22/d27/f30 0 2024-03-29T04:45:09.967 INFO:tasks.workunit.client.0.smithi142.stdout:2/770: dread - de/d11/f87 zero size 2024-03-29T04:45:09.968 INFO:tasks.workunit.client.1.smithi152.stdout:0/196: creat d2/d29/f44 x:0 0 0 2024-03-29T04:45:09.969 INFO:tasks.workunit.client.1.smithi152.stdout:2/169: unlink d0/d3/df/f1e 0 2024-03-29T04:45:09.970 INFO:tasks.workunit.client.0.smithi142.stdout:1/738: mknod d8/db/dc/d77/db9/dbe/d2d/d44/c100 0 2024-03-29T04:45:09.971 INFO:tasks.workunit.client.1.smithi152.stdout:1/191: readlink d5/d7/d8/d13/d19/l1a 0 2024-03-29T04:45:09.974 INFO:tasks.workunit.client.0.smithi142.stdout:1/739: dread d8/db/dc/d77/db9/dbe/d2d/d44/d7f/fb0 [0,4194304] 0 2024-03-29T04:45:09.974 INFO:tasks.workunit.client.0.smithi142.stdout:7/862: rename d7/f31 to d7/d27/d3a/d92/d97/dea/f11d 0 2024-03-29T04:45:09.976 INFO:tasks.workunit.client.1.smithi152.stdout:4/166: rename d0/f12 to d0/d7/f29 0 2024-03-29T04:45:09.976 INFO:tasks.workunit.client.0.smithi142.stdout:3/637: creat d3/d29/d2b/d40/d67/d8b/fdd x:0 0 0 2024-03-29T04:45:09.977 INFO:tasks.workunit.client.1.smithi152.stdout:1/192: dwrite d5/f3c [0,4194304] 0 2024-03-29T04:45:09.977 INFO:tasks.workunit.client.1.smithi152.stdout:3/182: write d7/d19/d1b/f1d [1544130,4534] 0 2024-03-29T04:45:09.978 INFO:tasks.workunit.client.1.smithi152.stdout:2/170: creat d0/d3/f39 x:0 0 0 2024-03-29T04:45:09.979 INFO:tasks.workunit.client.0.smithi142.stdout:4/774: creat d0/d9/d11/d4c/d5f/ffd x:0 0 0 2024-03-29T04:45:09.981 INFO:tasks.workunit.client.1.smithi152.stdout:4/167: write d0/d7/f29 [2886233,99756] 0 2024-03-29T04:45:09.982 INFO:tasks.workunit.client.1.smithi152.stdout:1/193: creat d5/d7/d9/f46 x:0 0 0 2024-03-29T04:45:09.983 INFO:tasks.workunit.client.1.smithi152.stdout:4/168: mkdir d0/d7/de/d2a 0 2024-03-29T04:45:09.985 INFO:tasks.workunit.client.0.smithi142.stdout:0/778: truncate d7/d13/dbf/d72/d75/fb5 807440 0 2024-03-29T04:45:09.985 INFO:tasks.workunit.client.1.smithi152.stdout:0/197: rename d2/d29/c3b to d2/da/c45 0 2024-03-29T04:45:09.987 INFO:tasks.workunit.client.1.smithi152.stdout:1/194: write d5/d7/d8/f1e [958704,63386] 0 2024-03-29T04:45:09.988 INFO:tasks.workunit.client.0.smithi142.stdout:6/753: rename d1c/d33/db6/d52 to d1c/d45/d8a/df6 0 2024-03-29T04:45:09.989 INFO:tasks.workunit.client.1.smithi152.stdout:3/183: link d7/d19/d1b/f1d d7/d19/d1b/f2f 0 2024-03-29T04:45:09.991 INFO:tasks.workunit.client.0.smithi142.stdout:7/863: creat d7/d27/d44/db2/db8/de/d20/ddf/f11e x:0 0 0 2024-03-29T04:45:09.992 INFO:tasks.workunit.client.1.smithi152.stdout:4/169: creat d0/d27/f2b x:0 0 0 2024-03-29T04:45:09.992 INFO:tasks.workunit.client.1.smithi152.stdout:1/195: rmdir d5/d7/d8/d13/d19 39 2024-03-29T04:45:09.994 INFO:tasks.workunit.client.1.smithi152.stdout:0/198: dwrite d2/d31/f3c [0,4194304] 0 2024-03-29T04:45:09.995 INFO:tasks.workunit.client.0.smithi142.stdout:8/899: symlink d5/d1d/d24/d66/dce/l13c 0 2024-03-29T04:45:09.995 INFO:tasks.workunit.client.1.smithi152.stdout:2/171: write d0/d3/d30/d15/f20 [774522,80818] 0 2024-03-29T04:45:09.998 INFO:tasks.workunit.client.0.smithi142.stdout:2/771: mkdir de/d11/d17/d60/de6/d101 0 2024-03-29T04:45:10.000 INFO:tasks.workunit.client.1.smithi152.stdout:1/196: stat d5/d7/d8/d13/d19/f3d 0 2024-03-29T04:45:10.000 INFO:tasks.workunit.client.1.smithi152.stdout:0/199: mknod d2/d29/c46 0 2024-03-29T04:45:10.002 INFO:tasks.workunit.client.1.smithi152.stdout:3/184: write d7/d1c/f2c [752314,32842] 0 2024-03-29T04:45:10.003 INFO:tasks.workunit.client.0.smithi142.stdout:1/740: symlink d8/db/dc/d77/de9/l101 0 2024-03-29T04:45:10.006 INFO:tasks.workunit.client.1.smithi152.stdout:1/197: creat d5/d7/d1b/f47 x:0 0 0 2024-03-29T04:45:10.006 INFO:tasks.workunit.client.1.smithi152.stdout:4/170: dwrite d0/d7/d25/f26 [0,4194304] 0 2024-03-29T04:45:10.007 INFO:tasks.workunit.client.0.smithi142.stdout:9/635: truncate d0/d12/d50/d52/f78 569414 0 2024-03-29T04:45:10.007 INFO:tasks.workunit.client.0.smithi142.stdout:9/636: fdatasync d0/d2/d39/fc5 0 2024-03-29T04:45:10.008 INFO:tasks.workunit.client.1.smithi152.stdout:4/171: fdatasync d0/f15 0 2024-03-29T04:45:10.008 INFO:tasks.workunit.client.0.smithi142.stdout:7/864: creat d7/d27/d44/db2/db8/de/dc1/f11f x:0 0 0 2024-03-29T04:45:10.010 INFO:tasks.workunit.client.0.smithi142.stdout:4/775: fdatasync d0/d9/db2/dba/fe6 0 2024-03-29T04:45:10.012 INFO:tasks.workunit.client.0.smithi142.stdout:4/776: write d0/d9/d11/d4c/d5f/f6f [487411,121929] 0 2024-03-29T04:45:10.013 INFO:tasks.workunit.client.1.smithi152.stdout:1/198: write d5/d7/d9/f10 [299141,35616] 0 2024-03-29T04:45:10.013 INFO:tasks.workunit.client.1.smithi152.stdout:3/185: rename d7/ff to d7/d19/f30 0 2024-03-29T04:45:10.013 INFO:tasks.workunit.client.1.smithi152.stdout:2/172: dread d0/d3/d30/d15/f25 [0,4194304] 0 2024-03-29T04:45:10.016 INFO:tasks.workunit.client.0.smithi142.stdout:0/779: symlink d7/d9/lfd 0 2024-03-29T04:45:10.019 INFO:tasks.workunit.client.0.smithi142.stdout:8/900: dread d5/d1d/f85 [0,4194304] 0 2024-03-29T04:45:10.020 INFO:tasks.workunit.client.0.smithi142.stdout:9/637: fdatasync d0/d2/d6/d16/faf 0 2024-03-29T04:45:10.021 INFO:tasks.workunit.client.0.smithi142.stdout:7/865: symlink d7/d27/d44/db2/db8/de/d20/d86/dd4/l120 0 2024-03-29T04:45:10.022 INFO:tasks.workunit.client.0.smithi142.stdout:5/984: dwrite df/d16/d24/d2e/d47/d4d/d102/d75/fbb [0,4194304] 0 2024-03-29T04:45:10.028 INFO:tasks.workunit.client.0.smithi142.stdout:7/866: dwrite d7/d62/f114 [0,4194304] 0 2024-03-29T04:45:10.029 INFO:tasks.workunit.client.0.smithi142.stdout:2/772: rename de/d11/d17/d60/ld7 to de/d11/d23/dd1/l102 0 2024-03-29T04:45:10.031 INFO:tasks.workunit.client.0.smithi142.stdout:0/780: creat d7/d9/d55/d42/ffe x:0 0 0 2024-03-29T04:45:10.032 INFO:tasks.workunit.client.0.smithi142.stdout:2/773: truncate de/d11/d1e/d22/d50/fdb 1134335 0 2024-03-29T04:45:10.032 INFO:tasks.workunit.client.0.smithi142.stdout:0/781: chown d7/d9/d54/d58/f6b 0 1 2024-03-29T04:45:10.033 INFO:tasks.workunit.client.0.smithi142.stdout:3/638: link d3/d29/d2b/d40/cac d3/d29/d2b/d40/d67/d8b/cde 0 2024-03-29T04:45:10.035 INFO:tasks.workunit.client.0.smithi142.stdout:8/901: mkdir d5/d1d/d81/d13d 0 2024-03-29T04:45:10.038 INFO:tasks.workunit.client.0.smithi142.stdout:9/638: mkdir d0/d2/d6/d27/dc6 0 2024-03-29T04:45:10.039 INFO:tasks.workunit.client.0.smithi142.stdout:9/639: dread - d0/d6c/fa1 zero size 2024-03-29T04:45:10.039 INFO:tasks.workunit.client.0.smithi142.stdout:5/985: truncate df/d1a/d21/d27/f36 345279 0 2024-03-29T04:45:10.039 INFO:tasks.workunit.client.0.smithi142.stdout:9/640: write d0/d2/d6/d27/d3c/f72 [25286,98388] 0 2024-03-29T04:45:10.040 INFO:tasks.workunit.client.0.smithi142.stdout:6/754: write d1c/d45/d8a/df6/f8d [954126,87142] 0 2024-03-29T04:45:10.042 INFO:tasks.workunit.client.0.smithi142.stdout:5/986: stat df/d16/d25/d120/d139/df1 0 2024-03-29T04:45:10.044 INFO:tasks.workunit.client.0.smithi142.stdout:5/987: stat df/d1a/d21/d27/d45/dbf/lc7 0 2024-03-29T04:45:10.045 INFO:tasks.workunit.client.0.smithi142.stdout:5/988: write df/d16/f39 [769750,123966] 0 2024-03-29T04:45:10.047 INFO:tasks.workunit.client.0.smithi142.stdout:0/782: chown d7/d13/dbf/d72/fb8 11101 1 2024-03-29T04:45:10.047 INFO:tasks.workunit.client.0.smithi142.stdout:6/755: dwrite d1c/f62 [0,4194304] 0 2024-03-29T04:45:10.048 INFO:tasks.workunit.client.0.smithi142.stdout:8/902: stat d5/d2e/dd2/c11a 0 2024-03-29T04:45:10.051 INFO:tasks.workunit.client.0.smithi142.stdout:8/903: write d5/d2e/dd2/d60/f8f [5322087,13722] 0 2024-03-29T04:45:10.054 INFO:tasks.workunit.client.0.smithi142.stdout:1/741: dwrite d8/db/dc/d77/db9/dbe/d1f/d50/fc3 [0,4194304] 0 2024-03-29T04:45:10.056 INFO:tasks.workunit.client.0.smithi142.stdout:1/742: write d8/db/dc/d77/db9/dbe/d2d/d44/d53/f7b [847439,68941] 0 2024-03-29T04:45:10.058 INFO:tasks.workunit.client.0.smithi142.stdout:1/743: write d8/db/d5e/f73 [1829084,57775] 0 2024-03-29T04:45:10.067 INFO:tasks.workunit.client.0.smithi142.stdout:7/867: rename d7/d27/d44/db2/db8/de to d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121 0 2024-03-29T04:45:10.068 INFO:tasks.workunit.client.0.smithi142.stdout:3/639: dread d3/d29/d2b/d40/d67/d8b/f91 [0,4194304] 0 2024-03-29T04:45:10.072 INFO:tasks.workunit.client.0.smithi142.stdout:2/774: link de/d11/d1e/d22/ff3 de/d11/d1e/d84/f103 0 2024-03-29T04:45:10.073 INFO:tasks.workunit.client.0.smithi142.stdout:0/783: dread - d7/d13/dbf/dca/fc3 zero size 2024-03-29T04:45:10.075 INFO:tasks.workunit.client.0.smithi142.stdout:6/756: mkdir d1c/d45/d8a/df6/df7 0 2024-03-29T04:45:10.078 INFO:tasks.workunit.client.0.smithi142.stdout:8/904: symlink d5/d2e/dd2/d60/d12d/l13e 0 2024-03-29T04:45:10.082 INFO:tasks.workunit.client.0.smithi142.stdout:1/744: mknod d8/db/dc/d77/db9/dbe/d2d/d8b/dda/c102 0 2024-03-29T04:45:10.083 INFO:tasks.workunit.client.0.smithi142.stdout:4/777: dwrite d0/d9/db2/fb3 [0,4194304] 0 2024-03-29T04:45:10.083 INFO:tasks.workunit.client.0.smithi142.stdout:7/868: stat d7/d27/d44/dab/fda 0 2024-03-29T04:45:10.085 INFO:tasks.workunit.client.0.smithi142.stdout:1/745: readlink d8/db/dc/d77/db9/dbe/d2d/d44/d7f/lc7 0 2024-03-29T04:45:10.086 INFO:tasks.workunit.client.0.smithi142.stdout:0/784: dread d7/d9/d6e/f6f [0,4194304] 0 2024-03-29T04:45:10.089 INFO:tasks.workunit.client.0.smithi142.stdout:3/640: dread d3/d29/d2b/d40/d67/d8b/d6/f2f [0,4194304] 0 2024-03-29T04:45:10.095 INFO:tasks.workunit.client.0.smithi142.stdout:4/778: unlink d0/d9/d19/d2c/fb9 0 2024-03-29T04:45:10.095 INFO:tasks.workunit.client.0.smithi142.stdout:4/779: readlink d0/l46 0 2024-03-29T04:45:10.097 INFO:tasks.workunit.client.0.smithi142.stdout:2/775: link de/d11/d17/d60/ff8 de/d11/d1e/d84/f104 0 2024-03-29T04:45:10.097 INFO:tasks.workunit.client.0.smithi142.stdout:5/989: link df/d16/d25/d120/d139/de0/fe3 df/d29/d6e/f149 0 2024-03-29T04:45:10.098 INFO:tasks.workunit.client.0.smithi142.stdout:5/990: read df/d29/d6e/f101 [339018,73370] 0 2024-03-29T04:45:10.099 INFO:tasks.workunit.client.0.smithi142.stdout:8/905: unlink d5/d1d/d2f/d5f/fe0 0 2024-03-29T04:45:10.100 INFO:tasks.workunit.client.0.smithi142.stdout:8/906: write d5/d1d/d2f/d76/f88 [1272243,103555] 0 2024-03-29T04:45:10.102 INFO:tasks.workunit.client.0.smithi142.stdout:3/641: readlink d3/d29/d8f/d5f/d64/lbc 0 2024-03-29T04:45:10.104 INFO:tasks.workunit.client.0.smithi142.stdout:9/641: getdents d0/d2/d6/d27/d3c/d7b 0 2024-03-29T04:45:10.105 INFO:tasks.workunit.client.0.smithi142.stdout:8/907: dread d5/d1d/d2f/fc8 [0,4194304] 0 2024-03-29T04:45:10.106 INFO:tasks.workunit.client.0.smithi142.stdout:4/780: stat d0/d9/d11/d97/d9e 0 2024-03-29T04:45:10.107 INFO:tasks.workunit.client.0.smithi142.stdout:2/776: creat de/d11/d3d/f105 x:0 0 0 2024-03-29T04:45:10.108 INFO:tasks.workunit.client.0.smithi142.stdout:7/869: dread d7/d27/d44/fd7 [0,4194304] 0 2024-03-29T04:45:10.114 INFO:tasks.workunit.client.1.smithi152.stdout:1/199: readlink d5/d7/d9/l33 0 2024-03-29T04:45:10.115 INFO:tasks.workunit.client.1.smithi152.stdout:4/172: rename d0/d7/de/l1b to d0/d7/de/d2a/l2c 0 2024-03-29T04:45:10.116 INFO:tasks.workunit.client.0.smithi142.stdout:9/642: rename d0/d2/d6/d16 to d0/d2/d6/d27/dc7 0 2024-03-29T04:45:10.116 INFO:tasks.workunit.client.0.smithi142.stdout:1/746: dwrite d8/db/dc/d77/d31/d4d/d56/f58 [4194304,4194304] 0 2024-03-29T04:45:10.117 INFO:tasks.workunit.client.1.smithi152.stdout:2/173: unlink d0/d3/df/c1f 0 2024-03-29T04:45:10.117 INFO:tasks.workunit.client.0.smithi142.stdout:0/785: write d7/d13/dbf/dca/dab/dc1/fce [845783,3385] 0 2024-03-29T04:45:10.118 INFO:tasks.workunit.client.0.smithi142.stdout:8/908: mknod d5/d1d/d24/db4/c13f 0 2024-03-29T04:45:10.122 INFO:tasks.workunit.client.0.smithi142.stdout:6/757: getdents d1c/d33/d46/d61/d9e 0 2024-03-29T04:45:10.123 INFO:tasks.workunit.client.1.smithi152.stdout:1/200: unlink d5/d7/d1b/d2e/l31 0 2024-03-29T04:45:10.123 INFO:tasks.workunit.client.0.smithi142.stdout:7/870: creat d7/d27/d3a/d92/f122 x:0 0 0 2024-03-29T04:45:10.124 INFO:tasks.workunit.client.1.smithi152.stdout:2/174: write d0/d3/f10 [123739,107264] 0 2024-03-29T04:45:10.125 INFO:tasks.workunit.client.1.smithi152.stdout:3/186: rmdir d7/d19/d1b/d1e 39 2024-03-29T04:45:10.125 INFO:tasks.workunit.client.1.smithi152.stdout:0/200: dread d2/f8 [0,4194304] 0 2024-03-29T04:45:10.126 INFO:tasks.workunit.client.0.smithi142.stdout:1/747: dwrite d8/db/dc/d77/db9/dbe/d2d/d8b/da4/fba [0,4194304] 0 2024-03-29T04:45:10.126 INFO:tasks.workunit.client.0.smithi142.stdout:5/991: unlink df/d1a/c8e 0 2024-03-29T04:45:10.127 INFO:tasks.workunit.client.1.smithi152.stdout:1/201: symlink d5/d7/l48 0 2024-03-29T04:45:10.127 INFO:tasks.workunit.client.1.smithi152.stdout:3/187: readlink d7/l2b 0 2024-03-29T04:45:10.128 INFO:tasks.workunit.client.1.smithi152.stdout:0/201: write d2/d29/f2f [598990,46839] 0 2024-03-29T04:45:10.128 INFO:tasks.workunit.client.1.smithi152.stdout:1/202: read d5/d7/d9/f11 [87554,87996] 0 2024-03-29T04:45:10.129 INFO:tasks.workunit.client.1.smithi152.stdout:2/175: creat d0/d3/d30/d15/f3a x:0 0 0 2024-03-29T04:45:10.129 INFO:tasks.workunit.client.1.smithi152.stdout:1/203: chown d5/d7/d9/d2b 390 1 2024-03-29T04:45:10.131 INFO:tasks.workunit.client.1.smithi152.stdout:3/188: write d7/f10 [1245500,12657] 0 2024-03-29T04:45:10.132 INFO:tasks.workunit.client.1.smithi152.stdout:1/204: write d5/d7/f1f [1223027,101673] 0 2024-03-29T04:45:10.132 INFO:tasks.workunit.client.1.smithi152.stdout:0/202: unlink d2/l34 0 2024-03-29T04:45:10.134 INFO:tasks.workunit.client.0.smithi142.stdout:0/786: rename d7/d13/dbf/dca/dab/dc1/dd3/f7f to d7/d13/dbf/fff 0 2024-03-29T04:45:10.138 INFO:tasks.workunit.client.1.smithi152.stdout:3/189: read d7/d19/f30 [2499436,117087] 0 2024-03-29T04:45:10.138 INFO:tasks.workunit.client.0.smithi142.stdout:9/643: mkdir d0/d2/d6/d31/dc8 0 2024-03-29T04:45:10.139 INFO:tasks.workunit.client.0.smithi142.stdout:1/748: creat d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/f103 x:0 0 0 2024-03-29T04:45:10.140 INFO:tasks.workunit.client.1.smithi152.stdout:3/190: write d7/d19/f25 [61945,2676] 0 2024-03-29T04:45:10.140 INFO:tasks.workunit.client.0.smithi142.stdout:7/871: creat d7/d27/d44/db2/db8/d4b/d71/d30/d33/df5/f123 x:0 0 0 2024-03-29T04:45:10.140 INFO:tasks.workunit.client.0.smithi142.stdout:5/992: dwrite df/d2c/d112/f11e [0,4194304] 0 2024-03-29T04:45:10.141 INFO:tasks.workunit.client.1.smithi152.stdout:1/205: rmdir d5/d7 39 2024-03-29T04:45:10.141 INFO:tasks.workunit.client.0.smithi142.stdout:4/781: creat d0/d9/db2/ffe x:0 0 0 2024-03-29T04:45:10.141 INFO:tasks.workunit.client.1.smithi152.stdout:3/191: chown d7/d19/d1b 0 1 2024-03-29T04:45:10.142 INFO:tasks.workunit.client.1.smithi152.stdout:4/173: dread d0/f13 [0,4194304] 0 2024-03-29T04:45:10.143 INFO:tasks.workunit.client.1.smithi152.stdout:1/206: chown d5/d7/fe 24164 1 2024-03-29T04:45:10.145 INFO:tasks.workunit.client.1.smithi152.stdout:3/192: symlink d7/l31 0 2024-03-29T04:45:10.147 INFO:tasks.workunit.client.1.smithi152.stdout:1/207: symlink d5/d7/d9/d12/l49 0 2024-03-29T04:45:10.148 INFO:tasks.workunit.client.1.smithi152.stdout:1/208: dread - d5/f3f zero size 2024-03-29T04:45:10.149 INFO:tasks.workunit.client.1.smithi152.stdout:4/174: getdents d0/d7/d25 0 2024-03-29T04:45:10.150 INFO:tasks.workunit.client.1.smithi152.stdout:1/209: read - d5/d7/d8/d13/f45 zero size 2024-03-29T04:45:10.153 INFO:tasks.workunit.client.1.smithi152.stdout:1/210: creat d5/d7/d9/f4a x:0 0 0 2024-03-29T04:45:10.154 INFO:tasks.workunit.client.1.smithi152.stdout:0/203: mknod d2/d29/c47 0 2024-03-29T04:45:10.154 INFO:tasks.workunit.client.1.smithi152.stdout:4/175: link d0/lc d0/d7/l2d 0 2024-03-29T04:45:10.155 INFO:tasks.workunit.client.1.smithi152.stdout:1/211: write d5/d7/f25 [2607396,5152] 0 2024-03-29T04:45:10.156 INFO:tasks.workunit.client.1.smithi152.stdout:3/193: dwrite d7/d19/f29 [0,4194304] 0 2024-03-29T04:45:10.157 INFO:tasks.workunit.client.1.smithi152.stdout:1/212: write d5/f3f [106512,91948] 0 2024-03-29T04:45:10.158 INFO:tasks.workunit.client.1.smithi152.stdout:4/176: unlink d0/d7/f14 0 2024-03-29T04:45:10.160 INFO:tasks.workunit.client.1.smithi152.stdout:4/177: truncate d0/f11 976799 0 2024-03-29T04:45:10.161 INFO:tasks.workunit.client.1.smithi152.stdout:3/194: fsync d7/d19/d1b/d1e/d22/f2d 0 2024-03-29T04:45:10.161 INFO:tasks.workunit.client.1.smithi152.stdout:1/213: unlink c4 0 2024-03-29T04:45:10.163 INFO:tasks.workunit.client.1.smithi152.stdout:0/204: dwrite d2/da/d17/f2b [0,4194304] 0 2024-03-29T04:45:10.166 INFO:tasks.workunit.client.1.smithi152.stdout:1/214: dread d5/d7/d8/fb [0,4194304] 0 2024-03-29T04:45:10.166 INFO:tasks.workunit.client.1.smithi152.stdout:3/195: creat d7/d19/f32 x:0 0 0 2024-03-29T04:45:10.167 INFO:tasks.workunit.client.1.smithi152.stdout:0/205: mknod d2/d29/c48 0 2024-03-29T04:45:10.168 INFO:tasks.workunit.client.1.smithi152.stdout:1/215: chown d5/d7/d8/f2d 7864073 1 2024-03-29T04:45:10.169 INFO:tasks.workunit.client.1.smithi152.stdout:0/206: write d2/da/d17/f2b [2732978,46882] 0 2024-03-29T04:45:10.169 INFO:tasks.workunit.client.1.smithi152.stdout:1/216: chown d5/d7/d8/d13/c41 111976851 1 2024-03-29T04:45:10.170 INFO:tasks.workunit.client.1.smithi152.stdout:3/196: read d7/d19/d1b/f23 [2571604,42664] 0 2024-03-29T04:45:10.170 INFO:tasks.workunit.client.1.smithi152.stdout:1/217: chown d5/d7/f25 957 1 2024-03-29T04:45:10.172 INFO:tasks.workunit.client.1.smithi152.stdout:1/218: write d5/d7/d8/d13/d19/f30 [787130,24975] 0 2024-03-29T04:45:10.176 INFO:tasks.workunit.client.1.smithi152.stdout:0/207: dread d2/d31/f3c [0,4194304] 0 2024-03-29T04:45:10.178 INFO:tasks.workunit.client.1.smithi152.stdout:0/208: mknod d2/da/df/c49 0 2024-03-29T04:45:10.181 INFO:tasks.workunit.client.1.smithi152.stdout:1/219: dwrite d5/d7/d9/f11 [0,4194304] 0 2024-03-29T04:45:10.181 INFO:tasks.workunit.client.1.smithi152.stdout:0/209: creat d2/da/df/f4a x:0 0 0 2024-03-29T04:45:10.182 INFO:tasks.workunit.client.1.smithi152.stdout:1/220: stat d5/d7/f25 0 2024-03-29T04:45:10.184 INFO:tasks.workunit.client.1.smithi152.stdout:0/210: creat d2/da/df/f4b x:0 0 0 2024-03-29T04:45:10.185 INFO:tasks.workunit.client.1.smithi152.stdout:0/211: fdatasync d2/da/df/f4a 0 2024-03-29T04:45:10.187 INFO:tasks.workunit.client.1.smithi152.stdout:0/212: write d2/d29/f37 [502082,20548] 0 2024-03-29T04:45:10.192 INFO:tasks.workunit.client.1.smithi152.stdout:4/178: truncate d0/d7/d25/f26 3416904 0 2024-03-29T04:45:10.193 INFO:tasks.workunit.client.1.smithi152.stdout:0/213: creat d2/da/d17/f4c x:0 0 0 2024-03-29T04:45:10.194 INFO:tasks.workunit.client.1.smithi152.stdout:3/197: write d7/d19/d1b/f1d [2307636,13764] 0 2024-03-29T04:45:10.195 INFO:tasks.workunit.client.1.smithi152.stdout:3/198: write d7/d19/f32 [933734,22803] 0 2024-03-29T04:45:10.197 INFO:tasks.workunit.client.1.smithi152.stdout:4/179: unlink d0/d7/c8 0 2024-03-29T04:45:10.197 INFO:tasks.workunit.client.1.smithi152.stdout:4/180: fdatasync d0/d7/f29 0 2024-03-29T04:45:10.198 INFO:tasks.workunit.client.1.smithi152.stdout:0/214: symlink d2/da/l4d 0 2024-03-29T04:45:10.199 INFO:tasks.workunit.client.1.smithi152.stdout:4/181: truncate d0/f15 428720 0 2024-03-29T04:45:10.200 INFO:tasks.workunit.client.1.smithi152.stdout:1/221: rename d5/c17 to d5/c4b 0 2024-03-29T04:45:10.203 INFO:tasks.workunit.client.1.smithi152.stdout:3/199: mkdir d7/d19/d1b/d33 0 2024-03-29T04:45:10.204 INFO:tasks.workunit.client.1.smithi152.stdout:3/200: dread - d7/d19/d1b/d1e/d22/f2d zero size 2024-03-29T04:45:10.204 INFO:tasks.workunit.client.1.smithi152.stdout:0/215: symlink d2/da/d17/l4e 0 2024-03-29T04:45:10.205 INFO:tasks.workunit.client.1.smithi152.stdout:4/182: write d0/f13 [3383998,106383] 0 2024-03-29T04:45:10.208 INFO:tasks.workunit.client.1.smithi152.stdout:3/201: write d7/f10 [485046,73973] 0 2024-03-29T04:45:10.209 INFO:tasks.workunit.client.1.smithi152.stdout:3/202: dread - d7/d19/d1b/d1e/d22/f2d zero size 2024-03-29T04:45:10.214 INFO:tasks.workunit.client.1.smithi152.stdout:1/222: fdatasync d5/d7/d9/f46 0 2024-03-29T04:45:10.219 INFO:tasks.workunit.client.1.smithi152.stdout:3/203: dread d7/f18 [0,4194304] 0 2024-03-29T04:45:10.221 INFO:tasks.workunit.client.1.smithi152.stdout:4/183: truncate d0/d7/de/ff 1868849 0 2024-03-29T04:45:10.224 INFO:tasks.workunit.client.1.smithi152.stdout:0/216: link d2/da/d17/l13 d2/da/d3a/l4f 0 2024-03-29T04:45:10.225 INFO:tasks.workunit.client.1.smithi152.stdout:1/223: truncate d5/d7/d8/f2d 353534 0 2024-03-29T04:45:10.227 INFO:tasks.workunit.client.1.smithi152.stdout:0/217: mkdir d2/da/d17/d50 0 2024-03-29T04:45:10.229 INFO:tasks.workunit.client.1.smithi152.stdout:0/218: dread - d2/f38 zero size 2024-03-29T04:45:10.229 INFO:tasks.workunit.client.1.smithi152.stdout:1/224: dread d5/d7/d8/d13/f35 [0,4194304] 0 2024-03-29T04:45:10.230 INFO:tasks.workunit.client.1.smithi152.stdout:0/219: dread - d2/da/d17/f4c zero size 2024-03-29T04:45:10.231 INFO:tasks.workunit.client.1.smithi152.stdout:1/225: chown d5/d7/d9/l33 0 1 2024-03-29T04:45:10.232 INFO:tasks.workunit.client.1.smithi152.stdout:0/220: chown d2/da/d17/f2b 205191 1 2024-03-29T04:45:10.232 INFO:tasks.workunit.client.1.smithi152.stdout:3/204: link d7/c11 d7/d19/c34 0 2024-03-29T04:45:10.233 INFO:tasks.workunit.client.0.smithi142.stdout:0/787: fsync d7/d9/d54/d58/f6b 0 2024-03-29T04:45:10.233 INFO:tasks.workunit.client.0.smithi142.stdout:1/749: read - d8/db/dc/d77/db9/dbe/d2d/f90 zero size 2024-03-29T04:45:10.234 INFO:tasks.workunit.client.1.smithi152.stdout:1/226: write d5/d7/d9/f46 [230564,123949] 0 2024-03-29T04:45:10.234 INFO:tasks.workunit.client.0.smithi142.stdout:2/777: write de/f29 [1626337,42246] 0 2024-03-29T04:45:10.235 INFO:tasks.workunit.client.0.smithi142.stdout:6/758: write d1c/d33/f67 [604315,9633] 0 2024-03-29T04:45:10.235 INFO:tasks.workunit.client.1.smithi152.stdout:0/221: mknod d2/da/d17/c51 0 2024-03-29T04:45:10.235 INFO:tasks.workunit.client.1.smithi152.stdout:4/184: dwrite d0/d7/d25/f1a [0,4194304] 0 2024-03-29T04:45:10.242 INFO:tasks.workunit.client.1.smithi152.stdout:3/205: rename d7/ce to d7/c35 0 2024-03-29T04:45:10.242 INFO:tasks.workunit.client.1.smithi152.stdout:1/227: creat d5/d7/d38/f4c x:0 0 0 2024-03-29T04:45:10.242 INFO:tasks.workunit.client.1.smithi152.stdout:0/222: mknod d2/da/d17/d50/c52 0 2024-03-29T04:45:10.242 INFO:tasks.workunit.client.1.smithi152.stdout:4/185: mknod d0/d7/d25/d17/c2e 0 2024-03-29T04:45:10.242 INFO:tasks.workunit.client.1.smithi152.stdout:0/223: chown d2/da/d3a/c43 353 1 2024-03-29T04:45:10.245 INFO:tasks.workunit.client.1.smithi152.stdout:0/224: mknod d2/d29/c53 0 2024-03-29T04:45:10.248 INFO:tasks.workunit.client.1.smithi152.stdout:0/225: mknod d2/c54 0 2024-03-29T04:45:10.249 INFO:tasks.workunit.client.1.smithi152.stdout:1/228: dwrite d5/f20 [0,4194304] 0 2024-03-29T04:45:10.251 INFO:tasks.workunit.client.1.smithi152.stdout:5/189: sync 2024-03-29T04:45:10.252 INFO:tasks.workunit.client.1.smithi152.stdout:1/229: mkdir d5/d7/d4d 0 2024-03-29T04:45:10.254 INFO:tasks.workunit.client.1.smithi152.stdout:1/230: fdatasync d5/d7/d9/d2b/f2f 0 2024-03-29T04:45:10.254 INFO:tasks.workunit.client.1.smithi152.stdout:3/206: dread d7/d19/d1b/f1d [0,4194304] 0 2024-03-29T04:45:10.255 INFO:tasks.workunit.client.1.smithi152.stdout:5/190: creat d6/d16/d25/d2e/f44 x:0 0 0 2024-03-29T04:45:10.258 INFO:tasks.workunit.client.1.smithi152.stdout:0/226: rename d2/da/d17/l1e to d2/l55 0 2024-03-29T04:45:10.260 INFO:tasks.workunit.client.1.smithi152.stdout:5/191: readlink d6/lb 0 2024-03-29T04:45:10.262 INFO:tasks.workunit.client.1.smithi152.stdout:4/186: write d0/d7/d25/f26 [857443,19976] 0 2024-03-29T04:45:10.263 INFO:tasks.workunit.client.0.smithi142.stdout:5/993: creat df/d16/d24/d2e/d47/d4d/d102/d75/f14a x:0 0 0 2024-03-29T04:45:10.264 INFO:tasks.workunit.client.0.smithi142.stdout:8/909: getdents d5/d1d/d2f/d76/d3c 0 2024-03-29T04:45:10.265 INFO:tasks.workunit.client.1.smithi152.stdout:4/187: chown d0/lc 2787 1 2024-03-29T04:45:10.266 INFO:tasks.workunit.client.0.smithi142.stdout:0/788: readlink d7/d9/d1e/d81/le3 0 2024-03-29T04:45:10.268 INFO:tasks.workunit.client.0.smithi142.stdout:9/644: dwrite d0/d2/d6/d27/dc7/d6a/fae [0,4194304] 0 2024-03-29T04:45:10.268 INFO:tasks.workunit.client.0.smithi142.stdout:7/872: rename d7/d27/d44/d4c/c11a to d7/d27/d44/db2/db8/d4b/d71/d30/d33/d10b/c124 0 2024-03-29T04:45:10.268 INFO:tasks.workunit.client.1.smithi152.stdout:3/207: getdents d7/d19/d1b 0 2024-03-29T04:45:10.270 INFO:tasks.workunit.client.0.smithi142.stdout:0/789: fdatasync d7/d13/d4a/de7/ff4 0 2024-03-29T04:45:10.270 INFO:tasks.workunit.client.0.smithi142.stdout:4/782: mkdir d0/d9/d11/d97/dff 0 2024-03-29T04:45:10.271 INFO:tasks.workunit.client.0.smithi142.stdout:6/759: write d1c/d33/f5a [258072,15164] 0 2024-03-29T04:45:10.271 INFO:tasks.workunit.client.1.smithi152.stdout:0/227: getdents d2/da 0 2024-03-29T04:45:10.271 INFO:tasks.workunit.client.0.smithi142.stdout:1/750: fsync d8/fa 0 2024-03-29T04:45:10.274 INFO:tasks.workunit.client.1.smithi152.stdout:4/188: mknod d0/d27/c2f 0 2024-03-29T04:45:10.275 INFO:tasks.workunit.client.0.smithi142.stdout:2/778: dwrite de/d11/d2b/d3e/f76 [0,4194304] 0 2024-03-29T04:45:10.275 INFO:tasks.workunit.client.0.smithi142.stdout:0/790: chown d7/d9/d55/f98 52 1 2024-03-29T04:45:10.275 INFO:tasks.workunit.client.0.smithi142.stdout:5/994: unlink df/d16/l83 0 2024-03-29T04:45:10.276 INFO:tasks.workunit.client.1.smithi152.stdout:4/189: write d0/d7/d25/f21 [1027124,59056] 0 2024-03-29T04:45:10.279 INFO:tasks.workunit.client.0.smithi142.stdout:4/783: write d0/d9/d11/d4c/d5f/d81/dc9/d47/fda [3832234,90032] 0 2024-03-29T04:45:10.279 INFO:tasks.workunit.client.0.smithi142.stdout:8/910: fdatasync d5/d1d/d2f/d45/fcf 0 2024-03-29T04:45:10.282 INFO:tasks.workunit.client.0.smithi142.stdout:9/645: creat d0/d6c/dc2/fc9 x:0 0 0 2024-03-29T04:45:10.283 INFO:tasks.workunit.client.1.smithi152.stdout:3/208: link d7/d19/f26 d7/d19/f36 0 2024-03-29T04:45:10.283 INFO:tasks.workunit.client.0.smithi142.stdout:0/791: creat d7/d9/d1e/f100 x:0 0 0 2024-03-29T04:45:10.284 INFO:tasks.workunit.client.0.smithi142.stdout:8/911: chown d5/d1d/d24/d66/dce/feb 48119 1 2024-03-29T04:45:10.284 INFO:tasks.workunit.client.0.smithi142.stdout:5/995: read df/d29/d6e/f149 [127136,97549] 0 2024-03-29T04:45:10.285 INFO:tasks.workunit.client.0.smithi142.stdout:2/779: symlink de/d11/d1e/d22/dbf/l106 0 2024-03-29T04:45:10.286 INFO:tasks.workunit.client.1.smithi152.stdout:5/192: dwrite d6/dd/f11 [0,4194304] 0 2024-03-29T04:45:10.287 INFO:tasks.workunit.client.0.smithi142.stdout:2/780: dread - de/d11/d1e/d22/d27/dba/f6b zero size 2024-03-29T04:45:10.287 INFO:tasks.workunit.client.0.smithi142.stdout:6/760: mknod d1c/d45/d8a/df6/df7/cf8 0 2024-03-29T04:45:10.288 INFO:tasks.workunit.client.1.smithi152.stdout:0/228: rename d2/c1c to d2/d29/c56 0 2024-03-29T04:45:10.289 INFO:tasks.workunit.client.0.smithi142.stdout:7/873: unlink d7/d27/d3a/d92/d97/dea/f11d 0 2024-03-29T04:45:10.289 INFO:tasks.workunit.client.1.smithi152.stdout:4/190: link d0/c1d d0/d7/d25/c30 0 2024-03-29T04:45:10.290 INFO:tasks.workunit.client.1.smithi152.stdout:5/193: readlink d6/d16/d25/d2e/l37 0 2024-03-29T04:45:10.290 INFO:tasks.workunit.client.0.smithi142.stdout:4/784: read - d0/d12/f7b zero size 2024-03-29T04:45:10.293 INFO:tasks.workunit.client.0.smithi142.stdout:9/646: symlink d0/d12/d50/d52/da3/lca 0 2024-03-29T04:45:10.293 INFO:tasks.workunit.client.1.smithi152.stdout:4/191: write d0/f11 [1147175,44537] 0 2024-03-29T04:45:10.293 INFO:tasks.workunit.client.1.smithi152.stdout:5/194: creat d6/d1b/f45 x:0 0 0 2024-03-29T04:45:10.294 INFO:tasks.workunit.client.0.smithi142.stdout:0/792: mkdir d7/d13/dbf/dca/d101 0 2024-03-29T04:45:10.295 INFO:tasks.workunit.client.1.smithi152.stdout:5/195: write d6/f14 [1916396,126602] 0 2024-03-29T04:45:10.295 INFO:tasks.workunit.client.1.smithi152.stdout:3/209: rename d7/c24 to d7/d19/d1b/d33/c37 0 2024-03-29T04:45:10.296 INFO:tasks.workunit.client.0.smithi142.stdout:8/912: unlink d5/d1d/d81/cc1 0 2024-03-29T04:45:10.297 INFO:tasks.workunit.client.0.smithi142.stdout:4/785: dread d0/d9/d11/d4c/d64/d94/fab [0,4194304] 0 2024-03-29T04:45:10.298 INFO:tasks.workunit.client.1.smithi152.stdout:5/196: truncate d6/d1d/d3b/f3f 970849 0 2024-03-29T04:45:10.299 INFO:tasks.workunit.client.1.smithi152.stdout:4/192: write d0/f13 [2488588,40841] 0 2024-03-29T04:45:10.302 INFO:tasks.workunit.client.0.smithi142.stdout:9/647: creat d0/d2/d6/d27/d3c/d6f/fcb x:0 0 0 2024-03-29T04:45:10.302 INFO:tasks.workunit.client.0.smithi142.stdout:8/913: unlink d5/f94 0 2024-03-29T04:45:10.303 INFO:tasks.workunit.client.1.smithi152.stdout:5/197: unlink d6/f23 0 2024-03-29T04:45:10.303 INFO:tasks.workunit.client.0.smithi142.stdout:2/781: symlink de/d11/d1e/l107 0 2024-03-29T04:45:10.305 INFO:tasks.workunit.client.1.smithi152.stdout:3/210: rename d7/d19/d1b/d1e/f2a to d7/d19/f38 0 2024-03-29T04:45:10.306 INFO:tasks.workunit.client.0.smithi142.stdout:7/874: creat d7/d27/d44/dab/dcc/f125 x:0 0 0 2024-03-29T04:45:10.306 INFO:tasks.workunit.client.0.smithi142.stdout:1/751: truncate d8/db/dc/d77/db9/dbe/d15/f47 592330 0 2024-03-29T04:45:10.306 INFO:tasks.workunit.client.0.smithi142.stdout:7/875: chown d7/d27 746094 1 2024-03-29T04:45:10.308 INFO:tasks.workunit.client.0.smithi142.stdout:4/786: fsync d0/d9/d11/d4c/d5f/d81/dc9/d47/d7a/de4/ff7 0 2024-03-29T04:45:10.309 INFO:tasks.workunit.client.1.smithi152.stdout:5/198: dwrite d6/dd/f11 [0,4194304] 0 2024-03-29T04:45:10.310 INFO:tasks.workunit.client.1.smithi152.stdout:3/211: symlink d7/d19/d1b/d1e/d22/l39 0 2024-03-29T04:45:10.312 INFO:tasks.workunit.client.1.smithi152.stdout:5/199: creat d6/d1d/d30/f46 x:0 0 0 2024-03-29T04:45:10.312 INFO:tasks.workunit.client.0.smithi142.stdout:5/996: rename df/d16/d24/cdc to df/d29/c14b 0 2024-03-29T04:45:10.313 INFO:tasks.workunit.client.1.smithi152.stdout:5/200: chown d6/d16/d25/d2e 0 1 2024-03-29T04:45:10.316 INFO:tasks.workunit.client.0.smithi142.stdout:1/752: dwrite d8/ff7 [0,4194304] 0 2024-03-29T04:45:10.318 INFO:tasks.workunit.client.0.smithi142.stdout:2/782: truncate fd 2215058 0 2024-03-29T04:45:10.320 INFO:tasks.workunit.client.1.smithi152.stdout:5/201: dwrite d6/f14 [0,4194304] 0 2024-03-29T04:45:10.320 INFO:tasks.workunit.client.0.smithi142.stdout:0/793: mknod d7/d13/d4a/d9a/da7/c102 0 2024-03-29T04:45:10.323 INFO:tasks.workunit.client.0.smithi142.stdout:6/761: dread d1c/d33/d46/d66/f6b [0,4194304] 0 2024-03-29T04:45:10.324 INFO:tasks.workunit.client.0.smithi142.stdout:9/648: creat d0/d2/d6/d27/dc7/daa/d95/fcc x:0 0 0 2024-03-29T04:45:10.324 INFO:tasks.workunit.client.1.smithi152.stdout:3/212: rename d7/d19/d1b/d33 to d7/d19/d3a 0 2024-03-29T04:45:10.326 INFO:tasks.workunit.client.0.smithi142.stdout:9/649: write d0/d12/d50/d8e/fa4 [286257,105486] 0 2024-03-29T04:45:10.328 INFO:tasks.workunit.client.1.smithi152.stdout:0/229: truncate d2/f20 1115932 0 2024-03-29T04:45:10.330 INFO:tasks.workunit.client.0.smithi142.stdout:1/753: mknod d8/db/dd6/c104 0 2024-03-29T04:45:10.332 INFO:tasks.workunit.client.1.smithi152.stdout:3/213: dread d7/d19/f25 [0,4194304] 0 2024-03-29T04:45:10.333 INFO:tasks.workunit.client.1.smithi152.stdout:0/230: write d2/f1f [4536834,65443] 0 2024-03-29T04:45:10.334 INFO:tasks.workunit.client.0.smithi142.stdout:0/794: truncate d7/d9/d1e/f53 1328613 0 2024-03-29T04:45:10.336 INFO:tasks.workunit.client.1.smithi152.stdout:0/231: truncate d2/d29/f44 121121 0 2024-03-29T04:45:10.336 INFO:tasks.workunit.client.0.smithi142.stdout:6/762: dwrite d1c/d90/d94/fd2 [0,4194304] 0 2024-03-29T04:45:10.340 INFO:tasks.workunit.client.0.smithi142.stdout:8/914: dread d5/d93/fb8 [0,4194304] 0 2024-03-29T04:45:10.340 INFO:tasks.workunit.client.0.smithi142.stdout:0/795: readlink d7/d13/d4a/l85 0 2024-03-29T04:45:10.341 INFO:tasks.workunit.client.1.smithi152.stdout:4/193: truncate d0/d7/de/d23/f28 7895670 0 2024-03-29T04:45:10.344 INFO:tasks.workunit.client.1.smithi152.stdout:0/232: chown d2/f2e 12661912 1 2024-03-29T04:45:10.345 INFO:tasks.workunit.client.1.smithi152.stdout:5/202: dread d6/d1d/f2c [0,4194304] 0 2024-03-29T04:45:10.345 INFO:tasks.workunit.client.1.smithi152.stdout:4/194: rmdir d0/d7/d25 39 2024-03-29T04:45:10.347 INFO:tasks.workunit.client.1.smithi152.stdout:0/233: mkdir d2/da/d17/d57 0 2024-03-29T04:45:10.348 INFO:tasks.workunit.client.0.smithi142.stdout:9/650: dwrite d0/d12/d50/f63 [0,4194304] 0 2024-03-29T04:45:10.348 INFO:tasks.workunit.client.1.smithi152.stdout:5/203: mkdir d6/d16/d47 0 2024-03-29T04:45:10.349 INFO:tasks.workunit.client.0.smithi142.stdout:9/651: chown d0/l42 16478053 1 2024-03-29T04:45:10.351 INFO:tasks.workunit.client.1.smithi152.stdout:0/234: symlink d2/da/d17/d50/l58 0 2024-03-29T04:45:10.352 INFO:tasks.workunit.client.1.smithi152.stdout:5/204: mknod d6/d16/d25/d2e/c48 0 2024-03-29T04:45:10.352 INFO:tasks.workunit.client.1.smithi152.stdout:4/195: link d0/d7/l2d d0/l31 0 2024-03-29T04:45:10.354 INFO:tasks.workunit.client.1.smithi152.stdout:5/205: symlink d6/d7/l49 0 2024-03-29T04:45:10.356 INFO:tasks.workunit.client.1.smithi152.stdout:0/235: creat d2/f59 x:0 0 0 2024-03-29T04:45:10.357 INFO:tasks.workunit.client.1.smithi152.stdout:4/196: truncate d0/f15 1193015 0 2024-03-29T04:45:10.358 INFO:tasks.workunit.client.1.smithi152.stdout:0/236: read d2/d31/f36 [387686,26258] 0 2024-03-29T04:45:10.360 INFO:tasks.workunit.client.0.smithi142.stdout:4/787: truncate d0/d9/d19/f58 265882 0 2024-03-29T04:45:10.361 INFO:tasks.workunit.client.1.smithi152.stdout:5/206: getdents d6/dd 0 2024-03-29T04:45:10.362 INFO:tasks.workunit.client.0.smithi142.stdout:9/652: fsync d0/d12/f24 0 2024-03-29T04:45:10.363 INFO:tasks.workunit.client.0.smithi142.stdout:7/876: truncate d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/f98 7322813 0 2024-03-29T04:45:10.364 INFO:tasks.workunit.client.0.smithi142.stdout:5/997: link df/c3a df/d16/d25/d120/d139/c14c 0 2024-03-29T04:45:10.365 INFO:tasks.workunit.client.1.smithi152.stdout:0/237: truncate d2/f8 1763287 0 2024-03-29T04:45:10.366 INFO:tasks.workunit.client.0.smithi142.stdout:1/754: creat d8/db/dc/d77/d31/d4d/d8c/df9/f105 x:0 0 0 2024-03-29T04:45:10.367 INFO:tasks.workunit.client.1.smithi152.stdout:4/197: dwrite d0/f15 [0,4194304] 0 2024-03-29T04:45:10.369 INFO:tasks.workunit.client.0.smithi142.stdout:2/783: getdents de/d11/d1e/d22/d27/dba/d61/dcd 0 2024-03-29T04:45:10.372 INFO:tasks.workunit.client.1.smithi152.stdout:3/214: creat d7/d19/d1b/d1e/d22/f3b x:0 0 0 2024-03-29T04:45:10.375 INFO:tasks.workunit.client.1.smithi152.stdout:0/238: mknod d2/d31/c5a 0 2024-03-29T04:45:10.376 INFO:tasks.workunit.client.1.smithi152.stdout:5/207: dwrite d6/d1d/f40 [0,4194304] 0 2024-03-29T04:45:10.378 INFO:tasks.workunit.client.0.smithi142.stdout:2/784: symlink de/d11/d17/d4a/d79/da8/l108 0 2024-03-29T04:45:10.379 INFO:tasks.workunit.client.0.smithi142.stdout:6/763: dwrite d1c/d33/db6/f4c [0,4194304] 0 2024-03-29T04:45:10.382 INFO:tasks.workunit.client.1.smithi152.stdout:4/198: symlink d0/d7/d25/d17/l32 0 2024-03-29T04:45:10.387 INFO:tasks.workunit.client.0.smithi142.stdout:1/755: mkdir d8/db/dc/d77/db9/dbe/d1f/d50/d96/d106 0 2024-03-29T04:45:10.388 INFO:tasks.workunit.client.0.smithi142.stdout:9/653: rename d0/d2/d6/d31/f75 to d0/d2/d6/fcd 0 2024-03-29T04:45:10.388 INFO:tasks.workunit.client.1.smithi152.stdout:0/239: rename d2/f7 to d2/da/df/f5b 0 2024-03-29T04:45:10.389 INFO:tasks.workunit.client.0.smithi142.stdout:6/764: creat d1c/d33/d46/d61/ff9 x:0 0 0 2024-03-29T04:45:10.389 INFO:tasks.workunit.client.1.smithi152.stdout:0/240: fdatasync d2/f59 0 2024-03-29T04:45:10.392 INFO:tasks.workunit.client.0.smithi142.stdout:6/765: readlink la 0 2024-03-29T04:45:10.393 INFO:tasks.workunit.client.1.smithi152.stdout:4/199: stat d0/d7/d25/c30 0 2024-03-29T04:45:10.393 INFO:tasks.workunit.client.0.smithi142.stdout:7/877: write d7/d27/d44/db2/dc4/fc6 [436515,20019] 0 2024-03-29T04:45:10.394 INFO:tasks.workunit.client.1.smithi152.stdout:5/208: truncate f5 2243944 0 2024-03-29T04:45:10.394 INFO:tasks.workunit.client.1.smithi152.stdout:4/200: chown d0/d7 13459 1 2024-03-29T04:45:10.394 INFO:tasks.workunit.client.1.smithi152.stdout:7/160: sync 2024-03-29T04:45:10.394 INFO:tasks.workunit.client.1.smithi152.stdout:9/270: sync 2024-03-29T04:45:10.395 INFO:tasks.workunit.client.1.smithi152.stdout:5/209: stat d6/d1d/d30/f46 0 2024-03-29T04:45:10.395 INFO:tasks.workunit.client.0.smithi142.stdout:7/878: readlink d7/l66 0 2024-03-29T04:45:10.396 INFO:tasks.workunit.client.1.smithi152.stdout:3/215: creat d7/d19/f3c x:0 0 0 2024-03-29T04:45:10.396 INFO:tasks.workunit.client.0.smithi142.stdout:1/756: chown d8/db/dc/d77/db9/dbe/d2d/d8b/da4/fba 352 1 2024-03-29T04:45:10.396 INFO:tasks.workunit.client.0.smithi142.stdout:2/785: dwrite de/d11/d1e/d22/d27/dba/d61/d7d/fbe [0,4194304] 0 2024-03-29T04:45:10.398 INFO:tasks.workunit.client.1.smithi152.stdout:5/210: write d6/d16/d25/d2e/f44 [636805,107906] 0 2024-03-29T04:45:10.399 INFO:tasks.workunit.client.0.smithi142.stdout:5/998: link df/d1a/d21/d55/d66/f108 df/d16/d24/d2e/d47/d4d/d102/d75/d90/f14d 0 2024-03-29T04:45:10.400 INFO:tasks.workunit.client.0.smithi142.stdout:6/766: write d1c/d33/d46/d61/d7e/fbd [2106510,12413] 0 2024-03-29T04:45:10.400 INFO:tasks.workunit.client.0.smithi142.stdout:0/796: dwrite d7/d13/d4a/d9a/fb3 [0,4194304] 0 2024-03-29T04:45:10.401 INFO:tasks.workunit.client.0.smithi142.stdout:2/786: chown de/d11/d3d/f65 418071329 1 2024-03-29T04:45:10.401 INFO:tasks.workunit.client.0.smithi142.stdout:1/757: write d8/db/dc/d77/db9/dbe/fc5 [439411,43529] 0 2024-03-29T04:45:10.406 INFO:tasks.workunit.client.0.smithi142.stdout:9/654: rename d0/d2/f2f to d0/d2/d6/d27/dc7/d6a/fce 0 2024-03-29T04:45:10.407 INFO:tasks.workunit.client.1.smithi152.stdout:4/201: creat d0/d7/de/d2a/f33 x:0 0 0 2024-03-29T04:45:10.410 INFO:tasks.workunit.client.1.smithi152.stdout:7/161: dwrite d7/d8/db/df/f13 [0,4194304] 0 2024-03-29T04:45:10.410 INFO:tasks.workunit.client.1.smithi152.stdout:5/211: mknod d6/d16/d25/c4a 0 2024-03-29T04:45:10.412 INFO:tasks.workunit.client.1.smithi152.stdout:7/162: chown d7/d8/d12 163 1 2024-03-29T04:45:10.413 INFO:tasks.workunit.client.0.smithi142.stdout:2/787: dwrite de/d11/d17/d60/de6/fef [0,4194304] 0 2024-03-29T04:45:10.414 INFO:tasks.workunit.client.0.smithi142.stdout:2/788: chown de/d11/d1e/d22/dbf 2 1 2024-03-29T04:45:10.414 INFO:tasks.workunit.client.0.smithi142.stdout:5/999: symlink df/d1a/l14e 0 2024-03-29T04:45:10.415 INFO:tasks.workunit.client.1.smithi152.stdout:0/241: dwrite d2/da/df/f5b [0,4194304] 0 2024-03-29T04:45:10.416 INFO:tasks.workunit.client.1.smithi152.stdout:9/271: link d5/d8/c4a d5/d12/d2c/c62 0 2024-03-29T04:45:10.417 INFO:tasks.workunit.client.0.smithi142.stdout:0/797: mknod d7/d13/c103 0 2024-03-29T04:45:10.418 INFO:tasks.workunit.client.0.smithi142.stdout:0/798: fsync d7/d9/d1e/f88 0 2024-03-29T04:45:10.419 INFO:tasks.workunit.client.0.smithi142.stdout:9/655: mkdir d0/d2/dcf 0 2024-03-29T04:45:10.422 INFO:tasks.workunit.client.0.smithi142.stdout:2/789: fdatasync de/d11/d23/f28 0 2024-03-29T04:45:10.424 INFO:tasks.workunit.client.1.smithi152.stdout:2/176: sync 2024-03-29T04:45:10.425 INFO:tasks.workunit.client.1.smithi152.stdout:8/138: sync 2024-03-29T04:45:10.425 INFO:tasks.workunit.client.1.smithi152.stdout:1/231: sync 2024-03-29T04:45:10.425 INFO:tasks.workunit.client.1.smithi152.stdout:6/263: sync 2024-03-29T04:45:10.425 INFO:tasks.workunit.client.0.smithi142.stdout:2/790: read de/d11/d17/d5d/d71/f9f [538113,118502] 0 2024-03-29T04:45:10.425 INFO:tasks.workunit.client.1.smithi152.stdout:3/216: dwrite d7/d19/d1b/d1e/d22/f3b [0,4194304] 0 2024-03-29T04:45:10.426 INFO:tasks.workunit.client.0.smithi142.stdout:1/758: truncate d8/db/dc/d77/db9/dbe/d2d/d44/d53/f5f 2305208 0 2024-03-29T04:45:10.429 INFO:tasks.workunit.client.1.smithi152.stdout:0/242: truncate d2/f1f 5389010 0 2024-03-29T04:45:10.429 INFO:tasks.workunit.client.1.smithi152.stdout:8/139: chown d1/ld 1 1 2024-03-29T04:45:10.431 INFO:tasks.workunit.client.1.smithi152.stdout:8/140: rename d1 to d1/d2/d3/d28 22 2024-03-29T04:45:10.431 INFO:tasks.workunit.client.1.smithi152.stdout:9/272: creat d5/d12/d2c/f63 x:0 0 0 2024-03-29T04:45:10.432 INFO:tasks.workunit.client.1.smithi152.stdout:5/212: symlink d6/d16/d47/l4b 0 2024-03-29T04:45:10.438 INFO:tasks.workunit.client.1.smithi152.stdout:3/217: dread d7/d19/f36 [0,4194304] 0 2024-03-29T04:45:10.438 INFO:tasks.workunit.client.1.smithi152.stdout:1/232: rmdir d5 39 2024-03-29T04:45:10.438 INFO:tasks.workunit.client.1.smithi152.stdout:0/243: readlink d2/l42 0 2024-03-29T04:45:10.438 INFO:tasks.workunit.client.1.smithi152.stdout:9/273: creat d5/d12/d2c/f64 x:0 0 0 2024-03-29T04:45:10.439 INFO:tasks.workunit.client.0.smithi142.stdout:9/656: creat d0/d2/d6/d27/dc7/daa/d95/fd0 x:0 0 0 2024-03-29T04:45:10.444 INFO:tasks.workunit.client.1.smithi152.stdout:3/218: readlink d7/d1c/l28 0 2024-03-29T04:45:10.444 INFO:tasks.workunit.client.1.smithi152.stdout:6/264: getdents d1/d3/d9/d13/d28/d45/d55 0 2024-03-29T04:45:10.444 INFO:tasks.workunit.client.1.smithi152.stdout:2/177: write d0/d3/f18 [462668,35937] 0 2024-03-29T04:45:10.444 INFO:tasks.workunit.client.0.smithi142.stdout:1/759: rmdir d8/db/dc/d77/d31/d4d/d56 39 2024-03-29T04:45:10.444 INFO:tasks.workunit.client.1.smithi152.stdout:2/178: readlink d0/le 0 2024-03-29T04:45:10.445 INFO:tasks.workunit.client.1.smithi152.stdout:8/141: dread d1/d2/d1d/f23 [0,4194304] 0 2024-03-29T04:45:10.445 INFO:tasks.workunit.client.1.smithi152.stdout:0/244: mkdir d2/da/d17/d50/d5c 0 2024-03-29T04:45:10.446 INFO:tasks.workunit.client.0.smithi142.stdout:7/879: dwrite d7/d27/d44/d4c/f7b [0,4194304] 0 2024-03-29T04:45:10.446 INFO:tasks.workunit.client.1.smithi152.stdout:6/265: write d1/d3/d9/d13/d3e/f4d [503396,126051] 0 2024-03-29T04:45:10.447 INFO:tasks.workunit.client.0.smithi142.stdout:6/767: write d1c/d45/d51/f83 [802864,78219] 0 2024-03-29T04:45:10.448 INFO:tasks.workunit.client.0.smithi142.stdout:1/760: rmdir d8/db/dc/d77/db9/dbe/d1f 39 2024-03-29T04:45:10.449 INFO:tasks.workunit.client.1.smithi152.stdout:0/245: read d2/da/d17/f2b [2553578,119521] 0 2024-03-29T04:45:10.450 INFO:tasks.workunit.client.1.smithi152.stdout:4/202: truncate d0/f11 594379 0 2024-03-29T04:45:10.451 INFO:tasks.workunit.client.0.smithi142.stdout:7/880: mkdir d7/d27/d3a/d92/d97/dea/d126 0 2024-03-29T04:45:10.452 INFO:tasks.workunit.client.1.smithi152.stdout:2/179: fdatasync d0/d3/d30/f1b 0 2024-03-29T04:45:10.455 INFO:tasks.workunit.client.1.smithi152.stdout:4/203: chown d0/d7/de/c20 28 1 2024-03-29T04:45:10.455 INFO:tasks.workunit.client.1.smithi152.stdout:0/246: dread d2/d29/f44 [0,4194304] 0 2024-03-29T04:45:10.456 INFO:tasks.workunit.client.0.smithi142.stdout:6/768: unlink d1c/f6e 0 2024-03-29T04:45:10.458 INFO:tasks.workunit.client.1.smithi152.stdout:8/142: creat d1/d2/d3/f29 x:0 0 0 2024-03-29T04:45:10.459 INFO:tasks.workunit.client.0.smithi142.stdout:2/791: link de/d11/d1e/d22/f8e de/d11/d17/f109 0 2024-03-29T04:45:10.459 INFO:tasks.workunit.client.0.smithi142.stdout:6/769: mknod d1c/d33/db6/da4/cfa 0 2024-03-29T04:45:10.459 INFO:tasks.workunit.client.1.smithi152.stdout:2/180: creat d0/f3b x:0 0 0 2024-03-29T04:45:10.460 INFO:tasks.workunit.client.0.smithi142.stdout:2/792: write de/d11/d17/fcc [411475,66014] 0 2024-03-29T04:45:10.461 INFO:tasks.workunit.client.1.smithi152.stdout:9/274: rename d5/d12/d2c/f35 to d5/db/f65 0 2024-03-29T04:45:10.461 INFO:tasks.workunit.client.1.smithi152.stdout:1/233: dwrite d5/d7/f1f [0,4194304] 0 2024-03-29T04:45:10.461 INFO:tasks.workunit.client.0.smithi142.stdout:2/793: fdatasync de/d11/d23/fb1 0 2024-03-29T04:45:10.462 INFO:tasks.workunit.client.1.smithi152.stdout:8/143: write d1/d2/d3/f29 [942960,79295] 0 2024-03-29T04:45:10.466 INFO:tasks.workunit.client.1.smithi152.stdout:0/247: mknod d2/da/d17/d57/c5d 0 2024-03-29T04:45:10.466 INFO:tasks.workunit.client.1.smithi152.stdout:7/163: dwrite d7/d8/db/df/f17 [4194304,4194304] 0 2024-03-29T04:45:10.466 INFO:tasks.workunit.client.0.smithi142.stdout:6/770: getdents d1c/d45 0 2024-03-29T04:45:10.468 INFO:tasks.workunit.client.0.smithi142.stdout:2/794: mknod de/d11/d23/c10a 0 2024-03-29T04:45:10.468 INFO:tasks.workunit.client.1.smithi152.stdout:0/248: write d2/f1f [3100743,106570] 0 2024-03-29T04:45:10.469 INFO:tasks.workunit.client.1.smithi152.stdout:6/266: rename d1/fd to d1/d3/d1a/d1b/d29/f5f 0 2024-03-29T04:45:10.470 INFO:tasks.workunit.client.1.smithi152.stdout:2/181: truncate d0/d3/df/f36 963475 0 2024-03-29T04:45:10.471 INFO:tasks.workunit.client.1.smithi152.stdout:8/144: creat d1/d21/f2a x:0 0 0 2024-03-29T04:45:10.476 INFO:tasks.workunit.client.0.smithi142.stdout:6/771: dwrite d1c/d90/d94/fd2 [0,4194304] 0 2024-03-29T04:45:10.478 INFO:tasks.workunit.client.0.smithi142.stdout:0/799: write d7/d13/dbf/d72/d37/f73 [3568300,60232] 0 2024-03-29T04:45:10.479 INFO:tasks.workunit.client.0.smithi142.stdout:6/772: fdatasync d1c/d33/d46/f65 0 2024-03-29T04:45:10.481 INFO:tasks.workunit.client.1.smithi152.stdout:2/182: rmdir d0/d3/df/d2c 39 2024-03-29T04:45:10.482 INFO:tasks.workunit.client.1.smithi152.stdout:9/275: dwrite d5/db/d1b/f55 [0,4194304] 0 2024-03-29T04:45:10.484 INFO:tasks.workunit.client.1.smithi152.stdout:4/204: rename d0/d7/d25/f26 to d0/d27/f34 0 2024-03-29T04:45:10.485 INFO:tasks.workunit.client.1.smithi152.stdout:3/219: dwrite d7/d19/f26 [0,4194304] 0 2024-03-29T04:45:10.485 INFO:tasks.workunit.client.1.smithi152.stdout:0/249: dread d2/d31/f3c [0,4194304] 0 2024-03-29T04:45:10.486 INFO:tasks.workunit.client.0.smithi142.stdout:9/657: dwrite d0/d2/d6/d27/f48 [0,4194304] 0 2024-03-29T04:45:10.488 INFO:tasks.workunit.client.1.smithi152.stdout:2/183: truncate d0/d3/df/f24 1665826 0 2024-03-29T04:45:10.489 INFO:tasks.workunit.client.0.smithi142.stdout:2/795: rename de/d11/d1e to de/d11/d17/d10b 0 2024-03-29T04:45:10.490 INFO:tasks.workunit.client.1.smithi152.stdout:6/267: symlink d1/d3/d9/d13/l60 0 2024-03-29T04:45:10.490 INFO:tasks.workunit.client.1.smithi152.stdout:1/234: unlink d5/d7/d8/d13/d19/l3b 0 2024-03-29T04:45:10.491 INFO:tasks.workunit.client.1.smithi152.stdout:8/145: symlink d1/l2b 0 2024-03-29T04:45:10.492 INFO:tasks.workunit.client.0.smithi142.stdout:3/642: sync 2024-03-29T04:45:10.492 INFO:tasks.workunit.client.1.smithi152.stdout:8/146: chown d1/d2/d3/f17 0 1 2024-03-29T04:45:10.494 INFO:tasks.workunit.client.1.smithi152.stdout:9/276: symlink d5/d8/l66 0 2024-03-29T04:45:10.495 INFO:tasks.workunit.client.1.smithi152.stdout:4/205: dread d0/d7/d25/f1a [0,4194304] 0 2024-03-29T04:45:10.496 INFO:tasks.workunit.client.0.smithi142.stdout:1/761: write d8/db/dc/d77/f7c [449658,49131] 0 2024-03-29T04:45:10.497 INFO:tasks.workunit.client.1.smithi152.stdout:7/164: symlink d7/d8/d12/d15/l2f 0 2024-03-29T04:45:10.497 INFO:tasks.workunit.client.1.smithi152.stdout:9/277: write d5/d12/d2c/f39 [16276,21472] 0 2024-03-29T04:45:10.498 INFO:tasks.workunit.client.0.smithi142.stdout:9/658: rename d0/d2/d6/d27/d3c/d4f/l5c to d0/d2/d6/d31/d83/ld1 0 2024-03-29T04:45:10.498 INFO:tasks.workunit.client.1.smithi152.stdout:4/206: read - d0/d7/de/d2a/f33 zero size 2024-03-29T04:45:10.500 INFO:tasks.workunit.client.0.smithi142.stdout:2/796: stat de/d11/d2b/d7b/lf9 0 2024-03-29T04:45:10.501 INFO:tasks.workunit.client.0.smithi142.stdout:2/797: fdatasync de/d11/d17/d10b/d22/dbf/ffc 0 2024-03-29T04:45:10.502 INFO:tasks.workunit.client.1.smithi152.stdout:6/268: unlink d1/d3/d1a/f53 0 2024-03-29T04:45:10.503 INFO:tasks.workunit.client.1.smithi152.stdout:6/269: write f0 [1052727,75511] 0 2024-03-29T04:45:10.504 INFO:tasks.workunit.client.0.smithi142.stdout:6/773: mkdir d1c/d90/ddc/dfb 0 2024-03-29T04:45:10.505 INFO:tasks.workunit.client.1.smithi152.stdout:3/220: rmdir d7/d19/d1b/d1e 39 2024-03-29T04:45:10.506 INFO:tasks.workunit.client.1.smithi152.stdout:8/147: rmdir d1/d2/d3 39 2024-03-29T04:45:10.507 INFO:tasks.workunit.client.1.smithi152.stdout:0/250: fdatasync d2/da/d17/f2b 0 2024-03-29T04:45:10.508 INFO:tasks.workunit.client.0.smithi142.stdout:9/659: mkdir d0/d2/d39/dd2 0 2024-03-29T04:45:10.509 INFO:tasks.workunit.client.1.smithi152.stdout:8/148: write d1/f26 [124134,110083] 0 2024-03-29T04:45:10.510 INFO:tasks.workunit.client.1.smithi152.stdout:7/165: dwrite d7/d8/db/fe [0,4194304] 0 2024-03-29T04:45:10.510 INFO:tasks.workunit.client.0.smithi142.stdout:0/800: creat d7/d13/f104 x:0 0 0 2024-03-29T04:45:10.511 INFO:tasks.workunit.client.1.smithi152.stdout:4/207: unlink d0/d7/f29 0 2024-03-29T04:45:10.511 INFO:tasks.workunit.client.0.smithi142.stdout:0/801: stat d7/d13/dbf/d72/d75/c93 0 2024-03-29T04:45:10.512 INFO:tasks.workunit.client.1.smithi152.stdout:8/149: dread - d1/d21/f24 zero size 2024-03-29T04:45:10.514 INFO:tasks.workunit.client.0.smithi142.stdout:6/774: mknod d1c/d33/d46/cfc 0 2024-03-29T04:45:10.514 INFO:tasks.workunit.client.1.smithi152.stdout:2/184: creat d0/d3/df/d2c/f3c x:0 0 0 2024-03-29T04:45:10.515 INFO:tasks.workunit.client.1.smithi152.stdout:1/235: mknod d5/c4e 0 2024-03-29T04:45:10.515 INFO:tasks.workunit.client.1.smithi152.stdout:2/185: dread - d0/d3/d30/d15/f3a zero size 2024-03-29T04:45:10.516 INFO:tasks.workunit.client.1.smithi152.stdout:6/270: creat d1/d3/d1a/f61 x:0 0 0 2024-03-29T04:45:10.516 INFO:tasks.workunit.client.0.smithi142.stdout:2/798: dread de/d11/d2b/d7b/fa7 [0,4194304] 0 2024-03-29T04:45:10.516 INFO:tasks.workunit.client.0.smithi142.stdout:0/802: creat d7/d13/d4a/d9a/da7/f105 x:0 0 0 2024-03-29T04:45:10.518 INFO:tasks.workunit.client.1.smithi152.stdout:2/186: write d0/d3/f10 [950980,56977] 0 2024-03-29T04:45:10.521 INFO:tasks.workunit.client.0.smithi142.stdout:9/660: dwrite d0/d2/d6/d27/dc7/f40 [4194304,4194304] 0 2024-03-29T04:45:10.521 INFO:tasks.workunit.client.0.smithi142.stdout:2/799: rmdir de/d11/da1 39 2024-03-29T04:45:10.523 INFO:tasks.workunit.client.1.smithi152.stdout:7/166: mkdir d7/d8/d12/d1d/d30 0 2024-03-29T04:45:10.524 INFO:tasks.workunit.client.0.smithi142.stdout:6/775: getdents d1c/dc9 0 2024-03-29T04:45:10.527 INFO:tasks.workunit.client.0.smithi142.stdout:9/661: creat d0/d2/d6/d27/d3c/d7b/fd3 x:0 0 0 2024-03-29T04:45:10.528 INFO:tasks.workunit.client.1.smithi152.stdout:9/278: symlink d5/d8/d31/d59/l67 0 2024-03-29T04:45:10.529 INFO:tasks.workunit.client.0.smithi142.stdout:2/800: fsync de/d11/d17/d60/f6d 0 2024-03-29T04:45:10.530 INFO:tasks.workunit.client.1.smithi152.stdout:9/279: write d5/db/d1b/f55 [5175008,93020] 0 2024-03-29T04:45:10.530 INFO:tasks.workunit.client.0.smithi142.stdout:3/643: truncate d3/d1f/d24/d47/f75 109485 0 2024-03-29T04:45:10.530 INFO:tasks.workunit.client.0.smithi142.stdout:9/662: readlink d0/d2/d6/d31/l9b 0 2024-03-29T04:45:10.531 INFO:tasks.workunit.client.0.smithi142.stdout:1/762: write d8/db/f20 [1009154,126668] 0 2024-03-29T04:45:10.531 INFO:tasks.workunit.client.1.smithi152.stdout:9/280: truncate d5/db/d1b/f49 274231 0 2024-03-29T04:45:10.532 INFO:tasks.workunit.client.0.smithi142.stdout:9/663: stat d0/d2/lba 0 2024-03-29T04:45:10.532 INFO:tasks.workunit.client.0.smithi142.stdout:3/644: fdatasync d3/d29/d2d/f3f 0 2024-03-29T04:45:10.536 INFO:tasks.workunit.client.1.smithi152.stdout:4/208: truncate d0/f4 898733 0 2024-03-29T04:45:10.537 INFO:tasks.workunit.client.0.smithi142.stdout:6/776: dwrite d1c/d33/d46/d61/fee [0,4194304] 0 2024-03-29T04:45:10.537 INFO:tasks.workunit.client.0.smithi142.stdout:6/777: readlink d1c/l2f 0 2024-03-29T04:45:10.538 INFO:tasks.workunit.client.1.smithi152.stdout:0/251: getdents d2/da/d17/d50/d5c 0 2024-03-29T04:45:10.539 INFO:tasks.workunit.client.0.smithi142.stdout:9/664: rename d0/d2/d39/c44 to d0/cd4 0 2024-03-29T04:45:10.539 INFO:tasks.workunit.client.1.smithi152.stdout:8/150: unlink d1/d2/d3/l22 0 2024-03-29T04:45:10.541 INFO:tasks.workunit.client.0.smithi142.stdout:1/763: dread - d8/db/dc/d77/db9/dbe/d1f/d97/fd5 zero size 2024-03-29T04:45:10.543 INFO:tasks.workunit.client.1.smithi152.stdout:1/236: unlink d5/ca 0 2024-03-29T04:45:10.544 INFO:tasks.workunit.client.0.smithi142.stdout:3/645: truncate d3/d29/d8f/d5f/d64/f76 860010 0 2024-03-29T04:45:10.544 INFO:tasks.workunit.client.0.smithi142.stdout:1/764: write d8/db/dc/d77/fdf [1328970,2699] 0 2024-03-29T04:45:10.546 INFO:tasks.workunit.client.1.smithi152.stdout:6/271: mknod d1/d3/c62 0 2024-03-29T04:45:10.548 INFO:tasks.workunit.client.1.smithi152.stdout:6/272: write d1/d3/d1a/d49/f5c [102443,22254] 0 2024-03-29T04:45:10.549 INFO:tasks.workunit.client.1.smithi152.stdout:6/273: chown d1 573614 1 2024-03-29T04:45:10.549 INFO:tasks.workunit.client.1.smithi152.stdout:2/187: rename d0/c2a to d0/d3/d30/c3d 0 2024-03-29T04:45:10.550 INFO:tasks.workunit.client.0.smithi142.stdout:3/646: mknod d3/d29/d8f/d5f/cdf 0 2024-03-29T04:45:10.550 INFO:tasks.workunit.client.0.smithi142.stdout:0/803: dread d7/d9/d55/f87 [0,4194304] 0 2024-03-29T04:45:10.551 INFO:tasks.workunit.client.1.smithi152.stdout:0/252: creat d2/da/df/f5e x:0 0 0 2024-03-29T04:45:10.552 INFO:tasks.workunit.client.0.smithi142.stdout:6/778: link d1c/l4a d1c/d33/lfd 0 2024-03-29T04:45:10.552 INFO:tasks.workunit.client.1.smithi152.stdout:7/167: dread d7/f10 [0,4194304] 0 2024-03-29T04:45:10.554 INFO:tasks.workunit.client.1.smithi152.stdout:3/221: creat d7/d19/d1b/f3d x:0 0 0 2024-03-29T04:45:10.555 INFO:tasks.workunit.client.0.smithi142.stdout:9/665: dwrite d0/d2/fa2 [0,4194304] 0 2024-03-29T04:45:10.556 INFO:tasks.workunit.client.0.smithi142.stdout:0/804: chown d7/d13/dbf/dca/dab/ddf 77 1 2024-03-29T04:45:10.556 INFO:tasks.workunit.client.0.smithi142.stdout:2/801: dwrite de/d11/d17/d60/dc4/ff5 [0,4194304] 0 2024-03-29T04:45:10.557 INFO:tasks.workunit.client.0.smithi142.stdout:6/779: read d1c/f40 [1633548,33530] 0 2024-03-29T04:45:10.557 INFO:tasks.workunit.client.0.smithi142.stdout:0/805: stat d7/d9/lac 0 2024-03-29T04:45:10.557 INFO:tasks.workunit.client.1.smithi152.stdout:6/274: write d1/d3/d1a/f2f [1680099,4765] 0 2024-03-29T04:45:10.559 INFO:tasks.workunit.client.0.smithi142.stdout:2/802: write de/d11/d3d/fd8 [1313872,128133] 0 2024-03-29T04:45:10.559 INFO:tasks.workunit.client.0.smithi142.stdout:1/765: rename d8/db/dc/d77/d31/f61 to d8/db/f107 0 2024-03-29T04:45:10.559 INFO:tasks.workunit.client.0.smithi142.stdout:6/780: write d1c/d45/d51/f7f [2967325,50347] 0 2024-03-29T04:45:10.562 INFO:tasks.workunit.client.0.smithi142.stdout:9/666: creat d0/d12/d50/fd5 x:0 0 0 2024-03-29T04:45:10.564 INFO:tasks.workunit.client.0.smithi142.stdout:3/647: chown d3/d29/d2b/d40/d67/d8b/cde 27 1 2024-03-29T04:45:10.566 INFO:tasks.workunit.client.1.smithi152.stdout:4/209: creat d0/d7/f35 x:0 0 0 2024-03-29T04:45:10.568 INFO:tasks.workunit.client.0.smithi142.stdout:9/667: creat d0/d2/d6/d27/d3c/d4f/fd6 x:0 0 0 2024-03-29T04:45:10.568 INFO:tasks.workunit.client.1.smithi152.stdout:8/151: link d1/d2/d3/l11 d1/d2/d3/l2c 0 2024-03-29T04:45:10.569 INFO:tasks.workunit.client.0.smithi142.stdout:1/766: dread - d8/db/dc/d77/db9/dbe/d1f/d4c/f81 zero size 2024-03-29T04:45:10.569 INFO:tasks.workunit.client.1.smithi152.stdout:1/237: dwrite d5/d7/d8/d13/f35 [4194304,4194304] 0 2024-03-29T04:45:10.570 INFO:tasks.workunit.client.0.smithi142.stdout:6/781: truncate d1c/f39 24479 0 2024-03-29T04:45:10.572 INFO:tasks.workunit.client.1.smithi152.stdout:7/168: dwrite d7/d8/db/df/f17 [4194304,4194304] 0 2024-03-29T04:45:10.572 INFO:tasks.workunit.client.1.smithi152.stdout:3/222: dwrite d7/d19/f26 [0,4194304] 0 2024-03-29T04:45:10.573 INFO:tasks.workunit.client.1.smithi152.stdout:7/169: fdatasync d7/d8/d12/f28 0 2024-03-29T04:45:10.575 INFO:tasks.workunit.client.0.smithi142.stdout:2/803: truncate de/d11/d23/fb1 2915619 0 2024-03-29T04:45:10.576 INFO:tasks.workunit.client.0.smithi142.stdout:9/668: symlink d0/d2/ld7 0 2024-03-29T04:45:10.578 INFO:tasks.workunit.client.0.smithi142.stdout:9/669: fsync d0/d2/d6/d27/d3c/d7b/f8f 0 2024-03-29T04:45:10.579 INFO:tasks.workunit.client.1.smithi152.stdout:7/170: chown d7/d8 10328815 1 2024-03-29T04:45:10.581 INFO:tasks.workunit.client.0.smithi142.stdout:9/670: fsync d0/d2/d6/d27/d3c/f72 0 2024-03-29T04:45:10.583 INFO:tasks.workunit.client.1.smithi152.stdout:4/210: rmdir d0/d7/d25/d17 39 2024-03-29T04:45:10.583 INFO:tasks.workunit.client.1.smithi152.stdout:2/188: creat d0/d3/df/f3e x:0 0 0 2024-03-29T04:45:10.583 INFO:tasks.workunit.client.1.smithi152.stdout:8/152: rmdir d1/d21 39 2024-03-29T04:45:10.583 INFO:tasks.workunit.client.1.smithi152.stdout:4/211: dread - d0/d27/f2b zero size 2024-03-29T04:45:10.583 INFO:tasks.workunit.client.1.smithi152.stdout:4/212: dread - d0/d27/f2b zero size 2024-03-29T04:45:10.584 INFO:tasks.workunit.client.0.smithi142.stdout:6/782: dread d1c/d33/d46/d61/fa1 [0,4194304] 0 2024-03-29T04:45:10.586 INFO:tasks.workunit.client.1.smithi152.stdout:0/253: link d2/da/d17/d57/c5d d2/da/c5f 0 2024-03-29T04:45:10.586 INFO:tasks.workunit.client.1.smithi152.stdout:9/281: write d5/d12/f19 [935099,109509] 0 2024-03-29T04:45:10.587 INFO:tasks.workunit.client.1.smithi152.stdout:7/171: creat d7/d8/d12/d15/f31 x:0 0 0 2024-03-29T04:45:10.589 INFO:tasks.workunit.client.1.smithi152.stdout:6/275: rmdir d1/d3/d9/d13/d28/d45/d55 0 2024-03-29T04:45:10.590 INFO:tasks.workunit.client.0.smithi142.stdout:9/671: link d0/d2/d61/d64/lb2 d0/d6c/dc2/ld8 0 2024-03-29T04:45:10.592 INFO:tasks.workunit.client.1.smithi152.stdout:8/153: dread d1/d2/d3/f17 [0,4194304] 0 2024-03-29T04:45:10.593 INFO:tasks.workunit.client.1.smithi152.stdout:6/276: chown d1/f17 16148 1 2024-03-29T04:45:10.593 INFO:tasks.workunit.client.1.smithi152.stdout:8/154: fsync d1/f10 0 2024-03-29T04:45:10.598 INFO:tasks.workunit.client.1.smithi152.stdout:7/172: dwrite d7/d8/d12/d1d/f29 [0,4194304] 0 2024-03-29T04:45:10.601 INFO:tasks.workunit.client.0.smithi142.stdout:2/804: write de/f16 [4364009,116436] 0 2024-03-29T04:45:10.602 INFO:tasks.workunit.client.1.smithi152.stdout:7/173: stat d7/d8/d12/d1d/f1e 0 2024-03-29T04:45:10.603 INFO:tasks.workunit.client.0.smithi142.stdout:6/783: rmdir d1c/dc9/dce 0 2024-03-29T04:45:10.604 INFO:tasks.workunit.client.1.smithi152.stdout:9/282: mkdir d5/db/d1b/d68 0 2024-03-29T04:45:10.605 INFO:tasks.workunit.client.0.smithi142.stdout:2/805: stat de/d11/d17/d10b/d22/d27/d3a 0 2024-03-29T04:45:10.605 INFO:tasks.workunit.client.0.smithi142.stdout:6/784: stat d1c/d33/d46/d61/d7e 0 2024-03-29T04:45:10.605 INFO:tasks.workunit.client.0.smithi142.stdout:9/672: mknod d0/d2/d6/d27/dc7/daa/d95/cd9 0 2024-03-29T04:45:10.606 INFO:tasks.workunit.client.0.smithi142.stdout:9/673: chown d0/d12/f86 7726399 1 2024-03-29T04:45:10.606 INFO:tasks.workunit.client.0.smithi142.stdout:0/806: dread d7/d9/fcb [0,4194304] 0 2024-03-29T04:45:10.606 INFO:tasks.workunit.client.0.smithi142.stdout:2/806: write de/d11/d17/d5d/d71/fe8 [1214042,71042] 0 2024-03-29T04:45:10.607 INFO:tasks.workunit.client.1.smithi152.stdout:9/283: dread d5/f20 [0,4194304] 0 2024-03-29T04:45:10.611 INFO:tasks.workunit.client.1.smithi152.stdout:3/223: creat d7/f3e x:0 0 0 2024-03-29T04:45:10.613 INFO:tasks.workunit.client.1.smithi152.stdout:6/277: creat d1/d3/d9/d3b/f63 x:0 0 0 2024-03-29T04:45:10.613 INFO:tasks.workunit.client.1.smithi152.stdout:7/174: dwrite d7/d8/db/df/d21/f2c [0,4194304] 0 2024-03-29T04:45:10.613 INFO:tasks.workunit.client.0.smithi142.stdout:3/648: dread d3/d29/d2b/d40/f82 [0,4194304] 0 2024-03-29T04:45:10.615 INFO:tasks.workunit.client.0.smithi142.stdout:0/807: fsync d7/d9/d55/f8f 0 2024-03-29T04:45:10.616 INFO:tasks.workunit.client.0.smithi142.stdout:9/674: rmdir d0/d2/d39 39 2024-03-29T04:45:10.618 INFO:tasks.workunit.client.1.smithi152.stdout:9/284: dread d5/db/d1b/f55 [4194304,4194304] 0 2024-03-29T04:45:10.622 INFO:tasks.workunit.client.1.smithi152.stdout:8/155: mknod d1/d21/c2d 0 2024-03-29T04:45:10.623 INFO:tasks.workunit.client.1.smithi152.stdout:9/285: dread d5/d8/d31/f4c [0,4194304] 0 2024-03-29T04:45:10.623 INFO:tasks.workunit.client.0.smithi142.stdout:0/808: unlink d7/d13/dbf/dca/ff0 0 2024-03-29T04:45:10.624 INFO:tasks.workunit.client.1.smithi152.stdout:9/286: chown d5/d12/d2c/f5e 0 1 2024-03-29T04:45:10.624 INFO:tasks.workunit.client.1.smithi152.stdout:4/213: link d0/d27/f2b d0/d7/d25/f36 0 2024-03-29T04:45:10.625 INFO:tasks.workunit.client.1.smithi152.stdout:6/278: symlink d1/d3/d9/d13/d28/l64 0 2024-03-29T04:45:10.626 INFO:tasks.workunit.client.1.smithi152.stdout:3/224: write d7/f1a [4295588,3230] 0 2024-03-29T04:45:10.627 INFO:tasks.workunit.client.1.smithi152.stdout:6/279: chown d1/d3/d9/f21 1 1 2024-03-29T04:45:10.635 INFO:tasks.workunit.client.0.smithi142.stdout:3/649: creat d3/d29/d2b/d40/d67/fe0 x:0 0 0 2024-03-29T04:45:10.638 INFO:tasks.workunit.client.1.smithi152.stdout:6/280: symlink d1/d3/d9/d13/d3e/l65 0 2024-03-29T04:45:10.638 INFO:tasks.workunit.client.1.smithi152.stdout:6/281: fsync d1/d3/d1a/d49/f57 0 2024-03-29T04:45:10.642 INFO:tasks.workunit.client.1.smithi152.stdout:4/214: mkdir d0/d7/d25/d17/d37 0 2024-03-29T04:45:10.643 INFO:tasks.workunit.client.0.smithi142.stdout:0/809: dwrite d7/d13/dbf/d72/d37/f43 [0,4194304] 0 2024-03-29T04:45:10.646 INFO:tasks.workunit.client.1.smithi152.stdout:9/287: getdents d5/d12 0 2024-03-29T04:45:10.647 INFO:tasks.workunit.client.1.smithi152.stdout:6/282: dwrite d1/d3/d1a/f61 [0,4194304] 0 2024-03-29T04:45:10.648 INFO:tasks.workunit.client.1.smithi152.stdout:8/156: symlink d1/d2/l2e 0 2024-03-29T04:45:10.650 INFO:tasks.workunit.client.0.smithi142.stdout:9/675: link d0/d2/c20 d0/d2/d6/d27/d3c/cda 0 2024-03-29T04:45:10.654 INFO:tasks.workunit.client.0.smithi142.stdout:9/676: fdatasync d0/d2/d6/d27/dc7/fc0 0 2024-03-29T04:45:10.654 INFO:tasks.workunit.client.0.smithi142.stdout:9/677: chown d0/d12/d50/d52 518 1 2024-03-29T04:45:10.654 INFO:tasks.workunit.client.1.smithi152.stdout:7/175: dread d7/f14 [0,4194304] 0 2024-03-29T04:45:10.656 INFO:tasks.workunit.client.0.smithi142.stdout:0/810: creat d7/d13/dbf/dca/d101/f106 x:0 0 0 2024-03-29T04:45:10.656 INFO:tasks.workunit.client.0.smithi142.stdout:9/678: mknod d0/d2/d6/d27/cdb 0 2024-03-29T04:45:10.657 INFO:tasks.workunit.client.0.smithi142.stdout:3/650: getdents d3/db9 0 2024-03-29T04:45:10.658 INFO:tasks.workunit.client.0.smithi142.stdout:9/679: readlink d0/d2/ld7 0 2024-03-29T04:45:10.658 INFO:tasks.workunit.client.0.smithi142.stdout:0/811: fsync d7/d13/f78 0 2024-03-29T04:45:10.659 INFO:tasks.workunit.client.0.smithi142.stdout:0/812: chown d7/d13/dbf 2027520 1 2024-03-29T04:45:10.660 INFO:tasks.workunit.client.0.smithi142.stdout:3/651: fsync d3/d29/d2b/d40/d67/d8b/d4d/f9d 0 2024-03-29T04:45:10.661 INFO:tasks.workunit.client.0.smithi142.stdout:3/652: read - d3/d29/d2b/d40/d67/fb6 zero size 2024-03-29T04:45:10.666 INFO:tasks.workunit.client.1.smithi152.stdout:8/157: symlink d1/d2/d3/l2f 0 2024-03-29T04:45:10.667 INFO:tasks.workunit.client.0.smithi142.stdout:9/680: symlink d0/d2/d6/d27/d3c/d6f/ldc 0 2024-03-29T04:45:10.667 INFO:tasks.workunit.client.1.smithi152.stdout:7/176: dread d7/d8/d12/f22 [0,4194304] 0 2024-03-29T04:45:10.668 INFO:tasks.workunit.client.1.smithi152.stdout:6/283: truncate d1/d3/f18 1090987 0 2024-03-29T04:45:10.668 INFO:tasks.workunit.client.0.smithi142.stdout:9/681: write d0/d12/d50/f90 [4322393,116729] 0 2024-03-29T04:45:10.669 INFO:tasks.workunit.client.0.smithi142.stdout:0/813: symlink d7/d13/dbf/dca/dab/l107 0 2024-03-29T04:45:10.669 INFO:tasks.workunit.client.1.smithi152.stdout:6/284: write d1/f8 [2902839,66508] 0 2024-03-29T04:45:10.669 INFO:tasks.workunit.client.1.smithi152.stdout:7/177: stat d7/d8/db/fe 0 2024-03-29T04:45:10.670 INFO:tasks.workunit.client.0.smithi142.stdout:3/653: mknod d3/d29/d8f/d5f/db4/ce1 0 2024-03-29T04:45:10.670 INFO:tasks.workunit.client.0.smithi142.stdout:9/682: stat d0/d2/d61/l8d 0 2024-03-29T04:45:10.670 INFO:tasks.workunit.client.0.smithi142.stdout:3/654: chown d3/d1f/d24/c74 223 1 2024-03-29T04:45:10.672 INFO:tasks.workunit.client.0.smithi142.stdout:9/683: chown d0/d2/d6/fcd 11234800 1 2024-03-29T04:45:10.672 INFO:tasks.workunit.client.0.smithi142.stdout:3/655: write d3/d29/d2b/d6a/f7b [1408700,93337] 0 2024-03-29T04:45:10.673 INFO:tasks.workunit.client.0.smithi142.stdout:0/814: truncate d7/d13/f7d 150227 0 2024-03-29T04:45:10.675 INFO:tasks.workunit.client.0.smithi142.stdout:0/815: truncate d7/d13/dbf/dca/dab/dc1/fdd 814835 0 2024-03-29T04:45:10.676 INFO:tasks.workunit.client.0.smithi142.stdout:9/684: dread d0/d12/d50/d8e/fa4 [0,4194304] 0 2024-03-29T04:45:10.677 INFO:tasks.workunit.client.0.smithi142.stdout:9/685: dread - d0/d2/d6/d27/d3c/d4f/fd6 zero size 2024-03-29T04:45:10.677 INFO:tasks.workunit.client.0.smithi142.stdout:3/656: unlink d3/d29/cd7 0 2024-03-29T04:45:10.678 INFO:tasks.workunit.client.0.smithi142.stdout:9/686: stat d0/d2/d6/d27/d3c/d7b/f8f 0 2024-03-29T04:45:10.678 INFO:tasks.workunit.client.0.smithi142.stdout:0/816: mknod d7/d13/d4a/c108 0 2024-03-29T04:45:10.679 INFO:tasks.workunit.client.0.smithi142.stdout:3/657: write d3/d29/d2d/d6c/fa8 [1373260,130012] 0 2024-03-29T04:45:10.679 INFO:tasks.workunit.client.0.smithi142.stdout:9/687: dread - d0/d12/d50/d52/fb4 zero size 2024-03-29T04:45:10.680 INFO:tasks.workunit.client.0.smithi142.stdout:0/817: chown d7/d13/dbf/dca/dab/ddf 215 1 2024-03-29T04:45:10.682 INFO:tasks.workunit.client.0.smithi142.stdout:9/688: write d0/d2/d6/d27/dc7/daa/d95/fcc [129431,52955] 0 2024-03-29T04:45:10.684 INFO:tasks.workunit.client.0.smithi142.stdout:9/689: truncate d0/d2/d6/d27/d3c/d4f/fd6 680269 0 2024-03-29T04:45:10.684 INFO:tasks.workunit.client.1.smithi152.stdout:3/225: getdents d7/d19/d1b/d1e/d22 0 2024-03-29T04:45:10.685 INFO:tasks.workunit.client.0.smithi142.stdout:0/818: unlink d7/d13/d4a/d9a/da7/ld9 0 2024-03-29T04:45:10.687 INFO:tasks.workunit.client.0.smithi142.stdout:9/690: creat d0/d12/d50/d52/fdd x:0 0 0 2024-03-29T04:45:10.687 INFO:tasks.workunit.client.0.smithi142.stdout:9/691: chown d0/l26 289682 1 2024-03-29T04:45:10.688 INFO:tasks.workunit.client.1.smithi152.stdout:3/226: rmdir d7/d19/d1b/d1e 39 2024-03-29T04:45:10.690 INFO:tasks.workunit.client.1.smithi152.stdout:6/285: link d1/d3/d1a/d1b/f58 d1/d3/df/f66 0 2024-03-29T04:45:10.690 INFO:tasks.workunit.client.0.smithi142.stdout:0/819: truncate d7/d9/d55/f57 600533 0 2024-03-29T04:45:10.690 INFO:tasks.workunit.client.0.smithi142.stdout:8/915: sync 2024-03-29T04:45:10.690 INFO:tasks.workunit.client.0.smithi142.stdout:4/788: sync 2024-03-29T04:45:10.691 INFO:tasks.workunit.client.1.smithi152.stdout:7/178: dwrite d7/f14 [0,4194304] 0 2024-03-29T04:45:10.691 INFO:tasks.workunit.client.1.smithi152.stdout:8/158: link d1/f26 d1/d2/d1d/f30 0 2024-03-29T04:45:10.691 INFO:tasks.workunit.client.0.smithi142.stdout:4/789: chown d0/d12/f5d 1 1 2024-03-29T04:45:10.695 INFO:tasks.workunit.client.0.smithi142.stdout:3/658: getdents d3/d1f/d24/d47 0 2024-03-29T04:45:10.696 INFO:tasks.workunit.client.1.smithi152.stdout:7/179: fdatasync d7/d8/d12/d1d/f29 0 2024-03-29T04:45:10.698 INFO:tasks.workunit.client.0.smithi142.stdout:0/820: chown d7/d13/dbf/dca/dab/dc1/dd3/c9d 993 1 2024-03-29T04:45:10.698 INFO:tasks.workunit.client.1.smithi152.stdout:8/159: write d1/d21/f24 [400201,99923] 0 2024-03-29T04:45:10.698 INFO:tasks.workunit.client.0.smithi142.stdout:9/692: dread d0/d12/f24 [0,4194304] 0 2024-03-29T04:45:10.702 INFO:tasks.workunit.client.0.smithi142.stdout:8/916: read d5/d1d/fd8 [173506,1740] 0 2024-03-29T04:45:10.702 INFO:tasks.workunit.client.1.smithi152.stdout:7/180: creat d7/d8/d18/f32 x:0 0 0 2024-03-29T04:45:10.703 INFO:tasks.workunit.client.1.smithi152.stdout:3/227: dwrite d7/d19/f29 [0,4194304] 0 2024-03-29T04:45:10.703 INFO:tasks.workunit.client.0.smithi142.stdout:3/659: dwrite d3/d1f/fbd [0,4194304] 0 2024-03-29T04:45:10.704 INFO:tasks.workunit.client.0.smithi142.stdout:9/693: write d0/d2/d6/d27/dc7/d6a/fae [2501108,106806] 0 2024-03-29T04:45:10.705 INFO:tasks.workunit.client.1.smithi152.stdout:6/286: dread d1/d3/d9/f27 [0,4194304] 0 2024-03-29T04:45:10.708 INFO:tasks.workunit.client.1.smithi152.stdout:7/181: creat d7/d8/d12/f33 x:0 0 0 2024-03-29T04:45:10.711 INFO:tasks.workunit.client.1.smithi152.stdout:7/182: chown d7/d8/d18/f2a 3 1 2024-03-29T04:45:10.712 INFO:tasks.workunit.client.0.smithi142.stdout:4/790: mkdir d0/d9/d11/d97/d100 0 2024-03-29T04:45:10.713 INFO:tasks.workunit.client.1.smithi152.stdout:8/160: rename d1/ld to d1/d2/d1d/l31 0 2024-03-29T04:45:10.713 INFO:tasks.workunit.client.1.smithi152.stdout:7/183: readlink d7/d8/d12/d1d/l26 0 2024-03-29T04:45:10.715 INFO:tasks.workunit.client.0.smithi142.stdout:8/917: chown d5/d1d/d2f/ce8 33383 1 2024-03-29T04:45:10.716 INFO:tasks.workunit.client.1.smithi152.stdout:4/215: dwrite d0/d7/de/d23/f28 [0,4194304] 0 2024-03-29T04:45:10.717 INFO:tasks.workunit.client.1.smithi152.stdout:7/184: write d7/d8/f2d [257328,82024] 0 2024-03-29T04:45:10.717 INFO:tasks.workunit.client.1.smithi152.stdout:3/228: write d7/d19/d1b/d1e/d22/f2d [770325,50583] 0 2024-03-29T04:45:10.718 INFO:tasks.workunit.client.0.smithi142.stdout:3/660: dread d3/d1f/f7e [0,4194304] 0 2024-03-29T04:45:10.719 INFO:tasks.workunit.client.0.smithi142.stdout:3/661: chown d3/f3c 237 1 2024-03-29T04:45:10.720 INFO:tasks.workunit.client.1.smithi152.stdout:8/161: creat d1/d21/f32 x:0 0 0 2024-03-29T04:45:10.722 INFO:tasks.workunit.client.1.smithi152.stdout:4/216: truncate d0/d7/de/d2a/f33 1002672 0 2024-03-29T04:45:10.722 INFO:tasks.workunit.client.1.smithi152.stdout:8/162: fdatasync d1/d2/d3/f17 0 2024-03-29T04:45:10.723 INFO:tasks.workunit.client.0.smithi142.stdout:9/694: symlink d0/d2/d61/d64/lde 0 2024-03-29T04:45:10.727 INFO:tasks.workunit.client.0.smithi142.stdout:4/791: truncate d0/d9/d11/f23 550136 0 2024-03-29T04:45:10.727 INFO:tasks.workunit.client.1.smithi152.stdout:3/229: symlink d7/d19/d1b/d1e/l3f 0 2024-03-29T04:45:10.728 INFO:tasks.workunit.client.1.smithi152.stdout:8/163: mknod d1/d21/c33 0 2024-03-29T04:45:10.729 INFO:tasks.workunit.client.1.smithi152.stdout:8/164: read d1/d2/d3/f17 [3249516,33606] 0 2024-03-29T04:45:10.730 INFO:tasks.workunit.client.1.smithi152.stdout:3/230: write d7/d19/f30 [4187477,62761] 0 2024-03-29T04:45:10.734 INFO:tasks.workunit.client.1.smithi152.stdout:7/185: dread d7/d8/d18/f1a [0,4194304] 0 2024-03-29T04:45:10.757 INFO:tasks.workunit.client.1.smithi152.stdout:7/186: creat d7/d8/d12/d1d/f34 x:0 0 0 2024-03-29T04:45:10.757 INFO:tasks.workunit.client.0.smithi142.stdout:9/695: creat d0/d2/d61/d64/fdf x:0 0 0 2024-03-29T04:45:10.758 INFO:tasks.workunit.client.1.smithi152.stdout:7/187: chown d7/d8/d18/f2a 9171605 1 2024-03-29T04:45:10.769 INFO:tasks.workunit.client.1.smithi152.stdout:8/165: creat d1/f34 x:0 0 0 2024-03-29T04:45:10.771 INFO:tasks.workunit.client.1.smithi152.stdout:8/166: write d1/d21/f32 [990448,122068] 0 2024-03-29T04:45:10.779 INFO:tasks.workunit.client.0.smithi142.stdout:7/881: sync 2024-03-29T04:45:10.779 INFO:tasks.workunit.client.0.smithi142.stdout:1/767: sync 2024-03-29T04:45:10.783 INFO:tasks.workunit.client.0.smithi142.stdout:9/696: dread d0/f94 [0,4194304] 0 2024-03-29T04:45:10.783 INFO:tasks.workunit.client.0.smithi142.stdout:4/792: mkdir d0/d9/d11/d101 0 2024-03-29T04:45:10.784 INFO:tasks.workunit.client.0.smithi142.stdout:8/918: mkdir d5/d1d/d2f/d76/d3c/d70/d13b/d140 0 2024-03-29T04:45:10.784 INFO:tasks.workunit.client.0.smithi142.stdout:9/697: chown d0/d2/d6/d27/dc7/daa/d95 50497 1 2024-03-29T04:45:10.785 INFO:tasks.workunit.client.1.smithi152.stdout:4/217: write d0/d27/f2b [560260,126230] 0 2024-03-29T04:45:10.785 INFO:tasks.workunit.client.0.smithi142.stdout:8/919: chown d5/d1d/d2f/d45/fe4 49748646 1 2024-03-29T04:45:10.786 INFO:tasks.workunit.client.1.smithi152.stdout:4/218: write d0/d27/f2b [143966,112987] 0 2024-03-29T04:45:10.787 INFO:tasks.workunit.client.1.smithi152.stdout:7/188: rename d7/d8/d18 to d7/d8/d12/d35 0 2024-03-29T04:45:10.788 INFO:tasks.workunit.client.0.smithi142.stdout:0/821: write d7/d13/dbf/d72/fb1 [852237,120937] 0 2024-03-29T04:45:10.788 INFO:tasks.workunit.client.0.smithi142.stdout:1/768: stat c2 0 2024-03-29T04:45:10.790 INFO:tasks.workunit.client.0.smithi142.stdout:7/882: mknod d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d8d/c127 0 2024-03-29T04:45:10.792 INFO:tasks.workunit.client.0.smithi142.stdout:8/920: fsync d5/d1d/d24/f2b 0 2024-03-29T04:45:10.793 INFO:tasks.workunit.client.0.smithi142.stdout:9/698: dwrite d0/d6c/fa1 [0,4194304] 0 2024-03-29T04:45:10.794 INFO:tasks.workunit.client.0.smithi142.stdout:8/921: chown d5/d2e/l5a 2659 1 2024-03-29T04:45:10.798 INFO:tasks.workunit.client.0.smithi142.stdout:4/793: unlink d0/d9/db2/dba/dbc/d8d/da5/fbf 0 2024-03-29T04:45:10.800 INFO:tasks.workunit.client.1.smithi152.stdout:4/219: symlink d0/d7/d25/d17/l38 0 2024-03-29T04:45:10.800 INFO:tasks.workunit.client.0.smithi142.stdout:4/794: chown d0/d9/d11/d4c/d64/d94/ddd 424 1 2024-03-29T04:45:10.802 INFO:tasks.workunit.client.0.smithi142.stdout:7/883: creat d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/f128 x:0 0 0 2024-03-29T04:45:10.802 INFO:tasks.workunit.client.1.smithi152.stdout:7/189: symlink d7/d8/d12/d1d/d30/l36 0 2024-03-29T04:45:10.803 INFO:tasks.workunit.client.0.smithi142.stdout:3/662: dwrite d3/d29/d8f/d5f/db4/fa1 [0,4194304] 0 2024-03-29T04:45:10.804 INFO:tasks.workunit.client.1.smithi152.stdout:4/220: rename d0/d7/de/d2a/f33 to d0/d27/f39 0 2024-03-29T04:45:10.805 INFO:tasks.workunit.client.0.smithi142.stdout:9/699: mknod d0/d12/d50/d52/da3/ce0 0 2024-03-29T04:45:10.805 INFO:tasks.workunit.client.1.smithi152.stdout:4/221: fsync d0/d7/d25/f21 0 2024-03-29T04:45:10.806 INFO:tasks.workunit.client.1.smithi152.stdout:8/167: creat d1/d2/d1d/f35 x:0 0 0 2024-03-29T04:45:10.810 INFO:tasks.workunit.client.1.smithi152.stdout:7/190: mkdir d7/d8/d37 0 2024-03-29T04:45:10.812 INFO:tasks.workunit.client.0.smithi142.stdout:3/663: creat d3/d29/d2b/fe2 x:0 0 0 2024-03-29T04:45:10.812 INFO:tasks.workunit.client.1.smithi152.stdout:4/222: mkdir d0/d7/de/d3a 0 2024-03-29T04:45:10.814 INFO:tasks.workunit.client.0.smithi142.stdout:8/922: symlink d5/d1d/d2f/d76/d2d/d13a/l141 0 2024-03-29T04:45:10.814 INFO:tasks.workunit.client.1.smithi152.stdout:8/168: dread d1/d21/f25 [0,4194304] 0 2024-03-29T04:45:10.814 INFO:tasks.workunit.client.1.smithi152.stdout:8/169: fdatasync d1/f10 0 2024-03-29T04:45:10.815 INFO:tasks.workunit.client.0.smithi142.stdout:7/884: dwrite d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/f7c [4194304,4194304] 0 2024-03-29T04:45:10.815 INFO:tasks.workunit.client.1.smithi152.stdout:4/223: creat d0/d7/d25/d17/f3b x:0 0 0 2024-03-29T04:45:10.816 INFO:tasks.workunit.client.1.smithi152.stdout:7/191: getdents d7/d8/d37 0 2024-03-29T04:45:10.816 INFO:tasks.workunit.client.0.smithi142.stdout:8/923: chown d5/d1d/d24/f2b 23487 1 2024-03-29T04:45:10.816 INFO:tasks.workunit.client.0.smithi142.stdout:9/700: unlink d0/d2/d39/fbf 0 2024-03-29T04:45:10.819 INFO:tasks.workunit.client.1.smithi152.stdout:7/192: mknod d7/d8/d12/d1d/c38 0 2024-03-29T04:45:10.819 INFO:tasks.workunit.client.0.smithi142.stdout:3/664: creat d3/fe3 x:0 0 0 2024-03-29T04:45:10.821 INFO:tasks.workunit.client.0.smithi142.stdout:7/885: truncate d7/d27/f48 806637 0 2024-03-29T04:45:10.822 INFO:tasks.workunit.client.1.smithi152.stdout:8/170: mkdir d1/d2/d1d/d36 0 2024-03-29T04:45:10.823 INFO:tasks.workunit.client.1.smithi152.stdout:8/171: truncate d1/d21/f25 1191909 0 2024-03-29T04:45:10.823 INFO:tasks.workunit.client.0.smithi142.stdout:3/665: dread - d3/d29/d2b/d40/d67/fdb zero size 2024-03-29T04:45:10.824 INFO:tasks.workunit.client.0.smithi142.stdout:8/924: unlink d5/d1d/d2f/d45/dc3/l123 0 2024-03-29T04:45:10.824 INFO:tasks.workunit.client.0.smithi142.stdout:9/701: read d0/d12/d50/f90 [3672256,46197] 0 2024-03-29T04:45:10.824 INFO:tasks.workunit.client.0.smithi142.stdout:0/822: dwrite d7/d13/d4a/d9a/fcc [0,4194304] 0 2024-03-29T04:45:10.827 INFO:tasks.workunit.client.0.smithi142.stdout:7/886: readlink d7/d27/d44/db2/db8/d4b/d71/d30/d72/l8a 0 2024-03-29T04:45:10.827 INFO:tasks.workunit.client.0.smithi142.stdout:1/769: dread d8/f2a [4194304,4194304] 0 2024-03-29T04:45:10.827 INFO:tasks.workunit.client.0.smithi142.stdout:3/666: chown d3/d29/d2d/d6c/lae 523708423 1 2024-03-29T04:45:10.828 INFO:tasks.workunit.client.0.smithi142.stdout:9/702: dread - d0/d2/d6/d31/f71 zero size 2024-03-29T04:45:10.828 INFO:tasks.workunit.client.1.smithi152.stdout:8/172: symlink d1/d21/l37 0 2024-03-29T04:45:10.829 INFO:tasks.workunit.client.0.smithi142.stdout:3/667: read - d3/d1f/d59/fbb zero size 2024-03-29T04:45:10.829 INFO:tasks.workunit.client.0.smithi142.stdout:8/925: chown d5/d1d/d2f/d76/d3c/d70 46491435 1 2024-03-29T04:45:10.830 INFO:tasks.workunit.client.1.smithi152.stdout:4/224: creat d0/d7/f3c x:0 0 0 2024-03-29T04:45:10.830 INFO:tasks.workunit.client.1.smithi152.stdout:4/225: fdatasync d0/d7/f35 0 2024-03-29T04:45:10.831 INFO:tasks.workunit.client.0.smithi142.stdout:8/926: stat d5/d1d/d2f/fa9 0 2024-03-29T04:45:10.832 INFO:tasks.workunit.client.0.smithi142.stdout:7/887: symlink d7/d27/d44/l129 0 2024-03-29T04:45:10.835 INFO:tasks.workunit.client.1.smithi152.stdout:8/173: symlink d1/d2/l38 0 2024-03-29T04:45:10.838 INFO:tasks.workunit.client.0.smithi142.stdout:7/888: dwrite d7/d27/d44/dab/fda [0,4194304] 0 2024-03-29T04:45:10.843 INFO:tasks.workunit.client.1.smithi152.stdout:8/174: symlink d1/d2/l39 0 2024-03-29T04:45:10.844 INFO:tasks.workunit.client.0.smithi142.stdout:0/823: rename d7/d9/f1c to d7/df6/f109 0 2024-03-29T04:45:10.845 INFO:tasks.workunit.client.1.smithi152.stdout:4/226: dread d0/d7/d25/f21 [0,4194304] 0 2024-03-29T04:45:10.845 INFO:tasks.workunit.client.0.smithi142.stdout:2/807: truncate de/d11/d17/d60/dc4/ff5 2050184 0 2024-03-29T04:45:10.846 INFO:tasks.workunit.client.1.smithi152.stdout:8/175: truncate d1/f34 598400 0 2024-03-29T04:45:10.847 INFO:tasks.workunit.client.0.smithi142.stdout:2/808: mknod de/d11/d17/d60/d6f/c10c 0 2024-03-29T04:45:10.848 INFO:tasks.workunit.client.1.smithi152.stdout:7/193: dwrite d7/d8/d12/f22 [0,4194304] 0 2024-03-29T04:45:10.850 INFO:tasks.workunit.client.0.smithi142.stdout:7/889: symlink d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/dc1/l12a 0 2024-03-29T04:45:10.853 INFO:tasks.workunit.client.1.smithi152.stdout:8/176: creat d1/d2/d1d/f3a x:0 0 0 2024-03-29T04:45:10.855 INFO:tasks.workunit.client.0.smithi142.stdout:9/703: rename d0/d2/d6/f30 to d0/fe1 0 2024-03-29T04:45:10.857 INFO:tasks.workunit.client.1.smithi152.stdout:8/177: dread d1/f34 [0,4194304] 0 2024-03-29T04:45:10.858 INFO:tasks.workunit.client.1.smithi152.stdout:7/194: dwrite d7/d8/d12/f33 [0,4194304] 0 2024-03-29T04:45:10.858 INFO:tasks.workunit.client.1.smithi152.stdout:8/178: write d1/d21/f24 [297273,12984] 0 2024-03-29T04:45:10.860 INFO:tasks.workunit.client.0.smithi142.stdout:0/824: rename d7/d13/d4a/d9a/da7/fad to d7/d9/d6e/f10a 0 2024-03-29T04:45:10.860 INFO:tasks.workunit.client.1.smithi152.stdout:8/179: chown d1/d21/f25 34556846 1 2024-03-29T04:45:10.860 INFO:tasks.workunit.client.0.smithi142.stdout:7/890: dwrite d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/f40 [0,4194304] 0 2024-03-29T04:45:10.862 INFO:tasks.workunit.client.0.smithi142.stdout:0/825: chown d7/d13/dbf/dca/dab/dc1/fce 2195 1 2024-03-29T04:45:10.862 INFO:tasks.workunit.client.1.smithi152.stdout:8/180: dread d1/f34 [0,4194304] 0 2024-03-29T04:45:10.863 INFO:tasks.workunit.client.0.smithi142.stdout:0/826: chown d7/d13/dbf/d72/d75/c93 2 1 2024-03-29T04:45:10.865 INFO:tasks.workunit.client.1.smithi152.stdout:8/181: write d1/d2/d1d/f23 [425093,56785] 0 2024-03-29T04:45:10.867 INFO:tasks.workunit.client.1.smithi152.stdout:5/213: sync 2024-03-29T04:45:10.868 INFO:tasks.workunit.client.1.smithi152.stdout:7/195: dwrite d7/d8/db/df/f17 [0,4194304] 0 2024-03-29T04:45:10.869 INFO:tasks.workunit.client.0.smithi142.stdout:8/927: dread d5/d2e/dd2/f8b [0,4194304] 0 2024-03-29T04:45:10.870 INFO:tasks.workunit.client.1.smithi152.stdout:5/214: creat d6/d7/f4c x:0 0 0 2024-03-29T04:45:10.870 INFO:tasks.workunit.client.1.smithi152.stdout:5/215: chown d6 51 1 2024-03-29T04:45:10.872 INFO:tasks.workunit.client.1.smithi152.stdout:8/182: rename d1/d21/c33 to d1/d21/c3b 0 2024-03-29T04:45:10.874 INFO:tasks.workunit.client.1.smithi152.stdout:7/196: truncate d7/f14 4630304 0 2024-03-29T04:45:10.877 INFO:tasks.workunit.client.0.smithi142.stdout:0/827: rmdir d7/d13/dbf/dca/dab/dc1/dd3/d90 39 2024-03-29T04:45:10.877 INFO:tasks.workunit.client.0.smithi142.stdout:7/891: rmdir d7/d27/d44/db2/db8/d4b/d71/d30/d33/d10c 0 2024-03-29T04:45:10.879 INFO:tasks.workunit.client.1.smithi152.stdout:7/197: write d7/d8/db/df/f2b [4602472,74695] 0 2024-03-29T04:45:10.883 INFO:tasks.workunit.client.1.smithi152.stdout:5/216: creat d6/d16/f4d x:0 0 0 2024-03-29T04:45:10.884 INFO:tasks.workunit.client.1.smithi152.stdout:5/217: truncate d6/d7/f4c 979297 0 2024-03-29T04:45:10.888 INFO:tasks.workunit.client.1.smithi152.stdout:5/218: creat d6/d16/f4e x:0 0 0 2024-03-29T04:45:10.890 INFO:tasks.workunit.client.0.smithi142.stdout:2/809: dread de/d11/d17/d60/fc6 [0,4194304] 0 2024-03-29T04:45:10.892 INFO:tasks.workunit.client.0.smithi142.stdout:1/770: write d8/db/dc/d77/d5a/fe6 [1565404,63160] 0 2024-03-29T04:45:10.892 INFO:tasks.workunit.client.0.smithi142.stdout:3/668: write d3/d1f/d24/d47/f4f [610285,127631] 0 2024-03-29T04:45:10.893 INFO:tasks.workunit.client.0.smithi142.stdout:2/810: symlink de/d11/d17/l10d 0 2024-03-29T04:45:10.903 INFO:tasks.workunit.client.0.smithi142.stdout:0/828: creat d7/d13/dbf/dca/f10b x:0 0 0 2024-03-29T04:45:10.904 INFO:tasks.workunit.client.0.smithi142.stdout:7/892: dread d7/d27/d44/db2/db8/d4b/d71/d30/d33/f11c [4194304,4194304] 0 2024-03-29T04:45:10.905 INFO:tasks.workunit.client.1.smithi152.stdout:5/219: write d6/dd/fe [1932870,124225] 0 2024-03-29T04:45:10.907 INFO:tasks.workunit.client.1.smithi152.stdout:5/220: write d6/d1d/d30/f46 [21176,130271] 0 2024-03-29T04:45:10.907 INFO:tasks.workunit.client.0.smithi142.stdout:3/669: rename d3/d29/d2b/d40/ca0 to d3/d29/d8f/d95/ce4 0 2024-03-29T04:45:10.908 INFO:tasks.workunit.client.0.smithi142.stdout:0/829: rename d7 to d7/d13/dbf/dca/d101/d10c 22 2024-03-29T04:45:10.909 INFO:tasks.workunit.client.0.smithi142.stdout:0/830: readlink d7/d9/d55/l4f 0 2024-03-29T04:45:10.909 INFO:tasks.workunit.client.1.smithi152.stdout:5/221: rmdir d6/d1b 39 2024-03-29T04:45:10.910 INFO:tasks.workunit.client.1.smithi152.stdout:5/222: fsync d6/f41 0 2024-03-29T04:45:10.910 INFO:tasks.workunit.client.0.smithi142.stdout:3/670: write d3/d29/d2b/d40/d67/fb6 [356563,59870] 0 2024-03-29T04:45:10.911 INFO:tasks.workunit.client.0.smithi142.stdout:0/831: write d7/d9/d54/fed [627003,80848] 0 2024-03-29T04:45:10.914 INFO:tasks.workunit.client.1.smithi152.stdout:5/223: rename d6/d16/d47/l4b to d6/dd/l4f 0 2024-03-29T04:45:10.916 INFO:tasks.workunit.client.0.smithi142.stdout:2/811: unlink de/d11/d17/d10b/d22/d27/f66 0 2024-03-29T04:45:10.916 INFO:tasks.workunit.client.0.smithi142.stdout:1/771: getdents d8/db 0 2024-03-29T04:45:10.917 INFO:tasks.workunit.client.0.smithi142.stdout:0/832: symlink d7/d9/d55/df7/l10d 0 2024-03-29T04:45:10.919 INFO:tasks.workunit.client.1.smithi152.stdout:5/224: symlink d6/d1b/l50 0 2024-03-29T04:45:10.920 INFO:tasks.workunit.client.1.smithi152.stdout:5/225: fsync d6/d1b/f45 0 2024-03-29T04:45:10.920 INFO:tasks.workunit.client.1.smithi152.stdout:5/226: stat d6/d16/d25/d2e/l37 0 2024-03-29T04:45:10.921 INFO:tasks.workunit.client.0.smithi142.stdout:3/671: rename d3/d29/d2b/d40/d67/d8b/lc3 to d3/d1f/d24/d47/d87/le5 0 2024-03-29T04:45:10.921 INFO:tasks.workunit.client.1.smithi152.stdout:5/227: chown d6/dd/f35 62425512 1 2024-03-29T04:45:10.922 INFO:tasks.workunit.client.0.smithi142.stdout:1/772: mkdir d8/db/dc/d77/db9/dbe/d2d/d44/dff/d108 0 2024-03-29T04:45:10.923 INFO:tasks.workunit.client.0.smithi142.stdout:7/893: dread d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/f21 [0,4194304] 0 2024-03-29T04:45:10.923 INFO:tasks.workunit.client.0.smithi142.stdout:0/833: creat d7/f10e x:0 0 0 2024-03-29T04:45:10.924 INFO:tasks.workunit.client.1.smithi152.stdout:5/228: symlink d6/dd/l51 0 2024-03-29T04:45:10.925 INFO:tasks.workunit.client.0.smithi142.stdout:3/672: mkdir d3/d1f/de6 0 2024-03-29T04:45:10.927 INFO:tasks.workunit.client.1.smithi152.stdout:5/229: link l4 d6/d1d/l52 0 2024-03-29T04:45:10.929 INFO:tasks.workunit.client.0.smithi142.stdout:7/894: dread - d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/f128 zero size 2024-03-29T04:45:10.929 INFO:tasks.workunit.client.0.smithi142.stdout:3/673: rmdir d3/d29/d2b/d40/d67/d8b/d6 39 2024-03-29T04:45:10.929 INFO:tasks.workunit.client.1.smithi152.stdout:5/230: unlink d6/d1b/f42 0 2024-03-29T04:45:10.930 INFO:tasks.workunit.client.0.smithi142.stdout:3/674: chown d3/d29/d2d/c65 91393104 1 2024-03-29T04:45:10.931 INFO:tasks.workunit.client.0.smithi142.stdout:0/834: creat d7/d13/dbf/dca/dab/dc1/dd3/d90/f10f x:0 0 0 2024-03-29T04:45:10.932 INFO:tasks.workunit.client.1.smithi152.stdout:5/231: dread d6/d7/f4c [0,4194304] 0 2024-03-29T04:45:10.933 INFO:tasks.workunit.client.0.smithi142.stdout:8/928: truncate d5/d1d/d24/d66/dce/f105 150044 0 2024-03-29T04:45:10.934 INFO:tasks.workunit.client.0.smithi142.stdout:2/812: dread de/d11/d17/d10b/d22/d50/f62 [0,4194304] 0 2024-03-29T04:45:10.935 INFO:tasks.workunit.client.0.smithi142.stdout:8/929: stat d5/d1d/d2f/d5f/d12c/lf5 0 2024-03-29T04:45:10.935 INFO:tasks.workunit.client.0.smithi142.stdout:7/895: symlink d7/d27/d44/db2/db8/d4b/d71/d30/d33/d10b/l12b 0 2024-03-29T04:45:10.937 INFO:tasks.workunit.client.0.smithi142.stdout:2/813: readlink de/d11/d17/d10b/d22/d27/dba/lc1 0 2024-03-29T04:45:10.940 INFO:tasks.workunit.client.0.smithi142.stdout:2/814: read - de/d11/d3d/fbc zero size 2024-03-29T04:45:10.942 INFO:tasks.workunit.client.0.smithi142.stdout:8/930: rename d5/d2e/dd2/d60/ffe to d5/d1d/d2f/d76/d3c/d70/de9/f142 0 2024-03-29T04:45:10.944 INFO:tasks.workunit.client.0.smithi142.stdout:1/773: dread d8/db/f14 [0,4194304] 0 2024-03-29T04:45:10.947 INFO:tasks.workunit.client.1.smithi152.stdout:5/232: truncate d6/d1d/f40 4114532 0 2024-03-29T04:45:10.948 INFO:tasks.workunit.client.1.smithi152.stdout:5/233: write d6/d1d/d30/f38 [323825,60557] 0 2024-03-29T04:45:10.951 INFO:tasks.workunit.client.1.smithi152.stdout:5/234: mknod d6/d16/d47/c53 0 2024-03-29T04:45:10.951 INFO:tasks.workunit.client.0.smithi142.stdout:8/931: write d5/d2e/dd2/f8b [132883,52719] 0 2024-03-29T04:45:10.953 INFO:tasks.workunit.client.0.smithi142.stdout:7/896: truncate d7/d27/d44/db2/db8/d4b/d71/f28 1818656 0 2024-03-29T04:45:10.953 INFO:tasks.workunit.client.0.smithi142.stdout:2/815: dwrite de/f16 [0,4194304] 0 2024-03-29T04:45:10.954 INFO:tasks.workunit.client.0.smithi142.stdout:1/774: dread d8/f70 [0,4194304] 0 2024-03-29T04:45:10.956 INFO:tasks.workunit.client.0.smithi142.stdout:6/785: sync 2024-03-29T04:45:10.956 INFO:tasks.workunit.client.1.smithi152.stdout:5/235: getdents d6/dd 0 2024-03-29T04:45:10.957 INFO:tasks.workunit.client.1.smithi152.stdout:5/236: chown d6/d1d/f2c 129770159 1 2024-03-29T04:45:10.958 INFO:tasks.workunit.client.0.smithi142.stdout:0/835: dwrite d7/d13/dbf/d72/fd6 [0,4194304] 0 2024-03-29T04:45:10.961 INFO:tasks.workunit.client.0.smithi142.stdout:3/675: dwrite d3/d29/d2d/f48 [0,4194304] 0 2024-03-29T04:45:10.961 INFO:tasks.workunit.client.0.smithi142.stdout:6/786: read - d1c/d45/d8a/fea zero size 2024-03-29T04:45:10.962 INFO:tasks.workunit.client.1.smithi152.stdout:5/237: dread d6/d16/d25/d2e/f44 [0,4194304] 0 2024-03-29T04:45:10.963 INFO:tasks.workunit.client.1.smithi152.stdout:5/238: write d6/d1d/d30/f46 [696048,74160] 0 2024-03-29T04:45:10.967 INFO:tasks.workunit.client.1.smithi152.stdout:5/239: creat d6/d16/d25/d2e/f54 x:0 0 0 2024-03-29T04:45:10.987 INFO:tasks.workunit.client.0.smithi142.stdout:8/932: fdatasync d5/d1d/d2f/d76/d3c/d70/fda 0 2024-03-29T04:45:10.987 INFO:tasks.workunit.client.0.smithi142.stdout:6/787: rmdir d1c/d45 39 2024-03-29T04:45:10.987 INFO:tasks.workunit.client.0.smithi142.stdout:3/676: mkdir d3/d1f/d59/de7 0 2024-03-29T04:45:10.989 INFO:tasks.workunit.client.0.smithi142.stdout:3/677: fsync d3/d29/d8f/d5f/db4/f8d 0 2024-03-29T04:45:11.028 INFO:tasks.workunit.client.0.smithi142.stdout:6/788: creat d1c/d45/d8a/df6/ffe x:0 0 0 2024-03-29T04:45:11.028 INFO:tasks.workunit.client.0.smithi142.stdout:2/816: dread de/d11/d17/d10b/d22/d27/f4c [0,4194304] 0 2024-03-29T04:45:11.028 INFO:tasks.workunit.client.0.smithi142.stdout:3/678: stat d3/d1f/d59/de7 0 2024-03-29T04:45:11.029 INFO:tasks.workunit.client.0.smithi142.stdout:6/789: fdatasync f18 0 2024-03-29T04:45:11.030 INFO:tasks.workunit.client.0.smithi142.stdout:0/836: dwrite d7/d9/d1e/f100 [0,4194304] 0 2024-03-29T04:45:11.034 INFO:tasks.workunit.client.0.smithi142.stdout:0/837: dread - d7/d13/dbf/dca/dab/ddf/ffb zero size 2024-03-29T04:45:11.035 INFO:tasks.workunit.client.0.smithi142.stdout:8/933: dwrite d5/d1d/f85 [0,4194304] 0 2024-03-29T04:45:11.036 INFO:tasks.workunit.client.0.smithi142.stdout:2/817: mkdir de/d11/d17/d10b/d22/d27/d3a/d10e 0 2024-03-29T04:45:11.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:10 smithi152 ceph-mon[37177]: pgmap v271: 65 pgs: 65 active+clean; 1.8 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 65 MiB/s rd, 220 MiB/s wr, 421 op/s 2024-03-29T04:45:11.040 INFO:tasks.workunit.client.0.smithi142.stdout:7/897: dread d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/f98 [4194304,4194304] 0 2024-03-29T04:45:11.041 INFO:tasks.workunit.client.0.smithi142.stdout:8/934: mknod d5/d1d/d2f/d5f/d12c/c143 0 2024-03-29T04:45:11.042 INFO:tasks.workunit.client.0.smithi142.stdout:0/838: mknod d7/d9/d55/c110 0 2024-03-29T04:45:11.044 INFO:tasks.workunit.client.0.smithi142.stdout:3/679: dwrite d3/d29/d2b/d40/d67/fb7 [0,4194304] 0 2024-03-29T04:45:11.046 INFO:tasks.workunit.client.0.smithi142.stdout:6/790: dwrite d1c/d33/d46/d61/fee [0,4194304] 0 2024-03-29T04:45:11.048 INFO:tasks.workunit.client.0.smithi142.stdout:2/818: getdents de/d11/d23 0 2024-03-29T04:45:11.048 INFO:tasks.workunit.client.0.smithi142.stdout:8/935: write d5/d1d/d59/f7c [1361460,89488] 0 2024-03-29T04:45:11.048 INFO:tasks.workunit.client.0.smithi142.stdout:7/898: fsync d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/d3f/f41 0 2024-03-29T04:45:11.051 INFO:tasks.workunit.client.0.smithi142.stdout:6/791: mkdir d1c/dc7/df2/dff 0 2024-03-29T04:45:11.053 INFO:tasks.workunit.client.0.smithi142.stdout:3/680: rename d3/d29/d2b/d40/f4a to d3/d29/d2b/fe8 0 2024-03-29T04:45:11.053 INFO:tasks.workunit.client.0.smithi142.stdout:8/936: mkdir d5/d2e/d144 0 2024-03-29T04:45:11.055 INFO:tasks.workunit.client.0.smithi142.stdout:8/937: truncate d5/f128 428187 0 2024-03-29T04:45:11.056 INFO:tasks.workunit.client.0.smithi142.stdout:2/819: write de/d11/d17/d10b/d22/d27/dba/d61/fa0 [803339,4290] 0 2024-03-29T04:45:11.059 INFO:tasks.workunit.client.0.smithi142.stdout:3/681: mkdir d3/d29/d2d/de9 0 2024-03-29T04:45:11.059 INFO:tasks.workunit.client.0.smithi142.stdout:8/938: write d5/d1d/d81/f133 [491137,125669] 0 2024-03-29T04:45:11.062 INFO:tasks.workunit.client.0.smithi142.stdout:2/820: creat de/d11/d17/d10b/d22/d27/f10f x:0 0 0 2024-03-29T04:45:11.063 INFO:tasks.workunit.client.0.smithi142.stdout:3/682: write d3/d1f/d24/fa5 [698300,11207] 0 2024-03-29T04:45:11.066 INFO:tasks.workunit.client.0.smithi142.stdout:6/792: dwrite d1c/d45/d8a/df6/f95 [0,4194304] 0 2024-03-29T04:45:11.067 INFO:tasks.workunit.client.0.smithi142.stdout:6/793: chown d1c/l3b 14 1 2024-03-29T04:45:11.067 INFO:tasks.workunit.client.0.smithi142.stdout:8/939: truncate d5/d1d/d24/d66/dce/f105 814844 0 2024-03-29T04:45:11.067 INFO:tasks.workunit.client.0.smithi142.stdout:6/794: fdatasync d1c/d33/f3e 0 2024-03-29T04:45:11.073 INFO:tasks.workunit.client.0.smithi142.stdout:0/839: dread d7/d13/d4a/f5e [0,4194304] 0 2024-03-29T04:45:11.074 INFO:tasks.workunit.client.0.smithi142.stdout:6/795: getdents d1c/d45/d51 0 2024-03-29T04:45:11.074 INFO:tasks.workunit.client.0.smithi142.stdout:2/821: write de/d11/d17/f25 [3039782,38312] 0 2024-03-29T04:45:11.075 INFO:tasks.workunit.client.0.smithi142.stdout:0/840: readlink d7/l12 0 2024-03-29T04:45:11.075 INFO:tasks.workunit.client.0.smithi142.stdout:6/796: chown d1c/dc7/lca 35 1 2024-03-29T04:45:11.077 INFO:tasks.workunit.client.0.smithi142.stdout:6/797: fdatasync fc 0 2024-03-29T04:45:11.078 INFO:tasks.workunit.client.0.smithi142.stdout:0/841: mknod d7/d9/d54/d58/c111 0 2024-03-29T04:45:11.079 INFO:tasks.workunit.client.0.smithi142.stdout:7/899: write d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/ddf/fe1 [869805,123694] 0 2024-03-29T04:45:11.079 INFO:tasks.workunit.client.0.smithi142.stdout:0/842: readlink d7/d9/d1e/d81/le3 0 2024-03-29T04:45:11.082 INFO:tasks.workunit.client.0.smithi142.stdout:7/900: write d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/dc1/f11f [50150,16515] 0 2024-03-29T04:45:11.086 INFO:tasks.workunit.client.0.smithi142.stdout:6/798: link d1c/f39 d1c/d33/d46/d66/dc1/f100 0 2024-03-29T04:45:11.088 INFO:tasks.workunit.client.0.smithi142.stdout:2/822: dwrite de/d11/d23/f46 [0,4194304] 0 2024-03-29T04:45:11.089 INFO:tasks.workunit.client.0.smithi142.stdout:6/799: mknod d1c/d33/d46/c101 0 2024-03-29T04:45:11.089 INFO:tasks.workunit.client.0.smithi142.stdout:8/940: write d5/d1d/d24/d66/dce/f105 [584030,118073] 0 2024-03-29T04:45:11.094 INFO:tasks.workunit.client.0.smithi142.stdout:2/823: fdatasync de/d11/d17/d10b/d22/d27/dba/d61/d7d/fbe 0 2024-03-29T04:45:11.094 INFO:tasks.workunit.client.0.smithi142.stdout:0/843: dwrite d7/d9/d55/f98 [0,4194304] 0 2024-03-29T04:45:11.095 INFO:tasks.workunit.client.0.smithi142.stdout:8/941: chown d5/d1d/d2f/d5f/d12c/c143 455 1 2024-03-29T04:45:11.100 INFO:tasks.workunit.client.0.smithi142.stdout:2/824: write de/d11/d3d/fd8 [1289114,73410] 0 2024-03-29T04:45:11.100 INFO:tasks.workunit.client.0.smithi142.stdout:0/844: unlink d7/d9/d54/d58/f6b 0 2024-03-29T04:45:11.104 INFO:tasks.workunit.client.0.smithi142.stdout:6/800: dwrite f4 [0,4194304] 0 2024-03-29T04:45:11.104 INFO:tasks.workunit.client.0.smithi142.stdout:8/942: write d5/d1d/d2f/fc8 [936294,91445] 0 2024-03-29T04:45:11.105 INFO:tasks.workunit.client.0.smithi142.stdout:0/845: stat d7/d13/dbf/dca/dab/dc1/dd3 0 2024-03-29T04:45:11.105 INFO:tasks.workunit.client.0.smithi142.stdout:7/901: dread d7/d27/d44/db2/db8/d4b/d71/d30/d33/f38 [0,4194304] 0 2024-03-29T04:45:11.108 INFO:tasks.workunit.client.0.smithi142.stdout:6/801: symlink d1c/dc7/df2/dff/l102 0 2024-03-29T04:45:11.109 INFO:tasks.workunit.client.0.smithi142.stdout:0/846: mknod d7/d13/dbf/dca/dab/dc1/dd0/c112 0 2024-03-29T04:45:11.109 INFO:tasks.workunit.client.0.smithi142.stdout:2/825: rmdir de/d11/d17/d10b/df4 0 2024-03-29T04:45:11.114 INFO:tasks.workunit.client.0.smithi142.stdout:0/847: rename d7/d13/lbd to d7/l113 0 2024-03-29T04:45:11.117 INFO:tasks.workunit.client.0.smithi142.stdout:0/848: mkdir d7/d13/dbf/d72/d37/da4/d114 0 2024-03-29T04:45:11.120 INFO:tasks.workunit.client.0.smithi142.stdout:6/802: truncate d1c/d33/d46/d61/d9e/fd1 1067446 0 2024-03-29T04:45:11.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:10 smithi142 ceph-mon[29027]: pgmap v271: 65 pgs: 65 active+clean; 1.8 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 65 MiB/s rd, 220 MiB/s wr, 421 op/s 2024-03-29T04:45:11.121 INFO:tasks.workunit.client.0.smithi142.stdout:6/803: fsync d1c/d33/d46/d71/f60 0 2024-03-29T04:45:11.122 INFO:tasks.workunit.client.0.smithi142.stdout:6/804: dread - d1c/d90/fd5 zero size 2024-03-29T04:45:11.123 INFO:tasks.workunit.client.0.smithi142.stdout:7/902: dwrite d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/dc1/fee [0,4194304] 0 2024-03-29T04:45:11.136 INFO:tasks.workunit.client.1.smithi152.stdout:1/238: sync 2024-03-29T04:45:11.137 INFO:tasks.workunit.client.1.smithi152.stdout:1/239: chown d5/d7/d9/d2b/f2f 1 1 2024-03-29T04:45:11.140 INFO:tasks.workunit.client.1.smithi152.stdout:1/240: link d5/d7/d9/d12/c29 d5/d7/d1b/c4f 0 2024-03-29T04:45:11.146 INFO:tasks.workunit.client.1.smithi152.stdout:1/241: truncate d5/d7/d9/d2b/f37 2563876 0 2024-03-29T04:45:11.148 INFO:tasks.workunit.client.1.smithi152.stdout:1/242: rmdir d5/d7/d8 39 2024-03-29T04:45:11.150 INFO:tasks.workunit.client.1.smithi152.stdout:1/243: mknod d5/d7/d38/c50 0 2024-03-29T04:45:11.152 INFO:tasks.workunit.client.1.smithi152.stdout:1/244: write d5/d7/d8/f2d [515701,16088] 0 2024-03-29T04:45:11.154 INFO:tasks.workunit.client.1.smithi152.stdout:1/245: symlink d5/d7/d1b/d2e/l51 0 2024-03-29T04:45:11.158 INFO:tasks.workunit.client.1.smithi152.stdout:1/246: truncate d5/d7/d9/d2b/f37 1593690 0 2024-03-29T04:45:11.164 INFO:tasks.workunit.client.1.smithi152.stdout:1/247: dwrite d5/d7/fe [0,4194304] 0 2024-03-29T04:45:11.168 INFO:tasks.workunit.client.1.smithi152.stdout:1/248: creat d5/d7/d8/d13/d19/d21/f52 x:0 0 0 2024-03-29T04:45:11.177 INFO:tasks.workunit.client.1.smithi152.stdout:1/249: dwrite d5/d7/d8/d13/d19/f30 [0,4194304] 0 2024-03-29T04:45:11.177 INFO:tasks.workunit.client.0.smithi142.stdout:6/805: creat d1c/dc9/f103 x:0 0 0 2024-03-29T04:45:11.180 INFO:tasks.workunit.client.1.smithi152.stdout:1/250: symlink d5/d7/d1b/d2e/l53 0 2024-03-29T04:45:11.187 INFO:tasks.workunit.client.1.smithi152.stdout:1/251: dread d5/d7/d9/d2b/f2f [0,4194304] 0 2024-03-29T04:45:11.190 INFO:tasks.workunit.client.1.smithi152.stdout:1/252: creat d5/d7/f54 x:0 0 0 2024-03-29T04:45:11.193 INFO:tasks.workunit.client.0.smithi142.stdout:0/849: dread d7/d13/dbf/dca/f97 [0,4194304] 0 2024-03-29T04:45:11.194 INFO:tasks.workunit.client.0.smithi142.stdout:2/826: dread de/f19 [0,4194304] 0 2024-03-29T04:45:11.194 INFO:tasks.workunit.client.1.smithi152.stdout:1/253: dread d5/d7/d8/f2d [0,4194304] 0 2024-03-29T04:45:11.195 INFO:tasks.workunit.client.0.smithi142.stdout:2/827: dread - de/d11/d17/d10b/d84/f103 zero size 2024-03-29T04:45:11.198 INFO:tasks.workunit.client.1.smithi152.stdout:1/254: fdatasync d5/d7/d9/d2b/f37 0 2024-03-29T04:45:11.200 INFO:tasks.workunit.client.0.smithi142.stdout:2/828: creat de/d11/d17/d10b/d22/f110 x:0 0 0 2024-03-29T04:45:11.201 INFO:tasks.workunit.client.1.smithi152.stdout:1/255: truncate d5/d7/d8/d13/f35 8774782 0 2024-03-29T04:45:11.203 INFO:tasks.workunit.client.1.smithi152.stdout:1/256: dread d5/d7/d9/d2b/f2f [0,4194304] 0 2024-03-29T04:45:11.205 INFO:tasks.workunit.client.1.smithi152.stdout:1/257: read d5/d7/d8/f1e [2883013,121663] 0 2024-03-29T04:45:11.207 INFO:tasks.workunit.client.1.smithi152.stdout:1/258: chown d5/d7/d9/d2b/f2f 268 1 2024-03-29T04:45:11.212 INFO:tasks.workunit.client.1.smithi152.stdout:1/259: link d5/d7/d9/l33 d5/d7/d8/d13/l55 0 2024-03-29T04:45:11.214 INFO:tasks.workunit.client.1.smithi152.stdout:1/260: creat d5/d7/d8/d13/d19/d21/f56 x:0 0 0 2024-03-29T04:45:11.217 INFO:tasks.workunit.client.1.smithi152.stdout:1/261: dread d5/d7/d9/d2b/f37 [0,4194304] 0 2024-03-29T04:45:11.218 INFO:tasks.workunit.client.1.smithi152.stdout:1/262: fsync d5/d7/fe 0 2024-03-29T04:45:11.219 INFO:tasks.workunit.client.1.smithi152.stdout:1/263: write d5/d7/d9/f11 [4739806,117953] 0 2024-03-29T04:45:11.223 INFO:tasks.workunit.client.0.smithi142.stdout:2/829: write de/d11/d17/f25 [7486405,84529] 0 2024-03-29T04:45:11.227 INFO:tasks.workunit.client.1.smithi152.stdout:1/264: dwrite d5/d7/f1f [0,4194304] 0 2024-03-29T04:45:11.228 INFO:tasks.workunit.client.1.smithi152.stdout:1/265: write d5/d7/f25 [5877836,105366] 0 2024-03-29T04:45:11.229 INFO:tasks.workunit.client.0.smithi142.stdout:2/830: dwrite de/d11/d17/d10b/d22/d27/dba/d61/fa0 [0,4194304] 0 2024-03-29T04:45:11.231 INFO:tasks.workunit.client.0.smithi142.stdout:2/831: truncate de/d11/d3d/f105 497658 0 2024-03-29T04:45:11.232 INFO:tasks.workunit.client.0.smithi142.stdout:6/806: creat d1c/d45/d8a/df6/df4/f104 x:0 0 0 2024-03-29T04:45:11.234 INFO:tasks.workunit.client.1.smithi152.stdout:1/266: symlink d5/d7/d8/d13/d19/l57 0 2024-03-29T04:45:11.239 INFO:tasks.workunit.client.0.smithi142.stdout:2/832: rename de/d11/d17/d10b/d22/d27/dba/c8a to de/d11/d3d/c111 0 2024-03-29T04:45:11.241 INFO:tasks.workunit.client.1.smithi152.stdout:1/267: symlink d5/d7/d38/l58 0 2024-03-29T04:45:11.243 INFO:tasks.workunit.client.0.smithi142.stdout:2/833: creat de/d11/d17/d10b/d22/f112 x:0 0 0 2024-03-29T04:45:11.245 INFO:tasks.workunit.client.1.smithi152.stdout:1/268: unlink d5/d7/d8/d13/d19/d21/c26 0 2024-03-29T04:45:11.246 INFO:tasks.workunit.client.0.smithi142.stdout:2/834: truncate de/d11/d17/f31 2768069 0 2024-03-29T04:45:11.247 INFO:tasks.workunit.client.1.smithi152.stdout:1/269: mkdir d5/d7/d9/d59 0 2024-03-29T04:45:11.248 INFO:tasks.workunit.client.0.smithi142.stdout:2/835: symlink de/d11/d17/d10b/l113 0 2024-03-29T04:45:11.252 INFO:tasks.workunit.client.1.smithi152.stdout:1/270: rename d5/d7/d38/f4c to d5/d7/f5a 0 2024-03-29T04:45:11.264 INFO:tasks.workunit.client.1.smithi152.stdout:1/271: rmdir d5/d7 39 2024-03-29T04:45:11.264 INFO:tasks.workunit.client.0.smithi142.stdout:2/836: dread de/d11/d17/d10b/d22/f6c [0,4194304] 0 2024-03-29T04:45:11.265 INFO:tasks.workunit.client.1.smithi152.stdout:1/272: chown d5/d7/d38/l58 30 1 2024-03-29T04:45:11.266 INFO:tasks.workunit.client.1.smithi152.stdout:1/273: readlink d5/d7/d8/d13/d19/d21/l39 0 2024-03-29T04:45:11.269 INFO:tasks.workunit.client.1.smithi152.stdout:1/274: creat d5/d7/d9/d59/f5b x:0 0 0 2024-03-29T04:45:11.270 INFO:tasks.workunit.client.1.smithi152.stdout:1/275: mknod d5/d7/d4d/c5c 0 2024-03-29T04:45:11.272 INFO:tasks.workunit.client.1.smithi152.stdout:1/276: write d5/d7/f1f [2957965,125161] 0 2024-03-29T04:45:11.272 INFO:tasks.workunit.client.1.smithi152.stdout:2/189: sync 2024-03-29T04:45:11.273 INFO:tasks.workunit.client.1.smithi152.stdout:1/277: chown d5/d7/d8/d13/d19/d21/l39 171 1 2024-03-29T04:45:11.276 INFO:tasks.workunit.client.1.smithi152.stdout:2/190: rename d0/d3/df/d2c/f3c to d0/d3/d30/f3f 0 2024-03-29T04:45:11.277 INFO:tasks.workunit.client.1.smithi152.stdout:1/278: readlink d5/d7/d9/l33 0 2024-03-29T04:45:11.278 INFO:tasks.workunit.client.0.smithi142.stdout:6/807: dwrite d1c/d45/d51/f97 [0,4194304] 0 2024-03-29T04:45:11.279 INFO:tasks.workunit.client.1.smithi152.stdout:1/279: stat d5/d7/d8/d13/d19/d21 0 2024-03-29T04:45:11.279 INFO:tasks.workunit.client.1.smithi152.stdout:1/280: dread - d5/d7/f54 zero size 2024-03-29T04:45:11.279 INFO:tasks.workunit.client.1.smithi152.stdout:2/191: symlink d0/d3/d30/l40 0 2024-03-29T04:45:11.280 INFO:tasks.workunit.client.1.smithi152.stdout:2/192: dread - d0/f3b zero size 2024-03-29T04:45:11.281 INFO:tasks.workunit.client.0.smithi142.stdout:6/808: chown d1c/d33/d46/d66/dcd/le8 77 1 2024-03-29T04:45:11.282 INFO:tasks.workunit.client.0.smithi142.stdout:6/809: fsync d1c/d33/d46/d71/f9f 0 2024-03-29T04:45:11.283 INFO:tasks.workunit.client.0.smithi142.stdout:6/810: chown d1c/d33/d46/cfc 4969694 1 2024-03-29T04:45:11.283 INFO:tasks.workunit.client.0.smithi142.stdout:6/811: chown d1c/l91 447296279 1 2024-03-29T04:45:11.284 INFO:tasks.workunit.client.1.smithi152.stdout:2/193: write d0/d3/d30/d15/f25 [781563,99246] 0 2024-03-29T04:45:11.285 INFO:tasks.workunit.client.0.smithi142.stdout:6/812: fdatasync d1c/f49 0 2024-03-29T04:45:11.287 INFO:tasks.workunit.client.1.smithi152.stdout:2/194: mknod d0/c41 0 2024-03-29T04:45:11.287 INFO:tasks.workunit.client.1.smithi152.stdout:1/281: dwrite d5/d7/d8/d13/d19/d21/f56 [0,4194304] 0 2024-03-29T04:45:11.287 INFO:tasks.workunit.client.0.smithi142.stdout:1/775: sync 2024-03-29T04:45:11.288 INFO:tasks.workunit.client.0.smithi142.stdout:4/795: sync 2024-03-29T04:45:11.288 INFO:tasks.workunit.client.0.smithi142.stdout:9/704: sync 2024-03-29T04:45:11.288 INFO:tasks.workunit.client.0.smithi142.stdout:6/813: creat d1c/dc9/f105 x:0 0 0 2024-03-29T04:45:11.289 INFO:tasks.workunit.client.0.smithi142.stdout:9/705: write d0/d2/d39/fc5 [385332,76915] 0 2024-03-29T04:45:11.290 INFO:tasks.workunit.client.0.smithi142.stdout:4/796: dread - d0/d9/db2/dba/fc0 zero size 2024-03-29T04:45:11.294 INFO:tasks.workunit.client.1.smithi152.stdout:1/282: rename d5/d7/d1b/c4f to d5/d7/c5d 0 2024-03-29T04:45:11.295 INFO:tasks.workunit.client.0.smithi142.stdout:4/797: mkdir d0/d9/db2/dba/dbc/d8d/da5/d102 0 2024-03-29T04:45:11.296 INFO:tasks.workunit.client.0.smithi142.stdout:9/706: creat d0/d2/d6/d31/dc8/fe2 x:0 0 0 2024-03-29T04:45:11.297 INFO:tasks.workunit.client.1.smithi152.stdout:1/283: creat d5/d7/d1b/f5e x:0 0 0 2024-03-29T04:45:11.302 INFO:tasks.workunit.client.1.smithi152.stdout:1/284: dwrite d5/d7/f54 [0,4194304] 0 2024-03-29T04:45:11.304 INFO:tasks.workunit.client.1.smithi152.stdout:1/285: write d5/d7/d8/d13/d19/f30 [3677611,13997] 0 2024-03-29T04:45:11.308 INFO:tasks.workunit.client.0.smithi142.stdout:1/776: rmdir d8/db/dc/d77/d31 39 2024-03-29T04:45:11.309 INFO:tasks.workunit.client.0.smithi142.stdout:6/814: write d1c/f21 [453241,88283] 0 2024-03-29T04:45:11.311 INFO:tasks.workunit.client.0.smithi142.stdout:4/798: unlink d0/d9/d11/d4c/d5f/d81/dc9/d47/d7a/de4/ff7 0 2024-03-29T04:45:11.312 INFO:tasks.workunit.client.0.smithi142.stdout:9/707: stat d0/l42 0 2024-03-29T04:45:11.313 INFO:tasks.workunit.client.0.smithi142.stdout:9/708: fsync d0/d6c/fa1 0 2024-03-29T04:45:11.313 INFO:tasks.workunit.client.1.smithi152.stdout:1/286: dread - d5/f3e zero size 2024-03-29T04:45:11.314 INFO:tasks.workunit.client.0.smithi142.stdout:9/709: stat d0/d2/d6/d27/f97 0 2024-03-29T04:45:11.314 INFO:tasks.workunit.client.1.smithi152.stdout:1/287: dread - d5/d7/d8/d13/f45 zero size 2024-03-29T04:45:11.316 INFO:tasks.workunit.client.0.smithi142.stdout:4/799: creat d0/d45/f103 x:0 0 0 2024-03-29T04:45:11.317 INFO:tasks.workunit.client.0.smithi142.stdout:4/800: stat d0/d9/d11/d4c/d5f/d81/c83 0 2024-03-29T04:45:11.318 INFO:tasks.workunit.client.1.smithi152.stdout:2/195: dread d0/d3/df/f13 [0,4194304] 0 2024-03-29T04:45:11.319 INFO:tasks.workunit.client.0.smithi142.stdout:6/815: creat d1c/d90/ddc/dfb/f106 x:0 0 0 2024-03-29T04:45:11.319 INFO:tasks.workunit.client.1.smithi152.stdout:2/196: stat d0/d3/l7 0 2024-03-29T04:45:11.320 INFO:tasks.workunit.client.1.smithi152.stdout:2/197: read d0/d3/d30/d15/d22/f37 [1193267,53783] 0 2024-03-29T04:45:11.321 INFO:tasks.workunit.client.1.smithi152.stdout:2/198: chown d0/d3/d30/f19 794822 1 2024-03-29T04:45:11.321 INFO:tasks.workunit.client.0.smithi142.stdout:6/816: dread d1c/f21 [0,4194304] 0 2024-03-29T04:45:11.323 INFO:tasks.workunit.client.0.smithi142.stdout:6/817: fdatasync d1c/d45/d51/f7f 0 2024-03-29T04:45:11.323 INFO:tasks.workunit.client.0.smithi142.stdout:1/777: write d8/db/d37/f76 [655377,82793] 0 2024-03-29T04:45:11.324 INFO:tasks.workunit.client.1.smithi152.stdout:2/199: creat d0/d3/f42 x:0 0 0 2024-03-29T04:45:11.326 INFO:tasks.workunit.client.1.smithi152.stdout:2/200: write d0/d3/d30/d15/f20 [224231,99122] 0 2024-03-29T04:45:11.326 INFO:tasks.workunit.client.0.smithi142.stdout:4/801: dwrite d0/d9/d11/d84/fca [0,4194304] 0 2024-03-29T04:45:11.328 INFO:tasks.workunit.client.0.smithi142.stdout:6/818: fsync d1c/d33/d46/d61/fa1 0 2024-03-29T04:45:11.330 INFO:tasks.workunit.client.0.smithi142.stdout:1/778: unlink d8/db/dc/d77/db9/dbe/c2f 0 2024-03-29T04:45:11.332 INFO:tasks.workunit.client.0.smithi142.stdout:1/779: chown d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/fd8 68536413 1 2024-03-29T04:45:11.333 INFO:tasks.workunit.client.1.smithi152.stdout:2/201: unlink d0/d3/d30/d15/f3a 0 2024-03-29T04:45:11.335 INFO:tasks.workunit.client.0.smithi142.stdout:6/819: rename d1c/d33/f3e to d1c/dc7/df2/f107 0 2024-03-29T04:45:11.335 INFO:tasks.workunit.client.1.smithi152.stdout:2/202: write d0/d3/d30/f19 [7999698,91885] 0 2024-03-29T04:45:11.336 INFO:tasks.workunit.client.0.smithi142.stdout:4/802: mknod d0/d9/d11/d4c/d5f/d81/dc9/d47/dea/c104 0 2024-03-29T04:45:11.338 INFO:tasks.workunit.client.1.smithi152.stdout:1/288: dread d5/d7/d8/f1e [0,4194304] 0 2024-03-29T04:45:11.338 INFO:tasks.workunit.client.0.smithi142.stdout:6/820: rmdir d1c/d33/d46/d61/d7e 39 2024-03-29T04:45:11.339 INFO:tasks.workunit.client.0.smithi142.stdout:6/821: chown l15 893100694 1 2024-03-29T04:45:11.339 INFO:tasks.workunit.client.1.smithi152.stdout:1/289: dread - d5/d7/d1b/f47 zero size 2024-03-29T04:45:11.340 INFO:tasks.workunit.client.1.smithi152.stdout:1/290: chown d5/d7/d1b/d2e/l53 1851865150 1 2024-03-29T04:45:11.341 INFO:tasks.workunit.client.0.smithi142.stdout:1/780: mkdir d8/db/dc/d109 0 2024-03-29T04:45:11.342 INFO:tasks.workunit.client.0.smithi142.stdout:6/822: dread d1c/d33/f5a [0,4194304] 0 2024-03-29T04:45:11.342 INFO:tasks.workunit.client.1.smithi152.stdout:1/291: creat d5/d7/d1b/d2e/f5f x:0 0 0 2024-03-29T04:45:11.346 INFO:tasks.workunit.client.0.smithi142.stdout:4/803: link d0/d9/d11/d4c/d64/d94/ddd/ff4 d0/dc/d15/d3d/d65/f105 0 2024-03-29T04:45:11.349 INFO:tasks.workunit.client.1.smithi152.stdout:1/292: truncate d5/d7/d8/d13/f35 6547068 0 2024-03-29T04:45:11.351 INFO:tasks.workunit.client.0.smithi142.stdout:1/781: rename d8/db/dc/d77/d31/d4d/d8c/df9 to d8/db/dc/d77/db9/dbe/d2d/d8b/dda/d10a 0 2024-03-29T04:45:11.352 INFO:tasks.workunit.client.1.smithi152.stdout:1/293: link d5/d7/d9/c43 d5/d7/d9/d2b/c60 0 2024-03-29T04:45:11.352 INFO:tasks.workunit.client.1.smithi152.stdout:1/294: dread - d5/d7/d1b/f47 zero size 2024-03-29T04:45:11.354 INFO:tasks.workunit.client.1.smithi152.stdout:1/295: creat d5/d7/d9/d59/f61 x:0 0 0 2024-03-29T04:45:11.356 INFO:tasks.workunit.client.1.smithi152.stdout:1/296: fdatasync d5/f3c 0 2024-03-29T04:45:11.358 INFO:tasks.workunit.client.0.smithi142.stdout:1/782: dwrite d8/db/dc/d77/d31/d4d/f7a [0,4194304] 0 2024-03-29T04:45:11.358 INFO:tasks.workunit.client.1.smithi152.stdout:1/297: dread - d5/d7/d8/d13/f45 zero size 2024-03-29T04:45:11.358 INFO:tasks.workunit.client.1.smithi152.stdout:1/298: stat d5/d7/d9/f46 0 2024-03-29T04:45:11.359 INFO:tasks.workunit.client.0.smithi142.stdout:6/823: dwrite d1c/f23 [0,4194304] 0 2024-03-29T04:45:11.361 INFO:tasks.workunit.client.1.smithi152.stdout:1/299: write d5/d7/d8/f1e [3031664,37071] 0 2024-03-29T04:45:11.364 INFO:tasks.workunit.client.0.smithi142.stdout:6/824: unlink d1c/f5d 0 2024-03-29T04:45:11.367 INFO:tasks.workunit.client.0.smithi142.stdout:6/825: write d1c/d45/fa5 [573874,45120] 0 2024-03-29T04:45:11.368 INFO:tasks.workunit.client.0.smithi142.stdout:1/783: write d8/db/dc/d77/db9/dbe/d1f/f6d [1165593,117661] 0 2024-03-29T04:45:11.370 INFO:tasks.workunit.client.1.smithi152.stdout:1/300: rename d5/d7/d8/d13/d19/d21/f56 to d5/d7/d8/d13/f62 0 2024-03-29T04:45:11.372 INFO:tasks.workunit.client.1.smithi152.stdout:1/301: rmdir d5/d7/d38 39 2024-03-29T04:45:11.373 INFO:tasks.workunit.client.0.smithi142.stdout:1/784: mknod d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/ded/c10b 0 2024-03-29T04:45:11.374 INFO:tasks.workunit.client.1.smithi152.stdout:1/302: creat d5/d7/d9/d2b/f63 x:0 0 0 2024-03-29T04:45:11.375 INFO:tasks.workunit.client.1.smithi152.stdout:1/303: write d5/d7/d9/f4a [414930,53856] 0 2024-03-29T04:45:11.376 INFO:tasks.workunit.client.0.smithi142.stdout:1/785: write d8/db/dc/d77/d31/d4d/d56/faf [2415967,34796] 0 2024-03-29T04:45:11.379 INFO:tasks.workunit.client.0.smithi142.stdout:1/786: truncate d8/db/dc/d77/db9/dbe/d1f/d4c/fb1 806595 0 2024-03-29T04:45:11.438 INFO:tasks.workunit.client.1.smithi152.stdout:7/198: sync 2024-03-29T04:45:11.438 INFO:tasks.workunit.client.1.smithi152.stdout:0/254: sync 2024-03-29T04:45:11.438 INFO:tasks.workunit.client.1.smithi152.stdout:9/288: sync 2024-03-29T04:45:11.438 INFO:tasks.workunit.client.1.smithi152.stdout:3/231: sync 2024-03-29T04:45:11.438 INFO:tasks.workunit.client.1.smithi152.stdout:5/240: sync 2024-03-29T04:45:11.438 INFO:tasks.workunit.client.1.smithi152.stdout:8/183: sync 2024-03-29T04:45:11.438 INFO:tasks.workunit.client.1.smithi152.stdout:4/227: sync 2024-03-29T04:45:11.439 INFO:tasks.workunit.client.1.smithi152.stdout:6/287: sync 2024-03-29T04:45:11.439 INFO:tasks.workunit.client.0.smithi142.stdout:3/683: sync 2024-03-29T04:45:11.439 INFO:tasks.workunit.client.0.smithi142.stdout:8/943: sync 2024-03-29T04:45:11.439 INFO:tasks.workunit.client.1.smithi152.stdout:8/184: truncate d1/f26 353123 0 2024-03-29T04:45:11.440 INFO:tasks.workunit.client.1.smithi152.stdout:7/199: truncate d7/d8/d12/d1d/f1e 347198 0 2024-03-29T04:45:11.445 INFO:tasks.workunit.client.1.smithi152.stdout:7/200: dwrite d7/d8/db/df/f17 [4194304,4194304] 0 2024-03-29T04:45:11.451 INFO:tasks.workunit.client.0.smithi142.stdout:3/684: dread d3/d29/d2b/d40/d67/d8b/d6/f68 [0,4194304] 0 2024-03-29T04:45:11.452 INFO:tasks.workunit.client.0.smithi142.stdout:3/685: write d3/fd0 [136889,127561] 0 2024-03-29T04:45:11.454 INFO:tasks.workunit.client.0.smithi142.stdout:8/944: dread d5/fc5 [0,4194304] 0 2024-03-29T04:45:11.455 INFO:tasks.workunit.client.0.smithi142.stdout:3/686: creat d3/d29/d2b/d40/fea x:0 0 0 2024-03-29T04:45:11.457 INFO:tasks.workunit.client.0.smithi142.stdout:8/945: creat d5/d1d/d81/d13d/f145 x:0 0 0 2024-03-29T04:45:11.457 INFO:tasks.workunit.client.0.smithi142.stdout:8/946: dread - d5/d1d/d2f/d76/d3c/d70/de9/f126 zero size 2024-03-29T04:45:11.459 INFO:tasks.workunit.client.0.smithi142.stdout:8/947: mkdir d5/d1d/d81/d146 0 2024-03-29T04:45:11.459 INFO:tasks.workunit.client.0.smithi142.stdout:8/948: readlink d5/d1d/dd9/l139 0 2024-03-29T04:45:11.461 INFO:tasks.workunit.client.0.smithi142.stdout:8/949: truncate d5/d1d/d2f/fe1 949848 0 2024-03-29T04:45:11.467 INFO:tasks.workunit.client.0.smithi142.stdout:8/950: truncate d5/d1d/d2f/d76/d3c/d70/de9/f142 3348299 0 2024-03-29T04:45:11.471 INFO:tasks.workunit.client.0.smithi142.stdout:8/951: dread d5/d1d/d24/f2b [0,4194304] 0 2024-03-29T04:45:11.472 INFO:tasks.workunit.client.0.smithi142.stdout:8/952: chown d5/d2e/cbf 14 1 2024-03-29T04:45:11.478 INFO:tasks.workunit.client.0.smithi142.stdout:8/953: dwrite d5/d1d/d2f/d45/f129 [0,4194304] 0 2024-03-29T04:45:11.478 INFO:tasks.workunit.client.0.smithi142.stdout:8/954: chown d5/d1d/d2f/d76/d9c 206463018 1 2024-03-29T04:45:11.480 INFO:tasks.workunit.client.0.smithi142.stdout:8/955: creat d5/d2e/dd2/d60/f147 x:0 0 0 2024-03-29T04:45:11.487 INFO:tasks.workunit.client.0.smithi142.stdout:8/956: dwrite d5/d1d/d81/fa4 [0,4194304] 0 2024-03-29T04:45:11.490 INFO:tasks.workunit.client.0.smithi142.stdout:8/957: mknod d5/d2e/dd2/df6/c148 0 2024-03-29T04:45:11.492 INFO:tasks.workunit.client.0.smithi142.stdout:8/958: truncate d5/d1d/d2f/d76/d3c/f112 419517 0 2024-03-29T04:45:11.497 INFO:tasks.workunit.client.0.smithi142.stdout:8/959: getdents d5/d1d/d2f/d76/d2d/d13a 0 2024-03-29T04:45:11.499 INFO:tasks.workunit.client.0.smithi142.stdout:8/960: mknod d5/d2e/c149 0 2024-03-29T04:45:11.499 INFO:tasks.workunit.client.0.smithi142.stdout:8/961: chown d5/d2e 3 1 2024-03-29T04:45:11.500 INFO:tasks.workunit.client.0.smithi142.stdout:8/962: dread - d5/d1d/d2f/d76/d3c/d70/f137 zero size 2024-03-29T04:45:11.502 INFO:tasks.workunit.client.0.smithi142.stdout:8/963: mkdir d5/d1d/d2f/d5f/d14a 0 2024-03-29T04:45:11.505 INFO:tasks.workunit.client.0.smithi142.stdout:8/964: link d5/d1d/d81/lee d5/d1d/d2f/d76/d2d/d13a/l14b 0 2024-03-29T04:45:11.511 INFO:tasks.workunit.client.0.smithi142.stdout:8/965: truncate d5/d1d/d2f/fc8 99732 0 2024-03-29T04:45:11.512 INFO:tasks.workunit.client.0.smithi142.stdout:8/966: write d5/d1d/d2f/d45/f129 [3014810,34720] 0 2024-03-29T04:45:11.514 INFO:tasks.workunit.client.0.smithi142.stdout:8/967: mkdir d5/d1d/d24/db4/d14c 0 2024-03-29T04:45:11.527 INFO:tasks.workunit.client.0.smithi142.stdout:8/968: dread d5/d2e/dd2/d60/fd6 [0,4194304] 0 2024-03-29T04:45:11.529 INFO:tasks.workunit.client.0.smithi142.stdout:8/969: truncate d5/d1d/d2f/f50 4189689 0 2024-03-29T04:45:11.530 INFO:tasks.workunit.client.0.smithi142.stdout:8/970: truncate d5/d2e/dd2/d60/d121/f124 938244 0 2024-03-29T04:45:11.535 INFO:tasks.workunit.client.1.smithi152.stdout:3/232: creat d7/d19/d1b/d1e/f40 x:0 0 0 2024-03-29T04:45:11.535 INFO:tasks.workunit.client.0.smithi142.stdout:8/971: dwrite d5/d1d/d2f/d76/f106 [0,4194304] 0 2024-03-29T04:45:11.535 INFO:tasks.workunit.client.1.smithi152.stdout:1/304: symlink d5/d7/d8/l64 0 2024-03-29T04:45:11.537 INFO:tasks.workunit.client.1.smithi152.stdout:0/255: dread - d2/da/d17/f4c zero size 2024-03-29T04:45:11.539 INFO:tasks.workunit.client.1.smithi152.stdout:8/185: mkdir d1/d2/d3c 0 2024-03-29T04:45:11.541 INFO:tasks.workunit.client.1.smithi152.stdout:3/233: mkdir d7/d1c/d41 0 2024-03-29T04:45:11.542 INFO:tasks.workunit.client.0.smithi142.stdout:8/972: dwrite d5/d1d/d24/d66/dce/f105 [0,4194304] 0 2024-03-29T04:45:11.546 INFO:tasks.workunit.client.1.smithi152.stdout:4/228: rename d0/d7/d25/d17/c2e to d0/c3d 0 2024-03-29T04:45:11.548 INFO:tasks.workunit.client.0.smithi142.stdout:8/973: creat d5/d1d/d24/db4/d14c/f14d x:0 0 0 2024-03-29T04:45:11.548 INFO:tasks.workunit.client.1.smithi152.stdout:1/305: unlink d5/d7/d1b/d2e/c40 0 2024-03-29T04:45:11.551 INFO:tasks.workunit.client.1.smithi152.stdout:1/306: write d5/d7/d9/d2b/f63 [662472,74828] 0 2024-03-29T04:45:11.551 INFO:tasks.workunit.client.1.smithi152.stdout:3/234: write d7/d19/f25 [1144576,86152] 0 2024-03-29T04:45:11.552 INFO:tasks.workunit.client.1.smithi152.stdout:6/288: rename d1/d3/l4c to d1/d3/d9/d13/d28/d45/l67 0 2024-03-29T04:45:11.553 INFO:tasks.workunit.client.1.smithi152.stdout:7/201: truncate d7/f14 1850583 0 2024-03-29T04:45:11.554 INFO:tasks.workunit.client.1.smithi152.stdout:6/289: stat d1/d3/d1a/d1b 0 2024-03-29T04:45:11.555 INFO:tasks.workunit.client.1.smithi152.stdout:1/307: fsync d5/f3f 0 2024-03-29T04:45:11.556 INFO:tasks.workunit.client.1.smithi152.stdout:5/241: dwrite d6/dd/f17 [0,4194304] 0 2024-03-29T04:45:11.557 INFO:tasks.workunit.client.1.smithi152.stdout:3/235: write d7/d19/d1b/f3d [495971,15401] 0 2024-03-29T04:45:11.557 INFO:tasks.workunit.client.1.smithi152.stdout:0/256: rename d2/da/d17/l13 to d2/d29/l60 0 2024-03-29T04:45:11.558 INFO:tasks.workunit.client.1.smithi152.stdout:7/202: fdatasync d7/d8/db/df/d21/f2c 0 2024-03-29T04:45:11.559 INFO:tasks.workunit.client.1.smithi152.stdout:4/229: mknod d0/d7/de/d3a/c3e 0 2024-03-29T04:45:11.559 INFO:tasks.workunit.client.1.smithi152.stdout:8/186: creat d1/d2/d1d/f3d x:0 0 0 2024-03-29T04:45:11.559 INFO:tasks.workunit.client.1.smithi152.stdout:6/290: rmdir d1/d3/d9 39 2024-03-29T04:45:11.559 INFO:tasks.workunit.client.1.smithi152.stdout:5/242: fsync d6/dd/f35 0 2024-03-29T04:45:11.560 INFO:tasks.workunit.client.1.smithi152.stdout:1/308: write d5/d7/d8/f1e [859330,119720] 0 2024-03-29T04:45:11.561 INFO:tasks.workunit.client.1.smithi152.stdout:1/309: chown d5 584 1 2024-03-29T04:45:11.562 INFO:tasks.workunit.client.1.smithi152.stdout:3/236: write d7/d19/d1b/d1e/d22/f2d [1272387,20501] 0 2024-03-29T04:45:11.562 INFO:tasks.workunit.client.1.smithi152.stdout:7/203: rename d7/d8 to d7/d8/d12/d1d/d39 22 2024-03-29T04:45:11.562 INFO:tasks.workunit.client.1.smithi152.stdout:8/187: write d1/d2/d1d/f35 [107312,48416] 0 2024-03-29T04:45:11.563 INFO:tasks.workunit.client.1.smithi152.stdout:5/243: truncate d6/f26 605161 0 2024-03-29T04:45:11.563 INFO:tasks.workunit.client.1.smithi152.stdout:0/257: mkdir d2/da/d3a/d61 0 2024-03-29T04:45:11.565 INFO:tasks.workunit.client.0.smithi142.stdout:7/903: sync 2024-03-29T04:45:11.565 INFO:tasks.workunit.client.1.smithi152.stdout:1/310: write d5/f3c [5506917,6837] 0 2024-03-29T04:45:11.568 INFO:tasks.workunit.client.1.smithi152.stdout:8/188: mknod d1/d21/c3e 0 2024-03-29T04:45:11.569 INFO:tasks.workunit.client.1.smithi152.stdout:3/237: mknod d7/d19/c42 0 2024-03-29T04:45:11.569 INFO:tasks.workunit.client.1.smithi152.stdout:8/189: chown d1/l2b 5 1 2024-03-29T04:45:11.570 INFO:tasks.workunit.client.0.smithi142.stdout:8/974: dwrite d5/d93/fb2 [0,4194304] 0 2024-03-29T04:45:11.570 INFO:tasks.workunit.client.1.smithi152.stdout:4/230: rename d0/d7/d25/f1a to d0/d7/d25/f3f 0 2024-03-29T04:45:11.571 INFO:tasks.workunit.client.0.smithi142.stdout:8/975: write d5/d1d/d2f/d76/f106 [4998253,2834] 0 2024-03-29T04:45:11.571 INFO:tasks.workunit.client.0.smithi142.stdout:7/904: rename d7/d27/d44/db2/db8/d4b/fd0 to d7/d27/d3a/f12c 0 2024-03-29T04:45:11.572 INFO:tasks.workunit.client.1.smithi152.stdout:4/231: fsync d0/f4 0 2024-03-29T04:45:11.572 INFO:tasks.workunit.client.1.smithi152.stdout:5/244: rename d6/dd to d6/d1d/d3b/d55 0 2024-03-29T04:45:11.572 INFO:tasks.workunit.client.1.smithi152.stdout:8/190: write d1/d2/d3/f29 [163624,15846] 0 2024-03-29T04:45:11.574 INFO:tasks.workunit.client.0.smithi142.stdout:8/976: mkdir d5/d1d/dd9/d14e 0 2024-03-29T04:45:11.578 INFO:tasks.workunit.client.1.smithi152.stdout:3/238: chown d7/c35 102652519 1 2024-03-29T04:45:11.579 INFO:tasks.workunit.client.1.smithi152.stdout:0/258: getdents d2/da/d17/d57 0 2024-03-29T04:45:11.580 INFO:tasks.workunit.client.0.smithi142.stdout:8/977: truncate d5/d2e/f65 491982 0 2024-03-29T04:45:11.581 INFO:tasks.workunit.client.0.smithi142.stdout:7/905: dwrite d7/d27/d44/db2/db8/d4b/d71/dec/f117 [0,4194304] 0 2024-03-29T04:45:11.581 INFO:tasks.workunit.client.1.smithi152.stdout:1/311: dwrite d5/d7/d8/f1e [0,4194304] 0 2024-03-29T04:45:11.581 INFO:tasks.workunit.client.1.smithi152.stdout:5/245: mkdir d6/d16/d47/d56 0 2024-03-29T04:45:11.582 INFO:tasks.workunit.client.0.smithi142.stdout:8/978: stat d5/d1d/d2f/d5f/d48/l10f 0 2024-03-29T04:45:11.585 INFO:tasks.workunit.client.1.smithi152.stdout:0/259: dread - d2/da/df/f5e zero size 2024-03-29T04:45:11.588 INFO:tasks.workunit.client.1.smithi152.stdout:0/260: chown d2/da/df/c33 3849275 1 2024-03-29T04:45:11.589 INFO:tasks.workunit.client.1.smithi152.stdout:1/312: chown d5/c4b 241903 1 2024-03-29T04:45:11.589 INFO:tasks.workunit.client.1.smithi152.stdout:8/191: dwrite d1/d2/d3/f29 [0,4194304] 0 2024-03-29T04:45:11.591 INFO:tasks.workunit.client.0.smithi142.stdout:8/979: dwrite d5/d1d/d2f/fd0 [0,4194304] 0 2024-03-29T04:45:11.592 INFO:tasks.workunit.client.1.smithi152.stdout:1/313: dread - d5/d7/d9/d59/f61 zero size 2024-03-29T04:45:11.594 INFO:tasks.workunit.client.0.smithi142.stdout:8/980: write d5/d2e/dd2/d60/f147 [6810,122231] 0 2024-03-29T04:45:11.598 INFO:tasks.workunit.client.1.smithi152.stdout:5/246: rename d6/lb to d6/d16/d25/l57 0 2024-03-29T04:45:11.599 INFO:tasks.workunit.client.1.smithi152.stdout:5/247: read - d6/d16/f4e zero size 2024-03-29T04:45:11.600 INFO:tasks.workunit.client.1.smithi152.stdout:5/248: dread - d6/d1d/d3b/d55/f35 zero size 2024-03-29T04:45:11.602 INFO:tasks.workunit.client.1.smithi152.stdout:4/232: rename d0/d7/de to d0/d7/d40 0 2024-03-29T04:45:11.604 INFO:tasks.workunit.client.1.smithi152.stdout:6/291: dread d1/f8 [0,4194304] 0 2024-03-29T04:45:11.605 INFO:tasks.workunit.client.1.smithi152.stdout:7/204: truncate d7/d8/d12/d35/f2a 655990 0 2024-03-29T04:45:11.607 INFO:tasks.workunit.client.1.smithi152.stdout:1/314: unlink d5/f20 0 2024-03-29T04:45:11.608 INFO:tasks.workunit.client.1.smithi152.stdout:0/261: getdents d2/da/d17/d57 0 2024-03-29T04:45:11.610 INFO:tasks.workunit.client.1.smithi152.stdout:4/233: symlink d0/d7/d40/d3a/l41 0 2024-03-29T04:45:11.613 INFO:tasks.workunit.client.1.smithi152.stdout:7/205: creat d7/d8/db/df/f3a x:0 0 0 2024-03-29T04:45:11.613 INFO:tasks.workunit.client.1.smithi152.stdout:4/234: dread d0/d7/d40/d23/f28 [0,4194304] 0 2024-03-29T04:45:11.614 INFO:tasks.workunit.client.1.smithi152.stdout:0/262: creat d2/da/d17/d57/f62 x:0 0 0 2024-03-29T04:45:11.614 INFO:tasks.workunit.client.1.smithi152.stdout:2/203: sync 2024-03-29T04:45:11.614 INFO:tasks.workunit.client.1.smithi152.stdout:6/292: mknod d1/d3/c68 0 2024-03-29T04:45:11.617 INFO:tasks.workunit.client.1.smithi152.stdout:7/206: creat d7/d8/f3b x:0 0 0 2024-03-29T04:45:11.617 INFO:tasks.workunit.client.1.smithi152.stdout:0/263: readlink d2/da/d17/d50/l58 0 2024-03-29T04:45:11.618 INFO:tasks.workunit.client.0.smithi142.stdout:7/906: dread d7/d27/d44/db2/db8/d4b/d71/f2c [0,4194304] 0 2024-03-29T04:45:11.619 INFO:tasks.workunit.client.1.smithi152.stdout:0/264: read d2/d31/f3c [3667764,123597] 0 2024-03-29T04:45:11.619 INFO:tasks.workunit.client.1.smithi152.stdout:8/192: rename d1/d21/c3b to d1/d2/c3f 0 2024-03-29T04:45:11.620 INFO:tasks.workunit.client.1.smithi152.stdout:2/204: dread d0/d3/df/f13 [0,4194304] 0 2024-03-29T04:45:11.621 INFO:tasks.workunit.client.1.smithi152.stdout:8/193: write d1/d2/d1d/f23 [1357334,129844] 0 2024-03-29T04:45:11.622 INFO:tasks.workunit.client.1.smithi152.stdout:7/207: fdatasync d7/d8/d12/d35/f1a 0 2024-03-29T04:45:11.624 INFO:tasks.workunit.client.1.smithi152.stdout:7/208: dread - d7/d8/d12/d35/f32 zero size 2024-03-29T04:45:11.625 INFO:tasks.workunit.client.1.smithi152.stdout:3/239: dread d7/d19/d1b/f21 [0,4194304] 0 2024-03-29T04:45:11.628 INFO:tasks.workunit.client.1.smithi152.stdout:6/293: creat d1/d3/d9/d3b/f69 x:0 0 0 2024-03-29T04:45:11.628 INFO:tasks.workunit.client.1.smithi152.stdout:8/194: mkdir d1/d21/d40 0 2024-03-29T04:45:11.629 INFO:tasks.workunit.client.0.smithi142.stdout:7/907: write d7/d27/d44/d4c/f80 [1691232,127042] 0 2024-03-29T04:45:11.630 INFO:tasks.workunit.client.1.smithi152.stdout:5/249: stat d6/d16/d25/l57 0 2024-03-29T04:45:11.631 INFO:tasks.workunit.client.1.smithi152.stdout:0/265: rmdir d2/da/d17/d57 39 2024-03-29T04:45:11.632 INFO:tasks.workunit.client.0.smithi142.stdout:7/908: creat d7/d27/d3a/d92/d97/f12d x:0 0 0 2024-03-29T04:45:11.632 INFO:tasks.workunit.client.1.smithi152.stdout:3/240: dread d7/d19/d1b/d1e/d22/f2d [0,4194304] 0 2024-03-29T04:45:11.633 INFO:tasks.workunit.client.1.smithi152.stdout:3/241: read - d7/d19/f3c zero size 2024-03-29T04:45:11.634 INFO:tasks.workunit.client.0.smithi142.stdout:7/909: mkdir d7/d27/d44/d12e 0 2024-03-29T04:45:11.635 INFO:tasks.workunit.client.1.smithi152.stdout:4/235: dread d0/d7/d40/ff [0,4194304] 0 2024-03-29T04:45:11.637 INFO:tasks.workunit.client.1.smithi152.stdout:6/294: chown d1/d3/d9/d13/d28/d45/l67 0 1 2024-03-29T04:45:11.638 INFO:tasks.workunit.client.1.smithi152.stdout:5/250: chown d6/d1d/d30/c3d 28 1 2024-03-29T04:45:11.638 INFO:tasks.workunit.client.0.smithi142.stdout:7/910: getdents d7/d27/d44/db2/dd2 0 2024-03-29T04:45:11.639 INFO:tasks.workunit.client.1.smithi152.stdout:0/266: dwrite d2/f1f [0,4194304] 0 2024-03-29T04:45:11.639 INFO:tasks.workunit.client.1.smithi152.stdout:1/315: rename d5/d7/f5a to d5/d7/d8/d13/d19/f65 0 2024-03-29T04:45:11.640 INFO:tasks.workunit.client.1.smithi152.stdout:5/251: dread d6/d16/d25/d2e/f44 [0,4194304] 0 2024-03-29T04:45:11.641 INFO:tasks.workunit.client.0.smithi142.stdout:7/911: dread - d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d8d/fa7 zero size 2024-03-29T04:45:11.641 INFO:tasks.workunit.client.1.smithi152.stdout:5/252: truncate d6/d1d/d30/f38 658579 0 2024-03-29T04:45:11.642 INFO:tasks.workunit.client.0.smithi142.stdout:7/912: chown d7/d27/d44/db2/db8/d4b/d71/dec/lbf 964770 1 2024-03-29T04:45:11.642 INFO:tasks.workunit.client.1.smithi152.stdout:2/205: truncate d0/d3/f5 1586017 0 2024-03-29T04:45:11.644 INFO:tasks.workunit.client.1.smithi152.stdout:4/236: creat d0/d7/d40/d2a/f42 x:0 0 0 2024-03-29T04:45:11.647 INFO:tasks.workunit.client.1.smithi152.stdout:5/253: mknod d6/d16/d25/c58 0 2024-03-29T04:45:11.647 INFO:tasks.workunit.client.1.smithi152.stdout:6/295: unlink d1/f43 0 2024-03-29T04:45:11.648 INFO:tasks.workunit.client.1.smithi152.stdout:3/242: mknod d7/d1c/d41/c43 0 2024-03-29T04:45:11.649 INFO:tasks.workunit.client.1.smithi152.stdout:6/296: readlink d1/d3/d9/d13/l60 0 2024-03-29T04:45:11.649 INFO:tasks.workunit.client.1.smithi152.stdout:8/195: creat d1/d2/d1d/f41 x:0 0 0 2024-03-29T04:45:11.650 INFO:tasks.workunit.client.1.smithi152.stdout:8/196: fdatasync d1/d2/d1d/f3a 0 2024-03-29T04:45:11.651 INFO:tasks.workunit.client.1.smithi152.stdout:6/297: read d1/d3/d1a/d1b/d29/f5f [40876,18223] 0 2024-03-29T04:45:11.652 INFO:tasks.workunit.client.1.smithi152.stdout:1/316: symlink d5/d7/d8/d13/d19/l66 0 2024-03-29T04:45:11.653 INFO:tasks.workunit.client.1.smithi152.stdout:6/298: write d1/d3/d9/d3b/f63 [679124,98034] 0 2024-03-29T04:45:11.653 INFO:tasks.workunit.client.1.smithi152.stdout:5/254: dread d6/d1d/d30/f46 [0,4194304] 0 2024-03-29T04:45:11.654 INFO:tasks.workunit.client.0.smithi142.stdout:7/913: dwrite d7/d27/d44/db2/db8/d4b/d71/d30/f4a [4194304,4194304] 0 2024-03-29T04:45:11.656 INFO:tasks.workunit.client.1.smithi152.stdout:7/209: truncate d7/d8/db/df/f13 828403 0 2024-03-29T04:45:11.658 INFO:tasks.workunit.client.0.smithi142.stdout:7/914: symlink d7/d27/d3a/d92/l12f 0 2024-03-29T04:45:11.658 INFO:tasks.workunit.client.1.smithi152.stdout:4/237: dwrite d0/d27/f34 [0,4194304] 0 2024-03-29T04:45:11.659 INFO:tasks.workunit.client.0.smithi142.stdout:7/915: chown d7/d27/fa8 3 1 2024-03-29T04:45:11.659 INFO:tasks.workunit.client.1.smithi152.stdout:2/206: rename d0/d3/d30/d15/d22/f38 to d0/d3/df/f43 0 2024-03-29T04:45:11.659 INFO:tasks.workunit.client.1.smithi152.stdout:4/238: chown d0/d7/d40 3 1 2024-03-29T04:45:11.660 INFO:tasks.workunit.client.1.smithi152.stdout:4/239: fsync d0/d7/d40/d23/f28 0 2024-03-29T04:45:11.661 INFO:tasks.workunit.client.0.smithi142.stdout:7/916: chown d7/d27/d44/l58 4836496 1 2024-03-29T04:45:11.662 INFO:tasks.workunit.client.0.smithi142.stdout:7/917: stat d7/d27/d44/db2/db8/la4 0 2024-03-29T04:45:11.663 INFO:tasks.workunit.client.1.smithi152.stdout:3/243: symlink d7/d19/d1b/l44 0 2024-03-29T04:45:11.663 INFO:tasks.workunit.client.1.smithi152.stdout:8/197: mkdir d1/d2/d1d/d42 0 2024-03-29T04:45:11.665 INFO:tasks.workunit.client.1.smithi152.stdout:6/299: mknod d1/d3/d1a/d1b/d29/c6a 0 2024-03-29T04:45:11.667 INFO:tasks.workunit.client.1.smithi152.stdout:6/300: write d1/d3/d9/f3a [837484,113666] 0 2024-03-29T04:45:11.667 INFO:tasks.workunit.client.1.smithi152.stdout:7/210: creat d7/d8/d12/d1d/d30/f3c x:0 0 0 2024-03-29T04:45:11.669 INFO:tasks.workunit.client.0.smithi142.stdout:7/918: rename d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/d86/dd4/l120 to d7/l130 0 2024-03-29T04:45:11.670 INFO:tasks.workunit.client.1.smithi152.stdout:5/255: rename d6/d1d/d3b/d55/f27 to d6/d16/d47/f59 0 2024-03-29T04:45:11.671 INFO:tasks.workunit.client.1.smithi152.stdout:3/244: rmdir d7/d19/d1b/d1e/d22 39 2024-03-29T04:45:11.672 INFO:tasks.workunit.client.1.smithi152.stdout:0/267: dwrite d2/d29/f44 [0,4194304] 0 2024-03-29T04:45:11.672 INFO:tasks.workunit.client.1.smithi152.stdout:8/198: readlink d1/d2/d3/l11 0 2024-03-29T04:45:11.673 INFO:tasks.workunit.client.0.smithi142.stdout:7/919: creat d7/d27/d44/db2/db8/f131 x:0 0 0 2024-03-29T04:45:11.674 INFO:tasks.workunit.client.1.smithi152.stdout:3/245: fdatasync d7/d19/d1b/f3d 0 2024-03-29T04:45:11.677 INFO:tasks.workunit.client.1.smithi152.stdout:3/246: chown d7/d1c/d41/c43 242 1 2024-03-29T04:45:11.678 INFO:tasks.workunit.client.1.smithi152.stdout:7/211: dwrite d7/d8/d12/d1d/f34 [0,4194304] 0 2024-03-29T04:45:11.681 INFO:tasks.workunit.client.1.smithi152.stdout:7/212: readlink d7/d8/lc 0 2024-03-29T04:45:11.682 INFO:tasks.workunit.client.1.smithi152.stdout:5/256: creat d6/d1b/f5a x:0 0 0 2024-03-29T04:45:11.682 INFO:tasks.workunit.client.0.smithi142.stdout:7/920: dread d7/d27/d44/db2/db8/d4b/f4f [0,4194304] 0 2024-03-29T04:45:11.682 INFO:tasks.workunit.client.0.smithi142.stdout:2/837: sync 2024-03-29T04:45:11.682 INFO:tasks.workunit.client.0.smithi142.stdout:0/850: sync 2024-03-29T04:45:11.683 INFO:tasks.workunit.client.0.smithi142.stdout:7/921: chown d7/d27/d3a/d92/f122 42137975 1 2024-03-29T04:45:11.685 INFO:tasks.workunit.client.1.smithi152.stdout:5/257: chown d6/d1d/d30/f46 420817 1 2024-03-29T04:45:11.686 INFO:tasks.workunit.client.0.smithi142.stdout:7/922: fsync d7/d27/d44/db2/db8/d89/fdc 0 2024-03-29T04:45:11.687 INFO:tasks.workunit.client.1.smithi152.stdout:5/258: truncate d6/d1b/f5a 869641 0 2024-03-29T04:45:11.689 INFO:tasks.workunit.client.1.smithi152.stdout:0/268: creat d2/da/df/f63 x:0 0 0 2024-03-29T04:45:11.691 INFO:tasks.workunit.client.0.smithi142.stdout:0/851: dwrite d7/df6/f109 [0,4194304] 0 2024-03-29T04:45:11.691 INFO:tasks.workunit.client.1.smithi152.stdout:2/207: creat d0/f44 x:0 0 0 2024-03-29T04:45:11.692 INFO:tasks.workunit.client.1.smithi152.stdout:2/208: write d0/d3/d30/f33 [897848,37722] 0 2024-03-29T04:45:11.693 INFO:tasks.workunit.client.0.smithi142.stdout:7/923: symlink d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/ddf/l132 0 2024-03-29T04:45:11.694 INFO:tasks.workunit.client.0.smithi142.stdout:2/838: chown de/d11/d17/d10b/d22/d27/dba/d61/d7d/db4/fc5 854 1 2024-03-29T04:45:11.696 INFO:tasks.workunit.client.1.smithi152.stdout:1/317: truncate d5/d7/d8/d13/d19/f30 1174177 0 2024-03-29T04:45:11.698 INFO:tasks.workunit.client.1.smithi152.stdout:7/213: rmdir d7/d8/d12/d1d 39 2024-03-29T04:45:11.700 INFO:tasks.workunit.client.1.smithi152.stdout:6/301: dread d1/d3/df/f2e [0,4194304] 0 2024-03-29T04:45:11.701 INFO:tasks.workunit.client.1.smithi152.stdout:6/302: chown d1/d3/df/f4f 7020563 1 2024-03-29T04:45:11.701 INFO:tasks.workunit.client.0.smithi142.stdout:2/839: dread de/d11/d2b/d7b/fa7 [0,4194304] 0 2024-03-29T04:45:11.703 INFO:tasks.workunit.client.0.smithi142.stdout:2/840: chown de/d11/d17/d10b/d22/dbf/l106 37 1 2024-03-29T04:45:11.704 INFO:tasks.workunit.client.1.smithi152.stdout:8/199: symlink d1/d2/d3c/l43 0 2024-03-29T04:45:11.705 INFO:tasks.workunit.client.0.smithi142.stdout:0/852: write d7/d9/d6e/f10a [1012331,128289] 0 2024-03-29T04:45:11.706 INFO:tasks.workunit.client.0.smithi142.stdout:0/853: chown d7/d13/dbf/d72/d37/le9 7693 1 2024-03-29T04:45:11.709 INFO:tasks.workunit.client.0.smithi142.stdout:0/854: rmdir d7/d13/dbf/dca/dab/ddf 39 2024-03-29T04:45:11.710 INFO:tasks.workunit.client.1.smithi152.stdout:1/318: chown d5/d7/d38/l58 11 1 2024-03-29T04:45:11.711 INFO:tasks.workunit.client.0.smithi142.stdout:0/855: chown d7/d13/dbf/d72/d75/c93 0 1 2024-03-29T04:45:11.713 INFO:tasks.workunit.client.1.smithi152.stdout:7/214: symlink d7/l3d 0 2024-03-29T04:45:11.714 INFO:tasks.workunit.client.0.smithi142.stdout:2/841: rmdir de/d11/d17/d10b/d22/d27 39 2024-03-29T04:45:11.714 INFO:tasks.workunit.client.0.smithi142.stdout:0/856: getdents d7/d9/d55/df7 0 2024-03-29T04:45:11.714 INFO:tasks.workunit.client.0.smithi142.stdout:2/842: readlink de/d11/d17/d10b/l107 0 2024-03-29T04:45:11.716 INFO:tasks.workunit.client.0.smithi142.stdout:4/804: sync 2024-03-29T04:45:11.716 INFO:tasks.workunit.client.0.smithi142.stdout:3/687: sync 2024-03-29T04:45:11.716 INFO:tasks.workunit.client.0.smithi142.stdout:9/710: sync 2024-03-29T04:45:11.716 INFO:tasks.workunit.client.0.smithi142.stdout:6/826: sync 2024-03-29T04:45:11.716 INFO:tasks.workunit.client.0.smithi142.stdout:8/981: sync 2024-03-29T04:45:11.716 INFO:tasks.workunit.client.0.smithi142.stdout:1/787: sync 2024-03-29T04:45:11.716 INFO:tasks.workunit.client.0.smithi142.stdout:0/857: write d7/d9/d1e/d81/fc2 [4207338,51387] 0 2024-03-29T04:45:11.717 INFO:tasks.workunit.client.1.smithi152.stdout:1/319: dwrite d5/f3e [0,4194304] 0 2024-03-29T04:45:11.717 INFO:tasks.workunit.client.1.smithi152.stdout:6/303: mknod d1/d3/d1a/d1b/d29/c6b 0 2024-03-29T04:45:11.719 INFO:tasks.workunit.client.0.smithi142.stdout:6/827: write d1c/d45/d8a/df6/ffe [431455,102782] 0 2024-03-29T04:45:11.719 INFO:tasks.workunit.client.1.smithi152.stdout:0/269: creat d2/da/d17/d57/f64 x:0 0 0 2024-03-29T04:45:11.720 INFO:tasks.workunit.client.1.smithi152.stdout:3/247: creat d7/d19/d1b/d1e/f45 x:0 0 0 2024-03-29T04:45:11.720 INFO:tasks.workunit.client.0.smithi142.stdout:1/788: chown d8/db/dc/d77/db9/dbd 259955 1 2024-03-29T04:45:11.721 INFO:tasks.workunit.client.1.smithi152.stdout:3/248: chown d7/d1c/d41/c43 5903539 1 2024-03-29T04:45:11.722 INFO:tasks.workunit.client.1.smithi152.stdout:3/249: stat d7/d1c/d41/c43 0 2024-03-29T04:45:11.724 INFO:tasks.workunit.client.1.smithi152.stdout:3/250: write d7/d19/f30 [5831549,80705] 0 2024-03-29T04:45:11.725 INFO:tasks.workunit.client.0.smithi142.stdout:7/924: dread d7/d27/d44/db2/db8/d4b/d71/d30/d33/f77 [0,4194304] 0 2024-03-29T04:45:11.726 INFO:tasks.workunit.client.0.smithi142.stdout:3/688: mknod d3/d29/d8f/d5f/d64/d6b/ceb 0 2024-03-29T04:45:11.726 INFO:tasks.workunit.client.0.smithi142.stdout:2/843: rename de/f8b to de/d11/d17/d10b/d22/dbf/f114 0 2024-03-29T04:45:11.726 INFO:tasks.workunit.client.0.smithi142.stdout:9/711: creat d0/d2/d61/d64/fe3 x:0 0 0 2024-03-29T04:45:11.727 INFO:tasks.workunit.client.1.smithi152.stdout:3/251: truncate d7/d19/f29 4976583 0 2024-03-29T04:45:11.729 INFO:tasks.workunit.client.0.smithi142.stdout:0/858: readlink d7/d13/dbf/dca/dab/dc1/dd3/de1/lf9 0 2024-03-29T04:45:11.729 INFO:tasks.workunit.client.0.smithi142.stdout:9/712: chown d0/l4e 32496 1 2024-03-29T04:45:11.729 INFO:tasks.workunit.client.1.smithi152.stdout:5/259: creat d6/d16/f5b x:0 0 0 2024-03-29T04:45:11.729 INFO:tasks.workunit.client.1.smithi152.stdout:7/215: unlink d7/d8/d12/l27 0 2024-03-29T04:45:11.731 INFO:tasks.workunit.client.1.smithi152.stdout:0/270: dwrite d2/da/df/f63 [0,4194304] 0 2024-03-29T04:45:11.732 INFO:tasks.workunit.client.0.smithi142.stdout:4/805: creat d0/d9/d11/d4c/d5f/d81/dc9/de5/f106 x:0 0 0 2024-03-29T04:45:11.735 INFO:tasks.workunit.client.0.smithi142.stdout:6/828: creat d1c/d33/d46/d61/d7e/f108 x:0 0 0 2024-03-29T04:45:11.736 INFO:tasks.workunit.client.0.smithi142.stdout:8/982: link d5/d1d/d2f/d45/f95 d5/d1d/dd9/d120/f14f 0 2024-03-29T04:45:11.737 INFO:tasks.workunit.client.1.smithi152.stdout:6/304: symlink d1/d3/l6c 0 2024-03-29T04:45:11.737 INFO:tasks.workunit.client.0.smithi142.stdout:9/713: dwrite d0/d2/d6/f9c [0,4194304] 0 2024-03-29T04:45:11.739 INFO:tasks.workunit.client.1.smithi152.stdout:6/305: fdatasync f0 0 2024-03-29T04:45:11.740 INFO:tasks.workunit.client.0.smithi142.stdout:1/789: creat d8/db/dc/d77/db9/dbe/d2d/d44/d7f/df4/dfb/f10c x:0 0 0 2024-03-29T04:45:11.740 INFO:tasks.workunit.client.0.smithi142.stdout:0/859: readlink d7/d9/d54/d58/lb2 0 2024-03-29T04:45:11.742 INFO:tasks.workunit.client.1.smithi152.stdout:6/306: readlink d1/l32 0 2024-03-29T04:45:11.743 INFO:tasks.workunit.client.0.smithi142.stdout:0/860: chown d7/d9/d54/fea 64476 1 2024-03-29T04:45:11.744 INFO:tasks.workunit.client.1.smithi152.stdout:3/252: dwrite d7/d1c/f2c [0,4194304] 0 2024-03-29T04:45:11.744 INFO:tasks.workunit.client.1.smithi152.stdout:7/216: creat d7/d8/d12/d15/f3e x:0 0 0 2024-03-29T04:45:11.744 INFO:tasks.workunit.client.1.smithi152.stdout:5/260: dread d6/d1d/d3b/d55/f11 [0,4194304] 0 2024-03-29T04:45:11.745 INFO:tasks.workunit.client.0.smithi142.stdout:4/806: symlink d0/d9/d11/d4c/d5f/d81/dc9/d47/d7a/de4/l107 0 2024-03-29T04:45:11.745 INFO:tasks.workunit.client.0.smithi142.stdout:1/790: write d8/db/dc/d77/d5a/fe6 [4839555,118252] 0 2024-03-29T04:45:11.745 INFO:tasks.workunit.client.1.smithi152.stdout:0/271: unlink d2/da/df/f63 0 2024-03-29T04:45:11.746 INFO:tasks.workunit.client.0.smithi142.stdout:9/714: dread d0/d12/d50/f93 [0,4194304] 0 2024-03-29T04:45:11.746 INFO:tasks.workunit.client.1.smithi152.stdout:8/200: fsync d1/d21/f25 0 2024-03-29T04:45:11.749 INFO:tasks.workunit.client.0.smithi142.stdout:8/983: dread - d5/d1d/d59/ffc zero size 2024-03-29T04:45:11.750 INFO:tasks.workunit.client.1.smithi152.stdout:0/272: chown d2/da/df/f5e 174 1 2024-03-29T04:45:11.750 INFO:tasks.workunit.client.1.smithi152.stdout:5/261: chown d6/d1d/d30/f46 243272561 1 2024-03-29T04:45:11.751 INFO:tasks.workunit.client.1.smithi152.stdout:2/209: dwrite d0/d3/f5 [0,4194304] 0 2024-03-29T04:45:11.752 INFO:tasks.workunit.client.0.smithi142.stdout:3/689: link d3/d29/d2d/f3f d3/d29/d2b/d40/fec 0 2024-03-29T04:45:11.753 INFO:tasks.workunit.client.1.smithi152.stdout:3/253: write d7/d19/f3c [579839,83699] 0 2024-03-29T04:45:11.755 INFO:tasks.workunit.client.1.smithi152.stdout:3/254: truncate d7/d19/f25 1746110 0 2024-03-29T04:45:11.755 INFO:tasks.workunit.client.1.smithi152.stdout:7/217: symlink d7/d8/l3f 0 2024-03-29T04:45:11.757 INFO:tasks.workunit.client.1.smithi152.stdout:8/201: creat d1/d2/d3/f44 x:0 0 0 2024-03-29T04:45:11.757 INFO:tasks.workunit.client.0.smithi142.stdout:4/807: symlink d0/d9/d11/d4c/d64/d94/ddd/l108 0 2024-03-29T04:45:11.758 INFO:tasks.workunit.client.0.smithi142.stdout:1/791: truncate d8/db/dc/d77/db9/dbe/d2d/d8b/dda/fde 167675 0 2024-03-29T04:45:11.758 INFO:tasks.workunit.client.1.smithi152.stdout:1/320: getdents d5/d7/d8/d13/d19/d21 0 2024-03-29T04:45:11.760 INFO:tasks.workunit.client.0.smithi142.stdout:6/829: dwrite d1c/d45/f4e [0,4194304] 0 2024-03-29T04:45:11.760 INFO:tasks.workunit.client.1.smithi152.stdout:8/202: fdatasync d1/d2/d1d/f3a 0 2024-03-29T04:45:11.760 INFO:tasks.workunit.client.1.smithi152.stdout:2/210: fdatasync d0/d3/df/f13 0 2024-03-29T04:45:11.764 INFO:tasks.workunit.client.0.smithi142.stdout:0/861: creat d7/d13/dbf/dca/dab/ddf/f115 x:0 0 0 2024-03-29T04:45:11.765 INFO:tasks.workunit.client.0.smithi142.stdout:0/862: readlink d7/d13/l51 0 2024-03-29T04:45:11.765 INFO:tasks.workunit.client.1.smithi152.stdout:7/218: write d7/d8/d12/d1d/f29 [2493073,116229] 0 2024-03-29T04:45:11.766 INFO:tasks.workunit.client.1.smithi152.stdout:0/273: dwrite d2/da/df/f5b [0,4194304] 0 2024-03-29T04:45:11.766 INFO:tasks.workunit.client.1.smithi152.stdout:3/255: mkdir d7/d46 0 2024-03-29T04:45:11.766 INFO:tasks.workunit.client.0.smithi142.stdout:8/984: symlink d5/d1d/dd9/d14e/l150 0 2024-03-29T04:45:11.767 INFO:tasks.workunit.client.0.smithi142.stdout:7/925: rename d7/d27/d44/db2/db8/d4b/d71/d30/d33/df5/c119 to d7/c133 0 2024-03-29T04:45:11.768 INFO:tasks.workunit.client.1.smithi152.stdout:8/203: symlink d1/d21/l45 0 2024-03-29T04:45:11.768 INFO:tasks.workunit.client.0.smithi142.stdout:2/844: link de/d11/d2b/d7b/cac de/d11/d17/d10b/d22/dbf/c115 0 2024-03-29T04:45:11.771 INFO:tasks.workunit.client.1.smithi152.stdout:1/321: dread d5/d7/d9/d2b/f37 [0,4194304] 0 2024-03-29T04:45:11.771 INFO:tasks.workunit.client.1.smithi152.stdout:2/211: mkdir d0/d3/df/d2c/d45 0 2024-03-29T04:45:11.772 INFO:tasks.workunit.client.0.smithi142.stdout:2/845: stat de/d11/d23/dd1/ff0 0 2024-03-29T04:45:11.772 INFO:tasks.workunit.client.0.smithi142.stdout:6/830: dwrite d1c/d45/d8a/df6/f87 [0,4194304] 0 2024-03-29T04:45:11.773 INFO:tasks.workunit.client.1.smithi152.stdout:2/212: read - d0/d3/df/d2c/f2d zero size 2024-03-29T04:45:11.774 INFO:tasks.workunit.client.0.smithi142.stdout:3/690: truncate d3/d29/d2b/d40/d67/d8b/d4d/f9d 2223310 0 2024-03-29T04:45:11.775 INFO:tasks.workunit.client.0.smithi142.stdout:9/715: getdents d0/d2/d6/d31 0 2024-03-29T04:45:11.776 INFO:tasks.workunit.client.1.smithi152.stdout:3/256: truncate d7/d19/d1b/f2f 2430233 0 2024-03-29T04:45:11.777 INFO:tasks.workunit.client.0.smithi142.stdout:4/808: creat d0/d9/db2/dc6/f109 x:0 0 0 2024-03-29T04:45:11.777 INFO:tasks.workunit.client.1.smithi152.stdout:0/274: symlink d2/da/d17/l65 0 2024-03-29T04:45:11.779 INFO:tasks.workunit.client.0.smithi142.stdout:9/716: write d0/d2/d6/d27/d3c/f72 [365263,16039] 0 2024-03-29T04:45:11.780 INFO:tasks.workunit.client.0.smithi142.stdout:7/926: readlink d7/d27/d44/db2/db8/d4b/d71/l32 0 2024-03-29T04:45:11.781 INFO:tasks.workunit.client.1.smithi152.stdout:7/219: rename l1 to d7/d8/d37/l40 0 2024-03-29T04:45:11.781 INFO:tasks.workunit.client.1.smithi152.stdout:5/262: dwrite d6/f21 [0,4194304] 0 2024-03-29T04:45:11.781 INFO:tasks.workunit.client.0.smithi142.stdout:2/846: mkdir de/d11/d17/d60/d6f/d116 0 2024-03-29T04:45:11.782 INFO:tasks.workunit.client.1.smithi152.stdout:1/322: read d5/d7/d9/f10 [6179113,101625] 0 2024-03-29T04:45:11.782 INFO:tasks.workunit.client.0.smithi142.stdout:3/691: chown d3/d29/d2b/d40/d67/d8b/d4d/f9d 0 1 2024-03-29T04:45:11.784 INFO:tasks.workunit.client.1.smithi152.stdout:3/257: rmdir d7/d1c 39 2024-03-29T04:45:11.784 INFO:tasks.workunit.client.0.smithi142.stdout:7/927: stat d7/d27/d44/db2/db8/d4b/d71/f108 0 2024-03-29T04:45:11.784 INFO:tasks.workunit.client.0.smithi142.stdout:3/692: chown d3/d1f/d59/fbb 4 1 2024-03-29T04:45:11.787 INFO:tasks.workunit.client.0.smithi142.stdout:8/985: rmdir d5/d2e/dd2/d60/d12d/d136 0 2024-03-29T04:45:11.789 INFO:tasks.workunit.client.0.smithi142.stdout:6/831: dwrite d1c/f21 [0,4194304] 0 2024-03-29T04:45:11.790 INFO:tasks.workunit.client.0.smithi142.stdout:2/847: creat de/d11/d17/d60/de6/d101/f117 x:0 0 0 2024-03-29T04:45:11.790 INFO:tasks.workunit.client.0.smithi142.stdout:8/986: write d5/d2e/ff4 [4324655,35500] 0 2024-03-29T04:45:11.791 INFO:tasks.workunit.client.1.smithi152.stdout:2/213: dread d0/d3/df/f13 [0,4194304] 0 2024-03-29T04:45:11.793 INFO:tasks.workunit.client.1.smithi152.stdout:2/214: readlink d0/d3/l11 0 2024-03-29T04:45:11.794 INFO:tasks.workunit.client.0.smithi142.stdout:7/928: chown d7/d27/d44/c96 43 1 2024-03-29T04:45:11.796 INFO:tasks.workunit.client.0.smithi142.stdout:6/832: creat d1c/d33/d46/d61/d7e/f109 x:0 0 0 2024-03-29T04:45:11.799 INFO:tasks.workunit.client.1.smithi152.stdout:0/275: dwrite d2/da/df/f4b [0,4194304] 0 2024-03-29T04:45:11.804 INFO:tasks.workunit.client.0.smithi142.stdout:6/833: creat d1c/d45/d8a/df6/df7/f10a x:0 0 0 2024-03-29T04:45:11.804 INFO:tasks.workunit.client.1.smithi152.stdout:1/323: unlink d5/d7/d8/l64 0 2024-03-29T04:45:11.805 INFO:tasks.workunit.client.0.smithi142.stdout:3/693: link d3/d29/d8f/d5f/d64/d6b/lab d3/d29/d2b/led 0 2024-03-29T04:45:11.805 INFO:tasks.workunit.client.0.smithi142.stdout:2/848: fdatasync de/d11/d17/d10b/d22/d27/dba/f6b 0 2024-03-29T04:45:11.805 INFO:tasks.workunit.client.0.smithi142.stdout:8/987: dwrite d5/d1d/d81/fa4 [0,4194304] 0 2024-03-29T04:45:11.807 INFO:tasks.workunit.client.1.smithi152.stdout:5/263: dwrite d6/d1b/f5a [0,4194304] 0 2024-03-29T04:45:11.807 INFO:tasks.workunit.client.0.smithi142.stdout:8/988: chown d5/d1d/d2f/d76/d3c/f5e 1 1 2024-03-29T04:45:11.808 INFO:tasks.workunit.client.1.smithi152.stdout:7/220: dread d7/d8/db/df/d21/f2c [0,4194304] 0 2024-03-29T04:45:11.809 INFO:tasks.workunit.client.1.smithi152.stdout:7/221: dread - d7/d8/d12/d15/f31 zero size 2024-03-29T04:45:11.810 INFO:tasks.workunit.client.0.smithi142.stdout:7/929: creat d7/d27/d44/d4c/f134 x:0 0 0 2024-03-29T04:45:11.810 INFO:tasks.workunit.client.1.smithi152.stdout:7/222: dread - d7/d8/d12/d15/f31 zero size 2024-03-29T04:45:11.811 INFO:tasks.workunit.client.1.smithi152.stdout:3/258: unlink d7/c16 0 2024-03-29T04:45:11.815 INFO:tasks.workunit.client.1.smithi152.stdout:3/259: read d7/d19/f26 [1405459,63523] 0 2024-03-29T04:45:11.816 INFO:tasks.workunit.client.0.smithi142.stdout:6/834: mknod d1c/d33/d46/d66/dc1/c10b 0 2024-03-29T04:45:11.819 INFO:tasks.workunit.client.0.smithi142.stdout:8/989: creat d5/d1d/d2f/d76/d9c/f151 x:0 0 0 2024-03-29T04:45:11.819 INFO:tasks.workunit.client.1.smithi152.stdout:5/264: write d6/d1d/d3b/f3f [1930619,113272] 0 2024-03-29T04:45:11.820 INFO:tasks.workunit.client.1.smithi152.stdout:1/324: mknod d5/d7/d8/d13/d19/c67 0 2024-03-29T04:45:11.821 INFO:tasks.workunit.client.1.smithi152.stdout:0/276: rename d2/l18 to d2/da/l66 0 2024-03-29T04:45:11.822 INFO:tasks.workunit.client.0.smithi142.stdout:7/930: mkdir d7/d27/d3a/d135 0 2024-03-29T04:45:11.822 INFO:tasks.workunit.client.1.smithi152.stdout:3/260: chown d7/d19/d1b/l44 455410 1 2024-03-29T04:45:11.822 INFO:tasks.workunit.client.1.smithi152.stdout:6/307: dread d1/d3/d1a/f1d [0,4194304] 0 2024-03-29T04:45:11.826 INFO:tasks.workunit.client.1.smithi152.stdout:6/308: symlink d1/d3/d1a/d1b/l6d 0 2024-03-29T04:45:11.827 INFO:tasks.workunit.client.0.smithi142.stdout:3/694: rename d3/d29/d8f/d5f/d64/fb8 to d3/d29/d2b/d40/d67/d8b/d4d/fee 0 2024-03-29T04:45:11.828 INFO:tasks.workunit.client.1.smithi152.stdout:7/223: dwrite d7/d8/d12/d1d/f1e [0,4194304] 0 2024-03-29T04:45:11.829 INFO:tasks.workunit.client.1.smithi152.stdout:1/325: fsync d5/d7/d8/d13/d19/d21/f52 0 2024-03-29T04:45:11.830 INFO:tasks.workunit.client.1.smithi152.stdout:7/224: readlink d7/d8/l2e 0 2024-03-29T04:45:11.832 INFO:tasks.workunit.client.1.smithi152.stdout:6/309: fsync d1/d3/d1a/f61 0 2024-03-29T04:45:11.832 INFO:tasks.workunit.client.0.smithi142.stdout:0/863: dread d7/d13/dbf/f8e [0,4194304] 0 2024-03-29T04:45:11.833 INFO:tasks.workunit.client.1.smithi152.stdout:7/225: chown d7/d8/d12/d1d/d30 857 1 2024-03-29T04:45:11.835 INFO:tasks.workunit.client.1.smithi152.stdout:0/277: symlink d2/da/d3a/d61/l67 0 2024-03-29T04:45:11.836 INFO:tasks.workunit.client.1.smithi152.stdout:6/310: readlink d1/d3/d9/d13/d28/d45/l52 0 2024-03-29T04:45:11.836 INFO:tasks.workunit.client.1.smithi152.stdout:1/326: dread - d5/d7/d1b/d2e/f5f zero size 2024-03-29T04:45:11.836 INFO:tasks.workunit.client.0.smithi142.stdout:0/864: chown d7/d13/dbf/dca/dab/dc1/dd3/d90/f10f 140721 1 2024-03-29T04:45:11.836 INFO:tasks.workunit.client.0.smithi142.stdout:6/835: rename d1c/d90/fa3 to d1c/d45/d8a/f10c 0 2024-03-29T04:45:11.838 INFO:tasks.workunit.client.0.smithi142.stdout:7/931: truncate d7/d27/d44/db2/db8/d4b/d71/d30/d72/fac 1138026 0 2024-03-29T04:45:11.838 INFO:tasks.workunit.client.0.smithi142.stdout:3/695: symlink d3/d29/d2d/de9/lef 0 2024-03-29T04:45:11.839 INFO:tasks.workunit.client.1.smithi152.stdout:3/261: creat d7/d1c/f47 x:0 0 0 2024-03-29T04:45:11.839 INFO:tasks.workunit.client.0.smithi142.stdout:7/932: write d7/d27/d44/d4c/f7b [4951146,63155] 0 2024-03-29T04:45:11.840 INFO:tasks.workunit.client.1.smithi152.stdout:7/226: chown d7/d8/d12/f22 88 1 2024-03-29T04:45:11.840 INFO:tasks.workunit.client.1.smithi152.stdout:9/289: sync 2024-03-29T04:45:11.841 INFO:tasks.workunit.client.1.smithi152.stdout:0/278: chown d2/f9 116 1 2024-03-29T04:45:11.841 INFO:tasks.workunit.client.1.smithi152.stdout:5/265: dwrite d6/d1d/d3b/d55/f17 [0,4194304] 0 2024-03-29T04:45:11.842 INFO:tasks.workunit.client.0.smithi142.stdout:1/792: write d8/db/dc/d77/db9/dbe/d1f/d4c/fc4 [152274,79428] 0 2024-03-29T04:45:11.842 INFO:tasks.workunit.client.0.smithi142.stdout:3/696: truncate d3/d1f/d59/fb3 982847 0 2024-03-29T04:45:11.844 INFO:tasks.workunit.client.1.smithi152.stdout:9/290: write d5/f7 [632535,43915] 0 2024-03-29T04:45:11.847 INFO:tasks.workunit.client.0.smithi142.stdout:6/836: dwrite f4 [4194304,4194304] 0 2024-03-29T04:45:11.847 INFO:tasks.workunit.client.0.smithi142.stdout:7/933: creat d7/d27/d44/dab/f136 x:0 0 0 2024-03-29T04:45:11.849 INFO:tasks.workunit.client.1.smithi152.stdout:3/262: symlink d7/d1c/d41/l48 0 2024-03-29T04:45:11.849 INFO:tasks.workunit.client.1.smithi152.stdout:1/327: dwrite d5/d7/f25 [0,4194304] 0 2024-03-29T04:45:11.851 INFO:tasks.workunit.client.1.smithi152.stdout:8/204: truncate d1/d2/d3/f29 3484543 0 2024-03-29T04:45:11.851 INFO:tasks.workunit.client.0.smithi142.stdout:3/697: mknod d3/d29/d8f/d5f/cf0 0 2024-03-29T04:45:11.852 INFO:tasks.workunit.client.0.smithi142.stdout:9/717: write d0/f32 [539371,66726] 0 2024-03-29T04:45:11.854 INFO:tasks.workunit.client.0.smithi142.stdout:6/837: symlink d1c/d45/l10d 0 2024-03-29T04:45:11.856 INFO:tasks.workunit.client.0.smithi142.stdout:8/990: dread d5/d2e/dd2/fd7 [0,4194304] 0 2024-03-29T04:45:11.856 INFO:tasks.workunit.client.0.smithi142.stdout:6/838: write d1c/f62 [2506066,39830] 0 2024-03-29T04:45:11.858 INFO:tasks.workunit.client.0.smithi142.stdout:3/698: read d3/d1f/d59/f6e [633456,113930] 0 2024-03-29T04:45:11.859 INFO:tasks.workunit.client.1.smithi152.stdout:9/291: write d5/db/f58 [838303,64321] 0 2024-03-29T04:45:11.860 INFO:tasks.workunit.client.0.smithi142.stdout:2/849: write de/d11/d17/d10b/d22/d50/f57 [192002,59682] 0 2024-03-29T04:45:11.860 INFO:tasks.workunit.client.1.smithi152.stdout:5/266: mkdir d6/d16/d5c 0 2024-03-29T04:45:11.862 INFO:tasks.workunit.client.0.smithi142.stdout:6/839: dread - d1c/d33/d46/d61/ff9 zero size 2024-03-29T04:45:11.862 INFO:tasks.workunit.client.0.smithi142.stdout:9/718: symlink d0/d2/d39/le4 0 2024-03-29T04:45:11.863 INFO:tasks.workunit.client.0.smithi142.stdout:4/809: dwrite d0/d12/f44 [0,4194304] 0 2024-03-29T04:45:11.866 INFO:tasks.workunit.client.1.smithi152.stdout:5/267: fsync d6/d16/d25/f3a 0 2024-03-29T04:45:11.867 INFO:tasks.workunit.client.0.smithi142.stdout:4/810: stat d0/d9/db2/dba/dbc/daf 0 2024-03-29T04:45:11.867 INFO:tasks.workunit.client.0.smithi142.stdout:9/719: write d0/d2/d6/d27/dc7/d6a/fae [2452540,38000] 0 2024-03-29T04:45:11.867 INFO:tasks.workunit.client.0.smithi142.stdout:8/991: truncate d5/d1d/d2f/d76/d2d/f56 4637177 0 2024-03-29T04:45:11.868 INFO:tasks.workunit.client.0.smithi142.stdout:3/699: fsync d3/d29/d2b/d40/d67/d8b/f4e 0 2024-03-29T04:45:11.870 INFO:tasks.workunit.client.1.smithi152.stdout:1/328: dwrite d5/d7/f1f [0,4194304] 0 2024-03-29T04:45:11.870 INFO:tasks.workunit.client.1.smithi152.stdout:3/263: unlink d7/d1c/l28 0 2024-03-29T04:45:11.870 INFO:tasks.workunit.client.0.smithi142.stdout:2/850: creat de/d11/d3d/f118 x:0 0 0 2024-03-29T04:45:11.872 INFO:tasks.workunit.client.1.smithi152.stdout:0/279: dwrite d2/f5 [0,4194304] 0 2024-03-29T04:45:11.874 INFO:tasks.workunit.client.1.smithi152.stdout:1/329: fsync d5/d7/d8/d13/f45 0 2024-03-29T04:45:11.874 INFO:tasks.workunit.client.0.smithi142.stdout:3/700: write d3/d29/d2b/d40/d67/fb7 [2226511,103146] 0 2024-03-29T04:45:11.876 INFO:tasks.workunit.client.1.smithi152.stdout:9/292: creat d5/d8/f69 x:0 0 0 2024-03-29T04:45:11.876 INFO:tasks.workunit.client.0.smithi142.stdout:2/851: mkdir de/d11/d17/d5d/d71/d119 0 2024-03-29T04:45:11.876 INFO:tasks.workunit.client.0.smithi142.stdout:7/934: rename d7/d27/d44/db2/db8/d4b/d71/d30/d72/d105 to d7/d27/d44/db2/db8/d4b/d71/d30/d33/d137 0 2024-03-29T04:45:11.878 INFO:tasks.workunit.client.1.smithi152.stdout:3/264: dread - d7/d19/d1b/d1e/f45 zero size 2024-03-29T04:45:11.879 INFO:tasks.workunit.client.1.smithi152.stdout:6/311: write d1/d3/d9/d13/f31 [1224680,37933] 0 2024-03-29T04:45:11.880 INFO:tasks.workunit.client.1.smithi152.stdout:8/205: dwrite d1/d2/d1d/f3d [0,4194304] 0 2024-03-29T04:45:11.883 INFO:tasks.workunit.client.1.smithi152.stdout:0/280: creat d2/da/d3a/f68 x:0 0 0 2024-03-29T04:45:11.886 INFO:tasks.workunit.client.1.smithi152.stdout:5/268: symlink d6/d16/l5d 0 2024-03-29T04:45:11.886 INFO:tasks.workunit.client.0.smithi142.stdout:2/852: mkdir de/d11/d17/d4a/d11a 0 2024-03-29T04:45:11.889 INFO:tasks.workunit.client.0.smithi142.stdout:9/720: creat d0/d2/d61/fe5 x:0 0 0 2024-03-29T04:45:11.889 INFO:tasks.workunit.client.0.smithi142.stdout:8/992: mknod d5/d1d/d2f/d5f/d48/c152 0 2024-03-29T04:45:11.891 INFO:tasks.workunit.client.0.smithi142.stdout:7/935: readlink d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/d86/l5f 0 2024-03-29T04:45:11.891 INFO:tasks.workunit.client.1.smithi152.stdout:0/281: dwrite d2/d29/f37 [0,4194304] 0 2024-03-29T04:45:11.892 INFO:tasks.workunit.client.0.smithi142.stdout:0/865: write d7/f21 [339997,121157] 0 2024-03-29T04:45:11.893 INFO:tasks.workunit.client.1.smithi152.stdout:8/206: rename d1/d2/f1a to d1/d21/f46 0 2024-03-29T04:45:11.894 INFO:tasks.workunit.client.0.smithi142.stdout:8/993: readlink d5/d1d/d2f/ldc 0 2024-03-29T04:45:11.896 INFO:tasks.workunit.client.0.smithi142.stdout:9/721: read d0/d2/f11 [2106594,123232] 0 2024-03-29T04:45:11.898 INFO:tasks.workunit.client.0.smithi142.stdout:8/994: chown d5/d1d/d2f/d76/d9c/l6a 1132759 1 2024-03-29T04:45:11.898 INFO:tasks.workunit.client.0.smithi142.stdout:2/853: dwrite de/d11/d23/dd1/ff0 [0,4194304] 0 2024-03-29T04:45:11.900 INFO:tasks.workunit.client.1.smithi152.stdout:0/282: read d2/d29/f2f [528578,99946] 0 2024-03-29T04:45:11.901 INFO:tasks.workunit.client.1.smithi152.stdout:5/269: symlink d6/d1b/l5e 0 2024-03-29T04:45:11.902 INFO:tasks.workunit.client.1.smithi152.stdout:7/227: truncate d7/d8/d12/f33 571875 0 2024-03-29T04:45:11.902 INFO:tasks.workunit.client.0.smithi142.stdout:9/722: truncate d0/d2/d6/d27/f97 838889 0 2024-03-29T04:45:11.903 INFO:tasks.workunit.client.1.smithi152.stdout:5/270: write d6/d1d/d3b/d55/f17 [3280550,40901] 0 2024-03-29T04:45:11.904 INFO:tasks.workunit.client.0.smithi142.stdout:9/723: chown d0/d12/d50/d52/fdd 17226 1 2024-03-29T04:45:11.905 INFO:tasks.workunit.client.1.smithi152.stdout:3/265: link d7/f10 d7/d19/d3a/f49 0 2024-03-29T04:45:11.906 INFO:tasks.workunit.client.0.smithi142.stdout:0/866: dread d7/d13/dbf/dca/dab/fd8 [0,4194304] 0 2024-03-29T04:45:11.908 INFO:tasks.workunit.client.0.smithi142.stdout:0/867: chown d7/d13/dbf/dca/dab/dc1/dd3/d90/fe5 1706301201 1 2024-03-29T04:45:11.909 INFO:tasks.workunit.client.0.smithi142.stdout:0/868: chown d7/d9/d55/c76 0 1 2024-03-29T04:45:11.910 INFO:tasks.workunit.client.0.smithi142.stdout:0/869: write d7/d13/dbf/d72/fd6 [3835907,9114] 0 2024-03-29T04:45:11.911 INFO:tasks.workunit.client.1.smithi152.stdout:1/330: getdents d5/d7/d8/d13 0 2024-03-29T04:45:11.912 INFO:tasks.workunit.client.0.smithi142.stdout:1/793: getdents d8/db/dc/d77/db9/dbe/d2d/d8b/da4/dd1 0 2024-03-29T04:45:11.912 INFO:tasks.workunit.client.1.smithi152.stdout:0/283: mknod d2/d29/c69 0 2024-03-29T04:45:11.913 INFO:tasks.workunit.client.0.smithi142.stdout:3/701: mkdir d3/d1f/de6/df1 0 2024-03-29T04:45:11.915 INFO:tasks.workunit.client.0.smithi142.stdout:2/854: mkdir de/d11/d17/d10b/d22/d27/dba/d61/d7d/d11b 0 2024-03-29T04:45:11.918 INFO:tasks.workunit.client.1.smithi152.stdout:5/271: chown d6/d7/lc 530 1 2024-03-29T04:45:11.919 INFO:tasks.workunit.client.0.smithi142.stdout:6/840: rename d1c/d33/db6/c4d to d1c/d33/c10e 0 2024-03-29T04:45:11.919 INFO:tasks.workunit.client.1.smithi152.stdout:1/331: dwrite d5/d7/d9/f11 [0,4194304] 0 2024-03-29T04:45:11.919 INFO:tasks.workunit.client.0.smithi142.stdout:2/855: mkdir de/d11/d17/d10b/d22/dbf/d11c 0 2024-03-29T04:45:11.920 INFO:tasks.workunit.client.1.smithi152.stdout:1/332: fsync d5/d7/d9/f46 0 2024-03-29T04:45:11.920 INFO:tasks.workunit.client.0.smithi142.stdout:1/794: symlink d8/db/dc/d77/db9/dbe/d2d/d8b/dda/l10d 0 2024-03-29T04:45:11.921 INFO:tasks.workunit.client.1.smithi152.stdout:8/207: mkdir d1/d21/d40/d47 0 2024-03-29T04:45:11.921 INFO:tasks.workunit.client.0.smithi142.stdout:1/795: readlink d8/db/dc/d77/db9/dbe/d15/l4e 0 2024-03-29T04:45:11.922 INFO:tasks.workunit.client.1.smithi152.stdout:8/208: fdatasync d1/d2/d1d/f23 0 2024-03-29T04:45:11.922 INFO:tasks.workunit.client.0.smithi142.stdout:2/856: symlink de/d11/d23/dd1/l11d 0 2024-03-29T04:45:11.925 INFO:tasks.workunit.client.1.smithi152.stdout:0/284: symlink d2/da/d17/d57/l6a 0 2024-03-29T04:45:11.927 INFO:tasks.workunit.client.1.smithi152.stdout:0/285: write d2/d29/f44 [716058,47289] 0 2024-03-29T04:45:11.928 INFO:tasks.workunit.client.0.smithi142.stdout:6/841: rename d1c/d45/l47 to d1c/d33/d46/d61/d9b/l10f 0 2024-03-29T04:45:11.930 INFO:tasks.workunit.client.0.smithi142.stdout:7/936: getdents d7/d27/d3a/d92/de5 0 2024-03-29T04:45:11.932 INFO:tasks.workunit.client.0.smithi142.stdout:0/870: getdents d7/d13/d4a/d9a/da7 0 2024-03-29T04:45:11.932 INFO:tasks.workunit.client.0.smithi142.stdout:2/857: link de/d11/d17/d60/d6f/f100 de/d11/d17/d4a/d11a/f11e 0 2024-03-29T04:45:11.933 INFO:tasks.workunit.client.0.smithi142.stdout:6/842: creat d1c/d45/d8a/f110 x:0 0 0 2024-03-29T04:45:11.936 INFO:tasks.workunit.client.0.smithi142.stdout:6/843: creat d1c/d33/db6/f111 x:0 0 0 2024-03-29T04:45:11.937 INFO:tasks.workunit.client.0.smithi142.stdout:0/871: mkdir d7/d13/dbf/d72/d116 0 2024-03-29T04:45:11.939 INFO:tasks.workunit.client.0.smithi142.stdout:2/858: symlink de/d11/d17/d60/d6f/d116/l11f 0 2024-03-29T04:45:11.939 INFO:tasks.workunit.client.0.smithi142.stdout:3/702: dread d3/d29/d2b/d40/d67/d8b/d4d/f9d [0,4194304] 0 2024-03-29T04:45:11.941 INFO:tasks.workunit.client.0.smithi142.stdout:2/859: write de/d11/d3d/f82 [2082144,29722] 0 2024-03-29T04:45:11.941 INFO:tasks.workunit.client.1.smithi152.stdout:0/286: read d2/f8 [8841,13242] 0 2024-03-29T04:45:11.943 INFO:tasks.workunit.client.0.smithi142.stdout:6/844: mkdir d1c/d33/d46/d61/d112 0 2024-03-29T04:45:11.943 INFO:tasks.workunit.client.0.smithi142.stdout:7/937: link d7/d27/d44/db2/db8/d4b/d71/l6a d7/d27/d3a/d92/l138 0 2024-03-29T04:45:11.944 INFO:tasks.workunit.client.0.smithi142.stdout:0/872: mknod d7/d13/dbf/d72/d37/da4/d114/c117 0 2024-03-29T04:45:11.945 INFO:tasks.workunit.client.0.smithi142.stdout:6/845: stat d1c/dc9 0 2024-03-29T04:45:11.945 INFO:tasks.workunit.client.1.smithi152.stdout:6/312: write d1/d3/d1a/d1b/f58 [62579,35513] 0 2024-03-29T04:45:11.947 INFO:tasks.workunit.client.0.smithi142.stdout:2/860: read - de/d11/d17/d5d/fe5 zero size 2024-03-29T04:45:11.950 INFO:tasks.workunit.client.1.smithi152.stdout:1/333: rename d5/d7/d9/f4a to d5/d7/d8/d13/d19/f68 0 2024-03-29T04:45:11.950 INFO:tasks.workunit.client.0.smithi142.stdout:3/703: dwrite d3/d1f/d24/d47/f4f [0,4194304] 0 2024-03-29T04:45:11.950 INFO:tasks.workunit.client.0.smithi142.stdout:6/846: mkdir d1c/d33/d46/d86/dc0/d113 0 2024-03-29T04:45:11.950 INFO:tasks.workunit.client.1.smithi152.stdout:1/334: chown d5/d7/d1b/c42 51772582 1 2024-03-29T04:45:11.951 INFO:tasks.workunit.client.1.smithi152.stdout:6/313: symlink d1/d3/d9/d3b/l6e 0 2024-03-29T04:45:11.951 INFO:tasks.workunit.client.1.smithi152.stdout:1/335: fsync d5/f3c 0 2024-03-29T04:45:11.952 INFO:tasks.workunit.client.0.smithi142.stdout:6/847: dread - d1c/d33/d46/d61/d7e/f109 zero size 2024-03-29T04:45:11.952 INFO:tasks.workunit.client.0.smithi142.stdout:0/873: getdents d7/d9/d55/df7 0 2024-03-29T04:45:11.953 INFO:tasks.workunit.client.0.smithi142.stdout:0/874: chown d7/d13/dbf/d72/d75/la3 123 1 2024-03-29T04:45:11.953 INFO:tasks.workunit.client.1.smithi152.stdout:5/272: getdents d6/d16 0 2024-03-29T04:45:11.955 INFO:tasks.workunit.client.0.smithi142.stdout:3/704: mknod d3/d29/d2d/d79/dcc/cf2 0 2024-03-29T04:45:11.956 INFO:tasks.workunit.client.1.smithi152.stdout:8/209: dread d1/f10 [0,4194304] 0 2024-03-29T04:45:11.956 INFO:tasks.workunit.client.0.smithi142.stdout:0/875: stat d7/d13/f7d 0 2024-03-29T04:45:11.960 INFO:tasks.workunit.client.0.smithi142.stdout:8/995: dwrite d5/d2e/fd5 [0,4194304] 0 2024-03-29T04:45:11.961 INFO:tasks.workunit.client.1.smithi152.stdout:1/336: chown d5/d7/d9/c43 1275 1 2024-03-29T04:45:11.962 INFO:tasks.workunit.client.1.smithi152.stdout:1/337: write d5/d7/f54 [4931411,38098] 0 2024-03-29T04:45:11.963 INFO:tasks.workunit.client.0.smithi142.stdout:0/876: read - d7/d13/dbf/d72/fc7 zero size 2024-03-29T04:45:11.964 INFO:tasks.workunit.client.0.smithi142.stdout:3/705: dread d3/d29/d2d/f48 [0,4194304] 0 2024-03-29T04:45:11.965 INFO:tasks.workunit.client.1.smithi152.stdout:0/287: creat d2/f6b x:0 0 0 2024-03-29T04:45:11.966 INFO:tasks.workunit.client.0.smithi142.stdout:3/706: readlink d3/d29/d2d/d6c/lae 0 2024-03-29T04:45:11.968 INFO:tasks.workunit.client.0.smithi142.stdout:7/938: write d7/d27/f91 [126130,44943] 0 2024-03-29T04:45:11.970 INFO:tasks.workunit.client.1.smithi152.stdout:8/210: dread d1/f34 [0,4194304] 0 2024-03-29T04:45:11.972 INFO:tasks.workunit.client.1.smithi152.stdout:8/211: read d1/d21/f32 [779419,88734] 0 2024-03-29T04:45:11.973 INFO:tasks.workunit.client.1.smithi152.stdout:1/338: creat d5/d7/d8/d13/d19/d21/f69 x:0 0 0 2024-03-29T04:45:11.973 INFO:tasks.workunit.client.0.smithi142.stdout:0/877: creat d7/d13/f118 x:0 0 0 2024-03-29T04:45:11.974 INFO:tasks.workunit.client.0.smithi142.stdout:8/996: dwrite d5/d2e/dd2/d60/f116 [0,4194304] 0 2024-03-29T04:45:11.974 INFO:tasks.workunit.client.1.smithi152.stdout:0/288: rmdir d2/da/df 39 2024-03-29T04:45:11.975 INFO:tasks.workunit.client.0.smithi142.stdout:7/939: symlink d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/l139 0 2024-03-29T04:45:11.978 INFO:tasks.workunit.client.1.smithi152.stdout:5/273: unlink d6/d7/f4c 0 2024-03-29T04:45:11.979 INFO:tasks.workunit.client.0.smithi142.stdout:0/878: unlink d7/d13/dbf/d72/d75/la3 0 2024-03-29T04:45:11.981 INFO:tasks.workunit.client.0.smithi142.stdout:0/879: fsync d7/d13/f50 0 2024-03-29T04:45:11.981 INFO:tasks.workunit.client.1.smithi152.stdout:8/212: truncate d1/f34 549877 0 2024-03-29T04:45:11.984 INFO:tasks.workunit.client.1.smithi152.stdout:0/289: rmdir d2/d29 39 2024-03-29T04:45:11.985 INFO:tasks.workunit.client.0.smithi142.stdout:8/997: rename d5/d1d/d2f/d76/d9c/fb9 to d5/d1d/d24/d66/dce/f153 0 2024-03-29T04:45:11.986 INFO:tasks.workunit.client.1.smithi152.stdout:3/266: write d7/d19/f36 [4663230,59861] 0 2024-03-29T04:45:11.986 INFO:tasks.workunit.client.0.smithi142.stdout:3/707: dwrite d3/d29/d8f/d5f/db4/f97 [0,4194304] 0 2024-03-29T04:45:11.987 INFO:tasks.workunit.client.1.smithi152.stdout:8/213: dread d1/d21/f24 [0,4194304] 0 2024-03-29T04:45:11.987 INFO:tasks.workunit.client.0.smithi142.stdout:0/880: mkdir d7/d9/d55/df7/d119 0 2024-03-29T04:45:11.994 INFO:tasks.workunit.client.0.smithi142.stdout:0/881: creat d7/d13/d4a/d9a/da7/f11a x:0 0 0 2024-03-29T04:45:11.994 INFO:tasks.workunit.client.0.smithi142.stdout:3/708: dread d3/f3c [0,4194304] 0 2024-03-29T04:45:11.995 INFO:tasks.workunit.client.0.smithi142.stdout:8/998: unlink d5/d1d/d2f/d45/l4d 0 2024-03-29T04:45:11.995 INFO:tasks.workunit.client.1.smithi152.stdout:3/267: write d7/d19/d1b/d1e/d22/f3b [1347505,52283] 0 2024-03-29T04:45:11.996 INFO:tasks.workunit.client.1.smithi152.stdout:5/274: symlink d6/d16/d5c/l5f 0 2024-03-29T04:45:11.998 INFO:tasks.workunit.client.1.smithi152.stdout:7/228: write d7/f14 [666411,34324] 0 2024-03-29T04:45:11.999 INFO:tasks.workunit.client.0.smithi142.stdout:3/709: creat d3/d29/d2b/d40/ff3 x:0 0 0 2024-03-29T04:45:12.000 INFO:tasks.workunit.client.0.smithi142.stdout:8/999: mkdir d5/d2e/dd2/d154 0 2024-03-29T04:45:12.002 INFO:tasks.workunit.client.1.smithi152.stdout:0/290: creat d2/d29/f6c x:0 0 0 2024-03-29T04:45:12.003 INFO:tasks.workunit.client.0.smithi142.stdout:0/882: unlink d7/d13/dbf/d72/d37/f43 0 2024-03-29T04:45:12.004 INFO:tasks.workunit.client.0.smithi142.stdout:3/710: rmdir d3/d1f/d59 39 2024-03-29T04:45:12.005 INFO:tasks.workunit.client.1.smithi152.stdout:7/229: dwrite d7/d8/db/df/f2b [0,4194304] 0 2024-03-29T04:45:12.007 INFO:tasks.workunit.client.0.smithi142.stdout:7/940: dread d7/d27/d44/db2/db8/d4b/d71/d30/d33/f93 [0,4194304] 0 2024-03-29T04:45:12.008 INFO:tasks.workunit.client.0.smithi142.stdout:0/883: rename d7/d9/d1e/d81/fc4 to d7/d9/d55/df7/d119/f11b 0 2024-03-29T04:45:12.008 INFO:tasks.workunit.client.0.smithi142.stdout:3/711: dread d3/d29/d2b/d40/d67/fb6 [0,4194304] 0 2024-03-29T04:45:12.013 INFO:tasks.workunit.client.1.smithi152.stdout:5/275: creat d6/d1d/d3b/d55/f60 x:0 0 0 2024-03-29T04:45:12.014 INFO:tasks.workunit.client.1.smithi152.stdout:5/276: write d6/d7/f8 [3453329,69231] 0 2024-03-29T04:45:12.015 INFO:tasks.workunit.client.1.smithi152.stdout:8/214: creat d1/d2/d1d/d36/f48 x:0 0 0 2024-03-29T04:45:12.016 INFO:tasks.workunit.client.1.smithi152.stdout:5/277: write d6/d1b/f31 [1856087,46520] 0 2024-03-29T04:45:12.016 INFO:tasks.workunit.client.0.smithi142.stdout:3/712: mkdir d3/d29/d2b/d6a/df4 0 2024-03-29T04:45:12.017 INFO:tasks.workunit.client.1.smithi152.stdout:1/339: getdents d5/d7/d8/d13/d19 0 2024-03-29T04:45:12.017 INFO:tasks.workunit.client.1.smithi152.stdout:5/278: fsync d6/d1d/d3b/f39 0 2024-03-29T04:45:12.018 INFO:tasks.workunit.client.0.smithi142.stdout:0/884: creat d7/d13/d4a/d9a/da7/f11c x:0 0 0 2024-03-29T04:45:12.019 INFO:tasks.workunit.client.1.smithi152.stdout:1/340: dread - d5/d7/d8/d13/d19/d21/f69 zero size 2024-03-29T04:45:12.019 INFO:tasks.workunit.client.1.smithi152.stdout:6/314: rename d1/d3/d9/d3b to d1/d3/d1a/d1b/d29/d6f 0 2024-03-29T04:45:12.019 INFO:tasks.workunit.client.1.smithi152.stdout:3/268: mknod d7/d46/c4a 0 2024-03-29T04:45:12.021 INFO:tasks.workunit.client.1.smithi152.stdout:7/230: truncate d7/d8/db/df/d21/f2c 162325 0 2024-03-29T04:45:12.021 INFO:tasks.workunit.client.1.smithi152.stdout:1/341: chown d5/d7/d9/f10 21 1 2024-03-29T04:45:12.021 INFO:tasks.workunit.client.1.smithi152.stdout:3/269: fdatasync d7/f1a 0 2024-03-29T04:45:12.023 INFO:tasks.workunit.client.1.smithi152.stdout:3/270: write d7/d19/f36 [2664826,34025] 0 2024-03-29T04:45:12.026 INFO:tasks.workunit.client.1.smithi152.stdout:3/271: fdatasync d7/d19/f29 0 2024-03-29T04:45:12.027 INFO:tasks.workunit.client.0.smithi142.stdout:0/885: dwrite d7/d9/d54/fed [0,4194304] 0 2024-03-29T04:45:12.027 INFO:tasks.workunit.client.0.smithi142.stdout:3/713: rmdir d3/d1f/d59/de7 0 2024-03-29T04:45:12.029 INFO:tasks.workunit.client.1.smithi152.stdout:6/315: dwrite d1/d3/d1a/d1b/f58 [0,4194304] 0 2024-03-29T04:45:12.030 INFO:tasks.workunit.client.0.smithi142.stdout:0/886: symlink d7/d9/d1e/l11d 0 2024-03-29T04:45:12.030 INFO:tasks.workunit.client.0.smithi142.stdout:3/714: symlink d3/d29/d2d/d56/lf5 0 2024-03-29T04:45:12.035 INFO:tasks.workunit.client.0.smithi142.stdout:7/941: dwrite d7/d27/d44/f60 [0,4194304] 0 2024-03-29T04:45:12.035 INFO:tasks.workunit.client.1.smithi152.stdout:5/279: unlink d6/d1b/l24 0 2024-03-29T04:45:12.037 INFO:tasks.workunit.client.0.smithi142.stdout:0/887: creat d7/d9/d55/d42/f11e x:0 0 0 2024-03-29T04:45:12.037 INFO:tasks.workunit.client.1.smithi152.stdout:6/316: rmdir d1/d3/d9/d13 39 2024-03-29T04:45:12.039 INFO:tasks.workunit.client.1.smithi152.stdout:6/317: write d1/d3/f4e [154170,3123] 0 2024-03-29T04:45:12.039 INFO:tasks.workunit.client.1.smithi152.stdout:0/291: rename d2/f59 to d2/da/d17/f6d 0 2024-03-29T04:45:12.040 INFO:tasks.workunit.client.1.smithi152.stdout:8/215: rename d1/d2/d1d to d1/d2/d1d/d36/d49 22 2024-03-29T04:45:12.040 INFO:tasks.workunit.client.0.smithi142.stdout:3/715: symlink d3/d1f/d59/dc7/lf6 0 2024-03-29T04:45:12.041 INFO:tasks.workunit.client.1.smithi152.stdout:0/292: dread - d2/d29/f6c zero size 2024-03-29T04:45:12.042 INFO:tasks.workunit.client.0.smithi142.stdout:3/716: fdatasync d3/d29/d2d/faa 0 2024-03-29T04:45:12.044 INFO:tasks.workunit.client.0.smithi142.stdout:0/888: read d7/d9/d1e/f53 [853440,42723] 0 2024-03-29T04:45:12.044 INFO:tasks.workunit.client.1.smithi152.stdout:0/293: readlink d2/da/d17/d50/l58 0 2024-03-29T04:45:12.046 INFO:tasks.workunit.client.1.smithi152.stdout:3/272: dwrite d7/d19/d1b/f2f [0,4194304] 0 2024-03-29T04:45:12.047 INFO:tasks.workunit.client.0.smithi142.stdout:3/717: write d3/d29/d2b/d40/d67/d8b/d4d/fee [986899,17808] 0 2024-03-29T04:45:12.048 INFO:tasks.workunit.client.1.smithi152.stdout:6/318: chown d1/d3/d9/d13/d3e/f4d 920 1 2024-03-29T04:45:12.049 INFO:tasks.workunit.client.1.smithi152.stdout:8/216: symlink d1/d2/d1d/d36/l4a 0 2024-03-29T04:45:12.049 INFO:tasks.workunit.client.0.smithi142.stdout:0/889: mknod d7/d9/d6e/df2/c11f 0 2024-03-29T04:45:12.050 INFO:tasks.workunit.client.0.smithi142.stdout:0/890: write d7/d13/f28 [4134141,57467] 0 2024-03-29T04:45:12.051 INFO:tasks.workunit.client.0.smithi142.stdout:3/718: unlink d3/l7a 0 2024-03-29T04:45:12.053 INFO:tasks.workunit.client.1.smithi152.stdout:1/342: rename d5/d7/d8/d13/d19/d21 to d5/d7/d6a 0 2024-03-29T04:45:12.053 INFO:tasks.workunit.client.1.smithi152.stdout:1/343: chown d5/d7/d38 13853951 1 2024-03-29T04:45:12.053 INFO:tasks.workunit.client.1.smithi152.stdout:3/273: truncate d7/f18 730306 0 2024-03-29T04:45:12.053 INFO:tasks.workunit.client.0.smithi142.stdout:0/891: rmdir d7/d13/dbf/dca 39 2024-03-29T04:45:12.054 INFO:tasks.workunit.client.1.smithi152.stdout:6/319: mknod d1/d3/d1a/d49/c70 0 2024-03-29T04:45:12.055 INFO:tasks.workunit.client.1.smithi152.stdout:3/274: write d7/d1c/f47 [459928,38194] 0 2024-03-29T04:45:12.055 INFO:tasks.workunit.client.1.smithi152.stdout:0/294: symlink d2/l6e 0 2024-03-29T04:45:12.057 INFO:tasks.workunit.client.1.smithi152.stdout:7/231: rename d7/d8/db/df/d21 to d7/d8/d12/d1d/d41 0 2024-03-29T04:45:12.058 INFO:tasks.workunit.client.1.smithi152.stdout:6/320: truncate d1/d3/d9/d13/d3e/f4d 1676699 0 2024-03-29T04:45:12.058 INFO:tasks.workunit.client.1.smithi152.stdout:8/217: mkdir d1/d2/d1d/d36/d4b 0 2024-03-29T04:45:12.058 INFO:tasks.workunit.client.0.smithi142.stdout:3/719: mkdir d3/d29/d2b/d40/d67/d8b/df7 0 2024-03-29T04:45:12.059 INFO:tasks.workunit.client.1.smithi152.stdout:1/344: mknod d5/d7/d9/d12/c6b 0 2024-03-29T04:45:12.059 INFO:tasks.workunit.client.1.smithi152.stdout:6/321: chown d1/d3 0 1 2024-03-29T04:45:12.059 INFO:tasks.workunit.client.0.smithi142.stdout:0/892: mkdir d7/d13/d4a/d9a/da7/d120 0 2024-03-29T04:45:12.061 INFO:tasks.workunit.client.1.smithi152.stdout:0/295: mkdir d2/da/d3a/d6f 0 2024-03-29T04:45:12.062 INFO:tasks.workunit.client.1.smithi152.stdout:6/322: write f0 [792606,93254] 0 2024-03-29T04:45:12.062 INFO:tasks.workunit.client.1.smithi152.stdout:7/232: write d7/d8/d12/d35/f1a [1027056,109802] 0 2024-03-29T04:45:12.063 INFO:tasks.workunit.client.1.smithi152.stdout:1/345: dread - d5/d7/d9/d59/f61 zero size 2024-03-29T04:45:12.063 INFO:tasks.workunit.client.1.smithi152.stdout:5/280: rename d6/f26 to d6/d16/d47/f61 0 2024-03-29T04:45:12.065 INFO:tasks.workunit.client.1.smithi152.stdout:5/281: stat d6/d1d/d30/f3e 0 2024-03-29T04:45:12.066 INFO:tasks.workunit.client.1.smithi152.stdout:8/218: symlink d1/d2/d1d/d42/l4c 0 2024-03-29T04:45:12.066 INFO:tasks.workunit.client.1.smithi152.stdout:1/346: chown d5 1 1 2024-03-29T04:45:12.068 INFO:tasks.workunit.client.1.smithi152.stdout:0/296: mknod d2/da/d3a/d61/c70 0 2024-03-29T04:45:12.069 INFO:tasks.workunit.client.1.smithi152.stdout:6/323: dread - d1/d3/d1a/d49/f5e zero size 2024-03-29T04:45:12.070 INFO:tasks.workunit.client.1.smithi152.stdout:3/275: dwrite d7/d19/d1b/f2f [4194304,4194304] 0 2024-03-29T04:45:12.072 INFO:tasks.workunit.client.0.smithi142.stdout:7/942: write d7/fa [3381510,40726] 0 2024-03-29T04:45:12.072 INFO:tasks.workunit.client.1.smithi152.stdout:8/219: readlink d1/d2/d3/l8 0 2024-03-29T04:45:12.073 INFO:tasks.workunit.client.0.smithi142.stdout:7/943: chown d7/d27/d44/f60 16191 1 2024-03-29T04:45:12.073 INFO:tasks.workunit.client.1.smithi152.stdout:3/276: chown d7/c35 1692249897 1 2024-03-29T04:45:12.076 INFO:tasks.workunit.client.1.smithi152.stdout:3/277: read d7/d19/d1b/f21 [1302967,46115] 0 2024-03-29T04:45:12.076 INFO:tasks.workunit.client.1.smithi152.stdout:1/347: mkdir d5/d7/d38/d6c 0 2024-03-29T04:45:12.078 INFO:tasks.workunit.client.0.smithi142.stdout:0/893: truncate d7/d13/dbf/dca/dab/dc1/dd3/d90/fe5 2969500 0 2024-03-29T04:45:12.078 INFO:tasks.workunit.client.1.smithi152.stdout:3/278: truncate d7/d19/d1b/f21 4220682 0 2024-03-29T04:45:12.079 INFO:tasks.workunit.client.1.smithi152.stdout:6/324: creat d1/d3/d1a/d1b/d29/f71 x:0 0 0 2024-03-29T04:45:12.080 INFO:tasks.workunit.client.0.smithi142.stdout:0/894: readlink d7/d13/dbf/d72/l29 0 2024-03-29T04:45:12.082 INFO:tasks.workunit.client.0.smithi142.stdout:0/895: creat d7/d13/dbf/d72/f121 x:0 0 0 2024-03-29T04:45:12.085 INFO:tasks.workunit.client.0.smithi142.stdout:0/896: rmdir d7/d9/d54 39 2024-03-29T04:45:12.085 INFO:tasks.workunit.client.1.smithi152.stdout:0/297: rename d2/da/f24 to d2/da/d17/d57/f71 0 2024-03-29T04:45:12.089 INFO:tasks.workunit.client.1.smithi152.stdout:7/233: link d7/d8/db/df/f17 d7/d8/db/f42 0 2024-03-29T04:45:12.090 INFO:tasks.workunit.client.0.smithi142.stdout:0/897: symlink d7/d13/dbf/dca/dab/dc1/dd3/de1/l122 0 2024-03-29T04:45:12.091 INFO:tasks.workunit.client.1.smithi152.stdout:1/348: truncate d5/d7/d8/d13/d19/f68 669061 0 2024-03-29T04:45:12.092 INFO:tasks.workunit.client.0.smithi142.stdout:0/898: unlink d7/d9/d55/d42/f74 0 2024-03-29T04:45:12.095 INFO:tasks.workunit.client.0.smithi142.stdout:0/899: write d7/d13/dbf/d72/d37/f83 [522023,52570] 0 2024-03-29T04:45:12.098 INFO:tasks.workunit.client.0.smithi142.stdout:7/944: dwrite d7/d27/d3a/d92/d97/ff2 [0,4194304] 0 2024-03-29T04:45:12.098 INFO:tasks.workunit.client.0.smithi142.stdout:0/900: write d7/d13/dbf/dca/dab/dc1/fce [733783,79014] 0 2024-03-29T04:45:12.099 INFO:tasks.workunit.client.1.smithi152.stdout:5/282: link d6/d1b/l1a d6/d1d/d3b/l62 0 2024-03-29T04:45:12.100 INFO:tasks.workunit.client.0.smithi142.stdout:7/945: truncate d7/d27/fa8 1125540 0 2024-03-29T04:45:12.100 INFO:tasks.workunit.client.1.smithi152.stdout:8/220: dwrite d1/f26 [0,4194304] 0 2024-03-29T04:45:12.106 INFO:tasks.workunit.client.1.smithi152.stdout:7/234: rmdir d7/d8/db 39 2024-03-29T04:45:12.106 INFO:tasks.workunit.client.1.smithi152.stdout:5/283: write d6/d1d/d3b/d55/f11 [3516944,117741] 0 2024-03-29T04:45:12.106 INFO:tasks.workunit.client.1.smithi152.stdout:6/325: dread d1/d3/d9/f21 [0,4194304] 0 2024-03-29T04:45:12.107 INFO:tasks.workunit.client.0.smithi142.stdout:0/901: rename d7/d13/dbf/dca/dab/dc1/dd3 to d7/d9/d1e/d123 0 2024-03-29T04:45:12.109 INFO:tasks.workunit.client.1.smithi152.stdout:0/298: mknod d2/da/d3a/d6f/c72 0 2024-03-29T04:45:12.112 INFO:tasks.workunit.client.1.smithi152.stdout:3/279: unlink d7/f18 0 2024-03-29T04:45:12.112 INFO:tasks.workunit.client.1.smithi152.stdout:0/299: fsync d2/f6b 0 2024-03-29T04:45:12.113 INFO:tasks.workunit.client.1.smithi152.stdout:1/349: write d5/d7/d8/d13/d19/f65 [995624,102671] 0 2024-03-29T04:45:12.114 INFO:tasks.workunit.client.1.smithi152.stdout:8/221: unlink d1/d2/d1d/d36/f48 0 2024-03-29T04:45:12.115 INFO:tasks.workunit.client.1.smithi152.stdout:8/222: stat d1/d21/c3e 0 2024-03-29T04:45:12.116 INFO:tasks.workunit.client.1.smithi152.stdout:6/326: unlink d1/d3/d1a/d49/f5c 0 2024-03-29T04:45:12.117 INFO:tasks.workunit.client.1.smithi152.stdout:4/240: sync 2024-03-29T04:45:12.118 INFO:tasks.workunit.client.0.smithi142.stdout:0/902: read d7/d13/f9c [2628263,113288] 0 2024-03-29T04:45:12.118 INFO:tasks.workunit.client.1.smithi152.stdout:0/300: creat d2/da/d17/d57/f73 x:0 0 0 2024-03-29T04:45:12.119 INFO:tasks.workunit.client.1.smithi152.stdout:1/350: symlink d5/d7/d4d/l6d 0 2024-03-29T04:45:12.120 INFO:tasks.workunit.client.1.smithi152.stdout:7/235: dwrite d7/d8/f2d [0,4194304] 0 2024-03-29T04:45:12.121 INFO:tasks.workunit.client.1.smithi152.stdout:3/280: truncate d7/d19/d3a/f49 2785496 0 2024-03-29T04:45:12.121 INFO:tasks.workunit.client.1.smithi152.stdout:8/223: getdents d1/d21/d40/d47 0 2024-03-29T04:45:12.121 INFO:tasks.workunit.client.0.smithi142.stdout:0/903: rename d7/d9/lc9 to d7/d9/d6e/l124 0 2024-03-29T04:45:12.122 INFO:tasks.workunit.client.1.smithi152.stdout:0/301: fsync d2/da/d17/f4c 0 2024-03-29T04:45:12.127 INFO:tasks.workunit.client.0.smithi142.stdout:0/904: creat d7/d9/d54/f125 x:0 0 0 2024-03-29T04:45:12.127 INFO:tasks.workunit.client.1.smithi152.stdout:8/224: fdatasync d1/d2/d1d/f3a 0 2024-03-29T04:45:12.127 INFO:tasks.workunit.client.0.smithi142.stdout:0/905: rename d7/d13 to d7/d13/dbf/dca/d126 22 2024-03-29T04:45:12.129 INFO:tasks.workunit.client.1.smithi152.stdout:0/302: write d2/da/d17/f3d [1016794,788] 0 2024-03-29T04:45:12.129 INFO:tasks.workunit.client.1.smithi152.stdout:3/281: rmdir d7/d19/d1b/d1e 39 2024-03-29T04:45:12.130 INFO:tasks.workunit.client.1.smithi152.stdout:4/241: mknod d0/d7/d25/d17/d37/c43 0 2024-03-29T04:45:12.131 INFO:tasks.workunit.client.1.smithi152.stdout:8/225: stat d1/d2/d1d/f3d 0 2024-03-29T04:45:12.132 INFO:tasks.workunit.client.0.smithi142.stdout:0/906: creat d7/f127 x:0 0 0 2024-03-29T04:45:12.135 INFO:tasks.workunit.client.0.smithi142.stdout:7/946: dwrite d7/d27/d3a/f12c [0,4194304] 0 2024-03-29T04:45:12.135 INFO:tasks.workunit.client.1.smithi152.stdout:6/327: dwrite d1/d3/d1a/d1b/d29/f71 [0,4194304] 0 2024-03-29T04:45:12.137 INFO:tasks.workunit.client.1.smithi152.stdout:1/351: rename d5/d7/d9/d2b/f63 to d5/d7/d6a/f6e 0 2024-03-29T04:45:12.139 INFO:tasks.workunit.client.0.smithi142.stdout:0/907: rename d7/df6 to d7/d13/dbf/dca/d101/d128 0 2024-03-29T04:45:12.141 INFO:tasks.workunit.client.1.smithi152.stdout:6/328: write d1/d3/d1a/f2c [395992,122358] 0 2024-03-29T04:45:12.144 INFO:tasks.workunit.client.1.smithi152.stdout:8/226: symlink d1/d2/d1d/d36/d4b/l4d 0 2024-03-29T04:45:12.147 INFO:tasks.workunit.client.1.smithi152.stdout:0/303: dwrite d2/d29/f37 [0,4194304] 0 2024-03-29T04:45:12.153 INFO:tasks.workunit.client.1.smithi152.stdout:5/284: dread d6/d7/f8 [0,4194304] 0 2024-03-29T04:45:12.153 INFO:tasks.workunit.client.1.smithi152.stdout:1/352: rename d5/d7/f1f to d5/d7/d38/f6f 0 2024-03-29T04:45:12.154 INFO:tasks.workunit.client.1.smithi152.stdout:5/285: chown d6/d1d/d30/f46 28872439 1 2024-03-29T04:45:12.154 INFO:tasks.workunit.client.1.smithi152.stdout:8/227: creat d1/d2/d3/f4e x:0 0 0 2024-03-29T04:45:12.155 INFO:tasks.workunit.client.1.smithi152.stdout:6/329: unlink d1/d3/d1a/d1b/d29/d6f/f63 0 2024-03-29T04:45:12.156 INFO:tasks.workunit.client.1.smithi152.stdout:0/304: fsync d2/da/df/f5e 0 2024-03-29T04:45:12.156 INFO:tasks.workunit.client.1.smithi152.stdout:1/353: creat d5/d7/d1b/f70 x:0 0 0 2024-03-29T04:45:12.157 INFO:tasks.workunit.client.1.smithi152.stdout:5/286: write d6/d1d/d3b/d55/f60 [238822,21858] 0 2024-03-29T04:45:12.158 INFO:tasks.workunit.client.1.smithi152.stdout:6/330: write d1/d3/d9/f3a [813455,49986] 0 2024-03-29T04:45:12.158 INFO:tasks.workunit.client.1.smithi152.stdout:0/305: stat d2/da/d17/d57/f62 0 2024-03-29T04:45:12.159 INFO:tasks.workunit.client.1.smithi152.stdout:4/242: getdents d0/d7/d25/d17/d37 0 2024-03-29T04:45:12.163 INFO:tasks.workunit.client.1.smithi152.stdout:5/287: dread d6/d1d/d30/f3e [0,4194304] 0 2024-03-29T04:45:12.163 INFO:tasks.workunit.client.1.smithi152.stdout:4/243: mknod d0/d7/d40/d3a/c44 0 2024-03-29T04:45:12.167 INFO:tasks.workunit.client.1.smithi152.stdout:1/354: dwrite d5/d7/d9/d2b/f2f [0,4194304] 0 2024-03-29T04:45:12.168 INFO:tasks.workunit.client.1.smithi152.stdout:0/306: link d2/d31/f35 d2/da/d17/f74 0 2024-03-29T04:45:12.170 INFO:tasks.workunit.client.1.smithi152.stdout:6/331: link d1/d3/d9/d13/d3e/l4b d1/d3/d1a/d1b/d29/l72 0 2024-03-29T04:45:12.170 INFO:tasks.workunit.client.1.smithi152.stdout:0/307: write d2/da/d17/d57/f64 [64339,11985] 0 2024-03-29T04:45:12.173 INFO:tasks.workunit.client.1.smithi152.stdout:5/288: getdents d6/d16/d47 0 2024-03-29T04:45:12.175 INFO:tasks.workunit.client.1.smithi152.stdout:7/236: dread d7/d8/d12/f20 [0,4194304] 0 2024-03-29T04:45:12.176 INFO:tasks.workunit.client.1.smithi152.stdout:5/289: read d6/d7/f8 [1963887,61479] 0 2024-03-29T04:45:12.177 INFO:tasks.workunit.client.1.smithi152.stdout:3/282: truncate d7/d19/d1b/f21 977744 0 2024-03-29T04:45:12.177 INFO:tasks.workunit.client.0.smithi142.stdout:0/908: dread d7/d13/dbf/d72/fd6 [0,4194304] 0 2024-03-29T04:45:12.178 INFO:tasks.workunit.client.1.smithi152.stdout:7/237: dread - d7/d8/d12/d35/f32 zero size 2024-03-29T04:45:12.179 INFO:tasks.workunit.client.0.smithi142.stdout:0/909: readlink d7/d9/d55/l4f 0 2024-03-29T04:45:12.180 INFO:tasks.workunit.client.1.smithi152.stdout:0/308: rename d2/d29/c48 to d2/da/d17/d50/c75 0 2024-03-29T04:45:12.180 INFO:tasks.workunit.client.1.smithi152.stdout:1/355: link d5/d7/d9/c43 d5/d7/c71 0 2024-03-29T04:45:12.181 INFO:tasks.workunit.client.0.smithi142.stdout:7/947: dwrite d7/d27/d44/db2/db8/d89/fdc [0,4194304] 0 2024-03-29T04:45:12.182 INFO:tasks.workunit.client.1.smithi152.stdout:6/332: dwrite d1/d3/d1a/d1b/d29/f71 [0,4194304] 0 2024-03-29T04:45:12.183 INFO:tasks.workunit.client.1.smithi152.stdout:6/333: readlink d1/d3/l6c 0 2024-03-29T04:45:12.184 INFO:tasks.workunit.client.0.smithi142.stdout:0/910: creat d7/d13/dbf/f129 x:0 0 0 2024-03-29T04:45:12.184 INFO:tasks.workunit.client.1.smithi152.stdout:3/283: rename d7/d1c/d41/l48 to d7/d1c/l4b 0 2024-03-29T04:45:12.185 INFO:tasks.workunit.client.0.smithi142.stdout:7/948: mknod d7/d27/d44/db2/db8/d4b/d71/d30/d33/d137/c13a 0 2024-03-29T04:45:12.189 INFO:tasks.workunit.client.1.smithi152.stdout:0/309: symlink d2/da/d3a/d61/l76 0 2024-03-29T04:45:12.189 INFO:tasks.workunit.client.1.smithi152.stdout:7/238: chown d7/d8/db/df/c16 334 1 2024-03-29T04:45:12.190 INFO:tasks.workunit.client.1.smithi152.stdout:7/239: chown d7/d8/db/f42 51 1 2024-03-29T04:45:12.192 INFO:tasks.workunit.client.0.smithi142.stdout:7/949: creat d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/d3f/f13b x:0 0 0 2024-03-29T04:45:12.192 INFO:tasks.workunit.client.1.smithi152.stdout:5/290: mkdir d6/d16/d47/d56/d63 0 2024-03-29T04:45:12.193 INFO:tasks.workunit.client.1.smithi152.stdout:6/334: unlink d1/d3/c62 0 2024-03-29T04:45:12.194 INFO:tasks.workunit.client.0.smithi142.stdout:0/911: dwrite d7/d13/dbf/dca/dab/ddf/f115 [0,4194304] 0 2024-03-29T04:45:12.194 INFO:tasks.workunit.client.1.smithi152.stdout:4/244: dread d0/d7/d25/f3f [0,4194304] 0 2024-03-29T04:45:12.194 INFO:tasks.workunit.client.1.smithi152.stdout:5/291: truncate d6/d16/d25/f3a 371978 0 2024-03-29T04:45:12.195 INFO:tasks.workunit.client.0.smithi142.stdout:0/912: fdatasync d7/d13/dbf/d72/d37/f56 0 2024-03-29T04:45:12.197 INFO:tasks.workunit.client.1.smithi152.stdout:0/310: chown d2/f20 1106 1 2024-03-29T04:45:12.199 INFO:tasks.workunit.client.0.smithi142.stdout:0/913: mknod d7/d9/d1e/c12a 0 2024-03-29T04:45:12.200 INFO:tasks.workunit.client.0.smithi142.stdout:4/811: sync 2024-03-29T04:45:12.202 INFO:tasks.workunit.client.1.smithi152.stdout:5/292: dwrite d6/d1d/d3b/d55/f60 [0,4194304] 0 2024-03-29T04:45:12.205 INFO:tasks.workunit.client.1.smithi152.stdout:7/240: rmdir d7/d8/d12/d15 39 2024-03-29T04:45:12.207 INFO:tasks.workunit.client.1.smithi152.stdout:7/241: read - d7/d8/d12/d1d/d30/f3c zero size 2024-03-29T04:45:12.208 INFO:tasks.workunit.client.0.smithi142.stdout:7/950: rename d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/d86/dd4/ddb/df6/f102 to d7/d27/d44/db2/db8/d4b/d71/d30/dd9/f13c 0 2024-03-29T04:45:12.211 INFO:tasks.workunit.client.1.smithi152.stdout:4/245: mknod d0/d7/d40/d3a/c45 0 2024-03-29T04:45:12.212 INFO:tasks.workunit.client.0.smithi142.stdout:0/914: getdents d7/d13/dbf/dca/d101 0 2024-03-29T04:45:12.212 INFO:tasks.workunit.client.1.smithi152.stdout:1/356: creat d5/f72 x:0 0 0 2024-03-29T04:45:12.213 INFO:tasks.workunit.client.0.smithi142.stdout:7/951: write d7/d27/d44/db2/db8/d4b/d71/d30/d33/f38 [1992768,16844] 0 2024-03-29T04:45:12.213 INFO:tasks.workunit.client.0.smithi142.stdout:4/812: link d0/d9/db2/dba/fc0 d0/d9/d11/d84/f10a 0 2024-03-29T04:45:12.214 INFO:tasks.workunit.client.1.smithi152.stdout:3/284: truncate d7/f10 2785728 0 2024-03-29T04:45:12.215 INFO:tasks.workunit.client.1.smithi152.stdout:7/242: stat d7/d8/d12/f33 0 2024-03-29T04:45:12.216 INFO:tasks.workunit.client.0.smithi142.stdout:4/813: symlink d0/d45/l10b 0 2024-03-29T04:45:12.217 INFO:tasks.workunit.client.1.smithi152.stdout:7/243: stat d7/d8/d12/d1d/d30 0 2024-03-29T04:45:12.218 INFO:tasks.workunit.client.0.smithi142.stdout:7/952: unlink d7/d27/d44/db2/db8/d4b/d71/d30/d72/db3/le8 0 2024-03-29T04:45:12.219 INFO:tasks.workunit.client.1.smithi152.stdout:5/293: truncate d6/d7/f2d 2894703 0 2024-03-29T04:45:12.219 INFO:tasks.workunit.client.1.smithi152.stdout:3/285: mknod d7/d46/c4c 0 2024-03-29T04:45:12.219 INFO:tasks.workunit.client.0.smithi142.stdout:0/915: getdents d7 0 2024-03-29T04:45:12.220 INFO:tasks.workunit.client.1.smithi152.stdout:0/311: creat d2/da/d17/f77 x:0 0 0 2024-03-29T04:45:12.220 INFO:tasks.workunit.client.1.smithi152.stdout:1/357: dread d5/d7/f54 [0,4194304] 0 2024-03-29T04:45:12.221 INFO:tasks.workunit.client.1.smithi152.stdout:3/286: stat d7/d19/d1b/c2e 0 2024-03-29T04:45:12.222 INFO:tasks.workunit.client.1.smithi152.stdout:0/312: chown d2/da/d17/f6d 0 1 2024-03-29T04:45:12.222 INFO:tasks.workunit.client.1.smithi152.stdout:1/358: chown d5/d7/d38/d6c 30914032 1 2024-03-29T04:45:12.222 INFO:tasks.workunit.client.0.smithi142.stdout:4/814: truncate d0/d9/d11/d97/fe1 4075579 0 2024-03-29T04:45:12.223 INFO:tasks.workunit.client.1.smithi152.stdout:0/313: dread - d2/d29/f6c zero size 2024-03-29T04:45:12.224 INFO:tasks.workunit.client.1.smithi152.stdout:4/246: creat d0/d7/d25/f46 x:0 0 0 2024-03-29T04:45:12.226 INFO:tasks.workunit.client.1.smithi152.stdout:1/359: fdatasync d5/d7/d8/d13/d19/f65 0 2024-03-29T04:45:12.226 INFO:tasks.workunit.client.0.smithi142.stdout:0/916: rename d7/d13/dbf/f129 to d7/d13/dbf/dca/dab/f12b 0 2024-03-29T04:45:12.227 INFO:tasks.workunit.client.0.smithi142.stdout:0/917: chown d7 48119187 1 2024-03-29T04:45:12.228 INFO:tasks.workunit.client.1.smithi152.stdout:6/335: dread d1/d3/d9/d13/f33 [0,4194304] 0 2024-03-29T04:45:12.228 INFO:tasks.workunit.client.1.smithi152.stdout:3/287: chown d7/l13 1606 1 2024-03-29T04:45:12.230 INFO:tasks.workunit.client.1.smithi152.stdout:4/247: creat d0/d7/d40/d3a/f47 x:0 0 0 2024-03-29T04:45:12.231 INFO:tasks.workunit.client.1.smithi152.stdout:5/294: dread d6/d1d/d3b/f3f [0,4194304] 0 2024-03-29T04:45:12.232 INFO:tasks.workunit.client.1.smithi152.stdout:1/360: creat d5/d7/d9/f73 x:0 0 0 2024-03-29T04:45:12.233 INFO:tasks.workunit.client.0.smithi142.stdout:0/918: rename d7/d9/d1e/d81/ce6 to d7/d13/dbf/d72/d116/c12c 0 2024-03-29T04:45:12.235 INFO:tasks.workunit.client.1.smithi152.stdout:6/336: stat d1/d3/d1a/d1b/d29/l3f 0 2024-03-29T04:45:12.235 INFO:tasks.workunit.client.0.smithi142.stdout:0/919: read - d7/d13/dbf/d72/d37/faa zero size 2024-03-29T04:45:12.237 INFO:tasks.workunit.client.0.smithi142.stdout:7/953: dread d7/d62/f114 [0,4194304] 0 2024-03-29T04:45:12.238 INFO:tasks.workunit.client.0.smithi142.stdout:4/815: write d0/d9/d11/d4c/d5f/d81/dc9/f3a [188714,6177] 0 2024-03-29T04:45:12.239 INFO:tasks.workunit.client.1.smithi152.stdout:0/314: creat d2/f78 x:0 0 0 2024-03-29T04:45:12.240 INFO:tasks.workunit.client.0.smithi142.stdout:4/816: chown d0/dc/d15/d3d/d65/f95 5527927 1 2024-03-29T04:45:12.240 INFO:tasks.workunit.client.1.smithi152.stdout:5/295: dread d6/d1b/f31 [0,4194304] 0 2024-03-29T04:45:12.241 INFO:tasks.workunit.client.1.smithi152.stdout:5/296: fsync d6/d16/d25/d2e/f54 0 2024-03-29T04:45:12.241 INFO:tasks.workunit.client.1.smithi152.stdout:1/361: creat d5/d7/d8/d13/f74 x:0 0 0 2024-03-29T04:45:12.242 INFO:tasks.workunit.client.1.smithi152.stdout:5/297: fsync d6/d16/d25/d2e/f54 0 2024-03-29T04:45:12.242 INFO:tasks.workunit.client.1.smithi152.stdout:3/288: creat d7/d19/d1b/d1e/d22/f4d x:0 0 0 2024-03-29T04:45:12.243 INFO:tasks.workunit.client.0.smithi142.stdout:7/954: fdatasync d7/d27/d44/db2/db8/d4b/d71/f84 0 2024-03-29T04:45:12.243 INFO:tasks.workunit.client.1.smithi152.stdout:3/289: stat d7/d1c 0 2024-03-29T04:45:12.244 INFO:tasks.workunit.client.0.smithi142.stdout:4/817: creat d0/d9/d11/d4c/d64/d94/f10c x:0 0 0 2024-03-29T04:45:12.244 INFO:tasks.workunit.client.1.smithi152.stdout:5/298: write d6/d1d/d3b/d55/f17 [4001692,61855] 0 2024-03-29T04:45:12.244 INFO:tasks.workunit.client.0.smithi142.stdout:0/920: write d7/d13/dbf/d72/fc7 [126246,51537] 0 2024-03-29T04:45:12.245 INFO:tasks.workunit.client.0.smithi142.stdout:4/818: chown d0/dc/d15/d3d/f43 404430 1 2024-03-29T04:45:12.247 INFO:tasks.workunit.client.1.smithi152.stdout:4/248: mkdir d0/d7/d25/d48 0 2024-03-29T04:45:12.248 INFO:tasks.workunit.client.1.smithi152.stdout:4/249: dread - d0/d7/d40/d2a/f42 zero size 2024-03-29T04:45:12.249 INFO:tasks.workunit.client.1.smithi152.stdout:7/244: link d7/d8/d12/d1d/d41/f2c d7/d8/d12/f43 0 2024-03-29T04:45:12.250 INFO:tasks.workunit.client.0.smithi142.stdout:0/921: getdents d7/d13/dbf/d72/d37 0 2024-03-29T04:45:12.251 INFO:tasks.workunit.client.1.smithi152.stdout:5/299: creat d6/d16/d5c/f64 x:0 0 0 2024-03-29T04:45:12.252 INFO:tasks.workunit.client.0.smithi142.stdout:4/819: dwrite d0/d9/d11/d4c/d5f/d81/ff0 [0,4194304] 0 2024-03-29T04:45:12.254 INFO:tasks.workunit.client.0.smithi142.stdout:0/922: rename d7/d9/fbb to d7/d9/d1e/f12d 0 2024-03-29T04:45:12.255 INFO:tasks.workunit.client.0.smithi142.stdout:4/820: creat d0/d9/d19/d2c/f10d x:0 0 0 2024-03-29T04:45:12.256 INFO:tasks.workunit.client.1.smithi152.stdout:0/315: rename d2/da/d3a/l4f to d2/da/d3a/l79 0 2024-03-29T04:45:12.257 INFO:tasks.workunit.client.1.smithi152.stdout:7/245: mknod d7/d8/d12/d35/c44 0 2024-03-29T04:45:12.258 INFO:tasks.workunit.client.1.smithi152.stdout:6/337: link d1/d3/d1a/d1b/d29/l3f d1/d3/d9/d13/d28/d45/l73 0 2024-03-29T04:45:12.259 INFO:tasks.workunit.client.0.smithi142.stdout:4/821: rename d0/d12/f5d to d0/d9/d19/d2c/f10e 0 2024-03-29T04:45:12.259 INFO:tasks.workunit.client.1.smithi152.stdout:4/250: symlink d0/d7/d25/l49 0 2024-03-29T04:45:12.260 INFO:tasks.workunit.client.1.smithi152.stdout:1/362: getdents d5/d7/d38 0 2024-03-29T04:45:12.261 INFO:tasks.workunit.client.1.smithi152.stdout:7/246: mkdir d7/d8/d12/d1d/d41/d45 0 2024-03-29T04:45:12.266 INFO:tasks.workunit.client.1.smithi152.stdout:4/251: stat d0/l31 0 2024-03-29T04:45:12.266 INFO:tasks.workunit.client.1.smithi152.stdout:0/316: creat d2/f7a x:0 0 0 2024-03-29T04:45:12.269 INFO:tasks.workunit.client.1.smithi152.stdout:0/317: fdatasync d2/da/d17/d57/f64 0 2024-03-29T04:45:12.270 INFO:tasks.workunit.client.1.smithi152.stdout:5/300: dwrite d6/d16/d47/f61 [0,4194304] 0 2024-03-29T04:45:12.273 INFO:tasks.workunit.client.1.smithi152.stdout:7/247: dwrite d7/f14 [0,4194304] 0 2024-03-29T04:45:12.277 INFO:tasks.workunit.client.1.smithi152.stdout:5/301: creat d6/d16/f65 x:0 0 0 2024-03-29T04:45:12.279 INFO:tasks.workunit.client.1.smithi152.stdout:7/248: chown d7/d8/db/df/f2b 7 1 2024-03-29T04:45:12.282 INFO:tasks.workunit.client.0.smithi142.stdout:4/822: dwrite d0/d9/db2/dba/fe6 [0,4194304] 0 2024-03-29T04:45:12.282 INFO:tasks.workunit.client.1.smithi152.stdout:4/252: dwrite d0/d7/d25/f21 [0,4194304] 0 2024-03-29T04:45:12.284 INFO:tasks.workunit.client.0.smithi142.stdout:4/823: stat d0/d9/d11/d4c/d5f/d81/dc9/d47/d7a/fc8 0 2024-03-29T04:45:12.289 INFO:tasks.workunit.client.1.smithi152.stdout:7/249: write d7/d8/d12/f20 [4111619,76964] 0 2024-03-29T04:45:12.290 INFO:tasks.workunit.client.0.smithi142.stdout:4/824: creat d0/d9/d11/d4c/d5f/d81/dc9/d47/dea/f10f x:0 0 0 2024-03-29T04:45:12.291 INFO:tasks.workunit.client.1.smithi152.stdout:6/338: write d1/d3/d9/d13/f1f [4381280,31573] 0 2024-03-29T04:45:12.292 INFO:tasks.workunit.client.1.smithi152.stdout:0/318: dwrite d2/d29/f6c [0,4194304] 0 2024-03-29T04:45:12.293 INFO:tasks.workunit.client.1.smithi152.stdout:1/363: truncate d5/d7/d9/d2b/f2f 1061755 0 2024-03-29T04:45:12.295 INFO:tasks.workunit.client.1.smithi152.stdout:7/250: stat d7/d8/d12/f20 0 2024-03-29T04:45:12.295 INFO:tasks.workunit.client.1.smithi152.stdout:4/253: rename d0/d7/f3c to d0/d7/d40/d2a/f4a 0 2024-03-29T04:45:12.298 INFO:tasks.workunit.client.1.smithi152.stdout:5/302: dwrite d6/f21 [0,4194304] 0 2024-03-29T04:45:12.298 INFO:tasks.workunit.client.1.smithi152.stdout:6/339: mknod d1/d3/d9/d13/d28/d45/c74 0 2024-03-29T04:45:12.302 INFO:tasks.workunit.client.1.smithi152.stdout:0/319: truncate d2/d29/f2f 633208 0 2024-03-29T04:45:12.303 INFO:tasks.workunit.client.0.smithi142.stdout:4/825: dwrite d0/d9/db2/dba/de9/ff8 [0,4194304] 0 2024-03-29T04:45:12.304 INFO:tasks.workunit.client.1.smithi152.stdout:0/320: fdatasync d2/da/d3a/f68 0 2024-03-29T04:45:12.305 INFO:tasks.workunit.client.1.smithi152.stdout:7/251: dread d7/d8/d12/d1d/f34 [0,4194304] 0 2024-03-29T04:45:12.306 INFO:tasks.workunit.client.1.smithi152.stdout:0/321: read d2/da/d17/f3d [428142,15626] 0 2024-03-29T04:45:12.307 INFO:tasks.workunit.client.0.smithi142.stdout:4/826: link d0/d9/d19/la9 d0/d9/d11/d78/l110 0 2024-03-29T04:45:12.307 INFO:tasks.workunit.client.1.smithi152.stdout:6/340: rmdir d1/d3/d9 39 2024-03-29T04:45:12.308 INFO:tasks.workunit.client.1.smithi152.stdout:7/252: truncate d7/d8/d12/f28 817548 0 2024-03-29T04:45:12.309 INFO:tasks.workunit.client.1.smithi152.stdout:0/322: write d2/da/d17/d57/f62 [320184,60210] 0 2024-03-29T04:45:12.310 INFO:tasks.workunit.client.1.smithi152.stdout:5/303: dread d6/d1d/d3b/d55/f11 [0,4194304] 0 2024-03-29T04:45:12.312 INFO:tasks.workunit.client.0.smithi142.stdout:4/827: link d0/d9/db2/ffe d0/d9/d11/d4c/d5f/d81/f111 0 2024-03-29T04:45:12.313 INFO:tasks.workunit.client.1.smithi152.stdout:7/253: creat d7/d8/d37/f46 x:0 0 0 2024-03-29T04:45:12.314 INFO:tasks.workunit.client.0.smithi142.stdout:4/828: symlink d0/d9/d11/d4c/l112 0 2024-03-29T04:45:12.316 INFO:tasks.workunit.client.1.smithi152.stdout:5/304: fsync d6/d7/f8 0 2024-03-29T04:45:12.317 INFO:tasks.workunit.client.1.smithi152.stdout:7/254: mknod d7/c47 0 2024-03-29T04:45:12.318 INFO:tasks.workunit.client.1.smithi152.stdout:5/305: creat d6/d1d/d3b/d55/f66 x:0 0 0 2024-03-29T04:45:12.319 INFO:tasks.workunit.client.1.smithi152.stdout:7/255: symlink d7/d8/d12/d1d/d41/l48 0 2024-03-29T04:45:12.319 INFO:tasks.workunit.client.0.smithi142.stdout:4/829: link d0/dc/d15/d3d/d65/c7e d0/d9/d11/c113 0 2024-03-29T04:45:12.319 INFO:tasks.workunit.client.1.smithi152.stdout:5/306: dread - d6/d16/f5b zero size 2024-03-29T04:45:12.322 INFO:tasks.workunit.client.1.smithi152.stdout:5/307: mknod d6/d16/d25/d2e/c67 0 2024-03-29T04:45:12.326 INFO:tasks.workunit.client.1.smithi152.stdout:4/254: rmdir d0/d7 39 2024-03-29T04:45:12.330 INFO:tasks.workunit.client.0.smithi142.stdout:4/830: dwrite d0/dc/f36 [0,4194304] 0 2024-03-29T04:45:12.331 INFO:tasks.workunit.client.1.smithi152.stdout:4/255: symlink d0/d7/d25/d48/l4b 0 2024-03-29T04:45:12.333 INFO:tasks.workunit.client.1.smithi152.stdout:4/256: creat d0/d7/d25/d17/f4c x:0 0 0 2024-03-29T04:45:12.335 INFO:tasks.workunit.client.1.smithi152.stdout:4/257: dread - d0/d7/d25/d17/f3b zero size 2024-03-29T04:45:12.335 INFO:tasks.workunit.client.0.smithi142.stdout:4/831: write d0/d9/d11/f25 [1158429,68605] 0 2024-03-29T04:45:12.336 INFO:tasks.workunit.client.1.smithi152.stdout:4/258: stat d0/d7/d40/d2a 0 2024-03-29T04:45:12.337 INFO:tasks.workunit.client.0.smithi142.stdout:4/832: fsync d0/d9/d11/d4c/d5f/d81/dc9/d47/fda 0 2024-03-29T04:45:12.339 INFO:tasks.workunit.client.1.smithi152.stdout:4/259: creat d0/d7/d25/d17/f4d x:0 0 0 2024-03-29T04:45:12.341 INFO:tasks.workunit.client.1.smithi152.stdout:4/260: rmdir d0/d7/d40 39 2024-03-29T04:45:12.342 INFO:tasks.workunit.client.1.smithi152.stdout:0/323: dread d2/d31/f35 [0,4194304] 0 2024-03-29T04:45:12.343 INFO:tasks.workunit.client.1.smithi152.stdout:4/261: write d0/d7/d25/d17/f3b [66284,120181] 0 2024-03-29T04:45:12.344 INFO:tasks.workunit.client.1.smithi152.stdout:7/256: write d7/f10 [201002,41403] 0 2024-03-29T04:45:12.345 INFO:tasks.workunit.client.1.smithi152.stdout:1/364: write d5/d7/d9/d2b/f2f [366163,49894] 0 2024-03-29T04:45:12.346 INFO:tasks.workunit.client.1.smithi152.stdout:7/257: chown d7/f14 2 1 2024-03-29T04:45:12.351 INFO:tasks.workunit.client.1.smithi152.stdout:0/324: creat d2/da/d17/d50/d5c/f7b x:0 0 0 2024-03-29T04:45:12.353 INFO:tasks.workunit.client.1.smithi152.stdout:7/258: write d7/d8/d12/f22 [629328,43277] 0 2024-03-29T04:45:12.353 INFO:tasks.workunit.client.1.smithi152.stdout:4/262: symlink d0/d7/d25/d17/l4e 0 2024-03-29T04:45:12.356 INFO:tasks.workunit.client.1.smithi152.stdout:6/341: dwrite d1/d3/d9/f27 [0,4194304] 0 2024-03-29T04:45:12.358 INFO:tasks.workunit.client.1.smithi152.stdout:0/325: mkdir d2/d31/d7c 0 2024-03-29T04:45:12.359 INFO:tasks.workunit.client.1.smithi152.stdout:1/365: mknod d5/c75 0 2024-03-29T04:45:12.362 INFO:tasks.workunit.client.1.smithi152.stdout:4/263: creat d0/d7/d25/d17/d37/f4f x:0 0 0 2024-03-29T04:45:12.363 INFO:tasks.workunit.client.1.smithi152.stdout:5/308: dwrite d6/d1d/f2c [0,4194304] 0 2024-03-29T04:45:12.364 INFO:tasks.workunit.client.0.smithi142.stdout:9/724: sync 2024-03-29T04:45:12.365 INFO:tasks.workunit.client.1.smithi152.stdout:0/326: symlink d2/da/d17/d50/l7d 0 2024-03-29T04:45:12.368 INFO:tasks.workunit.client.1.smithi152.stdout:1/366: truncate d5/d7/d1b/d2e/f5f 360821 0 2024-03-29T04:45:12.368 INFO:tasks.workunit.client.1.smithi152.stdout:6/342: fdatasync d1/d3/d1a/d1b/d29/f5f 0 2024-03-29T04:45:12.368 INFO:tasks.workunit.client.1.smithi152.stdout:7/259: rename d7/d8/d12/d1d/f29 to d7/d8/d12/d1d/f49 0 2024-03-29T04:45:12.369 INFO:tasks.workunit.client.0.smithi142.stdout:9/725: read d0/d2/d6/d27/f82 [1227441,58300] 0 2024-03-29T04:45:12.369 INFO:tasks.workunit.client.1.smithi152.stdout:4/264: creat d0/d7/d25/d48/f50 x:0 0 0 2024-03-29T04:45:12.370 INFO:tasks.workunit.client.1.smithi152.stdout:6/343: write d1/d3/d9/d13/d28/f34 [3812240,17821] 0 2024-03-29T04:45:12.372 INFO:tasks.workunit.client.1.smithi152.stdout:1/367: symlink d5/d7/d6a/l76 0 2024-03-29T04:45:12.373 INFO:tasks.workunit.client.1.smithi152.stdout:7/260: truncate d7/d8/d12/d35/f32 825671 0 2024-03-29T04:45:12.373 INFO:tasks.workunit.client.1.smithi152.stdout:0/327: symlink d2/da/d17/d57/l7e 0 2024-03-29T04:45:12.374 INFO:tasks.workunit.client.1.smithi152.stdout:5/309: getdents d6/d1d 0 2024-03-29T04:45:12.374 INFO:tasks.workunit.client.0.smithi142.stdout:9/726: creat d0/d2/fe6 x:0 0 0 2024-03-29T04:45:12.376 INFO:tasks.workunit.client.1.smithi152.stdout:7/261: write d7/d8/f3b [431468,96924] 0 2024-03-29T04:45:12.376 INFO:tasks.workunit.client.1.smithi152.stdout:0/328: readlink d2/d29/l60 0 2024-03-29T04:45:12.377 INFO:tasks.workunit.client.0.smithi142.stdout:9/727: write d0/d2/d6/d27/d3c/d4f/f8c [633815,113439] 0 2024-03-29T04:45:12.377 INFO:tasks.workunit.client.1.smithi152.stdout:6/344: symlink d1/d3/l75 0 2024-03-29T04:45:12.377 INFO:tasks.workunit.client.0.smithi142.stdout:9/728: stat d0/d12/d50/d52/da3/lca 0 2024-03-29T04:45:12.378 INFO:tasks.workunit.client.0.smithi142.stdout:4/833: dread d0/dc/d15/d91/ffc [0,4194304] 0 2024-03-29T04:45:12.378 INFO:tasks.workunit.client.0.smithi142.stdout:9/729: chown d0/d12/d50/f93 230301696 1 2024-03-29T04:45:12.379 INFO:tasks.workunit.client.1.smithi152.stdout:5/310: mknod d6/d1d/c68 0 2024-03-29T04:45:12.381 INFO:tasks.workunit.client.1.smithi152.stdout:7/262: write d7/d8/d12/d1d/d30/f3c [879192,71793] 0 2024-03-29T04:45:12.381 INFO:tasks.workunit.client.1.smithi152.stdout:5/311: chown d6/d16/f5b 89 1 2024-03-29T04:45:12.381 INFO:tasks.workunit.client.1.smithi152.stdout:1/368: dwrite d5/d7/f25 [0,4194304] 0 2024-03-29T04:45:12.383 INFO:tasks.workunit.client.0.smithi142.stdout:4/834: creat d0/d9/db2/dba/dbc/d8d/da5/d102/f114 x:0 0 0 2024-03-29T04:45:12.384 INFO:tasks.workunit.client.1.smithi152.stdout:0/329: rmdir d2/d31/d7c 0 2024-03-29T04:45:12.386 INFO:tasks.workunit.client.0.smithi142.stdout:9/730: dwrite d0/d12/d50/fd5 [0,4194304] 0 2024-03-29T04:45:12.390 INFO:tasks.workunit.client.0.smithi142.stdout:4/835: rename d0/d9/d11/c57 to d0/d9/db2/dba/de9/c115 0 2024-03-29T04:45:12.392 INFO:tasks.workunit.client.1.smithi152.stdout:1/369: mkdir d5/d7/d9/d12/d77 0 2024-03-29T04:45:12.394 INFO:tasks.workunit.client.1.smithi152.stdout:1/370: chown d5/d7/d38 69 1 2024-03-29T04:45:12.394 INFO:tasks.workunit.client.1.smithi152.stdout:5/312: creat d6/d16/d5c/f69 x:0 0 0 2024-03-29T04:45:12.396 INFO:tasks.workunit.client.1.smithi152.stdout:5/313: dread - d6/f41 zero size 2024-03-29T04:45:12.397 INFO:tasks.workunit.client.0.smithi142.stdout:4/836: creat d0/d9/db2/dba/dbc/daf/f116 x:0 0 0 2024-03-29T04:45:12.399 INFO:tasks.workunit.client.1.smithi152.stdout:1/371: unlink d5/d7/d9/f11 0 2024-03-29T04:45:12.400 INFO:tasks.workunit.client.1.smithi152.stdout:6/345: dwrite d1/d3/d9/d13/f33 [4194304,4194304] 0 2024-03-29T04:45:12.402 INFO:tasks.workunit.client.1.smithi152.stdout:7/263: dwrite d7/d8/d12/d35/f1a [0,4194304] 0 2024-03-29T04:45:12.402 INFO:tasks.workunit.client.1.smithi152.stdout:1/372: write d5/f3f [373063,123092] 0 2024-03-29T04:45:12.403 INFO:tasks.workunit.client.1.smithi152.stdout:1/373: fsync d5/d7/d9/f73 0 2024-03-29T04:45:12.405 INFO:tasks.workunit.client.1.smithi152.stdout:6/346: symlink d1/d3/d9/d13/d28/l76 0 2024-03-29T04:45:12.408 INFO:tasks.workunit.client.0.smithi142.stdout:9/731: rmdir d0 39 2024-03-29T04:45:12.411 INFO:tasks.workunit.client.0.smithi142.stdout:9/732: write d0/d12/d50/f76 [1659139,7324] 0 2024-03-29T04:45:12.412 INFO:tasks.workunit.client.0.smithi142.stdout:9/733: dread - d0/d2/d61/fc3 zero size 2024-03-29T04:45:12.413 INFO:tasks.workunit.client.1.smithi152.stdout:5/314: dwrite d6/d1d/d30/f46 [0,4194304] 0 2024-03-29T04:45:12.414 INFO:tasks.workunit.client.0.smithi142.stdout:9/734: dread - d0/d12/f86 zero size 2024-03-29T04:45:12.415 INFO:tasks.workunit.client.1.smithi152.stdout:7/264: dread d7/d8/db/df/f13 [0,4194304] 0 2024-03-29T04:45:12.417 INFO:tasks.workunit.client.1.smithi152.stdout:6/347: dread d1/d3/d9/f3a [0,4194304] 0 2024-03-29T04:45:12.418 INFO:tasks.workunit.client.1.smithi152.stdout:0/330: truncate d2/da/df/f5b 1987314 0 2024-03-29T04:45:12.420 INFO:tasks.workunit.client.0.smithi142.stdout:4/837: dwrite d0/d9/d11/d84/f10a [0,4194304] 0 2024-03-29T04:45:12.420 INFO:tasks.workunit.client.1.smithi152.stdout:1/374: dwrite d5/d7/d8/d13/f74 [0,4194304] 0 2024-03-29T04:45:12.421 INFO:tasks.workunit.client.1.smithi152.stdout:5/315: mkdir d6/d1d/d30/d6a 0 2024-03-29T04:45:12.424 INFO:tasks.workunit.client.0.smithi142.stdout:4/838: stat d0/d9/d11/lc2 0 2024-03-29T04:45:12.425 INFO:tasks.workunit.client.0.smithi142.stdout:9/735: dread d0/d2/d6/d27/d3c/f69 [0,4194304] 0 2024-03-29T04:45:12.427 INFO:tasks.workunit.client.1.smithi152.stdout:5/316: symlink d6/d16/d47/l6b 0 2024-03-29T04:45:12.429 INFO:tasks.workunit.client.0.smithi142.stdout:9/736: dread d0/d12/f60 [0,4194304] 0 2024-03-29T04:45:12.430 INFO:tasks.workunit.client.1.smithi152.stdout:5/317: write d6/d16/d47/f61 [2246255,12400] 0 2024-03-29T04:45:12.431 INFO:tasks.workunit.client.1.smithi152.stdout:7/265: symlink d7/d8/d12/d1d/d41/d45/l4a 0 2024-03-29T04:45:12.432 INFO:tasks.workunit.client.0.smithi142.stdout:9/737: write d0/d2/d6/d27/d3c/f69 [33786,12505] 0 2024-03-29T04:45:12.433 INFO:tasks.workunit.client.1.smithi152.stdout:5/318: chown d6/d16/d47/f59 2608 1 2024-03-29T04:45:12.434 INFO:tasks.workunit.client.0.smithi142.stdout:9/738: truncate d0/d2/d61/fc3 972454 0 2024-03-29T04:45:12.436 INFO:tasks.workunit.client.1.smithi152.stdout:1/375: dwrite d5/d7/d38/f6f [0,4194304] 0 2024-03-29T04:45:12.440 INFO:tasks.workunit.client.1.smithi152.stdout:7/266: creat d7/d8/db/f4b x:0 0 0 2024-03-29T04:45:12.440 INFO:tasks.workunit.client.1.smithi152.stdout:7/267: chown d7/d8/l2e 110699693 1 2024-03-29T04:45:12.444 INFO:tasks.workunit.client.1.smithi152.stdout:5/319: rename d6/d1d/d3b/d55/f11 to d6/d16/d25/d2e/f6c 0 2024-03-29T04:45:12.447 INFO:tasks.workunit.client.1.smithi152.stdout:7/268: dwrite d7/d8/d37/f46 [0,4194304] 0 2024-03-29T04:45:12.448 INFO:tasks.workunit.client.1.smithi152.stdout:7/269: fdatasync d7/d8/d12/d35/f1a 0 2024-03-29T04:45:12.450 INFO:tasks.workunit.client.1.smithi152.stdout:5/320: creat d6/d16/d47/d56/d63/f6d x:0 0 0 2024-03-29T04:45:12.451 INFO:tasks.workunit.client.1.smithi152.stdout:7/270: chown d7/d8/d12/d1d/d41/d45/l4a 22813765 1 2024-03-29T04:45:12.456 INFO:tasks.workunit.client.1.smithi152.stdout:5/321: getdents d6/d16/d5c 0 2024-03-29T04:45:12.457 INFO:tasks.workunit.client.1.smithi152.stdout:7/271: link d7/l3d d7/d8/d12/d1d/d41/l4c 0 2024-03-29T04:45:12.457 INFO:tasks.workunit.client.1.smithi152.stdout:5/322: chown d6/d1d/d30/f3e 6 1 2024-03-29T04:45:12.459 INFO:tasks.workunit.client.1.smithi152.stdout:5/323: rmdir d6/d7 39 2024-03-29T04:45:12.462 INFO:tasks.workunit.client.1.smithi152.stdout:7/272: rename d7/f10 to d7/d8/d12/d1d/d41/f4d 0 2024-03-29T04:45:12.465 INFO:tasks.workunit.client.1.smithi152.stdout:7/273: mknod d7/d8/db/df/c4e 0 2024-03-29T04:45:12.466 INFO:tasks.workunit.client.1.smithi152.stdout:5/324: getdents d6/d1d/d30/d6a 0 2024-03-29T04:45:12.466 INFO:tasks.workunit.client.0.smithi142.stdout:9/739: dread d0/f1d [0,4194304] 0 2024-03-29T04:45:12.467 INFO:tasks.workunit.client.1.smithi152.stdout:0/331: write d2/f8 [1823789,21045] 0 2024-03-29T04:45:12.467 INFO:tasks.workunit.client.1.smithi152.stdout:5/325: write d6/d16/d5c/f69 [159017,13331] 0 2024-03-29T04:45:12.470 INFO:tasks.workunit.client.1.smithi152.stdout:6/348: dread f0 [0,4194304] 0 2024-03-29T04:45:12.470 INFO:tasks.workunit.client.1.smithi152.stdout:7/274: rename d7/d8/db/fe to d7/d8/db/f4f 0 2024-03-29T04:45:12.470 INFO:tasks.workunit.client.0.smithi142.stdout:9/740: unlink d0/d12/f86 0 2024-03-29T04:45:12.471 INFO:tasks.workunit.client.1.smithi152.stdout:0/332: rmdir d2/da/d17/d50 39 2024-03-29T04:45:12.472 INFO:tasks.workunit.client.0.smithi142.stdout:9/741: symlink d0/d2/d6/d27/d3c/d7b/le7 0 2024-03-29T04:45:12.473 INFO:tasks.workunit.client.0.smithi142.stdout:9/742: stat d0/d12/d50 0 2024-03-29T04:45:12.475 INFO:tasks.workunit.client.0.smithi142.stdout:9/743: chown d0/d2/d6/d27/c9d 673 1 2024-03-29T04:45:12.477 INFO:tasks.workunit.client.0.smithi142.stdout:9/744: creat d0/d12/d50/fe8 x:0 0 0 2024-03-29T04:45:12.478 INFO:tasks.workunit.client.1.smithi152.stdout:7/275: creat d7/d8/d12/d15/f50 x:0 0 0 2024-03-29T04:45:12.479 INFO:tasks.workunit.client.0.smithi142.stdout:9/745: truncate d0/d2/f1f 2496733 0 2024-03-29T04:45:12.480 INFO:tasks.workunit.client.1.smithi152.stdout:5/326: rename d6/d16/d25/d2e/f6c to d6/d1d/f6e 0 2024-03-29T04:45:12.481 INFO:tasks.workunit.client.0.smithi142.stdout:9/746: symlink d0/d2/d6/le9 0 2024-03-29T04:45:12.481 INFO:tasks.workunit.client.1.smithi152.stdout:1/376: write d5/d7/d8/d13/d19/f3d [8177439,98827] 0 2024-03-29T04:45:12.483 INFO:tasks.workunit.client.1.smithi152.stdout:1/377: write d5/d7/d1b/f70 [182512,90588] 0 2024-03-29T04:45:12.483 INFO:tasks.workunit.client.1.smithi152.stdout:7/276: creat d7/d8/d12/d1d/d41/f51 x:0 0 0 2024-03-29T04:45:12.484 INFO:tasks.workunit.client.1.smithi152.stdout:0/333: link d2/da/d17/d50/d5c/f7b d2/da/d3a/d61/f7f 0 2024-03-29T04:45:12.486 INFO:tasks.workunit.client.1.smithi152.stdout:5/327: rename d6/d16/d25/c2b to d6/d16/c6f 0 2024-03-29T04:45:12.487 INFO:tasks.workunit.client.1.smithi152.stdout:5/328: write d6/d1d/d3b/d55/f35 [400557,64958] 0 2024-03-29T04:45:12.488 INFO:tasks.workunit.client.0.smithi142.stdout:9/747: dwrite d0/d2/d6/d27/f82 [0,4194304] 0 2024-03-29T04:45:12.488 INFO:tasks.workunit.client.1.smithi152.stdout:1/378: creat d5/d7/d1b/f78 x:0 0 0 2024-03-29T04:45:12.489 INFO:tasks.workunit.client.1.smithi152.stdout:1/379: dread - d5/d7/d6a/f69 zero size 2024-03-29T04:45:12.490 INFO:tasks.workunit.client.1.smithi152.stdout:0/334: mknod d2/da/d17/d57/c80 0 2024-03-29T04:45:12.492 INFO:tasks.workunit.client.1.smithi152.stdout:0/335: write d2/d29/f37 [419847,52134] 0 2024-03-29T04:45:12.494 INFO:tasks.workunit.client.1.smithi152.stdout:0/336: write d2/f8 [1360467,71358] 0 2024-03-29T04:45:12.494 INFO:tasks.workunit.client.0.smithi142.stdout:9/748: rename d0/d2/d6/d27/d3c/c7f to d0/d2/d6/d27/d3c/d7b/cea 0 2024-03-29T04:45:12.497 INFO:tasks.workunit.client.1.smithi152.stdout:6/349: truncate d1/d3/d1a/f1d 3579393 0 2024-03-29T04:45:12.498 INFO:tasks.workunit.client.0.smithi142.stdout:9/749: creat d0/d12/feb x:0 0 0 2024-03-29T04:45:12.499 INFO:tasks.workunit.client.1.smithi152.stdout:1/380: link d5/d7/f54 d5/d7/d6a/f79 0 2024-03-29T04:45:12.499 INFO:tasks.workunit.client.0.smithi142.stdout:9/750: dread - d0/d2/d6/d27/f92 zero size 2024-03-29T04:45:12.501 INFO:tasks.workunit.client.0.smithi142.stdout:9/751: write d0/d2/d6/d31/dc8/fe2 [804837,67718] 0 2024-03-29T04:45:12.501 INFO:tasks.workunit.client.1.smithi152.stdout:2/215: sync 2024-03-29T04:45:12.501 INFO:tasks.workunit.client.1.smithi152.stdout:9/293: sync 2024-03-29T04:45:12.506 INFO:tasks.workunit.client.0.smithi142.stdout:9/752: creat d0/d2/d6/d27/d3c/d6f/fec x:0 0 0 2024-03-29T04:45:12.506 INFO:tasks.workunit.client.1.smithi152.stdout:0/337: rename d2/da/d17/f2b to d2/da/d3a/f81 0 2024-03-29T04:45:12.506 INFO:tasks.workunit.client.1.smithi152.stdout:0/338: chown d2/da/d3a/d61/l67 7 1 2024-03-29T04:45:12.507 INFO:tasks.workunit.client.1.smithi152.stdout:5/329: getdents d6/d16/d25/d2e 0 2024-03-29T04:45:12.507 INFO:tasks.workunit.client.0.smithi142.stdout:9/753: mknod d0/d2/d61/d64/ced 0 2024-03-29T04:45:12.507 INFO:tasks.workunit.client.0.smithi142.stdout:9/754: chown d0/d2/d39/le4 16181 1 2024-03-29T04:45:12.509 INFO:tasks.workunit.client.0.smithi142.stdout:9/755: mkdir d0/d2/d61/d64/dee 0 2024-03-29T04:45:12.510 INFO:tasks.workunit.client.1.smithi152.stdout:9/294: mkdir d5/d12/d2c/d6a 0 2024-03-29T04:45:12.511 INFO:tasks.workunit.client.1.smithi152.stdout:1/381: symlink d5/d7/d8/d13/d19/l7a 0 2024-03-29T04:45:12.512 INFO:tasks.workunit.client.1.smithi152.stdout:7/277: truncate d7/d8/d12/f28 642157 0 2024-03-29T04:45:12.516 INFO:tasks.workunit.client.1.smithi152.stdout:7/278: readlink d7/d8/d12/d1d/d41/l48 0 2024-03-29T04:45:12.516 INFO:tasks.workunit.client.1.smithi152.stdout:1/382: truncate d5/d7/d1b/f70 529522 0 2024-03-29T04:45:12.516 INFO:tasks.workunit.client.1.smithi152.stdout:5/330: dwrite d6/d16/d25/f3a [0,4194304] 0 2024-03-29T04:45:12.520 INFO:tasks.workunit.client.1.smithi152.stdout:5/331: write d6/d1d/d3b/d55/f66 [58819,25322] 0 2024-03-29T04:45:12.523 INFO:tasks.workunit.client.1.smithi152.stdout:5/332: write d6/d1d/d3b/d55/f35 [534559,34527] 0 2024-03-29T04:45:12.526 INFO:tasks.workunit.client.1.smithi152.stdout:6/350: link d1/d3/d9/d13/d28/d45/l73 d1/d3/d9/d13/d3e/d54/l77 0 2024-03-29T04:45:12.526 INFO:tasks.workunit.client.1.smithi152.stdout:1/383: chown d5/l1d 15092 1 2024-03-29T04:45:12.527 INFO:tasks.workunit.client.1.smithi152.stdout:9/295: rmdir d5/d12/d2c/d6a 0 2024-03-29T04:45:12.529 INFO:tasks.workunit.client.1.smithi152.stdout:2/216: dwrite d0/d3/f1d [0,4194304] 0 2024-03-29T04:45:12.530 INFO:tasks.workunit.client.0.smithi142.stdout:9/756: dwrite d0/d12/d50/d52/fb4 [0,4194304] 0 2024-03-29T04:45:12.530 INFO:tasks.workunit.client.1.smithi152.stdout:7/279: mknod d7/d8/db/c52 0 2024-03-29T04:45:12.533 INFO:tasks.workunit.client.1.smithi152.stdout:1/384: truncate d5/d7/d8/f2d 878275 0 2024-03-29T04:45:12.534 INFO:tasks.workunit.client.0.smithi142.stdout:9/757: symlink d0/d2/d6/d27/d3c/d6f/lef 0 2024-03-29T04:45:12.537 INFO:tasks.workunit.client.0.smithi142.stdout:9/758: creat d0/d12/d50/ff0 x:0 0 0 2024-03-29T04:45:12.537 INFO:tasks.workunit.client.1.smithi152.stdout:5/333: dread d6/f21 [0,4194304] 0 2024-03-29T04:45:12.540 INFO:tasks.workunit.client.0.smithi142.stdout:1/796: sync 2024-03-29T04:45:12.541 INFO:tasks.workunit.client.1.smithi152.stdout:0/339: dwrite d2/da/fb [0,4194304] 0 2024-03-29T04:45:12.541 INFO:tasks.workunit.client.1.smithi152.stdout:2/217: symlink d0/d3/l46 0 2024-03-29T04:45:12.542 INFO:tasks.workunit.client.0.smithi142.stdout:1/797: dread - d8/db/dc/d19/ff0 zero size 2024-03-29T04:45:12.543 INFO:tasks.workunit.client.1.smithi152.stdout:1/385: rename d5/d7/f54 to d5/d7/d9/f7b 0 2024-03-29T04:45:12.543 INFO:tasks.workunit.client.0.smithi142.stdout:9/759: creat d0/d2/ff1 x:0 0 0 2024-03-29T04:45:12.546 INFO:tasks.workunit.client.1.smithi152.stdout:0/340: read - d2/da/d17/d57/f73 zero size 2024-03-29T04:45:12.549 INFO:tasks.workunit.client.1.smithi152.stdout:2/218: write d0/d3/d30/d15/f29 [834574,77670] 0 2024-03-29T04:45:12.549 INFO:tasks.workunit.client.0.smithi142.stdout:9/760: dwrite d0/d2/d6/d27/dc7/daa/f87 [0,4194304] 0 2024-03-29T04:45:12.552 INFO:tasks.workunit.client.1.smithi152.stdout:9/296: rename d5/db/c38 to d5/db/c6b 0 2024-03-29T04:45:12.554 INFO:tasks.workunit.client.0.smithi142.stdout:9/761: creat d0/d12/d50/d52/ff2 x:0 0 0 2024-03-29T04:45:12.555 INFO:tasks.workunit.client.1.smithi152.stdout:5/334: read d6/d7/f2d [2019092,118420] 0 2024-03-29T04:45:12.556 INFO:tasks.workunit.client.0.smithi142.stdout:9/762: fsync d0/d12/d50/d52/da3/fb1 0 2024-03-29T04:45:12.560 INFO:tasks.workunit.client.0.smithi142.stdout:9/763: read d0/d2/d61/f55 [3101008,78702] 0 2024-03-29T04:45:12.561 INFO:tasks.workunit.client.1.smithi152.stdout:5/335: fsync d6/d16/d47/f61 0 2024-03-29T04:45:12.561 INFO:tasks.workunit.client.1.smithi152.stdout:2/219: link d0/d3/f1d d0/d3/df/d2c/d45/f47 0 2024-03-29T04:45:12.562 INFO:tasks.workunit.client.1.smithi152.stdout:7/280: dwrite d7/d8/d12/d1d/f34 [0,4194304] 0 2024-03-29T04:45:12.563 INFO:tasks.workunit.client.0.smithi142.stdout:9/764: symlink d0/d12/lf3 0 2024-03-29T04:45:12.565 INFO:tasks.workunit.client.1.smithi152.stdout:9/297: getdents d5 0 2024-03-29T04:45:12.568 INFO:tasks.workunit.client.1.smithi152.stdout:2/220: mknod d0/d3/d30/c48 0 2024-03-29T04:45:12.569 INFO:tasks.workunit.client.1.smithi152.stdout:1/386: dwrite d5/d7/d9/f46 [0,4194304] 0 2024-03-29T04:45:12.570 INFO:tasks.workunit.client.0.smithi142.stdout:9/765: dwrite d0/d2/d6/d27/dc7/d6a/fae [0,4194304] 0 2024-03-29T04:45:12.571 INFO:tasks.workunit.client.1.smithi152.stdout:6/351: dwrite d1/d3/f18 [0,4194304] 0 2024-03-29T04:45:12.572 INFO:tasks.workunit.client.1.smithi152.stdout:0/341: link d2/da/df/c23 d2/da/d17/d50/c82 0 2024-03-29T04:45:12.572 INFO:tasks.workunit.client.1.smithi152.stdout:9/298: write d5/db/d1b/f4f [2932155,25509] 0 2024-03-29T04:45:12.573 INFO:tasks.workunit.client.1.smithi152.stdout:5/336: write d6/d1b/f31 [958952,104938] 0 2024-03-29T04:45:12.574 INFO:tasks.workunit.client.0.smithi142.stdout:9/766: mkdir d0/d12/d50/d8e/df4 0 2024-03-29T04:45:12.577 INFO:tasks.workunit.client.1.smithi152.stdout:7/281: chown d7/d8/d37/l40 14311 1 2024-03-29T04:45:12.580 INFO:tasks.workunit.client.1.smithi152.stdout:9/299: write d5/db/f4e [3647844,106582] 0 2024-03-29T04:45:12.581 INFO:tasks.workunit.client.1.smithi152.stdout:2/221: write d0/d3/d30/d15/d22/f37 [4671273,126749] 0 2024-03-29T04:45:12.582 INFO:tasks.workunit.client.0.smithi142.stdout:9/767: rename d0/d2/d6/d27/d3c/d7b/l8b to d0/d2/d61/lf5 0 2024-03-29T04:45:12.585 INFO:tasks.workunit.client.0.smithi142.stdout:9/768: truncate d0/d12/f59 3016580 0 2024-03-29T04:45:12.585 INFO:tasks.workunit.client.1.smithi152.stdout:7/282: read d7/d8/d12/d1d/f49 [4048484,77329] 0 2024-03-29T04:45:12.586 INFO:tasks.workunit.client.1.smithi152.stdout:1/387: dread d5/d7/d1b/f70 [0,4194304] 0 2024-03-29T04:45:12.586 INFO:tasks.workunit.client.1.smithi152.stdout:5/337: rename d6/d16/d25/d2e/c48 to d6/d1d/d30/c70 0 2024-03-29T04:45:12.589 INFO:tasks.workunit.client.1.smithi152.stdout:0/342: dwrite d2/da/d17/d57/f62 [0,4194304] 0 2024-03-29T04:45:12.590 INFO:tasks.workunit.client.1.smithi152.stdout:2/222: creat d0/d3/d30/d15/d22/f49 x:0 0 0 2024-03-29T04:45:12.591 INFO:tasks.workunit.client.1.smithi152.stdout:2/223: readlink d0/d3/l46 0 2024-03-29T04:45:12.591 INFO:tasks.workunit.client.1.smithi152.stdout:0/343: stat d2/da/d3a/d61/l67 0 2024-03-29T04:45:12.596 INFO:tasks.workunit.client.1.smithi152.stdout:2/224: symlink d0/d3/d30/l4a 0 2024-03-29T04:45:12.596 INFO:tasks.workunit.client.1.smithi152.stdout:6/352: link d1/l32 d1/d3/d9/d13/d28/l78 0 2024-03-29T04:45:12.598 INFO:tasks.workunit.client.0.smithi142.stdout:9/769: write d0/d2/d6/d27/dc7/faf [2786122,8435] 0 2024-03-29T04:45:12.600 INFO:tasks.workunit.client.1.smithi152.stdout:7/283: link d7/d8/db/c52 d7/d8/d12/d1d/d41/d45/c53 0 2024-03-29T04:45:12.601 INFO:tasks.workunit.client.0.smithi142.stdout:9/770: truncate d0/d2/d6/d27/d3c/d6f/fcb 578205 0 2024-03-29T04:45:12.601 INFO:tasks.workunit.client.0.smithi142.stdout:9/771: stat d0/d12/d50 0 2024-03-29T04:45:12.602 INFO:tasks.workunit.client.1.smithi152.stdout:9/300: rename d5/d8/f23 to d5/db/f6c 0 2024-03-29T04:45:12.602 INFO:tasks.workunit.client.1.smithi152.stdout:2/225: mknod d0/c4b 0 2024-03-29T04:45:12.603 INFO:tasks.workunit.client.0.smithi142.stdout:9/772: mknod d0/d6c/cf6 0 2024-03-29T04:45:12.604 INFO:tasks.workunit.client.1.smithi152.stdout:0/344: dwrite d2/d31/f35 [0,4194304] 0 2024-03-29T04:45:12.604 INFO:tasks.workunit.client.1.smithi152.stdout:2/226: dread - d0/d3/f2e zero size 2024-03-29T04:45:12.605 INFO:tasks.workunit.client.1.smithi152.stdout:5/338: rename d6/d1d/f2c to d6/d16/d5c/f71 0 2024-03-29T04:45:12.607 INFO:tasks.workunit.client.1.smithi152.stdout:2/227: dread - d0/d3/df/f3e zero size 2024-03-29T04:45:12.608 INFO:tasks.workunit.client.0.smithi142.stdout:9/773: getdents d0/d2/d61/d64 0 2024-03-29T04:45:12.609 INFO:tasks.workunit.client.1.smithi152.stdout:7/284: fdatasync d7/d8/d12/f43 0 2024-03-29T04:45:12.611 INFO:tasks.workunit.client.1.smithi152.stdout:6/353: rename d1/d3/d1a/d1b/f58 to d1/d3/d1a/d1b/d29/f79 0 2024-03-29T04:45:12.611 INFO:tasks.workunit.client.1.smithi152.stdout:8/228: sync 2024-03-29T04:45:12.613 INFO:tasks.workunit.client.1.smithi152.stdout:0/345: mknod d2/da/d3a/d6f/c83 0 2024-03-29T04:45:12.614 INFO:tasks.workunit.client.0.smithi142.stdout:9/774: dread d0/d2/d6/d27/dc7/f3e [0,4194304] 0 2024-03-29T04:45:12.615 INFO:tasks.workunit.client.1.smithi152.stdout:9/301: dwrite d5/d12/d2c/f63 [0,4194304] 0 2024-03-29T04:45:12.618 INFO:tasks.workunit.client.1.smithi152.stdout:5/339: creat d6/d16/d25/d2e/f72 x:0 0 0 2024-03-29T04:45:12.618 INFO:tasks.workunit.client.0.smithi142.stdout:9/775: rename d0/d2/d6/d27/dc6 to d0/d2/d61/df7 0 2024-03-29T04:45:12.619 INFO:tasks.workunit.client.1.smithi152.stdout:2/228: mkdir d0/d3/d30/d15/d22/d4c 0 2024-03-29T04:45:12.620 INFO:tasks.workunit.client.1.smithi152.stdout:7/285: symlink d7/d8/d12/d15/l54 0 2024-03-29T04:45:12.621 INFO:tasks.workunit.client.0.smithi142.stdout:9/776: creat d0/d12/d50/ff8 x:0 0 0 2024-03-29T04:45:12.624 INFO:tasks.workunit.client.0.smithi142.stdout:9/777: fdatasync d0/d6c/f79 0 2024-03-29T04:45:12.625 INFO:tasks.workunit.client.1.smithi152.stdout:1/388: write d5/d7/d6a/f79 [3943636,124838] 0 2024-03-29T04:45:12.627 INFO:tasks.workunit.client.1.smithi152.stdout:6/354: creat d1/d3/d9/d13/d3e/f7a x:0 0 0 2024-03-29T04:45:12.628 INFO:tasks.workunit.client.1.smithi152.stdout:9/302: dwrite d5/db/d1b/f2b [4194304,4194304] 0 2024-03-29T04:45:12.631 INFO:tasks.workunit.client.1.smithi152.stdout:0/346: write d2/da/d17/d57/f71 [4656924,55753] 0 2024-03-29T04:45:12.632 INFO:tasks.workunit.client.0.smithi142.stdout:9/778: dwrite d0/d2/d6/d27/dc7/d6a/fce [0,4194304] 0 2024-03-29T04:45:12.633 INFO:tasks.workunit.client.1.smithi152.stdout:5/340: dread d6/d7/f8 [0,4194304] 0 2024-03-29T04:45:12.636 INFO:tasks.workunit.client.1.smithi152.stdout:2/229: dwrite d0/d3/d30/f1a [0,4194304] 0 2024-03-29T04:45:12.639 INFO:tasks.workunit.client.0.smithi142.stdout:9/779: creat d0/d2/d6/d27/d3c/ff9 x:0 0 0 2024-03-29T04:45:12.640 INFO:tasks.workunit.client.0.smithi142.stdout:9/780: dread - d0/d2/fe6 zero size 2024-03-29T04:45:12.641 INFO:tasks.workunit.client.1.smithi152.stdout:5/341: write d6/d1d/d3b/d55/fe [1674137,115198] 0 2024-03-29T04:45:12.642 INFO:tasks.workunit.client.1.smithi152.stdout:0/347: mknod d2/c84 0 2024-03-29T04:45:12.648 INFO:tasks.workunit.client.1.smithi152.stdout:7/286: rmdir d7/d8/d12/d15 39 2024-03-29T04:45:12.648 INFO:tasks.workunit.client.0.smithi142.stdout:2/861: sync 2024-03-29T04:45:12.650 INFO:tasks.workunit.client.1.smithi152.stdout:5/342: creat d6/d16/f73 x:0 0 0 2024-03-29T04:45:12.651 INFO:tasks.workunit.client.0.smithi142.stdout:9/781: read d0/d2/d61/f2d [2513467,91852] 0 2024-03-29T04:45:12.652 INFO:tasks.workunit.client.1.smithi152.stdout:0/348: dwrite d2/f8 [0,4194304] 0 2024-03-29T04:45:12.653 INFO:tasks.workunit.client.1.smithi152.stdout:5/343: chown l4 36 1 2024-03-29T04:45:12.653 INFO:tasks.workunit.client.0.smithi142.stdout:2/862: rename de/d11/d23/dd1/ff0 to de/f120 0 2024-03-29T04:45:12.655 INFO:tasks.workunit.client.0.smithi142.stdout:6/848: sync 2024-03-29T04:45:12.657 INFO:tasks.workunit.client.1.smithi152.stdout:7/287: dread - d7/d8/d12/d15/f3e zero size 2024-03-29T04:45:12.658 INFO:tasks.workunit.client.0.smithi142.stdout:6/849: truncate d1c/d45/d8a/fa0 463475 0 2024-03-29T04:45:12.659 INFO:tasks.workunit.client.1.smithi152.stdout:5/344: creat d6/d16/d5c/f74 x:0 0 0 2024-03-29T04:45:12.659 INFO:tasks.workunit.client.1.smithi152.stdout:0/349: creat d2/da/d3a/d6f/f85 x:0 0 0 2024-03-29T04:45:12.660 INFO:tasks.workunit.client.0.smithi142.stdout:6/850: dread - d1c/d45/d8a/f110 zero size 2024-03-29T04:45:12.660 INFO:tasks.workunit.client.0.smithi142.stdout:2/863: getdents de/d11/da1 0 2024-03-29T04:45:12.661 INFO:tasks.workunit.client.0.smithi142.stdout:6/851: chown d1c/d45/d8a/fe0 2680 1 2024-03-29T04:45:12.661 INFO:tasks.workunit.client.1.smithi152.stdout:5/345: truncate d6/d16/d5c/f69 642851 0 2024-03-29T04:45:12.662 INFO:tasks.workunit.client.0.smithi142.stdout:6/852: chown d1c/f21 245 1 2024-03-29T04:45:12.662 INFO:tasks.workunit.client.1.smithi152.stdout:7/288: creat d7/d8/d12/d1d/d41/d45/f55 x:0 0 0 2024-03-29T04:45:12.665 INFO:tasks.workunit.client.1.smithi152.stdout:5/346: dread - d6/d16/d25/d2e/f72 zero size 2024-03-29T04:45:12.666 INFO:tasks.workunit.client.0.smithi142.stdout:2/864: mknod de/d11/d17/d10b/d22/d27/dba/d61/d7d/c121 0 2024-03-29T04:45:12.666 INFO:tasks.workunit.client.1.smithi152.stdout:9/303: dread d5/db/d1b/f46 [0,4194304] 0 2024-03-29T04:45:12.667 INFO:tasks.workunit.client.1.smithi152.stdout:1/389: truncate d5/d7/d9/f10 1746721 0 2024-03-29T04:45:12.667 INFO:tasks.workunit.client.0.smithi142.stdout:2/865: chown de/d11/d17/d60/ff8 17551 1 2024-03-29T04:45:12.668 INFO:tasks.workunit.client.1.smithi152.stdout:7/289: read d7/d8/d12/d1d/d41/f2c [2282,9075] 0 2024-03-29T04:45:12.668 INFO:tasks.workunit.client.0.smithi142.stdout:6/853: dwrite d1c/d45/d51/f7f [0,4194304] 0 2024-03-29T04:45:12.670 INFO:tasks.workunit.client.1.smithi152.stdout:0/350: dwrite d2/da/d17/f74 [0,4194304] 0 2024-03-29T04:45:12.670 INFO:tasks.workunit.client.0.smithi142.stdout:2/866: fdatasync f7 0 2024-03-29T04:45:12.672 INFO:tasks.workunit.client.1.smithi152.stdout:9/304: write d5/db/f65 [500675,95186] 0 2024-03-29T04:45:12.673 INFO:tasks.workunit.client.1.smithi152.stdout:1/390: creat d5/d7/d38/f7c x:0 0 0 2024-03-29T04:45:12.674 INFO:tasks.workunit.client.1.smithi152.stdout:7/290: readlink d7/d8/d12/d1d/d41/l48 0 2024-03-29T04:45:12.674 INFO:tasks.workunit.client.0.smithi142.stdout:2/867: chown de/d11/d17/d10b/d22/dbf/c115 113 1 2024-03-29T04:45:12.675 INFO:tasks.workunit.client.0.smithi142.stdout:6/854: getdents d1c/d33/d46/d61/d9e/dda 0 2024-03-29T04:45:12.675 INFO:tasks.workunit.client.1.smithi152.stdout:9/305: creat d5/d12/f6d x:0 0 0 2024-03-29T04:45:12.676 INFO:tasks.workunit.client.1.smithi152.stdout:6/355: dread d1/d3/d9/d13/f26 [0,4194304] 0 2024-03-29T04:45:12.677 INFO:tasks.workunit.client.1.smithi152.stdout:7/291: fsync d7/d8/d12/d15/f3e 0 2024-03-29T04:45:12.677 INFO:tasks.workunit.client.1.smithi152.stdout:0/351: mkdir d2/da/df/d86 0 2024-03-29T04:45:12.679 INFO:tasks.workunit.client.0.smithi142.stdout:6/855: chown d1c/d33/d46/d66/dcd/le8 28565 1 2024-03-29T04:45:12.680 INFO:tasks.workunit.client.1.smithi152.stdout:6/356: truncate d1/d3/d9/f3a 1571811 0 2024-03-29T04:45:12.680 INFO:tasks.workunit.client.1.smithi152.stdout:1/391: symlink d5/d7/d4d/l7d 0 2024-03-29T04:45:12.681 INFO:tasks.workunit.client.1.smithi152.stdout:7/292: write d7/d8/d12/d1d/d30/f3c [774290,80531] 0 2024-03-29T04:45:12.682 INFO:tasks.workunit.client.1.smithi152.stdout:9/306: dread d5/d12/d2c/f39 [0,4194304] 0 2024-03-29T04:45:12.683 INFO:tasks.workunit.client.0.smithi142.stdout:9/782: dwrite d0/d2/f1f [0,4194304] 0 2024-03-29T04:45:12.683 INFO:tasks.workunit.client.0.smithi142.stdout:2/868: creat de/d11/d17/d60/d6f/d116/f122 x:0 0 0 2024-03-29T04:45:12.687 INFO:tasks.workunit.client.1.smithi152.stdout:2/230: dread d0/d3/d30/f19 [4194304,4194304] 0 2024-03-29T04:45:12.690 INFO:tasks.workunit.client.0.smithi142.stdout:2/869: write de/f16 [1611833,36481] 0 2024-03-29T04:45:12.691 INFO:tasks.workunit.client.0.smithi142.stdout:6/856: dwrite d1c/d45/d51/fb4 [0,4194304] 0 2024-03-29T04:45:12.693 INFO:tasks.workunit.client.0.smithi142.stdout:9/783: dread d0/d2/d6/f9c [0,4194304] 0 2024-03-29T04:45:12.693 INFO:tasks.workunit.client.1.smithi152.stdout:1/392: dwrite d5/d7/d6a/f69 [0,4194304] 0 2024-03-29T04:45:12.693 INFO:tasks.workunit.client.1.smithi152.stdout:7/293: mknod d7/d8/d12/d1d/d30/c56 0 2024-03-29T04:45:12.694 INFO:tasks.workunit.client.1.smithi152.stdout:5/347: dread d6/d7/f2d [0,4194304] 0 2024-03-29T04:45:12.697 INFO:tasks.workunit.client.0.smithi142.stdout:2/870: symlink de/d11/da1/de2/l123 0 2024-03-29T04:45:12.698 INFO:tasks.workunit.client.0.smithi142.stdout:2/871: fdatasync de/d11/da1/fff 0 2024-03-29T04:45:12.698 INFO:tasks.workunit.client.0.smithi142.stdout:9/784: truncate d0/d12/f24 178743 0 2024-03-29T04:45:12.699 INFO:tasks.workunit.client.1.smithi152.stdout:5/348: chown d6/d16/f4d 25 1 2024-03-29T04:45:12.699 INFO:tasks.workunit.client.0.smithi142.stdout:2/872: write de/f16 [1726452,65596] 0 2024-03-29T04:45:12.700 INFO:tasks.workunit.client.0.smithi142.stdout:6/857: creat d1c/d33/d46/d61/d9e/dda/f114 x:0 0 0 2024-03-29T04:45:12.702 INFO:tasks.workunit.client.1.smithi152.stdout:0/352: link d2/d29/f6c d2/da/d3a/f87 0 2024-03-29T04:45:12.703 INFO:tasks.workunit.client.0.smithi142.stdout:9/785: mkdir d0/d2/d61/d64/dfa 0 2024-03-29T04:45:12.705 INFO:tasks.workunit.client.1.smithi152.stdout:5/349: write d6/d1d/d3b/d55/fe [2668626,120126] 0 2024-03-29T04:45:12.707 INFO:tasks.workunit.client.1.smithi152.stdout:7/294: mknod d7/d8/d12/d1d/d30/c57 0 2024-03-29T04:45:12.710 INFO:tasks.workunit.client.0.smithi142.stdout:2/873: rename de/d11/d17/d10b/d22/d50/l9c to de/d11/d17/d60/d6f/d116/l124 0 2024-03-29T04:45:12.710 INFO:tasks.workunit.client.0.smithi142.stdout:9/786: unlink d0/d2/d6/d27/c9d 0 2024-03-29T04:45:12.710 INFO:tasks.workunit.client.0.smithi142.stdout:2/874: write de/d11/d3d/fd8 [411084,4779] 0 2024-03-29T04:45:12.710 INFO:tasks.workunit.client.0.smithi142.stdout:9/787: chown d0/d2/fe6 818540084 1 2024-03-29T04:45:12.711 INFO:tasks.workunit.client.0.smithi142.stdout:2/875: readlink de/d11/d17/l41 0 2024-03-29T04:45:12.711 INFO:tasks.workunit.client.1.smithi152.stdout:2/231: dwrite d0/d3/df/f24 [0,4194304] 0 2024-03-29T04:45:12.712 INFO:tasks.workunit.client.0.smithi142.stdout:6/858: getdents d1c/d45 0 2024-03-29T04:45:12.713 INFO:tasks.workunit.client.0.smithi142.stdout:2/876: dread - de/d11/d17/d60/d6f/d116/f122 zero size 2024-03-29T04:45:12.713 INFO:tasks.workunit.client.1.smithi152.stdout:1/393: dwrite d5/d7/d1b/f5e [0,4194304] 0 2024-03-29T04:45:12.718 INFO:tasks.workunit.client.0.smithi142.stdout:6/859: mknod d1c/c115 0 2024-03-29T04:45:12.720 INFO:tasks.workunit.client.0.smithi142.stdout:9/788: dwrite d0/d2/d6/d27/d3c/f72 [0,4194304] 0 2024-03-29T04:45:12.720 INFO:tasks.workunit.client.0.smithi142.stdout:9/789: chown d0/d6c 226904808 1 2024-03-29T04:45:12.722 INFO:tasks.workunit.client.0.smithi142.stdout:6/860: write d1c/d33/db6/f111 [522724,109466] 0 2024-03-29T04:45:12.723 INFO:tasks.workunit.client.0.smithi142.stdout:6/861: chown d1c/d45/d51/fb4 419970190 1 2024-03-29T04:45:12.724 INFO:tasks.workunit.client.0.smithi142.stdout:6/862: chown d1c 5 1 2024-03-29T04:45:12.724 INFO:tasks.workunit.client.0.smithi142.stdout:9/790: write d0/d2/d6/fcd [3544026,8915] 0 2024-03-29T04:45:12.726 INFO:tasks.workunit.client.0.smithi142.stdout:6/863: chown d1c/d45/d8a/df6/f95 1 1 2024-03-29T04:45:12.726 INFO:tasks.workunit.client.1.smithi152.stdout:0/353: symlink d2/da/d3a/d61/l88 0 2024-03-29T04:45:12.727 INFO:tasks.workunit.client.0.smithi142.stdout:6/864: fdatasync d1c/d33/f67 0 2024-03-29T04:45:12.729 INFO:tasks.workunit.client.1.smithi152.stdout:5/350: dwrite d6/d1b/f5a [0,4194304] 0 2024-03-29T04:45:12.732 INFO:tasks.workunit.client.1.smithi152.stdout:7/295: dwrite d7/d8/d12/f20 [0,4194304] 0 2024-03-29T04:45:12.734 INFO:tasks.workunit.client.1.smithi152.stdout:7/296: dread - d7/d8/d12/d15/f31 zero size 2024-03-29T04:45:12.735 INFO:tasks.workunit.client.1.smithi152.stdout:1/394: dread d5/d7/d8/d13/f74 [0,4194304] 0 2024-03-29T04:45:12.738 INFO:tasks.workunit.client.1.smithi152.stdout:5/351: mkdir d6/d7/d75 0 2024-03-29T04:45:12.738 INFO:tasks.workunit.client.1.smithi152.stdout:7/297: chown d7/d8/d12/d1d/d41/f51 634 1 2024-03-29T04:45:12.738 INFO:tasks.workunit.client.0.smithi142.stdout:9/791: dwrite d0/d2/d6/d27/d3c/d4f/f56 [0,4194304] 0 2024-03-29T04:45:12.740 INFO:tasks.workunit.client.0.smithi142.stdout:9/792: write d0/d2/d6/d27/dc7/daa/f87 [1082079,111419] 0 2024-03-29T04:45:12.740 INFO:tasks.workunit.client.1.smithi152.stdout:9/307: dread d5/d8/f11 [0,4194304] 0 2024-03-29T04:45:12.742 INFO:tasks.workunit.client.1.smithi152.stdout:9/308: read d5/f29 [2315402,12408] 0 2024-03-29T04:45:12.743 INFO:tasks.workunit.client.1.smithi152.stdout:1/395: write d5/d7/d8/d13/f74 [191228,113583] 0 2024-03-29T04:45:12.743 INFO:tasks.workunit.client.1.smithi152.stdout:7/298: read d7/d8/fa [1932933,85623] 0 2024-03-29T04:45:12.744 INFO:tasks.workunit.client.0.smithi142.stdout:9/793: getdents d0/d2/d61/d64/dee 0 2024-03-29T04:45:12.746 INFO:tasks.workunit.client.1.smithi152.stdout:0/354: link d2/da/df/c23 d2/da/df/c89 0 2024-03-29T04:45:12.749 INFO:tasks.workunit.client.1.smithi152.stdout:6/357: truncate d1/f8 4448846 0 2024-03-29T04:45:12.749 INFO:tasks.workunit.client.0.smithi142.stdout:9/794: rename d0/d2/d61/d64/c84 to d0/d2/d6/d27/cfb 0 2024-03-29T04:45:12.751 INFO:tasks.workunit.client.0.smithi142.stdout:9/795: write d0/d2/d61/fc3 [1427205,35166] 0 2024-03-29T04:45:12.751 INFO:tasks.workunit.client.0.smithi142.stdout:3/720: sync 2024-03-29T04:45:12.751 INFO:tasks.workunit.client.0.smithi142.stdout:7/955: sync 2024-03-29T04:45:12.752 INFO:tasks.workunit.client.0.smithi142.stdout:9/796: chown d0/d2/d6/d31 283323471 1 2024-03-29T04:45:12.752 INFO:tasks.workunit.client.1.smithi152.stdout:6/358: dread d1/d3/d9/f3a [0,4194304] 0 2024-03-29T04:45:12.752 INFO:tasks.workunit.client.0.smithi142.stdout:7/956: stat d7/d27/d3a/c54 0 2024-03-29T04:45:12.753 INFO:tasks.workunit.client.0.smithi142.stdout:9/797: readlink d0/d2/d6/d27/d3c/d6f/lef 0 2024-03-29T04:45:12.754 INFO:tasks.workunit.client.1.smithi152.stdout:0/355: symlink d2/da/d3a/l8a 0 2024-03-29T04:45:12.754 INFO:tasks.workunit.client.0.smithi142.stdout:2/877: write de/d11/d23/f28 [1053193,101973] 0 2024-03-29T04:45:12.759 INFO:tasks.workunit.client.0.smithi142.stdout:9/798: creat d0/d2/d61/d64/dfa/ffc x:0 0 0 2024-03-29T04:45:12.759 INFO:tasks.workunit.client.0.smithi142.stdout:2/878: creat de/d11/d17/d10b/d22/d27/dba/d61/d7d/db4/f125 x:0 0 0 2024-03-29T04:45:12.762 INFO:tasks.workunit.client.0.smithi142.stdout:9/799: creat d0/d12/d50/d52/ffd x:0 0 0 2024-03-29T04:45:12.762 INFO:tasks.workunit.client.0.smithi142.stdout:9/800: dread - d0/d12/d50/d52/ff2 zero size 2024-03-29T04:45:12.763 INFO:tasks.workunit.client.1.smithi152.stdout:0/356: symlink d2/da/d17/d50/d5c/l8b 0 2024-03-29T04:45:12.763 INFO:tasks.workunit.client.0.smithi142.stdout:9/801: fdatasync d0/d12/d50/d52/da3/fb1 0 2024-03-29T04:45:12.763 INFO:tasks.workunit.client.1.smithi152.stdout:0/357: chown d2/da/d3a/f68 6640 1 2024-03-29T04:45:12.765 INFO:tasks.workunit.client.0.smithi142.stdout:9/802: mknod d0/d6c/dc2/cfe 0 2024-03-29T04:45:12.766 INFO:tasks.workunit.client.0.smithi142.stdout:9/803: write d0/d12/d50/d52/ffd [709492,51754] 0 2024-03-29T04:45:12.766 INFO:tasks.workunit.client.1.smithi152.stdout:9/309: getdents d5/db/d1b 0 2024-03-29T04:45:12.767 INFO:tasks.workunit.client.1.smithi152.stdout:7/299: write d7/d8/db/df/f13 [1771993,47674] 0 2024-03-29T04:45:12.769 INFO:tasks.workunit.client.0.smithi142.stdout:9/804: creat d0/d12/d50/d52/da3/fff x:0 0 0 2024-03-29T04:45:12.770 INFO:tasks.workunit.client.1.smithi152.stdout:9/310: mkdir d5/d12/d2c/d6e 0 2024-03-29T04:45:12.771 INFO:tasks.workunit.client.0.smithi142.stdout:9/805: dread d0/d6c/f79 [0,4194304] 0 2024-03-29T04:45:12.771 INFO:tasks.workunit.client.1.smithi152.stdout:9/311: fsync d5/d8/f69 0 2024-03-29T04:45:12.772 INFO:tasks.workunit.client.1.smithi152.stdout:9/312: truncate d5/f2f 1112907 0 2024-03-29T04:45:12.772 INFO:tasks.workunit.client.1.smithi152.stdout:1/396: truncate d5/d7/d9/f7b 696568 0 2024-03-29T04:45:12.772 INFO:tasks.workunit.client.1.smithi152.stdout:7/300: creat d7/d8/db/df/f58 x:0 0 0 2024-03-29T04:45:12.773 INFO:tasks.workunit.client.1.smithi152.stdout:7/301: fsync d7/d8/d12/d15/f31 0 2024-03-29T04:45:12.774 INFO:tasks.workunit.client.0.smithi142.stdout:2/879: getdents de/d11/d17/d10b 0 2024-03-29T04:45:12.775 INFO:tasks.workunit.client.1.smithi152.stdout:9/313: fsync d5/db/d1b/f37 0 2024-03-29T04:45:12.776 INFO:tasks.workunit.client.1.smithi152.stdout:1/397: symlink d5/d7/d8/d13/l7e 0 2024-03-29T04:45:12.778 INFO:tasks.workunit.client.1.smithi152.stdout:7/302: creat d7/f59 x:0 0 0 2024-03-29T04:45:12.781 INFO:tasks.workunit.client.0.smithi142.stdout:2/880: dwrite de/d11/d17/d10b/d22/f110 [0,4194304] 0 2024-03-29T04:45:12.782 INFO:tasks.workunit.client.0.smithi142.stdout:2/881: chown de/d11/d17/d10b/lad 2085847 1 2024-03-29T04:45:12.788 INFO:tasks.workunit.client.1.smithi152.stdout:0/358: link d2/da/df/f5b d2/da/d17/d57/f8c 0 2024-03-29T04:45:12.790 INFO:tasks.workunit.client.0.smithi142.stdout:2/882: rename de/d11/d23/fb1 to de/d11/d17/d10b/d22/d27/dba/d61/d7d/db4/f126 0 2024-03-29T04:45:12.790 INFO:tasks.workunit.client.0.smithi142.stdout:2/883: chown de/d11/d17/d60/d6f/f95 3 1 2024-03-29T04:45:12.792 INFO:tasks.workunit.client.1.smithi152.stdout:9/314: dwrite d5/d8/f11 [0,4194304] 0 2024-03-29T04:45:12.795 INFO:tasks.workunit.client.1.smithi152.stdout:0/359: mknod d2/da/c8d 0 2024-03-29T04:45:12.797 INFO:tasks.workunit.client.1.smithi152.stdout:9/315: mkdir d5/d8/d31/d59/d6f 0 2024-03-29T04:45:12.798 INFO:tasks.workunit.client.1.smithi152.stdout:0/360: creat d2/da/df/f8e x:0 0 0 2024-03-29T04:45:12.803 INFO:tasks.workunit.client.1.smithi152.stdout:9/316: unlink d5/db/c36 0 2024-03-29T04:45:12.806 INFO:tasks.workunit.client.1.smithi152.stdout:9/317: mkdir d5/d70 0 2024-03-29T04:45:12.806 INFO:tasks.workunit.client.1.smithi152.stdout:7/303: dwrite d7/d8/db/f42 [4194304,4194304] 0 2024-03-29T04:45:12.807 INFO:tasks.workunit.client.1.smithi152.stdout:1/398: dread d5/f3e [0,4194304] 0 2024-03-29T04:45:12.809 INFO:tasks.workunit.client.1.smithi152.stdout:1/399: write d5/d7/d8/d13/f35 [6663744,89252] 0 2024-03-29T04:45:12.810 INFO:tasks.workunit.client.1.smithi152.stdout:1/400: stat d5/d7/d8/d13/d19/f68 0 2024-03-29T04:45:12.814 INFO:tasks.workunit.client.1.smithi152.stdout:0/361: dread d2/f1f [0,4194304] 0 2024-03-29T04:45:12.814 INFO:tasks.workunit.client.1.smithi152.stdout:1/401: rename d5/d7/d9/d12/c6b to d5/d7/c7f 0 2024-03-29T04:45:12.814 INFO:tasks.workunit.client.1.smithi152.stdout:7/304: dread d7/d8/d12/f33 [0,4194304] 0 2024-03-29T04:45:12.816 INFO:tasks.workunit.client.1.smithi152.stdout:1/402: stat d5/d7/d1b/f78 0 2024-03-29T04:45:12.817 INFO:tasks.workunit.client.1.smithi152.stdout:7/305: write d7/d8/d12/d1d/d30/f3c [1512739,90436] 0 2024-03-29T04:45:12.820 INFO:tasks.workunit.client.1.smithi152.stdout:1/403: write d5/d7/d8/d13/d19/f68 [494313,101761] 0 2024-03-29T04:45:12.820 INFO:tasks.workunit.client.1.smithi152.stdout:9/318: creat d5/d8/d31/d59/d6f/f71 x:0 0 0 2024-03-29T04:45:12.824 INFO:tasks.workunit.client.1.smithi152.stdout:0/362: dwrite d2/f8 [0,4194304] 0 2024-03-29T04:45:12.826 INFO:tasks.workunit.client.1.smithi152.stdout:7/306: getdents d7/d8/db/df 0 2024-03-29T04:45:12.829 INFO:tasks.workunit.client.1.smithi152.stdout:9/319: write d5/d12/d2c/f3f [1801199,23493] 0 2024-03-29T04:45:12.832 INFO:tasks.workunit.client.1.smithi152.stdout:1/404: dwrite d5/d7/d8/d13/f74 [4194304,4194304] 0 2024-03-29T04:45:12.833 INFO:tasks.workunit.client.1.smithi152.stdout:9/320: truncate d5/d8/f9 1423980 0 2024-03-29T04:45:12.834 INFO:tasks.workunit.client.1.smithi152.stdout:7/307: symlink d7/d8/l5a 0 2024-03-29T04:45:12.834 INFO:tasks.workunit.client.1.smithi152.stdout:4/265: sync 2024-03-29T04:45:12.834 INFO:tasks.workunit.client.1.smithi152.stdout:3/290: sync 2024-03-29T04:45:12.843 INFO:tasks.workunit.client.1.smithi152.stdout:6/359: read d1/d3/d1a/d1b/d29/d6f/f3c [490044,123758] 0 2024-03-29T04:45:12.846 INFO:tasks.workunit.client.1.smithi152.stdout:1/405: mkdir d5/d7/d8/d13/d19/d80 0 2024-03-29T04:45:12.847 INFO:tasks.workunit.client.1.smithi152.stdout:6/360: fsync d1/d3/d1a/d49/f5e 0 2024-03-29T04:45:12.851 INFO:tasks.workunit.client.0.smithi142.stdout:2/884: dread de/d11/d17/d10b/d22/d50/fdb [0,4194304] 0 2024-03-29T04:45:12.856 INFO:tasks.workunit.client.1.smithi152.stdout:3/291: mkdir d7/d19/d1b/d1e/d4e 0 2024-03-29T04:45:12.857 INFO:tasks.workunit.client.1.smithi152.stdout:4/266: creat d0/d7/d25/d48/f51 x:0 0 0 2024-03-29T04:45:12.860 INFO:tasks.workunit.client.0.smithi142.stdout:6/865: fsync d1c/d33/d46/d61/d9e/dda/f114 0 2024-03-29T04:45:12.862 INFO:tasks.workunit.client.1.smithi152.stdout:7/308: getdents d7/d8/d12/d15 0 2024-03-29T04:45:12.863 INFO:tasks.workunit.client.1.smithi152.stdout:0/363: dwrite d2/da/d17/f41 [0,4194304] 0 2024-03-29T04:45:12.864 INFO:tasks.workunit.client.1.smithi152.stdout:3/292: write d7/d19/f36 [4445496,120718] 0 2024-03-29T04:45:12.865 INFO:tasks.workunit.client.1.smithi152.stdout:6/361: dread d1/f50 [0,4194304] 0 2024-03-29T04:45:12.867 INFO:tasks.workunit.client.0.smithi142.stdout:6/866: link d1c/d45/d8a/df6/l79 d1c/d45/d51/l116 0 2024-03-29T04:45:12.870 INFO:tasks.workunit.client.1.smithi152.stdout:7/309: unlink d7/d8/d12/d15/l2f 0 2024-03-29T04:45:12.870 INFO:tasks.workunit.client.1.smithi152.stdout:6/362: chown d1/d3/d1a/d1b/l44 21466 1 2024-03-29T04:45:12.871 INFO:tasks.workunit.client.1.smithi152.stdout:1/406: dread d5/d7/d8/f2d [0,4194304] 0 2024-03-29T04:45:12.872 INFO:tasks.workunit.client.0.smithi142.stdout:3/721: write d3/d1f/d24/faf [1064503,17079] 0 2024-03-29T04:45:12.873 INFO:tasks.workunit.client.1.smithi152.stdout:7/310: symlink d7/d8/d12/d15/l5b 0 2024-03-29T04:45:12.874 INFO:tasks.workunit.client.1.smithi152.stdout:1/407: chown d5/c4e 787624 1 2024-03-29T04:45:12.874 INFO:tasks.workunit.client.0.smithi142.stdout:6/867: dwrite d1c/d45/d51/fb4 [0,4194304] 0 2024-03-29T04:45:12.878 INFO:tasks.workunit.client.1.smithi152.stdout:6/363: rename d1/d3/d1a/d1b/l44 to d1/d3/d9/d13/l7b 0 2024-03-29T04:45:12.880 INFO:tasks.workunit.client.0.smithi142.stdout:6/868: mkdir d1c/d45/d8a/d117 0 2024-03-29T04:45:12.880 INFO:tasks.workunit.client.0.smithi142.stdout:6/869: stat d1c/d33/d46/d61 0 2024-03-29T04:45:12.883 INFO:tasks.workunit.client.0.smithi142.stdout:6/870: mkdir d1c/d45/d8a/d118 0 2024-03-29T04:45:12.884 INFO:tasks.workunit.client.0.smithi142.stdout:6/871: write d1c/d45/d8a/f110 [290920,2077] 0 2024-03-29T04:45:12.884 INFO:tasks.workunit.client.1.smithi152.stdout:9/321: dread d5/db/f6c [0,4194304] 0 2024-03-29T04:45:12.885 INFO:tasks.workunit.client.0.smithi142.stdout:6/872: write d1c/d90/ddc/dfb/f106 [585481,93912] 0 2024-03-29T04:45:12.886 INFO:tasks.workunit.client.1.smithi152.stdout:0/364: fdatasync d2/f8 0 2024-03-29T04:45:12.888 INFO:tasks.workunit.client.1.smithi152.stdout:0/365: dread - d2/da/df/f4a zero size 2024-03-29T04:45:12.891 INFO:tasks.workunit.client.1.smithi152.stdout:0/366: write d2/da/d17/f41 [1946614,67288] 0 2024-03-29T04:45:12.898 INFO:tasks.workunit.client.0.smithi142.stdout:7/957: dwrite d7/d27/d44/db2/db8/d4b/d71/d30/f3e [0,4194304] 0 2024-03-29T04:45:12.903 INFO:tasks.workunit.client.1.smithi152.stdout:4/267: write d0/d7/d40/ff [174668,46694] 0 2024-03-29T04:45:12.903 INFO:tasks.workunit.client.0.smithi142.stdout:4/839: sync 2024-03-29T04:45:12.903 INFO:tasks.workunit.client.0.smithi142.stdout:1/798: sync 2024-03-29T04:45:12.903 INFO:tasks.workunit.client.0.smithi142.stdout:0/923: sync 2024-03-29T04:45:12.904 INFO:tasks.workunit.client.0.smithi142.stdout:0/924: stat d7/l12 0 2024-03-29T04:45:12.905 INFO:tasks.workunit.client.1.smithi152.stdout:0/367: dread d2/d29/f2f [0,4194304] 0 2024-03-29T04:45:12.906 INFO:tasks.workunit.client.0.smithi142.stdout:1/799: chown d8/db/dc/d77/d31/d4d/d8c/f95 0 1 2024-03-29T04:45:12.908 INFO:tasks.workunit.client.0.smithi142.stdout:3/722: truncate d3/d1f/f46 3855913 0 2024-03-29T04:45:12.913 INFO:tasks.workunit.client.1.smithi152.stdout:3/293: dwrite d7/d19/d3a/f49 [0,4194304] 0 2024-03-29T04:45:12.913 INFO:tasks.workunit.client.1.smithi152.stdout:0/368: mknod d2/da/d17/d57/c8f 0 2024-03-29T04:45:12.915 INFO:tasks.workunit.client.1.smithi152.stdout:6/364: write d1/f8 [5076831,100460] 0 2024-03-29T04:45:12.915 INFO:tasks.workunit.client.1.smithi152.stdout:9/322: write d5/f20 [509415,25770] 0 2024-03-29T04:45:12.916 INFO:tasks.workunit.client.1.smithi152.stdout:4/268: mknod d0/d7/c52 0 2024-03-29T04:45:12.916 INFO:tasks.workunit.client.1.smithi152.stdout:7/311: dread d7/d8/d12/f22 [0,4194304] 0 2024-03-29T04:45:12.918 INFO:tasks.workunit.client.1.smithi152.stdout:1/408: dread d5/f3c [4194304,4194304] 0 2024-03-29T04:45:12.919 INFO:tasks.workunit.client.1.smithi152.stdout:3/294: chown d7/d19/d1b/d1e/d22 257447178 1 2024-03-29T04:45:12.923 INFO:tasks.workunit.client.1.smithi152.stdout:7/312: rmdir d7/d8/d12/d1d/d41 39 2024-03-29T04:45:12.925 INFO:tasks.workunit.client.1.smithi152.stdout:4/269: creat d0/d7/d40/d23/f53 x:0 0 0 2024-03-29T04:45:12.926 INFO:tasks.workunit.client.0.smithi142.stdout:7/958: mkdir d7/d27/d44/db2/db8/d4b/d71/d30/d13d 0 2024-03-29T04:45:12.926 INFO:tasks.workunit.client.1.smithi152.stdout:9/323: creat d5/d8/d31/f72 x:0 0 0 2024-03-29T04:45:12.929 INFO:tasks.workunit.client.0.smithi142.stdout:4/840: symlink d0/d9/d11/d4c/d64/d94/ddd/l117 0 2024-03-29T04:45:12.930 INFO:tasks.workunit.client.1.smithi152.stdout:9/324: chown d5/db/d1b/c24 0 1 2024-03-29T04:45:12.930 INFO:tasks.workunit.client.1.smithi152.stdout:0/369: rename d2/da/d17/d50/c82 to d2/da/d3a/d6f/c90 0 2024-03-29T04:45:12.931 INFO:tasks.workunit.client.1.smithi152.stdout:1/409: dwrite d5/d7/d1b/f5e [0,4194304] 0 2024-03-29T04:45:12.933 INFO:tasks.workunit.client.0.smithi142.stdout:1/800: rename d8/db/dc/d77/db9/dbe/d1f/d50 to d8/db/dc/d77/db9/dbe/d2d/d44/d7f/d10e 0 2024-03-29T04:45:12.933 INFO:tasks.workunit.client.1.smithi152.stdout:0/370: creat d2/da/d3a/f91 x:0 0 0 2024-03-29T04:45:12.934 INFO:tasks.workunit.client.0.smithi142.stdout:1/801: write d8/db/dc/d77/db9/dbe/d2d/d44/d53/f7b [2677914,68618] 0 2024-03-29T04:45:12.935 INFO:tasks.workunit.client.1.smithi152.stdout:4/270: rename d0/d27/f39 to d0/d7/d25/d17/f54 0 2024-03-29T04:45:12.935 INFO:tasks.workunit.client.1.smithi152.stdout:9/325: mkdir d5/d12/d2c/d6e/d73 0 2024-03-29T04:45:12.936 INFO:tasks.workunit.client.1.smithi152.stdout:0/371: readlink d2/l6 0 2024-03-29T04:45:12.938 INFO:tasks.workunit.client.1.smithi152.stdout:4/271: mkdir d0/d7/d40/d23/d55 0 2024-03-29T04:45:12.939 INFO:tasks.workunit.client.0.smithi142.stdout:0/925: mknod d7/d13/c12e 0 2024-03-29T04:45:12.940 INFO:tasks.workunit.client.0.smithi142.stdout:3/723: dwrite d3/d29/d2b/d40/d67/d8b/f91 [0,4194304] 0 2024-03-29T04:45:12.941 INFO:tasks.workunit.client.0.smithi142.stdout:6/873: link d1c/l24 d1c/d45/d8a/df6/df4/l119 0 2024-03-29T04:45:12.942 INFO:tasks.workunit.client.1.smithi152.stdout:4/272: fsync d0/f13 0 2024-03-29T04:45:12.942 INFO:tasks.workunit.client.0.smithi142.stdout:3/724: fsync d3/d1f/d24/faf 0 2024-03-29T04:45:12.943 INFO:tasks.workunit.client.0.smithi142.stdout:4/841: symlink d0/l118 0 2024-03-29T04:45:12.943 INFO:tasks.workunit.client.1.smithi152.stdout:1/410: dwrite d5/d7/d9/d59/f61 [0,4194304] 0 2024-03-29T04:45:12.944 INFO:tasks.workunit.client.1.smithi152.stdout:0/372: symlink d2/da/d17/l92 0 2024-03-29T04:45:12.946 INFO:tasks.workunit.client.0.smithi142.stdout:4/842: truncate d0/d9/db2/dba/dbc/d8d/da5/d102/f114 104382 0 2024-03-29T04:45:12.947 INFO:tasks.workunit.client.0.smithi142.stdout:3/725: chown d3/d29/d2b/fe8 1 1 2024-03-29T04:45:12.948 INFO:tasks.workunit.client.0.smithi142.stdout:6/874: truncate d1c/d33/d46/f65 630782 0 2024-03-29T04:45:12.948 INFO:tasks.workunit.client.0.smithi142.stdout:4/843: truncate d0/d9/d11/ff9 126772 0 2024-03-29T04:45:12.949 INFO:tasks.workunit.client.1.smithi152.stdout:4/273: mknod d0/d27/c56 0 2024-03-29T04:45:12.950 INFO:tasks.workunit.client.0.smithi142.stdout:0/926: mkdir d7/d13/dbf/d12f 0 2024-03-29T04:45:12.950 INFO:tasks.workunit.client.0.smithi142.stdout:6/875: write d1c/d45/d8a/df6/f95 [5126865,16116] 0 2024-03-29T04:45:12.951 INFO:tasks.workunit.client.1.smithi152.stdout:9/326: creat d5/d70/f74 x:0 0 0 2024-03-29T04:45:12.951 INFO:tasks.workunit.client.0.smithi142.stdout:4/844: write d0/dc/d15/d3d/d65/f7c [5742071,6483] 0 2024-03-29T04:45:12.951 INFO:tasks.workunit.client.1.smithi152.stdout:0/373: creat d2/da/df/f93 x:0 0 0 2024-03-29T04:45:12.953 INFO:tasks.workunit.client.0.smithi142.stdout:6/876: chown d1c/d33/db6/l4b 1004 1 2024-03-29T04:45:12.958 INFO:tasks.workunit.client.1.smithi152.stdout:4/274: rename d0/d7/d40/d3a to d0/d7/d25/d48/d57 0 2024-03-29T04:45:12.959 INFO:tasks.workunit.client.1.smithi152.stdout:6/365: dread d1/d3/df/f66 [0,4194304] 0 2024-03-29T04:45:12.960 INFO:tasks.workunit.client.0.smithi142.stdout:1/802: getdents d8/db/dc/d77/d5a 0 2024-03-29T04:45:12.961 INFO:tasks.workunit.client.1.smithi152.stdout:1/411: dwrite d5/f3f [0,4194304] 0 2024-03-29T04:45:12.962 INFO:tasks.workunit.client.1.smithi152.stdout:6/366: chown d1/d3/d1a/d1b 3580297 1 2024-03-29T04:45:12.964 INFO:tasks.workunit.client.0.smithi142.stdout:4/845: mknod d0/d9/d11/d4c/d5f/d81/dc9/d47/c119 0 2024-03-29T04:45:12.965 INFO:tasks.workunit.client.0.smithi142.stdout:4/846: chown d0/d12/f3b 205093904 1 2024-03-29T04:45:12.967 INFO:tasks.workunit.client.0.smithi142.stdout:1/803: chown d8/db/dc/d77/db9/dbe/d1f/d4c/fb1 4653 1 2024-03-29T04:45:12.968 INFO:tasks.workunit.client.1.smithi152.stdout:4/275: mknod d0/d7/d40/d2a/c58 0 2024-03-29T04:45:12.969 INFO:tasks.workunit.client.1.smithi152.stdout:3/295: truncate d7/f10 4152336 0 2024-03-29T04:45:12.970 INFO:tasks.workunit.client.1.smithi152.stdout:1/412: dwrite d5/d7/f25 [4194304,4194304] 0 2024-03-29T04:45:12.971 INFO:tasks.workunit.client.1.smithi152.stdout:6/367: write d1/d3/df/f4f [4428187,79893] 0 2024-03-29T04:45:12.973 INFO:tasks.workunit.client.1.smithi152.stdout:1/413: read - d5/f72 zero size 2024-03-29T04:45:12.973 INFO:tasks.workunit.client.0.smithi142.stdout:3/726: rename d3/d29/d2b/d40/d67/fdb to d3/d29/d2d/ff8 0 2024-03-29T04:45:12.974 INFO:tasks.workunit.client.0.smithi142.stdout:3/727: chown d3/d29/d2b/d40/d67/d8b/d4d/f60 1664 1 2024-03-29T04:45:12.975 INFO:tasks.workunit.client.1.smithi152.stdout:0/374: getdents d2/da/d17/d57 0 2024-03-29T04:45:12.976 INFO:tasks.workunit.client.0.smithi142.stdout:4/847: unlink d0/dc/d15/f59 0 2024-03-29T04:45:12.978 INFO:tasks.workunit.client.1.smithi152.stdout:3/296: unlink d7/d19/f38 0 2024-03-29T04:45:12.979 INFO:tasks.workunit.client.1.smithi152.stdout:0/375: truncate d2/da/d17/d50/d5c/f7b 763145 0 2024-03-29T04:45:12.979 INFO:tasks.workunit.client.1.smithi152.stdout:1/414: symlink d5/d7/d9/l81 0 2024-03-29T04:45:12.980 INFO:tasks.workunit.client.0.smithi142.stdout:6/877: creat d1c/f11a x:0 0 0 2024-03-29T04:45:12.983 INFO:tasks.workunit.client.0.smithi142.stdout:3/728: dwrite d3/d29/d2b/d40/ff3 [0,4194304] 0 2024-03-29T04:45:12.983 INFO:tasks.workunit.client.1.smithi152.stdout:9/327: write d5/d8/d31/f4c [910008,44529] 0 2024-03-29T04:45:12.984 INFO:tasks.workunit.client.0.smithi142.stdout:4/848: creat d0/d9/db2/dba/dbc/daf/f11a x:0 0 0 2024-03-29T04:45:12.984 INFO:tasks.workunit.client.1.smithi152.stdout:0/376: mknod d2/da/d3a/d61/c94 0 2024-03-29T04:45:12.986 INFO:tasks.workunit.client.0.smithi142.stdout:4/849: chown d0/d9/d11/d4c/d5f/d81/dc9/d47/d7a/f8b 1361702169 1 2024-03-29T04:45:12.986 INFO:tasks.workunit.client.1.smithi152.stdout:1/415: write d5/d7/d8/d13/f62 [5163173,82962] 0 2024-03-29T04:45:12.987 INFO:tasks.workunit.client.1.smithi152.stdout:9/328: chown d5/f2f 1375 1 2024-03-29T04:45:12.988 INFO:tasks.workunit.client.1.smithi152.stdout:0/377: symlink d2/da/d3a/d61/l95 0 2024-03-29T04:45:12.989 INFO:tasks.workunit.client.1.smithi152.stdout:0/378: truncate d2/f78 191188 0 2024-03-29T04:45:12.989 INFO:tasks.workunit.client.0.smithi142.stdout:6/878: creat d1c/dc7/df2/f11b x:0 0 0 2024-03-29T04:45:12.990 INFO:tasks.workunit.client.0.smithi142.stdout:1/804: creat d8/db/dc/d77/d31/d4d/f10f x:0 0 0 2024-03-29T04:45:12.992 INFO:tasks.workunit.client.1.smithi152.stdout:9/329: mkdir d5/d8/d31/d75 0 2024-03-29T04:45:12.993 INFO:tasks.workunit.client.1.smithi152.stdout:0/379: write d2/d31/f3c [881082,92389] 0 2024-03-29T04:45:12.994 INFO:tasks.workunit.client.1.smithi152.stdout:9/330: read - d5/d8/f69 zero size 2024-03-29T04:45:12.995 INFO:tasks.workunit.client.0.smithi142.stdout:0/927: dread d7/d9/d55/d42/f46 [4194304,4194304] 0 2024-03-29T04:45:12.995 INFO:tasks.workunit.client.0.smithi142.stdout:4/850: read d0/d9/db2/dba/dbc/d6d/d6e/fc4 [46919,41949] 0 2024-03-29T04:45:12.995 INFO:tasks.workunit.client.1.smithi152.stdout:1/416: rename d5/d7/d8/d13/d19/l57 to d5/d7/d9/d2b/l82 0 2024-03-29T04:45:12.996 INFO:tasks.workunit.client.0.smithi142.stdout:6/879: mkdir d1c/d33/d46/d66/dcd/d11c 0 2024-03-29T04:45:12.996 INFO:tasks.workunit.client.1.smithi152.stdout:0/380: mknod d2/da/d17/d50/c96 0 2024-03-29T04:45:12.997 INFO:tasks.workunit.client.1.smithi152.stdout:1/417: chown d5/d7/d8/d13/d19/d80 483852 1 2024-03-29T04:45:12.998 INFO:tasks.workunit.client.1.smithi152.stdout:0/381: dread - d2/da/df/f93 zero size 2024-03-29T04:45:12.999 INFO:tasks.workunit.client.0.smithi142.stdout:4/851: chown d0/d9/d11/d4c/d64/d94/ddd/ff4 886324122 1 2024-03-29T04:45:13.000 INFO:tasks.workunit.client.0.smithi142.stdout:1/805: dwrite d8/db/dc/d77/db9/dbe/d1f/d4c/fc4 [0,4194304] 0 2024-03-29T04:45:13.001 INFO:tasks.workunit.client.1.smithi152.stdout:0/382: creat d2/da/d3a/f97 x:0 0 0 2024-03-29T04:45:13.001 INFO:tasks.workunit.client.1.smithi152.stdout:1/418: symlink d5/d7/d9/l83 0 2024-03-29T04:45:13.002 INFO:tasks.workunit.client.0.smithi142.stdout:1/806: chown d8/db/dc/d77/d31/d4d/d8c/f95 24528 1 2024-03-29T04:45:13.002 INFO:tasks.workunit.client.0.smithi142.stdout:0/928: symlink d7/d9/d1e/d81/l130 0 2024-03-29T04:45:13.003 INFO:tasks.workunit.client.0.smithi142.stdout:6/880: read d1c/d33/d46/d61/d9e/fef [3316738,121278] 0 2024-03-29T04:45:13.004 INFO:tasks.workunit.client.1.smithi152.stdout:9/331: rename d5/d12/f6d to d5/d12/d2c/f76 0 2024-03-29T04:45:13.004 INFO:tasks.workunit.client.0.smithi142.stdout:0/929: fdatasync d7/d13/d4a/d9a/da7/fd4 0 2024-03-29T04:45:13.004 INFO:tasks.workunit.client.0.smithi142.stdout:6/881: dread - d1c/d45/d8a/fea zero size 2024-03-29T04:45:13.004 INFO:tasks.workunit.client.1.smithi152.stdout:0/383: dread d2/f78 [0,4194304] 0 2024-03-29T04:45:13.006 INFO:tasks.workunit.client.0.smithi142.stdout:6/882: truncate d1c/d33/d46/d71/fb3 4818629 0 2024-03-29T04:45:13.008 INFO:tasks.workunit.client.1.smithi152.stdout:1/419: rename d5/d7/d1b/c42 to d5/d7/d9/d2b/c84 0 2024-03-29T04:45:13.009 INFO:tasks.workunit.client.1.smithi152.stdout:0/384: unlink d2/d29/l60 0 2024-03-29T04:45:13.009 INFO:tasks.workunit.client.0.smithi142.stdout:7/959: write d7/d27/d44/db2/db8/d4b/d71/d30/f99 [714366,92137] 0 2024-03-29T04:45:13.010 INFO:tasks.workunit.client.0.smithi142.stdout:0/930: mknod d7/d9/d6e/c131 0 2024-03-29T04:45:13.011 INFO:tasks.workunit.client.1.smithi152.stdout:1/420: dread - d5/d7/d38/f7c zero size 2024-03-29T04:45:13.011 INFO:tasks.workunit.client.1.smithi152.stdout:9/332: mknod d5/d8/d31/c77 0 2024-03-29T04:45:13.012 INFO:tasks.workunit.client.1.smithi152.stdout:6/368: dread d1/d3/d1a/f61 [0,4194304] 0 2024-03-29T04:45:13.013 INFO:tasks.workunit.client.1.smithi152.stdout:9/333: write d5/db/d1b/f2b [579401,129268] 0 2024-03-29T04:45:13.014 INFO:tasks.workunit.client.1.smithi152.stdout:1/421: chown d5/d7/d8/d13/f45 10 1 2024-03-29T04:45:13.016 INFO:tasks.workunit.client.1.smithi152.stdout:1/422: fsync d5/d7/d8/d13/f62 0 2024-03-29T04:45:13.024 INFO:tasks.workunit.client.1.smithi152.stdout:9/334: rmdir d5/d12 39 2024-03-29T04:45:13.027 INFO:tasks.workunit.client.1.smithi152.stdout:1/423: dread d5/d7/d9/d2b/f2f [0,4194304] 0 2024-03-29T04:45:13.032 INFO:tasks.workunit.client.1.smithi152.stdout:9/335: creat d5/d12/d2c/f78 x:0 0 0 2024-03-29T04:45:13.033 INFO:tasks.workunit.client.0.smithi142.stdout:6/883: link d1c/d33/d46/d61/d9b/ccb d1c/d33/d46/d86/dc0/d113/c11d 0 2024-03-29T04:45:13.034 INFO:tasks.workunit.client.0.smithi142.stdout:3/729: dwrite d3/d1f/f7e [0,4194304] 0 2024-03-29T04:45:13.034 INFO:tasks.workunit.client.1.smithi152.stdout:9/336: dread - d5/d8/f69 zero size 2024-03-29T04:45:13.036 INFO:tasks.workunit.client.1.smithi152.stdout:0/385: dread d2/f9 [4194304,4194304] 0 2024-03-29T04:45:13.036 INFO:tasks.workunit.client.0.smithi142.stdout:7/960: getdents d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/dc1 0 2024-03-29T04:45:13.036 INFO:tasks.workunit.client.0.smithi142.stdout:6/884: read d1c/f49 [2698936,48987] 0 2024-03-29T04:45:13.038 INFO:tasks.workunit.client.0.smithi142.stdout:6/885: chown d1c/d33/d46/d71/fdd 114097 1 2024-03-29T04:45:13.039 INFO:tasks.workunit.client.0.smithi142.stdout:0/931: getdents d7/d9/d6e 0 2024-03-29T04:45:13.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:12 smithi152 ceph-mon[37177]: pgmap v272: 65 pgs: 65 active+clean; 2.1 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 72 MiB/s rd, 223 MiB/s wr, 390 op/s 2024-03-29T04:45:13.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:12 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:13.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:12 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi152.usoolh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:45:13.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:12 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:45:13.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:12 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:45:13.040 INFO:tasks.workunit.client.0.smithi142.stdout:7/961: chown d7/d27/d44/db2/db8/d4b/d71/d30/d33/f4d 1966 1 2024-03-29T04:45:13.041 INFO:tasks.workunit.client.0.smithi142.stdout:0/932: chown d7/f52 0 1 2024-03-29T04:45:13.042 INFO:tasks.workunit.client.1.smithi152.stdout:9/337: truncate d5/db/f21 4284464 0 2024-03-29T04:45:13.044 INFO:tasks.workunit.client.1.smithi152.stdout:8/229: sync 2024-03-29T04:45:13.044 INFO:tasks.workunit.client.1.smithi152.stdout:5/352: sync 2024-03-29T04:45:13.044 INFO:tasks.workunit.client.1.smithi152.stdout:2/232: sync 2024-03-29T04:45:13.046 INFO:tasks.workunit.client.0.smithi142.stdout:6/886: mknod d1c/d45/d8a/d118/c11e 0 2024-03-29T04:45:13.047 INFO:tasks.workunit.client.0.smithi142.stdout:3/730: dread d3/d29/d2b/d40/d67/d8b/d4d/d50/f51 [0,4194304] 0 2024-03-29T04:45:13.047 INFO:tasks.workunit.client.1.smithi152.stdout:0/386: dwrite d2/da/d3a/f97 [0,4194304] 0 2024-03-29T04:45:13.048 INFO:tasks.workunit.client.0.smithi142.stdout:1/807: dwrite d8/db/dc/d77/db9/dbe/d2d/d44/d53/f6a [0,4194304] 0 2024-03-29T04:45:13.049 INFO:tasks.workunit.client.1.smithi152.stdout:5/353: write d6/d16/d47/d56/d63/f6d [288970,78220] 0 2024-03-29T04:45:13.050 INFO:tasks.workunit.client.0.smithi142.stdout:6/887: chown d1c/d33/d46/d61/fee 19773641 1 2024-03-29T04:45:13.050 INFO:tasks.workunit.client.1.smithi152.stdout:2/233: write d0/d3/f39 [230722,125123] 0 2024-03-29T04:45:13.052 INFO:tasks.workunit.client.1.smithi152.stdout:1/424: dwrite d5/d7/d1b/f70 [0,4194304] 0 2024-03-29T04:45:13.053 INFO:tasks.workunit.client.1.smithi152.stdout:9/338: getdents d5/d8/d31/d59 0 2024-03-29T04:45:13.056 INFO:tasks.workunit.client.0.smithi142.stdout:0/933: dwrite d7/d13/d4a/d9a/da7/f105 [0,4194304] 0 2024-03-29T04:45:13.056 INFO:tasks.workunit.client.1.smithi152.stdout:8/230: creat d1/d2/d1d/f4f x:0 0 0 2024-03-29T04:45:13.057 INFO:tasks.workunit.client.0.smithi142.stdout:6/888: unlink d1c/d45/d8a/df6/f95 0 2024-03-29T04:45:13.060 INFO:tasks.workunit.client.1.smithi152.stdout:8/231: chown d1/d2/d3/l14 105596 1 2024-03-29T04:45:13.061 INFO:tasks.workunit.client.1.smithi152.stdout:0/387: creat d2/da/f98 x:0 0 0 2024-03-29T04:45:13.063 INFO:tasks.workunit.client.1.smithi152.stdout:0/388: dread - d2/da/d3a/d6f/f85 zero size 2024-03-29T04:45:13.066 INFO:tasks.workunit.client.1.smithi152.stdout:8/232: read - d1/d21/f2a zero size 2024-03-29T04:45:13.067 INFO:tasks.workunit.client.1.smithi152.stdout:2/234: dwrite d0/d3/f39 [0,4194304] 0 2024-03-29T04:45:13.068 INFO:tasks.workunit.client.0.smithi142.stdout:6/889: rename d1c/d33/d46/d66/ld8 to d1c/d33/l11f 0 2024-03-29T04:45:13.069 INFO:tasks.workunit.client.0.smithi142.stdout:0/934: dwrite d7/d13/d4a/d9a/da7/f11a [0,4194304] 0 2024-03-29T04:45:13.069 INFO:tasks.workunit.client.1.smithi152.stdout:9/339: creat d5/db/f79 x:0 0 0 2024-03-29T04:45:13.070 INFO:tasks.workunit.client.1.smithi152.stdout:0/389: symlink d2/da/d3a/d6f/l99 0 2024-03-29T04:45:13.070 INFO:tasks.workunit.client.0.smithi142.stdout:0/935: chown d7/d9/d6e/fec 100678002 1 2024-03-29T04:45:13.071 INFO:tasks.workunit.client.1.smithi152.stdout:5/354: dwrite d6/f21 [0,4194304] 0 2024-03-29T04:45:13.071 INFO:tasks.workunit.client.1.smithi152.stdout:8/233: mknod d1/d21/c50 0 2024-03-29T04:45:13.072 INFO:tasks.workunit.client.0.smithi142.stdout:6/890: truncate d1c/d33/d46/d61/fe6 4698453 0 2024-03-29T04:45:13.073 INFO:tasks.workunit.client.1.smithi152.stdout:0/390: write d2/d29/f37 [219042,111054] 0 2024-03-29T04:45:13.075 INFO:tasks.workunit.client.0.smithi142.stdout:6/891: symlink d1c/d33/d46/d61/d9e/l120 0 2024-03-29T04:45:13.076 INFO:tasks.workunit.client.0.smithi142.stdout:0/936: symlink d7/d9/d55/l132 0 2024-03-29T04:45:13.079 INFO:tasks.workunit.client.1.smithi152.stdout:2/235: rename d0/d3/d30/f1b to d0/d3/d30/d15/d22/d4c/f4d 0 2024-03-29T04:45:13.080 INFO:tasks.workunit.client.1.smithi152.stdout:8/234: fsync d1/f34 0 2024-03-29T04:45:13.085 INFO:tasks.workunit.client.1.smithi152.stdout:0/391: rename d2/f1f to d2/da/d3a/d61/f9a 0 2024-03-29T04:45:13.088 INFO:tasks.workunit.client.1.smithi152.stdout:8/235: mkdir d1/d2/d1d/d36/d4b/d51 0 2024-03-29T04:45:13.089 INFO:tasks.workunit.client.0.smithi142.stdout:0/937: dwrite d7/d13/dbf/f8e [4194304,4194304] 0 2024-03-29T04:45:13.091 INFO:tasks.workunit.client.1.smithi152.stdout:9/340: getdents d5/d12 0 2024-03-29T04:45:13.092 INFO:tasks.workunit.client.1.smithi152.stdout:9/341: readlink d5/d8/l14 0 2024-03-29T04:45:13.092 INFO:tasks.workunit.client.1.smithi152.stdout:2/236: rename d0/d3/f2e to d0/d3/df/d2c/d45/f4e 0 2024-03-29T04:45:13.095 INFO:tasks.workunit.client.1.smithi152.stdout:2/237: dread - d0/d3/df/f3e zero size 2024-03-29T04:45:13.097 INFO:tasks.workunit.client.0.smithi142.stdout:0/938: symlink d7/d13/dbf/dca/dab/l133 0 2024-03-29T04:45:13.097 INFO:tasks.workunit.client.0.smithi142.stdout:7/962: write d7/d27/d3a/d92/d97/faf [621408,123926] 0 2024-03-29T04:45:13.100 INFO:tasks.workunit.client.1.smithi152.stdout:8/236: unlink d1/d21/c3e 0 2024-03-29T04:45:13.102 INFO:tasks.workunit.client.1.smithi152.stdout:2/238: symlink d0/d3/df/d2c/l4f 0 2024-03-29T04:45:13.103 INFO:tasks.workunit.client.1.smithi152.stdout:1/425: dread d5/d7/d8/d13/d19/f30 [0,4194304] 0 2024-03-29T04:45:13.103 INFO:tasks.workunit.client.1.smithi152.stdout:9/342: mkdir d5/d12/d2c/d6e/d73/d7a 0 2024-03-29T04:45:13.103 INFO:tasks.workunit.client.0.smithi142.stdout:7/963: dread d7/d27/d44/db2/db8/d4b/d71/d30/fb1 [0,4194304] 0 2024-03-29T04:45:13.104 INFO:tasks.workunit.client.1.smithi152.stdout:9/343: fsync d5/d8/f11 0 2024-03-29T04:45:13.105 INFO:tasks.workunit.client.1.smithi152.stdout:8/237: creat d1/d21/d40/f52 x:0 0 0 2024-03-29T04:45:13.105 INFO:tasks.workunit.client.1.smithi152.stdout:1/426: write d5/d7/d9/d2b/f37 [2466942,107730] 0 2024-03-29T04:45:13.106 INFO:tasks.workunit.client.0.smithi142.stdout:1/808: dread d8/db/dc/d77/d5a/fc6 [0,4194304] 0 2024-03-29T04:45:13.106 INFO:tasks.workunit.client.0.smithi142.stdout:9/806: sync 2024-03-29T04:45:13.108 INFO:tasks.workunit.client.1.smithi152.stdout:2/239: symlink d0/d3/df/d2c/l50 0 2024-03-29T04:45:13.108 INFO:tasks.workunit.client.0.smithi142.stdout:7/964: creat d7/d27/d44/db2/f13e x:0 0 0 2024-03-29T04:45:13.109 INFO:tasks.workunit.client.1.smithi152.stdout:2/240: chown d0/d3/d30/d15/f25 0 1 2024-03-29T04:45:13.110 INFO:tasks.workunit.client.0.smithi142.stdout:9/807: symlink d0/d2/d6/d31/d83/l100 0 2024-03-29T04:45:13.110 INFO:tasks.workunit.client.1.smithi152.stdout:9/344: read d5/db/d1b/f40 [92378,12136] 0 2024-03-29T04:45:13.110 INFO:tasks.workunit.client.1.smithi152.stdout:1/427: rename d5/d7/d4d/l7d to d5/d7/d9/l85 0 2024-03-29T04:45:13.111 INFO:tasks.workunit.client.1.smithi152.stdout:2/241: dread - d0/d3/d30/d15/f26 zero size 2024-03-29T04:45:13.111 INFO:tasks.workunit.client.0.smithi142.stdout:7/965: unlink d7/c106 0 2024-03-29T04:45:13.113 INFO:tasks.workunit.client.0.smithi142.stdout:9/808: write d0/fe1 [2179996,124601] 0 2024-03-29T04:45:13.113 INFO:tasks.workunit.client.0.smithi142.stdout:1/809: dwrite d8/db/dc/d77/fdf [0,4194304] 0 2024-03-29T04:45:13.115 INFO:tasks.workunit.client.1.smithi152.stdout:1/428: unlink d5/d7/d9/l44 0 2024-03-29T04:45:13.118 INFO:tasks.workunit.client.1.smithi152.stdout:8/238: dwrite d1/d21/f46 [4194304,4194304] 0 2024-03-29T04:45:13.118 INFO:tasks.workunit.client.0.smithi142.stdout:9/809: write d0/d12/d50/d52/da3/fff [567519,27174] 0 2024-03-29T04:45:13.118 INFO:tasks.workunit.client.0.smithi142.stdout:7/966: chown d7/d27/d44/db2/db8/d4b/d71/d30/d33/df5/f123 18 1 2024-03-29T04:45:13.119 INFO:tasks.workunit.client.1.smithi152.stdout:9/345: rename d5/db/d1b/f55 to d5/d70/f7b 0 2024-03-29T04:45:13.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:12 smithi142 ceph-mon[29027]: pgmap v272: 65 pgs: 65 active+clean; 2.1 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 72 MiB/s rd, 223 MiB/s wr, 390 op/s 2024-03-29T04:45:13.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:12 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:13.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:12 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi152.usoolh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:45:13.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:12 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:45:13.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:12 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:45:13.121 INFO:tasks.workunit.client.1.smithi152.stdout:2/242: creat d0/d3/d30/d15/d22/d4c/f51 x:0 0 0 2024-03-29T04:45:13.121 INFO:tasks.workunit.client.0.smithi142.stdout:9/810: truncate d0/d2/d6/d27/dc7/daa/d95/fd0 28028 0 2024-03-29T04:45:13.122 INFO:tasks.workunit.client.1.smithi152.stdout:1/429: creat d5/d7/d1b/d2e/f86 x:0 0 0 2024-03-29T04:45:13.130 INFO:tasks.workunit.client.1.smithi152.stdout:8/239: truncate d1/f10 4031273 0 2024-03-29T04:45:13.132 INFO:tasks.workunit.client.0.smithi142.stdout:9/811: creat d0/d2/d6/d31/f101 x:0 0 0 2024-03-29T04:45:13.133 INFO:tasks.workunit.client.1.smithi152.stdout:9/346: read d5/f5c [2053931,80586] 0 2024-03-29T04:45:13.134 INFO:tasks.workunit.client.1.smithi152.stdout:0/392: write d2/da/d3a/f87 [2675304,82104] 0 2024-03-29T04:45:13.135 INFO:tasks.workunit.client.1.smithi152.stdout:8/240: write d1/d21/f25 [110672,66594] 0 2024-03-29T04:45:13.136 INFO:tasks.workunit.client.0.smithi142.stdout:0/939: dread d7/d13/f28 [0,4194304] 0 2024-03-29T04:45:13.137 INFO:tasks.workunit.client.0.smithi142.stdout:9/812: symlink d0/d12/l102 0 2024-03-29T04:45:13.139 INFO:tasks.workunit.client.1.smithi152.stdout:0/393: readlink d2/da/d3a/d61/l88 0 2024-03-29T04:45:13.140 INFO:tasks.workunit.client.1.smithi152.stdout:2/243: dwrite d0/d3/f1d [0,4194304] 0 2024-03-29T04:45:13.140 INFO:tasks.workunit.client.1.smithi152.stdout:9/347: write d5/db/d1b/f51 [891182,100402] 0 2024-03-29T04:45:13.140 INFO:tasks.workunit.client.1.smithi152.stdout:1/430: dwrite d5/d7/d9/f46 [0,4194304] 0 2024-03-29T04:45:13.140 INFO:tasks.workunit.client.1.smithi152.stdout:8/241: fdatasync d1/d2/d3/f44 0 2024-03-29T04:45:13.140 INFO:tasks.workunit.client.0.smithi142.stdout:6/892: dwrite d1c/f39 [0,4194304] 0 2024-03-29T04:45:13.140 INFO:tasks.workunit.client.1.smithi152.stdout:8/242: dread - d1/d2/d3/f44 zero size 2024-03-29T04:45:13.141 INFO:tasks.workunit.client.0.smithi142.stdout:6/893: fdatasync d1c/d45/d8a/fe0 0 2024-03-29T04:45:13.143 INFO:tasks.workunit.client.0.smithi142.stdout:0/940: rmdir d7/d9/d1e/d123/de1 39 2024-03-29T04:45:13.143 INFO:tasks.workunit.client.0.smithi142.stdout:7/967: write d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/d3f/fae [1582296,5391] 0 2024-03-29T04:45:13.144 INFO:tasks.workunit.client.1.smithi152.stdout:8/243: write d1/d21/f2a [803653,41012] 0 2024-03-29T04:45:13.145 INFO:tasks.workunit.client.0.smithi142.stdout:9/813: unlink d0/d2/d61/d64/ced 0 2024-03-29T04:45:13.148 INFO:tasks.workunit.client.1.smithi152.stdout:2/244: mkdir d0/d3/d30/d15/d22/d4c/d52 0 2024-03-29T04:45:13.149 INFO:tasks.workunit.client.0.smithi142.stdout:1/810: dread d8/db/dc/d77/db9/dbe/d2d/d44/d53/f5f [0,4194304] 0 2024-03-29T04:45:13.150 INFO:tasks.workunit.client.0.smithi142.stdout:1/811: stat f5 0 2024-03-29T04:45:13.153 INFO:tasks.workunit.client.1.smithi152.stdout:0/394: creat d2/f9b x:0 0 0 2024-03-29T04:45:13.156 INFO:tasks.workunit.client.1.smithi152.stdout:8/244: write d1/f34 [872946,2922] 0 2024-03-29T04:45:13.157 INFO:tasks.workunit.client.0.smithi142.stdout:6/894: rename d1c/d90/d94/fa2 to d1c/d33/d46/d61/d112/f121 0 2024-03-29T04:45:13.158 INFO:tasks.workunit.client.1.smithi152.stdout:9/348: dwrite d5/f61 [0,4194304] 0 2024-03-29T04:45:13.158 INFO:tasks.workunit.client.1.smithi152.stdout:1/431: dread d5/d7/d6a/f6e [0,4194304] 0 2024-03-29T04:45:13.158 INFO:tasks.workunit.client.1.smithi152.stdout:8/245: fsync d1/d2/d1d/f3a 0 2024-03-29T04:45:13.161 INFO:tasks.workunit.client.1.smithi152.stdout:0/395: mkdir d2/da/d17/d57/d9c 0 2024-03-29T04:45:13.162 INFO:tasks.workunit.client.1.smithi152.stdout:0/396: stat d2/d29/f2f 0 2024-03-29T04:45:13.164 INFO:tasks.workunit.client.1.smithi152.stdout:1/432: symlink d5/d7/d8/d13/l87 0 2024-03-29T04:45:13.164 INFO:tasks.workunit.client.1.smithi152.stdout:0/397: read d2/da/d17/f74 [2554962,130605] 0 2024-03-29T04:45:13.165 INFO:tasks.workunit.client.1.smithi152.stdout:9/349: symlink d5/d12/d2c/d6e/l7c 0 2024-03-29T04:45:13.166 INFO:tasks.workunit.client.1.smithi152.stdout:0/398: dread - d2/da/d3a/f91 zero size 2024-03-29T04:45:13.167 INFO:tasks.workunit.client.0.smithi142.stdout:9/814: getdents d0/d2/d61 0 2024-03-29T04:45:13.168 INFO:tasks.workunit.client.1.smithi152.stdout:1/433: creat d5/d7/d6a/f88 x:0 0 0 2024-03-29T04:45:13.169 INFO:tasks.workunit.client.0.smithi142.stdout:0/941: dwrite d7/d13/f104 [0,4194304] 0 2024-03-29T04:45:13.169 INFO:tasks.workunit.client.1.smithi152.stdout:8/246: dwrite d1/d2/d3/f17 [0,4194304] 0 2024-03-29T04:45:13.171 INFO:tasks.workunit.client.0.smithi142.stdout:9/815: fdatasync d0/d12/d50/fe8 0 2024-03-29T04:45:13.172 INFO:tasks.workunit.client.1.smithi152.stdout:2/245: truncate d0/d3/df/f35 247305 0 2024-03-29T04:45:13.172 INFO:tasks.workunit.client.0.smithi142.stdout:7/968: dwrite d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/d3f/f41 [4194304,4194304] 0 2024-03-29T04:45:13.172 INFO:tasks.workunit.client.0.smithi142.stdout:1/812: rename d8/db/c71 to d8/db/dc/d77/db9/dbe/d2d/d44/d7f/d10e/c110 0 2024-03-29T04:45:13.173 INFO:tasks.workunit.client.1.smithi152.stdout:9/350: creat d5/d12/d2c/d6e/d73/f7d x:0 0 0 2024-03-29T04:45:13.174 INFO:tasks.workunit.client.1.smithi152.stdout:2/246: write d0/d3/df/f23 [5093463,81408] 0 2024-03-29T04:45:13.174 INFO:tasks.workunit.client.0.smithi142.stdout:0/942: chown d7/d9/d54/cdc 3 1 2024-03-29T04:45:13.174 INFO:tasks.workunit.client.1.smithi152.stdout:0/399: dread d2/da/d17/d57/f62 [0,4194304] 0 2024-03-29T04:45:13.175 INFO:tasks.workunit.client.0.smithi142.stdout:6/895: dwrite d1c/d45/d51/f97 [0,4194304] 0 2024-03-29T04:45:13.175 INFO:tasks.workunit.client.1.smithi152.stdout:0/400: stat d2/c54 0 2024-03-29T04:45:13.177 INFO:tasks.workunit.client.0.smithi142.stdout:7/969: fsync d7/d27/d44/d4c/f7b 0 2024-03-29T04:45:13.178 INFO:tasks.workunit.client.0.smithi142.stdout:7/970: write d7/d27/d44/fd7 [4089168,59617] 0 2024-03-29T04:45:13.180 INFO:tasks.workunit.client.1.smithi152.stdout:8/247: mknod d1/d21/c53 0 2024-03-29T04:45:13.181 INFO:tasks.workunit.client.0.smithi142.stdout:0/943: fsync d7/d9/ff 0 2024-03-29T04:45:13.181 INFO:tasks.workunit.client.0.smithi142.stdout:7/971: chown d7/d27/d44/db2/db8/d4b/d71/dec/cb7 8 1 2024-03-29T04:45:13.182 INFO:tasks.workunit.client.1.smithi152.stdout:2/247: mkdir d0/d3/d30/d15/d53 0 2024-03-29T04:45:13.183 INFO:tasks.workunit.client.0.smithi142.stdout:6/896: unlink d1c/f55 0 2024-03-29T04:45:13.186 INFO:tasks.workunit.client.1.smithi152.stdout:0/401: creat d2/da/d17/d57/d9c/f9d x:0 0 0 2024-03-29T04:45:13.186 INFO:tasks.workunit.client.1.smithi152.stdout:0/402: fsync d2/da/d17/f41 0 2024-03-29T04:45:13.187 INFO:tasks.workunit.client.1.smithi152.stdout:0/403: chown d2/l42 206 1 2024-03-29T04:45:13.188 INFO:tasks.workunit.client.1.smithi152.stdout:9/351: dread d5/db/f58 [0,4194304] 0 2024-03-29T04:45:13.188 INFO:tasks.workunit.client.0.smithi142.stdout:7/972: creat d7/d27/d3a/d92/de5/f13f x:0 0 0 2024-03-29T04:45:13.189 INFO:tasks.workunit.client.1.smithi152.stdout:2/248: mkdir d0/d3/d30/d15/d53/d54 0 2024-03-29T04:45:13.189 INFO:tasks.workunit.client.1.smithi152.stdout:9/352: fdatasync d5/d8/f11 0 2024-03-29T04:45:13.191 INFO:tasks.workunit.client.0.smithi142.stdout:9/816: getdents d0 0 2024-03-29T04:45:13.191 INFO:tasks.workunit.client.1.smithi152.stdout:2/249: chown d0/d3/d30/d15/d22/f2f 329588257 1 2024-03-29T04:45:13.194 INFO:tasks.workunit.client.1.smithi152.stdout:1/434: rmdir d5/d7/d8/d13 39 2024-03-29T04:45:13.196 INFO:tasks.workunit.client.1.smithi152.stdout:9/353: creat d5/d12/d2c/d6e/d73/d7a/f7e x:0 0 0 2024-03-29T04:45:13.197 INFO:tasks.workunit.client.0.smithi142.stdout:7/973: dwrite d7/d27/d44/db2/db8/d4b/d71/f108 [0,4194304] 0 2024-03-29T04:45:13.197 INFO:tasks.workunit.client.1.smithi152.stdout:8/248: link d1/lf d1/d2/l54 0 2024-03-29T04:45:13.198 INFO:tasks.workunit.client.1.smithi152.stdout:9/354: write d5/db/d1b/f51 [474420,86899] 0 2024-03-29T04:45:13.199 INFO:tasks.workunit.client.1.smithi152.stdout:2/250: dwrite d0/d3/df/f3e [0,4194304] 0 2024-03-29T04:45:13.201 INFO:tasks.workunit.client.1.smithi152.stdout:1/435: fsync d5/d7/d8/d13/d19/f65 0 2024-03-29T04:45:13.204 INFO:tasks.workunit.client.1.smithi152.stdout:8/249: fdatasync d1/d21/f32 0 2024-03-29T04:45:13.204 INFO:tasks.workunit.client.0.smithi142.stdout:7/974: stat d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/d86/dd4/cf0 0 2024-03-29T04:45:13.206 INFO:tasks.workunit.client.0.smithi142.stdout:7/975: write d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/f34 [4682654,66648] 0 2024-03-29T04:45:13.210 INFO:tasks.workunit.client.1.smithi152.stdout:8/250: mknod d1/d2/d1d/d42/c55 0 2024-03-29T04:45:13.210 INFO:tasks.workunit.client.1.smithi152.stdout:2/251: dread d0/d3/df/f24 [0,4194304] 0 2024-03-29T04:45:13.220 INFO:tasks.workunit.client.1.smithi152.stdout:8/251: dwrite d1/d21/d40/f52 [0,4194304] 0 2024-03-29T04:45:13.221 INFO:tasks.workunit.client.1.smithi152.stdout:8/252: stat d1/f34 0 2024-03-29T04:45:13.224 INFO:tasks.workunit.client.0.smithi142.stdout:1/813: dwrite d8/db/d37/f76 [0,4194304] 0 2024-03-29T04:45:13.226 INFO:tasks.workunit.client.1.smithi152.stdout:9/355: truncate d5/d8/d31/f4c 217787 0 2024-03-29T04:45:13.226 INFO:tasks.workunit.client.1.smithi152.stdout:0/404: dwrite d2/da/d17/d50/d5c/f7b [0,4194304] 0 2024-03-29T04:45:13.227 INFO:tasks.workunit.client.0.smithi142.stdout:7/976: dread d7/d27/f9f [0,4194304] 0 2024-03-29T04:45:13.227 INFO:tasks.workunit.client.1.smithi152.stdout:1/436: write d5/d7/d8/d13/d19/f3d [2149158,16600] 0 2024-03-29T04:45:13.229 INFO:tasks.workunit.client.1.smithi152.stdout:9/356: chown d5/db/f6c 44669 1 2024-03-29T04:45:13.229 INFO:tasks.workunit.client.1.smithi152.stdout:8/253: mknod d1/d2/d1d/d36/d4b/c56 0 2024-03-29T04:45:13.232 INFO:tasks.workunit.client.0.smithi142.stdout:0/944: dread d7/d9/d1e/d123/fd1 [0,4194304] 0 2024-03-29T04:45:13.233 INFO:tasks.workunit.client.0.smithi142.stdout:0/945: dread - d7/d9/ff1 zero size 2024-03-29T04:45:13.233 INFO:tasks.workunit.client.1.smithi152.stdout:1/437: unlink d5/d7/d6a/f52 0 2024-03-29T04:45:13.235 INFO:tasks.workunit.client.0.smithi142.stdout:1/814: symlink d8/db/dc/d77/d5a/l111 0 2024-03-29T04:45:13.236 INFO:tasks.workunit.client.1.smithi152.stdout:9/357: creat d5/d12/d2c/d6e/d73/f7f x:0 0 0 2024-03-29T04:45:13.236 INFO:tasks.workunit.client.1.smithi152.stdout:0/405: truncate d2/da/d17/d50/d5c/f7b 4689039 0 2024-03-29T04:45:13.237 INFO:tasks.workunit.client.0.smithi142.stdout:7/977: creat d7/d27/d44/db2/db8/d4b/d71/d30/d13d/f140 x:0 0 0 2024-03-29T04:45:13.237 INFO:tasks.workunit.client.1.smithi152.stdout:8/254: mkdir d1/d2/d57 0 2024-03-29T04:45:13.238 INFO:tasks.workunit.client.1.smithi152.stdout:9/358: write d5/db/f79 [909205,49027] 0 2024-03-29T04:45:13.239 INFO:tasks.workunit.client.0.smithi142.stdout:0/946: mknod d7/d13/dbf/c134 0 2024-03-29T04:45:13.240 INFO:tasks.workunit.client.0.smithi142.stdout:1/815: symlink d8/db/dd6/l112 0 2024-03-29T04:45:13.241 INFO:tasks.workunit.client.1.smithi152.stdout:1/438: read d5/d7/d8/fb [1488963,89419] 0 2024-03-29T04:45:13.241 INFO:tasks.workunit.client.0.smithi142.stdout:0/947: write d7/d9/d55/d42/f7c [1148863,62847] 0 2024-03-29T04:45:13.242 INFO:tasks.workunit.client.1.smithi152.stdout:9/359: symlink d5/d8/l80 0 2024-03-29T04:45:13.243 INFO:tasks.workunit.client.0.smithi142.stdout:9/817: dread d0/d2/d6/d27/dc7/fc0 [0,4194304] 0 2024-03-29T04:45:13.243 INFO:tasks.workunit.client.0.smithi142.stdout:1/816: write d8/db/dc/d77/d31/d4d/f7a [2850348,124854] 0 2024-03-29T04:45:13.243 INFO:tasks.workunit.client.0.smithi142.stdout:7/978: rmdir d7/d27/d44/dab 39 2024-03-29T04:45:13.244 INFO:tasks.workunit.client.1.smithi152.stdout:9/360: chown d5/d12/d2c/f76 10623545 1 2024-03-29T04:45:13.244 INFO:tasks.workunit.client.0.smithi142.stdout:9/818: dread - d0/d12/d50/ff8 zero size 2024-03-29T04:45:13.245 INFO:tasks.workunit.client.1.smithi152.stdout:2/252: truncate d0/d3/df/f35 562396 0 2024-03-29T04:45:13.246 INFO:tasks.workunit.client.1.smithi152.stdout:0/406: rename d2/d31/f3c to d2/da/df/d86/f9e 0 2024-03-29T04:45:13.246 INFO:tasks.workunit.client.0.smithi142.stdout:9/819: read d0/d2/d6/d27/dc7/daa/f87 [2324863,25165] 0 2024-03-29T04:45:13.247 INFO:tasks.workunit.client.1.smithi152.stdout:9/361: write d5/d12/d2c/f5f [958922,97538] 0 2024-03-29T04:45:13.247 INFO:tasks.workunit.client.0.smithi142.stdout:1/817: write d8/db/dc/d77/f7c [1385755,64814] 0 2024-03-29T04:45:13.250 INFO:tasks.workunit.client.1.smithi152.stdout:0/407: creat d2/da/d17/d57/d9c/f9f x:0 0 0 2024-03-29T04:45:13.250 INFO:tasks.workunit.client.1.smithi152.stdout:8/255: dread d1/d21/f24 [0,4194304] 0 2024-03-29T04:45:13.250 INFO:tasks.workunit.client.1.smithi152.stdout:9/362: symlink d5/d70/l81 0 2024-03-29T04:45:13.251 INFO:tasks.workunit.client.0.smithi142.stdout:7/979: mkdir d7/d27/d44/db2/db8/d4b/d71/d30/d72/db3/d141 0 2024-03-29T04:45:13.253 INFO:tasks.workunit.client.0.smithi142.stdout:9/820: creat d0/d12/d50/d52/f103 x:0 0 0 2024-03-29T04:45:13.253 INFO:tasks.workunit.client.0.smithi142.stdout:0/948: link d7/d13/d4a/c108 d7/d13/dbf/d72/d37/da4/c135 0 2024-03-29T04:45:13.253 INFO:tasks.workunit.client.1.smithi152.stdout:0/408: symlink d2/da/df/la0 0 2024-03-29T04:45:13.256 INFO:tasks.workunit.client.0.smithi142.stdout:7/980: creat d7/d27/d44/db2/dd2/f142 x:0 0 0 2024-03-29T04:45:13.257 INFO:tasks.workunit.client.1.smithi152.stdout:2/253: getdents d0/d3/df/d2c 0 2024-03-29T04:45:13.257 INFO:tasks.workunit.client.0.smithi142.stdout:0/949: mknod d7/d13/dbf/dca/d101/c136 0 2024-03-29T04:45:13.258 INFO:tasks.workunit.client.0.smithi142.stdout:9/821: dread d0/d2/d6/d27/d3c/d4f/f8c [0,4194304] 0 2024-03-29T04:45:13.260 INFO:tasks.workunit.client.1.smithi152.stdout:2/254: chown d0/d3/f31 5333 1 2024-03-29T04:45:13.262 INFO:tasks.workunit.client.0.smithi142.stdout:9/822: mkdir d0/d2/d61/d64/d104 0 2024-03-29T04:45:13.266 INFO:tasks.workunit.client.1.smithi152.stdout:2/255: fdatasync d0/d3/df/f23 0 2024-03-29T04:45:13.267 INFO:tasks.workunit.client.1.smithi152.stdout:9/363: dwrite d5/db/f21 [0,4194304] 0 2024-03-29T04:45:13.268 INFO:tasks.workunit.client.1.smithi152.stdout:8/256: dwrite d1/d2/d1d/f35 [0,4194304] 0 2024-03-29T04:45:13.268 INFO:tasks.workunit.client.0.smithi142.stdout:7/981: dwrite d7/d27/d44/fd7 [0,4194304] 0 2024-03-29T04:45:13.268 INFO:tasks.workunit.client.0.smithi142.stdout:9/823: creat d0/d2/d61/d64/f105 x:0 0 0 2024-03-29T04:45:13.268 INFO:tasks.workunit.client.0.smithi142.stdout:1/818: dread d8/db/dc/d77/db9/dbe/d2d/d8b/da4/fba [0,4194304] 0 2024-03-29T04:45:13.271 INFO:tasks.workunit.client.1.smithi152.stdout:0/409: dwrite d2/f5 [0,4194304] 0 2024-03-29T04:45:13.273 INFO:tasks.workunit.client.0.smithi142.stdout:0/950: dwrite d7/d13/d4a/d9a/da7/f11a [4194304,4194304] 0 2024-03-29T04:45:13.275 INFO:tasks.workunit.client.1.smithi152.stdout:1/439: dread d5/d7/d8/f1e [0,4194304] 0 2024-03-29T04:45:13.280 INFO:tasks.workunit.client.0.smithi142.stdout:1/819: rename d8/db/d37 to d8/db/dd6/d113 0 2024-03-29T04:45:13.280 INFO:tasks.workunit.client.0.smithi142.stdout:2/885: sync 2024-03-29T04:45:13.283 INFO:tasks.workunit.client.0.smithi142.stdout:2/886: rmdir de/d11/d17/d5d 39 2024-03-29T04:45:13.284 INFO:tasks.workunit.client.1.smithi152.stdout:9/364: unlink d5/d70/f7b 0 2024-03-29T04:45:13.284 INFO:tasks.workunit.client.1.smithi152.stdout:9/365: dread - d5/d12/d2c/f78 zero size 2024-03-29T04:45:13.287 INFO:tasks.workunit.client.1.smithi152.stdout:9/366: creat d5/d8/d31/d59/f82 x:0 0 0 2024-03-29T04:45:13.287 INFO:tasks.workunit.client.1.smithi152.stdout:7/313: sync 2024-03-29T04:45:13.288 INFO:tasks.workunit.client.1.smithi152.stdout:7/314: chown d7/d8/l2e 4024116 1 2024-03-29T04:45:13.290 INFO:tasks.workunit.client.1.smithi152.stdout:7/315: chown d7/d8/c1b 1072496 1 2024-03-29T04:45:13.291 INFO:tasks.workunit.client.0.smithi142.stdout:9/824: rename d0/d6c/dc2/ld8 to d0/d2/d6/d27/dc7/daa/d95/l106 0 2024-03-29T04:45:13.292 INFO:tasks.workunit.client.1.smithi152.stdout:9/367: truncate d5/d12/d2c/d6e/d73/d7a/f7e 81081 0 2024-03-29T04:45:13.295 INFO:tasks.workunit.client.1.smithi152.stdout:7/316: rmdir d7/d8/d12/d1d 39 2024-03-29T04:45:13.295 INFO:tasks.workunit.client.1.smithi152.stdout:7/317: write d7/d8/d12/d15/f50 [648699,55900] 0 2024-03-29T04:45:13.296 INFO:tasks.workunit.client.1.smithi152.stdout:9/368: mknod d5/d12/d2c/d6e/d73/d7a/c83 0 2024-03-29T04:45:13.298 INFO:tasks.workunit.client.0.smithi142.stdout:9/825: dread d0/d12/d50/d52/f78 [0,4194304] 0 2024-03-29T04:45:13.299 INFO:tasks.workunit.client.1.smithi152.stdout:9/369: unlink d5/db/f13 0 2024-03-29T04:45:13.299 INFO:tasks.workunit.client.1.smithi152.stdout:7/318: mknod d7/d8/d37/c5c 0 2024-03-29T04:45:13.300 INFO:tasks.workunit.client.0.smithi142.stdout:9/826: fsync d0/d12/fa0 0 2024-03-29T04:45:13.301 INFO:tasks.workunit.client.0.smithi142.stdout:9/827: write d0/d2/d6/fcd [3502752,91827] 0 2024-03-29T04:45:13.302 INFO:tasks.workunit.client.1.smithi152.stdout:9/370: unlink d5/db/f25 0 2024-03-29T04:45:13.304 INFO:tasks.workunit.client.1.smithi152.stdout:9/371: truncate d5/db/d1b/f43 265571 0 2024-03-29T04:45:13.306 INFO:tasks.workunit.client.1.smithi152.stdout:9/372: mknod d5/d8/d31/d59/d6f/c84 0 2024-03-29T04:45:13.310 INFO:tasks.workunit.client.1.smithi152.stdout:2/256: dread d0/d3/d30/d15/f20 [0,4194304] 0 2024-03-29T04:45:13.312 INFO:tasks.workunit.client.1.smithi152.stdout:2/257: truncate d0/d3/f31 754456 0 2024-03-29T04:45:13.312 INFO:tasks.workunit.client.1.smithi152.stdout:0/410: dread d2/d29/f44 [0,4194304] 0 2024-03-29T04:45:13.312 INFO:tasks.workunit.client.1.smithi152.stdout:0/411: readlink d2/da/df/la0 0 2024-03-29T04:45:13.314 INFO:tasks.workunit.client.1.smithi152.stdout:0/412: creat d2/da/df/fa1 x:0 0 0 2024-03-29T04:45:13.316 INFO:tasks.workunit.client.0.smithi142.stdout:9/828: dread d0/d2/d61/f55 [0,4194304] 0 2024-03-29T04:45:13.316 INFO:tasks.workunit.client.1.smithi152.stdout:0/413: write d2/f5 [5414936,30636] 0 2024-03-29T04:45:13.318 INFO:tasks.workunit.client.1.smithi152.stdout:2/258: dwrite d0/d3/df/f23 [0,4194304] 0 2024-03-29T04:45:13.319 INFO:tasks.workunit.client.0.smithi142.stdout:9/829: mknod d0/d2/d6/d27/d3c/c107 0 2024-03-29T04:45:13.320 INFO:tasks.workunit.client.1.smithi152.stdout:0/414: creat d2/da/d3a/d6f/fa2 x:0 0 0 2024-03-29T04:45:13.325 INFO:tasks.workunit.client.1.smithi152.stdout:2/259: rename d0/d3/df/f3e to d0/d3/df/d2c/f55 0 2024-03-29T04:45:13.326 INFO:tasks.workunit.client.1.smithi152.stdout:0/415: creat d2/da/d17/d57/d9c/fa3 x:0 0 0 2024-03-29T04:45:13.327 INFO:tasks.workunit.client.0.smithi142.stdout:9/830: link d0/d2/d6/d27/d3c/f72 d0/d2/d61/d64/f108 0 2024-03-29T04:45:13.327 INFO:tasks.workunit.client.1.smithi152.stdout:0/416: truncate d2/da/f98 183745 0 2024-03-29T04:45:13.327 INFO:tasks.workunit.client.0.smithi142.stdout:7/982: write d7/d27/fa8 [1779339,17795] 0 2024-03-29T04:45:13.331 INFO:tasks.workunit.client.1.smithi152.stdout:8/257: write d1/d2/d3/f29 [1006211,85902] 0 2024-03-29T04:45:13.331 INFO:tasks.workunit.client.1.smithi152.stdout:0/417: truncate d2/da/d3a/f91 50827 0 2024-03-29T04:45:13.333 INFO:tasks.workunit.client.0.smithi142.stdout:9/831: mkdir d0/d2/dcf/d109 0 2024-03-29T04:45:13.334 INFO:tasks.workunit.client.1.smithi152.stdout:1/440: truncate d5/d7/f25 2689903 0 2024-03-29T04:45:13.335 INFO:tasks.workunit.client.1.smithi152.stdout:8/258: mkdir d1/d21/d40/d58 0 2024-03-29T04:45:13.336 INFO:tasks.workunit.client.1.smithi152.stdout:1/441: stat d5/d7/d8/d13/c41 0 2024-03-29T04:45:13.336 INFO:tasks.workunit.client.1.smithi152.stdout:0/418: creat d2/da/d17/d50/d5c/fa4 x:0 0 0 2024-03-29T04:45:13.336 INFO:tasks.workunit.client.0.smithi142.stdout:0/951: dwrite d7/d13/d4a/fa6 [0,4194304] 0 2024-03-29T04:45:13.337 INFO:tasks.workunit.client.1.smithi152.stdout:7/319: dread d7/d8/db/df/f17 [0,4194304] 0 2024-03-29T04:45:13.337 INFO:tasks.workunit.client.1.smithi152.stdout:1/442: readlink d5/d7/d38/l58 0 2024-03-29T04:45:13.338 INFO:tasks.workunit.client.1.smithi152.stdout:7/320: chown d7/d8/l2e 490193 1 2024-03-29T04:45:13.339 INFO:tasks.workunit.client.1.smithi152.stdout:8/259: mkdir d1/d2/d59 0 2024-03-29T04:45:13.341 INFO:tasks.workunit.client.0.smithi142.stdout:9/832: truncate d0/d6c/fa1 8878 0 2024-03-29T04:45:13.341 INFO:tasks.workunit.client.1.smithi152.stdout:8/260: truncate d1/d2/d1d/f4f 835428 0 2024-03-29T04:45:13.341 INFO:tasks.workunit.client.1.smithi152.stdout:1/443: creat d5/d7/d9/d59/f89 x:0 0 0 2024-03-29T04:45:13.341 INFO:tasks.workunit.client.1.smithi152.stdout:8/261: read - d1/d2/d3/f44 zero size 2024-03-29T04:45:13.342 INFO:tasks.workunit.client.0.smithi142.stdout:7/983: dwrite d7/d27/d44/d4c/f80 [0,4194304] 0 2024-03-29T04:45:13.343 INFO:tasks.workunit.client.1.smithi152.stdout:1/444: truncate d5/d7/d8/d13/d19/f65 1172637 0 2024-03-29T04:45:13.344 INFO:tasks.workunit.client.1.smithi152.stdout:1/445: chown d5 2 1 2024-03-29T04:45:13.347 INFO:tasks.workunit.client.0.smithi142.stdout:7/984: mkdir d7/d27/d3a/d143 0 2024-03-29T04:45:13.349 INFO:tasks.workunit.client.0.smithi142.stdout:9/833: write d0/d2/d6/d27/dc7/faf [4041088,122277] 0 2024-03-29T04:45:13.349 INFO:tasks.workunit.client.1.smithi152.stdout:8/262: dwrite d1/d21/f2a [0,4194304] 0 2024-03-29T04:45:13.351 INFO:tasks.workunit.client.1.smithi152.stdout:1/446: readlink d5/d7/d9/l85 0 2024-03-29T04:45:13.351 INFO:tasks.workunit.client.0.smithi142.stdout:2/887: dwrite de/d11/d17/d10b/d22/d27/f45 [0,4194304] 0 2024-03-29T04:45:13.352 INFO:tasks.workunit.client.1.smithi152.stdout:1/447: fdatasync d5/d7/d8/d13/d19/f68 0 2024-03-29T04:45:13.353 INFO:tasks.workunit.client.1.smithi152.stdout:0/419: link d2/da/d17/d50/c96 d2/da/d17/d57/ca5 0 2024-03-29T04:45:13.354 INFO:tasks.workunit.client.1.smithi152.stdout:1/448: stat d5/d7/d1b/d2e/f86 0 2024-03-29T04:45:13.355 INFO:tasks.workunit.client.0.smithi142.stdout:7/985: creat d7/d27/d44/d4c/f144 x:0 0 0 2024-03-29T04:45:13.359 INFO:tasks.workunit.client.0.smithi142.stdout:0/952: dwrite d7/d9/d55/f87 [0,4194304] 0 2024-03-29T04:45:13.360 INFO:tasks.workunit.client.0.smithi142.stdout:7/986: unlink d7/f1a 0 2024-03-29T04:45:13.360 INFO:tasks.workunit.client.1.smithi152.stdout:8/263: mkdir d1/d2/d1d/d5a 0 2024-03-29T04:45:13.364 INFO:tasks.workunit.client.0.smithi142.stdout:9/834: getdents d0/d2/d61/df7 0 2024-03-29T04:45:13.364 INFO:tasks.workunit.client.0.smithi142.stdout:0/953: mknod d7/d13/d4a/d9a/da7/c137 0 2024-03-29T04:45:13.367 INFO:tasks.workunit.client.1.smithi152.stdout:1/449: rename d5/d7/d6a/l76 to d5/d7/d8/d13/d19/d80/l8a 0 2024-03-29T04:45:13.368 INFO:tasks.workunit.client.1.smithi152.stdout:1/450: truncate d5/d7/d8/f2d 1641499 0 2024-03-29T04:45:13.369 INFO:tasks.workunit.client.0.smithi142.stdout:1/820: dread d8/db/f20 [0,4194304] 0 2024-03-29T04:45:13.371 INFO:tasks.workunit.client.0.smithi142.stdout:1/821: dread - d8/db/dc/d77/db9/dbe/d2d/fad zero size 2024-03-29T04:45:13.371 INFO:tasks.workunit.client.1.smithi152.stdout:7/321: getdents d7/d8/d12/d1d 0 2024-03-29T04:45:13.372 INFO:tasks.workunit.client.0.smithi142.stdout:1/822: chown d8/db/dc/d77/d5a/fce 12 1 2024-03-29T04:45:13.372 INFO:tasks.workunit.client.1.smithi152.stdout:2/260: write d0/d3/d30/f1a [4758130,30354] 0 2024-03-29T04:45:13.373 INFO:tasks.workunit.client.1.smithi152.stdout:1/451: dread d5/d7/d8/d13/f35 [4194304,4194304] 0 2024-03-29T04:45:13.374 INFO:tasks.workunit.client.1.smithi152.stdout:8/264: fdatasync d1/f10 0 2024-03-29T04:45:13.377 INFO:tasks.workunit.client.1.smithi152.stdout:0/420: getdents d2/da/d3a 0 2024-03-29T04:45:13.378 INFO:tasks.workunit.client.1.smithi152.stdout:8/265: mknod d1/d21/d40/c5b 0 2024-03-29T04:45:13.378 INFO:tasks.workunit.client.1.smithi152.stdout:7/322: mkdir d7/d8/d5d 0 2024-03-29T04:45:13.379 INFO:tasks.workunit.client.1.smithi152.stdout:1/452: symlink d5/d7/d8/l8b 0 2024-03-29T04:45:13.380 INFO:tasks.workunit.client.1.smithi152.stdout:1/453: readlink d5/d7/d8/d13/l7e 0 2024-03-29T04:45:13.381 INFO:tasks.workunit.client.0.smithi142.stdout:0/954: truncate d7/d13/dbf/d72/fd6 1887806 0 2024-03-29T04:45:13.383 INFO:tasks.workunit.client.1.smithi152.stdout:0/421: write d2/da/df/d86/f9e [1728347,71502] 0 2024-03-29T04:45:13.384 INFO:tasks.workunit.client.0.smithi142.stdout:0/955: chown d7/d9/d1e/d81/fb4 2 1 2024-03-29T04:45:13.386 INFO:tasks.workunit.client.0.smithi142.stdout:0/956: write d7/d13/d4a/de7/ff4 [988435,103666] 0 2024-03-29T04:45:13.387 INFO:tasks.workunit.client.0.smithi142.stdout:1/823: dwrite d8/db/dd6/d113/f94 [0,4194304] 0 2024-03-29T04:45:13.389 INFO:tasks.workunit.client.1.smithi152.stdout:8/266: rename d1/d2/d57 to d1/d2/d1d/d36/d4b/d51/d5c 0 2024-03-29T04:45:13.389 INFO:tasks.workunit.client.0.smithi142.stdout:1/824: chown d8/db/dc/d77/d5a/f98 1 1 2024-03-29T04:45:13.391 INFO:tasks.workunit.client.0.smithi142.stdout:0/957: read d7/d13/dbf/dca/f97 [1021220,60751] 0 2024-03-29T04:45:13.393 INFO:tasks.workunit.client.1.smithi152.stdout:1/454: dwrite d5/d7/d8/d13/d19/f3d [0,4194304] 0 2024-03-29T04:45:13.394 INFO:tasks.workunit.client.0.smithi142.stdout:0/958: chown d7/d13/f118 109540 1 2024-03-29T04:45:13.394 INFO:tasks.workunit.client.1.smithi152.stdout:0/422: chown d2/d29/c56 46633 1 2024-03-29T04:45:13.394 INFO:tasks.workunit.client.1.smithi152.stdout:1/455: chown d5/c75 1 1 2024-03-29T04:45:13.395 INFO:tasks.workunit.client.1.smithi152.stdout:2/261: link d0/le d0/d3/d30/d15/l56 0 2024-03-29T04:45:13.396 INFO:tasks.workunit.client.1.smithi152.stdout:1/456: chown d5/d7/d1b/f5e 176 1 2024-03-29T04:45:13.398 INFO:tasks.workunit.client.0.smithi142.stdout:2/888: write de/d11/d17/d10b/d22/d27/dba/d61/d7d/db4/f126 [60147,54965] 0 2024-03-29T04:45:13.399 INFO:tasks.workunit.client.0.smithi142.stdout:0/959: creat d7/d9/d54/d58/f138 x:0 0 0 2024-03-29T04:45:13.401 INFO:tasks.workunit.client.0.smithi142.stdout:1/825: rename d8/db/dc/d77/db9/dbe/l36 to d8/db/dc/d77/d31/d4d/l114 0 2024-03-29T04:45:13.403 INFO:tasks.workunit.client.0.smithi142.stdout:0/960: symlink d7/d9/d55/df7/l139 0 2024-03-29T04:45:13.404 INFO:tasks.workunit.client.0.smithi142.stdout:9/835: dread d0/d2/d6/d27/d3c/f4c [0,4194304] 0 2024-03-29T04:45:13.408 INFO:tasks.workunit.client.1.smithi152.stdout:8/267: mkdir d1/d2/d1d/d36/d5d 0 2024-03-29T04:45:13.409 INFO:tasks.workunit.client.0.smithi142.stdout:9/836: rename d0/d2/d6/d27/d3c/d7b to d0/d12/d50/d52/da3/d10a 0 2024-03-29T04:45:13.409 INFO:tasks.workunit.client.0.smithi142.stdout:9/837: read - d0/d12/d50/ff0 zero size 2024-03-29T04:45:13.410 INFO:tasks.workunit.client.1.smithi152.stdout:0/423: dread - d2/f38 zero size 2024-03-29T04:45:13.411 INFO:tasks.workunit.client.0.smithi142.stdout:9/838: dread d0/d2/d6/d27/d3c/d4f/fd6 [0,4194304] 0 2024-03-29T04:45:13.413 INFO:tasks.workunit.client.1.smithi152.stdout:1/457: unlink d5/d7/d1b/d2e/l51 0 2024-03-29T04:45:13.414 INFO:tasks.workunit.client.1.smithi152.stdout:2/262: mkdir d0/d3/d30/d15/d53/d54/d57 0 2024-03-29T04:45:13.415 INFO:tasks.workunit.client.0.smithi142.stdout:1/826: write d8/db/dc/d77/db9/dbe/d2d/d44/d7f/d10e/f87 [697414,62382] 0 2024-03-29T04:45:13.415 INFO:tasks.workunit.client.1.smithi152.stdout:8/268: rename d1/d2/d3/f17 to d1/d2/d1d/d36/d4b/d51/f5e 0 2024-03-29T04:45:13.415 INFO:tasks.workunit.client.1.smithi152.stdout:7/323: dwrite d7/d8/d12/d35/f2a [0,4194304] 0 2024-03-29T04:45:13.416 INFO:tasks.workunit.client.0.smithi142.stdout:2/889: getdents de/d11/d17/d10b/d22/dbf 0 2024-03-29T04:45:13.416 INFO:tasks.workunit.client.0.smithi142.stdout:9/839: chown d0/d2/d6/d27/d3c/cda 265525 1 2024-03-29T04:45:13.416 INFO:tasks.workunit.client.1.smithi152.stdout:0/424: write d2/da/d17/f41 [1468313,91519] 0 2024-03-29T04:45:13.418 INFO:tasks.workunit.client.1.smithi152.stdout:1/458: truncate d5/d7/d1b/f78 899745 0 2024-03-29T04:45:13.419 INFO:tasks.workunit.client.0.smithi142.stdout:9/840: write d0/f1d [6532938,77989] 0 2024-03-29T04:45:13.420 INFO:tasks.workunit.client.1.smithi152.stdout:1/459: mknod d5/d7/d9/d59/c8c 0 2024-03-29T04:45:13.425 INFO:tasks.workunit.client.1.smithi152.stdout:2/263: link d0/d3/d30/l40 d0/d3/d30/d15/d22/d4c/l58 0 2024-03-29T04:45:13.426 INFO:tasks.workunit.client.1.smithi152.stdout:7/324: dwrite d7/d8/d12/d1d/d41/f2c [0,4194304] 0 2024-03-29T04:45:13.427 INFO:tasks.workunit.client.1.smithi152.stdout:1/460: getdents d5/d7/d9/d12/d77 0 2024-03-29T04:45:13.428 INFO:tasks.workunit.client.0.smithi142.stdout:9/841: mknod d0/d2/d6/d31/dc8/c10b 0 2024-03-29T04:45:13.428 INFO:tasks.workunit.client.1.smithi152.stdout:2/264: chown d0/d3/df 12822 1 2024-03-29T04:45:13.429 INFO:tasks.workunit.client.1.smithi152.stdout:1/461: chown d5/d7/fe 29532 1 2024-03-29T04:45:13.430 INFO:tasks.workunit.client.0.smithi142.stdout:0/961: dread d7/d9/d6e/f10a [0,4194304] 0 2024-03-29T04:45:13.431 INFO:tasks.workunit.client.0.smithi142.stdout:9/842: mkdir d0/d2/d61/d64/d104/d10c 0 2024-03-29T04:45:13.432 INFO:tasks.workunit.client.1.smithi152.stdout:1/462: creat d5/d7/d1b/d2e/f8d x:0 0 0 2024-03-29T04:45:13.432 INFO:tasks.workunit.client.1.smithi152.stdout:2/265: write d0/d3/d30/d15/d22/d4c/f4d [4794571,30595] 0 2024-03-29T04:45:13.432 INFO:tasks.workunit.client.0.smithi142.stdout:9/843: chown d0/d2/d6/d27/d3c/d6f/ldc 2 1 2024-03-29T04:45:13.433 INFO:tasks.workunit.client.0.smithi142.stdout:0/962: symlink d7/d9/d55/df7/d119/l13a 0 2024-03-29T04:45:13.435 INFO:tasks.workunit.client.1.smithi152.stdout:7/325: getdents d7/d8/d12/d1d/d41 0 2024-03-29T04:45:13.435 INFO:tasks.workunit.client.1.smithi152.stdout:1/463: creat d5/d7/d1b/d2e/f8e x:0 0 0 2024-03-29T04:45:13.436 INFO:tasks.workunit.client.0.smithi142.stdout:9/844: dread d0/d2/d6/f9c [0,4194304] 0 2024-03-29T04:45:13.436 INFO:tasks.workunit.client.0.smithi142.stdout:0/963: mkdir d7/d9/d1e/d81/d13b 0 2024-03-29T04:45:13.437 INFO:tasks.workunit.client.1.smithi152.stdout:2/266: symlink d0/d3/d30/d15/d53/l59 0 2024-03-29T04:45:13.438 INFO:tasks.workunit.client.0.smithi142.stdout:0/964: chown d7/d9/d6e/f6f 1307 1 2024-03-29T04:45:13.439 INFO:tasks.workunit.client.1.smithi152.stdout:1/464: write d5/d7/d1b/f5e [4151650,113415] 0 2024-03-29T04:45:13.439 INFO:tasks.workunit.client.0.smithi142.stdout:2/890: dread de/d11/d17/d5d/d71/fe8 [0,4194304] 0 2024-03-29T04:45:13.440 INFO:tasks.workunit.client.0.smithi142.stdout:0/965: read d7/d13/dbf/f8e [7462016,51172] 0 2024-03-29T04:45:13.441 INFO:tasks.workunit.client.1.smithi152.stdout:2/267: mkdir d0/d3/d30/d15/d22/d4c/d5a 0 2024-03-29T04:45:13.442 INFO:tasks.workunit.client.0.smithi142.stdout:0/966: fsync d7/d9/f2e 0 2024-03-29T04:45:13.442 INFO:tasks.workunit.client.0.smithi142.stdout:2/891: rmdir de/d11/d17/d60/de6 39 2024-03-29T04:45:13.443 INFO:tasks.workunit.client.1.smithi152.stdout:1/465: truncate d5/d7/d8/d13/d19/f30 1051191 0 2024-03-29T04:45:13.445 INFO:tasks.workunit.client.0.smithi142.stdout:2/892: symlink de/d11/d17/d5d/d71/l127 0 2024-03-29T04:45:13.445 INFO:tasks.workunit.client.1.smithi152.stdout:1/466: write d5/d7/d9/d59/f89 [214975,91451] 0 2024-03-29T04:45:13.446 INFO:tasks.workunit.client.0.smithi142.stdout:0/967: getdents d7/d13/dbf/dca/dab/ddf 0 2024-03-29T04:45:13.447 INFO:tasks.workunit.client.0.smithi142.stdout:2/893: rmdir de/d11/d2b 39 2024-03-29T04:45:13.449 INFO:tasks.workunit.client.1.smithi152.stdout:2/268: dwrite d0/d3/f10 [0,4194304] 0 2024-03-29T04:45:13.451 INFO:tasks.workunit.client.0.smithi142.stdout:2/894: symlink de/d11/d17/d10b/d22/dbf/d11c/l128 0 2024-03-29T04:45:13.457 INFO:tasks.workunit.client.1.smithi152.stdout:1/467: dread d5/f3f [0,4194304] 0 2024-03-29T04:45:13.458 INFO:tasks.workunit.client.0.smithi142.stdout:2/895: dwrite de/d11/d17/d10b/d84/f98 [0,4194304] 0 2024-03-29T04:45:13.458 INFO:tasks.workunit.client.1.smithi152.stdout:1/468: readlink d5/d7/d8/d13/l55 0 2024-03-29T04:45:13.459 INFO:tasks.workunit.client.1.smithi152.stdout:0/425: dwrite d2/da/d3a/f81 [0,4194304] 0 2024-03-29T04:45:13.459 INFO:tasks.workunit.client.1.smithi152.stdout:2/269: rename d0/d3/d30/d15/d53/l59 to d0/l5b 0 2024-03-29T04:45:13.460 INFO:tasks.workunit.client.0.smithi142.stdout:9/845: rename d0/d2/d6/d31 to d0/d2/d61/d64/dee/d10d 0 2024-03-29T04:45:13.462 INFO:tasks.workunit.client.1.smithi152.stdout:2/270: read d0/d3/d30/d15/f2b [723912,5631] 0 2024-03-29T04:45:13.463 INFO:tasks.workunit.client.1.smithi152.stdout:0/426: chown d2/da/df/c23 945446 1 2024-03-29T04:45:13.465 INFO:tasks.workunit.client.1.smithi152.stdout:2/271: truncate d0/f3b 93795 0 2024-03-29T04:45:13.465 INFO:tasks.workunit.client.0.smithi142.stdout:9/846: write d0/d2/d61/d64/fdf [972653,68428] 0 2024-03-29T04:45:13.465 INFO:tasks.workunit.client.0.smithi142.stdout:0/968: dread d7/d13/dbf/dca/dab/ddf/f115 [0,4194304] 0 2024-03-29T04:45:13.468 INFO:tasks.workunit.client.1.smithi152.stdout:1/469: unlink d5/d7/d38/c50 0 2024-03-29T04:45:13.469 INFO:tasks.workunit.client.0.smithi142.stdout:2/896: truncate de/d11/d23/f28 1218192 0 2024-03-29T04:45:13.471 INFO:tasks.workunit.client.0.smithi142.stdout:1/827: dwrite d8/db/dc/d77/f35 [0,4194304] 0 2024-03-29T04:45:13.472 INFO:tasks.workunit.client.1.smithi152.stdout:2/272: mknod d0/d3/c5c 0 2024-03-29T04:45:13.473 INFO:tasks.workunit.client.1.smithi152.stdout:1/470: write d5/d7/d1b/d2e/f8d [602753,69998] 0 2024-03-29T04:45:13.473 INFO:tasks.workunit.client.1.smithi152.stdout:0/427: dwrite d2/da/d17/f4c [0,4194304] 0 2024-03-29T04:45:13.474 INFO:tasks.workunit.client.0.smithi142.stdout:9/847: rename d0/d2/d6/d27 to d0/d12/d10e 0 2024-03-29T04:45:13.475 INFO:tasks.workunit.client.1.smithi152.stdout:7/326: dread d7/d8/fa [0,4194304] 0 2024-03-29T04:45:13.477 INFO:tasks.workunit.client.0.smithi142.stdout:2/897: mkdir de/d11/d17/d10b/d22/d27/dba/d61/dcd/d129 0 2024-03-29T04:45:13.477 INFO:tasks.workunit.client.0.smithi142.stdout:0/969: dwrite d7/d13/dbf/dca/d101/f106 [0,4194304] 0 2024-03-29T04:45:13.478 INFO:tasks.workunit.client.1.smithi152.stdout:2/273: mknod d0/d3/d30/d15/d22/d4c/c5d 0 2024-03-29T04:45:13.479 INFO:tasks.workunit.client.1.smithi152.stdout:7/327: dread - d7/d8/d12/d15/f31 zero size 2024-03-29T04:45:13.481 INFO:tasks.workunit.client.0.smithi142.stdout:0/970: write d7/d9/d1e/d81/fb4 [1310168,41819] 0 2024-03-29T04:45:13.482 INFO:tasks.workunit.client.0.smithi142.stdout:2/898: chown de/d11/d17/d10b/d22/d27/dba/d61/f8f 47162 1 2024-03-29T04:45:13.483 INFO:tasks.workunit.client.0.smithi142.stdout:0/971: dread - d7/d13/f118 zero size 2024-03-29T04:45:13.485 INFO:tasks.workunit.client.0.smithi142.stdout:9/848: dwrite d0/d12/d10e/dc7/fc0 [0,4194304] 0 2024-03-29T04:45:13.486 INFO:tasks.workunit.client.1.smithi152.stdout:0/428: dwrite d2/f38 [0,4194304] 0 2024-03-29T04:45:13.488 INFO:tasks.workunit.client.0.smithi142.stdout:9/849: truncate d0/d12/d50/fe8 599288 0 2024-03-29T04:45:13.491 INFO:tasks.workunit.client.1.smithi152.stdout:0/429: mknod d2/da/d17/ca6 0 2024-03-29T04:45:13.492 INFO:tasks.workunit.client.0.smithi142.stdout:0/972: fdatasync d7/d9/d1e/d123/d90/fe5 0 2024-03-29T04:45:13.493 INFO:tasks.workunit.client.1.smithi152.stdout:0/430: write d2/f9b [428467,112386] 0 2024-03-29T04:45:13.498 INFO:tasks.workunit.client.1.smithi152.stdout:0/431: mknod d2/da/d17/ca7 0 2024-03-29T04:45:13.500 INFO:tasks.workunit.client.0.smithi142.stdout:2/899: dread de/d11/d17/d10b/d22/d27/dba/d61/d7d/db4/fc5 [0,4194304] 0 2024-03-29T04:45:13.501 INFO:tasks.workunit.client.0.smithi142.stdout:9/850: getdents d0/d2/d61/d64/dfa 0 2024-03-29T04:45:13.504 INFO:tasks.workunit.client.0.smithi142.stdout:0/973: read d7/d9/d55/fdb [3680476,73008] 0 2024-03-29T04:45:13.504 INFO:tasks.workunit.client.0.smithi142.stdout:9/851: mkdir d0/d12/d10e/d3c/d4f/d10f 0 2024-03-29T04:45:13.506 INFO:tasks.workunit.client.0.smithi142.stdout:0/974: creat d7/f13c x:0 0 0 2024-03-29T04:45:13.508 INFO:tasks.workunit.client.0.smithi142.stdout:9/852: dread d0/d12/d10e/dc7/fc0 [0,4194304] 0 2024-03-29T04:45:13.509 INFO:tasks.workunit.client.0.smithi142.stdout:9/853: fdatasync d0/d12/d50/d52/da3/d10a/fd3 0 2024-03-29T04:45:13.532 INFO:tasks.workunit.client.1.smithi152.stdout:7/328: dread d7/d8/db/df/f2b [0,4194304] 0 2024-03-29T04:45:13.533 INFO:tasks.workunit.client.1.smithi152.stdout:7/329: mkdir d7/d8/d12/d35/d5e 0 2024-03-29T04:45:13.534 INFO:tasks.workunit.client.1.smithi152.stdout:7/330: dread - d7/d8/db/df/f3a zero size 2024-03-29T04:45:13.538 INFO:tasks.workunit.client.1.smithi152.stdout:7/331: link d7/d8/l5a d7/d8/d12/d1d/d41/l5f 0 2024-03-29T04:45:13.541 INFO:tasks.workunit.client.0.smithi142.stdout:0/975: chown d7/d9/d1e/c3f 1 1 2024-03-29T04:45:13.541 INFO:tasks.workunit.client.1.smithi152.stdout:7/332: symlink d7/d8/d5d/l60 0 2024-03-29T04:45:13.542 INFO:tasks.workunit.client.1.smithi152.stdout:7/333: readlink d7/d8/d12/d1d/d41/l5f 0 2024-03-29T04:45:13.543 INFO:tasks.workunit.client.1.smithi152.stdout:7/334: chown d7/d8/d12/d1d/l26 205095648 1 2024-03-29T04:45:13.545 INFO:tasks.workunit.client.0.smithi142.stdout:0/976: rename d7/d13/d4a/d9a to d7/d13/dbf/d13d 0 2024-03-29T04:45:13.546 INFO:tasks.workunit.client.1.smithi152.stdout:7/335: rename d7/d8/db/df/f17 to d7/d8/d12/f61 0 2024-03-29T04:45:13.549 INFO:tasks.workunit.client.1.smithi152.stdout:7/336: rename d7/d8/db/df/f13 to d7/d8/d12/d1d/f62 0 2024-03-29T04:45:13.552 INFO:tasks.workunit.client.1.smithi152.stdout:7/337: dread d7/d8/fa [0,4194304] 0 2024-03-29T04:45:13.553 INFO:tasks.workunit.client.1.smithi152.stdout:7/338: write d7/d8/f3b [1536665,54514] 0 2024-03-29T04:45:13.556 INFO:tasks.workunit.client.1.smithi152.stdout:7/339: creat d7/d8/d12/d35/f63 x:0 0 0 2024-03-29T04:45:13.556 INFO:tasks.workunit.client.1.smithi152.stdout:7/340: read d7/d8/d12/f22 [3109816,63209] 0 2024-03-29T04:45:13.558 INFO:tasks.workunit.client.0.smithi142.stdout:3/731: sync 2024-03-29T04:45:13.558 INFO:tasks.workunit.client.0.smithi142.stdout:4/852: sync 2024-03-29T04:45:13.561 INFO:tasks.workunit.client.0.smithi142.stdout:4/853: symlink d0/d9/d11/d97/l11b 0 2024-03-29T04:45:13.562 INFO:tasks.workunit.client.0.smithi142.stdout:0/977: dread d7/d9/d55/f3d [0,4194304] 0 2024-03-29T04:45:13.562 INFO:tasks.workunit.client.0.smithi142.stdout:3/732: creat d3/ff9 x:0 0 0 2024-03-29T04:45:13.564 INFO:tasks.workunit.client.0.smithi142.stdout:0/978: read d7/d13/dbf/dca/dab/dc1/fde [8852,120102] 0 2024-03-29T04:45:13.565 INFO:tasks.workunit.client.0.smithi142.stdout:3/733: write d3/d29/d2b/d40/d67/d8b/d6/f2f [4786093,88782] 0 2024-03-29T04:45:13.565 INFO:tasks.workunit.client.1.smithi152.stdout:7/341: rmdir d7/d8/d12/d35 39 2024-03-29T04:45:13.567 INFO:tasks.workunit.client.1.smithi152.stdout:7/342: fdatasync d7/d8/d12/f61 0 2024-03-29T04:45:13.567 INFO:tasks.workunit.client.0.smithi142.stdout:4/854: rename d0/d9/d11/d4c/d5f/d81/dc9/d47/l96 to d0/d9/l11c 0 2024-03-29T04:45:13.567 INFO:tasks.workunit.client.1.smithi152.stdout:7/343: read - d7/d8/db/df/f3a zero size 2024-03-29T04:45:13.569 INFO:tasks.workunit.client.0.smithi142.stdout:0/979: unlink d7/d13/dbf/dca/dab/dc1/lcf 0 2024-03-29T04:45:13.569 INFO:tasks.workunit.client.0.smithi142.stdout:4/855: write d0/d9/d11/d4c/d5f/d81/ff0 [4333291,10569] 0 2024-03-29T04:45:13.570 INFO:tasks.workunit.client.0.smithi142.stdout:0/980: readlink d7/d13/dbf/d72/d37/le9 0 2024-03-29T04:45:13.571 INFO:tasks.workunit.client.0.smithi142.stdout:3/734: dwrite d3/d1f/d24/d47/f4f [0,4194304] 0 2024-03-29T04:45:13.576 INFO:tasks.workunit.client.0.smithi142.stdout:0/981: mknod d7/d9/d55/df7/c13e 0 2024-03-29T04:45:13.578 INFO:tasks.workunit.client.0.smithi142.stdout:4/856: creat d0/d9/d11/d97/f11d x:0 0 0 2024-03-29T04:45:13.580 INFO:tasks.workunit.client.0.smithi142.stdout:4/857: symlink d0/d9/d11/d84/l11e 0 2024-03-29T04:45:13.582 INFO:tasks.workunit.client.0.smithi142.stdout:0/982: creat d7/d9/d1e/d123/de1/f13f x:0 0 0 2024-03-29T04:45:13.582 INFO:tasks.workunit.client.0.smithi142.stdout:3/735: link d3/d29/d2b/d40/cac d3/d1f/de6/df1/cfa 0 2024-03-29T04:45:13.583 INFO:tasks.workunit.client.0.smithi142.stdout:0/983: chown d7/d9/d1e/c12a 143384 1 2024-03-29T04:45:13.583 INFO:tasks.workunit.client.0.smithi142.stdout:3/736: write d3/d1f/d59/fbb [257271,103889] 0 2024-03-29T04:45:13.585 INFO:tasks.workunit.client.0.smithi142.stdout:4/858: link d0/d9/db2/ffe d0/d9/d11/d4c/d5f/f11f 0 2024-03-29T04:45:13.587 INFO:tasks.workunit.client.0.smithi142.stdout:3/737: symlink d3/d29/d2b/d40/d67/d8b/d4d/lfb 0 2024-03-29T04:45:13.591 INFO:tasks.workunit.client.0.smithi142.stdout:3/738: getdents d3/d29 0 2024-03-29T04:45:13.594 INFO:tasks.workunit.client.0.smithi142.stdout:4/859: truncate d0/d9/d11/d4c/d5f/f73 1670232 0 2024-03-29T04:45:13.598 INFO:tasks.workunit.client.0.smithi142.stdout:3/739: dwrite d3/d29/d2d/f48 [0,4194304] 0 2024-03-29T04:45:13.598 INFO:tasks.workunit.client.0.smithi142.stdout:0/984: dwrite d7/d13/dbf/d72/fd6 [0,4194304] 0 2024-03-29T04:45:13.600 INFO:tasks.workunit.client.0.smithi142.stdout:4/860: truncate d0/d9/db2/dba/dbc/d8d/da5/d102/f114 795767 0 2024-03-29T04:45:13.603 INFO:tasks.workunit.client.0.smithi142.stdout:3/740: fdatasync d3/d29/d2b/fe8 0 2024-03-29T04:45:13.604 INFO:tasks.workunit.client.0.smithi142.stdout:3/741: write d3/d29/d8f/d5f/db4/f97 [1580466,7216] 0 2024-03-29T04:45:13.605 INFO:tasks.workunit.client.0.smithi142.stdout:0/985: unlink d7/d13/d4a/la1 0 2024-03-29T04:45:13.606 INFO:tasks.workunit.client.0.smithi142.stdout:0/986: chown d7/d9/d1e/l9b 247414453 1 2024-03-29T04:45:13.607 INFO:tasks.workunit.client.0.smithi142.stdout:0/987: read - d7/d13/dbf/d72/d37/faa zero size 2024-03-29T04:45:13.611 INFO:tasks.workunit.client.0.smithi142.stdout:3/742: dwrite d3/d1f/d24/faf [0,4194304] 0 2024-03-29T04:45:13.614 INFO:tasks.workunit.client.0.smithi142.stdout:0/988: creat d7/d9/d55/f140 x:0 0 0 2024-03-29T04:45:13.617 INFO:tasks.workunit.client.0.smithi142.stdout:3/743: creat d3/d29/d2d/ffc x:0 0 0 2024-03-29T04:45:13.618 INFO:tasks.workunit.client.0.smithi142.stdout:3/744: write d3/d29/d2d/f48 [3582448,86855] 0 2024-03-29T04:45:13.619 INFO:tasks.workunit.client.0.smithi142.stdout:0/989: symlink d7/d9/d6e/df2/l141 0 2024-03-29T04:45:13.619 INFO:tasks.workunit.client.0.smithi142.stdout:3/745: dread - d3/fe3 zero size 2024-03-29T04:45:13.622 INFO:tasks.workunit.client.0.smithi142.stdout:4/861: write d0/d9/d11/d4c/d5f/d81/dc9/d47/f74 [843634,25231] 0 2024-03-29T04:45:13.624 INFO:tasks.workunit.client.0.smithi142.stdout:0/990: dread d7/d9/d54/f64 [4194304,4194304] 0 2024-03-29T04:45:13.625 INFO:tasks.workunit.client.0.smithi142.stdout:4/862: creat d0/d9/d11/d4c/d5f/d81/f120 x:0 0 0 2024-03-29T04:45:13.629 INFO:tasks.workunit.client.0.smithi142.stdout:0/991: rename d7/d9/d6e/c9e to d7/d13/dbf/d72/d116/c142 0 2024-03-29T04:45:13.631 INFO:tasks.workunit.client.0.smithi142.stdout:0/992: mkdir d7/d9/d1e/d81/d143 0 2024-03-29T04:45:13.633 INFO:tasks.workunit.client.0.smithi142.stdout:0/993: creat d7/d9/d1e/d123/f144 x:0 0 0 2024-03-29T04:45:13.636 INFO:tasks.workunit.client.0.smithi142.stdout:0/994: rename d7/d13/f7d to d7/d9/d55/df7/f145 0 2024-03-29T04:45:13.640 INFO:tasks.workunit.client.1.smithi152.stdout:7/344: truncate d7/d8/d12/d1d/d41/f2c 130853 0 2024-03-29T04:45:13.643 INFO:tasks.workunit.client.1.smithi152.stdout:7/345: mknod d7/d8/db/c64 0 2024-03-29T04:45:13.646 INFO:tasks.workunit.client.1.smithi152.stdout:7/346: dread d7/d8/d12/f61 [0,4194304] 0 2024-03-29T04:45:13.646 INFO:tasks.workunit.client.0.smithi142.stdout:0/995: dwrite d7/d9/d1e/d123/d90/fe5 [0,4194304] 0 2024-03-29T04:45:13.649 INFO:tasks.workunit.client.0.smithi142.stdout:0/996: creat d7/d13/d4a/de7/f146 x:0 0 0 2024-03-29T04:45:13.652 INFO:tasks.workunit.client.1.smithi152.stdout:7/347: write d7/d8/d12/f33 [1493415,54127] 0 2024-03-29T04:45:13.653 INFO:tasks.workunit.client.0.smithi142.stdout:0/997: rename d7/d9/d54/d58/c111 to d7/d13/dbf/dca/d101/c147 0 2024-03-29T04:45:13.655 INFO:tasks.workunit.client.1.smithi152.stdout:7/348: symlink d7/d8/d12/d15/l65 0 2024-03-29T04:45:13.656 INFO:tasks.workunit.client.0.smithi142.stdout:0/998: mknod d7/d9/d1e/d81/d13b/c148 0 2024-03-29T04:45:13.657 INFO:tasks.workunit.client.1.smithi152.stdout:7/349: dread - d7/d8/d12/d15/f3e zero size 2024-03-29T04:45:13.657 INFO:tasks.workunit.client.0.smithi142.stdout:0/999: fsync d7/d9/d1e/f100 0 2024-03-29T04:45:13.658 INFO:tasks.workunit.client.1.smithi152.stdout:7/350: creat d7/d8/d37/f66 x:0 0 0 2024-03-29T04:45:13.667 INFO:tasks.workunit.client.1.smithi152.stdout:7/351: truncate d7/d8/d12/d1d/f34 3402863 0 2024-03-29T04:45:13.673 INFO:tasks.workunit.client.1.smithi152.stdout:7/352: dwrite d7/d8/d12/d1d/d41/d45/f55 [0,4194304] 0 2024-03-29T04:45:13.675 INFO:tasks.workunit.client.1.smithi152.stdout:7/353: fsync d7/d8/db/df/f2b 0 2024-03-29T04:45:13.676 INFO:tasks.workunit.client.1.smithi152.stdout:4/276: sync 2024-03-29T04:45:13.676 INFO:tasks.workunit.client.1.smithi152.stdout:3/297: sync 2024-03-29T04:45:13.678 INFO:tasks.workunit.client.1.smithi152.stdout:4/277: creat d0/d27/f59 x:0 0 0 2024-03-29T04:45:13.686 INFO:tasks.workunit.client.1.smithi152.stdout:4/278: dwrite d0/d7/d25/d17/f4c [0,4194304] 0 2024-03-29T04:45:13.691 INFO:tasks.workunit.client.1.smithi152.stdout:4/279: getdents d0/d27 0 2024-03-29T04:45:13.693 INFO:tasks.workunit.client.1.smithi152.stdout:4/280: mknod d0/c5a 0 2024-03-29T04:45:13.697 INFO:tasks.workunit.client.1.smithi152.stdout:4/281: getdents d0/d7/d25/d17 0 2024-03-29T04:45:13.699 INFO:tasks.workunit.client.1.smithi152.stdout:7/354: dwrite d7/d8/d12/d1d/f34 [0,4194304] 0 2024-03-29T04:45:13.705 INFO:tasks.workunit.client.1.smithi152.stdout:4/282: getdents d0/d7/d25/d48/d57 0 2024-03-29T04:45:13.707 INFO:tasks.workunit.client.1.smithi152.stdout:4/283: creat d0/d7/d25/d17/f5b x:0 0 0 2024-03-29T04:45:13.709 INFO:tasks.workunit.client.1.smithi152.stdout:4/284: creat d0/d7/d25/d48/d57/f5c x:0 0 0 2024-03-29T04:45:13.710 INFO:tasks.workunit.client.1.smithi152.stdout:4/285: rename d0/d7/d40/d2a to d0/d7/d40/d2a/d5d 22 2024-03-29T04:45:13.711 INFO:tasks.workunit.client.1.smithi152.stdout:4/286: dread - d0/d7/d25/d17/d37/f4f zero size 2024-03-29T04:45:13.713 INFO:tasks.workunit.client.0.smithi142.stdout:7/987: sync 2024-03-29T04:45:13.713 INFO:tasks.workunit.client.0.smithi142.stdout:6/897: sync 2024-03-29T04:45:13.715 INFO:tasks.workunit.client.0.smithi142.stdout:7/988: creat d7/d27/d3a/d135/f145 x:0 0 0 2024-03-29T04:45:13.720 INFO:tasks.workunit.client.0.smithi142.stdout:7/989: dwrite d7/fa [0,4194304] 0 2024-03-29T04:45:13.725 INFO:tasks.workunit.client.0.smithi142.stdout:7/990: read d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/d86/f104 [372564,19991] 0 2024-03-29T04:45:13.730 INFO:tasks.workunit.client.0.smithi142.stdout:7/991: rename d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/d20/f21 to d7/d27/d3a/d92/f146 0 2024-03-29T04:45:13.731 INFO:tasks.workunit.client.0.smithi142.stdout:7/992: write d7/d27/d44/db2/db8/d4b/d71/d30/d72/dc0/d121/dc1/f10a [9317,29747] 0 2024-03-29T04:45:13.737 INFO:tasks.workunit.client.0.smithi142.stdout:7/993: dwrite d7/d27/d44/db2/db8/d89/fdc [0,4194304] 0 2024-03-29T04:45:13.740 INFO:tasks.workunit.client.0.smithi142.stdout:7/994: creat d7/d27/d44/d4c/da3/d10f/f147 x:0 0 0 2024-03-29T04:45:13.743 INFO:tasks.workunit.client.0.smithi142.stdout:7/995: fsync d7/d27/d44/db2/db8/d4b/f9a 0 2024-03-29T04:45:13.745 INFO:tasks.workunit.client.0.smithi142.stdout:7/996: symlink d7/d27/d3a/d92/d97/dea/l148 0 2024-03-29T04:45:13.746 INFO:tasks.workunit.client.0.smithi142.stdout:7/997: chown d7/d27 16510334 1 2024-03-29T04:45:13.746 INFO:tasks.workunit.client.0.smithi142.stdout:7/998: stat d7/d27/d3a/d92/d97 0 2024-03-29T04:45:13.748 INFO:tasks.workunit.client.0.smithi142.stdout:7/999: creat d7/d27/d3a/f149 x:0 0 0 2024-03-29T04:45:13.818 INFO:tasks.workunit.client.1.smithi152.stdout:4/287: write d0/d7/d25/d17/f54 [1476885,78480] 0 2024-03-29T04:45:13.821 INFO:tasks.workunit.client.1.smithi152.stdout:4/288: mkdir d0/d7/d25/d17/d5e 0 2024-03-29T04:45:13.825 INFO:tasks.workunit.client.1.smithi152.stdout:4/289: rmdir d0/d7/d25/d48/d57 39 2024-03-29T04:45:13.825 INFO:tasks.workunit.client.1.smithi152.stdout:7/355: dread d7/f14 [0,4194304] 0 2024-03-29T04:45:13.828 INFO:tasks.workunit.client.1.smithi152.stdout:7/356: symlink d7/d8/d12/d35/l67 0 2024-03-29T04:45:13.831 INFO:tasks.workunit.client.1.smithi152.stdout:7/357: dread d7/d8/d12/d1d/f34 [0,4194304] 0 2024-03-29T04:45:13.834 INFO:tasks.workunit.client.1.smithi152.stdout:7/358: rename d7/d8/d5d to d7/d8/d12/d35/d68 0 2024-03-29T04:45:13.835 INFO:tasks.workunit.client.1.smithi152.stdout:7/359: mknod d7/d8/d37/c69 0 2024-03-29T04:45:13.873 INFO:tasks.workunit.client.0.smithi142.stdout:1/828: sync 2024-03-29T04:45:13.873 INFO:tasks.workunit.client.1.smithi152.stdout:6/369: sync 2024-03-29T04:45:13.874 INFO:tasks.workunit.client.0.smithi142.stdout:1/829: chown d8/db/dc/d77/db9/dbe/d2d/d44/d53/cc1 380 1 2024-03-29T04:45:13.874 INFO:tasks.workunit.client.1.smithi152.stdout:6/370: fsync d1/d3/d9/d13/f46 0 2024-03-29T04:45:13.875 INFO:tasks.workunit.client.1.smithi152.stdout:6/371: readlink d1/d3/d9/l47 0 2024-03-29T04:45:13.876 INFO:tasks.workunit.client.0.smithi142.stdout:1/830: stat d8/db/dc/d77/db9/dbe/d1f/d4c/cfd 0 2024-03-29T04:45:13.876 INFO:tasks.workunit.client.0.smithi142.stdout:1/831: dread - d8/db/dc/d19/ff0 zero size 2024-03-29T04:45:13.879 INFO:tasks.workunit.client.1.smithi152.stdout:6/372: dread d1/d3/d9/f3a [0,4194304] 0 2024-03-29T04:45:13.880 INFO:tasks.workunit.client.0.smithi142.stdout:1/832: rename d8/db/dc/d77/db9/dbe/d2d/d44/d53/f7b to d8/db/dc/d77/db9/dbe/d2d/f115 0 2024-03-29T04:45:13.880 INFO:tasks.workunit.client.1.smithi152.stdout:6/373: write d1/d3/d1a/d1b/d29/f5f [3581924,107202] 0 2024-03-29T04:45:13.882 INFO:tasks.workunit.client.0.smithi142.stdout:1/833: mknod d8/db/dc/d77/db9/dbe/d2d/d8b/c116 0 2024-03-29T04:45:13.883 INFO:tasks.workunit.client.1.smithi152.stdout:6/374: creat d1/d3/d9/d13/d3e/d54/f7c x:0 0 0 2024-03-29T04:45:13.884 INFO:tasks.workunit.client.1.smithi152.stdout:6/375: fsync d1/d3/d9/f21 0 2024-03-29T04:45:13.886 INFO:tasks.workunit.client.0.smithi142.stdout:1/834: link d8/db/dc/d77/db9/dbe/d2d/d44/d7f/lc7 d8/db/dc/d109/l117 0 2024-03-29T04:45:13.888 INFO:tasks.workunit.client.0.smithi142.stdout:1/835: creat d8/db/dc/d77/d31/d4d/d56/f118 x:0 0 0 2024-03-29T04:45:13.888 INFO:tasks.workunit.client.1.smithi152.stdout:6/376: truncate f0 3971475 0 2024-03-29T04:45:13.890 INFO:tasks.workunit.client.0.smithi142.stdout:1/836: read d8/db/dc/d77/db9/dbe/d1f/f29 [489694,2205] 0 2024-03-29T04:45:13.891 INFO:tasks.workunit.client.0.smithi142.stdout:1/837: unlink d8/f2a 0 2024-03-29T04:45:13.900 INFO:tasks.workunit.client.0.smithi142.stdout:1/838: fsync d8/fa 0 2024-03-29T04:45:13.961 INFO:tasks.workunit.client.0.smithi142.stdout:1/839: truncate d8/f2b 552322 0 2024-03-29T04:45:13.961 INFO:tasks.workunit.client.0.smithi142.stdout:1/840: unlink d8/db/dc/d77/db9/dbe/d1f/d97/fd5 0 2024-03-29T04:45:13.962 INFO:tasks.workunit.client.0.smithi142.stdout:1/841: creat d8/db/dc/d77/db9/dbe/d2d/d44/d53/f119 x:0 0 0 2024-03-29T04:45:13.962 INFO:tasks.workunit.client.0.smithi142.stdout:1/842: write d8/db/dc/d77/db9/dbe/d2d/f115 [4840531,59968] 0 2024-03-29T04:45:13.962 INFO:tasks.workunit.client.0.smithi142.stdout:1/843: fdatasync d8/db/dc/d77/db9/dbe/d1f/f33 0 2024-03-29T04:45:13.964 INFO:tasks.workunit.client.1.smithi152.stdout:6/377: dread - d1/d3/d1a/d49/f57 zero size 2024-03-29T04:45:13.964 INFO:tasks.workunit.client.1.smithi152.stdout:6/378: symlink d1/d3/d9/d13/d28/l7d 0 2024-03-29T04:45:13.964 INFO:tasks.workunit.client.1.smithi152.stdout:6/379: creat d1/d3/d1a/d1b/d29/f7e x:0 0 0 2024-03-29T04:45:13.964 INFO:tasks.workunit.client.1.smithi152.stdout:6/380: truncate d1/d3/d9/d13/f46 729371 0 2024-03-29T04:45:13.964 INFO:tasks.workunit.client.1.smithi152.stdout:6/381: rmdir d1/d3/d9/d13/d28/d45 39 2024-03-29T04:45:13.964 INFO:tasks.workunit.client.1.smithi152.stdout:6/382: mknod d1/d3/d9/d13/d28/c7f 0 2024-03-29T04:45:13.965 INFO:tasks.workunit.client.1.smithi152.stdout:6/383: creat d1/d3/d9/d13/d28/d45/f80 x:0 0 0 2024-03-29T04:45:13.965 INFO:tasks.workunit.client.1.smithi152.stdout:6/384: dwrite d1/d3/d1a/d1b/d29/f5f [0,4194304] 0 2024-03-29T04:45:13.965 INFO:tasks.workunit.client.1.smithi152.stdout:6/385: read d1/d3/d1a/d1b/f1e [1061218,105492] 0 2024-03-29T04:45:13.965 INFO:tasks.workunit.client.1.smithi152.stdout:6/386: creat d1/d3/df/f81 x:0 0 0 2024-03-29T04:45:13.965 INFO:tasks.workunit.client.1.smithi152.stdout:6/387: write d1/d3/d1a/d1b/d29/f71 [3729313,86595] 0 2024-03-29T04:45:13.965 INFO:tasks.workunit.client.1.smithi152.stdout:6/388: write d1/f17 [4765398,109842] 0 2024-03-29T04:45:13.965 INFO:tasks.workunit.client.1.smithi152.stdout:6/389: chown d1/d3/d1a/d1b/d29 20500012 1 2024-03-29T04:45:13.965 INFO:tasks.workunit.client.1.smithi152.stdout:6/390: creat d1/d3/d9/d13/d3e/d54/f82 x:0 0 0 2024-03-29T04:45:13.965 INFO:tasks.workunit.client.1.smithi152.stdout:9/373: sync 2024-03-29T04:45:13.965 INFO:tasks.workunit.client.1.smithi152.stdout:5/355: sync 2024-03-29T04:45:13.965 INFO:tasks.workunit.client.1.smithi152.stdout:6/391: getdents d1/d3/d1a 0 2024-03-29T04:45:13.966 INFO:tasks.workunit.client.1.smithi152.stdout:9/374: mkdir d5/d8/d31/d59/d85 0 2024-03-29T04:45:13.966 INFO:tasks.workunit.client.1.smithi152.stdout:6/392: mkdir d1/d3/d1a/d1b/d29/d6f/d83 0 2024-03-29T04:45:13.966 INFO:tasks.workunit.client.1.smithi152.stdout:9/375: fsync d5/d8/f1a 0 2024-03-29T04:45:13.966 INFO:tasks.workunit.client.1.smithi152.stdout:5/356: dwrite d6/d1d/d3b/d55/f66 [0,4194304] 0 2024-03-29T04:45:13.966 INFO:tasks.workunit.client.1.smithi152.stdout:6/393: rename d1/d3/d1a/d49 to d1/d3/df/d84 0 2024-03-29T04:45:13.966 INFO:tasks.workunit.client.1.smithi152.stdout:9/376: creat d5/d12/f86 x:0 0 0 2024-03-29T04:45:13.966 INFO:tasks.workunit.client.1.smithi152.stdout:5/357: symlink d6/l76 0 2024-03-29T04:45:13.966 INFO:tasks.workunit.client.1.smithi152.stdout:9/377: symlink d5/d12/d2c/d6e/d73/d7a/l87 0 2024-03-29T04:45:13.966 INFO:tasks.workunit.client.1.smithi152.stdout:6/394: creat d1/d3/d1a/d1b/d29/d6f/d83/f85 x:0 0 0 2024-03-29T04:45:13.968 INFO:tasks.workunit.client.1.smithi152.stdout:9/378: rename d5/db/d1b/c24 to d5/d8/d31/d75/c88 0 2024-03-29T04:45:13.969 INFO:tasks.workunit.client.1.smithi152.stdout:6/395: mknod d1/d3/d1a/c86 0 2024-03-29T04:45:13.970 INFO:tasks.workunit.client.1.smithi152.stdout:9/379: rmdir d5/db/d1b 39 2024-03-29T04:45:13.972 INFO:tasks.workunit.client.1.smithi152.stdout:5/358: dwrite d6/d16/d25/d2e/f44 [0,4194304] 0 2024-03-29T04:45:13.974 INFO:tasks.workunit.client.1.smithi152.stdout:6/396: mkdir d1/d3/d1a/d1b/d87 0 2024-03-29T04:45:13.978 INFO:tasks.workunit.client.0.smithi142.stdout:9/854: sync 2024-03-29T04:45:13.978 INFO:tasks.workunit.client.0.smithi142.stdout:2/900: sync 2024-03-29T04:45:13.979 INFO:tasks.workunit.client.1.smithi152.stdout:5/359: rmdir d6 39 2024-03-29T04:45:13.984 INFO:tasks.workunit.client.1.smithi152.stdout:6/397: dread d1/d3/d9/d13/f31 [0,4194304] 0 2024-03-29T04:45:13.985 INFO:tasks.workunit.client.0.smithi142.stdout:2/901: creat de/d11/d17/d10b/d22/f12a x:0 0 0 2024-03-29T04:45:13.986 INFO:tasks.workunit.client.0.smithi142.stdout:9/855: dread d0/f94 [0,4194304] 0 2024-03-29T04:45:13.988 INFO:tasks.workunit.client.1.smithi152.stdout:6/398: truncate d1/d3/d9/d13/f46 1007051 0 2024-03-29T04:45:13.990 INFO:tasks.workunit.client.1.smithi152.stdout:6/399: fdatasync d1/d3/df/f48 0 2024-03-29T04:45:13.992 INFO:tasks.workunit.client.1.smithi152.stdout:6/400: fsync d1/d3/d9/d13/f31 0 2024-03-29T04:45:13.992 INFO:tasks.workunit.client.0.smithi142.stdout:9/856: dwrite d0/d2/d61/d64/dee/d10d/dc8/fe2 [0,4194304] 0 2024-03-29T04:45:13.993 INFO:tasks.workunit.client.0.smithi142.stdout:2/902: rename de/d11/d23 to de/d11/d17/d4a/d79/d12b 0 2024-03-29T04:45:13.995 INFO:tasks.workunit.client.0.smithi142.stdout:9/857: stat d0/c19 0 2024-03-29T04:45:13.996 INFO:tasks.workunit.client.1.smithi152.stdout:5/360: dread f5 [0,4194304] 0 2024-03-29T04:45:14.002 INFO:tasks.workunit.client.1.smithi152.stdout:5/361: symlink d6/d16/d25/d2e/l77 0 2024-03-29T04:45:14.002 INFO:tasks.workunit.client.0.smithi142.stdout:2/903: dwrite de/d11/d17/d4a/d79/d12b/f46 [0,4194304] 0 2024-03-29T04:45:14.003 INFO:tasks.workunit.client.1.smithi152.stdout:6/401: truncate d1/d3/f18 1648962 0 2024-03-29T04:45:14.003 INFO:tasks.workunit.client.1.smithi152.stdout:6/402: fdatasync d1/f17 0 2024-03-29T04:45:14.004 INFO:tasks.workunit.client.1.smithi152.stdout:5/362: creat d6/d1b/f78 x:0 0 0 2024-03-29T04:45:14.005 INFO:tasks.workunit.client.1.smithi152.stdout:6/403: readlink d1/d3/d1a/d1b/d29/l72 0 2024-03-29T04:45:14.006 INFO:tasks.workunit.client.0.smithi142.stdout:9/858: dwrite d0/d6c/dc2/fc9 [0,4194304] 0 2024-03-29T04:45:14.009 INFO:tasks.workunit.client.0.smithi142.stdout:9/859: write d0/d12/d50/d8e/fa4 [1195593,110276] 0 2024-03-29T04:45:14.010 INFO:tasks.workunit.client.1.smithi152.stdout:5/363: rename d6/d16/d25/d2e/f44 to d6/d1d/f79 0 2024-03-29T04:45:14.011 INFO:tasks.workunit.client.0.smithi142.stdout:2/904: read de/d11/d17/f109 [4934327,11482] 0 2024-03-29T04:45:14.013 INFO:tasks.workunit.client.0.smithi142.stdout:9/860: fdatasync d0/d2/d61/d64/fac 0 2024-03-29T04:45:14.019 INFO:tasks.workunit.client.0.smithi142.stdout:9/861: rename d0/d2/ff1 to d0/d2/f110 0 2024-03-29T04:45:14.021 INFO:tasks.workunit.client.1.smithi152.stdout:5/364: dwrite d6/d16/d47/f61 [4194304,4194304] 0 2024-03-29T04:45:14.022 INFO:tasks.workunit.client.1.smithi152.stdout:5/365: truncate d6/f41 916541 0 2024-03-29T04:45:14.023 INFO:tasks.workunit.client.0.smithi142.stdout:9/862: symlink d0/d12/d10e/dc7/l111 0 2024-03-29T04:45:14.023 INFO:tasks.workunit.client.1.smithi152.stdout:5/366: chown d6/d1d/d3b/d55/l4f 350803 1 2024-03-29T04:45:14.026 INFO:tasks.workunit.client.0.smithi142.stdout:9/863: rename d0/d12/d50/d52/c77 to d0/d12/d10e/dc7/d6a/c112 0 2024-03-29T04:45:14.026 INFO:tasks.workunit.client.1.smithi152.stdout:5/367: rename d6/d16/d25/d2e/f72 to d6/d1d/d3b/d55/f7a 0 2024-03-29T04:45:14.029 INFO:tasks.workunit.client.0.smithi142.stdout:9/864: mknod d0/d12/d10e/dc7/daa/c113 0 2024-03-29T04:45:14.033 INFO:tasks.workunit.client.0.smithi142.stdout:9/865: symlink d0/d2/d61/d64/dee/d10d/d83/l114 0 2024-03-29T04:45:14.033 INFO:tasks.workunit.client.0.smithi142.stdout:3/746: sync 2024-03-29T04:45:14.033 INFO:tasks.workunit.client.0.smithi142.stdout:4/863: sync 2024-03-29T04:45:14.033 INFO:tasks.workunit.client.0.smithi142.stdout:6/898: sync 2024-03-29T04:45:14.033 INFO:tasks.workunit.client.1.smithi152.stdout:5/368: dwrite d6/d1b/f5a [0,4194304] 0 2024-03-29T04:45:14.034 INFO:tasks.workunit.client.0.smithi142.stdout:3/747: write d3/d1f/fbd [3736505,103618] 0 2024-03-29T04:45:14.038 INFO:tasks.workunit.client.1.smithi152.stdout:5/369: getdents d6/d16/d25 0 2024-03-29T04:45:14.039 INFO:tasks.workunit.client.0.smithi142.stdout:4/864: creat d0/d9/d11/d4c/d64/f121 x:0 0 0 2024-03-29T04:45:14.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:13 smithi152 ceph-mon[37177]: Upgrade: Updating mgr.smithi152.usoolh 2024-03-29T04:45:14.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:13 smithi152 ceph-mon[37177]: Deploying daemon mgr.smithi152.usoolh on smithi152 2024-03-29T04:45:14.040 INFO:tasks.workunit.client.1.smithi152.stdout:6/404: dread d1/d3/d9/f11 [0,4194304] 0 2024-03-29T04:45:14.041 INFO:tasks.workunit.client.0.smithi142.stdout:9/866: creat d0/d2/d61/d64/dee/d10d/dc8/f115 x:0 0 0 2024-03-29T04:45:14.043 INFO:tasks.workunit.client.0.smithi142.stdout:6/899: rename d1c/d45/d8a/d118 to d1c/d33/d46/d61/d9e/d122 0 2024-03-29T04:45:14.043 INFO:tasks.workunit.client.1.smithi152.stdout:6/405: mknod d1/d3/d1a/d1b/d29/d6f/c88 0 2024-03-29T04:45:14.045 INFO:tasks.workunit.client.0.smithi142.stdout:4/865: rmdir d0/d9/d19/d2c 39 2024-03-29T04:45:14.047 INFO:tasks.workunit.client.0.smithi142.stdout:6/900: dread d1c/f31 [0,4194304] 0 2024-03-29T04:45:14.048 INFO:tasks.workunit.client.1.smithi152.stdout:5/370: dwrite d6/d1d/d3b/d55/f7a [0,4194304] 0 2024-03-29T04:45:14.048 INFO:tasks.workunit.client.1.smithi152.stdout:6/406: mknod d1/d3/d9/d13/d3e/c89 0 2024-03-29T04:45:14.048 INFO:tasks.workunit.client.0.smithi142.stdout:6/901: write d1c/d45/fa5 [24299,50001] 0 2024-03-29T04:45:14.048 INFO:tasks.workunit.client.0.smithi142.stdout:9/867: unlink d0/d12/d10e/cdb 0 2024-03-29T04:45:14.049 INFO:tasks.workunit.client.0.smithi142.stdout:9/868: read - d0/d2/d61/fe5 zero size 2024-03-29T04:45:14.051 INFO:tasks.workunit.client.1.smithi152.stdout:5/371: symlink d6/d16/l7b 0 2024-03-29T04:45:14.052 INFO:tasks.workunit.client.0.smithi142.stdout:9/869: dread d0/f94 [0,4194304] 0 2024-03-29T04:45:14.053 INFO:tasks.workunit.client.1.smithi152.stdout:5/372: chown d6/d16/d47/f61 1799853212 1 2024-03-29T04:45:14.053 INFO:tasks.workunit.client.0.smithi142.stdout:9/870: fdatasync d0/d12/d50/d52/fdd 0 2024-03-29T04:45:14.054 INFO:tasks.workunit.client.1.smithi152.stdout:5/373: truncate d6/d16/f73 407463 0 2024-03-29T04:45:14.054 INFO:tasks.workunit.client.0.smithi142.stdout:9/871: chown d0/f32 0 1 2024-03-29T04:45:14.054 INFO:tasks.workunit.client.1.smithi152.stdout:5/374: dread - d6/d1b/f78 zero size 2024-03-29T04:45:14.056 INFO:tasks.workunit.client.0.smithi142.stdout:9/872: chown d0/d12/d50/ff8 4081 1 2024-03-29T04:45:14.056 INFO:tasks.workunit.client.0.smithi142.stdout:3/748: dread d3/d29/d2d/f3f [0,4194304] 0 2024-03-29T04:45:14.057 INFO:tasks.workunit.client.0.smithi142.stdout:3/749: stat d3/d29/d2b/d6a/l9b 0 2024-03-29T04:45:14.058 INFO:tasks.workunit.client.1.smithi152.stdout:5/375: rename d6/d16/d5c/l5f to d6/d1d/d30/d6a/l7c 0 2024-03-29T04:45:14.061 INFO:tasks.workunit.client.0.smithi142.stdout:9/873: creat d0/d2/d39/f116 x:0 0 0 2024-03-29T04:45:14.063 INFO:tasks.workunit.client.0.smithi142.stdout:3/750: truncate d3/d29/d2b/d40/d67/d8b/d4d/d50/f51 3272384 0 2024-03-29T04:45:14.064 INFO:tasks.workunit.client.0.smithi142.stdout:4/866: write d0/d9/d11/d4c/d5f/f11f [885785,49796] 0 2024-03-29T04:45:14.065 INFO:tasks.workunit.client.0.smithi142.stdout:6/902: write d1c/d33/d46/d61/d9e/fd1 [1632040,31225] 0 2024-03-29T04:45:14.066 INFO:tasks.workunit.client.1.smithi152.stdout:5/376: dwrite d6/d16/f4d [0,4194304] 0 2024-03-29T04:45:14.067 INFO:tasks.workunit.client.0.smithi142.stdout:9/874: link d0/d6c/la9 d0/l117 0 2024-03-29T04:45:14.067 INFO:tasks.workunit.client.1.smithi152.stdout:5/377: fsync d6/d16/d5c/f74 0 2024-03-29T04:45:14.067 INFO:tasks.workunit.client.0.smithi142.stdout:9/875: stat d0/d12/fbc 0 2024-03-29T04:45:14.069 INFO:tasks.workunit.client.0.smithi142.stdout:4/867: rename d0/d9/d11/f8c to d0/dc/d15/d3d/f122 0 2024-03-29T04:45:14.071 INFO:tasks.workunit.client.0.smithi142.stdout:6/903: getdents d1c/d45/d8a/df6/df0 0 2024-03-29T04:45:14.073 INFO:tasks.workunit.client.0.smithi142.stdout:4/868: write d0/d9/d11/d4c/d5f/d81/dc9/d47/fda [1236108,47160] 0 2024-03-29T04:45:14.073 INFO:tasks.workunit.client.1.smithi152.stdout:5/378: dread d6/d16/d47/d56/d63/f6d [0,4194304] 0 2024-03-29T04:45:14.074 INFO:tasks.workunit.client.0.smithi142.stdout:3/751: rename d3/d1f/d24/d47/d87/cc9 to d3/d29/d2b/d40/d67/d8b/d4d/cfd 0 2024-03-29T04:45:14.075 INFO:tasks.workunit.client.0.smithi142.stdout:9/876: dwrite d0/d12/d10e/f82 [0,4194304] 0 2024-03-29T04:45:14.076 INFO:tasks.workunit.client.0.smithi142.stdout:9/877: chown d0/d2/d61/fc3 33822959 1 2024-03-29T04:45:14.077 INFO:tasks.workunit.client.0.smithi142.stdout:4/869: symlink d0/d9/db2/dba/l123 0 2024-03-29T04:45:14.078 INFO:tasks.workunit.client.0.smithi142.stdout:3/752: fdatasync d3/d1f/d24/d47/f90 0 2024-03-29T04:45:14.079 INFO:tasks.workunit.client.1.smithi152.stdout:5/379: dread d6/d1b/f5a [0,4194304] 0 2024-03-29T04:45:14.080 INFO:tasks.workunit.client.1.smithi152.stdout:5/380: stat d6/d1d/f6e 0 2024-03-29T04:45:14.080 INFO:tasks.workunit.client.1.smithi152.stdout:6/407: dread d1/d3/d1a/d1b/d29/d6f/f3c [0,4194304] 0 2024-03-29T04:45:14.080 INFO:tasks.workunit.client.1.smithi152.stdout:5/381: readlink d6/d7/lc 0 2024-03-29T04:45:14.083 INFO:tasks.workunit.client.1.smithi152.stdout:5/382: chown d6/d16/d25/f3a 4 1 2024-03-29T04:45:14.084 INFO:tasks.workunit.client.0.smithi142.stdout:3/753: mkdir d3/dfe 0 2024-03-29T04:45:14.085 INFO:tasks.workunit.client.0.smithi142.stdout:3/754: truncate d3/f3c 4263716 0 2024-03-29T04:45:14.088 INFO:tasks.workunit.client.0.smithi142.stdout:3/755: write d3/d29/d2d/ff8 [861183,50207] 0 2024-03-29T04:45:14.090 INFO:tasks.workunit.client.1.smithi152.stdout:6/408: dwrite d1/d3/d1a/d1b/d29/f5f [4194304,4194304] 0 2024-03-29T04:45:14.092 INFO:tasks.workunit.client.0.smithi142.stdout:3/756: rename d3/d1f/d59/dc7/lf6 to d3/d29/d2d/d6c/lff 0 2024-03-29T04:45:14.093 INFO:tasks.workunit.client.1.smithi152.stdout:6/409: mkdir d1/d3/df/d8a 0 2024-03-29T04:45:14.095 INFO:tasks.workunit.client.0.smithi142.stdout:6/904: write d1c/d33/f99 [526290,13547] 0 2024-03-29T04:45:14.096 INFO:tasks.workunit.client.1.smithi152.stdout:8/269: sync 2024-03-29T04:45:14.096 INFO:tasks.workunit.client.0.smithi142.stdout:6/905: write d1c/f11a [11919,110357] 0 2024-03-29T04:45:14.097 INFO:tasks.workunit.client.0.smithi142.stdout:9/878: write d0/d2/d61/d64/fac [408240,49862] 0 2024-03-29T04:45:14.098 INFO:tasks.workunit.client.1.smithi152.stdout:8/270: chown d1/d2/d3/l14 756 1 2024-03-29T04:45:14.098 INFO:tasks.workunit.client.0.smithi142.stdout:9/879: stat d0/d2/d61/d64/dfa/ffc 0 2024-03-29T04:45:14.099 INFO:tasks.workunit.client.0.smithi142.stdout:6/906: mkdir d1c/d33/d46/d66/dc1/d123 0 2024-03-29T04:45:14.101 INFO:tasks.workunit.client.0.smithi142.stdout:4/870: truncate d0/d9/db2/fb3 3910046 0 2024-03-29T04:45:14.101 INFO:tasks.workunit.client.1.smithi152.stdout:8/271: mknod d1/c5f 0 2024-03-29T04:45:14.102 INFO:tasks.workunit.client.0.smithi142.stdout:9/880: stat d0/lb 0 2024-03-29T04:45:14.102 INFO:tasks.workunit.client.0.smithi142.stdout:6/907: rmdir d1c/d33/d46 39 2024-03-29T04:45:14.103 INFO:tasks.workunit.client.1.smithi152.stdout:5/383: truncate d6/d16/f4d 3912172 0 2024-03-29T04:45:14.103 INFO:tasks.workunit.client.0.smithi142.stdout:9/881: dread d0/d12/d10e/d3c/d4f/f8c [0,4194304] 0 2024-03-29T04:45:14.103 INFO:tasks.workunit.client.1.smithi152.stdout:5/384: dread - d6/d16/f4e zero size 2024-03-29T04:45:14.104 INFO:tasks.workunit.client.1.smithi152.stdout:6/410: write d1/d3/d1a/f1d [3867982,120714] 0 2024-03-29T04:45:14.105 INFO:tasks.workunit.client.1.smithi152.stdout:8/272: mknod d1/d21/d40/d47/c60 0 2024-03-29T04:45:14.105 INFO:tasks.workunit.client.0.smithi142.stdout:9/882: write d0/d6c/dc2/fc9 [1733651,13334] 0 2024-03-29T04:45:14.105 INFO:tasks.workunit.client.0.smithi142.stdout:6/908: dread d1c/f34 [0,4194304] 0 2024-03-29T04:45:14.106 INFO:tasks.workunit.client.1.smithi152.stdout:5/385: mknod d6/d1b/c7d 0 2024-03-29T04:45:14.108 INFO:tasks.workunit.client.0.smithi142.stdout:9/883: mkdir d0/d6c/d118 0 2024-03-29T04:45:14.108 INFO:tasks.workunit.client.1.smithi152.stdout:8/273: symlink d1/d2/d1d/d36/d4b/l61 0 2024-03-29T04:45:14.109 INFO:tasks.workunit.client.1.smithi152.stdout:6/411: fdatasync d1/d3/f18 0 2024-03-29T04:45:14.110 INFO:tasks.workunit.client.0.smithi142.stdout:9/884: write d0/d12/d50/f63 [4638460,39182] 0 2024-03-29T04:45:14.110 INFO:tasks.workunit.client.0.smithi142.stdout:4/871: dwrite d0/d12/f3b [0,4194304] 0 2024-03-29T04:45:14.111 INFO:tasks.workunit.client.1.smithi152.stdout:8/274: fsync d1/f10 0 2024-03-29T04:45:14.111 INFO:tasks.workunit.client.1.smithi152.stdout:8/275: dread - d1/d2/d1d/f41 zero size 2024-03-29T04:45:14.112 INFO:tasks.workunit.client.1.smithi152.stdout:6/412: symlink d1/d3/d1a/l8b 0 2024-03-29T04:45:14.112 INFO:tasks.workunit.client.0.smithi142.stdout:4/872: write d0/d9/d11/d97/f11d [187997,95636] 0 2024-03-29T04:45:14.114 INFO:tasks.workunit.client.1.smithi152.stdout:6/413: chown f0 11 1 2024-03-29T04:45:14.116 INFO:tasks.workunit.client.0.smithi142.stdout:9/885: mknod d0/c119 0 2024-03-29T04:45:14.116 INFO:tasks.workunit.client.0.smithi142.stdout:6/909: getdents d1c/d45 0 2024-03-29T04:45:14.118 INFO:tasks.workunit.client.0.smithi142.stdout:6/910: write d1c/d45/d8a/df6/f87 [222409,128646] 0 2024-03-29T04:45:14.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:13 smithi142 ceph-mon[29027]: Upgrade: Updating mgr.smithi152.usoolh 2024-03-29T04:45:14.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:13 smithi142 ceph-mon[29027]: Deploying daemon mgr.smithi152.usoolh on smithi152 2024-03-29T04:45:14.121 INFO:tasks.workunit.client.0.smithi142.stdout:9/886: link d0/f4b d0/d12/d10e/d3c/d6f/f11a 0 2024-03-29T04:45:14.124 INFO:tasks.workunit.client.0.smithi142.stdout:9/887: creat d0/d2/d61/d64/dee/d10d/d83/f11b x:0 0 0 2024-03-29T04:45:14.126 INFO:tasks.workunit.client.0.smithi142.stdout:9/888: read d0/d2/f1f [3481537,91572] 0 2024-03-29T04:45:14.127 INFO:tasks.workunit.client.0.smithi142.stdout:9/889: fsync d0/d2/d61/d64/dee/d10d/f101 0 2024-03-29T04:45:14.129 INFO:tasks.workunit.client.0.smithi142.stdout:9/890: getdents d0/d6c/d118 0 2024-03-29T04:45:14.130 INFO:tasks.workunit.client.1.smithi152.stdout:6/414: dread d1/d3/d1a/f61 [0,4194304] 0 2024-03-29T04:45:14.130 INFO:tasks.workunit.client.0.smithi142.stdout:9/891: dread - d0/d12/d50/d52/da3/d10a/fd3 zero size 2024-03-29T04:45:14.131 INFO:tasks.workunit.client.1.smithi152.stdout:6/415: readlink d1/d3/d9/d13/d3e/l4b 0 2024-03-29T04:45:14.132 INFO:tasks.workunit.client.1.smithi152.stdout:6/416: chown d1/d3/df/d8a 71393970 1 2024-03-29T04:45:14.135 INFO:tasks.workunit.client.1.smithi152.stdout:8/276: write d1/d21/f32 [645523,368] 0 2024-03-29T04:45:14.136 INFO:tasks.workunit.client.1.smithi152.stdout:6/417: dread d1/d3/d9/f11 [0,4194304] 0 2024-03-29T04:45:14.138 INFO:tasks.workunit.client.1.smithi152.stdout:6/418: rmdir d1/d3/df 39 2024-03-29T04:45:14.139 INFO:tasks.workunit.client.1.smithi152.stdout:6/419: fdatasync d1/d3/d9/d13/f31 0 2024-03-29T04:45:14.139 INFO:tasks.workunit.client.1.smithi152.stdout:8/277: creat d1/d2/d1d/d36/d4b/d51/d5c/f62 x:0 0 0 2024-03-29T04:45:14.142 INFO:tasks.workunit.client.0.smithi142.stdout:6/911: truncate d1c/d45/f4e 1404684 0 2024-03-29T04:45:14.142 INFO:tasks.workunit.client.1.smithi152.stdout:8/278: creat d1/d2/d1d/d42/f63 x:0 0 0 2024-03-29T04:45:14.143 INFO:tasks.workunit.client.1.smithi152.stdout:8/279: dread - d1/d2/d1d/f41 zero size 2024-03-29T04:45:14.143 INFO:tasks.workunit.client.0.smithi142.stdout:4/873: dwrite d0/d9/d11/d4c/d5f/fd0 [0,4194304] 0 2024-03-29T04:45:14.144 INFO:tasks.workunit.client.0.smithi142.stdout:6/912: write d1c/d33/d46/d66/dc1/fd6 [287362,85770] 0 2024-03-29T04:45:14.145 INFO:tasks.workunit.client.1.smithi152.stdout:6/420: rename d1/d3/d1a/d1b/d29/d6f/d83 to d1/d3/d9/d13/d28/d45/d8c 0 2024-03-29T04:45:14.147 INFO:tasks.workunit.client.1.smithi152.stdout:8/280: mknod d1/d2/d3/c64 0 2024-03-29T04:45:14.148 INFO:tasks.workunit.client.0.smithi142.stdout:4/874: creat d0/d9/d11/d4c/d5f/f124 x:0 0 0 2024-03-29T04:45:14.148 INFO:tasks.workunit.client.1.smithi152.stdout:8/281: chown d1/d2/d3/l14 9253 1 2024-03-29T04:45:14.148 INFO:tasks.workunit.client.1.smithi152.stdout:6/421: write d1/d3/d1a/d1b/d29/d6f/f69 [665328,87711] 0 2024-03-29T04:45:14.150 INFO:tasks.workunit.client.1.smithi152.stdout:6/422: chown d1/d3/d9/d13/d28/d45/f80 1927967171 1 2024-03-29T04:45:14.151 INFO:tasks.workunit.client.0.smithi142.stdout:6/913: unlink d1c/d45/d51/f83 0 2024-03-29T04:45:14.152 INFO:tasks.workunit.client.1.smithi152.stdout:6/423: stat d1/d3/d1a/d1b/d29/d6f/l6e 0 2024-03-29T04:45:14.153 INFO:tasks.workunit.client.1.smithi152.stdout:6/424: stat d1/d3/d1a/d1b/l6d 0 2024-03-29T04:45:14.154 INFO:tasks.workunit.client.1.smithi152.stdout:8/282: truncate d1/d2/d1d/f23 352882 0 2024-03-29T04:45:14.155 INFO:tasks.workunit.client.0.smithi142.stdout:4/875: mknod d0/d9/d11/d4c/d5f/d81/c125 0 2024-03-29T04:45:14.156 INFO:tasks.workunit.client.0.smithi142.stdout:6/914: mknod d1c/d33/d46/d61/d112/c124 0 2024-03-29T04:45:14.156 INFO:tasks.workunit.client.1.smithi152.stdout:8/283: mknod d1/d21/d40/d47/c65 0 2024-03-29T04:45:14.158 INFO:tasks.workunit.client.0.smithi142.stdout:6/915: read - d1c/d45/d8a/df6/df4/f104 zero size 2024-03-29T04:45:14.158 INFO:tasks.workunit.client.1.smithi152.stdout:6/425: creat d1/d3/df/d84/f8d x:0 0 0 2024-03-29T04:45:14.160 INFO:tasks.workunit.client.0.smithi142.stdout:6/916: truncate d1c/d33/db6/f81 488544 0 2024-03-29T04:45:14.160 INFO:tasks.workunit.client.1.smithi152.stdout:6/426: mknod d1/d3/d9/d13/d3e/c8e 0 2024-03-29T04:45:14.161 INFO:tasks.workunit.client.1.smithi152.stdout:6/427: dread - d1/d3/d9/d13/d3e/d54/f7c zero size 2024-03-29T04:45:14.162 INFO:tasks.workunit.client.0.smithi142.stdout:6/917: symlink d1c/d33/db6/l125 0 2024-03-29T04:45:14.164 INFO:tasks.workunit.client.0.smithi142.stdout:6/918: mkdir d1c/d90/d94/d126 0 2024-03-29T04:45:14.164 INFO:tasks.workunit.client.1.smithi152.stdout:6/428: mkdir d1/d3/d1a/d1b/d87/d8f 0 2024-03-29T04:45:14.165 INFO:tasks.workunit.client.1.smithi152.stdout:6/429: dread - d1/d3/df/d84/f57 zero size 2024-03-29T04:45:14.166 INFO:tasks.workunit.client.1.smithi152.stdout:6/430: chown d1/f50 129 1 2024-03-29T04:45:14.166 INFO:tasks.workunit.client.0.smithi142.stdout:6/919: mknod d1c/d33/c127 0 2024-03-29T04:45:14.167 INFO:tasks.workunit.client.1.smithi152.stdout:8/284: read d1/d2/d1d/f23 [95341,14429] 0 2024-03-29T04:45:14.169 INFO:tasks.workunit.client.0.smithi142.stdout:6/920: symlink d1c/d90/ddc/l128 0 2024-03-29T04:45:14.169 INFO:tasks.workunit.client.1.smithi152.stdout:8/285: fdatasync d1/d2/d1d/f30 0 2024-03-29T04:45:14.169 INFO:tasks.workunit.client.1.smithi152.stdout:6/431: symlink d1/d3/d1a/d1b/d87/d8f/l90 0 2024-03-29T04:45:14.172 INFO:tasks.workunit.client.1.smithi152.stdout:8/286: chown d1/la 1515 1 2024-03-29T04:45:14.172 INFO:tasks.workunit.client.1.smithi152.stdout:8/287: fdatasync d1/d2/d3/f44 0 2024-03-29T04:45:14.173 INFO:tasks.workunit.client.1.smithi152.stdout:6/432: write d1/d3/d1a/d1b/d29/d6f/f3c [2674829,26245] 0 2024-03-29T04:45:14.174 INFO:tasks.workunit.client.1.smithi152.stdout:6/433: chown d1/d3/d1a/d1b/c51 122455119 1 2024-03-29T04:45:14.175 INFO:tasks.workunit.client.1.smithi152.stdout:8/288: creat d1/d2/d1d/d42/f66 x:0 0 0 2024-03-29T04:45:14.175 INFO:tasks.workunit.client.1.smithi152.stdout:6/434: write d1/f17 [603743,6211] 0 2024-03-29T04:45:14.176 INFO:tasks.workunit.client.0.smithi142.stdout:6/921: dwrite d1c/d45/d8a/fea [0,4194304] 0 2024-03-29T04:45:14.177 INFO:tasks.workunit.client.1.smithi152.stdout:6/435: chown d1/d3/d1a/d1b/d29/d6f/c88 31 1 2024-03-29T04:45:14.178 INFO:tasks.workunit.client.0.smithi142.stdout:4/876: dread d0/d9/d11/d4c/d5f/d81/dc9/d47/f4b [4194304,4194304] 0 2024-03-29T04:45:14.178 INFO:tasks.workunit.client.1.smithi152.stdout:8/289: mknod d1/d2/d1d/d42/c67 0 2024-03-29T04:45:14.181 INFO:tasks.workunit.client.0.smithi142.stdout:4/877: dread - d0/dc/d15/d3d/fe8 zero size 2024-03-29T04:45:14.186 INFO:tasks.workunit.client.0.smithi142.stdout:6/922: symlink d1c/d33/d46/d86/dc0/d113/l129 0 2024-03-29T04:45:14.186 INFO:tasks.workunit.client.0.smithi142.stdout:4/878: mkdir d0/d9/d11/d4c/d5f/d81/dc9/d126 0 2024-03-29T04:45:14.187 INFO:tasks.workunit.client.0.smithi142.stdout:4/879: truncate d0/d9/db2/ffe 1854024 0 2024-03-29T04:45:14.188 INFO:tasks.workunit.client.0.smithi142.stdout:6/923: fsync fc 0 2024-03-29T04:45:14.191 INFO:tasks.workunit.client.1.smithi152.stdout:8/290: dwrite d1/d2/d1d/f23 [0,4194304] 0 2024-03-29T04:45:14.194 INFO:tasks.workunit.client.1.smithi152.stdout:8/291: chown d1/f26 504720 1 2024-03-29T04:45:14.194 INFO:tasks.workunit.client.0.smithi142.stdout:4/880: truncate d0/d9/db2/dba/dbc/f92 4530529 0 2024-03-29T04:45:14.195 INFO:tasks.workunit.client.1.smithi152.stdout:6/436: dwrite d1/d3/d9/d13/f46 [0,4194304] 0 2024-03-29T04:45:14.195 INFO:tasks.workunit.client.0.smithi142.stdout:4/881: chown d0/d45/f60 38 1 2024-03-29T04:45:14.196 INFO:tasks.workunit.client.0.smithi142.stdout:6/924: mkdir d1c/d12a 0 2024-03-29T04:45:14.198 INFO:tasks.workunit.client.0.smithi142.stdout:4/882: creat d0/d9/db2/dba/dbc/daf/f127 x:0 0 0 2024-03-29T04:45:14.198 INFO:tasks.workunit.client.0.smithi142.stdout:1/844: sync 2024-03-29T04:45:14.198 INFO:tasks.workunit.client.1.smithi152.stdout:8/292: creat d1/d2/d1d/d36/d4b/f68 x:0 0 0 2024-03-29T04:45:14.199 INFO:tasks.workunit.client.1.smithi152.stdout:8/293: dread - d1/d2/d1d/f3a zero size 2024-03-29T04:45:14.200 INFO:tasks.workunit.client.1.smithi152.stdout:6/437: mkdir d1/d3/d91 0 2024-03-29T04:45:14.201 INFO:tasks.workunit.client.0.smithi142.stdout:4/883: write d0/d9/d11/d4c/d5f/f124 [760656,52746] 0 2024-03-29T04:45:14.201 INFO:tasks.workunit.client.0.smithi142.stdout:1/845: chown d8/db/dc/d77/db9/dbe/d15/d9c 51 1 2024-03-29T04:45:14.202 INFO:tasks.workunit.client.1.smithi152.stdout:8/294: getdents d1/d2/d1d/d36/d5d 0 2024-03-29T04:45:14.202 INFO:tasks.workunit.client.0.smithi142.stdout:1/846: read - d8/db/dd6/d113/fe2 zero size 2024-03-29T04:45:14.203 INFO:tasks.workunit.client.1.smithi152.stdout:6/438: truncate d1/d3/d9/d13/d3e/d54/f82 866872 0 2024-03-29T04:45:14.204 INFO:tasks.workunit.client.1.smithi152.stdout:6/439: write d1/d3/d1a/d1b/d29/f7e [703681,123367] 0 2024-03-29T04:45:14.209 INFO:tasks.workunit.client.1.smithi152.stdout:6/440: mknod d1/d3/d9/d13/d28/d45/c92 0 2024-03-29T04:45:14.209 INFO:tasks.workunit.client.0.smithi142.stdout:4/884: dwrite d0/d9/d11/d4c/d5f/f6c [0,4194304] 0 2024-03-29T04:45:14.211 INFO:tasks.workunit.client.1.smithi152.stdout:8/295: link d1/d2/d1d/f3d d1/d2/d3c/f69 0 2024-03-29T04:45:14.213 INFO:tasks.workunit.client.0.smithi142.stdout:4/885: fsync d0/dc/d15/d3d/d65/f90 0 2024-03-29T04:45:14.216 INFO:tasks.workunit.client.1.smithi152.stdout:8/296: rename d1/d2/d1d/d36/d4b/d51/d5c to d1/d21/d40/d6a 0 2024-03-29T04:45:14.217 INFO:tasks.workunit.client.1.smithi152.stdout:6/441: getdents d1/d3/d1a/d1b/d87 0 2024-03-29T04:45:14.222 INFO:tasks.workunit.client.1.smithi152.stdout:6/442: dread d1/d3/d9/d13/d3e/f4d [0,4194304] 0 2024-03-29T04:45:14.222 INFO:tasks.workunit.client.1.smithi152.stdout:8/297: rename d1/d2/d1d/d5a to d1/d2/d6b 0 2024-03-29T04:45:14.223 INFO:tasks.workunit.client.0.smithi142.stdout:1/847: write d8/db/dc/d77/db9/dbe/f11 [1867775,35718] 0 2024-03-29T04:45:14.224 INFO:tasks.workunit.client.0.smithi142.stdout:4/886: write d0/d45/fad [2250312,29602] 0 2024-03-29T04:45:14.226 INFO:tasks.workunit.client.1.smithi152.stdout:6/443: rmdir d1/d3/d1a/d1b/d29/d6f 39 2024-03-29T04:45:14.227 INFO:tasks.workunit.client.0.smithi142.stdout:4/887: fdatasync d0/d9/f4d 0 2024-03-29T04:45:14.230 INFO:tasks.workunit.client.0.smithi142.stdout:4/888: write d0/d9/f49 [3154477,12017] 0 2024-03-29T04:45:14.230 INFO:tasks.workunit.client.1.smithi152.stdout:8/298: dwrite d1/d2/d3/f44 [0,4194304] 0 2024-03-29T04:45:14.231 INFO:tasks.workunit.client.0.smithi142.stdout:1/848: dwrite d8/db/dc/d77/d31/d4d/fef [0,4194304] 0 2024-03-29T04:45:14.232 INFO:tasks.workunit.client.0.smithi142.stdout:4/889: chown d0/d9/d11/d4c/d64 9889 1 2024-03-29T04:45:14.234 INFO:tasks.workunit.client.1.smithi152.stdout:8/299: unlink d1/c19 0 2024-03-29T04:45:14.234 INFO:tasks.workunit.client.0.smithi142.stdout:1/849: symlink d8/db/dd6/l11a 0 2024-03-29T04:45:14.235 INFO:tasks.workunit.client.0.smithi142.stdout:4/890: truncate d0/dc/f68 161789 0 2024-03-29T04:45:14.236 INFO:tasks.workunit.client.0.smithi142.stdout:1/850: chown d8/db/dd6/c104 2 1 2024-03-29T04:45:14.236 INFO:tasks.workunit.client.1.smithi152.stdout:6/444: getdents d1/d3/d9/d13/d28 0 2024-03-29T04:45:14.240 INFO:tasks.workunit.client.0.smithi142.stdout:1/851: mkdir d8/db/dc/d77/d5a/d11b 0 2024-03-29T04:45:14.240 INFO:tasks.workunit.client.0.smithi142.stdout:4/891: dread d0/d9/d11/ff9 [0,4194304] 0 2024-03-29T04:45:14.241 INFO:tasks.workunit.client.1.smithi152.stdout:6/445: mkdir d1/d3/d9/d13/d3e/d93 0 2024-03-29T04:45:14.242 INFO:tasks.workunit.client.1.smithi152.stdout:8/300: dwrite d1/d2/d3/f44 [4194304,4194304] 0 2024-03-29T04:45:14.244 INFO:tasks.workunit.client.0.smithi142.stdout:1/852: rmdir d8/db/dc/d66 39 2024-03-29T04:45:14.246 INFO:tasks.workunit.client.1.smithi152.stdout:2/274: sync 2024-03-29T04:45:14.246 INFO:tasks.workunit.client.1.smithi152.stdout:9/380: sync 2024-03-29T04:45:14.246 INFO:tasks.workunit.client.1.smithi152.stdout:4/290: sync 2024-03-29T04:45:14.247 INFO:tasks.workunit.client.1.smithi152.stdout:0/432: sync 2024-03-29T04:45:14.247 INFO:tasks.workunit.client.1.smithi152.stdout:1/471: sync 2024-03-29T04:45:14.247 INFO:tasks.workunit.client.1.smithi152.stdout:3/298: sync 2024-03-29T04:45:14.247 INFO:tasks.workunit.client.1.smithi152.stdout:7/360: sync 2024-03-29T04:45:14.247 INFO:tasks.workunit.client.1.smithi152.stdout:6/446: creat d1/f94 x:0 0 0 2024-03-29T04:45:14.249 INFO:tasks.workunit.client.0.smithi142.stdout:1/853: rename d8/db/dc/d77/db9/dbe/d2d/f115 to d8/db/dc/d77/de9/f11c 0 2024-03-29T04:45:14.250 INFO:tasks.workunit.client.0.smithi142.stdout:4/892: truncate d0/d9/d11/d97/fe1 3789230 0 2024-03-29T04:45:14.251 INFO:tasks.workunit.client.1.smithi152.stdout:6/447: chown d1/d3/d1a/d1b/l6d 23545706 1 2024-03-29T04:45:14.251 INFO:tasks.workunit.client.1.smithi152.stdout:2/275: mknod d0/d3/c5e 0 2024-03-29T04:45:14.252 INFO:tasks.workunit.client.0.smithi142.stdout:4/893: readlink d0/d9/d11/d4c/d64/d94/ddd/l108 0 2024-03-29T04:45:14.253 INFO:tasks.workunit.client.0.smithi142.stdout:4/894: chown d0/d9/d11/d78/ca0 1006 1 2024-03-29T04:45:14.253 INFO:tasks.workunit.client.1.smithi152.stdout:3/299: mknod d7/d19/d1b/c4f 0 2024-03-29T04:45:14.254 INFO:tasks.workunit.client.0.smithi142.stdout:1/854: dread d8/db/f20 [0,4194304] 0 2024-03-29T04:45:14.254 INFO:tasks.workunit.client.1.smithi152.stdout:8/301: write d1/d2/d1d/f3d [2812024,154] 0 2024-03-29T04:45:14.254 INFO:tasks.workunit.client.1.smithi152.stdout:3/300: dread - d7/f3e zero size 2024-03-29T04:45:14.256 INFO:tasks.workunit.client.1.smithi152.stdout:2/276: fsync d0/d3/d30/d15/f20 0 2024-03-29T04:45:14.257 INFO:tasks.workunit.client.1.smithi152.stdout:7/361: rmdir d7/d8/d12/d35/d5e 0 2024-03-29T04:45:14.257 INFO:tasks.workunit.client.1.smithi152.stdout:8/302: dread - d1/d21/d40/d6a/f62 zero size 2024-03-29T04:45:14.258 INFO:tasks.workunit.client.1.smithi152.stdout:7/362: write d7/f59 [16166,2485] 0 2024-03-29T04:45:14.259 INFO:tasks.workunit.client.1.smithi152.stdout:8/303: rmdir d1/d21/d40/d6a 39 2024-03-29T04:45:14.261 INFO:tasks.workunit.client.1.smithi152.stdout:0/433: dwrite d2/da/d17/f6d [0,4194304] 0 2024-03-29T04:45:14.265 INFO:tasks.workunit.client.1.smithi152.stdout:3/301: dread - d7/d19/d1b/d1e/d22/f4d zero size 2024-03-29T04:45:14.270 INFO:tasks.workunit.client.1.smithi152.stdout:9/381: creat d5/db/d1b/d68/f89 x:0 0 0 2024-03-29T04:45:14.270 INFO:tasks.workunit.client.1.smithi152.stdout:0/434: unlink d2/d29/c47 0 2024-03-29T04:45:14.271 INFO:tasks.workunit.client.1.smithi152.stdout:7/363: write d7/d8/d12/d1d/f34 [4602618,44154] 0 2024-03-29T04:45:14.272 INFO:tasks.workunit.client.1.smithi152.stdout:9/382: read - d5/d12/d2c/f78 zero size 2024-03-29T04:45:14.273 INFO:tasks.workunit.client.0.smithi142.stdout:4/895: dread d0/d9/db2/dba/de9/ff8 [0,4194304] 0 2024-03-29T04:45:14.274 INFO:tasks.workunit.client.1.smithi152.stdout:8/304: creat d1/d21/d40/f6c x:0 0 0 2024-03-29T04:45:14.274 INFO:tasks.workunit.client.0.smithi142.stdout:1/855: write d8/db/dc/d77/db9/dbe/d1f/f29 [6119178,10651] 0 2024-03-29T04:45:14.276 INFO:tasks.workunit.client.1.smithi152.stdout:3/302: getdents d7/d19/d1b/d1e/d4e 0 2024-03-29T04:45:14.278 INFO:tasks.workunit.client.0.smithi142.stdout:1/856: rename f5 to d8/db/dd6/d113/f11d 0 2024-03-29T04:45:14.278 INFO:tasks.workunit.client.1.smithi152.stdout:1/472: dread d5/d7/d8/d13/d19/f65 [0,4194304] 0 2024-03-29T04:45:14.279 INFO:tasks.workunit.client.1.smithi152.stdout:2/277: dwrite d0/d3/d30/f33 [0,4194304] 0 2024-03-29T04:45:14.281 INFO:tasks.workunit.client.1.smithi152.stdout:3/303: write d7/f1a [235520,110933] 0 2024-03-29T04:45:14.281 INFO:tasks.workunit.client.1.smithi152.stdout:4/291: dwrite d0/d7/d25/d17/f5b [0,4194304] 0 2024-03-29T04:45:14.282 INFO:tasks.workunit.client.0.smithi142.stdout:4/896: rename d0/d9/d19/d2c/c42 to d0/d9/d11/d4c/d64/c128 0 2024-03-29T04:45:14.282 INFO:tasks.workunit.client.1.smithi152.stdout:0/435: mknod d2/da/d3a/d61/ca8 0 2024-03-29T04:45:14.284 INFO:tasks.workunit.client.0.smithi142.stdout:1/857: creat d8/db/dc/d77/db9/dbe/d2d/f11e x:0 0 0 2024-03-29T04:45:14.285 INFO:tasks.workunit.client.0.smithi142.stdout:1/858: dread - d8/db/dc/d77/db9/dbe/d1f/d4c/f81 zero size 2024-03-29T04:45:14.287 INFO:tasks.workunit.client.0.smithi142.stdout:1/859: chown d8/db/dc/d77/db9/dbe/d15/c18 58505741 1 2024-03-29T04:45:14.291 INFO:tasks.workunit.client.0.smithi142.stdout:1/860: mknod d8/db/dc/d77/db9/dbe/d2d/d44/d4b/c11f 0 2024-03-29T04:45:14.293 INFO:tasks.workunit.client.1.smithi152.stdout:8/305: rmdir d1/d2/d6b 0 2024-03-29T04:45:14.294 INFO:tasks.workunit.client.0.smithi142.stdout:4/897: fsync d0/d9/d11/d97/fe1 0 2024-03-29T04:45:14.295 INFO:tasks.workunit.client.0.smithi142.stdout:1/861: fdatasync d8/db/dc/d77/db9/dbe/d2d/f7d 0 2024-03-29T04:45:14.296 INFO:tasks.workunit.client.1.smithi152.stdout:7/364: dwrite d7/d8/d37/f46 [0,4194304] 0 2024-03-29T04:45:14.297 INFO:tasks.workunit.client.1.smithi152.stdout:3/304: creat d7/d1c/d41/f50 x:0 0 0 2024-03-29T04:45:14.298 INFO:tasks.workunit.client.1.smithi152.stdout:1/473: symlink d5/d7/d8/d13/d19/d80/l8f 0 2024-03-29T04:45:14.298 INFO:tasks.workunit.client.1.smithi152.stdout:2/278: mkdir d0/d3/d5f 0 2024-03-29T04:45:14.299 INFO:tasks.workunit.client.1.smithi152.stdout:7/365: chown d7/d8/d12/d15/l54 4 1 2024-03-29T04:45:14.300 INFO:tasks.workunit.client.1.smithi152.stdout:9/383: dwrite d5/db/d1b/f51 [0,4194304] 0 2024-03-29T04:45:14.302 INFO:tasks.workunit.client.1.smithi152.stdout:2/279: fsync d0/d3/d30/d15/d22/d4c/f4d 0 2024-03-29T04:45:14.303 INFO:tasks.workunit.client.1.smithi152.stdout:8/306: mkdir d1/d2/d6d 0 2024-03-29T04:45:14.304 INFO:tasks.workunit.client.0.smithi142.stdout:1/862: creat d8/db/dc/d77/db9/dbe/d2d/d8b/da4/dd1/f120 x:0 0 0 2024-03-29T04:45:14.305 INFO:tasks.workunit.client.0.smithi142.stdout:4/898: mkdir d0/d9/d11/d97/d129 0 2024-03-29T04:45:14.305 INFO:tasks.workunit.client.1.smithi152.stdout:4/292: dread d0/f11 [0,4194304] 0 2024-03-29T04:45:14.309 INFO:tasks.workunit.client.1.smithi152.stdout:7/366: creat d7/d8/d12/d1d/d41/d45/f6a x:0 0 0 2024-03-29T04:45:14.311 INFO:tasks.workunit.client.1.smithi152.stdout:1/474: dwrite d5/d7/fe [4194304,4194304] 0 2024-03-29T04:45:14.312 INFO:tasks.workunit.client.0.smithi142.stdout:1/863: mknod d8/db/dc/d77/db9/dbe/d2d/d44/d7f/d10e/d96/c121 0 2024-03-29T04:45:14.313 INFO:tasks.workunit.client.1.smithi152.stdout:8/307: dread - d1/d21/d40/d6a/f62 zero size 2024-03-29T04:45:14.315 INFO:tasks.workunit.client.1.smithi152.stdout:2/280: mknod d0/d3/c60 0 2024-03-29T04:45:14.318 INFO:tasks.workunit.client.1.smithi152.stdout:7/367: rename d7/d8/db/f4f to d7/d8/db/df/f6b 0 2024-03-29T04:45:14.320 INFO:tasks.workunit.client.1.smithi152.stdout:9/384: dread d5/db/d1b/f46 [0,4194304] 0 2024-03-29T04:45:14.322 INFO:tasks.workunit.client.1.smithi152.stdout:7/368: chown d7/d8/db/f42 48 1 2024-03-29T04:45:14.322 INFO:tasks.workunit.client.1.smithi152.stdout:8/308: link d1/d21/l37 d1/d2/l6e 0 2024-03-29T04:45:14.323 INFO:tasks.workunit.client.1.smithi152.stdout:2/281: symlink d0/d3/d30/d15/d53/d54/d57/l61 0 2024-03-29T04:45:14.323 INFO:tasks.workunit.client.0.smithi142.stdout:4/899: write d0/d9/d11/ff9 [347241,89764] 0 2024-03-29T04:45:14.324 INFO:tasks.workunit.client.1.smithi152.stdout:8/309: fdatasync d1/f34 0 2024-03-29T04:45:14.324 INFO:tasks.workunit.client.0.smithi142.stdout:1/864: truncate d8/db/dc/d77/d31/d4d/d56/f58 4853901 0 2024-03-29T04:45:14.325 INFO:tasks.workunit.client.1.smithi152.stdout:2/282: fsync d0/d3/df/f36 0 2024-03-29T04:45:14.327 INFO:tasks.workunit.client.1.smithi152.stdout:7/369: mknod d7/d8/d12/d1d/d41/d45/c6c 0 2024-03-29T04:45:14.328 INFO:tasks.workunit.client.1.smithi152.stdout:1/475: dwrite d5/d7/d8/d13/f35 [0,4194304] 0 2024-03-29T04:45:14.328 INFO:tasks.workunit.client.1.smithi152.stdout:2/283: write d0/d3/f39 [454770,82393] 0 2024-03-29T04:45:14.330 INFO:tasks.workunit.client.0.smithi142.stdout:1/865: creat d8/db/dc/d77/db9/f122 x:0 0 0 2024-03-29T04:45:14.337 INFO:tasks.workunit.client.1.smithi152.stdout:0/436: truncate d2/da/d17/d50/d5c/f7b 140925 0 2024-03-29T04:45:14.338 INFO:tasks.workunit.client.0.smithi142.stdout:4/900: dwrite d0/d45/f6a [0,4194304] 0 2024-03-29T04:45:14.338 INFO:tasks.workunit.client.0.smithi142.stdout:1/866: dwrite d8/db/dc/d77/d31/d4d/d56/faf [0,4194304] 0 2024-03-29T04:45:14.339 INFO:tasks.workunit.client.1.smithi152.stdout:3/305: truncate d7/d19/d1b/f23 3453115 0 2024-03-29T04:45:14.339 INFO:tasks.workunit.client.1.smithi152.stdout:1/476: symlink d5/d7/d9/d2b/l90 0 2024-03-29T04:45:14.341 INFO:tasks.workunit.client.1.smithi152.stdout:1/477: readlink d5/l1d 0 2024-03-29T04:45:14.342 INFO:tasks.workunit.client.0.smithi142.stdout:1/867: chown d8/db/dc/d77/db9/dbe/d2d/d8b/dda/fde 12 1 2024-03-29T04:45:14.343 INFO:tasks.workunit.client.1.smithi152.stdout:7/370: dwrite d7/d8/d12/d15/f50 [0,4194304] 0 2024-03-29T04:45:14.343 INFO:tasks.workunit.client.0.smithi142.stdout:4/901: mkdir d0/d9/db2/dba/dbc/d6d/d6e/d12a 0 2024-03-29T04:45:14.345 INFO:tasks.workunit.client.0.smithi142.stdout:1/868: creat d8/db/dc/d77/de9/f123 x:0 0 0 2024-03-29T04:45:14.346 INFO:tasks.workunit.client.1.smithi152.stdout:9/385: link d5/d12/c48 d5/d8/d31/d59/d6f/c8a 0 2024-03-29T04:45:14.350 INFO:tasks.workunit.client.1.smithi152.stdout:2/284: dwrite d0/d3/d30/d15/f29 [0,4194304] 0 2024-03-29T04:45:14.350 INFO:tasks.workunit.client.0.smithi142.stdout:1/869: chown d8/db/dc/d77/db9/dbe/d1f/d4c/f81 431 1 2024-03-29T04:45:14.350 INFO:tasks.workunit.client.0.smithi142.stdout:2/905: sync 2024-03-29T04:45:14.350 INFO:tasks.workunit.client.0.smithi142.stdout:4/902: symlink d0/d9/d11/d97/d129/l12b 0 2024-03-29T04:45:14.354 INFO:tasks.workunit.client.0.smithi142.stdout:2/906: mknod de/d11/d17/d10b/d22/d27/d3a/c12c 0 2024-03-29T04:45:14.355 INFO:tasks.workunit.client.0.smithi142.stdout:2/907: dread - de/d11/d17/d60/f6d zero size 2024-03-29T04:45:14.355 INFO:tasks.workunit.client.1.smithi152.stdout:9/386: read d5/db/d1b/f46 [885382,9305] 0 2024-03-29T04:45:14.356 INFO:tasks.workunit.client.0.smithi142.stdout:4/903: truncate d0/d9/db2/dba/dbc/d6d/d6e/fc4 1134880 0 2024-03-29T04:45:14.357 INFO:tasks.workunit.client.1.smithi152.stdout:8/310: getdents d1/d21 0 2024-03-29T04:45:14.358 INFO:tasks.workunit.client.1.smithi152.stdout:3/306: unlink d7/d19/d1b/f1d 0 2024-03-29T04:45:14.360 INFO:tasks.workunit.client.1.smithi152.stdout:2/285: write d0/d3/df/f43 [665153,107969] 0 2024-03-29T04:45:14.360 INFO:tasks.workunit.client.0.smithi142.stdout:4/904: symlink d0/d9/db2/dba/dbc/d6d/l12c 0 2024-03-29T04:45:14.361 INFO:tasks.workunit.client.0.smithi142.stdout:4/905: readlink d0/d9/d11/l35 0 2024-03-29T04:45:14.362 INFO:tasks.workunit.client.1.smithi152.stdout:9/387: dread - d5/d12/d2c/f76 zero size 2024-03-29T04:45:14.363 INFO:tasks.workunit.client.1.smithi152.stdout:4/293: dread d0/f13 [0,4194304] 0 2024-03-29T04:45:14.364 INFO:tasks.workunit.client.0.smithi142.stdout:1/870: dwrite d8/db/f14 [0,4194304] 0 2024-03-29T04:45:14.365 INFO:tasks.workunit.client.1.smithi152.stdout:1/478: symlink d5/d7/l91 0 2024-03-29T04:45:14.366 INFO:tasks.workunit.client.1.smithi152.stdout:3/307: write d7/f1a [3344372,72827] 0 2024-03-29T04:45:14.366 INFO:tasks.workunit.client.1.smithi152.stdout:2/286: readlink d0/d3/ld 0 2024-03-29T04:45:14.367 INFO:tasks.workunit.client.0.smithi142.stdout:4/906: creat d0/d9/d11/d4c/d5f/d81/dc9/d126/f12d x:0 0 0 2024-03-29T04:45:14.367 INFO:tasks.workunit.client.1.smithi152.stdout:7/371: dread d7/d8/db/df/f2b [0,4194304] 0 2024-03-29T04:45:14.368 INFO:tasks.workunit.client.1.smithi152.stdout:0/437: rename d2/da/d17/d50/c96 to d2/d31/ca9 0 2024-03-29T04:45:14.368 INFO:tasks.workunit.client.0.smithi142.stdout:1/871: write d8/db/dc/d77/db9/dbe/d2d/d8b/da4/fba [5221530,97955] 0 2024-03-29T04:45:14.369 INFO:tasks.workunit.client.1.smithi152.stdout:4/294: chown d0/d7/d40/d2a/f4a 5204908 1 2024-03-29T04:45:14.370 INFO:tasks.workunit.client.1.smithi152.stdout:3/308: dread - d7/d19/d1b/d1e/f45 zero size 2024-03-29T04:45:14.370 INFO:tasks.workunit.client.0.smithi142.stdout:4/907: chown d0/d9/d11/d4c/d5f/ffd 9 1 2024-03-29T04:45:14.370 INFO:tasks.workunit.client.1.smithi152.stdout:2/287: write d0/d3/df/f13 [228567,80394] 0 2024-03-29T04:45:14.371 INFO:tasks.workunit.client.0.smithi142.stdout:4/908: readlink d0/d9/d19/l2f 0 2024-03-29T04:45:14.371 INFO:tasks.workunit.client.1.smithi152.stdout:9/388: creat d5/db/d1b/f8b x:0 0 0 2024-03-29T04:45:14.372 INFO:tasks.workunit.client.1.smithi152.stdout:8/311: creat d1/d2/d1d/d36/d5d/f6f x:0 0 0 2024-03-29T04:45:14.373 INFO:tasks.workunit.client.0.smithi142.stdout:1/872: creat d8/db/dc/d77/db9/dbe/d2d/d8b/f124 x:0 0 0 2024-03-29T04:45:14.374 INFO:tasks.workunit.client.1.smithi152.stdout:9/389: chown d5/d8/d31/d75 872 1 2024-03-29T04:45:14.376 INFO:tasks.workunit.client.0.smithi142.stdout:4/909: symlink d0/d9/d11/d97/dff/l12e 0 2024-03-29T04:45:14.376 INFO:tasks.workunit.client.1.smithi152.stdout:7/372: creat d7/d8/d12/d35/f6d x:0 0 0 2024-03-29T04:45:14.376 INFO:tasks.workunit.client.1.smithi152.stdout:3/309: creat d7/d1c/f51 x:0 0 0 2024-03-29T04:45:14.377 INFO:tasks.workunit.client.1.smithi152.stdout:1/479: rename d5/d7/d9/d2b/l82 to d5/d7/d9/d59/l92 0 2024-03-29T04:45:14.377 INFO:tasks.workunit.client.0.smithi142.stdout:4/910: write d0/dc/d15/d91/ffc [5510617,77460] 0 2024-03-29T04:45:14.379 INFO:tasks.workunit.client.0.smithi142.stdout:1/873: truncate d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/fd8 264429 0 2024-03-29T04:45:14.379 INFO:tasks.workunit.client.1.smithi152.stdout:2/288: mknod d0/d3/d30/d15/c62 0 2024-03-29T04:45:14.380 INFO:tasks.workunit.client.0.smithi142.stdout:1/874: chown d8/db/dc/d77/db9/dbe/d2d/d44 523994 1 2024-03-29T04:45:14.383 INFO:tasks.workunit.client.1.smithi152.stdout:1/480: dread d5/d7/d9/d2b/f2f [0,4194304] 0 2024-03-29T04:45:14.383 INFO:tasks.workunit.client.1.smithi152.stdout:9/390: mknod d5/d8/d31/c8c 0 2024-03-29T04:45:14.384 INFO:tasks.workunit.client.0.smithi142.stdout:1/875: creat d8/db/f125 x:0 0 0 2024-03-29T04:45:14.386 INFO:tasks.workunit.client.1.smithi152.stdout:3/310: mknod d7/d19/d1b/c52 0 2024-03-29T04:45:14.386 INFO:tasks.workunit.client.0.smithi142.stdout:1/876: read d8/ff7 [2277611,6997] 0 2024-03-29T04:45:14.387 INFO:tasks.workunit.client.1.smithi152.stdout:4/295: rename d0/ca to d0/d7/d25/d17/d37/c5f 0 2024-03-29T04:45:14.388 INFO:tasks.workunit.client.0.smithi142.stdout:9/892: sync 2024-03-29T04:45:14.388 INFO:tasks.workunit.client.0.smithi142.stdout:3/757: sync 2024-03-29T04:45:14.389 INFO:tasks.workunit.client.0.smithi142.stdout:6/925: sync 2024-03-29T04:45:14.389 INFO:tasks.workunit.client.1.smithi152.stdout:8/312: getdents d1/d21/d40 0 2024-03-29T04:45:14.392 INFO:tasks.workunit.client.1.smithi152.stdout:6/448: sync 2024-03-29T04:45:14.392 INFO:tasks.workunit.client.1.smithi152.stdout:5/386: sync 2024-03-29T04:45:14.392 INFO:tasks.workunit.client.1.smithi152.stdout:3/311: mkdir d7/d53 0 2024-03-29T04:45:14.396 INFO:tasks.workunit.client.1.smithi152.stdout:2/289: dwrite d0/d3/d30/d15/f25 [0,4194304] 0 2024-03-29T04:45:14.396 INFO:tasks.workunit.client.0.smithi142.stdout:3/758: creat d3/d29/d2b/d40/d67/d8b/d6/f100 x:0 0 0 2024-03-29T04:45:14.396 INFO:tasks.workunit.client.0.smithi142.stdout:6/926: dwrite d1c/d33/d46/d86/dc0/ff3 [0,4194304] 0 2024-03-29T04:45:14.399 INFO:tasks.workunit.client.1.smithi152.stdout:8/313: dread - d1/d21/d40/d6a/f62 zero size 2024-03-29T04:45:14.402 INFO:tasks.workunit.client.0.smithi142.stdout:6/927: rmdir d1c/d33/d46/d86/dc0/d113 39 2024-03-29T04:45:14.403 INFO:tasks.workunit.client.1.smithi152.stdout:9/391: dwrite d5/f61 [0,4194304] 0 2024-03-29T04:45:14.404 INFO:tasks.workunit.client.0.smithi142.stdout:6/928: dread - d1c/d33/d46/d61/d7e/f108 zero size 2024-03-29T04:45:14.405 INFO:tasks.workunit.client.1.smithi152.stdout:4/296: read d0/d7/d25/f21 [1908406,29870] 0 2024-03-29T04:45:14.408 INFO:tasks.workunit.client.0.smithi142.stdout:1/877: write d8/db/dc/d77/db9/dbe/d15/f27 [612342,6433] 0 2024-03-29T04:45:14.412 INFO:tasks.workunit.client.0.smithi142.stdout:3/759: creat d3/d29/d8f/d5f/dc5/f101 x:0 0 0 2024-03-29T04:45:14.412 INFO:tasks.workunit.client.1.smithi152.stdout:9/392: mkdir d5/d12/d2c/d6e/d73/d8d 0 2024-03-29T04:45:14.414 INFO:tasks.workunit.client.0.smithi142.stdout:2/908: dwrite de/d11/d17/d10b/d22/d27/dba/d61/f8f [0,4194304] 0 2024-03-29T04:45:14.417 INFO:tasks.workunit.client.0.smithi142.stdout:3/760: readlink d3/d1f/d24/d47/l92 0 2024-03-29T04:45:14.417 INFO:tasks.workunit.client.0.smithi142.stdout:9/893: truncate d0/d2/d61/d64/dee/d10d/dc8/fe2 1141624 0 2024-03-29T04:45:14.417 INFO:tasks.workunit.client.0.smithi142.stdout:1/878: chown d8/db/dc/d77/db9/dbe/d2d/d44/d4b/c11f 15808 1 2024-03-29T04:45:14.418 INFO:tasks.workunit.client.1.smithi152.stdout:5/387: dwrite d6/d16/d25/d2e/f54 [0,4194304] 0 2024-03-29T04:45:14.419 INFO:tasks.workunit.client.0.smithi142.stdout:9/894: write d0/f1d [1740934,95163] 0 2024-03-29T04:45:14.419 INFO:tasks.workunit.client.0.smithi142.stdout:4/911: dwrite d0/dc/d15/d3d/d65/f90 [0,4194304] 0 2024-03-29T04:45:14.421 INFO:tasks.workunit.client.0.smithi142.stdout:6/929: getdents d1c/dc7/df2/dff 0 2024-03-29T04:45:14.421 INFO:tasks.workunit.client.1.smithi152.stdout:5/388: chown d6/d16/l7b 35096 1 2024-03-29T04:45:14.424 INFO:tasks.workunit.client.1.smithi152.stdout:0/438: dwrite d2/f20 [0,4194304] 0 2024-03-29T04:45:14.426 INFO:tasks.workunit.client.1.smithi152.stdout:5/389: chown d6/d1b/f78 508 1 2024-03-29T04:45:14.427 INFO:tasks.workunit.client.0.smithi142.stdout:6/930: mknod d1c/c12b 0 2024-03-29T04:45:14.429 INFO:tasks.workunit.client.0.smithi142.stdout:9/895: creat d0/d2/d61/d64/d104/f11c x:0 0 0 2024-03-29T04:45:14.429 INFO:tasks.workunit.client.1.smithi152.stdout:5/390: fsync d6/d1b/f45 0 2024-03-29T04:45:14.433 INFO:tasks.workunit.client.0.smithi142.stdout:4/912: getdents d0/d9/d11/d4c/d5f 0 2024-03-29T04:45:14.434 INFO:tasks.workunit.client.0.smithi142.stdout:4/913: read d0/d9/d11/d4c/d5f/d81/dc9/d47/f4b [2347656,125375] 0 2024-03-29T04:45:14.436 INFO:tasks.workunit.client.0.smithi142.stdout:4/914: write d0/d9/db2/dba/dbc/d8d/da5/d102/f114 [261047,47625] 0 2024-03-29T04:45:14.436 INFO:tasks.workunit.client.0.smithi142.stdout:3/761: dread d3/d29/d2b/d6a/f7b [0,4194304] 0 2024-03-29T04:45:14.438 INFO:tasks.workunit.client.0.smithi142.stdout:6/931: link d1c/f21 d1c/dc7/df2/dff/f12c 0 2024-03-29T04:45:14.439 INFO:tasks.workunit.client.1.smithi152.stdout:0/439: mknod d2/da/d17/d50/caa 0 2024-03-29T04:45:14.440 INFO:tasks.workunit.client.1.smithi152.stdout:7/373: dwrite d7/d8/d12/d35/f2a [0,4194304] 0 2024-03-29T04:45:14.441 INFO:tasks.workunit.client.0.smithi142.stdout:9/896: link d0/d12/d10e/d3c/f4c d0/d12/d10e/d3c/d4f/f11d 0 2024-03-29T04:45:14.441 INFO:tasks.workunit.client.0.smithi142.stdout:9/897: chown d0/l42 7519937 1 2024-03-29T04:45:14.445 INFO:tasks.workunit.client.0.smithi142.stdout:3/762: mknod d3/db9/c102 0 2024-03-29T04:45:14.445 INFO:tasks.workunit.client.0.smithi142.stdout:4/915: symlink d0/d9/db2/dba/dbc/d6d/l12f 0 2024-03-29T04:45:14.446 INFO:tasks.workunit.client.0.smithi142.stdout:4/916: chown d0/d9/d11/d97/fe1 73458 1 2024-03-29T04:45:14.447 INFO:tasks.workunit.client.0.smithi142.stdout:6/932: rename d1c/d33/db6 to d1c/dc7/d12d 0 2024-03-29T04:45:14.448 INFO:tasks.workunit.client.1.smithi152.stdout:5/391: dread d6/d16/d25/d2e/f54 [0,4194304] 0 2024-03-29T04:45:14.449 INFO:tasks.workunit.client.0.smithi142.stdout:3/763: rmdir d3/d29/d2b/d40/d67/d8b 39 2024-03-29T04:45:14.449 INFO:tasks.workunit.client.1.smithi152.stdout:9/393: dread d5/d8/f9 [0,4194304] 0 2024-03-29T04:45:14.451 INFO:tasks.workunit.client.1.smithi152.stdout:1/481: dread d5/d7/fe [0,4194304] 0 2024-03-29T04:45:14.451 INFO:tasks.workunit.client.1.smithi152.stdout:0/440: write d2/da/d17/d57/d9c/f9f [210294,75585] 0 2024-03-29T04:45:14.451 INFO:tasks.workunit.client.0.smithi142.stdout:4/917: mknod d0/d9/d11/d4c/d5f/d81/dc9/d126/c130 0 2024-03-29T04:45:14.452 INFO:tasks.workunit.client.1.smithi152.stdout:6/449: dread d1/d3/d1a/f1d [0,4194304] 0 2024-03-29T04:45:14.454 INFO:tasks.workunit.client.1.smithi152.stdout:1/482: write d5/d7/d9/f73 [165175,105674] 0 2024-03-29T04:45:14.456 INFO:tasks.workunit.client.0.smithi142.stdout:6/933: creat d1c/d12a/f12e x:0 0 0 2024-03-29T04:45:14.457 INFO:tasks.workunit.client.1.smithi152.stdout:6/450: unlink d1/d3/d9/d13/d3e/c8e 0 2024-03-29T04:45:14.458 INFO:tasks.workunit.client.0.smithi142.stdout:9/898: getdents d0/d12/d10e 0 2024-03-29T04:45:14.459 INFO:tasks.workunit.client.1.smithi152.stdout:9/394: rename d5/d70 to d5/d12/d2c/d6e/d73/d8e 0 2024-03-29T04:45:14.459 INFO:tasks.workunit.client.0.smithi142.stdout:9/899: fdatasync d0/d12/d50/d52/da3/d10a/f8f 0 2024-03-29T04:45:14.460 INFO:tasks.workunit.client.0.smithi142.stdout:4/918: getdents d0/d9/db2/dc6 0 2024-03-29T04:45:14.460 INFO:tasks.workunit.client.1.smithi152.stdout:1/483: rename d5/d7/d1b/f78 to d5/d7/d1b/f93 0 2024-03-29T04:45:14.461 INFO:tasks.workunit.client.0.smithi142.stdout:6/934: dread d1c/f41 [0,4194304] 0 2024-03-29T04:45:14.462 INFO:tasks.workunit.client.0.smithi142.stdout:6/935: chown f4 89123 1 2024-03-29T04:45:14.463 INFO:tasks.workunit.client.1.smithi152.stdout:6/451: unlink d1/d3/df/c22 0 2024-03-29T04:45:14.464 INFO:tasks.workunit.client.0.smithi142.stdout:9/900: getdents d0/d2/d61/d64/dee 0 2024-03-29T04:45:14.465 INFO:tasks.workunit.client.0.smithi142.stdout:6/936: chown d1c/d45/d8a/f10c 59516988 1 2024-03-29T04:45:14.466 INFO:tasks.workunit.client.1.smithi152.stdout:1/484: write d5/d7/d8/fb [2014198,100561] 0 2024-03-29T04:45:14.468 INFO:tasks.workunit.client.0.smithi142.stdout:4/919: dwrite d0/d9/db2/dc6/f109 [0,4194304] 0 2024-03-29T04:45:14.472 INFO:tasks.workunit.client.1.smithi152.stdout:9/395: getdents d5/d8/d31/d59/d85 0 2024-03-29T04:45:14.473 INFO:tasks.workunit.client.0.smithi142.stdout:2/909: write de/d11/d17/d10b/d22/dbf/f114 [4432214,80989] 0 2024-03-29T04:45:14.476 INFO:tasks.workunit.client.1.smithi152.stdout:2/290: dwrite d0/d3/df/f35 [0,4194304] 0 2024-03-29T04:45:14.476 INFO:tasks.workunit.client.1.smithi152.stdout:6/452: mkdir d1/d3/d1a/d1b/d29/d6f/d95 0 2024-03-29T04:45:14.478 INFO:tasks.workunit.client.1.smithi152.stdout:8/314: truncate d1/d2/d1d/f35 2298358 0 2024-03-29T04:45:14.479 INFO:tasks.workunit.client.0.smithi142.stdout:4/920: dwrite d0/d12/f3b [0,4194304] 0 2024-03-29T04:45:14.479 INFO:tasks.workunit.client.0.smithi142.stdout:2/910: rmdir de/d11/d17/d5d 39 2024-03-29T04:45:14.479 INFO:tasks.workunit.client.1.smithi152.stdout:5/392: getdents d6/d1d/d3b 0 2024-03-29T04:45:14.480 INFO:tasks.workunit.client.1.smithi152.stdout:8/315: fdatasync d1/d2/d1d/d36/d5d/f6f 0 2024-03-29T04:45:14.484 INFO:tasks.workunit.client.1.smithi152.stdout:6/453: fsync d1/d3/df/f48 0 2024-03-29T04:45:14.485 INFO:tasks.workunit.client.0.smithi142.stdout:1/879: dwrite d8/db/dc/d77/db9/dbe/d1f/d97/d9e/fa6 [0,4194304] 0 2024-03-29T04:45:14.485 INFO:tasks.workunit.client.0.smithi142.stdout:2/911: rmdir de/d11/d17/d10b/d22/dbf/d11c 39 2024-03-29T04:45:14.487 INFO:tasks.workunit.client.0.smithi142.stdout:1/880: chown d8/db/dc/d77/db9/dbe/d1f/f6d 903 1 2024-03-29T04:45:14.490 INFO:tasks.workunit.client.0.smithi142.stdout:1/881: chown d8/db/dc/d77/db9/dbe/d15/fa0 3052179 1 2024-03-29T04:45:14.492 INFO:tasks.workunit.client.1.smithi152.stdout:5/393: dwrite d6/d1b/f31 [0,4194304] 0 2024-03-29T04:45:14.493 INFO:tasks.workunit.client.0.smithi142.stdout:6/937: write d1c/d45/fd9 [373055,47002] 0 2024-03-29T04:45:14.496 INFO:tasks.workunit.client.0.smithi142.stdout:2/912: dwrite de/d11/d17/d10b/d84/f98 [0,4194304] 0 2024-03-29T04:45:14.497 INFO:tasks.workunit.client.1.smithi152.stdout:1/485: getdents d5 0 2024-03-29T04:45:14.500 INFO:tasks.workunit.client.1.smithi152.stdout:8/316: rename d1/d2/c12 to d1/d2/d1d/d36/d5d/c70 0 2024-03-29T04:45:14.503 INFO:tasks.workunit.client.1.smithi152.stdout:0/441: dwrite d2/da/d17/d57/f8c [0,4194304] 0 2024-03-29T04:45:14.503 INFO:tasks.workunit.client.1.smithi152.stdout:2/291: link d0/d3/d30/d15/d22/d4c/f51 d0/d3/d5f/f63 0 2024-03-29T04:45:14.504 INFO:tasks.workunit.client.0.smithi142.stdout:3/764: dwrite d3/d1f/fc4 [0,4194304] 0 2024-03-29T04:45:14.508 INFO:tasks.workunit.client.0.smithi142.stdout:9/901: dwrite d0/f4b [0,4194304] 0 2024-03-29T04:45:14.508 INFO:tasks.workunit.client.1.smithi152.stdout:6/454: unlink d1/d3/f4e 0 2024-03-29T04:45:14.508 INFO:tasks.workunit.client.1.smithi152.stdout:0/442: write d2/da/d17/d57/f71 [2252759,40971] 0 2024-03-29T04:45:14.508 INFO:tasks.workunit.client.1.smithi152.stdout:5/394: rmdir d6/d16/d5c 39 2024-03-29T04:45:14.509 INFO:tasks.workunit.client.0.smithi142.stdout:3/765: truncate d3/d1f/d24/fa5 848353 0 2024-03-29T04:45:14.510 INFO:tasks.workunit.client.1.smithi152.stdout:0/443: chown d2/f38 0 1 2024-03-29T04:45:14.513 INFO:tasks.workunit.client.1.smithi152.stdout:8/317: dwrite d1/f26 [0,4194304] 0 2024-03-29T04:45:14.515 INFO:tasks.workunit.client.0.smithi142.stdout:3/766: truncate d3/d29/d2b/d40/fea 295410 0 2024-03-29T04:45:14.515 INFO:tasks.workunit.client.0.smithi142.stdout:2/913: chown de/d11/d17/d5d/d71/ca2 345 1 2024-03-29T04:45:14.520 INFO:tasks.workunit.client.0.smithi142.stdout:9/902: mkdir d0/d11e 0 2024-03-29T04:45:14.522 INFO:tasks.workunit.client.1.smithi152.stdout:2/292: mkdir d0/d3/d64 0 2024-03-29T04:45:14.523 INFO:tasks.workunit.client.0.smithi142.stdout:2/914: getdents de/d11/d17/d60/dc4 0 2024-03-29T04:45:14.525 INFO:tasks.workunit.client.0.smithi142.stdout:3/767: dwrite d3/fd0 [0,4194304] 0 2024-03-29T04:45:14.526 INFO:tasks.workunit.client.1.smithi152.stdout:0/444: dwrite d2/da/d3a/d6f/f85 [0,4194304] 0 2024-03-29T04:45:14.529 INFO:tasks.workunit.client.1.smithi152.stdout:0/445: chown d2/d29/c69 38 1 2024-03-29T04:45:14.529 INFO:tasks.workunit.client.1.smithi152.stdout:8/318: link d1/d21/f25 d1/d2/d1d/d36/d5d/f71 0 2024-03-29T04:45:14.533 INFO:tasks.workunit.client.0.smithi142.stdout:9/903: dwrite d0/d12/d10e/d3c/d6f/fec [0,4194304] 0 2024-03-29T04:45:14.534 INFO:tasks.workunit.client.0.smithi142.stdout:2/915: link de/d11/d3d/fbc de/d11/d17/d10b/d22/d27/dba/d61/dcd/d129/f12d 0 2024-03-29T04:45:14.535 INFO:tasks.workunit.client.1.smithi152.stdout:0/446: mknod d2/da/d3a/d61/cab 0 2024-03-29T04:45:14.536 INFO:tasks.workunit.client.0.smithi142.stdout:3/768: creat d3/d29/d8f/d5f/f103 x:0 0 0 2024-03-29T04:45:14.540 INFO:tasks.workunit.client.0.smithi142.stdout:4/921: dwrite d0/d9/d11/fde [0,4194304] 0 2024-03-29T04:45:14.541 INFO:tasks.workunit.client.1.smithi152.stdout:0/447: truncate d2/f78 167861 0 2024-03-29T04:45:14.541 INFO:tasks.workunit.client.0.smithi142.stdout:4/922: write d0/d9/db2/dba/dbc/d8d/da5/d102/f114 [111916,16216] 0 2024-03-29T04:45:14.543 INFO:tasks.workunit.client.0.smithi142.stdout:2/916: mkdir de/d11/d2b/d3e/d12e 0 2024-03-29T04:45:14.543 INFO:tasks.workunit.client.1.smithi152.stdout:0/448: fdatasync d2/da/df/d86/f9e 0 2024-03-29T04:45:14.543 INFO:tasks.workunit.client.1.smithi152.stdout:8/319: dwrite d1/d2/d1d/f30 [0,4194304] 0 2024-03-29T04:45:14.546 INFO:tasks.workunit.client.0.smithi142.stdout:3/769: rename d3/d29/d2b/d40/db1/fcf to d3/d29/d8f/d5f/dc5/f104 0 2024-03-29T04:45:14.547 INFO:tasks.workunit.client.0.smithi142.stdout:3/770: chown d3/d29/d8f/d5f/d64/l88 5735 1 2024-03-29T04:45:14.548 INFO:tasks.workunit.client.1.smithi152.stdout:5/395: getdents d6/d16/d5c 0 2024-03-29T04:45:14.549 INFO:tasks.workunit.client.0.smithi142.stdout:2/917: unlink de/d11/d17/d10b/d22/d27/dba/d61/fa0 0 2024-03-29T04:45:14.549 INFO:tasks.workunit.client.0.smithi142.stdout:9/904: creat d0/d12/f11f x:0 0 0 2024-03-29T04:45:14.550 INFO:tasks.workunit.client.1.smithi152.stdout:8/320: dread - d1/d2/d1d/d36/d4b/f68 zero size 2024-03-29T04:45:14.551 INFO:tasks.workunit.client.0.smithi142.stdout:4/923: mknod d0/d9/d11/d4c/d5f/d81/dc9/d47/d7a/de4/c131 0 2024-03-29T04:45:14.554 INFO:tasks.workunit.client.1.smithi152.stdout:8/321: write d1/d21/f46 [2582585,108887] 0 2024-03-29T04:45:14.554 INFO:tasks.workunit.client.0.smithi142.stdout:9/905: symlink d0/d2/d61/df7/l120 0 2024-03-29T04:45:14.555 INFO:tasks.workunit.client.1.smithi152.stdout:0/449: mkdir d2/da/d17/d50/dac 0 2024-03-29T04:45:14.555 INFO:tasks.workunit.client.0.smithi142.stdout:2/918: fsync de/d11/d17/d5d/f74 0 2024-03-29T04:45:14.556 INFO:tasks.workunit.client.0.smithi142.stdout:3/771: creat d3/d1f/de6/df1/f105 x:0 0 0 2024-03-29T04:45:14.558 INFO:tasks.workunit.client.0.smithi142.stdout:2/919: symlink de/d11/d3d/l12f 0 2024-03-29T04:45:14.560 INFO:tasks.workunit.client.1.smithi152.stdout:0/450: creat d2/da/d17/d57/d9c/fad x:0 0 0 2024-03-29T04:45:14.564 INFO:tasks.workunit.client.0.smithi142.stdout:4/924: rename d0/dc/d15/d91/ffc to d0/d9/db2/dba/dbc/f132 0 2024-03-29T04:45:14.565 INFO:tasks.workunit.client.1.smithi152.stdout:0/451: stat d2/da/d17/d50/d5c/f7b 0 2024-03-29T04:45:14.565 INFO:tasks.workunit.client.1.smithi152.stdout:5/396: dwrite d6/d16/f5b [0,4194304] 0 2024-03-29T04:45:14.566 INFO:tasks.workunit.client.0.smithi142.stdout:3/772: symlink d3/d29/d2b/d40/d67/d8b/df7/l106 0 2024-03-29T04:45:14.568 INFO:tasks.workunit.client.1.smithi152.stdout:8/322: dwrite d1/d2/d3/f29 [0,4194304] 0 2024-03-29T04:45:14.569 INFO:tasks.workunit.client.1.smithi152.stdout:0/452: stat d2/da/c28 0 2024-03-29T04:45:14.570 INFO:tasks.workunit.client.1.smithi152.stdout:8/323: dread - d1/d2/d1d/f3a zero size 2024-03-29T04:45:14.570 INFO:tasks.workunit.client.0.smithi142.stdout:3/773: mkdir d3/d29/d2b/d40/d67/d107 0 2024-03-29T04:45:14.571 INFO:tasks.workunit.client.1.smithi152.stdout:5/397: rename d6/d16/d5c to d6/d16/d25/d7e 0 2024-03-29T04:45:14.572 INFO:tasks.workunit.client.1.smithi152.stdout:0/453: symlink d2/da/d3a/d6f/lae 0 2024-03-29T04:45:14.572 INFO:tasks.workunit.client.1.smithi152.stdout:0/454: dread - d2/da/df/f4a zero size 2024-03-29T04:45:14.574 INFO:tasks.workunit.client.0.smithi142.stdout:9/906: rename d0/d2/d61/d64/dee/d10d/d83/ld1 to d0/d12/d10e/dc7/l121 0 2024-03-29T04:45:14.574 INFO:tasks.workunit.client.1.smithi152.stdout:0/455: write d2/da/df/f4b [2445491,56406] 0 2024-03-29T04:45:14.575 INFO:tasks.workunit.client.0.smithi142.stdout:9/907: stat d0/d12/d50/f76 0 2024-03-29T04:45:14.576 INFO:tasks.workunit.client.0.smithi142.stdout:3/774: dread d3/fd0 [0,4194304] 0 2024-03-29T04:45:14.577 INFO:tasks.workunit.client.1.smithi152.stdout:8/324: rename d1/f26 to d1/d21/d40/d47/f72 0 2024-03-29T04:45:14.578 INFO:tasks.workunit.client.1.smithi152.stdout:8/325: stat d1/d2/d3c/f69 0 2024-03-29T04:45:14.578 INFO:tasks.workunit.client.1.smithi152.stdout:1/486: dread d5/d7/d6a/f69 [0,4194304] 0 2024-03-29T04:45:14.579 INFO:tasks.workunit.client.1.smithi152.stdout:5/398: mknod d6/d16/d25/c7f 0 2024-03-29T04:45:14.580 INFO:tasks.workunit.client.1.smithi152.stdout:8/326: read - d1/d2/d1d/d36/d4b/f68 zero size 2024-03-29T04:45:14.582 INFO:tasks.workunit.client.1.smithi152.stdout:9/396: truncate d5/db/f6c 3981787 0 2024-03-29T04:45:14.584 INFO:tasks.workunit.client.1.smithi152.stdout:0/456: mkdir d2/da/d17/d50/d5c/daf 0 2024-03-29T04:45:14.585 INFO:tasks.workunit.client.1.smithi152.stdout:0/457: write d2/f8 [3859158,41574] 0 2024-03-29T04:45:14.586 INFO:tasks.workunit.client.0.smithi142.stdout:2/920: rename de/d11/dc7/ffd to de/f130 0 2024-03-29T04:45:14.587 INFO:tasks.workunit.client.1.smithi152.stdout:5/399: symlink d6/d1d/d30/d6a/l80 0 2024-03-29T04:45:14.589 INFO:tasks.workunit.client.1.smithi152.stdout:8/327: write d1/d21/f25 [729762,20437] 0 2024-03-29T04:45:14.589 INFO:tasks.workunit.client.0.smithi142.stdout:2/921: fdatasync de/d11/d17/d10b/d22/d50/fdb 0 2024-03-29T04:45:14.590 INFO:tasks.workunit.client.0.smithi142.stdout:4/925: rename d0/d9/d11/d4c/d5f/d81/dc9/d126/f12d to d0/d9/d11/d4c/d5f/d81/dc9/d47/d7a/de4/f133 0 2024-03-29T04:45:14.590 INFO:tasks.workunit.client.0.smithi142.stdout:2/922: fsync de/d11/d2b/d3e/fcf 0 2024-03-29T04:45:14.591 INFO:tasks.workunit.client.0.smithi142.stdout:4/926: chown d0/d12/f3c 13 1 2024-03-29T04:45:14.593 INFO:tasks.workunit.client.1.smithi152.stdout:5/400: dread d6/d16/d25/d7e/f69 [0,4194304] 0 2024-03-29T04:45:14.593 INFO:tasks.workunit.client.0.smithi142.stdout:3/775: rename d3/d29/d2b/d40/d67/d8b/c2c to d3/d29/d8f/d5f/d64/d6b/c108 0 2024-03-29T04:45:14.596 INFO:tasks.workunit.client.0.smithi142.stdout:4/927: rename d0/d9/d19/d2c/f10e to d0/d9/d11/d4c/d5f/d81/dc9/d126/f134 0 2024-03-29T04:45:14.597 INFO:tasks.workunit.client.0.smithi142.stdout:2/923: fdatasync de/d11/d17/d5d/f86 0 2024-03-29T04:45:14.598 INFO:tasks.workunit.client.1.smithi152.stdout:5/401: mknod d6/d16/d47/d56/c81 0 2024-03-29T04:45:14.599 INFO:tasks.workunit.client.1.smithi152.stdout:1/487: getdents d5/d7/d8/d13/d19/d80 0 2024-03-29T04:45:14.600 INFO:tasks.workunit.client.0.smithi142.stdout:9/908: dread d0/d12/d50/fd5 [0,4194304] 0 2024-03-29T04:45:14.601 INFO:tasks.workunit.client.1.smithi152.stdout:1/488: chown d5/c15 4895 1 2024-03-29T04:45:14.601 INFO:tasks.workunit.client.1.smithi152.stdout:1/489: truncate d5/f72 510071 0 2024-03-29T04:45:14.602 INFO:tasks.workunit.client.1.smithi152.stdout:1/490: write d5/d7/d8/d13/f35 [7063388,52763] 0 2024-03-29T04:45:14.603 INFO:tasks.workunit.client.1.smithi152.stdout:5/402: rename d6/d1d/d30/l3c to d6/d16/d25/d2e/l82 0 2024-03-29T04:45:14.604 INFO:tasks.workunit.client.0.smithi142.stdout:3/776: dwrite d3/d29/d2b/d40/d67/fe0 [0,4194304] 0 2024-03-29T04:45:14.604 INFO:tasks.workunit.client.0.smithi142.stdout:9/909: chown d0/d6c/fa1 6614145 1 2024-03-29T04:45:14.606 INFO:tasks.workunit.client.0.smithi142.stdout:2/924: truncate de/d11/d17/d10b/d22/d27/dba/f6b 650054 0 2024-03-29T04:45:14.608 INFO:tasks.workunit.client.0.smithi142.stdout:4/928: dwrite d0/d9/d11/d4c/d5f/fd0 [0,4194304] 0 2024-03-29T04:45:14.608 INFO:tasks.workunit.client.0.smithi142.stdout:9/910: creat d0/d12/d50/d52/da3/f122 x:0 0 0 2024-03-29T04:45:14.610 INFO:tasks.workunit.client.0.smithi142.stdout:3/777: mknod d3/d29/d8f/d5f/d64/d6b/c109 0 2024-03-29T04:45:14.613 INFO:tasks.workunit.client.0.smithi142.stdout:2/925: rename de/d11/d17/d10b/d22/d27/dba/d61/f8f to de/d11/d17/d10b/d22/f131 0 2024-03-29T04:45:14.620 INFO:tasks.workunit.client.0.smithi142.stdout:2/926: dwrite de/d11/d2b/d3e/f76 [0,4194304] 0 2024-03-29T04:45:14.624 INFO:tasks.workunit.client.0.smithi142.stdout:9/911: mkdir d0/d12/d10e/dc7/daa/d123 0 2024-03-29T04:45:14.624 INFO:tasks.workunit.client.1.smithi152.stdout:5/403: symlink d6/d1b/l83 0 2024-03-29T04:45:14.625 INFO:tasks.workunit.client.0.smithi142.stdout:9/912: readlink d0/d6c/la9 0 2024-03-29T04:45:14.626 INFO:tasks.workunit.client.0.smithi142.stdout:6/938: write d1c/d33/d46/d71/f68 [11309,15139] 0 2024-03-29T04:45:14.627 INFO:tasks.workunit.client.1.smithi152.stdout:6/455: dread d1/d3/df/f66 [0,4194304] 0 2024-03-29T04:45:14.629 INFO:tasks.workunit.client.0.smithi142.stdout:6/939: fsync d1c/d45/d51/f97 0 2024-03-29T04:45:14.629 INFO:tasks.workunit.client.0.smithi142.stdout:9/913: chown d0/d12/d10e/d3c/d6f/ldc 94997 1 2024-03-29T04:45:14.629 INFO:tasks.workunit.client.0.smithi142.stdout:2/927: dwrite de/d11/d17/d5d/fe5 [0,4194304] 0 2024-03-29T04:45:14.630 INFO:tasks.workunit.client.1.smithi152.stdout:1/491: link d5/d7/fe d5/d7/d8/d13/d19/f94 0 2024-03-29T04:45:14.632 INFO:tasks.workunit.client.0.smithi142.stdout:2/928: write de/d11/d2b/d3e/f76 [1642511,9768] 0 2024-03-29T04:45:14.634 INFO:tasks.workunit.client.1.smithi152.stdout:1/492: mkdir d5/d7/d1b/d2e/d95 0 2024-03-29T04:45:14.635 INFO:tasks.workunit.client.0.smithi142.stdout:9/914: creat d0/d6c/f124 x:0 0 0 2024-03-29T04:45:14.635 INFO:tasks.workunit.client.0.smithi142.stdout:4/929: dread d0/dc/f93 [0,4194304] 0 2024-03-29T04:45:14.636 INFO:tasks.workunit.client.0.smithi142.stdout:2/929: fsync de/d11/d17/d10b/d22/dbf/f114 0 2024-03-29T04:45:14.639 INFO:tasks.workunit.client.0.smithi142.stdout:6/940: dwrite d1c/dc7/df2/f11b [0,4194304] 0 2024-03-29T04:45:14.639 INFO:tasks.workunit.client.1.smithi152.stdout:1/493: unlink d5/d7/d8/d13/f62 0 2024-03-29T04:45:14.641 INFO:tasks.workunit.client.1.smithi152.stdout:2/293: dwrite d0/f16 [0,4194304] 0 2024-03-29T04:45:14.642 INFO:tasks.workunit.client.0.smithi142.stdout:4/930: dread d0/d9/d11/f38 [0,4194304] 0 2024-03-29T04:45:14.643 INFO:tasks.workunit.client.0.smithi142.stdout:9/915: creat d0/f125 x:0 0 0 2024-03-29T04:45:14.645 INFO:tasks.workunit.client.0.smithi142.stdout:4/931: read d0/d9/db2/dba/fe6 [4927423,52389] 0 2024-03-29T04:45:14.645 INFO:tasks.workunit.client.1.smithi152.stdout:6/456: creat d1/d3/f96 x:0 0 0 2024-03-29T04:45:14.649 INFO:tasks.workunit.client.0.smithi142.stdout:6/941: unlink d1c/dc7/d12d/l4b 0 2024-03-29T04:45:14.651 INFO:tasks.workunit.client.1.smithi152.stdout:1/494: write d5/d7/d6a/f69 [1136363,59699] 0 2024-03-29T04:45:14.653 INFO:tasks.workunit.client.1.smithi152.stdout:8/328: dread d1/d21/f25 [0,4194304] 0 2024-03-29T04:45:14.655 INFO:tasks.workunit.client.0.smithi142.stdout:2/930: rename de/d11/d17/d10b/d22/d27/f10f to de/d11/d17/d10b/d22/d27/dba/d61/f132 0 2024-03-29T04:45:14.658 INFO:tasks.workunit.client.0.smithi142.stdout:9/916: symlink d0/d2/d39/dd2/l126 0 2024-03-29T04:45:14.659 INFO:tasks.workunit.client.1.smithi152.stdout:2/294: mknod d0/d3/d30/d15/d22/d4c/d52/c65 0 2024-03-29T04:45:14.660 INFO:tasks.workunit.client.1.smithi152.stdout:6/457: mknod d1/d3/d9/d13/d3e/d93/c97 0 2024-03-29T04:45:14.661 INFO:tasks.workunit.client.1.smithi152.stdout:1/495: mkdir d5/d7/d8/d13/d96 0 2024-03-29T04:45:14.662 INFO:tasks.workunit.client.0.smithi142.stdout:9/917: creat d0/d12/d10e/d3c/f127 x:0 0 0 2024-03-29T04:45:14.662 INFO:tasks.workunit.client.1.smithi152.stdout:1/496: chown d5/d7/d8/d13/d19/f94 213 1 2024-03-29T04:45:14.663 INFO:tasks.workunit.client.1.smithi152.stdout:8/329: unlink d1/d21/f32 0 2024-03-29T04:45:14.665 INFO:tasks.workunit.client.1.smithi152.stdout:8/330: chown d1/d2/l2e 34 1 2024-03-29T04:45:14.666 INFO:tasks.workunit.client.1.smithi152.stdout:2/295: rename d0/d3/l32 to d0/d3/d30/d15/d53/l66 0 2024-03-29T04:45:14.669 INFO:tasks.workunit.client.0.smithi142.stdout:9/918: getdents d0/d12/d10e/dc7/daa 0 2024-03-29T04:45:14.670 INFO:tasks.workunit.client.1.smithi152.stdout:1/497: dwrite d5/d7/d9/d59/f89 [0,4194304] 0 2024-03-29T04:45:14.674 INFO:tasks.workunit.client.1.smithi152.stdout:6/458: mkdir d1/d3/df/d8a/d98 0 2024-03-29T04:45:14.674 INFO:tasks.workunit.client.0.smithi142.stdout:9/919: rename d0/d12/c2e to d0/d12/d10e/d3c/d6f/c128 0 2024-03-29T04:45:14.674 INFO:tasks.workunit.client.1.smithi152.stdout:0/458: rmdir d2 39 2024-03-29T04:45:14.675 INFO:tasks.workunit.client.0.smithi142.stdout:4/932: write d0/d9/d19/d2c/f37 [3885744,61563] 0 2024-03-29T04:45:14.676 INFO:tasks.workunit.client.0.smithi142.stdout:9/920: read d0/d12/d50/fd5 [2338846,34037] 0 2024-03-29T04:45:14.676 INFO:tasks.workunit.client.1.smithi152.stdout:6/459: write d1/d3/d1a/d1b/d29/d6f/f3c [4489295,113214] 0 2024-03-29T04:45:14.677 INFO:tasks.workunit.client.0.smithi142.stdout:9/921: write d0/d6c/f124 [185252,76608] 0 2024-03-29T04:45:14.679 INFO:tasks.workunit.client.0.smithi142.stdout:4/933: readlink d0/d9/l11c 0 2024-03-29T04:45:14.680 INFO:tasks.workunit.client.1.smithi152.stdout:8/331: creat d1/f73 x:0 0 0 2024-03-29T04:45:14.681 INFO:tasks.workunit.client.0.smithi142.stdout:9/922: creat d0/d12/d50/d8e/f129 x:0 0 0 2024-03-29T04:45:14.682 INFO:tasks.workunit.client.1.smithi152.stdout:2/296: dwrite d0/d3/f10 [0,4194304] 0 2024-03-29T04:45:14.684 INFO:tasks.workunit.client.0.smithi142.stdout:4/934: symlink d0/d9/d11/d97/d129/l135 0 2024-03-29T04:45:14.684 INFO:tasks.workunit.client.1.smithi152.stdout:5/404: getdents d6/d16/d25/d2e 0 2024-03-29T04:45:14.684 INFO:tasks.workunit.client.0.smithi142.stdout:6/942: dwrite d1c/d45/d8a/fa0 [0,4194304] 0 2024-03-29T04:45:14.686 INFO:tasks.workunit.client.1.smithi152.stdout:6/460: mkdir d1/d3/d9/d99 0 2024-03-29T04:45:14.687 INFO:tasks.workunit.client.0.smithi142.stdout:4/935: read d0/d9/d11/d4c/d5f/d81/dc9/d126/f134 [824115,59374] 0 2024-03-29T04:45:14.688 INFO:tasks.workunit.client.0.smithi142.stdout:4/936: chown d0/d9/d11/d4c/d5f/fd0 71843 1 2024-03-29T04:45:14.688 INFO:tasks.workunit.client.0.smithi142.stdout:9/923: dread d0/d2/d61/d64/fdf [0,4194304] 0 2024-03-29T04:45:14.689 INFO:tasks.workunit.client.1.smithi152.stdout:1/498: dread d5/d7/d9/d59/f89 [0,4194304] 0 2024-03-29T04:45:14.689 INFO:tasks.workunit.client.1.smithi152.stdout:9/397: dwrite d5/db/d1b/f4d [0,4194304] 0 2024-03-29T04:45:14.691 INFO:tasks.workunit.client.0.smithi142.stdout:6/943: getdents d1c/d90/d94 0 2024-03-29T04:45:14.692 INFO:tasks.workunit.client.1.smithi152.stdout:9/398: write d5/d12/d2c/f54 [9262716,113776] 0 2024-03-29T04:45:14.695 INFO:tasks.workunit.client.1.smithi152.stdout:6/461: readlink d1/d3/d9/la 0 2024-03-29T04:45:14.698 INFO:tasks.workunit.client.0.smithi142.stdout:9/924: link d0/d2/d61/d64/dee/d10d/dc8/c10b d0/d12/d10e/dc7/daa/c12a 0 2024-03-29T04:45:14.699 INFO:tasks.workunit.client.1.smithi152.stdout:0/459: rename d2/da/d3a/d61/l67 to d2/da/d17/d50/d5c/daf/lb0 0 2024-03-29T04:45:14.700 INFO:tasks.workunit.client.1.smithi152.stdout:9/399: write d5/db/d1b/f4f [3481035,100221] 0 2024-03-29T04:45:14.701 INFO:tasks.workunit.client.0.smithi142.stdout:6/944: dwrite d1c/d33/d46/d61/d7e/f109 [0,4194304] 0 2024-03-29T04:45:14.703 INFO:tasks.workunit.client.1.smithi152.stdout:6/462: fsync d1/d3/d9/d13/f33 0 2024-03-29T04:45:14.704 INFO:tasks.workunit.client.1.smithi152.stdout:1/499: mknod d5/d7/d8/d13/c97 0 2024-03-29T04:45:14.706 INFO:tasks.workunit.client.1.smithi152.stdout:0/460: truncate d2/f9b 1541722 0 2024-03-29T04:45:14.706 INFO:tasks.workunit.client.1.smithi152.stdout:2/297: dwrite d0/d3/d30/f1a [0,4194304] 0 2024-03-29T04:45:14.707 INFO:tasks.workunit.client.0.smithi142.stdout:9/925: fdatasync d0/d12/d10e/fa8 0 2024-03-29T04:45:14.708 INFO:tasks.workunit.client.0.smithi142.stdout:9/926: write d0/d12/d10e/d3c/d6f/f11a [3756420,81487] 0 2024-03-29T04:45:14.709 INFO:tasks.workunit.client.1.smithi152.stdout:9/400: write d5/db/d1b/f49 [716530,71232] 0 2024-03-29T04:45:14.709 INFO:tasks.workunit.client.0.smithi142.stdout:6/945: creat d1c/d90/ddc/f12f x:0 0 0 2024-03-29T04:45:14.711 INFO:tasks.workunit.client.1.smithi152.stdout:5/405: dwrite d6/d1b/f78 [0,4194304] 0 2024-03-29T04:45:14.712 INFO:tasks.workunit.client.1.smithi152.stdout:8/332: creat d1/d2/d1d/f74 x:0 0 0 2024-03-29T04:45:14.718 INFO:tasks.workunit.client.1.smithi152.stdout:5/406: chown d6/d16/d25/d2e 1938 1 2024-03-29T04:45:14.719 INFO:tasks.workunit.client.1.smithi152.stdout:1/500: creat d5/f98 x:0 0 0 2024-03-29T04:45:14.720 INFO:tasks.workunit.client.1.smithi152.stdout:6/463: symlink d1/d3/d1a/d1b/l9a 0 2024-03-29T04:45:14.726 INFO:tasks.workunit.client.0.smithi142.stdout:6/946: getdents d1c/d33/d46/d66 0 2024-03-29T04:45:14.726 INFO:tasks.workunit.client.1.smithi152.stdout:2/298: dwrite d0/f44 [0,4194304] 0 2024-03-29T04:45:14.726 INFO:tasks.workunit.client.1.smithi152.stdout:5/407: chown d6/d16/d25/d2e/f54 48 1 2024-03-29T04:45:14.728 INFO:tasks.workunit.client.0.smithi142.stdout:6/947: chown d1c/d33/d46/d61/d7e/fb2 1514 1 2024-03-29T04:45:14.729 INFO:tasks.workunit.client.1.smithi152.stdout:5/408: dread d6/d16/d47/d56/d63/f6d [0,4194304] 0 2024-03-29T04:45:14.729 INFO:tasks.workunit.client.1.smithi152.stdout:8/333: fdatasync d1/d2/d3/f4e 0 2024-03-29T04:45:14.729 INFO:tasks.workunit.client.0.smithi142.stdout:6/948: dread - d1c/dc9/f103 zero size 2024-03-29T04:45:14.732 INFO:tasks.workunit.client.0.smithi142.stdout:6/949: fdatasync d1c/f31 0 2024-03-29T04:45:14.733 INFO:tasks.workunit.client.1.smithi152.stdout:9/401: link d5/d8/d31/f3b d5/d12/d2c/f8f 0 2024-03-29T04:45:14.733 INFO:tasks.workunit.client.1.smithi152.stdout:5/409: unlink d6/d16/l5d 0 2024-03-29T04:45:14.734 INFO:tasks.workunit.client.1.smithi152.stdout:8/334: truncate d1/d21/d40/d47/f72 5080885 0 2024-03-29T04:45:14.736 INFO:tasks.workunit.client.1.smithi152.stdout:8/335: dread - d1/d2/d1d/d36/d4b/f68 zero size 2024-03-29T04:45:14.737 INFO:tasks.workunit.client.1.smithi152.stdout:0/461: getdents d2/da/d17/d50/d5c 0 2024-03-29T04:45:14.738 INFO:tasks.workunit.client.0.smithi142.stdout:6/950: link d1c/d33/d46/d86/dc0/ff3 d1c/d90/ddc/dfb/f130 0 2024-03-29T04:45:14.738 INFO:tasks.workunit.client.1.smithi152.stdout:3/312: sync 2024-03-29T04:45:14.738 INFO:tasks.workunit.client.1.smithi152.stdout:3/313: chown d7/d46 1770947633 1 2024-03-29T04:45:14.739 INFO:tasks.workunit.client.1.smithi152.stdout:5/410: dread d6/d16/d25/d7e/f69 [0,4194304] 0 2024-03-29T04:45:14.740 INFO:tasks.workunit.client.1.smithi152.stdout:0/462: rename d2/da/d17 to d2/da/d17/d50/dac/db1 22 2024-03-29T04:45:14.741 INFO:tasks.workunit.client.1.smithi152.stdout:9/402: dread d5/db/d1b/f49 [0,4194304] 0 2024-03-29T04:45:14.741 INFO:tasks.workunit.client.1.smithi152.stdout:0/463: fdatasync d2/da/d17/d57/d9c/f9f 0 2024-03-29T04:45:14.742 INFO:tasks.workunit.client.1.smithi152.stdout:9/403: dread - d5/d12/d2c/f78 zero size 2024-03-29T04:45:14.742 INFO:tasks.workunit.client.1.smithi152.stdout:3/314: creat d7/d1c/f54 x:0 0 0 2024-03-29T04:45:14.743 INFO:tasks.workunit.client.1.smithi152.stdout:5/411: mknod d6/d1d/d3b/d55/c84 0 2024-03-29T04:45:14.744 INFO:tasks.workunit.client.1.smithi152.stdout:8/336: getdents d1/d2/d3c 0 2024-03-29T04:45:14.745 INFO:tasks.workunit.client.0.smithi142.stdout:6/951: dwrite d1c/d33/d46/d66/dc1/f100 [0,4194304] 0 2024-03-29T04:45:14.749 INFO:tasks.workunit.client.1.smithi152.stdout:5/412: symlink d6/d1d/d30/l85 0 2024-03-29T04:45:14.750 INFO:tasks.workunit.client.1.smithi152.stdout:9/404: mknod d5/db/c90 0 2024-03-29T04:45:14.750 INFO:tasks.workunit.client.0.smithi142.stdout:4/937: dread d0/d12/f1e [0,4194304] 0 2024-03-29T04:45:14.750 INFO:tasks.workunit.client.0.smithi142.stdout:6/952: truncate d1c/f1d 2316327 0 2024-03-29T04:45:14.750 INFO:tasks.workunit.client.0.smithi142.stdout:9/927: write d0/f94 [679845,15930] 0 2024-03-29T04:45:14.754 INFO:tasks.workunit.client.0.smithi142.stdout:6/953: truncate d1c/dc9/f103 566958 0 2024-03-29T04:45:14.754 INFO:tasks.workunit.client.1.smithi152.stdout:3/315: rename d7/d19/f30 to d7/d19/d3a/f55 0 2024-03-29T04:45:14.757 INFO:tasks.workunit.client.0.smithi142.stdout:4/938: mkdir d0/d9/d11/d4c/d5f/d81/dc9/d47/dea/d136 0 2024-03-29T04:45:14.758 INFO:tasks.workunit.client.1.smithi152.stdout:0/464: dwrite d2/f5 [4194304,4194304] 0 2024-03-29T04:45:14.758 INFO:tasks.workunit.client.1.smithi152.stdout:9/405: chown d5/db/d1b/f37 30819067 1 2024-03-29T04:45:14.760 INFO:tasks.workunit.client.0.smithi142.stdout:4/939: fsync d0/dc/d15/d3d/f43 0 2024-03-29T04:45:14.762 INFO:tasks.workunit.client.0.smithi142.stdout:4/940: truncate d0/d45/f103 372713 0 2024-03-29T04:45:14.762 INFO:tasks.workunit.client.0.smithi142.stdout:9/928: dread d0/f32 [0,4194304] 0 2024-03-29T04:45:14.763 INFO:tasks.workunit.client.0.smithi142.stdout:9/929: readlink d0/d2/d39/l9f 0 2024-03-29T04:45:14.763 INFO:tasks.workunit.client.1.smithi152.stdout:8/337: rename d1/d2/l6e to d1/d21/d40/d47/l75 0 2024-03-29T04:45:14.764 INFO:tasks.workunit.client.1.smithi152.stdout:5/413: dwrite d6/d16/d47/f61 [4194304,4194304] 0 2024-03-29T04:45:14.765 INFO:tasks.workunit.client.1.smithi152.stdout:0/465: dread d2/f5 [4194304,4194304] 0 2024-03-29T04:45:14.765 INFO:tasks.workunit.client.0.smithi142.stdout:4/941: symlink d0/d9/db2/dba/de9/l137 0 2024-03-29T04:45:14.766 INFO:tasks.workunit.client.1.smithi152.stdout:3/316: rmdir d7/d53 0 2024-03-29T04:45:14.767 INFO:tasks.workunit.client.1.smithi152.stdout:6/464: dread d1/d3/d1a/d1b/d29/f5f [0,4194304] 0 2024-03-29T04:45:14.769 INFO:tasks.workunit.client.0.smithi142.stdout:6/954: write d1c/d33/d46/d71/fdd [87609,23377] 0 2024-03-29T04:45:14.770 INFO:tasks.workunit.client.1.smithi152.stdout:0/466: write d2/da/d17/d57/d9c/fad [69984,110623] 0 2024-03-29T04:45:14.772 INFO:tasks.workunit.client.0.smithi142.stdout:9/930: rmdir d0/d2/d61/d64/d104/d10c 0 2024-03-29T04:45:14.773 INFO:tasks.workunit.client.0.smithi142.stdout:1/882: sync 2024-03-29T04:45:14.773 INFO:tasks.workunit.client.1.smithi152.stdout:0/467: write d2/da/f98 [609283,76400] 0 2024-03-29T04:45:14.773 INFO:tasks.workunit.client.1.smithi152.stdout:8/338: mkdir d1/d2/d3c/d76 0 2024-03-29T04:45:14.774 INFO:tasks.workunit.client.0.smithi142.stdout:4/942: creat d0/d9/d11/d4c/d5f/d81/dc9/d47/dea/d136/f138 x:0 0 0 2024-03-29T04:45:14.774 INFO:tasks.workunit.client.1.smithi152.stdout:5/414: fsync d6/d16/d25/d2e/f54 0 2024-03-29T04:45:14.774 INFO:tasks.workunit.client.0.smithi142.stdout:6/955: creat d1c/d90/ddc/f131 x:0 0 0 2024-03-29T04:45:14.775 INFO:tasks.workunit.client.0.smithi142.stdout:9/931: fdatasync d0/d6c/fa1 0 2024-03-29T04:45:14.776 INFO:tasks.workunit.client.1.smithi152.stdout:1/501: write d5/d7/fe [1073690,74567] 0 2024-03-29T04:45:14.779 INFO:tasks.workunit.client.1.smithi152.stdout:9/406: dwrite d5/db/d1b/f46 [0,4194304] 0 2024-03-29T04:45:14.781 INFO:tasks.workunit.client.1.smithi152.stdout:3/317: stat d7/d19/d1b/d1e/d22/l39 0 2024-03-29T04:45:14.783 INFO:tasks.workunit.client.0.smithi142.stdout:6/956: unlink d1c/d33/d46/d61/d9e/d122/c11e 0 2024-03-29T04:45:14.783 INFO:tasks.workunit.client.1.smithi152.stdout:5/415: mknod d6/d1b/c86 0 2024-03-29T04:45:14.785 INFO:tasks.workunit.client.0.smithi142.stdout:4/943: symlink d0/d9/d11/d97/d100/l139 0 2024-03-29T04:45:14.785 INFO:tasks.workunit.client.1.smithi152.stdout:0/468: rename d2/da/d17/d50/d5c/fa4 to d2/da/d17/d50/dac/fb2 0 2024-03-29T04:45:14.786 INFO:tasks.workunit.client.1.smithi152.stdout:6/465: creat d1/f9b x:0 0 0 2024-03-29T04:45:14.788 INFO:tasks.workunit.client.0.smithi142.stdout:9/932: rename d0/d2/d61/d64/dee/d10d/dc8/fe2 to d0/d2/d61/d64/dee/f12b 0 2024-03-29T04:45:14.788 INFO:tasks.workunit.client.1.smithi152.stdout:8/339: creat d1/d2/d1d/d36/d4b/d51/f77 x:0 0 0 2024-03-29T04:45:14.790 INFO:tasks.workunit.client.1.smithi152.stdout:9/407: mkdir d5/d12/d2c/d6e/d91 0 2024-03-29T04:45:14.790 INFO:tasks.workunit.client.0.smithi142.stdout:9/933: write d0/d2/d61/d64/dfa/ffc [513817,88945] 0 2024-03-29T04:45:14.791 INFO:tasks.workunit.client.0.smithi142.stdout:9/934: chown d0/d12/d50/cad 1305 1 2024-03-29T04:45:14.791 INFO:tasks.workunit.client.0.smithi142.stdout:4/944: mknod d0/d9/d11/d4c/d5f/d81/dc9/de5/c13a 0 2024-03-29T04:45:14.792 INFO:tasks.workunit.client.0.smithi142.stdout:1/883: write d8/db/dc/d77/d31/d4d/d8c/f95 [2688238,102998] 0 2024-03-29T04:45:14.793 INFO:tasks.workunit.client.1.smithi152.stdout:3/318: chown d7/d19/d1b/f21 163885 1 2024-03-29T04:45:14.795 INFO:tasks.workunit.client.1.smithi152.stdout:5/416: rename d6/d1d/d3b/d55/c1f to d6/d7/d75/c87 0 2024-03-29T04:45:14.796 INFO:tasks.workunit.client.0.smithi142.stdout:1/884: readlink d8/db/dd6/l11a 0 2024-03-29T04:45:14.797 INFO:tasks.workunit.client.1.smithi152.stdout:1/502: dwrite d5/d7/d9/f46 [0,4194304] 0 2024-03-29T04:45:14.797 INFO:tasks.workunit.client.0.smithi142.stdout:9/935: dwrite d0/d12/d50/f63 [0,4194304] 0 2024-03-29T04:45:14.799 INFO:tasks.workunit.client.1.smithi152.stdout:6/466: mknod d1/d3/d9/d99/c9c 0 2024-03-29T04:45:14.799 INFO:tasks.workunit.client.0.smithi142.stdout:1/885: readlink d8/db/dc/d77/db9/dbe/d2d/d44/la8 0 2024-03-29T04:45:14.800 INFO:tasks.workunit.client.0.smithi142.stdout:4/945: getdents d0/d9/db2/dba/dbc/daf 0 2024-03-29T04:45:14.803 INFO:tasks.workunit.client.0.smithi142.stdout:1/886: fdatasync d8/db/dc/f1c 0 2024-03-29T04:45:14.805 INFO:tasks.workunit.client.1.smithi152.stdout:9/408: dread d5/db/f58 [0,4194304] 0 2024-03-29T04:45:14.807 INFO:tasks.workunit.client.0.smithi142.stdout:3/778: sync 2024-03-29T04:45:14.807 INFO:tasks.workunit.client.0.smithi142.stdout:2/931: sync 2024-03-29T04:45:14.808 INFO:tasks.workunit.client.0.smithi142.stdout:4/946: creat d0/d9/d11/d4c/d5f/d81/f13b x:0 0 0 2024-03-29T04:45:14.808 INFO:tasks.workunit.client.1.smithi152.stdout:0/469: dwrite d2/f8 [0,4194304] 0 2024-03-29T04:45:14.809 INFO:tasks.workunit.client.0.smithi142.stdout:3/779: readlink d3/d29/d2d/d6c/lae 0 2024-03-29T04:45:14.809 INFO:tasks.workunit.client.1.smithi152.stdout:6/467: readlink d1/d3/d9/d13/l2a 0 2024-03-29T04:45:14.810 INFO:tasks.workunit.client.0.smithi142.stdout:6/957: dread d1c/d45/f4e [0,4194304] 0 2024-03-29T04:45:14.811 INFO:tasks.workunit.client.0.smithi142.stdout:6/958: chown d1c/d33/d46 2323 1 2024-03-29T04:45:14.812 INFO:tasks.workunit.client.0.smithi142.stdout:1/887: symlink d8/db/dc/d77/db9/dbe/d2d/d44/d7f/d10e/l126 0 2024-03-29T04:45:14.813 INFO:tasks.workunit.client.1.smithi152.stdout:3/319: dread d7/d19/f25 [0,4194304] 0 2024-03-29T04:45:14.814 INFO:tasks.workunit.client.0.smithi142.stdout:2/932: symlink de/d11/d17/d10b/d22/d27/dba/d61/dcd/d129/l133 0 2024-03-29T04:45:14.815 INFO:tasks.workunit.client.1.smithi152.stdout:8/340: dwrite d1/f34 [0,4194304] 0 2024-03-29T04:45:14.817 INFO:tasks.workunit.client.0.smithi142.stdout:6/959: truncate d1c/f3c 981830 0 2024-03-29T04:45:14.817 INFO:tasks.workunit.client.0.smithi142.stdout:2/933: fsync de/d11/d17/d10b/d84/f104 0 2024-03-29T04:45:14.818 INFO:tasks.workunit.client.1.smithi152.stdout:9/409: mkdir d5/db/d92 0 2024-03-29T04:45:14.818 INFO:tasks.workunit.client.1.smithi152.stdout:3/320: chown d7/d1c/f54 2 1 2024-03-29T04:45:14.818 INFO:tasks.workunit.client.0.smithi142.stdout:4/947: dwrite d0/d9/db2/ffe [0,4194304] 0 2024-03-29T04:45:14.820 INFO:tasks.workunit.client.1.smithi152.stdout:6/468: creat d1/d3/df/d84/f9d x:0 0 0 2024-03-29T04:45:14.821 INFO:tasks.workunit.client.0.smithi142.stdout:3/780: read d3/d29/d2b/d40/f4c [409681,49982] 0 2024-03-29T04:45:14.822 INFO:tasks.workunit.client.1.smithi152.stdout:3/321: write d7/d19/f32 [678704,39324] 0 2024-03-29T04:45:14.822 INFO:tasks.workunit.client.0.smithi142.stdout:2/934: unlink de/d11/d17/d10b/d22/dbf/l106 0 2024-03-29T04:45:14.822 INFO:tasks.workunit.client.1.smithi152.stdout:9/410: write d5/db/d1b/f46 [2129532,66648] 0 2024-03-29T04:45:14.824 INFO:tasks.workunit.client.1.smithi152.stdout:0/470: creat d2/da/df/fb3 x:0 0 0 2024-03-29T04:45:14.825 INFO:tasks.workunit.client.1.smithi152.stdout:5/417: dwrite d6/d16/d47/f61 [0,4194304] 0 2024-03-29T04:45:14.825 INFO:tasks.workunit.client.1.smithi152.stdout:6/469: read - d1/d3/d9/d13/d28/d45/f80 zero size 2024-03-29T04:45:14.826 INFO:tasks.workunit.client.0.smithi142.stdout:1/888: mkdir d8/db/dc/d77/db9/dbe/d2d/d44/d7f/d10e/d96/d106/d127 0 2024-03-29T04:45:14.826 INFO:tasks.workunit.client.0.smithi142.stdout:4/948: truncate d0/dc/d15/d3d/fe8 631476 0 2024-03-29T04:45:14.827 INFO:tasks.workunit.client.1.smithi152.stdout:9/411: chown d5/db/d1b/f4f 95307341 1 2024-03-29T04:45:14.830 INFO:tasks.workunit.client.1.smithi152.stdout:8/341: creat d1/d21/f78 x:0 0 0 2024-03-29T04:45:14.830 INFO:tasks.workunit.client.1.smithi152.stdout:3/322: readlink d7/d1c/l4b 0 2024-03-29T04:45:14.831 INFO:tasks.workunit.client.1.smithi152.stdout:8/342: readlink d1/d2/d3/l16 0 2024-03-29T04:45:14.831 INFO:tasks.workunit.client.1.smithi152.stdout:6/470: fsync d1/d3/d9/f11 0 2024-03-29T04:45:14.832 INFO:tasks.workunit.client.0.smithi142.stdout:2/935: symlink de/d11/d17/d10b/d22/d50/l134 0 2024-03-29T04:45:14.833 INFO:tasks.workunit.client.1.smithi152.stdout:5/418: symlink d6/d1b/l88 0 2024-03-29T04:45:14.834 INFO:tasks.workunit.client.1.smithi152.stdout:5/419: chown d6/d16/d25/d7e/f69 178877316 1 2024-03-29T04:45:14.834 INFO:tasks.workunit.client.1.smithi152.stdout:8/343: chown d1/d2/d1d/d36/d5d/f71 10790816 1 2024-03-29T04:45:14.835 INFO:tasks.workunit.client.0.smithi142.stdout:1/889: truncate d8/db/dc/d77/d31/d4d/d56/f118 528493 0 2024-03-29T04:45:14.835 INFO:tasks.workunit.client.0.smithi142.stdout:4/949: read d0/dc/f68 [144434,53378] 0 2024-03-29T04:45:14.835 INFO:tasks.workunit.client.0.smithi142.stdout:3/781: dwrite d3/d1f/fc4 [0,4194304] 0 2024-03-29T04:45:14.835 INFO:tasks.workunit.client.0.smithi142.stdout:3/782: chown d3/db9 722774 1 2024-03-29T04:45:14.836 INFO:tasks.workunit.client.0.smithi142.stdout:2/936: fsync de/d11/d3d/f65 0 2024-03-29T04:45:14.836 INFO:tasks.workunit.client.1.smithi152.stdout:6/471: write d1/d3/d9/d13/d28/d45/d8c/f85 [498607,15864] 0 2024-03-29T04:45:14.837 INFO:tasks.workunit.client.1.smithi152.stdout:0/471: mkdir d2/da/d3a/d61/db4 0 2024-03-29T04:45:14.840 INFO:tasks.workunit.client.1.smithi152.stdout:0/472: fdatasync d2/da/d17/d57/d9c/fa3 0 2024-03-29T04:45:14.842 INFO:tasks.workunit.client.1.smithi152.stdout:3/323: write d7/d19/d3a/f55 [4082504,116588] 0 2024-03-29T04:45:14.845 INFO:tasks.workunit.client.1.smithi152.stdout:5/420: creat d6/d1d/d3b/d55/f89 x:0 0 0 2024-03-29T04:45:14.845 INFO:tasks.workunit.client.0.smithi142.stdout:9/936: truncate d0/d12/d10e/f82 17322 0 2024-03-29T04:45:14.846 INFO:tasks.workunit.client.0.smithi142.stdout:3/783: dwrite d3/d29/d8f/d5f/db4/f8d [0,4194304] 0 2024-03-29T04:45:14.846 INFO:tasks.workunit.client.0.smithi142.stdout:4/950: dread d0/d9/d11/d4c/d64/f77 [0,4194304] 0 2024-03-29T04:45:14.849 INFO:tasks.workunit.client.1.smithi152.stdout:6/472: creat d1/d3/df/f9e x:0 0 0 2024-03-29T04:45:14.852 INFO:tasks.workunit.client.0.smithi142.stdout:1/890: dwrite d8/db/d5e/f73 [0,4194304] 0 2024-03-29T04:45:14.853 INFO:tasks.workunit.client.0.smithi142.stdout:9/937: creat d0/d12/d50/d52/da3/f12c x:0 0 0 2024-03-29T04:45:14.853 INFO:tasks.workunit.client.1.smithi152.stdout:8/344: creat d1/d21/d40/d47/f79 x:0 0 0 2024-03-29T04:45:14.854 INFO:tasks.workunit.client.0.smithi142.stdout:3/784: fsync d3/d29/d2b/d40/d67/d8b/f3e 0 2024-03-29T04:45:14.856 INFO:tasks.workunit.client.1.smithi152.stdout:6/473: creat d1/d3/d1a/f9f x:0 0 0 2024-03-29T04:45:14.857 INFO:tasks.workunit.client.0.smithi142.stdout:4/951: mkdir d0/d9/d11/d101/d13c 0 2024-03-29T04:45:14.858 INFO:tasks.workunit.client.1.smithi152.stdout:9/412: dread d5/d8/d31/f3b [0,4194304] 0 2024-03-29T04:45:14.858 INFO:tasks.workunit.client.1.smithi152.stdout:0/473: mkdir d2/da/d17/db5 0 2024-03-29T04:45:14.859 INFO:tasks.workunit.client.1.smithi152.stdout:6/474: chown d1/d3/df/l39 12811 1 2024-03-29T04:45:14.860 INFO:tasks.workunit.client.1.smithi152.stdout:8/345: creat d1/d2/d3c/f7a x:0 0 0 2024-03-29T04:45:14.860 INFO:tasks.workunit.client.1.smithi152.stdout:9/413: dread - d5/d12/f86 zero size 2024-03-29T04:45:14.861 INFO:tasks.workunit.client.0.smithi142.stdout:1/891: mknod d8/db/dc/c128 0 2024-03-29T04:45:14.863 INFO:tasks.workunit.client.1.smithi152.stdout:3/324: creat d7/d19/d1b/d1e/d4e/f56 x:0 0 0 2024-03-29T04:45:14.864 INFO:tasks.workunit.client.1.smithi152.stdout:3/325: fsync d7/d19/f26 0 2024-03-29T04:45:14.865 INFO:tasks.workunit.client.1.smithi152.stdout:0/474: fsync d2/d31/f36 0 2024-03-29T04:45:14.865 INFO:tasks.workunit.client.0.smithi142.stdout:1/892: fdatasync d8/db/dc/d77/db9/dbe/d2d/d44/d53/f6a 0 2024-03-29T04:45:14.865 INFO:tasks.workunit.client.1.smithi152.stdout:9/414: mknod d5/db/d1b/c93 0 2024-03-29T04:45:14.866 INFO:tasks.workunit.client.1.smithi152.stdout:9/415: dread - d5/d12/d2c/f76 zero size 2024-03-29T04:45:14.867 INFO:tasks.workunit.client.1.smithi152.stdout:9/416: dread - d5/d12/d2c/f50 zero size 2024-03-29T04:45:14.867 INFO:tasks.workunit.client.1.smithi152.stdout:6/475: rename d1/d3/d9/d13/d3e/f7a to d1/d3/df/d8a/fa0 0 2024-03-29T04:45:14.868 INFO:tasks.workunit.client.0.smithi142.stdout:4/952: dwrite d0/d9/d11/d97/f11d [0,4194304] 0 2024-03-29T04:45:14.870 INFO:tasks.workunit.client.0.smithi142.stdout:1/893: rmdir d8/db/dc/d77 39 2024-03-29T04:45:14.870 INFO:tasks.workunit.client.0.smithi142.stdout:2/937: rename de/d11/d17/d10b/d22/d27/dba/d61/dcd/d129 to de/d11/d2b/d3e/d135 0 2024-03-29T04:45:14.873 INFO:tasks.workunit.client.0.smithi142.stdout:4/953: write d0/d9/db2/dba/de9/ff8 [5014193,90746] 0 2024-03-29T04:45:14.875 INFO:tasks.workunit.client.1.smithi152.stdout:9/417: dwrite d5/db/f4e [0,4194304] 0 2024-03-29T04:45:14.876 INFO:tasks.workunit.client.0.smithi142.stdout:1/894: dread d8/db/dc/d77/d5a/f63 [0,4194304] 0 2024-03-29T04:45:14.878 INFO:tasks.workunit.client.1.smithi152.stdout:9/418: dread - d5/d12/d2c/d6e/d73/f7d zero size 2024-03-29T04:45:14.879 INFO:tasks.workunit.client.1.smithi152.stdout:3/326: chown d7/d19/c34 34634 1 2024-03-29T04:45:14.880 INFO:tasks.workunit.client.0.smithi142.stdout:1/895: mknod d8/db/dc/d77/d31/d4d/d8c/c129 0 2024-03-29T04:45:14.883 INFO:tasks.workunit.client.0.smithi142.stdout:2/938: getdents de/d11/d17/d10b/d22/d27/d3a 0 2024-03-29T04:45:14.884 INFO:tasks.workunit.client.1.smithi152.stdout:9/419: dread - d5/d8/d31/d59/f82 zero size 2024-03-29T04:45:14.885 INFO:tasks.workunit.client.1.smithi152.stdout:1/503: truncate d5/d7/d9/d59/f89 3109690 0 2024-03-29T04:45:14.886 INFO:tasks.workunit.client.1.smithi152.stdout:8/346: creat d1/f7b x:0 0 0 2024-03-29T04:45:14.887 INFO:tasks.workunit.client.0.smithi142.stdout:6/960: dread d1c/d45/d51/fb4 [0,4194304] 0 2024-03-29T04:45:14.887 INFO:tasks.workunit.client.0.smithi142.stdout:1/896: rename d8/db/dd6/d113/f94 to d8/db/dc/d66/f12a 0 2024-03-29T04:45:14.888 INFO:tasks.workunit.client.1.smithi152.stdout:3/327: write d7/d19/f25 [877833,72933] 0 2024-03-29T04:45:14.889 INFO:tasks.workunit.client.1.smithi152.stdout:1/504: readlink d5/d7/d38/l58 0 2024-03-29T04:45:14.890 INFO:tasks.workunit.client.1.smithi152.stdout:8/347: dread - d1/d2/d3/f4e zero size 2024-03-29T04:45:14.890 INFO:tasks.workunit.client.1.smithi152.stdout:5/421: dread d6/d1d/f79 [0,4194304] 0 2024-03-29T04:45:14.891 INFO:tasks.workunit.client.0.smithi142.stdout:2/939: dread de/d11/d17/d5d/d71/f9f [0,4194304] 0 2024-03-29T04:45:14.891 INFO:tasks.workunit.client.0.smithi142.stdout:3/785: dread d3/d29/d8f/d5f/db4/fa1 [0,4194304] 0 2024-03-29T04:45:14.891 INFO:tasks.workunit.client.1.smithi152.stdout:3/328: dread - d7/d19/d1b/d1e/f40 zero size 2024-03-29T04:45:14.892 INFO:tasks.workunit.client.1.smithi152.stdout:8/348: read - d1/d2/d1d/f74 zero size 2024-03-29T04:45:14.892 INFO:tasks.workunit.client.0.smithi142.stdout:6/961: mkdir d1c/d33/d46/d66/dcd/d132 0 2024-03-29T04:45:14.893 INFO:tasks.workunit.client.1.smithi152.stdout:9/420: creat d5/d12/d2c/d6e/d91/f94 x:0 0 0 2024-03-29T04:45:14.893 INFO:tasks.workunit.client.1.smithi152.stdout:3/329: read - d7/d1c/d41/f50 zero size 2024-03-29T04:45:14.893 INFO:tasks.workunit.client.0.smithi142.stdout:3/786: fdatasync d3/d29/d2b/d40/d67/d8b/fdd 0 2024-03-29T04:45:14.894 INFO:tasks.workunit.client.0.smithi142.stdout:1/897: truncate d8/db/dc/d77/db9/dbe/d2d/f11e 137216 0 2024-03-29T04:45:14.894 INFO:tasks.workunit.client.1.smithi152.stdout:5/422: creat d6/d7/f8a x:0 0 0 2024-03-29T04:45:14.895 INFO:tasks.workunit.client.0.smithi142.stdout:3/787: write d3/d29/d2d/ffc [993948,125012] 0 2024-03-29T04:45:14.895 INFO:tasks.workunit.client.0.smithi142.stdout:3/788: chown d3 16 1 2024-03-29T04:45:14.896 INFO:tasks.workunit.client.1.smithi152.stdout:5/423: dread - d6/d16/d25/d7e/f74 zero size 2024-03-29T04:45:14.896 INFO:tasks.workunit.client.0.smithi142.stdout:6/962: creat d1c/d33/d46/d71/f133 x:0 0 0 2024-03-29T04:45:14.897 INFO:tasks.workunit.client.1.smithi152.stdout:9/421: creat d5/db/d1b/f95 x:0 0 0 2024-03-29T04:45:14.897 INFO:tasks.workunit.client.0.smithi142.stdout:1/898: rmdir d8/db/dc/d77/db9/dbe/d2d/d44/dff 39 2024-03-29T04:45:14.898 INFO:tasks.workunit.client.0.smithi142.stdout:3/789: creat d3/d29/d8f/f10a x:0 0 0 2024-03-29T04:45:14.899 INFO:tasks.workunit.client.0.smithi142.stdout:3/790: readlink d3/d29/d2d/de9/lef 0 2024-03-29T04:45:14.900 INFO:tasks.workunit.client.0.smithi142.stdout:6/963: creat d1c/d33/d46/d61/d9b/f134 x:0 0 0 2024-03-29T04:45:14.900 INFO:tasks.workunit.client.1.smithi152.stdout:5/424: mknod d6/d16/d47/c8b 0 2024-03-29T04:45:14.901 INFO:tasks.workunit.client.1.smithi152.stdout:1/505: link d5/d7/c7f d5/c99 0 2024-03-29T04:45:14.905 INFO:tasks.workunit.client.1.smithi152.stdout:5/425: creat d6/d16/d25/d2e/f8c x:0 0 0 2024-03-29T04:45:14.907 INFO:tasks.workunit.client.1.smithi152.stdout:1/506: fsync d5/f3e 0 2024-03-29T04:45:14.909 INFO:tasks.workunit.client.1.smithi152.stdout:6/476: dread d1/d3/d9/d13/d28/f34 [0,4194304] 0 2024-03-29T04:45:14.910 INFO:tasks.workunit.client.1.smithi152.stdout:1/507: write d5/d7/d8/d13/f35 [4435841,126396] 0 2024-03-29T04:45:14.912 INFO:tasks.workunit.client.1.smithi152.stdout:5/426: dread d6/d1b/f78 [0,4194304] 0 2024-03-29T04:45:14.914 INFO:tasks.workunit.client.0.smithi142.stdout:6/964: dread d1c/dc7/d12d/f111 [0,4194304] 0 2024-03-29T04:45:14.915 INFO:tasks.workunit.client.0.smithi142.stdout:4/954: dwrite d0/d9/d11/d78/fb1 [0,4194304] 0 2024-03-29T04:45:14.917 INFO:tasks.workunit.client.1.smithi152.stdout:3/330: dread d7/f15 [0,4194304] 0 2024-03-29T04:45:14.918 INFO:tasks.workunit.client.0.smithi142.stdout:9/938: link d0/d12/d10e/f82 d0/d12/d10e/dc7/f12d 0 2024-03-29T04:45:14.919 INFO:tasks.workunit.client.1.smithi152.stdout:0/475: write d2/da/d17/d57/f62 [4877308,22047] 0 2024-03-29T04:45:14.920 INFO:tasks.workunit.client.1.smithi152.stdout:6/477: dwrite d1/d3/d9/f27 [0,4194304] 0 2024-03-29T04:45:14.929 INFO:tasks.workunit.client.1.smithi152.stdout:6/478: mkdir d1/d3/df/d84/da1 0 2024-03-29T04:45:14.933 INFO:tasks.workunit.client.0.smithi142.stdout:1/899: write d8/f2b [841726,117734] 0 2024-03-29T04:45:14.936 INFO:tasks.workunit.client.1.smithi152.stdout:3/331: creat d7/d19/f57 x:0 0 0 2024-03-29T04:45:14.936 INFO:tasks.workunit.client.1.smithi152.stdout:5/427: dwrite d6/d16/d47/d56/d63/f6d [0,4194304] 0 2024-03-29T04:45:14.936 INFO:tasks.workunit.client.1.smithi152.stdout:5/428: chown d6/d1d/d30/c3d 0 1 2024-03-29T04:45:14.939 INFO:tasks.workunit.client.1.smithi152.stdout:1/508: dread d5/d7/d8/d13/d19/f3d [0,4194304] 0 2024-03-29T04:45:14.941 INFO:tasks.workunit.client.0.smithi142.stdout:1/900: dwrite d8/db/dc/d77/d31/d4d/d56/faf [0,4194304] 0 2024-03-29T04:45:14.942 INFO:tasks.workunit.client.1.smithi152.stdout:1/509: write d5/d7/d1b/d2e/f5f [756834,35200] 0 2024-03-29T04:45:14.943 INFO:tasks.workunit.client.1.smithi152.stdout:5/429: mknod d6/d16/d47/d56/c8d 0 2024-03-29T04:45:14.945 INFO:tasks.workunit.client.1.smithi152.stdout:5/430: chown d6/d16/d25/d7e/f69 242689937 1 2024-03-29T04:45:14.946 INFO:tasks.workunit.client.1.smithi152.stdout:5/431: fdatasync d6/d16/d25/d7e/f64 0 2024-03-29T04:45:14.950 INFO:tasks.workunit.client.0.smithi142.stdout:2/940: rename de/d11/d17/d10b/d22/d27/dba/c5c to de/d11/d17/d10b/d22/d27/dba/d61/c136 0 2024-03-29T04:45:14.950 INFO:tasks.workunit.client.1.smithi152.stdout:3/332: truncate d7/d19/d1b/f23 1360451 0 2024-03-29T04:45:14.951 INFO:tasks.workunit.client.0.smithi142.stdout:1/901: dwrite d8/db/dc/d77/d31/d4d/d56/f118 [0,4194304] 0 2024-03-29T04:45:14.954 INFO:tasks.workunit.client.1.smithi152.stdout:1/510: rename d5/d7/d8/d13/l7e to d5/d7/d9/l9a 0 2024-03-29T04:45:14.955 INFO:tasks.workunit.client.1.smithi152.stdout:5/432: creat d6/d16/d25/f8e x:0 0 0 2024-03-29T04:45:14.958 INFO:tasks.workunit.client.0.smithi142.stdout:9/939: rmdir d0/d2/d61/d64/dee/d10d 39 2024-03-29T04:45:14.962 INFO:tasks.workunit.client.0.smithi142.stdout:6/965: rename d1c/d45/d8a/df6/ffe to d1c/d45/d51/f135 0 2024-03-29T04:45:14.963 INFO:tasks.workunit.client.1.smithi152.stdout:3/333: dwrite d7/f1a [4194304,4194304] 0 2024-03-29T04:45:14.969 INFO:tasks.workunit.client.1.smithi152.stdout:6/479: getdents d1/d3/df/d84 0 2024-03-29T04:45:14.969 INFO:tasks.workunit.client.1.smithi152.stdout:1/511: mkdir d5/d7/d8/d9b 0 2024-03-29T04:45:14.969 INFO:tasks.workunit.client.1.smithi152.stdout:5/433: dread d6/d1b/f31 [0,4194304] 0 2024-03-29T04:45:14.969 INFO:tasks.workunit.client.1.smithi152.stdout:0/476: dread d2/da/df/d86/f9e [0,4194304] 0 2024-03-29T04:45:14.969 INFO:tasks.workunit.client.1.smithi152.stdout:3/334: chown d7/l31 4 1 2024-03-29T04:45:14.969 INFO:tasks.workunit.client.1.smithi152.stdout:6/480: readlink d1/d3/d9/la 0 2024-03-29T04:45:14.972 INFO:tasks.workunit.client.0.smithi142.stdout:3/791: getdents d3/d29/d2b/d40/d67/d8b/d4d/d50 0 2024-03-29T04:45:14.973 INFO:tasks.workunit.client.0.smithi142.stdout:9/940: creat d0/d2/f12e x:0 0 0 2024-03-29T04:45:14.974 INFO:tasks.workunit.client.0.smithi142.stdout:3/792: creat d3/d1f/d24/f10b x:0 0 0 2024-03-29T04:45:14.977 INFO:tasks.workunit.client.0.smithi142.stdout:3/793: creat d3/d1f/d24/d47/d87/f10c x:0 0 0 2024-03-29T04:45:14.978 INFO:tasks.workunit.client.1.smithi152.stdout:5/434: dwrite d6/d1d/d3b/d55/f17 [4194304,4194304] 0 2024-03-29T04:45:14.979 INFO:tasks.workunit.client.1.smithi152.stdout:1/512: rename d5/d7/c71 to d5/d7/d9/d12/c9c 0 2024-03-29T04:45:14.980 INFO:tasks.workunit.client.1.smithi152.stdout:5/435: read d6/d1d/d30/f38 [625394,95055] 0 2024-03-29T04:45:14.981 INFO:tasks.workunit.client.1.smithi152.stdout:5/436: readlink d6/d1d/d30/l85 0 2024-03-29T04:45:14.983 INFO:tasks.workunit.client.1.smithi152.stdout:3/335: creat d7/d19/d1b/f58 x:0 0 0 2024-03-29T04:45:14.984 INFO:tasks.workunit.client.0.smithi142.stdout:3/794: getdents d3/d29/d2d/d79/dcc 0 2024-03-29T04:45:14.987 INFO:tasks.workunit.client.0.smithi142.stdout:3/795: mknod d3/d29/d2b/d6a/df4/c10d 0 2024-03-29T04:45:14.989 INFO:tasks.workunit.client.1.smithi152.stdout:0/477: rename d2/da/d3a/d61/l76 to d2/da/d3a/lb6 0 2024-03-29T04:45:14.989 INFO:tasks.workunit.client.1.smithi152.stdout:0/478: chown d2/da/d17/c51 0 1 2024-03-29T04:45:14.990 INFO:tasks.workunit.client.0.smithi142.stdout:2/941: link de/d11/d17/d10b/d22/d27/dba/lc1 de/d11/d3d/l137 0 2024-03-29T04:45:14.991 INFO:tasks.workunit.client.0.smithi142.stdout:6/966: dread d1c/d33/d46/d61/d9e/fef [0,4194304] 0 2024-03-29T04:45:14.993 INFO:tasks.workunit.client.1.smithi152.stdout:5/437: symlink d6/l8f 0 2024-03-29T04:45:14.994 INFO:tasks.workunit.client.0.smithi142.stdout:6/967: mknod d1c/d33/d46/d66/dc1/c136 0 2024-03-29T04:45:14.994 INFO:tasks.workunit.client.1.smithi152.stdout:3/336: mkdir d7/d19/d1b/d59 0 2024-03-29T04:45:14.995 INFO:tasks.workunit.client.0.smithi142.stdout:2/942: truncate de/d11/d17/d10b/d22/d27/f4c 1959543 0 2024-03-29T04:45:14.996 INFO:tasks.workunit.client.1.smithi152.stdout:1/513: dwrite d5/d7/d6a/f6e [0,4194304] 0 2024-03-29T04:45:14.997 INFO:tasks.workunit.client.1.smithi152.stdout:0/479: rename d2/f9b to d2/da/d17/d50/fb7 0 2024-03-29T04:45:14.997 INFO:tasks.workunit.client.1.smithi152.stdout:3/337: truncate d7/d1c/f51 235813 0 2024-03-29T04:45:15.000 INFO:tasks.workunit.client.0.smithi142.stdout:2/943: mkdir de/d11/d17/d5d/d138 0 2024-03-29T04:45:15.001 INFO:tasks.workunit.client.1.smithi152.stdout:5/438: chown d6/d16/d25/d2e/l82 8 1 2024-03-29T04:45:15.001 INFO:tasks.workunit.client.0.smithi142.stdout:6/968: dwrite d1c/d45/fd9 [0,4194304] 0 2024-03-29T04:45:15.002 INFO:tasks.workunit.client.0.smithi142.stdout:2/944: stat de/d11/d17/d5d/fe5 0 2024-03-29T04:45:15.002 INFO:tasks.workunit.client.1.smithi152.stdout:1/514: dread d5/d7/d1b/d2e/f8d [0,4194304] 0 2024-03-29T04:45:15.002 INFO:tasks.workunit.client.1.smithi152.stdout:5/439: fdatasync d6/d16/d25/d2e/f54 0 2024-03-29T04:45:15.004 INFO:tasks.workunit.client.0.smithi142.stdout:1/902: write d8/db/dc/d77/d31/d4d/d8c/fd7 [650680,65119] 0 2024-03-29T04:45:15.004 INFO:tasks.workunit.client.0.smithi142.stdout:4/955: write d0/d9/d11/d4c/d5f/d81/dc9/d47/d7a/fa1 [437263,12645] 0 2024-03-29T04:45:15.006 INFO:tasks.workunit.client.0.smithi142.stdout:9/941: write d0/d6c/f79 [330286,23820] 0 2024-03-29T04:45:15.007 INFO:tasks.workunit.client.0.smithi142.stdout:6/969: creat d1c/d12a/f137 x:0 0 0 2024-03-29T04:45:15.009 INFO:tasks.workunit.client.0.smithi142.stdout:1/903: symlink d8/db/dc/d77/db9/dbe/d2d/l12b 0 2024-03-29T04:45:15.014 INFO:tasks.workunit.client.1.smithi152.stdout:3/338: getdents d7 0 2024-03-29T04:45:15.014 INFO:tasks.workunit.client.0.smithi142.stdout:6/970: rename d1c/dc7/df2/f107 to d1c/d33/d46/d71/f138 0 2024-03-29T04:45:15.015 INFO:tasks.workunit.client.1.smithi152.stdout:1/515: dwrite d5/d7/d1b/f70 [0,4194304] 0 2024-03-29T04:45:15.016 INFO:tasks.workunit.client.1.smithi152.stdout:3/339: dread - d7/d19/d1b/d1e/f40 zero size 2024-03-29T04:45:15.016 INFO:tasks.workunit.client.1.smithi152.stdout:1/516: stat d5/c15 0 2024-03-29T04:45:15.017 INFO:tasks.workunit.client.0.smithi142.stdout:6/971: creat d1c/dc7/d12d/da4/f139 x:0 0 0 2024-03-29T04:45:15.020 INFO:tasks.workunit.client.1.smithi152.stdout:3/340: fdatasync d7/d1c/f2c 0 2024-03-29T04:45:15.020 INFO:tasks.workunit.client.1.smithi152.stdout:1/517: dread d5/d7/d1b/d2e/f8d [0,4194304] 0 2024-03-29T04:45:15.020 INFO:tasks.workunit.client.1.smithi152.stdout:1/518: chown c1 3830 1 2024-03-29T04:45:15.026 INFO:tasks.workunit.client.1.smithi152.stdout:1/519: mkdir d5/d7/d1b/d2e/d95/d9d 0 2024-03-29T04:45:15.027 INFO:tasks.workunit.client.1.smithi152.stdout:3/341: write d7/d19/d1b/f23 [761466,191] 0 2024-03-29T04:45:15.029 INFO:tasks.workunit.client.0.smithi142.stdout:4/956: write d0/dc/f33 [1143578,6033] 0 2024-03-29T04:45:15.030 INFO:tasks.workunit.client.0.smithi142.stdout:1/904: write d8/db/dc/d77/db9/dbe/d15/f60 [1024822,15562] 0 2024-03-29T04:45:15.032 INFO:tasks.workunit.client.1.smithi152.stdout:1/520: link d5/d7/d9/d59/f5b d5/d7/d1b/d2e/f9e 0 2024-03-29T04:45:15.035 INFO:tasks.workunit.client.0.smithi142.stdout:4/957: mkdir d0/d9/db2/dba/dbc/d6d/d6e/d12a/d13d 0 2024-03-29T04:45:15.035 INFO:tasks.workunit.client.1.smithi152.stdout:1/521: symlink d5/d7/d9/l9f 0 2024-03-29T04:45:15.037 INFO:tasks.workunit.client.0.smithi142.stdout:6/972: dwrite d1c/f1d [0,4194304] 0 2024-03-29T04:45:15.038 INFO:tasks.workunit.client.1.smithi152.stdout:3/342: dwrite d7/d1c/f51 [0,4194304] 0 2024-03-29T04:45:15.039 INFO:tasks.workunit.client.0.smithi142.stdout:4/958: readlink d0/d9/d11/d4c/d5f/d81/ldf 0 2024-03-29T04:45:15.040 INFO:tasks.workunit.client.1.smithi152.stdout:3/343: readlink d7/d1c/l4b 0 2024-03-29T04:45:15.041 INFO:tasks.workunit.client.0.smithi142.stdout:6/973: mkdir d1c/d45/d8a/df6/d13a 0 2024-03-29T04:45:15.044 INFO:tasks.workunit.client.0.smithi142.stdout:9/942: dread d0/d12/d10e/dc7/f66 [0,4194304] 0 2024-03-29T04:45:15.046 INFO:tasks.workunit.client.0.smithi142.stdout:6/974: creat d1c/dc7/df2/f13b x:0 0 0 2024-03-29T04:45:15.049 INFO:tasks.workunit.client.1.smithi152.stdout:3/344: dwrite d7/d1c/f2c [0,4194304] 0 2024-03-29T04:45:15.049 INFO:tasks.workunit.client.0.smithi142.stdout:4/959: getdents d0/d9/db2/dba/dbc 0 2024-03-29T04:45:15.050 INFO:tasks.workunit.client.0.smithi142.stdout:1/905: dwrite d8/db/dc/f2c [0,4194304] 0 2024-03-29T04:45:15.051 INFO:tasks.workunit.client.0.smithi142.stdout:4/960: chown d0/d9/d11/cd7 525 1 2024-03-29T04:45:15.052 INFO:tasks.workunit.client.0.smithi142.stdout:2/945: dread de/d11/d17/d4a/d79/d12b/f46 [4194304,4194304] 0 2024-03-29T04:45:15.052 INFO:tasks.workunit.client.0.smithi142.stdout:6/975: rmdir d1c/d33/d46/d61 39 2024-03-29T04:45:15.056 INFO:tasks.workunit.client.0.smithi142.stdout:9/943: rename d0/d12/d50/d52/fb7 to d0/d2/dcf/f12f 0 2024-03-29T04:45:15.056 INFO:tasks.workunit.client.1.smithi152.stdout:3/345: creat d7/d19/d1b/d1e/d4e/f5a x:0 0 0 2024-03-29T04:45:15.056 INFO:tasks.workunit.client.0.smithi142.stdout:2/946: rmdir de/d11/d17/d60/d6f/d116 39 2024-03-29T04:45:15.057 INFO:tasks.workunit.client.0.smithi142.stdout:9/944: stat d0/d12/d10e/dc7/f3b 0 2024-03-29T04:45:15.057 INFO:tasks.workunit.client.1.smithi152.stdout:1/522: dread d5/d7/d1b/f5e [0,4194304] 0 2024-03-29T04:45:15.061 INFO:tasks.workunit.client.0.smithi142.stdout:2/947: chown de/d11/d17/d10b/d22/d27/dba/d61/d7d/db4/f126 1925 1 2024-03-29T04:45:15.061 INFO:tasks.workunit.client.0.smithi142.stdout:1/906: creat d8/db/dc/d77/d31/d4d/d8c/f12c x:0 0 0 2024-03-29T04:45:15.061 INFO:tasks.workunit.client.1.smithi152.stdout:1/523: creat d5/d7/d1b/d2e/fa0 x:0 0 0 2024-03-29T04:45:15.062 INFO:tasks.workunit.client.0.smithi142.stdout:6/976: readlink d1c/dc7/d12d/ldf 0 2024-03-29T04:45:15.064 INFO:tasks.workunit.client.0.smithi142.stdout:6/977: write d1c/d45/d51/f7f [939449,106712] 0 2024-03-29T04:45:15.065 INFO:tasks.workunit.client.0.smithi142.stdout:9/945: mkdir d0/d2/dcf/d109/d130 0 2024-03-29T04:45:15.065 INFO:tasks.workunit.client.1.smithi152.stdout:1/524: mknod d5/d7/d38/ca1 0 2024-03-29T04:45:15.065 INFO:tasks.workunit.client.1.smithi152.stdout:3/346: getdents d7/d19/d1b 0 2024-03-29T04:45:15.066 INFO:tasks.workunit.client.0.smithi142.stdout:6/978: write d1c/d90/ddc/f131 [37805,77846] 0 2024-03-29T04:45:15.067 INFO:tasks.workunit.client.0.smithi142.stdout:1/907: creat d8/db/dc/d77/d5a/f12d x:0 0 0 2024-03-29T04:45:15.068 INFO:tasks.workunit.client.0.smithi142.stdout:4/961: creat d0/f13e x:0 0 0 2024-03-29T04:45:15.068 INFO:tasks.workunit.client.1.smithi152.stdout:3/347: rmdir d7/d1c/d41 39 2024-03-29T04:45:15.070 INFO:tasks.workunit.client.0.smithi142.stdout:6/979: creat d1c/d45/d8a/df6/f13c x:0 0 0 2024-03-29T04:45:15.070 INFO:tasks.workunit.client.1.smithi152.stdout:1/525: symlink d5/d7/d1b/d2e/d95/d9d/la2 0 2024-03-29T04:45:15.071 INFO:tasks.workunit.client.0.smithi142.stdout:4/962: creat d0/d45/f13f x:0 0 0 2024-03-29T04:45:15.075 INFO:tasks.workunit.client.1.smithi152.stdout:1/526: dread d5/d7/d8/d13/d19/f94 [4194304,4194304] 0 2024-03-29T04:45:15.076 INFO:tasks.workunit.client.0.smithi142.stdout:1/908: getdents d8/db/dc/d77/db9/dbe/d2d/d8b/da4 0 2024-03-29T04:45:15.077 INFO:tasks.workunit.client.1.smithi152.stdout:1/527: creat d5/d7/d1b/d2e/fa3 x:0 0 0 2024-03-29T04:45:15.077 INFO:tasks.workunit.client.0.smithi142.stdout:4/963: getdents d0/d9 0 2024-03-29T04:45:15.078 INFO:tasks.workunit.client.1.smithi152.stdout:1/528: fsync d5/d7/d38/f7c 0 2024-03-29T04:45:15.078 INFO:tasks.workunit.client.0.smithi142.stdout:4/964: read - d0/d9/d11/d4c/d64/d94/f10c zero size 2024-03-29T04:45:15.080 INFO:tasks.workunit.client.1.smithi152.stdout:1/529: symlink d5/d7/d4d/la4 0 2024-03-29T04:45:15.080 INFO:tasks.workunit.client.0.smithi142.stdout:1/909: creat d8/db/dc/d77/db9/dbe/d2d/d44/d4b/f12e x:0 0 0 2024-03-29T04:45:15.081 INFO:tasks.workunit.client.0.smithi142.stdout:2/948: write de/d11/d17/d10b/d22/f26 [596236,77493] 0 2024-03-29T04:45:15.082 INFO:tasks.workunit.client.1.smithi152.stdout:1/530: chown d5/d7/d1b/d2e/f9e 56 1 2024-03-29T04:45:15.082 INFO:tasks.workunit.client.1.smithi152.stdout:3/348: truncate d7/d19/d1b/f23 1115280 0 2024-03-29T04:45:15.084 INFO:tasks.workunit.client.1.smithi152.stdout:3/349: symlink d7/d19/d3a/l5b 0 2024-03-29T04:45:15.085 INFO:tasks.workunit.client.0.smithi142.stdout:6/980: write d1c/d33/d46/f65 [43882,19734] 0 2024-03-29T04:45:15.086 INFO:tasks.workunit.client.0.smithi142.stdout:2/949: fsync de/d11/d17/d4a/d79/d12b/f46 0 2024-03-29T04:45:15.086 INFO:tasks.workunit.client.0.smithi142.stdout:1/910: rmdir d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4 39 2024-03-29T04:45:15.087 INFO:tasks.workunit.client.0.smithi142.stdout:4/965: dwrite d0/d9/d19/d2c/f10d [0,4194304] 0 2024-03-29T04:45:15.088 INFO:tasks.workunit.client.0.smithi142.stdout:4/966: stat d0/lae 0 2024-03-29T04:45:15.088 INFO:tasks.workunit.client.1.smithi152.stdout:3/350: creat d7/d19/f5c x:0 0 0 2024-03-29T04:45:15.089 INFO:tasks.workunit.client.1.smithi152.stdout:1/531: dread d5/d7/d6a/f79 [0,4194304] 0 2024-03-29T04:45:15.091 INFO:tasks.workunit.client.0.smithi142.stdout:6/981: dread - d1c/d33/d46/d61/d9b/f134 zero size 2024-03-29T04:45:15.096 INFO:tasks.workunit.client.0.smithi142.stdout:1/911: dwrite d8/db/dc/d77/d31/d4d/f7a [0,4194304] 0 2024-03-29T04:45:15.099 INFO:tasks.workunit.client.1.smithi152.stdout:1/532: rmdir d5/d7/d8/d9b 0 2024-03-29T04:45:15.101 INFO:tasks.workunit.client.1.smithi152.stdout:4/297: sync 2024-03-29T04:45:15.101 INFO:tasks.workunit.client.1.smithi152.stdout:7/374: sync 2024-03-29T04:45:15.103 INFO:tasks.workunit.client.1.smithi152.stdout:4/298: write d0/d7/d25/d17/f5b [1323589,48015] 0 2024-03-29T04:45:15.105 INFO:tasks.workunit.client.0.smithi142.stdout:9/946: dread d0/d2/f11 [0,4194304] 0 2024-03-29T04:45:15.105 INFO:tasks.workunit.client.0.smithi142.stdout:4/967: dwrite d0/d9/d11/f23 [0,4194304] 0 2024-03-29T04:45:15.107 INFO:tasks.workunit.client.1.smithi152.stdout:7/375: write d7/d8/d12/d15/f31 [5175,123563] 0 2024-03-29T04:45:15.109 INFO:tasks.workunit.client.1.smithi152.stdout:1/533: truncate d5/d7/d8/d13/d19/f30 2064462 0 2024-03-29T04:45:15.110 INFO:tasks.workunit.client.1.smithi152.stdout:3/351: dwrite d7/d19/d1b/f21 [0,4194304] 0 2024-03-29T04:45:15.111 INFO:tasks.workunit.client.0.smithi142.stdout:9/947: rmdir d0/d2 39 2024-03-29T04:45:15.112 INFO:tasks.workunit.client.0.smithi142.stdout:2/950: dread de/d11/d3d/fd9 [0,4194304] 0 2024-03-29T04:45:15.113 INFO:tasks.workunit.client.1.smithi152.stdout:4/299: mkdir d0/d7/d25/d17/d5e/d60 0 2024-03-29T04:45:15.114 INFO:tasks.workunit.client.1.smithi152.stdout:1/534: dread d5/f72 [0,4194304] 0 2024-03-29T04:45:15.115 INFO:tasks.workunit.client.0.smithi142.stdout:1/912: write d8/db/dd6/d113/f38 [1080379,93761] 0 2024-03-29T04:45:15.115 INFO:tasks.workunit.client.0.smithi142.stdout:2/951: read de/d11/d2b/d7b/fa7 [575175,63799] 0 2024-03-29T04:45:15.116 INFO:tasks.workunit.client.1.smithi152.stdout:3/352: mkdir d7/d19/d3a/d5d 0 2024-03-29T04:45:15.120 INFO:tasks.workunit.client.0.smithi142.stdout:1/913: mkdir d8/db/dc/d77/db9/dbe/d2d/d44/d53/d12f 0 2024-03-29T04:45:15.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:14 smithi142 ceph-mon[29027]: pgmap v273: 65 pgs: 65 active+clean; 2.3 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 83 MiB/s rd, 241 MiB/s wr, 473 op/s 2024-03-29T04:45:15.123 INFO:tasks.workunit.client.0.smithi142.stdout:9/948: dwrite d0/d2/d61/d64/fe3 [0,4194304] 0 2024-03-29T04:45:15.123 INFO:tasks.workunit.client.1.smithi152.stdout:3/353: rename d7/d19/d1b/c2e to d7/d19/d1b/d1e/d22/c5e 0 2024-03-29T04:45:15.125 INFO:tasks.workunit.client.1.smithi152.stdout:7/376: getdents d7 0 2024-03-29T04:45:15.126 INFO:tasks.workunit.client.1.smithi152.stdout:1/535: mkdir d5/d7/d8/da5 0 2024-03-29T04:45:15.128 INFO:tasks.workunit.client.0.smithi142.stdout:1/914: symlink d8/db/dc/d109/l130 0 2024-03-29T04:45:15.129 INFO:tasks.workunit.client.0.smithi142.stdout:2/952: dwrite de/d11/d17/d60/de6/d101/f117 [0,4194304] 0 2024-03-29T04:45:15.132 INFO:tasks.workunit.client.1.smithi152.stdout:4/300: rename d0/c1d to d0/d7/d25/d48/d57/c61 0 2024-03-29T04:45:15.136 INFO:tasks.workunit.client.1.smithi152.stdout:3/354: dread - d7/d1c/d41/f50 zero size 2024-03-29T04:45:15.138 INFO:tasks.workunit.client.0.smithi142.stdout:1/915: symlink d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/l131 0 2024-03-29T04:45:15.139 INFO:tasks.workunit.client.0.smithi142.stdout:4/968: dwrite d0/dc/d15/d3d/d65/f105 [0,4194304] 0 2024-03-29T04:45:15.140 INFO:tasks.workunit.client.1.smithi152.stdout:3/355: rename d7/d19 to d7/d19/d3a/d5d/d5f 22 2024-03-29T04:45:15.140 INFO:tasks.workunit.client.1.smithi152.stdout:7/377: creat d7/d8/f6e x:0 0 0 2024-03-29T04:45:15.141 INFO:tasks.workunit.client.0.smithi142.stdout:9/949: write d0/d12/d10e/dc7/f12d [451352,61524] 0 2024-03-29T04:45:15.142 INFO:tasks.workunit.client.1.smithi152.stdout:4/301: dwrite d0/f13 [0,4194304] 0 2024-03-29T04:45:15.144 INFO:tasks.workunit.client.1.smithi152.stdout:4/302: read - d0/d7/d25/d48/d57/f5c zero size 2024-03-29T04:45:15.144 INFO:tasks.workunit.client.0.smithi142.stdout:1/916: mkdir d8/db/dc/d77/d5a/d132 0 2024-03-29T04:45:15.145 INFO:tasks.workunit.client.1.smithi152.stdout:4/303: chown d0/f15 94 1 2024-03-29T04:45:15.146 INFO:tasks.workunit.client.0.smithi142.stdout:4/969: mknod d0/d9/d11/d97/d9e/c140 0 2024-03-29T04:45:15.146 INFO:tasks.workunit.client.0.smithi142.stdout:4/970: readlink d0/d9/d11/d97/d129/l135 0 2024-03-29T04:45:15.148 INFO:tasks.workunit.client.1.smithi152.stdout:1/536: getdents d5/d7/d9 0 2024-03-29T04:45:15.148 INFO:tasks.workunit.client.1.smithi152.stdout:4/304: readlink d0/l9 0 2024-03-29T04:45:15.151 INFO:tasks.workunit.client.0.smithi142.stdout:1/917: mkdir d8/db/dc/d77/db9/dbe/d2d/d44/d7f/d10e/d133 0 2024-03-29T04:45:15.151 INFO:tasks.workunit.client.1.smithi152.stdout:3/356: mknod d7/d1c/c60 0 2024-03-29T04:45:15.153 INFO:tasks.workunit.client.0.smithi142.stdout:1/918: write d8/db/dc/d77/d31/d4d/d8c/f12c [127486,22405] 0 2024-03-29T04:45:15.154 INFO:tasks.workunit.client.0.smithi142.stdout:4/971: unlink d0/d9/d11/d97/d100/l139 0 2024-03-29T04:45:15.154 INFO:tasks.workunit.client.1.smithi152.stdout:3/357: fsync d7/d19/f26 0 2024-03-29T04:45:15.154 INFO:tasks.workunit.client.1.smithi152.stdout:7/378: creat d7/d8/d12/f6f x:0 0 0 2024-03-29T04:45:15.156 INFO:tasks.workunit.client.0.smithi142.stdout:1/919: fdatasync d8/db/dd6/d113/fe2 0 2024-03-29T04:45:15.156 INFO:tasks.workunit.client.1.smithi152.stdout:1/537: link d5/d7/d9/d12/l49 d5/d7/d9/d12/la6 0 2024-03-29T04:45:15.157 INFO:tasks.workunit.client.0.smithi142.stdout:4/972: truncate d0/dc/d15/d3d/d65/f85 4092186 0 2024-03-29T04:45:15.158 INFO:tasks.workunit.client.0.smithi142.stdout:1/920: fdatasync d8/db/dc/d77/db9/dbe/d2d/d44/d7f/df4/dfb/f10c 0 2024-03-29T04:45:15.158 INFO:tasks.workunit.client.1.smithi152.stdout:4/305: dwrite d0/d27/f34 [0,4194304] 0 2024-03-29T04:45:15.162 INFO:tasks.workunit.client.0.smithi142.stdout:1/921: unlink d8/db/dc/d77/db9/dbe/d2d/d8b/da4/fdc 0 2024-03-29T04:45:15.164 INFO:tasks.workunit.client.1.smithi152.stdout:1/538: mkdir d5/d7/d1b/d2e/d95/d9d/da7 0 2024-03-29T04:45:15.166 INFO:tasks.workunit.client.0.smithi142.stdout:1/922: mkdir d8/db/dc/d19/d134 0 2024-03-29T04:45:15.171 INFO:tasks.workunit.client.1.smithi152.stdout:3/358: fdatasync d7/d19/d1b/f23 0 2024-03-29T04:45:15.172 INFO:tasks.workunit.client.1.smithi152.stdout:3/359: readlink d7/l31 0 2024-03-29T04:45:15.172 INFO:tasks.workunit.client.1.smithi152.stdout:7/379: dwrite d7/d8/d12/f20 [0,4194304] 0 2024-03-29T04:45:15.172 INFO:tasks.workunit.client.1.smithi152.stdout:7/380: write d7/d8/d12/d1d/d41/f2c [4036322,43453] 0 2024-03-29T04:45:15.173 INFO:tasks.workunit.client.0.smithi142.stdout:1/923: stat d8/db/dc/d77/db9/dbe/d2d/d44/d80/c83 0 2024-03-29T04:45:15.174 INFO:tasks.workunit.client.0.smithi142.stdout:2/953: dwrite de/d11/d17/d5d/d71/fe8 [0,4194304] 0 2024-03-29T04:45:15.174 INFO:tasks.workunit.client.0.smithi142.stdout:2/954: chown de/d11/d17/d4a/d79/d12b/dd1/ce9 156 1 2024-03-29T04:45:15.174 INFO:tasks.workunit.client.0.smithi142.stdout:1/924: mkdir d8/db/dc/d77/db9/dbe/d2d/d44/d53/d135 0 2024-03-29T04:45:15.175 INFO:tasks.workunit.client.0.smithi142.stdout:1/925: truncate d8/db/dc/d77/de9/f123 396529 0 2024-03-29T04:45:15.179 INFO:tasks.workunit.client.0.smithi142.stdout:1/926: mknod d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/ded/c136 0 2024-03-29T04:45:15.179 INFO:tasks.workunit.client.0.smithi142.stdout:2/955: creat de/d11/d17/d5d/f139 x:0 0 0 2024-03-29T04:45:15.180 INFO:tasks.workunit.client.1.smithi152.stdout:3/360: dwrite d7/d1c/d41/f50 [0,4194304] 0 2024-03-29T04:45:15.181 INFO:tasks.workunit.client.0.smithi142.stdout:2/956: fsync de/d11/d17/d10b/d22/d27/dba/d61/d7d/f7e 0 2024-03-29T04:45:15.182 INFO:tasks.workunit.client.0.smithi142.stdout:1/927: mknod d8/db/dc/d109/c137 0 2024-03-29T04:45:15.182 INFO:tasks.workunit.client.0.smithi142.stdout:2/957: chown de/d11/d2b/d3e/f76 206087012 1 2024-03-29T04:45:15.184 INFO:tasks.workunit.client.0.smithi142.stdout:2/958: read - de/d11/d17/d10b/d84/f103 zero size 2024-03-29T04:45:15.184 INFO:tasks.workunit.client.0.smithi142.stdout:2/959: chown de/d11/d2b/lcb 915 1 2024-03-29T04:45:15.185 INFO:tasks.workunit.client.1.smithi152.stdout:7/381: dwrite d7/d8/d12/d15/f50 [0,4194304] 0 2024-03-29T04:45:15.186 INFO:tasks.workunit.client.1.smithi152.stdout:3/361: mkdir d7/d1c/d41/d61 0 2024-03-29T04:45:15.187 INFO:tasks.workunit.client.1.smithi152.stdout:3/362: fdatasync d7/d1c/f54 0 2024-03-29T04:45:15.187 INFO:tasks.workunit.client.0.smithi142.stdout:1/928: mkdir d8/db/dc/d77/db9/dbe/d2d/d44/d7f/d10e/d133/d138 0 2024-03-29T04:45:15.188 INFO:tasks.workunit.client.1.smithi152.stdout:1/539: dread d5/f3e [0,4194304] 0 2024-03-29T04:45:15.188 INFO:tasks.workunit.client.0.smithi142.stdout:2/960: getdents de/d11/d17/d60/d6f/d116 0 2024-03-29T04:45:15.189 INFO:tasks.workunit.client.0.smithi142.stdout:2/961: chown de/d11/d17/d10b/d22/d27/dba 0 1 2024-03-29T04:45:15.189 INFO:tasks.workunit.client.1.smithi152.stdout:1/540: write d5/d7/d6a/f88 [332529,69824] 0 2024-03-29T04:45:15.190 INFO:tasks.workunit.client.0.smithi142.stdout:1/929: mknod d8/db/dc/d77/db9/dbe/d2d/c139 0 2024-03-29T04:45:15.190 INFO:tasks.workunit.client.1.smithi152.stdout:7/382: creat d7/d8/d12/d1d/d41/d45/f70 x:0 0 0 2024-03-29T04:45:15.190 INFO:tasks.workunit.client.1.smithi152.stdout:1/541: stat d5/d7/d9/l81 0 2024-03-29T04:45:15.191 INFO:tasks.workunit.client.1.smithi152.stdout:7/383: fdatasync d7/d8/f3b 0 2024-03-29T04:45:15.191 INFO:tasks.workunit.client.1.smithi152.stdout:3/363: readlink d7/d19/d1b/d1e/d22/l39 0 2024-03-29T04:45:15.192 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:14 smithi152 ceph-mon[37177]: pgmap v273: 65 pgs: 65 active+clean; 2.3 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 83 MiB/s rd, 241 MiB/s wr, 473 op/s 2024-03-29T04:45:15.192 INFO:tasks.workunit.client.0.smithi142.stdout:1/930: fdatasync d8/db/dc/d77/db9/dbe/d2d/d44/d53/f6a 0 2024-03-29T04:45:15.193 INFO:tasks.workunit.client.0.smithi142.stdout:2/962: symlink de/d11/d2b/d3e/d12e/l13a 0 2024-03-29T04:45:15.193 INFO:tasks.workunit.client.1.smithi152.stdout:3/364: chown d7/d19/f25 252088 1 2024-03-29T04:45:15.193 INFO:tasks.workunit.client.1.smithi152.stdout:1/542: write d5/d7/d8/d13/f35 [5108532,113359] 0 2024-03-29T04:45:15.193 INFO:tasks.workunit.client.1.smithi152.stdout:7/384: write d7/d8/d12/d35/f2a [355497,71771] 0 2024-03-29T04:45:15.193 INFO:tasks.workunit.client.1.smithi152.stdout:3/365: write d7/d19/f26 [834597,129976] 0 2024-03-29T04:45:15.195 INFO:tasks.workunit.client.0.smithi142.stdout:2/963: symlink de/d11/d17/l13b 0 2024-03-29T04:45:15.195 INFO:tasks.workunit.client.0.smithi142.stdout:1/931: creat d8/db/dc/d77/db9/dbe/d2d/d44/d80/f13a x:0 0 0 2024-03-29T04:45:15.197 INFO:tasks.workunit.client.1.smithi152.stdout:1/543: mknod d5/d7/d4d/ca8 0 2024-03-29T04:45:15.197 INFO:tasks.workunit.client.1.smithi152.stdout:1/544: read - d5/f98 zero size 2024-03-29T04:45:15.197 INFO:tasks.workunit.client.1.smithi152.stdout:2/299: sync 2024-03-29T04:45:15.200 INFO:tasks.workunit.client.0.smithi142.stdout:2/964: rename de/d11/d17/d10b/d22/d27/dba/d61/d7d to de/d11/d17/d60/de6/d101/d13c 0 2024-03-29T04:45:15.202 INFO:tasks.workunit.client.1.smithi152.stdout:7/385: rename d7/d8/l3f to d7/d8/d12/d35/d68/l71 0 2024-03-29T04:45:15.202 INFO:tasks.workunit.client.1.smithi152.stdout:2/300: read - d0/d3/d30/d15/d22/d4c/f51 zero size 2024-03-29T04:45:15.204 INFO:tasks.workunit.client.0.smithi142.stdout:1/932: rename d8/db/dc/d77/d31/d4d/c59 to d8/db/dc/d77/db9/dbe/d2d/d8b/da4/c13b 0 2024-03-29T04:45:15.204 INFO:tasks.workunit.client.1.smithi152.stdout:7/386: mkdir d7/d8/d12/d15/d72 0 2024-03-29T04:45:15.204 INFO:tasks.workunit.client.1.smithi152.stdout:1/545: creat d5/d7/d9/d2b/fa9 x:0 0 0 2024-03-29T04:45:15.205 INFO:tasks.workunit.client.1.smithi152.stdout:2/301: read - d0/d3/d30/d15/f26 zero size 2024-03-29T04:45:15.206 INFO:tasks.workunit.client.0.smithi142.stdout:1/933: chown d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/fe7 17687093 1 2024-03-29T04:45:15.208 INFO:tasks.workunit.client.0.smithi142.stdout:2/965: dwrite de/f19 [0,4194304] 0 2024-03-29T04:45:15.210 INFO:tasks.workunit.client.1.smithi152.stdout:7/387: truncate d7/d8/d12/d35/f6d 607608 0 2024-03-29T04:45:15.210 INFO:tasks.workunit.client.1.smithi152.stdout:1/546: creat d5/d7/d4d/faa x:0 0 0 2024-03-29T04:45:15.211 INFO:tasks.workunit.client.0.smithi142.stdout:9/950: write d0/d6c/fa1 [256511,103676] 0 2024-03-29T04:45:15.211 INFO:tasks.workunit.client.1.smithi152.stdout:2/302: symlink d0/d3/d30/d15/d22/d4c/l67 0 2024-03-29T04:45:15.212 INFO:tasks.workunit.client.1.smithi152.stdout:7/388: chown d7/d8/d12/d1d/d41/l5f 22 1 2024-03-29T04:45:15.213 INFO:tasks.workunit.client.0.smithi142.stdout:9/951: chown d0/d12/d50/d52/f103 23956139 1 2024-03-29T04:45:15.214 INFO:tasks.workunit.client.0.smithi142.stdout:4/973: getdents d0/d9/d11/d97/d100 0 2024-03-29T04:45:15.214 INFO:tasks.workunit.client.1.smithi152.stdout:7/389: fsync d7/d8/d12/d35/f1a 0 2024-03-29T04:45:15.214 INFO:tasks.workunit.client.1.smithi152.stdout:1/547: fdatasync d5/f72 0 2024-03-29T04:45:15.215 INFO:tasks.workunit.client.0.smithi142.stdout:4/974: fdatasync d0/d9/db2/dba/de9/ff8 0 2024-03-29T04:45:15.215 INFO:tasks.workunit.client.1.smithi152.stdout:2/303: mkdir d0/d3/df/d2c/d45/d68 0 2024-03-29T04:45:15.216 INFO:tasks.workunit.client.0.smithi142.stdout:2/966: creat de/d11/d17/d10b/d22/dbf/f13d x:0 0 0 2024-03-29T04:45:15.216 INFO:tasks.workunit.client.1.smithi152.stdout:7/390: chown d7/f14 25543162 1 2024-03-29T04:45:15.216 INFO:tasks.workunit.client.1.smithi152.stdout:2/304: chown d0/d3/f10 364846 1 2024-03-29T04:45:15.217 INFO:tasks.workunit.client.1.smithi152.stdout:3/366: dread d7/d19/f3c [0,4194304] 0 2024-03-29T04:45:15.218 INFO:tasks.workunit.client.0.smithi142.stdout:2/967: symlink de/d11/d17/d60/de6/d101/l13e 0 2024-03-29T04:45:15.219 INFO:tasks.workunit.client.0.smithi142.stdout:9/952: creat d0/d2/d61/f131 x:0 0 0 2024-03-29T04:45:15.219 INFO:tasks.workunit.client.0.smithi142.stdout:2/968: fdatasync de/d11/d17/d4a/d11a/f11e 0 2024-03-29T04:45:15.220 INFO:tasks.workunit.client.1.smithi152.stdout:3/367: fsync d7/d1c/f47 0 2024-03-29T04:45:15.222 INFO:tasks.workunit.client.1.smithi152.stdout:1/548: creat d5/d7/d9/d2b/fab x:0 0 0 2024-03-29T04:45:15.222 INFO:tasks.workunit.client.0.smithi142.stdout:2/969: symlink de/d11/d17/d4a/d79/l13f 0 2024-03-29T04:45:15.225 INFO:tasks.workunit.client.0.smithi142.stdout:1/934: link d8/db/dc/d77/db9/dbe/d15/f47 d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/f13c 0 2024-03-29T04:45:15.225 INFO:tasks.workunit.client.0.smithi142.stdout:2/970: mknod de/d11/d17/c140 0 2024-03-29T04:45:15.228 INFO:tasks.workunit.client.0.smithi142.stdout:9/953: link d0/d12/d50/d52/da3/f12c d0/d2/dcf/f132 0 2024-03-29T04:45:15.230 INFO:tasks.workunit.client.0.smithi142.stdout:9/954: fsync d0/d6c/dc2/fc9 0 2024-03-29T04:45:15.231 INFO:tasks.workunit.client.0.smithi142.stdout:2/971: creat de/d11/d17/d10b/d22/d27/dba/f141 x:0 0 0 2024-03-29T04:45:15.232 INFO:tasks.workunit.client.1.smithi152.stdout:7/391: dwrite d7/d8/fa [0,4194304] 0 2024-03-29T04:45:15.232 INFO:tasks.workunit.client.0.smithi142.stdout:1/935: dwrite d8/db/dc/d77/d31/d4d/d8c/fd7 [0,4194304] 0 2024-03-29T04:45:15.235 INFO:tasks.workunit.client.0.smithi142.stdout:9/955: unlink d0/d12/d50/d52/da3/f122 0 2024-03-29T04:45:15.236 INFO:tasks.workunit.client.0.smithi142.stdout:2/972: truncate de/d11/d17/d5d/f86 934344 0 2024-03-29T04:45:15.236 INFO:tasks.workunit.client.1.smithi152.stdout:1/549: creat d5/d7/d1b/d2e/d95/fac x:0 0 0 2024-03-29T04:45:15.238 INFO:tasks.workunit.client.0.smithi142.stdout:1/936: creat d8/db/dc/d77/db9/dbe/d15/f13d x:0 0 0 2024-03-29T04:45:15.241 INFO:tasks.workunit.client.1.smithi152.stdout:1/550: readlink d5/d7/d8/d13/l32 0 2024-03-29T04:45:15.243 INFO:tasks.workunit.client.0.smithi142.stdout:9/956: rename d0/d2/l4d to d0/d12/d50/d52/da3/l133 0 2024-03-29T04:45:15.244 INFO:tasks.workunit.client.0.smithi142.stdout:9/957: chown d0/d2/d61/d64/fe3 1 1 2024-03-29T04:45:15.245 INFO:tasks.workunit.client.0.smithi142.stdout:1/937: symlink d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/dea/l13e 0 2024-03-29T04:45:15.246 INFO:tasks.workunit.client.1.smithi152.stdout:3/368: dwrite d7/f15 [4194304,4194304] 0 2024-03-29T04:45:15.249 INFO:tasks.workunit.client.1.smithi152.stdout:2/305: dwrite d0/d3/f31 [0,4194304] 0 2024-03-29T04:45:15.249 INFO:tasks.workunit.client.1.smithi152.stdout:1/551: chown d5/d7/d8/d13/f45 6 1 2024-03-29T04:45:15.249 INFO:tasks.workunit.client.0.smithi142.stdout:1/938: mkdir d8/db/dc/d77/db9/dbe/d2d/d8b/d13f 0 2024-03-29T04:45:15.251 INFO:tasks.workunit.client.0.smithi142.stdout:9/958: dwrite d0/d12/d10e/f97 [0,4194304] 0 2024-03-29T04:45:15.253 INFO:tasks.workunit.client.1.smithi152.stdout:7/392: dread d7/d8/db/f42 [0,4194304] 0 2024-03-29T04:45:15.257 INFO:tasks.workunit.client.1.smithi152.stdout:3/369: truncate d7/d19/d1b/d1e/d22/f4d 556893 0 2024-03-29T04:45:15.257 INFO:tasks.workunit.client.1.smithi152.stdout:7/393: read d7/d8/d12/d15/f50 [2176909,9931] 0 2024-03-29T04:45:15.258 INFO:tasks.workunit.client.1.smithi152.stdout:4/306: dread d0/f15 [0,4194304] 0 2024-03-29T04:45:15.260 INFO:tasks.workunit.client.1.smithi152.stdout:4/307: truncate d0/d7/d25/d17/f4d 17727 0 2024-03-29T04:45:15.263 INFO:tasks.workunit.client.1.smithi152.stdout:4/308: truncate d0/d7/d25/d48/d57/f5c 107754 0 2024-03-29T04:45:15.264 INFO:tasks.workunit.client.0.smithi142.stdout:4/975: write d0/d9/d11/d4c/d5f/d81/ff1 [837873,7742] 0 2024-03-29T04:45:15.265 INFO:tasks.workunit.client.0.smithi142.stdout:4/976: chown d0/d9/d11/lc2 1272 1 2024-03-29T04:45:15.266 INFO:tasks.workunit.client.1.smithi152.stdout:3/370: read d7/d19/d3a/f55 [324004,72761] 0 2024-03-29T04:45:15.266 INFO:tasks.workunit.client.1.smithi152.stdout:7/394: creat d7/d8/d12/d35/d68/f73 x:0 0 0 2024-03-29T04:45:15.267 INFO:tasks.workunit.client.0.smithi142.stdout:3/796: sync 2024-03-29T04:45:15.267 INFO:tasks.workunit.client.1.smithi152.stdout:4/309: rmdir d0/d27 39 2024-03-29T04:45:15.267 INFO:tasks.workunit.client.1.smithi152.stdout:3/371: chown d7/cc 14165 1 2024-03-29T04:45:15.269 INFO:tasks.workunit.client.0.smithi142.stdout:9/959: dwrite d0/d2/f1f [0,4194304] 0 2024-03-29T04:45:15.270 INFO:tasks.workunit.client.1.smithi152.stdout:4/310: mkdir d0/d7/d40/d62 0 2024-03-29T04:45:15.271 INFO:tasks.workunit.client.0.smithi142.stdout:4/977: creat d0/d9/d11/d84/d9f/f141 x:0 0 0 2024-03-29T04:45:15.271 INFO:tasks.workunit.client.1.smithi152.stdout:2/306: rmdir d0/d3/df/d2c/d45/d68 0 2024-03-29T04:45:15.272 INFO:tasks.workunit.client.0.smithi142.stdout:4/978: write d0/d9/d11/fde [2026255,59798] 0 2024-03-29T04:45:15.272 INFO:tasks.workunit.client.0.smithi142.stdout:3/797: symlink d3/d29/d2b/d40/d67/d8b/d6/l10e 0 2024-03-29T04:45:15.273 INFO:tasks.workunit.client.1.smithi152.stdout:7/395: rename d7/d8/d37/l40 to d7/d8/d12/d15/l74 0 2024-03-29T04:45:15.274 INFO:tasks.workunit.client.0.smithi142.stdout:3/798: write d3/d1f/de6/df1/f105 [217674,76121] 0 2024-03-29T04:45:15.274 INFO:tasks.workunit.client.1.smithi152.stdout:2/307: chown d0/d3/d30/c48 793 1 2024-03-29T04:45:15.274 INFO:tasks.workunit.client.1.smithi152.stdout:3/372: symlink d7/d19/d1b/l62 0 2024-03-29T04:45:15.277 INFO:tasks.workunit.client.1.smithi152.stdout:4/311: getdents d0/d7/d25/d48/d57 0 2024-03-29T04:45:15.279 INFO:tasks.workunit.client.1.smithi152.stdout:2/308: creat d0/d3/df/d2c/f69 x:0 0 0 2024-03-29T04:45:15.281 INFO:tasks.workunit.client.1.smithi152.stdout:4/312: write d0/d7/d25/d48/d57/f47 [555965,40257] 0 2024-03-29T04:45:15.281 INFO:tasks.workunit.client.1.smithi152.stdout:2/309: write d0/d3/f10 [1107501,40515] 0 2024-03-29T04:45:15.281 INFO:tasks.workunit.client.1.smithi152.stdout:3/373: fdatasync d7/d19/d1b/f23 0 2024-03-29T04:45:15.282 INFO:tasks.workunit.client.1.smithi152.stdout:7/396: getdents d7/d8/d12/d1d/d30 0 2024-03-29T04:45:15.282 INFO:tasks.workunit.client.0.smithi142.stdout:4/979: creat d0/d9/d11/d4c/d5f/d81/dc9/d47/dea/f142 x:0 0 0 2024-03-29T04:45:15.284 INFO:tasks.workunit.client.0.smithi142.stdout:3/799: fsync d3/d29/d8f/d5f/d64/f76 0 2024-03-29T04:45:15.286 INFO:tasks.workunit.client.1.smithi152.stdout:4/313: unlink d0/d7/f35 0 2024-03-29T04:45:15.287 INFO:tasks.workunit.client.0.smithi142.stdout:1/939: write d8/db/dc/d77/db9/dbe/d2d/d44/d53/fb7 [260905,60965] 0 2024-03-29T04:45:15.288 INFO:tasks.workunit.client.1.smithi152.stdout:7/397: mkdir d7/d8/d37/d75 0 2024-03-29T04:45:15.288 INFO:tasks.workunit.client.0.smithi142.stdout:1/940: chown d8/db/d5e/f8d 303551800 1 2024-03-29T04:45:15.289 INFO:tasks.workunit.client.1.smithi152.stdout:2/310: dwrite d0/d3/df/f23 [0,4194304] 0 2024-03-29T04:45:15.290 INFO:tasks.workunit.client.0.smithi142.stdout:4/980: rmdir d0/d9/db2/dba/dbc/d6d/d6e/d12a/d13d 0 2024-03-29T04:45:15.291 INFO:tasks.workunit.client.1.smithi152.stdout:7/398: write d7/d8/db/f42 [1818890,22389] 0 2024-03-29T04:45:15.291 INFO:tasks.workunit.client.0.smithi142.stdout:1/941: creat d8/db/dc/d77/d31/f140 x:0 0 0 2024-03-29T04:45:15.292 INFO:tasks.workunit.client.1.smithi152.stdout:3/374: mkdir d7/d1c/d41/d61/d63 0 2024-03-29T04:45:15.294 INFO:tasks.workunit.client.0.smithi142.stdout:1/942: chown d8/db/dc/d77/de9 1 1 2024-03-29T04:45:15.294 INFO:tasks.workunit.client.1.smithi152.stdout:2/311: mknod d0/d3/df/d2c/d45/c6a 0 2024-03-29T04:45:15.295 INFO:tasks.workunit.client.1.smithi152.stdout:2/312: dread - d0/d3/df/d2c/d45/f4e zero size 2024-03-29T04:45:15.298 INFO:tasks.workunit.client.1.smithi152.stdout:7/399: rename d7/d8/d12/d35/d68/f73 to d7/d8/d12/d15/f76 0 2024-03-29T04:45:15.300 INFO:tasks.workunit.client.0.smithi142.stdout:1/943: stat d8/db/dc/d77/db9/dbe/d1f/l92 0 2024-03-29T04:45:15.300 INFO:tasks.workunit.client.1.smithi152.stdout:7/400: chown d7/d8/fa 1983358 1 2024-03-29T04:45:15.300 INFO:tasks.workunit.client.1.smithi152.stdout:3/375: unlink d7/d46/c4a 0 2024-03-29T04:45:15.301 INFO:tasks.workunit.client.1.smithi152.stdout:2/313: unlink d0/d3/df/d2c/d45/f47 0 2024-03-29T04:45:15.303 INFO:tasks.workunit.client.1.smithi152.stdout:3/376: dread d7/d19/f25 [0,4194304] 0 2024-03-29T04:45:15.303 INFO:tasks.workunit.client.0.smithi142.stdout:1/944: chown d8/f2b 83914591 1 2024-03-29T04:45:15.303 INFO:tasks.workunit.client.0.smithi142.stdout:4/981: dwrite d0/dc/f33 [4194304,4194304] 0 2024-03-29T04:45:15.303 INFO:tasks.workunit.client.1.smithi152.stdout:7/401: fsync d7/d8/d12/d1d/f49 0 2024-03-29T04:45:15.305 INFO:tasks.workunit.client.0.smithi142.stdout:2/973: dwrite de/d11/d17/d10b/d22/d27/dba/d61/f78 [0,4194304] 0 2024-03-29T04:45:15.306 INFO:tasks.workunit.client.0.smithi142.stdout:4/982: chown d0/dc/d15/l2a 3259 1 2024-03-29T04:45:15.308 INFO:tasks.workunit.client.0.smithi142.stdout:1/945: chown d8/db/dc/d77/de9/f123 68049206 1 2024-03-29T04:45:15.308 INFO:tasks.workunit.client.0.smithi142.stdout:9/960: dwrite d0/d2/d61/f55 [0,4194304] 0 2024-03-29T04:45:15.310 INFO:tasks.workunit.client.0.smithi142.stdout:9/961: stat d0/d12/d10e/dc7/l111 0 2024-03-29T04:45:15.312 INFO:tasks.workunit.client.0.smithi142.stdout:1/946: chown d8/db/dc/d77/d31/d4d/d56/f118 0 1 2024-03-29T04:45:15.314 INFO:tasks.workunit.client.1.smithi152.stdout:3/377: link d7/d19/d1b/d1e/d22/f4d d7/d46/f64 0 2024-03-29T04:45:15.315 INFO:tasks.workunit.client.0.smithi142.stdout:4/983: creat d0/d9/d11/d97/dff/f143 x:0 0 0 2024-03-29T04:45:15.316 INFO:tasks.workunit.client.0.smithi142.stdout:9/962: creat d0/d12/d10e/d3c/d6f/f134 x:0 0 0 2024-03-29T04:45:15.319 INFO:tasks.workunit.client.0.smithi142.stdout:1/947: chown d8/db/cd3 4188339 1 2024-03-29T04:45:15.319 INFO:tasks.workunit.client.0.smithi142.stdout:9/963: readlink d0/d12/d10e/l88 0 2024-03-29T04:45:15.320 INFO:tasks.workunit.client.1.smithi152.stdout:1/552: dwrite d5/d7/d1b/d2e/f9e [0,4194304] 0 2024-03-29T04:45:15.323 INFO:tasks.workunit.client.0.smithi142.stdout:4/984: mknod d0/d9/d11/d4c/d5f/d81/c144 0 2024-03-29T04:45:15.323 INFO:tasks.workunit.client.1.smithi152.stdout:7/402: rename d7/d8/d12/d1d/d41/d45/l4a to d7/d8/l77 0 2024-03-29T04:45:15.325 INFO:tasks.workunit.client.0.smithi142.stdout:4/985: dread - d0/d9/db2/dba/dbc/daf/f127 zero size 2024-03-29T04:45:15.325 INFO:tasks.workunit.client.0.smithi142.stdout:6/982: sync 2024-03-29T04:45:15.325 INFO:tasks.workunit.client.1.smithi152.stdout:7/403: write d7/d8/db/df/f2b [4742387,88964] 0 2024-03-29T04:45:15.326 INFO:tasks.workunit.client.1.smithi152.stdout:3/378: symlink d7/d46/l65 0 2024-03-29T04:45:15.330 INFO:tasks.workunit.client.0.smithi142.stdout:4/986: mknod d0/d9/d11/d4c/d5f/d81/dc9/d47/dea/d136/c145 0 2024-03-29T04:45:15.331 INFO:tasks.workunit.client.0.smithi142.stdout:3/800: dread d3/d1f/d24/d47/f4f [0,4194304] 0 2024-03-29T04:45:15.332 INFO:tasks.workunit.client.0.smithi142.stdout:9/964: write d0/d2/d61/d64/dee/d10d/d83/f11b [127754,20691] 0 2024-03-29T04:45:15.332 INFO:tasks.workunit.client.0.smithi142.stdout:6/983: symlink d1c/d33/d46/d86/l13d 0 2024-03-29T04:45:15.332 INFO:tasks.workunit.client.0.smithi142.stdout:2/974: link de/d11/d2b/d3e/fcf de/d11/d17/d4a/d79/d12b/f142 0 2024-03-29T04:45:15.333 INFO:tasks.workunit.client.0.smithi142.stdout:4/987: write d0/d9/d11/f23 [2237802,116048] 0 2024-03-29T04:45:15.335 INFO:tasks.workunit.client.0.smithi142.stdout:3/801: creat d3/d1f/d24/d47/d87/f10f x:0 0 0 2024-03-29T04:45:15.337 INFO:tasks.workunit.client.1.smithi152.stdout:1/553: rename d5/d7/d9 to d5/d7/d1b/d2e/d95/dad 0 2024-03-29T04:45:15.339 INFO:tasks.workunit.client.1.smithi152.stdout:7/404: dread d7/d8/d12/d1d/d30/f3c [0,4194304] 0 2024-03-29T04:45:15.340 INFO:tasks.workunit.client.0.smithi142.stdout:3/802: unlink d3/d1f/d24/d47/c9c 0 2024-03-29T04:45:15.340 INFO:tasks.workunit.client.1.smithi152.stdout:7/405: chown d7/d8/d12/d15 8 1 2024-03-29T04:45:15.341 INFO:tasks.workunit.client.1.smithi152.stdout:7/406: readlink d7/d8/d12/d1d/d41/l48 0 2024-03-29T04:45:15.343 INFO:tasks.workunit.client.1.smithi152.stdout:3/379: mknod d7/d1c/d41/d61/d63/c66 0 2024-03-29T04:45:15.344 INFO:tasks.workunit.client.0.smithi142.stdout:4/988: dwrite d0/dc/d15/d3d/d65/f7c [0,4194304] 0 2024-03-29T04:45:15.344 INFO:tasks.workunit.client.1.smithi152.stdout:1/554: dwrite d5/d7/d8/fb [0,4194304] 0 2024-03-29T04:45:15.344 INFO:tasks.workunit.client.1.smithi152.stdout:7/407: symlink d7/d8/d12/d35/l78 0 2024-03-29T04:45:15.346 INFO:tasks.workunit.client.1.smithi152.stdout:7/408: fsync d7/d8/f2d 0 2024-03-29T04:45:15.346 INFO:tasks.workunit.client.0.smithi142.stdout:1/948: write d8/db/dc/d77/db9/dbe/d1f/d4c/fb1 [546052,50046] 0 2024-03-29T04:45:15.349 INFO:tasks.workunit.client.0.smithi142.stdout:1/949: read d8/db/dc/d77/d5a/fe6 [1869081,39663] 0 2024-03-29T04:45:15.349 INFO:tasks.workunit.client.1.smithi152.stdout:3/380: write d7/d19/d1b/f58 [585813,28438] 0 2024-03-29T04:45:15.350 INFO:tasks.workunit.client.1.smithi152.stdout:4/314: truncate d0/f15 3712573 0 2024-03-29T04:45:15.355 INFO:tasks.workunit.client.0.smithi142.stdout:3/803: write d3/d1f/d59/f6e [48940,75952] 0 2024-03-29T04:45:15.357 INFO:tasks.workunit.client.0.smithi142.stdout:9/965: dwrite d0/d12/f24 [0,4194304] 0 2024-03-29T04:45:15.359 INFO:tasks.workunit.client.0.smithi142.stdout:9/966: chown d0/d12/d50/d52/fb4 4647 1 2024-03-29T04:45:15.362 INFO:tasks.workunit.client.0.smithi142.stdout:3/804: unlink d3/d29/d2d/d56/l73 0 2024-03-29T04:45:15.365 INFO:tasks.workunit.client.1.smithi152.stdout:2/314: truncate d0/d3/df/f23 3333659 0 2024-03-29T04:45:15.367 INFO:tasks.workunit.client.0.smithi142.stdout:4/989: write d0/d9/db2/dba/dbc/d6d/d6e/fec [370293,72761] 0 2024-03-29T04:45:15.367 INFO:tasks.workunit.client.0.smithi142.stdout:1/950: rmdir d8/db/dc/d77/db9/dbe/d2d/d44/d7f/d10e/d133/d138 0 2024-03-29T04:45:15.367 INFO:tasks.workunit.client.1.smithi152.stdout:2/315: write d0/d3/f31 [2417899,62664] 0 2024-03-29T04:45:15.369 INFO:tasks.workunit.client.1.smithi152.stdout:3/381: rmdir d7/d19/d3a/d5d 0 2024-03-29T04:45:15.374 INFO:tasks.workunit.client.0.smithi142.stdout:4/990: write d0/dc/d15/d3d/d65/f105 [2770997,102417] 0 2024-03-29T04:45:15.379 INFO:tasks.workunit.client.1.smithi152.stdout:3/382: creat d7/d1c/d41/d61/d63/f67 x:0 0 0 2024-03-29T04:45:15.379 INFO:tasks.workunit.client.0.smithi142.stdout:3/805: link d3/d1f/d24/d47/d87/le5 d3/d29/d2d/de9/l110 0 2024-03-29T04:45:15.379 INFO:tasks.workunit.client.0.smithi142.stdout:1/951: dwrite d8/db/dc/d77/db9/dbe/d2d/d8b/f124 [0,4194304] 0 2024-03-29T04:45:15.380 INFO:tasks.workunit.client.0.smithi142.stdout:3/806: chown d3/d1f/de6 113490 1 2024-03-29T04:45:15.380 INFO:tasks.workunit.client.1.smithi152.stdout:2/316: creat d0/d3/d30/d15/d22/d4c/d5a/f6b x:0 0 0 2024-03-29T04:45:15.381 INFO:tasks.workunit.client.0.smithi142.stdout:9/967: write d0/d12/d10e/d3c/d4f/fd6 [421441,14579] 0 2024-03-29T04:45:15.381 INFO:tasks.workunit.client.1.smithi152.stdout:1/555: rmdir d5/d7/d1b/d2e/d95/dad 39 2024-03-29T04:45:15.381 INFO:tasks.workunit.client.1.smithi152.stdout:2/317: truncate d0/d3/f42 662116 0 2024-03-29T04:45:15.383 INFO:tasks.workunit.client.1.smithi152.stdout:1/556: chown d5/c4e 9 1 2024-03-29T04:45:15.383 INFO:tasks.workunit.client.0.smithi142.stdout:4/991: rename d0/d9/d11/d4c/d5f/d81/dc9/d47/dea/f142 to d0/dc/d15/d3d/d65/f146 0 2024-03-29T04:45:15.387 INFO:tasks.workunit.client.0.smithi142.stdout:9/968: creat d0/d2/d61/df7/f135 x:0 0 0 2024-03-29T04:45:15.388 INFO:tasks.workunit.client.1.smithi152.stdout:3/383: creat d7/d19/d1b/d59/f68 x:0 0 0 2024-03-29T04:45:15.391 INFO:tasks.workunit.client.0.smithi142.stdout:3/807: truncate d3/d1f/d24/d47/d87/fa3 3984982 0 2024-03-29T04:45:15.391 INFO:tasks.workunit.client.1.smithi152.stdout:3/384: symlink d7/d46/l69 0 2024-03-29T04:45:15.392 INFO:tasks.workunit.client.0.smithi142.stdout:3/808: chown d3/d1f/d59/fb3 215 1 2024-03-29T04:45:15.393 INFO:tasks.workunit.client.1.smithi152.stdout:1/557: rename d5/d7/d8/d13/l87 to d5/d7/d1b/d2e/d95/dad/d59/lae 0 2024-03-29T04:45:15.393 INFO:tasks.workunit.client.0.smithi142.stdout:1/952: rmdir d8/db/dc/d19/d134 0 2024-03-29T04:45:15.395 INFO:tasks.workunit.client.1.smithi152.stdout:7/409: truncate d7/d8/db/f42 2006034 0 2024-03-29T04:45:15.396 INFO:tasks.workunit.client.1.smithi152.stdout:3/385: dread d7/f10 [0,4194304] 0 2024-03-29T04:45:15.398 INFO:tasks.workunit.client.1.smithi152.stdout:1/558: symlink d5/d7/d1b/d2e/d95/d9d/da7/laf 0 2024-03-29T04:45:15.399 INFO:tasks.workunit.client.0.smithi142.stdout:9/969: creat d0/f136 x:0 0 0 2024-03-29T04:45:15.399 INFO:tasks.workunit.client.0.smithi142.stdout:4/992: getdents d0/d9/db2/dba/dbc/d8d/da5/d102 0 2024-03-29T04:45:15.402 INFO:tasks.workunit.client.0.smithi142.stdout:3/809: dwrite d3/d1f/d24/fa5 [0,4194304] 0 2024-03-29T04:45:15.402 INFO:tasks.workunit.client.1.smithi152.stdout:4/315: truncate d0/d7/d25/f21 3153743 0 2024-03-29T04:45:15.406 INFO:tasks.workunit.client.0.smithi142.stdout:9/970: readlink d0/d2/d61/d64/dee/d10d/d83/l114 0 2024-03-29T04:45:15.406 INFO:tasks.workunit.client.0.smithi142.stdout:1/953: dwrite d8/db/dc/d77/db9/f122 [0,4194304] 0 2024-03-29T04:45:15.407 INFO:tasks.workunit.client.0.smithi142.stdout:9/971: fsync d0/d2/d61/f131 0 2024-03-29T04:45:15.410 INFO:tasks.workunit.client.0.smithi142.stdout:4/993: creat d0/d9/d11/d4c/d5f/d81/dc9/d47/dea/f147 x:0 0 0 2024-03-29T04:45:15.412 INFO:tasks.workunit.client.1.smithi152.stdout:2/318: truncate d0/d3/d30/f1a 2457481 0 2024-03-29T04:45:15.414 INFO:tasks.workunit.client.0.smithi142.stdout:4/994: mkdir d0/d9/d19/d2c/d148 0 2024-03-29T04:45:15.415 INFO:tasks.workunit.client.1.smithi152.stdout:7/410: creat d7/d8/d12/d15/d72/f79 x:0 0 0 2024-03-29T04:45:15.417 INFO:tasks.workunit.client.0.smithi142.stdout:9/972: read d0/d2/d6/f81 [3981030,90676] 0 2024-03-29T04:45:15.417 INFO:tasks.workunit.client.0.smithi142.stdout:1/954: getdents d8/db/dc/d77/db9/dbe/d2d/d44/d7f/d10e 0 2024-03-29T04:45:15.418 INFO:tasks.workunit.client.1.smithi152.stdout:4/316: creat d0/d7/d40/d2a/f63 x:0 0 0 2024-03-29T04:45:15.419 INFO:tasks.workunit.client.0.smithi142.stdout:9/973: chown d0/d2/d61/d64/fe3 5437 1 2024-03-29T04:45:15.421 INFO:tasks.workunit.client.0.smithi142.stdout:4/995: link d0/d9/db2/dba/dbc/d6d/d6e/cf5 d0/d9/db2/dba/dbc/d6d/c149 0 2024-03-29T04:45:15.423 INFO:tasks.workunit.client.1.smithi152.stdout:7/411: dwrite d7/d8/d12/d1d/d41/f2c [0,4194304] 0 2024-03-29T04:45:15.423 INFO:tasks.workunit.client.0.smithi142.stdout:9/974: getdents d0/d2/d61/d64/dfa 0 2024-03-29T04:45:15.424 INFO:tasks.workunit.client.0.smithi142.stdout:4/996: creat d0/d9/d11/d84/f14a x:0 0 0 2024-03-29T04:45:15.425 INFO:tasks.workunit.client.1.smithi152.stdout:1/559: creat d5/d7/d38/d6c/fb0 x:0 0 0 2024-03-29T04:45:15.425 INFO:tasks.workunit.client.0.smithi142.stdout:1/955: getdents d8/db/dc/d77/d31 0 2024-03-29T04:45:15.425 INFO:tasks.workunit.client.0.smithi142.stdout:4/997: write d0/dc/f33 [7011902,87082] 0 2024-03-29T04:45:15.431 INFO:tasks.workunit.client.0.smithi142.stdout:4/998: symlink d0/d9/d11/d4c/d5f/d81/dc9/l14b 0 2024-03-29T04:45:15.435 INFO:tasks.workunit.client.1.smithi152.stdout:7/412: getdents d7/d8/d12/d1d/d30 0 2024-03-29T04:45:15.436 INFO:tasks.workunit.client.1.smithi152.stdout:3/386: dwrite d7/d46/f64 [0,4194304] 0 2024-03-29T04:45:15.436 INFO:tasks.workunit.client.1.smithi152.stdout:2/319: dwrite d0/d3/df/d2c/f55 [4194304,4194304] 0 2024-03-29T04:45:15.439 INFO:tasks.workunit.client.1.smithi152.stdout:3/387: mkdir d7/d1c/d41/d6a 0 2024-03-29T04:45:15.441 INFO:tasks.workunit.client.1.smithi152.stdout:3/388: readlink d7/d46/l65 0 2024-03-29T04:45:15.443 INFO:tasks.workunit.client.1.smithi152.stdout:7/413: unlink d7/d8/l77 0 2024-03-29T04:45:15.447 INFO:tasks.workunit.client.0.smithi142.stdout:4/999: dread d0/f14 [0,4194304] 0 2024-03-29T04:45:15.448 INFO:tasks.workunit.client.1.smithi152.stdout:2/320: dread d0/d3/df/f43 [0,4194304] 0 2024-03-29T04:45:15.449 INFO:tasks.workunit.client.1.smithi152.stdout:3/389: dread d7/d19/d3a/f49 [0,4194304] 0 2024-03-29T04:45:15.451 INFO:tasks.workunit.client.0.smithi142.stdout:1/956: dread d8/db/f20 [0,4194304] 0 2024-03-29T04:45:15.452 INFO:tasks.workunit.client.0.smithi142.stdout:3/810: dwrite d3/d29/d2b/d40/d67/d8b/d6/f68 [0,4194304] 0 2024-03-29T04:45:15.453 INFO:tasks.workunit.client.1.smithi152.stdout:3/390: symlink d7/d19/d1b/d1e/d4e/l6b 0 2024-03-29T04:45:15.455 INFO:tasks.workunit.client.1.smithi152.stdout:3/391: dread - d7/d19/f57 zero size 2024-03-29T04:45:15.457 INFO:tasks.workunit.client.1.smithi152.stdout:2/321: dwrite d0/d3/df/d2c/f69 [0,4194304] 0 2024-03-29T04:45:15.458 INFO:tasks.workunit.client.0.smithi142.stdout:9/975: dwrite d0/d2/d61/d64/dee/d10d/f49 [0,4194304] 0 2024-03-29T04:45:15.461 INFO:tasks.workunit.client.0.smithi142.stdout:1/957: getdents d8/db/dc/d77/db9/dbe/d1f/d97 0 2024-03-29T04:45:15.466 INFO:tasks.workunit.client.1.smithi152.stdout:4/317: write d0/f15 [3955116,53523] 0 2024-03-29T04:45:15.468 INFO:tasks.workunit.client.0.smithi142.stdout:1/958: creat d8/db/dc/d77/d31/d4d/f141 x:0 0 0 2024-03-29T04:45:15.471 INFO:tasks.workunit.client.1.smithi152.stdout:1/560: dread d5/d7/d8/f1e [0,4194304] 0 2024-03-29T04:45:15.474 INFO:tasks.workunit.client.1.smithi152.stdout:4/318: dwrite d0/d7/d40/d23/f28 [0,4194304] 0 2024-03-29T04:45:15.479 INFO:tasks.workunit.client.0.smithi142.stdout:3/811: dwrite d3/d1f/d24/d47/f90 [0,4194304] 0 2024-03-29T04:45:15.480 INFO:tasks.workunit.client.0.smithi142.stdout:3/812: chown d3/d29/d2b/d6a 113 1 2024-03-29T04:45:15.481 INFO:tasks.workunit.client.1.smithi152.stdout:4/319: dread d0/d7/d25/d17/f3b [0,4194304] 0 2024-03-29T04:45:15.482 INFO:tasks.workunit.client.1.smithi152.stdout:4/320: write d0/f15 [3700399,43037] 0 2024-03-29T04:45:15.483 INFO:tasks.workunit.client.1.smithi152.stdout:4/321: fdatasync d0/f13 0 2024-03-29T04:45:15.485 INFO:tasks.workunit.client.1.smithi152.stdout:4/322: write d0/d7/d25/d17/f54 [1082961,38486] 0 2024-03-29T04:45:15.487 INFO:tasks.workunit.client.0.smithi142.stdout:3/813: link d3/d29/d2b/d40/d67/d8b/d4d/f60 d3/d29/d8f/f111 0 2024-03-29T04:45:15.489 INFO:tasks.workunit.client.0.smithi142.stdout:3/814: creat d3/d1f/de6/df1/f112 x:0 0 0 2024-03-29T04:45:15.491 INFO:tasks.workunit.client.0.smithi142.stdout:3/815: symlink d3/d1f/d24/l113 0 2024-03-29T04:45:15.493 INFO:tasks.workunit.client.1.smithi152.stdout:4/323: dwrite d0/d7/d40/d2a/f63 [0,4194304] 0 2024-03-29T04:45:15.494 INFO:tasks.workunit.client.0.smithi142.stdout:3/816: mknod d3/d29/d2b/d6a/df4/c114 0 2024-03-29T04:45:15.494 INFO:tasks.workunit.client.0.smithi142.stdout:3/817: fdatasync d3/d29/d2b/f7d 0 2024-03-29T04:45:15.497 INFO:tasks.workunit.client.1.smithi152.stdout:2/322: dread d0/f16 [0,4194304] 0 2024-03-29T04:45:15.499 INFO:tasks.workunit.client.1.smithi152.stdout:4/324: write d0/d27/f34 [1171395,117046] 0 2024-03-29T04:45:15.499 INFO:tasks.workunit.client.0.smithi142.stdout:1/959: dread d8/db/dc/d77/db9/dbe/d15/f16 [0,4194304] 0 2024-03-29T04:45:15.500 INFO:tasks.workunit.client.1.smithi152.stdout:8/349: sync 2024-03-29T04:45:15.500 INFO:tasks.workunit.client.0.smithi142.stdout:9/976: dread d0/d12/d10e/dc7/f58 [0,4194304] 0 2024-03-29T04:45:15.502 INFO:tasks.workunit.client.0.smithi142.stdout:1/960: truncate d8/db/dc/d77/de9/f11c 5322419 0 2024-03-29T04:45:15.503 INFO:tasks.workunit.client.0.smithi142.stdout:9/977: unlink d0/f136 0 2024-03-29T04:45:15.504 INFO:tasks.workunit.client.1.smithi152.stdout:4/325: creat d0/f64 x:0 0 0 2024-03-29T04:45:15.505 INFO:tasks.workunit.client.1.smithi152.stdout:8/350: creat d1/f7c x:0 0 0 2024-03-29T04:45:15.508 INFO:tasks.workunit.client.1.smithi152.stdout:8/351: dread - d1/d21/d40/d47/f79 zero size 2024-03-29T04:45:15.508 INFO:tasks.workunit.client.0.smithi142.stdout:9/978: fdatasync d0/d12/d10e/dc7/f3b 0 2024-03-29T04:45:15.509 INFO:tasks.workunit.client.1.smithi152.stdout:2/323: dwrite d0/f44 [0,4194304] 0 2024-03-29T04:45:15.511 INFO:tasks.workunit.client.1.smithi152.stdout:1/561: dwrite d5/d7/d8/d13/d19/f65 [0,4194304] 0 2024-03-29T04:45:15.511 INFO:tasks.workunit.client.1.smithi152.stdout:8/352: stat d1/la 0 2024-03-29T04:45:15.512 INFO:tasks.workunit.client.0.smithi142.stdout:9/979: mknod d0/d2/d61/c137 0 2024-03-29T04:45:15.514 INFO:tasks.workunit.client.1.smithi152.stdout:2/324: chown d0/d3/df/f23 6996 1 2024-03-29T04:45:15.515 INFO:tasks.workunit.client.1.smithi152.stdout:2/325: read - d0/d3/d5f/f63 zero size 2024-03-29T04:45:15.517 INFO:tasks.workunit.client.1.smithi152.stdout:4/326: dwrite d0/d7/d40/d23/f28 [0,4194304] 0 2024-03-29T04:45:15.518 INFO:tasks.workunit.client.1.smithi152.stdout:4/327: chown d0/d7/d25 665 1 2024-03-29T04:45:15.532 INFO:tasks.workunit.client.1.smithi152.stdout:1/562: mkdir d5/d7/d4d/db1 0 2024-03-29T04:45:15.532 INFO:tasks.workunit.client.0.smithi142.stdout:9/980: creat d0/d12/d10e/d3c/d4f/f138 x:0 0 0 2024-03-29T04:45:15.533 INFO:tasks.workunit.client.0.smithi142.stdout:9/981: dread - d0/d12/d10e/d3c/ff9 zero size 2024-03-29T04:45:15.535 INFO:tasks.workunit.client.1.smithi152.stdout:2/326: unlink d0/d3/f39 0 2024-03-29T04:45:15.536 INFO:tasks.workunit.client.1.smithi152.stdout:4/328: mkdir d0/d7/d25/d17/d65 0 2024-03-29T04:45:15.537 INFO:tasks.workunit.client.1.smithi152.stdout:8/353: mknod d1/d2/d6d/c7d 0 2024-03-29T04:45:15.538 INFO:tasks.workunit.client.0.smithi142.stdout:9/982: rename d0/fe1 to d0/d2/dcf/d109/d130/f139 0 2024-03-29T04:45:15.538 INFO:tasks.workunit.client.1.smithi152.stdout:8/354: write d1/d2/d3/f44 [5095063,71261] 0 2024-03-29T04:45:15.541 INFO:tasks.workunit.client.1.smithi152.stdout:4/329: fdatasync d0/d7/d25/f3f 0 2024-03-29T04:45:15.542 INFO:tasks.workunit.client.0.smithi142.stdout:9/983: dread d0/d12/d10e/d3c/f69 [0,4194304] 0 2024-03-29T04:45:15.542 INFO:tasks.workunit.client.1.smithi152.stdout:4/330: readlink d0/d7/d25/d48/d57/l41 0 2024-03-29T04:45:15.543 INFO:tasks.workunit.client.1.smithi152.stdout:1/563: link d5/d7/d38/f6f d5/d7/d8/d13/fb2 0 2024-03-29T04:45:15.543 INFO:tasks.workunit.client.1.smithi152.stdout:2/327: dread d0/d3/f18 [0,4194304] 0 2024-03-29T04:45:15.544 INFO:tasks.workunit.client.0.smithi142.stdout:9/984: dread d0/d12/d10e/d3c/d6f/fcb [0,4194304] 0 2024-03-29T04:45:15.545 INFO:tasks.workunit.client.1.smithi152.stdout:2/328: write d0/d3/d30/d15/f25 [4343496,12683] 0 2024-03-29T04:45:15.545 INFO:tasks.workunit.client.1.smithi152.stdout:8/355: symlink d1/d2/d1d/d36/d4b/l7e 0 2024-03-29T04:45:15.546 INFO:tasks.workunit.client.1.smithi152.stdout:8/356: dread - d1/d2/d3/f4e zero size 2024-03-29T04:45:15.547 INFO:tasks.workunit.client.0.smithi142.stdout:9/985: mkdir d0/d2/d61/d64/d104/d13a 0 2024-03-29T04:45:15.548 INFO:tasks.workunit.client.0.smithi142.stdout:9/986: symlink d0/d6c/l13b 0 2024-03-29T04:45:15.549 INFO:tasks.workunit.client.0.smithi142.stdout:9/987: chown d0/d12/d50/d52/f103 125092 1 2024-03-29T04:45:15.550 INFO:tasks.workunit.client.0.smithi142.stdout:9/988: write d0/d12/f54 [2841110,66353] 0 2024-03-29T04:45:15.552 INFO:tasks.workunit.client.1.smithi152.stdout:8/357: mkdir d1/d21/d7f 0 2024-03-29T04:45:15.554 INFO:tasks.workunit.client.1.smithi152.stdout:2/329: dwrite d0/d3/d30/d15/d22/f49 [0,4194304] 0 2024-03-29T04:45:15.558 INFO:tasks.workunit.client.1.smithi152.stdout:8/358: creat d1/d2/d3/f80 x:0 0 0 2024-03-29T04:45:15.558 INFO:tasks.workunit.client.0.smithi142.stdout:9/989: dwrite d0/d12/d50/ff8 [0,4194304] 0 2024-03-29T04:45:15.558 INFO:tasks.workunit.client.0.smithi142.stdout:1/961: dwrite d8/db/dc/d77/d5a/fe6 [0,4194304] 0 2024-03-29T04:45:15.559 INFO:tasks.workunit.client.1.smithi152.stdout:2/330: dread d0/d3/f42 [0,4194304] 0 2024-03-29T04:45:15.561 INFO:tasks.workunit.client.0.smithi142.stdout:1/962: write d8/db/dc/d77/d5a/fe6 [1299650,6559] 0 2024-03-29T04:45:15.563 INFO:tasks.workunit.client.0.smithi142.stdout:1/963: chown d8/db/dc/d77/d5a/fc6 3162 1 2024-03-29T04:45:15.564 INFO:tasks.workunit.client.1.smithi152.stdout:1/564: truncate d5/d7/d6a/f6e 3910327 0 2024-03-29T04:45:15.566 INFO:tasks.workunit.client.1.smithi152.stdout:2/331: symlink d0/d3/l6c 0 2024-03-29T04:45:15.568 INFO:tasks.workunit.client.1.smithi152.stdout:1/565: creat d5/d7/d6a/fb3 x:0 0 0 2024-03-29T04:45:15.568 INFO:tasks.workunit.client.0.smithi142.stdout:9/990: rmdir d0 39 2024-03-29T04:45:15.569 INFO:tasks.workunit.client.1.smithi152.stdout:4/331: dwrite d0/d7/d25/f21 [0,4194304] 0 2024-03-29T04:45:15.572 INFO:tasks.workunit.client.0.smithi142.stdout:1/964: read d8/db/dc/d77/d31/d4d/f8f [859094,7035] 0 2024-03-29T04:45:15.573 INFO:tasks.workunit.client.1.smithi152.stdout:2/332: mknod d0/d3/d30/d15/d22/c6d 0 2024-03-29T04:45:15.574 INFO:tasks.workunit.client.0.smithi142.stdout:1/965: chown d8/db/dc/d77/db9/dbe/d2d/d8b/d13f 244537 1 2024-03-29T04:45:15.574 INFO:tasks.workunit.client.0.smithi142.stdout:1/966: stat d8/db/dc/ld9 0 2024-03-29T04:45:15.575 INFO:tasks.workunit.client.1.smithi152.stdout:1/566: creat d5/d7/d1b/d2e/d95/dad/d12/d77/fb4 x:0 0 0 2024-03-29T04:45:15.578 INFO:tasks.workunit.client.1.smithi152.stdout:1/567: write d5/d7/d8/d13/d19/f3d [2067626,14140] 0 2024-03-29T04:45:15.579 INFO:tasks.workunit.client.1.smithi152.stdout:2/333: getdents d0/d3/d30/d15/d53/d54 0 2024-03-29T04:45:15.583 INFO:tasks.workunit.client.0.smithi142.stdout:1/967: dwrite d8/db/dc/d77/db9/dbe/d2d/d8b/dda/fde [0,4194304] 0 2024-03-29T04:45:15.583 INFO:tasks.workunit.client.1.smithi152.stdout:1/568: dread d5/d7/d1b/d2e/d95/dad/f46 [0,4194304] 0 2024-03-29T04:45:15.586 INFO:tasks.workunit.client.1.smithi152.stdout:1/569: mknod d5/d7/d8/d13/d19/d80/cb5 0 2024-03-29T04:45:15.588 INFO:tasks.workunit.client.1.smithi152.stdout:1/570: readlink d5/d7/d1b/d2e/d95/dad/d12/la6 0 2024-03-29T04:45:15.600 INFO:tasks.workunit.client.0.smithi142.stdout:1/968: dwrite d8/db/dc/d77/d5a/f63 [0,4194304] 0 2024-03-29T04:45:15.606 INFO:tasks.workunit.client.0.smithi142.stdout:9/991: dread d0/d2/d6/f81 [0,4194304] 0 2024-03-29T04:45:15.610 INFO:tasks.workunit.client.1.smithi152.stdout:4/332: dread d0/d7/d25/d17/f5b [0,4194304] 0 2024-03-29T04:45:15.610 INFO:tasks.workunit.client.0.smithi142.stdout:9/992: read d0/d2/dcf/d109/d130/f139 [22010,22278] 0 2024-03-29T04:45:15.611 INFO:tasks.workunit.client.0.smithi142.stdout:1/969: dwrite d8/db/dc/d66/f12a [4194304,4194304] 0 2024-03-29T04:45:15.611 INFO:tasks.workunit.client.1.smithi152.stdout:4/333: chown d0/d7/d25/d48/d57/f5c 14 1 2024-03-29T04:45:15.613 INFO:tasks.workunit.client.0.smithi142.stdout:9/993: symlink d0/d12/d10e/d3c/d6f/l13c 0 2024-03-29T04:45:15.615 INFO:tasks.workunit.client.0.smithi142.stdout:9/994: dread - d0/d12/d10e/d3c/ff9 zero size 2024-03-29T04:45:15.615 INFO:tasks.workunit.client.0.smithi142.stdout:1/970: dread - d8/db/dc/d77/db9/dbe/d2d/d44/d4b/fb2 zero size 2024-03-29T04:45:15.616 INFO:tasks.workunit.client.0.smithi142.stdout:9/995: chown d0/d2/d61/d64/dee/d10d 1809 1 2024-03-29T04:45:15.618 INFO:tasks.workunit.client.0.smithi142.stdout:9/996: chown d0/d2/d6/l91 376 1 2024-03-29T04:45:15.619 INFO:tasks.workunit.client.0.smithi142.stdout:1/971: mkdir d8/db/dc/d77/db9/dbd/d142 0 2024-03-29T04:45:15.620 INFO:tasks.workunit.client.0.smithi142.stdout:2/975: sync 2024-03-29T04:45:15.620 INFO:tasks.workunit.client.0.smithi142.stdout:6/984: sync 2024-03-29T04:45:15.622 INFO:tasks.workunit.client.0.smithi142.stdout:2/976: write de/d11/d17/d4a/d79/d12b/f28 [352267,46049] 0 2024-03-29T04:45:15.622 INFO:tasks.workunit.client.0.smithi142.stdout:6/985: mkdir d1c/d90/d94/d13e 0 2024-03-29T04:45:15.625 INFO:tasks.workunit.client.0.smithi142.stdout:1/972: symlink d8/db/dc/d77/db9/dbe/d2d/d44/l143 0 2024-03-29T04:45:15.629 INFO:tasks.workunit.client.0.smithi142.stdout:6/986: rmdir d1c/d90/d94/d13e 0 2024-03-29T04:45:15.630 INFO:tasks.workunit.client.0.smithi142.stdout:1/973: mknod d8/db/dc/d77/db9/dbe/d2d/d44/d53/d135/c144 0 2024-03-29T04:45:15.630 INFO:tasks.workunit.client.1.smithi152.stdout:4/334: symlink d0/d7/d40/d62/l66 0 2024-03-29T04:45:15.631 INFO:tasks.workunit.client.0.smithi142.stdout:6/987: truncate d1c/dc9/f103 681994 0 2024-03-29T04:45:15.631 INFO:tasks.workunit.client.0.smithi142.stdout:1/974: write d8/db/dc/d66/f12a [5149814,64229] 0 2024-03-29T04:45:15.632 INFO:tasks.workunit.client.0.smithi142.stdout:2/977: symlink de/d11/d17/d5d/l143 0 2024-03-29T04:45:15.634 INFO:tasks.workunit.client.1.smithi152.stdout:4/335: unlink d0/d7/d25/d48/d57/c3e 0 2024-03-29T04:45:15.636 INFO:tasks.workunit.client.0.smithi142.stdout:1/975: symlink d8/l145 0 2024-03-29T04:45:15.636 INFO:tasks.workunit.client.0.smithi142.stdout:6/988: link la d1c/d90/ddc/l13f 0 2024-03-29T04:45:15.636 INFO:tasks.workunit.client.0.smithi142.stdout:2/978: unlink de/d11/d17/d4a/d79/d12b/dd1/ce9 0 2024-03-29T04:45:15.637 INFO:tasks.workunit.client.1.smithi152.stdout:2/334: write d0/d3/d5f/f63 [486873,23254] 0 2024-03-29T04:45:15.640 INFO:tasks.workunit.client.0.smithi142.stdout:2/979: write de/d11/d17/d60/fc6 [3145864,7459] 0 2024-03-29T04:45:15.640 INFO:tasks.workunit.client.0.smithi142.stdout:1/976: creat d8/db/dc/d77/db9/dbe/d2d/d8b/dda/d10a/f146 x:0 0 0 2024-03-29T04:45:15.642 INFO:tasks.workunit.client.0.smithi142.stdout:9/997: dread d0/f4b [0,4194304] 0 2024-03-29T04:45:15.644 INFO:tasks.workunit.client.0.smithi142.stdout:2/980: rmdir de/d11/d2b/d3e 39 2024-03-29T04:45:15.645 INFO:tasks.workunit.client.0.smithi142.stdout:6/989: dwrite d1c/dc9/f103 [0,4194304] 0 2024-03-29T04:45:15.650 INFO:tasks.workunit.client.0.smithi142.stdout:6/990: mknod d1c/d33/d46/d66/dcd/d11c/c140 0 2024-03-29T04:45:15.651 INFO:tasks.workunit.client.0.smithi142.stdout:3/818: sync 2024-03-29T04:45:15.652 INFO:tasks.workunit.client.0.smithi142.stdout:6/991: fdatasync d1c/d33/d46/d61/d7e/f109 0 2024-03-29T04:45:15.656 INFO:tasks.workunit.client.0.smithi142.stdout:3/819: rmdir d3/d1f/de6/df1 39 2024-03-29T04:45:15.658 INFO:tasks.workunit.client.0.smithi142.stdout:6/992: truncate d1c/f34 1406525 0 2024-03-29T04:45:15.660 INFO:tasks.workunit.client.0.smithi142.stdout:3/820: dread - d3/d1f/d59/fb2 zero size 2024-03-29T04:45:15.661 INFO:tasks.workunit.client.0.smithi142.stdout:3/821: chown d3/d1f/fbd 1 1 2024-03-29T04:45:15.665 INFO:tasks.workunit.client.0.smithi142.stdout:6/993: dwrite d1c/d33/d46/d66/dc1/f100 [0,4194304] 0 2024-03-29T04:45:15.666 INFO:tasks.workunit.client.0.smithi142.stdout:6/994: fdatasync d1c/d33/d46/d61/fa1 0 2024-03-29T04:45:15.670 INFO:tasks.workunit.client.0.smithi142.stdout:1/977: dread d8/db/dc/d77/db9/dbe/d15/f27 [4194304,4194304] 0 2024-03-29T04:45:15.670 INFO:tasks.workunit.client.0.smithi142.stdout:9/998: dread d0/d2/d61/d64/f108 [0,4194304] 0 2024-03-29T04:45:15.674 INFO:tasks.workunit.client.0.smithi142.stdout:9/999: dread d0/d12/d10e/dc7/daa/d95/fd0 [0,4194304] 0 2024-03-29T04:45:15.676 INFO:tasks.workunit.client.0.smithi142.stdout:1/978: dread d8/db/dc/f2c [0,4194304] 0 2024-03-29T04:45:15.677 INFO:tasks.workunit.client.1.smithi152.stdout:1/571: link d5/d7/d1b/d2e/d95/dad/f10 d5/d7/d8/d13/d19/fb6 0 2024-03-29T04:45:15.677 INFO:tasks.workunit.client.0.smithi142.stdout:3/822: write d3/d29/d2d/d79/fce [917918,97744] 0 2024-03-29T04:45:15.678 INFO:tasks.workunit.client.0.smithi142.stdout:3/823: stat d3/d29/d2b/d6a/l9b 0 2024-03-29T04:45:15.679 INFO:tasks.workunit.client.1.smithi152.stdout:1/572: mknod d5/d7/d1b/d2e/d95/dad/d59/cb7 0 2024-03-29T04:45:15.680 INFO:tasks.workunit.client.1.smithi152.stdout:1/573: write d5/d7/d1b/d2e/f9e [2305956,46990] 0 2024-03-29T04:45:15.680 INFO:tasks.workunit.client.0.smithi142.stdout:1/979: mknod d8/db/dc/d77/d31/d4d/d8c/c147 0 2024-03-29T04:45:15.680 INFO:tasks.workunit.client.0.smithi142.stdout:6/995: dread d1c/f31 [0,4194304] 0 2024-03-29T04:45:15.684 INFO:tasks.workunit.client.1.smithi152.stdout:4/336: creat d0/d7/d40/f67 x:0 0 0 2024-03-29T04:45:15.686 INFO:tasks.workunit.client.0.smithi142.stdout:2/981: dwrite de/d11/d17/d60/de6/d101/d13c/db4/ffe [0,4194304] 0 2024-03-29T04:45:15.690 INFO:tasks.workunit.client.0.smithi142.stdout:2/982: creat de/d11/d17/d60/d6f/d116/f144 x:0 0 0 2024-03-29T04:45:15.734 INFO:tasks.workunit.client.1.smithi152.stdout:1/574: dwrite d5/d7/d1b/d2e/d95/dad/d2b/fab [0,4194304] 0 2024-03-29T04:45:15.738 INFO:tasks.workunit.client.1.smithi152.stdout:4/337: getdents d0/d7/d25/d17/d37 0 2024-03-29T04:45:15.741 INFO:tasks.workunit.client.1.smithi152.stdout:4/338: mknod d0/d7/d40/d62/c68 0 2024-03-29T04:45:15.742 INFO:tasks.workunit.client.1.smithi152.stdout:4/339: stat d0/d7 0 2024-03-29T04:45:15.742 INFO:tasks.workunit.client.1.smithi152.stdout:1/575: dwrite d5/d7/d8/fb [0,4194304] 0 2024-03-29T04:45:15.750 INFO:tasks.workunit.client.0.smithi142.stdout:6/996: truncate d1c/f34 1995644 0 2024-03-29T04:45:15.752 INFO:tasks.workunit.client.0.smithi142.stdout:6/997: creat d1c/d33/d46/d61/d9b/f141 x:0 0 0 2024-03-29T04:45:15.753 INFO:tasks.workunit.client.0.smithi142.stdout:2/983: write de/d11/d17/d60/de6/fef [1543470,46365] 0 2024-03-29T04:45:15.758 INFO:tasks.workunit.client.0.smithi142.stdout:2/984: getdents de/d11/d17/d10b/d22/d27/dba/d61 0 2024-03-29T04:45:15.759 INFO:tasks.workunit.client.1.smithi152.stdout:1/576: write d5/d7/d8/d13/d19/f30 [2966618,36042] 0 2024-03-29T04:45:15.761 INFO:tasks.workunit.client.0.smithi142.stdout:2/985: creat de/d11/d17/d5d/d71/d119/f145 x:0 0 0 2024-03-29T04:45:15.762 INFO:tasks.workunit.client.0.smithi142.stdout:2/986: dread - de/d11/d17/d10b/d22/ff3 zero size 2024-03-29T04:45:15.762 INFO:tasks.workunit.client.1.smithi152.stdout:1/577: rename d5/d7/d6a/fb3 to d5/fb8 0 2024-03-29T04:45:15.763 INFO:tasks.workunit.client.0.smithi142.stdout:2/987: fsync de/f120 0 2024-03-29T04:45:15.764 INFO:tasks.workunit.client.1.smithi152.stdout:1/578: fdatasync d5/d7/d1b/d2e/d95/dad/f7b 0 2024-03-29T04:45:15.765 INFO:tasks.workunit.client.0.smithi142.stdout:6/998: dwrite d1c/d33/d46/f48 [0,4194304] 0 2024-03-29T04:45:15.766 INFO:tasks.workunit.client.1.smithi152.stdout:1/579: mknod d5/d7/d38/d6c/cb9 0 2024-03-29T04:45:15.767 INFO:tasks.workunit.client.0.smithi142.stdout:2/988: truncate de/d11/d17/d60/de6/d101/d13c/f7e 4840900 0 2024-03-29T04:45:15.767 INFO:tasks.workunit.client.1.smithi152.stdout:1/580: stat d5/d7/d4d/l6d 0 2024-03-29T04:45:15.770 INFO:tasks.workunit.client.1.smithi152.stdout:1/581: getdents d5/d7/d8/da5 0 2024-03-29T04:45:15.774 INFO:tasks.workunit.client.0.smithi142.stdout:2/989: dwrite de/d11/d17/f25 [4194304,4194304] 0 2024-03-29T04:45:15.774 INFO:tasks.workunit.client.1.smithi152.stdout:1/582: rename d5/d7/d6a/f79 to d5/d7/d38/fba 0 2024-03-29T04:45:15.776 INFO:tasks.workunit.client.1.smithi152.stdout:1/583: write d5/d7/d8/d13/d19/f65 [3088165,43655] 0 2024-03-29T04:45:15.777 INFO:tasks.workunit.client.1.smithi152.stdout:1/584: chown d5/d7/d38/f7c 3339 1 2024-03-29T04:45:15.778 INFO:tasks.workunit.client.0.smithi142.stdout:2/990: creat de/d11/d17/d4a/f146 x:0 0 0 2024-03-29T04:45:15.778 INFO:tasks.workunit.client.1.smithi152.stdout:1/585: chown d5/d7/d1b/d2e/d95/dad/f73 2 1 2024-03-29T04:45:15.782 INFO:tasks.workunit.client.0.smithi142.stdout:6/999: dwrite d1c/d33/d46/f69 [0,4194304] 0 2024-03-29T04:45:15.789 INFO:tasks.workunit.client.0.smithi142.stdout:2/991: dwrite de/d11/d2b/d3e/fcf [4194304,4194304] 0 2024-03-29T04:45:15.793 INFO:tasks.workunit.client.0.smithi142.stdout:2/992: dread - de/d11/d17/d60/de6/d101/d13c/db4/f125 zero size 2024-03-29T04:45:15.794 INFO:tasks.workunit.client.0.smithi142.stdout:2/993: readlink de/d11/d17/d10b/d22/d27/dba/d61/l70 0 2024-03-29T04:45:15.806 INFO:tasks.workunit.client.1.smithi152.stdout:1/586: dread d5/f3f [0,4194304] 0 2024-03-29T04:45:15.808 INFO:tasks.workunit.client.1.smithi152.stdout:1/587: write d5/d7/d1b/f93 [1136022,78484] 0 2024-03-29T04:45:15.809 INFO:tasks.workunit.client.1.smithi152.stdout:1/588: write d5/d7/d38/f7c [209635,107769] 0 2024-03-29T04:45:15.811 INFO:tasks.workunit.client.1.smithi152.stdout:1/589: rmdir d5/d7/d8/d13/d19/d80 39 2024-03-29T04:45:15.812 INFO:tasks.workunit.client.1.smithi152.stdout:1/590: stat d5/d7/d8/d13/f74 0 2024-03-29T04:45:15.814 INFO:tasks.workunit.client.1.smithi152.stdout:1/591: symlink d5/d7/lbb 0 2024-03-29T04:45:15.815 INFO:tasks.workunit.client.1.smithi152.stdout:1/592: symlink d5/d7/d1b/d2e/d95/dad/d2b/lbc 0 2024-03-29T04:45:15.817 INFO:tasks.workunit.client.1.smithi152.stdout:1/593: creat d5/d7/d1b/d2e/d95/dad/d12/fbd x:0 0 0 2024-03-29T04:45:15.819 INFO:tasks.workunit.client.1.smithi152.stdout:1/594: symlink d5/d7/lbe 0 2024-03-29T04:45:15.819 INFO:tasks.workunit.client.1.smithi152.stdout:1/595: stat d5/f72 0 2024-03-29T04:45:15.820 INFO:tasks.workunit.client.1.smithi152.stdout:1/596: creat d5/d7/d38/d6c/fbf x:0 0 0 2024-03-29T04:45:15.822 INFO:tasks.workunit.client.1.smithi152.stdout:1/597: chown d5/d7/d1b/d2e/d95/dad/d12/c9c 17319232 1 2024-03-29T04:45:15.823 INFO:tasks.workunit.client.1.smithi152.stdout:1/598: mknod d5/d7/d1b/cc0 0 2024-03-29T04:45:15.825 INFO:tasks.workunit.client.1.smithi152.stdout:1/599: creat d5/d7/d8/fc1 x:0 0 0 2024-03-29T04:45:15.827 INFO:tasks.workunit.client.1.smithi152.stdout:1/600: creat d5/d7/fc2 x:0 0 0 2024-03-29T04:45:15.834 INFO:tasks.workunit.client.1.smithi152.stdout:1/601: dread d5/d7/d8/d13/d19/f30 [0,4194304] 0 2024-03-29T04:45:15.837 INFO:tasks.workunit.client.1.smithi152.stdout:1/602: dread d5/d7/d1b/d2e/f9e [0,4194304] 0 2024-03-29T04:45:15.840 INFO:tasks.workunit.client.1.smithi152.stdout:1/603: rename d5/d7/d8/d13/d19/c67 to d5/d7/d8/d13/d96/cc3 0 2024-03-29T04:45:15.841 INFO:tasks.workunit.client.1.smithi152.stdout:1/604: write d5/d7/d1b/d2e/d95/dad/f46 [4035876,78409] 0 2024-03-29T04:45:15.842 INFO:tasks.workunit.client.1.smithi152.stdout:1/605: chown d5/d7/d8/d13/d19/f94 3313001 1 2024-03-29T04:45:15.843 INFO:tasks.workunit.client.1.smithi152.stdout:1/606: write d5/d7/d38/f7c [555481,11753] 0 2024-03-29T04:45:15.847 INFO:tasks.workunit.client.1.smithi152.stdout:1/607: rename d5/d7/d8/d13/c97 to d5/d7/d1b/d2e/d95/cc4 0 2024-03-29T04:45:15.848 INFO:tasks.workunit.client.1.smithi152.stdout:1/608: mkdir d5/d7/d1b/dc5 0 2024-03-29T04:45:15.851 INFO:tasks.workunit.client.1.smithi152.stdout:1/609: dread d5/d7/d1b/d2e/d95/dad/f46 [0,4194304] 0 2024-03-29T04:45:15.853 INFO:tasks.workunit.client.1.smithi152.stdout:9/422: sync 2024-03-29T04:45:15.853 INFO:tasks.workunit.client.1.smithi152.stdout:0/480: sync 2024-03-29T04:45:15.853 INFO:tasks.workunit.client.1.smithi152.stdout:5/440: sync 2024-03-29T04:45:15.853 INFO:tasks.workunit.client.1.smithi152.stdout:6/481: sync 2024-03-29T04:45:15.854 INFO:tasks.workunit.client.1.smithi152.stdout:6/482: dread - d1/f94 zero size 2024-03-29T04:45:15.854 INFO:tasks.workunit.client.1.smithi152.stdout:5/441: dread - d6/d16/f65 zero size 2024-03-29T04:45:15.854 INFO:tasks.workunit.client.1.smithi152.stdout:0/481: stat d2/da/d17/d57/f62 0 2024-03-29T04:45:15.855 INFO:tasks.workunit.client.1.smithi152.stdout:1/610: link d5/d7/d1b/d2e/d95/dad/d59/cb7 d5/d7/d1b/d2e/d95/dad/d2b/cc6 0 2024-03-29T04:45:15.855 INFO:tasks.workunit.client.1.smithi152.stdout:9/423: read d5/d12/d2c/f54 [4290115,54813] 0 2024-03-29T04:45:15.856 INFO:tasks.workunit.client.1.smithi152.stdout:1/611: dread - d5/d7/d1b/d2e/f86 zero size 2024-03-29T04:45:15.857 INFO:tasks.workunit.client.1.smithi152.stdout:5/442: rmdir d6/d16/d25/d2e 39 2024-03-29T04:45:15.858 INFO:tasks.workunit.client.1.smithi152.stdout:6/483: getdents d1/d3/df/d8a/d98 0 2024-03-29T04:45:15.859 INFO:tasks.workunit.client.1.smithi152.stdout:9/424: fdatasync d5/d12/d2c/f8f 0 2024-03-29T04:45:15.860 INFO:tasks.workunit.client.1.smithi152.stdout:6/484: chown d1/d3/d9/d13/d28/d45/c74 868029314 1 2024-03-29T04:45:15.860 INFO:tasks.workunit.client.1.smithi152.stdout:1/612: fdatasync d5/d7/d1b/d2e/d95/dad/f7b 0 2024-03-29T04:45:15.861 INFO:tasks.workunit.client.1.smithi152.stdout:6/485: dread - d1/f9b zero size 2024-03-29T04:45:15.862 INFO:tasks.workunit.client.1.smithi152.stdout:1/613: fsync d5/d7/d8/d13/d19/f68 0 2024-03-29T04:45:15.863 INFO:tasks.workunit.client.1.smithi152.stdout:1/614: readlink d5/d7/d8/d13/d19/l7a 0 2024-03-29T04:45:15.866 INFO:tasks.workunit.client.1.smithi152.stdout:9/425: link d5/db/d1b/c93 d5/d12/d2c/d6e/d73/c96 0 2024-03-29T04:45:15.868 INFO:tasks.workunit.client.1.smithi152.stdout:9/426: chown d5/db/d1b/f49 7874806 1 2024-03-29T04:45:15.869 INFO:tasks.workunit.client.1.smithi152.stdout:6/486: link d1/d3/d9/d13/d28/l76 d1/d3/d9/d13/d28/d45/d8c/la2 0 2024-03-29T04:45:15.871 INFO:tasks.workunit.client.1.smithi152.stdout:9/427: mknod d5/d12/d2c/d6e/c97 0 2024-03-29T04:45:15.871 INFO:tasks.workunit.client.1.smithi152.stdout:1/615: dwrite d5/f3c [0,4194304] 0 2024-03-29T04:45:15.873 INFO:tasks.workunit.client.1.smithi152.stdout:9/428: write d5/f20 [579572,122565] 0 2024-03-29T04:45:15.873 INFO:tasks.workunit.client.1.smithi152.stdout:6/487: rename d1/f50 to d1/d3/df/d84/fa3 0 2024-03-29T04:45:15.876 INFO:tasks.workunit.client.1.smithi152.stdout:0/482: dread d2/d29/f6c [0,4194304] 0 2024-03-29T04:45:15.876 INFO:tasks.workunit.client.1.smithi152.stdout:5/443: dread d6/d16/f4d [0,4194304] 0 2024-03-29T04:45:15.877 INFO:tasks.workunit.client.1.smithi152.stdout:1/616: mkdir d5/d7/d8/d13/d19/dc7 0 2024-03-29T04:45:15.877 INFO:tasks.workunit.client.1.smithi152.stdout:6/488: chown d1/d3 1263 1 2024-03-29T04:45:15.878 INFO:tasks.workunit.client.1.smithi152.stdout:9/429: symlink d5/d12/l98 0 2024-03-29T04:45:15.879 INFO:tasks.workunit.client.1.smithi152.stdout:6/489: write d1/d3/d1a/f61 [5152131,44950] 0 2024-03-29T04:45:15.880 INFO:tasks.workunit.client.1.smithi152.stdout:5/444: creat d6/d16/d25/d7e/f90 x:0 0 0 2024-03-29T04:45:15.882 INFO:tasks.workunit.client.1.smithi152.stdout:0/483: rename d2/da/d17/d50/d5c to d2/da/df/d86/db8 0 2024-03-29T04:45:15.884 INFO:tasks.workunit.client.1.smithi152.stdout:5/445: dread d6/d16/d25/d7e/f69 [0,4194304] 0 2024-03-29T04:45:15.884 INFO:tasks.workunit.client.1.smithi152.stdout:9/430: unlink d5/db/d1b/f2b 0 2024-03-29T04:45:15.886 INFO:tasks.workunit.client.1.smithi152.stdout:6/490: unlink d1/d3/d9/d13/d28/d45/c5b 0 2024-03-29T04:45:15.887 INFO:tasks.workunit.client.1.smithi152.stdout:1/617: dwrite d5/d7/d1b/d2e/d95/dad/f73 [0,4194304] 0 2024-03-29T04:45:15.888 INFO:tasks.workunit.client.1.smithi152.stdout:0/484: mknod d2/da/d3a/d6f/cb9 0 2024-03-29T04:45:15.888 INFO:tasks.workunit.client.1.smithi152.stdout:6/491: write d1/d3/df/d84/f57 [159052,128057] 0 2024-03-29T04:45:15.889 INFO:tasks.workunit.client.1.smithi152.stdout:1/618: read d5/d7/d8/f2d [1196449,75837] 0 2024-03-29T04:45:15.890 INFO:tasks.workunit.client.1.smithi152.stdout:5/446: creat d6/d1b/f91 x:0 0 0 2024-03-29T04:45:15.892 INFO:tasks.workunit.client.1.smithi152.stdout:6/492: fdatasync d1/d3/d1a/d1b/d29/f71 0 2024-03-29T04:45:15.892 INFO:tasks.workunit.client.1.smithi152.stdout:1/619: creat d5/d7/d4d/fc8 x:0 0 0 2024-03-29T04:45:15.893 INFO:tasks.workunit.client.1.smithi152.stdout:9/431: link d5/d12/d2c/d6e/c97 d5/d12/c99 0 2024-03-29T04:45:15.894 INFO:tasks.workunit.client.1.smithi152.stdout:5/447: mkdir d6/d16/d25/d7e/d92 0 2024-03-29T04:45:15.895 INFO:tasks.workunit.client.1.smithi152.stdout:1/620: mknod d5/d7/d1b/cc9 0 2024-03-29T04:45:15.895 INFO:tasks.workunit.client.1.smithi152.stdout:9/432: write d5/db/d1b/f95 [259446,78360] 0 2024-03-29T04:45:15.897 INFO:tasks.workunit.client.1.smithi152.stdout:5/448: unlink d6/d1b/f78 0 2024-03-29T04:45:15.898 INFO:tasks.workunit.client.1.smithi152.stdout:6/493: link d1/d3/df/f2e d1/d3/d9/d13/d28/fa4 0 2024-03-29T04:45:15.899 INFO:tasks.workunit.client.1.smithi152.stdout:1/621: symlink d5/d7/d4d/lca 0 2024-03-29T04:45:15.900 INFO:tasks.workunit.client.1.smithi152.stdout:6/494: fdatasync d1/d3/d1a/d1b/d29/f71 0 2024-03-29T04:45:15.902 INFO:tasks.workunit.client.0.smithi142.stdout:1/980: sync 2024-03-29T04:45:15.903 INFO:tasks.workunit.client.0.smithi142.stdout:3/824: sync 2024-03-29T04:45:15.903 INFO:tasks.workunit.client.0.smithi142.stdout:2/994: sync 2024-03-29T04:45:15.903 INFO:tasks.workunit.client.1.smithi152.stdout:1/622: write d5/d7/d8/fc1 [536770,80523] 0 2024-03-29T04:45:15.903 INFO:tasks.workunit.client.0.smithi142.stdout:1/981: dread - d8/db/dc/d77/d31/f140 zero size 2024-03-29T04:45:15.905 INFO:tasks.workunit.client.1.smithi152.stdout:1/623: stat d5/d7/d6a/f69 0 2024-03-29T04:45:15.905 INFO:tasks.workunit.client.1.smithi152.stdout:9/433: dwrite d5/db/d1b/f4d [4194304,4194304] 0 2024-03-29T04:45:15.905 INFO:tasks.workunit.client.0.smithi142.stdout:2/995: creat de/d11/da1/de2/f147 x:0 0 0 2024-03-29T04:45:15.906 INFO:tasks.workunit.client.1.smithi152.stdout:1/624: chown d5/d7/d6a/f69 252468288 1 2024-03-29T04:45:15.913 INFO:tasks.workunit.client.0.smithi142.stdout:1/982: unlink d8/lc0 0 2024-03-29T04:45:15.913 INFO:tasks.workunit.client.0.smithi142.stdout:3/825: unlink d3/d29/d8f/l9e 0 2024-03-29T04:45:15.915 INFO:tasks.workunit.client.0.smithi142.stdout:1/983: truncate d8/db/dd6/d113/f38 2160595 0 2024-03-29T04:45:15.916 INFO:tasks.workunit.client.1.smithi152.stdout:9/434: dwrite d5/d12/d2c/f39 [0,4194304] 0 2024-03-29T04:45:15.916 INFO:tasks.workunit.client.1.smithi152.stdout:1/625: mknod d5/d7/d8/da5/ccb 0 2024-03-29T04:45:15.923 INFO:tasks.workunit.client.1.smithi152.stdout:5/449: dread d6/d1d/d3b/d55/f66 [0,4194304] 0 2024-03-29T04:45:15.924 INFO:tasks.workunit.client.1.smithi152.stdout:9/435: truncate d5/d8/f9 1689642 0 2024-03-29T04:45:15.926 INFO:tasks.workunit.client.1.smithi152.stdout:1/626: link d5/d7/d38/f7c d5/fcc 0 2024-03-29T04:45:15.926 INFO:tasks.workunit.client.1.smithi152.stdout:1/627: chown d5/d7/d1b/f93 86136670 1 2024-03-29T04:45:15.927 INFO:tasks.workunit.client.0.smithi142.stdout:2/996: mkdir de/d11/d17/d10b/d22/d27/d148 0 2024-03-29T04:45:15.928 INFO:tasks.workunit.client.1.smithi152.stdout:1/628: dread d5/d7/d38/fba [0,4194304] 0 2024-03-29T04:45:15.928 INFO:tasks.workunit.client.0.smithi142.stdout:2/997: write de/d11/d3d/f105 [1151101,64470] 0 2024-03-29T04:45:15.930 INFO:tasks.workunit.client.1.smithi152.stdout:1/629: truncate d5/d7/d8/d13/f45 210064 0 2024-03-29T04:45:15.934 INFO:tasks.workunit.client.0.smithi142.stdout:3/826: symlink d3/d1f/d59/l115 0 2024-03-29T04:45:15.935 INFO:tasks.workunit.client.0.smithi142.stdout:1/984: mkdir d8/db/dc/d77/db9/dbe/d2d/d148 0 2024-03-29T04:45:15.936 INFO:tasks.workunit.client.1.smithi152.stdout:5/450: rename d6/d1d/l52 to d6/d16/d25/d2e/l93 0 2024-03-29T04:45:15.937 INFO:tasks.workunit.client.0.smithi142.stdout:1/985: write d8/db/dc/d77/d31/d4d/d56/f118 [3563686,13569] 0 2024-03-29T04:45:15.939 INFO:tasks.workunit.client.0.smithi142.stdout:2/998: creat de/d11/d17/d10b/d22/dbf/d11c/f149 x:0 0 0 2024-03-29T04:45:15.939 INFO:tasks.workunit.client.0.smithi142.stdout:3/827: dread d3/d29/d2d/d79/fce [0,4194304] 0 2024-03-29T04:45:15.940 INFO:tasks.workunit.client.1.smithi152.stdout:6/495: chown d1/d3/d9/d13/d28/d45/d8c 1 1 2024-03-29T04:45:15.941 INFO:tasks.workunit.client.1.smithi152.stdout:9/436: creat d5/db/d92/f9a x:0 0 0 2024-03-29T04:45:15.942 INFO:tasks.workunit.client.0.smithi142.stdout:3/828: truncate d3/d29/d8f/d5f/f103 742114 0 2024-03-29T04:45:15.942 INFO:tasks.workunit.client.0.smithi142.stdout:1/986: mknod d8/db/dc/d77/db9/dbe/d2d/d44/d7f/df4/dfb/c149 0 2024-03-29T04:45:15.942 INFO:tasks.workunit.client.0.smithi142.stdout:3/829: stat d3/d29/d2b/d40 0 2024-03-29T04:45:15.942 INFO:tasks.workunit.client.1.smithi152.stdout:1/630: mknod d5/d7/d4d/db1/ccd 0 2024-03-29T04:45:15.942 INFO:tasks.workunit.client.1.smithi152.stdout:9/437: fdatasync d5/d8/d31/d59/d6f/f71 0 2024-03-29T04:45:15.943 INFO:tasks.workunit.client.1.smithi152.stdout:1/631: dread - d5/d7/fc2 zero size 2024-03-29T04:45:15.943 INFO:tasks.workunit.client.0.smithi142.stdout:2/999: unlink de/d11/d17/d10b/d22/dbf/c115 0 2024-03-29T04:45:15.944 INFO:tasks.workunit.client.1.smithi152.stdout:1/632: write d5/f3c [3917225,89084] 0 2024-03-29T04:45:15.945 INFO:tasks.workunit.client.0.smithi142.stdout:1/987: unlink d8/db/dc/d77/de9/f123 0 2024-03-29T04:45:15.945 INFO:tasks.workunit.client.1.smithi152.stdout:5/451: creat d6/d16/d47/d56/f94 x:0 0 0 2024-03-29T04:45:15.946 INFO:tasks.workunit.client.0.smithi142.stdout:1/988: dread - d8/db/dc/d77/d31/f140 zero size 2024-03-29T04:45:15.949 INFO:tasks.workunit.client.1.smithi152.stdout:1/633: symlink d5/d7/d6a/lce 0 2024-03-29T04:45:15.950 INFO:tasks.workunit.client.0.smithi142.stdout:3/830: mkdir d3/d29/d2d/d79/d116 0 2024-03-29T04:45:15.950 INFO:tasks.workunit.client.1.smithi152.stdout:5/452: creat d6/d1d/d3b/f95 x:0 0 0 2024-03-29T04:45:15.953 INFO:tasks.workunit.client.0.smithi142.stdout:1/989: symlink d8/db/dd6/l14a 0 2024-03-29T04:45:15.956 INFO:tasks.workunit.client.0.smithi142.stdout:3/831: creat d3/d29/d2b/f117 x:0 0 0 2024-03-29T04:45:15.956 INFO:tasks.workunit.client.1.smithi152.stdout:1/634: truncate d5/d7/d8/d13/d19/fb6 4095490 0 2024-03-29T04:45:15.956 INFO:tasks.workunit.client.1.smithi152.stdout:6/496: dread d1/d3/d9/d13/d28/f34 [0,4194304] 0 2024-03-29T04:45:15.960 INFO:tasks.workunit.client.1.smithi152.stdout:6/497: rmdir d1/d3/df/d84 39 2024-03-29T04:45:15.960 INFO:tasks.workunit.client.1.smithi152.stdout:1/635: read d5/d7/d1b/d2e/d95/dad/d59/f61 [2317543,119142] 0 2024-03-29T04:45:15.960 INFO:tasks.workunit.client.0.smithi142.stdout:3/832: creat d3/d29/d8f/d5f/dc5/f118 x:0 0 0 2024-03-29T04:45:15.961 INFO:tasks.workunit.client.1.smithi152.stdout:9/438: write d5/f29 [4345702,129564] 0 2024-03-29T04:45:15.963 INFO:tasks.workunit.client.0.smithi142.stdout:3/833: creat d3/d29/d2d/d6c/f119 x:0 0 0 2024-03-29T04:45:15.966 INFO:tasks.workunit.client.0.smithi142.stdout:3/834: creat d3/d29/d2b/d6a/df4/f11a x:0 0 0 2024-03-29T04:45:15.974 INFO:tasks.workunit.client.1.smithi152.stdout:1/636: link d5/d7/d1b/d2e/fa3 d5/d7/d8/da5/fcf 0 2024-03-29T04:45:15.974 INFO:tasks.workunit.client.1.smithi152.stdout:6/498: truncate d1/f8 314718 0 2024-03-29T04:45:15.974 INFO:tasks.workunit.client.1.smithi152.stdout:5/453: dwrite d6/d16/d25/d7e/f69 [0,4194304] 0 2024-03-29T04:45:15.975 INFO:tasks.workunit.client.1.smithi152.stdout:9/439: write d5/d12/d2c/d6e/d73/d7a/f7e [588743,18527] 0 2024-03-29T04:45:15.977 INFO:tasks.workunit.client.0.smithi142.stdout:3/835: symlink d3/d29/d2b/d40/d67/d8b/df7/l11b 0 2024-03-29T04:45:15.980 INFO:tasks.workunit.client.1.smithi152.stdout:5/454: symlink d6/d1d/d30/d6a/l96 0 2024-03-29T04:45:15.980 INFO:tasks.workunit.client.0.smithi142.stdout:1/990: truncate d8/db/dc/d77/db9/f122 4025571 0 2024-03-29T04:45:15.981 INFO:tasks.workunit.client.0.smithi142.stdout:1/991: dread - d8/db/dc/d77/db9/dbe/d2d/d8b/da4/dd1/f120 zero size 2024-03-29T04:45:15.982 INFO:tasks.workunit.client.1.smithi152.stdout:9/440: mknod d5/d8/c9b 0 2024-03-29T04:45:15.982 INFO:tasks.workunit.client.1.smithi152.stdout:9/441: chown d5/d8 928232 1 2024-03-29T04:45:15.982 INFO:tasks.workunit.client.1.smithi152.stdout:1/637: rename d5/d7/d1b/dc5 to d5/d7/d8/d13/dd0 0 2024-03-29T04:45:15.983 INFO:tasks.workunit.client.1.smithi152.stdout:5/455: rmdir d6/d1b 39 2024-03-29T04:45:15.984 INFO:tasks.workunit.client.1.smithi152.stdout:9/442: symlink d5/d8/d31/l9c 0 2024-03-29T04:45:15.986 INFO:tasks.workunit.client.1.smithi152.stdout:6/499: dwrite d1/d3/d1a/d1b/d29/d6f/f3c [0,4194304] 0 2024-03-29T04:45:15.991 INFO:tasks.workunit.client.1.smithi152.stdout:1/638: dread d5/d7/d8/d13/d19/f68 [0,4194304] 0 2024-03-29T04:45:15.992 INFO:tasks.workunit.client.1.smithi152.stdout:6/500: mknod d1/ca5 0 2024-03-29T04:45:15.995 INFO:tasks.workunit.client.1.smithi152.stdout:1/639: chown d5/d7/d1b/d2e/d95/dad/d12/la6 14 1 2024-03-29T04:45:15.995 INFO:tasks.workunit.client.1.smithi152.stdout:9/443: dwrite d5/db/d1b/f43 [0,4194304] 0 2024-03-29T04:45:15.997 INFO:tasks.workunit.client.1.smithi152.stdout:6/501: creat d1/d3/d9/d99/fa6 x:0 0 0 2024-03-29T04:45:15.998 INFO:tasks.workunit.client.1.smithi152.stdout:9/444: write d5/d12/d2c/d6e/d73/f7f [894689,90595] 0 2024-03-29T04:45:15.998 INFO:tasks.workunit.client.1.smithi152.stdout:1/640: rmdir d5/d7/d8/da5 39 2024-03-29T04:45:16.000 INFO:tasks.workunit.client.1.smithi152.stdout:6/502: mkdir d1/d3/d9/d13/da7 0 2024-03-29T04:45:16.000 INFO:tasks.workunit.client.1.smithi152.stdout:1/641: mknod d5/d7/d1b/d2e/cd1 0 2024-03-29T04:45:16.003 INFO:tasks.workunit.client.1.smithi152.stdout:6/503: rename d1 to d1/d3/df/da8 22 2024-03-29T04:45:16.004 INFO:tasks.workunit.client.1.smithi152.stdout:1/642: chown d5/d7/d1b/d2e/d95/dad/d2b/c60 31 1 2024-03-29T04:45:16.007 INFO:tasks.workunit.client.1.smithi152.stdout:9/445: dwrite d5/d8/d31/d59/d6f/f71 [0,4194304] 0 2024-03-29T04:45:16.007 INFO:tasks.workunit.client.1.smithi152.stdout:1/643: chown d5/c99 3297514 1 2024-03-29T04:45:16.008 INFO:tasks.workunit.client.1.smithi152.stdout:6/504: symlink d1/d3/d1a/d1b/d29/d6f/la9 0 2024-03-29T04:45:16.009 INFO:tasks.workunit.client.1.smithi152.stdout:9/446: chown d5/d12/d2c/d6e/d73/d8e/l81 38 1 2024-03-29T04:45:16.012 INFO:tasks.workunit.client.1.smithi152.stdout:5/456: dread d6/f21 [0,4194304] 0 2024-03-29T04:45:16.013 INFO:tasks.workunit.client.1.smithi152.stdout:6/505: truncate d1/d3/df/d84/f57 1017301 0 2024-03-29T04:45:16.015 INFO:tasks.workunit.client.1.smithi152.stdout:1/644: mkdir d5/d7/d8/d13/d19/dc7/dd2 0 2024-03-29T04:45:16.016 INFO:tasks.workunit.client.1.smithi152.stdout:5/457: mkdir d6/d7/d75/d97 0 2024-03-29T04:45:16.017 INFO:tasks.workunit.client.1.smithi152.stdout:9/447: mknod d5/d12/d2c/c9d 0 2024-03-29T04:45:16.018 INFO:tasks.workunit.client.1.smithi152.stdout:1/645: write d5/d7/d8/f1e [4907290,31476] 0 2024-03-29T04:45:16.019 INFO:tasks.workunit.client.1.smithi152.stdout:5/458: chown d6/d1b/l50 2869 1 2024-03-29T04:45:16.020 INFO:tasks.workunit.client.1.smithi152.stdout:6/506: creat d1/d3/d9/d13/d28/faa x:0 0 0 2024-03-29T04:45:16.020 INFO:tasks.workunit.client.1.smithi152.stdout:5/459: mknod d6/d1b/c98 0 2024-03-29T04:45:16.021 INFO:tasks.workunit.client.1.smithi152.stdout:1/646: mkdir d5/d7/d8/d13/dd0/dd3 0 2024-03-29T04:45:16.021 INFO:tasks.workunit.client.1.smithi152.stdout:6/507: read - d1/d3/df/f9e zero size 2024-03-29T04:45:16.022 INFO:tasks.workunit.client.1.smithi152.stdout:6/508: chown d1/d3/d9/d13/d3e 15 1 2024-03-29T04:45:16.022 INFO:tasks.workunit.client.1.smithi152.stdout:5/460: symlink d6/l99 0 2024-03-29T04:45:16.022 INFO:tasks.workunit.client.1.smithi152.stdout:1/647: write d5/d7/d8/d13/d19/f68 [931446,51933] 0 2024-03-29T04:45:16.024 INFO:tasks.workunit.client.1.smithi152.stdout:5/461: rmdir d6/d1d/d3b/d55 39 2024-03-29T04:45:16.025 INFO:tasks.workunit.client.1.smithi152.stdout:1/648: stat d5/d7/d1b/d2e/d95/d9d/la2 0 2024-03-29T04:45:16.027 INFO:tasks.workunit.client.1.smithi152.stdout:1/649: fdatasync d5/d7/d6a/f6e 0 2024-03-29T04:45:16.028 INFO:tasks.workunit.client.1.smithi152.stdout:1/650: chown d5/d7 35055629 1 2024-03-29T04:45:16.029 INFO:tasks.workunit.client.1.smithi152.stdout:6/509: dwrite d1/d3/d9/f21 [0,4194304] 0 2024-03-29T04:45:16.031 INFO:tasks.workunit.client.1.smithi152.stdout:5/462: dread d6/d1d/d3b/d55/f66 [0,4194304] 0 2024-03-29T04:45:16.033 INFO:tasks.workunit.client.1.smithi152.stdout:6/510: link d1/d3/d1a/f9f d1/d3/d1a/d1b/fab 0 2024-03-29T04:45:16.038 INFO:tasks.workunit.client.1.smithi152.stdout:1/651: dwrite d5/d7/d8/f2d [0,4194304] 0 2024-03-29T04:45:16.038 INFO:tasks.workunit.client.1.smithi152.stdout:6/511: truncate d1/d3/d9/d13/d28/d45/d8c/f85 679486 0 2024-03-29T04:45:16.041 INFO:tasks.workunit.client.1.smithi152.stdout:1/652: chown d5/d7/d1b/d2e/d95/dad/d2b/cc6 122966246 1 2024-03-29T04:45:16.044 INFO:tasks.workunit.client.1.smithi152.stdout:1/653: mkdir d5/d7/d4d/dd4 0 2024-03-29T04:45:16.046 INFO:tasks.workunit.client.1.smithi152.stdout:1/654: creat d5/d7/d38/d6c/fd5 x:0 0 0 2024-03-29T04:45:16.048 INFO:tasks.workunit.client.1.smithi152.stdout:6/512: truncate d1/d3/d1a/d1b/d29/f79 1917015 0 2024-03-29T04:45:16.052 INFO:tasks.workunit.client.1.smithi152.stdout:1/655: dwrite d5/d7/d1b/d2e/f8e [0,4194304] 0 2024-03-29T04:45:16.057 INFO:tasks.workunit.client.1.smithi152.stdout:1/656: write d5/d7/d1b/d2e/d95/dad/d59/f5b [3501704,8124] 0 2024-03-29T04:45:16.059 INFO:tasks.workunit.client.1.smithi152.stdout:1/657: write d5/d7/d1b/d2e/f8d [833351,64736] 0 2024-03-29T04:45:16.064 INFO:tasks.workunit.client.1.smithi152.stdout:1/658: getdents d5/d7/d8/d13/d19/dc7/dd2 0 2024-03-29T04:45:16.067 INFO:tasks.workunit.client.1.smithi152.stdout:1/659: link d5/d7/d1b/c2c d5/d7/d8/d13/d96/cd6 0 2024-03-29T04:45:16.072 INFO:tasks.workunit.client.1.smithi152.stdout:1/660: link d5/d7/d8/d13/d19/f94 d5/d7/d8/d13/d19/fd7 0 2024-03-29T04:45:16.072 INFO:tasks.workunit.client.0.smithi142.stdout:3/836: mknod d3/d29/d2d/d79/dcc/c11c 0 2024-03-29T04:45:16.074 INFO:tasks.workunit.client.1.smithi152.stdout:1/661: mkdir d5/d7/d1b/d2e/dd8 0 2024-03-29T04:45:16.074 INFO:tasks.workunit.client.0.smithi142.stdout:3/837: truncate d3/d29/d2b/d6a/df4/f11a 211225 0 2024-03-29T04:45:16.075 INFO:tasks.workunit.client.0.smithi142.stdout:1/992: dread d8/db/dc/d77/db9/dbe/d2d/d44/d4b/db4/db8/fd8 [0,4194304] 0 2024-03-29T04:45:16.076 INFO:tasks.workunit.client.1.smithi152.stdout:1/662: write d5/d7/d1b/d2e/d95/dad/d12/d77/fb4 [627200,98406] 0 2024-03-29T04:45:16.077 INFO:tasks.workunit.client.0.smithi142.stdout:3/838: mkdir d3/d1f/de6/d11d 0 2024-03-29T04:45:16.078 INFO:tasks.workunit.client.1.smithi152.stdout:6/513: dread d1/d3/d1a/f25 [0,4194304] 0 2024-03-29T04:45:16.079 INFO:tasks.workunit.client.1.smithi152.stdout:1/663: creat d5/d7/d1b/d2e/d95/dad/d59/fd9 x:0 0 0 2024-03-29T04:45:16.079 INFO:tasks.workunit.client.0.smithi142.stdout:3/839: mkdir d3/db9/d11e 0 2024-03-29T04:45:16.080 INFO:tasks.workunit.client.1.smithi152.stdout:6/514: dread - d1/d3/df/f48 zero size 2024-03-29T04:45:16.084 INFO:tasks.workunit.client.1.smithi152.stdout:6/515: mknod d1/d3/d9/d13/d28/cac 0 2024-03-29T04:45:16.085 INFO:tasks.workunit.client.1.smithi152.stdout:6/516: fdatasync d1/d3/d9/d13/f1f 0 2024-03-29T04:45:16.087 INFO:tasks.workunit.client.1.smithi152.stdout:6/517: mknod d1/d3/d91/cad 0 2024-03-29T04:45:16.090 INFO:tasks.workunit.client.1.smithi152.stdout:6/518: symlink d1/d3/d9/lae 0 2024-03-29T04:45:16.091 INFO:tasks.workunit.client.0.smithi142.stdout:3/840: dwrite d3/d1f/f46 [0,4194304] 0 2024-03-29T04:45:16.093 INFO:tasks.workunit.client.0.smithi142.stdout:1/993: dread d8/db/dc/d77/db9/dbe/d1f/d4c/fc4 [0,4194304] 0 2024-03-29T04:45:16.094 INFO:tasks.workunit.client.1.smithi152.stdout:1/664: dwrite d5/d7/d38/fba [0,4194304] 0 2024-03-29T04:45:16.096 INFO:tasks.workunit.client.1.smithi152.stdout:6/519: mknod d1/d3/d9/d13/d28/d45/d8c/caf 0 2024-03-29T04:45:16.101 INFO:tasks.workunit.client.1.smithi152.stdout:6/520: creat d1/d3/d9/d13/da7/fb0 x:0 0 0 2024-03-29T04:45:16.101 INFO:tasks.workunit.client.1.smithi152.stdout:1/665: dwrite d5/d7/d1b/d2e/f8d [0,4194304] 0 2024-03-29T04:45:16.103 INFO:tasks.workunit.client.0.smithi142.stdout:1/994: link d8/db/l24 d8/db/dc/d77/db9/dbe/d2d/d8b/da4/l14b 0 2024-03-29T04:45:16.103 INFO:tasks.workunit.client.1.smithi152.stdout:1/666: chown d5/d7/d8/f1e 0 1 2024-03-29T04:45:16.105 INFO:tasks.workunit.client.1.smithi152.stdout:1/667: fdatasync d5/d7/d8/d13/d19/f30 0 2024-03-29T04:45:16.107 INFO:tasks.workunit.client.0.smithi142.stdout:3/841: dwrite d3/d1f/d24/d47/f57 [0,4194304] 0 2024-03-29T04:45:16.107 INFO:tasks.workunit.client.1.smithi152.stdout:1/668: fsync d5/d7/d8/d13/d19/f65 0 2024-03-29T04:45:16.108 INFO:tasks.workunit.client.1.smithi152.stdout:6/521: rename d1/d3/d1a/d1b/c51 to d1/d3/d9/d13/d28/cb1 0 2024-03-29T04:45:16.109 INFO:tasks.workunit.client.0.smithi142.stdout:1/995: unlink d8/db/dc/d77/d5a/c3c 0 2024-03-29T04:45:16.114 INFO:tasks.workunit.client.1.smithi152.stdout:1/669: read d5/d7/d8/d13/d19/fb6 [3966798,84883] 0 2024-03-29T04:45:16.117 INFO:tasks.workunit.client.1.smithi152.stdout:1/670: mknod d5/d7/d4d/db1/cda 0 2024-03-29T04:45:16.117 INFO:tasks.workunit.client.0.smithi142.stdout:1/996: dwrite d8/db/f125 [0,4194304] 0 2024-03-29T04:45:16.121 INFO:tasks.workunit.client.1.smithi152.stdout:1/671: link d5/d7/d1b/d2e/d95/dad/l9f d5/d7/d4d/ldb 0 2024-03-29T04:45:16.122 INFO:tasks.workunit.client.0.smithi142.stdout:1/997: getdents d8/db/dc/d77 0 2024-03-29T04:45:16.125 INFO:tasks.workunit.client.0.smithi142.stdout:1/998: write d8/db/dc/d77/db9/dbe/d2d/d44/d7f/d10e/f87 [852328,124300] 0 2024-03-29T04:45:16.125 INFO:tasks.workunit.client.1.smithi152.stdout:1/672: unlink d5/d7/d1b/d2e/l53 0 2024-03-29T04:45:16.127 INFO:tasks.workunit.client.0.smithi142.stdout:1/999: rmdir d8/db/dd6 39 2024-03-29T04:45:16.128 INFO:tasks.workunit.client.1.smithi152.stdout:1/673: rename d5/d7/d6a/l39 to d5/d7/d6a/ldc 0 2024-03-29T04:45:16.131 INFO:tasks.workunit.client.1.smithi152.stdout:1/674: rename d5/d7/d1b/d2e/d95/dad/d12/fbd to d5/d7/d1b/d2e/d95/dad/d12/fdd 0 2024-03-29T04:45:16.132 INFO:tasks.workunit.client.1.smithi152.stdout:1/675: write d5/d7/d1b/f47 [171538,39058] 0 2024-03-29T04:45:16.135 INFO:tasks.workunit.client.0.smithi142.stdout:3/842: dwrite d3/d29/d8f/f111 [0,4194304] 0 2024-03-29T04:45:16.136 INFO:tasks.workunit.client.0.smithi142.stdout:3/843: write d3/d1f/d24/faf [4577253,86328] 0 2024-03-29T04:45:16.137 INFO:tasks.workunit.client.1.smithi152.stdout:1/676: link d5/c75 d5/d7/d8/d13/d19/cde 0 2024-03-29T04:45:16.137 INFO:tasks.workunit.client.1.smithi152.stdout:6/522: dread d1/d3/d9/d13/f33 [0,4194304] 0 2024-03-29T04:45:16.138 INFO:tasks.workunit.client.1.smithi152.stdout:1/677: fsync d5/d7/d1b/d2e/d95/dad/f73 0 2024-03-29T04:45:16.141 INFO:tasks.workunit.client.0.smithi142.stdout:3/844: rename d3/d29/d2b/d6a/df4 to d3/d29/d8f/d95/d11f 0 2024-03-29T04:45:16.142 INFO:tasks.workunit.client.1.smithi152.stdout:1/678: unlink d5/d7/d1b/f47 0 2024-03-29T04:45:16.145 INFO:tasks.workunit.client.0.smithi142.stdout:3/845: mkdir d3/d29/d2d/d56/d120 0 2024-03-29T04:45:16.146 INFO:tasks.workunit.client.1.smithi152.stdout:1/679: creat d5/d7/d8/d13/d19/dc7/fdf x:0 0 0 2024-03-29T04:45:16.149 INFO:tasks.workunit.client.0.smithi142.stdout:3/846: mknod d3/d29/d2b/d40/d67/c121 0 2024-03-29T04:45:16.149 INFO:tasks.workunit.client.1.smithi152.stdout:6/523: dread d1/d3/df/f66 [0,4194304] 0 2024-03-29T04:45:16.150 INFO:tasks.workunit.client.0.smithi142.stdout:3/847: fsync d3/d29/d8f/d5f/dc5/f118 0 2024-03-29T04:45:16.151 INFO:tasks.workunit.client.1.smithi152.stdout:1/680: dread d5/d7/d1b/d2e/d95/dad/d59/f5b [0,4194304] 0 2024-03-29T04:45:16.152 INFO:tasks.workunit.client.1.smithi152.stdout:6/524: write d1/d3/d1a/f1d [694107,28] 0 2024-03-29T04:45:16.152 INFO:tasks.workunit.client.1.smithi152.stdout:1/681: read - d5/d7/d1b/d2e/fa0 zero size 2024-03-29T04:45:16.152 INFO:tasks.workunit.client.0.smithi142.stdout:3/848: creat d3/d1f/d59/dc7/f122 x:0 0 0 2024-03-29T04:45:16.153 INFO:tasks.workunit.client.1.smithi152.stdout:1/682: write d5/d7/d38/d6c/fb0 [534432,124470] 0 2024-03-29T04:45:16.153 INFO:tasks.workunit.client.0.smithi142.stdout:3/849: fdatasync d3/d29/d2b/d40/ff3 0 2024-03-29T04:45:16.155 INFO:tasks.workunit.client.1.smithi152.stdout:6/525: creat d1/d3/d1a/fb2 x:0 0 0 2024-03-29T04:45:16.156 INFO:tasks.workunit.client.0.smithi142.stdout:3/850: symlink d3/d29/d2d/d79/dcc/l123 0 2024-03-29T04:45:16.157 INFO:tasks.workunit.client.1.smithi152.stdout:1/683: mknod d5/d7/d8/d13/d19/d80/ce0 0 2024-03-29T04:45:16.159 INFO:tasks.workunit.client.0.smithi142.stdout:3/851: truncate d3/d29/d2b/d40/f4c 2019663 0 2024-03-29T04:45:16.160 INFO:tasks.workunit.client.0.smithi142.stdout:3/852: truncate d3/d1f/d59/dc7/f122 210232 0 2024-03-29T04:45:16.160 INFO:tasks.workunit.client.1.smithi152.stdout:1/684: creat d5/d7/d1b/d2e/d95/fe1 x:0 0 0 2024-03-29T04:45:16.162 INFO:tasks.workunit.client.1.smithi152.stdout:1/685: mkdir d5/d7/d1b/d2e/de2 0 2024-03-29T04:45:16.164 INFO:tasks.workunit.client.1.smithi152.stdout:1/686: fdatasync d5/fcc 0 2024-03-29T04:45:16.165 INFO:tasks.workunit.client.1.smithi152.stdout:6/526: truncate d1/d3/d1a/f61 1062103 0 2024-03-29T04:45:16.168 INFO:tasks.workunit.client.1.smithi152.stdout:6/527: mknod d1/d3/df/d8a/d98/cb3 0 2024-03-29T04:45:16.168 INFO:tasks.workunit.client.1.smithi152.stdout:1/687: symlink d5/d7/d1b/d2e/d95/d9d/le3 0 2024-03-29T04:45:16.169 INFO:tasks.workunit.client.1.smithi152.stdout:6/528: fdatasync d1/d3/d9/d13/da7/fb0 0 2024-03-29T04:45:16.171 INFO:tasks.workunit.client.1.smithi152.stdout:1/688: write d5/d7/d38/f7c [1272378,123612] 0 2024-03-29T04:45:16.171 INFO:tasks.workunit.client.1.smithi152.stdout:6/529: mknod d1/d3/d91/cb4 0 2024-03-29T04:45:16.174 INFO:tasks.workunit.client.1.smithi152.stdout:6/530: rmdir d1/d3/d1a 39 2024-03-29T04:45:16.180 INFO:tasks.workunit.client.1.smithi152.stdout:6/531: link d1/d3/d9/d13/d3e/l65 d1/d3/d1a/d1b/d87/lb5 0 2024-03-29T04:45:16.183 INFO:tasks.workunit.client.1.smithi152.stdout:6/532: truncate d1/d3/d9/d13/d3e/f42 197282 0 2024-03-29T04:45:16.189 INFO:tasks.workunit.client.1.smithi152.stdout:6/533: dread d1/d3/d1a/f2f [0,4194304] 0 2024-03-29T04:45:16.193 INFO:tasks.workunit.client.1.smithi152.stdout:6/534: truncate d1/d3/d1a/d1b/d29/f79 623097 0 2024-03-29T04:45:16.195 INFO:tasks.workunit.client.1.smithi152.stdout:6/535: creat d1/d3/df/d8a/fb6 x:0 0 0 2024-03-29T04:45:16.197 INFO:tasks.workunit.client.1.smithi152.stdout:6/536: chown d1/d3/d9/d13/d3e/f42 6417767 1 2024-03-29T04:45:16.213 INFO:tasks.workunit.client.1.smithi152.stdout:6/537: dread d1/d3/d9/f41 [0,4194304] 0 2024-03-29T04:45:16.214 INFO:tasks.workunit.client.1.smithi152.stdout:6/538: mknod d1/d3/d9/d13/cb7 0 2024-03-29T04:45:16.218 INFO:tasks.workunit.client.1.smithi152.stdout:6/539: rename d1/d3/d1a/d1b/d29/l72 to d1/d3/df/d84/da1/lb8 0 2024-03-29T04:45:16.220 INFO:tasks.workunit.client.1.smithi152.stdout:6/540: creat d1/d3/d1a/fb9 x:0 0 0 2024-03-29T04:45:16.231 INFO:tasks.workunit.client.0.smithi142.stdout:3/853: getdents d3/d29/d2b/d40/d67/d8b 0 2024-03-29T04:45:16.233 INFO:tasks.workunit.client.1.smithi152.stdout:6/541: truncate d1/d3/d1a/d1b/d29/f5f 5554541 0 2024-03-29T04:45:16.234 INFO:tasks.workunit.client.1.smithi152.stdout:6/542: fdatasync d1/d3/d1a/d1b/d29/d6f/f3c 0 2024-03-29T04:45:16.355 INFO:tasks.workunit.client.1.smithi152.stdout:7/414: sync 2024-03-29T04:45:16.355 INFO:tasks.workunit.client.1.smithi152.stdout:7/415: readlink d7/d8/d12/d15/l5b 0 2024-03-29T04:45:16.357 INFO:tasks.workunit.client.1.smithi152.stdout:7/416: mknod d7/d8/d12/d1d/d30/c7a 0 2024-03-29T04:45:16.364 INFO:tasks.workunit.client.1.smithi152.stdout:7/417: truncate d7/d8/d12/d1d/f1e 3337077 0 2024-03-29T04:45:16.367 INFO:tasks.workunit.client.1.smithi152.stdout:7/418: link d7/d8/d12/d1d/f62 d7/d8/d12/f7b 0 2024-03-29T04:45:16.368 INFO:tasks.workunit.client.1.smithi152.stdout:7/419: unlink d7/d8/d12/d1d/d30/c57 0 2024-03-29T04:45:16.371 INFO:tasks.workunit.client.1.smithi152.stdout:7/420: unlink d7/d8/lc 0 2024-03-29T04:45:16.419 INFO:tasks.workunit.client.1.smithi152.stdout:7/421: dwrite d7/d8/d12/d1d/d41/d45/f70 [0,4194304] 0 2024-03-29T04:45:16.424 INFO:tasks.workunit.client.1.smithi152.stdout:7/422: mknod d7/d8/d12/d1d/c7c 0 2024-03-29T04:45:16.424 INFO:tasks.workunit.client.1.smithi152.stdout:7/423: stat d7/d8/fa 0 2024-03-29T04:45:16.425 INFO:tasks.workunit.client.1.smithi152.stdout:7/424: write d7/d8/fa [2901142,49910] 0 2024-03-29T04:45:16.435 INFO:tasks.workunit.client.1.smithi152.stdout:7/425: dread d7/d8/d12/d1d/f34 [4194304,4194304] 0 2024-03-29T04:45:16.437 INFO:tasks.workunit.client.1.smithi152.stdout:7/426: unlink d7/c47 0 2024-03-29T04:45:16.438 INFO:tasks.workunit.client.1.smithi152.stdout:7/427: creat d7/d8/d37/f7d x:0 0 0 2024-03-29T04:45:16.441 INFO:tasks.workunit.client.1.smithi152.stdout:7/428: getdents d7/d8 0 2024-03-29T04:45:16.444 INFO:tasks.workunit.client.0.smithi142.stdout:3/854: sync 2024-03-29T04:45:16.449 INFO:tasks.workunit.client.0.smithi142.stdout:3/855: dread d3/d29/d2b/d6a/f7b [0,4194304] 0 2024-03-29T04:45:16.452 INFO:tasks.workunit.client.0.smithi142.stdout:3/856: rename d3/d29/d2b/d40/d67/d8b/d4d to d3/d29/d2b/d40/db1/d124 0 2024-03-29T04:45:16.452 INFO:tasks.workunit.client.1.smithi152.stdout:7/429: dread d7/d8/db/df/f6b [0,4194304] 0 2024-03-29T04:45:16.457 INFO:tasks.workunit.client.1.smithi152.stdout:7/430: dwrite d7/d8/d12/f20 [0,4194304] 0 2024-03-29T04:45:16.467 INFO:tasks.workunit.client.0.smithi142.stdout:3/857: dread d3/d1f/fc4 [0,4194304] 0 2024-03-29T04:45:16.468 INFO:tasks.workunit.client.0.smithi142.stdout:3/858: chown d3/d29/d8f/d5f/db4/l81 13421 1 2024-03-29T04:45:16.474 INFO:tasks.workunit.client.0.smithi142.stdout:3/859: read d3/d29/d2b/d40/ff3 [1690879,104764] 0 2024-03-29T04:45:16.475 INFO:tasks.workunit.client.0.smithi142.stdout:3/860: symlink d3/d29/d2d/l125 0 2024-03-29T04:45:16.482 INFO:tasks.workunit.client.0.smithi142.stdout:3/861: rename d3/d29/d2d to d3/d29/d2b/d40/db1/d124/d50/d126 0 2024-03-29T04:45:16.483 INFO:tasks.workunit.client.0.smithi142.stdout:3/862: getdents d3/dfe 0 2024-03-29T04:45:16.491 INFO:tasks.workunit.client.0.smithi142.stdout:3/863: creat d3/d29/d2b/d40/d67/d8b/f127 x:0 0 0 2024-03-29T04:45:16.495 INFO:tasks.workunit.client.0.smithi142.stdout:3/864: dread d3/d29/d2b/d40/d67/fb6 [0,4194304] 0 2024-03-29T04:45:16.502 INFO:tasks.workunit.client.0.smithi142.stdout:3/865: dwrite d3/d29/d2b/d40/db1/d124/d50/d126/d56/fd1 [0,4194304] 0 2024-03-29T04:45:16.506 INFO:tasks.workunit.client.0.smithi142.stdout:3/866: chown d3/d29/d8f/d5f/db4 29136880 1 2024-03-29T04:45:16.507 INFO:tasks.workunit.client.0.smithi142.stdout:3/867: mknod d3/d1f/d59/c128 0 2024-03-29T04:45:16.509 INFO:tasks.workunit.client.0.smithi142.stdout:3/868: symlink d3/d29/d8f/d5f/d64/l129 0 2024-03-29T04:45:16.516 INFO:tasks.workunit.client.0.smithi142.stdout:3/869: dwrite d3/d29/f77 [0,4194304] 0 2024-03-29T04:45:16.530 INFO:tasks.workunit.client.0.smithi142.stdout:3/870: dwrite d3/d29/d2b/d40/db1/d124/d50/d126/f8a [0,4194304] 0 2024-03-29T04:45:16.530 INFO:tasks.workunit.client.0.smithi142.stdout:3/871: readlink d3/d1f/d24/l113 0 2024-03-29T04:45:16.531 INFO:tasks.workunit.client.0.smithi142.stdout:3/872: chown d3/db9/c102 22337 1 2024-03-29T04:45:16.535 INFO:tasks.workunit.client.0.smithi142.stdout:3/873: creat d3/d29/d2b/d40/d67/f12a x:0 0 0 2024-03-29T04:45:16.540 INFO:tasks.workunit.client.0.smithi142.stdout:3/874: mkdir d3/d29/d2b/d40/d67/d8b/df7/d12b 0 2024-03-29T04:45:16.541 INFO:tasks.workunit.client.0.smithi142.stdout:3/875: read d3/d29/d2b/d40/db1/d124/fee [289506,56088] 0 2024-03-29T04:45:16.542 INFO:tasks.workunit.client.0.smithi142.stdout:3/876: chown d3/d1f/d59/fb3 21 1 2024-03-29T04:45:16.543 INFO:tasks.workunit.client.0.smithi142.stdout:3/877: rmdir d3/d1f/d59/dc7 39 2024-03-29T04:45:16.554 INFO:tasks.workunit.client.1.smithi152.stdout:3/392: sync 2024-03-29T04:45:16.555 INFO:tasks.workunit.client.1.smithi152.stdout:3/393: mkdir d7/d6c 0 2024-03-29T04:45:16.561 INFO:tasks.workunit.client.1.smithi152.stdout:3/394: unlink d7/d46/f64 0 2024-03-29T04:45:16.563 INFO:tasks.workunit.client.1.smithi152.stdout:3/395: fsync d7/d19/d3a/f49 0 2024-03-29T04:45:16.564 INFO:tasks.workunit.client.1.smithi152.stdout:3/396: write d7/f15 [8953447,21775] 0 2024-03-29T04:45:16.574 INFO:tasks.workunit.client.1.smithi152.stdout:3/397: dwrite d7/d19/d1b/f23 [0,4194304] 0 2024-03-29T04:45:16.579 INFO:tasks.workunit.client.1.smithi152.stdout:3/398: getdents d7/d1c/d41/d61 0 2024-03-29T04:45:16.581 INFO:tasks.workunit.client.1.smithi152.stdout:3/399: creat d7/d46/f6d x:0 0 0 2024-03-29T04:45:16.584 INFO:tasks.workunit.client.1.smithi152.stdout:3/400: rename d7/d19/d1b/f3d to d7/d19/d3a/f6e 0 2024-03-29T04:45:16.586 INFO:tasks.workunit.client.1.smithi152.stdout:3/401: unlink d7/d19/f32 0 2024-03-29T04:45:16.587 INFO:tasks.workunit.client.1.smithi152.stdout:3/402: creat d7/d19/d1b/d1e/d22/f6f x:0 0 0 2024-03-29T04:45:16.589 INFO:tasks.workunit.client.1.smithi152.stdout:3/403: creat d7/d19/d1b/d1e/d4e/f70 x:0 0 0 2024-03-29T04:45:16.589 INFO:tasks.workunit.client.1.smithi152.stdout:3/404: chown d7/d46/l69 19 1 2024-03-29T04:45:16.592 INFO:tasks.workunit.client.1.smithi152.stdout:3/405: unlink d7/d19/d1b/d1e/d22/c5e 0 2024-03-29T04:45:16.597 INFO:tasks.workunit.client.1.smithi152.stdout:3/406: dwrite d7/d19/d3a/f6e [0,4194304] 0 2024-03-29T04:45:16.598 INFO:tasks.workunit.client.1.smithi152.stdout:3/407: chown d7/d46/l69 3832485 1 2024-03-29T04:45:16.601 INFO:tasks.workunit.client.1.smithi152.stdout:3/408: stat d7/f10 0 2024-03-29T04:45:16.604 INFO:tasks.workunit.client.1.smithi152.stdout:3/409: unlink d7/l31 0 2024-03-29T04:45:16.605 INFO:tasks.workunit.client.1.smithi152.stdout:3/410: stat d7/d19/l20 0 2024-03-29T04:45:16.610 INFO:tasks.workunit.client.1.smithi152.stdout:3/411: dwrite d7/d19/d1b/d1e/d22/f3b [0,4194304] 0 2024-03-29T04:45:16.625 INFO:tasks.workunit.client.1.smithi152.stdout:3/412: dread d7/f15 [0,4194304] 0 2024-03-29T04:45:16.627 INFO:tasks.workunit.client.1.smithi152.stdout:3/413: mknod d7/d1c/d41/c71 0 2024-03-29T04:45:16.628 INFO:tasks.workunit.client.1.smithi152.stdout:3/414: write d7/d19/f5c [600182,10784] 0 2024-03-29T04:45:16.629 INFO:tasks.workunit.client.1.smithi152.stdout:3/415: fdatasync d7/d19/d1b/d1e/d4e/f5a 0 2024-03-29T04:45:16.632 INFO:tasks.workunit.client.1.smithi152.stdout:3/416: symlink d7/d46/l72 0 2024-03-29T04:45:16.635 INFO:tasks.workunit.client.1.smithi152.stdout:3/417: dread d7/d19/f5c [0,4194304] 0 2024-03-29T04:45:16.640 INFO:tasks.workunit.client.1.smithi152.stdout:3/418: dwrite d7/d1c/d41/d61/d63/f67 [0,4194304] 0 2024-03-29T04:45:16.642 INFO:tasks.workunit.client.1.smithi152.stdout:3/419: chown d7/d19/d1b/d1e/d4e/f70 8 1 2024-03-29T04:45:16.654 INFO:tasks.workunit.client.0.smithi142.stdout:3/878: sync 2024-03-29T04:45:16.654 INFO:tasks.workunit.client.1.smithi152.stdout:3/420: dread d7/d19/d1b/d1e/d22/f4d [0,4194304] 0 2024-03-29T04:45:16.655 INFO:tasks.workunit.client.0.smithi142.stdout:3/879: dread - d3/d29/d2b/d40/d67/d8b/fdd zero size 2024-03-29T04:45:16.656 INFO:tasks.workunit.client.1.smithi152.stdout:3/421: mkdir d7/d1c/d41/d73 0 2024-03-29T04:45:16.659 INFO:tasks.workunit.client.0.smithi142.stdout:3/880: dread d3/d29/f77 [0,4194304] 0 2024-03-29T04:45:16.661 INFO:tasks.workunit.client.0.smithi142.stdout:3/881: rename d3/d29/d8f/d5f/f86 to d3/d29/d8f/d5f/f12c 0 2024-03-29T04:45:16.663 INFO:tasks.workunit.client.0.smithi142.stdout:3/882: rename d3/d1f/d24/c74 to d3/d29/d2b/d40/db1/d124/c12d 0 2024-03-29T04:45:16.665 INFO:tasks.workunit.client.0.smithi142.stdout:3/883: rename d3/d29/d8f/d95 to d3/d29/d2b/d40/db1/d12e 0 2024-03-29T04:45:16.667 INFO:tasks.workunit.client.1.smithi152.stdout:8/359: sync 2024-03-29T04:45:16.668 INFO:tasks.workunit.client.1.smithi152.stdout:8/360: write d1/d21/d40/f52 [157176,82161] 0 2024-03-29T04:45:16.671 INFO:tasks.workunit.client.1.smithi152.stdout:8/361: mkdir d1/d2/d81 0 2024-03-29T04:45:16.672 INFO:tasks.workunit.client.0.smithi142.stdout:3/884: write d3/d29/d2b/d40/d67/fb6 [538757,55380] 0 2024-03-29T04:45:16.674 INFO:tasks.workunit.client.0.smithi142.stdout:3/885: truncate d3/d29/d2b/d40/db1/d124/d50/d126/faa 1698084 0 2024-03-29T04:45:16.674 INFO:tasks.workunit.client.0.smithi142.stdout:3/886: stat d3/d1f/d24/f10b 0 2024-03-29T04:45:16.675 INFO:tasks.workunit.client.1.smithi152.stdout:8/362: write d1/f10 [4755998,4792] 0 2024-03-29T04:45:16.676 INFO:tasks.workunit.client.0.smithi142.stdout:3/887: truncate d3/d29/d2b/d40/db1/d124/fee 1043734 0 2024-03-29T04:45:16.677 INFO:tasks.workunit.client.1.smithi152.stdout:8/363: mknod d1/d2/d6d/c82 0 2024-03-29T04:45:16.677 INFO:tasks.workunit.client.1.smithi152.stdout:8/364: chown d1/d2/l39 433726 1 2024-03-29T04:45:16.677 INFO:tasks.workunit.client.0.smithi142.stdout:3/888: creat d3/d1f/d24/d47/d87/f12f x:0 0 0 2024-03-29T04:45:16.678 INFO:tasks.workunit.client.1.smithi152.stdout:8/365: chown d1/d2/d1d/d36/d4b/l61 12584071 1 2024-03-29T04:45:16.679 INFO:tasks.workunit.client.0.smithi142.stdout:3/889: mkdir d3/db9/d130 0 2024-03-29T04:45:16.680 INFO:tasks.workunit.client.0.smithi142.stdout:3/890: mknod d3/d1f/d24/c131 0 2024-03-29T04:45:16.681 INFO:tasks.workunit.client.1.smithi152.stdout:8/366: rename d1/d2/d1d/d36/d5d/f6f to d1/d2/d1d/f83 0 2024-03-29T04:45:16.682 INFO:tasks.workunit.client.0.smithi142.stdout:3/891: mkdir d3/d132 0 2024-03-29T04:45:16.683 INFO:tasks.workunit.client.1.smithi152.stdout:8/367: unlink d1/d2/d3/l18 0 2024-03-29T04:45:16.684 INFO:tasks.workunit.client.1.smithi152.stdout:8/368: write d1/d2/d1d/d36/d4b/d51/f77 [133604,29230] 0 2024-03-29T04:45:16.685 INFO:tasks.workunit.client.0.smithi142.stdout:3/892: link d3/d1f/d24/faf d3/d29/d8f/d5f/db4/f133 0 2024-03-29T04:45:16.686 INFO:tasks.workunit.client.1.smithi152.stdout:8/369: stat d1/d2/d3/l8 0 2024-03-29T04:45:16.687 INFO:tasks.workunit.client.1.smithi152.stdout:8/370: write d1/d21/f46 [8602041,13577] 0 2024-03-29T04:45:16.688 INFO:tasks.workunit.client.0.smithi142.stdout:3/893: rename d3/d1f/d59/fb2 to d3/d1f/d24/d47/f134 0 2024-03-29T04:45:16.693 INFO:tasks.workunit.client.1.smithi152.stdout:8/371: dwrite d1/d21/d40/f52 [0,4194304] 0 2024-03-29T04:45:16.694 INFO:tasks.workunit.client.1.smithi152.stdout:8/372: chown d1/d2/l38 135 1 2024-03-29T04:45:16.699 INFO:tasks.workunit.client.0.smithi142.stdout:3/894: write d3/d1f/d24/d47/f134 [468928,30746] 0 2024-03-29T04:45:16.700 INFO:tasks.workunit.client.0.smithi142.stdout:3/895: readlink d3/d29/d8f/d5f/d64/l129 0 2024-03-29T04:45:16.701 INFO:tasks.workunit.client.1.smithi152.stdout:8/373: dwrite d1/f7b [0,4194304] 0 2024-03-29T04:45:16.701 INFO:tasks.workunit.client.0.smithi142.stdout:3/896: chown d3/d29/d2b/d40/db1/d124/d50/d126 6 1 2024-03-29T04:45:16.704 INFO:tasks.workunit.client.0.smithi142.stdout:3/897: mkdir d3/d29/d8f/d5f/db4/d135 0 2024-03-29T04:45:16.705 INFO:tasks.workunit.client.1.smithi152.stdout:8/374: dread d1/d2/d1d/f30 [4194304,4194304] 0 2024-03-29T04:45:16.705 INFO:tasks.workunit.client.0.smithi142.stdout:3/898: rename d3/d29 to d3/d29/d8f/d5f/d64/d136 22 2024-03-29T04:45:16.708 INFO:tasks.workunit.client.1.smithi152.stdout:8/375: creat d1/d2/d81/f84 x:0 0 0 2024-03-29T04:45:16.709 INFO:tasks.workunit.client.0.smithi142.stdout:3/899: rename d3/d29/d8f/d5f/d64/d6b/la7 to d3/d29/d2b/d40/db1/d124/d50/l137 0 2024-03-29T04:45:16.714 INFO:tasks.workunit.client.1.smithi152.stdout:8/376: dwrite d1/d21/d40/f6c [0,4194304] 0 2024-03-29T04:45:16.733 INFO:tasks.workunit.client.0.smithi142.stdout:3/900: dread d3/d29/d2b/d40/d67/d8b/f4e [0,4194304] 0 2024-03-29T04:45:16.737 INFO:tasks.workunit.client.1.smithi152.stdout:8/377: dread d1/d2/d1d/f23 [0,4194304] 0 2024-03-29T04:45:16.739 INFO:tasks.workunit.client.1.smithi152.stdout:8/378: mkdir d1/d2/d1d/d36/d85 0 2024-03-29T04:45:16.739 INFO:tasks.workunit.client.0.smithi142.stdout:3/901: dread d3/d29/d2b/d40/db1/d124/d50/d126/ffc [0,4194304] 0 2024-03-29T04:45:16.739 INFO:tasks.workunit.client.0.smithi142.stdout:3/902: dread - d3/d1f/d24/f10b zero size 2024-03-29T04:45:16.741 INFO:tasks.workunit.client.1.smithi152.stdout:8/379: dread d1/d2/d1d/f4f [0,4194304] 0 2024-03-29T04:45:16.741 INFO:tasks.workunit.client.0.smithi142.stdout:3/903: symlink d3/d29/d2b/d40/db1/d124/d50/d126/de9/l138 0 2024-03-29T04:45:16.744 INFO:tasks.workunit.client.0.smithi142.stdout:3/904: mknod d3/d29/d2b/d40/db1/d124/d50/d126/d79/dcc/c139 0 2024-03-29T04:45:16.746 INFO:tasks.workunit.client.0.smithi142.stdout:3/905: creat d3/d29/d2b/d40/d67/d8b/f13a x:0 0 0 2024-03-29T04:45:16.747 INFO:tasks.workunit.client.1.smithi152.stdout:8/380: dwrite d1/d2/d1d/d36/d5d/f71 [0,4194304] 0 2024-03-29T04:45:16.748 INFO:tasks.workunit.client.0.smithi142.stdout:3/906: rmdir d3/d29/d2b/d6a 39 2024-03-29T04:45:16.749 INFO:tasks.workunit.client.1.smithi152.stdout:8/381: fdatasync d1/d2/d1d/d42/f66 0 2024-03-29T04:45:16.751 INFO:tasks.workunit.client.0.smithi142.stdout:3/907: fdatasync d3/d29/d8f/d5f/db4/f133 0 2024-03-29T04:45:16.754 INFO:tasks.workunit.client.0.smithi142.stdout:3/908: fsync d3/d29/d2b/d40/db1/d124/d50/d126/d79/fce 0 2024-03-29T04:45:16.755 INFO:tasks.workunit.client.0.smithi142.stdout:3/909: write d3/fe3 [259404,92594] 0 2024-03-29T04:45:16.758 INFO:tasks.workunit.client.1.smithi152.stdout:8/382: dwrite d1/d2/d1d/f23 [4194304,4194304] 0 2024-03-29T04:45:16.763 INFO:tasks.workunit.client.1.smithi152.stdout:8/383: link d1/d2/c1e d1/d2/d59/c86 0 2024-03-29T04:45:16.763 INFO:tasks.workunit.client.1.smithi152.stdout:8/384: dread - d1/d2/d1d/f3a zero size 2024-03-29T04:45:16.763 INFO:tasks.workunit.client.1.smithi152.stdout:8/385: read - d1/d21/f78 zero size 2024-03-29T04:45:16.764 INFO:tasks.workunit.client.1.smithi152.stdout:8/386: fsync d1/d2/d3c/f69 0 2024-03-29T04:45:16.766 INFO:tasks.workunit.client.1.smithi152.stdout:8/387: symlink d1/d21/d40/d47/l87 0 2024-03-29T04:45:16.770 INFO:tasks.workunit.client.0.smithi142.stdout:3/910: dwrite d3/d1f/d24/d47/d87/fa3 [0,4194304] 0 2024-03-29T04:45:16.771 INFO:tasks.workunit.client.1.smithi152.stdout:4/340: sync 2024-03-29T04:45:16.771 INFO:tasks.workunit.client.1.smithi152.stdout:9/448: sync 2024-03-29T04:45:16.771 INFO:tasks.workunit.client.1.smithi152.stdout:7/431: sync 2024-03-29T04:45:16.771 INFO:tasks.workunit.client.1.smithi152.stdout:0/485: sync 2024-03-29T04:45:16.771 INFO:tasks.workunit.client.1.smithi152.stdout:6/543: sync 2024-03-29T04:45:16.771 INFO:tasks.workunit.client.1.smithi152.stdout:5/463: sync 2024-03-29T04:45:16.771 INFO:tasks.workunit.client.1.smithi152.stdout:2/335: sync 2024-03-29T04:45:16.771 INFO:tasks.workunit.client.1.smithi152.stdout:1/689: sync 2024-03-29T04:45:16.771 INFO:tasks.workunit.client.0.smithi142.stdout:3/911: stat d3/d1f/d24/d47/d87/fa3 0 2024-03-29T04:45:16.775 INFO:tasks.workunit.client.0.smithi142.stdout:3/912: creat d3/d29/d2b/d40/db1/d124/d50/d126/d79/d116/f13b x:0 0 0 2024-03-29T04:45:16.776 INFO:tasks.workunit.client.1.smithi152.stdout:0/486: fsync d2/f78 0 2024-03-29T04:45:16.778 INFO:tasks.workunit.client.1.smithi152.stdout:1/690: dread d5/f72 [0,4194304] 0 2024-03-29T04:45:16.779 INFO:tasks.workunit.client.0.smithi142.stdout:3/913: creat d3/d29/f13c x:0 0 0 2024-03-29T04:45:16.780 INFO:tasks.workunit.client.1.smithi152.stdout:2/336: unlink d0/ca 0 2024-03-29T04:45:16.782 INFO:tasks.workunit.client.1.smithi152.stdout:7/432: symlink d7/d8/d12/d35/d68/l7e 0 2024-03-29T04:45:16.783 INFO:tasks.workunit.client.1.smithi152.stdout:4/341: symlink d0/d7/d25/d17/d37/l69 0 2024-03-29T04:45:16.783 INFO:tasks.workunit.client.0.smithi142.stdout:3/914: mkdir d3/d29/d2b/d6a/d13d 0 2024-03-29T04:45:16.784 INFO:tasks.workunit.client.1.smithi152.stdout:1/691: chown d5/d7/d1b/d2e/fa3 2579066 1 2024-03-29T04:45:16.785 INFO:tasks.workunit.client.1.smithi152.stdout:9/449: unlink d5/d8/d31/d59/d6f/c8a 0 2024-03-29T04:45:16.788 INFO:tasks.workunit.client.1.smithi152.stdout:2/337: rmdir d0/d3/d30/d15/d22/d4c/d5a 39 2024-03-29T04:45:16.789 INFO:tasks.workunit.client.1.smithi152.stdout:5/464: dwrite d6/d16/d25/d2e/f8c [0,4194304] 0 2024-03-29T04:45:16.789 INFO:tasks.workunit.client.1.smithi152.stdout:0/487: truncate d2/da/df/d86/db8/f7b 250240 0 2024-03-29T04:45:16.790 INFO:tasks.workunit.client.0.smithi142.stdout:3/915: rmdir d3/d29/d2b 39 2024-03-29T04:45:16.790 INFO:tasks.workunit.client.1.smithi152.stdout:6/544: dwrite d1/d3/d1a/f2c [0,4194304] 0 2024-03-29T04:45:16.791 INFO:tasks.workunit.client.1.smithi152.stdout:7/433: fdatasync d7/d8/d12/d1d/d30/f3c 0 2024-03-29T04:45:16.793 INFO:tasks.workunit.client.0.smithi142.stdout:3/916: dread d3/d1f/d24/d47/f75 [0,4194304] 0 2024-03-29T04:45:16.793 INFO:tasks.workunit.client.1.smithi152.stdout:7/434: chown d7/d8/db 1 1 2024-03-29T04:45:16.796 INFO:tasks.workunit.client.1.smithi152.stdout:9/450: mkdir d5/d12/d2c/d9e 0 2024-03-29T04:45:16.796 INFO:tasks.workunit.client.0.smithi142.stdout:3/917: fsync d3/d29/d2b/d40/fea 0 2024-03-29T04:45:16.797 INFO:tasks.workunit.client.1.smithi152.stdout:5/465: mknod d6/d16/d25/d7e/c9a 0 2024-03-29T04:45:16.797 INFO:tasks.workunit.client.0.smithi142.stdout:3/918: stat d3/d29/d2b/f117 0 2024-03-29T04:45:16.797 INFO:tasks.workunit.client.1.smithi152.stdout:8/388: dread d1/d2/d3/f29 [0,4194304] 0 2024-03-29T04:45:16.798 INFO:tasks.workunit.client.1.smithi152.stdout:8/389: fdatasync d1/d2/d1d/d36/d4b/f68 0 2024-03-29T04:45:16.799 INFO:tasks.workunit.client.0.smithi142.stdout:3/919: fdatasync d3/d29/d2b/d40/db1/d124/d50/d126/ff8 0 2024-03-29T04:45:16.799 INFO:tasks.workunit.client.1.smithi152.stdout:8/390: write d1/d2/d3/f4e [658668,106291] 0 2024-03-29T04:45:16.801 INFO:tasks.workunit.client.1.smithi152.stdout:0/488: fsync d2/da/d3a/d61/f9a 0 2024-03-29T04:45:16.805 INFO:tasks.workunit.client.0.smithi142.stdout:3/920: rename d3/d29/d2b/d6a/f7b to d3/d29/d2b/d40/d67/f13e 0 2024-03-29T04:45:16.805 INFO:tasks.workunit.client.1.smithi152.stdout:7/435: rename d7/d8/db/df/f6b to d7/d8/d37/f7f 0 2024-03-29T04:45:16.807 INFO:tasks.workunit.client.1.smithi152.stdout:1/692: truncate d5/d7/d1b/d2e/d95/dad/d59/f89 2309532 0 2024-03-29T04:45:16.808 INFO:tasks.workunit.client.0.smithi142.stdout:3/921: chown d3/d29/d2b/d40/d67/d8b/d6/f100 286617920 1 2024-03-29T04:45:16.809 INFO:tasks.workunit.client.1.smithi152.stdout:7/436: write d7/f59 [246918,114795] 0 2024-03-29T04:45:16.810 INFO:tasks.workunit.client.0.smithi142.stdout:3/922: chown d3/d29/d8f/d5f/db4/d135 0 1 2024-03-29T04:45:16.810 INFO:tasks.workunit.client.1.smithi152.stdout:5/466: symlink d6/d1d/d3b/l9b 0 2024-03-29T04:45:16.811 INFO:tasks.workunit.client.0.smithi142.stdout:3/923: rmdir d3/d29/d2b/d40 39 2024-03-29T04:45:16.812 INFO:tasks.workunit.client.1.smithi152.stdout:0/489: link d2/da/d17/d57/f64 d2/da/df/d86/fba 0 2024-03-29T04:45:16.813 INFO:tasks.workunit.client.1.smithi152.stdout:1/693: fsync d5/d7/d1b/d2e/d95/dad/f46 0 2024-03-29T04:45:16.813 INFO:tasks.workunit.client.1.smithi152.stdout:2/338: dwrite d0/d3/d30/f33 [4194304,4194304] 0 2024-03-29T04:45:16.813 INFO:tasks.workunit.client.0.smithi142.stdout:3/924: creat d3/db9/d11e/f13f x:0 0 0 2024-03-29T04:45:16.815 INFO:tasks.workunit.client.0.smithi142.stdout:3/925: truncate d3/d1f/d24/d47/f90 4712878 0 2024-03-29T04:45:16.816 INFO:tasks.workunit.client.1.smithi152.stdout:6/545: rename d1/d3/d9/d13/d3e/d54/l77 to d1/d3/df/d8a/lba 0 2024-03-29T04:45:16.816 INFO:tasks.workunit.client.1.smithi152.stdout:4/342: truncate d0/d7/d40/d2a/f63 1139889 0 2024-03-29T04:45:16.818 INFO:tasks.workunit.client.1.smithi152.stdout:4/343: chown d0/d7/d40/d2a/f4a 0 1 2024-03-29T04:45:16.819 INFO:tasks.workunit.client.0.smithi142.stdout:3/926: fdatasync d3/d29/d2b/d40/db1/d124/d50/d126/f3f 0 2024-03-29T04:45:16.820 INFO:tasks.workunit.client.1.smithi152.stdout:0/490: mkdir d2/da/df/dbb 0 2024-03-29T04:45:16.822 INFO:tasks.workunit.client.1.smithi152.stdout:4/344: chown d0/d7/d25/l49 1421624184 1 2024-03-29T04:45:16.823 INFO:tasks.workunit.client.1.smithi152.stdout:1/694: creat d5/d7/d38/fe4 x:0 0 0 2024-03-29T04:45:16.823 INFO:tasks.workunit.client.1.smithi152.stdout:9/451: rename d5/d12/d2c/c9d to d5/d8/d31/d75/c9f 0 2024-03-29T04:45:16.825 INFO:tasks.workunit.client.1.smithi152.stdout:7/437: creat d7/d8/d37/d75/f80 x:0 0 0 2024-03-29T04:45:16.826 INFO:tasks.workunit.client.0.smithi142.stdout:3/927: symlink d3/d29/d2b/d40/d67/d107/l140 0 2024-03-29T04:45:16.826 INFO:tasks.workunit.client.1.smithi152.stdout:4/345: chown d0/d7/d40/d23/f53 4893 1 2024-03-29T04:45:16.829 INFO:tasks.workunit.client.0.smithi142.stdout:3/928: creat d3/d29/d2b/d40/d67/d8b/d6/f141 x:0 0 0 2024-03-29T04:45:16.830 INFO:tasks.workunit.client.1.smithi152.stdout:2/339: creat d0/d3/d64/f6e x:0 0 0 2024-03-29T04:45:16.830 INFO:tasks.workunit.client.1.smithi152.stdout:0/491: unlink d2/f9 0 2024-03-29T04:45:16.830 INFO:tasks.workunit.client.1.smithi152.stdout:6/546: dwrite d1/d3/d1a/f1d [0,4194304] 0 2024-03-29T04:45:16.832 INFO:tasks.workunit.client.1.smithi152.stdout:6/547: chown d1/d3/df/d84/f57 126 1 2024-03-29T04:45:16.832 INFO:tasks.workunit.client.1.smithi152.stdout:5/467: rename d6/d16/d47/c8b to d6/d1d/d3b/d55/c9c 0 2024-03-29T04:45:16.833 INFO:tasks.workunit.client.1.smithi152.stdout:6/548: stat d1/d3/d9/d13/f31 0 2024-03-29T04:45:16.833 INFO:tasks.workunit.client.1.smithi152.stdout:1/695: creat d5/d7/d4d/db1/fe5 x:0 0 0 2024-03-29T04:45:16.834 INFO:tasks.workunit.client.1.smithi152.stdout:9/452: creat d5/d12/d2c/d6e/fa0 x:0 0 0 2024-03-29T04:45:16.835 INFO:tasks.workunit.client.0.smithi142.stdout:3/929: unlink d3/d29/d2b/d40/d67/d8b/f4e 0 2024-03-29T04:45:16.835 INFO:tasks.workunit.client.1.smithi152.stdout:1/696: read d5/d7/d1b/d2e/f8d [1762067,42470] 0 2024-03-29T04:45:16.837 INFO:tasks.workunit.client.1.smithi152.stdout:8/391: dread d1/d2/d3c/f69 [0,4194304] 0 2024-03-29T04:45:16.838 INFO:tasks.workunit.client.1.smithi152.stdout:4/346: mkdir d0/d7/d40/d23/d6a 0 2024-03-29T04:45:16.838 INFO:tasks.workunit.client.1.smithi152.stdout:8/392: chown d1/f7b 0 1 2024-03-29T04:45:16.838 INFO:tasks.workunit.client.1.smithi152.stdout:4/347: readlink d0/l9 0 2024-03-29T04:45:16.839 INFO:tasks.workunit.client.1.smithi152.stdout:8/393: dread - d1/d2/d1d/d36/d4b/f68 zero size 2024-03-29T04:45:16.840 INFO:tasks.workunit.client.1.smithi152.stdout:0/492: symlink d2/d31/lbc 0 2024-03-29T04:45:16.840 INFO:tasks.workunit.client.1.smithi152.stdout:4/348: read d0/f4 [1993973,41365] 0 2024-03-29T04:45:16.840 INFO:tasks.workunit.client.1.smithi152.stdout:8/394: stat d1/d21/d40/d47/l87 0 2024-03-29T04:45:16.841 INFO:tasks.workunit.client.0.smithi142.stdout:3/930: rename d3/d29/d2b/d40/db1/d12e/d11f/c10d to d3/c142 0 2024-03-29T04:45:16.844 INFO:tasks.workunit.client.1.smithi152.stdout:9/453: rename d5/db/c90 to d5/d12/d2c/d6e/d73/d8e/ca1 0 2024-03-29T04:45:16.846 INFO:tasks.workunit.client.1.smithi152.stdout:2/340: symlink d0/d3/d30/d15/d22/d4c/l6f 0 2024-03-29T04:45:16.847 INFO:tasks.workunit.client.0.smithi142.stdout:3/931: dwrite d3/d29/d2b/d40/db1/d124/d50/d126/d6c/f119 [0,4194304] 0 2024-03-29T04:45:16.848 INFO:tasks.workunit.client.1.smithi152.stdout:8/395: write d1/d2/d3/f29 [4862895,48645] 0 2024-03-29T04:45:16.850 INFO:tasks.workunit.client.1.smithi152.stdout:6/549: truncate d1/d3/d9/d13/f33 7633894 0 2024-03-29T04:45:16.850 INFO:tasks.workunit.client.0.smithi142.stdout:3/932: symlink d3/d1f/l143 0 2024-03-29T04:45:16.851 INFO:tasks.workunit.client.1.smithi152.stdout:4/349: dwrite d0/f4 [0,4194304] 0 2024-03-29T04:45:16.853 INFO:tasks.workunit.client.1.smithi152.stdout:7/438: creat d7/d8/d12/f81 x:0 0 0 2024-03-29T04:45:16.857 INFO:tasks.workunit.client.1.smithi152.stdout:9/454: symlink d5/d12/d2c/d6e/d73/d8e/la2 0 2024-03-29T04:45:16.861 INFO:tasks.workunit.client.1.smithi152.stdout:4/350: symlink d0/d27/l6b 0 2024-03-29T04:45:16.862 INFO:tasks.workunit.client.1.smithi152.stdout:5/468: getdents d6/d16/d47/d56 0 2024-03-29T04:45:16.863 INFO:tasks.workunit.client.1.smithi152.stdout:8/396: creat d1/d2/d59/f88 x:0 0 0 2024-03-29T04:45:16.865 INFO:tasks.workunit.client.1.smithi152.stdout:2/341: link d0/d3/d30/d15/d22/f37 d0/d3/d30/d15/d22/f70 0 2024-03-29T04:45:16.867 INFO:tasks.workunit.client.1.smithi152.stdout:9/455: rename d5/d12/d2c/d6e/d73/f7d to d5/d12/d2c/d6e/d73/d8e/fa3 0 2024-03-29T04:45:16.867 INFO:tasks.workunit.client.0.smithi142.stdout:3/933: write d3/d29/d2b/d40/db1/d124/f9d [2785126,117181] 0 2024-03-29T04:45:16.868 INFO:tasks.workunit.client.1.smithi152.stdout:1/697: dwrite d5/d7/d1b/d2e/d95/dad/f10 [4194304,4194304] 0 2024-03-29T04:45:16.869 INFO:tasks.workunit.client.1.smithi152.stdout:6/550: mknod d1/d3/d1a/d1b/d29/cbb 0 2024-03-29T04:45:16.869 INFO:tasks.workunit.client.1.smithi152.stdout:2/342: dread - d0/d3/df/d2c/f2d zero size 2024-03-29T04:45:16.871 INFO:tasks.workunit.client.1.smithi152.stdout:7/439: truncate d7/d8/d12/f7b 1142796 0 2024-03-29T04:45:16.871 INFO:tasks.workunit.client.1.smithi152.stdout:5/469: read d6/d16/d47/d56/d63/f6d [1028735,114372] 0 2024-03-29T04:45:16.872 INFO:tasks.workunit.client.0.smithi142.stdout:3/934: creat d3/d132/f144 x:0 0 0 2024-03-29T04:45:16.872 INFO:tasks.workunit.client.1.smithi152.stdout:1/698: fdatasync d5/d7/d8/fb 0 2024-03-29T04:45:16.875 INFO:tasks.workunit.client.0.smithi142.stdout:3/935: dread d3/d29/d2b/d40/d67/fb6 [0,4194304] 0 2024-03-29T04:45:16.876 INFO:tasks.workunit.client.1.smithi152.stdout:8/397: rename d1/f10 to d1/d21/d40/d6a/f89 0 2024-03-29T04:45:16.878 INFO:tasks.workunit.client.1.smithi152.stdout:4/351: mkdir d0/d7/d40/d23/d55/d6c 0 2024-03-29T04:45:16.880 INFO:tasks.workunit.client.1.smithi152.stdout:6/551: mkdir d1/d3/d1a/d1b/dbc 0 2024-03-29T04:45:16.880 INFO:tasks.workunit.client.1.smithi152.stdout:1/699: creat d5/d7/d8/d13/d96/fe6 x:0 0 0 2024-03-29T04:45:16.882 INFO:tasks.workunit.client.1.smithi152.stdout:0/493: truncate d2/d29/f37 6670 0 2024-03-29T04:45:16.883 INFO:tasks.workunit.client.0.smithi142.stdout:3/936: dwrite d3/d29/d2b/d40/d67/fe0 [0,4194304] 0 2024-03-29T04:45:16.885 INFO:tasks.workunit.client.1.smithi152.stdout:6/552: creat d1/d3/d1a/fbd x:0 0 0 2024-03-29T04:45:16.885 INFO:tasks.workunit.client.0.smithi142.stdout:3/937: fdatasync d3/d29/d2b/d40/db1/d124/d50/d126/d56/fcd 0 2024-03-29T04:45:16.886 INFO:tasks.workunit.client.1.smithi152.stdout:3/422: sync 2024-03-29T04:45:16.888 INFO:tasks.workunit.client.1.smithi152.stdout:4/352: chown d0/d7/d25/d17/d37/c5f 2530 1 2024-03-29T04:45:16.891 INFO:tasks.workunit.client.0.smithi142.stdout:3/938: rename d3/d29/d2b/d40/db1/d124/f9d to d3/d29/d2b/d6a/d13d/f145 0 2024-03-29T04:45:16.892 INFO:tasks.workunit.client.1.smithi152.stdout:5/470: dwrite d6/d16/d25/d2e/f54 [0,4194304] 0 2024-03-29T04:45:16.892 INFO:tasks.workunit.client.1.smithi152.stdout:0/494: write d2/f20 [2247197,56711] 0 2024-03-29T04:45:16.893 INFO:tasks.workunit.client.1.smithi152.stdout:7/440: dwrite d7/d8/d12/f6f [0,4194304] 0 2024-03-29T04:45:16.897 INFO:tasks.workunit.client.1.smithi152.stdout:2/343: rename d0/d3/d30/l4a to d0/l71 0 2024-03-29T04:45:16.897 INFO:tasks.workunit.client.0.smithi142.stdout:3/939: readlink d3/d29/d2b/d40/d67/d8b/df7/l106 0 2024-03-29T04:45:16.898 INFO:tasks.workunit.client.1.smithi152.stdout:2/344: truncate d0/d3/d5f/f63 882224 0 2024-03-29T04:45:16.902 INFO:tasks.workunit.client.0.smithi142.stdout:3/940: fsync d3/d29/d8f/d5f/db4/f133 0 2024-03-29T04:45:16.902 INFO:tasks.workunit.client.1.smithi152.stdout:3/423: read - d7/d19/d1b/d1e/d4e/f70 zero size 2024-03-29T04:45:16.903 INFO:tasks.workunit.client.0.smithi142.stdout:3/941: dread - d3/db9/d11e/f13f zero size 2024-03-29T04:45:16.905 INFO:tasks.workunit.client.1.smithi152.stdout:0/495: chown d2/da/c28 311 1 2024-03-29T04:45:16.906 INFO:tasks.workunit.client.0.smithi142.stdout:3/942: mknod d3/d29/d2b/d40/db1/d124/d50/c146 0 2024-03-29T04:45:16.909 INFO:tasks.workunit.client.1.smithi152.stdout:5/471: mkdir d6/d16/d25/d2e/d9d 0 2024-03-29T04:45:16.909 INFO:tasks.workunit.client.0.smithi142.stdout:3/943: mknod d3/d1f/c147 0 2024-03-29T04:45:16.910 INFO:tasks.workunit.client.1.smithi152.stdout:8/398: dwrite d1/d2/d3/f44 [0,4194304] 0 2024-03-29T04:45:16.912 INFO:tasks.workunit.client.1.smithi152.stdout:9/456: rename d5/d12/f19 to d5/d8/d31/d59/fa4 0 2024-03-29T04:45:16.912 INFO:tasks.workunit.client.0.smithi142.stdout:3/944: chown d3/d29/d2b/d40/db1/d124/d50/d126/de9/l138 1727393 1 2024-03-29T04:45:16.914 INFO:tasks.workunit.client.1.smithi152.stdout:1/700: dwrite d5/d7/d8/fc1 [0,4194304] 0 2024-03-29T04:45:16.917 INFO:tasks.workunit.client.0.smithi142.stdout:3/945: rename d3/d1f to d3/d1f/d24/d47/d87/d148 22 2024-03-29T04:45:16.918 INFO:tasks.workunit.client.1.smithi152.stdout:0/496: write d2/da/d17/f4c [4412929,118495] 0 2024-03-29T04:45:16.930 INFO:tasks.workunit.client.1.smithi152.stdout:3/424: dwrite d7/d46/f6d [0,4194304] 0 2024-03-29T04:45:16.937 INFO:tasks.workunit.client.1.smithi152.stdout:2/345: dwrite d0/d3/df/d2c/f55 [0,4194304] 0 2024-03-29T04:45:16.939 INFO:tasks.workunit.client.1.smithi152.stdout:6/553: dwrite d1/d3/d9/d13/f26 [0,4194304] 0 2024-03-29T04:45:16.941 INFO:tasks.workunit.client.1.smithi152.stdout:7/441: dwrite d7/d8/d12/d35/f63 [0,4194304] 0 2024-03-29T04:45:16.942 INFO:tasks.workunit.client.1.smithi152.stdout:5/472: mknod d6/d16/d25/d2e/d9d/c9e 0 2024-03-29T04:45:16.943 INFO:tasks.workunit.client.1.smithi152.stdout:1/701: mknod d5/d7/d1b/d2e/d95/dad/d12/d77/ce7 0 2024-03-29T04:45:16.945 INFO:tasks.workunit.client.1.smithi152.stdout:4/353: dread d0/d7/d40/ff [0,4194304] 0 2024-03-29T04:45:16.947 INFO:tasks.workunit.client.1.smithi152.stdout:1/702: write d5/fcc [901380,118344] 0 2024-03-29T04:45:16.947 INFO:tasks.workunit.client.1.smithi152.stdout:7/442: rmdir d7/d8/d12/d15 39 2024-03-29T04:45:16.948 INFO:tasks.workunit.client.1.smithi152.stdout:6/554: dread - d1/d3/d1a/fbd zero size 2024-03-29T04:45:16.957 INFO:tasks.workunit.client.1.smithi152.stdout:1/703: unlink d5/d7/d1b/f5e 0 2024-03-29T04:45:16.963 INFO:tasks.workunit.client.1.smithi152.stdout:1/704: chown d5/fcc 805 1 2024-03-29T04:45:16.964 INFO:tasks.workunit.client.1.smithi152.stdout:3/425: dwrite d7/d19/d1b/d1e/d22/f6f [0,4194304] 0 2024-03-29T04:45:16.968 INFO:tasks.workunit.client.1.smithi152.stdout:0/497: dwrite d2/da/d17/d57/f62 [0,4194304] 0 2024-03-29T04:45:16.969 INFO:tasks.workunit.client.1.smithi152.stdout:9/457: truncate d5/f61 3223287 0 2024-03-29T04:45:16.971 INFO:tasks.workunit.client.1.smithi152.stdout:8/399: dread d1/d21/f46 [0,4194304] 0 2024-03-29T04:45:16.974 INFO:tasks.workunit.client.1.smithi152.stdout:5/473: rename d6/d1b/f31 to d6/d16/d25/f9f 0 2024-03-29T04:45:16.975 INFO:tasks.workunit.client.0.smithi142.stdout:3/946: truncate d3/d29/d2b/d40/db1/d124/d50/d126/d56/fcd 170522 0 2024-03-29T04:45:16.975 INFO:tasks.workunit.client.0.smithi142.stdout:3/947: write d3/fe3 [788908,118596] 0 2024-03-29T04:45:16.976 INFO:tasks.workunit.client.1.smithi152.stdout:7/443: creat d7/d8/db/f82 x:0 0 0 2024-03-29T04:45:16.977 INFO:tasks.workunit.client.1.smithi152.stdout:2/346: dwrite d0/d3/df/f43 [0,4194304] 0 2024-03-29T04:45:16.977 INFO:tasks.workunit.client.0.smithi142.stdout:3/948: write d3/d29/d8f/d5f/f103 [1251664,52682] 0 2024-03-29T04:45:16.979 INFO:tasks.workunit.client.0.smithi142.stdout:3/949: rmdir d3/d29/d8f/d5f/db4 39 2024-03-29T04:45:16.981 INFO:tasks.workunit.client.1.smithi152.stdout:2/347: chown d0/d3/df/d2c/l4f 97484 1 2024-03-29T04:45:16.984 INFO:tasks.workunit.client.1.smithi152.stdout:0/498: rmdir d2/da/d3a/d6f 39 2024-03-29T04:45:16.985 INFO:tasks.workunit.client.0.smithi142.stdout:3/950: mkdir d3/d29/d2b/d40/d67/d8b/d149 0 2024-03-29T04:45:16.986 INFO:tasks.workunit.client.1.smithi152.stdout:1/705: rename d5/d7/d1b/d2e/d95/dad/d2b/fab to d5/d7/d1b/d2e/d95/dad/d2b/fe8 0 2024-03-29T04:45:16.987 INFO:tasks.workunit.client.0.smithi142.stdout:3/951: chown d3/d29/d8f/d5f/d64/c70 5826723 1 2024-03-29T04:45:16.989 INFO:tasks.workunit.client.1.smithi152.stdout:1/706: stat d5/d7/d1b/d2e/d95 0 2024-03-29T04:45:16.989 INFO:tasks.workunit.client.1.smithi152.stdout:7/444: truncate d7/d8/d12/d15/f76 389319 0 2024-03-29T04:45:16.990 INFO:tasks.workunit.client.1.smithi152.stdout:8/400: mkdir d1/d8a 0 2024-03-29T04:45:16.992 INFO:tasks.workunit.client.0.smithi142.stdout:3/952: unlink d3/d29/d2b/d40/db1/d124/d50/d126/de9/l110 0 2024-03-29T04:45:16.994 INFO:tasks.workunit.client.1.smithi152.stdout:2/348: dwrite d0/d3/d64/f6e [0,4194304] 0 2024-03-29T04:45:16.994 INFO:tasks.workunit.client.1.smithi152.stdout:8/401: chown d1/d2/d1d/d36/d4b/f68 2349252 1 2024-03-29T04:45:16.994 INFO:tasks.workunit.client.1.smithi152.stdout:9/458: rename d5/l10 to d5/db/la5 0 2024-03-29T04:45:16.995 INFO:tasks.workunit.client.1.smithi152.stdout:7/445: mkdir d7/d8/d12/d1d/d41/d83 0 2024-03-29T04:45:16.998 INFO:tasks.workunit.client.1.smithi152.stdout:8/402: mkdir d1/d21/d8b 0 2024-03-29T04:45:16.998 INFO:tasks.workunit.client.1.smithi152.stdout:2/349: fdatasync d0/d3/f42 0 2024-03-29T04:45:16.998 INFO:tasks.workunit.client.1.smithi152.stdout:9/459: truncate d5/f5c 5013953 0 2024-03-29T04:45:17.000 INFO:tasks.workunit.client.1.smithi152.stdout:7/446: readlink d7/d8/d12/d35/l23 0 2024-03-29T04:45:17.000 INFO:tasks.workunit.client.1.smithi152.stdout:5/474: rename d6/d16/d25/d7e to d6/d16/d25/d2e/da0 0 2024-03-29T04:45:17.000 INFO:tasks.workunit.client.0.smithi142.stdout:3/953: dwrite d3/d29/d2b/d40/d67/f12a [0,4194304] 0 2024-03-29T04:45:17.003 INFO:tasks.workunit.client.1.smithi152.stdout:1/707: getdents d5/d7/d1b/d2e/d95/d9d/da7 0 2024-03-29T04:45:17.004 INFO:tasks.workunit.client.0.smithi142.stdout:3/954: creat d3/d29/d8f/d5f/d64/d6b/f14a x:0 0 0 2024-03-29T04:45:17.006 INFO:tasks.workunit.client.1.smithi152.stdout:0/499: dwrite d2/da/d3a/d6f/f85 [0,4194304] 0 2024-03-29T04:45:17.007 INFO:tasks.workunit.client.1.smithi152.stdout:9/460: write d5/d12/d2c/f5f [1791303,63445] 0 2024-03-29T04:45:17.008 INFO:tasks.workunit.client.0.smithi142.stdout:3/955: fdatasync d3/d29/d2b/d40/d67/d8b/d6/f20 0 2024-03-29T04:45:17.008 INFO:tasks.workunit.client.1.smithi152.stdout:2/350: creat d0/d3/d30/d15/d53/d54/d57/f72 x:0 0 0 2024-03-29T04:45:17.013 INFO:tasks.workunit.client.1.smithi152.stdout:6/555: write d1/d3/df/f66 [610672,41314] 0 2024-03-29T04:45:17.014 INFO:tasks.workunit.client.1.smithi152.stdout:5/475: rename d6/d16/d25/d2e/da0/f90 to d6/d16/d47/fa1 0 2024-03-29T04:45:17.015 INFO:tasks.workunit.client.1.smithi152.stdout:9/461: mknod d5/d12/d2c/d6e/d91/ca6 0 2024-03-29T04:45:17.016 INFO:tasks.workunit.client.0.smithi142.stdout:3/956: unlink d3/d29/d2b/d40/d67/d8b/l10 0 2024-03-29T04:45:17.016 INFO:tasks.workunit.client.1.smithi152.stdout:2/351: symlink d0/d3/df/d2c/d45/l73 0 2024-03-29T04:45:17.018 INFO:tasks.workunit.client.0.smithi142.stdout:3/957: rmdir d3/d29/d2b/d40/db1/d124/d50 39 2024-03-29T04:45:17.018 INFO:tasks.workunit.client.1.smithi152.stdout:6/556: unlink d1/d3/d91/cb4 0 2024-03-29T04:45:17.024 INFO:tasks.workunit.client.1.smithi152.stdout:5/476: creat d6/d1d/d30/fa2 x:0 0 0 2024-03-29T04:45:17.026 INFO:tasks.workunit.client.1.smithi152.stdout:8/403: dwrite d1/d2/d1d/f23 [4194304,4194304] 0 2024-03-29T04:45:17.026 INFO:tasks.workunit.client.0.smithi142.stdout:3/958: dwrite d3/d29/d2b/d40/d67/d8b/f91 [4194304,4194304] 0 2024-03-29T04:45:17.027 INFO:tasks.workunit.client.1.smithi152.stdout:2/352: mkdir d0/d3/d30/d15/d22/d74 0 2024-03-29T04:45:17.028 INFO:tasks.workunit.client.1.smithi152.stdout:0/500: dwrite d2/d31/f36 [0,4194304] 0 2024-03-29T04:45:17.028 INFO:tasks.workunit.client.1.smithi152.stdout:9/462: mknod d5/ca7 0 2024-03-29T04:45:17.033 INFO:tasks.workunit.client.0.smithi142.stdout:3/959: creat d3/d29/d2b/d40/d67/d8b/df7/d12b/f14b x:0 0 0 2024-03-29T04:45:17.034 INFO:tasks.workunit.client.1.smithi152.stdout:3/426: dwrite d7/d19/d1b/d1e/d22/f2d [0,4194304] 0 2024-03-29T04:45:17.034 INFO:tasks.workunit.client.1.smithi152.stdout:9/463: chown d5/d8/d31/c77 8 1 2024-03-29T04:45:17.041 INFO:tasks.workunit.client.1.smithi152.stdout:2/353: truncate d0/d3/df/f24 4954882 0 2024-03-29T04:45:17.042 INFO:tasks.workunit.client.0.smithi142.stdout:3/960: write d3/d29/d8f/d5f/db4/f8d [2234466,65249] 0 2024-03-29T04:45:17.044 INFO:tasks.workunit.client.1.smithi152.stdout:7/447: truncate d7/d8/d12/d1d/d41/d45/f55 2485143 0 2024-03-29T04:45:17.044 INFO:tasks.workunit.client.1.smithi152.stdout:3/427: symlink d7/d46/l74 0 2024-03-29T04:45:17.047 INFO:tasks.workunit.client.0.smithi142.stdout:3/961: readlink d3/d29/d2b/d40/db1/d124/d50/d126/de9/lef 0 2024-03-29T04:45:17.047 INFO:tasks.workunit.client.0.smithi142.stdout:3/962: chown d3/d29/d2b/d40/d67/d8b/d149 752678783 1 2024-03-29T04:45:17.051 INFO:tasks.workunit.client.1.smithi152.stdout:1/708: dread d5/d7/d6a/f6e [0,4194304] 0 2024-03-29T04:45:17.052 INFO:tasks.workunit.client.1.smithi152.stdout:3/428: mkdir d7/d1c/d41/d61/d75 0 2024-03-29T04:45:17.053 INFO:tasks.workunit.client.1.smithi152.stdout:6/557: write d1/d3/d1a/d1b/d29/f5f [4395977,19717] 0 2024-03-29T04:45:17.053 INFO:tasks.workunit.client.0.smithi142.stdout:3/963: creat d3/d29/d2b/d40/db1/d124/f14c x:0 0 0 2024-03-29T04:45:17.055 INFO:tasks.workunit.client.1.smithi152.stdout:8/404: truncate d1/d2/d1d/f3d 1711472 0 2024-03-29T04:45:17.056 INFO:tasks.workunit.client.1.smithi152.stdout:0/501: creat d2/da/d17/fbd x:0 0 0 2024-03-29T04:45:17.056 INFO:tasks.workunit.client.1.smithi152.stdout:2/354: mknod d0/d3/c75 0 2024-03-29T04:45:17.057 INFO:tasks.workunit.client.1.smithi152.stdout:3/429: truncate d7/d19/f5c 904182 0 2024-03-29T04:45:17.059 INFO:tasks.workunit.client.1.smithi152.stdout:5/477: link d6/d16/d25/d2e/l82 d6/d1d/la3 0 2024-03-29T04:45:17.059 INFO:tasks.workunit.client.1.smithi152.stdout:6/558: mknod d1/d3/d1a/d1b/cbe 0 2024-03-29T04:45:17.060 INFO:tasks.workunit.client.1.smithi152.stdout:1/709: dread d5/d7/d8/d13/d19/f68 [0,4194304] 0 2024-03-29T04:45:17.060 INFO:tasks.workunit.client.1.smithi152.stdout:5/478: chown d6/d1d/c68 61963 1 2024-03-29T04:45:17.061 INFO:tasks.workunit.client.1.smithi152.stdout:0/502: mkdir d2/da/d17/d57/d9c/dbe 0 2024-03-29T04:45:17.061 INFO:tasks.workunit.client.1.smithi152.stdout:8/405: creat d1/d8a/f8c x:0 0 0 2024-03-29T04:45:17.063 INFO:tasks.workunit.client.1.smithi152.stdout:6/559: stat d1/d3/d9/d13/d28/f34 0 2024-03-29T04:45:17.063 INFO:tasks.workunit.client.1.smithi152.stdout:8/406: read - d1/d2/d1d/d42/f66 zero size 2024-03-29T04:45:17.064 INFO:tasks.workunit.client.1.smithi152.stdout:2/355: creat d0/d3/df/f76 x:0 0 0 2024-03-29T04:45:17.064 INFO:tasks.workunit.client.1.smithi152.stdout:6/560: read - d1/d3/d1a/f9f zero size 2024-03-29T04:45:17.065 INFO:tasks.workunit.client.1.smithi152.stdout:3/430: getdents d7/d1c/d41/d73 0 2024-03-29T04:45:17.068 INFO:tasks.workunit.client.0.smithi142.stdout:3/964: read d3/d29/d8f/d5f/db4/f8d [3475836,49654] 0 2024-03-29T04:45:17.069 INFO:tasks.workunit.client.0.smithi142.stdout:3/965: stat d3/d29/d2b/f21 0 2024-03-29T04:45:17.069 INFO:tasks.workunit.client.1.smithi152.stdout:8/407: mknod d1/d2/d1d/d42/c8d 0 2024-03-29T04:45:17.070 INFO:tasks.workunit.client.1.smithi152.stdout:2/356: dread - d0/d3/d30/d15/f26 zero size 2024-03-29T04:45:17.071 INFO:tasks.workunit.client.1.smithi152.stdout:6/561: mknod d1/d3/d9/d99/cbf 0 2024-03-29T04:45:17.072 INFO:tasks.workunit.client.1.smithi152.stdout:1/710: rename d5/d7/d8/d13/d19/l66 to d5/d7/le9 0 2024-03-29T04:45:17.073 INFO:tasks.workunit.client.1.smithi152.stdout:8/408: rmdir d1/d21 39 2024-03-29T04:45:17.074 INFO:tasks.workunit.client.1.smithi152.stdout:0/503: dwrite d2/da/d3a/f87 [0,4194304] 0 2024-03-29T04:45:17.075 INFO:tasks.workunit.client.1.smithi152.stdout:3/431: truncate d7/d19/d1b/d1e/d4e/f56 25094 0 2024-03-29T04:45:17.076 INFO:tasks.workunit.client.1.smithi152.stdout:6/562: mkdir d1/d3/d9/d13/dc0 0 2024-03-29T04:45:17.078 INFO:tasks.workunit.client.1.smithi152.stdout:7/448: write d7/d8/d12/d1d/f62 [2004106,45722] 0 2024-03-29T04:45:17.078 INFO:tasks.workunit.client.1.smithi152.stdout:9/464: write d5/d12/d2c/f8f [1721472,118084] 0 2024-03-29T04:45:17.079 INFO:tasks.workunit.client.1.smithi152.stdout:3/432: dread - d7/d19/d1b/d59/f68 zero size 2024-03-29T04:45:17.080 INFO:tasks.workunit.client.1.smithi152.stdout:1/711: read d5/d7/d1b/d2e/d95/dad/f73 [2397307,8072] 0 2024-03-29T04:45:17.080 INFO:tasks.workunit.client.1.smithi152.stdout:8/409: dread - d1/d2/d3/f80 zero size 2024-03-29T04:45:17.081 INFO:tasks.workunit.client.1.smithi152.stdout:2/357: dread d0/d3/d30/d15/f2b [0,4194304] 0 2024-03-29T04:45:17.084 INFO:tasks.workunit.client.1.smithi152.stdout:7/449: mkdir d7/d8/d12/d35/d68/d84 0 2024-03-29T04:45:17.084 INFO:tasks.workunit.client.1.smithi152.stdout:0/504: creat d2/da/d17/db5/fbf x:0 0 0 2024-03-29T04:45:17.085 INFO:tasks.workunit.client.1.smithi152.stdout:8/410: write d1/d2/d3c/f7a [795006,66711] 0 2024-03-29T04:45:17.085 INFO:tasks.workunit.client.1.smithi152.stdout:8/411: readlink d1/la 0 2024-03-29T04:45:17.086 INFO:tasks.workunit.client.1.smithi152.stdout:6/563: rename d1/d3/d9/c35 to d1/d3/df/d8a/d98/cc1 0 2024-03-29T04:45:17.086 INFO:tasks.workunit.client.1.smithi152.stdout:0/505: fdatasync d2/d31/f36 0 2024-03-29T04:45:17.086 INFO:tasks.workunit.client.1.smithi152.stdout:3/433: creat d7/d19/d1b/d1e/d4e/f76 x:0 0 0 2024-03-29T04:45:17.088 INFO:tasks.workunit.client.1.smithi152.stdout:1/712: unlink d5/d7/d38/fe4 0 2024-03-29T04:45:17.088 INFO:tasks.workunit.client.1.smithi152.stdout:2/358: mkdir d0/d3/d30/d15/d77 0 2024-03-29T04:45:17.088 INFO:tasks.workunit.client.1.smithi152.stdout:8/412: chown d1/d2/d3/c27 4622 1 2024-03-29T04:45:17.089 INFO:tasks.workunit.client.1.smithi152.stdout:6/564: write d1/d3/d9/d13/f31 [1549755,25788] 0 2024-03-29T04:45:17.091 INFO:tasks.workunit.client.1.smithi152.stdout:7/450: mknod d7/d8/d12/d35/d68/d84/c85 0 2024-03-29T04:45:17.092 INFO:tasks.workunit.client.1.smithi152.stdout:3/434: rename d7/d19/f57 to d7/d1c/d41/d73/f77 0 2024-03-29T04:45:17.092 INFO:tasks.workunit.client.0.smithi142.stdout:3/966: dread d3/f3c [0,4194304] 0 2024-03-29T04:45:17.093 INFO:tasks.workunit.client.1.smithi152.stdout:2/359: write d0/d3/d30/d15/d53/d54/d57/f72 [257203,90897] 0 2024-03-29T04:45:17.093 INFO:tasks.workunit.client.1.smithi152.stdout:9/465: dread d5/d12/d2c/d6e/d73/f7f [0,4194304] 0 2024-03-29T04:45:17.093 INFO:tasks.workunit.client.1.smithi152.stdout:8/413: creat d1/d2/d3/f8e x:0 0 0 2024-03-29T04:45:17.097 INFO:tasks.workunit.client.0.smithi142.stdout:3/967: creat d3/d29/d2b/d40/db1/d124/d50/d126/de9/f14d x:0 0 0 2024-03-29T04:45:17.097 INFO:tasks.workunit.client.1.smithi152.stdout:1/713: mknod d5/d7/d1b/d2e/d95/dad/d12/d77/cea 0 2024-03-29T04:45:17.097 INFO:tasks.workunit.client.1.smithi152.stdout:7/451: write d7/d8/db/f82 [481728,7865] 0 2024-03-29T04:45:17.097 INFO:tasks.workunit.client.1.smithi152.stdout:0/506: creat d2/da/d3a/fc0 x:0 0 0 2024-03-29T04:45:17.098 INFO:tasks.workunit.client.1.smithi152.stdout:6/565: dread d1/d3/d1a/f2f [0,4194304] 0 2024-03-29T04:45:17.100 INFO:tasks.workunit.client.1.smithi152.stdout:7/452: write d7/d8/d37/f66 [624081,28164] 0 2024-03-29T04:45:17.100 INFO:tasks.workunit.client.1.smithi152.stdout:9/466: read d5/db/d1b/f43 [2847408,41488] 0 2024-03-29T04:45:17.100 INFO:tasks.workunit.client.0.smithi142.stdout:3/968: rename d3/d29/d2b/d40/fec to d3/db9/d11e/f14e 0 2024-03-29T04:45:17.101 INFO:tasks.workunit.client.0.smithi142.stdout:3/969: fsync d3/db9/d11e/f13f 0 2024-03-29T04:45:17.101 INFO:tasks.workunit.client.1.smithi152.stdout:0/507: chown d2/da/d17/d50/dac/fb2 11836963 1 2024-03-29T04:45:17.102 INFO:tasks.workunit.client.0.smithi142.stdout:3/970: readlink d3/d29/d2b/d40/l66 0 2024-03-29T04:45:17.102 INFO:tasks.workunit.client.1.smithi152.stdout:3/435: mkdir d7/d1c/d41/d6a/d78 0 2024-03-29T04:45:17.102 INFO:tasks.workunit.client.1.smithi152.stdout:2/360: symlink d0/d3/d30/d15/d22/d74/l78 0 2024-03-29T04:45:17.103 INFO:tasks.workunit.client.0.smithi142.stdout:3/971: fdatasync d3/d29/d8f/f10a 0 2024-03-29T04:45:17.103 INFO:tasks.workunit.client.0.smithi142.stdout:3/972: readlink d3/d1f/l143 0 2024-03-29T04:45:17.104 INFO:tasks.workunit.client.0.smithi142.stdout:3/973: fdatasync d3/ff9 0 2024-03-29T04:45:17.104 INFO:tasks.workunit.client.1.smithi152.stdout:3/436: fdatasync d7/d19/d1b/d59/f68 0 2024-03-29T04:45:17.104 INFO:tasks.workunit.client.1.smithi152.stdout:1/714: mkdir d5/d7/d1b/d2e/deb 0 2024-03-29T04:45:17.105 INFO:tasks.workunit.client.1.smithi152.stdout:1/715: dread - d5/d7/d4d/fc8 zero size 2024-03-29T04:45:17.106 INFO:tasks.workunit.client.1.smithi152.stdout:7/453: read d7/d8/d12/f28 [91361,104167] 0 2024-03-29T04:45:17.106 INFO:tasks.workunit.client.0.smithi142.stdout:3/974: creat d3/d29/d2b/d6a/f14f x:0 0 0 2024-03-29T04:45:17.106 INFO:tasks.workunit.client.1.smithi152.stdout:9/467: chown d5/db/f6c 16 1 2024-03-29T04:45:17.108 INFO:tasks.workunit.client.1.smithi152.stdout:1/716: write d5/d7/d1b/d2e/f86 [407146,118590] 0 2024-03-29T04:45:17.109 INFO:tasks.workunit.client.1.smithi152.stdout:6/566: unlink d1/d3/d9/d13/d28/d45/d8c/la2 0 2024-03-29T04:45:17.110 INFO:tasks.workunit.client.1.smithi152.stdout:2/361: creat d0/d3/d30/d15/d22/d4c/d52/f79 x:0 0 0 2024-03-29T04:45:17.111 INFO:tasks.workunit.client.0.smithi142.stdout:3/975: rename d3/d29/d2b/d40/d67/f13e to d3/d29/d2b/d40/db1/d124/d50/d126/d56/f150 0 2024-03-29T04:45:17.111 INFO:tasks.workunit.client.1.smithi152.stdout:1/717: chown d5/d7/d8/d13/fb2 31 1 2024-03-29T04:45:17.111 INFO:tasks.workunit.client.1.smithi152.stdout:6/567: write d1/d3/df/f81 [654322,20668] 0 2024-03-29T04:45:17.112 INFO:tasks.workunit.client.0.smithi142.stdout:3/976: write d3/d1f/d24/d47/d87/f10f [325517,102555] 0 2024-03-29T04:45:17.112 INFO:tasks.workunit.client.1.smithi152.stdout:8/414: getdents d1/d2/d1d/d36/d4b/d51 0 2024-03-29T04:45:17.114 INFO:tasks.workunit.client.1.smithi152.stdout:0/508: mknod d2/da/d3a/cc1 0 2024-03-29T04:45:17.114 INFO:tasks.workunit.client.1.smithi152.stdout:7/454: creat d7/d8/d12/d1d/f86 x:0 0 0 2024-03-29T04:45:17.115 INFO:tasks.workunit.client.0.smithi142.stdout:3/977: mkdir d3/d29/d2b/d151 0 2024-03-29T04:45:17.116 INFO:tasks.workunit.client.1.smithi152.stdout:0/509: chown d2/da 30 1 2024-03-29T04:45:17.116 INFO:tasks.workunit.client.0.smithi142.stdout:3/978: stat d3/d29/d8f/d5f/d64/d6b 0 2024-03-29T04:45:17.117 INFO:tasks.workunit.client.0.smithi142.stdout:3/979: truncate d3/d29/d2b/d6a/f14f 221659 0 2024-03-29T04:45:17.118 INFO:tasks.workunit.client.1.smithi152.stdout:9/468: rename d5/d12/c48 to d5/db/d1b/ca8 0 2024-03-29T04:45:17.119 INFO:tasks.workunit.client.1.smithi152.stdout:3/437: getdents d7/d1c/d41/d61 0 2024-03-29T04:45:17.119 INFO:tasks.workunit.client.1.smithi152.stdout:0/510: symlink d2/d29/lc2 0 2024-03-29T04:45:17.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:16 smithi142 ceph-mon[29027]: pgmap v274: 65 pgs: 65 active+clean; 2.5 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 78 MiB/s rd, 229 MiB/s wr, 439 op/s 2024-03-29T04:45:17.122 INFO:tasks.workunit.client.1.smithi152.stdout:3/438: fsync d7/d1c/f54 0 2024-03-29T04:45:17.122 INFO:tasks.workunit.client.1.smithi152.stdout:1/718: link d5/d7/d8/d13/d19/l7a d5/d7/d38/d6c/lec 0 2024-03-29T04:45:17.122 INFO:tasks.workunit.client.0.smithi142.stdout:3/980: link d3/d29/d2b/d40/d67/c121 d3/d29/d2b/d40/db1/c152 0 2024-03-29T04:45:17.123 INFO:tasks.workunit.client.1.smithi152.stdout:9/469: read - d5/d12/d2c/f78 zero size 2024-03-29T04:45:17.124 INFO:tasks.workunit.client.0.smithi142.stdout:3/981: read d3/d29/d8f/d5f/db4/f58 [2402906,12246] 0 2024-03-29T04:45:17.125 INFO:tasks.workunit.client.1.smithi152.stdout:6/568: rename d1/d3/d9/d13/d28/l7d to d1/d3/df/d84/lc2 0 2024-03-29T04:45:17.125 INFO:tasks.workunit.client.0.smithi142.stdout:3/982: chown d3/d29/d2b/d40/d67/c8c 468611 1 2024-03-29T04:45:17.127 INFO:tasks.workunit.client.1.smithi152.stdout:0/511: unlink d2/da/d17/d57/d9c/f9d 0 2024-03-29T04:45:17.127 INFO:tasks.workunit.client.0.smithi142.stdout:3/983: chown d3/d29/d8f/c6f 6 1 2024-03-29T04:45:17.130 INFO:tasks.workunit.client.1.smithi152.stdout:9/470: fdatasync d5/db/f5a 0 2024-03-29T04:45:17.130 INFO:tasks.workunit.client.0.smithi142.stdout:3/984: mkdir d3/d29/d8f/d5f/d64/d6b/d153 0 2024-03-29T04:45:17.130 INFO:tasks.workunit.client.1.smithi152.stdout:8/415: dwrite d1/d21/f78 [0,4194304] 0 2024-03-29T04:45:17.131 INFO:tasks.workunit.client.1.smithi152.stdout:9/471: fdatasync d5/db/f79 0 2024-03-29T04:45:17.131 INFO:tasks.workunit.client.1.smithi152.stdout:1/719: chown d5/d7/d8/d13/d19/d80/cb5 26 1 2024-03-29T04:45:17.132 INFO:tasks.workunit.client.1.smithi152.stdout:2/362: dwrite d0/d3/f18 [0,4194304] 0 2024-03-29T04:45:17.133 INFO:tasks.workunit.client.1.smithi152.stdout:3/439: symlink d7/d19/l79 0 2024-03-29T04:45:17.138 INFO:tasks.workunit.client.1.smithi152.stdout:8/416: read - d1/d2/d1d/d36/d4b/f68 zero size 2024-03-29T04:45:17.140 INFO:tasks.workunit.client.1.smithi152.stdout:2/363: mknod d0/d3/df/d2c/c7a 0 2024-03-29T04:45:17.145 INFO:tasks.workunit.client.1.smithi152.stdout:7/455: rename d7/d8/d12/d1d/d41/f51 to d7/d8/d12/f87 0 2024-03-29T04:45:17.148 INFO:tasks.workunit.client.1.smithi152.stdout:0/512: dwrite d2/da/d17/f4c [4194304,4194304] 0 2024-03-29T04:45:17.150 INFO:tasks.workunit.client.0.smithi142.stdout:3/985: dwrite d3/d1f/d24/f37 [0,4194304] 0 2024-03-29T04:45:17.151 INFO:tasks.workunit.client.1.smithi152.stdout:7/456: write d7/d8/d12/d1d/d41/f2c [3669939,3896] 0 2024-03-29T04:45:17.158 INFO:tasks.workunit.client.1.smithi152.stdout:3/440: dwrite d7/d46/f6d [0,4194304] 0 2024-03-29T04:45:17.159 INFO:tasks.workunit.client.1.smithi152.stdout:2/364: truncate d0/f16 506035 0 2024-03-29T04:45:17.164 INFO:tasks.workunit.client.1.smithi152.stdout:7/457: creat d7/d8/d12/d35/d68/f88 x:0 0 0 2024-03-29T04:45:17.168 INFO:tasks.workunit.client.1.smithi152.stdout:6/569: dwrite d1/fe [0,4194304] 0 2024-03-29T04:45:17.170 INFO:tasks.workunit.client.1.smithi152.stdout:2/365: write d0/d3/d64/f6e [4818270,11369] 0 2024-03-29T04:45:17.170 INFO:tasks.workunit.client.1.smithi152.stdout:3/441: chown d7/d19/d1b/d1e/d22/f4d 39977619 1 2024-03-29T04:45:17.171 INFO:tasks.workunit.client.1.smithi152.stdout:8/417: dwrite d1/d21/d40/d6a/f89 [0,4194304] 0 2024-03-29T04:45:17.172 INFO:tasks.workunit.client.0.smithi142.stdout:3/986: truncate d3/d1f/de6/df1/f105 125979 0 2024-03-29T04:45:17.172 INFO:tasks.workunit.client.1.smithi152.stdout:9/472: dwrite d5/d8/f9 [0,4194304] 0 2024-03-29T04:45:17.173 INFO:tasks.workunit.client.0.smithi142.stdout:3/987: dread - d3/d29/d8f/d5f/dc5/f101 zero size 2024-03-29T04:45:17.174 INFO:tasks.workunit.client.1.smithi152.stdout:8/418: write d1/f34 [90792,104368] 0 2024-03-29T04:45:17.177 INFO:tasks.workunit.client.0.smithi142.stdout:3/988: stat d3/d29/d2b/d40/d67/d8b/fdd 0 2024-03-29T04:45:17.178 INFO:tasks.workunit.client.1.smithi152.stdout:3/442: write d7/d19/d1b/d1e/d4e/f70 [277294,116306] 0 2024-03-29T04:45:17.181 INFO:tasks.workunit.client.1.smithi152.stdout:9/473: mknod d5/db/ca9 0 2024-03-29T04:45:17.181 INFO:tasks.workunit.client.1.smithi152.stdout:8/419: chown d1/d2/d6d/c7d 302 1 2024-03-29T04:45:17.181 INFO:tasks.workunit.client.1.smithi152.stdout:6/570: mkdir d1/d3/d9/d13/dc0/dc3 0 2024-03-29T04:45:17.183 INFO:tasks.workunit.client.1.smithi152.stdout:7/458: dwrite d7/d8/d12/d15/d72/f79 [0,4194304] 0 2024-03-29T04:45:17.186 INFO:tasks.workunit.client.0.smithi142.stdout:3/989: dwrite d3/d1f/d24/d47/f90 [0,4194304] 0 2024-03-29T04:45:17.187 INFO:tasks.workunit.client.1.smithi152.stdout:3/443: creat d7/d19/d3a/f7a x:0 0 0 2024-03-29T04:45:17.188 INFO:tasks.workunit.client.1.smithi152.stdout:7/459: fdatasync d7/d8/f6e 0 2024-03-29T04:45:17.190 INFO:tasks.workunit.client.0.smithi142.stdout:3/990: fsync d3/d29/d2b/d40/d67/d8b/f91 0 2024-03-29T04:45:17.192 INFO:tasks.workunit.client.1.smithi152.stdout:9/474: mkdir d5/db/d1b/d68/daa 0 2024-03-29T04:45:17.194 INFO:tasks.workunit.client.1.smithi152.stdout:2/366: dwrite d0/d3/d30/f3f [0,4194304] 0 2024-03-29T04:45:17.196 INFO:tasks.workunit.client.0.smithi142.stdout:3/991: symlink d3/d29/d2b/d40/l154 0 2024-03-29T04:45:17.197 INFO:tasks.workunit.client.1.smithi152.stdout:1/720: dread d5/d7/d8/d13/d19/f65 [0,4194304] 0 2024-03-29T04:45:17.200 INFO:tasks.workunit.client.1.smithi152.stdout:8/420: mknod d1/d2/d1d/d36/d85/c8f 0 2024-03-29T04:45:17.202 INFO:tasks.workunit.client.1.smithi152.stdout:9/475: dread - d5/d12/d2c/f50 zero size 2024-03-29T04:45:17.203 INFO:tasks.workunit.client.0.smithi142.stdout:3/992: getdents d3/d29/d2b/d40/db1/d124 0 2024-03-29T04:45:17.204 INFO:tasks.workunit.client.1.smithi152.stdout:1/721: mknod d5/d7/d1b/d2e/deb/ced 0 2024-03-29T04:45:17.206 INFO:tasks.workunit.client.1.smithi152.stdout:3/444: dwrite d7/d19/d3a/f55 [4194304,4194304] 0 2024-03-29T04:45:17.211 INFO:tasks.workunit.client.0.smithi142.stdout:3/993: link d3/d29/d2b/d40/db1/d124/d50/d126/d79/dcc/c11c d3/d1f/d24/d47/d87/c155 0 2024-03-29T04:45:17.211 INFO:tasks.workunit.client.1.smithi152.stdout:3/445: creat d7/d19/d1b/f7b x:0 0 0 2024-03-29T04:45:17.212 INFO:tasks.workunit.client.1.smithi152.stdout:3/446: chown c4 3 1 2024-03-29T04:45:17.214 INFO:tasks.workunit.client.0.smithi142.stdout:3/994: truncate d3/d29/d2b/fe2 85839 0 2024-03-29T04:45:17.215 INFO:tasks.workunit.client.1.smithi152.stdout:9/476: dwrite d5/db/f58 [0,4194304] 0 2024-03-29T04:45:17.217 INFO:tasks.workunit.client.1.smithi152.stdout:9/477: chown d5/d12/c99 349790891 1 2024-03-29T04:45:17.219 INFO:tasks.workunit.client.1.smithi152.stdout:3/447: read d7/d19/f25 [1361374,78374] 0 2024-03-29T04:45:17.221 INFO:tasks.workunit.client.0.smithi142.stdout:3/995: getdents d3/d29/d8f/d5f/db4 0 2024-03-29T04:45:17.222 INFO:tasks.workunit.client.0.smithi142.stdout:3/996: truncate d3/d29/d2b/d40/d67/f12a 4969897 0 2024-03-29T04:45:17.246 INFO:tasks.workunit.client.0.smithi142.stdout:3/997: dread d3/d29/d8f/d5f/db4/f97 [0,4194304] 0 2024-03-29T04:45:17.252 INFO:tasks.workunit.client.1.smithi152.stdout:9/478: fsync d5/d8/f1a 0 2024-03-29T04:45:17.253 INFO:tasks.workunit.client.1.smithi152.stdout:9/479: creat d5/db/d92/fab x:0 0 0 2024-03-29T04:45:17.254 INFO:tasks.workunit.client.0.smithi142.stdout:3/998: creat d3/d29/d8f/f156 x:0 0 0 2024-03-29T04:45:17.255 INFO:tasks.workunit.client.1.smithi152.stdout:3/448: link d7/d1c/d41/d73/f77 d7/d1c/d41/d61/d75/f7c 0 2024-03-29T04:45:17.255 INFO:tasks.workunit.client.1.smithi152.stdout:6/571: dread f0 [0,4194304] 0 2024-03-29T04:45:17.255 INFO:tasks.workunit.client.0.smithi142.stdout:3/999: readlink d3/d29/d2b/d40/d67/d107/l140 0 2024-03-29T04:45:17.258 INFO:tasks.workunit.client.1.smithi152.stdout:6/572: unlink d1/d3/d1a/d1b/cbe 0 2024-03-29T04:45:17.260 INFO:tasks.workunit.client.1.smithi152.stdout:6/573: stat d1/d3/d1a/f1d 0 2024-03-29T04:45:17.263 INFO:tasks.workunit.client.1.smithi152.stdout:9/480: dwrite d5/d8/f69 [0,4194304] 0 2024-03-29T04:45:17.263 INFO:tasks.workunit.client.0.smithi142.stderr:+ rm -rf -- ./tmp.847eakZwIb 2024-03-29T04:45:17.265 INFO:tasks.workunit.client.1.smithi152.stdout:6/574: truncate d1/d3/df/f9e 160457 0 2024-03-29T04:45:17.265 INFO:tasks.workunit.client.1.smithi152.stdout:2/367: dwrite d0/d3/d30/d15/f2b [0,4194304] 0 2024-03-29T04:45:17.269 INFO:tasks.workunit.client.1.smithi152.stdout:6/575: stat d1/d3/d1a/fbd 0 2024-03-29T04:45:17.269 INFO:tasks.workunit.client.1.smithi152.stdout:9/481: write d5/d12/d2c/f50 [244752,69804] 0 2024-03-29T04:45:17.269 INFO:tasks.workunit.client.1.smithi152.stdout:1/722: rmdir d5/d7/d1b 39 2024-03-29T04:45:17.269 INFO:tasks.workunit.client.1.smithi152.stdout:6/576: fsync d1/d3/d9/f27 0 2024-03-29T04:45:17.270 INFO:tasks.workunit.client.1.smithi152.stdout:1/723: chown d5/d7/d8/fc1 251149 1 2024-03-29T04:45:17.273 INFO:tasks.workunit.client.1.smithi152.stdout:6/577: write d1/d3/df/f81 [456661,76217] 0 2024-03-29T04:45:17.275 INFO:tasks.workunit.client.1.smithi152.stdout:9/482: rmdir d5/d8/d31/d59 39 2024-03-29T04:45:17.275 INFO:tasks.workunit.client.1.smithi152.stdout:1/724: creat d5/d7/d4d/db1/fee x:0 0 0 2024-03-29T04:45:17.276 INFO:tasks.workunit.client.1.smithi152.stdout:9/483: dread - d5/db/f5a zero size 2024-03-29T04:45:17.277 INFO:tasks.workunit.client.1.smithi152.stdout:6/578: chown d1/d3/df/f4f 197203 1 2024-03-29T04:45:17.278 INFO:tasks.workunit.client.1.smithi152.stdout:3/449: truncate d7/f1a 3989959 0 2024-03-29T04:45:17.280 INFO:tasks.workunit.client.1.smithi152.stdout:3/450: write d7/d1c/f47 [869777,41468] 0 2024-03-29T04:45:17.284 INFO:tasks.workunit.client.1.smithi152.stdout:1/725: dwrite d5/f98 [0,4194304] 0 2024-03-29T04:45:17.286 INFO:tasks.workunit.client.1.smithi152.stdout:1/726: fsync d5/f3e 0 2024-03-29T04:45:17.289 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:16 smithi152 ceph-mon[37177]: pgmap v274: 65 pgs: 65 active+clean; 2.5 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 78 MiB/s rd, 229 MiB/s wr, 439 op/s 2024-03-29T04:45:17.291 INFO:tasks.workunit.client.1.smithi152.stdout:3/451: dwrite d7/d46/f6d [0,4194304] 0 2024-03-29T04:45:17.308 INFO:tasks.workunit.client.1.smithi152.stdout:6/579: dread d1/d3/d1a/d1b/f1e [0,4194304] 0 2024-03-29T04:45:17.311 INFO:tasks.workunit.client.1.smithi152.stdout:6/580: creat d1/d3/d1a/d1b/d29/fc4 x:0 0 0 2024-03-29T04:45:17.316 INFO:tasks.workunit.client.1.smithi152.stdout:6/581: write d1/d3/d1a/f20 [1960658,92546] 0 2024-03-29T04:45:17.319 INFO:tasks.workunit.client.1.smithi152.stdout:1/727: dread d5/d7/d8/d13/d19/fd7 [4194304,4194304] 0 2024-03-29T04:45:17.322 INFO:tasks.workunit.client.1.smithi152.stdout:3/452: dread d7/d19/d3a/f6e [0,4194304] 0 2024-03-29T04:45:17.325 INFO:tasks.workunit.client.1.smithi152.stdout:6/582: dwrite d1/d3/d1a/d1b/d29/f7e [0,4194304] 0 2024-03-29T04:45:17.326 INFO:tasks.workunit.client.1.smithi152.stdout:6/583: readlink d1/d3/d9/d13/l2a 0 2024-03-29T04:45:17.327 INFO:tasks.workunit.client.1.smithi152.stdout:1/728: truncate d5/d7/fe 1232821 0 2024-03-29T04:45:17.327 INFO:tasks.workunit.client.1.smithi152.stdout:6/584: read - d1/d3/d9/d99/fa6 zero size 2024-03-29T04:45:17.329 INFO:tasks.workunit.client.1.smithi152.stdout:6/585: fdatasync d1/d3/d9/f21 0 2024-03-29T04:45:17.329 INFO:tasks.workunit.client.1.smithi152.stdout:3/453: truncate d7/d1c/f2c 1921974 0 2024-03-29T04:45:17.330 INFO:tasks.workunit.client.1.smithi152.stdout:1/729: creat d5/d7/d4d/db1/fef x:0 0 0 2024-03-29T04:45:17.340 INFO:tasks.workunit.client.1.smithi152.stdout:1/730: dwrite d5/d7/d8/d13/d19/dc7/fdf [0,4194304] 0 2024-03-29T04:45:17.344 INFO:tasks.workunit.client.1.smithi152.stdout:1/731: rename d5 to d5/d7/d8/da5/df0 22 2024-03-29T04:45:17.347 INFO:tasks.workunit.client.1.smithi152.stdout:6/586: dread d1/d3/f18 [0,4194304] 0 2024-03-29T04:45:17.350 INFO:tasks.workunit.client.1.smithi152.stdout:1/732: read d5/d7/d1b/d2e/d95/dad/d12/d77/fb4 [362596,5684] 0 2024-03-29T04:45:17.354 INFO:tasks.workunit.client.1.smithi152.stdout:3/454: write d7/f10 [2137506,71584] 0 2024-03-29T04:45:17.359 INFO:tasks.workunit.client.1.smithi152.stdout:3/455: mknod d7/d19/d1b/d1e/d4e/c7d 0 2024-03-29T04:45:17.360 INFO:tasks.workunit.client.1.smithi152.stdout:6/587: dwrite d1/d3/df/f9e [0,4194304] 0 2024-03-29T04:45:17.364 INFO:tasks.workunit.client.1.smithi152.stdout:1/733: dwrite d5/d7/fc2 [0,4194304] 0 2024-03-29T04:45:17.366 INFO:tasks.workunit.client.1.smithi152.stdout:6/588: symlink d1/d3/df/d84/lc5 0 2024-03-29T04:45:17.367 INFO:tasks.workunit.client.1.smithi152.stdout:3/456: rename d7/d19/f3c to d7/d19/d1b/f7e 0 2024-03-29T04:45:17.368 INFO:tasks.workunit.client.1.smithi152.stdout:3/457: readlink d7/d19/l20 0 2024-03-29T04:45:17.371 INFO:tasks.workunit.client.1.smithi152.stdout:1/734: mkdir d5/d7/d1b/d2e/d95/df1 0 2024-03-29T04:45:17.376 INFO:tasks.workunit.client.1.smithi152.stdout:1/735: truncate d5/d7/f25 1699092 0 2024-03-29T04:45:17.381 INFO:tasks.workunit.client.1.smithi152.stdout:1/736: rename d5/d7/d1b/d2e/d95/dad/d59/f89 to d5/d7/d8/d13/d19/ff2 0 2024-03-29T04:45:17.383 INFO:tasks.workunit.client.1.smithi152.stdout:1/737: mknod d5/d7/d1b/d2e/de2/cf3 0 2024-03-29T04:45:17.383 INFO:tasks.workunit.client.1.smithi152.stdout:1/738: chown d5/d7/lbe 250194416 1 2024-03-29T04:45:17.385 INFO:tasks.workunit.client.1.smithi152.stdout:6/589: truncate d1/d3/df/f66 184351 0 2024-03-29T04:45:17.386 INFO:tasks.workunit.client.1.smithi152.stdout:3/458: truncate d7/d19/f26 3346849 0 2024-03-29T04:45:17.387 INFO:tasks.workunit.client.1.smithi152.stdout:3/459: readlink d7/l2b 0 2024-03-29T04:45:17.388 INFO:tasks.workunit.client.1.smithi152.stdout:3/460: fdatasync d7/d19/d3a/f7a 0 2024-03-29T04:45:17.390 INFO:tasks.workunit.client.1.smithi152.stdout:6/590: rename d1/d3/d9/d99/c9c to d1/cc6 0 2024-03-29T04:45:17.391 INFO:tasks.workunit.client.1.smithi152.stdout:1/739: dread d5/d7/d8/d13/d19/f3d [4194304,4194304] 0 2024-03-29T04:45:17.398 INFO:tasks.workunit.client.1.smithi152.stdout:6/591: rmdir d1/d3/d1a/d1b/dbc 0 2024-03-29T04:45:17.399 INFO:tasks.workunit.client.1.smithi152.stdout:3/461: dwrite d7/d19/d1b/f21 [4194304,4194304] 0 2024-03-29T04:45:17.400 INFO:tasks.workunit.client.1.smithi152.stdout:1/740: creat d5/d7/d8/d13/d19/ff4 x:0 0 0 2024-03-29T04:45:17.404 INFO:tasks.workunit.client.1.smithi152.stdout:1/741: write d5/d7/fc2 [5100086,78040] 0 2024-03-29T04:45:17.407 INFO:tasks.workunit.client.1.smithi152.stdout:3/462: rename d7/d19/d1b/d1e/f40 to d7/d19/d3a/f7f 0 2024-03-29T04:45:17.410 INFO:tasks.workunit.client.1.smithi152.stdout:3/463: write d7/d19/d1b/f21 [4592283,41765] 0 2024-03-29T04:45:17.410 INFO:tasks.workunit.client.1.smithi152.stdout:6/592: dwrite d1/d3/d1a/fbd [0,4194304] 0 2024-03-29T04:45:17.410 INFO:tasks.workunit.client.1.smithi152.stdout:1/742: mknod d5/d7/d8/d13/d19/dc7/cf5 0 2024-03-29T04:45:17.412 INFO:tasks.workunit.client.1.smithi152.stdout:3/464: write d7/d1c/f47 [1943420,59479] 0 2024-03-29T04:45:17.414 INFO:tasks.workunit.client.1.smithi152.stdout:1/743: creat d5/d7/d1b/d2e/de2/ff6 x:0 0 0 2024-03-29T04:45:17.428 INFO:tasks.workunit.client.1.smithi152.stdout:3/465: rename d7/f3e to d7/d19/d1b/f80 0 2024-03-29T04:45:17.429 INFO:tasks.workunit.client.1.smithi152.stdout:3/466: dread - d7/d19/d3a/f7f zero size 2024-03-29T04:45:17.431 INFO:tasks.workunit.client.1.smithi152.stdout:3/467: mknod d7/d1c/d41/d6a/d78/c81 0 2024-03-29T04:45:17.434 INFO:tasks.workunit.client.1.smithi152.stdout:6/593: write d1/f8 [773371,91871] 0 2024-03-29T04:45:17.434 INFO:tasks.workunit.client.1.smithi152.stdout:1/744: write d5/d7/d1b/d2e/fa3 [628265,7715] 0 2024-03-29T04:45:17.436 INFO:tasks.workunit.client.1.smithi152.stdout:6/594: fsync f0 0 2024-03-29T04:45:17.438 INFO:tasks.workunit.client.1.smithi152.stdout:1/745: unlink d5/d7/d8/d13/d19/f65 0 2024-03-29T04:45:17.442 INFO:tasks.workunit.client.1.smithi152.stdout:1/746: creat d5/d7/d1b/d2e/d95/dad/d12/ff7 x:0 0 0 2024-03-29T04:45:17.445 INFO:tasks.workunit.client.1.smithi152.stdout:6/595: getdents d1/d3/d1a 0 2024-03-29T04:45:17.445 INFO:tasks.workunit.client.1.smithi152.stdout:1/747: read - d5/d7/d4d/db1/fef zero size 2024-03-29T04:45:17.447 INFO:tasks.workunit.client.1.smithi152.stdout:3/468: dwrite d7/d1c/d41/d61/d75/f7c [0,4194304] 0 2024-03-29T04:45:17.448 INFO:tasks.workunit.client.1.smithi152.stdout:1/748: readlink d5/d7/d1b/d2e/d95/dad/l83 0 2024-03-29T04:45:17.449 INFO:tasks.workunit.client.1.smithi152.stdout:6/596: mkdir d1/d3/df/d8a/d98/dc7 0 2024-03-29T04:45:17.451 INFO:tasks.workunit.client.1.smithi152.stdout:6/597: write d1/d3/df/d84/f9d [209156,52035] 0 2024-03-29T04:45:17.455 INFO:tasks.workunit.client.1.smithi152.stdout:3/469: link d7/d1c/d41/d73/f77 d7/d19/d1b/d1e/f82 0 2024-03-29T04:45:17.458 INFO:tasks.workunit.client.1.smithi152.stdout:6/598: mknod d1/d3/d1a/d1b/d29/cc8 0 2024-03-29T04:45:17.459 INFO:tasks.workunit.client.1.smithi152.stdout:1/749: dwrite d5/fb8 [0,4194304] 0 2024-03-29T04:45:17.461 INFO:tasks.workunit.client.1.smithi152.stdout:1/750: truncate d5/d7/d38/d6c/fd5 395364 0 2024-03-29T04:45:17.461 INFO:tasks.workunit.client.1.smithi152.stdout:3/470: creat d7/d1c/d41/d6a/d78/f83 x:0 0 0 2024-03-29T04:45:17.462 INFO:tasks.workunit.client.1.smithi152.stdout:3/471: stat c4 0 2024-03-29T04:45:17.463 INFO:tasks.workunit.client.1.smithi152.stdout:6/599: rmdir d1/d3/df/d8a/d98 39 2024-03-29T04:45:17.464 INFO:tasks.workunit.client.1.smithi152.stdout:3/472: chown d7/d1c/d41/f50 281722134 1 2024-03-29T04:45:17.465 INFO:tasks.workunit.client.1.smithi152.stdout:1/751: getdents d5/d7/d1b/d2e/d95/df1 0 2024-03-29T04:45:17.467 INFO:tasks.workunit.client.1.smithi152.stdout:3/473: creat d7/d1c/d41/d73/f84 x:0 0 0 2024-03-29T04:45:17.469 INFO:tasks.workunit.client.1.smithi152.stdout:1/752: write d5/d7/d1b/d2e/f8e [48334,7759] 0 2024-03-29T04:45:17.470 INFO:tasks.workunit.client.1.smithi152.stdout:3/474: mkdir d7/d1c/d41/d61/d85 0 2024-03-29T04:45:17.472 INFO:tasks.workunit.client.1.smithi152.stdout:6/600: dwrite d1/f8 [0,4194304] 0 2024-03-29T04:45:17.475 INFO:tasks.workunit.client.1.smithi152.stdout:6/601: symlink d1/d3/d9/d13/d3e/lc9 0 2024-03-29T04:45:17.477 INFO:tasks.workunit.client.1.smithi152.stdout:3/475: rename d7/l13 to d7/d19/d1b/d1e/d4e/l86 0 2024-03-29T04:45:17.481 INFO:tasks.workunit.client.1.smithi152.stdout:1/753: rename d5/d7/d1b/d2e/d95/dad/d12/d77/ce7 to d5/d7/d8/da5/cf8 0 2024-03-29T04:45:17.484 INFO:tasks.workunit.client.1.smithi152.stdout:6/602: creat d1/d3/d1a/d1b/d29/fca x:0 0 0 2024-03-29T04:45:17.484 INFO:tasks.workunit.client.1.smithi152.stdout:1/754: rename d5/d7/lbb to d5/d7/d1b/d2e/de2/lf9 0 2024-03-29T04:45:17.487 INFO:tasks.workunit.client.1.smithi152.stdout:6/603: dread d1/d3/f18 [0,4194304] 0 2024-03-29T04:45:17.488 INFO:tasks.workunit.client.1.smithi152.stdout:6/604: write d1/d3/df/f81 [321392,107549] 0 2024-03-29T04:45:17.492 INFO:tasks.workunit.client.1.smithi152.stdout:6/605: mkdir d1/d3/d9/dcb 0 2024-03-29T04:45:17.497 INFO:tasks.workunit.client.1.smithi152.stdout:1/755: dwrite d5/d7/d1b/d2e/f9e [4194304,4194304] 0 2024-03-29T04:45:17.497 INFO:tasks.workunit.client.1.smithi152.stdout:3/476: dwrite d7/d19/f36 [4194304,4194304] 0 2024-03-29T04:45:17.500 INFO:tasks.workunit.client.1.smithi152.stdout:6/606: creat d1/d3/d1a/d1b/d29/fcc x:0 0 0 2024-03-29T04:45:17.506 INFO:tasks.workunit.client.1.smithi152.stdout:3/477: creat d7/f87 x:0 0 0 2024-03-29T04:45:17.507 INFO:tasks.workunit.client.1.smithi152.stdout:1/756: dwrite d5/d7/d1b/d2e/f9e [4194304,4194304] 0 2024-03-29T04:45:17.507 INFO:tasks.workunit.client.1.smithi152.stdout:6/607: creat d1/d3/d9/d13/d3e/fcd x:0 0 0 2024-03-29T04:45:17.508 INFO:tasks.workunit.client.1.smithi152.stdout:3/478: chown d7/d19/d1b/l44 2830123 1 2024-03-29T04:45:17.509 INFO:tasks.workunit.client.1.smithi152.stdout:6/608: write d1/d3/d1a/f2c [691304,21083] 0 2024-03-29T04:45:17.512 INFO:tasks.workunit.client.1.smithi152.stdout:1/757: creat d5/d7/ffa x:0 0 0 2024-03-29T04:45:17.514 INFO:tasks.workunit.client.1.smithi152.stdout:6/609: mkdir d1/d3/d1a/d1b/d29/dce 0 2024-03-29T04:45:17.516 INFO:tasks.workunit.client.1.smithi152.stdout:3/479: dread d7/d19/d1b/d1e/d22/f2d [0,4194304] 0 2024-03-29T04:45:17.518 INFO:tasks.workunit.client.1.smithi152.stdout:3/480: read d7/d1c/f47 [1595090,520] 0 2024-03-29T04:45:17.519 INFO:tasks.workunit.client.1.smithi152.stdout:3/481: write d7/d19/d1b/f58 [617184,98033] 0 2024-03-29T04:45:17.519 INFO:tasks.workunit.client.1.smithi152.stdout:6/610: truncate d1/d3/d1a/d1b/d29/f79 151439 0 2024-03-29T04:45:17.522 INFO:tasks.workunit.client.1.smithi152.stdout:3/482: mkdir d7/d1c/d41/d61/d85/d88 0 2024-03-29T04:45:17.523 INFO:tasks.workunit.client.1.smithi152.stdout:6/611: creat d1/fcf x:0 0 0 2024-03-29T04:45:17.524 INFO:tasks.workunit.client.1.smithi152.stdout:3/483: chown d7/d19/d3a/c37 87 1 2024-03-29T04:45:17.526 INFO:tasks.workunit.client.1.smithi152.stdout:3/484: mknod d7/d1c/d41/d61/d63/c89 0 2024-03-29T04:45:17.528 INFO:tasks.workunit.client.1.smithi152.stdout:6/612: getdents d1/d3/df/d84/da1 0 2024-03-29T04:45:17.528 INFO:tasks.workunit.client.1.smithi152.stdout:3/485: truncate d7/d1c/d41/d6a/d78/f83 764379 0 2024-03-29T04:45:17.529 INFO:tasks.workunit.client.1.smithi152.stdout:3/486: fsync d7/d19/d1b/d59/f68 0 2024-03-29T04:45:17.530 INFO:tasks.workunit.client.1.smithi152.stdout:6/613: rmdir d1/d3/d1a/d1b/d87/d8f 39 2024-03-29T04:45:17.533 INFO:tasks.workunit.client.1.smithi152.stdout:3/487: truncate d7/f1a 4110397 0 2024-03-29T04:45:17.534 INFO:tasks.workunit.client.1.smithi152.stdout:3/488: dread - d7/d19/d1b/d59/f68 zero size 2024-03-29T04:45:17.536 INFO:tasks.workunit.client.1.smithi152.stdout:1/758: dwrite d5/d7/d1b/d2e/f8d [0,4194304] 0 2024-03-29T04:45:17.537 INFO:tasks.workunit.client.1.smithi152.stdout:3/489: mknod d7/d46/c8a 0 2024-03-29T04:45:17.538 INFO:tasks.workunit.client.1.smithi152.stdout:3/490: write d7/d19/d1b/d1e/d4e/f70 [60979,13066] 0 2024-03-29T04:45:17.539 INFO:tasks.workunit.client.1.smithi152.stdout:1/759: rmdir d5/d7/d38 39 2024-03-29T04:45:17.546 INFO:tasks.workunit.client.1.smithi152.stdout:6/614: dread d1/d3/d9/d13/f1f [0,4194304] 0 2024-03-29T04:45:17.547 INFO:tasks.workunit.client.1.smithi152.stdout:6/615: chown d1/d3/d1a/d1b/d29/fc4 395534 1 2024-03-29T04:45:17.551 INFO:tasks.workunit.client.1.smithi152.stdout:1/760: dread d5/d7/d1b/d2e/d95/dad/d2b/f2f [0,4194304] 0 2024-03-29T04:45:17.553 INFO:tasks.workunit.client.1.smithi152.stdout:6/616: read d1/d3/d9/f11 [1693455,30399] 0 2024-03-29T04:45:17.555 INFO:tasks.workunit.client.1.smithi152.stdout:6/617: write d1/d3/d9/d13/f31 [1379155,52784] 0 2024-03-29T04:45:17.556 INFO:tasks.workunit.client.1.smithi152.stdout:1/761: truncate d5/d7/d8/d13/d19/fd7 617762 0 2024-03-29T04:45:17.559 INFO:tasks.workunit.client.1.smithi152.stdout:1/762: truncate d5/d7/d8/d13/fb2 3671808 0 2024-03-29T04:45:17.561 INFO:tasks.workunit.client.1.smithi152.stdout:1/763: creat d5/d7/d8/d13/dd0/ffb x:0 0 0 2024-03-29T04:45:17.566 INFO:tasks.workunit.client.1.smithi152.stdout:1/764: dwrite d5/d7/d6a/f88 [0,4194304] 0 2024-03-29T04:45:17.567 INFO:tasks.workunit.client.1.smithi152.stdout:1/765: dread - d5/d7/d4d/fc8 zero size 2024-03-29T04:45:17.571 INFO:tasks.workunit.client.1.smithi152.stdout:1/766: link d5/d7/d4d/db1/fee d5/d7/d8/d13/dd0/dd3/ffc 0 2024-03-29T04:45:17.571 INFO:tasks.workunit.client.1.smithi152.stdout:1/767: fsync d5/fb8 0 2024-03-29T04:45:17.581 INFO:tasks.workunit.client.1.smithi152.stdout:1/768: rename d5/d7/d1b/d2e/de2/lf9 to d5/d7/d8/lfd 0 2024-03-29T04:45:17.599 INFO:tasks.workunit.client.1.smithi152.stdout:1/769: dwrite d5/d7/f25 [0,4194304] 0 2024-03-29T04:45:17.606 INFO:tasks.workunit.client.1.smithi152.stdout:1/770: dwrite d5/d7/d8/d13/f45 [0,4194304] 0 2024-03-29T04:45:17.607 INFO:tasks.workunit.client.1.smithi152.stdout:1/771: chown d5/d7/d8/d13/d19/d80/l8f 121358342 1 2024-03-29T04:45:17.611 INFO:tasks.workunit.client.1.smithi152.stdout:1/772: mkdir d5/d7/d1b/d2e/d95/df1/dfe 0 2024-03-29T04:45:17.616 INFO:tasks.workunit.client.1.smithi152.stdout:1/773: link d5/d7/d4d/la4 d5/d7/d1b/d2e/d95/d9d/da7/lff 0 2024-03-29T04:45:17.624 INFO:tasks.workunit.client.1.smithi152.stdout:1/774: dwrite d5/d7/d38/d6c/fd5 [0,4194304] 0 2024-03-29T04:45:17.626 INFO:tasks.workunit.client.1.smithi152.stdout:1/775: mkdir d5/d7/d1b/d2e/de2/d100 0 2024-03-29T04:45:17.631 INFO:tasks.workunit.client.1.smithi152.stdout:1/776: rename d5/d7/d38/d6c/fb0 to d5/d7/d1b/d2e/d95/d9d/f101 0 2024-03-29T04:45:17.632 INFO:tasks.workunit.client.1.smithi152.stdout:1/777: chown d5/d7/fe 495260487 1 2024-03-29T04:45:17.640 INFO:tasks.workunit.client.1.smithi152.stdout:1/778: write d5/d7/d8/d13/d19/f94 [324409,52090] 0 2024-03-29T04:45:17.641 INFO:tasks.workunit.client.1.smithi152.stdout:1/779: readlink d5/d7/d1b/d2e/d95/dad/d2b/lbc 0 2024-03-29T04:45:17.645 INFO:tasks.workunit.client.1.smithi152.stdout:1/780: rename d5/d7/d1b/d2e/d95/dad/d59 to d5/d7/d1b/d2e/de2/d102 0 2024-03-29T04:45:17.722 INFO:tasks.workunit.client.1.smithi152.stdout:5/479: sync 2024-03-29T04:45:17.722 INFO:tasks.workunit.client.1.smithi152.stdout:4/354: sync 2024-03-29T04:45:17.723 INFO:tasks.workunit.client.1.smithi152.stdout:4/355: readlink d0/d7/d25/d17/l32 0 2024-03-29T04:45:17.725 INFO:tasks.workunit.client.1.smithi152.stdout:4/356: unlink d0/d7/d25/f46 0 2024-03-29T04:45:17.727 INFO:tasks.workunit.client.1.smithi152.stdout:4/357: read - d0/d7/d40/d2a/f4a zero size 2024-03-29T04:45:17.728 INFO:tasks.workunit.client.1.smithi152.stdout:4/358: mkdir d0/d7/d40/d6d 0 2024-03-29T04:45:17.730 INFO:tasks.workunit.client.1.smithi152.stdout:4/359: mkdir d0/d27/d6e 0 2024-03-29T04:45:17.733 INFO:tasks.workunit.client.1.smithi152.stdout:4/360: symlink d0/d7/d25/d17/d37/l6f 0 2024-03-29T04:45:17.736 INFO:tasks.workunit.client.1.smithi152.stdout:4/361: mkdir d0/d7/d40/d23/d55/d70 0 2024-03-29T04:45:17.737 INFO:tasks.workunit.client.1.smithi152.stdout:4/362: write d0/d7/d25/d17/f5b [2660445,433] 0 2024-03-29T04:45:17.738 INFO:tasks.workunit.client.1.smithi152.stdout:4/363: chown d0/d7/d40/d23 18 1 2024-03-29T04:45:17.740 INFO:tasks.workunit.client.1.smithi152.stdout:5/480: dread d6/d16/d47/f61 [0,4194304] 0 2024-03-29T04:45:17.741 INFO:tasks.workunit.client.1.smithi152.stdout:4/364: unlink d0/d7/d40/d2a/f4a 0 2024-03-29T04:45:17.743 INFO:tasks.workunit.client.1.smithi152.stdout:5/481: getdents d6/d16/d47/d56/d63 0 2024-03-29T04:45:17.744 INFO:tasks.workunit.client.1.smithi152.stdout:5/482: readlink d6/d1d/d30/l33 0 2024-03-29T04:45:17.748 INFO:tasks.workunit.client.1.smithi152.stdout:5/483: link d6/d16/d25/d2e/da0/f71 d6/d7/fa4 0 2024-03-29T04:45:17.754 INFO:tasks.workunit.client.1.smithi152.stdout:4/365: dread d0/f15 [0,4194304] 0 2024-03-29T04:45:17.754 INFO:tasks.workunit.client.1.smithi152.stdout:5/484: dwrite d6/d16/f65 [0,4194304] 0 2024-03-29T04:45:17.755 INFO:tasks.workunit.client.1.smithi152.stdout:4/366: chown d0/d7/d40/d23 228777738 1 2024-03-29T04:45:17.758 INFO:tasks.workunit.client.1.smithi152.stdout:5/485: write d6/d1d/d3b/d55/f66 [4631079,91342] 0 2024-03-29T04:45:17.758 INFO:tasks.workunit.client.1.smithi152.stdout:4/367: creat d0/d7/d25/d17/f71 x:0 0 0 2024-03-29T04:45:17.759 INFO:tasks.workunit.client.1.smithi152.stdout:5/486: readlink d6/l8f 0 2024-03-29T04:45:17.760 INFO:tasks.workunit.client.1.smithi152.stdout:4/368: dread - d0/d7/d40/f67 zero size 2024-03-29T04:45:17.760 INFO:tasks.workunit.client.1.smithi152.stdout:5/487: chown d6/d1b/l29 6 1 2024-03-29T04:45:17.762 INFO:tasks.workunit.client.1.smithi152.stdout:5/488: write d6/d16/d47/f61 [4415291,74359] 0 2024-03-29T04:45:17.763 INFO:tasks.workunit.client.1.smithi152.stdout:4/369: mknod d0/d7/d25/d17/c72 0 2024-03-29T04:45:17.765 INFO:tasks.workunit.client.1.smithi152.stdout:4/370: chown d0/d7/d25/d17/d37/l6f 1665 1 2024-03-29T04:45:17.765 INFO:tasks.workunit.client.1.smithi152.stdout:5/489: mkdir d6/d1d/da5 0 2024-03-29T04:45:17.766 INFO:tasks.workunit.client.1.smithi152.stdout:4/371: rename d0/d7/d40 to d0/d7/d40/d62/d73 22 2024-03-29T04:45:17.767 INFO:tasks.workunit.client.1.smithi152.stdout:5/490: creat d6/d1d/d30/d6a/fa6 x:0 0 0 2024-03-29T04:45:17.768 INFO:tasks.workunit.client.1.smithi152.stdout:5/491: chown d6/d16/d25/f9f 1283162 1 2024-03-29T04:45:17.768 INFO:tasks.workunit.client.1.smithi152.stdout:4/372: rmdir d0/d7/d40/d2a 39 2024-03-29T04:45:17.769 INFO:tasks.workunit.client.1.smithi152.stdout:4/373: write d0/d7/d40/f67 [798358,87194] 0 2024-03-29T04:45:17.770 INFO:tasks.workunit.client.1.smithi152.stdout:5/492: creat d6/d1d/d30/d6a/fa7 x:0 0 0 2024-03-29T04:45:17.771 INFO:tasks.workunit.client.1.smithi152.stdout:5/493: mknod d6/d1b/ca8 0 2024-03-29T04:45:17.774 INFO:tasks.workunit.client.1.smithi152.stdout:4/374: rename d0/d7/d25/d48 to d0/d7/d25/d17/d5e/d60/d74 0 2024-03-29T04:45:17.776 INFO:tasks.workunit.client.1.smithi152.stdout:5/494: creat d6/d1d/fa9 x:0 0 0 2024-03-29T04:45:17.778 INFO:tasks.workunit.client.1.smithi152.stdout:5/495: mkdir d6/d1d/daa 0 2024-03-29T04:45:17.778 INFO:tasks.workunit.client.1.smithi152.stdout:5/496: read - d6/d1d/fa9 zero size 2024-03-29T04:45:17.779 INFO:tasks.workunit.client.1.smithi152.stdout:4/375: mkdir d0/d7/d40/d75 0 2024-03-29T04:45:17.780 INFO:tasks.workunit.client.1.smithi152.stdout:5/497: readlink d6/d1d/d3b/l62 0 2024-03-29T04:45:17.786 INFO:tasks.workunit.client.1.smithi152.stdout:4/376: dwrite d0/d7/d25/d17/f4d [0,4194304] 0 2024-03-29T04:45:17.788 INFO:tasks.workunit.client.1.smithi152.stdout:4/377: getdents d0/d27/d6e 0 2024-03-29T04:45:17.790 INFO:tasks.workunit.client.1.smithi152.stdout:4/378: fdatasync d0/d7/d40/ff 0 2024-03-29T04:45:17.793 INFO:tasks.workunit.client.1.smithi152.stdout:5/498: dwrite d6/d16/d25/d2e/da0/f71 [0,4194304] 0 2024-03-29T04:45:17.797 INFO:tasks.workunit.client.1.smithi152.stdout:5/499: mkdir d6/d16/d25/d2e/da0/dab 0 2024-03-29T04:45:17.799 INFO:tasks.workunit.client.1.smithi152.stdout:4/379: dread d0/d7/d40/f67 [0,4194304] 0 2024-03-29T04:45:17.800 INFO:tasks.workunit.client.1.smithi152.stdout:5/500: creat d6/d16/d47/fac x:0 0 0 2024-03-29T04:45:17.805 INFO:tasks.workunit.client.1.smithi152.stdout:5/501: link d6/d16/d25/c7f d6/d16/d25/d2e/d9d/cad 0 2024-03-29T04:45:17.808 INFO:tasks.workunit.client.1.smithi152.stdout:5/502: read d6/d16/d25/f3a [2076157,56203] 0 2024-03-29T04:45:17.810 INFO:tasks.workunit.client.1.smithi152.stdout:5/503: truncate d6/d1d/d30/fa2 939017 0 2024-03-29T04:45:17.810 INFO:tasks.workunit.client.1.smithi152.stdout:4/380: write d0/d7/d40/d2a/f63 [2018204,56259] 0 2024-03-29T04:45:17.811 INFO:tasks.workunit.client.1.smithi152.stdout:5/504: fdatasync d6/d1b/f5a 0 2024-03-29T04:45:17.814 INFO:tasks.workunit.client.1.smithi152.stdout:5/505: unlink d6/d16/d47/f59 0 2024-03-29T04:45:17.816 INFO:tasks.workunit.client.1.smithi152.stdout:5/506: mknod d6/d16/d25/d2e/da0/d92/cae 0 2024-03-29T04:45:17.817 INFO:tasks.workunit.client.1.smithi152.stdout:4/381: dwrite d0/d7/d40/d23/f28 [0,4194304] 0 2024-03-29T04:45:17.818 INFO:tasks.workunit.client.1.smithi152.stdout:4/382: dread - d0/d7/d40/d23/f53 zero size 2024-03-29T04:45:17.819 INFO:tasks.workunit.client.1.smithi152.stdout:4/383: fdatasync d0/f13 0 2024-03-29T04:45:17.819 INFO:tasks.workunit.client.1.smithi152.stdout:5/507: creat d6/d1d/d30/d6a/faf x:0 0 0 2024-03-29T04:45:17.820 INFO:tasks.workunit.client.1.smithi152.stdout:5/508: chown d6/d16/d25/l2a 13541069 1 2024-03-29T04:45:17.820 INFO:tasks.workunit.client.1.smithi152.stdout:4/384: write d0/d7/d40/d23/f53 [1001756,70132] 0 2024-03-29T04:45:17.823 INFO:tasks.workunit.client.1.smithi152.stdout:4/385: symlink d0/d7/d25/d17/d5e/l76 0 2024-03-29T04:45:17.824 INFO:tasks.workunit.client.1.smithi152.stdout:5/509: fsync d6/d16/d47/d56/d63/f6d 0 2024-03-29T04:45:17.826 INFO:tasks.workunit.client.1.smithi152.stdout:5/510: write d6/d1b/f45 [344189,96288] 0 2024-03-29T04:45:17.829 INFO:tasks.workunit.client.1.smithi152.stdout:5/511: read d6/d1d/d30/f38 [100576,86598] 0 2024-03-29T04:45:17.830 INFO:tasks.workunit.client.1.smithi152.stdout:5/512: read d6/d7/fa4 [1923568,45547] 0 2024-03-29T04:45:17.832 INFO:tasks.workunit.client.1.smithi152.stdout:5/513: unlink d6/f21 0 2024-03-29T04:45:17.836 INFO:tasks.workunit.client.1.smithi152.stdout:4/386: write d0/d27/f2b [968535,113797] 0 2024-03-29T04:45:17.838 INFO:tasks.workunit.client.1.smithi152.stdout:4/387: mkdir d0/d7/d25/d17/d37/d77 0 2024-03-29T04:45:17.842 INFO:tasks.workunit.client.1.smithi152.stdout:4/388: write d0/f11 [21091,109877] 0 2024-03-29T04:45:17.845 INFO:tasks.workunit.client.1.smithi152.stdout:4/389: getdents d0/d7 0 2024-03-29T04:45:17.846 INFO:tasks.workunit.client.1.smithi152.stdout:4/390: dread - d0/d27/f59 zero size 2024-03-29T04:45:17.858 INFO:tasks.workunit.client.1.smithi152.stdout:4/391: dread d0/d27/f34 [0,4194304] 0 2024-03-29T04:45:17.860 INFO:tasks.workunit.client.1.smithi152.stdout:4/392: mkdir d0/d27/d6e/d78 0 2024-03-29T04:45:17.860 INFO:tasks.workunit.client.1.smithi152.stdout:4/393: chown d0/d7/d25/d17 113061 1 2024-03-29T04:45:17.862 INFO:tasks.workunit.client.1.smithi152.stdout:4/394: dread - d0/d7/d25/d17/d5e/d60/d74/f51 zero size 2024-03-29T04:45:17.877 INFO:tasks.workunit.client.1.smithi152.stdout:0/513: sync 2024-03-29T04:45:17.880 INFO:tasks.workunit.client.1.smithi152.stdout:0/514: write d2/d31/f35 [1294955,28222] 0 2024-03-29T04:45:17.882 INFO:tasks.workunit.client.1.smithi152.stdout:0/515: rmdir d2/da 39 2024-03-29T04:45:17.882 INFO:tasks.workunit.client.1.smithi152.stdout:0/516: chown d2/d29/f44 0 1 2024-03-29T04:45:17.888 INFO:tasks.workunit.client.1.smithi152.stdout:0/517: dwrite d2/da/d17/d57/f71 [0,4194304] 0 2024-03-29T04:45:17.889 INFO:tasks.workunit.client.1.smithi152.stdout:0/518: chown d2/da/d3a/d6f/lae 38340345 1 2024-03-29T04:45:17.899 INFO:tasks.workunit.client.1.smithi152.stdout:0/519: dwrite d2/d29/f2f [0,4194304] 0 2024-03-29T04:45:17.902 INFO:tasks.workunit.client.1.smithi152.stdout:0/520: symlink d2/d31/lc3 0 2024-03-29T04:45:17.904 INFO:tasks.workunit.client.1.smithi152.stdout:0/521: truncate d2/da/df/f5e 996256 0 2024-03-29T04:45:17.905 INFO:tasks.workunit.client.1.smithi152.stdout:0/522: truncate d2/da/d17/d57/f64 649385 0 2024-03-29T04:45:17.987 INFO:tasks.workunit.client.1.smithi152.stdout:7/460: sync 2024-03-29T04:45:17.989 INFO:tasks.workunit.client.1.smithi152.stdout:7/461: rename d7/d8/c1b to d7/c89 0 2024-03-29T04:45:17.996 INFO:tasks.workunit.client.1.smithi152.stdout:7/462: dread - d7/d8/d12/d1d/d41/d45/f6a zero size 2024-03-29T04:45:17.998 INFO:tasks.workunit.client.1.smithi152.stdout:7/463: read d7/d8/d12/f61 [1111611,53683] 0 2024-03-29T04:45:18.005 INFO:tasks.workunit.client.1.smithi152.stdout:7/464: dwrite d7/d8/d12/d1d/f86 [0,4194304] 0 2024-03-29T04:45:18.015 INFO:tasks.workunit.client.1.smithi152.stdout:7/465: dwrite d7/d8/d12/f28 [0,4194304] 0 2024-03-29T04:45:18.018 INFO:tasks.workunit.client.1.smithi152.stdout:7/466: write d7/d8/d12/d1d/f86 [1549628,73299] 0 2024-03-29T04:45:18.020 INFO:tasks.workunit.client.1.smithi152.stdout:7/467: mknod d7/d8/d12/d1d/d41/c8a 0 2024-03-29T04:45:18.020 INFO:tasks.workunit.client.1.smithi152.stdout:7/468: fsync d7/f59 0 2024-03-29T04:45:18.029 INFO:tasks.workunit.client.1.smithi152.stdout:7/469: dwrite d7/d8/db/df/f3a [0,4194304] 0 2024-03-29T04:45:18.035 INFO:tasks.workunit.client.1.smithi152.stdout:7/470: truncate d7/d8/d12/d1d/d41/f4d 856298 0 2024-03-29T04:45:18.039 INFO:tasks.workunit.client.1.smithi152.stdout:7/471: creat d7/d8/f8b x:0 0 0 2024-03-29T04:45:18.041 INFO:tasks.workunit.client.1.smithi152.stdout:7/472: creat d7/d8/d12/d1d/f8c x:0 0 0 2024-03-29T04:45:18.042 INFO:tasks.workunit.client.1.smithi152.stdout:7/473: fdatasync d7/d8/d12/f43 0 2024-03-29T04:45:18.043 INFO:tasks.workunit.client.1.smithi152.stdout:7/474: creat d7/d8/d12/d15/f8d x:0 0 0 2024-03-29T04:45:18.045 INFO:tasks.workunit.client.1.smithi152.stdout:7/475: symlink d7/d8/d12/d1d/d41/l8e 0 2024-03-29T04:45:18.047 INFO:tasks.workunit.client.1.smithi152.stdout:7/476: mknod d7/d8/d37/c8f 0 2024-03-29T04:45:18.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:17 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:45:18.289 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:17 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:45:18.319 INFO:tasks.workunit.client.1.smithi152.stdout:2/368: sync 2024-03-29T04:45:18.319 INFO:tasks.workunit.client.1.smithi152.stdout:8/421: sync 2024-03-29T04:45:18.323 INFO:tasks.workunit.client.1.smithi152.stdout:8/422: rename d1/f73 to d1/d21/f90 0 2024-03-29T04:45:18.324 INFO:tasks.workunit.client.1.smithi152.stdout:8/423: write d1/d21/f2a [2962916,78271] 0 2024-03-29T04:45:18.325 INFO:tasks.workunit.client.1.smithi152.stdout:2/369: unlink d0/d3/d30/d15/d22/d4c/l6f 0 2024-03-29T04:45:18.330 INFO:tasks.workunit.client.1.smithi152.stdout:2/370: symlink d0/d3/df/l7b 0 2024-03-29T04:45:18.331 INFO:tasks.workunit.client.1.smithi152.stdout:8/424: dwrite d1/d2/d3/f29 [0,4194304] 0 2024-03-29T04:45:18.333 INFO:tasks.workunit.client.1.smithi152.stdout:8/425: truncate d1/d2/d3/f4e 1215856 0 2024-03-29T04:45:18.337 INFO:tasks.workunit.client.1.smithi152.stdout:2/371: fdatasync d0/d3/d30/d15/d22/f70 0 2024-03-29T04:45:18.338 INFO:tasks.workunit.client.1.smithi152.stdout:2/372: write d0/d3/f10 [771145,89026] 0 2024-03-29T04:45:18.339 INFO:tasks.workunit.client.1.smithi152.stdout:8/426: mkdir d1/d8a/d91 0 2024-03-29T04:45:18.343 INFO:tasks.workunit.client.1.smithi152.stdout:2/373: fdatasync d0/d3/d30/d15/d22/f70 0 2024-03-29T04:45:18.347 INFO:tasks.workunit.client.1.smithi152.stdout:2/374: rmdir d0/d3/d30/d15/d53/d54/d57 39 2024-03-29T04:45:18.353 INFO:tasks.workunit.client.1.smithi152.stdout:2/375: getdents d0/d3/d30/d15/d22/d4c/d5a 0 2024-03-29T04:45:18.354 INFO:tasks.workunit.client.1.smithi152.stdout:2/376: chown d0/d3/d30/d15/d22/d4c/f51 745117497 1 2024-03-29T04:45:18.355 INFO:tasks.workunit.client.1.smithi152.stdout:2/377: stat d0/f44 0 2024-03-29T04:45:18.356 INFO:tasks.workunit.client.1.smithi152.stdout:2/378: stat d0/d3/f1d 0 2024-03-29T04:45:18.358 INFO:tasks.workunit.client.1.smithi152.stdout:2/379: mknod d0/d3/df/d2c/c7c 0 2024-03-29T04:45:18.359 INFO:tasks.workunit.client.1.smithi152.stdout:2/380: symlink d0/d3/d30/d15/d22/d4c/d52/l7d 0 2024-03-29T04:45:18.360 INFO:tasks.workunit.client.1.smithi152.stdout:2/381: rename d0 to d0/d3/d30/d15/d53/d54/d7e 22 2024-03-29T04:45:18.362 INFO:tasks.workunit.client.1.smithi152.stdout:2/382: mknod d0/d3/d30/d15/d22/d4c/c7f 0 2024-03-29T04:45:18.374 INFO:tasks.workunit.client.1.smithi152.stdout:2/383: dread d0/d3/d30/d15/f29 [0,4194304] 0 2024-03-29T04:45:18.376 INFO:tasks.workunit.client.1.smithi152.stdout:2/384: creat d0/d3/d30/d15/d53/f80 x:0 0 0 2024-03-29T04:45:18.377 INFO:tasks.workunit.client.1.smithi152.stdout:2/385: write d0/d3/df/d2c/f55 [7919491,110399] 0 2024-03-29T04:45:18.381 INFO:tasks.workunit.client.1.smithi152.stdout:2/386: rmdir d0/d3/d30/d15/d22/d4c 39 2024-03-29T04:45:18.384 INFO:tasks.workunit.client.1.smithi152.stdout:2/387: creat d0/d3/f81 x:0 0 0 2024-03-29T04:45:18.396 INFO:tasks.workunit.client.1.smithi152.stdout:2/388: dread d0/d3/df/d2c/f55 [0,4194304] 0 2024-03-29T04:45:18.402 INFO:tasks.workunit.client.1.smithi152.stdout:2/389: dwrite d0/d3/d30/d15/d22/f49 [0,4194304] 0 2024-03-29T04:45:18.413 INFO:tasks.workunit.client.1.smithi152.stdout:2/390: dwrite d0/d3/d30/d15/d22/f70 [0,4194304] 0 2024-03-29T04:45:18.425 INFO:tasks.workunit.client.1.smithi152.stdout:2/391: dwrite d0/d3/df/f36 [0,4194304] 0 2024-03-29T04:45:18.439 INFO:tasks.workunit.client.1.smithi152.stdout:2/392: creat d0/d3/d30/d15/d22/d4c/d52/f82 x:0 0 0 2024-03-29T04:45:18.441 INFO:tasks.workunit.client.1.smithi152.stdout:2/393: unlink d0/d3/df/f24 0 2024-03-29T04:45:18.446 INFO:tasks.workunit.client.1.smithi152.stdout:2/394: rename d0/d3/d30/d15/d22/d4c/l58 to d0/d3/d30/d15/l83 0 2024-03-29T04:45:18.448 INFO:tasks.workunit.client.1.smithi152.stdout:2/395: fdatasync d0/f16 0 2024-03-29T04:45:18.450 INFO:tasks.workunit.client.1.smithi152.stdout:2/396: creat d0/d3/d30/d15/d22/d74/f84 x:0 0 0 2024-03-29T04:45:18.451 INFO:tasks.workunit.client.1.smithi152.stdout:2/397: truncate d0/d3/df/f76 278750 0 2024-03-29T04:45:18.462 INFO:tasks.workunit.client.1.smithi152.stdout:2/398: write d0/d3/d30/f1a [3266586,78680] 0 2024-03-29T04:45:18.465 INFO:tasks.workunit.client.1.smithi152.stdout:9/484: sync 2024-03-29T04:45:18.465 INFO:tasks.workunit.client.1.smithi152.stdout:3/491: sync 2024-03-29T04:45:18.467 INFO:tasks.workunit.client.1.smithi152.stdout:3/492: mknod d7/d46/c8b 0 2024-03-29T04:45:18.468 INFO:tasks.workunit.client.1.smithi152.stdout:9/485: stat d5/d8/d31/d59/d6f/f71 0 2024-03-29T04:45:18.469 INFO:tasks.workunit.client.1.smithi152.stdout:2/399: dwrite d0/d3/f18 [0,4194304] 0 2024-03-29T04:45:18.474 INFO:tasks.workunit.client.1.smithi152.stdout:3/493: dread d7/d1c/d41/d61/d75/f7c [0,4194304] 0 2024-03-29T04:45:18.474 INFO:tasks.workunit.client.1.smithi152.stdout:9/486: write d5/d12/d2c/f76 [531941,123524] 0 2024-03-29T04:45:18.475 INFO:tasks.workunit.client.1.smithi152.stdout:3/494: chown d7/d19/d3a/f49 1 1 2024-03-29T04:45:18.477 INFO:tasks.workunit.client.1.smithi152.stdout:2/400: dwrite d0/d3/d30/d15/d22/f49 [0,4194304] 0 2024-03-29T04:45:18.478 INFO:tasks.workunit.client.1.smithi152.stdout:9/487: creat d5/d12/d2c/d6e/d73/d7a/fac x:0 0 0 2024-03-29T04:45:18.480 INFO:tasks.workunit.client.1.smithi152.stdout:3/495: mkdir d7/d1c/d41/d61/d75/d8c 0 2024-03-29T04:45:18.480 INFO:tasks.workunit.client.1.smithi152.stdout:2/401: truncate d0/d3/d30/d15/f2b 4974407 0 2024-03-29T04:45:18.480 INFO:tasks.workunit.client.1.smithi152.stdout:9/488: creat d5/db/d1b/d68/fad x:0 0 0 2024-03-29T04:45:18.486 INFO:tasks.workunit.client.1.smithi152.stdout:3/496: creat d7/d1c/d41/d61/d85/f8d x:0 0 0 2024-03-29T04:45:18.487 INFO:tasks.workunit.client.1.smithi152.stdout:2/402: rmdir d0/d3/d30/d15/d77 0 2024-03-29T04:45:18.488 INFO:tasks.workunit.client.1.smithi152.stdout:9/489: dwrite d5/db/d1b/f4f [0,4194304] 0 2024-03-29T04:45:18.491 INFO:tasks.workunit.client.1.smithi152.stdout:2/403: readlink d0/d3/d30/l40 0 2024-03-29T04:45:18.492 INFO:tasks.workunit.client.1.smithi152.stdout:9/490: dread - d5/d12/d2c/d6e/d91/f94 zero size 2024-03-29T04:45:18.492 INFO:tasks.workunit.client.1.smithi152.stdout:3/497: dread d7/d1c/d41/d61/d75/f7c [0,4194304] 0 2024-03-29T04:45:18.493 INFO:tasks.workunit.client.1.smithi152.stdout:9/491: chown d5/f29 2963 1 2024-03-29T04:45:18.495 INFO:tasks.workunit.client.1.smithi152.stdout:9/492: truncate d5/db/d1b/f95 1073998 0 2024-03-29T04:45:18.500 INFO:tasks.workunit.client.1.smithi152.stdout:3/498: rename d7/d19/d1b/l44 to d7/d1c/d41/d61/d75/d8c/l8e 0 2024-03-29T04:45:18.503 INFO:tasks.workunit.client.1.smithi152.stdout:3/499: mkdir d7/d19/d1b/d1e/d4e/d8f 0 2024-03-29T04:45:18.506 INFO:tasks.workunit.client.1.smithi152.stdout:9/493: link d5/c22 d5/d12/d2c/d6e/d73/d8e/cae 0 2024-03-29T04:45:18.509 INFO:tasks.workunit.client.1.smithi152.stdout:9/494: dread - d5/d12/d2c/d6e/d73/d7a/fac zero size 2024-03-29T04:45:18.512 INFO:tasks.workunit.client.1.smithi152.stdout:9/495: symlink d5/db/laf 0 2024-03-29T04:45:18.513 INFO:tasks.workunit.client.1.smithi152.stdout:2/404: dwrite d0/d3/f5 [4194304,4194304] 0 2024-03-29T04:45:18.515 INFO:tasks.workunit.client.1.smithi152.stdout:2/405: chown d0/d3/df/d2c/l4f 9 1 2024-03-29T04:45:18.516 INFO:tasks.workunit.client.1.smithi152.stdout:9/496: creat d5/d8/d31/d59/fb0 x:0 0 0 2024-03-29T04:45:18.520 INFO:tasks.workunit.client.1.smithi152.stdout:3/500: dwrite d7/d1c/f47 [0,4194304] 0 2024-03-29T04:45:18.522 INFO:tasks.workunit.client.1.smithi152.stdout:3/501: fdatasync d7/d1c/f51 0 2024-03-29T04:45:18.522 INFO:tasks.workunit.client.1.smithi152.stdout:9/497: mkdir d5/d8/d31/d75/db1 0 2024-03-29T04:45:18.524 INFO:tasks.workunit.client.1.smithi152.stdout:8/427: sync 2024-03-29T04:45:18.524 INFO:tasks.workunit.client.1.smithi152.stdout:1/781: sync 2024-03-29T04:45:18.524 INFO:tasks.workunit.client.1.smithi152.stdout:0/523: sync 2024-03-29T04:45:18.524 INFO:tasks.workunit.client.1.smithi152.stdout:5/514: sync 2024-03-29T04:45:18.524 INFO:tasks.workunit.client.1.smithi152.stdout:4/395: sync 2024-03-29T04:45:18.524 INFO:tasks.workunit.client.1.smithi152.stdout:7/477: sync 2024-03-29T04:45:18.524 INFO:tasks.workunit.client.1.smithi152.stdout:6/618: sync 2024-03-29T04:45:18.529 INFO:tasks.workunit.client.1.smithi152.stdout:6/619: fsync d1/d3/d1a/f2c 0 2024-03-29T04:45:18.532 INFO:tasks.workunit.client.1.smithi152.stdout:5/515: stat d6/d16/d25/d2e/da0/f64 0 2024-03-29T04:45:18.536 INFO:tasks.workunit.client.1.smithi152.stdout:8/428: creat d1/d2/d1d/d42/f92 x:0 0 0 2024-03-29T04:45:18.536 INFO:tasks.workunit.client.1.smithi152.stdout:9/498: write d5/d12/d2c/f54 [4339555,76639] 0 2024-03-29T04:45:18.537 INFO:tasks.workunit.client.1.smithi152.stdout:0/524: mknod d2/da/d17/cc4 0 2024-03-29T04:45:18.537 INFO:tasks.workunit.client.1.smithi152.stdout:2/406: dwrite d0/f16 [0,4194304] 0 2024-03-29T04:45:18.541 INFO:tasks.workunit.client.1.smithi152.stdout:5/516: mkdir d6/d7/d75/db0 0 2024-03-29T04:45:18.542 INFO:tasks.workunit.client.1.smithi152.stdout:1/782: fsync d5/d7/d1b/d2e/d95/dad/d2b/f37 0 2024-03-29T04:45:18.542 INFO:tasks.workunit.client.1.smithi152.stdout:0/525: unlink d2/d29/f44 0 2024-03-29T04:45:18.544 INFO:tasks.workunit.client.1.smithi152.stdout:2/407: fsync d0/d3/d30/d15/f26 0 2024-03-29T04:45:18.544 INFO:tasks.workunit.client.1.smithi152.stdout:9/499: unlink d5/d8/f11 0 2024-03-29T04:45:18.545 INFO:tasks.workunit.client.1.smithi152.stdout:8/429: symlink d1/d2/d1d/d36/l93 0 2024-03-29T04:45:18.547 INFO:tasks.workunit.client.1.smithi152.stdout:3/502: truncate d7/d19/d1b/f21 6708329 0 2024-03-29T04:45:18.547 INFO:tasks.workunit.client.1.smithi152.stdout:5/517: dread - d6/d1d/d30/d6a/faf zero size 2024-03-29T04:45:18.549 INFO:tasks.workunit.client.1.smithi152.stdout:1/783: stat d5/d7/d8/d13/d19/d80/cb5 0 2024-03-29T04:45:18.549 INFO:tasks.workunit.client.1.smithi152.stdout:9/500: readlink d5/d12/d2c/d6e/d73/d8e/la2 0 2024-03-29T04:45:18.552 INFO:tasks.workunit.client.1.smithi152.stdout:6/620: rename d1/d3/d9/d99/fa6 to d1/fd0 0 2024-03-29T04:45:18.556 INFO:tasks.workunit.client.1.smithi152.stdout:3/503: unlink d7/d46/l72 0 2024-03-29T04:45:18.556 INFO:tasks.workunit.client.1.smithi152.stdout:1/784: fsync d5/d7/d8/d13/d19/dc7/fdf 0 2024-03-29T04:45:18.556 INFO:tasks.workunit.client.1.smithi152.stdout:4/396: dwrite d0/f13 [0,4194304] 0 2024-03-29T04:45:18.559 INFO:tasks.workunit.client.1.smithi152.stdout:7/478: dwrite d7/d8/d12/d35/f1a [0,4194304] 0 2024-03-29T04:45:18.562 INFO:tasks.workunit.client.1.smithi152.stdout:5/518: creat d6/d16/d47/d56/d63/fb1 x:0 0 0 2024-03-29T04:45:18.562 INFO:tasks.workunit.client.1.smithi152.stdout:0/526: dread d2/da/df/f5e [0,4194304] 0 2024-03-29T04:45:18.564 INFO:tasks.workunit.client.1.smithi152.stdout:3/504: symlink d7/d1c/d41/d6a/l90 0 2024-03-29T04:45:18.565 INFO:tasks.workunit.client.1.smithi152.stdout:0/527: fdatasync d2/da/d17/d57/f62 0 2024-03-29T04:45:18.566 INFO:tasks.workunit.client.1.smithi152.stdout:5/519: rmdir d6 39 2024-03-29T04:45:18.567 INFO:tasks.workunit.client.1.smithi152.stdout:2/408: dwrite d0/d3/d30/d15/d22/d4c/f4d [4194304,4194304] 0 2024-03-29T04:45:18.569 INFO:tasks.workunit.client.1.smithi152.stdout:2/409: chown d0/d3/df/d2c/l50 50581 1 2024-03-29T04:45:18.572 INFO:tasks.workunit.client.1.smithi152.stdout:6/621: creat d1/d3/d9/d13/d3e/d93/fd1 x:0 0 0 2024-03-29T04:45:18.572 INFO:tasks.workunit.client.1.smithi152.stdout:4/397: creat d0/d7/d25/d17/d65/f79 x:0 0 0 2024-03-29T04:45:18.579 INFO:tasks.workunit.client.1.smithi152.stdout:4/398: write d0/d7/d25/d17/f4c [2998308,123835] 0 2024-03-29T04:45:18.580 INFO:tasks.workunit.client.1.smithi152.stdout:0/528: mkdir d2/da/d17/d57/dc5 0 2024-03-29T04:45:18.582 INFO:tasks.workunit.client.1.smithi152.stdout:1/785: dwrite d5/d7/d1b/d2e/de2/d102/fd9 [0,4194304] 0 2024-03-29T04:45:18.585 INFO:tasks.workunit.client.1.smithi152.stdout:6/622: chown d1/d3/d9/d13/d28/c30 1 1 2024-03-29T04:45:18.587 INFO:tasks.workunit.client.1.smithi152.stdout:9/501: write d5/db/f21 [1863779,6820] 0 2024-03-29T04:45:18.590 INFO:tasks.workunit.client.1.smithi152.stdout:7/479: getdents d7/d8/db 0 2024-03-29T04:45:18.592 INFO:tasks.workunit.client.1.smithi152.stdout:8/430: dread d1/d21/d40/f52 [0,4194304] 0 2024-03-29T04:45:18.592 INFO:tasks.workunit.client.1.smithi152.stdout:2/410: getdents d0/d3 0 2024-03-29T04:45:18.593 INFO:tasks.workunit.client.1.smithi152.stdout:3/505: dwrite d7/f10 [0,4194304] 0 2024-03-29T04:45:18.596 INFO:tasks.workunit.client.1.smithi152.stdout:4/399: creat d0/d7/d25/d17/d5e/d60/d74/d57/f7a x:0 0 0 2024-03-29T04:45:18.596 INFO:tasks.workunit.client.1.smithi152.stdout:1/786: readlink d5/d7/d1b/d2e/d95/d9d/da7/laf 0 2024-03-29T04:45:18.597 INFO:tasks.workunit.client.1.smithi152.stdout:0/529: mkdir d2/da/d17/d50/dc6 0 2024-03-29T04:45:18.597 INFO:tasks.workunit.client.1.smithi152.stdout:2/411: mkdir d0/d3/d64/d85 0 2024-03-29T04:45:18.600 INFO:tasks.workunit.client.1.smithi152.stdout:0/530: rmdir d2/da/d17/d50 39 2024-03-29T04:45:18.600 INFO:tasks.workunit.client.1.smithi152.stdout:1/787: chown d5/d7/d4d/db1/fe5 452 1 2024-03-29T04:45:18.601 INFO:tasks.workunit.client.1.smithi152.stdout:2/412: write d0/d3/d30/f33 [3226287,91503] 0 2024-03-29T04:45:18.602 INFO:tasks.workunit.client.1.smithi152.stdout:7/480: read d7/d8/d12/d15/f50 [46603,95058] 0 2024-03-29T04:45:18.602 INFO:tasks.workunit.client.1.smithi152.stdout:6/623: dread d1/d3/d9/d13/d3e/f42 [0,4194304] 0 2024-03-29T04:45:18.602 INFO:tasks.workunit.client.1.smithi152.stdout:0/531: dread - d2/da/df/fb3 zero size 2024-03-29T04:45:18.607 INFO:tasks.workunit.client.1.smithi152.stdout:8/431: rename d1/d21/d40/d47/l75 to d1/d21/d40/d6a/l94 0 2024-03-29T04:45:18.655 INFO:tasks.workunit.client.1.smithi152.stdout:6/624: mkdir d1/d3/df/dd2 0 2024-03-29T04:45:18.655 INFO:tasks.workunit.client.1.smithi152.stdout:0/532: dread - d2/f6b zero size 2024-03-29T04:45:18.655 INFO:tasks.workunit.client.1.smithi152.stdout:7/481: creat d7/d8/d12/d15/f90 x:0 0 0 2024-03-29T04:45:18.656 INFO:tasks.workunit.client.1.smithi152.stdout:4/400: dread d0/d7/d25/d17/f54 [0,4194304] 0 2024-03-29T04:45:18.660 INFO:tasks.workunit.client.1.smithi152.stdout:6/625: chown d1/d3/df/l39 11 1 2024-03-29T04:45:18.665 INFO:tasks.workunit.client.1.smithi152.stdout:1/788: dwrite d5/d7/d1b/d2e/d95/dad/f46 [0,4194304] 0 2024-03-29T04:45:18.670 INFO:tasks.workunit.client.1.smithi152.stdout:8/432: dwrite d1/d2/d3/f8e [0,4194304] 0 2024-03-29T04:45:18.673 INFO:tasks.workunit.client.1.smithi152.stdout:4/401: creat d0/d27/d6e/d78/f7b x:0 0 0 2024-03-29T04:45:18.679 INFO:tasks.workunit.client.1.smithi152.stdout:6/626: dread d1/d3/df/f81 [0,4194304] 0 2024-03-29T04:45:18.680 INFO:tasks.workunit.client.1.smithi152.stdout:8/433: mknod d1/d2/d6d/c95 0 2024-03-29T04:45:18.681 INFO:tasks.workunit.client.1.smithi152.stdout:0/533: dwrite d2/da/d17/d50/fb7 [0,4194304] 0 2024-03-29T04:45:18.682 INFO:tasks.workunit.client.1.smithi152.stdout:7/482: dwrite d7/d8/fa [0,4194304] 0 2024-03-29T04:45:18.685 INFO:tasks.workunit.client.1.smithi152.stdout:3/506: dread d7/d19/d1b/d1e/d22/f3b [0,4194304] 0 2024-03-29T04:45:18.687 INFO:tasks.workunit.client.1.smithi152.stdout:4/402: mknod d0/d7/d40/d23/d55/c7c 0 2024-03-29T04:45:18.688 INFO:tasks.workunit.client.1.smithi152.stdout:8/434: creat d1/d2/d3/f96 x:0 0 0 2024-03-29T04:45:18.690 INFO:tasks.workunit.client.1.smithi152.stdout:3/507: read d7/d19/d3a/f49 [1910581,23776] 0 2024-03-29T04:45:18.691 INFO:tasks.workunit.client.1.smithi152.stdout:2/413: dread d0/d3/d30/f33 [0,4194304] 0 2024-03-29T04:45:18.692 INFO:tasks.workunit.client.1.smithi152.stdout:6/627: creat d1/d3/d9/d13/d3e/fd3 x:0 0 0 2024-03-29T04:45:18.694 INFO:tasks.workunit.client.1.smithi152.stdout:6/628: fdatasync d1/d3/d9/d13/d3e/d54/f82 0 2024-03-29T04:45:18.698 INFO:tasks.workunit.client.1.smithi152.stdout:0/534: link d2/da/df/c49 d2/da/d17/d50/dac/cc7 0 2024-03-29T04:45:18.698 INFO:tasks.workunit.client.1.smithi152.stdout:8/435: symlink d1/d2/d1d/d42/l97 0 2024-03-29T04:45:18.700 INFO:tasks.workunit.client.1.smithi152.stdout:4/403: creat d0/d7/d40/d75/f7d x:0 0 0 2024-03-29T04:45:18.700 INFO:tasks.workunit.client.1.smithi152.stdout:2/414: symlink d0/d3/d30/d15/d22/d4c/l86 0 2024-03-29T04:45:18.703 INFO:tasks.workunit.client.1.smithi152.stdout:8/436: write d1/d2/d3/f80 [819143,113496] 0 2024-03-29T04:45:18.703 INFO:tasks.workunit.client.1.smithi152.stdout:3/508: creat d7/f91 x:0 0 0 2024-03-29T04:45:18.703 INFO:tasks.workunit.client.1.smithi152.stdout:4/404: dread - d0/d7/d25/d17/d5e/d60/d74/f51 zero size 2024-03-29T04:45:18.704 INFO:tasks.workunit.client.1.smithi152.stdout:2/415: creat d0/d3/d30/f87 x:0 0 0 2024-03-29T04:45:18.705 INFO:tasks.workunit.client.1.smithi152.stdout:2/416: dread - d0/d3/f81 zero size 2024-03-29T04:45:18.706 INFO:tasks.workunit.client.1.smithi152.stdout:8/437: rename d1/d21/c2d to d1/d2/d6d/c98 0 2024-03-29T04:45:18.707 INFO:tasks.workunit.client.1.smithi152.stdout:3/509: rename d7/d1c/d41 to d7/d1c/d41/d61/d85/d92 22 2024-03-29T04:45:18.708 INFO:tasks.workunit.client.1.smithi152.stdout:2/417: symlink d0/d3/d30/d15/d22/d74/l88 0 2024-03-29T04:45:18.709 INFO:tasks.workunit.client.1.smithi152.stdout:8/438: creat d1/d2/d1d/d36/d85/f99 x:0 0 0 2024-03-29T04:45:18.710 INFO:tasks.workunit.client.1.smithi152.stdout:3/510: symlink d7/d19/d1b/d1e/d4e/l93 0 2024-03-29T04:45:18.711 INFO:tasks.workunit.client.1.smithi152.stdout:2/418: readlink d0/d3/df/d2c/d45/l73 0 2024-03-29T04:45:18.713 INFO:tasks.workunit.client.1.smithi152.stdout:3/511: symlink d7/d19/d1b/l94 0 2024-03-29T04:45:18.715 INFO:tasks.workunit.client.1.smithi152.stdout:2/419: creat d0/d3/d30/d15/d22/d4c/f89 x:0 0 0 2024-03-29T04:45:18.726 INFO:tasks.workunit.client.1.smithi152.stdout:3/512: dwrite d7/f87 [0,4194304] 0 2024-03-29T04:45:18.729 INFO:tasks.workunit.client.1.smithi152.stdout:7/483: dread d7/d8/d12/f43 [0,4194304] 0 2024-03-29T04:45:18.732 INFO:tasks.workunit.client.1.smithi152.stdout:2/420: dwrite d0/d3/df/f35 [0,4194304] 0 2024-03-29T04:45:18.737 INFO:tasks.workunit.client.1.smithi152.stdout:7/484: chown d7/d8/db/df/f3a 578378639 1 2024-03-29T04:45:18.739 INFO:tasks.workunit.client.1.smithi152.stdout:2/421: write d0/f3b [967916,38954] 0 2024-03-29T04:45:18.740 INFO:tasks.workunit.client.1.smithi152.stdout:3/513: creat d7/d1c/d41/d6a/d78/f95 x:0 0 0 2024-03-29T04:45:18.740 INFO:tasks.workunit.client.1.smithi152.stdout:0/535: dwrite d2/da/df/f5e [0,4194304] 0 2024-03-29T04:45:18.744 INFO:tasks.workunit.client.1.smithi152.stdout:7/485: creat d7/d8/d12/f91 x:0 0 0 2024-03-29T04:45:18.747 INFO:tasks.workunit.client.1.smithi152.stdout:4/405: dwrite d0/d7/d25/f3f [0,4194304] 0 2024-03-29T04:45:18.749 INFO:tasks.workunit.client.1.smithi152.stdout:8/439: dwrite d1/d21/f24 [0,4194304] 0 2024-03-29T04:45:18.752 INFO:tasks.workunit.client.1.smithi152.stdout:2/422: symlink d0/d3/df/d2c/l8a 0 2024-03-29T04:45:18.752 INFO:tasks.workunit.client.1.smithi152.stdout:3/514: creat d7/d1c/d41/d6a/d78/f96 x:0 0 0 2024-03-29T04:45:18.755 INFO:tasks.workunit.client.1.smithi152.stdout:8/440: unlink d1/d2/d3/l2c 0 2024-03-29T04:45:18.756 INFO:tasks.workunit.client.1.smithi152.stdout:7/486: read d7/d8/db/f42 [1334884,3119] 0 2024-03-29T04:45:18.758 INFO:tasks.workunit.client.1.smithi152.stdout:4/406: link d0/d27/f34 d0/d7/d40/d23/d55/d70/f7e 0 2024-03-29T04:45:18.758 INFO:tasks.workunit.client.1.smithi152.stdout:8/441: chown d1/d2/d3/c64 189733658 1 2024-03-29T04:45:18.759 INFO:tasks.workunit.client.1.smithi152.stdout:2/423: unlink d0/d3/df/d2c/d45/f4e 0 2024-03-29T04:45:18.760 INFO:tasks.workunit.client.1.smithi152.stdout:0/536: link d2/da/d3a/c43 d2/da/d3a/d6f/cc8 0 2024-03-29T04:45:18.760 INFO:tasks.workunit.client.1.smithi152.stdout:8/442: dread - d1/d21/d40/d47/f79 zero size 2024-03-29T04:45:18.761 INFO:tasks.workunit.client.1.smithi152.stdout:3/515: mknod d7/d1c/d41/d61/c97 0 2024-03-29T04:45:18.762 INFO:tasks.workunit.client.1.smithi152.stdout:3/516: truncate d7/f87 4829796 0 2024-03-29T04:45:18.763 INFO:tasks.workunit.client.1.smithi152.stdout:8/443: rmdir d1 39 2024-03-29T04:45:18.768 INFO:tasks.workunit.client.1.smithi152.stdout:8/444: fdatasync d1/d2/d1d/f30 0 2024-03-29T04:45:18.770 INFO:tasks.workunit.client.1.smithi152.stdout:0/537: truncate d2/d31/f36 2227097 0 2024-03-29T04:45:18.771 INFO:tasks.workunit.client.1.smithi152.stdout:4/407: link d0/d7/d25/d17/d5e/d60/d74/d57/l41 d0/d7/d40/d23/d55/d6c/l7f 0 2024-03-29T04:45:18.772 INFO:tasks.workunit.client.1.smithi152.stdout:8/445: chown d1/d2/d1d/l31 1952415 1 2024-03-29T04:45:18.775 INFO:tasks.workunit.client.1.smithi152.stdout:7/487: dread d7/d8/d12/d1d/f62 [0,4194304] 0 2024-03-29T04:45:18.775 INFO:tasks.workunit.client.1.smithi152.stdout:4/408: link d0/d7/d40/f67 d0/d7/d40/f80 0 2024-03-29T04:45:18.776 INFO:tasks.workunit.client.1.smithi152.stdout:0/538: read d2/da/d3a/f97 [3034342,120590] 0 2024-03-29T04:45:18.777 INFO:tasks.workunit.client.1.smithi152.stdout:7/488: rename d7/f14 to d7/f92 0 2024-03-29T04:45:18.780 INFO:tasks.workunit.client.1.smithi152.stdout:7/489: stat d7/d8/d12/d1d/f8c 0 2024-03-29T04:45:18.784 INFO:tasks.workunit.client.1.smithi152.stdout:4/409: dwrite d0/f13 [0,4194304] 0 2024-03-29T04:45:18.784 INFO:tasks.workunit.client.1.smithi152.stdout:7/490: chown d7/d8/d12/d35/d68/d84 647498307 1 2024-03-29T04:45:18.785 INFO:tasks.workunit.client.1.smithi152.stdout:4/410: chown d0/d7/d40/d2a/c58 77 1 2024-03-29T04:45:18.787 INFO:tasks.workunit.client.1.smithi152.stdout:3/517: truncate d7/d46/f6d 1246769 0 2024-03-29T04:45:18.790 INFO:tasks.workunit.client.1.smithi152.stdout:7/491: write d7/d8/d12/d35/d68/f88 [803329,42226] 0 2024-03-29T04:45:18.791 INFO:tasks.workunit.client.1.smithi152.stdout:2/424: dread d0/d3/df/f43 [0,4194304] 0 2024-03-29T04:45:18.792 INFO:tasks.workunit.client.1.smithi152.stdout:3/518: write d7/d19/d1b/f7b [538156,103652] 0 2024-03-29T04:45:18.795 INFO:tasks.workunit.client.1.smithi152.stdout:3/519: chown d7/d19/d1b/d1e/d4e/f76 1292402 1 2024-03-29T04:45:18.798 INFO:tasks.workunit.client.1.smithi152.stdout:4/411: link d0/d7/d25/f3f d0/d7/d25/d17/d5e/d60/d74/f81 0 2024-03-29T04:45:18.799 INFO:tasks.workunit.client.1.smithi152.stdout:0/539: dread d2/da/d17/d57/f8c [0,4194304] 0 2024-03-29T04:45:18.800 INFO:tasks.workunit.client.1.smithi152.stdout:7/492: rename d7/d8/d37/d75 to d7/d8/d93 0 2024-03-29T04:45:18.801 INFO:tasks.workunit.client.1.smithi152.stdout:0/540: dread - d2/da/d17/fbd zero size 2024-03-29T04:45:18.802 INFO:tasks.workunit.client.1.smithi152.stdout:3/520: dread d7/d19/d1b/f7b [0,4194304] 0 2024-03-29T04:45:18.804 INFO:tasks.workunit.client.1.smithi152.stdout:0/541: stat d2/da/d17/d57/d9c/dbe 0 2024-03-29T04:45:18.806 INFO:tasks.workunit.client.1.smithi152.stdout:4/412: creat d0/d7/d40/f82 x:0 0 0 2024-03-29T04:45:18.809 INFO:tasks.workunit.client.1.smithi152.stdout:7/493: mknod d7/d8/c94 0 2024-03-29T04:45:18.810 INFO:tasks.workunit.client.1.smithi152.stdout:0/542: unlink d2/d29/c69 0 2024-03-29T04:45:18.813 INFO:tasks.workunit.client.1.smithi152.stdout:4/413: symlink d0/d7/d40/d6d/l83 0 2024-03-29T04:45:18.816 INFO:tasks.workunit.client.1.smithi152.stdout:2/425: dread d0/d3/d30/f19 [0,4194304] 0 2024-03-29T04:45:18.817 INFO:tasks.workunit.client.1.smithi152.stdout:4/414: creat d0/d7/f84 x:0 0 0 2024-03-29T04:45:18.818 INFO:tasks.workunit.client.1.smithi152.stdout:0/543: getdents d2/da 0 2024-03-29T04:45:18.818 INFO:tasks.workunit.client.1.smithi152.stdout:3/521: dwrite d7/d19/d3a/f49 [0,4194304] 0 2024-03-29T04:45:18.819 INFO:tasks.workunit.client.1.smithi152.stdout:2/426: chown d0/f16 172234 1 2024-03-29T04:45:18.822 INFO:tasks.workunit.client.1.smithi152.stdout:0/544: creat d2/da/d3a/d6f/fc9 x:0 0 0 2024-03-29T04:45:18.827 INFO:tasks.workunit.client.1.smithi152.stdout:2/427: link d0/d3/d30/c48 d0/c8b 0 2024-03-29T04:45:18.827 INFO:tasks.workunit.client.1.smithi152.stdout:4/415: dwrite d0/f11 [0,4194304] 0 2024-03-29T04:45:18.829 INFO:tasks.workunit.client.1.smithi152.stdout:3/522: dwrite d7/d19/d3a/f55 [4194304,4194304] 0 2024-03-29T04:45:18.829 INFO:tasks.workunit.client.1.smithi152.stdout:2/428: dread - d0/d3/f81 zero size 2024-03-29T04:45:18.831 INFO:tasks.workunit.client.1.smithi152.stdout:3/523: creat d7/d46/f98 x:0 0 0 2024-03-29T04:45:18.833 INFO:tasks.workunit.client.1.smithi152.stdout:3/524: fsync d7/d19/d1b/d59/f68 0 2024-03-29T04:45:18.836 INFO:tasks.workunit.client.1.smithi152.stdout:3/525: mkdir d7/d1c/d41/d61/d99 0 2024-03-29T04:45:18.837 INFO:tasks.workunit.client.1.smithi152.stdout:0/545: write d2/d31/f36 [1740451,86961] 0 2024-03-29T04:45:18.872 INFO:tasks.workunit.client.1.smithi152.stdout:4/416: mknod d0/d7/d25/d17/d5e/d60/c85 0 2024-03-29T04:45:18.873 INFO:tasks.workunit.client.1.smithi152.stdout:0/546: mknod d2/da/d3a/d6f/cca 0 2024-03-29T04:45:18.887 INFO:tasks.workunit.client.1.smithi152.stdout:4/417: dwrite d0/d7/d25/d17/d5e/d60/d74/f50 [0,4194304] 0 2024-03-29T04:45:18.887 INFO:tasks.workunit.client.1.smithi152.stdout:3/526: dwrite d7/d19/d1b/f2f [0,4194304] 0 2024-03-29T04:45:18.890 INFO:tasks.workunit.client.1.smithi152.stdout:3/527: chown d7/d1c/d41/d61/d75/d8c/l8e 497049 1 2024-03-29T04:45:18.891 INFO:tasks.workunit.client.1.smithi152.stdout:3/528: stat d7/d19/d1b/d1e/d22/f3b 0 2024-03-29T04:45:18.892 INFO:tasks.workunit.client.1.smithi152.stdout:4/418: mknod d0/d7/d40/d23/d55/c86 0 2024-03-29T04:45:18.895 INFO:tasks.workunit.client.1.smithi152.stdout:4/419: creat d0/d27/d6e/f87 x:0 0 0 2024-03-29T04:45:18.896 INFO:tasks.workunit.client.1.smithi152.stdout:3/529: mknod d7/d1c/d41/d61/d99/c9a 0 2024-03-29T04:45:18.897 INFO:tasks.workunit.client.1.smithi152.stdout:3/530: chown d7/d19/f26 7127331 1 2024-03-29T04:45:18.899 INFO:tasks.workunit.client.1.smithi152.stdout:3/531: symlink d7/d19/d3a/l9b 0 2024-03-29T04:45:18.901 INFO:tasks.workunit.client.1.smithi152.stdout:3/532: creat d7/d1c/f9c x:0 0 0 2024-03-29T04:45:18.903 INFO:tasks.workunit.client.1.smithi152.stdout:3/533: truncate d7/d19/d1b/d1e/d4e/f5a 943198 0 2024-03-29T04:45:18.903 INFO:tasks.workunit.client.1.smithi152.stdout:3/534: chown d7/d19/f25 6972205 1 2024-03-29T04:45:18.911 INFO:tasks.workunit.client.1.smithi152.stdout:3/535: truncate d7/d19/f26 1236579 0 2024-03-29T04:45:18.912 INFO:tasks.workunit.client.1.smithi152.stdout:3/536: symlink d7/d1c/d41/d61/d75/d8c/l9d 0 2024-03-29T04:45:18.914 INFO:tasks.workunit.client.1.smithi152.stdout:3/537: write d7/d19/d1b/d1e/d22/f2d [2291084,129651] 0 2024-03-29T04:45:18.919 INFO:tasks.workunit.client.1.smithi152.stdout:3/538: getdents d7 0 2024-03-29T04:45:18.933 INFO:tasks.workunit.client.1.smithi152.stdout:3/539: dread d7/d19/d1b/d1e/d22/f4d [0,4194304] 0 2024-03-29T04:45:18.939 INFO:tasks.workunit.client.1.smithi152.stdout:3/540: truncate d7/d19/d1b/f7b 320984 0 2024-03-29T04:45:18.940 INFO:tasks.workunit.client.1.smithi152.stdout:3/541: creat d7/d46/f9e x:0 0 0 2024-03-29T04:45:18.942 INFO:tasks.workunit.client.1.smithi152.stdout:3/542: write d7/d19/d3a/f7f [95292,5911] 0 2024-03-29T04:45:18.944 INFO:tasks.workunit.client.1.smithi152.stdout:3/543: creat d7/d46/f9f x:0 0 0 2024-03-29T04:45:18.945 INFO:tasks.workunit.client.1.smithi152.stdout:3/544: write d7/d19/d1b/f58 [1281925,42416] 0 2024-03-29T04:45:18.947 INFO:tasks.workunit.client.1.smithi152.stdout:3/545: creat d7/d1c/d41/d61/d75/d8c/fa0 x:0 0 0 2024-03-29T04:45:18.948 INFO:tasks.workunit.client.1.smithi152.stdout:3/546: rmdir d7/d1c/d41/d61 39 2024-03-29T04:45:18.949 INFO:tasks.workunit.client.1.smithi152.stdout:3/547: stat d7/d19/l79 0 2024-03-29T04:45:18.951 INFO:tasks.workunit.client.1.smithi152.stdout:3/548: rmdir d7/d1c/d41/d61/d99 39 2024-03-29T04:45:18.953 INFO:tasks.workunit.client.1.smithi152.stdout:3/549: write d7/d19/d1b/d1e/d22/f3b [157411,129027] 0 2024-03-29T04:45:18.960 INFO:tasks.workunit.client.1.smithi152.stdout:3/550: rename d7/d1c to d7/d19/d1b/d1e/da1 0 2024-03-29T04:45:18.966 INFO:tasks.workunit.client.1.smithi152.stdout:3/551: dread d7/d19/d1b/d1e/d4e/f56 [0,4194304] 0 2024-03-29T04:45:18.967 INFO:tasks.workunit.client.1.smithi152.stdout:3/552: fdatasync d7/d19/d1b/d1e/d4e/f56 0 2024-03-29T04:45:18.969 INFO:tasks.workunit.client.1.smithi152.stdout:3/553: fsync d7/d19/d1b/d1e/da1/f47 0 2024-03-29T04:45:18.973 INFO:tasks.workunit.client.1.smithi152.stdout:3/554: rename d7/c35 to d7/d19/d1b/ca2 0 2024-03-29T04:45:18.979 INFO:tasks.workunit.client.1.smithi152.stdout:3/555: dread d7/d19/d3a/f7f [0,4194304] 0 2024-03-29T04:45:18.981 INFO:tasks.workunit.client.1.smithi152.stdout:3/556: creat d7/fa3 x:0 0 0 2024-03-29T04:45:18.986 INFO:tasks.workunit.client.1.smithi152.stdout:3/557: rmdir d7/d19/d1b/d1e/da1/d41/d61/d85/d88 0 2024-03-29T04:45:18.987 INFO:tasks.workunit.client.1.smithi152.stdout:3/558: stat d7/d19/d1b/f21 0 2024-03-29T04:45:18.988 INFO:tasks.workunit.client.1.smithi152.stdout:3/559: fsync d7/d19/d1b/d1e/d4e/f76 0 2024-03-29T04:45:18.989 INFO:tasks.workunit.client.1.smithi152.stdout:3/560: read d7/d19/d1b/d1e/d4e/f56 [2025,128278] 0 2024-03-29T04:45:19.003 INFO:tasks.workunit.client.1.smithi152.stdout:3/561: dread d7/d19/d1b/f2f [4194304,4194304] 0 2024-03-29T04:45:19.003 INFO:tasks.workunit.client.1.smithi152.stdout:3/562: stat d7/d19/d3a/c37 0 2024-03-29T04:45:19.006 INFO:tasks.workunit.client.1.smithi152.stdout:3/563: creat d7/fa4 x:0 0 0 2024-03-29T04:45:19.007 INFO:tasks.workunit.client.1.smithi152.stdout:3/564: chown d7/d46/f9e 3654647 1 2024-03-29T04:45:19.009 INFO:tasks.workunit.client.1.smithi152.stdout:3/565: symlink d7/d19/d1b/d1e/da1/d41/d6a/d78/la5 0 2024-03-29T04:45:19.039 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:18 smithi152 ceph-mon[37177]: pgmap v275: 65 pgs: 65 active+clean; 2.8 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 85 MiB/s rd, 264 MiB/s wr, 478 op/s 2024-03-29T04:45:19.088 INFO:tasks.workunit.client.1.smithi152.stdout:5/520: sync 2024-03-29T04:45:19.090 INFO:tasks.workunit.client.1.smithi152.stdout:5/521: truncate d6/d16/d25/f8e 421211 0 2024-03-29T04:45:19.091 INFO:tasks.workunit.client.1.smithi152.stdout:5/522: dread - d6/d1d/d3b/f95 zero size 2024-03-29T04:45:19.091 INFO:tasks.workunit.client.1.smithi152.stdout:5/523: chown d6/d7/lc 805541 1 2024-03-29T04:45:19.094 INFO:tasks.workunit.client.1.smithi152.stdout:5/524: mkdir d6/d16/db2 0 2024-03-29T04:45:19.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:18 smithi142 ceph-mon[29027]: pgmap v275: 65 pgs: 65 active+clean; 2.8 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 85 MiB/s rd, 264 MiB/s wr, 478 op/s 2024-03-29T04:45:19.317 INFO:tasks.workunit.client.1.smithi152.stdout:9/502: sync 2024-03-29T04:45:19.318 INFO:tasks.workunit.client.1.smithi152.stdout:9/503: rename d5/db/d1b/d68 to d5/db/d1b/d68/daa/db2 22 2024-03-29T04:45:19.320 INFO:tasks.workunit.client.1.smithi152.stdout:9/504: creat d5/d8/d31/d59/fb3 x:0 0 0 2024-03-29T04:45:19.321 INFO:tasks.workunit.client.1.smithi152.stdout:9/505: stat d5/db/d1b/f37 0 2024-03-29T04:45:19.323 INFO:tasks.workunit.client.1.smithi152.stdout:9/506: mkdir d5/d12/d2c/d6e/db4 0 2024-03-29T04:45:19.325 INFO:tasks.workunit.client.1.smithi152.stdout:9/507: creat d5/fb5 x:0 0 0 2024-03-29T04:45:19.339 INFO:tasks.workunit.client.1.smithi152.stdout:9/508: dwrite d5/f61 [0,4194304] 0 2024-03-29T04:45:19.340 INFO:tasks.workunit.client.1.smithi152.stdout:9/509: stat d5/d8/f53 0 2024-03-29T04:45:19.346 INFO:tasks.workunit.client.1.smithi152.stdout:9/510: getdents d5/d12/d2c/d6e/d73/d7a 0 2024-03-29T04:45:19.349 INFO:tasks.workunit.client.1.smithi152.stdout:9/511: creat d5/db/d1b/d68/daa/fb6 x:0 0 0 2024-03-29T04:45:19.351 INFO:tasks.workunit.client.1.smithi152.stdout:9/512: truncate d5/d12/d2c/d6e/fa0 838738 0 2024-03-29T04:45:19.352 INFO:tasks.workunit.client.1.smithi152.stdout:9/513: mkdir d5/d12/db7 0 2024-03-29T04:45:19.356 INFO:tasks.workunit.client.1.smithi152.stdout:9/514: rename d5/d12/d2c/d6e/d73/d8e/cae to d5/d12/db7/cb8 0 2024-03-29T04:45:19.357 INFO:tasks.workunit.client.1.smithi152.stdout:9/515: chown d5/d12/d2c/d6e/d91/ca6 964643586 1 2024-03-29T04:45:19.359 INFO:tasks.workunit.client.1.smithi152.stdout:9/516: symlink d5/d12/d2c/d6e/d91/lb9 0 2024-03-29T04:45:19.364 INFO:tasks.workunit.client.1.smithi152.stdout:9/517: dwrite d5/db/d1b/f95 [0,4194304] 0 2024-03-29T04:45:19.369 INFO:tasks.workunit.client.1.smithi152.stdout:9/518: rename d5/d12/d2c/d6e/d91/lb9 to d5/d8/d31/d75/db1/lba 0 2024-03-29T04:45:19.371 INFO:tasks.workunit.client.1.smithi152.stdout:9/519: mkdir d5/d12/d2c/d6e/dbb 0 2024-03-29T04:45:19.376 INFO:tasks.workunit.client.1.smithi152.stdout:9/520: rename d5/d12/d2c/f64 to d5/d8/d31/d75/db1/fbc 0 2024-03-29T04:45:19.385 INFO:tasks.workunit.client.1.smithi152.stdout:9/521: truncate d5/db/f4e 1571577 0 2024-03-29T04:45:19.386 INFO:tasks.workunit.client.1.smithi152.stdout:9/522: write d5/d12/d2c/f78 [943484,57116] 0 2024-03-29T04:45:19.388 INFO:tasks.workunit.client.1.smithi152.stdout:9/523: symlink d5/d8/d31/d75/db1/lbd 0 2024-03-29T04:45:19.399 INFO:tasks.workunit.client.1.smithi152.stdout:9/524: dwrite d5/d8/d31/f4c [0,4194304] 0 2024-03-29T04:45:19.401 INFO:tasks.workunit.client.1.smithi152.stdout:1/789: sync 2024-03-29T04:45:19.402 INFO:tasks.workunit.client.1.smithi152.stdout:1/790: stat d5/d7/d1b/d2e/d95/dad/d12/ff7 0 2024-03-29T04:45:19.403 INFO:tasks.workunit.client.1.smithi152.stdout:9/525: rmdir d5/d12/d2c/d6e/dbb 0 2024-03-29T04:45:19.408 INFO:tasks.workunit.client.1.smithi152.stdout:1/791: creat d5/d7/d1b/d2e/d95/dad/d12/d77/f103 x:0 0 0 2024-03-29T04:45:19.410 INFO:tasks.workunit.client.1.smithi152.stdout:9/526: dread d5/f20 [0,4194304] 0 2024-03-29T04:45:19.411 INFO:tasks.workunit.client.1.smithi152.stdout:9/527: read - d5/db/d1b/d68/f89 zero size 2024-03-29T04:45:19.412 INFO:tasks.workunit.client.1.smithi152.stdout:9/528: chown d5/d12/d2c/d6e/d73/d8e/f74 404478482 1 2024-03-29T04:45:19.412 INFO:tasks.workunit.client.1.smithi152.stdout:9/529: dread - d5/fb5 zero size 2024-03-29T04:45:19.624 INFO:tasks.workunit.client.1.smithi152.stdout:6/629: sync 2024-03-29T04:45:19.624 INFO:tasks.workunit.client.1.smithi152.stdout:2/429: sync 2024-03-29T04:45:19.626 INFO:tasks.workunit.client.1.smithi152.stdout:2/430: mkdir d0/d3/df/d8c 0 2024-03-29T04:45:19.628 INFO:tasks.workunit.client.1.smithi152.stdout:2/431: mkdir d0/d3/d64/d8d 0 2024-03-29T04:45:19.629 INFO:tasks.workunit.client.1.smithi152.stdout:2/432: read d0/d3/df/f35 [2288065,15452] 0 2024-03-29T04:45:19.636 INFO:tasks.workunit.client.1.smithi152.stdout:2/433: dwrite d0/d3/d30/d15/d22/f70 [0,4194304] 0 2024-03-29T04:45:19.640 INFO:tasks.workunit.client.1.smithi152.stdout:6/630: dread d1/d3/df/f4f [0,4194304] 0 2024-03-29T04:45:19.643 INFO:tasks.workunit.client.1.smithi152.stdout:2/434: dread d0/d3/d30/d15/d53/d54/d57/f72 [0,4194304] 0 2024-03-29T04:45:19.649 INFO:tasks.workunit.client.1.smithi152.stdout:6/631: write d1/d3/d9/d13/d3e/d54/f7c [93366,37913] 0 2024-03-29T04:45:19.650 INFO:tasks.workunit.client.1.smithi152.stdout:6/632: stat d1/d3/df/f4f 0 2024-03-29T04:45:19.654 INFO:tasks.workunit.client.1.smithi152.stdout:2/435: dwrite d0/d3/f42 [0,4194304] 0 2024-03-29T04:45:19.657 INFO:tasks.workunit.client.1.smithi152.stdout:2/436: symlink d0/d3/l8e 0 2024-03-29T04:45:19.669 INFO:tasks.workunit.client.1.smithi152.stdout:2/437: rename d0/d3/d30/d15/d22/d74/l88 to d0/d3/df/d8c/l8f 0 2024-03-29T04:45:19.670 INFO:tasks.workunit.client.1.smithi152.stdout:2/438: fdatasync d0/d3/d30/d15/d22/d4c/d5a/f6b 0 2024-03-29T04:45:19.674 INFO:tasks.workunit.client.1.smithi152.stdout:2/439: getdents d0/d3/df/d2c 0 2024-03-29T04:45:19.676 INFO:tasks.workunit.client.1.smithi152.stdout:2/440: fsync d0/d3/d30/f19 0 2024-03-29T04:45:19.678 INFO:tasks.workunit.client.1.smithi152.stdout:2/441: write d0/d3/df/d2c/f55 [1399788,79974] 0 2024-03-29T04:45:19.678 INFO:tasks.workunit.client.1.smithi152.stdout:6/633: dread d1/fe [0,4194304] 0 2024-03-29T04:45:19.679 INFO:tasks.workunit.client.1.smithi152.stdout:6/634: stat d1/ca5 0 2024-03-29T04:45:19.681 INFO:tasks.workunit.client.1.smithi152.stdout:6/635: symlink d1/d3/ld4 0 2024-03-29T04:45:19.682 INFO:tasks.workunit.client.1.smithi152.stdout:2/442: creat d0/d3/d30/d15/d22/d4c/f90 x:0 0 0 2024-03-29T04:45:19.682 INFO:tasks.workunit.client.1.smithi152.stdout:6/636: rename d1/d3 to d1/d3/d9/d13/d3e/d93/dd5 22 2024-03-29T04:45:19.859 INFO:tasks.workunit.client.1.smithi152.stdout:8/446: sync 2024-03-29T04:45:19.860 INFO:tasks.workunit.client.1.smithi152.stdout:7/494: sync 2024-03-29T04:45:19.862 INFO:tasks.workunit.client.1.smithi152.stdout:8/447: mkdir d1/d2/d3c/d76/d9a 0 2024-03-29T04:45:19.863 INFO:tasks.workunit.client.1.smithi152.stdout:8/448: getdents d1/d8a/d91 0 2024-03-29T04:45:19.868 INFO:tasks.workunit.client.1.smithi152.stdout:8/449: write d1/d2/d1d/f4f [1599380,48349] 0 2024-03-29T04:45:19.872 INFO:tasks.workunit.client.1.smithi152.stdout:8/450: getdents d1/d21/d40/d47 0 2024-03-29T04:45:19.872 INFO:tasks.workunit.client.1.smithi152.stdout:8/451: dread - d1/d2/d1d/f83 zero size 2024-03-29T04:45:19.878 INFO:tasks.workunit.client.1.smithi152.stdout:8/452: truncate d1/d21/f25 2382393 0 2024-03-29T04:45:19.881 INFO:tasks.workunit.client.1.smithi152.stdout:8/453: read d1/d21/d40/f6c [785035,61672] 0 2024-03-29T04:45:19.885 INFO:tasks.workunit.client.1.smithi152.stdout:8/454: rename d1/d2/d3/f80 to d1/d2/d3c/d76/f9b 0 2024-03-29T04:45:19.888 INFO:tasks.workunit.client.1.smithi152.stdout:8/455: getdents d1/d2/d1d/d36 0 2024-03-29T04:45:19.895 INFO:tasks.workunit.client.1.smithi152.stdout:8/456: dwrite d1/d2/d3/f96 [0,4194304] 0 2024-03-29T04:45:19.931 INFO:tasks.workunit.client.1.smithi152.stdout:8/457: dread d1/d21/d40/f6c [0,4194304] 0 2024-03-29T04:45:19.933 INFO:tasks.workunit.client.1.smithi152.stdout:8/458: creat d1/d2/d6d/f9c x:0 0 0 2024-03-29T04:45:19.946 INFO:tasks.workunit.client.1.smithi152.stdout:8/459: dread d1/d21/f2a [0,4194304] 0 2024-03-29T04:45:19.948 INFO:tasks.workunit.client.1.smithi152.stdout:8/460: mknod d1/d21/d40/d6a/c9d 0 2024-03-29T04:45:19.954 INFO:tasks.workunit.client.1.smithi152.stdout:8/461: dwrite d1/d2/d81/f84 [0,4194304] 0 2024-03-29T04:45:19.956 INFO:tasks.workunit.client.1.smithi152.stdout:8/462: write d1/d2/d1d/d36/d85/f99 [651936,25773] 0 2024-03-29T04:45:19.956 INFO:tasks.workunit.client.1.smithi152.stdout:8/463: chown d1/d2/d1d/d36/d5d 2 1 2024-03-29T04:45:19.959 INFO:tasks.workunit.client.1.smithi152.stdout:8/464: creat d1/d2/d1d/d42/f9e x:0 0 0 2024-03-29T04:45:19.960 INFO:tasks.workunit.client.1.smithi152.stdout:8/465: readlink d1/la 0 2024-03-29T04:45:19.961 INFO:tasks.workunit.client.1.smithi152.stdout:8/466: symlink d1/d2/d1d/d42/l9f 0 2024-03-29T04:45:19.964 INFO:tasks.workunit.client.1.smithi152.stdout:8/467: creat d1/d2/d1d/d36/fa0 x:0 0 0 2024-03-29T04:45:19.966 INFO:tasks.workunit.client.1.smithi152.stdout:8/468: mkdir d1/d2/d1d/d36/d4b/da1 0 2024-03-29T04:45:19.967 INFO:tasks.workunit.client.1.smithi152.stdout:8/469: creat d1/d21/d8b/fa2 x:0 0 0 2024-03-29T04:45:19.970 INFO:tasks.workunit.client.1.smithi152.stdout:8/470: truncate d1/d21/d40/d47/f72 426290 0 2024-03-29T04:45:19.972 INFO:tasks.workunit.client.1.smithi152.stdout:8/471: creat d1/d21/d40/d6a/fa3 x:0 0 0 2024-03-29T04:45:19.973 INFO:tasks.workunit.client.1.smithi152.stdout:8/472: readlink d1/d2/d3/l8 0 2024-03-29T04:45:19.974 INFO:tasks.workunit.client.1.smithi152.stdout:8/473: mknod d1/d21/d8b/ca4 0 2024-03-29T04:45:19.975 INFO:tasks.workunit.client.1.smithi152.stdout:8/474: read - d1/d2/d1d/d36/d4b/f68 zero size 2024-03-29T04:45:19.976 INFO:tasks.workunit.client.1.smithi152.stdout:8/475: mknod d1/d2/d1d/d36/d5d/ca5 0 2024-03-29T04:45:19.978 INFO:tasks.workunit.client.1.smithi152.stdout:8/476: rename d1/d2/d6d/c82 to d1/d2/d3c/d76/ca6 0 2024-03-29T04:45:19.980 INFO:tasks.workunit.client.1.smithi152.stdout:8/477: creat d1/d2/d1d/d36/d85/fa7 x:0 0 0 2024-03-29T04:45:19.981 INFO:tasks.workunit.client.1.smithi152.stdout:8/478: creat d1/d2/d59/fa8 x:0 0 0 2024-03-29T04:45:19.982 INFO:tasks.workunit.client.1.smithi152.stdout:8/479: write d1/d21/d40/d6a/f89 [4326744,21699] 0 2024-03-29T04:45:19.986 INFO:tasks.workunit.client.1.smithi152.stdout:8/480: rename d1/d2/d81 to d1/d21/da9 0 2024-03-29T04:45:19.991 INFO:tasks.workunit.client.1.smithi152.stdout:8/481: getdents d1/d2/d3c/d76 0 2024-03-29T04:45:20.001 INFO:tasks.workunit.client.1.smithi152.stdout:8/482: dwrite d1/d2/d1d/d36/d5d/f71 [0,4194304] 0 2024-03-29T04:45:20.010 INFO:tasks.workunit.client.1.smithi152.stdout:8/483: dwrite d1/d2/d1d/f41 [0,4194304] 0 2024-03-29T04:45:20.011 INFO:tasks.workunit.client.1.smithi152.stdout:0/547: sync 2024-03-29T04:45:20.012 INFO:tasks.workunit.client.1.smithi152.stdout:2/443: sync 2024-03-29T04:45:20.012 INFO:tasks.workunit.client.1.smithi152.stdout:3/566: sync 2024-03-29T04:45:20.012 INFO:tasks.workunit.client.1.smithi152.stdout:6/637: sync 2024-03-29T04:45:20.012 INFO:tasks.workunit.client.1.smithi152.stdout:4/420: sync 2024-03-29T04:45:20.013 INFO:tasks.workunit.client.1.smithi152.stdout:5/525: sync 2024-03-29T04:45:20.013 INFO:tasks.workunit.client.1.smithi152.stdout:9/530: sync 2024-03-29T04:45:20.013 INFO:tasks.workunit.client.1.smithi152.stdout:1/792: sync 2024-03-29T04:45:20.017 INFO:tasks.workunit.client.1.smithi152.stdout:1/793: fdatasync d5/f98 0 2024-03-29T04:45:20.017 INFO:tasks.workunit.client.1.smithi152.stdout:0/548: truncate d2/f6b 360254 0 2024-03-29T04:45:20.017 INFO:tasks.workunit.client.1.smithi152.stdout:5/526: chown d6/d7/d75/c87 303 1 2024-03-29T04:45:20.019 INFO:tasks.workunit.client.1.smithi152.stdout:8/484: mknod d1/d2/d1d/d36/d85/caa 0 2024-03-29T04:45:20.021 INFO:tasks.workunit.client.1.smithi152.stdout:3/567: mknod d7/d19/d1b/d1e/da1/d41/d6a/ca6 0 2024-03-29T04:45:20.023 INFO:tasks.workunit.client.1.smithi152.stdout:8/485: chown d1/d21/d40/c5b 74 1 2024-03-29T04:45:20.023 INFO:tasks.workunit.client.1.smithi152.stdout:4/421: creat d0/d7/d25/d17/d5e/d60/f88 x:0 0 0 2024-03-29T04:45:20.024 INFO:tasks.workunit.client.1.smithi152.stdout:2/444: dwrite d0/d3/f5 [0,4194304] 0 2024-03-29T04:45:20.029 INFO:tasks.workunit.client.1.smithi152.stdout:6/638: rename d1/d3/d9/d13/d28/c30 to d1/d3/d9/d13/d28/cd6 0 2024-03-29T04:45:20.032 INFO:tasks.workunit.client.1.smithi152.stdout:4/422: write d0/d7/d25/d17/d5e/d60/d74/f51 [997028,96237] 0 2024-03-29T04:45:20.032 INFO:tasks.workunit.client.1.smithi152.stdout:5/527: dwrite d6/d1b/f91 [0,4194304] 0 2024-03-29T04:45:20.033 INFO:tasks.workunit.client.1.smithi152.stdout:6/639: dread - d1/d3/d9/d13/d3e/fcd zero size 2024-03-29T04:45:20.039 INFO:tasks.workunit.client.1.smithi152.stdout:2/445: unlink d0/d3/df/f13 0 2024-03-29T04:45:20.039 INFO:tasks.workunit.client.1.smithi152.stdout:1/794: truncate d5/d7/d8/d13/d19/dc7/fdf 1746684 0 2024-03-29T04:45:20.041 INFO:tasks.workunit.client.1.smithi152.stdout:8/486: link d1/d21/d40/f6c d1/fab 0 2024-03-29T04:45:20.042 INFO:tasks.workunit.client.1.smithi152.stdout:4/423: symlink d0/d7/d40/d62/l89 0 2024-03-29T04:45:20.043 INFO:tasks.workunit.client.1.smithi152.stdout:8/487: readlink d1/d2/d1d/l31 0 2024-03-29T04:45:20.043 INFO:tasks.workunit.client.1.smithi152.stdout:0/549: dwrite d2/da/df/f93 [0,4194304] 0 2024-03-29T04:45:20.045 INFO:tasks.workunit.client.1.smithi152.stdout:4/424: dread - d0/d27/f59 zero size 2024-03-29T04:45:20.045 INFO:tasks.workunit.client.1.smithi152.stdout:5/528: mkdir d6/d16/db3 0 2024-03-29T04:45:20.045 INFO:tasks.workunit.client.1.smithi152.stdout:6/640: write d1/d3/d9/d13/d28/faa [352496,99386] 0 2024-03-29T04:45:20.054 INFO:tasks.workunit.client.1.smithi152.stdout:1/795: rename d5/d7/d1b/d2e/d95/dad/d2b to d5/d7/d1b/d2e/de2/d102/d104 0 2024-03-29T04:45:20.055 INFO:tasks.workunit.client.1.smithi152.stdout:2/446: creat d0/d3/d30/d15/d22/f91 x:0 0 0 2024-03-29T04:45:20.055 INFO:tasks.workunit.client.1.smithi152.stdout:4/425: dread d0/d7/d25/d17/f3b [0,4194304] 0 2024-03-29T04:45:20.060 INFO:tasks.workunit.client.1.smithi152.stdout:8/488: dwrite d1/d2/d1d/f4f [0,4194304] 0 2024-03-29T04:45:20.061 INFO:tasks.workunit.client.1.smithi152.stdout:0/550: write d2/da/d17/f41 [2137841,36863] 0 2024-03-29T04:45:20.067 INFO:tasks.workunit.client.1.smithi152.stdout:2/447: truncate d0/d3/d30/d15/d22/d4c/d52/f82 171799 0 2024-03-29T04:45:20.068 INFO:tasks.workunit.client.1.smithi152.stdout:4/426: chown d0/d7/d25/d17/d5e/d60/d74/d57/l41 40070 1 2024-03-29T04:45:20.068 INFO:tasks.workunit.client.1.smithi152.stdout:8/489: creat d1/d2/d3c/fac x:0 0 0 2024-03-29T04:45:20.070 INFO:tasks.workunit.client.1.smithi152.stdout:6/641: dwrite d1/d3/d9/d13/d28/f34 [0,4194304] 0 2024-03-29T04:45:20.073 INFO:tasks.workunit.client.1.smithi152.stdout:1/796: getdents d5/d7/d1b/d2e/d95 0 2024-03-29T04:45:20.074 INFO:tasks.workunit.client.1.smithi152.stdout:3/568: dread d7/f1a [0,4194304] 0 2024-03-29T04:45:20.077 INFO:tasks.workunit.client.1.smithi152.stdout:5/529: dwrite d6/f41 [0,4194304] 0 2024-03-29T04:45:20.078 INFO:tasks.workunit.client.1.smithi152.stdout:1/797: mknod d5/d7/d4d/db1/c105 0 2024-03-29T04:45:20.080 INFO:tasks.workunit.client.1.smithi152.stdout:0/551: symlink d2/da/d17/d57/dc5/lcb 0 2024-03-29T04:45:20.081 INFO:tasks.workunit.client.1.smithi152.stdout:8/490: creat d1/d2/d3c/d76/d9a/fad x:0 0 0 2024-03-29T04:45:20.084 INFO:tasks.workunit.client.1.smithi152.stdout:3/569: creat d7/d19/d1b/d59/fa7 x:0 0 0 2024-03-29T04:45:20.085 INFO:tasks.workunit.client.1.smithi152.stdout:4/427: creat d0/d7/d40/f8a x:0 0 0 2024-03-29T04:45:20.086 INFO:tasks.workunit.client.1.smithi152.stdout:8/491: write d1/d2/d3/f29 [1451076,100934] 0 2024-03-29T04:45:20.086 INFO:tasks.workunit.client.1.smithi152.stdout:1/798: symlink d5/d7/d1b/d2e/d95/dad/d12/d77/l106 0 2024-03-29T04:45:20.087 INFO:tasks.workunit.client.1.smithi152.stdout:6/642: mkdir d1/d3/d1a/d1b/d29/dce/dd7 0 2024-03-29T04:45:20.087 INFO:tasks.workunit.client.1.smithi152.stdout:4/428: readlink d0/d7/d25/d17/d37/l6f 0 2024-03-29T04:45:20.088 INFO:tasks.workunit.client.1.smithi152.stdout:0/552: creat d2/da/d17/d57/fcc x:0 0 0 2024-03-29T04:45:20.090 INFO:tasks.workunit.client.1.smithi152.stdout:2/448: dwrite d0/d3/df/f35 [4194304,4194304] 0 2024-03-29T04:45:20.090 INFO:tasks.workunit.client.1.smithi152.stdout:5/530: rmdir d6/d7/d75/db0 0 2024-03-29T04:45:20.091 INFO:tasks.workunit.client.1.smithi152.stdout:6/643: symlink d1/d3/df/ld8 0 2024-03-29T04:45:20.092 INFO:tasks.workunit.client.1.smithi152.stdout:0/553: truncate d2/da/d17/db5/fbf 7086 0 2024-03-29T04:45:20.093 INFO:tasks.workunit.client.1.smithi152.stdout:4/429: fsync d0/d7/d40/f80 0 2024-03-29T04:45:20.094 INFO:tasks.workunit.client.1.smithi152.stdout:0/554: chown d2/da/d17/cc4 468744 1 2024-03-29T04:45:20.098 INFO:tasks.workunit.client.1.smithi152.stdout:2/449: dread d0/d3/f42 [0,4194304] 0 2024-03-29T04:45:20.100 INFO:tasks.workunit.client.1.smithi152.stdout:5/531: creat d6/d1d/daa/fb4 x:0 0 0 2024-03-29T04:45:20.102 INFO:tasks.workunit.client.1.smithi152.stdout:0/555: rename d2/da/d3a/f87 to d2/da/d17/d57/dc5/fcd 0 2024-03-29T04:45:20.102 INFO:tasks.workunit.client.1.smithi152.stdout:5/532: read d6/d1b/f5a [273346,56726] 0 2024-03-29T04:45:20.103 INFO:tasks.workunit.client.1.smithi152.stdout:1/799: link d5/d7/d8/d13/fb2 d5/d7/d8/d13/dd0/f107 0 2024-03-29T04:45:20.104 INFO:tasks.workunit.client.1.smithi152.stdout:5/533: readlink d6/d7/lc 0 2024-03-29T04:45:20.109 INFO:tasks.workunit.client.1.smithi152.stdout:5/534: symlink d6/d16/d25/d2e/da0/lb5 0 2024-03-29T04:45:20.109 INFO:tasks.workunit.client.1.smithi152.stdout:0/556: dwrite d2/f20 [0,4194304] 0 2024-03-29T04:45:20.113 INFO:tasks.workunit.client.1.smithi152.stdout:1/800: dread d5/d7/d1b/d2e/de2/d102/fd9 [0,4194304] 0 2024-03-29T04:45:20.115 INFO:tasks.workunit.client.1.smithi152.stdout:4/430: dread d0/d27/f34 [0,4194304] 0 2024-03-29T04:45:20.115 INFO:tasks.workunit.client.1.smithi152.stdout:4/431: fsync d0/f64 0 2024-03-29T04:45:20.117 INFO:tasks.workunit.client.1.smithi152.stdout:5/535: link d6/d1d/d3b/d55/c84 d6/d1d/daa/cb6 0 2024-03-29T04:45:20.118 INFO:tasks.workunit.client.1.smithi152.stdout:5/536: chown d6/d16/d25/d2e/d9d/c9e 1 1 2024-03-29T04:45:20.119 INFO:tasks.workunit.client.1.smithi152.stdout:5/537: truncate d6/d1d/daa/fb4 324501 0 2024-03-29T04:45:20.120 INFO:tasks.workunit.client.1.smithi152.stdout:5/538: chown d6/d1d/d3b/d55/l51 2489049 1 2024-03-29T04:45:20.121 INFO:tasks.workunit.client.1.smithi152.stdout:5/539: chown d6/d1d/d3b/d55/fe 287654505 1 2024-03-29T04:45:20.123 INFO:tasks.workunit.client.1.smithi152.stdout:6/644: dread d1/d3/d9/d13/f26 [0,4194304] 0 2024-03-29T04:45:20.123 INFO:tasks.workunit.client.1.smithi152.stdout:5/540: truncate d6/d16/d47/d56/f94 88027 0 2024-03-29T04:45:20.124 INFO:tasks.workunit.client.1.smithi152.stdout:5/541: chown d6/d1d/fa9 41039216 1 2024-03-29T04:45:20.124 INFO:tasks.workunit.client.1.smithi152.stdout:6/645: chown d1/d3/d1a/d1b/d29/fc4 0 1 2024-03-29T04:45:20.127 INFO:tasks.workunit.client.1.smithi152.stdout:6/646: creat d1/d3/d9/d13/d3e/fd9 x:0 0 0 2024-03-29T04:45:20.129 INFO:tasks.workunit.client.1.smithi152.stdout:5/542: rename d6/d16/d47 to d6/d1d/daa/db7 0 2024-03-29T04:45:20.133 INFO:tasks.workunit.client.1.smithi152.stdout:4/432: dwrite d0/d7/d40/f80 [0,4194304] 0 2024-03-29T04:45:20.134 INFO:tasks.workunit.client.1.smithi152.stdout:4/433: chown d0/d7/d40 991644142 1 2024-03-29T04:45:20.135 INFO:tasks.workunit.client.1.smithi152.stdout:4/434: write d0/d7/d40/f67 [2224788,65250] 0 2024-03-29T04:45:20.136 INFO:tasks.workunit.client.1.smithi152.stdout:6/647: write d1/d3/d1a/d1b/d29/d6f/f69 [780351,110528] 0 2024-03-29T04:45:20.138 INFO:tasks.workunit.client.1.smithi152.stdout:4/435: write d0/d7/d25/d17/f4d [4842615,48928] 0 2024-03-29T04:45:20.140 INFO:tasks.workunit.client.1.smithi152.stdout:5/543: getdents d6/d1d/d3b/d55 0 2024-03-29T04:45:20.141 INFO:tasks.workunit.client.1.smithi152.stdout:1/801: dread d5/fb8 [0,4194304] 0 2024-03-29T04:45:20.141 INFO:tasks.workunit.client.1.smithi152.stdout:6/648: creat d1/d3/d1a/d1b/d29/dce/fda x:0 0 0 2024-03-29T04:45:20.142 INFO:tasks.workunit.client.1.smithi152.stdout:4/436: fdatasync d0/f15 0 2024-03-29T04:45:20.145 INFO:tasks.workunit.client.1.smithi152.stdout:6/649: symlink d1/d3/d91/ldb 0 2024-03-29T04:45:20.145 INFO:tasks.workunit.client.1.smithi152.stdout:1/802: creat d5/d7/d1b/f108 x:0 0 0 2024-03-29T04:45:20.149 INFO:tasks.workunit.client.1.smithi152.stdout:6/650: symlink d1/d3/d1a/d1b/d29/d6f/d95/ldc 0 2024-03-29T04:45:20.152 INFO:tasks.workunit.client.1.smithi152.stdout:5/544: dread d6/d1d/d3b/d55/f66 [0,4194304] 0 2024-03-29T04:45:20.153 INFO:tasks.workunit.client.1.smithi152.stdout:4/437: dwrite d0/d7/d25/d17/d5e/d60/d74/d57/f47 [0,4194304] 0 2024-03-29T04:45:20.157 INFO:tasks.workunit.client.1.smithi152.stdout:7/495: sync 2024-03-29T04:45:20.160 INFO:tasks.workunit.client.1.smithi152.stdout:6/651: dwrite d1/d3/d9/d13/f31 [0,4194304] 0 2024-03-29T04:45:20.160 INFO:tasks.workunit.client.1.smithi152.stdout:7/496: write d7/d8/d37/f7d [1009652,26018] 0 2024-03-29T04:45:20.160 INFO:tasks.workunit.client.1.smithi152.stdout:4/438: rename d0/f4 to d0/d7/d25/d17/d5e/d60/f8b 0 2024-03-29T04:45:20.162 INFO:tasks.workunit.client.1.smithi152.stdout:1/803: dread d5/d7/d8/d13/d19/dc7/fdf [0,4194304] 0 2024-03-29T04:45:20.163 INFO:tasks.workunit.client.1.smithi152.stdout:4/439: chown d0/d27/c56 20355 1 2024-03-29T04:45:20.164 INFO:tasks.workunit.client.1.smithi152.stdout:6/652: creat d1/d3/d1a/d1b/d29/dce/fdd x:0 0 0 2024-03-29T04:45:20.166 INFO:tasks.workunit.client.1.smithi152.stdout:4/440: chown d0/d7/d40/l18 93 1 2024-03-29T04:45:20.166 INFO:tasks.workunit.client.1.smithi152.stdout:7/497: write d7/d8/d12/f87 [432372,1294] 0 2024-03-29T04:45:20.167 INFO:tasks.workunit.client.1.smithi152.stdout:5/545: truncate d6/d1b/f91 1679534 0 2024-03-29T04:45:20.168 INFO:tasks.workunit.client.1.smithi152.stdout:7/498: stat d7/d8/d12/d15/d72/f79 0 2024-03-29T04:45:20.170 INFO:tasks.workunit.client.1.smithi152.stdout:4/441: mkdir d0/d27/d6e/d8c 0 2024-03-29T04:45:20.173 INFO:tasks.workunit.client.1.smithi152.stdout:5/546: link l4 d6/d1d/d3b/lb8 0 2024-03-29T04:45:20.176 INFO:tasks.workunit.client.1.smithi152.stdout:4/442: creat d0/d7/f8d x:0 0 0 2024-03-29T04:45:20.177 INFO:tasks.workunit.client.1.smithi152.stdout:5/547: mkdir d6/d1d/daa/db7/d56/d63/db9 0 2024-03-29T04:45:20.179 INFO:tasks.workunit.client.1.smithi152.stdout:5/548: symlink d6/d1b/lba 0 2024-03-29T04:45:20.180 INFO:tasks.workunit.client.1.smithi152.stdout:7/499: write d7/d8/d12/d35/f32 [850155,31540] 0 2024-03-29T04:45:20.184 INFO:tasks.workunit.client.1.smithi152.stdout:4/443: dwrite d0/f15 [0,4194304] 0 2024-03-29T04:45:20.185 INFO:tasks.workunit.client.1.smithi152.stdout:7/500: creat d7/d8/d12/d1d/d30/f95 x:0 0 0 2024-03-29T04:45:20.187 INFO:tasks.workunit.client.1.smithi152.stdout:7/501: write d7/d8/d12/f7b [585627,56172] 0 2024-03-29T04:45:20.192 INFO:tasks.workunit.client.1.smithi152.stdout:4/444: mkdir d0/d7/d25/d8e 0 2024-03-29T04:45:20.195 INFO:tasks.workunit.client.1.smithi152.stdout:1/804: dread d5/d7/d8/fb [0,4194304] 0 2024-03-29T04:45:20.196 INFO:tasks.workunit.client.1.smithi152.stdout:4/445: write d0/d27/d6e/f87 [641181,61834] 0 2024-03-29T04:45:20.200 INFO:tasks.workunit.client.1.smithi152.stdout:4/446: chown d0/d7/d25/d17/d5e/d60/f8b 6941 1 2024-03-29T04:45:20.200 INFO:tasks.workunit.client.1.smithi152.stdout:1/805: mkdir d5/d7/d38/d109 0 2024-03-29T04:45:20.201 INFO:tasks.workunit.client.1.smithi152.stdout:7/502: dwrite d7/d8/d12/f81 [0,4194304] 0 2024-03-29T04:45:20.202 INFO:tasks.workunit.client.1.smithi152.stdout:6/653: dread d1/d3/df/f2e [0,4194304] 0 2024-03-29T04:45:20.204 INFO:tasks.workunit.client.1.smithi152.stdout:1/806: dread - d5/d7/d4d/fc8 zero size 2024-03-29T04:45:20.207 INFO:tasks.workunit.client.1.smithi152.stdout:6/654: creat d1/d3/d1a/d1b/d29/d6f/d95/fde x:0 0 0 2024-03-29T04:45:20.215 INFO:tasks.workunit.client.1.smithi152.stdout:5/549: dread d6/d1d/d30/f46 [0,4194304] 0 2024-03-29T04:45:20.216 INFO:tasks.workunit.client.1.smithi152.stdout:4/447: dwrite d0/d27/f2b [0,4194304] 0 2024-03-29T04:45:20.220 INFO:tasks.workunit.client.1.smithi152.stdout:1/807: dwrite d5/d7/d8/d13/d19/ff4 [0,4194304] 0 2024-03-29T04:45:20.224 INFO:tasks.workunit.client.1.smithi152.stdout:7/503: dread d7/d8/d12/f20 [0,4194304] 0 2024-03-29T04:45:20.228 INFO:tasks.workunit.client.1.smithi152.stdout:7/504: write d7/d8/db/df/f3a [2329519,6938] 0 2024-03-29T04:45:20.230 INFO:tasks.workunit.client.1.smithi152.stdout:1/808: fdatasync d5/d7/d38/f6f 0 2024-03-29T04:45:20.231 INFO:tasks.workunit.client.1.smithi152.stdout:4/448: symlink d0/d7/d25/d17/d5e/d60/d74/d57/l8f 0 2024-03-29T04:45:20.231 INFO:tasks.workunit.client.1.smithi152.stdout:6/655: link d1/d3/d1a/d1b/d29/c2b d1/d3/d9/d13/dc0/dc3/cdf 0 2024-03-29T04:45:20.233 INFO:tasks.workunit.client.1.smithi152.stdout:5/550: dwrite d6/d1d/d3b/f95 [0,4194304] 0 2024-03-29T04:45:20.234 INFO:tasks.workunit.client.1.smithi152.stdout:7/505: write d7/d8/d12/d15/f76 [324179,61042] 0 2024-03-29T04:45:20.237 INFO:tasks.workunit.client.1.smithi152.stdout:1/809: dread - d5/d7/d8/d13/dd0/ffb zero size 2024-03-29T04:45:20.237 INFO:tasks.workunit.client.1.smithi152.stdout:1/810: fdatasync d5/d7/d4d/faa 0 2024-03-29T04:45:20.242 INFO:tasks.workunit.client.1.smithi152.stdout:4/449: mknod d0/d7/d40/d6d/c90 0 2024-03-29T04:45:20.242 INFO:tasks.workunit.client.1.smithi152.stdout:7/506: rename d7/d8/d12/d1d/c38 to d7/d8/d12/d1d/c96 0 2024-03-29T04:45:20.244 INFO:tasks.workunit.client.1.smithi152.stdout:1/811: symlink d5/d7/d8/d13/d19/l10a 0 2024-03-29T04:45:20.244 INFO:tasks.workunit.client.1.smithi152.stdout:7/507: write d7/d8/d12/d1d/f49 [1617211,77266] 0 2024-03-29T04:45:20.244 INFO:tasks.workunit.client.1.smithi152.stdout:5/551: link d6/d16/d25/d2e/da0/c9a d6/d16/db3/cbb 0 2024-03-29T04:45:20.246 INFO:tasks.workunit.client.1.smithi152.stdout:4/450: mkdir d0/d7/d40/d23/d55/d6c/d91 0 2024-03-29T04:45:20.246 INFO:tasks.workunit.client.1.smithi152.stdout:7/508: chown d7/d8/d12/d1d/d41 316 1 2024-03-29T04:45:20.248 INFO:tasks.workunit.client.1.smithi152.stdout:4/451: unlink d0/f64 0 2024-03-29T04:45:20.249 INFO:tasks.workunit.client.1.smithi152.stdout:1/812: rename d5/d7/d4d/faa to d5/d7/d4d/f10b 0 2024-03-29T04:45:20.256 INFO:tasks.workunit.client.1.smithi152.stdout:6/656: dread d1/d3/d9/f3a [0,4194304] 0 2024-03-29T04:45:20.256 INFO:tasks.workunit.client.1.smithi152.stdout:1/813: write d5/d7/d1b/d2e/de2/d102/d104/f2f [1216616,59687] 0 2024-03-29T04:45:20.257 INFO:tasks.workunit.client.1.smithi152.stdout:5/552: dwrite d6/d7/f8a [0,4194304] 0 2024-03-29T04:45:20.260 INFO:tasks.workunit.client.1.smithi152.stdout:5/553: readlink d6/d16/d25/d2e/l77 0 2024-03-29T04:45:20.263 INFO:tasks.workunit.client.1.smithi152.stdout:6/657: mknod d1/d3/d91/ce0 0 2024-03-29T04:45:20.265 INFO:tasks.workunit.client.1.smithi152.stdout:4/452: dwrite d0/d7/d40/d2a/f63 [0,4194304] 0 2024-03-29T04:45:20.269 INFO:tasks.workunit.client.1.smithi152.stdout:1/814: creat d5/d7/d1b/d2e/dd8/f10c x:0 0 0 2024-03-29T04:45:20.273 INFO:tasks.workunit.client.1.smithi152.stdout:7/509: dread d7/d8/d12/d1d/f1e [0,4194304] 0 2024-03-29T04:45:20.276 INFO:tasks.workunit.client.1.smithi152.stdout:7/510: creat d7/d8/d12/d15/f97 x:0 0 0 2024-03-29T04:45:20.276 INFO:tasks.workunit.client.1.smithi152.stdout:1/815: read d5/d7/d8/f2d [1738680,97657] 0 2024-03-29T04:45:20.278 INFO:tasks.workunit.client.1.smithi152.stdout:4/453: dread d0/d7/d25/d17/d5e/d60/d74/f51 [0,4194304] 0 2024-03-29T04:45:20.278 INFO:tasks.workunit.client.1.smithi152.stdout:1/816: write d5/d7/ffa [611321,29667] 0 2024-03-29T04:45:20.282 INFO:tasks.workunit.client.1.smithi152.stdout:7/511: creat d7/d8/d12/d1d/f98 x:0 0 0 2024-03-29T04:45:20.282 INFO:tasks.workunit.client.1.smithi152.stdout:6/658: truncate d1/d3/d1a/f2f 238909 0 2024-03-29T04:45:20.282 INFO:tasks.workunit.client.1.smithi152.stdout:7/512: mkdir d7/d8/d93/d99 0 2024-03-29T04:45:20.283 INFO:tasks.workunit.client.1.smithi152.stdout:5/554: dwrite d6/d1b/f91 [0,4194304] 0 2024-03-29T04:45:20.285 INFO:tasks.workunit.client.1.smithi152.stdout:4/454: write d0/d7/d25/d17/f3b [27723,19611] 0 2024-03-29T04:45:20.286 INFO:tasks.workunit.client.1.smithi152.stdout:5/555: fdatasync d6/d1d/d3b/d55/f17 0 2024-03-29T04:45:20.286 INFO:tasks.workunit.client.1.smithi152.stdout:6/659: mkdir d1/d3/d9/d13/dc0/dc3/de1 0 2024-03-29T04:45:20.289 INFO:tasks.workunit.client.1.smithi152.stdout:4/455: chown d0/d7/d25/d17/d37/l69 0 1 2024-03-29T04:45:20.291 INFO:tasks.workunit.client.1.smithi152.stdout:5/556: creat d6/d16/d25/d2e/da0/d92/fbc x:0 0 0 2024-03-29T04:45:20.292 INFO:tasks.workunit.client.1.smithi152.stdout:4/456: write d0/d7/d25/d17/d5e/d60/d74/f51 [2096855,109406] 0 2024-03-29T04:45:20.296 INFO:tasks.workunit.client.1.smithi152.stdout:5/557: creat d6/d1d/daa/db7/d56/fbd x:0 0 0 2024-03-29T04:45:20.298 INFO:tasks.workunit.client.1.smithi152.stdout:6/660: dwrite d1/d3/d1a/d1b/d29/d6f/d95/fde [0,4194304] 0 2024-03-29T04:45:20.298 INFO:tasks.workunit.client.1.smithi152.stdout:4/457: mknod d0/d7/d40/d23/d55/d70/c92 0 2024-03-29T04:45:20.299 INFO:tasks.workunit.client.1.smithi152.stdout:5/558: creat d6/d1d/d3b/d55/fbe x:0 0 0 2024-03-29T04:45:20.300 INFO:tasks.workunit.client.1.smithi152.stdout:4/458: write d0/d7/d25/f21 [726960,128519] 0 2024-03-29T04:45:20.300 INFO:tasks.workunit.client.1.smithi152.stdout:5/559: readlink d6/d1b/l1a 0 2024-03-29T04:45:20.303 INFO:tasks.workunit.client.1.smithi152.stdout:6/661: rename d1/d3/d9/f41 to d1/d3/df/dd2/fe2 0 2024-03-29T04:45:20.304 INFO:tasks.workunit.client.1.smithi152.stdout:4/459: stat d0/d7/d25/d17/d5e/d60/d74/d57/l8f 0 2024-03-29T04:45:20.305 INFO:tasks.workunit.client.1.smithi152.stdout:5/560: creat d6/d1d/d30/fbf x:0 0 0 2024-03-29T04:45:20.307 INFO:tasks.workunit.client.1.smithi152.stdout:5/561: fdatasync d6/d1d/daa/db7/f61 0 2024-03-29T04:45:20.307 INFO:tasks.workunit.client.1.smithi152.stdout:6/662: truncate d1/f8 4697018 0 2024-03-29T04:45:20.311 INFO:tasks.workunit.client.1.smithi152.stdout:4/460: rename d0/d7/d25/d8e to d0/d7/d40/d23/d55/d6c/d91/d93 0 2024-03-29T04:45:20.313 INFO:tasks.workunit.client.1.smithi152.stdout:4/461: unlink d0/d27/c56 0 2024-03-29T04:45:20.315 INFO:tasks.workunit.client.1.smithi152.stdout:6/663: dread d1/d3/df/d84/fa3 [0,4194304] 0 2024-03-29T04:45:20.317 INFO:tasks.workunit.client.1.smithi152.stdout:4/462: creat d0/d7/d40/d2a/f94 x:0 0 0 2024-03-29T04:45:20.317 INFO:tasks.workunit.client.1.smithi152.stdout:7/513: dread d7/d8/d12/d1d/d41/d45/f55 [0,4194304] 0 2024-03-29T04:45:20.320 INFO:tasks.workunit.client.1.smithi152.stdout:5/562: dwrite d6/d7/f2d [0,4194304] 0 2024-03-29T04:45:20.321 INFO:tasks.workunit.client.1.smithi152.stdout:4/463: mknod d0/d7/d40/d23/d55/d6c/d91/d93/c95 0 2024-03-29T04:45:20.324 INFO:tasks.workunit.client.1.smithi152.stdout:7/514: rename d7/d8/d12/d15 to d7/d8/d12/d1d/d41/d83/d9a 0 2024-03-29T04:45:20.326 INFO:tasks.workunit.client.1.smithi152.stdout:6/664: link d1/d3/d1a/d1b/d29/d6f/d95/fde d1/d3/d9/fe3 0 2024-03-29T04:45:20.326 INFO:tasks.workunit.client.1.smithi152.stdout:7/515: chown d7/d8/d12/d1d/d41/d83/d9a/f3e 490871 1 2024-03-29T04:45:20.327 INFO:tasks.workunit.client.1.smithi152.stdout:4/464: mkdir d0/d7/d40/d23/d55/d70/d96 0 2024-03-29T04:45:20.327 INFO:tasks.workunit.client.1.smithi152.stdout:6/665: write d1/d3/d1a/d1b/d29/fc4 [864119,110391] 0 2024-03-29T04:45:20.328 INFO:tasks.workunit.client.1.smithi152.stdout:5/563: mkdir d6/d16/d25/d2e/da0/d92/dc0 0 2024-03-29T04:45:20.329 INFO:tasks.workunit.client.1.smithi152.stdout:6/666: dread - d1/d3/d1a/fb2 zero size 2024-03-29T04:45:20.329 INFO:tasks.workunit.client.1.smithi152.stdout:7/516: creat d7/d8/d12/d1d/d30/f9b x:0 0 0 2024-03-29T04:45:20.330 INFO:tasks.workunit.client.1.smithi152.stdout:6/667: readlink d1/d3/df/ld8 0 2024-03-29T04:45:20.330 INFO:tasks.workunit.client.1.smithi152.stdout:4/465: chown d0/d7/d25/d17/d5e/d60/d74/d57/f47 90349426 1 2024-03-29T04:45:20.332 INFO:tasks.workunit.client.1.smithi152.stdout:5/564: mkdir d6/d1d/daa/db7/d56/dc1 0 2024-03-29T04:45:20.333 INFO:tasks.workunit.client.1.smithi152.stdout:6/668: truncate d1/d3/d1a/d1b/d29/dce/fdd 951294 0 2024-03-29T04:45:20.333 INFO:tasks.workunit.client.1.smithi152.stdout:7/517: rmdir d7/d8/d12/d1d/d41/d83/d9a/d72 39 2024-03-29T04:45:20.337 INFO:tasks.workunit.client.1.smithi152.stdout:7/518: read d7/d8/d12/d35/f63 [871288,120417] 0 2024-03-29T04:45:20.344 INFO:tasks.workunit.client.1.smithi152.stdout:6/669: creat d1/d3/df/d8a/fe4 x:0 0 0 2024-03-29T04:45:20.345 INFO:tasks.workunit.client.1.smithi152.stdout:6/670: dread - d1/d3/d9/d13/da7/fb0 zero size 2024-03-29T04:45:20.346 INFO:tasks.workunit.client.1.smithi152.stdout:4/466: write d0/d7/d25/d17/d37/f4f [384290,69919] 0 2024-03-29T04:45:20.346 INFO:tasks.workunit.client.1.smithi152.stdout:7/519: creat d7/d8/f9c x:0 0 0 2024-03-29T04:45:20.346 INFO:tasks.workunit.client.1.smithi152.stdout:5/565: write d6/d16/f4e [831507,28334] 0 2024-03-29T04:45:20.347 INFO:tasks.workunit.client.1.smithi152.stdout:5/566: dread - d6/d1d/daa/db7/d56/fbd zero size 2024-03-29T04:45:20.348 INFO:tasks.workunit.client.1.smithi152.stdout:6/671: mkdir d1/d3/d9/d99/de5 0 2024-03-29T04:45:20.349 INFO:tasks.workunit.client.1.smithi152.stdout:4/467: creat d0/d27/d6e/f97 x:0 0 0 2024-03-29T04:45:20.349 INFO:tasks.workunit.client.1.smithi152.stdout:5/567: fdatasync d6/d16/d25/d2e/da0/f64 0 2024-03-29T04:45:20.408 INFO:tasks.workunit.client.1.smithi152.stdout:9/531: sync 2024-03-29T04:45:20.409 INFO:tasks.workunit.client.1.smithi152.stdout:5/568: chown d6/d7/fa4 20260 1 2024-03-29T04:45:20.410 INFO:tasks.workunit.client.1.smithi152.stdout:9/532: dread - d5/d12/d2c/f5e zero size 2024-03-29T04:45:20.410 INFO:tasks.workunit.client.1.smithi152.stdout:4/468: mknod d0/d7/d25/d17/d37/d77/c98 0 2024-03-29T04:45:20.412 INFO:tasks.workunit.client.1.smithi152.stdout:5/569: mknod d6/d16/cc2 0 2024-03-29T04:45:20.414 INFO:tasks.workunit.client.1.smithi152.stdout:4/469: mknod d0/d27/c99 0 2024-03-29T04:45:20.415 INFO:tasks.workunit.client.1.smithi152.stdout:6/672: dwrite d1/d3/df/f81 [0,4194304] 0 2024-03-29T04:45:20.417 INFO:tasks.workunit.client.1.smithi152.stdout:5/570: creat d6/d16/d25/d2e/da0/d92/fc3 x:0 0 0 2024-03-29T04:45:20.419 INFO:tasks.workunit.client.1.smithi152.stdout:9/533: dread d5/db/d1b/f95 [0,4194304] 0 2024-03-29T04:45:20.420 INFO:tasks.workunit.client.1.smithi152.stdout:6/673: chown d1/d3/d9/d13/d28/cb1 0 1 2024-03-29T04:45:20.421 INFO:tasks.workunit.client.1.smithi152.stdout:9/534: chown d5/d8/d31/d59/d6f/f71 7167385 1 2024-03-29T04:45:20.423 INFO:tasks.workunit.client.1.smithi152.stdout:5/571: read d6/d16/d25/d2e/f8c [2219359,109638] 0 2024-03-29T04:45:20.423 INFO:tasks.workunit.client.1.smithi152.stdout:4/470: dread d0/d7/d25/d17/d5e/d60/d74/f51 [0,4194304] 0 2024-03-29T04:45:20.424 INFO:tasks.workunit.client.1.smithi152.stdout:6/674: creat d1/d3/d9/d13/dc0/fe6 x:0 0 0 2024-03-29T04:45:20.424 INFO:tasks.workunit.client.1.smithi152.stdout:5/572: write d6/d16/f5b [979000,120083] 0 2024-03-29T04:45:20.426 INFO:tasks.workunit.client.1.smithi152.stdout:9/535: creat d5/d8/d31/d75/db1/fbe x:0 0 0 2024-03-29T04:45:20.426 INFO:tasks.workunit.client.1.smithi152.stdout:6/675: fsync d1/d3/d1a/d1b/d29/f71 0 2024-03-29T04:45:20.431 INFO:tasks.workunit.client.1.smithi152.stdout:6/676: stat d1/d3/d9/d13/f46 0 2024-03-29T04:45:20.431 INFO:tasks.workunit.client.1.smithi152.stdout:5/573: rmdir d6/d1d/d3b/d55 39 2024-03-29T04:45:20.431 INFO:tasks.workunit.client.1.smithi152.stdout:9/536: write d5/db/d92/fab [819828,36686] 0 2024-03-29T04:45:20.433 INFO:tasks.workunit.client.1.smithi152.stdout:5/574: readlink d6/d1b/l88 0 2024-03-29T04:45:20.433 INFO:tasks.workunit.client.1.smithi152.stdout:6/677: fdatasync d1/d3/d9/d13/f1f 0 2024-03-29T04:45:20.433 INFO:tasks.workunit.client.1.smithi152.stdout:9/537: read d5/f20 [142489,41841] 0 2024-03-29T04:45:20.443 INFO:tasks.workunit.client.1.smithi152.stdout:4/471: dwrite d0/d7/d25/f3f [0,4194304] 0 2024-03-29T04:45:20.446 INFO:tasks.workunit.client.1.smithi152.stdout:5/575: dwrite d6/d1b/f45 [0,4194304] 0 2024-03-29T04:45:20.449 INFO:tasks.workunit.client.1.smithi152.stdout:6/678: dwrite d1/d3/d9/d13/f1f [4194304,4194304] 0 2024-03-29T04:45:20.453 INFO:tasks.workunit.client.1.smithi152.stdout:5/576: mknod d6/d7/d75/cc4 0 2024-03-29T04:45:20.458 INFO:tasks.workunit.client.1.smithi152.stdout:5/577: mkdir d6/d1d/d30/dc5 0 2024-03-29T04:45:20.460 INFO:tasks.workunit.client.1.smithi152.stdout:9/538: dread d5/db/d1b/f37 [0,4194304] 0 2024-03-29T04:45:20.464 INFO:tasks.workunit.client.1.smithi152.stdout:4/472: dwrite d0/d7/d25/d17/d65/f79 [0,4194304] 0 2024-03-29T04:45:20.464 INFO:tasks.workunit.client.1.smithi152.stdout:9/539: write d5/d8/d31/d59/fb3 [967391,28572] 0 2024-03-29T04:45:20.466 INFO:tasks.workunit.client.1.smithi152.stdout:5/578: creat d6/d16/d25/d2e/da0/d92/fc6 x:0 0 0 2024-03-29T04:45:20.469 INFO:tasks.workunit.client.1.smithi152.stdout:4/473: creat d0/d7/d40/d62/f9a x:0 0 0 2024-03-29T04:45:20.469 INFO:tasks.workunit.client.1.smithi152.stdout:6/679: dwrite d1/d3/d9/d13/f26 [0,4194304] 0 2024-03-29T04:45:20.470 INFO:tasks.workunit.client.1.smithi152.stdout:9/540: mkdir d5/d8/d31/d59/d6f/dbf 0 2024-03-29T04:45:20.470 INFO:tasks.workunit.client.1.smithi152.stdout:9/541: stat d5/d12/l1d 0 2024-03-29T04:45:20.471 INFO:tasks.workunit.client.1.smithi152.stdout:5/579: truncate d6/d7/f8 1132770 0 2024-03-29T04:45:20.473 INFO:tasks.workunit.client.1.smithi152.stdout:4/474: write d0/d7/d40/d75/f7d [20741,24414] 0 2024-03-29T04:45:20.473 INFO:tasks.workunit.client.1.smithi152.stdout:6/680: mknod d1/d3/d9/d13/dc0/dc3/ce7 0 2024-03-29T04:45:20.475 INFO:tasks.workunit.client.1.smithi152.stdout:4/475: chown d0/d7/d25/d17/f54 9138299 1 2024-03-29T04:45:20.476 INFO:tasks.workunit.client.1.smithi152.stdout:9/542: mknod d5/db/cc0 0 2024-03-29T04:45:20.477 INFO:tasks.workunit.client.1.smithi152.stdout:5/580: dread d6/d1d/d30/f46 [0,4194304] 0 2024-03-29T04:45:20.477 INFO:tasks.workunit.client.1.smithi152.stdout:6/681: creat d1/d3/d9/d13/da7/fe8 x:0 0 0 2024-03-29T04:45:20.479 INFO:tasks.workunit.client.1.smithi152.stdout:4/476: unlink d0/d7/d40/d23/f53 0 2024-03-29T04:45:20.481 INFO:tasks.workunit.client.1.smithi152.stdout:4/477: write d0/d27/f59 [234251,68633] 0 2024-03-29T04:45:20.486 INFO:tasks.workunit.client.1.smithi152.stdout:5/581: truncate d6/d1b/f5a 659238 0 2024-03-29T04:45:20.487 INFO:tasks.workunit.client.1.smithi152.stdout:4/478: creat d0/d7/d25/d17/d5e/d60/f9b x:0 0 0 2024-03-29T04:45:20.488 INFO:tasks.workunit.client.1.smithi152.stdout:5/582: mkdir d6/d16/d25/d2e/dc7 0 2024-03-29T04:45:20.489 INFO:tasks.workunit.client.1.smithi152.stdout:4/479: mknod d0/d7/d40/d75/c9c 0 2024-03-29T04:45:20.489 INFO:tasks.workunit.client.1.smithi152.stdout:6/682: dwrite d1/d3/d1a/d1b/d29/fca [0,4194304] 0 2024-03-29T04:45:20.492 INFO:tasks.workunit.client.1.smithi152.stdout:4/480: write d0/d7/d25/d17/f4c [4174177,118964] 0 2024-03-29T04:45:20.502 INFO:tasks.workunit.client.1.smithi152.stdout:6/683: dwrite d1/d3/d9/d13/d28/f34 [0,4194304] 0 2024-03-29T04:45:20.506 INFO:tasks.workunit.client.1.smithi152.stdout:4/481: dwrite d0/d7/d40/ff [0,4194304] 0 2024-03-29T04:45:20.512 INFO:tasks.workunit.client.1.smithi152.stdout:9/543: dwrite d5/db/f4e [0,4194304] 0 2024-03-29T04:45:20.513 INFO:tasks.workunit.client.1.smithi152.stdout:4/482: rename d0/d7/d25/d17/d5e/d60 to d0/d7/d25/d17/d5e/d60/d9d 22 2024-03-29T04:45:20.515 INFO:tasks.workunit.client.1.smithi152.stdout:4/483: write d0/d7/d40/f8a [853573,113000] 0 2024-03-29T04:45:20.519 INFO:tasks.workunit.client.1.smithi152.stdout:9/544: unlink d5/d8/c9b 0 2024-03-29T04:45:20.521 INFO:tasks.workunit.client.1.smithi152.stdout:9/545: symlink d5/db/d92/lc1 0 2024-03-29T04:45:20.523 INFO:tasks.workunit.client.1.smithi152.stdout:9/546: chown d5/d12/d2c/c41 87 1 2024-03-29T04:45:20.525 INFO:tasks.workunit.client.1.smithi152.stdout:9/547: fsync d5/d8/f53 0 2024-03-29T04:45:20.525 INFO:tasks.workunit.client.1.smithi152.stdout:5/583: dread d6/d16/d25/d2e/da0/f71 [0,4194304] 0 2024-03-29T04:45:20.528 INFO:tasks.workunit.client.1.smithi152.stdout:9/548: rmdir d5/db/d1b 39 2024-03-29T04:45:20.530 INFO:tasks.workunit.client.1.smithi152.stdout:9/549: mknod d5/d8/d31/d59/cc2 0 2024-03-29T04:45:20.532 INFO:tasks.workunit.client.1.smithi152.stdout:5/584: dwrite d6/d16/f4e [0,4194304] 0 2024-03-29T04:45:20.534 INFO:tasks.workunit.client.1.smithi152.stdout:9/550: mkdir d5/d8/d31/d59/dc3 0 2024-03-29T04:45:20.536 INFO:tasks.workunit.client.1.smithi152.stdout:9/551: mkdir d5/d12/d2c/dc4 0 2024-03-29T04:45:20.539 INFO:tasks.workunit.client.1.smithi152.stdout:9/552: fsync d5/d12/d2c/d6e/d73/d8e/fa3 0 2024-03-29T04:45:20.540 INFO:tasks.workunit.client.1.smithi152.stdout:6/684: dread d1/d3/df/f9e [0,4194304] 0 2024-03-29T04:45:20.541 INFO:tasks.workunit.client.1.smithi152.stdout:9/553: write d5/db/d92/f9a [834024,52931] 0 2024-03-29T04:45:20.542 INFO:tasks.workunit.client.1.smithi152.stdout:9/554: chown d5/d12/db7 1108 1 2024-03-29T04:45:20.545 INFO:tasks.workunit.client.1.smithi152.stdout:6/685: dread d1/d3/d9/d13/d28/d45/d8c/f85 [0,4194304] 0 2024-03-29T04:45:20.545 INFO:tasks.workunit.client.1.smithi152.stdout:9/555: rename d5/d8/d31/d59/cc2 to d5/d12/d2c/cc5 0 2024-03-29T04:45:20.547 INFO:tasks.workunit.client.1.smithi152.stdout:6/686: mkdir d1/d3/d9/d13/d3e/de9 0 2024-03-29T04:45:20.548 INFO:tasks.workunit.client.1.smithi152.stdout:9/556: creat d5/d12/d2c/d6e/d91/fc6 x:0 0 0 2024-03-29T04:45:20.548 INFO:tasks.workunit.client.1.smithi152.stdout:9/557: stat d5/db/f4e 0 2024-03-29T04:45:20.549 INFO:tasks.workunit.client.1.smithi152.stdout:9/558: chown d5/d8/d31/c8c 119 1 2024-03-29T04:45:20.554 INFO:tasks.workunit.client.1.smithi152.stdout:9/559: rename d5/d8/d31/d75/db1/fbe to d5/d12/d2c/d6e/fc7 0 2024-03-29T04:45:20.554 INFO:tasks.workunit.client.1.smithi152.stdout:6/687: dwrite d1/d3/d9/d13/f31 [0,4194304] 0 2024-03-29T04:45:20.555 INFO:tasks.workunit.client.1.smithi152.stdout:5/585: write d6/d16/d25/f43 [271764,121502] 0 2024-03-29T04:45:20.556 INFO:tasks.workunit.client.1.smithi152.stdout:9/560: dread - d5/db/f5a zero size 2024-03-29T04:45:20.562 INFO:tasks.workunit.client.1.smithi152.stdout:5/586: truncate d6/d7/f8 117454 0 2024-03-29T04:45:20.562 INFO:tasks.workunit.client.1.smithi152.stdout:6/688: dread d1/d3/df/d84/fa3 [0,4194304] 0 2024-03-29T04:45:20.564 INFO:tasks.workunit.client.1.smithi152.stdout:5/587: unlink d6/d1d/f40 0 2024-03-29T04:45:20.564 INFO:tasks.workunit.client.1.smithi152.stdout:6/689: mkdir d1/d3/d9/d13/dc0/dea 0 2024-03-29T04:45:20.566 INFO:tasks.workunit.client.1.smithi152.stdout:9/561: dwrite d5/d12/d2c/d6e/fa0 [0,4194304] 0 2024-03-29T04:45:20.566 INFO:tasks.workunit.client.1.smithi152.stdout:6/690: fsync d1/d3/df/d8a/fb6 0 2024-03-29T04:45:20.567 INFO:tasks.workunit.client.1.smithi152.stdout:5/588: symlink d6/d1d/da5/lc8 0 2024-03-29T04:45:20.568 INFO:tasks.workunit.client.1.smithi152.stdout:6/691: write d1/d3/df/d84/f9d [758352,72582] 0 2024-03-29T04:45:20.570 INFO:tasks.workunit.client.1.smithi152.stdout:5/589: write d6/d1d/d30/fbf [821603,93221] 0 2024-03-29T04:45:20.573 INFO:tasks.workunit.client.1.smithi152.stdout:6/692: creat d1/d3/d9/d13/d28/d45/d8c/feb x:0 0 0 2024-03-29T04:45:20.573 INFO:tasks.workunit.client.1.smithi152.stdout:6/693: chown d1/d3/d9/l40 1 1 2024-03-29T04:45:20.574 INFO:tasks.workunit.client.1.smithi152.stdout:9/562: truncate d5/db/d1b/f51 3302385 0 2024-03-29T04:45:20.575 INFO:tasks.workunit.client.1.smithi152.stdout:5/590: mkdir d6/d1d/daa/db7/d56/dc9 0 2024-03-29T04:45:20.577 INFO:tasks.workunit.client.1.smithi152.stdout:5/591: chown d6/d1d/daa/db7/d56/d63 0 1 2024-03-29T04:45:20.578 INFO:tasks.workunit.client.1.smithi152.stdout:5/592: fdatasync d6/d1d/daa/db7/f61 0 2024-03-29T04:45:20.581 INFO:tasks.workunit.client.1.smithi152.stdout:6/694: dwrite d1/d3/d1a/d1b/d29/fcc [0,4194304] 0 2024-03-29T04:45:20.581 INFO:tasks.workunit.client.1.smithi152.stdout:9/563: dwrite d5/db/d1b/f4f [0,4194304] 0 2024-03-29T04:45:20.583 INFO:tasks.workunit.client.1.smithi152.stdout:5/593: creat d6/d1d/daa/fca x:0 0 0 2024-03-29T04:45:20.588 INFO:tasks.workunit.client.1.smithi152.stdout:9/564: creat d5/d12/d2c/d6e/d73/d8d/fc8 x:0 0 0 2024-03-29T04:45:20.588 INFO:tasks.workunit.client.1.smithi152.stdout:5/594: getdents d6/d1d/daa/db7/d56/dc1 0 2024-03-29T04:45:20.636 INFO:tasks.workunit.client.1.smithi152.stdout:9/565: symlink d5/d8/d31/d59/d6f/lc9 0 2024-03-29T04:45:20.638 INFO:tasks.workunit.client.1.smithi152.stdout:5/595: chown d6/d1d/d3b/d55/l51 357349 1 2024-03-29T04:45:20.639 INFO:tasks.workunit.client.1.smithi152.stdout:5/596: dread - d6/d1d/daa/db7/d56/d63/fb1 zero size 2024-03-29T04:45:20.640 INFO:tasks.workunit.client.1.smithi152.stdout:6/695: write d1/d3/d9/d13/d28/d45/d8c/f85 [1413749,129413] 0 2024-03-29T04:45:20.641 INFO:tasks.workunit.client.1.smithi152.stdout:6/696: stat d1/d3/d9/d13/d28/d45/f80 0 2024-03-29T04:45:20.658 INFO:tasks.workunit.client.1.smithi152.stdout:6/697: dread d1/d3/d1a/d1b/d29/d6f/f3c [0,4194304] 0 2024-03-29T04:45:20.665 INFO:tasks.workunit.client.1.smithi152.stdout:6/698: dread - d1/d3/d9/d13/d28/d45/f80 zero size 2024-03-29T04:45:20.666 INFO:tasks.workunit.client.1.smithi152.stdout:6/699: truncate d1/d3/d1a/fb2 421973 0 2024-03-29T04:45:20.671 INFO:tasks.workunit.client.1.smithi152.stdout:6/700: mkdir d1/d3/d1a/d1b/d29/dce/dec 0 2024-03-29T04:45:20.689 INFO:tasks.workunit.client.1.smithi152.stdout:6/701: rmdir d1/d3/df/d8a/d98/dc7 0 2024-03-29T04:45:20.714 INFO:tasks.workunit.client.1.smithi152.stdout:9/566: rename d5/d12/d2c/d6e/d73/d8d to d5/db/d1b/d68/dca 0 2024-03-29T04:45:20.720 INFO:tasks.workunit.client.1.smithi152.stdout:9/567: getdents d5/db/d92 0 2024-03-29T04:45:20.724 INFO:tasks.workunit.client.1.smithi152.stdout:5/597: dwrite d6/d1d/d30/f46 [0,4194304] 0 2024-03-29T04:45:20.724 INFO:tasks.workunit.client.1.smithi152.stdout:9/568: unlink d5/d12/d2c/d6e/d91/f94 0 2024-03-29T04:45:20.735 INFO:tasks.workunit.client.1.smithi152.stdout:5/598: dread d6/d1d/d3b/f3f [0,4194304] 0 2024-03-29T04:45:20.738 INFO:tasks.workunit.client.1.smithi152.stdout:5/599: creat d6/d1b/fcb x:0 0 0 2024-03-29T04:45:20.740 INFO:tasks.workunit.client.1.smithi152.stdout:6/702: unlink d1/fe 0 2024-03-29T04:45:20.749 INFO:tasks.workunit.client.1.smithi152.stdout:9/569: truncate d5/db/d1b/f95 3119263 0 2024-03-29T04:45:20.750 INFO:tasks.workunit.client.1.smithi152.stdout:9/570: stat d5/db/l3d 0 2024-03-29T04:45:20.751 INFO:tasks.workunit.client.1.smithi152.stdout:6/703: write d1/d3/d1a/f2f [837268,71896] 0 2024-03-29T04:45:20.758 INFO:tasks.workunit.client.1.smithi152.stdout:5/600: dwrite d6/d16/d25/f9f [0,4194304] 0 2024-03-29T04:45:20.759 INFO:tasks.workunit.client.1.smithi152.stdout:9/571: dwrite d5/db/f58 [0,4194304] 0 2024-03-29T04:45:20.760 INFO:tasks.workunit.client.1.smithi152.stdout:6/704: dread d1/d3/df/f9e [0,4194304] 0 2024-03-29T04:45:20.762 INFO:tasks.workunit.client.1.smithi152.stdout:9/572: rmdir d5/d8/d31/d75 39 2024-03-29T04:45:20.763 INFO:tasks.workunit.client.1.smithi152.stdout:9/573: readlink d5/d12/d2c/d6e/d73/d8e/la2 0 2024-03-29T04:45:20.764 INFO:tasks.workunit.client.1.smithi152.stdout:5/601: mknod d6/d1d/daa/db7/d56/ccc 0 2024-03-29T04:45:20.767 INFO:tasks.workunit.client.1.smithi152.stdout:6/705: symlink d1/d3/d1a/d1b/d87/d8f/led 0 2024-03-29T04:45:20.769 INFO:tasks.workunit.client.1.smithi152.stdout:5/602: creat d6/d16/d25/d2e/dc7/fcd x:0 0 0 2024-03-29T04:45:20.770 INFO:tasks.workunit.client.1.smithi152.stdout:6/706: mkdir d1/d3/d1a/d1b/d29/d6f/dee 0 2024-03-29T04:45:20.774 INFO:tasks.workunit.client.1.smithi152.stdout:6/707: rename d1/d3/d9/d13/dc0/dc3/de1 to d1/d3/d9/d13/dc0/dea/def 0 2024-03-29T04:45:20.774 INFO:tasks.workunit.client.1.smithi152.stdout:5/603: link d6/d1d/d30/d6a/l96 d6/d16/d25/d2e/d9d/lce 0 2024-03-29T04:45:20.776 INFO:tasks.workunit.client.1.smithi152.stdout:5/604: creat d6/d1b/fcf x:0 0 0 2024-03-29T04:45:20.777 INFO:tasks.workunit.client.1.smithi152.stdout:5/605: read d6/d16/f4e [3192561,117165] 0 2024-03-29T04:45:20.780 INFO:tasks.workunit.client.1.smithi152.stdout:5/606: unlink d6/d16/l7b 0 2024-03-29T04:45:20.788 INFO:tasks.workunit.client.1.smithi152.stdout:6/708: write d1/d3/d9/d13/d28/d45/f80 [1020776,27849] 0 2024-03-29T04:45:20.791 INFO:tasks.workunit.client.1.smithi152.stdout:9/574: dread d5/d12/d2c/f63 [0,4194304] 0 2024-03-29T04:45:20.792 INFO:tasks.workunit.client.1.smithi152.stdout:6/709: write d1/d3/df/f66 [241461,92350] 0 2024-03-29T04:45:20.793 INFO:tasks.workunit.client.1.smithi152.stdout:9/575: symlink d5/lcb 0 2024-03-29T04:45:20.795 INFO:tasks.workunit.client.1.smithi152.stdout:9/576: read - d5/db/f5a zero size 2024-03-29T04:45:20.796 INFO:tasks.workunit.client.1.smithi152.stdout:5/607: dwrite d6/d1d/d3b/d55/f35 [0,4194304] 0 2024-03-29T04:45:20.798 INFO:tasks.workunit.client.1.smithi152.stdout:5/608: read - d6/d1d/daa/db7/d56/fbd zero size 2024-03-29T04:45:20.799 INFO:tasks.workunit.client.1.smithi152.stdout:5/609: stat d6/d1d/daa/db7/d56/d63 0 2024-03-29T04:45:20.799 INFO:tasks.workunit.client.1.smithi152.stdout:9/577: readlink d5/d8/d31/d75/db1/lbd 0 2024-03-29T04:45:20.801 INFO:tasks.workunit.client.1.smithi152.stdout:9/578: stat d5/d12/d2c/d6e/d73/d7a/fac 0 2024-03-29T04:45:20.802 INFO:tasks.workunit.client.1.smithi152.stdout:5/610: truncate d6/d16/f73 1356293 0 2024-03-29T04:45:20.803 INFO:tasks.workunit.client.1.smithi152.stdout:6/710: write d1/d3/d1a/d1b/d29/d6f/d95/fde [4494078,9668] 0 2024-03-29T04:45:20.804 INFO:tasks.workunit.client.1.smithi152.stdout:9/579: dread - d5/d12/d2c/d6e/d73/d8e/f74 zero size 2024-03-29T04:45:20.805 INFO:tasks.workunit.client.1.smithi152.stdout:6/711: chown d1/l32 867994 1 2024-03-29T04:45:20.806 INFO:tasks.workunit.client.1.smithi152.stdout:3/570: sync 2024-03-29T04:45:20.809 INFO:tasks.workunit.client.1.smithi152.stdout:6/712: creat d1/d3/d9/d13/d3e/d93/ff0 x:0 0 0 2024-03-29T04:45:20.810 INFO:tasks.workunit.client.1.smithi152.stdout:3/571: symlink d7/d19/d1b/d59/la8 0 2024-03-29T04:45:20.811 INFO:tasks.workunit.client.1.smithi152.stdout:6/713: write d1/d3/d9/d13/d28/f34 [1413108,61728] 0 2024-03-29T04:45:20.814 INFO:tasks.workunit.client.1.smithi152.stdout:3/572: rmdir d7/d19/d1b/d1e/d22 39 2024-03-29T04:45:20.819 INFO:tasks.workunit.client.1.smithi152.stdout:5/611: truncate d6/d1d/d3b/d55/f35 2981216 0 2024-03-29T04:45:20.820 INFO:tasks.workunit.client.1.smithi152.stdout:6/714: dwrite d1/d3/df/d84/f57 [0,4194304] 0 2024-03-29T04:45:20.824 INFO:tasks.workunit.client.1.smithi152.stdout:6/715: dread - d1/d3/df/d8a/fe4 zero size 2024-03-29T04:45:20.828 INFO:tasks.workunit.client.1.smithi152.stdout:5/612: symlink d6/d16/d25/d2e/dc7/ld0 0 2024-03-29T04:45:20.828 INFO:tasks.workunit.client.1.smithi152.stdout:9/580: dwrite d5/db/d1b/f40 [0,4194304] 0 2024-03-29T04:45:20.829 INFO:tasks.workunit.client.1.smithi152.stdout:5/613: chown d6/d1d/d3b/d55 69517066 1 2024-03-29T04:45:20.832 INFO:tasks.workunit.client.1.smithi152.stdout:3/573: rename d7/d19/d1b/d1e/da1/d41/d61/d75/f7c to d7/d19/d1b/d1e/d4e/d8f/fa9 0 2024-03-29T04:45:20.833 INFO:tasks.workunit.client.1.smithi152.stdout:5/614: creat d6/d1b/fd1 x:0 0 0 2024-03-29T04:45:20.838 INFO:tasks.workunit.client.1.smithi152.stdout:9/581: mknod d5/d8/d31/ccc 0 2024-03-29T04:45:20.840 INFO:tasks.workunit.client.1.smithi152.stdout:6/716: dwrite d1/d3/d1a/d1b/d29/fca [4194304,4194304] 0 2024-03-29T04:45:20.841 INFO:tasks.workunit.client.1.smithi152.stdout:5/615: unlink d6/d7/f8 0 2024-03-29T04:45:20.843 INFO:tasks.workunit.client.1.smithi152.stdout:5/616: chown d6/d1d/daa/db7/d56/f94 667691 1 2024-03-29T04:45:20.843 INFO:tasks.workunit.client.1.smithi152.stdout:3/574: link d7/d19/d1b/d1e/d4e/f76 d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/faa 0 2024-03-29T04:45:20.847 INFO:tasks.workunit.client.1.smithi152.stdout:5/617: dread d6/d1d/d3b/d55/f35 [0,4194304] 0 2024-03-29T04:45:20.848 INFO:tasks.workunit.client.1.smithi152.stdout:9/582: rename d5/d12/d2c to d5/d8/d31/d75/db1/dcd 0 2024-03-29T04:45:20.849 INFO:tasks.workunit.client.1.smithi152.stdout:6/717: dread d1/d3/d1a/f61 [0,4194304] 0 2024-03-29T04:45:20.851 INFO:tasks.workunit.client.1.smithi152.stdout:9/583: chown d5/d8/d31/d75/db1/dcd/d6e/db4 43751226 1 2024-03-29T04:45:20.856 INFO:tasks.workunit.client.1.smithi152.stdout:6/718: chown d1/d3/d9/d13/d28/l76 335504757 1 2024-03-29T04:45:20.859 INFO:tasks.workunit.client.1.smithi152.stdout:5/618: dwrite d6/d16/d25/f3a [0,4194304] 0 2024-03-29T04:45:20.866 INFO:tasks.workunit.client.1.smithi152.stdout:6/719: write d1/d3/d9/d13/d28/f34 [218338,16993] 0 2024-03-29T04:45:20.867 INFO:tasks.workunit.client.1.smithi152.stdout:9/584: dwrite d5/d8/d31/d59/d6f/f71 [4194304,4194304] 0 2024-03-29T04:45:20.870 INFO:tasks.workunit.client.1.smithi152.stdout:6/720: truncate d1/f9b 805115 0 2024-03-29T04:45:20.876 INFO:tasks.workunit.client.1.smithi152.stdout:9/585: creat d5/d8/d31/d59/d6f/fce x:0 0 0 2024-03-29T04:45:20.878 INFO:tasks.workunit.client.1.smithi152.stdout:5/619: dwrite d6/d16/d25/d2e/dc7/fcd [0,4194304] 0 2024-03-29T04:45:20.883 INFO:tasks.workunit.client.1.smithi152.stdout:9/586: rename d5/d12/c2e to d5/d8/d31/d75/ccf 0 2024-03-29T04:45:20.886 INFO:tasks.workunit.client.1.smithi152.stdout:4/484: sync 2024-03-29T04:45:20.886 INFO:tasks.workunit.client.1.smithi152.stdout:7/520: sync 2024-03-29T04:45:20.886 INFO:tasks.workunit.client.1.smithi152.stdout:2/450: sync 2024-03-29T04:45:20.887 INFO:tasks.workunit.client.1.smithi152.stdout:8/492: sync 2024-03-29T04:45:20.887 INFO:tasks.workunit.client.1.smithi152.stdout:1/817: sync 2024-03-29T04:45:20.887 INFO:tasks.workunit.client.1.smithi152.stdout:0/557: sync 2024-03-29T04:45:20.887 INFO:tasks.workunit.client.1.smithi152.stdout:3/575: dread d7/d19/d1b/d1e/f82 [0,4194304] 0 2024-03-29T04:45:20.888 INFO:tasks.workunit.client.1.smithi152.stdout:5/620: getdents d6/d16/d25/d2e/da0/d92 0 2024-03-29T04:45:20.894 INFO:tasks.workunit.client.1.smithi152.stdout:9/587: rename d5/d8/d31/f72 to d5/d8/d31/d59/dc3/fd0 0 2024-03-29T04:45:20.894 INFO:tasks.workunit.client.1.smithi152.stdout:0/558: chown d2/f20 1236 1 2024-03-29T04:45:20.894 INFO:tasks.workunit.client.1.smithi152.stdout:4/485: readlink d0/d7/d25/d17/l38 0 2024-03-29T04:45:20.894 INFO:tasks.workunit.client.1.smithi152.stdout:1/818: fsync d5/d7/d1b/d2e/f5f 0 2024-03-29T04:45:20.895 INFO:tasks.workunit.client.1.smithi152.stdout:9/588: fdatasync d5/db/d1b/f4d 0 2024-03-29T04:45:20.898 INFO:tasks.workunit.client.1.smithi152.stdout:5/621: truncate d6/d7/fa4 4605623 0 2024-03-29T04:45:20.899 INFO:tasks.workunit.client.1.smithi152.stdout:8/493: readlink d1/d2/d1d/d36/l4a 0 2024-03-29T04:45:20.902 INFO:tasks.workunit.client.1.smithi152.stdout:3/576: creat d7/d19/d3a/fab x:0 0 0 2024-03-29T04:45:20.902 INFO:tasks.workunit.client.1.smithi152.stdout:0/559: chown d2/da/d17/d57/dc5/fcd 136736 1 2024-03-29T04:45:20.903 INFO:tasks.workunit.client.1.smithi152.stdout:8/494: read - d1/d21/d40/d47/f79 zero size 2024-03-29T04:45:20.905 INFO:tasks.workunit.client.1.smithi152.stdout:2/451: getdents d0/d3/d30/d15 0 2024-03-29T04:45:20.908 INFO:tasks.workunit.client.1.smithi152.stdout:3/577: chown d7/d19/d1b/d1e/d22/f3b 188 1 2024-03-29T04:45:20.912 INFO:tasks.workunit.client.1.smithi152.stdout:4/486: link d0/d27/d6e/d78/f7b d0/d7/d40/d23/d55/d6c/f9e 0 2024-03-29T04:45:20.913 INFO:tasks.workunit.client.1.smithi152.stdout:5/622: creat d6/d1d/daa/db7/d56/d63/db9/fd2 x:0 0 0 2024-03-29T04:45:20.920 INFO:tasks.workunit.client.1.smithi152.stdout:6/721: dread f0 [0,4194304] 0 2024-03-29T04:45:20.922 INFO:tasks.workunit.client.1.smithi152.stdout:3/578: mknod d7/cac 0 2024-03-29T04:45:20.926 INFO:tasks.workunit.client.1.smithi152.stdout:4/487: creat d0/d7/d40/d23/d55/d70/d96/f9f x:0 0 0 2024-03-29T04:45:20.927 INFO:tasks.workunit.client.1.smithi152.stdout:0/560: rename d2/da/d17/d57/f71 to d2/da/d17/fce 0 2024-03-29T04:45:20.933 INFO:tasks.workunit.client.1.smithi152.stdout:2/452: dwrite d0/d3/f42 [4194304,4194304] 0 2024-03-29T04:45:20.933 INFO:tasks.workunit.client.1.smithi152.stdout:5/623: getdents d6/d1d/daa/db7/d56/d63/db9 0 2024-03-29T04:45:20.934 INFO:tasks.workunit.client.1.smithi152.stdout:4/488: stat d0/d7/d25/d17/d5e/d60/d74/f81 0 2024-03-29T04:45:20.943 INFO:tasks.workunit.client.1.smithi152.stdout:6/722: readlink d1/d3/d9/d13/d28/l78 0 2024-03-29T04:45:20.944 INFO:tasks.workunit.client.1.smithi152.stdout:0/561: mkdir d2/d29/dcf 0 2024-03-29T04:45:20.945 INFO:tasks.workunit.client.1.smithi152.stdout:1/819: dread d5/d7/d8/d13/f45 [0,4194304] 0 2024-03-29T04:45:20.945 INFO:tasks.workunit.client.1.smithi152.stdout:8/495: dwrite d1/d2/d1d/f3d [0,4194304] 0 2024-03-29T04:45:20.945 INFO:tasks.workunit.client.1.smithi152.stdout:3/579: creat d7/d19/d1b/d1e/d4e/fad x:0 0 0 2024-03-29T04:45:20.946 INFO:tasks.workunit.client.1.smithi152.stdout:7/521: dread d7/f92 [0,4194304] 0 2024-03-29T04:45:20.948 INFO:tasks.workunit.client.1.smithi152.stdout:1/820: stat d5/c4b 0 2024-03-29T04:45:20.953 INFO:tasks.workunit.client.1.smithi152.stdout:2/453: unlink d0/d3/d30/d15/d22/d4c/l86 0 2024-03-29T04:45:20.953 INFO:tasks.workunit.client.1.smithi152.stdout:4/489: dwrite d0/d7/d40/ff [0,4194304] 0 2024-03-29T04:45:20.954 INFO:tasks.workunit.client.1.smithi152.stdout:4/490: fsync d0/d7/d25/d17/d5e/d60/f9b 0 2024-03-29T04:45:20.955 INFO:tasks.workunit.client.1.smithi152.stdout:6/723: fdatasync d1/d3/df/d84/f5e 0 2024-03-29T04:45:20.956 INFO:tasks.workunit.client.1.smithi152.stdout:8/496: rmdir d1/d2/d1d/d36 39 2024-03-29T04:45:20.959 INFO:tasks.workunit.client.1.smithi152.stdout:4/491: write d0/d7/d40/f80 [3040564,16715] 0 2024-03-29T04:45:20.968 INFO:tasks.workunit.client.1.smithi152.stdout:3/580: symlink d7/d19/d1b/d1e/da1/d41/d61/d85/lae 0 2024-03-29T04:45:20.970 INFO:tasks.workunit.client.1.smithi152.stdout:1/821: dread d5/d7/d1b/f93 [0,4194304] 0 2024-03-29T04:45:20.970 INFO:tasks.workunit.client.1.smithi152.stdout:4/492: dwrite d0/d7/f8d [0,4194304] 0 2024-03-29T04:45:20.971 INFO:tasks.workunit.client.1.smithi152.stdout:0/562: dread d2/da/d3a/d61/f7f [0,4194304] 0 2024-03-29T04:45:20.976 INFO:tasks.workunit.client.1.smithi152.stdout:8/497: stat d1/d2/c1e 0 2024-03-29T04:45:20.978 INFO:tasks.workunit.client.1.smithi152.stdout:7/522: write d7/d8/db/f4b [83282,112726] 0 2024-03-29T04:45:20.980 INFO:tasks.workunit.client.1.smithi152.stdout:3/581: dwrite d7/d19/d1b/d1e/d4e/f70 [0,4194304] 0 2024-03-29T04:45:20.981 INFO:tasks.workunit.client.1.smithi152.stdout:1/822: rmdir d5/d7/d6a 39 2024-03-29T04:45:20.982 INFO:tasks.workunit.client.1.smithi152.stdout:0/563: creat d2/da/d3a/d61/fd0 x:0 0 0 2024-03-29T04:45:20.983 INFO:tasks.workunit.client.1.smithi152.stdout:5/624: getdents d6/d16/d25/d2e/d9d 0 2024-03-29T04:45:20.985 INFO:tasks.workunit.client.1.smithi152.stdout:7/523: creat d7/f9d x:0 0 0 2024-03-29T04:45:20.986 INFO:tasks.workunit.client.1.smithi152.stdout:5/625: chown d6/d1d/d3b/l62 726 1 2024-03-29T04:45:20.986 INFO:tasks.workunit.client.1.smithi152.stdout:7/524: fdatasync d7/f59 0 2024-03-29T04:45:20.987 INFO:tasks.workunit.client.1.smithi152.stdout:2/454: creat d0/d3/d5f/f92 x:0 0 0 2024-03-29T04:45:20.987 INFO:tasks.workunit.client.1.smithi152.stdout:6/724: creat d1/d3/d1a/d1b/d29/ff1 x:0 0 0 2024-03-29T04:45:20.988 INFO:tasks.workunit.client.1.smithi152.stdout:8/498: symlink d1/d2/d1d/d36/lae 0 2024-03-29T04:45:20.989 INFO:tasks.workunit.client.1.smithi152.stdout:1/823: getdents d5/d7/d4d/dd4 0 2024-03-29T04:45:20.989 INFO:tasks.workunit.client.1.smithi152.stdout:6/725: readlink d1/d3/d9/l47 0 2024-03-29T04:45:20.989 INFO:tasks.workunit.client.1.smithi152.stdout:5/626: creat d6/d7/fd3 x:0 0 0 2024-03-29T04:45:20.990 INFO:tasks.workunit.client.1.smithi152.stdout:2/455: read - d0/d3/d30/d15/f26 zero size 2024-03-29T04:45:20.991 INFO:tasks.workunit.client.1.smithi152.stdout:0/564: creat d2/da/d17/db5/fd1 x:0 0 0 2024-03-29T04:45:20.992 INFO:tasks.workunit.client.1.smithi152.stdout:8/499: stat d1/d2/d3c/d76/ca6 0 2024-03-29T04:45:20.992 INFO:tasks.workunit.client.1.smithi152.stdout:4/493: link d0/f13 d0/d7/d25/d17/d5e/d60/d74/fa0 0 2024-03-29T04:45:20.993 INFO:tasks.workunit.client.1.smithi152.stdout:2/456: write d0/d3/d30/f87 [695482,6933] 0 2024-03-29T04:45:20.993 INFO:tasks.workunit.client.1.smithi152.stdout:7/525: symlink d7/d8/db/l9e 0 2024-03-29T04:45:20.997 INFO:tasks.workunit.client.1.smithi152.stdout:5/627: creat d6/d1d/daa/fd4 x:0 0 0 2024-03-29T04:45:21.001 INFO:tasks.workunit.client.1.smithi152.stdout:6/726: truncate d1/d3/d9/d13/d3e/d93/fd1 516483 0 2024-03-29T04:45:21.003 INFO:tasks.workunit.client.1.smithi152.stdout:1/824: getdents d5/d7/d4d 0 2024-03-29T04:45:21.004 INFO:tasks.workunit.client.1.smithi152.stdout:5/628: rmdir d6/d1d/da5 39 2024-03-29T04:45:21.005 INFO:tasks.workunit.client.1.smithi152.stdout:2/457: symlink d0/d3/d64/l93 0 2024-03-29T04:45:21.006 INFO:tasks.workunit.client.1.smithi152.stdout:0/565: write d2/da/d3a/f91 [929641,50369] 0 2024-03-29T04:45:21.007 INFO:tasks.workunit.client.1.smithi152.stdout:6/727: read - d1/d3/d9/d13/da7/fe8 zero size 2024-03-29T04:45:21.011 INFO:tasks.workunit.client.1.smithi152.stdout:4/494: dwrite d0/d7/d40/f80 [0,4194304] 0 2024-03-29T04:45:21.013 INFO:tasks.workunit.client.1.smithi152.stdout:8/500: dwrite d1/d2/d1d/d36/d4b/f68 [0,4194304] 0 2024-03-29T04:45:21.015 INFO:tasks.workunit.client.1.smithi152.stdout:3/582: dread d7/f10 [0,4194304] 0 2024-03-29T04:45:21.016 INFO:tasks.workunit.client.1.smithi152.stdout:6/728: creat d1/d3/df/dd2/ff2 x:0 0 0 2024-03-29T04:45:21.016 INFO:tasks.workunit.client.1.smithi152.stdout:2/458: mkdir d0/d3/d94 0 2024-03-29T04:45:21.019 INFO:tasks.workunit.client.1.smithi152.stdout:6/729: chown d1/d3/d1a/f25 493 1 2024-03-29T04:45:21.019 INFO:tasks.workunit.client.1.smithi152.stdout:0/566: creat d2/da/d3a/fd2 x:0 0 0 2024-03-29T04:45:21.019 INFO:tasks.workunit.client.1.smithi152.stdout:1/825: dread d5/d7/d38/f7c [0,4194304] 0 2024-03-29T04:45:21.020 INFO:tasks.workunit.client.1.smithi152.stdout:2/459: chown d0/d3/df/d2c/d45 239 1 2024-03-29T04:45:21.021 INFO:tasks.workunit.client.1.smithi152.stdout:4/495: mknod d0/d7/d40/d23/d55/d6c/d91/ca1 0 2024-03-29T04:45:21.022 INFO:tasks.workunit.client.1.smithi152.stdout:3/583: unlink d7/c14 0 2024-03-29T04:45:21.022 INFO:tasks.workunit.client.1.smithi152.stdout:4/496: fsync d0/d7/d25/d17/d65/f79 0 2024-03-29T04:45:21.023 INFO:tasks.workunit.client.1.smithi152.stdout:3/584: fdatasync d7/fa3 0 2024-03-29T04:45:21.025 INFO:tasks.workunit.client.1.smithi152.stdout:0/567: rmdir d2/da/d17/d57/dc5 39 2024-03-29T04:45:21.028 INFO:tasks.workunit.client.1.smithi152.stdout:2/460: chown d0/d3/d30/d15/d22/c6d 450018 1 2024-03-29T04:45:21.028 INFO:tasks.workunit.client.1.smithi152.stdout:6/730: mkdir d1/d3/d1a/d1b/d87/d8f/df3 0 2024-03-29T04:45:21.029 INFO:tasks.workunit.client.1.smithi152.stdout:2/461: read - d0/d3/d30/d15/d22/d4c/f89 zero size 2024-03-29T04:45:21.029 INFO:tasks.workunit.client.1.smithi152.stdout:8/501: getdents d1/d2/d1d/d36/d4b/da1 0 2024-03-29T04:45:21.031 INFO:tasks.workunit.client.1.smithi152.stdout:5/629: link d6/d1d/daa/db7/d56/c8d d6/d1d/daa/db7/d56/d63/db9/cd5 0 2024-03-29T04:45:21.031 INFO:tasks.workunit.client.1.smithi152.stdout:8/502: chown d1/d2/d3/f96 863723463 1 2024-03-29T04:45:21.032 INFO:tasks.workunit.client.1.smithi152.stdout:0/568: unlink c1 0 2024-03-29T04:45:21.033 INFO:tasks.workunit.client.1.smithi152.stdout:6/731: creat d1/d3/d9/d13/d3e/d54/ff4 x:0 0 0 2024-03-29T04:45:21.033 INFO:tasks.workunit.client.1.smithi152.stdout:3/585: write d7/f15 [8397939,93782] 0 2024-03-29T04:45:21.034 INFO:tasks.workunit.client.1.smithi152.stdout:4/497: mkdir d0/d7/d25/d17/d5e/d60/d74/da2 0 2024-03-29T04:45:21.035 INFO:tasks.workunit.client.1.smithi152.stdout:6/732: fdatasync d1/d3/d9/d13/d3e/d93/fd1 0 2024-03-29T04:45:21.036 INFO:tasks.workunit.client.1.smithi152.stdout:4/498: fdatasync d0/d27/f59 0 2024-03-29T04:45:21.036 INFO:tasks.workunit.client.1.smithi152.stdout:5/630: creat d6/d1d/d30/fd6 x:0 0 0 2024-03-29T04:45:21.036 INFO:tasks.workunit.client.1.smithi152.stdout:8/503: symlink d1/d2/d1d/d36/laf 0 2024-03-29T04:45:21.037 INFO:tasks.workunit.client.1.smithi152.stdout:1/826: rmdir d5/d7/d38/d109 0 2024-03-29T04:45:21.038 INFO:tasks.workunit.client.1.smithi152.stdout:0/569: mknod d2/da/d17/d50/cd3 0 2024-03-29T04:45:21.041 INFO:tasks.workunit.client.1.smithi152.stdout:5/631: mknod d6/d1d/daa/db7/d56/d63/cd7 0 2024-03-29T04:45:21.041 INFO:tasks.workunit.client.1.smithi152.stdout:3/586: truncate d7/d19/d1b/d1e/d22/f4d 4172824 0 2024-03-29T04:45:21.046 INFO:tasks.workunit.client.1.smithi152.stdout:3/587: read d7/d19/d3a/f49 [1859282,36357] 0 2024-03-29T04:45:21.051 INFO:tasks.workunit.client.1.smithi152.stdout:4/499: dwrite d0/d27/d6e/f97 [0,4194304] 0 2024-03-29T04:45:21.051 INFO:tasks.workunit.client.1.smithi152.stdout:3/588: read - d7/d19/d1b/d1e/da1/f54 zero size 2024-03-29T04:45:21.054 INFO:tasks.workunit.client.1.smithi152.stdout:6/733: truncate d1/d3/df/f81 896801 0 2024-03-29T04:45:21.059 INFO:tasks.workunit.client.1.smithi152.stdout:1/827: dwrite d5/d7/d8/d13/f35 [4194304,4194304] 0 2024-03-29T04:45:21.060 INFO:tasks.workunit.client.1.smithi152.stdout:2/462: dread d0/d3/d30/f1a [0,4194304] 0 2024-03-29T04:45:21.063 INFO:tasks.workunit.client.1.smithi152.stdout:1/828: read d5/d7/d8/d13/f45 [3533806,19522] 0 2024-03-29T04:45:21.063 INFO:tasks.workunit.client.1.smithi152.stdout:6/734: mkdir d1/d3/d9/d13/da7/df5 0 2024-03-29T04:45:21.064 INFO:tasks.workunit.client.1.smithi152.stdout:4/500: dread d0/d7/d40/d75/f7d [0,4194304] 0 2024-03-29T04:45:21.064 INFO:tasks.workunit.client.1.smithi152.stdout:8/504: dread d1/f7b [0,4194304] 0 2024-03-29T04:45:21.064 INFO:tasks.workunit.client.1.smithi152.stdout:2/463: fsync d0/d3/d30/f1a 0 2024-03-29T04:45:21.065 INFO:tasks.workunit.client.1.smithi152.stdout:3/589: mkdir d7/d19/d1b/d1e/da1/d41/daf 0 2024-03-29T04:45:21.067 INFO:tasks.workunit.client.1.smithi152.stdout:5/632: dwrite d6/d1d/d3b/d55/f35 [0,4194304] 0 2024-03-29T04:45:21.067 INFO:tasks.workunit.client.1.smithi152.stdout:8/505: fsync d1/d2/d3/f8e 0 2024-03-29T04:45:21.069 INFO:tasks.workunit.client.1.smithi152.stdout:3/590: dread - d7/f91 zero size 2024-03-29T04:45:21.071 INFO:tasks.workunit.client.1.smithi152.stdout:8/506: write d1/d2/d59/f88 [83966,48250] 0 2024-03-29T04:45:21.071 INFO:tasks.workunit.client.1.smithi152.stdout:1/829: mknod d5/d7/c10d 0 2024-03-29T04:45:21.073 INFO:tasks.workunit.client.1.smithi152.stdout:4/501: mknod d0/d7/d25/d17/d37/ca3 0 2024-03-29T04:45:21.074 INFO:tasks.workunit.client.1.smithi152.stdout:1/830: stat d5/d7/d8/d13/d19/f3d 0 2024-03-29T04:45:21.075 INFO:tasks.workunit.client.1.smithi152.stdout:2/464: creat d0/d3/d30/d15/d22/d4c/d52/f95 x:0 0 0 2024-03-29T04:45:21.077 INFO:tasks.workunit.client.1.smithi152.stdout:8/507: unlink d1/d2/d3/l2f 0 2024-03-29T04:45:21.078 INFO:tasks.workunit.client.1.smithi152.stdout:3/591: dread d7/d19/f5c [0,4194304] 0 2024-03-29T04:45:21.081 INFO:tasks.workunit.client.1.smithi152.stdout:6/735: getdents d1/d3/d9/d13/dc0/dea 0 2024-03-29T04:45:21.083 INFO:tasks.workunit.client.1.smithi152.stdout:8/508: rename d1/d21 to d1/d2/d1d/d36/d4b/da1/db0 0 2024-03-29T04:45:21.085 INFO:tasks.workunit.client.1.smithi152.stdout:3/592: creat d7/d19/d1b/d1e/d22/fb0 x:0 0 0 2024-03-29T04:45:21.085 INFO:tasks.workunit.client.1.smithi152.stdout:6/736: rmdir d1/d3/df/d8a/d98 39 2024-03-29T04:45:21.086 INFO:tasks.workunit.client.1.smithi152.stdout:2/465: symlink d0/d3/d64/d85/l96 0 2024-03-29T04:45:21.086 INFO:tasks.workunit.client.1.smithi152.stdout:8/509: rmdir d1/d2 39 2024-03-29T04:45:21.087 INFO:tasks.workunit.client.1.smithi152.stdout:2/466: chown d0/d3/df/d2c/c7a 16 1 2024-03-29T04:45:21.087 INFO:tasks.workunit.client.1.smithi152.stdout:3/593: mknod d7/d19/d1b/d1e/da1/d41/d6a/cb1 0 2024-03-29T04:45:21.089 INFO:tasks.workunit.client.1.smithi152.stdout:2/467: rmdir d0/d3/df/d8c 39 2024-03-29T04:45:21.089 INFO:tasks.workunit.client.1.smithi152.stdout:6/737: mknod d1/d3/d9/cf6 0 2024-03-29T04:45:21.090 INFO:tasks.workunit.client.1.smithi152.stdout:8/510: write d1/d2/d59/fa8 [786989,80933] 0 2024-03-29T04:45:21.092 INFO:tasks.workunit.client.1.smithi152.stdout:3/594: truncate d7/d19/d1b/d1e/f45 18996 0 2024-03-29T04:45:21.097 INFO:tasks.workunit.client.1.smithi152.stdout:8/511: rename d1/d2/d1d/l31 to d1/d2/d1d/d36/d4b/d51/lb1 0 2024-03-29T04:45:21.098 INFO:tasks.workunit.client.1.smithi152.stdout:6/738: creat d1/d3/df/d84/ff7 x:0 0 0 2024-03-29T04:45:21.099 INFO:tasks.workunit.client.1.smithi152.stdout:8/512: creat d1/d2/d1d/d36/d85/fb2 x:0 0 0 2024-03-29T04:45:21.102 INFO:tasks.workunit.client.1.smithi152.stdout:8/513: creat d1/d8a/fb3 x:0 0 0 2024-03-29T04:45:21.102 INFO:tasks.workunit.client.1.smithi152.stdout:3/595: dread d7/d19/f26 [0,4194304] 0 2024-03-29T04:45:21.104 INFO:tasks.workunit.client.1.smithi152.stdout:1/831: write d5/d7/d8/d13/f74 [966501,12623] 0 2024-03-29T04:45:21.105 INFO:tasks.workunit.client.1.smithi152.stdout:3/596: fsync d7/d19/d1b/d1e/da1/d41/d6a/d78/f95 0 2024-03-29T04:45:21.105 INFO:tasks.workunit.client.1.smithi152.stdout:6/739: truncate d1/d3/d9/f21 1284209 0 2024-03-29T04:45:21.106 INFO:tasks.workunit.client.1.smithi152.stdout:8/514: creat d1/d2/d1d/d36/d85/fb4 x:0 0 0 2024-03-29T04:45:21.109 INFO:tasks.workunit.client.1.smithi152.stdout:4/502: dread d0/f15 [0,4194304] 0 2024-03-29T04:45:21.109 INFO:tasks.workunit.client.1.smithi152.stdout:3/597: creat d7/d19/d3a/fb2 x:0 0 0 2024-03-29T04:45:21.113 INFO:tasks.workunit.client.1.smithi152.stdout:4/503: chown d0/d7/d25/d17/d37 377950 1 2024-03-29T04:45:21.113 INFO:tasks.workunit.client.1.smithi152.stdout:8/515: fsync d1/d2/d1d/d36/d4b/da1/db0/d8b/fa2 0 2024-03-29T04:45:21.113 INFO:tasks.workunit.client.1.smithi152.stdout:1/832: creat d5/d7/f10e x:0 0 0 2024-03-29T04:45:21.115 INFO:tasks.workunit.client.1.smithi152.stdout:5/633: dwrite d6/d1d/d30/f38 [0,4194304] 0 2024-03-29T04:45:21.116 INFO:tasks.workunit.client.1.smithi152.stdout:2/468: dread d0/d3/d30/d15/f2b [0,4194304] 0 2024-03-29T04:45:21.116 INFO:tasks.workunit.client.1.smithi152.stdout:8/516: fsync d1/d8a/f8c 0 2024-03-29T04:45:21.119 INFO:tasks.workunit.client.1.smithi152.stdout:1/833: mkdir d5/d7/d1b/d10f 0 2024-03-29T04:45:21.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:20 smithi142 ceph-mon[29027]: pgmap v276: 65 pgs: 65 active+clean; 3.1 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 102 MiB/s rd, 300 MiB/s wr, 613 op/s 2024-03-29T04:45:21.121 INFO:tasks.workunit.client.1.smithi152.stdout:1/834: chown d5/d7/d1b/d2e/de2/d102/d104/lbc 0 1 2024-03-29T04:45:21.122 INFO:tasks.workunit.client.1.smithi152.stdout:2/469: chown d0/d3/c5c 5799 1 2024-03-29T04:45:21.122 INFO:tasks.workunit.client.1.smithi152.stdout:6/740: truncate d1/d3/d1a/d1b/d29/f7e 957054 0 2024-03-29T04:45:21.124 INFO:tasks.workunit.client.1.smithi152.stdout:4/504: truncate d0/d7/d25/d17/d5e/d60/d74/d57/f5c 682578 0 2024-03-29T04:45:21.124 INFO:tasks.workunit.client.1.smithi152.stdout:6/741: stat d1/d3/d1a/fbd 0 2024-03-29T04:45:21.125 INFO:tasks.workunit.client.1.smithi152.stdout:3/598: creat d7/d19/d1b/d1e/fb3 x:0 0 0 2024-03-29T04:45:21.126 INFO:tasks.workunit.client.1.smithi152.stdout:2/470: readlink d0/l5b 0 2024-03-29T04:45:21.126 INFO:tasks.workunit.client.1.smithi152.stdout:8/517: dwrite d1/d2/d1d/d36/d85/fb2 [0,4194304] 0 2024-03-29T04:45:21.128 INFO:tasks.workunit.client.1.smithi152.stdout:5/634: creat d6/d1d/daa/db7/d56/dc9/fd8 x:0 0 0 2024-03-29T04:45:21.130 INFO:tasks.workunit.client.1.smithi152.stdout:6/742: write d1/d3/d1a/d1b/d29/f79 [414524,105869] 0 2024-03-29T04:45:21.132 INFO:tasks.workunit.client.1.smithi152.stdout:8/518: write d1/d2/d1d/d36/fa0 [772506,88618] 0 2024-03-29T04:45:21.133 INFO:tasks.workunit.client.1.smithi152.stdout:3/599: mknod d7/d19/d1b/d1e/cb4 0 2024-03-29T04:45:21.134 INFO:tasks.workunit.client.1.smithi152.stdout:1/835: dwrite d5/fb8 [0,4194304] 0 2024-03-29T04:45:21.136 INFO:tasks.workunit.client.1.smithi152.stdout:2/471: mkdir d0/d3/df/d8c/d97 0 2024-03-29T04:45:21.136 INFO:tasks.workunit.client.1.smithi152.stdout:5/635: rmdir d6/d1d/daa/db7/d56/d63 39 2024-03-29T04:45:21.136 INFO:tasks.workunit.client.1.smithi152.stdout:8/519: truncate d1/d2/d6d/f9c 721051 0 2024-03-29T04:45:21.138 INFO:tasks.workunit.client.1.smithi152.stdout:5/636: dread - d6/d7/fd3 zero size 2024-03-29T04:45:21.139 INFO:tasks.workunit.client.1.smithi152.stdout:6/743: creat d1/d3/d9/d13/d3e/d93/ff8 x:0 0 0 2024-03-29T04:45:21.140 INFO:tasks.workunit.client.1.smithi152.stdout:8/520: write d1/d2/d59/f88 [1160812,12423] 0 2024-03-29T04:45:21.142 INFO:tasks.workunit.client.1.smithi152.stdout:3/600: mkdir d7/d19/db5 0 2024-03-29T04:45:21.142 INFO:tasks.workunit.client.1.smithi152.stdout:5/637: read d6/d16/d25/d2e/dc7/fcd [1055125,59318] 0 2024-03-29T04:45:21.142 INFO:tasks.workunit.client.1.smithi152.stdout:3/601: chown d7/d46 67763 1 2024-03-29T04:45:21.143 INFO:tasks.workunit.client.1.smithi152.stdout:1/836: symlink d5/d7/d1b/d10f/l110 0 2024-03-29T04:45:21.143 INFO:tasks.workunit.client.1.smithi152.stdout:6/744: fdatasync d1/d3/d1a/d1b/d29/fcc 0 2024-03-29T04:45:21.146 INFO:tasks.workunit.client.1.smithi152.stdout:3/602: chown d7/d19/c42 6 1 2024-03-29T04:45:21.146 INFO:tasks.workunit.client.1.smithi152.stdout:1/837: read - d5/d7/f10e zero size 2024-03-29T04:45:21.151 INFO:tasks.workunit.client.1.smithi152.stdout:2/472: dwrite d0/d3/d30/d15/d22/f49 [0,4194304] 0 2024-03-29T04:45:21.153 INFO:tasks.workunit.client.1.smithi152.stdout:3/603: creat d7/d19/fb6 x:0 0 0 2024-03-29T04:45:21.154 INFO:tasks.workunit.client.1.smithi152.stdout:1/838: read d5/d7/d8/d13/d19/f94 [540974,76430] 0 2024-03-29T04:45:21.155 INFO:tasks.workunit.client.1.smithi152.stdout:2/473: stat d0/d3/d30/f33 0 2024-03-29T04:45:21.157 INFO:tasks.workunit.client.1.smithi152.stdout:6/745: write d1/d3/d9/f3a [1075967,17818] 0 2024-03-29T04:45:21.159 INFO:tasks.workunit.client.1.smithi152.stdout:8/521: dwrite d1/d2/d1d/d36/d4b/da1/db0/f2a [0,4194304] 0 2024-03-29T04:45:21.159 INFO:tasks.workunit.client.1.smithi152.stdout:3/604: rmdir d7/d19/d1b/d1e 39 2024-03-29T04:45:21.160 INFO:tasks.workunit.client.1.smithi152.stdout:5/638: dwrite d6/d16/d25/d2e/da0/d92/fbc [0,4194304] 0 2024-03-29T04:45:21.161 INFO:tasks.workunit.client.1.smithi152.stdout:5/639: readlink d6/l8f 0 2024-03-29T04:45:21.162 INFO:tasks.workunit.client.1.smithi152.stdout:6/746: fsync d1/d3/d9/d13/d3e/d54/f7c 0 2024-03-29T04:45:21.164 INFO:tasks.workunit.client.1.smithi152.stdout:4/505: dread d0/d27/f2b [0,4194304] 0 2024-03-29T04:45:21.165 INFO:tasks.workunit.client.1.smithi152.stdout:3/605: fdatasync d7/d19/d1b/d1e/d4e/fad 0 2024-03-29T04:45:21.169 INFO:tasks.workunit.client.1.smithi152.stdout:2/474: dwrite d0/d3/d30/d15/d22/f49 [0,4194304] 0 2024-03-29T04:45:21.172 INFO:tasks.workunit.client.1.smithi152.stdout:8/522: creat d1/d2/d1d/d36/d5d/fb5 x:0 0 0 2024-03-29T04:45:21.173 INFO:tasks.workunit.client.1.smithi152.stdout:6/747: creat d1/d3/d9/d13/dc0/ff9 x:0 0 0 2024-03-29T04:45:21.179 INFO:tasks.workunit.client.1.smithi152.stdout:3/606: creat d7/d19/d1b/d1e/da1/d41/d61/d75/fb7 x:0 0 0 2024-03-29T04:45:21.180 INFO:tasks.workunit.client.1.smithi152.stdout:8/523: symlink d1/d2/d1d/d36/d4b/da1/db0/d40/lb6 0 2024-03-29T04:45:21.184 INFO:tasks.workunit.client.1.smithi152.stdout:3/607: stat d7/d19/d1b/d1e/da1/d41/d6a/d78/c81 0 2024-03-29T04:45:21.186 INFO:tasks.workunit.client.1.smithi152.stdout:1/839: getdents d5/d7/d1b/d2e/de2/d102 0 2024-03-29T04:45:21.187 INFO:tasks.workunit.client.1.smithi152.stdout:6/748: creat d1/ffa x:0 0 0 2024-03-29T04:45:21.188 INFO:tasks.workunit.client.1.smithi152.stdout:6/749: chown d1 2706740 1 2024-03-29T04:45:21.188 INFO:tasks.workunit.client.1.smithi152.stdout:4/506: dwrite d0/d27/f34 [4194304,4194304] 0 2024-03-29T04:45:21.191 INFO:tasks.workunit.client.1.smithi152.stdout:8/524: creat d1/d2/d1d/d36/d4b/da1/db0/d7f/fb7 x:0 0 0 2024-03-29T04:45:21.192 INFO:tasks.workunit.client.1.smithi152.stdout:3/608: truncate d7/d19/f25 1568228 0 2024-03-29T04:45:21.192 INFO:tasks.workunit.client.1.smithi152.stdout:1/840: fsync d5/d7/d8/d13/d19/fd7 0 2024-03-29T04:45:21.193 INFO:tasks.workunit.client.1.smithi152.stdout:4/507: chown d0/d7/c19 64771 1 2024-03-29T04:45:21.195 INFO:tasks.workunit.client.1.smithi152.stdout:8/525: mknod d1/d2/d3c/d76/d9a/cb8 0 2024-03-29T04:45:21.196 INFO:tasks.workunit.client.1.smithi152.stdout:1/841: chown d5/d7/ffa 20005658 1 2024-03-29T04:45:21.196 INFO:tasks.workunit.client.1.smithi152.stdout:4/508: write d0/d7/d40/f8a [1624117,34150] 0 2024-03-29T04:45:21.198 INFO:tasks.workunit.client.1.smithi152.stdout:1/842: chown d5/d7/d1b/d2e/de2/d102/d104/l90 1 1 2024-03-29T04:45:21.198 INFO:tasks.workunit.client.1.smithi152.stdout:4/509: chown d0/d7/d25/f21 180000053 1 2024-03-29T04:45:21.199 INFO:tasks.workunit.client.1.smithi152.stdout:6/750: getdents d1/d3/d1a/d1b 0 2024-03-29T04:45:21.200 INFO:tasks.workunit.client.1.smithi152.stdout:8/526: creat d1/d2/d59/fb9 x:0 0 0 2024-03-29T04:45:21.202 INFO:tasks.workunit.client.1.smithi152.stdout:1/843: write d5/d7/d1b/d2e/de2/d102/d104/fe8 [2842378,73907] 0 2024-03-29T04:45:21.203 INFO:tasks.workunit.client.1.smithi152.stdout:4/510: creat d0/d7/d40/d6d/fa4 x:0 0 0 2024-03-29T04:45:21.205 INFO:tasks.workunit.client.1.smithi152.stdout:6/751: mkdir d1/d3/d9/d13/dc0/dc3/dfb 0 2024-03-29T04:45:21.206 INFO:tasks.workunit.client.1.smithi152.stdout:3/609: link d7/d19/d1b/d1e/da1/d41/f50 d7/d19/d1b/d1e/fb8 0 2024-03-29T04:45:21.208 INFO:tasks.workunit.client.1.smithi152.stdout:1/844: mkdir d5/d7/d4d/db1/d111 0 2024-03-29T04:45:21.208 INFO:tasks.workunit.client.1.smithi152.stdout:8/527: creat d1/d2/d3c/fba x:0 0 0 2024-03-29T04:45:21.212 INFO:tasks.workunit.client.1.smithi152.stdout:4/511: creat d0/d7/d40/d23/d6a/fa5 x:0 0 0 2024-03-29T04:45:21.212 INFO:tasks.workunit.client.1.smithi152.stdout:1/845: write d5/d7/d8/d13/d19/ff4 [206358,101473] 0 2024-03-29T04:45:21.216 INFO:tasks.workunit.client.1.smithi152.stdout:3/610: dwrite d7/f15 [0,4194304] 0 2024-03-29T04:45:21.217 INFO:tasks.workunit.client.1.smithi152.stdout:4/512: chown d0/d7/d40/d75/c9c 1175 1 2024-03-29T04:45:21.219 INFO:tasks.workunit.client.1.smithi152.stdout:2/475: dread d0/d3/f10 [0,4194304] 0 2024-03-29T04:45:21.221 INFO:tasks.workunit.client.1.smithi152.stdout:8/528: creat d1/d2/d1d/d36/d4b/da1/fbb x:0 0 0 2024-03-29T04:45:21.225 INFO:tasks.workunit.client.1.smithi152.stdout:1/846: symlink d5/d7/d38/d6c/l112 0 2024-03-29T04:45:21.227 INFO:tasks.workunit.client.1.smithi152.stdout:4/513: chown d0/c3d 52140 1 2024-03-29T04:45:21.227 INFO:tasks.workunit.client.1.smithi152.stdout:1/847: chown d5/d7/d1b/d2e/de2/d102/f5b 386842319 1 2024-03-29T04:45:21.228 INFO:tasks.workunit.client.1.smithi152.stdout:4/514: chown d0/c3d 1 1 2024-03-29T04:45:21.229 INFO:tasks.workunit.client.1.smithi152.stdout:4/515: read d0/d27/d6e/f97 [455656,102397] 0 2024-03-29T04:45:21.231 INFO:tasks.workunit.client.1.smithi152.stdout:6/752: link d1/d3/d1a/d1b/d29/ff1 d1/d3/d9/d13/d3e/d93/ffc 0 2024-03-29T04:45:21.232 INFO:tasks.workunit.client.1.smithi152.stdout:6/753: fsync d1/d3/d9/d13/dc0/fe6 0 2024-03-29T04:45:21.232 INFO:tasks.workunit.client.1.smithi152.stdout:8/529: creat d1/d2/d1d/d36/d4b/da1/db0/da9/fbc x:0 0 0 2024-03-29T04:45:21.233 INFO:tasks.workunit.client.1.smithi152.stdout:3/611: mkdir d7/d19/d1b/d1e/db9 0 2024-03-29T04:45:21.237 INFO:tasks.workunit.client.1.smithi152.stdout:2/476: write d0/d3/d30/d15/d22/f2f [5066531,53396] 0 2024-03-29T04:45:21.241 INFO:tasks.workunit.client.1.smithi152.stdout:1/848: dwrite d5/d7/d1b/d2e/de2/d102/f61 [0,4194304] 0 2024-03-29T04:45:21.244 INFO:tasks.workunit.client.1.smithi152.stdout:1/849: write d5/d7/d1b/d2e/f8e [134045,120086] 0 2024-03-29T04:45:21.245 INFO:tasks.workunit.client.1.smithi152.stdout:4/516: mkdir d0/d7/d25/d17/d5e/d60/d74/da2/da6 0 2024-03-29T04:45:21.246 INFO:tasks.workunit.client.1.smithi152.stdout:1/850: dread d5/f72 [0,4194304] 0 2024-03-29T04:45:21.247 INFO:tasks.workunit.client.1.smithi152.stdout:6/754: mkdir d1/d3/d9/d13/da7/df5/dfd 0 2024-03-29T04:45:21.247 INFO:tasks.workunit.client.1.smithi152.stdout:2/477: write d0/d3/f81 [733559,14609] 0 2024-03-29T04:45:21.248 INFO:tasks.workunit.client.1.smithi152.stdout:3/612: fdatasync d7/d19/d1b/f7b 0 2024-03-29T04:45:21.250 INFO:tasks.workunit.client.1.smithi152.stdout:8/530: rename d1/d2/d1d/d36/d4b/da1/db0/l37 to d1/d2/d1d/d36/d85/lbd 0 2024-03-29T04:45:21.253 INFO:tasks.workunit.client.1.smithi152.stdout:2/478: readlink d0/d3/l6c 0 2024-03-29T04:45:21.256 INFO:tasks.workunit.client.1.smithi152.stdout:1/851: mkdir d5/d7/d1b/d2e/de2/d102/d113 0 2024-03-29T04:45:21.258 INFO:tasks.workunit.client.1.smithi152.stdout:8/531: mkdir d1/d2/d1d/d36/d85/dbe 0 2024-03-29T04:45:21.258 INFO:tasks.workunit.client.1.smithi152.stdout:4/517: dwrite d0/d7/d40/d75/f7d [0,4194304] 0 2024-03-29T04:45:21.259 INFO:tasks.workunit.client.1.smithi152.stdout:6/755: symlink d1/d3/df/lfe 0 2024-03-29T04:45:21.261 INFO:tasks.workunit.client.1.smithi152.stdout:3/613: symlink d7/d6c/lba 0 2024-03-29T04:45:21.261 INFO:tasks.workunit.client.1.smithi152.stdout:2/479: mknod d0/d3/df/d8c/d97/c98 0 2024-03-29T04:45:21.263 INFO:tasks.workunit.client.1.smithi152.stdout:2/480: fdatasync d0/d3/d64/f6e 0 2024-03-29T04:45:21.263 INFO:tasks.workunit.client.1.smithi152.stdout:8/532: mknod d1/d2/d3/cbf 0 2024-03-29T04:45:21.264 INFO:tasks.workunit.client.1.smithi152.stdout:4/518: symlink d0/d7/d40/d23/d55/d70/d96/la7 0 2024-03-29T04:45:21.266 INFO:tasks.workunit.client.1.smithi152.stdout:1/852: symlink d5/d7/d4d/dd4/l114 0 2024-03-29T04:45:21.273 INFO:tasks.workunit.client.1.smithi152.stdout:8/533: dwrite d1/d2/d1d/d36/d4b/da1/db0/da9/f84 [0,4194304] 0 2024-03-29T04:45:21.274 INFO:tasks.workunit.client.1.smithi152.stdout:6/756: creat d1/d3/df/fff x:0 0 0 2024-03-29T04:45:21.276 INFO:tasks.workunit.client.1.smithi152.stdout:3/614: dwrite d7/f87 [0,4194304] 0 2024-03-29T04:45:21.276 INFO:tasks.workunit.client.1.smithi152.stdout:1/853: creat d5/d7/d1b/d2e/d95/dad/d12/d77/f115 x:0 0 0 2024-03-29T04:45:21.276 INFO:tasks.workunit.client.1.smithi152.stdout:4/519: unlink d0/cb 0 2024-03-29T04:45:21.278 INFO:tasks.workunit.client.1.smithi152.stdout:2/481: creat d0/d3/d30/d15/d22/f99 x:0 0 0 2024-03-29T04:45:21.281 INFO:tasks.workunit.client.1.smithi152.stdout:3/615: unlink d7/d19/d1b/d1e/da1/d41/d6a/d78/f83 0 2024-03-29T04:45:21.282 INFO:tasks.workunit.client.1.smithi152.stdout:4/520: symlink d0/d7/d40/d23/d55/la8 0 2024-03-29T04:45:21.283 INFO:tasks.workunit.client.1.smithi152.stdout:4/521: readlink d0/d7/d25/d17/l4e 0 2024-03-29T04:45:21.286 INFO:tasks.workunit.client.1.smithi152.stdout:8/534: mknod d1/d2/d1d/d36/d4b/da1/db0/d40/d6a/cc0 0 2024-03-29T04:45:21.286 INFO:tasks.workunit.client.1.smithi152.stdout:8/535: read - d1/d2/d1d/d36/d4b/da1/fbb zero size 2024-03-29T04:45:21.286 INFO:tasks.workunit.client.1.smithi152.stdout:6/757: rename d1/d3/d9/d13/d3e/f42 to d1/d3/d9/d13/dc0/dc3/dfb/f100 0 2024-03-29T04:45:21.288 INFO:tasks.workunit.client.1.smithi152.stdout:8/536: chown d1/d2/d1d/d42/f9e 573230 1 2024-03-29T04:45:21.288 INFO:tasks.workunit.client.1.smithi152.stdout:2/482: mkdir d0/d3/d30/d15/d22/d4c/d9a 0 2024-03-29T04:45:21.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:20 smithi152 ceph-mon[37177]: pgmap v276: 65 pgs: 65 active+clean; 3.1 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 102 MiB/s rd, 300 MiB/s wr, 613 op/s 2024-03-29T04:45:21.291 INFO:tasks.workunit.client.1.smithi152.stdout:3/616: creat d7/d19/d1b/d1e/da1/d41/d73/fbb x:0 0 0 2024-03-29T04:45:21.292 INFO:tasks.workunit.client.1.smithi152.stdout:1/854: unlink d5/d7/d1b/d2e/d95/d9d/da7/lff 0 2024-03-29T04:45:21.297 INFO:tasks.workunit.client.1.smithi152.stdout:8/537: rename d1/d2/d3c/fba to d1/d2/d1d/d36/d85/fc1 0 2024-03-29T04:45:21.298 INFO:tasks.workunit.client.1.smithi152.stdout:6/758: dwrite d1/d3/d1a/d1b/d29/d6f/f69 [0,4194304] 0 2024-03-29T04:45:21.301 INFO:tasks.workunit.client.1.smithi152.stdout:3/617: mknod d7/d19/d1b/d1e/d4e/d8f/cbc 0 2024-03-29T04:45:21.302 INFO:tasks.workunit.client.1.smithi152.stdout:3/618: read d7/d19/d1b/f2f [7234599,27783] 0 2024-03-29T04:45:21.302 INFO:tasks.workunit.client.1.smithi152.stdout:1/855: unlink d5/d7/d8/d13/dd0/dd3/ffc 0 2024-03-29T04:45:21.303 INFO:tasks.workunit.client.1.smithi152.stdout:4/522: getdents d0/d7/d25 0 2024-03-29T04:45:21.305 INFO:tasks.workunit.client.1.smithi152.stdout:3/619: dread - d7/d19/d1b/d1e/da1/d41/d6a/d78/f96 zero size 2024-03-29T04:45:21.306 INFO:tasks.workunit.client.1.smithi152.stdout:8/538: mkdir d1/d2/d1d/d36/d4b/da1/dc2 0 2024-03-29T04:45:21.306 INFO:tasks.workunit.client.1.smithi152.stdout:4/523: dread - d0/d7/d25/d17/d5e/d60/f88 zero size 2024-03-29T04:45:21.307 INFO:tasks.workunit.client.1.smithi152.stdout:3/620: dread - d7/f91 zero size 2024-03-29T04:45:21.307 INFO:tasks.workunit.client.1.smithi152.stdout:1/856: dread d5/fcc [0,4194304] 0 2024-03-29T04:45:21.310 INFO:tasks.workunit.client.1.smithi152.stdout:8/539: stat d1/d2/d1d/d36/d4b/da1/db0/da9 0 2024-03-29T04:45:21.312 INFO:tasks.workunit.client.1.smithi152.stdout:4/524: unlink d0/d27/d6e/d78/f7b 0 2024-03-29T04:45:21.313 INFO:tasks.workunit.client.1.smithi152.stdout:6/759: write f0 [965335,88685] 0 2024-03-29T04:45:21.316 INFO:tasks.workunit.client.1.smithi152.stdout:1/857: read d5/d7/d6a/f88 [984408,79783] 0 2024-03-29T04:45:21.316 INFO:tasks.workunit.client.1.smithi152.stdout:3/621: write d7/d19/d1b/d1e/da1/f2c [2217625,57727] 0 2024-03-29T04:45:21.319 INFO:tasks.workunit.client.1.smithi152.stdout:8/540: read d1/d2/d1d/d36/d4b/da1/db0/d40/f6c [2682611,14385] 0 2024-03-29T04:45:21.319 INFO:tasks.workunit.client.1.smithi152.stdout:2/483: dwrite d0/d3/d30/f19 [4194304,4194304] 0 2024-03-29T04:45:21.319 INFO:tasks.workunit.client.1.smithi152.stdout:4/525: write d0/d7/d25/d17/f3b [333059,52552] 0 2024-03-29T04:45:21.323 INFO:tasks.workunit.client.1.smithi152.stdout:4/526: write d0/d7/f8d [3608709,67388] 0 2024-03-29T04:45:21.323 INFO:tasks.workunit.client.1.smithi152.stdout:2/484: dread - d0/d3/df/d2c/f2d zero size 2024-03-29T04:45:21.328 INFO:tasks.workunit.client.1.smithi152.stdout:2/485: write d0/d3/d30/d15/d22/f99 [288248,52259] 0 2024-03-29T04:45:21.331 INFO:tasks.workunit.client.1.smithi152.stdout:3/622: rename d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/fa0 to d7/d19/d1b/d1e/da1/d41/d61/d75/fbd 0 2024-03-29T04:45:21.332 INFO:tasks.workunit.client.1.smithi152.stdout:1/858: write d5/d7/d1b/f70 [3638965,115385] 0 2024-03-29T04:45:21.333 INFO:tasks.workunit.client.1.smithi152.stdout:6/760: dwrite d1/d3/d1a/d1b/d29/d6f/d95/fde [0,4194304] 0 2024-03-29T04:45:21.335 INFO:tasks.workunit.client.1.smithi152.stdout:2/486: chown d0/d3/d30/d15/d22/d4c/d5a/f6b 325 1 2024-03-29T04:45:21.337 INFO:tasks.workunit.client.1.smithi152.stdout:4/527: truncate d0/d7/d25/d17/d5e/d60/d74/d57/f7a 619217 0 2024-03-29T04:45:21.341 INFO:tasks.workunit.client.1.smithi152.stdout:6/761: symlink d1/d3/d9/d13/da7/l101 0 2024-03-29T04:45:21.344 INFO:tasks.workunit.client.1.smithi152.stdout:2/487: mknod d0/d3/d5f/c9b 0 2024-03-29T04:45:21.344 INFO:tasks.workunit.client.1.smithi152.stdout:2/488: readlink d0/d3/df/d2c/l4f 0 2024-03-29T04:45:21.346 INFO:tasks.workunit.client.1.smithi152.stdout:3/623: creat d7/d19/d1b/fbe x:0 0 0 2024-03-29T04:45:21.347 INFO:tasks.workunit.client.1.smithi152.stdout:6/762: mkdir d1/d3/d1a/d1b/d87/d8f/d102 0 2024-03-29T04:45:21.347 INFO:tasks.workunit.client.1.smithi152.stdout:4/528: dread d0/d7/d40/d75/f7d [0,4194304] 0 2024-03-29T04:45:21.348 INFO:tasks.workunit.client.1.smithi152.stdout:2/489: rmdir d0/d3/d64 39 2024-03-29T04:45:21.353 INFO:tasks.workunit.client.1.smithi152.stdout:1/859: truncate d5/d7/d6a/f88 3856881 0 2024-03-29T04:45:21.356 INFO:tasks.workunit.client.1.smithi152.stdout:4/529: getdents d0/d7/d25/d17/d37/d77 0 2024-03-29T04:45:21.360 INFO:tasks.workunit.client.1.smithi152.stdout:6/763: dwrite d1/d3/d1a/f2c [0,4194304] 0 2024-03-29T04:45:21.366 INFO:tasks.workunit.client.1.smithi152.stdout:2/490: dwrite d0/d3/d30/d15/d22/d4c/f89 [0,4194304] 0 2024-03-29T04:45:21.367 INFO:tasks.workunit.client.1.smithi152.stdout:6/764: creat d1/d3/df/dd2/f103 x:0 0 0 2024-03-29T04:45:21.368 INFO:tasks.workunit.client.1.smithi152.stdout:2/491: write d0/d3/d30/d15/d22/f99 [613576,102050] 0 2024-03-29T04:45:21.372 INFO:tasks.workunit.client.1.smithi152.stdout:4/530: creat d0/d7/d25/d17/d5e/d60/d74/da2/da6/fa9 x:0 0 0 2024-03-29T04:45:21.373 INFO:tasks.workunit.client.1.smithi152.stdout:4/531: write d0/f11 [2264614,103333] 0 2024-03-29T04:45:21.375 INFO:tasks.workunit.client.1.smithi152.stdout:4/532: chown d0/d7/d25/d17/d37/f4f 42600781 1 2024-03-29T04:45:21.376 INFO:tasks.workunit.client.1.smithi152.stdout:4/533: dread - d0/d7/d40/d23/d55/d70/d96/f9f zero size 2024-03-29T04:45:21.379 INFO:tasks.workunit.client.1.smithi152.stdout:6/765: link d1/d3/d9/d13/d28/l64 d1/d3/d1a/d1b/d29/d6f/d95/l104 0 2024-03-29T04:45:21.380 INFO:tasks.workunit.client.1.smithi152.stdout:3/624: dread d7/d19/d1b/d1e/d22/f3b [0,4194304] 0 2024-03-29T04:45:21.381 INFO:tasks.workunit.client.1.smithi152.stdout:2/492: getdents d0/d3/d30/d15/d22/d4c 0 2024-03-29T04:45:21.383 INFO:tasks.workunit.client.1.smithi152.stdout:4/534: write d0/d7/d25/d17/d5e/d60/f8b [4425472,73099] 0 2024-03-29T04:45:21.383 INFO:tasks.workunit.client.1.smithi152.stdout:2/493: mknod d0/d3/d5f/c9c 0 2024-03-29T04:45:21.386 INFO:tasks.workunit.client.1.smithi152.stdout:3/625: getdents d7/d19/d1b/d1e/da1/d41/daf 0 2024-03-29T04:45:21.387 INFO:tasks.workunit.client.1.smithi152.stdout:2/494: chown d0/d3/d30/d15/d22/c6d 55 1 2024-03-29T04:45:21.390 INFO:tasks.workunit.client.1.smithi152.stdout:1/860: dread d5/d7/d8/d13/d19/f30 [0,4194304] 0 2024-03-29T04:45:21.391 INFO:tasks.workunit.client.1.smithi152.stdout:6/766: dwrite d1/d3/d1a/f2c [0,4194304] 0 2024-03-29T04:45:21.392 INFO:tasks.workunit.client.1.smithi152.stdout:2/495: chown d0/d3/d30/d15/d22/d4c/f89 48008 1 2024-03-29T04:45:21.393 INFO:tasks.workunit.client.1.smithi152.stdout:4/535: dread d0/d7/d25/d17/d5e/d60/d74/d57/f5c [0,4194304] 0 2024-03-29T04:45:21.396 INFO:tasks.workunit.client.1.smithi152.stdout:3/626: symlink d7/d19/d1b/d1e/da1/d41/d61/d99/lbf 0 2024-03-29T04:45:21.397 INFO:tasks.workunit.client.1.smithi152.stdout:4/536: creat d0/d7/d40/d2a/faa x:0 0 0 2024-03-29T04:45:21.399 INFO:tasks.workunit.client.1.smithi152.stdout:6/767: symlink d1/d3/d9/d13/d28/l105 0 2024-03-29T04:45:21.401 INFO:tasks.workunit.client.1.smithi152.stdout:2/496: rename d0/d3/d30/d15/d22/d4c/d9a to d0/d3/d64/d85/d9d 0 2024-03-29T04:45:21.407 INFO:tasks.workunit.client.1.smithi152.stdout:6/768: stat d1/d3/d9/d13/d3e/fd3 0 2024-03-29T04:45:21.408 INFO:tasks.workunit.client.1.smithi152.stdout:3/627: dwrite d7/f91 [0,4194304] 0 2024-03-29T04:45:21.412 INFO:tasks.workunit.client.1.smithi152.stdout:4/537: dwrite d0/d7/d40/f67 [0,4194304] 0 2024-03-29T04:45:21.413 INFO:tasks.workunit.client.1.smithi152.stdout:1/861: dread d5/d7/d8/fb [0,4194304] 0 2024-03-29T04:45:21.419 INFO:tasks.workunit.client.1.smithi152.stdout:6/769: unlink d1/d3/d9/d13/dc0/dc3/cdf 0 2024-03-29T04:45:21.422 INFO:tasks.workunit.client.1.smithi152.stdout:3/628: dwrite d7/fa3 [0,4194304] 0 2024-03-29T04:45:21.425 INFO:tasks.workunit.client.1.smithi152.stdout:6/770: chown d1/d3/d9/d13/d3e/l65 434180524 1 2024-03-29T04:45:21.425 INFO:tasks.workunit.client.1.smithi152.stdout:3/629: write d7/d19/d3a/fb2 [1017210,11657] 0 2024-03-29T04:45:21.428 INFO:tasks.workunit.client.1.smithi152.stdout:4/538: dwrite d0/d27/d6e/f87 [0,4194304] 0 2024-03-29T04:45:21.428 INFO:tasks.workunit.client.1.smithi152.stdout:4/539: stat d0 0 2024-03-29T04:45:21.439 INFO:tasks.workunit.client.1.smithi152.stdout:3/630: creat d7/d19/d1b/d1e/d4e/d8f/fc0 x:0 0 0 2024-03-29T04:45:21.447 INFO:tasks.workunit.client.1.smithi152.stdout:2/497: dread d0/d3/df/f35 [4194304,4194304] 0 2024-03-29T04:45:21.448 INFO:tasks.workunit.client.1.smithi152.stdout:4/540: creat d0/d7/fab x:0 0 0 2024-03-29T04:45:21.449 INFO:tasks.workunit.client.1.smithi152.stdout:2/498: rmdir d0/d3/d30/d15/d53 39 2024-03-29T04:45:21.451 INFO:tasks.workunit.client.1.smithi152.stdout:6/771: getdents d1/d3/df/d8a/d98 0 2024-03-29T04:45:21.453 INFO:tasks.workunit.client.1.smithi152.stdout:4/541: getdents d0/d7/d25/d17/d5e/d60/d74 0 2024-03-29T04:45:21.454 INFO:tasks.workunit.client.1.smithi152.stdout:6/772: fdatasync d1/d3/df/f48 0 2024-03-29T04:45:21.455 INFO:tasks.workunit.client.1.smithi152.stdout:2/499: read d0/d3/d30/d15/d22/f37 [3876596,130198] 0 2024-03-29T04:45:21.455 INFO:tasks.workunit.client.1.smithi152.stdout:6/773: stat d1/d3/d9/d13/l2a 0 2024-03-29T04:45:21.457 INFO:tasks.workunit.client.1.smithi152.stdout:4/542: mkdir d0/d7/d25/d17/d5e/d60/d74/da2/da6/dac 0 2024-03-29T04:45:21.457 INFO:tasks.workunit.client.1.smithi152.stdout:6/774: fsync d1/d3/d9/d13/d28/d45/d8c/feb 0 2024-03-29T04:45:21.458 INFO:tasks.workunit.client.1.smithi152.stdout:1/862: dread d5/d7/d6a/f88 [0,4194304] 0 2024-03-29T04:45:21.458 INFO:tasks.workunit.client.1.smithi152.stdout:4/543: stat d0/d7/d40/d23 0 2024-03-29T04:45:21.461 INFO:tasks.workunit.client.1.smithi152.stdout:1/863: mknod d5/d7/d38/c116 0 2024-03-29T04:45:21.462 INFO:tasks.workunit.client.1.smithi152.stdout:4/544: creat d0/d27/d6e/fad x:0 0 0 2024-03-29T04:45:21.467 INFO:tasks.workunit.client.1.smithi152.stdout:4/545: rmdir d0/d7/d40/d23/d6a 39 2024-03-29T04:45:21.470 INFO:tasks.workunit.client.1.smithi152.stdout:4/546: readlink d0/d7/d40/d62/l89 0 2024-03-29T04:45:21.471 INFO:tasks.workunit.client.1.smithi152.stdout:2/500: dwrite d0/d3/d30/d15/d22/d4c/d52/f82 [0,4194304] 0 2024-03-29T04:45:21.474 INFO:tasks.workunit.client.1.smithi152.stdout:1/864: dwrite d5/d7/d8/d13/f45 [4194304,4194304] 0 2024-03-29T04:45:21.474 INFO:tasks.workunit.client.1.smithi152.stdout:6/775: dread d1/d3/d9/d13/f31 [0,4194304] 0 2024-03-29T04:45:21.475 INFO:tasks.workunit.client.1.smithi152.stdout:1/865: chown d5/f98 4538036 1 2024-03-29T04:45:21.475 INFO:tasks.workunit.client.1.smithi152.stdout:2/501: chown d0/d3/d30/d15/d53/d54/d57/f72 4008 1 2024-03-29T04:45:21.476 INFO:tasks.workunit.client.1.smithi152.stdout:6/776: chown d1/d3/d9/d99/de5 0 1 2024-03-29T04:45:21.479 INFO:tasks.workunit.client.1.smithi152.stdout:4/547: symlink d0/d27/d6e/d8c/lae 0 2024-03-29T04:45:21.481 INFO:tasks.workunit.client.1.smithi152.stdout:2/502: write d0/d3/d30/d15/f29 [1290762,95503] 0 2024-03-29T04:45:21.482 INFO:tasks.workunit.client.1.smithi152.stdout:4/548: write d0/f15 [3691249,62983] 0 2024-03-29T04:45:21.485 INFO:tasks.workunit.client.1.smithi152.stdout:1/866: dread d5/d7/d8/d13/f45 [4194304,4194304] 0 2024-03-29T04:45:21.486 INFO:tasks.workunit.client.1.smithi152.stdout:4/549: unlink d0/d27/f34 0 2024-03-29T04:45:21.486 INFO:tasks.workunit.client.1.smithi152.stdout:1/867: fsync d5/d7/f10e 0 2024-03-29T04:45:21.488 INFO:tasks.workunit.client.1.smithi152.stdout:2/503: dread d0/d3/d30/f87 [0,4194304] 0 2024-03-29T04:45:21.489 INFO:tasks.workunit.client.1.smithi152.stdout:4/550: read d0/d7/d25/d17/d5e/d60/d74/d57/f5c [643332,83635] 0 2024-03-29T04:45:21.489 INFO:tasks.workunit.client.1.smithi152.stdout:1/868: mkdir d5/d7/d117 0 2024-03-29T04:45:21.491 INFO:tasks.workunit.client.1.smithi152.stdout:2/504: mkdir d0/d3/df/d9e 0 2024-03-29T04:45:21.492 INFO:tasks.workunit.client.1.smithi152.stdout:1/869: mknod d5/d7/d1b/d2e/d95/d9d/c118 0 2024-03-29T04:45:21.494 INFO:tasks.workunit.client.1.smithi152.stdout:1/870: write d5/d7/fe [720212,76103] 0 2024-03-29T04:45:21.494 INFO:tasks.workunit.client.1.smithi152.stdout:6/777: write d1/d3/d9/d13/f33 [6555655,73939] 0 2024-03-29T04:45:21.496 INFO:tasks.workunit.client.1.smithi152.stdout:2/505: rename d0/d3/d64/d85/l96 to d0/d3/df/l9f 0 2024-03-29T04:45:21.497 INFO:tasks.workunit.client.1.smithi152.stdout:2/506: chown d0/d3/df/f36 90034 1 2024-03-29T04:45:21.498 INFO:tasks.workunit.client.1.smithi152.stdout:2/507: truncate d0/d3/df/f76 689218 0 2024-03-29T04:45:21.500 INFO:tasks.workunit.client.1.smithi152.stdout:6/778: getdents d1/d3/d1a/d1b 0 2024-03-29T04:45:21.500 INFO:tasks.workunit.client.1.smithi152.stdout:2/508: write d0/d3/d5f/f63 [1728722,100682] 0 2024-03-29T04:45:21.503 INFO:tasks.workunit.client.1.smithi152.stdout:6/779: mknod d1/d3/d9/d13/d3e/c106 0 2024-03-29T04:45:21.507 INFO:tasks.workunit.client.1.smithi152.stdout:2/509: dwrite d0/d3/f42 [8388608,4194304] 0 2024-03-29T04:45:21.508 INFO:tasks.workunit.client.1.smithi152.stdout:6/780: mknod d1/d3/d9/d13/dc0/dc3/dfb/c107 0 2024-03-29T04:45:21.508 INFO:tasks.workunit.client.1.smithi152.stdout:1/871: truncate d5/d7/d38/f7c 1199724 0 2024-03-29T04:45:21.509 INFO:tasks.workunit.client.1.smithi152.stdout:2/510: write d0/d3/d30/d15/f26 [274860,70202] 0 2024-03-29T04:45:21.513 INFO:tasks.workunit.client.1.smithi152.stdout:6/781: symlink d1/d3/df/d8a/d98/l108 0 2024-03-29T04:45:21.516 INFO:tasks.workunit.client.1.smithi152.stdout:6/782: dread - d1/d3/d9/d13/d3e/fd3 zero size 2024-03-29T04:45:21.518 INFO:tasks.workunit.client.1.smithi152.stdout:1/872: dwrite d5/d7/d1b/f70 [0,4194304] 0 2024-03-29T04:45:21.521 INFO:tasks.workunit.client.1.smithi152.stdout:1/873: mkdir d5/d7/d8/d13/d19/d80/d119 0 2024-03-29T04:45:21.523 INFO:tasks.workunit.client.1.smithi152.stdout:1/874: dread d5/d7/d1b/f93 [0,4194304] 0 2024-03-29T04:45:21.524 INFO:tasks.workunit.client.1.smithi152.stdout:6/783: rename d1/d3/d9/d13/d28/d45/d8c/caf to d1/d3/d9/c109 0 2024-03-29T04:45:21.528 INFO:tasks.workunit.client.1.smithi152.stdout:2/511: dread d0/d3/d30/d15/d22/f37 [0,4194304] 0 2024-03-29T04:45:21.529 INFO:tasks.workunit.client.1.smithi152.stdout:6/784: creat d1/f10a x:0 0 0 2024-03-29T04:45:21.530 INFO:tasks.workunit.client.1.smithi152.stdout:1/875: rename d5/d7/d1b/c2c to d5/d7/d1b/d2e/d95/dad/c11a 0 2024-03-29T04:45:21.534 INFO:tasks.workunit.client.1.smithi152.stdout:6/785: creat d1/d3/d9/d99/de5/f10b x:0 0 0 2024-03-29T04:45:21.542 INFO:tasks.workunit.client.1.smithi152.stdout:2/512: truncate d0/d3/d30/f3f 3072901 0 2024-03-29T04:45:21.543 INFO:tasks.workunit.client.1.smithi152.stdout:6/786: write d1/f94 [217116,59948] 0 2024-03-29T04:45:21.545 INFO:tasks.workunit.client.1.smithi152.stdout:1/876: dread d5/d7/d38/d6c/fd5 [0,4194304] 0 2024-03-29T04:45:21.546 INFO:tasks.workunit.client.1.smithi152.stdout:1/877: fsync d5/d7/ffa 0 2024-03-29T04:45:21.546 INFO:tasks.workunit.client.1.smithi152.stdout:2/513: rmdir d0/d3/d5f 39 2024-03-29T04:45:21.547 INFO:tasks.workunit.client.1.smithi152.stdout:6/787: creat d1/d3/d9/d13/d28/f10c x:0 0 0 2024-03-29T04:45:21.549 INFO:tasks.workunit.client.1.smithi152.stdout:1/878: fdatasync d5/d7/d1b/d2e/de2/d102/fd9 0 2024-03-29T04:45:21.550 INFO:tasks.workunit.client.1.smithi152.stdout:1/879: readlink d5/d7/d8/d13/l55 0 2024-03-29T04:45:21.551 INFO:tasks.workunit.client.1.smithi152.stdout:1/880: chown d5/d7/d8/f1e 56005708 1 2024-03-29T04:45:21.552 INFO:tasks.workunit.client.1.smithi152.stdout:6/788: rename d1/d3/d1a/d1b/d29/dce/dec to d1/d3/d1a/d1b/d10d 0 2024-03-29T04:45:21.554 INFO:tasks.workunit.client.1.smithi152.stdout:1/881: mkdir d5/d7/d1b/d10f/d11b 0 2024-03-29T04:45:21.555 INFO:tasks.workunit.client.1.smithi152.stdout:2/514: truncate d0/d3/d30/d15/f2b 3066766 0 2024-03-29T04:45:21.556 INFO:tasks.workunit.client.1.smithi152.stdout:6/789: mknod d1/d3/d1a/d1b/d29/c10e 0 2024-03-29T04:45:21.557 INFO:tasks.workunit.client.1.smithi152.stdout:1/882: truncate d5/d7/d8/d13/d19/dc7/fdf 553972 0 2024-03-29T04:45:21.558 INFO:tasks.workunit.client.1.smithi152.stdout:2/515: dread d0/d3/d30/d15/f26 [0,4194304] 0 2024-03-29T04:45:21.559 INFO:tasks.workunit.client.1.smithi152.stdout:2/516: write d0/d3/d30/d15/d22/f49 [2432658,129471] 0 2024-03-29T04:45:21.560 INFO:tasks.workunit.client.1.smithi152.stdout:6/790: getdents d1/d3/df/d8a 0 2024-03-29T04:45:21.567 INFO:tasks.workunit.client.1.smithi152.stdout:9/589: sync 2024-03-29T04:45:21.568 INFO:tasks.workunit.client.1.smithi152.stdout:1/883: getdents d5/d7/d8/da5 0 2024-03-29T04:45:21.569 INFO:tasks.workunit.client.1.smithi152.stdout:9/590: write d5/d8/d31/d59/fb0 [421568,18606] 0 2024-03-29T04:45:21.570 INFO:tasks.workunit.client.1.smithi152.stdout:1/884: chown d5/d7/d8/d13/f35 110654709 1 2024-03-29T04:45:21.572 INFO:tasks.workunit.client.1.smithi152.stdout:2/517: write d0/d3/d30/d15/f2b [1138933,53837] 0 2024-03-29T04:45:21.572 INFO:tasks.workunit.client.1.smithi152.stdout:6/791: write d1/d3/d9/d13/f46 [874399,97834] 0 2024-03-29T04:45:21.573 INFO:tasks.workunit.client.1.smithi152.stdout:9/591: unlink d5/d8/d31/d59/d6f/c84 0 2024-03-29T04:45:21.575 INFO:tasks.workunit.client.1.smithi152.stdout:9/592: fsync d5/d8/d31/d75/db1/dcd/f50 0 2024-03-29T04:45:21.575 INFO:tasks.workunit.client.1.smithi152.stdout:1/885: mkdir d5/d7/d1b/d2e/d95/dad/d12/d77/d11c 0 2024-03-29T04:45:21.577 INFO:tasks.workunit.client.1.smithi152.stdout:6/792: mkdir d1/d3/d1a/d1b/d29/d6f/dee/d10f 0 2024-03-29T04:45:21.578 INFO:tasks.workunit.client.1.smithi152.stdout:6/793: chown d1/d3/d1a/fb9 27549019 1 2024-03-29T04:45:21.578 INFO:tasks.workunit.client.1.smithi152.stdout:2/518: getdents d0/d3/d64 0 2024-03-29T04:45:21.579 INFO:tasks.workunit.client.1.smithi152.stdout:1/886: creat d5/d7/d1b/d2e/d95/df1/dfe/f11d x:0 0 0 2024-03-29T04:45:21.582 INFO:tasks.workunit.client.1.smithi152.stdout:1/887: mkdir d5/d7/d4d/db1/d111/d11e 0 2024-03-29T04:45:21.582 INFO:tasks.workunit.client.1.smithi152.stdout:2/519: write d0/d3/d30/d15/d22/d4c/f4d [6284158,46928] 0 2024-03-29T04:45:21.585 INFO:tasks.workunit.client.1.smithi152.stdout:2/520: chown d0/d3/d30/f1a 6868 1 2024-03-29T04:45:21.585 INFO:tasks.workunit.client.1.smithi152.stdout:6/794: dwrite d1/d3/d1a/f2c [0,4194304] 0 2024-03-29T04:45:21.587 INFO:tasks.workunit.client.1.smithi152.stdout:2/521: write d0/f3b [952650,125802] 0 2024-03-29T04:45:21.588 INFO:tasks.workunit.client.1.smithi152.stdout:2/522: readlink d0/d3/l6c 0 2024-03-29T04:45:21.588 INFO:tasks.workunit.client.1.smithi152.stdout:6/795: write d1/d3/d9/d13/f46 [4764595,103785] 0 2024-03-29T04:45:21.591 INFO:tasks.workunit.client.1.smithi152.stdout:1/888: rename d5/d7/d1b/d2e/d95/dad/l9a to d5/d7/d8/d13/d19/d80/d119/l11f 0 2024-03-29T04:45:21.592 INFO:tasks.workunit.client.1.smithi152.stdout:6/796: truncate d1/d3/d1a/d1b/f1e 776986 0 2024-03-29T04:45:21.594 INFO:tasks.workunit.client.1.smithi152.stdout:1/889: creat d5/f120 x:0 0 0 2024-03-29T04:45:21.594 INFO:tasks.workunit.client.1.smithi152.stdout:2/523: getdents d0/d3/d94 0 2024-03-29T04:45:21.596 INFO:tasks.workunit.client.1.smithi152.stdout:2/524: write d0/d3/d64/f6e [674074,41446] 0 2024-03-29T04:45:21.598 INFO:tasks.workunit.client.1.smithi152.stdout:1/890: rename d5/d7/d4d/dd4 to d5/d7/d4d/db1/d121 0 2024-03-29T04:45:21.601 INFO:tasks.workunit.client.1.smithi152.stdout:1/891: mkdir d5/d7/d4d/db1/d122 0 2024-03-29T04:45:21.603 INFO:tasks.workunit.client.1.smithi152.stdout:2/525: dwrite d0/f3b [0,4194304] 0 2024-03-29T04:45:21.604 INFO:tasks.workunit.client.1.smithi152.stdout:2/526: chown d0/d3/d30/f33 676504 1 2024-03-29T04:45:21.607 INFO:tasks.workunit.client.1.smithi152.stdout:1/892: link d5/d7/d1b/d2e/f5f d5/d7/d1b/d2e/d95/df1/f123 0 2024-03-29T04:45:21.610 INFO:tasks.workunit.client.1.smithi152.stdout:1/893: rmdir d5/d7/d1b/d2e/d95/dad 39 2024-03-29T04:45:21.616 INFO:tasks.workunit.client.1.smithi152.stdout:1/894: creat d5/d7/d1b/d2e/de2/d102/d113/f124 x:0 0 0 2024-03-29T04:45:21.617 INFO:tasks.workunit.client.1.smithi152.stdout:6/797: write d1/d3/d1a/d1b/d29/ff1 [184718,33339] 0 2024-03-29T04:45:21.620 INFO:tasks.workunit.client.1.smithi152.stdout:6/798: mkdir d1/d3/d9/d13/d110 0 2024-03-29T04:45:21.622 INFO:tasks.workunit.client.1.smithi152.stdout:6/799: creat d1/d3/d1a/d1b/d29/dce/f111 x:0 0 0 2024-03-29T04:45:21.624 INFO:tasks.workunit.client.1.smithi152.stdout:2/527: dread d0/d3/f1d [0,4194304] 0 2024-03-29T04:45:21.625 INFO:tasks.workunit.client.1.smithi152.stdout:1/895: creat d5/d7/d1b/d2e/d95/dad/d12/d77/d11c/f125 x:0 0 0 2024-03-29T04:45:21.626 INFO:tasks.workunit.client.1.smithi152.stdout:2/528: chown d0/d3/d5f/c9b 14 1 2024-03-29T04:45:21.629 INFO:tasks.workunit.client.1.smithi152.stdout:2/529: mknod d0/d3/d30/d15/d22/d4c/d5a/ca0 0 2024-03-29T04:45:21.629 INFO:tasks.workunit.client.1.smithi152.stdout:6/800: dread d1/d3/d1a/d1b/d29/fc4 [0,4194304] 0 2024-03-29T04:45:21.630 INFO:tasks.workunit.client.1.smithi152.stdout:6/801: dread - d1/d3/f96 zero size 2024-03-29T04:45:21.631 INFO:tasks.workunit.client.1.smithi152.stdout:2/530: write d0/d3/d30/d15/d22/d4c/d5a/f6b [124483,110195] 0 2024-03-29T04:45:21.634 INFO:tasks.workunit.client.1.smithi152.stdout:2/531: getdents d0/d3/df/d9e 0 2024-03-29T04:45:21.634 INFO:tasks.workunit.client.1.smithi152.stdout:6/802: dread d1/d3/d9/d13/d3e/d93/ffc [0,4194304] 0 2024-03-29T04:45:21.636 INFO:tasks.workunit.client.1.smithi152.stdout:2/532: fdatasync d0/d3/df/d2c/f69 0 2024-03-29T04:45:21.639 INFO:tasks.workunit.client.1.smithi152.stdout:2/533: mknod d0/d3/df/d8c/ca1 0 2024-03-29T04:45:21.641 INFO:tasks.workunit.client.1.smithi152.stdout:6/803: dwrite d1/f9b [0,4194304] 0 2024-03-29T04:45:21.643 INFO:tasks.workunit.client.1.smithi152.stdout:2/534: creat d0/d3/df/fa2 x:0 0 0 2024-03-29T04:45:21.648 INFO:tasks.workunit.client.1.smithi152.stdout:2/535: truncate d0/f44 4008704 0 2024-03-29T04:45:21.649 INFO:tasks.workunit.client.1.smithi152.stdout:6/804: dwrite d1/d3/d9/d13/f33 [0,4194304] 0 2024-03-29T04:45:21.650 INFO:tasks.workunit.client.1.smithi152.stdout:2/536: chown d0/d3/d30/d15/d22/d4c/d52/f95 117301 1 2024-03-29T04:45:21.651 INFO:tasks.workunit.client.1.smithi152.stdout:6/805: readlink d1/d3/d9/d13/l60 0 2024-03-29T04:45:21.654 INFO:tasks.workunit.client.1.smithi152.stdout:2/537: mknod d0/d3/d30/d15/d22/d4c/d52/ca3 0 2024-03-29T04:45:21.658 INFO:tasks.workunit.client.1.smithi152.stdout:2/538: creat d0/d3/d30/d15/d22/d4c/d5a/fa4 x:0 0 0 2024-03-29T04:45:21.660 INFO:tasks.workunit.client.1.smithi152.stdout:2/539: fsync d0/d3/d30/d15/f29 0 2024-03-29T04:45:21.660 INFO:tasks.workunit.client.1.smithi152.stdout:6/806: dwrite d1/d3/d9/d13/d3e/d54/f7c [0,4194304] 0 2024-03-29T04:45:21.663 INFO:tasks.workunit.client.1.smithi152.stdout:6/807: mkdir d1/d3/d9/d13/dc0/dc3/dfb/d112 0 2024-03-29T04:45:21.663 INFO:tasks.workunit.client.1.smithi152.stdout:2/540: write d0/d3/d30/d15/d22/d4c/d5a/f6b [469928,57063] 0 2024-03-29T04:45:21.665 INFO:tasks.workunit.client.1.smithi152.stdout:6/808: chown d1/d3/d9/d13/da7 29377 1 2024-03-29T04:45:21.667 INFO:tasks.workunit.client.1.smithi152.stdout:6/809: write d1/d3/d1a/d1b/d29/fc4 [1642992,114046] 0 2024-03-29T04:45:21.668 INFO:tasks.workunit.client.1.smithi152.stdout:2/541: mkdir d0/d3/d30/d15/da5 0 2024-03-29T04:45:21.671 INFO:tasks.workunit.client.1.smithi152.stdout:2/542: creat d0/d3/df/d2c/fa6 x:0 0 0 2024-03-29T04:45:21.673 INFO:tasks.workunit.client.1.smithi152.stdout:6/810: rename d1/d3/df/fff to d1/d3/d1a/d1b/d29/d6f/dee/d10f/f113 0 2024-03-29T04:45:21.674 INFO:tasks.workunit.client.1.smithi152.stdout:6/811: stat d1/ca5 0 2024-03-29T04:45:21.675 INFO:tasks.workunit.client.1.smithi152.stdout:2/543: dread d0/d3/df/f35 [4194304,4194304] 0 2024-03-29T04:45:21.679 INFO:tasks.workunit.client.1.smithi152.stdout:6/812: rename d1/d3/d9/d13/da7/fe8 to d1/d3/d9/d13/d28/f114 0 2024-03-29T04:45:21.682 INFO:tasks.workunit.client.1.smithi152.stdout:6/813: symlink d1/d3/d1a/d1b/d87/d8f/d102/l115 0 2024-03-29T04:45:21.682 INFO:tasks.workunit.client.1.smithi152.stdout:2/544: dwrite d0/d3/df/f43 [0,4194304] 0 2024-03-29T04:45:21.683 INFO:tasks.workunit.client.1.smithi152.stdout:2/545: rmdir d0/d3 39 2024-03-29T04:45:21.685 INFO:tasks.workunit.client.1.smithi152.stdout:6/814: creat d1/d3/d1a/d1b/d10d/f116 x:0 0 0 2024-03-29T04:45:21.688 INFO:tasks.workunit.client.1.smithi152.stdout:6/815: chown d1/d3/d9/f21 112280 1 2024-03-29T04:45:21.688 INFO:tasks.workunit.client.1.smithi152.stdout:2/546: mkdir d0/d3/df/da7 0 2024-03-29T04:45:21.690 INFO:tasks.workunit.client.1.smithi152.stdout:6/816: dread - d1/d3/df/d84/f8d zero size 2024-03-29T04:45:21.691 INFO:tasks.workunit.client.1.smithi152.stdout:2/547: symlink d0/d3/d30/d15/d22/d4c/la8 0 2024-03-29T04:45:21.692 INFO:tasks.workunit.client.1.smithi152.stdout:2/548: dread - d0/d3/df/fa2 zero size 2024-03-29T04:45:21.699 INFO:tasks.workunit.client.1.smithi152.stdout:6/817: write d1/d3/d1a/f25 [1758595,107993] 0 2024-03-29T04:45:21.700 INFO:tasks.workunit.client.1.smithi152.stdout:2/549: dread d0/d3/df/d2c/f55 [0,4194304] 0 2024-03-29T04:45:21.701 INFO:tasks.workunit.client.1.smithi152.stdout:6/818: write d1/d3/d1a/d1b/d29/d6f/f69 [3310122,121677] 0 2024-03-29T04:45:21.702 INFO:tasks.workunit.client.1.smithi152.stdout:2/550: read d0/d3/d30/d15/d22/f99 [498586,49281] 0 2024-03-29T04:45:21.705 INFO:tasks.workunit.client.1.smithi152.stdout:6/819: symlink d1/d3/d9/d13/d28/d45/l117 0 2024-03-29T04:45:21.706 INFO:tasks.workunit.client.1.smithi152.stdout:2/551: dread d0/d3/d30/f19 [4194304,4194304] 0 2024-03-29T04:45:21.709 INFO:tasks.workunit.client.1.smithi152.stdout:6/820: getdents d1/d3/d1a/d1b/d87 0 2024-03-29T04:45:21.713 INFO:tasks.workunit.client.1.smithi152.stdout:6/821: read d1/d3/d1a/f2c [2467795,121371] 0 2024-03-29T04:45:21.713 INFO:tasks.workunit.client.1.smithi152.stdout:2/552: dwrite d0/d3/df/d2c/fa6 [0,4194304] 0 2024-03-29T04:45:21.715 INFO:tasks.workunit.client.1.smithi152.stdout:2/553: chown d0/d3/d5f/c9c 7007530 1 2024-03-29T04:45:21.721 INFO:tasks.workunit.client.1.smithi152.stdout:6/822: write d1/d3/f18 [601950,103858] 0 2024-03-29T04:45:21.721 INFO:tasks.workunit.client.1.smithi152.stdout:2/554: link d0/d3/d30/d15/d53/d54/d57/l61 d0/d3/df/d2c/la9 0 2024-03-29T04:45:21.725 INFO:tasks.workunit.client.1.smithi152.stdout:2/555: rmdir d0/d3/d30/d15/d22/d74 39 2024-03-29T04:45:21.732 INFO:tasks.workunit.client.1.smithi152.stdout:2/556: write d0/d3/d30/f3f [105190,53631] 0 2024-03-29T04:45:21.733 INFO:tasks.workunit.client.1.smithi152.stdout:6/823: dread d1/d3/f18 [0,4194304] 0 2024-03-29T04:45:21.736 INFO:tasks.workunit.client.1.smithi152.stdout:6/824: creat d1/d3/d1a/d1b/d87/d8f/d102/f118 x:0 0 0 2024-03-29T04:45:21.738 INFO:tasks.workunit.client.1.smithi152.stdout:6/825: write d1/d3/d1a/d1b/d29/fc4 [1503714,12384] 0 2024-03-29T04:45:21.742 INFO:tasks.workunit.client.1.smithi152.stdout:2/557: dread d0/d3/d30/f19 [0,4194304] 0 2024-03-29T04:45:21.742 INFO:tasks.workunit.client.1.smithi152.stdout:6/826: symlink d1/d3/d1a/d1b/d29/dce/dd7/l119 0 2024-03-29T04:45:21.744 INFO:tasks.workunit.client.1.smithi152.stdout:2/558: creat d0/faa x:0 0 0 2024-03-29T04:45:21.748 INFO:tasks.workunit.client.1.smithi152.stdout:6/827: dread d1/d3/d9/f3a [0,4194304] 0 2024-03-29T04:45:21.750 INFO:tasks.workunit.client.1.smithi152.stdout:6/828: creat d1/d3/d1a/d1b/d29/dce/f11a x:0 0 0 2024-03-29T04:45:21.752 INFO:tasks.workunit.client.1.smithi152.stdout:2/559: dwrite d0/d3/d30/d15/d22/d4c/f51 [0,4194304] 0 2024-03-29T04:45:21.754 INFO:tasks.workunit.client.1.smithi152.stdout:6/829: creat d1/d3/d9/d13/da7/df5/dfd/f11b x:0 0 0 2024-03-29T04:45:21.756 INFO:tasks.workunit.client.1.smithi152.stdout:2/560: write d0/d3/d30/d15/d22/d74/f84 [923377,102626] 0 2024-03-29T04:45:21.756 INFO:tasks.workunit.client.1.smithi152.stdout:6/830: read d1/d3/d1a/f25 [577879,122121] 0 2024-03-29T04:45:21.761 INFO:tasks.workunit.client.1.smithi152.stdout:2/561: write d0/d3/d30/d15/d22/d4c/d52/f79 [502635,85982] 0 2024-03-29T04:45:21.762 INFO:tasks.workunit.client.1.smithi152.stdout:2/562: write d0/d3/d30/d15/d53/f80 [919800,55982] 0 2024-03-29T04:45:21.763 INFO:tasks.workunit.client.1.smithi152.stdout:2/563: chown d0/d3/c5c 0 1 2024-03-29T04:45:21.765 INFO:tasks.workunit.client.1.smithi152.stdout:2/564: write d0/d3/d30/d15/d22/f70 [140325,108903] 0 2024-03-29T04:45:21.772 INFO:tasks.workunit.client.1.smithi152.stdout:2/565: dwrite d0/d3/d30/d15/d22/f70 [0,4194304] 0 2024-03-29T04:45:21.772 INFO:tasks.workunit.client.1.smithi152.stdout:2/566: chown d0/f3b 1 1 2024-03-29T04:45:21.776 INFO:tasks.workunit.client.1.smithi152.stdout:2/567: read d0/d3/d30/d15/d22/d74/f84 [922078,67231] 0 2024-03-29T04:45:21.779 INFO:tasks.workunit.client.1.smithi152.stdout:2/568: chown d0/d3/d30/d15/d53/d54/d57/l61 22139 1 2024-03-29T04:45:21.784 INFO:tasks.workunit.client.1.smithi152.stdout:2/569: dread d0/d3/d30/d15/d22/f70 [0,4194304] 0 2024-03-29T04:45:21.790 INFO:tasks.workunit.client.1.smithi152.stdout:6/831: dwrite d1/d3/d1a/d1b/d29/f7e [0,4194304] 0 2024-03-29T04:45:21.791 INFO:tasks.workunit.client.1.smithi152.stdout:6/832: chown d1/d3/df/d8a/fe4 2604 1 2024-03-29T04:45:21.795 INFO:tasks.workunit.client.1.smithi152.stdout:6/833: creat d1/d3/d1a/d1b/d29/d6f/dee/f11c x:0 0 0 2024-03-29T04:45:21.823 INFO:tasks.workunit.client.1.smithi152.stdout:2/570: dread d0/d3/f31 [0,4194304] 0 2024-03-29T04:45:21.824 INFO:tasks.workunit.client.1.smithi152.stdout:2/571: chown d0/d3/d30/d15/f29 4758 1 2024-03-29T04:45:21.826 INFO:tasks.workunit.client.1.smithi152.stdout:2/572: chown d0/d3/d30/d15/d22/d4c/f89 10186970 1 2024-03-29T04:45:21.827 INFO:tasks.workunit.client.1.smithi152.stdout:2/573: fsync d0/f3b 0 2024-03-29T04:45:21.832 INFO:tasks.workunit.client.1.smithi152.stdout:2/574: getdents d0/d3/df/d2c/d45 0 2024-03-29T04:45:21.836 INFO:tasks.workunit.client.1.smithi152.stdout:2/575: write d0/d3/f31 [3788947,73771] 0 2024-03-29T04:45:21.841 INFO:tasks.workunit.client.1.smithi152.stdout:2/576: getdents d0/d3/d30/d15 0 2024-03-29T04:45:21.849 INFO:tasks.workunit.client.1.smithi152.stdout:2/577: dwrite d0/d3/df/f43 [0,4194304] 0 2024-03-29T04:45:21.850 INFO:tasks.workunit.client.1.smithi152.stdout:2/578: chown d0/f44 96 1 2024-03-29T04:45:21.851 INFO:tasks.workunit.client.1.smithi152.stdout:2/579: readlink d0/d3/d64/l93 0 2024-03-29T04:45:21.852 INFO:tasks.workunit.client.1.smithi152.stdout:2/580: fdatasync d0/faa 0 2024-03-29T04:45:21.855 INFO:tasks.workunit.client.1.smithi152.stdout:2/581: truncate d0/d3/d30/d15/d53/d54/d57/f72 1190527 0 2024-03-29T04:45:21.857 INFO:tasks.workunit.client.1.smithi152.stdout:2/582: fsync d0/d3/d30/d15/d22/d4c/d52/f79 0 2024-03-29T04:45:21.860 INFO:tasks.workunit.client.1.smithi152.stdout:2/583: symlink d0/d3/df/d2c/d45/lab 0 2024-03-29T04:45:21.861 INFO:tasks.workunit.client.1.smithi152.stdout:2/584: write d0/d3/d30/d15/d53/f80 [277122,93722] 0 2024-03-29T04:45:21.867 INFO:tasks.workunit.client.1.smithi152.stdout:2/585: rmdir d0/d3/d64/d8d 0 2024-03-29T04:45:21.868 INFO:tasks.workunit.client.1.smithi152.stdout:2/586: mknod d0/d3/df/cac 0 2024-03-29T04:45:21.871 INFO:tasks.workunit.client.1.smithi152.stdout:2/587: mkdir d0/d3/d30/d15/d53/d54/dad 0 2024-03-29T04:45:21.876 INFO:tasks.workunit.client.1.smithi152.stdout:2/588: rename d0/d3/c6 to d0/d3/d30/d15/da5/cae 0 2024-03-29T04:45:21.881 INFO:tasks.workunit.client.1.smithi152.stdout:2/589: rename d0/d3/d30/d15/d22/d74/f84 to d0/d3/df/da7/faf 0 2024-03-29T04:45:21.883 INFO:tasks.workunit.client.1.smithi152.stdout:2/590: mknod d0/d3/df/d9e/cb0 0 2024-03-29T04:45:21.885 INFO:tasks.workunit.client.1.smithi152.stdout:2/591: dread d0/d3/df/f76 [0,4194304] 0 2024-03-29T04:45:21.887 INFO:tasks.workunit.client.1.smithi152.stdout:2/592: write d0/d3/df/d2c/f2d [1907,36138] 0 2024-03-29T04:45:21.889 INFO:tasks.workunit.client.1.smithi152.stdout:2/593: write d0/d3/d30/d15/d22/d4c/d52/f82 [3755500,85792] 0 2024-03-29T04:45:21.897 INFO:tasks.workunit.client.1.smithi152.stdout:2/594: dwrite d0/d3/d5f/f92 [0,4194304] 0 2024-03-29T04:45:21.901 INFO:tasks.workunit.client.1.smithi152.stdout:2/595: dread d0/d3/f81 [0,4194304] 0 2024-03-29T04:45:21.904 INFO:tasks.workunit.client.1.smithi152.stdout:2/596: creat d0/d3/df/d8c/fb1 x:0 0 0 2024-03-29T04:45:21.906 INFO:tasks.workunit.client.1.smithi152.stdout:2/597: readlink d0/d3/d30/d15/d53/l66 0 2024-03-29T04:45:21.907 INFO:tasks.workunit.client.1.smithi152.stdout:2/598: stat d0/d3/ld 0 2024-03-29T04:45:21.909 INFO:tasks.workunit.client.1.smithi152.stdout:2/599: write d0/d3/df/f76 [1497945,89729] 0 2024-03-29T04:45:21.912 INFO:tasks.workunit.client.1.smithi152.stdout:2/600: creat d0/d3/d64/d85/d9d/fb2 x:0 0 0 2024-03-29T04:45:22.029 INFO:tasks.workunit.client.1.smithi152.stdout:0/570: sync 2024-03-29T04:45:22.029 INFO:tasks.workunit.client.1.smithi152.stdout:7/526: sync 2024-03-29T04:45:22.030 INFO:tasks.workunit.client.1.smithi152.stdout:0/571: write d2/da/d17/f74 [2083944,105289] 0 2024-03-29T04:45:22.035 INFO:tasks.workunit.client.1.smithi152.stdout:7/527: creat d7/d8/d12/f9f x:0 0 0 2024-03-29T04:45:22.040 INFO:tasks.workunit.client.1.smithi152.stdout:7/528: truncate d7/d8/d12/d1d/f62 2013870 0 2024-03-29T04:45:22.042 INFO:tasks.workunit.client.1.smithi152.stdout:7/529: truncate d7/d8/d12/f6f 4499495 0 2024-03-29T04:45:22.042 INFO:tasks.workunit.client.1.smithi152.stdout:7/530: stat d7 0 2024-03-29T04:45:22.045 INFO:tasks.workunit.client.1.smithi152.stdout:7/531: truncate d7/d8/d37/f66 970473 0 2024-03-29T04:45:22.045 INFO:tasks.workunit.client.1.smithi152.stdout:0/572: dwrite d2/d29/f37 [0,4194304] 0 2024-03-29T04:45:22.047 INFO:tasks.workunit.client.1.smithi152.stdout:7/532: symlink d7/la0 0 2024-03-29T04:45:22.050 INFO:tasks.workunit.client.1.smithi152.stdout:7/533: symlink d7/d8/d12/d1d/d41/la1 0 2024-03-29T04:45:22.052 INFO:tasks.workunit.client.1.smithi152.stdout:0/573: write d2/da/d3a/d61/f7f [673450,25943] 0 2024-03-29T04:45:22.053 INFO:tasks.workunit.client.1.smithi152.stdout:7/534: read d7/d8/d12/d35/d68/f88 [67293,114516] 0 2024-03-29T04:45:22.065 INFO:tasks.workunit.client.1.smithi152.stdout:7/535: dwrite d7/f92 [0,4194304] 0 2024-03-29T04:45:22.075 INFO:tasks.workunit.client.1.smithi152.stdout:7/536: dwrite d7/d8/d12/d35/d68/f88 [0,4194304] 0 2024-03-29T04:45:22.076 INFO:tasks.workunit.client.1.smithi152.stdout:7/537: dread - d7/d8/d12/d1d/d41/d83/d9a/f90 zero size 2024-03-29T04:45:22.077 INFO:tasks.workunit.client.1.smithi152.stdout:7/538: stat d7/d8/d12/d35/c44 0 2024-03-29T04:45:22.083 INFO:tasks.workunit.client.1.smithi152.stdout:0/574: dread d2/da/df/f93 [0,4194304] 0 2024-03-29T04:45:22.084 INFO:tasks.workunit.client.1.smithi152.stdout:0/575: write d2/da/d17/d57/f62 [1060448,72110] 0 2024-03-29T04:45:22.085 INFO:tasks.workunit.client.1.smithi152.stdout:7/539: truncate d7/d8/d12/d35/f1a 3353912 0 2024-03-29T04:45:22.086 INFO:tasks.workunit.client.1.smithi152.stdout:7/540: chown d7/d8/d12/d1d/l26 191329247 1 2024-03-29T04:45:22.087 INFO:tasks.workunit.client.1.smithi152.stdout:0/576: creat d2/da/d17/db5/fd4 x:0 0 0 2024-03-29T04:45:22.093 INFO:tasks.workunit.client.1.smithi152.stdout:7/541: write d7/d8/d12/d1d/f62 [866330,14430] 0 2024-03-29T04:45:22.094 INFO:tasks.workunit.client.1.smithi152.stdout:7/542: dread - d7/d8/d12/d1d/d41/d83/d9a/f97 zero size 2024-03-29T04:45:22.099 INFO:tasks.workunit.client.1.smithi152.stdout:7/543: write d7/d8/d12/d1d/d41/d45/f70 [4422331,119701] 0 2024-03-29T04:45:22.102 INFO:tasks.workunit.client.1.smithi152.stdout:0/577: dread d2/f38 [0,4194304] 0 2024-03-29T04:45:22.104 INFO:tasks.workunit.client.1.smithi152.stdout:0/578: creat d2/da/df/fd5 x:0 0 0 2024-03-29T04:45:22.104 INFO:tasks.workunit.client.1.smithi152.stdout:7/544: write d7/d8/f3b [1477974,75954] 0 2024-03-29T04:45:22.108 INFO:tasks.workunit.client.1.smithi152.stdout:0/579: rename d2/da/df/fa1 to d2/da/d3a/d61/fd6 0 2024-03-29T04:45:22.108 INFO:tasks.workunit.client.1.smithi152.stdout:0/580: dread - d2/da/d3a/d61/fd0 zero size 2024-03-29T04:45:22.109 INFO:tasks.workunit.client.1.smithi152.stdout:7/545: unlink d7/d8/d12/d35/d68/l71 0 2024-03-29T04:45:22.111 INFO:tasks.workunit.client.1.smithi152.stdout:0/581: creat d2/da/df/d86/db8/daf/fd7 x:0 0 0 2024-03-29T04:45:22.111 INFO:tasks.workunit.client.1.smithi152.stdout:7/546: mknod d7/d8/d12/d35/d68/ca2 0 2024-03-29T04:45:22.113 INFO:tasks.workunit.client.1.smithi152.stdout:7/547: write d7/d8/d12/d1d/d41/d45/f70 [1077646,39966] 0 2024-03-29T04:45:22.117 INFO:tasks.workunit.client.1.smithi152.stdout:7/548: mknod d7/d8/db/ca3 0 2024-03-29T04:45:22.119 INFO:tasks.workunit.client.1.smithi152.stdout:7/549: dread - d7/d8/f6e zero size 2024-03-29T04:45:22.120 INFO:tasks.workunit.client.1.smithi152.stdout:0/582: rename d2/da/df/d86 to d2/da/d17/dd8 0 2024-03-29T04:45:22.126 INFO:tasks.workunit.client.1.smithi152.stdout:0/583: mknod d2/da/d17/d57/cd9 0 2024-03-29T04:45:22.126 INFO:tasks.workunit.client.1.smithi152.stdout:0/584: creat d2/d31/fda x:0 0 0 2024-03-29T04:45:22.126 INFO:tasks.workunit.client.1.smithi152.stdout:0/585: readlink d2/da/d17/l92 0 2024-03-29T04:45:22.129 INFO:tasks.workunit.client.1.smithi152.stdout:0/586: mkdir d2/da/d17/dd8/db8/daf/ddb 0 2024-03-29T04:45:22.129 INFO:tasks.workunit.client.1.smithi152.stdout:7/550: dwrite d7/d8/db/f4b [0,4194304] 0 2024-03-29T04:45:22.130 INFO:tasks.workunit.client.1.smithi152.stdout:0/587: chown d2/da/d17/d50/caa 1741 1 2024-03-29T04:45:22.133 INFO:tasks.workunit.client.1.smithi152.stdout:7/551: getdents d7/d8/d93 0 2024-03-29T04:45:22.135 INFO:tasks.workunit.client.1.smithi152.stdout:7/552: creat d7/d8/d12/d35/d68/d84/fa4 x:0 0 0 2024-03-29T04:45:22.140 INFO:tasks.workunit.client.1.smithi152.stdout:7/553: read d7/d8/f2d [1842462,110399] 0 2024-03-29T04:45:22.141 INFO:tasks.workunit.client.1.smithi152.stdout:5/640: sync 2024-03-29T04:45:22.141 INFO:tasks.workunit.client.1.smithi152.stdout:7/554: rmdir d7/d8/d12/d35 39 2024-03-29T04:45:22.142 INFO:tasks.workunit.client.1.smithi152.stdout:5/641: chown d6/d1d/daa/fd4 294307286 1 2024-03-29T04:45:22.145 INFO:tasks.workunit.client.1.smithi152.stdout:7/555: symlink d7/d8/d12/d35/d68/la5 0 2024-03-29T04:45:22.149 INFO:tasks.workunit.client.1.smithi152.stdout:5/642: dwrite d6/d16/f65 [4194304,4194304] 0 2024-03-29T04:45:22.150 INFO:tasks.workunit.client.1.smithi152.stdout:7/556: write d7/d8/d12/d1d/f1e [3763179,3595] 0 2024-03-29T04:45:22.151 INFO:tasks.workunit.client.1.smithi152.stdout:5/643: write d6/d7/f2d [2425460,79679] 0 2024-03-29T04:45:22.159 INFO:tasks.workunit.client.1.smithi152.stdout:5/644: dwrite d6/d1b/f91 [0,4194304] 0 2024-03-29T04:45:22.162 INFO:tasks.workunit.client.1.smithi152.stdout:5/645: fdatasync d6/d16/f73 0 2024-03-29T04:45:22.166 INFO:tasks.workunit.client.1.smithi152.stdout:5/646: mknod d6/d1d/daa/db7/d56/cd9 0 2024-03-29T04:45:22.168 INFO:tasks.workunit.client.1.smithi152.stdout:5/647: truncate d6/d1d/d3b/d55/fbe 627597 0 2024-03-29T04:45:22.170 INFO:tasks.workunit.client.1.smithi152.stdout:5/648: mkdir d6/d1d/daa/db7/dda 0 2024-03-29T04:45:22.172 INFO:tasks.workunit.client.1.smithi152.stdout:7/557: dread d7/d8/d12/f81 [0,4194304] 0 2024-03-29T04:45:22.174 INFO:tasks.workunit.client.1.smithi152.stdout:5/649: creat d6/d7/d75/d97/fdb x:0 0 0 2024-03-29T04:45:22.175 INFO:tasks.workunit.client.1.smithi152.stdout:5/650: write d6/d1d/d3b/d55/fe [607049,123513] 0 2024-03-29T04:45:22.177 INFO:tasks.workunit.client.1.smithi152.stdout:7/558: creat d7/d8/d12/d1d/d41/d83/d9a/fa6 x:0 0 0 2024-03-29T04:45:22.179 INFO:tasks.workunit.client.1.smithi152.stdout:5/651: creat d6/d16/d25/fdc x:0 0 0 2024-03-29T04:45:22.180 INFO:tasks.workunit.client.1.smithi152.stdout:7/559: mknod d7/d8/d12/d35/d68/ca7 0 2024-03-29T04:45:22.180 INFO:tasks.workunit.client.1.smithi152.stdout:5/652: dread - d6/d1d/daa/db7/d56/dc9/fd8 zero size 2024-03-29T04:45:22.186 INFO:tasks.workunit.client.1.smithi152.stdout:5/653: rename d6/d16/f65 to d6/d16/d25/d2e/d9d/fdd 0 2024-03-29T04:45:22.189 INFO:tasks.workunit.client.1.smithi152.stdout:5/654: symlink d6/d1d/lde 0 2024-03-29T04:45:22.191 INFO:tasks.workunit.client.1.smithi152.stdout:7/560: getdents d7/d8/d12/d35/d68 0 2024-03-29T04:45:22.196 INFO:tasks.workunit.client.1.smithi152.stdout:7/561: rename d7/d8/d12/d1d/d41/d83/d9a/l65 to d7/d8/d37/la8 0 2024-03-29T04:45:22.197 INFO:tasks.workunit.client.1.smithi152.stdout:5/655: truncate d6/d16/d25/f9f 799101 0 2024-03-29T04:45:22.198 INFO:tasks.workunit.client.1.smithi152.stdout:7/562: link d7/d8/d12/f6f d7/d8/d93/fa9 0 2024-03-29T04:45:22.199 INFO:tasks.workunit.client.1.smithi152.stdout:7/563: readlink d7/d8/d12/d35/l23 0 2024-03-29T04:45:22.199 INFO:tasks.workunit.client.1.smithi152.stdout:5/656: rename d6/d1d/daa/db7/d56/ccc to d6/d16/d25/d2e/da0/d92/cdf 0 2024-03-29T04:45:22.200 INFO:tasks.workunit.client.1.smithi152.stdout:7/564: write d7/d8/d12/d35/d68/d84/fa4 [148826,52916] 0 2024-03-29T04:45:22.202 INFO:tasks.workunit.client.1.smithi152.stdout:5/657: creat d6/fe0 x:0 0 0 2024-03-29T04:45:22.203 INFO:tasks.workunit.client.1.smithi152.stdout:7/565: truncate d7/d8/f2d 5025266 0 2024-03-29T04:45:22.205 INFO:tasks.workunit.client.1.smithi152.stdout:7/566: creat d7/d8/d12/d35/d68/faa x:0 0 0 2024-03-29T04:45:22.205 INFO:tasks.workunit.client.1.smithi152.stdout:5/658: rmdir d6/d1d/daa/db7/d56/d63 39 2024-03-29T04:45:22.206 INFO:tasks.workunit.client.1.smithi152.stdout:7/567: write d7/d8/db/f4b [3106340,86173] 0 2024-03-29T04:45:22.209 INFO:tasks.workunit.client.1.smithi152.stdout:7/568: chown d7/d8/d12/d1d/d41/d83/d9a/d72/f79 27751 1 2024-03-29T04:45:22.212 INFO:tasks.workunit.client.1.smithi152.stdout:7/569: mknod d7/d8/d37/cab 0 2024-03-29T04:45:22.213 INFO:tasks.workunit.client.1.smithi152.stdout:5/659: dread d6/d16/f4d [0,4194304] 0 2024-03-29T04:45:22.214 INFO:tasks.workunit.client.1.smithi152.stdout:7/570: readlink d7/d8/l5a 0 2024-03-29T04:45:22.215 INFO:tasks.workunit.client.1.smithi152.stdout:7/571: chown d7/d8/d12/d35/c44 126086 1 2024-03-29T04:45:22.215 INFO:tasks.workunit.client.1.smithi152.stdout:5/660: creat d6/d1d/d30/d6a/fe1 x:0 0 0 2024-03-29T04:45:22.216 INFO:tasks.workunit.client.1.smithi152.stdout:5/661: dread - d6/d1d/d30/d6a/faf zero size 2024-03-29T04:45:22.218 INFO:tasks.workunit.client.1.smithi152.stdout:7/572: mknod d7/d8/d12/d1d/d41/d83/d9a/cac 0 2024-03-29T04:45:22.218 INFO:tasks.workunit.client.1.smithi152.stdout:5/662: chown d6/d1d/daa/db7/d56/d63/fb1 3884 1 2024-03-29T04:45:22.220 INFO:tasks.workunit.client.1.smithi152.stdout:5/663: rmdir d6/d7 39 2024-03-29T04:45:22.221 INFO:tasks.workunit.client.1.smithi152.stdout:5/664: dread - d6/d1d/daa/db7/fa1 zero size 2024-03-29T04:45:22.222 INFO:tasks.workunit.client.1.smithi152.stdout:5/665: fsync d6/d1d/daa/db7/f61 0 2024-03-29T04:45:22.224 INFO:tasks.workunit.client.1.smithi152.stdout:5/666: symlink d6/d16/d25/d2e/da0/d92/le2 0 2024-03-29T04:45:22.224 INFO:tasks.workunit.client.1.smithi152.stdout:5/667: truncate d6/d1b/fcf 897115 0 2024-03-29T04:45:22.226 INFO:tasks.workunit.client.1.smithi152.stdout:5/668: chown d6/d16/d25/f9f 382126 1 2024-03-29T04:45:22.228 INFO:tasks.workunit.client.1.smithi152.stdout:5/669: symlink d6/d7/d75/d97/le3 0 2024-03-29T04:45:22.231 INFO:tasks.workunit.client.1.smithi152.stdout:5/670: unlink d6/d16/d25/d2e/da0/d92/cdf 0 2024-03-29T04:45:22.233 INFO:tasks.workunit.client.1.smithi152.stdout:5/671: mkdir d6/d1d/daa/db7/d56/dc1/de4 0 2024-03-29T04:45:22.238 INFO:tasks.workunit.client.1.smithi152.stdout:5/672: rename d6/d7/d75/d97 to d6/d1d/daa/de5 0 2024-03-29T04:45:22.238 INFO:tasks.workunit.client.1.smithi152.stdout:5/673: fsync d6/d16/d25/f3a 0 2024-03-29T04:45:22.239 INFO:tasks.workunit.client.1.smithi152.stdout:5/674: stat d6/d1b/fd1 0 2024-03-29T04:45:22.241 INFO:tasks.workunit.client.1.smithi152.stdout:5/675: creat d6/d1d/d3b/d55/fe6 x:0 0 0 2024-03-29T04:45:22.248 INFO:tasks.workunit.client.1.smithi152.stdout:5/676: dwrite d6/d1d/d3b/d55/f7a [0,4194304] 0 2024-03-29T04:45:22.250 INFO:tasks.workunit.client.1.smithi152.stdout:5/677: mkdir d6/d7/d75/de7 0 2024-03-29T04:45:22.253 INFO:tasks.workunit.client.1.smithi152.stdout:5/678: mknod d6/d16/d25/d2e/da0/d92/dc0/ce8 0 2024-03-29T04:45:22.254 INFO:tasks.workunit.client.1.smithi152.stdout:5/679: write d6/d1d/d3b/f39 [65120,24359] 0 2024-03-29T04:45:22.255 INFO:tasks.workunit.client.1.smithi152.stdout:5/680: write d6/d16/d25/f3a [1205259,65581] 0 2024-03-29T04:45:22.256 INFO:tasks.workunit.client.1.smithi152.stdout:5/681: chown d6/d16/d25/d2e/da0/lb5 147 1 2024-03-29T04:45:22.265 INFO:tasks.workunit.client.1.smithi152.stdout:5/682: write d6/d7/fa4 [2909592,5929] 0 2024-03-29T04:45:22.266 INFO:tasks.workunit.client.1.smithi152.stdout:5/683: write d6/d16/d25/d2e/f8c [1927558,17904] 0 2024-03-29T04:45:22.270 INFO:tasks.workunit.client.1.smithi152.stdout:5/684: creat d6/d1d/daa/db7/dda/fe9 x:0 0 0 2024-03-29T04:45:22.273 INFO:tasks.workunit.client.1.smithi152.stdout:5/685: mknod d6/d1d/daa/db7/d56/dc1/de4/cea 0 2024-03-29T04:45:22.275 INFO:tasks.workunit.client.1.smithi152.stdout:5/686: creat d6/d16/db3/feb x:0 0 0 2024-03-29T04:45:22.275 INFO:tasks.workunit.client.1.smithi152.stdout:5/687: chown d6/d16/d25 376338039 1 2024-03-29T04:45:22.276 INFO:tasks.workunit.client.1.smithi152.stdout:5/688: write d6/d16/d25/d2e/da0/d92/fc6 [62115,49490] 0 2024-03-29T04:45:22.279 INFO:tasks.workunit.client.1.smithi152.stdout:5/689: mkdir d6/d16/d25/d2e/da0/d92/dc0/dec 0 2024-03-29T04:45:22.281 INFO:tasks.workunit.client.1.smithi152.stdout:5/690: mknod d6/d16/d25/d2e/da0/dab/ced 0 2024-03-29T04:45:22.290 INFO:tasks.workunit.client.1.smithi152.stdout:5/691: dwrite d6/d1d/daa/db7/d56/d63/f6d [0,4194304] 0 2024-03-29T04:45:22.292 INFO:tasks.workunit.client.1.smithi152.stdout:5/692: symlink d6/d1d/daa/db7/d56/d63/lee 0 2024-03-29T04:45:22.292 INFO:tasks.workunit.client.1.smithi152.stdout:5/693: stat d6/d16/d25/l57 0 2024-03-29T04:45:22.293 INFO:tasks.workunit.client.1.smithi152.stdout:5/694: readlink d6/d1d/daa/db7/l6b 0 2024-03-29T04:45:22.297 INFO:tasks.workunit.client.1.smithi152.stdout:5/695: creat d6/d16/d25/d2e/da0/d92/dc0/dec/fef x:0 0 0 2024-03-29T04:45:22.297 INFO:tasks.workunit.client.1.smithi152.stdout:5/696: fsync d6/d1d/d3b/f95 0 2024-03-29T04:45:22.298 INFO:tasks.workunit.client.1.smithi152.stdout:5/697: chown d6/d1d 516373088 1 2024-03-29T04:45:22.300 INFO:tasks.workunit.client.1.smithi152.stdout:5/698: mknod d6/d16/d25/d2e/dc7/cf0 0 2024-03-29T04:45:22.302 INFO:tasks.workunit.client.1.smithi152.stdout:5/699: symlink d6/d1d/daa/db7/d56/dc1/de4/lf1 0 2024-03-29T04:45:22.304 INFO:tasks.workunit.client.1.smithi152.stdout:5/700: creat d6/d1d/daa/db7/d56/d63/db9/ff2 x:0 0 0 2024-03-29T04:45:22.306 INFO:tasks.workunit.client.1.smithi152.stdout:5/701: mknod d6/d16/d25/d2e/da0/d92/cf3 0 2024-03-29T04:45:22.308 INFO:tasks.workunit.client.1.smithi152.stdout:5/702: symlink d6/d7/d75/lf4 0 2024-03-29T04:45:22.311 INFO:tasks.workunit.client.1.smithi152.stdout:5/703: link d6/d1d/d3b/l9b d6/d16/d25/d2e/da0/d92/lf5 0 2024-03-29T04:45:22.313 INFO:tasks.workunit.client.1.smithi152.stdout:5/704: symlink d6/d1d/d30/d6a/lf6 0 2024-03-29T04:45:22.314 INFO:tasks.workunit.client.1.smithi152.stdout:5/705: chown d6/d1d/daa/cb6 16380425 1 2024-03-29T04:45:22.316 INFO:tasks.workunit.client.1.smithi152.stdout:5/706: symlink d6/d16/db3/lf7 0 2024-03-29T04:45:22.321 INFO:tasks.workunit.client.1.smithi152.stdout:5/707: write d6/d16/f4d [1430828,4551] 0 2024-03-29T04:45:22.322 INFO:tasks.workunit.client.1.smithi152.stdout:5/708: readlink d6/d1b/lba 0 2024-03-29T04:45:22.325 INFO:tasks.workunit.client.1.smithi152.stdout:8/541: sync 2024-03-29T04:45:22.326 INFO:tasks.workunit.client.1.smithi152.stdout:5/709: write d6/f14 [3074167,120329] 0 2024-03-29T04:45:22.328 INFO:tasks.workunit.client.1.smithi152.stdout:5/710: unlink d6/d16/d25/l2a 0 2024-03-29T04:45:22.334 INFO:tasks.workunit.client.1.smithi152.stdout:5/711: fsync d6/d16/f73 0 2024-03-29T04:45:22.334 INFO:tasks.workunit.client.1.smithi152.stdout:5/712: readlink d6/d1b/l29 0 2024-03-29T04:45:22.334 INFO:tasks.workunit.client.1.smithi152.stdout:5/713: truncate d6/d1d/daa/db7/d56/f94 1136161 0 2024-03-29T04:45:22.337 INFO:tasks.workunit.client.1.smithi152.stdout:5/714: mkdir d6/d16/df8 0 2024-03-29T04:45:22.339 INFO:tasks.workunit.client.1.smithi152.stdout:5/715: creat d6/d1d/d30/dc5/ff9 x:0 0 0 2024-03-29T04:45:22.340 INFO:tasks.workunit.client.1.smithi152.stdout:5/716: chown d6/d7 0 1 2024-03-29T04:45:22.343 INFO:tasks.workunit.client.1.smithi152.stdout:8/542: dread d1/d2/d1d/d36/d4b/da1/db0/d40/d6a/f89 [0,4194304] 0 2024-03-29T04:45:22.345 INFO:tasks.workunit.client.1.smithi152.stdout:5/717: dwrite d6/d1d/d3b/d55/f7a [0,4194304] 0 2024-03-29T04:45:22.346 INFO:tasks.workunit.client.1.smithi152.stdout:8/543: symlink d1/d2/d1d/d36/d4b/da1/dc2/lc3 0 2024-03-29T04:45:22.348 INFO:tasks.workunit.client.1.smithi152.stdout:5/718: chown d6/d1d/daa/db7/d56/d63/db9/cd5 234895 1 2024-03-29T04:45:22.351 INFO:tasks.workunit.client.1.smithi152.stdout:5/719: mknod d6/d16/cfa 0 2024-03-29T04:45:22.354 INFO:tasks.workunit.client.1.smithi152.stdout:5/720: mkdir d6/d1d/d30/d6a/dfb 0 2024-03-29T04:45:22.355 INFO:tasks.workunit.client.1.smithi152.stdout:8/544: dwrite d1/d2/d59/f88 [0,4194304] 0 2024-03-29T04:45:22.356 INFO:tasks.workunit.client.1.smithi152.stdout:5/721: write d6/d16/d25/d2e/da0/d92/fc6 [886935,646] 0 2024-03-29T04:45:22.359 INFO:tasks.workunit.client.1.smithi152.stdout:5/722: read d6/d1b/f45 [92178,36668] 0 2024-03-29T04:45:22.365 INFO:tasks.workunit.client.1.smithi152.stdout:8/545: dwrite d1/d2/d1d/d36/d4b/d51/f5e [0,4194304] 0 2024-03-29T04:45:22.367 INFO:tasks.workunit.client.1.smithi152.stdout:9/593: sync 2024-03-29T04:45:22.367 INFO:tasks.workunit.client.1.smithi152.stdout:4/551: sync 2024-03-29T04:45:22.367 INFO:tasks.workunit.client.1.smithi152.stdout:3/631: sync 2024-03-29T04:45:22.367 INFO:tasks.workunit.client.1.smithi152.stdout:1/896: sync 2024-03-29T04:45:22.369 INFO:tasks.workunit.client.1.smithi152.stdout:8/546: mknod d1/d2/d1d/d36/d4b/da1/db0/d40/d6a/cc4 0 2024-03-29T04:45:22.369 INFO:tasks.workunit.client.1.smithi152.stdout:4/552: stat d0/d7/d40/ff 0 2024-03-29T04:45:22.369 INFO:tasks.workunit.client.1.smithi152.stdout:5/723: link d6/d16/d25/c7f d6/d1d/d30/d6a/cfc 0 2024-03-29T04:45:22.372 INFO:tasks.workunit.client.1.smithi152.stdout:8/547: chown d1/d2/d1d/f41 2348204 1 2024-03-29T04:45:22.374 INFO:tasks.workunit.client.1.smithi152.stdout:5/724: readlink d6/d1d/d30/l33 0 2024-03-29T04:45:22.376 INFO:tasks.workunit.client.1.smithi152.stdout:3/632: creat d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/fc1 x:0 0 0 2024-03-29T04:45:22.378 INFO:tasks.workunit.client.1.smithi152.stdout:1/897: truncate d5/d7/d8/d13/d19/dc7/fdf 850072 0 2024-03-29T04:45:22.379 INFO:tasks.workunit.client.1.smithi152.stdout:5/725: mkdir d6/d16/db3/dfd 0 2024-03-29T04:45:22.379 INFO:tasks.workunit.client.1.smithi152.stdout:3/633: write d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/fc1 [199656,24915] 0 2024-03-29T04:45:22.381 INFO:tasks.workunit.client.1.smithi152.stdout:8/548: rename d1/d2/d1d/d36/d85/dbe to d1/dc5 0 2024-03-29T04:45:22.382 INFO:tasks.workunit.client.1.smithi152.stdout:8/549: stat d1/d2/d3/f96 0 2024-03-29T04:45:22.382 INFO:tasks.workunit.client.1.smithi152.stdout:1/898: symlink d5/d7/d1b/d10f/l126 0 2024-03-29T04:45:22.382 INFO:tasks.workunit.client.1.smithi152.stdout:3/634: write d7/fa4 [237998,9615] 0 2024-03-29T04:45:22.384 INFO:tasks.workunit.client.1.smithi152.stdout:5/726: rename d6/d7/d75/cc4 to d6/d1d/daa/db7/dda/cfe 0 2024-03-29T04:45:22.385 INFO:tasks.workunit.client.1.smithi152.stdout:3/635: chown d7/d6c/lba 196014328 1 2024-03-29T04:45:22.387 INFO:tasks.workunit.client.1.smithi152.stdout:1/899: mknod d5/d7/d8/d13/d19/c127 0 2024-03-29T04:45:22.390 INFO:tasks.workunit.client.1.smithi152.stdout:3/636: mknod d7/d19/d1b/d1e/cc2 0 2024-03-29T04:45:22.391 INFO:tasks.workunit.client.1.smithi152.stdout:1/900: symlink d5/d7/d8/da5/l128 0 2024-03-29T04:45:22.393 INFO:tasks.workunit.client.1.smithi152.stdout:8/550: dwrite d1/d2/d3/f44 [4194304,4194304] 0 2024-03-29T04:45:22.397 INFO:tasks.workunit.client.1.smithi152.stdout:5/727: dwrite d6/d7/fa4 [0,4194304] 0 2024-03-29T04:45:22.400 INFO:tasks.workunit.client.1.smithi152.stdout:1/901: chown d5/d7/d8/d13/d19/l1a 2751355 1 2024-03-29T04:45:22.402 INFO:tasks.workunit.client.1.smithi152.stdout:9/594: dwrite d5/f2f [0,4194304] 0 2024-03-29T04:45:22.405 INFO:tasks.workunit.client.1.smithi152.stdout:4/553: dwrite d0/d27/d6e/f97 [0,4194304] 0 2024-03-29T04:45:22.407 INFO:tasks.workunit.client.1.smithi152.stdout:8/551: truncate d1/d2/d3c/f7a 979935 0 2024-03-29T04:45:22.410 INFO:tasks.workunit.client.1.smithi152.stdout:1/902: creat d5/d7/d1b/d2e/de2/d102/f129 x:0 0 0 2024-03-29T04:45:22.413 INFO:tasks.workunit.client.1.smithi152.stdout:8/552: chown d1/d2/d1d/d36/d4b/l4d 165137822 1 2024-03-29T04:45:22.414 INFO:tasks.workunit.client.1.smithi152.stdout:4/554: chown d0/d27/d6e/d8c/lae 1 1 2024-03-29T04:45:22.415 INFO:tasks.workunit.client.1.smithi152.stdout:9/595: mkdir d5/d8/d31/d75/db1/dcd/d6e/d91/dd1 0 2024-03-29T04:45:22.417 INFO:tasks.workunit.client.1.smithi152.stdout:1/903: write d5/d7/d8/d13/d19/ff4 [4619897,48486] 0 2024-03-29T04:45:22.420 INFO:tasks.workunit.client.1.smithi152.stdout:8/553: mkdir d1/d2/d1d/dc6 0 2024-03-29T04:45:22.421 INFO:tasks.workunit.client.1.smithi152.stdout:9/596: write d5/d8/d31/f4c [2937610,78686] 0 2024-03-29T04:45:22.427 INFO:tasks.workunit.client.1.smithi152.stdout:4/555: link d0/d7/l2d d0/d7/d25/d17/d5e/d60/d74/da2/da6/dac/laf 0 2024-03-29T04:45:22.429 INFO:tasks.workunit.client.1.smithi152.stdout:1/904: creat d5/d7/d1b/d2e/d95/f12a x:0 0 0 2024-03-29T04:45:22.429 INFO:tasks.workunit.client.1.smithi152.stdout:8/554: dread d1/d2/d6d/f9c [0,4194304] 0 2024-03-29T04:45:22.429 INFO:tasks.workunit.client.1.smithi152.stdout:3/637: dwrite d7/d19/d1b/d1e/da1/d41/d61/d75/fbd [0,4194304] 0 2024-03-29T04:45:22.431 INFO:tasks.workunit.client.1.smithi152.stdout:4/556: write d0/d7/d40/ff [4730058,114105] 0 2024-03-29T04:45:22.432 INFO:tasks.workunit.client.1.smithi152.stdout:1/905: mkdir d5/d7/d8/da5/d12b 0 2024-03-29T04:45:22.448 INFO:tasks.workunit.client.1.smithi152.stdout:4/557: dwrite d0/d27/d6e/f97 [4194304,4194304] 0 2024-03-29T04:45:22.449 INFO:tasks.workunit.client.1.smithi152.stdout:3/638: dwrite d7/d19/d1b/d1e/da1/d41/d61/d75/fb7 [0,4194304] 0 2024-03-29T04:45:22.451 INFO:tasks.workunit.client.1.smithi152.stdout:3/639: dread - d7/d19/d1b/d1e/da1/d41/d6a/d78/f95 zero size 2024-03-29T04:45:22.452 INFO:tasks.workunit.client.1.smithi152.stdout:3/640: stat d7/d19/d1b/d1e/da1/l4b 0 2024-03-29T04:45:22.455 INFO:tasks.workunit.client.1.smithi152.stdout:3/641: chown d7/d19/d1b/d1e/da1/d41/d6a/d78/f95 23608 1 2024-03-29T04:45:22.458 INFO:tasks.workunit.client.1.smithi152.stdout:5/728: write d6/d16/d25/d2e/da0/f74 [228026,68282] 0 2024-03-29T04:45:22.464 INFO:tasks.workunit.client.1.smithi152.stdout:4/558: dread d0/d7/f8d [0,4194304] 0 2024-03-29T04:45:22.470 INFO:tasks.workunit.client.1.smithi152.stdout:8/555: truncate d1/d2/d1d/d36/d4b/d51/f5e 1393932 0 2024-03-29T04:45:22.471 INFO:tasks.workunit.client.1.smithi152.stdout:8/556: fdatasync d1/d2/d1d/f3d 0 2024-03-29T04:45:22.479 INFO:tasks.workunit.client.1.smithi152.stdout:1/906: write d5/d7/d1b/d2e/d95/d9d/f101 [1178642,16720] 0 2024-03-29T04:45:22.482 INFO:tasks.workunit.client.1.smithi152.stdout:5/729: unlink d6/d16/d25/d2e/l77 0 2024-03-29T04:45:22.484 INFO:tasks.workunit.client.1.smithi152.stdout:4/559: symlink d0/d7/d25/d17/d5e/d60/d74/d57/lb0 0 2024-03-29T04:45:22.485 INFO:tasks.workunit.client.1.smithi152.stdout:4/560: chown d0/d7/d25/d17/f71 328609 1 2024-03-29T04:45:22.486 INFO:tasks.workunit.client.1.smithi152.stdout:8/557: write d1/d2/d1d/d36/d4b/da1/db0/d40/d6a/f89 [2567937,24326] 0 2024-03-29T04:45:22.488 INFO:tasks.workunit.client.1.smithi152.stdout:3/642: symlink d7/lc3 0 2024-03-29T04:45:22.488 INFO:tasks.workunit.client.1.smithi152.stdout:8/558: read d1/d2/d3/f29 [640073,10769] 0 2024-03-29T04:45:22.494 INFO:tasks.workunit.client.1.smithi152.stdout:5/730: rename d6/d16/d25/d2e/l37 to d6/d1d/d3b/lff 0 2024-03-29T04:45:22.496 INFO:tasks.workunit.client.1.smithi152.stdout:4/561: symlink d0/d7/d40/d23/d55/d6c/d91/lb1 0 2024-03-29T04:45:22.498 INFO:tasks.workunit.client.1.smithi152.stdout:5/731: mknod d6/d1d/d30/d6a/c100 0 2024-03-29T04:45:22.500 INFO:tasks.workunit.client.1.smithi152.stdout:3/643: getdents d7/d19/d3a 0 2024-03-29T04:45:22.501 INFO:tasks.workunit.client.1.smithi152.stdout:5/732: dread - d6/d1d/d3b/d55/f89 zero size 2024-03-29T04:45:22.502 INFO:tasks.workunit.client.1.smithi152.stdout:5/733: stat d6/d1d/d30/d6a/l80 0 2024-03-29T04:45:22.503 INFO:tasks.workunit.client.1.smithi152.stdout:3/644: rename d7/d19/d1b/d1e/da1/d41/d61/c97 to d7/d19/d1b/d1e/da1/d41/d6a/cc4 0 2024-03-29T04:45:22.508 INFO:tasks.workunit.client.1.smithi152.stdout:3/645: rename d7/d19/d1b/d1e/da1/d41/c43 to d7/d46/cc5 0 2024-03-29T04:45:22.519 INFO:tasks.workunit.client.1.smithi152.stdout:5/734: rmdir d6/d1d 39 2024-03-29T04:45:22.519 INFO:tasks.workunit.client.1.smithi152.stdout:4/562: write d0/f13 [3056326,38560] 0 2024-03-29T04:45:22.522 INFO:tasks.workunit.client.1.smithi152.stdout:4/563: mkdir d0/d7/db2 0 2024-03-29T04:45:22.522 INFO:tasks.workunit.client.1.smithi152.stdout:5/735: fdatasync d6/d1d/d3b/d55/f66 0 2024-03-29T04:45:22.524 INFO:tasks.workunit.client.1.smithi152.stdout:5/736: fdatasync d6/d1d/daa/fd4 0 2024-03-29T04:45:22.524 INFO:tasks.workunit.client.1.smithi152.stdout:4/564: chown d0/d7/d40/d62/f9a 2 1 2024-03-29T04:45:22.525 INFO:tasks.workunit.client.1.smithi152.stdout:4/565: chown d0/d7 16577 1 2024-03-29T04:45:22.525 INFO:tasks.workunit.client.1.smithi152.stdout:3/646: dread d7/d19/d1b/f21 [0,4194304] 0 2024-03-29T04:45:22.525 INFO:tasks.workunit.client.1.smithi152.stdout:4/566: stat d0/c3d 0 2024-03-29T04:45:22.526 INFO:tasks.workunit.client.1.smithi152.stdout:4/567: write d0/d7/fab [285269,89649] 0 2024-03-29T04:45:22.530 INFO:tasks.workunit.client.1.smithi152.stdout:3/647: write d7/f1a [1494373,57636] 0 2024-03-29T04:45:22.533 INFO:tasks.workunit.client.1.smithi152.stdout:4/568: truncate d0/d7/d25/d17/f4d 1105159 0 2024-03-29T04:45:22.535 INFO:tasks.workunit.client.1.smithi152.stdout:3/648: getdents d7 0 2024-03-29T04:45:22.536 INFO:tasks.workunit.client.1.smithi152.stdout:5/737: dwrite d6/d1d/d3b/d55/fe [0,4194304] 0 2024-03-29T04:45:22.538 INFO:tasks.workunit.client.1.smithi152.stdout:6/834: sync 2024-03-29T04:45:22.539 INFO:tasks.workunit.client.1.smithi152.stdout:0/588: sync 2024-03-29T04:45:22.539 INFO:tasks.workunit.client.1.smithi152.stdout:2/601: sync 2024-03-29T04:45:22.539 INFO:tasks.workunit.client.1.smithi152.stdout:7/573: sync 2024-03-29T04:45:22.539 INFO:tasks.workunit.client.1.smithi152.stdout:4/569: unlink d0/d7/d25/d17/d5e/d60/d74/l4b 0 2024-03-29T04:45:22.541 INFO:tasks.workunit.client.1.smithi152.stdout:5/738: symlink d6/d1d/d30/dc5/l101 0 2024-03-29T04:45:22.543 INFO:tasks.workunit.client.1.smithi152.stdout:7/574: dread - d7/d8/d12/d1d/d41/d83/d9a/f8d zero size 2024-03-29T04:45:22.545 INFO:tasks.workunit.client.1.smithi152.stdout:0/589: mknod d2/da/d17/db5/cdc 0 2024-03-29T04:45:22.546 INFO:tasks.workunit.client.1.smithi152.stdout:2/602: mkdir d0/d3/d64/d85/db3 0 2024-03-29T04:45:22.546 INFO:tasks.workunit.client.1.smithi152.stdout:4/570: symlink d0/d7/d40/d23/d55/d6c/d91/lb3 0 2024-03-29T04:45:22.548 INFO:tasks.workunit.client.1.smithi152.stdout:5/739: truncate d6/d1d/daa/db7/d56/d63/db9/ff2 1042582 0 2024-03-29T04:45:22.549 INFO:tasks.workunit.client.1.smithi152.stdout:4/571: stat d0/d7/fab 0 2024-03-29T04:45:22.549 INFO:tasks.workunit.client.1.smithi152.stdout:0/590: mknod d2/d29/cdd 0 2024-03-29T04:45:22.549 INFO:tasks.workunit.client.1.smithi152.stdout:3/649: dread d7/d19/d1b/d1e/da1/d41/d73/f77 [0,4194304] 0 2024-03-29T04:45:22.550 INFO:tasks.workunit.client.1.smithi152.stdout:7/575: creat d7/d8/d93/d99/fad x:0 0 0 2024-03-29T04:45:22.551 INFO:tasks.workunit.client.1.smithi152.stdout:4/572: mknod d0/d7/d25/d17/d5e/cb4 0 2024-03-29T04:45:22.552 INFO:tasks.workunit.client.1.smithi152.stdout:3/650: chown d7/d19/d1b/d1e/f45 3351 1 2024-03-29T04:45:22.552 INFO:tasks.workunit.client.1.smithi152.stdout:5/740: symlink d6/d16/d25/d2e/da0/l102 0 2024-03-29T04:45:22.553 INFO:tasks.workunit.client.1.smithi152.stdout:7/576: mkdir d7/dae 0 2024-03-29T04:45:22.554 INFO:tasks.workunit.client.1.smithi152.stdout:7/577: chown d7/d8/d12/d1d/f1e 386562 1 2024-03-29T04:45:22.555 INFO:tasks.workunit.client.1.smithi152.stdout:0/591: rename d2/da/d17/d57/d9c/fa3 to d2/da/d3a/d61/fde 0 2024-03-29T04:45:22.558 INFO:tasks.workunit.client.1.smithi152.stdout:5/741: symlink d6/d1d/daa/db7/d56/dc1/l103 0 2024-03-29T04:45:22.559 INFO:tasks.workunit.client.1.smithi152.stdout:3/651: creat d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/fc6 x:0 0 0 2024-03-29T04:45:22.561 INFO:tasks.workunit.client.1.smithi152.stdout:6/835: truncate d1/d3/d1a/d1b/d29/d6f/f69 4105577 0 2024-03-29T04:45:22.565 INFO:tasks.workunit.client.1.smithi152.stdout:7/578: dread - d7/d8/db/df/f58 zero size 2024-03-29T04:45:22.566 INFO:tasks.workunit.client.1.smithi152.stdout:7/579: write d7/f59 [570481,56032] 0 2024-03-29T04:45:22.567 INFO:tasks.workunit.client.1.smithi152.stdout:3/652: write d7/d19/d3a/f7f [260137,124081] 0 2024-03-29T04:45:22.570 INFO:tasks.workunit.client.1.smithi152.stdout:6/836: mkdir d1/d3/d1a/d1b/d87/d8f/d102/d11d 0 2024-03-29T04:45:22.574 INFO:tasks.workunit.client.1.smithi152.stdout:2/603: truncate d0/d3/d30/d15/d22/d4c/f51 3857498 0 2024-03-29T04:45:22.577 INFO:tasks.workunit.client.1.smithi152.stdout:3/653: mkdir d7/d19/d1b/d1e/da1/d41/d61/d75/dc7 0 2024-03-29T04:45:22.581 INFO:tasks.workunit.client.1.smithi152.stdout:5/742: creat d6/d16/f104 x:0 0 0 2024-03-29T04:45:22.583 INFO:tasks.workunit.client.1.smithi152.stdout:6/837: dwrite d1/d3/d1a/d1b/d29/d6f/dee/f11c [0,4194304] 0 2024-03-29T04:45:22.584 INFO:tasks.workunit.client.1.smithi152.stdout:3/654: rmdir d7/d19/d1b/d1e 39 2024-03-29T04:45:22.591 INFO:tasks.workunit.client.1.smithi152.stdout:5/743: write d6/d1d/d3b/d55/f7a [520193,11223] 0 2024-03-29T04:45:22.593 INFO:tasks.workunit.client.1.smithi152.stdout:2/604: dwrite d0/d3/df/d2c/f2d [0,4194304] 0 2024-03-29T04:45:22.595 INFO:tasks.workunit.client.1.smithi152.stdout:0/592: dwrite d2/f6b [0,4194304] 0 2024-03-29T04:45:22.595 INFO:tasks.workunit.client.1.smithi152.stdout:7/580: dread d7/d8/d12/f28 [0,4194304] 0 2024-03-29T04:45:22.596 INFO:tasks.workunit.client.1.smithi152.stdout:3/655: symlink d7/d19/d1b/lc8 0 2024-03-29T04:45:22.597 INFO:tasks.workunit.client.1.smithi152.stdout:0/593: fsync d2/d31/f35 0 2024-03-29T04:45:22.601 INFO:tasks.workunit.client.1.smithi152.stdout:4/573: dread d0/d7/d25/d17/d5e/d60/d74/f50 [0,4194304] 0 2024-03-29T04:45:22.603 INFO:tasks.workunit.client.1.smithi152.stdout:5/744: creat d6/d16/d25/d2e/da0/dab/f105 x:0 0 0 2024-03-29T04:45:22.605 INFO:tasks.workunit.client.1.smithi152.stdout:2/605: mkdir d0/d3/d30/d15/d53/db4 0 2024-03-29T04:45:22.605 INFO:tasks.workunit.client.1.smithi152.stdout:3/656: rmdir d7/d6c 39 2024-03-29T04:45:22.605 INFO:tasks.workunit.client.1.smithi152.stdout:0/594: readlink d2/da/d3a/d6f/l99 0 2024-03-29T04:45:22.605 INFO:tasks.workunit.client.1.smithi152.stdout:7/581: creat d7/d8/d12/d35/faf x:0 0 0 2024-03-29T04:45:22.606 INFO:tasks.workunit.client.1.smithi152.stdout:5/745: dread - d6/d16/d25/fdc zero size 2024-03-29T04:45:22.609 INFO:tasks.workunit.client.1.smithi152.stdout:6/838: dwrite d1/d3/d9/d13/d3e/d93/ffc [0,4194304] 0 2024-03-29T04:45:22.609 INFO:tasks.workunit.client.1.smithi152.stdout:2/606: write d0/d3/d30/f87 [1244495,24846] 0 2024-03-29T04:45:22.610 INFO:tasks.workunit.client.1.smithi152.stdout:7/582: write d7/d8/d93/d99/fad [819978,97772] 0 2024-03-29T04:45:22.610 INFO:tasks.workunit.client.1.smithi152.stdout:0/595: mknod d2/da/d17/d57/dc5/cdf 0 2024-03-29T04:45:22.612 INFO:tasks.workunit.client.1.smithi152.stdout:5/746: mknod d6/d1d/d30/c106 0 2024-03-29T04:45:22.612 INFO:tasks.workunit.client.1.smithi152.stdout:0/596: chown d2/da/d17/d57/f8c 18503949 1 2024-03-29T04:45:22.615 INFO:tasks.workunit.client.1.smithi152.stdout:0/597: unlink d2/da/df/f93 0 2024-03-29T04:45:22.616 INFO:tasks.workunit.client.1.smithi152.stdout:3/657: symlink d7/d19/d1b/d1e/da1/d41/d61/lc9 0 2024-03-29T04:45:22.617 INFO:tasks.workunit.client.1.smithi152.stdout:0/598: readlink d2/da/le 0 2024-03-29T04:45:22.619 INFO:tasks.workunit.client.1.smithi152.stdout:6/839: dwrite d1/d3/f18 [0,4194304] 0 2024-03-29T04:45:22.620 INFO:tasks.workunit.client.1.smithi152.stdout:5/747: link d6/d1d/d30/d6a/cfc d6/d16/df8/c107 0 2024-03-29T04:45:22.621 INFO:tasks.workunit.client.1.smithi152.stdout:5/748: write d6/d1d/fa9 [11122,97276] 0 2024-03-29T04:45:22.624 INFO:tasks.workunit.client.1.smithi152.stdout:3/658: getdents d7/d19/d1b/d1e/d22 0 2024-03-29T04:45:22.625 INFO:tasks.workunit.client.1.smithi152.stdout:6/840: truncate d1/d3/df/f81 248648 0 2024-03-29T04:45:22.628 INFO:tasks.workunit.client.1.smithi152.stdout:6/841: getdents d1/d3/d9/d13/dc0/dc3/dfb 0 2024-03-29T04:45:22.631 INFO:tasks.workunit.client.1.smithi152.stdout:2/607: dread d0/d3/df/f23 [0,4194304] 0 2024-03-29T04:45:22.632 INFO:tasks.workunit.client.1.smithi152.stdout:7/583: dread d7/d8/d12/f6f [0,4194304] 0 2024-03-29T04:45:22.642 INFO:tasks.workunit.client.1.smithi152.stdout:2/608: dwrite d0/d3/d30/d15/d22/d4c/d52/f95 [0,4194304] 0 2024-03-29T04:45:22.643 INFO:tasks.workunit.client.1.smithi152.stdout:4/574: read d0/d7/d40/f8a [170448,124016] 0 2024-03-29T04:45:22.644 INFO:tasks.workunit.client.1.smithi152.stdout:2/609: readlink d0/l71 0 2024-03-29T04:45:22.649 INFO:tasks.workunit.client.1.smithi152.stdout:4/575: dread - d0/d7/d40/d23/d6a/fa5 zero size 2024-03-29T04:45:22.650 INFO:tasks.workunit.client.1.smithi152.stdout:4/576: stat d0/d7/d25/d17/f54 0 2024-03-29T04:45:22.650 INFO:tasks.workunit.client.1.smithi152.stdout:2/610: unlink d0/d3/d30/d15/d22/d4c/l67 0 2024-03-29T04:45:22.655 INFO:tasks.workunit.client.1.smithi152.stdout:2/611: symlink d0/lb5 0 2024-03-29T04:45:22.663 INFO:tasks.workunit.client.1.smithi152.stdout:2/612: dread d0/d3/d30/d15/d22/f37 [4194304,4194304] 0 2024-03-29T04:45:22.665 INFO:tasks.workunit.client.1.smithi152.stdout:0/599: dwrite d2/da/d3a/f97 [4194304,4194304] 0 2024-03-29T04:45:22.665 INFO:tasks.workunit.client.1.smithi152.stdout:5/749: dwrite d6/d1d/d3b/d55/f60 [4194304,4194304] 0 2024-03-29T04:45:22.670 INFO:tasks.workunit.client.1.smithi152.stdout:2/613: truncate d0/d3/d64/d85/d9d/fb2 210614 0 2024-03-29T04:45:22.671 INFO:tasks.workunit.client.1.smithi152.stdout:5/750: rename d6/d1d/d3b/d55 to d6/d1d/d30/d6a/d108 0 2024-03-29T04:45:22.671 INFO:tasks.workunit.client.1.smithi152.stdout:2/614: stat d0/d3/d30/d15/d53/d54/d57 0 2024-03-29T04:45:22.677 INFO:tasks.workunit.client.1.smithi152.stdout:6/842: truncate d1/d3/d9/f3a 1046938 0 2024-03-29T04:45:22.678 INFO:tasks.workunit.client.1.smithi152.stdout:7/584: truncate d7/d8/d12/f7b 1482283 0 2024-03-29T04:45:22.678 INFO:tasks.workunit.client.1.smithi152.stdout:5/751: mkdir d6/d16/d109 0 2024-03-29T04:45:22.679 INFO:tasks.workunit.client.1.smithi152.stdout:2/615: getdents d0/d3/d30/d15/d53/d54/dad 0 2024-03-29T04:45:22.681 INFO:tasks.workunit.client.1.smithi152.stdout:5/752: write d6/d1d/d30/d6a/fa6 [598397,25831] 0 2024-03-29T04:45:22.682 INFO:tasks.workunit.client.1.smithi152.stdout:6/843: rename d1/d3/l6c to d1/d3/d1a/d1b/d87/d8f/df3/l11e 0 2024-03-29T04:45:22.684 INFO:tasks.workunit.client.1.smithi152.stdout:6/844: write d1/d3/d9/d13/d28/d45/d8c/feb [851941,35998] 0 2024-03-29T04:45:22.684 INFO:tasks.workunit.client.1.smithi152.stdout:5/753: creat d6/d7/f10a x:0 0 0 2024-03-29T04:45:22.686 INFO:tasks.workunit.client.1.smithi152.stdout:5/754: dread - d6/d16/d25/d2e/da0/d92/dc0/dec/fef zero size 2024-03-29T04:45:22.687 INFO:tasks.workunit.client.1.smithi152.stdout:2/616: rename d0/d3/d30/d15/d22/d4c/d52/f82 to d0/d3/d30/d15/da5/fb6 0 2024-03-29T04:45:22.688 INFO:tasks.workunit.client.1.smithi152.stdout:7/585: link d7/d8/f8b d7/d8/d12/d1d/d30/fb0 0 2024-03-29T04:45:22.689 INFO:tasks.workunit.client.1.smithi152.stdout:5/755: write d6/d1d/d3b/f95 [4094420,35664] 0 2024-03-29T04:45:22.689 INFO:tasks.workunit.client.1.smithi152.stdout:6/845: symlink d1/d3/df/d84/da1/l11f 0 2024-03-29T04:45:22.689 INFO:tasks.workunit.client.1.smithi152.stdout:4/577: dread d0/d7/d25/d17/d5e/d60/d74/d57/f47 [0,4194304] 0 2024-03-29T04:45:22.692 INFO:tasks.workunit.client.1.smithi152.stdout:2/617: write d0/d3/d30/d15/d22/f91 [1009759,53483] 0 2024-03-29T04:45:22.693 INFO:tasks.workunit.client.1.smithi152.stdout:7/586: symlink d7/d8/d12/d1d/d41/lb1 0 2024-03-29T04:45:22.693 INFO:tasks.workunit.client.1.smithi152.stdout:0/600: write d2/da/d3a/d61/fde [921095,117487] 0 2024-03-29T04:45:22.696 INFO:tasks.workunit.client.1.smithi152.stdout:4/578: creat d0/d7/d40/d62/fb5 x:0 0 0 2024-03-29T04:45:22.696 INFO:tasks.workunit.client.1.smithi152.stdout:5/756: creat d6/d16/d25/d2e/da0/dab/f10b x:0 0 0 2024-03-29T04:45:22.696 INFO:tasks.workunit.client.1.smithi152.stdout:7/587: mknod d7/d8/d93/d99/cb2 0 2024-03-29T04:45:22.698 INFO:tasks.workunit.client.1.smithi152.stdout:6/846: symlink d1/d3/d9/d13/d3e/de9/l120 0 2024-03-29T04:45:22.699 INFO:tasks.workunit.client.1.smithi152.stdout:7/588: readlink d7/d8/d12/d35/d68/l60 0 2024-03-29T04:45:22.701 INFO:tasks.workunit.client.1.smithi152.stdout:6/847: fdatasync d1/d3/d1a/d1b/d29/f79 0 2024-03-29T04:45:22.703 INFO:tasks.workunit.client.1.smithi152.stdout:7/589: write d7/d8/d12/d35/f2a [2139960,38198] 0 2024-03-29T04:45:22.705 INFO:tasks.workunit.client.1.smithi152.stdout:4/579: symlink d0/d27/d6e/d8c/lb6 0 2024-03-29T04:45:22.705 INFO:tasks.workunit.client.1.smithi152.stdout:5/757: dread d6/d16/d25/d2e/d9d/fdd [4194304,4194304] 0 2024-03-29T04:45:22.707 INFO:tasks.workunit.client.1.smithi152.stdout:6/848: unlink d1/d3/d91/ldb 0 2024-03-29T04:45:22.709 INFO:tasks.workunit.client.1.smithi152.stdout:4/580: dread d0/d27/f59 [0,4194304] 0 2024-03-29T04:45:22.709 INFO:tasks.workunit.client.1.smithi152.stdout:5/758: mkdir d6/d16/d25/d2e/d9d/d10c 0 2024-03-29T04:45:22.712 INFO:tasks.workunit.client.1.smithi152.stdout:7/590: dwrite d7/d8/d12/d1d/d41/f2c [0,4194304] 0 2024-03-29T04:45:22.714 INFO:tasks.workunit.client.1.smithi152.stdout:6/849: dread d1/d3/df/d84/f9d [0,4194304] 0 2024-03-29T04:45:22.719 INFO:tasks.workunit.client.1.smithi152.stdout:7/591: chown d7/c89 78 1 2024-03-29T04:45:22.719 INFO:tasks.workunit.client.1.smithi152.stdout:5/759: dwrite d6/d1d/d30/d6a/fa7 [0,4194304] 0 2024-03-29T04:45:22.719 INFO:tasks.workunit.client.1.smithi152.stdout:2/618: fsync d0/d3/d30/d15/d53/d54/d57/f72 0 2024-03-29T04:45:22.721 INFO:tasks.workunit.client.1.smithi152.stdout:4/581: getdents d0/d7/d40/d62 0 2024-03-29T04:45:22.721 INFO:tasks.workunit.client.1.smithi152.stdout:2/619: readlink d0/d3/l8e 0 2024-03-29T04:45:22.722 INFO:tasks.workunit.client.1.smithi152.stdout:2/620: chown d0/f16 57491 1 2024-03-29T04:45:22.723 INFO:tasks.workunit.client.1.smithi152.stdout:0/601: write d2/da/d3a/d61/f9a [1213642,3248] 0 2024-03-29T04:45:22.724 INFO:tasks.workunit.client.1.smithi152.stdout:6/850: getdents d1/d3/d9/d13/da7/df5 0 2024-03-29T04:45:22.729 INFO:tasks.workunit.client.1.smithi152.stdout:2/621: write d0/d3/d30/d15/d22/d4c/f90 [220625,117143] 0 2024-03-29T04:45:22.730 INFO:tasks.workunit.client.1.smithi152.stdout:5/760: rename d6/d1b/l1a to d6/d1d/l10d 0 2024-03-29T04:45:22.730 INFO:tasks.workunit.client.1.smithi152.stdout:0/602: getdents d2/da/df/dbb 0 2024-03-29T04:45:22.731 INFO:tasks.workunit.client.1.smithi152.stdout:4/582: dread d0/d7/d25/d17/d5e/d60/d74/f50 [0,4194304] 0 2024-03-29T04:45:22.733 INFO:tasks.workunit.client.1.smithi152.stdout:5/761: read d6/d1d/d30/d6a/d108/fe [1373019,76072] 0 2024-03-29T04:45:22.733 INFO:tasks.workunit.client.1.smithi152.stdout:6/851: link d1/d3/df/d84/ff7 d1/d3/d9/d13/d28/d45/f121 0 2024-03-29T04:45:22.736 INFO:tasks.workunit.client.1.smithi152.stdout:6/852: chown d1/d3/d9/d13/d3e/d54/f7c 118455 1 2024-03-29T04:45:22.737 INFO:tasks.workunit.client.1.smithi152.stdout:4/583: symlink d0/d7/d40/d62/lb7 0 2024-03-29T04:45:22.739 INFO:tasks.workunit.client.1.smithi152.stdout:5/762: creat d6/d16/db3/dfd/f10e x:0 0 0 2024-03-29T04:45:22.739 INFO:tasks.workunit.client.1.smithi152.stdout:2/622: dwrite d0/d3/d30/f3f [0,4194304] 0 2024-03-29T04:45:22.741 INFO:tasks.workunit.client.1.smithi152.stdout:0/603: dread d2/da/d17/f3d [0,4194304] 0 2024-03-29T04:45:22.742 INFO:tasks.workunit.client.1.smithi152.stdout:4/584: write d0/d7/d25/d17/f3b [18097,117346] 0 2024-03-29T04:45:22.748 INFO:tasks.workunit.client.1.smithi152.stdout:2/623: creat d0/d3/d30/d15/da5/fb7 x:0 0 0 2024-03-29T04:45:22.749 INFO:tasks.workunit.client.1.smithi152.stdout:6/853: dwrite d1/d3/d9/d13/d3e/d54/ff4 [0,4194304] 0 2024-03-29T04:45:22.752 INFO:tasks.workunit.client.1.smithi152.stdout:5/763: rename d6/d16/db3/feb to d6/d1d/d30/d6a/dfb/f10f 0 2024-03-29T04:45:22.753 INFO:tasks.workunit.client.1.smithi152.stdout:5/764: stat d6/d1b/l50 0 2024-03-29T04:45:22.754 INFO:tasks.workunit.client.1.smithi152.stdout:0/604: fdatasync d2/da/d3a/d6f/f85 0 2024-03-29T04:45:22.759 INFO:tasks.workunit.client.1.smithi152.stdout:0/605: write d2/d31/fda [171465,31395] 0 2024-03-29T04:45:22.759 INFO:tasks.workunit.client.1.smithi152.stdout:4/585: dwrite d0/d7/d25/d17/f5b [0,4194304] 0 2024-03-29T04:45:22.760 INFO:tasks.workunit.client.1.smithi152.stdout:4/586: truncate d0/d7/d40/d2a/f94 666132 0 2024-03-29T04:45:22.762 INFO:tasks.workunit.client.1.smithi152.stdout:5/765: rename d6/d1d/d30/fbf to d6/d7/d75/f110 0 2024-03-29T04:45:22.762 INFO:tasks.workunit.client.1.smithi152.stdout:2/624: mkdir d0/d3/df/db8 0 2024-03-29T04:45:22.765 INFO:tasks.workunit.client.1.smithi152.stdout:4/587: creat d0/d7/d40/d23/d55/fb8 x:0 0 0 2024-03-29T04:45:22.765 INFO:tasks.workunit.client.1.smithi152.stdout:5/766: creat d6/d1d/daa/f111 x:0 0 0 2024-03-29T04:45:22.767 INFO:tasks.workunit.client.1.smithi152.stdout:2/625: unlink d0/d3/d30/d15/d22/f49 0 2024-03-29T04:45:22.768 INFO:tasks.workunit.client.1.smithi152.stdout:0/606: creat d2/da/d17/fe0 x:0 0 0 2024-03-29T04:45:22.770 INFO:tasks.workunit.client.1.smithi152.stdout:4/588: mknod d0/d7/cb9 0 2024-03-29T04:45:22.771 INFO:tasks.workunit.client.1.smithi152.stdout:5/767: mknod d6/d16/db2/c112 0 2024-03-29T04:45:22.771 INFO:tasks.workunit.client.1.smithi152.stdout:0/607: chown d2/da/d3a/d6f 56 1 2024-03-29T04:45:22.772 INFO:tasks.workunit.client.1.smithi152.stdout:6/854: dwrite d1/d3/d9/d13/d28/d45/f121 [0,4194304] 0 2024-03-29T04:45:22.775 INFO:tasks.workunit.client.1.smithi152.stdout:6/855: truncate d1/f10a 199340 0 2024-03-29T04:45:22.778 INFO:tasks.workunit.client.1.smithi152.stdout:0/608: mknod d2/da/d17/ce1 0 2024-03-29T04:45:22.779 INFO:tasks.workunit.client.1.smithi152.stdout:6/856: chown d1/d3/df/ld8 163835421 1 2024-03-29T04:45:22.785 INFO:tasks.workunit.client.1.smithi152.stdout:6/857: write f0 [1710226,9580] 0 2024-03-29T04:45:22.788 INFO:tasks.workunit.client.1.smithi152.stdout:2/626: dwrite d0/d3/d30/d15/d22/f37 [0,4194304] 0 2024-03-29T04:45:22.790 INFO:tasks.workunit.client.1.smithi152.stdout:5/768: dwrite d6/d16/d25/d2e/da0/d92/dc0/dec/fef [0,4194304] 0 2024-03-29T04:45:22.796 INFO:tasks.workunit.client.1.smithi152.stdout:0/609: dwrite d2/da/d17/f41 [4194304,4194304] 0 2024-03-29T04:45:22.798 INFO:tasks.workunit.client.1.smithi152.stdout:2/627: mkdir d0/d3/df/db9 0 2024-03-29T04:45:22.801 INFO:tasks.workunit.client.1.smithi152.stdout:6/858: link d1/d3/d1a/d1b/d29/f7e d1/d3/d9/d13/da7/df5/f122 0 2024-03-29T04:45:22.803 INFO:tasks.workunit.client.1.smithi152.stdout:6/859: truncate d1/d3/d1a/d1b/d29/dce/fda 356898 0 2024-03-29T04:45:22.803 INFO:tasks.workunit.client.1.smithi152.stdout:0/610: rename d2/da/df/la0 to d2/da/d17/d57/d9c/le2 0 2024-03-29T04:45:22.808 INFO:tasks.workunit.client.1.smithi152.stdout:5/769: creat d6/d1d/daa/db7/d56/f113 x:0 0 0 2024-03-29T04:45:22.809 INFO:tasks.workunit.client.1.smithi152.stdout:0/611: rmdir d2/da/d17/dd8/db8 39 2024-03-29T04:45:22.811 INFO:tasks.workunit.client.1.smithi152.stdout:5/770: chown d6/d1d/d30/d6a/d108/c9c 8289203 1 2024-03-29T04:45:22.812 INFO:tasks.workunit.client.1.smithi152.stdout:2/628: dwrite d0/d3/d30/d15/da5/fb6 [0,4194304] 0 2024-03-29T04:45:22.814 INFO:tasks.workunit.client.1.smithi152.stdout:4/589: dread d0/d7/d25/d17/d5e/d60/d74/f81 [0,4194304] 0 2024-03-29T04:45:22.816 INFO:tasks.workunit.client.1.smithi152.stdout:5/771: fsync d6/d16/f4e 0 2024-03-29T04:45:22.819 INFO:tasks.workunit.client.1.smithi152.stdout:0/612: rename d2/da/df/c23 to d2/da/d17/d57/dc5/ce3 0 2024-03-29T04:45:22.827 INFO:tasks.workunit.client.1.smithi152.stdout:2/629: creat d0/d3/d30/d15/d53/d54/dad/fba x:0 0 0 2024-03-29T04:45:22.829 INFO:tasks.workunit.client.1.smithi152.stdout:5/772: truncate d6/d1d/d30/d6a/d108/f66 2971116 0 2024-03-29T04:45:22.830 INFO:tasks.workunit.client.1.smithi152.stdout:0/613: creat d2/da/d17/d50/dc6/fe4 x:0 0 0 2024-03-29T04:45:22.831 INFO:tasks.workunit.client.1.smithi152.stdout:4/590: creat d0/fba x:0 0 0 2024-03-29T04:45:22.832 INFO:tasks.workunit.client.1.smithi152.stdout:5/773: creat d6/d1d/daa/db7/dda/f114 x:0 0 0 2024-03-29T04:45:22.833 INFO:tasks.workunit.client.1.smithi152.stdout:0/614: symlink d2/d29/le5 0 2024-03-29T04:45:22.834 INFO:tasks.workunit.client.1.smithi152.stdout:0/615: stat d2/da/d17/l65 0 2024-03-29T04:45:22.836 INFO:tasks.workunit.client.1.smithi152.stdout:5/774: rename d6/d16/d25/d2e/d9d/c9e to d6/d16/d25/d2e/d9d/c115 0 2024-03-29T04:45:22.837 INFO:tasks.workunit.client.1.smithi152.stdout:4/591: creat d0/d7/d40/fbb x:0 0 0 2024-03-29T04:45:22.839 INFO:tasks.workunit.client.1.smithi152.stdout:0/616: rename d2/da/df/c33 to d2/da/d17/ce6 0 2024-03-29T04:45:22.840 INFO:tasks.workunit.client.1.smithi152.stdout:5/775: creat d6/d16/d25/f116 x:0 0 0 2024-03-29T04:45:22.842 INFO:tasks.workunit.client.1.smithi152.stdout:5/776: creat d6/d1b/f117 x:0 0 0 2024-03-29T04:45:22.842 INFO:tasks.workunit.client.1.smithi152.stdout:5/777: dread - d6/fe0 zero size 2024-03-29T04:45:22.844 INFO:tasks.workunit.client.1.smithi152.stdout:6/860: dread d1/d3/d1a/f1d [0,4194304] 0 2024-03-29T04:45:22.846 INFO:tasks.workunit.client.1.smithi152.stdout:6/861: stat d1/d3/d1a/fbd 0 2024-03-29T04:45:22.847 INFO:tasks.workunit.client.1.smithi152.stdout:2/630: getdents d0/d3/d30/d15/d53/d54/dad 0 2024-03-29T04:45:22.849 INFO:tasks.workunit.client.1.smithi152.stdout:5/778: dwrite d6/d1b/f117 [0,4194304] 0 2024-03-29T04:45:22.850 INFO:tasks.workunit.client.1.smithi152.stdout:5/779: chown d6/d1d/d30/d6a/d108 23109 1 2024-03-29T04:45:22.852 INFO:tasks.workunit.client.1.smithi152.stdout:2/631: mknod d0/d3/df/db9/cbb 0 2024-03-29T04:45:22.854 INFO:tasks.workunit.client.1.smithi152.stdout:2/632: creat d0/d3/d30/d15/da5/fbc x:0 0 0 2024-03-29T04:45:22.855 INFO:tasks.workunit.client.1.smithi152.stdout:5/780: getdents d6/d1d/daa/db7/d56/dc9 0 2024-03-29T04:45:22.857 INFO:tasks.workunit.client.1.smithi152.stdout:2/633: write d0/d3/f5 [54294,14431] 0 2024-03-29T04:45:22.858 INFO:tasks.workunit.client.1.smithi152.stdout:2/634: write d0/d3/d30/d15/da5/fb7 [456213,13013] 0 2024-03-29T04:45:22.858 INFO:tasks.workunit.client.1.smithi152.stdout:5/781: creat d6/d16/d25/d2e/d9d/f118 x:0 0 0 2024-03-29T04:45:22.860 INFO:tasks.workunit.client.1.smithi152.stdout:6/862: dread d1/d3/df/f2e [0,4194304] 0 2024-03-29T04:45:22.862 INFO:tasks.workunit.client.1.smithi152.stdout:6/863: creat d1/d3/d9/d13/d28/d45/d8c/f123 x:0 0 0 2024-03-29T04:45:22.863 INFO:tasks.workunit.client.1.smithi152.stdout:5/782: write d6/d1d/d30/f3e [733979,3132] 0 2024-03-29T04:45:22.865 INFO:tasks.workunit.client.1.smithi152.stdout:6/864: unlink d1/d3/df/dd2/f103 0 2024-03-29T04:45:22.867 INFO:tasks.workunit.client.1.smithi152.stdout:5/783: creat d6/d7/d75/de7/f119 x:0 0 0 2024-03-29T04:45:22.867 INFO:tasks.workunit.client.1.smithi152.stdout:6/865: symlink d1/d3/d1a/d1b/d87/l124 0 2024-03-29T04:45:22.870 INFO:tasks.workunit.client.1.smithi152.stdout:6/866: rename d1/d3/df/d84/c70 to d1/d3/d9/d13/d3e/c125 0 2024-03-29T04:45:22.906 INFO:tasks.workunit.client.1.smithi152.stdout:6/867: dwrite d1/d3/d9/d13/d3e/d54/f7c [0,4194304] 0 2024-03-29T04:45:22.907 INFO:tasks.workunit.client.1.smithi152.stdout:5/784: dwrite d6/d1d/f79 [0,4194304] 0 2024-03-29T04:45:22.918 INFO:tasks.workunit.client.1.smithi152.stdout:6/868: symlink d1/d3/d9/d13/dc0/dea/def/l126 0 2024-03-29T04:45:22.918 INFO:tasks.workunit.client.1.smithi152.stdout:5/785: dwrite d6/d1d/f79 [0,4194304] 0 2024-03-29T04:45:22.922 INFO:tasks.workunit.client.1.smithi152.stdout:5/786: mkdir d6/d1d/daa/db7/d56/dc9/d11a 0 2024-03-29T04:45:22.925 INFO:tasks.workunit.client.1.smithi152.stdout:5/787: chown d6/d1d/d30/f3e 726 1 2024-03-29T04:45:22.927 INFO:tasks.workunit.client.1.smithi152.stdout:6/869: rename d1/d3/df/f48 to d1/d3/d1a/d1b/d29/d6f/d95/f127 0 2024-03-29T04:45:22.927 INFO:tasks.workunit.client.1.smithi152.stdout:5/788: dread d6/d1d/daa/fb4 [0,4194304] 0 2024-03-29T04:45:22.932 INFO:tasks.workunit.client.1.smithi152.stdout:5/789: rmdir d6/d16/d25/d2e/da0/d92/dc0/dec 39 2024-03-29T04:45:22.958 INFO:tasks.workunit.client.1.smithi152.stdout:5/790: truncate d6/d7/f8a 4135028 0 2024-03-29T04:45:22.959 INFO:tasks.workunit.client.1.smithi152.stdout:5/791: dread - d6/d1d/d30/d6a/fe1 zero size 2024-03-29T04:45:22.960 INFO:tasks.workunit.client.1.smithi152.stdout:5/792: fdatasync d6/d1d/daa/de5/fdb 0 2024-03-29T04:45:22.969 INFO:tasks.workunit.client.1.smithi152.stdout:5/793: dwrite d6/d1d/daa/fb4 [0,4194304] 0 2024-03-29T04:45:22.970 INFO:tasks.workunit.client.1.smithi152.stdout:5/794: chown d6/d1d/d30/d6a/d108/fbe 0 1 2024-03-29T04:45:22.972 INFO:tasks.workunit.client.1.smithi152.stdout:5/795: write d6/d16/d25/f8e [719825,37003] 0 2024-03-29T04:45:22.975 INFO:tasks.workunit.client.1.smithi152.stdout:5/796: symlink d6/d16/db3/dfd/l11b 0 2024-03-29T04:45:23.063 INFO:tasks.workunit.client.1.smithi152.stdout:9/597: sync 2024-03-29T04:45:23.064 INFO:tasks.workunit.client.1.smithi152.stdout:1/907: sync 2024-03-29T04:45:23.064 INFO:tasks.workunit.client.1.smithi152.stdout:9/598: stat d5/d8/d31/d75/db1/dcd/f54 0 2024-03-29T04:45:23.065 INFO:tasks.workunit.client.1.smithi152.stdout:9/599: chown d5/d8/l80 150 1 2024-03-29T04:45:23.066 INFO:tasks.workunit.client.1.smithi152.stdout:1/908: fdatasync d5/d7/d8/d13/d19/f3d 0 2024-03-29T04:45:23.067 INFO:tasks.workunit.client.1.smithi152.stdout:1/909: readlink d5/d7/d1b/d2e/de2/d102/l92 0 2024-03-29T04:45:23.069 INFO:tasks.workunit.client.1.smithi152.stdout:1/910: truncate d5/d7/d38/fba 4813021 0 2024-03-29T04:45:23.071 INFO:tasks.workunit.client.1.smithi152.stdout:1/911: chown d5/d7/d38/fba 13 1 2024-03-29T04:45:23.073 INFO:tasks.workunit.client.1.smithi152.stdout:1/912: write d5/d7/d1b/f93 [179262,89717] 0 2024-03-29T04:45:23.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:22 smithi142 ceph-mon[29027]: pgmap v277: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 95 MiB/s rd, 268 MiB/s wr, 550 op/s 2024-03-29T04:45:23.138 INFO:tasks.workunit.client.1.smithi152.stdout:1/913: creat d5/d7/d8/d13/d19/dc7/dd2/f12c x:0 0 0 2024-03-29T04:45:23.142 INFO:tasks.workunit.client.1.smithi152.stdout:1/914: write d5/d7/fc2 [4747854,126826] 0 2024-03-29T04:45:23.145 INFO:tasks.workunit.client.1.smithi152.stdout:1/915: mknod d5/d7/d1b/d2e/d95/dad/d12/d77/c12d 0 2024-03-29T04:45:23.145 INFO:tasks.workunit.client.1.smithi152.stdout:1/916: chown d5/c99 957823118 1 2024-03-29T04:45:23.150 INFO:tasks.workunit.client.1.smithi152.stdout:1/917: symlink d5/d7/d8/l12e 0 2024-03-29T04:45:23.152 INFO:tasks.workunit.client.1.smithi152.stdout:1/918: dwrite d5/d7/d1b/d2e/f9e [4194304,4194304] 0 2024-03-29T04:45:23.172 INFO:tasks.workunit.client.1.smithi152.stdout:1/919: dread d5/fb8 [0,4194304] 0 2024-03-29T04:45:23.174 INFO:tasks.workunit.client.1.smithi152.stdout:1/920: rmdir d5/d7/d8/d13/d19 39 2024-03-29T04:45:23.178 INFO:tasks.workunit.client.1.smithi152.stdout:1/921: unlink d5/d7/d8/d13/d19/d80/cb5 0 2024-03-29T04:45:23.188 INFO:tasks.workunit.client.1.smithi152.stdout:1/922: dwrite d5/d7/d1b/d2e/d95/df1/f123 [0,4194304] 0 2024-03-29T04:45:23.201 INFO:tasks.workunit.client.1.smithi152.stdout:8/559: sync 2024-03-29T04:45:23.203 INFO:tasks.workunit.client.1.smithi152.stdout:8/560: symlink d1/lc7 0 2024-03-29T04:45:23.210 INFO:tasks.workunit.client.1.smithi152.stdout:8/561: dwrite d1/d2/d1d/d36/d4b/da1/db0/f46 [4194304,4194304] 0 2024-03-29T04:45:23.212 INFO:tasks.workunit.client.1.smithi152.stdout:8/562: write d1/d2/d1d/d42/f9e [590055,14351] 0 2024-03-29T04:45:23.212 INFO:tasks.workunit.client.1.smithi152.stdout:1/923: dread d5/d7/d8/d13/d19/f3d [4194304,4194304] 0 2024-03-29T04:45:23.213 INFO:tasks.workunit.client.1.smithi152.stdout:8/563: fsync d1/d2/d1d/d36/d4b/da1/fbb 0 2024-03-29T04:45:23.214 INFO:tasks.workunit.client.1.smithi152.stdout:1/924: mkdir d5/d7/d8/d13/d19/d80/d119/d12f 0 2024-03-29T04:45:23.216 INFO:tasks.workunit.client.1.smithi152.stdout:8/564: creat d1/d2/d1d/d36/d4b/da1/db0/d40/d47/fc8 x:0 0 0 2024-03-29T04:45:23.221 INFO:tasks.workunit.client.1.smithi152.stdout:8/565: rename d1/d2/d1d/d36/d4b/da1/db0/d7f/fb7 to d1/d2/d1d/dc6/fc9 0 2024-03-29T04:45:23.221 INFO:tasks.workunit.client.1.smithi152.stdout:8/566: chown d1/c5f 45698 1 2024-03-29T04:45:23.223 INFO:tasks.workunit.client.1.smithi152.stdout:1/925: write d5/d7/d1b/d2e/d95/fac [133231,56096] 0 2024-03-29T04:45:23.224 INFO:tasks.workunit.client.1.smithi152.stdout:8/567: mkdir d1/d2/d1d/d36/d4b/da1/db0/d40/d47/dca 0 2024-03-29T04:45:23.225 INFO:tasks.workunit.client.1.smithi152.stdout:1/926: chown d5/d7/d1b/d2e/d95/dad/c11a 15867 1 2024-03-29T04:45:23.227 INFO:tasks.workunit.client.1.smithi152.stdout:1/927: mknod d5/d7/d38/d6c/c130 0 2024-03-29T04:45:23.232 INFO:tasks.workunit.client.1.smithi152.stdout:1/928: rmdir d5/d7/d8/d13/dd0/dd3 0 2024-03-29T04:45:23.233 INFO:tasks.workunit.client.1.smithi152.stdout:1/929: rename d5 to d5/d7/d8/d13/dd0/d131 22 2024-03-29T04:45:23.238 INFO:tasks.workunit.client.1.smithi152.stdout:8/568: dwrite d1/d2/d1d/d36/d4b/da1/db0/d40/d47/f72 [0,4194304] 0 2024-03-29T04:45:23.239 INFO:tasks.workunit.client.1.smithi152.stdout:1/930: write d5/d7/d8/d13/fb2 [59827,107619] 0 2024-03-29T04:45:23.240 INFO:tasks.workunit.client.1.smithi152.stdout:8/569: creat d1/d2/d3c/fcb x:0 0 0 2024-03-29T04:45:23.241 INFO:tasks.workunit.client.1.smithi152.stdout:1/931: symlink d5/d7/d8/d13/d96/l132 0 2024-03-29T04:45:23.244 INFO:tasks.workunit.client.1.smithi152.stdout:1/932: write d5/d7/d1b/d2e/de2/d102/d104/fe8 [3718434,99322] 0 2024-03-29T04:45:23.244 INFO:tasks.workunit.client.1.smithi152.stdout:8/570: unlink d1/d2/d1d/d36/lae 0 2024-03-29T04:45:23.245 INFO:tasks.workunit.client.1.smithi152.stdout:8/571: write d1/d2/d1d/d36/d5d/f71 [4232584,5521] 0 2024-03-29T04:45:23.246 INFO:tasks.workunit.client.1.smithi152.stdout:1/933: mknod d5/d7/d8/d13/d19/d80/c133 0 2024-03-29T04:45:23.246 INFO:tasks.workunit.client.1.smithi152.stdout:1/934: chown d5/d7 710563781 1 2024-03-29T04:45:23.248 INFO:tasks.workunit.client.1.smithi152.stdout:8/572: symlink d1/d2/d1d/d36/d4b/da1/db0/d40/lcc 0 2024-03-29T04:45:23.248 INFO:tasks.workunit.client.1.smithi152.stdout:1/935: symlink d5/l134 0 2024-03-29T04:45:23.265 INFO:tasks.workunit.client.1.smithi152.stdout:1/936: dread d5/d7/d1b/d2e/f8e [0,4194304] 0 2024-03-29T04:45:23.267 INFO:tasks.workunit.client.1.smithi152.stdout:1/937: mknod d5/d7/d8/da5/c135 0 2024-03-29T04:45:23.268 INFO:tasks.workunit.client.1.smithi152.stdout:1/938: rmdir d5/d7/d8/d13/d19/dc7/dd2 39 2024-03-29T04:45:23.289 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:22 smithi152 ceph-mon[37177]: pgmap v277: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 95 MiB/s rd, 268 MiB/s wr, 550 op/s 2024-03-29T04:45:23.337 INFO:tasks.workunit.client.1.smithi152.stdout:8/573: write d1/d2/d1d/f3a [727878,71612] 0 2024-03-29T04:45:23.337 INFO:tasks.workunit.client.1.smithi152.stdout:1/939: write d5/d7/d8/d13/d19/dc7/fdf [1003924,93667] 0 2024-03-29T04:45:23.341 INFO:tasks.workunit.client.1.smithi152.stdout:1/940: truncate d5/d7/d1b/d2e/d95/fac 229693 0 2024-03-29T04:45:23.344 INFO:tasks.workunit.client.1.smithi152.stdout:8/574: dwrite d1/d2/d3c/f7a [0,4194304] 0 2024-03-29T04:45:23.346 INFO:tasks.workunit.client.1.smithi152.stdout:1/941: getdents d5/d7/d4d/db1/d121 0 2024-03-29T04:45:23.350 INFO:tasks.workunit.client.1.smithi152.stdout:1/942: creat d5/d7/d1b/d2e/de2/d102/d104/f136 x:0 0 0 2024-03-29T04:45:23.353 INFO:tasks.workunit.client.1.smithi152.stdout:1/943: creat d5/d7/d1b/d2e/d95/df1/dfe/f137 x:0 0 0 2024-03-29T04:45:23.354 INFO:tasks.workunit.client.1.smithi152.stdout:8/575: dwrite d1/d2/d3c/d76/f9b [0,4194304] 0 2024-03-29T04:45:23.354 INFO:tasks.workunit.client.1.smithi152.stdout:1/944: fdatasync d5/d7/d8/d13/dd0/ffb 0 2024-03-29T04:45:23.356 INFO:tasks.workunit.client.1.smithi152.stdout:8/576: unlink d1/d2/d3/l16 0 2024-03-29T04:45:23.357 INFO:tasks.workunit.client.1.smithi152.stdout:1/945: fdatasync d5/d7/d8/d13/d96/fe6 0 2024-03-29T04:45:23.358 INFO:tasks.workunit.client.1.smithi152.stdout:8/577: creat d1/dc5/fcd x:0 0 0 2024-03-29T04:45:23.365 INFO:tasks.workunit.client.1.smithi152.stdout:1/946: write d5/d7/d6a/f6e [1430687,98928] 0 2024-03-29T04:45:23.366 INFO:tasks.workunit.client.1.smithi152.stdout:1/947: write d5/d7/f25 [3736976,36337] 0 2024-03-29T04:45:23.369 INFO:tasks.workunit.client.1.smithi152.stdout:1/948: unlink d5/d7/d1b/d2e/de2/d102/d104/f2f 0 2024-03-29T04:45:23.372 INFO:tasks.workunit.client.1.smithi152.stdout:1/949: link d5/l1d d5/d7/d1b/d2e/de2/d102/d113/l138 0 2024-03-29T04:45:23.378 INFO:tasks.workunit.client.1.smithi152.stdout:1/950: dwrite d5/d7/d1b/d2e/d95/dad/d12/d77/f115 [0,4194304] 0 2024-03-29T04:45:23.382 INFO:tasks.workunit.client.1.smithi152.stdout:1/951: symlink d5/d7/d8/d13/dd0/l139 0 2024-03-29T04:45:23.383 INFO:tasks.workunit.client.1.smithi152.stdout:1/952: mknod d5/d7/d6a/c13a 0 2024-03-29T04:45:23.386 INFO:tasks.workunit.client.1.smithi152.stdout:1/953: rename d5/d7/d1b/d2e/d95/dad/f46 to d5/d7/d6a/f13b 0 2024-03-29T04:45:23.389 INFO:tasks.workunit.client.1.smithi152.stdout:1/954: unlink d5/d7/d1b/d2e/d95/dad/c11a 0 2024-03-29T04:45:23.390 INFO:tasks.workunit.client.1.smithi152.stdout:1/955: mknod d5/d7/d4d/db1/d111/c13c 0 2024-03-29T04:45:23.394 INFO:tasks.workunit.client.1.smithi152.stdout:1/956: dread d5/d7/d1b/d2e/f86 [0,4194304] 0 2024-03-29T04:45:23.395 INFO:tasks.workunit.client.1.smithi152.stdout:1/957: chown d5/d7/d1b/d2e/d95/dad/d12/la6 120 1 2024-03-29T04:45:23.400 INFO:tasks.workunit.client.1.smithi152.stdout:1/958: dwrite d5/d7/fe [0,4194304] 0 2024-03-29T04:45:23.407 INFO:tasks.workunit.client.1.smithi152.stdout:1/959: dwrite d5/d7/d8/d13/f74 [0,4194304] 0 2024-03-29T04:45:23.408 INFO:tasks.workunit.client.1.smithi152.stdout:1/960: stat d5/d7/d6a/f88 0 2024-03-29T04:45:23.410 INFO:tasks.workunit.client.1.smithi152.stdout:1/961: creat d5/d7/d1b/d2e/d95/d9d/f13d x:0 0 0 2024-03-29T04:45:23.412 INFO:tasks.workunit.client.1.smithi152.stdout:1/962: creat d5/d7/d1b/d2e/d95/df1/dfe/f13e x:0 0 0 2024-03-29T04:45:23.415 INFO:tasks.workunit.client.1.smithi152.stdout:1/963: truncate d5/d7/d1b/d2e/fa3 253158 0 2024-03-29T04:45:23.417 INFO:tasks.workunit.client.1.smithi152.stdout:1/964: write d5/d7/d8/d13/d19/f94 [1425607,44160] 0 2024-03-29T04:45:23.417 INFO:tasks.workunit.client.1.smithi152.stdout:3/659: sync 2024-03-29T04:45:23.418 INFO:tasks.workunit.client.1.smithi152.stdout:1/965: write d5/d7/d1b/d2e/d95/fac [166945,93224] 0 2024-03-29T04:45:23.419 INFO:tasks.workunit.client.1.smithi152.stdout:3/660: write d7/fa4 [40540,68848] 0 2024-03-29T04:45:23.421 INFO:tasks.workunit.client.1.smithi152.stdout:1/966: symlink d5/d7/d1b/d10f/l13f 0 2024-03-29T04:45:23.423 INFO:tasks.workunit.client.1.smithi152.stdout:1/967: mknod d5/d7/d1b/d2e/deb/c140 0 2024-03-29T04:45:23.424 INFO:tasks.workunit.client.1.smithi152.stdout:1/968: chown d5/d7/d38/ca1 53 1 2024-03-29T04:45:23.427 INFO:tasks.workunit.client.1.smithi152.stdout:1/969: rename d5/c4e to d5/d7/d8/d13/d19/d80/c141 0 2024-03-29T04:45:23.436 INFO:tasks.workunit.client.1.smithi152.stdout:1/970: dwrite d5/d7/d1b/d2e/d95/dad/d12/fdd [0,4194304] 0 2024-03-29T04:45:23.437 INFO:tasks.workunit.client.1.smithi152.stdout:1/971: chown d5/d7/d1b/d2e/de2/d102/d113 2155376 1 2024-03-29T04:45:23.449 INFO:tasks.workunit.client.1.smithi152.stdout:3/661: dread d7/d19/d1b/d1e/d22/f2d [0,4194304] 0 2024-03-29T04:45:23.449 INFO:tasks.workunit.client.1.smithi152.stdout:3/662: chown d7/d19/c42 22231 1 2024-03-29T04:45:23.457 INFO:tasks.workunit.client.1.smithi152.stdout:3/663: write d7/d19/d1b/d1e/d4e/d8f/fa9 [4883090,81541] 0 2024-03-29T04:45:23.459 INFO:tasks.workunit.client.1.smithi152.stdout:3/664: write d7/d19/d1b/f58 [1225053,111562] 0 2024-03-29T04:45:23.463 INFO:tasks.workunit.client.1.smithi152.stdout:3/665: creat d7/d6c/fca x:0 0 0 2024-03-29T04:45:23.463 INFO:tasks.workunit.client.1.smithi152.stdout:1/972: dread d5/f3f [0,4194304] 0 2024-03-29T04:45:23.465 INFO:tasks.workunit.client.1.smithi152.stdout:3/666: creat d7/d19/d1b/d59/fcb x:0 0 0 2024-03-29T04:45:23.470 INFO:tasks.workunit.client.1.smithi152.stdout:3/667: dread d7/d19/d1b/d1e/da1/d41/d73/f77 [0,4194304] 0 2024-03-29T04:45:23.474 INFO:tasks.workunit.client.1.smithi152.stdout:1/973: truncate d5/d7/d1b/d2e/de2/d102/f5b 4155735 0 2024-03-29T04:45:23.477 INFO:tasks.workunit.client.1.smithi152.stdout:1/974: read d5/d7/ffa [396062,34933] 0 2024-03-29T04:45:23.477 INFO:tasks.workunit.client.1.smithi152.stdout:3/668: dwrite d7/d19/d1b/d1e/d4e/fad [0,4194304] 0 2024-03-29T04:45:23.483 INFO:tasks.workunit.client.1.smithi152.stdout:1/975: mkdir d5/d7/d4d/db1/d121/d142 0 2024-03-29T04:45:23.494 INFO:tasks.workunit.client.1.smithi152.stdout:3/669: write d7/d19/d1b/d1e/d22/f4d [1746043,31269] 0 2024-03-29T04:45:23.501 INFO:tasks.workunit.client.1.smithi152.stdout:1/976: dwrite d5/d7/d4d/db1/fee [0,4194304] 0 2024-03-29T04:45:23.502 INFO:tasks.workunit.client.1.smithi152.stdout:3/670: truncate d7/d19/d1b/d1e/d4e/fad 2281139 0 2024-03-29T04:45:23.503 INFO:tasks.workunit.client.1.smithi152.stdout:3/671: stat d7/d19/d3a/f7a 0 2024-03-29T04:45:23.504 INFO:tasks.workunit.client.1.smithi152.stdout:3/672: dread - d7/d19/d1b/d59/fa7 zero size 2024-03-29T04:45:23.504 INFO:tasks.workunit.client.1.smithi152.stdout:1/977: unlink d5/d7/d1b/d2e/de2/d102/d104/lbc 0 2024-03-29T04:45:23.507 INFO:tasks.workunit.client.1.smithi152.stdout:3/673: fdatasync d7/d19/d1b/d1e/d22/f6f 0 2024-03-29T04:45:23.507 INFO:tasks.workunit.client.1.smithi152.stdout:1/978: truncate d5/d7/d1b/d2e/f86 814409 0 2024-03-29T04:45:23.508 INFO:tasks.workunit.client.1.smithi152.stdout:1/979: chown d5/d7/d38/d6c/fd5 177 1 2024-03-29T04:45:23.510 INFO:tasks.workunit.client.1.smithi152.stdout:1/980: readlink d5/d7/d1b/d2e/d95/dad/l83 0 2024-03-29T04:45:23.512 INFO:tasks.workunit.client.1.smithi152.stdout:1/981: creat d5/d7/d1b/d2e/d95/dad/d12/d77/f143 x:0 0 0 2024-03-29T04:45:23.513 INFO:tasks.workunit.client.1.smithi152.stdout:1/982: write d5/d7/d8/d13/dd0/f107 [3078050,124983] 0 2024-03-29T04:45:23.515 INFO:tasks.workunit.client.1.smithi152.stdout:1/983: write d5/d7/d4d/f10b [956227,6045] 0 2024-03-29T04:45:23.518 INFO:tasks.workunit.client.1.smithi152.stdout:1/984: mkdir d5/d7/d4d/d144 0 2024-03-29T04:45:23.519 INFO:tasks.workunit.client.1.smithi152.stdout:7/592: sync 2024-03-29T04:45:23.519 INFO:tasks.workunit.client.1.smithi152.stdout:4/592: sync 2024-03-29T04:45:23.521 INFO:tasks.workunit.client.1.smithi152.stdout:4/593: dread - d0/d7/d40/d6d/fa4 zero size 2024-03-29T04:45:23.523 INFO:tasks.workunit.client.1.smithi152.stdout:1/985: getdents d5/d7/d8/d13/d96 0 2024-03-29T04:45:23.527 INFO:tasks.workunit.client.1.smithi152.stdout:3/674: dwrite d7/d19/d1b/f7b [0,4194304] 0 2024-03-29T04:45:23.527 INFO:tasks.workunit.client.1.smithi152.stdout:4/594: mknod d0/d7/d25/d17/d5e/d60/d74/cbc 0 2024-03-29T04:45:23.530 INFO:tasks.workunit.client.1.smithi152.stdout:3/675: write d7/d19/d1b/d59/f68 [43633,30764] 0 2024-03-29T04:45:23.533 INFO:tasks.workunit.client.1.smithi152.stdout:7/593: dwrite d7/d8/d12/f81 [0,4194304] 0 2024-03-29T04:45:23.535 INFO:tasks.workunit.client.1.smithi152.stdout:3/676: stat d7/d19/d1b/d1e/da1/d41/daf 0 2024-03-29T04:45:23.536 INFO:tasks.workunit.client.1.smithi152.stdout:1/986: dread d5/d7/d1b/d2e/d95/d9d/f101 [0,4194304] 0 2024-03-29T04:45:23.540 INFO:tasks.workunit.client.1.smithi152.stdout:4/595: dwrite d0/d27/f59 [0,4194304] 0 2024-03-29T04:45:23.540 INFO:tasks.workunit.client.1.smithi152.stdout:1/987: fsync d5/d7/d1b/d2e/d95/dad/d12/fdd 0 2024-03-29T04:45:23.541 INFO:tasks.workunit.client.1.smithi152.stdout:1/988: fdatasync d5/d7/d1b/d2e/d95/df1/f123 0 2024-03-29T04:45:23.542 INFO:tasks.workunit.client.1.smithi152.stdout:3/677: mkdir d7/d19/d1b/d1e/da1/d41/d6a/dcc 0 2024-03-29T04:45:23.544 INFO:tasks.workunit.client.1.smithi152.stdout:1/989: write d5/d7/d1b/d2e/de2/d102/d113/f124 [802506,35532] 0 2024-03-29T04:45:23.558 INFO:tasks.workunit.client.1.smithi152.stdout:3/678: rmdir d7/d19/d1b 39 2024-03-29T04:45:23.562 INFO:tasks.workunit.client.1.smithi152.stdout:3/679: rename d7/d19/d1b/d1e/da1/f2c to d7/d19/d1b/d1e/da1/fcd 0 2024-03-29T04:45:23.562 INFO:tasks.workunit.client.1.smithi152.stdout:7/594: truncate d7/d8/d12/d35/f32 415060 0 2024-03-29T04:45:23.563 INFO:tasks.workunit.client.1.smithi152.stdout:1/990: write d5/f72 [450496,66261] 0 2024-03-29T04:45:23.565 INFO:tasks.workunit.client.1.smithi152.stdout:3/680: creat d7/d19/d1b/d1e/da1/d41/d61/d85/fce x:0 0 0 2024-03-29T04:45:23.565 INFO:tasks.workunit.client.1.smithi152.stdout:7/595: read d7/d8/d12/d35/d68/f88 [562279,98914] 0 2024-03-29T04:45:23.565 INFO:tasks.workunit.client.1.smithi152.stdout:4/596: truncate d0/d7/d25/f36 3254868 0 2024-03-29T04:45:23.565 INFO:tasks.workunit.client.1.smithi152.stdout:1/991: write d5/d7/d1b/f93 [1389260,24479] 0 2024-03-29T04:45:23.566 INFO:tasks.workunit.client.1.smithi152.stdout:3/681: write d7/fa4 [673577,91797] 0 2024-03-29T04:45:23.572 INFO:tasks.workunit.client.1.smithi152.stdout:1/992: unlink d5/d7/d1b/d2e/d95/d9d/la2 0 2024-03-29T04:45:23.574 INFO:tasks.workunit.client.1.smithi152.stdout:4/597: mkdir d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd 0 2024-03-29T04:45:23.577 INFO:tasks.workunit.client.1.smithi152.stdout:4/598: chown d0/d7/c52 7355839 1 2024-03-29T04:45:23.579 INFO:tasks.workunit.client.1.smithi152.stdout:3/682: write d7/d19/d1b/f7e [622427,126660] 0 2024-03-29T04:45:23.585 INFO:tasks.workunit.client.1.smithi152.stdout:4/599: dwrite d0/d27/d6e/fad [0,4194304] 0 2024-03-29T04:45:23.586 INFO:tasks.workunit.client.1.smithi152.stdout:4/600: dread - d0/d7/d25/d17/d5e/d60/f9b zero size 2024-03-29T04:45:23.590 INFO:tasks.workunit.client.1.smithi152.stdout:1/993: dread d5/d7/d8/d13/fb2 [0,4194304] 0 2024-03-29T04:45:23.718 INFO:tasks.workunit.client.1.smithi152.stdout:9/600: sync 2024-03-29T04:45:23.719 INFO:tasks.workunit.client.1.smithi152.stdout:6/870: sync 2024-03-29T04:45:23.719 INFO:tasks.workunit.client.1.smithi152.stdout:2/635: sync 2024-03-29T04:45:23.719 INFO:tasks.workunit.client.1.smithi152.stdout:8/578: sync 2024-03-29T04:45:23.719 INFO:tasks.workunit.client.1.smithi152.stdout:5/797: sync 2024-03-29T04:45:23.719 INFO:tasks.workunit.client.1.smithi152.stdout:0/617: sync 2024-03-29T04:45:23.721 INFO:tasks.workunit.client.1.smithi152.stdout:9/601: mkdir d5/db/d92/dd2 0 2024-03-29T04:45:23.722 INFO:tasks.workunit.client.1.smithi152.stdout:8/579: mknod d1/d2/d1d/d36/cce 0 2024-03-29T04:45:23.723 INFO:tasks.workunit.client.1.smithi152.stdout:2/636: rmdir d0/d3/d30/d15/da5 39 2024-03-29T04:45:23.724 INFO:tasks.workunit.client.1.smithi152.stdout:8/580: write d1/d2/d1d/d36/d85/fa7 [675399,86771] 0 2024-03-29T04:45:23.724 INFO:tasks.workunit.client.1.smithi152.stdout:5/798: creat d6/d16/d25/d2e/dc7/f11c x:0 0 0 2024-03-29T04:45:23.725 INFO:tasks.workunit.client.1.smithi152.stdout:0/618: mkdir d2/da/d17/de7 0 2024-03-29T04:45:23.728 INFO:tasks.workunit.client.1.smithi152.stdout:0/619: dread - d2/da/d3a/fc0 zero size 2024-03-29T04:45:23.728 INFO:tasks.workunit.client.1.smithi152.stdout:5/799: read - d6/d1d/d30/d6a/d108/fe6 zero size 2024-03-29T04:45:23.729 INFO:tasks.workunit.client.1.smithi152.stdout:9/602: dwrite d5/f2f [0,4194304] 0 2024-03-29T04:45:23.730 INFO:tasks.workunit.client.1.smithi152.stdout:0/620: fsync d2/da/d17/fe0 0 2024-03-29T04:45:23.731 INFO:tasks.workunit.client.1.smithi152.stdout:5/800: fdatasync d6/d1d/d30/d6a/d108/fe6 0 2024-03-29T04:45:23.733 INFO:tasks.workunit.client.1.smithi152.stdout:5/801: write d6/d7/fa4 [2707082,26015] 0 2024-03-29T04:45:23.733 INFO:tasks.workunit.client.1.smithi152.stdout:8/581: rename d1/d2/l54 to d1/d2/d1d/d36/d4b/da1/db0/d40/d6a/lcf 0 2024-03-29T04:45:23.734 INFO:tasks.workunit.client.1.smithi152.stdout:9/603: rmdir d5/d12/db7 39 2024-03-29T04:45:23.736 INFO:tasks.workunit.client.1.smithi152.stdout:0/621: symlink d2/da/d17/d57/d9c/dbe/le8 0 2024-03-29T04:45:23.738 INFO:tasks.workunit.client.1.smithi152.stdout:2/637: rename d0/d3/d30/d15/d22/d4c/d52 to d0/d3/d30/d15/d22/d74/dbd 0 2024-03-29T04:45:23.741 INFO:tasks.workunit.client.1.smithi152.stdout:9/604: creat d5/db/d92/fd3 x:0 0 0 2024-03-29T04:45:23.743 INFO:tasks.workunit.client.1.smithi152.stdout:2/638: write d0/d3/df/f23 [3628655,86065] 0 2024-03-29T04:45:23.743 INFO:tasks.workunit.client.1.smithi152.stdout:8/582: dwrite d1/d2/d1d/d36/d5d/f71 [0,4194304] 0 2024-03-29T04:45:23.743 INFO:tasks.workunit.client.1.smithi152.stdout:2/639: stat d0/d3 0 2024-03-29T04:45:23.744 INFO:tasks.workunit.client.1.smithi152.stdout:5/802: getdents d6/d1d/daa/db7/d56/dc1/de4 0 2024-03-29T04:45:23.748 INFO:tasks.workunit.client.1.smithi152.stdout:5/803: symlink d6/d1d/daa/db7/l11d 0 2024-03-29T04:45:23.749 INFO:tasks.workunit.client.1.smithi152.stdout:8/583: creat d1/d2/d1d/d36/d4b/da1/db0/d40/d47/fd0 x:0 0 0 2024-03-29T04:45:23.751 INFO:tasks.workunit.client.1.smithi152.stdout:5/804: mknod d6/d16/db3/c11e 0 2024-03-29T04:45:23.752 INFO:tasks.workunit.client.1.smithi152.stdout:5/805: chown d6/d16/d25/d2e/da0/l102 1128635 1 2024-03-29T04:45:23.753 INFO:tasks.workunit.client.1.smithi152.stdout:0/622: write d2/da/d17/d57/f73 [472540,25473] 0 2024-03-29T04:45:23.756 INFO:tasks.workunit.client.1.smithi152.stdout:0/623: truncate d2/da/df/fd5 144972 0 2024-03-29T04:45:23.759 INFO:tasks.workunit.client.1.smithi152.stdout:9/605: dwrite d5/d8/d31/d75/db1/dcd/f39 [0,4194304] 0 2024-03-29T04:45:23.764 INFO:tasks.workunit.client.1.smithi152.stdout:0/624: getdents d2/da/d17/d57/dc5 0 2024-03-29T04:45:23.764 INFO:tasks.workunit.client.1.smithi152.stdout:9/606: rename d5/d8/d31/f3b to d5/d8/d31/d75/db1/dcd/d6e/d73/d8e/fd4 0 2024-03-29T04:45:23.766 INFO:tasks.workunit.client.1.smithi152.stdout:9/607: write d5/d8/d31/d75/db1/dcd/f39 [4430617,63099] 0 2024-03-29T04:45:23.767 INFO:tasks.workunit.client.1.smithi152.stdout:0/625: mkdir d2/da/d3a/de9 0 2024-03-29T04:45:23.770 INFO:tasks.workunit.client.1.smithi152.stdout:0/626: fsync d2/da/d17/d57/d9c/fad 0 2024-03-29T04:45:23.771 INFO:tasks.workunit.client.1.smithi152.stdout:8/584: dread d1/f34 [0,4194304] 0 2024-03-29T04:45:23.773 INFO:tasks.workunit.client.1.smithi152.stdout:8/585: mknod d1/dc5/cd1 0 2024-03-29T04:45:23.775 INFO:tasks.workunit.client.1.smithi152.stdout:8/586: unlink d1/c5f 0 2024-03-29T04:45:23.777 INFO:tasks.workunit.client.1.smithi152.stdout:8/587: readlink d1/d2/d3/l14 0 2024-03-29T04:45:23.779 INFO:tasks.workunit.client.1.smithi152.stdout:9/608: truncate d5/f61 603199 0 2024-03-29T04:45:23.780 INFO:tasks.workunit.client.1.smithi152.stdout:8/588: creat d1/d2/d1d/dc6/fd2 x:0 0 0 2024-03-29T04:45:23.780 INFO:tasks.workunit.client.1.smithi152.stdout:0/627: dwrite d2/da/d17/db5/fd1 [0,4194304] 0 2024-03-29T04:45:23.785 INFO:tasks.workunit.client.1.smithi152.stdout:0/628: creat d2/d29/fea x:0 0 0 2024-03-29T04:45:23.788 INFO:tasks.workunit.client.1.smithi152.stdout:9/609: link d5/d8/d31/d75/db1/dcd/c56 d5/d8/d31/d75/cd5 0 2024-03-29T04:45:23.792 INFO:tasks.workunit.client.1.smithi152.stdout:8/589: rename d1/d2/d1d/d36/d4b/da1 to d1/d2/d1d/dd3 0 2024-03-29T04:45:23.793 INFO:tasks.workunit.client.1.smithi152.stdout:0/629: creat d2/feb x:0 0 0 2024-03-29T04:45:23.794 INFO:tasks.workunit.client.1.smithi152.stdout:8/590: readlink d1/d2/d1d/dd3/db0/d40/d6a/l94 0 2024-03-29T04:45:23.795 INFO:tasks.workunit.client.1.smithi152.stdout:8/591: chown d1/d2/d1d/d36 0 1 2024-03-29T04:45:23.795 INFO:tasks.workunit.client.1.smithi152.stdout:0/630: write d2/da/d17/fbd [915248,45799] 0 2024-03-29T04:45:23.796 INFO:tasks.workunit.client.1.smithi152.stdout:9/610: rename d5/d8/d31/d75/db1/dcd/d6e/d73/d7a/f7e to d5/db/fd6 0 2024-03-29T04:45:23.798 INFO:tasks.workunit.client.1.smithi152.stdout:8/592: read d1/d2/d3/f4e [173058,35400] 0 2024-03-29T04:45:23.799 INFO:tasks.workunit.client.1.smithi152.stdout:9/611: symlink d5/d12/ld7 0 2024-03-29T04:45:23.807 INFO:tasks.workunit.client.1.smithi152.stdout:8/593: link d1/d2/d1d/dd3/db0/f78 d1/d8a/fd4 0 2024-03-29T04:45:23.807 INFO:tasks.workunit.client.1.smithi152.stdout:0/631: dwrite d2/da/d17/d57/f8c [4194304,4194304] 0 2024-03-29T04:45:23.810 INFO:tasks.workunit.client.1.smithi152.stdout:9/612: mknod d5/d8/d31/d75/db1/dcd/d6e/db4/cd8 0 2024-03-29T04:45:23.819 INFO:tasks.workunit.client.1.smithi152.stdout:0/632: creat d2/da/d17/fec x:0 0 0 2024-03-29T04:45:23.823 INFO:tasks.workunit.client.1.smithi152.stdout:9/613: rename d5/db/d1b/d68/fad to d5/d12/fd9 0 2024-03-29T04:45:23.825 INFO:tasks.workunit.client.1.smithi152.stdout:0/633: mknod d2/da/d17/d50/dac/ced 0 2024-03-29T04:45:23.825 INFO:tasks.workunit.client.1.smithi152.stdout:0/634: chown d2/da/d3a/de9 57619396 1 2024-03-29T04:45:23.826 INFO:tasks.workunit.client.1.smithi152.stdout:9/614: symlink d5/db/d92/lda 0 2024-03-29T04:45:23.830 INFO:tasks.workunit.client.1.smithi152.stdout:9/615: creat d5/d8/d31/d75/db1/dcd/d6e/d73/fdb x:0 0 0 2024-03-29T04:45:23.853 INFO:tasks.workunit.client.1.smithi152.stdout:9/616: mknod d5/d8/d31/d75/db1/dcd/d6e/d73/d8e/cdc 0 2024-03-29T04:45:23.855 INFO:tasks.workunit.client.1.smithi152.stdout:9/617: rmdir d5/d8/d31/d75/db1/dcd/d6e/d73/d7a 39 2024-03-29T04:45:23.858 INFO:tasks.workunit.client.1.smithi152.stdout:8/594: dread d1/d2/d1d/dd3/db0/f78 [0,4194304] 0 2024-03-29T04:45:23.863 INFO:tasks.workunit.client.1.smithi152.stdout:8/595: dread d1/d2/d1d/f30 [0,4194304] 0 2024-03-29T04:45:23.865 INFO:tasks.workunit.client.1.smithi152.stdout:9/618: link d5/db/d1b/f8b d5/d8/d31/d75/db1/dcd/d6e/fdd 0 2024-03-29T04:45:23.867 INFO:tasks.workunit.client.1.smithi152.stdout:9/619: fsync d5/db/d92/fab 0 2024-03-29T04:45:23.867 INFO:tasks.workunit.client.1.smithi152.stdout:8/596: mkdir d1/d2/d1d/dd3/db0/d40/d6a/dd5 0 2024-03-29T04:45:23.868 INFO:tasks.workunit.client.1.smithi152.stdout:9/620: chown d5/d8/d31/d75/db1/dcd/dc4 15066 1 2024-03-29T04:45:23.874 INFO:tasks.workunit.client.1.smithi152.stdout:0/635: dread d2/da/d17/f6d [0,4194304] 0 2024-03-29T04:45:23.876 INFO:tasks.workunit.client.1.smithi152.stdout:9/621: dwrite d5/d8/d31/d75/db1/dcd/d6e/d91/fc6 [0,4194304] 0 2024-03-29T04:45:23.876 INFO:tasks.workunit.client.1.smithi152.stdout:0/636: write d2/d31/fda [1210391,119902] 0 2024-03-29T04:45:23.877 INFO:tasks.workunit.client.1.smithi152.stdout:0/637: chown d2/f38 15160936 1 2024-03-29T04:45:23.878 INFO:tasks.workunit.client.1.smithi152.stdout:8/597: rmdir d1 39 2024-03-29T04:45:23.881 INFO:tasks.workunit.client.1.smithi152.stdout:0/638: fsync d2/d29/f6c 0 2024-03-29T04:45:23.886 INFO:tasks.workunit.client.1.smithi152.stdout:0/639: chown d2/da/d3a/c43 1 1 2024-03-29T04:45:23.888 INFO:tasks.workunit.client.1.smithi152.stdout:8/598: creat d1/d8a/d91/fd6 x:0 0 0 2024-03-29T04:45:23.891 INFO:tasks.workunit.client.1.smithi152.stdout:9/622: dwrite d5/db/f6c [0,4194304] 0 2024-03-29T04:45:23.891 INFO:tasks.workunit.client.1.smithi152.stdout:8/599: creat d1/d2/d1d/d42/fd7 x:0 0 0 2024-03-29T04:45:23.893 INFO:tasks.workunit.client.1.smithi152.stdout:9/623: fdatasync d5/f5c 0 2024-03-29T04:45:23.894 INFO:tasks.workunit.client.1.smithi152.stdout:8/600: readlink d1/d2/d3/l8 0 2024-03-29T04:45:23.895 INFO:tasks.workunit.client.1.smithi152.stdout:8/601: chown d1/d2/d1d/dd3/db0/d40 1 1 2024-03-29T04:45:23.896 INFO:tasks.workunit.client.1.smithi152.stdout:9/624: symlink d5/d8/d31/d75/db1/dcd/d6e/lde 0 2024-03-29T04:45:23.898 INFO:tasks.workunit.client.1.smithi152.stdout:0/640: write d2/da/d17/dd8/fba [282281,102288] 0 2024-03-29T04:45:23.899 INFO:tasks.workunit.client.1.smithi152.stdout:8/602: mkdir d1/d2/d1d/dd3/db0/d7f/dd8 0 2024-03-29T04:45:23.901 INFO:tasks.workunit.client.1.smithi152.stdout:0/641: truncate d2/da/df/fb3 715648 0 2024-03-29T04:45:23.901 INFO:tasks.workunit.client.1.smithi152.stdout:9/625: rename d5/db/d1b/d68/f89 to d5/d8/d31/d59/d85/fdf 0 2024-03-29T04:45:23.905 INFO:tasks.workunit.client.1.smithi152.stdout:0/642: link d2/d29/c32 d2/da/d17/db5/cee 0 2024-03-29T04:45:23.908 INFO:tasks.workunit.client.1.smithi152.stdout:8/603: dwrite d1/d2/d1d/dd3/db0/d40/d6a/fa3 [0,4194304] 0 2024-03-29T04:45:23.915 INFO:tasks.workunit.client.1.smithi152.stdout:8/604: link d1/d2/d59/f88 d1/d2/d1d/d36/d5d/fd9 0 2024-03-29T04:45:23.915 INFO:tasks.workunit.client.1.smithi152.stdout:0/643: dwrite d2/da/d17/dd8/db8/daf/fd7 [0,4194304] 0 2024-03-29T04:45:23.919 INFO:tasks.workunit.client.1.smithi152.stdout:8/605: mknod d1/d2/d1d/d36/d4b/d51/cda 0 2024-03-29T04:45:23.921 INFO:tasks.workunit.client.1.smithi152.stdout:8/606: creat d1/d8a/d91/fdb x:0 0 0 2024-03-29T04:45:23.923 INFO:tasks.workunit.client.1.smithi152.stdout:0/644: mkdir d2/da/d17/d57/def 0 2024-03-29T04:45:23.924 INFO:tasks.workunit.client.1.smithi152.stdout:8/607: unlink d1/d2/c20 0 2024-03-29T04:45:23.925 INFO:tasks.workunit.client.1.smithi152.stdout:8/608: dread - d1/d2/d1d/d36/d85/fb4 zero size 2024-03-29T04:45:23.925 INFO:tasks.workunit.client.1.smithi152.stdout:0/645: mkdir d2/da/d17/d57/d9c/df0 0 2024-03-29T04:45:23.926 INFO:tasks.workunit.client.1.smithi152.stdout:0/646: readlink d2/l6 0 2024-03-29T04:45:23.927 INFO:tasks.workunit.client.1.smithi152.stdout:9/626: dread d5/db/f4e [0,4194304] 0 2024-03-29T04:45:23.929 INFO:tasks.workunit.client.1.smithi152.stdout:0/647: mknod d2/da/d17/dd8/db8/daf/cf1 0 2024-03-29T04:45:23.930 INFO:tasks.workunit.client.1.smithi152.stdout:0/648: chown d2/da/d3a/fd2 46351821 1 2024-03-29T04:45:23.934 INFO:tasks.workunit.client.1.smithi152.stdout:8/609: dwrite d1/d2/d1d/d42/f92 [0,4194304] 0 2024-03-29T04:45:23.935 INFO:tasks.workunit.client.1.smithi152.stdout:0/649: mkdir d2/da/d17/d57/def/df2 0 2024-03-29T04:45:23.936 INFO:tasks.workunit.client.1.smithi152.stdout:9/627: dread d5/d8/d31/d75/db1/dcd/f39 [0,4194304] 0 2024-03-29T04:45:23.938 INFO:tasks.workunit.client.1.smithi152.stdout:0/650: write d2/da/d17/d50/dc6/fe4 [446180,120632] 0 2024-03-29T04:45:23.939 INFO:tasks.workunit.client.1.smithi152.stdout:9/628: chown d5/d8/d31/d75/db1/dcd/d6e/d73/d8e/cdc 953131 1 2024-03-29T04:45:23.939 INFO:tasks.workunit.client.1.smithi152.stdout:9/629: read d5/db/f6c [1365145,45248] 0 2024-03-29T04:45:23.940 INFO:tasks.workunit.client.1.smithi152.stdout:8/610: read d1/d2/d1d/d36/d4b/d51/f5e [306804,97654] 0 2024-03-29T04:45:23.942 INFO:tasks.workunit.client.1.smithi152.stdout:8/611: write d1/d2/d1d/dc6/fd2 [12845,41225] 0 2024-03-29T04:45:23.948 INFO:tasks.workunit.client.1.smithi152.stdout:8/612: truncate d1/d2/d3c/d76/d9a/fad 599829 0 2024-03-29T04:45:23.948 INFO:tasks.workunit.client.1.smithi152.stdout:9/630: dwrite d5/db/d1b/f4d [8388608,4194304] 0 2024-03-29T04:45:23.948 INFO:tasks.workunit.client.1.smithi152.stdout:0/651: truncate d2/da/d17/db5/fd1 1027909 0 2024-03-29T04:45:23.950 INFO:tasks.workunit.client.1.smithi152.stdout:0/652: stat d2/da/d17/d57/cd9 0 2024-03-29T04:45:23.950 INFO:tasks.workunit.client.1.smithi152.stdout:9/631: chown d5/d8/d31/d59/d85 1420005881 1 2024-03-29T04:45:23.954 INFO:tasks.workunit.client.1.smithi152.stdout:9/632: write d5/d8/d31/d75/db1/dcd/d6e/d73/d7a/fac [676054,1605] 0 2024-03-29T04:45:23.958 INFO:tasks.workunit.client.1.smithi152.stdout:8/613: truncate d1/f7b 2843058 0 2024-03-29T04:45:23.958 INFO:tasks.workunit.client.1.smithi152.stdout:0/653: dwrite d2/feb [0,4194304] 0 2024-03-29T04:45:23.962 INFO:tasks.workunit.client.1.smithi152.stdout:9/633: symlink d5/db/le0 0 2024-03-29T04:45:23.963 INFO:tasks.workunit.client.1.smithi152.stdout:8/614: write d1/d2/d1d/f83 [173774,37218] 0 2024-03-29T04:45:23.964 INFO:tasks.workunit.client.1.smithi152.stdout:0/654: mkdir d2/da/d17/d57/d9c/dbe/df3 0 2024-03-29T04:45:23.968 INFO:tasks.workunit.client.1.smithi152.stdout:8/615: getdents d1/d2/d1d/dd3/db0/d7f/dd8 0 2024-03-29T04:45:23.979 INFO:tasks.workunit.client.1.smithi152.stdout:9/634: write d5/f61 [1087483,24481] 0 2024-03-29T04:45:23.982 INFO:tasks.workunit.client.1.smithi152.stdout:9/635: mkdir d5/d8/d31/d75/db1/dcd/d6e/d91/de1 0 2024-03-29T04:45:23.988 INFO:tasks.workunit.client.1.smithi152.stdout:9/636: dread d5/db/d92/f9a [0,4194304] 0 2024-03-29T04:45:23.988 INFO:tasks.workunit.client.1.smithi152.stdout:7/596: sync 2024-03-29T04:45:23.990 INFO:tasks.workunit.client.1.smithi152.stdout:7/597: dread - d7/d8/d12/d1d/d41/d83/d9a/f8d zero size 2024-03-29T04:45:23.991 INFO:tasks.workunit.client.1.smithi152.stdout:9/637: creat d5/d8/d31/d75/db1/dcd/d6e/d91/fe2 x:0 0 0 2024-03-29T04:45:23.991 INFO:tasks.workunit.client.1.smithi152.stdout:8/616: dread d1/d2/d1d/dd3/db0/d40/f52 [0,4194304] 0 2024-03-29T04:45:23.991 INFO:tasks.workunit.client.1.smithi152.stdout:0/655: dread d2/d29/f2f [0,4194304] 0 2024-03-29T04:45:23.993 INFO:tasks.workunit.client.1.smithi152.stdout:0/656: fsync d2/da/d17/d50/fb7 0 2024-03-29T04:45:23.993 INFO:tasks.workunit.client.1.smithi152.stdout:0/657: fdatasync d2/d29/fea 0 2024-03-29T04:45:23.994 INFO:tasks.workunit.client.1.smithi152.stdout:7/598: unlink d7/d8/l2e 0 2024-03-29T04:45:23.995 INFO:tasks.workunit.client.1.smithi152.stdout:0/658: chown d2/da/d17/d57/d9c/dbe 1897662 1 2024-03-29T04:45:23.997 INFO:tasks.workunit.client.1.smithi152.stdout:9/638: symlink d5/d8/d31/d75/db1/dcd/d6e/d91/dd1/le3 0 2024-03-29T04:45:23.998 INFO:tasks.workunit.client.1.smithi152.stdout:0/659: write d2/da/d17/dd8/db8/daf/fd7 [857665,70559] 0 2024-03-29T04:45:24.000 INFO:tasks.workunit.client.1.smithi152.stdout:8/617: dwrite d1/d2/d1d/dd3/db0/d40/d47/fd0 [0,4194304] 0 2024-03-29T04:45:24.001 INFO:tasks.workunit.client.1.smithi152.stdout:0/660: write d2/da/d3a/f97 [3206847,22656] 0 2024-03-29T04:45:24.002 INFO:tasks.workunit.client.1.smithi152.stdout:8/618: read d1/d2/d1d/dd3/db0/d40/d6a/fa3 [3911511,6149] 0 2024-03-29T04:45:24.004 INFO:tasks.workunit.client.1.smithi152.stdout:0/661: truncate d2/da/d3a/d6f/fc9 435058 0 2024-03-29T04:45:24.007 INFO:tasks.workunit.client.1.smithi152.stdout:8/619: rmdir d1/d2/d1d/d42 39 2024-03-29T04:45:24.010 INFO:tasks.workunit.client.1.smithi152.stdout:9/639: dwrite d5/d8/d31/d75/db1/dcd/f50 [0,4194304] 0 2024-03-29T04:45:24.013 INFO:tasks.workunit.client.1.smithi152.stdout:8/620: mkdir d1/d2/d1d/d36/d4b/d51/ddc 0 2024-03-29T04:45:24.013 INFO:tasks.workunit.client.1.smithi152.stdout:7/599: dwrite d7/d8/d93/fa9 [0,4194304] 0 2024-03-29T04:45:24.014 INFO:tasks.workunit.client.1.smithi152.stdout:8/621: chown d1/d2/d1d/dd3/db0/d7f/dd8 12204 1 2024-03-29T04:45:24.023 INFO:tasks.workunit.client.1.smithi152.stdout:8/622: mkdir d1/d2/d1d/dd3/db0/d40/ddd 0 2024-03-29T04:45:24.026 INFO:tasks.workunit.client.1.smithi152.stdout:8/623: dread - d1/d2/d1d/d42/fd7 zero size 2024-03-29T04:45:24.028 INFO:tasks.workunit.client.1.smithi152.stdout:9/640: rmdir d5/d8/d31/d75/db1/dcd/d9e 0 2024-03-29T04:45:24.032 INFO:tasks.workunit.client.1.smithi152.stdout:7/600: rename d7/d8/d12/d1d/d41/d83/d9a to d7/d8/db3 0 2024-03-29T04:45:24.032 INFO:tasks.workunit.client.1.smithi152.stdout:8/624: dwrite d1/d2/d3/f8e [0,4194304] 0 2024-03-29T04:45:24.033 INFO:tasks.workunit.client.1.smithi152.stdout:9/641: dread d5/d8/d31/d75/db1/dcd/f78 [0,4194304] 0 2024-03-29T04:45:24.034 INFO:tasks.workunit.client.1.smithi152.stdout:8/625: chown d1/d2/d1d/dd3/db0/c53 118 1 2024-03-29T04:45:24.034 INFO:tasks.workunit.client.1.smithi152.stdout:7/601: creat d7/d8/d93/fb4 x:0 0 0 2024-03-29T04:45:24.035 INFO:tasks.workunit.client.1.smithi152.stdout:8/626: dread - d1/d2/d3c/fcb zero size 2024-03-29T04:45:24.037 INFO:tasks.workunit.client.1.smithi152.stdout:7/602: symlink d7/d8/d12/d1d/d41/d83/lb5 0 2024-03-29T04:45:24.037 INFO:tasks.workunit.client.1.smithi152.stdout:8/627: write d1/d2/d1d/dd3/db0/d40/d47/f79 [807757,13591] 0 2024-03-29T04:45:24.040 INFO:tasks.workunit.client.1.smithi152.stdout:8/628: creat d1/d2/d1d/dd3/db0/d40/d6a/dd5/fde x:0 0 0 2024-03-29T04:45:24.041 INFO:tasks.workunit.client.1.smithi152.stdout:7/603: creat d7/d8/d12/fb6 x:0 0 0 2024-03-29T04:45:24.046 INFO:tasks.workunit.client.1.smithi152.stdout:8/629: dwrite d1/d2/d3c/f7a [0,4194304] 0 2024-03-29T04:45:24.054 INFO:tasks.workunit.client.1.smithi152.stdout:8/630: getdents d1/d2/d1d/dd3/db0/d40 0 2024-03-29T04:45:24.057 INFO:tasks.workunit.client.1.smithi152.stdout:8/631: dread d1/d2/d1d/f83 [0,4194304] 0 2024-03-29T04:45:24.058 INFO:tasks.workunit.client.1.smithi152.stdout:9/642: write d5/db/f65 [118884,60386] 0 2024-03-29T04:45:24.058 INFO:tasks.workunit.client.1.smithi152.stdout:9/643: chown d5/d8/d31/d59/l67 24777257 1 2024-03-29T04:45:24.061 INFO:tasks.workunit.client.1.smithi152.stdout:8/632: creat d1/d8a/fdf x:0 0 0 2024-03-29T04:45:24.065 INFO:tasks.workunit.client.1.smithi152.stdout:9/644: link d5/d8/d31/d75/db1/dcd/d6e/d91/fc6 d5/db/d1b/fe4 0 2024-03-29T04:45:24.065 INFO:tasks.workunit.client.1.smithi152.stdout:8/633: mknod d1/d2/d1d/d36/d4b/d51/ddc/ce0 0 2024-03-29T04:45:24.066 INFO:tasks.workunit.client.1.smithi152.stdout:7/604: dwrite d7/d8/d12/d1d/d41/d45/f55 [0,4194304] 0 2024-03-29T04:45:24.068 INFO:tasks.workunit.client.1.smithi152.stdout:9/645: symlink d5/le5 0 2024-03-29T04:45:24.069 INFO:tasks.workunit.client.1.smithi152.stdout:8/634: truncate d1/d2/d1d/d36/d4b/d51/f5e 642094 0 2024-03-29T04:45:24.072 INFO:tasks.workunit.client.1.smithi152.stdout:7/605: getdents d7/d8/d93/d99 0 2024-03-29T04:45:24.072 INFO:tasks.workunit.client.1.smithi152.stdout:8/635: write d1/d2/d1d/dc6/fc9 [359295,47176] 0 2024-03-29T04:45:24.074 INFO:tasks.workunit.client.1.smithi152.stdout:9/646: getdents d5/d8/d31 0 2024-03-29T04:45:24.075 INFO:tasks.workunit.client.1.smithi152.stdout:9/647: chown d5/db/d1b/f46 3 1 2024-03-29T04:45:24.078 INFO:tasks.workunit.client.1.smithi152.stdout:9/648: rename d5/d8/d31/d59/dc3/fd0 to d5/d8/d31/d59/d85/fe6 0 2024-03-29T04:45:24.079 INFO:tasks.workunit.client.1.smithi152.stdout:8/636: getdents d1/d2/d6d 0 2024-03-29T04:45:24.080 INFO:tasks.workunit.client.1.smithi152.stdout:8/637: stat d1/d2/d3/f96 0 2024-03-29T04:45:24.082 INFO:tasks.workunit.client.1.smithi152.stdout:8/638: creat d1/d2/d1d/d36/d4b/d51/fe1 x:0 0 0 2024-03-29T04:45:24.085 INFO:tasks.workunit.client.1.smithi152.stdout:9/649: dwrite d5/d8/d31/d75/db1/dcd/d6e/fa0 [0,4194304] 0 2024-03-29T04:45:24.089 INFO:tasks.workunit.client.1.smithi152.stdout:9/650: truncate d5/d8/d31/d75/db1/dcd/f5e 626839 0 2024-03-29T04:45:24.091 INFO:tasks.workunit.client.1.smithi152.stdout:7/606: write d7/d8/d37/f7f [2076021,130690] 0 2024-03-29T04:45:24.091 INFO:tasks.workunit.client.1.smithi152.stdout:8/639: link d1/d2/d1d/d36/d5d/fd9 d1/fe2 0 2024-03-29T04:45:24.094 INFO:tasks.workunit.client.1.smithi152.stdout:8/640: write d1/d2/d1d/d36/d5d/f71 [1356500,122968] 0 2024-03-29T04:45:24.095 INFO:tasks.workunit.client.1.smithi152.stdout:9/651: rename d5/db/la5 to d5/d8/d31/d59/d85/le7 0 2024-03-29T04:45:24.099 INFO:tasks.workunit.client.1.smithi152.stdout:9/652: rmdir d5/d8/d31/d59 39 2024-03-29T04:45:24.101 INFO:tasks.workunit.client.1.smithi152.stdout:8/641: dwrite d1/d2/d1d/dd3/db0/d40/d6a/dd5/fde [0,4194304] 0 2024-03-29T04:45:24.105 INFO:tasks.workunit.client.1.smithi152.stdout:8/642: mknod d1/d2/d1d/dc6/ce3 0 2024-03-29T04:45:24.106 INFO:tasks.workunit.client.1.smithi152.stdout:8/643: readlink d1/d2/d1d/dd3/db0/l45 0 2024-03-29T04:45:24.106 INFO:tasks.workunit.client.1.smithi152.stdout:9/653: chown d5/d8/d31/d75/db1/dcd/d6e/d73/d7a 464 1 2024-03-29T04:45:24.109 INFO:tasks.workunit.client.1.smithi152.stdout:7/607: truncate d7/d8/d12/d1d/d41/d45/f70 2828052 0 2024-03-29T04:45:24.110 INFO:tasks.workunit.client.1.smithi152.stdout:8/644: mknod d1/d2/d1d/d36/d4b/d51/ddc/ce4 0 2024-03-29T04:45:24.111 INFO:tasks.workunit.client.1.smithi152.stdout:9/654: readlink d5/d8/d31/d59/d85/le7 0 2024-03-29T04:45:24.111 INFO:tasks.workunit.client.1.smithi152.stdout:7/608: symlink d7/d8/d12/d1d/d41/d83/lb7 0 2024-03-29T04:45:24.112 INFO:tasks.workunit.client.1.smithi152.stdout:8/645: creat d1/d8a/d91/fe5 x:0 0 0 2024-03-29T04:45:24.114 INFO:tasks.workunit.client.1.smithi152.stdout:7/609: creat d7/d8/db/df/fb8 x:0 0 0 2024-03-29T04:45:24.117 INFO:tasks.workunit.client.1.smithi152.stdout:9/655: dwrite d5/f61 [0,4194304] 0 2024-03-29T04:45:24.117 INFO:tasks.workunit.client.1.smithi152.stdout:7/610: write d7/d8/db/df/f3a [4927971,128256] 0 2024-03-29T04:45:24.120 INFO:tasks.workunit.client.1.smithi152.stdout:7/611: write d7/d8/d12/d1d/f98 [68671,30075] 0 2024-03-29T04:45:24.123 INFO:tasks.workunit.client.1.smithi152.stdout:9/656: creat d5/db/d1b/d68/fe8 x:0 0 0 2024-03-29T04:45:24.126 INFO:tasks.workunit.client.1.smithi152.stdout:7/612: rmdir d7/d8/d12/d1d 39 2024-03-29T04:45:24.130 INFO:tasks.workunit.client.1.smithi152.stdout:8/646: write d1/d2/d1d/f83 [688003,55703] 0 2024-03-29T04:45:24.131 INFO:tasks.workunit.client.1.smithi152.stdout:8/647: chown d1/d2/d1d/d36/d85/caa 0 1 2024-03-29T04:45:24.132 INFO:tasks.workunit.client.1.smithi152.stdout:7/613: dread d7/d8/d12/d35/f6d [0,4194304] 0 2024-03-29T04:45:24.133 INFO:tasks.workunit.client.1.smithi152.stdout:9/657: link d5/db/d1b/f49 d5/d8/d31/d59/d85/fe9 0 2024-03-29T04:45:24.138 INFO:tasks.workunit.client.1.smithi152.stdout:7/614: symlink d7/d8/lb9 0 2024-03-29T04:45:24.139 INFO:tasks.workunit.client.1.smithi152.stdout:8/648: rename d1/d2/d1d/dd3/db0/d40/d47/c60 to d1/ce6 0 2024-03-29T04:45:24.141 INFO:tasks.workunit.client.1.smithi152.stdout:7/615: mkdir d7/d8/d12/d35/dba 0 2024-03-29T04:45:24.144 INFO:tasks.workunit.client.1.smithi152.stdout:8/649: dwrite d1/d8a/fb3 [0,4194304] 0 2024-03-29T04:45:24.151 INFO:tasks.workunit.client.1.smithi152.stdout:4/601: sync 2024-03-29T04:45:24.151 INFO:tasks.workunit.client.1.smithi152.stdout:3/683: sync 2024-03-29T04:45:24.152 INFO:tasks.workunit.client.1.smithi152.stdout:9/658: dwrite d5/db/d1b/fe4 [0,4194304] 0 2024-03-29T04:45:24.156 INFO:tasks.workunit.client.1.smithi152.stdout:4/602: mknod d0/d7/db2/cbe 0 2024-03-29T04:45:24.156 INFO:tasks.workunit.client.1.smithi152.stdout:8/650: dread d1/d2/d1d/d36/d85/f99 [0,4194304] 0 2024-03-29T04:45:24.157 INFO:tasks.workunit.client.1.smithi152.stdout:9/659: mkdir d5/d8/d31/d75/db1/dcd/d6e/dea 0 2024-03-29T04:45:24.157 INFO:tasks.workunit.client.1.smithi152.stdout:4/603: chown d0/d7/d40/d62 633 1 2024-03-29T04:45:24.158 INFO:tasks.workunit.client.1.smithi152.stdout:7/616: dwrite d7/d8/db/f42 [0,4194304] 0 2024-03-29T04:45:24.162 INFO:tasks.workunit.client.1.smithi152.stdout:3/684: creat d7/d19/d1b/d1e/da1/d41/d61/fcf x:0 0 0 2024-03-29T04:45:24.164 INFO:tasks.workunit.client.1.smithi152.stdout:4/604: mknod d0/d7/d25/d17/d5e/cbf 0 2024-03-29T04:45:24.174 INFO:tasks.workunit.client.1.smithi152.stdout:8/651: mknod d1/d2/d1d/dd3/db0/d40/ddd/ce7 0 2024-03-29T04:45:24.174 INFO:tasks.workunit.client.1.smithi152.stdout:7/617: symlink d7/d8/d12/d1d/lbb 0 2024-03-29T04:45:24.175 INFO:tasks.workunit.client.1.smithi152.stdout:4/605: fdatasync d0/d7/d25/d17/d5e/d60/d74/f50 0 2024-03-29T04:45:24.182 INFO:tasks.workunit.client.1.smithi152.stdout:9/660: truncate d5/db/f21 3789247 0 2024-03-29T04:45:24.182 INFO:tasks.workunit.client.1.smithi152.stdout:9/661: readlink d5/d12/ld7 0 2024-03-29T04:45:24.182 INFO:tasks.workunit.client.1.smithi152.stdout:7/618: creat d7/d8/d93/d99/fbc x:0 0 0 2024-03-29T04:45:24.184 INFO:tasks.workunit.client.1.smithi152.stdout:7/619: write d7/d8/f3b [2152245,52833] 0 2024-03-29T04:45:24.184 INFO:tasks.workunit.client.1.smithi152.stdout:9/662: write d5/d8/d31/d75/db1/dcd/d6e/d73/fdb [436179,42651] 0 2024-03-29T04:45:24.184 INFO:tasks.workunit.client.1.smithi152.stdout:3/685: truncate d7/d19/d3a/f55 1112548 0 2024-03-29T04:45:24.186 INFO:tasks.workunit.client.1.smithi152.stdout:8/652: link d1/d8a/fdf d1/d2/d59/fe8 0 2024-03-29T04:45:24.187 INFO:tasks.workunit.client.1.smithi152.stdout:8/653: chown d1/d2/d6d/c7d 2070340 1 2024-03-29T04:45:24.189 INFO:tasks.workunit.client.1.smithi152.stdout:3/686: creat d7/d19/d1b/d1e/da1/d41/d61/d63/fd0 x:0 0 0 2024-03-29T04:45:24.192 INFO:tasks.workunit.client.1.smithi152.stdout:6/871: sync 2024-03-29T04:45:24.192 INFO:tasks.workunit.client.1.smithi152.stdout:0/662: sync 2024-03-29T04:45:24.192 INFO:tasks.workunit.client.1.smithi152.stdout:2/640: sync 2024-03-29T04:45:24.192 INFO:tasks.workunit.client.1.smithi152.stdout:1/994: sync 2024-03-29T04:45:24.193 INFO:tasks.workunit.client.1.smithi152.stdout:5/806: sync 2024-03-29T04:45:24.198 INFO:tasks.workunit.client.1.smithi152.stdout:6/872: write d1/d3/d9/d13/d28/d45/f80 [1718620,108664] 0 2024-03-29T04:45:24.200 INFO:tasks.workunit.client.1.smithi152.stdout:4/606: dread d0/d7/d40/d23/f28 [0,4194304] 0 2024-03-29T04:45:24.201 INFO:tasks.workunit.client.1.smithi152.stdout:2/641: symlink d0/d3/d64/lbe 0 2024-03-29T04:45:24.202 INFO:tasks.workunit.client.1.smithi152.stdout:8/654: dwrite d1/d2/d1d/d36/d85/fc1 [0,4194304] 0 2024-03-29T04:45:24.204 INFO:tasks.workunit.client.1.smithi152.stdout:5/807: creat d6/d16/d109/f11f x:0 0 0 2024-03-29T04:45:24.204 INFO:tasks.workunit.client.1.smithi152.stdout:6/873: write d1/d3/d1a/fb2 [639675,11055] 0 2024-03-29T04:45:24.207 INFO:tasks.workunit.client.1.smithi152.stdout:1/995: creat d5/d7/d1b/d2e/d95/d9d/f145 x:0 0 0 2024-03-29T04:45:24.209 INFO:tasks.workunit.client.1.smithi152.stdout:0/663: rename d2/da/d17/d50/dac/ced to d2/da/d17/d57/d9c/cf4 0 2024-03-29T04:45:24.209 INFO:tasks.workunit.client.1.smithi152.stdout:5/808: fdatasync d6/d1d/d3b/f39 0 2024-03-29T04:45:24.211 INFO:tasks.workunit.client.1.smithi152.stdout:1/996: chown d5/d7/d1b/d2e/d95/d9d/f145 13 1 2024-03-29T04:45:24.213 INFO:tasks.workunit.client.1.smithi152.stdout:7/620: dread d7/d8/fa [0,4194304] 0 2024-03-29T04:45:24.215 INFO:tasks.workunit.client.1.smithi152.stdout:2/642: write d0/d3/d30/d15/d53/d54/dad/fba [474022,32968] 0 2024-03-29T04:45:24.216 INFO:tasks.workunit.client.1.smithi152.stdout:7/621: chown d7/d8/d12/d1d/c7c 1 1 2024-03-29T04:45:24.216 INFO:tasks.workunit.client.1.smithi152.stdout:8/655: creat d1/d2/d3c/d76/fe9 x:0 0 0 2024-03-29T04:45:24.219 INFO:tasks.workunit.client.1.smithi152.stdout:3/687: dread d7/d19/d3a/f6e [0,4194304] 0 2024-03-29T04:45:24.219 INFO:tasks.workunit.client.1.smithi152.stdout:2/643: write d0/d3/df/d8c/fb1 [85501,116288] 0 2024-03-29T04:45:24.219 INFO:tasks.workunit.client.1.smithi152.stdout:0/664: dwrite d2/d29/f37 [4194304,4194304] 0 2024-03-29T04:45:24.219 INFO:tasks.workunit.client.1.smithi152.stdout:6/874: rmdir d1/d3/d1a/d1b/d29/d6f/dee/d10f 39 2024-03-29T04:45:24.220 INFO:tasks.workunit.client.1.smithi152.stdout:8/656: read - d1/d2/d1d/dd3/db0/da9/fbc zero size 2024-03-29T04:45:24.222 INFO:tasks.workunit.client.1.smithi152.stdout:3/688: chown d7/d19/d1b/d59/f68 5271 1 2024-03-29T04:45:24.222 INFO:tasks.workunit.client.1.smithi152.stdout:7/622: dread d7/d8/d12/d35/d68/d84/fa4 [0,4194304] 0 2024-03-29T04:45:24.223 INFO:tasks.workunit.client.1.smithi152.stdout:1/997: chown d5/d7/d8/d13/d19/d80/c141 6074699 1 2024-03-29T04:45:24.224 INFO:tasks.workunit.client.1.smithi152.stdout:4/607: write d0/d7/d40/d23/d55/d70/d96/f9f [954385,26255] 0 2024-03-29T04:45:24.229 INFO:tasks.workunit.client.1.smithi152.stdout:2/644: fsync d0/d3/df/d2c/f55 0 2024-03-29T04:45:24.231 INFO:tasks.workunit.client.1.smithi152.stdout:6/875: unlink d1/d3/d9/d13/f46 0 2024-03-29T04:45:24.233 INFO:tasks.workunit.client.1.smithi152.stdout:6/876: write d1/d3/d1a/d1b/d29/dce/fda [559273,5961] 0 2024-03-29T04:45:24.234 INFO:tasks.workunit.client.1.smithi152.stdout:8/657: creat d1/d2/d1d/d42/fea x:0 0 0 2024-03-29T04:45:24.236 INFO:tasks.workunit.client.1.smithi152.stdout:5/809: write d6/d1d/d30/fa2 [1799304,29041] 0 2024-03-29T04:45:24.238 INFO:tasks.workunit.client.1.smithi152.stdout:5/810: chown d6/d16/d25/d2e/da0/lb5 9723 1 2024-03-29T04:45:24.238 INFO:tasks.workunit.client.1.smithi152.stdout:1/998: dread - d5/d7/d1b/d2e/de2/ff6 zero size 2024-03-29T04:45:24.238 INFO:tasks.workunit.client.1.smithi152.stdout:0/665: write d2/da/d3a/f81 [702220,56672] 0 2024-03-29T04:45:24.242 INFO:tasks.workunit.client.1.smithi152.stdout:6/877: mknod d1/d3/d91/c128 0 2024-03-29T04:45:24.248 INFO:tasks.workunit.client.1.smithi152.stdout:2/645: unlink d0/d3/d30/d15/f20 0 2024-03-29T04:45:24.249 INFO:tasks.workunit.client.1.smithi152.stdout:0/666: rename d2/da/d3a/d61/l88 to d2/da/d17/d50/dc6/lf5 0 2024-03-29T04:45:24.250 INFO:tasks.workunit.client.1.smithi152.stdout:0/667: readlink d2/da/d3a/d6f/lae 0 2024-03-29T04:45:24.252 INFO:tasks.workunit.client.1.smithi152.stdout:8/658: write d1/f7c [334602,77451] 0 2024-03-29T04:45:24.253 INFO:tasks.workunit.client.1.smithi152.stdout:3/689: creat d7/d19/d1b/d1e/da1/fd1 x:0 0 0 2024-03-29T04:45:24.254 INFO:tasks.workunit.client.1.smithi152.stdout:7/623: creat d7/d8/d12/fbd x:0 0 0 2024-03-29T04:45:24.258 INFO:tasks.workunit.client.1.smithi152.stdout:7/624: dread - d7/d8/d12/d1d/f8c zero size 2024-03-29T04:45:24.260 INFO:tasks.workunit.client.1.smithi152.stdout:0/668: mknod d2/da/d3a/de9/cf6 0 2024-03-29T04:45:24.262 INFO:tasks.workunit.client.1.smithi152.stdout:6/878: creat d1/f129 x:0 0 0 2024-03-29T04:45:24.262 INFO:tasks.workunit.client.1.smithi152.stdout:6/879: stat d1/f94 0 2024-03-29T04:45:24.265 INFO:tasks.workunit.client.1.smithi152.stdout:8/659: link d1/d2/d3/f96 d1/d2/d1d/dd3/db0/d40/d58/feb 0 2024-03-29T04:45:24.267 INFO:tasks.workunit.client.1.smithi152.stdout:6/880: dwrite d1/d3/d9/d13/d28/d45/d8c/f123 [0,4194304] 0 2024-03-29T04:45:24.268 INFO:tasks.workunit.client.1.smithi152.stdout:3/690: symlink d7/d19/ld2 0 2024-03-29T04:45:24.272 INFO:tasks.workunit.client.1.smithi152.stdout:1/999: getdents d5/d7/d1b/d2e/d95/d9d 0 2024-03-29T04:45:24.276 INFO:tasks.workunit.client.1.smithi152.stdout:2/646: creat d0/d3/df/fbf x:0 0 0 2024-03-29T04:45:24.279 INFO:tasks.workunit.client.1.smithi152.stdout:0/669: creat d2/da/ff7 x:0 0 0 2024-03-29T04:45:24.281 INFO:tasks.workunit.client.1.smithi152.stdout:2/647: read d0/d3/df/d2c/f2d [283124,114406] 0 2024-03-29T04:45:24.284 INFO:tasks.workunit.client.1.smithi152.stdout:7/625: dwrite d7/d8/d37/f46 [4194304,4194304] 0 2024-03-29T04:45:24.288 INFO:tasks.workunit.client.1.smithi152.stdout:6/881: mknod d1/d3/d9/d13/da7/df5/dfd/c12a 0 2024-03-29T04:45:24.291 INFO:tasks.workunit.client.1.smithi152.stdout:3/691: fsync d7/d19/d1b/d1e/f82 0 2024-03-29T04:45:24.297 INFO:tasks.workunit.client.1.smithi152.stdout:7/626: symlink d7/d8/d12/d1d/d41/lbe 0 2024-03-29T04:45:24.297 INFO:tasks.workunit.client.1.smithi152.stdout:7/627: fdatasync d7/d8/db3/f76 0 2024-03-29T04:45:24.300 INFO:tasks.workunit.client.1.smithi152.stdout:6/882: rename d1/d3/d9/d13/d28/cb1 to d1/d3/df/d84/c12b 0 2024-03-29T04:45:24.302 INFO:tasks.workunit.client.1.smithi152.stdout:0/670: write d2/da/d3a/d6f/fa2 [498818,69902] 0 2024-03-29T04:45:24.303 INFO:tasks.workunit.client.1.smithi152.stdout:2/648: creat d0/d3/df/db8/fc0 x:0 0 0 2024-03-29T04:45:24.306 INFO:tasks.workunit.client.1.smithi152.stdout:7/628: rmdir d7/d8/d12/d35/d68 39 2024-03-29T04:45:24.307 INFO:tasks.workunit.client.1.smithi152.stdout:6/883: mknod d1/d3/d9/d13/d3e/d54/c12c 0 2024-03-29T04:45:24.309 INFO:tasks.workunit.client.1.smithi152.stdout:2/649: dread d0/d3/f81 [0,4194304] 0 2024-03-29T04:45:24.309 INFO:tasks.workunit.client.1.smithi152.stdout:3/692: dwrite d7/d46/f98 [0,4194304] 0 2024-03-29T04:45:24.310 INFO:tasks.workunit.client.1.smithi152.stdout:6/884: chown d1/d3/df/d8a 117734 1 2024-03-29T04:45:24.314 INFO:tasks.workunit.client.1.smithi152.stdout:0/671: dwrite d2/da/d3a/f91 [0,4194304] 0 2024-03-29T04:45:24.319 INFO:tasks.workunit.client.1.smithi152.stdout:7/629: dwrite d7/d8/db/f4b [0,4194304] 0 2024-03-29T04:45:24.320 INFO:tasks.workunit.client.1.smithi152.stdout:0/672: readlink d2/da/d3a/lb6 0 2024-03-29T04:45:24.321 INFO:tasks.workunit.client.1.smithi152.stdout:2/650: creat d0/d3/d30/d15/d53/db4/fc1 x:0 0 0 2024-03-29T04:45:24.323 INFO:tasks.workunit.client.1.smithi152.stdout:3/693: creat d7/fd3 x:0 0 0 2024-03-29T04:45:24.325 INFO:tasks.workunit.client.1.smithi152.stdout:2/651: write d0/d3/d30/d15/d53/d54/d57/f72 [1799506,125976] 0 2024-03-29T04:45:24.329 INFO:tasks.workunit.client.1.smithi152.stdout:0/673: creat d2/da/d17/db5/ff8 x:0 0 0 2024-03-29T04:45:24.329 INFO:tasks.workunit.client.1.smithi152.stdout:6/885: dread d1/d3/d9/f21 [0,4194304] 0 2024-03-29T04:45:24.330 INFO:tasks.workunit.client.1.smithi152.stdout:7/630: rename d7/d8/d12/f22 to d7/d8/d93/fbf 0 2024-03-29T04:45:24.332 INFO:tasks.workunit.client.1.smithi152.stdout:7/631: write d7/d8/db3/f76 [1215461,1960] 0 2024-03-29T04:45:24.333 INFO:tasks.workunit.client.1.smithi152.stdout:3/694: mknod d7/d19/d1b/d1e/cd4 0 2024-03-29T04:45:24.334 INFO:tasks.workunit.client.1.smithi152.stdout:6/886: creat d1/d3/d91/f12d x:0 0 0 2024-03-29T04:45:24.336 INFO:tasks.workunit.client.1.smithi152.stdout:0/674: mkdir d2/da/d17/d57/d9c/df9 0 2024-03-29T04:45:24.340 INFO:tasks.workunit.client.1.smithi152.stdout:3/695: rename d7/d19/f36 to d7/fd5 0 2024-03-29T04:45:24.345 INFO:tasks.workunit.client.1.smithi152.stdout:6/887: dwrite d1/d3/d9/d13/f31 [0,4194304] 0 2024-03-29T04:45:24.346 INFO:tasks.workunit.client.1.smithi152.stdout:0/675: dread d2/da/d3a/d61/fde [0,4194304] 0 2024-03-29T04:45:24.348 INFO:tasks.workunit.client.1.smithi152.stdout:3/696: dread d7/d19/d1b/d1e/f82 [0,4194304] 0 2024-03-29T04:45:24.349 INFO:tasks.workunit.client.1.smithi152.stdout:6/888: mknod d1/d3/df/dd2/c12e 0 2024-03-29T04:45:24.355 INFO:tasks.workunit.client.1.smithi152.stdout:6/889: creat d1/d3/d1a/d1b/d10d/f12f x:0 0 0 2024-03-29T04:45:24.357 INFO:tasks.workunit.client.1.smithi152.stdout:2/652: dread d0/d3/f5 [0,4194304] 0 2024-03-29T04:45:24.357 INFO:tasks.workunit.client.1.smithi152.stdout:6/890: stat d1/d3/d1a/d1b/d29/fcc 0 2024-03-29T04:45:24.357 INFO:tasks.workunit.client.1.smithi152.stdout:3/697: unlink d7/d19/d1b/d1e/da1/d41/d6a/d78/f96 0 2024-03-29T04:45:24.360 INFO:tasks.workunit.client.1.smithi152.stdout:3/698: rmdir d7/d19/d1b/d1e/d22 39 2024-03-29T04:45:24.362 INFO:tasks.workunit.client.1.smithi152.stdout:7/632: dwrite d7/d8/d12/d35/d68/d84/fa4 [0,4194304] 0 2024-03-29T04:45:24.365 INFO:tasks.workunit.client.1.smithi152.stdout:2/653: rename d0/d3/d30/d15/d22/f2f to d0/d3/df/db9/fc2 0 2024-03-29T04:45:24.369 INFO:tasks.workunit.client.1.smithi152.stdout:2/654: dread - d0/d3/df/fa2 zero size 2024-03-29T04:45:24.371 INFO:tasks.workunit.client.1.smithi152.stdout:3/699: creat d7/d19/d1b/d1e/d4e/d8f/fd6 x:0 0 0 2024-03-29T04:45:24.372 INFO:tasks.workunit.client.1.smithi152.stdout:3/700: fsync d7/d19/d1b/d59/fcb 0 2024-03-29T04:45:24.373 INFO:tasks.workunit.client.1.smithi152.stdout:7/633: dwrite d7/d8/d93/d99/fad [0,4194304] 0 2024-03-29T04:45:24.374 INFO:tasks.workunit.client.1.smithi152.stdout:2/655: fdatasync d0/d3/df/d2c/f2d 0 2024-03-29T04:45:24.374 INFO:tasks.workunit.client.1.smithi152.stdout:3/701: write d7/d19/d3a/f7f [811364,48063] 0 2024-03-29T04:45:24.376 INFO:tasks.workunit.client.1.smithi152.stdout:7/634: symlink d7/d8/db3/lc0 0 2024-03-29T04:45:24.377 INFO:tasks.workunit.client.1.smithi152.stdout:3/702: creat d7/d6c/fd7 x:0 0 0 2024-03-29T04:45:24.381 INFO:tasks.workunit.client.1.smithi152.stdout:0/676: dread d2/da/df/f4b [0,4194304] 0 2024-03-29T04:45:24.383 INFO:tasks.workunit.client.1.smithi152.stdout:3/703: write d7/d19/d1b/d1e/d22/fb0 [189759,3599] 0 2024-03-29T04:45:24.396 INFO:tasks.workunit.client.1.smithi152.stdout:3/704: dwrite d7/d46/f9f [0,4194304] 0 2024-03-29T04:45:24.397 INFO:tasks.workunit.client.1.smithi152.stdout:2/656: dread d0/d3/d30/d15/d22/f37 [0,4194304] 0 2024-03-29T04:45:24.398 INFO:tasks.workunit.client.1.smithi152.stdout:6/891: dwrite d1/d3/df/dd2/fe2 [0,4194304] 0 2024-03-29T04:45:24.400 INFO:tasks.workunit.client.1.smithi152.stdout:2/657: chown d0/d3/ld 30 1 2024-03-29T04:45:24.404 INFO:tasks.workunit.client.1.smithi152.stdout:7/635: dread d7/d8/d12/f28 [0,4194304] 0 2024-03-29T04:45:24.409 INFO:tasks.workunit.client.1.smithi152.stdout:2/658: rename d0/d3/f5 to d0/d3/d30/d15/d53/fc3 0 2024-03-29T04:45:24.409 INFO:tasks.workunit.client.1.smithi152.stdout:0/677: dread d2/da/d3a/f81 [0,4194304] 0 2024-03-29T04:45:24.411 INFO:tasks.workunit.client.1.smithi152.stdout:6/892: dwrite d1/d3/f18 [0,4194304] 0 2024-03-29T04:45:24.413 INFO:tasks.workunit.client.1.smithi152.stdout:2/659: symlink d0/d3/d5f/lc4 0 2024-03-29T04:45:24.416 INFO:tasks.workunit.client.1.smithi152.stdout:0/678: chown d2/da/d17/d57/ca5 100 1 2024-03-29T04:45:24.418 INFO:tasks.workunit.client.1.smithi152.stdout:2/660: creat d0/d3/d30/d15/d22/d4c/d5a/fc5 x:0 0 0 2024-03-29T04:45:24.419 INFO:tasks.workunit.client.1.smithi152.stdout:2/661: readlink d0/d3/df/d2c/d45/lab 0 2024-03-29T04:45:24.420 INFO:tasks.workunit.client.1.smithi152.stdout:6/893: truncate d1/d3/d1a/d1b/d29/d6f/f69 4433897 0 2024-03-29T04:45:24.421 INFO:tasks.workunit.client.1.smithi152.stdout:7/636: getdents d7/d8/d12/d1d/d41 0 2024-03-29T04:45:24.421 INFO:tasks.workunit.client.1.smithi152.stdout:0/679: mknod d2/da/d17/de7/cfa 0 2024-03-29T04:45:24.422 INFO:tasks.workunit.client.1.smithi152.stdout:7/637: readlink d7/d8/d12/d1d/d41/la1 0 2024-03-29T04:45:24.423 INFO:tasks.workunit.client.1.smithi152.stdout:0/680: write d2/da/d3a/f97 [7430615,55151] 0 2024-03-29T04:45:24.424 INFO:tasks.workunit.client.1.smithi152.stdout:7/638: fsync d7/d8/d12/f61 0 2024-03-29T04:45:24.424 INFO:tasks.workunit.client.1.smithi152.stdout:6/894: symlink d1/d3/d9/dcb/l130 0 2024-03-29T04:45:24.426 INFO:tasks.workunit.client.1.smithi152.stdout:0/681: creat d2/da/d3a/d61/db4/ffb x:0 0 0 2024-03-29T04:45:24.427 INFO:tasks.workunit.client.1.smithi152.stdout:6/895: symlink d1/d3/d9/d13/d3e/d93/l131 0 2024-03-29T04:45:24.427 INFO:tasks.workunit.client.1.smithi152.stdout:3/705: dread d7/d19/d3a/f49 [0,4194304] 0 2024-03-29T04:45:24.428 INFO:tasks.workunit.client.1.smithi152.stdout:9/663: sync 2024-03-29T04:45:24.428 INFO:tasks.workunit.client.1.smithi152.stdout:4/608: sync 2024-03-29T04:45:24.429 INFO:tasks.workunit.client.1.smithi152.stdout:7/639: link d7/d8/d12/d35/d68/d84/c85 d7/cc1 0 2024-03-29T04:45:24.430 INFO:tasks.workunit.client.1.smithi152.stdout:6/896: truncate d1/d3/d1a/d1b/d29/fc4 2218551 0 2024-03-29T04:45:24.432 INFO:tasks.workunit.client.1.smithi152.stdout:3/706: rmdir d7/d19/d1b/d1e/d4e/d8f 39 2024-03-29T04:45:24.433 INFO:tasks.workunit.client.1.smithi152.stdout:3/707: dread - d7/d6c/fca zero size 2024-03-29T04:45:24.437 INFO:tasks.workunit.client.1.smithi152.stdout:6/897: write d1/d3/d1a/d1b/f1e [186296,110304] 0 2024-03-29T04:45:24.438 INFO:tasks.workunit.client.1.smithi152.stdout:3/708: stat d7/d19/f26 0 2024-03-29T04:45:24.439 INFO:tasks.workunit.client.1.smithi152.stdout:7/640: dread d7/d8/d12/d1d/d41/f4d [0,4194304] 0 2024-03-29T04:45:24.439 INFO:tasks.workunit.client.1.smithi152.stdout:0/682: getdents d2/da/d17/d50/dac 0 2024-03-29T04:45:24.446 INFO:tasks.workunit.client.1.smithi152.stdout:2/662: dwrite d0/d3/d30/d15/f25 [0,4194304] 0 2024-03-29T04:45:24.446 INFO:tasks.workunit.client.1.smithi152.stdout:9/664: rename d5/d8/f69 to d5/db/feb 0 2024-03-29T04:45:24.448 INFO:tasks.workunit.client.1.smithi152.stdout:8/660: sync 2024-03-29T04:45:24.449 INFO:tasks.workunit.client.1.smithi152.stdout:5/811: sync 2024-03-29T04:45:24.456 INFO:tasks.workunit.client.1.smithi152.stdout:4/609: write d0/d7/d25/d17/d5e/d60/d74/d57/f5c [751877,103081] 0 2024-03-29T04:45:24.456 INFO:tasks.workunit.client.1.smithi152.stdout:0/683: write d2/da/d3a/d6f/f85 [3928009,37590] 0 2024-03-29T04:45:24.456 INFO:tasks.workunit.client.1.smithi152.stdout:6/898: creat d1/d3/d9/d13/dc0/dc3/dfb/d112/f132 x:0 0 0 2024-03-29T04:45:24.457 INFO:tasks.workunit.client.1.smithi152.stdout:7/641: truncate d7/d8/d12/d1d/f62 542559 0 2024-03-29T04:45:24.458 INFO:tasks.workunit.client.1.smithi152.stdout:3/709: rename d7/fd3 to d7/d19/d1b/d59/fd8 0 2024-03-29T04:45:24.458 INFO:tasks.workunit.client.1.smithi152.stdout:2/663: mkdir d0/d3/d30/d15/d53/db4/dc6 0 2024-03-29T04:45:24.459 INFO:tasks.workunit.client.1.smithi152.stdout:7/642: stat d7/d8/db3/f3e 0 2024-03-29T04:45:24.462 INFO:tasks.workunit.client.1.smithi152.stdout:7/643: write d7/d8/d12/f87 [698528,123840] 0 2024-03-29T04:45:24.468 INFO:tasks.workunit.client.1.smithi152.stdout:7/644: dwrite d7/d8/d37/f7d [0,4194304] 0 2024-03-29T04:45:24.472 INFO:tasks.workunit.client.1.smithi152.stdout:8/661: creat d1/dc5/fec x:0 0 0 2024-03-29T04:45:24.482 INFO:tasks.workunit.client.1.smithi152.stdout:3/710: mknod d7/d19/d1b/d1e/da1/d41/d61/d75/cd9 0 2024-03-29T04:45:24.485 INFO:tasks.workunit.client.1.smithi152.stdout:6/899: dread d1/d3/d9/d13/d3e/d54/ff4 [0,4194304] 0 2024-03-29T04:45:24.487 INFO:tasks.workunit.client.1.smithi152.stdout:7/645: symlink d7/d8/d12/d1d/d41/lc2 0 2024-03-29T04:45:24.492 INFO:tasks.workunit.client.1.smithi152.stdout:7/646: dwrite d7/d8/d93/fa9 [0,4194304] 0 2024-03-29T04:45:24.492 INFO:tasks.workunit.client.1.smithi152.stdout:0/684: getdents d2/da/d17/d57/d9c/df9 0 2024-03-29T04:45:24.496 INFO:tasks.workunit.client.1.smithi152.stdout:3/711: readlink d7/d19/d1b/d1e/d4e/l86 0 2024-03-29T04:45:24.499 INFO:tasks.workunit.client.1.smithi152.stdout:6/900: truncate d1/d3/d1a/f61 1030776 0 2024-03-29T04:45:24.499 INFO:tasks.workunit.client.1.smithi152.stdout:4/610: dwrite d0/d7/f8d [4194304,4194304] 0 2024-03-29T04:45:24.501 INFO:tasks.workunit.client.1.smithi152.stdout:8/662: mknod d1/d2/ced 0 2024-03-29T04:45:24.502 INFO:tasks.workunit.client.1.smithi152.stdout:7/647: dwrite d7/d8/d12/fbd [0,4194304] 0 2024-03-29T04:45:24.506 INFO:tasks.workunit.client.1.smithi152.stdout:9/665: rename d5/d8/d31/d75/db1/dcd/d6e/d73/f7f to d5/db/fec 0 2024-03-29T04:45:24.511 INFO:tasks.workunit.client.1.smithi152.stdout:2/664: truncate d0/d3/d5f/f92 640724 0 2024-03-29T04:45:24.511 INFO:tasks.workunit.client.1.smithi152.stdout:2/665: readlink d0/l71 0 2024-03-29T04:45:24.512 INFO:tasks.workunit.client.1.smithi152.stdout:6/901: creat d1/d3/d1a/d1b/d29/d6f/d95/f133 x:0 0 0 2024-03-29T04:45:24.513 INFO:tasks.workunit.client.1.smithi152.stdout:4/611: mkdir d0/d7/d40/d23/d55/d70/d96/dc0 0 2024-03-29T04:45:24.513 INFO:tasks.workunit.client.1.smithi152.stdout:6/902: fdatasync d1/d3/d1a/d1b/d29/f79 0 2024-03-29T04:45:24.514 INFO:tasks.workunit.client.1.smithi152.stdout:6/903: chown d1/d3/d1a/f2c 110 1 2024-03-29T04:45:24.515 INFO:tasks.workunit.client.1.smithi152.stdout:5/812: rename d6/d1d/d30/d6a/d108/fbe to d6/d16/d25/d2e/da0/d92/dc0/dec/f120 0 2024-03-29T04:45:24.516 INFO:tasks.workunit.client.1.smithi152.stdout:9/666: mkdir d5/d8/d31/d59/dc3/ded 0 2024-03-29T04:45:24.521 INFO:tasks.workunit.client.1.smithi152.stdout:0/685: rename d2/da/fb to d2/da/df/dbb/ffc 0 2024-03-29T04:45:24.523 INFO:tasks.workunit.client.1.smithi152.stdout:5/813: creat d6/d1d/d30/f121 x:0 0 0 2024-03-29T04:45:24.523 INFO:tasks.workunit.client.1.smithi152.stdout:9/667: creat d5/d8/d31/d75/db1/dcd/d6e/d91/fee x:0 0 0 2024-03-29T04:45:24.524 INFO:tasks.workunit.client.1.smithi152.stdout:5/814: chown d6/d1d/d30 119378143 1 2024-03-29T04:45:24.525 INFO:tasks.workunit.client.1.smithi152.stdout:3/712: rmdir d7/d19/d1b/d1e/da1/d41/daf 0 2024-03-29T04:45:24.528 INFO:tasks.workunit.client.1.smithi152.stdout:0/686: dwrite d2/d29/f37 [0,4194304] 0 2024-03-29T04:45:24.533 INFO:tasks.workunit.client.1.smithi152.stdout:8/663: rename d1/d2/d1d/d42 to d1/d8a/d91/dee 0 2024-03-29T04:45:24.538 INFO:tasks.workunit.client.1.smithi152.stdout:0/687: dwrite d2/da/d17/dd8/db8/daf/fd7 [0,4194304] 0 2024-03-29T04:45:24.540 INFO:tasks.workunit.client.1.smithi152.stdout:5/815: creat d6/d1d/daa/db7/d56/dc1/f122 x:0 0 0 2024-03-29T04:45:24.541 INFO:tasks.workunit.client.1.smithi152.stdout:3/713: truncate d7/d19/f26 348116 0 2024-03-29T04:45:24.544 INFO:tasks.workunit.client.1.smithi152.stdout:8/664: dwrite d1/d2/d1d/f41 [0,4194304] 0 2024-03-29T04:45:24.546 INFO:tasks.workunit.client.1.smithi152.stdout:5/816: dread d6/d1d/fa9 [0,4194304] 0 2024-03-29T04:45:24.547 INFO:tasks.workunit.client.1.smithi152.stdout:5/817: readlink d6/d1d/d30/l33 0 2024-03-29T04:45:24.550 INFO:tasks.workunit.client.1.smithi152.stdout:4/612: link d0/d27/l6b d0/d7/d40/d23/lc1 0 2024-03-29T04:45:24.551 INFO:tasks.workunit.client.1.smithi152.stdout:0/688: dwrite d2/da/d17/db5/ff8 [0,4194304] 0 2024-03-29T04:45:24.553 INFO:tasks.workunit.client.1.smithi152.stdout:4/613: chown d0/d7/d25/d17/d5e/l76 52675908 1 2024-03-29T04:45:24.557 INFO:tasks.workunit.client.1.smithi152.stdout:8/665: mkdir d1/d2/d3c/d76/d9a/def 0 2024-03-29T04:45:24.557 INFO:tasks.workunit.client.1.smithi152.stdout:6/904: dwrite d1/d3/d1a/fb9 [0,4194304] 0 2024-03-29T04:45:24.560 INFO:tasks.workunit.client.1.smithi152.stdout:8/666: write d1/d2/d1d/d36/d85/fc1 [3002572,108939] 0 2024-03-29T04:45:24.562 INFO:tasks.workunit.client.1.smithi152.stdout:5/818: mknod d6/d1d/daa/db7/d56/dc1/c123 0 2024-03-29T04:45:24.570 INFO:tasks.workunit.client.1.smithi152.stdout:4/614: creat d0/d7/d40/d75/fc2 x:0 0 0 2024-03-29T04:45:24.571 INFO:tasks.workunit.client.1.smithi152.stdout:9/668: write d5/fb5 [78665,33038] 0 2024-03-29T04:45:24.572 INFO:tasks.workunit.client.1.smithi152.stdout:0/689: truncate d2/da/d17/f77 302746 0 2024-03-29T04:45:24.573 INFO:tasks.workunit.client.1.smithi152.stdout:9/669: write d5/db/d1b/f40 [2547098,23746] 0 2024-03-29T04:45:24.574 INFO:tasks.workunit.client.1.smithi152.stdout:0/690: readlink d2/da/d3a/lb6 0 2024-03-29T04:45:24.576 INFO:tasks.workunit.client.1.smithi152.stdout:8/667: mknod d1/d2/d1d/dd3/db0/d8b/cf0 0 2024-03-29T04:45:24.578 INFO:tasks.workunit.client.1.smithi152.stdout:4/615: dwrite d0/d7/f8d [4194304,4194304] 0 2024-03-29T04:45:24.582 INFO:tasks.workunit.client.1.smithi152.stdout:3/714: rename d7/d19/d3a/f7f to d7/d19/d1b/d1e/d4e/fda 0 2024-03-29T04:45:24.587 INFO:tasks.workunit.client.1.smithi152.stdout:9/670: creat d5/db/d1b/fef x:0 0 0 2024-03-29T04:45:24.594 INFO:tasks.workunit.client.1.smithi152.stdout:3/715: mknod d7/d19/d1b/d1e/da1/d41/d61/d75/cdb 0 2024-03-29T04:45:24.596 INFO:tasks.workunit.client.1.smithi152.stdout:9/671: symlink d5/d8/d31/d75/db1/dcd/d6e/db4/lf0 0 2024-03-29T04:45:24.599 INFO:tasks.workunit.client.1.smithi152.stdout:8/668: dread d1/d2/d3c/d76/f9b [0,4194304] 0 2024-03-29T04:45:24.602 INFO:tasks.workunit.client.1.smithi152.stdout:3/716: mknod d7/d19/d1b/d1e/db9/cdc 0 2024-03-29T04:45:24.604 INFO:tasks.workunit.client.1.smithi152.stdout:8/669: mknod d1/d2/d1d/dd3/cf1 0 2024-03-29T04:45:24.605 INFO:tasks.workunit.client.1.smithi152.stdout:6/905: getdents d1/d3/d1a/d1b/d87/d8f/d102 0 2024-03-29T04:45:24.606 INFO:tasks.workunit.client.1.smithi152.stdout:0/691: write d2/f8 [1742829,45596] 0 2024-03-29T04:45:24.607 INFO:tasks.workunit.client.1.smithi152.stdout:5/819: dwrite d6/d1d/daa/db7/fac [0,4194304] 0 2024-03-29T04:45:24.613 INFO:tasks.workunit.client.1.smithi152.stdout:3/717: chown d7/d19/d1b/d1e/da1/d41/d6a/cc4 2240 1 2024-03-29T04:45:24.615 INFO:tasks.workunit.client.1.smithi152.stdout:5/820: creat d6/d1d/daa/db7/f124 x:0 0 0 2024-03-29T04:45:24.616 INFO:tasks.workunit.client.1.smithi152.stdout:6/906: symlink d1/d3/d1a/d1b/d29/l134 0 2024-03-29T04:45:24.618 INFO:tasks.workunit.client.1.smithi152.stdout:8/670: rename d1/d2/d59 to d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2 0 2024-03-29T04:45:24.618 INFO:tasks.workunit.client.1.smithi152.stdout:9/672: dread d5/db/f58 [0,4194304] 0 2024-03-29T04:45:24.620 INFO:tasks.workunit.client.1.smithi152.stdout:4/616: dwrite d0/d7/d25/d17/f54 [0,4194304] 0 2024-03-29T04:45:24.622 INFO:tasks.workunit.client.1.smithi152.stdout:0/692: dwrite d2/da/d3a/d6f/f85 [0,4194304] 0 2024-03-29T04:45:24.623 INFO:tasks.workunit.client.1.smithi152.stdout:3/718: mknod d7/d19/d1b/d1e/da1/d41/d6a/dcc/cdd 0 2024-03-29T04:45:24.625 INFO:tasks.workunit.client.1.smithi152.stdout:6/907: mknod d1/d3/d9/c135 0 2024-03-29T04:45:24.625 INFO:tasks.workunit.client.1.smithi152.stdout:8/671: mknod d1/d2/d1d/d36/d85/cf3 0 2024-03-29T04:45:24.626 INFO:tasks.workunit.client.1.smithi152.stdout:5/821: dread d6/d1d/d30/d6a/fa6 [0,4194304] 0 2024-03-29T04:45:24.630 INFO:tasks.workunit.client.1.smithi152.stdout:4/617: creat d0/d7/d40/d23/d6a/fc3 x:0 0 0 2024-03-29T04:45:24.634 INFO:tasks.workunit.client.1.smithi152.stdout:9/673: truncate d5/d8/d31/d75/db1/dcd/f63 1708077 0 2024-03-29T04:45:24.635 INFO:tasks.workunit.client.1.smithi152.stdout:6/908: creat d1/d3/d91/f136 x:0 0 0 2024-03-29T04:45:24.636 INFO:tasks.workunit.client.1.smithi152.stdout:5/822: dwrite d6/d1d/d30/d6a/d108/fe6 [0,4194304] 0 2024-03-29T04:45:24.637 INFO:tasks.workunit.client.1.smithi152.stdout:5/823: write d6/f14 [2572205,85171] 0 2024-03-29T04:45:24.638 INFO:tasks.workunit.client.1.smithi152.stdout:8/672: rename d1/d2/d1d/d36/d4b/l61 to d1/d2/d1d/dd3/db0/d7f/lf4 0 2024-03-29T04:45:24.640 INFO:tasks.workunit.client.1.smithi152.stdout:5/824: write d6/d1d/d3b/f95 [5234158,62275] 0 2024-03-29T04:45:24.640 INFO:tasks.workunit.client.1.smithi152.stdout:4/618: mknod d0/d7/db2/cc4 0 2024-03-29T04:45:24.641 INFO:tasks.workunit.client.1.smithi152.stdout:8/673: fsync d1/d2/d1d/dd3/db0/d40/d6a/f89 0 2024-03-29T04:45:24.643 INFO:tasks.workunit.client.1.smithi152.stdout:8/674: chown d1/d2/d3c/d76/ca6 49 1 2024-03-29T04:45:24.648 INFO:tasks.workunit.client.1.smithi152.stdout:4/619: symlink d0/d27/d6e/d78/lc5 0 2024-03-29T04:45:24.649 INFO:tasks.workunit.client.1.smithi152.stdout:8/675: symlink d1/d2/d1d/d36/d85/lf5 0 2024-03-29T04:45:24.650 INFO:tasks.workunit.client.1.smithi152.stdout:9/674: truncate d5/d8/d31/d75/db1/dcd/f3f 4266419 0 2024-03-29T04:45:24.651 INFO:tasks.workunit.client.1.smithi152.stdout:4/620: rmdir d0/d7/d40/d23/d55 39 2024-03-29T04:45:24.652 INFO:tasks.workunit.client.1.smithi152.stdout:8/676: rmdir d1/d2/d1d/dd3/dc2 39 2024-03-29T04:45:24.655 INFO:tasks.workunit.client.1.smithi152.stdout:9/675: rename d5/d8/d31/d59/l67 to d5/db/d1b/d68/lf1 0 2024-03-29T04:45:24.655 INFO:tasks.workunit.client.1.smithi152.stdout:5/825: dwrite d6/d1d/d30/d6a/dfb/f10f [0,4194304] 0 2024-03-29T04:45:24.657 INFO:tasks.workunit.client.1.smithi152.stdout:5/826: chown d6/d1d/d30/f38 2003 1 2024-03-29T04:45:24.661 INFO:tasks.workunit.client.1.smithi152.stdout:8/677: rename d1/ce6 to d1/cf6 0 2024-03-29T04:45:24.662 INFO:tasks.workunit.client.1.smithi152.stdout:5/827: rename d6/d16/db3/c11e to d6/d1d/d30/d6a/d108/c125 0 2024-03-29T04:45:24.663 INFO:tasks.workunit.client.1.smithi152.stdout:8/678: write d1/d2/d1d/dd3/db0/da9/fbc [23595,61116] 0 2024-03-29T04:45:24.666 INFO:tasks.workunit.client.1.smithi152.stdout:8/679: chown d1/d2/d1d/d36/d85/f99 3325 1 2024-03-29T04:45:24.666 INFO:tasks.workunit.client.1.smithi152.stdout:6/909: dread d1/d3/d9/d13/f33 [4194304,4194304] 0 2024-03-29T04:45:24.667 INFO:tasks.workunit.client.1.smithi152.stdout:5/828: creat d6/d1d/d30/d6a/dfb/f126 x:0 0 0 2024-03-29T04:45:24.669 INFO:tasks.workunit.client.1.smithi152.stdout:3/719: dwrite d7/d19/d1b/d1e/d4e/f76 [0,4194304] 0 2024-03-29T04:45:24.670 INFO:tasks.workunit.client.1.smithi152.stdout:3/720: chown d7/d19/d1b 0 1 2024-03-29T04:45:24.670 INFO:tasks.workunit.client.1.smithi152.stdout:5/829: readlink d6/d1d/daa/db7/d56/dc1/de4/lf1 0 2024-03-29T04:45:24.671 INFO:tasks.workunit.client.1.smithi152.stdout:8/680: creat d1/d2/d1d/dc6/ff7 x:0 0 0 2024-03-29T04:45:24.673 INFO:tasks.workunit.client.1.smithi152.stdout:8/681: chown d1/d2/d1d/d36/d85/lf5 3 1 2024-03-29T04:45:24.673 INFO:tasks.workunit.client.1.smithi152.stdout:8/682: chown d1/d2/d1d/f4f 0 1 2024-03-29T04:45:24.674 INFO:tasks.workunit.client.1.smithi152.stdout:5/830: symlink d6/d1b/l127 0 2024-03-29T04:45:24.675 INFO:tasks.workunit.client.1.smithi152.stdout:6/910: unlink d1/d3/d9/l47 0 2024-03-29T04:45:24.676 INFO:tasks.workunit.client.1.smithi152.stdout:8/683: truncate d1/d2/d1d/dd3/db0/da9/fbc 258491 0 2024-03-29T04:45:24.679 INFO:tasks.workunit.client.1.smithi152.stdout:5/831: rename d6/d1b/f45 to d6/d1d/daa/db7/f128 0 2024-03-29T04:45:24.683 INFO:tasks.workunit.client.1.smithi152.stdout:6/911: unlink d1/d3/d9/d13/d28/f114 0 2024-03-29T04:45:24.683 INFO:tasks.workunit.client.1.smithi152.stdout:9/676: dwrite d5/db/feb [0,4194304] 0 2024-03-29T04:45:24.693 INFO:tasks.workunit.client.1.smithi152.stdout:9/677: mkdir d5/d8/d31/d75/db1/dcd/d6e/db4/df2 0 2024-03-29T04:45:24.698 INFO:tasks.workunit.client.1.smithi152.stdout:3/721: dwrite d7/d19/d1b/d1e/da1/d41/d61/d85/f8d [0,4194304] 0 2024-03-29T04:45:24.700 INFO:tasks.workunit.client.1.smithi152.stdout:9/678: dread d5/d8/f1a [0,4194304] 0 2024-03-29T04:45:24.704 INFO:tasks.workunit.client.1.smithi152.stdout:9/679: truncate d5/d8/f1a 601510 0 2024-03-29T04:45:24.709 INFO:tasks.workunit.client.1.smithi152.stdout:9/680: link d5/db/d1b/f46 d5/d12/ff3 0 2024-03-29T04:45:24.710 INFO:tasks.workunit.client.1.smithi152.stdout:9/681: write d5/f2f [1057607,102436] 0 2024-03-29T04:45:24.710 INFO:tasks.workunit.client.1.smithi152.stdout:9/682: readlink d5/db/d1b/l27 0 2024-03-29T04:45:24.726 INFO:tasks.workunit.client.1.smithi152.stdout:8/684: dwrite d1/fe2 [0,4194304] 0 2024-03-29T04:45:24.729 INFO:tasks.workunit.client.1.smithi152.stdout:9/683: dread d5/d8/f53 [0,4194304] 0 2024-03-29T04:45:24.729 INFO:tasks.workunit.client.1.smithi152.stdout:5/832: truncate d6/d1d/d30/f38 2001789 0 2024-03-29T04:45:24.730 INFO:tasks.workunit.client.1.smithi152.stdout:8/685: creat d1/d2/d1d/d36/d4b/d51/ff8 x:0 0 0 2024-03-29T04:45:24.735 INFO:tasks.workunit.client.1.smithi152.stdout:5/833: symlink d6/d1d/daa/db7/dda/l129 0 2024-03-29T04:45:24.737 INFO:tasks.workunit.client.1.smithi152.stdout:8/686: rmdir d1/d2/d1d/dd3/db0/d40/d6a/dd5 39 2024-03-29T04:45:24.739 INFO:tasks.workunit.client.1.smithi152.stdout:9/684: mkdir d5/d8/d31/d59/d6f/dbf/df4 0 2024-03-29T04:45:24.740 INFO:tasks.workunit.client.1.smithi152.stdout:8/687: chown d1/d2/d1d/dd3/db0/d40/ddd/ce7 95 1 2024-03-29T04:45:24.742 INFO:tasks.workunit.client.1.smithi152.stdout:3/722: dwrite d7/d19/d1b/d1e/d4e/fad [0,4194304] 0 2024-03-29T04:45:24.743 INFO:tasks.workunit.client.1.smithi152.stdout:5/834: creat d6/d16/d25/d2e/da0/d92/f12a x:0 0 0 2024-03-29T04:45:24.745 INFO:tasks.workunit.client.1.smithi152.stdout:9/685: mknod d5/d8/d31/d59/d85/cf5 0 2024-03-29T04:45:24.751 INFO:tasks.workunit.client.1.smithi152.stdout:8/688: creat d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/ff9 x:0 0 0 2024-03-29T04:45:24.751 INFO:tasks.workunit.client.1.smithi152.stdout:3/723: dwrite d7/d19/d1b/d1e/f82 [0,4194304] 0 2024-03-29T04:45:24.762 INFO:tasks.workunit.client.1.smithi152.stdout:3/724: dread d7/d19/d1b/d1e/d22/f3b [0,4194304] 0 2024-03-29T04:45:24.764 INFO:tasks.workunit.client.1.smithi152.stdout:3/725: dread - d7/d19/d1b/d1e/da1/d41/d61/fcf zero size 2024-03-29T04:45:24.767 INFO:tasks.workunit.client.1.smithi152.stdout:3/726: link d7/d19/d1b/d1e/da1/d41/d73/fbb d7/d19/d1b/d1e/da1/d41/d6a/d78/fde 0 2024-03-29T04:45:24.771 INFO:tasks.workunit.client.1.smithi152.stdout:9/686: write d5/d12/ff3 [2714280,114440] 0 2024-03-29T04:45:24.772 INFO:tasks.workunit.client.1.smithi152.stdout:3/727: mknod d7/d19/db5/cdf 0 2024-03-29T04:45:24.775 INFO:tasks.workunit.client.1.smithi152.stdout:5/835: dread d6/d16/f4e [0,4194304] 0 2024-03-29T04:45:24.776 INFO:tasks.workunit.client.1.smithi152.stdout:3/728: creat d7/d19/d1b/d1e/da1/fe0 x:0 0 0 2024-03-29T04:45:24.777 INFO:tasks.workunit.client.1.smithi152.stdout:8/689: dread d1/d2/d3/f44 [4194304,4194304] 0 2024-03-29T04:45:24.777 INFO:tasks.workunit.client.1.smithi152.stdout:5/836: fdatasync d6/d16/d25/d2e/da0/d92/fc3 0 2024-03-29T04:45:24.780 INFO:tasks.workunit.client.1.smithi152.stdout:3/729: dread - d7/d19/d1b/d1e/da1/d41/d6a/d78/f95 zero size 2024-03-29T04:45:24.780 INFO:tasks.workunit.client.1.smithi152.stdout:8/690: stat d1/d2/d1d/dd3/db0/d40/d47/dca 0 2024-03-29T04:45:24.785 INFO:tasks.workunit.client.1.smithi152.stdout:3/730: symlink d7/d46/le1 0 2024-03-29T04:45:24.789 INFO:tasks.workunit.client.1.smithi152.stdout:3/731: symlink d7/d19/d1b/d1e/db9/le2 0 2024-03-29T04:45:24.789 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:24 smithi152 ceph-mon[37177]: pgmap v278: 65 pgs: 65 active+clean; 3.2 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 91 MiB/s rd, 241 MiB/s wr, 549 op/s 2024-03-29T04:45:24.791 INFO:tasks.workunit.client.1.smithi152.stdout:8/691: creat d1/ffa x:0 0 0 2024-03-29T04:45:24.794 INFO:tasks.workunit.client.1.smithi152.stdout:3/732: symlink d7/d19/d1b/d1e/da1/d41/d6a/dcc/le3 0 2024-03-29T04:45:24.800 INFO:tasks.workunit.client.1.smithi152.stdout:9/687: truncate d5/db/d1b/f4f 3064980 0 2024-03-29T04:45:24.807 INFO:tasks.workunit.client.1.smithi152.stdout:9/688: rmdir d5/d8/d31/d59/dc3 39 2024-03-29T04:45:24.815 INFO:tasks.workunit.client.1.smithi152.stdout:8/692: getdents d1/d2/d1d/d36/d4b/d51/ddc 0 2024-03-29T04:45:24.817 INFO:tasks.workunit.client.1.smithi152.stdout:5/837: dwrite d6/d1d/d30/f38 [0,4194304] 0 2024-03-29T04:45:24.817 INFO:tasks.workunit.client.1.smithi152.stdout:3/733: getdents d7/d19/d1b/d1e/da1/d41/d6a/dcc 0 2024-03-29T04:45:24.821 INFO:tasks.workunit.client.1.smithi152.stdout:5/838: write d6/d1d/daa/db7/d56/f94 [1660980,36393] 0 2024-03-29T04:45:24.824 INFO:tasks.workunit.client.1.smithi152.stdout:9/689: link d5/d12/l98 d5/d8/d31/d59/lf6 0 2024-03-29T04:45:24.825 INFO:tasks.workunit.client.1.smithi152.stdout:5/839: write d6/d1d/d30/d6a/d108/f35 [2845243,44358] 0 2024-03-29T04:45:24.827 INFO:tasks.workunit.client.1.smithi152.stdout:8/693: dwrite d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/fb9 [0,4194304] 0 2024-03-29T04:45:24.839 INFO:tasks.workunit.client.1.smithi152.stdout:5/840: mknod d6/d1d/d30/c12b 0 2024-03-29T04:45:24.841 INFO:tasks.workunit.client.1.smithi152.stdout:8/694: creat d1/d2/d3/ffb x:0 0 0 2024-03-29T04:45:24.843 INFO:tasks.workunit.client.1.smithi152.stdout:9/690: symlink d5/d8/d31/d75/db1/dcd/d6e/dea/lf7 0 2024-03-29T04:45:24.847 INFO:tasks.workunit.client.1.smithi152.stdout:5/841: dwrite d6/d1d/f79 [0,4194304] 0 2024-03-29T04:45:24.848 INFO:tasks.workunit.client.1.smithi152.stdout:7/648: sync 2024-03-29T04:45:24.850 INFO:tasks.workunit.client.1.smithi152.stdout:5/842: chown d6/d1d/d30/dc5/ff9 2 1 2024-03-29T04:45:24.853 INFO:tasks.workunit.client.1.smithi152.stdout:3/734: dwrite d7/d19/d1b/d1e/da1/d41/f50 [0,4194304] 0 2024-03-29T04:45:24.854 INFO:tasks.workunit.client.1.smithi152.stdout:5/843: write d6/f14 [4647101,107681] 0 2024-03-29T04:45:24.860 INFO:tasks.workunit.client.1.smithi152.stdout:8/695: getdents d1/d2/d1d/d36/d4b 0 2024-03-29T04:45:24.864 INFO:tasks.workunit.client.1.smithi152.stdout:9/691: dwrite d5/d8/d31/d75/db1/dcd/d6e/fc7 [0,4194304] 0 2024-03-29T04:45:24.865 INFO:tasks.workunit.client.1.smithi152.stdout:7/649: dread d7/d8/d37/f7d [0,4194304] 0 2024-03-29T04:45:24.867 INFO:tasks.workunit.client.1.smithi152.stdout:5/844: creat d6/d1d/daa/db7/d56/dc1/de4/f12c x:0 0 0 2024-03-29T04:45:24.868 INFO:tasks.workunit.client.1.smithi152.stdout:8/696: mkdir d1/d2/d3c/d76/d9a/dfc 0 2024-03-29T04:45:24.868 INFO:tasks.workunit.client.1.smithi152.stdout:9/692: chown d5/db/f21 316186793 1 2024-03-29T04:45:24.870 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:24 smithi142 ceph-mon[29027]: pgmap v278: 65 pgs: 65 active+clean; 3.2 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 91 MiB/s rd, 241 MiB/s wr, 549 op/s 2024-03-29T04:45:24.871 INFO:tasks.workunit.client.1.smithi152.stdout:3/735: rename d7/d19/d3a to d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4 0 2024-03-29T04:45:24.872 INFO:tasks.workunit.client.1.smithi152.stdout:7/650: mkdir d7/d8/d12/d35/d68/dc3 0 2024-03-29T04:45:24.873 INFO:tasks.workunit.client.1.smithi152.stdout:7/651: fdatasync d7/d8/db/f42 0 2024-03-29T04:45:24.874 INFO:tasks.workunit.client.1.smithi152.stdout:7/652: write d7/d8/d12/d1d/d30/f9b [880928,109827] 0 2024-03-29T04:45:24.881 INFO:tasks.workunit.client.1.smithi152.stdout:8/697: dwrite d1/d2/d1d/dd3/db0/f90 [0,4194304] 0 2024-03-29T04:45:24.881 INFO:tasks.workunit.client.1.smithi152.stdout:3/736: symlink d7/d19/d1b/d1e/da1/d41/d61/le5 0 2024-03-29T04:45:24.884 INFO:tasks.workunit.client.1.smithi152.stdout:3/737: dread - d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/fc6 zero size 2024-03-29T04:45:24.886 INFO:tasks.workunit.client.1.smithi152.stdout:8/698: symlink d1/d2/d1d/d36/d85/lfd 0 2024-03-29T04:45:24.887 INFO:tasks.workunit.client.1.smithi152.stdout:8/699: readlink d1/d2/d1d/d36/d85/lbd 0 2024-03-29T04:45:24.888 INFO:tasks.workunit.client.1.smithi152.stdout:8/700: dread - d1/d2/d3c/fcb zero size 2024-03-29T04:45:24.890 INFO:tasks.workunit.client.1.smithi152.stdout:8/701: write d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/f88 [2149909,98170] 0 2024-03-29T04:45:24.895 INFO:tasks.workunit.client.1.smithi152.stdout:3/738: link d7/d19/d1b/f23 d7/d6c/fe6 0 2024-03-29T04:45:24.896 INFO:tasks.workunit.client.1.smithi152.stdout:3/739: read d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/fb2 [48226,46991] 0 2024-03-29T04:45:24.899 INFO:tasks.workunit.client.1.smithi152.stdout:8/702: mknod d1/d2/d1d/d36/d4b/d51/ddc/cfe 0 2024-03-29T04:45:24.899 INFO:tasks.workunit.client.1.smithi152.stdout:7/653: write d7/d8/d12/d1d/f86 [1823253,107070] 0 2024-03-29T04:45:24.903 INFO:tasks.workunit.client.1.smithi152.stdout:3/740: mknod d7/d19/d1b/d1e/da1/d41/d61/d85/ce7 0 2024-03-29T04:45:24.904 INFO:tasks.workunit.client.1.smithi152.stdout:9/693: dwrite d5/d8/d31/d75/db1/dcd/f63 [0,4194304] 0 2024-03-29T04:45:24.904 INFO:tasks.workunit.client.1.smithi152.stdout:8/703: symlink d1/d2/d1d/d36/d5d/lff 0 2024-03-29T04:45:24.906 INFO:tasks.workunit.client.1.smithi152.stdout:8/704: dread - d1/d2/d1d/dd3/db0/d40/d47/fc8 zero size 2024-03-29T04:45:24.908 INFO:tasks.workunit.client.1.smithi152.stdout:8/705: truncate d1/d8a/d91/dee/f9e 1522884 0 2024-03-29T04:45:24.908 INFO:tasks.workunit.client.1.smithi152.stdout:8/706: stat d1/d2/d1d/dc6 0 2024-03-29T04:45:24.908 INFO:tasks.workunit.client.1.smithi152.stdout:3/741: dread d7/d19/d1b/d1e/d4e/fad [0,4194304] 0 2024-03-29T04:45:24.915 INFO:tasks.workunit.client.1.smithi152.stdout:8/707: dwrite d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/fa8 [0,4194304] 0 2024-03-29T04:45:24.919 INFO:tasks.workunit.client.1.smithi152.stdout:3/742: rmdir d7/d19/d1b/d1e/d22 39 2024-03-29T04:45:24.923 INFO:tasks.workunit.client.1.smithi152.stdout:9/694: truncate d5/d8/d31/d75/db1/dcd/f3f 1408566 0 2024-03-29T04:45:24.924 INFO:tasks.workunit.client.1.smithi152.stdout:7/654: link d7/d8/d93/f80 d7/d8/d12/d1d/fc4 0 2024-03-29T04:45:24.925 INFO:tasks.workunit.client.1.smithi152.stdout:9/695: truncate d5/d8/d31/d75/db1/dcd/d6e/d91/fee 377319 0 2024-03-29T04:45:24.926 INFO:tasks.workunit.client.1.smithi152.stdout:8/708: creat d1/d2/d1d/d36/d85/f100 x:0 0 0 2024-03-29T04:45:24.927 INFO:tasks.workunit.client.1.smithi152.stdout:3/743: symlink d7/d19/d1b/d1e/da1/d41/d6a/d78/le8 0 2024-03-29T04:45:24.928 INFO:tasks.workunit.client.1.smithi152.stdout:7/655: mknod d7/cc5 0 2024-03-29T04:45:24.929 INFO:tasks.workunit.client.1.smithi152.stdout:8/709: fsync d1/d2/d1d/f30 0 2024-03-29T04:45:24.931 INFO:tasks.workunit.client.1.smithi152.stdout:3/744: dread d7/d19/d1b/d59/f68 [0,4194304] 0 2024-03-29T04:45:24.932 INFO:tasks.workunit.client.1.smithi152.stdout:7/656: mknod d7/d8/d37/cc6 0 2024-03-29T04:45:24.932 INFO:tasks.workunit.client.1.smithi152.stdout:3/745: chown d7/d19/d1b/d1e/da1/d41/d61/lc9 1151834 1 2024-03-29T04:45:24.933 INFO:tasks.workunit.client.1.smithi152.stdout:3/746: truncate d7/d19/d1b/d1e/da1/d41/d61/fcf 995267 0 2024-03-29T04:45:24.933 INFO:tasks.workunit.client.1.smithi152.stdout:9/696: unlink d5/d8/d31/d75/db1/dcd/c47 0 2024-03-29T04:45:24.935 INFO:tasks.workunit.client.1.smithi152.stdout:8/710: mknod d1/dc5/c101 0 2024-03-29T04:45:24.938 INFO:tasks.workunit.client.1.smithi152.stdout:8/711: fdatasync d1/d2/d1d/dd3/db0/d40/d6a/f62 0 2024-03-29T04:45:24.939 INFO:tasks.workunit.client.1.smithi152.stdout:3/747: dread d7/d19/d1b/f21 [4194304,4194304] 0 2024-03-29T04:45:24.942 INFO:tasks.workunit.client.1.smithi152.stdout:9/697: dread d5/db/fec [0,4194304] 0 2024-03-29T04:45:24.943 INFO:tasks.workunit.client.1.smithi152.stdout:9/698: chown d5/d12/l1d 72 1 2024-03-29T04:45:24.950 INFO:tasks.workunit.client.1.smithi152.stdout:8/712: rmdir d1/dc5 39 2024-03-29T04:45:24.953 INFO:tasks.workunit.client.1.smithi152.stdout:9/699: mknod d5/d8/d31/d75/db1/dcd/d6e/d91/de1/cf8 0 2024-03-29T04:45:24.956 INFO:tasks.workunit.client.1.smithi152.stdout:3/748: truncate d7/d19/d1b/d1e/fb8 3117494 0 2024-03-29T04:45:24.957 INFO:tasks.workunit.client.1.smithi152.stdout:9/700: creat d5/d8/ff9 x:0 0 0 2024-03-29T04:45:24.957 INFO:tasks.workunit.client.1.smithi152.stdout:3/749: chown d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/l8e 34888 1 2024-03-29T04:45:24.962 INFO:tasks.workunit.client.1.smithi152.stdout:3/750: symlink d7/d19/d1b/d1e/d4e/le9 0 2024-03-29T04:45:24.964 INFO:tasks.workunit.client.1.smithi152.stdout:8/713: dread d1/d2/d1d/dd3/db0/d40/d47/fd0 [0,4194304] 0 2024-03-29T04:45:24.964 INFO:tasks.workunit.client.1.smithi152.stdout:8/714: chown d1/d2/l38 207 1 2024-03-29T04:45:24.965 INFO:tasks.workunit.client.1.smithi152.stdout:3/751: mknod d7/d19/d1b/d1e/da1/cea 0 2024-03-29T04:45:24.968 INFO:tasks.workunit.client.1.smithi152.stdout:8/715: creat d1/d2/d1d/d36/d4b/f102 x:0 0 0 2024-03-29T04:45:24.971 INFO:tasks.workunit.client.1.smithi152.stdout:8/716: creat d1/d2/d6d/f103 x:0 0 0 2024-03-29T04:45:24.971 INFO:tasks.workunit.client.1.smithi152.stdout:3/752: dwrite d7/d19/d1b/d1e/da1/d41/d61/d75/fbd [0,4194304] 0 2024-03-29T04:45:24.974 INFO:tasks.workunit.client.1.smithi152.stdout:3/753: write d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/fc6 [991390,123673] 0 2024-03-29T04:45:24.978 INFO:tasks.workunit.client.1.smithi152.stdout:8/717: dwrite d1/d2/d1d/dd3/db0/f2a [0,4194304] 0 2024-03-29T04:45:24.982 INFO:tasks.workunit.client.1.smithi152.stdout:8/718: creat d1/d8a/d91/f104 x:0 0 0 2024-03-29T04:45:24.982 INFO:tasks.workunit.client.1.smithi152.stdout:3/754: getdents d7/d19/d1b/d1e/da1/d41/d61/d75/d8c 0 2024-03-29T04:45:24.984 INFO:tasks.workunit.client.1.smithi152.stdout:8/719: mkdir d1/d2/d1d/dd3/db0/da9/d105 0 2024-03-29T04:45:24.985 INFO:tasks.workunit.client.1.smithi152.stdout:8/720: chown d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/c86 46515889 1 2024-03-29T04:45:24.986 INFO:tasks.workunit.client.1.smithi152.stdout:8/721: readlink d1/d2/l2e 0 2024-03-29T04:45:24.989 INFO:tasks.workunit.client.1.smithi152.stdout:3/755: link d7/cac d7/d6c/ceb 0 2024-03-29T04:45:24.993 INFO:tasks.workunit.client.1.smithi152.stdout:3/756: mkdir d7/d19/d1b/d59/dec 0 2024-03-29T04:45:24.994 INFO:tasks.workunit.client.1.smithi152.stdout:3/757: write d7/d19/d1b/d1e/da1/d41/d61/d63/fd0 [4928,109274] 0 2024-03-29T04:45:24.996 INFO:tasks.workunit.client.1.smithi152.stdout:8/722: getdents d1/d2 0 2024-03-29T04:45:24.996 INFO:tasks.workunit.client.1.smithi152.stdout:3/758: dread - d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/f7a zero size 2024-03-29T04:45:24.997 INFO:tasks.workunit.client.1.smithi152.stdout:8/723: chown d1/d2/d1d/dd3/db0/d40/d6a 644 1 2024-03-29T04:45:25.000 INFO:tasks.workunit.client.1.smithi152.stdout:3/759: mknod d7/d19/d1b/d1e/da1/d41/d61/ced 0 2024-03-29T04:45:25.001 INFO:tasks.workunit.client.1.smithi152.stdout:3/760: readlink d7/d19/d1b/d1e/da1/d41/d6a/dcc/le3 0 2024-03-29T04:45:25.003 INFO:tasks.workunit.client.1.smithi152.stdout:3/761: write d7/d19/d1b/d1e/da1/d41/d6a/d78/f95 [432898,51984] 0 2024-03-29T04:45:25.004 INFO:tasks.workunit.client.1.smithi152.stdout:3/762: read d7/d19/d1b/d59/f68 [31410,107257] 0 2024-03-29T04:45:25.008 INFO:tasks.workunit.client.1.smithi152.stdout:3/763: rename d7/f15 to d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/fee 0 2024-03-29T04:45:25.010 INFO:tasks.workunit.client.1.smithi152.stdout:3/764: creat d7/d19/d1b/d1e/da1/d41/d6a/fef x:0 0 0 2024-03-29T04:45:25.012 INFO:tasks.workunit.client.1.smithi152.stdout:8/724: truncate d1/d8a/d91/dee/f92 3803321 0 2024-03-29T04:45:25.013 INFO:tasks.workunit.client.1.smithi152.stdout:3/765: mkdir d7/d19/d1b/d1e/df0 0 2024-03-29T04:45:25.014 INFO:tasks.workunit.client.1.smithi152.stdout:8/725: creat d1/d2/d6d/f106 x:0 0 0 2024-03-29T04:45:25.016 INFO:tasks.workunit.client.1.smithi152.stdout:3/766: creat d7/d19/d1b/d1e/da1/d41/d61/d63/ff1 x:0 0 0 2024-03-29T04:45:25.018 INFO:tasks.workunit.client.1.smithi152.stdout:3/767: creat d7/d19/db5/ff2 x:0 0 0 2024-03-29T04:45:25.020 INFO:tasks.workunit.client.1.smithi152.stdout:3/768: creat d7/d19/d1b/d59/ff3 x:0 0 0 2024-03-29T04:45:25.025 INFO:tasks.workunit.client.1.smithi152.stdout:8/726: write d1/d8a/fd4 [1320861,97429] 0 2024-03-29T04:45:25.027 INFO:tasks.workunit.client.1.smithi152.stdout:3/769: dwrite d7/d19/d1b/d1e/da1/fcd [0,4194304] 0 2024-03-29T04:45:25.030 INFO:tasks.workunit.client.1.smithi152.stdout:8/727: dread d1/d2/d1d/d36/d5d/fd9 [0,4194304] 0 2024-03-29T04:45:25.031 INFO:tasks.workunit.client.1.smithi152.stdout:3/770: creat d7/d19/d1b/d1e/df0/ff4 x:0 0 0 2024-03-29T04:45:25.033 INFO:tasks.workunit.client.1.smithi152.stdout:8/728: mkdir d1/d2/d1d/d36/d107 0 2024-03-29T04:45:25.033 INFO:tasks.workunit.client.1.smithi152.stdout:8/729: fsync d1/d2/d1d/dd3/db0/f2a 0 2024-03-29T04:45:25.039 INFO:tasks.workunit.client.1.smithi152.stdout:8/730: rename d1/d2/d1d/dc6/ce3 to d1/d2/d3c/d76/d9a/dfc/c108 0 2024-03-29T04:45:25.040 INFO:tasks.workunit.client.1.smithi152.stdout:8/731: chown d1/d2/ced 1802 1 2024-03-29T04:45:25.041 INFO:tasks.workunit.client.1.smithi152.stdout:3/771: dwrite d7/d19/d1b/d1e/d4e/d8f/fc0 [0,4194304] 0 2024-03-29T04:45:25.042 INFO:tasks.workunit.client.1.smithi152.stdout:8/732: symlink d1/d2/d1d/dd3/db0/l109 0 2024-03-29T04:45:25.046 INFO:tasks.workunit.client.1.smithi152.stdout:8/733: creat d1/d2/d1d/d36/d4b/d51/ddc/f10a x:0 0 0 2024-03-29T04:45:25.050 INFO:tasks.workunit.client.1.smithi152.stdout:8/734: mkdir d1/d2/d1d/dd3/db0/d10b 0 2024-03-29T04:45:25.050 INFO:tasks.workunit.client.1.smithi152.stdout:3/772: dwrite d7/d19/d1b/d1e/da1/fcd [0,4194304] 0 2024-03-29T04:45:25.050 INFO:tasks.workunit.client.1.smithi152.stdout:8/735: stat d1/d2/d1d/f74 0 2024-03-29T04:45:25.052 INFO:tasks.workunit.client.1.smithi152.stdout:8/736: read d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/fa8 [3992691,41464] 0 2024-03-29T04:45:25.053 INFO:tasks.workunit.client.1.smithi152.stdout:8/737: write d1/d2/d1d/dd3/db0/f25 [4369670,115282] 0 2024-03-29T04:45:25.057 INFO:tasks.workunit.client.1.smithi152.stdout:3/773: symlink d7/d19/lf5 0 2024-03-29T04:45:25.062 INFO:tasks.workunit.client.1.smithi152.stdout:8/738: unlink d1/d2/d1d/f74 0 2024-03-29T04:45:25.063 INFO:tasks.workunit.client.1.smithi152.stdout:8/739: fsync d1/d8a/d91/dee/fd7 0 2024-03-29T04:45:25.064 INFO:tasks.workunit.client.1.smithi152.stdout:8/740: chown d1/d2/d3/f44 23759402 1 2024-03-29T04:45:25.067 INFO:tasks.workunit.client.1.smithi152.stdout:8/741: creat d1/d2/d3/f10c x:0 0 0 2024-03-29T04:45:25.068 INFO:tasks.workunit.client.1.smithi152.stdout:8/742: dread - d1/d2/d1d/d36/d4b/d51/ddc/f10a zero size 2024-03-29T04:45:25.072 INFO:tasks.workunit.client.1.smithi152.stdout:8/743: mknod d1/d2/d1d/dd3/db0/da9/d105/c10d 0 2024-03-29T04:45:25.077 INFO:tasks.workunit.client.1.smithi152.stdout:8/744: dwrite d1/d2/d1d/f3d [0,4194304] 0 2024-03-29T04:45:25.080 INFO:tasks.workunit.client.1.smithi152.stdout:8/745: write d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/f88 [3361518,70048] 0 2024-03-29T04:45:25.085 INFO:tasks.workunit.client.1.smithi152.stdout:8/746: symlink d1/d8a/d91/l10e 0 2024-03-29T04:45:25.096 INFO:tasks.workunit.client.1.smithi152.stdout:8/747: read d1/d2/d1d/d36/d4b/d51/f5e [189630,92522] 0 2024-03-29T04:45:25.102 INFO:tasks.workunit.client.1.smithi152.stdout:8/748: mknod d1/d2/d1d/dd3/dc2/c10f 0 2024-03-29T04:45:25.103 INFO:tasks.workunit.client.1.smithi152.stdout:8/749: mknod d1/d2/d1d/d36/d4b/c110 0 2024-03-29T04:45:25.105 INFO:tasks.workunit.client.1.smithi152.stdout:8/750: mkdir d1/d2/d1d/d36/d111 0 2024-03-29T04:45:25.106 INFO:tasks.workunit.client.1.smithi152.stdout:8/751: symlink d1/d2/d1d/dd3/db0/d10b/l112 0 2024-03-29T04:45:25.109 INFO:tasks.workunit.client.1.smithi152.stdout:8/752: rename d1/d2/d3c/l43 to d1/d2/d3c/d76/d9a/dfc/l113 0 2024-03-29T04:45:25.110 INFO:tasks.workunit.client.1.smithi152.stdout:8/753: read - d1/d8a/d91/dee/fea zero size 2024-03-29T04:45:25.111 INFO:tasks.workunit.client.1.smithi152.stdout:8/754: mknod d1/d2/d1d/dd3/db0/d40/ddd/c114 0 2024-03-29T04:45:25.123 INFO:tasks.workunit.client.1.smithi152.stdout:8/755: dread d1/d2/d1d/dd3/db0/d40/d47/f72 [0,4194304] 0 2024-03-29T04:45:25.125 INFO:tasks.workunit.client.1.smithi152.stdout:8/756: mknod d1/d2/d1d/d36/d5d/c115 0 2024-03-29T04:45:25.127 INFO:tasks.workunit.client.1.smithi152.stdout:8/757: symlink d1/d2/d1d/dd3/db0/d40/ddd/l116 0 2024-03-29T04:45:25.132 INFO:tasks.workunit.client.1.smithi152.stdout:8/758: dwrite d1/d2/d3c/f7a [4194304,4194304] 0 2024-03-29T04:45:25.137 INFO:tasks.workunit.client.1.smithi152.stdout:8/759: dread d1/d2/d3c/f69 [0,4194304] 0 2024-03-29T04:45:25.140 INFO:tasks.workunit.client.1.smithi152.stdout:8/760: creat d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/f117 x:0 0 0 2024-03-29T04:45:25.140 INFO:tasks.workunit.client.1.smithi152.stdout:8/761: truncate d1/d2/d3/f10c 682950 0 2024-03-29T04:45:25.141 INFO:tasks.workunit.client.1.smithi152.stdout:8/762: write d1/d8a/d91/dee/fea [104585,25084] 0 2024-03-29T04:45:25.162 INFO:tasks.workunit.client.1.smithi152.stdout:8/763: truncate d1/d2/d3c/f7a 396027 0 2024-03-29T04:45:25.167 INFO:tasks.workunit.client.1.smithi152.stdout:8/764: dwrite d1/d2/d3c/d76/d9a/fad [0,4194304] 0 2024-03-29T04:45:25.168 INFO:tasks.workunit.client.1.smithi152.stdout:8/765: chown d1/d2/d1d/dd3/db0/d40/d58 15185891 1 2024-03-29T04:45:25.172 INFO:tasks.workunit.client.1.smithi152.stdout:8/766: symlink d1/d2/d1d/dd3/l118 0 2024-03-29T04:45:25.177 INFO:tasks.workunit.client.1.smithi152.stdout:8/767: mkdir d1/d2/d1d/dd3/db0/d119 0 2024-03-29T04:45:25.182 INFO:tasks.workunit.client.1.smithi152.stdout:8/768: creat d1/d2/d1d/d36/d111/f11a x:0 0 0 2024-03-29T04:45:25.188 INFO:tasks.workunit.client.1.smithi152.stdout:0/693: sync 2024-03-29T04:45:25.188 INFO:tasks.workunit.client.1.smithi152.stdout:2/666: sync 2024-03-29T04:45:25.189 INFO:tasks.workunit.client.1.smithi152.stdout:8/769: getdents d1/d2/d1d/d36/d4b/d51/ddc 0 2024-03-29T04:45:25.190 INFO:tasks.workunit.client.1.smithi152.stdout:2/667: creat d0/d3/df/d8c/d97/fc7 x:0 0 0 2024-03-29T04:45:25.193 INFO:tasks.workunit.client.1.smithi152.stdout:0/694: dread d2/da/d17/dd8/db8/daf/fd7 [0,4194304] 0 2024-03-29T04:45:25.193 INFO:tasks.workunit.client.1.smithi152.stdout:2/668: mknod d0/d3/d30/d15/d22/cc8 0 2024-03-29T04:45:25.193 INFO:tasks.workunit.client.1.smithi152.stdout:8/770: dread d1/d2/d1d/dc6/fc9 [0,4194304] 0 2024-03-29T04:45:25.198 INFO:tasks.workunit.client.1.smithi152.stdout:2/669: fdatasync d0/d3/f1d 0 2024-03-29T04:45:25.200 INFO:tasks.workunit.client.1.smithi152.stdout:2/670: mknod d0/d3/d94/cc9 0 2024-03-29T04:45:25.200 INFO:tasks.workunit.client.1.smithi152.stdout:0/695: link d2/da/d17/db5/cdc d2/da/d17/dd8/db8/daf/cfd 0 2024-03-29T04:45:25.201 INFO:tasks.workunit.client.1.smithi152.stdout:0/696: truncate d2/da/d17/dd8/db8/f7b 880155 0 2024-03-29T04:45:25.204 INFO:tasks.workunit.client.1.smithi152.stdout:0/697: mknod d2/da/d17/d57/def/cfe 0 2024-03-29T04:45:25.204 INFO:tasks.workunit.client.1.smithi152.stdout:0/698: readlink d2/da/d17/d50/l58 0 2024-03-29T04:45:25.205 INFO:tasks.workunit.client.1.smithi152.stdout:2/671: link d0/d3/d30/d15/da5/fbc d0/d3/d30/d15/da5/fca 0 2024-03-29T04:45:25.205 INFO:tasks.workunit.client.1.smithi152.stdout:2/672: stat d0/d3/d64 0 2024-03-29T04:45:25.206 INFO:tasks.workunit.client.1.smithi152.stdout:2/673: fdatasync d0/d3/d30/f87 0 2024-03-29T04:45:25.210 INFO:tasks.workunit.client.1.smithi152.stdout:0/699: dwrite d2/da/d3a/d61/db4/ffb [0,4194304] 0 2024-03-29T04:45:25.213 INFO:tasks.workunit.client.1.smithi152.stdout:2/674: getdents d0/d3/d30/d15/d53/d54/d57 0 2024-03-29T04:45:25.216 INFO:tasks.workunit.client.1.smithi152.stdout:2/675: symlink d0/d3/d30/d15/d22/d4c/d5a/lcb 0 2024-03-29T04:45:25.218 INFO:tasks.workunit.client.1.smithi152.stdout:0/700: getdents d2/d29 0 2024-03-29T04:45:25.218 INFO:tasks.workunit.client.1.smithi152.stdout:2/676: chown d0/d3/d30/d15/d22/d4c/f51 1 1 2024-03-29T04:45:25.220 INFO:tasks.workunit.client.1.smithi152.stdout:2/677: mkdir d0/dcc 0 2024-03-29T04:45:25.224 INFO:tasks.workunit.client.1.smithi152.stdout:0/701: dread d2/da/d17/d57/d9c/f9f [0,4194304] 0 2024-03-29T04:45:25.224 INFO:tasks.workunit.client.1.smithi152.stdout:0/702: chown d2/c54 316708805 1 2024-03-29T04:45:25.228 INFO:tasks.workunit.client.1.smithi152.stdout:0/703: dread d2/da/d17/d57/f73 [0,4194304] 0 2024-03-29T04:45:25.234 INFO:tasks.workunit.client.1.smithi152.stdout:0/704: dwrite d2/da/d17/d50/dc6/fe4 [0,4194304] 0 2024-03-29T04:45:25.235 INFO:tasks.workunit.client.1.smithi152.stdout:0/705: chown d2/da/d17/dd8/db8 1 1 2024-03-29T04:45:25.236 INFO:tasks.workunit.client.1.smithi152.stdout:2/678: dread d0/d3/d30/d15/f29 [0,4194304] 0 2024-03-29T04:45:25.237 INFO:tasks.workunit.client.1.smithi152.stdout:2/679: chown d0/d3/df/db9 183 1 2024-03-29T04:45:25.238 INFO:tasks.workunit.client.1.smithi152.stdout:2/680: readlink d0/d3/d5f/lc4 0 2024-03-29T04:45:25.243 INFO:tasks.workunit.client.1.smithi152.stdout:2/681: dread d0/d3/d30/d15/d22/f91 [0,4194304] 0 2024-03-29T04:45:25.249 INFO:tasks.workunit.client.1.smithi152.stdout:2/682: getdents d0/d3/df/da7 0 2024-03-29T04:45:25.249 INFO:tasks.workunit.client.1.smithi152.stdout:2/683: stat d0/d3/df/da7 0 2024-03-29T04:45:25.250 INFO:tasks.workunit.client.1.smithi152.stdout:0/706: truncate d2/f8 3200041 0 2024-03-29T04:45:25.255 INFO:tasks.workunit.client.1.smithi152.stdout:2/684: dread d0/d3/df/d8c/fb1 [0,4194304] 0 2024-03-29T04:45:25.257 INFO:tasks.workunit.client.1.smithi152.stdout:0/707: dwrite d2/da/d17/d57/f64 [0,4194304] 0 2024-03-29T04:45:25.258 INFO:tasks.workunit.client.1.smithi152.stdout:0/708: chown d2/da/d17/d50/l7d 799876 1 2024-03-29T04:45:25.261 INFO:tasks.workunit.client.1.smithi152.stdout:2/685: symlink d0/d3/d30/d15/d53/d54/d57/lcd 0 2024-03-29T04:45:25.262 INFO:tasks.workunit.client.1.smithi152.stdout:2/686: chown d0/d3/d5f/f63 3837809 1 2024-03-29T04:45:25.263 INFO:tasks.workunit.client.1.smithi152.stdout:2/687: stat d0/d3/d30/d15/d53/d54/d57/f72 0 2024-03-29T04:45:25.263 INFO:tasks.workunit.client.1.smithi152.stdout:0/709: unlink d2/d29/fea 0 2024-03-29T04:45:25.264 INFO:tasks.workunit.client.1.smithi152.stdout:0/710: chown d2/da/d17/de7 4098157 1 2024-03-29T04:45:25.264 INFO:tasks.workunit.client.1.smithi152.stdout:2/688: chown d0/d3/d5f/lc4 1540749 1 2024-03-29T04:45:25.266 INFO:tasks.workunit.client.1.smithi152.stdout:0/711: symlink d2/lff 0 2024-03-29T04:45:25.271 INFO:tasks.workunit.client.1.smithi152.stdout:2/689: dwrite d0/d3/d30/d15/d53/f80 [0,4194304] 0 2024-03-29T04:45:25.284 INFO:tasks.workunit.client.1.smithi152.stdout:0/712: truncate d2/da/d3a/f97 1276216 0 2024-03-29T04:45:25.286 INFO:tasks.workunit.client.1.smithi152.stdout:2/690: write d0/d3/f1d [2578967,105831] 0 2024-03-29T04:45:25.287 INFO:tasks.workunit.client.1.smithi152.stdout:0/713: unlink d2/d29/lc2 0 2024-03-29T04:45:25.287 INFO:tasks.workunit.client.1.smithi152.stdout:0/714: chown d2 673 1 2024-03-29T04:45:25.289 INFO:tasks.workunit.client.1.smithi152.stdout:2/691: mkdir d0/d3/df/d9e/dce 0 2024-03-29T04:45:25.290 INFO:tasks.workunit.client.1.smithi152.stdout:0/715: mknod d2/da/d3a/d61/db4/c100 0 2024-03-29T04:45:25.292 INFO:tasks.workunit.client.1.smithi152.stdout:0/716: rmdir d2/da/d17/d57 39 2024-03-29T04:45:25.294 INFO:tasks.workunit.client.1.smithi152.stdout:2/692: rename d0/d3/d30/d15/da5/fb7 to d0/d3/d30/d15/d53/d54/fcf 0 2024-03-29T04:45:25.297 INFO:tasks.workunit.client.1.smithi152.stdout:4/621: sync 2024-03-29T04:45:25.297 INFO:tasks.workunit.client.1.smithi152.stdout:6/912: sync 2024-03-29T04:45:25.297 INFO:tasks.workunit.client.1.smithi152.stdout:3/774: sync 2024-03-29T04:45:25.297 INFO:tasks.workunit.client.1.smithi152.stdout:9/701: sync 2024-03-29T04:45:25.297 INFO:tasks.workunit.client.1.smithi152.stdout:7/657: sync 2024-03-29T04:45:25.297 INFO:tasks.workunit.client.1.smithi152.stdout:5/845: sync 2024-03-29T04:45:25.300 INFO:tasks.workunit.client.1.smithi152.stdout:5/846: fdatasync d6/d1d/daa/f111 0 2024-03-29T04:45:25.301 INFO:tasks.workunit.client.1.smithi152.stdout:2/693: readlink d0/d3/l7 0 2024-03-29T04:45:25.302 INFO:tasks.workunit.client.1.smithi152.stdout:9/702: creat d5/d8/ffa x:0 0 0 2024-03-29T04:45:25.302 INFO:tasks.workunit.client.1.smithi152.stdout:6/913: fsync d1/d3/d9/d13/dc0/ff9 0 2024-03-29T04:45:25.305 INFO:tasks.workunit.client.1.smithi152.stdout:5/847: write d6/d16/d25/d2e/da0/dab/f105 [972449,101247] 0 2024-03-29T04:45:25.306 INFO:tasks.workunit.client.1.smithi152.stdout:4/622: rename d0/d7/d25/d17/d37/l69 to d0/d7/d40/d62/lc6 0 2024-03-29T04:45:25.307 INFO:tasks.workunit.client.1.smithi152.stdout:2/694: mkdir d0/d3/d30/dd0 0 2024-03-29T04:45:25.309 INFO:tasks.workunit.client.1.smithi152.stdout:6/914: write d1/d3/d9/d13/d28/faa [1055849,4517] 0 2024-03-29T04:45:25.311 INFO:tasks.workunit.client.1.smithi152.stdout:0/717: dwrite d2/da/d17/d50/dac/fb2 [0,4194304] 0 2024-03-29T04:45:25.313 INFO:tasks.workunit.client.1.smithi152.stdout:2/695: write d0/d3/d30/d15/da5/fb6 [4917594,74495] 0 2024-03-29T04:45:25.316 INFO:tasks.workunit.client.1.smithi152.stdout:4/623: chown d0/d7/d25/f36 275874 1 2024-03-29T04:45:25.316 INFO:tasks.workunit.client.1.smithi152.stdout:3/775: dwrite d7/d19/d1b/d1e/da1/d41/d6a/fef [0,4194304] 0 2024-03-29T04:45:25.318 INFO:tasks.workunit.client.1.smithi152.stdout:9/703: rename d5/db/le0 to d5/d8/d31/d59/d6f/dbf/df4/lfb 0 2024-03-29T04:45:25.324 INFO:tasks.workunit.client.1.smithi152.stdout:5/848: dwrite d6/d1d/d30/d6a/fa6 [0,4194304] 0 2024-03-29T04:45:25.325 INFO:tasks.workunit.client.1.smithi152.stdout:5/849: stat d6/d1d/d30/d6a/d108/f7a 0 2024-03-29T04:45:25.326 INFO:tasks.workunit.client.1.smithi152.stdout:6/915: symlink d1/d3/d9/d13/d3e/l137 0 2024-03-29T04:45:25.330 INFO:tasks.workunit.client.1.smithi152.stdout:2/696: fdatasync d0/d3/df/db9/fc2 0 2024-03-29T04:45:25.332 INFO:tasks.workunit.client.1.smithi152.stdout:9/704: mknod d5/d8/d31/d75/db1/dcd/d6e/d73/d8e/cfc 0 2024-03-29T04:45:25.333 INFO:tasks.workunit.client.1.smithi152.stdout:7/658: link d7/d8/d12/d1d/c96 d7/d8/d12/d35/d68/dc3/cc7 0 2024-03-29T04:45:25.334 INFO:tasks.workunit.client.1.smithi152.stdout:9/705: stat d5/db/d92/f9a 0 2024-03-29T04:45:25.334 INFO:tasks.workunit.client.1.smithi152.stdout:0/718: dwrite d2/f5 [0,4194304] 0 2024-03-29T04:45:25.342 INFO:tasks.workunit.client.1.smithi152.stdout:9/706: fsync d5/d8/d31/d75/db1/dcd/f78 0 2024-03-29T04:45:25.343 INFO:tasks.workunit.client.1.smithi152.stdout:3/776: creat d7/d19/d1b/d1e/da1/d41/ff6 x:0 0 0 2024-03-29T04:45:25.343 INFO:tasks.workunit.client.1.smithi152.stdout:4/624: link d0/c3d d0/d7/d25/d17/d5e/cc7 0 2024-03-29T04:45:25.346 INFO:tasks.workunit.client.1.smithi152.stdout:5/850: creat d6/d1d/d3b/f12d x:0 0 0 2024-03-29T04:45:25.347 INFO:tasks.workunit.client.1.smithi152.stdout:0/719: mknod d2/da/d17/d57/d9c/dbe/c101 0 2024-03-29T04:45:25.348 INFO:tasks.workunit.client.1.smithi152.stdout:2/697: creat d0/d3/d30/d15/fd1 x:0 0 0 2024-03-29T04:45:25.349 INFO:tasks.workunit.client.1.smithi152.stdout:9/707: creat d5/d8/d31/d75/db1/ffd x:0 0 0 2024-03-29T04:45:25.349 INFO:tasks.workunit.client.1.smithi152.stdout:4/625: mkdir d0/d27/d6e/dc8 0 2024-03-29T04:45:25.350 INFO:tasks.workunit.client.1.smithi152.stdout:2/698: truncate d0/d3/df/d8c/fb1 863089 0 2024-03-29T04:45:25.353 INFO:tasks.workunit.client.1.smithi152.stdout:7/659: getdents d7/d8/d12/d1d/d41 0 2024-03-29T04:45:25.360 INFO:tasks.workunit.client.1.smithi152.stdout:4/626: dwrite d0/d27/f59 [0,4194304] 0 2024-03-29T04:45:25.361 INFO:tasks.workunit.client.1.smithi152.stdout:2/699: dwrite d0/d3/df/db8/fc0 [0,4194304] 0 2024-03-29T04:45:25.377 INFO:tasks.workunit.client.1.smithi152.stdout:0/720: write d2/d31/f36 [148204,5679] 0 2024-03-29T04:45:25.379 INFO:tasks.workunit.client.1.smithi152.stdout:2/700: creat d0/d3/d30/d15/d53/d54/d57/fd2 x:0 0 0 2024-03-29T04:45:25.379 INFO:tasks.workunit.client.1.smithi152.stdout:7/660: dread d7/d8/d93/fa9 [0,4194304] 0 2024-03-29T04:45:25.380 INFO:tasks.workunit.client.1.smithi152.stdout:3/777: truncate d7/d19/d1b/d1e/da1/d41/d61/d75/fb7 3305758 0 2024-03-29T04:45:25.381 INFO:tasks.workunit.client.1.smithi152.stdout:5/851: getdents d6/d1d/d30/d6a/dfb 0 2024-03-29T04:45:25.383 INFO:tasks.workunit.client.1.smithi152.stdout:5/852: read d6/d7/d75/f110 [488991,30412] 0 2024-03-29T04:45:25.384 INFO:tasks.workunit.client.1.smithi152.stdout:2/701: creat d0/d3/d30/fd3 x:0 0 0 2024-03-29T04:45:25.385 INFO:tasks.workunit.client.1.smithi152.stdout:9/708: write d5/db/d1b/f4f [1105155,42080] 0 2024-03-29T04:45:25.387 INFO:tasks.workunit.client.1.smithi152.stdout:7/661: mkdir d7/d8/d93/d99/dc8 0 2024-03-29T04:45:25.387 INFO:tasks.workunit.client.1.smithi152.stdout:3/778: rmdir d7/d19/d1b/d1e/da1/d41/d61/d75 39 2024-03-29T04:45:25.389 INFO:tasks.workunit.client.1.smithi152.stdout:0/721: symlink d2/da/d17/d57/l102 0 2024-03-29T04:45:25.390 INFO:tasks.workunit.client.1.smithi152.stdout:3/779: rename d7 to d7/d19/d1b/d1e/df0/df7 22 2024-03-29T04:45:25.392 INFO:tasks.workunit.client.1.smithi152.stdout:5/853: symlink d6/d1d/l12e 0 2024-03-29T04:45:25.392 INFO:tasks.workunit.client.1.smithi152.stdout:3/780: write d7/d19/d1b/d1e/da1/fd1 [850342,124935] 0 2024-03-29T04:45:25.393 INFO:tasks.workunit.client.1.smithi152.stdout:2/702: dwrite d0/faa [0,4194304] 0 2024-03-29T04:45:25.395 INFO:tasks.workunit.client.1.smithi152.stdout:2/703: write d0/d3/d30/d15/d22/d74/dbd/f79 [1471895,72799] 0 2024-03-29T04:45:25.397 INFO:tasks.workunit.client.1.smithi152.stdout:9/709: symlink d5/d12/db7/lfe 0 2024-03-29T04:45:25.402 INFO:tasks.workunit.client.1.smithi152.stdout:4/627: truncate d0/f11 3411907 0 2024-03-29T04:45:25.402 INFO:tasks.workunit.client.1.smithi152.stdout:5/854: symlink d6/d7/l12f 0 2024-03-29T04:45:25.402 INFO:tasks.workunit.client.1.smithi152.stdout:9/710: truncate d5/db/d1b/f46 4735141 0 2024-03-29T04:45:25.403 INFO:tasks.workunit.client.1.smithi152.stdout:2/704: write d0/d3/d30/d15/d53/d54/d57/fd2 [1023121,24813] 0 2024-03-29T04:45:25.403 INFO:tasks.workunit.client.1.smithi152.stdout:3/781: mkdir d7/d19/db5/df8 0 2024-03-29T04:45:25.407 INFO:tasks.workunit.client.1.smithi152.stdout:4/628: rename d0/d27/d6e/d8c/lae to d0/d7/d25/d17/d5e/d60/lc9 0 2024-03-29T04:45:25.414 INFO:tasks.workunit.client.1.smithi152.stdout:5/855: mknod d6/d16/d25/c130 0 2024-03-29T04:45:25.414 INFO:tasks.workunit.client.1.smithi152.stdout:5/856: fdatasync d6/d1d/daa/fb4 0 2024-03-29T04:45:25.416 INFO:tasks.workunit.client.1.smithi152.stdout:2/705: mknod d0/cd4 0 2024-03-29T04:45:25.416 INFO:tasks.workunit.client.1.smithi152.stdout:7/662: link d7/d8/d12/f7b d7/d8/db3/fc9 0 2024-03-29T04:45:25.417 INFO:tasks.workunit.client.1.smithi152.stdout:8/771: sync 2024-03-29T04:45:25.418 INFO:tasks.workunit.client.1.smithi152.stdout:3/782: mknod d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/cf9 0 2024-03-29T04:45:25.419 INFO:tasks.workunit.client.1.smithi152.stdout:8/772: write d1/d2/d3c/fac [797394,117765] 0 2024-03-29T04:45:25.421 INFO:tasks.workunit.client.1.smithi152.stdout:4/629: dread d0/d7/d25/d17/f4d [0,4194304] 0 2024-03-29T04:45:25.426 INFO:tasks.workunit.client.1.smithi152.stdout:2/706: rename d0/d3/d64/d85/d9d to d0/d3/d30/d15/d53/d54/dad/dd5 0 2024-03-29T04:45:25.427 INFO:tasks.workunit.client.1.smithi152.stdout:2/707: chown d0/d3/d30/d15/d53/fc3 0 1 2024-03-29T04:45:25.427 INFO:tasks.workunit.client.1.smithi152.stdout:5/857: dread f5 [0,4194304] 0 2024-03-29T04:45:25.429 INFO:tasks.workunit.client.1.smithi152.stdout:8/773: mkdir d1/d8a/d91/dee/d11b 0 2024-03-29T04:45:25.430 INFO:tasks.workunit.client.1.smithi152.stdout:2/708: chown d0/d3/d94/cc9 60 1 2024-03-29T04:45:25.431 INFO:tasks.workunit.client.1.smithi152.stdout:7/663: symlink d7/d8/db/lca 0 2024-03-29T04:45:25.431 INFO:tasks.workunit.client.1.smithi152.stdout:5/858: fdatasync d6/d1d/daa/db7/d56/dc1/de4/f12c 0 2024-03-29T04:45:25.432 INFO:tasks.workunit.client.1.smithi152.stdout:2/709: chown d0/d3/d64/d85/db3 18746 1 2024-03-29T04:45:25.438 INFO:tasks.workunit.client.1.smithi152.stdout:7/664: read - d7/d8/d12/d1d/d41/d45/f6a zero size 2024-03-29T04:45:25.438 INFO:tasks.workunit.client.1.smithi152.stdout:2/710: write d0/d3/df/fa2 [194957,47264] 0 2024-03-29T04:45:25.441 INFO:tasks.workunit.client.1.smithi152.stdout:8/774: dwrite d1/d2/d3/ffb [0,4194304] 0 2024-03-29T04:45:25.444 INFO:tasks.workunit.client.1.smithi152.stdout:3/783: dread d7/fa3 [0,4194304] 0 2024-03-29T04:45:25.444 INFO:tasks.workunit.client.1.smithi152.stdout:4/630: mknod d0/d7/d40/d23/d55/d70/d96/cca 0 2024-03-29T04:45:25.451 INFO:tasks.workunit.client.1.smithi152.stdout:7/665: dread d7/d8/d12/d1d/d41/f4d [0,4194304] 0 2024-03-29T04:45:25.451 INFO:tasks.workunit.client.1.smithi152.stdout:5/859: dwrite d6/d16/d25/d2e/da0/d92/fbc [0,4194304] 0 2024-03-29T04:45:25.452 INFO:tasks.workunit.client.1.smithi152.stdout:8/775: mknod d1/d2/d1d/dd3/db0/c11c 0 2024-03-29T04:45:25.462 INFO:tasks.workunit.client.1.smithi152.stdout:4/631: creat d0/d7/d40/d23/d55/d70/d96/fcb x:0 0 0 2024-03-29T04:45:25.466 INFO:tasks.workunit.client.1.smithi152.stdout:3/784: dwrite d7/f91 [0,4194304] 0 2024-03-29T04:45:25.466 INFO:tasks.workunit.client.1.smithi152.stdout:9/711: dwrite d5/d8/d31/d75/db1/dcd/f8f [0,4194304] 0 2024-03-29T04:45:25.468 INFO:tasks.workunit.client.1.smithi152.stdout:5/860: write d6/d16/d25/d2e/da0/f74 [915744,96171] 0 2024-03-29T04:45:25.473 INFO:tasks.workunit.client.1.smithi152.stdout:2/711: dwrite d0/d3/d30/d15/d22/d4c/f4d [4194304,4194304] 0 2024-03-29T04:45:25.479 INFO:tasks.workunit.client.1.smithi152.stdout:5/861: dwrite d6/d1d/daa/db7/d56/f94 [0,4194304] 0 2024-03-29T04:45:25.486 INFO:tasks.workunit.client.1.smithi152.stdout:7/666: rmdir d7/d8/d12/d1d/d41 39 2024-03-29T04:45:25.488 INFO:tasks.workunit.client.1.smithi152.stdout:4/632: write d0/d7/d40/d75/f7d [3517361,100455] 0 2024-03-29T04:45:25.488 INFO:tasks.workunit.client.1.smithi152.stdout:9/712: rmdir d5/d8/d31/d75/db1/dcd/d6e/d91/dd1 39 2024-03-29T04:45:25.489 INFO:tasks.workunit.client.1.smithi152.stdout:8/776: write d1/d2/d3c/f69 [1700557,85606] 0 2024-03-29T04:45:25.491 INFO:tasks.workunit.client.1.smithi152.stdout:2/712: mkdir d0/d3/d30/d15/d22/d4c/d5a/dd6 0 2024-03-29T04:45:25.493 INFO:tasks.workunit.client.1.smithi152.stdout:9/713: creat d5/d8/fff x:0 0 0 2024-03-29T04:45:25.496 INFO:tasks.workunit.client.1.smithi152.stdout:2/713: chown d0/d3/c75 614 1 2024-03-29T04:45:25.499 INFO:tasks.workunit.client.1.smithi152.stdout:3/785: dwrite d7/d19/f5c [0,4194304] 0 2024-03-29T04:45:25.502 INFO:tasks.workunit.client.1.smithi152.stdout:4/633: link d0/d7/d40/d23/d55/fb8 d0/d7/d40/d6d/fcc 0 2024-03-29T04:45:25.508 INFO:tasks.workunit.client.1.smithi152.stdout:8/777: link d1/d2/d1d/dc6/fc9 d1/d2/d1d/d36/d4b/f11d 0 2024-03-29T04:45:25.511 INFO:tasks.workunit.client.1.smithi152.stdout:2/714: creat d0/d3/d30/d15/da5/fd7 x:0 0 0 2024-03-29T04:45:25.512 INFO:tasks.workunit.client.1.smithi152.stdout:6/916: sync 2024-03-29T04:45:25.515 INFO:tasks.workunit.client.1.smithi152.stdout:4/634: write d0/d7/d40/d23/d55/fb8 [957061,94547] 0 2024-03-29T04:45:25.516 INFO:tasks.workunit.client.1.smithi152.stdout:5/862: link d6/d1d/d3b/l9b d6/d16/d109/l131 0 2024-03-29T04:45:25.517 INFO:tasks.workunit.client.1.smithi152.stdout:9/714: unlink d5/d8/d31/d75/db1/dcd/f5e 0 2024-03-29T04:45:25.517 INFO:tasks.workunit.client.1.smithi152.stdout:3/786: mknod d7/d19/d1b/d1e/da1/cfa 0 2024-03-29T04:45:25.517 INFO:tasks.workunit.client.1.smithi152.stdout:5/863: read - d6/d1d/d30/fd6 zero size 2024-03-29T04:45:25.518 INFO:tasks.workunit.client.1.smithi152.stdout:2/715: rmdir d0/d3/df/d8c/d97 39 2024-03-29T04:45:25.518 INFO:tasks.workunit.client.1.smithi152.stdout:8/778: write d1/dc5/fec [1038025,53712] 0 2024-03-29T04:45:25.519 INFO:tasks.workunit.client.1.smithi152.stdout:6/917: rmdir d1/d3/d1a/d1b/d87/d8f 39 2024-03-29T04:45:25.521 INFO:tasks.workunit.client.1.smithi152.stdout:9/715: write d5/db/d1b/fef [778186,55782] 0 2024-03-29T04:45:25.521 INFO:tasks.workunit.client.1.smithi152.stdout:5/864: write d6/d1d/daa/f111 [849191,65966] 0 2024-03-29T04:45:25.521 INFO:tasks.workunit.client.1.smithi152.stdout:4/635: dread d0/d7/d40/d2a/f94 [0,4194304] 0 2024-03-29T04:45:25.522 INFO:tasks.workunit.client.1.smithi152.stdout:3/787: rmdir d7/d6c 39 2024-03-29T04:45:25.522 INFO:tasks.workunit.client.1.smithi152.stdout:8/779: write d1/d8a/d91/dee/f9e [2015013,39064] 0 2024-03-29T04:45:25.527 INFO:tasks.workunit.client.1.smithi152.stdout:8/780: chown d1/d2/d1d/d36/d85/lbd 0 1 2024-03-29T04:45:25.528 INFO:tasks.workunit.client.1.smithi152.stdout:7/667: write d7/d8/d12/d35/f32 [1174158,105583] 0 2024-03-29T04:45:25.532 INFO:tasks.workunit.client.1.smithi152.stdout:2/716: rename d0/d3/df/l9f to d0/d3/d30/d15/d53/d54/dad/dd5/ld8 0 2024-03-29T04:45:25.535 INFO:tasks.workunit.client.1.smithi152.stdout:3/788: creat d7/d19/d1b/d1e/ffb x:0 0 0 2024-03-29T04:45:25.538 INFO:tasks.workunit.client.1.smithi152.stdout:5/865: link d6/d1d/d30/d6a/d108/f60 d6/d16/d25/d2e/d9d/d10c/f132 0 2024-03-29T04:45:25.539 INFO:tasks.workunit.client.1.smithi152.stdout:5/866: dread - d6/d1d/d30/fd6 zero size 2024-03-29T04:45:25.540 INFO:tasks.workunit.client.1.smithi152.stdout:5/867: dread - d6/d7/d75/de7/f119 zero size 2024-03-29T04:45:25.540 INFO:tasks.workunit.client.1.smithi152.stdout:5/868: stat d6/d1d/d30/d6a/d108/f7a 0 2024-03-29T04:45:25.540 INFO:tasks.workunit.client.1.smithi152.stdout:9/716: creat d5/d8/d31/d75/db1/dcd/f100 x:0 0 0 2024-03-29T04:45:25.543 INFO:tasks.workunit.client.1.smithi152.stdout:6/918: creat d1/d3/d9/d13/d28/d45/f138 x:0 0 0 2024-03-29T04:45:25.544 INFO:tasks.workunit.client.1.smithi152.stdout:8/781: write d1/d2/d1d/d36/d4b/f11d [908145,94280] 0 2024-03-29T04:45:25.545 INFO:tasks.workunit.client.1.smithi152.stdout:4/636: getdents d0/d27/d6e/d78 0 2024-03-29T04:45:25.547 INFO:tasks.workunit.client.1.smithi152.stdout:7/668: symlink d7/d8/d12/lcb 0 2024-03-29T04:45:25.547 INFO:tasks.workunit.client.1.smithi152.stdout:5/869: dread d6/d1d/d30/fa2 [0,4194304] 0 2024-03-29T04:45:25.548 INFO:tasks.workunit.client.1.smithi152.stdout:4/637: write d0/d7/d40/d6d/fa4 [3105,2148] 0 2024-03-29T04:45:25.549 INFO:tasks.workunit.client.1.smithi152.stdout:7/669: write d7/d8/d12/d35/d68/d84/fa4 [3289536,53195] 0 2024-03-29T04:45:25.556 INFO:tasks.workunit.client.1.smithi152.stdout:5/870: unlink d6/d1d/l12e 0 2024-03-29T04:45:25.557 INFO:tasks.workunit.client.1.smithi152.stdout:3/789: dwrite d7/d19/d1b/f21 [0,4194304] 0 2024-03-29T04:45:25.558 INFO:tasks.workunit.client.1.smithi152.stdout:2/717: rename d0/d3/d30/d15/fd1 to d0/d3/fd9 0 2024-03-29T04:45:25.564 INFO:tasks.workunit.client.1.smithi152.stdout:5/871: creat d6/d1d/daa/f133 x:0 0 0 2024-03-29T04:45:25.566 INFO:tasks.workunit.client.1.smithi152.stdout:3/790: rmdir d7/d19/d1b/d1e/da1/d41 39 2024-03-29T04:45:25.567 INFO:tasks.workunit.client.1.smithi152.stdout:3/791: stat d7/d46/f9f 0 2024-03-29T04:45:25.567 INFO:tasks.workunit.client.1.smithi152.stdout:7/670: mknod d7/d8/db3/ccc 0 2024-03-29T04:45:25.570 INFO:tasks.workunit.client.1.smithi152.stdout:8/782: link d1/d2/d1d/d36/d4b/l4d d1/d8a/d91/l11e 0 2024-03-29T04:45:25.572 INFO:tasks.workunit.client.1.smithi152.stdout:5/872: rename d6/d16/f4e to d6/d16/d25/d2e/dc7/f134 0 2024-03-29T04:45:25.575 INFO:tasks.workunit.client.1.smithi152.stdout:9/717: link d5/d8/d31/d75/db1/dcd/c56 d5/d8/d31/d75/db1/dcd/d6e/d73/d7a/c101 0 2024-03-29T04:45:25.576 INFO:tasks.workunit.client.1.smithi152.stdout:3/792: read d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/faa [3972360,55640] 0 2024-03-29T04:45:25.578 INFO:tasks.workunit.client.1.smithi152.stdout:4/638: getdents d0/d27 0 2024-03-29T04:45:25.584 INFO:tasks.workunit.client.1.smithi152.stdout:3/793: mknod d7/d19/d1b/d1e/da1/d41/d6a/cfc 0 2024-03-29T04:45:25.585 INFO:tasks.workunit.client.1.smithi152.stdout:9/718: dwrite d5/db/d1b/fe4 [0,4194304] 0 2024-03-29T04:45:25.588 INFO:tasks.workunit.client.1.smithi152.stdout:5/873: creat d6/d16/d25/d2e/d9d/d10c/f135 x:0 0 0 2024-03-29T04:45:25.590 INFO:tasks.workunit.client.1.smithi152.stdout:4/639: dwrite d0/d7/f8d [4194304,4194304] 0 2024-03-29T04:45:25.591 INFO:tasks.workunit.client.1.smithi152.stdout:5/874: dread - d6/d1d/daa/de5/fdb zero size 2024-03-29T04:45:25.593 INFO:tasks.workunit.client.1.smithi152.stdout:8/783: dread d1/d2/d1d/f35 [0,4194304] 0 2024-03-29T04:45:25.601 INFO:tasks.workunit.client.1.smithi152.stdout:7/671: dread d7/d8/d12/d1d/d41/f2c [0,4194304] 0 2024-03-29T04:45:25.601 INFO:tasks.workunit.client.1.smithi152.stdout:8/784: write d1/d2/d1d/f3a [572879,81818] 0 2024-03-29T04:45:25.605 INFO:tasks.workunit.client.1.smithi152.stdout:8/785: dread - d1/d2/d1d/dc6/ff7 zero size 2024-03-29T04:45:25.605 INFO:tasks.workunit.client.1.smithi152.stdout:5/875: dwrite d6/d1d/daa/de5/fdb [0,4194304] 0 2024-03-29T04:45:25.642 INFO:tasks.workunit.client.1.smithi152.stdout:3/794: dwrite d7/d19/d1b/d1e/d4e/f56 [0,4194304] 0 2024-03-29T04:45:25.643 INFO:tasks.workunit.client.1.smithi152.stdout:0/722: sync 2024-03-29T04:45:25.652 INFO:tasks.workunit.client.1.smithi152.stdout:0/723: creat d2/da/d17/d50/f103 x:0 0 0 2024-03-29T04:45:25.653 INFO:tasks.workunit.client.1.smithi152.stdout:0/724: fsync d2/da/d3a/d61/f9a 0 2024-03-29T04:45:25.655 INFO:tasks.workunit.client.1.smithi152.stdout:3/795: creat d7/d19/d1b/d1e/db9/ffd x:0 0 0 2024-03-29T04:45:25.659 INFO:tasks.workunit.client.1.smithi152.stdout:9/719: getdents d5/d8/d31/d59/d6f 0 2024-03-29T04:45:25.671 INFO:tasks.workunit.client.1.smithi152.stdout:5/876: getdents d6/d7/d75/de7 0 2024-03-29T04:45:25.672 INFO:tasks.workunit.client.1.smithi152.stdout:5/877: chown d6/d16/d25/d2e/dc7/cf0 11528984 1 2024-03-29T04:45:25.675 INFO:tasks.workunit.client.1.smithi152.stdout:5/878: mknod d6/d16/db3/dfd/c136 0 2024-03-29T04:45:25.677 INFO:tasks.workunit.client.1.smithi152.stdout:3/796: getdents d7/d19 0 2024-03-29T04:45:25.678 INFO:tasks.workunit.client.1.smithi152.stdout:5/879: fsync f5 0 2024-03-29T04:45:25.678 INFO:tasks.workunit.client.1.smithi152.stdout:3/797: write d7/d19/d1b/d1e/d4e/f70 [1670374,87221] 0 2024-03-29T04:45:25.679 INFO:tasks.workunit.client.1.smithi152.stdout:3/798: chown d7/d19/d1b/d1e/l3f 120795 1 2024-03-29T04:45:25.680 INFO:tasks.workunit.client.1.smithi152.stdout:3/799: chown d7/d19/d1b/d1e/da1 1 1 2024-03-29T04:45:25.681 INFO:tasks.workunit.client.1.smithi152.stdout:3/800: dread - d7/d19/d1b/d59/fd8 zero size 2024-03-29T04:45:25.682 INFO:tasks.workunit.client.1.smithi152.stdout:3/801: chown d7/d19/d1b/d1e/ffb 7032 1 2024-03-29T04:45:25.683 INFO:tasks.workunit.client.1.smithi152.stdout:5/880: mknod d6/d1d/d3b/c137 0 2024-03-29T04:45:25.683 INFO:tasks.workunit.client.1.smithi152.stdout:3/802: fsync d7/d19/d1b/fbe 0 2024-03-29T04:45:25.685 INFO:tasks.workunit.client.1.smithi152.stdout:5/881: symlink d6/d1d/d30/dc5/l138 0 2024-03-29T04:45:25.687 INFO:tasks.workunit.client.1.smithi152.stdout:3/803: creat d7/d19/d1b/d1e/da1/d41/d73/ffe x:0 0 0 2024-03-29T04:45:25.688 INFO:tasks.workunit.client.1.smithi152.stdout:3/804: creat d7/d46/fff x:0 0 0 2024-03-29T04:45:25.689 INFO:tasks.workunit.client.1.smithi152.stdout:3/805: readlink d7/d19/d1b/d1e/d4e/le9 0 2024-03-29T04:45:25.694 INFO:tasks.workunit.client.1.smithi152.stdout:3/806: link d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/fc1 d7/d19/d1b/d1e/d4e/d8f/f100 0 2024-03-29T04:45:25.695 INFO:tasks.workunit.client.1.smithi152.stdout:4/640: mknod d0/d27/d6e/ccd 0 2024-03-29T04:45:25.698 INFO:tasks.workunit.client.1.smithi152.stdout:4/641: rmdir d0/d27/d6e/d8c 39 2024-03-29T04:45:25.698 INFO:tasks.workunit.client.1.smithi152.stdout:3/807: mknod d7/d19/d1b/d1e/d22/c101 0 2024-03-29T04:45:25.706 INFO:tasks.workunit.client.1.smithi152.stdout:4/642: dwrite d0/d7/d40/d6d/fcc [0,4194304] 0 2024-03-29T04:45:25.709 INFO:tasks.workunit.client.1.smithi152.stdout:4/643: stat d0/d7/d40/d23/d55/d70/d96/fcb 0 2024-03-29T04:45:25.709 INFO:tasks.workunit.client.1.smithi152.stdout:3/808: dwrite d7/d19/d1b/d1e/da1/f54 [0,4194304] 0 2024-03-29T04:45:25.714 INFO:tasks.workunit.client.1.smithi152.stdout:4/644: unlink d0/d27/d6e/f87 0 2024-03-29T04:45:25.715 INFO:tasks.workunit.client.1.smithi152.stdout:3/809: symlink d7/d19/d1b/d1e/da1/d41/d61/d75/l102 0 2024-03-29T04:45:25.723 INFO:tasks.workunit.client.1.smithi152.stdout:7/672: mkdir d7/d8/dcd 0 2024-03-29T04:45:25.724 INFO:tasks.workunit.client.1.smithi152.stdout:7/673: truncate d7/f9d 549731 0 2024-03-29T04:45:25.729 INFO:tasks.workunit.client.1.smithi152.stdout:7/674: mknod d7/d8/d12/d35/cce 0 2024-03-29T04:45:25.730 INFO:tasks.workunit.client.1.smithi152.stdout:7/675: truncate d7/d8/d12/fbd 4902240 0 2024-03-29T04:45:25.732 INFO:tasks.workunit.client.1.smithi152.stdout:7/676: symlink d7/d8/d12/d1d/d41/lcf 0 2024-03-29T04:45:25.734 INFO:tasks.workunit.client.1.smithi152.stdout:7/677: rmdir d7/d8/db3/d72 39 2024-03-29T04:45:25.739 INFO:tasks.workunit.client.1.smithi152.stdout:8/786: rename d1/d2/d1d/dd3/db0/l109 to d1/l11f 0 2024-03-29T04:45:25.740 INFO:tasks.workunit.client.1.smithi152.stdout:8/787: rmdir d1/d2 39 2024-03-29T04:45:25.742 INFO:tasks.workunit.client.1.smithi152.stdout:9/720: rename d5/db/d1b/l27 to d5/db/d1b/d68/daa/l102 0 2024-03-29T04:45:25.745 INFO:tasks.workunit.client.1.smithi152.stdout:5/882: rename d6/d16/d25/d2e/d9d/d10c/f135 to d6/d16/db2/f139 0 2024-03-29T04:45:25.745 INFO:tasks.workunit.client.1.smithi152.stdout:5/883: chown d6/d1d/d3b/f39 15 1 2024-03-29T04:45:25.746 INFO:tasks.workunit.client.1.smithi152.stdout:5/884: truncate d6/d16/d25/d2e/dc7/f11c 884389 0 2024-03-29T04:45:25.750 INFO:tasks.workunit.client.1.smithi152.stdout:8/788: creat d1/d2/d1d/dd3/db0/d40/ddd/f120 x:0 0 0 2024-03-29T04:45:25.753 INFO:tasks.workunit.client.1.smithi152.stdout:5/885: dwrite d6/d16/d25/d2e/dc7/f11c [0,4194304] 0 2024-03-29T04:45:25.754 INFO:tasks.workunit.client.1.smithi152.stdout:8/789: creat d1/d2/d6d/f121 x:0 0 0 2024-03-29T04:45:25.756 INFO:tasks.workunit.client.1.smithi152.stdout:0/725: mkdir d2/da/d17/d104 0 2024-03-29T04:45:25.759 INFO:tasks.workunit.client.1.smithi152.stdout:8/790: dread d1/d2/d3c/f69 [0,4194304] 0 2024-03-29T04:45:25.760 INFO:tasks.workunit.client.1.smithi152.stdout:4/645: truncate d0/d7/d40/f80 2193082 0 2024-03-29T04:45:25.761 INFO:tasks.workunit.client.1.smithi152.stdout:8/791: read - d1/d2/d1d/d36/d4b/d51/fe1 zero size 2024-03-29T04:45:25.761 INFO:tasks.workunit.client.1.smithi152.stdout:4/646: chown d0/d27/d6e 523142735 1 2024-03-29T04:45:25.769 INFO:tasks.workunit.client.1.smithi152.stdout:4/647: mkdir d0/d7/d40/dce 0 2024-03-29T04:45:25.770 INFO:tasks.workunit.client.1.smithi152.stdout:8/792: dwrite d1/d2/d1d/f3a [0,4194304] 0 2024-03-29T04:45:25.772 INFO:tasks.workunit.client.1.smithi152.stdout:5/886: rmdir d6/d1d/daa/db7/d56/dc9/d11a 0 2024-03-29T04:45:25.773 INFO:tasks.workunit.client.1.smithi152.stdout:9/721: write d5/d8/f1a [121500,21870] 0 2024-03-29T04:45:25.774 INFO:tasks.workunit.client.1.smithi152.stdout:4/648: creat d0/d7/d25/d17/d5e/d60/d74/da2/fcf x:0 0 0 2024-03-29T04:45:25.774 INFO:tasks.workunit.client.1.smithi152.stdout:0/726: dread d2/d29/f6c [0,4194304] 0 2024-03-29T04:45:25.776 INFO:tasks.workunit.client.1.smithi152.stdout:5/887: creat d6/d7/f13a x:0 0 0 2024-03-29T04:45:25.776 INFO:tasks.workunit.client.1.smithi152.stdout:0/727: chown d2/d29/f37 13404560 1 2024-03-29T04:45:25.781 INFO:tasks.workunit.client.1.smithi152.stdout:0/728: write d2/da/d17/d50/f103 [349787,69883] 0 2024-03-29T04:45:25.783 INFO:tasks.workunit.client.1.smithi152.stdout:9/722: dwrite d5/f2f [0,4194304] 0 2024-03-29T04:45:25.785 INFO:tasks.workunit.client.1.smithi152.stdout:9/723: stat d5/d8/f53 0 2024-03-29T04:45:25.786 INFO:tasks.workunit.client.1.smithi152.stdout:5/888: getdents d6/d1d/daa/db7/dda 0 2024-03-29T04:45:25.788 INFO:tasks.workunit.client.1.smithi152.stdout:5/889: read - d6/d16/d25/d2e/d9d/f118 zero size 2024-03-29T04:45:25.794 INFO:tasks.workunit.client.1.smithi152.stdout:9/724: mkdir d5/d103 0 2024-03-29T04:45:25.794 INFO:tasks.workunit.client.1.smithi152.stdout:9/725: read - d5/d8/d31/d75/db1/dcd/d6e/d73/d8e/fa3 zero size 2024-03-29T04:45:25.795 INFO:tasks.workunit.client.1.smithi152.stdout:4/649: dread d0/d7/d40/d23/d55/d70/f7e [0,4194304] 0 2024-03-29T04:45:25.799 INFO:tasks.workunit.client.1.smithi152.stdout:4/650: rmdir d0 39 2024-03-29T04:45:25.800 INFO:tasks.workunit.client.1.smithi152.stdout:9/726: symlink d5/l104 0 2024-03-29T04:45:25.805 INFO:tasks.workunit.client.1.smithi152.stdout:4/651: read - d0/d7/f84 zero size 2024-03-29T04:45:25.806 INFO:tasks.workunit.client.1.smithi152.stdout:9/727: mknod d5/d8/d31/d75/db1/dcd/d6e/d91/dd1/c105 0 2024-03-29T04:45:25.807 INFO:tasks.workunit.client.1.smithi152.stdout:9/728: stat d5/db/d1b/d68 0 2024-03-29T04:45:25.808 INFO:tasks.workunit.client.1.smithi152.stdout:4/652: creat d0/d7/d40/d6d/fd0 x:0 0 0 2024-03-29T04:45:25.814 INFO:tasks.workunit.client.1.smithi152.stdout:9/729: dwrite d5/db/d1b/f40 [0,4194304] 0 2024-03-29T04:45:25.816 INFO:tasks.workunit.client.1.smithi152.stdout:5/890: dread d6/d1d/d30/f38 [0,4194304] 0 2024-03-29T04:45:25.816 INFO:tasks.workunit.client.1.smithi152.stdout:0/729: dread d2/da/df/dbb/ffc [0,4194304] 0 2024-03-29T04:45:25.823 INFO:tasks.workunit.client.1.smithi152.stdout:0/730: dwrite d2/da/d17/dd8/fba [0,4194304] 0 2024-03-29T04:45:25.824 INFO:tasks.workunit.client.1.smithi152.stdout:9/730: link d5/f2f d5/d8/d31/d75/db1/dcd/d6e/d73/f106 0 2024-03-29T04:45:25.828 INFO:tasks.workunit.client.1.smithi152.stdout:0/731: truncate d2/da/df/f4b 4937678 0 2024-03-29T04:45:25.830 INFO:tasks.workunit.client.1.smithi152.stdout:6/919: sync 2024-03-29T04:45:25.832 INFO:tasks.workunit.client.1.smithi152.stdout:9/731: creat d5/d8/d31/d59/dc3/ded/f107 x:0 0 0 2024-03-29T04:45:25.833 INFO:tasks.workunit.client.1.smithi152.stdout:9/732: chown d5/db/d92/lda 1 1 2024-03-29T04:45:25.834 INFO:tasks.workunit.client.1.smithi152.stdout:5/891: dread d6/d1d/d30/d6a/d108/fe [0,4194304] 0 2024-03-29T04:45:25.835 INFO:tasks.workunit.client.1.smithi152.stdout:0/732: getdents d2/da/d3a 0 2024-03-29T04:45:25.837 INFO:tasks.workunit.client.1.smithi152.stdout:5/892: creat d6/d1d/daa/f13b x:0 0 0 2024-03-29T04:45:25.839 INFO:tasks.workunit.client.1.smithi152.stdout:5/893: symlink d6/d7/l13c 0 2024-03-29T04:45:25.842 INFO:tasks.workunit.client.1.smithi152.stdout:5/894: chown d6/d16/d25/d2e/d9d/f118 11160 1 2024-03-29T04:45:25.842 INFO:tasks.workunit.client.1.smithi152.stdout:0/733: rename d2/f2e to d2/da/d17/d57/d9c/df9/f105 0 2024-03-29T04:45:25.843 INFO:tasks.workunit.client.1.smithi152.stdout:9/733: link d5/d8/d31/d75/db1/dcd/d6e/d73/d7a/l87 d5/d8/d31/d59/d85/l108 0 2024-03-29T04:45:25.844 INFO:tasks.workunit.client.1.smithi152.stdout:5/895: rmdir d6/d16/df8 39 2024-03-29T04:45:25.848 INFO:tasks.workunit.client.1.smithi152.stdout:9/734: fsync d5/db/d1b/f43 0 2024-03-29T04:45:25.849 INFO:tasks.workunit.client.1.smithi152.stdout:0/734: unlink d2/da/df/c89 0 2024-03-29T04:45:25.850 INFO:tasks.workunit.client.1.smithi152.stdout:6/920: read d1/f17 [901012,106593] 0 2024-03-29T04:45:25.850 INFO:tasks.workunit.client.1.smithi152.stdout:0/735: chown d2/d31/c5a 25569926 1 2024-03-29T04:45:25.853 INFO:tasks.workunit.client.1.smithi152.stdout:6/921: chown d1/d3/d1a/d1b/d87/d8f/d102/l115 742982559 1 2024-03-29T04:45:25.859 INFO:tasks.workunit.client.1.smithi152.stdout:9/735: creat d5/db/f109 x:0 0 0 2024-03-29T04:45:25.860 INFO:tasks.workunit.client.1.smithi152.stdout:5/896: getdents d6/d16/d25/d2e/dc7 0 2024-03-29T04:45:25.862 INFO:tasks.workunit.client.1.smithi152.stdout:5/897: read d6/d16/d25/f8e [654625,25925] 0 2024-03-29T04:45:25.863 INFO:tasks.workunit.client.1.smithi152.stdout:6/922: symlink d1/d3/df/d8a/d98/l139 0 2024-03-29T04:45:25.863 INFO:tasks.workunit.client.1.smithi152.stdout:4/653: write d0/f11 [214304,53426] 0 2024-03-29T04:45:25.865 INFO:tasks.workunit.client.1.smithi152.stdout:6/923: readlink d1/d3/d9/d13/d3e/l65 0 2024-03-29T04:45:25.866 INFO:tasks.workunit.client.1.smithi152.stdout:5/898: chown d6/d7/d75/f110 29 1 2024-03-29T04:45:25.867 INFO:tasks.workunit.client.1.smithi152.stdout:6/924: write d1/d3/d1a/d1b/d29/fc4 [678394,127747] 0 2024-03-29T04:45:25.868 INFO:tasks.workunit.client.1.smithi152.stdout:9/736: unlink d5/d8/d31/c3c 0 2024-03-29T04:45:25.869 INFO:tasks.workunit.client.1.smithi152.stdout:5/899: write d6/d1d/d30/d6a/dfb/f126 [139971,106675] 0 2024-03-29T04:45:25.877 INFO:tasks.workunit.client.1.smithi152.stdout:9/737: truncate d5/d8/d31/d75/db1/dcd/d6e/d73/d8e/f74 422297 0 2024-03-29T04:45:25.878 INFO:tasks.workunit.client.1.smithi152.stdout:4/654: mknod d0/d7/d25/d17/d5e/d60/d74/da2/da6/dac/cd1 0 2024-03-29T04:45:25.881 INFO:tasks.workunit.client.1.smithi152.stdout:5/900: dwrite d6/d16/d25/fdc [0,4194304] 0 2024-03-29T04:45:25.885 INFO:tasks.workunit.client.1.smithi152.stdout:4/655: creat d0/d7/d25/d17/d37/d77/fd2 x:0 0 0 2024-03-29T04:45:25.887 INFO:tasks.workunit.client.1.smithi152.stdout:4/656: readlink d0/d7/d25/d17/d5e/d60/d74/d57/l8f 0 2024-03-29T04:45:25.903 INFO:tasks.workunit.client.1.smithi152.stdout:0/736: rename d2/da/d17/d57/d9c/df9/f105 to d2/da/df/f106 0 2024-03-29T04:45:25.903 INFO:tasks.workunit.client.1.smithi152.stdout:0/737: fsync d2/d31/fda 0 2024-03-29T04:45:25.910 INFO:tasks.workunit.client.1.smithi152.stdout:0/738: creat d2/da/d17/dd8/db8/daf/f107 x:0 0 0 2024-03-29T04:45:25.912 INFO:tasks.workunit.client.1.smithi152.stdout:9/738: dread d5/db/d1b/f4d [8388608,4194304] 0 2024-03-29T04:45:25.912 INFO:tasks.workunit.client.1.smithi152.stdout:4/657: rename d0/d7/d25/d17/d37/c43 to d0/d7/d40/d23/d55/d6c/cd3 0 2024-03-29T04:45:25.913 INFO:tasks.workunit.client.1.smithi152.stdout:0/739: symlink d2/da/l108 0 2024-03-29T04:45:25.914 INFO:tasks.workunit.client.1.smithi152.stdout:6/925: write d1/d3/d1a/f20 [175407,9695] 0 2024-03-29T04:45:25.915 INFO:tasks.workunit.client.1.smithi152.stdout:9/739: dread - d5/d8/d31/d75/db1/dcd/d6e/d73/d8e/fa3 zero size 2024-03-29T04:45:25.915 INFO:tasks.workunit.client.1.smithi152.stdout:0/740: stat d2/da/d17/dd8 0 2024-03-29T04:45:25.916 INFO:tasks.workunit.client.1.smithi152.stdout:2/718: sync 2024-03-29T04:45:25.916 INFO:tasks.workunit.client.1.smithi152.stdout:0/741: chown d2/da/d17/f4c 9656627 1 2024-03-29T04:45:25.918 INFO:tasks.workunit.client.1.smithi152.stdout:4/658: creat d0/d7/d40/d23/d55/d6c/d91/d93/fd4 x:0 0 0 2024-03-29T04:45:25.921 INFO:tasks.workunit.client.1.smithi152.stdout:5/901: write d6/d16/d25/d2e/d9d/d10c/f132 [7995982,8473] 0 2024-03-29T04:45:25.925 INFO:tasks.workunit.client.1.smithi152.stdout:2/719: symlink d0/d3/d64/d85/db3/lda 0 2024-03-29T04:45:25.926 INFO:tasks.workunit.client.1.smithi152.stdout:9/740: dread d5/db/d92/f9a [0,4194304] 0 2024-03-29T04:45:25.927 INFO:tasks.workunit.client.1.smithi152.stdout:4/659: rename d0/d7/d40/d23/d55/c7c to d0/d7/d25/d17/d5e/d60/d74/da2/da6/dac/cd5 0 2024-03-29T04:45:25.929 INFO:tasks.workunit.client.1.smithi152.stdout:5/902: mkdir d6/d1d/d30/d6a/d108/d13d 0 2024-03-29T04:45:25.930 INFO:tasks.workunit.client.1.smithi152.stdout:2/720: stat d0/d3/d30/d15/l56 0 2024-03-29T04:45:25.933 INFO:tasks.workunit.client.1.smithi152.stdout:5/903: write d6/d16/d25/d2e/d9d/d10c/f132 [8372245,49767] 0 2024-03-29T04:45:25.935 INFO:tasks.workunit.client.1.smithi152.stdout:6/926: write d1/d3/d9/f11 [4546314,10320] 0 2024-03-29T04:45:25.937 INFO:tasks.workunit.client.1.smithi152.stdout:6/927: stat d1/f10a 0 2024-03-29T04:45:25.938 INFO:tasks.workunit.client.1.smithi152.stdout:4/660: dwrite d0/f13 [0,4194304] 0 2024-03-29T04:45:25.940 INFO:tasks.workunit.client.1.smithi152.stdout:5/904: truncate d6/d16/d25/d2e/f54 1880669 0 2024-03-29T04:45:25.943 INFO:tasks.workunit.client.1.smithi152.stdout:0/742: dread d2/f4 [0,4194304] 0 2024-03-29T04:45:25.944 INFO:tasks.workunit.client.1.smithi152.stdout:2/721: getdents d0/d3/df/db9 0 2024-03-29T04:45:25.947 INFO:tasks.workunit.client.1.smithi152.stdout:6/928: link d1/d3/d9/d13/d28/d45/f138 d1/d3/d91/f13a 0 2024-03-29T04:45:25.948 INFO:tasks.workunit.client.1.smithi152.stdout:5/905: fsync d6/d1d/daa/db7/d56/dc1/f122 0 2024-03-29T04:45:25.949 INFO:tasks.workunit.client.1.smithi152.stdout:9/741: dwrite d5/db/f5a [0,4194304] 0 2024-03-29T04:45:25.950 INFO:tasks.workunit.client.1.smithi152.stdout:2/722: mknod d0/d3/df/db9/cdb 0 2024-03-29T04:45:25.950 INFO:tasks.workunit.client.1.smithi152.stdout:4/661: dread d0/d7/d40/d23/d55/fb8 [0,4194304] 0 2024-03-29T04:45:25.951 INFO:tasks.workunit.client.1.smithi152.stdout:6/929: dread - d1/d3/d1a/d1b/d87/d8f/d102/f118 zero size 2024-03-29T04:45:25.952 INFO:tasks.workunit.client.1.smithi152.stdout:4/662: dread - d0/d7/d40/d2a/f42 zero size 2024-03-29T04:45:25.954 INFO:tasks.workunit.client.1.smithi152.stdout:9/742: readlink d5/d8/d31/d75/db1/dcd/d6e/d73/d7a/l87 0 2024-03-29T04:45:25.997 INFO:tasks.workunit.client.1.smithi152.stdout:2/723: rename d0/d3/d30/d15/da5/cae to d0/d3/d30/cdc 0 2024-03-29T04:45:25.998 INFO:tasks.workunit.client.1.smithi152.stdout:0/743: write d2/da/d17/f6d [3248931,63738] 0 2024-03-29T04:45:25.999 INFO:tasks.workunit.client.1.smithi152.stdout:4/663: truncate d0/d7/d40/d23/d55/d70/f7e 346041 0 2024-03-29T04:45:25.999 INFO:tasks.workunit.client.1.smithi152.stdout:9/743: write d5/d8/d31/d75/db1/dcd/f100 [614772,116681] 0 2024-03-29T04:45:26.001 INFO:tasks.workunit.client.1.smithi152.stdout:2/724: creat d0/d3/d30/d15/da5/fdd x:0 0 0 2024-03-29T04:45:26.004 INFO:tasks.workunit.client.1.smithi152.stdout:2/725: read d0/d3/f81 [78066,83334] 0 2024-03-29T04:45:26.006 INFO:tasks.workunit.client.1.smithi152.stdout:9/744: stat d5/d8/d31/c77 0 2024-03-29T04:45:26.006 INFO:tasks.workunit.client.1.smithi152.stdout:0/744: dwrite d2/d31/f36 [0,4194304] 0 2024-03-29T04:45:26.007 INFO:tasks.workunit.client.1.smithi152.stdout:4/664: dwrite d0/d27/d6e/fad [0,4194304] 0 2024-03-29T04:45:26.012 INFO:tasks.workunit.client.1.smithi152.stdout:2/726: rmdir d0/d3/d30/d15/d22 39 2024-03-29T04:45:26.016 INFO:tasks.workunit.client.1.smithi152.stdout:4/665: readlink d0/d7/d25/d17/d5e/d60/d74/da2/da6/dac/laf 0 2024-03-29T04:45:26.016 INFO:tasks.workunit.client.1.smithi152.stdout:0/745: dread d2/f78 [0,4194304] 0 2024-03-29T04:45:26.023 INFO:tasks.workunit.client.1.smithi152.stdout:2/727: mknod d0/d3/d30/dd0/cde 0 2024-03-29T04:45:26.024 INFO:tasks.workunit.client.1.smithi152.stdout:2/728: readlink d0/d3/ld 0 2024-03-29T04:45:26.025 INFO:tasks.workunit.client.1.smithi152.stdout:0/746: symlink d2/da/d17/d57/l109 0 2024-03-29T04:45:26.026 INFO:tasks.workunit.client.1.smithi152.stdout:0/747: chown d2/da/d3a/fd2 32673953 1 2024-03-29T04:45:26.026 INFO:tasks.workunit.client.1.smithi152.stdout:0/748: chown d2/da/d17/d57/d9c/df0 5214478 1 2024-03-29T04:45:26.028 INFO:tasks.workunit.client.1.smithi152.stdout:0/749: read d2/da/d17/f6d [3305836,29231] 0 2024-03-29T04:45:26.032 INFO:tasks.workunit.client.1.smithi152.stdout:9/745: dwrite d5/d8/d31/d75/db1/dcd/f5f [0,4194304] 0 2024-03-29T04:45:26.033 INFO:tasks.workunit.client.1.smithi152.stdout:9/746: stat d5/d8/d31/d75/db1/dcd/f50 0 2024-03-29T04:45:26.037 INFO:tasks.workunit.client.1.smithi152.stdout:0/750: mkdir d2/da/d17/d57/dc5/d10a 0 2024-03-29T04:45:26.037 INFO:tasks.workunit.client.1.smithi152.stdout:2/729: dwrite d0/d3/d30/d15/d22/d4c/f89 [0,4194304] 0 2024-03-29T04:45:26.039 INFO:tasks.workunit.client.1.smithi152.stdout:2/730: readlink d0/d3/d30/d15/d22/d74/dbd/l7d 0 2024-03-29T04:45:26.039 INFO:tasks.workunit.client.1.smithi152.stdout:2/731: fdatasync d0/d3/df/f43 0 2024-03-29T04:45:26.041 INFO:tasks.workunit.client.1.smithi152.stdout:4/666: truncate d0/d7/d25/f3f 276009 0 2024-03-29T04:45:26.044 INFO:tasks.workunit.client.1.smithi152.stdout:2/732: dread d0/d3/d30/d15/d22/d4c/f4d [4194304,4194304] 0 2024-03-29T04:45:26.049 INFO:tasks.workunit.client.1.smithi152.stdout:4/667: unlink d0/d7/d40/d62/c68 0 2024-03-29T04:45:26.058 INFO:tasks.workunit.client.1.smithi152.stdout:4/668: rmdir d0/d7/d25/d17/d5e/d60/d74/d57 39 2024-03-29T04:45:26.062 INFO:tasks.workunit.client.1.smithi152.stdout:2/733: mkdir d0/d3/d30/d15/d22/ddf 0 2024-03-29T04:45:26.062 INFO:tasks.workunit.client.1.smithi152.stdout:4/669: symlink d0/d7/d25/d17/d5e/ld6 0 2024-03-29T04:45:26.064 INFO:tasks.workunit.client.1.smithi152.stdout:3/810: sync 2024-03-29T04:45:26.064 INFO:tasks.workunit.client.1.smithi152.stdout:8/793: sync 2024-03-29T04:45:26.065 INFO:tasks.workunit.client.1.smithi152.stdout:7/678: sync 2024-03-29T04:45:26.066 INFO:tasks.workunit.client.1.smithi152.stdout:8/794: readlink d1/d8a/d91/dee/l4c 0 2024-03-29T04:45:26.066 INFO:tasks.workunit.client.1.smithi152.stdout:2/734: symlink d0/d3/d30/d15/d53/d54/dad/dd5/le0 0 2024-03-29T04:45:26.067 INFO:tasks.workunit.client.1.smithi152.stdout:0/751: dwrite d2/f38 [0,4194304] 0 2024-03-29T04:45:26.068 INFO:tasks.workunit.client.1.smithi152.stdout:7/679: fdatasync d7/d8/d12/f20 0 2024-03-29T04:45:26.069 INFO:tasks.workunit.client.1.smithi152.stdout:3/811: mknod d7/d19/d1b/d1e/db9/c103 0 2024-03-29T04:45:26.071 INFO:tasks.workunit.client.1.smithi152.stdout:9/747: dread d5/db/d1b/f51 [0,4194304] 0 2024-03-29T04:45:26.071 INFO:tasks.workunit.client.1.smithi152.stdout:4/670: creat d0/d7/d40/fd7 x:0 0 0 2024-03-29T04:45:26.075 INFO:tasks.workunit.client.1.smithi152.stdout:8/795: symlink d1/d2/d1d/d36/d107/l122 0 2024-03-29T04:45:26.075 INFO:tasks.workunit.client.1.smithi152.stdout:2/735: truncate d0/d3/d30/d15/d22/d74/dbd/f79 2290368 0 2024-03-29T04:45:26.077 INFO:tasks.workunit.client.1.smithi152.stdout:2/736: dread - d0/d3/d30/fd3 zero size 2024-03-29T04:45:26.077 INFO:tasks.workunit.client.1.smithi152.stdout:0/752: dread d2/da/d17/d50/dc6/fe4 [0,4194304] 0 2024-03-29T04:45:26.078 INFO:tasks.workunit.client.1.smithi152.stdout:7/680: mknod d7/d8/d12/d1d/d41/d45/cd0 0 2024-03-29T04:45:26.078 INFO:tasks.workunit.client.1.smithi152.stdout:8/796: read d1/d2/d1d/dc6/fc9 [518459,35340] 0 2024-03-29T04:45:26.079 INFO:tasks.workunit.client.1.smithi152.stdout:3/812: rename d7/d19/db5/cdf to d7/d19/d1b/d1e/d22/c104 0 2024-03-29T04:45:26.081 INFO:tasks.workunit.client.1.smithi152.stdout:9/748: unlink d5/d8/d31/d75/db1/dcd/d6e/d73/f106 0 2024-03-29T04:45:26.083 INFO:tasks.workunit.client.1.smithi152.stdout:4/671: rename d0/d27/l6b to d0/d27/ld8 0 2024-03-29T04:45:26.083 INFO:tasks.workunit.client.1.smithi152.stdout:4/672: dread - d0/fba zero size 2024-03-29T04:45:26.088 INFO:tasks.workunit.client.1.smithi152.stdout:4/673: creat d0/d7/d25/d17/d5e/fd9 x:0 0 0 2024-03-29T04:45:26.092 INFO:tasks.workunit.client.1.smithi152.stdout:3/813: mknod d7/d19/d1b/d1e/da1/d41/d61/c105 0 2024-03-29T04:45:26.093 INFO:tasks.workunit.client.1.smithi152.stdout:9/749: creat d5/d8/d31/d75/db1/dcd/d6e/db4/df2/f10a x:0 0 0 2024-03-29T04:45:26.094 INFO:tasks.workunit.client.1.smithi152.stdout:4/674: dwrite d0/d27/d6e/f97 [4194304,4194304] 0 2024-03-29T04:45:26.097 INFO:tasks.workunit.client.1.smithi152.stdout:7/681: rename d7/d8/db3/ccc to d7/d8/d12/cd1 0 2024-03-29T04:45:26.101 INFO:tasks.workunit.client.1.smithi152.stdout:3/814: unlink d7/d19/d1b/d1e/d22/f6f 0 2024-03-29T04:45:26.103 INFO:tasks.workunit.client.1.smithi152.stdout:9/750: mkdir d5/d8/d31/d59/d6f/d10b 0 2024-03-29T04:45:26.104 INFO:tasks.workunit.client.1.smithi152.stdout:4/675: dread - d0/d7/d40/d23/d55/d6c/f9e zero size 2024-03-29T04:45:26.105 INFO:tasks.workunit.client.1.smithi152.stdout:9/751: read d5/d8/d31/d75/db1/dcd/d6e/fc7 [3335632,61011] 0 2024-03-29T04:45:26.106 INFO:tasks.workunit.client.1.smithi152.stdout:9/752: chown d5/d12/fd9 92738 1 2024-03-29T04:45:26.106 INFO:tasks.workunit.client.1.smithi152.stdout:8/797: rename d1/d8a/d91/dee/f66 to d1/d2/d1d/dd3/db0/d40/d47/dca/f123 0 2024-03-29T04:45:26.107 INFO:tasks.workunit.client.1.smithi152.stdout:0/753: truncate d2/da/d17/db5/ff8 2466693 0 2024-03-29T04:45:26.120 INFO:tasks.workunit.client.1.smithi152.stdout:3/815: mknod d7/d19/db5/df8/c106 0 2024-03-29T04:45:26.123 INFO:tasks.workunit.client.1.smithi152.stdout:8/798: mknod d1/d2/d1d/dd3/db0/d7f/c124 0 2024-03-29T04:45:26.123 INFO:tasks.workunit.client.1.smithi152.stdout:7/682: truncate d7/d8/d12/d1d/d41/d45/f55 1135019 0 2024-03-29T04:45:26.123 INFO:tasks.workunit.client.1.smithi152.stdout:8/799: chown d1/d2/d3/c64 697521162 1 2024-03-29T04:45:26.126 INFO:tasks.workunit.client.1.smithi152.stdout:4/676: mkdir d0/d7/d40/d23/dda 0 2024-03-29T04:45:26.129 INFO:tasks.workunit.client.1.smithi152.stdout:9/753: dwrite d5/d8/f9 [0,4194304] 0 2024-03-29T04:45:26.129 INFO:tasks.workunit.client.1.smithi152.stdout:0/754: creat d2/da/d17/d57/d9c/dbe/df3/f10b x:0 0 0 2024-03-29T04:45:26.132 INFO:tasks.workunit.client.1.smithi152.stdout:8/800: dwrite d1/d2/d1d/d36/d85/fb4 [0,4194304] 0 2024-03-29T04:45:26.138 INFO:tasks.workunit.client.1.smithi152.stdout:4/677: dwrite d0/d7/d40/f8a [0,4194304] 0 2024-03-29T04:45:26.140 INFO:tasks.workunit.client.1.smithi152.stdout:9/754: creat d5/d8/d31/d75/db1/f10c x:0 0 0 2024-03-29T04:45:26.143 INFO:tasks.workunit.client.1.smithi152.stdout:8/801: symlink d1/d2/d1d/d36/d4b/d51/ddc/l125 0 2024-03-29T04:45:26.144 INFO:tasks.workunit.client.1.smithi152.stdout:8/802: chown d1/d2/d1d/d36/d5d 11598934 1 2024-03-29T04:45:26.144 INFO:tasks.workunit.client.1.smithi152.stdout:9/755: chown d5/d8/d31/d75/db1/dcd/d6e/d73/d7a/fac 112995212 1 2024-03-29T04:45:26.151 INFO:tasks.workunit.client.1.smithi152.stdout:3/816: link d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/fb2 d7/d19/d1b/d1e/da1/d41/f107 0 2024-03-29T04:45:26.152 INFO:tasks.workunit.client.1.smithi152.stdout:9/756: dwrite d5/fb5 [0,4194304] 0 2024-03-29T04:45:26.152 INFO:tasks.workunit.client.1.smithi152.stdout:4/678: unlink d0/d7/d40/d23/d55/d6c/d91/lb1 0 2024-03-29T04:45:26.155 INFO:tasks.workunit.client.1.smithi152.stdout:8/803: creat d1/d8a/f126 x:0 0 0 2024-03-29T04:45:26.157 INFO:tasks.workunit.client.1.smithi152.stdout:7/683: truncate d7/d8/d12/f6f 2295149 0 2024-03-29T04:45:26.160 INFO:tasks.workunit.client.1.smithi152.stdout:4/679: chown d0/d7/d25/d17/d5e/d60/d74/d57/f47 76532 1 2024-03-29T04:45:26.161 INFO:tasks.workunit.client.1.smithi152.stdout:8/804: mkdir d1/d2/d3c/d76/d127 0 2024-03-29T04:45:26.162 INFO:tasks.workunit.client.1.smithi152.stdout:4/680: write d0/d7/d40/fd7 [925506,13179] 0 2024-03-29T04:45:26.164 INFO:tasks.workunit.client.1.smithi152.stdout:0/755: dwrite d2/da/d17/d57/fcc [0,4194304] 0 2024-03-29T04:45:26.171 INFO:tasks.workunit.client.1.smithi152.stdout:4/681: dread - d0/d7/d25/d17/d5e/d60/d74/da2/fcf zero size 2024-03-29T04:45:26.171 INFO:tasks.workunit.client.1.smithi152.stdout:0/756: write d2/da/d3a/d6f/fc9 [597793,35016] 0 2024-03-29T04:45:26.174 INFO:tasks.workunit.client.1.smithi152.stdout:8/805: dwrite d1/d8a/d91/dee/fd7 [0,4194304] 0 2024-03-29T04:45:26.176 INFO:tasks.workunit.client.1.smithi152.stdout:8/806: write d1/d8a/fb3 [3556250,1240] 0 2024-03-29T04:45:26.179 INFO:tasks.workunit.client.1.smithi152.stdout:9/757: truncate d5/db/d1b/f4f 810550 0 2024-03-29T04:45:26.184 INFO:tasks.workunit.client.1.smithi152.stdout:9/758: creat d5/d8/d31/d59/d85/f10d x:0 0 0 2024-03-29T04:45:26.187 INFO:tasks.workunit.client.1.smithi152.stdout:7/684: dwrite d7/d8/d12/f7b [0,4194304] 0 2024-03-29T04:45:26.189 INFO:tasks.workunit.client.1.smithi152.stdout:7/685: write d7/d8/db3/f76 [620226,38616] 0 2024-03-29T04:45:26.192 INFO:tasks.workunit.client.1.smithi152.stdout:8/807: link d1/d2/d1d/d36/d5d/fb5 d1/d2/d1d/dd3/db0/d40/f128 0 2024-03-29T04:45:26.197 INFO:tasks.workunit.client.1.smithi152.stdout:9/759: unlink d5/d8/d31/d75/db1/dcd/d6e/d91/ca6 0 2024-03-29T04:45:26.197 INFO:tasks.workunit.client.1.smithi152.stdout:4/682: write d0/d7/d25/d17/f4d [847475,91088] 0 2024-03-29T04:45:26.198 INFO:tasks.workunit.client.1.smithi152.stdout:4/683: chown d0/d7/d25/d17/d5e/d60/d74/da2 2062164 1 2024-03-29T04:45:26.203 INFO:tasks.workunit.client.1.smithi152.stdout:8/808: mkdir d1/d2/d1d/d36/d85/d129 0 2024-03-29T04:45:26.205 INFO:tasks.workunit.client.1.smithi152.stdout:7/686: symlink d7/d8/d12/d35/ld2 0 2024-03-29T04:45:26.206 INFO:tasks.workunit.client.1.smithi152.stdout:8/809: write d1/d2/d1d/dd3/db0/d40/d47/f72 [1250637,30923] 0 2024-03-29T04:45:26.206 INFO:tasks.workunit.client.1.smithi152.stdout:4/684: rename d0/d7/d40/d23/d55/c86 to d0/d7/d25/d17/d37/d77/cdb 0 2024-03-29T04:45:26.209 INFO:tasks.workunit.client.1.smithi152.stdout:9/760: dwrite d5/d8/d31/d75/db1/dcd/f5f [4194304,4194304] 0 2024-03-29T04:45:26.209 INFO:tasks.workunit.client.1.smithi152.stdout:7/687: dread - d7/d8/db/df/fb8 zero size 2024-03-29T04:45:26.210 INFO:tasks.workunit.client.1.smithi152.stdout:9/761: readlink d5/d12/ld7 0 2024-03-29T04:45:26.210 INFO:tasks.workunit.client.1.smithi152.stdout:8/810: chown d1/d2/d3/f96 40120 1 2024-03-29T04:45:26.214 INFO:tasks.workunit.client.1.smithi152.stdout:7/688: read d7/d8/d12/f33 [371296,5759] 0 2024-03-29T04:45:26.215 INFO:tasks.workunit.client.1.smithi152.stdout:9/762: creat d5/d8/f10e x:0 0 0 2024-03-29T04:45:26.218 INFO:tasks.workunit.client.1.smithi152.stdout:4/685: creat d0/d7/d40/d23/dda/fdc x:0 0 0 2024-03-29T04:45:26.218 INFO:tasks.workunit.client.1.smithi152.stdout:8/811: readlink d1/d2/d1d/d36/d4b/l4d 0 2024-03-29T04:45:26.222 INFO:tasks.workunit.client.1.smithi152.stdout:9/763: symlink d5/d103/l10f 0 2024-03-29T04:45:26.223 INFO:tasks.workunit.client.1.smithi152.stdout:7/689: getdents d7/d8/d12/d35/d68/dc3 0 2024-03-29T04:45:26.224 INFO:tasks.workunit.client.1.smithi152.stdout:7/690: write d7/d8/d12/fb6 [285326,50492] 0 2024-03-29T04:45:26.231 INFO:tasks.workunit.client.1.smithi152.stdout:7/691: dread d7/d8/d12/d1d/d30/f9b [0,4194304] 0 2024-03-29T04:45:26.234 INFO:tasks.workunit.client.1.smithi152.stdout:7/692: write d7/d8/d12/d1d/d30/f9b [1517526,32646] 0 2024-03-29T04:45:26.235 INFO:tasks.workunit.client.1.smithi152.stdout:7/693: stat d7/d8/d12/d35/d68 0 2024-03-29T04:45:26.235 INFO:tasks.workunit.client.1.smithi152.stdout:7/694: stat d7/d8/fa 0 2024-03-29T04:45:26.240 INFO:tasks.workunit.client.1.smithi152.stdout:7/695: rename d7/d8/d12/d1d/d41/d83/lb5 to d7/d8/d93/d99/ld3 0 2024-03-29T04:45:26.241 INFO:tasks.workunit.client.1.smithi152.stdout:9/764: write d5/d8/d31/d59/d85/fe9 [1808092,100508] 0 2024-03-29T04:45:26.244 INFO:tasks.workunit.client.1.smithi152.stdout:9/765: mkdir d5/d8/d31/d75/db1/dcd/d6e/db4/d110 0 2024-03-29T04:45:26.244 INFO:tasks.workunit.client.1.smithi152.stdout:7/696: mknod d7/d8/d12/d35/cd4 0 2024-03-29T04:45:26.245 INFO:tasks.workunit.client.1.smithi152.stdout:7/697: write d7/d8/f6e [966354,52561] 0 2024-03-29T04:45:26.246 INFO:tasks.workunit.client.1.smithi152.stdout:7/698: write d7/f92 [2709058,53095] 0 2024-03-29T04:45:26.251 INFO:tasks.workunit.client.1.smithi152.stdout:9/766: write d5/f29 [3731960,129837] 0 2024-03-29T04:45:26.257 INFO:tasks.workunit.client.1.smithi152.stdout:7/699: link d7/d8/d12/d1d/f34 d7/d8/db/fd5 0 2024-03-29T04:45:26.258 INFO:tasks.workunit.client.1.smithi152.stdout:9/767: dwrite d5/d8/d31/d59/fb0 [0,4194304] 0 2024-03-29T04:45:26.260 INFO:tasks.workunit.client.1.smithi152.stdout:7/700: creat d7/d8/d12/d1d/d30/fd6 x:0 0 0 2024-03-29T04:45:26.261 INFO:tasks.workunit.client.1.smithi152.stdout:9/768: dread - d5/d8/d31/d75/db1/fbc zero size 2024-03-29T04:45:26.265 INFO:tasks.workunit.client.1.smithi152.stdout:7/701: creat d7/d8/d12/fd7 x:0 0 0 2024-03-29T04:45:26.268 INFO:tasks.workunit.client.1.smithi152.stdout:7/702: truncate d7/d8/d93/fbf 4967094 0 2024-03-29T04:45:26.274 INFO:tasks.workunit.client.1.smithi152.stdout:9/769: write d5/db/d1b/d68/daa/fb6 [497868,41057] 0 2024-03-29T04:45:26.282 INFO:tasks.workunit.client.1.smithi152.stdout:9/770: dwrite d5/db/d92/f9a [0,4194304] 0 2024-03-29T04:45:26.283 INFO:tasks.workunit.client.1.smithi152.stdout:9/771: truncate d5/db/d92/fd3 205447 0 2024-03-29T04:45:26.286 INFO:tasks.workunit.client.1.smithi152.stdout:7/703: dread d7/d8/db/f4b [0,4194304] 0 2024-03-29T04:45:26.286 INFO:tasks.workunit.client.1.smithi152.stdout:9/772: mknod d5/d8/d31/d75/db1/dcd/dc4/c111 0 2024-03-29T04:45:26.289 INFO:tasks.workunit.client.1.smithi152.stdout:7/704: creat d7/d8/db/df/fd8 x:0 0 0 2024-03-29T04:45:26.290 INFO:tasks.workunit.client.1.smithi152.stdout:7/705: write d7/d8/db3/f97 [982637,71494] 0 2024-03-29T04:45:26.292 INFO:tasks.workunit.client.1.smithi152.stdout:9/773: creat d5/d8/d31/f112 x:0 0 0 2024-03-29T04:45:26.293 INFO:tasks.workunit.client.1.smithi152.stdout:7/706: symlink d7/d8/d12/d1d/d41/ld9 0 2024-03-29T04:45:26.298 INFO:tasks.workunit.client.1.smithi152.stdout:9/774: dread d5/d8/d31/d59/fb3 [0,4194304] 0 2024-03-29T04:45:26.298 INFO:tasks.workunit.client.1.smithi152.stdout:7/707: rename d7/d8/d12/d1d/d30 to d7/d8/d12/d35/dda 0 2024-03-29T04:45:26.300 INFO:tasks.workunit.client.1.smithi152.stdout:7/708: read d7/d8/d93/d99/fad [4113109,8280] 0 2024-03-29T04:45:26.301 INFO:tasks.workunit.client.1.smithi152.stdout:7/709: chown d7/d8/d12/d1d/d41/d45/c6c 0 1 2024-03-29T04:45:26.302 INFO:tasks.workunit.client.1.smithi152.stdout:7/710: fsync d7/d8/d12/d35/faf 0 2024-03-29T04:45:26.304 INFO:tasks.workunit.client.1.smithi152.stdout:7/711: rename d7/d8 to d7/d8/d12/d35/d68/ddb 22 2024-03-29T04:45:26.305 INFO:tasks.workunit.client.1.smithi152.stdout:9/775: dwrite d5/d8/d31/d75/db1/dcd/f78 [0,4194304] 0 2024-03-29T04:45:26.309 INFO:tasks.workunit.client.1.smithi152.stdout:7/712: getdents d7/d8/d12/d35/dba 0 2024-03-29T04:45:26.310 INFO:tasks.workunit.client.1.smithi152.stdout:7/713: stat d7/d8/d12/d1d/d41/lcf 0 2024-03-29T04:45:26.315 INFO:tasks.workunit.client.1.smithi152.stdout:7/714: rename d7/d8/d12/d35/f2a to d7/d8/db3/d72/fdc 0 2024-03-29T04:45:26.318 INFO:tasks.workunit.client.1.smithi152.stdout:9/776: truncate d5/d8/d31/d75/db1/dcd/d6e/d91/fee 202330 0 2024-03-29T04:45:26.319 INFO:tasks.workunit.client.1.smithi152.stdout:7/715: fdatasync d7/d8/d12/d1d/d41/f2c 0 2024-03-29T04:45:26.321 INFO:tasks.workunit.client.1.smithi152.stdout:9/777: creat d5/db/d1b/d68/f113 x:0 0 0 2024-03-29T04:45:26.322 INFO:tasks.workunit.client.1.smithi152.stdout:7/716: rmdir d7/d8/db/df 39 2024-03-29T04:45:26.328 INFO:tasks.workunit.client.1.smithi152.stdout:9/778: dwrite d5/d8/d31/d75/db1/dcd/f63 [0,4194304] 0 2024-03-29T04:45:26.329 INFO:tasks.workunit.client.1.smithi152.stdout:7/717: dwrite d7/d8/db3/f76 [0,4194304] 0 2024-03-29T04:45:26.329 INFO:tasks.workunit.client.1.smithi152.stdout:9/779: read d5/db/f5a [1435344,96431] 0 2024-03-29T04:45:26.332 INFO:tasks.workunit.client.1.smithi152.stdout:5/906: sync 2024-03-29T04:45:26.332 INFO:tasks.workunit.client.1.smithi152.stdout:6/930: sync 2024-03-29T04:45:26.334 INFO:tasks.workunit.client.1.smithi152.stdout:5/907: write d6/d1d/daa/db7/d56/d63/db9/fd2 [623339,90946] 0 2024-03-29T04:45:26.337 INFO:tasks.workunit.client.1.smithi152.stdout:7/718: read d7/d8/d12/d1d/f34 [2012786,119830] 0 2024-03-29T04:45:26.348 INFO:tasks.workunit.client.1.smithi152.stdout:6/931: mkdir d1/d3/d1a/d1b/d29/d6f/d13b 0 2024-03-29T04:45:26.349 INFO:tasks.workunit.client.1.smithi152.stdout:5/908: rmdir d6/d1d 39 2024-03-29T04:45:26.352 INFO:tasks.workunit.client.1.smithi152.stdout:7/719: fdatasync d7/d8/db/f42 0 2024-03-29T04:45:26.354 INFO:tasks.workunit.client.1.smithi152.stdout:7/720: truncate d7/d8/d12/d35/d68/faa 493425 0 2024-03-29T04:45:26.355 INFO:tasks.workunit.client.1.smithi152.stdout:6/932: mknod d1/d3/d1a/d1b/d29/d6f/d95/c13c 0 2024-03-29T04:45:26.356 INFO:tasks.workunit.client.1.smithi152.stdout:6/933: dread - d1/d3/d91/f136 zero size 2024-03-29T04:45:26.359 INFO:tasks.workunit.client.1.smithi152.stdout:7/721: read d7/d8/d12/d35/dda/f3c [544079,1894] 0 2024-03-29T04:45:26.361 INFO:tasks.workunit.client.1.smithi152.stdout:7/722: truncate d7/d8/d12/d35/faf 405628 0 2024-03-29T04:45:26.361 INFO:tasks.workunit.client.1.smithi152.stdout:9/780: getdents d5/d8/d31/d75/db1/dcd/d6e/dea 0 2024-03-29T04:45:26.364 INFO:tasks.workunit.client.1.smithi152.stdout:5/909: symlink d6/d1d/d3b/l13e 0 2024-03-29T04:45:26.368 INFO:tasks.workunit.client.1.smithi152.stdout:7/723: rename d7/d8/c94 to d7/cdd 0 2024-03-29T04:45:26.369 INFO:tasks.workunit.client.1.smithi152.stdout:7/724: write d7/d8/d12/f81 [3546584,34146] 0 2024-03-29T04:45:26.370 INFO:tasks.workunit.client.1.smithi152.stdout:9/781: write d5/d12/fd9 [381023,129069] 0 2024-03-29T04:45:26.372 INFO:tasks.workunit.client.1.smithi152.stdout:9/782: read d5/f5c [1261307,120746] 0 2024-03-29T04:45:26.375 INFO:tasks.workunit.client.1.smithi152.stdout:2/737: sync 2024-03-29T04:45:26.375 INFO:tasks.workunit.client.1.smithi152.stdout:3/817: sync 2024-03-29T04:45:26.375 INFO:tasks.workunit.client.1.smithi152.stdout:0/757: sync 2024-03-29T04:45:26.377 INFO:tasks.workunit.client.1.smithi152.stdout:9/783: creat d5/db/d92/dd2/f114 x:0 0 0 2024-03-29T04:45:26.378 INFO:tasks.workunit.client.1.smithi152.stdout:3/818: read d7/d19/d1b/d1e/d22/f4d [2946042,16784] 0 2024-03-29T04:45:26.378 INFO:tasks.workunit.client.1.smithi152.stdout:7/725: creat d7/d8/d12/d1d/fde x:0 0 0 2024-03-29T04:45:26.380 INFO:tasks.workunit.client.1.smithi152.stdout:0/758: creat d2/da/df/dbb/f10c x:0 0 0 2024-03-29T04:45:26.380 INFO:tasks.workunit.client.1.smithi152.stdout:7/726: dread - d7/d8/d12/d1d/d41/d45/f6a zero size 2024-03-29T04:45:26.381 INFO:tasks.workunit.client.1.smithi152.stdout:6/934: write d1/d3/d1a/d1b/d29/d6f/f3c [507255,118863] 0 2024-03-29T04:45:26.383 INFO:tasks.workunit.client.1.smithi152.stdout:2/738: link d0/faa d0/d3/d64/fe1 0 2024-03-29T04:45:26.385 INFO:tasks.workunit.client.1.smithi152.stdout:9/784: dwrite d5/d12/fd9 [0,4194304] 0 2024-03-29T04:45:26.387 INFO:tasks.workunit.client.1.smithi152.stdout:3/819: mknod d7/d19/d1b/c108 0 2024-03-29T04:45:26.389 INFO:tasks.workunit.client.1.smithi152.stdout:2/739: rename d0/d3/df/d2c/f55 to d0/d3/d64/fe2 0 2024-03-29T04:45:26.390 INFO:tasks.workunit.client.1.smithi152.stdout:7/727: dwrite d7/d8/d37/f7f [0,4194304] 0 2024-03-29T04:45:26.402 INFO:tasks.workunit.client.1.smithi152.stdout:2/740: dwrite d0/d3/d30/d15/d53/fc3 [4194304,4194304] 0 2024-03-29T04:45:26.405 INFO:tasks.workunit.client.1.smithi152.stdout:5/910: dwrite d6/d1d/d30/d6a/d108/f66 [0,4194304] 0 2024-03-29T04:45:26.405 INFO:tasks.workunit.client.1.smithi152.stdout:7/728: unlink d7/d8/d12/d1d/f1e 0 2024-03-29T04:45:26.406 INFO:tasks.workunit.client.1.smithi152.stdout:5/911: write d6/d1d/daa/db7/d56/f94 [4137761,994] 0 2024-03-29T04:45:26.407 INFO:tasks.workunit.client.1.smithi152.stdout:3/820: fdatasync d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/f55 0 2024-03-29T04:45:26.411 INFO:tasks.workunit.client.1.smithi152.stdout:7/729: dread d7/d8/d12/f43 [0,4194304] 0 2024-03-29T04:45:26.414 INFO:tasks.workunit.client.1.smithi152.stdout:9/785: getdents d5/d103 0 2024-03-29T04:45:26.414 INFO:tasks.workunit.client.1.smithi152.stdout:6/935: link d1/d3/d9/d13/d28/d45/l67 d1/d3/d9/d13/d3e/d93/l13d 0 2024-03-29T04:45:26.415 INFO:tasks.workunit.client.1.smithi152.stdout:9/786: write d5/db/d92/f9a [189334,90427] 0 2024-03-29T04:45:26.415 INFO:tasks.workunit.client.1.smithi152.stdout:6/936: chown d1/d3/d9/d13/d28/fa4 41814 1 2024-03-29T04:45:26.417 INFO:tasks.workunit.client.1.smithi152.stdout:2/741: symlink d0/d3/d30/d15/d53/d54/d57/le3 0 2024-03-29T04:45:26.418 INFO:tasks.workunit.client.1.smithi152.stdout:9/787: write d5/d12/ff3 [4967264,78023] 0 2024-03-29T04:45:26.429 INFO:tasks.workunit.client.1.smithi152.stdout:2/742: creat d0/d3/d64/d85/fe4 x:0 0 0 2024-03-29T04:45:26.430 INFO:tasks.workunit.client.1.smithi152.stdout:3/821: dwrite d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/f6e [0,4194304] 0 2024-03-29T04:45:26.433 INFO:tasks.workunit.client.1.smithi152.stdout:4/686: sync 2024-03-29T04:45:26.433 INFO:tasks.workunit.client.1.smithi152.stdout:8/812: sync 2024-03-29T04:45:26.433 INFO:tasks.workunit.client.1.smithi152.stdout:9/788: mknod d5/d8/d31/d75/db1/dcd/d6e/d91/de1/c115 0 2024-03-29T04:45:26.436 INFO:tasks.workunit.client.1.smithi152.stdout:7/730: mkdir d7/dae/ddf 0 2024-03-29T04:45:26.438 INFO:tasks.workunit.client.1.smithi152.stdout:6/937: mknod d1/d3/d1a/d1b/d87/c13e 0 2024-03-29T04:45:26.439 INFO:tasks.workunit.client.1.smithi152.stdout:7/731: write d7/d8/f6e [1356544,94153] 0 2024-03-29T04:45:26.442 INFO:tasks.workunit.client.1.smithi152.stdout:2/743: read d0/d3/d30/d15/f25 [2118567,115523] 0 2024-03-29T04:45:26.447 INFO:tasks.workunit.client.1.smithi152.stdout:4/687: dwrite d0/d7/d40/d75/fc2 [0,4194304] 0 2024-03-29T04:45:26.449 INFO:tasks.workunit.client.1.smithi152.stdout:5/912: dread d6/d1d/d30/d6a/fa7 [0,4194304] 0 2024-03-29T04:45:26.452 INFO:tasks.workunit.client.1.smithi152.stdout:9/789: rmdir d5/d8/d31/d75/db1/dcd/d6e/d73 39 2024-03-29T04:45:26.455 INFO:tasks.workunit.client.1.smithi152.stdout:6/938: rmdir d1/d3/d1a/d1b/d10d 39 2024-03-29T04:45:26.457 INFO:tasks.workunit.client.1.smithi152.stdout:5/913: readlink d6/d1b/l29 0 2024-03-29T04:45:26.459 INFO:tasks.workunit.client.1.smithi152.stdout:3/822: dwrite d7/d19/d1b/d1e/ffb [0,4194304] 0 2024-03-29T04:45:26.461 INFO:tasks.workunit.client.1.smithi152.stdout:0/759: dwrite d2/f20 [0,4194304] 0 2024-03-29T04:45:26.477 INFO:tasks.workunit.client.1.smithi152.stdout:8/813: link d1/dc5/c101 d1/d2/d6d/c12a 0 2024-03-29T04:45:26.478 INFO:tasks.workunit.client.1.smithi152.stdout:4/688: readlink d0/d7/d40/d62/lc6 0 2024-03-29T04:45:26.478 INFO:tasks.workunit.client.1.smithi152.stdout:4/689: stat d0/d7/d40/l18 0 2024-03-29T04:45:26.514 INFO:tasks.workunit.client.1.smithi152.stdout:5/914: mknod d6/d1d/daa/db7/dda/c13f 0 2024-03-29T04:45:26.515 INFO:tasks.workunit.client.1.smithi152.stdout:9/790: mknod d5/d8/d31/d75/db1/dcd/d6e/d91/de1/c116 0 2024-03-29T04:45:26.515 INFO:tasks.workunit.client.1.smithi152.stdout:5/915: chown d6/d16/d25/d2e/da0/d92/cf3 1086850048 1 2024-03-29T04:45:26.515 INFO:tasks.workunit.client.1.smithi152.stdout:0/760: creat d2/da/df/f10d x:0 0 0 2024-03-29T04:45:26.516 INFO:tasks.workunit.client.1.smithi152.stdout:8/814: mkdir d1/d8a/d12b 0 2024-03-29T04:45:26.517 INFO:tasks.workunit.client.1.smithi152.stdout:3/823: mknod d7/d19/d1b/d1e/c109 0 2024-03-29T04:45:26.518 INFO:tasks.workunit.client.1.smithi152.stdout:4/690: mkdir d0/d7/d25/d17/d37/ddd 0 2024-03-29T04:45:26.520 INFO:tasks.workunit.client.1.smithi152.stdout:5/916: creat d6/d1d/d30/dc5/f140 x:0 0 0 2024-03-29T04:45:26.521 INFO:tasks.workunit.client.1.smithi152.stdout:0/761: dread d2/da/d17/fbd [0,4194304] 0 2024-03-29T04:45:26.522 INFO:tasks.workunit.client.1.smithi152.stdout:9/791: mknod d5/d8/d31/d75/db1/dcd/d6e/db4/c117 0 2024-03-29T04:45:26.523 INFO:tasks.workunit.client.1.smithi152.stdout:4/691: fdatasync d0/d7/d25/f36 0 2024-03-29T04:45:26.524 INFO:tasks.workunit.client.1.smithi152.stdout:3/824: mknod d7/d19/d1b/d1e/da1/d41/d6a/dcc/c10a 0 2024-03-29T04:45:26.526 INFO:tasks.workunit.client.1.smithi152.stdout:8/815: creat d1/d8a/d91/dee/d11b/f12c x:0 0 0 2024-03-29T04:45:26.527 INFO:tasks.workunit.client.1.smithi152.stdout:5/917: mknod d6/d16/d25/d2e/da0/dab/c141 0 2024-03-29T04:45:26.527 INFO:tasks.workunit.client.1.smithi152.stdout:8/816: fdatasync d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/f88 0 2024-03-29T04:45:26.529 INFO:tasks.workunit.client.1.smithi152.stdout:8/817: truncate d1/dc5/fec 1782814 0 2024-03-29T04:45:26.532 INFO:tasks.workunit.client.1.smithi152.stdout:5/918: symlink d6/d16/db3/l142 0 2024-03-29T04:45:26.535 INFO:tasks.workunit.client.1.smithi152.stdout:4/692: fdatasync d0/d7/d25/d17/d5e/d60/d74/f51 0 2024-03-29T04:45:26.536 INFO:tasks.workunit.client.1.smithi152.stdout:8/818: mkdir d1/d8a/d91/dee/d12d 0 2024-03-29T04:45:26.537 INFO:tasks.workunit.client.1.smithi152.stdout:3/825: dread d7/d19/d1b/d1e/da1/f54 [0,4194304] 0 2024-03-29T04:45:26.539 INFO:tasks.workunit.client.1.smithi152.stdout:5/919: unlink f5 0 2024-03-29T04:45:26.543 INFO:tasks.workunit.client.1.smithi152.stdout:8/819: truncate d1/d2/d1d/d36/fa0 1223741 0 2024-03-29T04:45:26.546 INFO:tasks.workunit.client.1.smithi152.stdout:9/792: getdents d5/d8/d31/d59 0 2024-03-29T04:45:26.548 INFO:tasks.workunit.client.1.smithi152.stdout:3/826: mknod d7/d19/d1b/d1e/c10b 0 2024-03-29T04:45:26.549 INFO:tasks.workunit.client.1.smithi152.stdout:6/939: truncate f0 1670335 0 2024-03-29T04:45:26.550 INFO:tasks.workunit.client.1.smithi152.stdout:2/744: dwrite d0/d3/f18 [4194304,4194304] 0 2024-03-29T04:45:26.554 INFO:tasks.workunit.client.1.smithi152.stdout:6/940: mkdir d1/d3/d1a/d1b/d29/dce/d13f 0 2024-03-29T04:45:26.555 INFO:tasks.workunit.client.1.smithi152.stdout:2/745: read d0/d3/d30/d15/d53/d54/dad/dd5/fb2 [59997,49997] 0 2024-03-29T04:45:26.557 INFO:tasks.workunit.client.1.smithi152.stdout:0/762: write d2/da/f98 [837583,108689] 0 2024-03-29T04:45:26.558 INFO:tasks.workunit.client.1.smithi152.stdout:5/920: dread d6/d16/d25/d2e/d9d/d10c/f132 [0,4194304] 0 2024-03-29T04:45:26.559 INFO:tasks.workunit.client.1.smithi152.stdout:0/763: dread - d2/da/d17/fec zero size 2024-03-29T04:45:26.559 INFO:tasks.workunit.client.1.smithi152.stdout:2/746: fdatasync d0/d3/d30/d15/d22/d4c/f4d 0 2024-03-29T04:45:26.561 INFO:tasks.workunit.client.1.smithi152.stdout:2/747: stat d0/d3/df/d9e/cb0 0 2024-03-29T04:45:26.566 INFO:tasks.workunit.client.1.smithi152.stdout:2/748: symlink d0/d3/d64/le5 0 2024-03-29T04:45:26.567 INFO:tasks.workunit.client.1.smithi152.stdout:0/764: dread d2/da/d17/f77 [0,4194304] 0 2024-03-29T04:45:26.568 INFO:tasks.workunit.client.1.smithi152.stdout:6/941: link d1/d3/d9/d13/dc0/dc3/dfb/f100 d1/d3/d9/d13/dc0/f140 0 2024-03-29T04:45:26.568 INFO:tasks.workunit.client.1.smithi152.stdout:4/693: write d0/d7/d40/f67 [900365,44704] 0 2024-03-29T04:45:26.573 INFO:tasks.workunit.client.1.smithi152.stdout:2/749: rmdir d0/d3/d30/d15/d22/d74/dbd 39 2024-03-29T04:45:26.574 INFO:tasks.workunit.client.1.smithi152.stdout:3/827: write d7/d46/f6d [32483,6158] 0 2024-03-29T04:45:26.576 INFO:tasks.workunit.client.1.smithi152.stdout:5/921: dwrite d6/d16/d25/d2e/da0/d92/dc0/dec/f120 [0,4194304] 0 2024-03-29T04:45:26.577 INFO:tasks.workunit.client.1.smithi152.stdout:9/793: dread d5/f7 [0,4194304] 0 2024-03-29T04:45:26.578 INFO:tasks.workunit.client.1.smithi152.stdout:0/765: creat d2/da/d17/d57/def/f10e x:0 0 0 2024-03-29T04:45:26.579 INFO:tasks.workunit.client.1.smithi152.stdout:5/922: chown d6/d7/d75 31507 1 2024-03-29T04:45:26.581 INFO:tasks.workunit.client.1.smithi152.stdout:9/794: write d5/db/f109 [128086,81685] 0 2024-03-29T04:45:26.582 INFO:tasks.workunit.client.1.smithi152.stdout:5/923: fdatasync d6/d1d/d30/d6a/faf 0 2024-03-29T04:45:26.582 INFO:tasks.workunit.client.1.smithi152.stdout:4/694: dread d0/d7/d25/d17/d5e/d60/d74/d57/f7a [0,4194304] 0 2024-03-29T04:45:26.583 INFO:tasks.workunit.client.1.smithi152.stdout:8/820: dread d1/d8a/d91/dee/f92 [0,4194304] 0 2024-03-29T04:45:26.590 INFO:tasks.workunit.client.1.smithi152.stdout:0/766: rename d2/da/d3a/d6f/c83 to d2/da/d17/d50/dc6/c10f 0 2024-03-29T04:45:26.590 INFO:tasks.workunit.client.1.smithi152.stdout:6/942: write d1/d3/df/d84/fa3 [822630,120093] 0 2024-03-29T04:45:26.593 INFO:tasks.workunit.client.1.smithi152.stdout:2/750: truncate d0/f3b 4038716 0 2024-03-29T04:45:26.596 INFO:tasks.workunit.client.1.smithi152.stdout:2/751: chown d0/d3/df/f76 52025 1 2024-03-29T04:45:26.597 INFO:tasks.workunit.client.1.smithi152.stdout:6/943: write d1/d3/df/d84/fa3 [1010413,63635] 0 2024-03-29T04:45:26.597 INFO:tasks.workunit.client.1.smithi152.stdout:3/828: dread d7/d19/d1b/d1e/d4e/f70 [0,4194304] 0 2024-03-29T04:45:26.597 INFO:tasks.workunit.client.1.smithi152.stdout:5/924: symlink d6/d1d/daa/db7/d56/dc1/l143 0 2024-03-29T04:45:26.598 INFO:tasks.workunit.client.1.smithi152.stdout:9/795: unlink d5/d8/d31/d75/db1/dcd/d6e/d73/d7a/c83 0 2024-03-29T04:45:26.599 INFO:tasks.workunit.client.1.smithi152.stdout:6/944: fsync d1/d3/d9/d13/f33 0 2024-03-29T04:45:26.601 INFO:tasks.workunit.client.1.smithi152.stdout:5/925: creat d6/d1d/d30/d6a/dfb/f144 x:0 0 0 2024-03-29T04:45:26.601 INFO:tasks.workunit.client.1.smithi152.stdout:6/945: stat d1/d3/d9/l40 0 2024-03-29T04:45:26.602 INFO:tasks.workunit.client.1.smithi152.stdout:9/796: fdatasync d5/d8/d31/d75/db1/dcd/f39 0 2024-03-29T04:45:26.603 INFO:tasks.workunit.client.1.smithi152.stdout:2/752: truncate d0/d3/df/db9/fc2 557717 0 2024-03-29T04:45:26.615 INFO:tasks.workunit.client.1.smithi152.stdout:9/797: chown d5/d8/d31/d59/d85 54694079 1 2024-03-29T04:45:26.616 INFO:tasks.workunit.client.1.smithi152.stdout:6/946: read d1/d3/d1a/d1b/d29/ff1 [3885781,78594] 0 2024-03-29T04:45:26.617 INFO:tasks.workunit.client.1.smithi152.stdout:2/753: stat d0/d3/d30/d15/d22/d74/l78 0 2024-03-29T04:45:26.618 INFO:tasks.workunit.client.1.smithi152.stdout:8/821: write d1/d8a/f8c [638783,119317] 0 2024-03-29T04:45:26.618 INFO:tasks.workunit.client.1.smithi152.stdout:4/695: write d0/d7/d25/d17/d5e/d60/f88 [868548,127094] 0 2024-03-29T04:45:26.620 INFO:tasks.workunit.client.1.smithi152.stdout:3/829: dwrite d7/d46/f9f [0,4194304] 0 2024-03-29T04:45:26.624 INFO:tasks.workunit.client.1.smithi152.stdout:5/926: dwrite d6/d1d/d30/d6a/dfb/f126 [0,4194304] 0 2024-03-29T04:45:26.625 INFO:tasks.workunit.client.1.smithi152.stdout:2/754: symlink d0/d3/d64/d85/le6 0 2024-03-29T04:45:26.626 INFO:tasks.workunit.client.1.smithi152.stdout:5/927: chown d6/d1d/daa/fca 61567 1 2024-03-29T04:45:26.636 INFO:tasks.workunit.client.1.smithi152.stdout:3/830: dwrite d7/d19/d1b/d1e/da1/d41/d6a/d78/f95 [0,4194304] 0 2024-03-29T04:45:26.638 INFO:tasks.workunit.client.1.smithi152.stdout:2/755: dwrite d0/d3/df/fa2 [0,4194304] 0 2024-03-29T04:45:26.643 INFO:tasks.workunit.client.1.smithi152.stdout:4/696: rename d0/d7/d25/d17/d5e/d60/d74/f81 to d0/d7/d40/d23/d55/d70/d96/fde 0 2024-03-29T04:45:26.647 INFO:tasks.workunit.client.1.smithi152.stdout:4/697: write d0/d7/d40/d23/d55/d6c/d91/d93/fd4 [703469,117865] 0 2024-03-29T04:45:26.647 INFO:tasks.workunit.client.1.smithi152.stdout:9/798: getdents d5/d8/d31/d59/dc3 0 2024-03-29T04:45:26.648 INFO:tasks.workunit.client.1.smithi152.stdout:9/799: dread - d5/d8/d31/f112 zero size 2024-03-29T04:45:26.649 INFO:tasks.workunit.client.1.smithi152.stdout:3/831: mkdir d7/d6c/d10c 0 2024-03-29T04:45:26.650 INFO:tasks.workunit.client.1.smithi152.stdout:2/756: write d0/d3/df/f35 [6076724,111991] 0 2024-03-29T04:45:26.652 INFO:tasks.workunit.client.1.smithi152.stdout:2/757: fdatasync d0/d3/d30/d15/d22/d4c/f90 0 2024-03-29T04:45:26.657 INFO:tasks.workunit.client.1.smithi152.stdout:2/758: write d0/d3/df/d8c/fb1 [1388151,130643] 0 2024-03-29T04:45:26.658 INFO:tasks.workunit.client.1.smithi152.stdout:9/800: dwrite d5/db/d1b/d68/daa/fb6 [0,4194304] 0 2024-03-29T04:45:26.658 INFO:tasks.workunit.client.1.smithi152.stdout:4/698: creat d0/d7/d25/d17/d5e/d60/d74/fdf x:0 0 0 2024-03-29T04:45:26.659 INFO:tasks.workunit.client.1.smithi152.stdout:6/947: dread d1/d3/d1a/d1b/d29/f7e [0,4194304] 0 2024-03-29T04:45:26.661 INFO:tasks.workunit.client.1.smithi152.stdout:9/801: stat d5/d8/d31/d75/db1/dcd/d6e/d91/de1/cf8 0 2024-03-29T04:45:26.661 INFO:tasks.workunit.client.1.smithi152.stdout:4/699: truncate d0/d7/d25/d17/f3b 639009 0 2024-03-29T04:45:26.662 INFO:tasks.workunit.client.1.smithi152.stdout:6/948: creat d1/d3/d9/d99/de5/f141 x:0 0 0 2024-03-29T04:45:26.662 INFO:tasks.workunit.client.1.smithi152.stdout:4/700: fsync d0/d7/d40/f67 0 2024-03-29T04:45:26.665 INFO:tasks.workunit.client.1.smithi152.stdout:8/822: dread d1/d2/d1d/f23 [4194304,4194304] 0 2024-03-29T04:45:26.666 INFO:tasks.workunit.client.1.smithi152.stdout:5/928: dread d6/d1d/f79 [0,4194304] 0 2024-03-29T04:45:26.666 INFO:tasks.workunit.client.1.smithi152.stdout:8/823: truncate d1/d8a/f126 411370 0 2024-03-29T04:45:26.667 INFO:tasks.workunit.client.1.smithi152.stdout:9/802: mknod d5/d12/c118 0 2024-03-29T04:45:26.668 INFO:tasks.workunit.client.1.smithi152.stdout:4/701: mkdir d0/d7/de0 0 2024-03-29T04:45:26.670 INFO:tasks.workunit.client.1.smithi152.stdout:9/803: fsync d5/d8/d31/f4c 0 2024-03-29T04:45:26.671 INFO:tasks.workunit.client.1.smithi152.stdout:8/824: rename d1/d2/d1d/d36/l93 to d1/d2/d3c/l12e 0 2024-03-29T04:45:26.672 INFO:tasks.workunit.client.1.smithi152.stdout:5/929: truncate d6/d16/d25/d2e/da0/f69 2178105 0 2024-03-29T04:45:26.672 INFO:tasks.workunit.client.1.smithi152.stdout:6/949: getdents d1/d3/d1a 0 2024-03-29T04:45:26.674 INFO:tasks.workunit.client.1.smithi152.stdout:9/804: mknod d5/d8/d31/d59/c119 0 2024-03-29T04:45:26.675 INFO:tasks.workunit.client.1.smithi152.stdout:6/950: stat d1/d3/df/dd2 0 2024-03-29T04:45:26.675 INFO:tasks.workunit.client.1.smithi152.stdout:8/825: fsync d1/d2/d1d/d36/d4b/d51/f77 0 2024-03-29T04:45:26.676 INFO:tasks.workunit.client.1.smithi152.stdout:5/930: mknod d6/d1d/daa/db7/dda/c145 0 2024-03-29T04:45:26.677 INFO:tasks.workunit.client.1.smithi152.stdout:2/759: dread d0/d3/d30/f1a [0,4194304] 0 2024-03-29T04:45:26.677 INFO:tasks.workunit.client.1.smithi152.stdout:8/826: write d1/d2/d1d/f41 [264281,56224] 0 2024-03-29T04:45:26.679 INFO:tasks.workunit.client.1.smithi152.stdout:5/931: write d6/d16/d25/d2e/da0/d92/fbc [1604065,69626] 0 2024-03-29T04:45:26.680 INFO:tasks.workunit.client.1.smithi152.stdout:5/932: chown d6/d7/c28 93861748 1 2024-03-29T04:45:26.680 INFO:tasks.workunit.client.1.smithi152.stdout:8/827: dread - d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/f117 zero size 2024-03-29T04:45:26.681 INFO:tasks.workunit.client.1.smithi152.stdout:8/828: read - d1/d2/d1d/dc6/ff7 zero size 2024-03-29T04:45:26.682 INFO:tasks.workunit.client.1.smithi152.stdout:8/829: chown d1/d2/d6d/f103 27223935 1 2024-03-29T04:45:26.685 INFO:tasks.workunit.client.1.smithi152.stdout:2/760: symlink d0/d3/df/da7/le7 0 2024-03-29T04:45:26.685 INFO:tasks.workunit.client.1.smithi152.stdout:2/761: chown d0/d3/df/d8c 73 1 2024-03-29T04:45:26.689 INFO:tasks.workunit.client.1.smithi152.stdout:5/933: rename d6/d1b/fd1 to d6/d16/d25/d2e/da0/f146 0 2024-03-29T04:45:26.691 INFO:tasks.workunit.client.1.smithi152.stdout:8/830: symlink d1/d2/d3/l12f 0 2024-03-29T04:45:26.691 INFO:tasks.workunit.client.1.smithi152.stdout:8/831: read - d1/d2/d3c/d76/fe9 zero size 2024-03-29T04:45:26.692 INFO:tasks.workunit.client.1.smithi152.stdout:8/832: write d1/f7c [1243242,67350] 0 2024-03-29T04:45:26.693 INFO:tasks.workunit.client.1.smithi152.stdout:8/833: readlink d1/d2/d1d/dd3/db0/d40/d47/l87 0 2024-03-29T04:45:26.697 INFO:tasks.workunit.client.1.smithi152.stdout:9/805: getdents d5/d8/d31/d75/db1/dcd/d6e 0 2024-03-29T04:45:26.699 INFO:tasks.workunit.client.1.smithi152.stdout:9/806: chown d5/db/f6c 12166495 1 2024-03-29T04:45:26.699 INFO:tasks.workunit.client.1.smithi152.stdout:4/702: write d0/d7/d40/d23/d55/d6c/f9e [487013,57104] 0 2024-03-29T04:45:26.700 INFO:tasks.workunit.client.1.smithi152.stdout:6/951: getdents d1/d3/d1a/d1b/d29/dce/dd7 0 2024-03-29T04:45:26.701 INFO:tasks.workunit.client.1.smithi152.stdout:4/703: chown d0/d7/d40/d2a/faa 1 1 2024-03-29T04:45:26.702 INFO:tasks.workunit.client.1.smithi152.stdout:4/704: chown d0/d7/d25/d17/d37/d77 229 1 2024-03-29T04:45:26.703 INFO:tasks.workunit.client.1.smithi152.stdout:5/934: rename d6/d1d/d30/c70 to d6/d1d/d30/d6a/d108/d13d/c147 0 2024-03-29T04:45:26.704 INFO:tasks.workunit.client.1.smithi152.stdout:5/935: write d6/d1d/d30/dc5/ff9 [90140,16247] 0 2024-03-29T04:45:26.705 INFO:tasks.workunit.client.1.smithi152.stdout:2/762: link d0/d3/d30/d15/da5/fd7 d0/fe8 0 2024-03-29T04:45:26.707 INFO:tasks.workunit.client.1.smithi152.stdout:2/763: write d0/d3/df/f43 [68731,78782] 0 2024-03-29T04:45:26.708 INFO:tasks.workunit.client.1.smithi152.stdout:6/952: chown d1/d3/d9/d13/d3e/l4b 13 1 2024-03-29T04:45:26.710 INFO:tasks.workunit.client.1.smithi152.stdout:8/834: dread d1/d2/d1d/dd3/db0/d40/d47/f72 [0,4194304] 0 2024-03-29T04:45:26.712 INFO:tasks.workunit.client.1.smithi152.stdout:5/936: mknod d6/d16/d25/d2e/d9d/c148 0 2024-03-29T04:45:26.712 INFO:tasks.workunit.client.1.smithi152.stdout:5/937: fdatasync d6/fe0 0 2024-03-29T04:45:26.715 INFO:tasks.workunit.client.1.smithi152.stdout:6/953: mkdir d1/d3/d9/d13/dc0/dc3/dfb/d142 0 2024-03-29T04:45:26.717 INFO:tasks.workunit.client.1.smithi152.stdout:8/835: write d1/d2/d1d/f4f [3231599,108129] 0 2024-03-29T04:45:26.718 INFO:tasks.workunit.client.1.smithi152.stdout:2/764: truncate d0/d3/df/db9/fc2 328861 0 2024-03-29T04:45:26.721 INFO:tasks.workunit.client.1.smithi152.stdout:9/807: dwrite d5/db/f60 [0,4194304] 0 2024-03-29T04:45:26.721 INFO:tasks.workunit.client.1.smithi152.stdout:4/705: rmdir d0/d7/de0 0 2024-03-29T04:45:26.725 INFO:tasks.workunit.client.1.smithi152.stdout:9/808: write d5/d8/d31/f4c [1524650,47743] 0 2024-03-29T04:45:26.731 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:26 smithi152 ceph-mon[37177]: pgmap v279: 65 pgs: 65 active+clean; 3.2 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 73 MiB/s rd, 201 MiB/s wr, 442 op/s 2024-03-29T04:45:26.731 INFO:tasks.workunit.client.1.smithi152.stdout:5/938: link d6/d1d/d30/c12b d6/d16/c149 0 2024-03-29T04:45:26.733 INFO:tasks.workunit.client.1.smithi152.stdout:4/706: chown d0/d7/d25/c30 20837 1 2024-03-29T04:45:26.733 INFO:tasks.workunit.client.1.smithi152.stdout:8/836: creat d1/d8a/d91/dee/d12d/f130 x:0 0 0 2024-03-29T04:45:26.735 INFO:tasks.workunit.client.1.smithi152.stdout:4/707: stat d0/d7/d40/dce 0 2024-03-29T04:45:26.736 INFO:tasks.workunit.client.1.smithi152.stdout:6/954: dwrite d1/d3/df/d8a/fb6 [0,4194304] 0 2024-03-29T04:45:26.738 INFO:tasks.workunit.client.1.smithi152.stdout:2/765: dwrite d0/d3/d30/d15/d53/d54/d57/fd2 [0,4194304] 0 2024-03-29T04:45:26.738 INFO:tasks.workunit.client.1.smithi152.stdout:5/939: creat d6/d16/d25/d2e/d9d/d10c/f14a x:0 0 0 2024-03-29T04:45:26.742 INFO:tasks.workunit.client.1.smithi152.stdout:5/940: dread - d6/d1d/d30/fd6 zero size 2024-03-29T04:45:26.742 INFO:tasks.workunit.client.1.smithi152.stdout:4/708: readlink d0/d7/d25/d17/d5e/ld6 0 2024-03-29T04:45:26.742 INFO:tasks.workunit.client.1.smithi152.stdout:8/837: symlink d1/d2/d1d/dd3/db0/d8b/l131 0 2024-03-29T04:45:26.744 INFO:tasks.workunit.client.1.smithi152.stdout:2/766: creat d0/d3/d30/d15/da5/fe9 x:0 0 0 2024-03-29T04:45:26.745 INFO:tasks.workunit.client.1.smithi152.stdout:6/955: rmdir d1/d3/d9/d13/dc0 39 2024-03-29T04:45:26.747 INFO:tasks.workunit.client.1.smithi152.stdout:8/838: symlink d1/d2/d1d/d36/d85/d129/l132 0 2024-03-29T04:45:26.749 INFO:tasks.workunit.client.1.smithi152.stdout:6/956: creat d1/d3/d1a/d1b/f143 x:0 0 0 2024-03-29T04:45:26.749 INFO:tasks.workunit.client.1.smithi152.stdout:6/957: truncate d1/f10a 285582 0 2024-03-29T04:45:26.751 INFO:tasks.workunit.client.1.smithi152.stdout:8/839: getdents d1/d2/d1d/d36/d107 0 2024-03-29T04:45:26.752 INFO:tasks.workunit.client.1.smithi152.stdout:6/958: creat d1/d3/d91/f144 x:0 0 0 2024-03-29T04:45:26.754 INFO:tasks.workunit.client.1.smithi152.stdout:6/959: symlink d1/d3/df/d84/l145 0 2024-03-29T04:45:26.754 INFO:tasks.workunit.client.1.smithi152.stdout:9/809: dread d5/d8/d31/d75/db1/dcd/d6e/d73/d8e/fd4 [0,4194304] 0 2024-03-29T04:45:26.754 INFO:tasks.workunit.client.1.smithi152.stdout:6/960: chown d1/d3/df/d8a/d98 12019722 1 2024-03-29T04:45:26.757 INFO:tasks.workunit.client.1.smithi152.stdout:6/961: write d1/d3/d1a/d1b/d10d/f116 [858459,56532] 0 2024-03-29T04:45:26.762 INFO:tasks.workunit.client.1.smithi152.stdout:9/810: dwrite d5/d8/d31/d59/dc3/ded/f107 [0,4194304] 0 2024-03-29T04:45:26.765 INFO:tasks.workunit.client.1.smithi152.stdout:9/811: fsync d5/d8/d31/d75/db1/dcd/d6e/d73/fdb 0 2024-03-29T04:45:26.769 INFO:tasks.workunit.client.1.smithi152.stdout:6/962: dwrite d1/d3/df/d8a/fa0 [0,4194304] 0 2024-03-29T04:45:26.773 INFO:tasks.workunit.client.1.smithi152.stdout:6/963: fsync d1/d3/d91/f13a 0 2024-03-29T04:45:26.773 INFO:tasks.workunit.client.1.smithi152.stdout:9/812: truncate d5/db/d1b/f8b 684330 0 2024-03-29T04:45:26.776 INFO:tasks.workunit.client.1.smithi152.stdout:6/964: mknod d1/d3/df/d8a/d98/c146 0 2024-03-29T04:45:26.788 INFO:tasks.workunit.client.1.smithi152.stdout:9/813: write d5/d8/d31/d75/db1/dcd/f54 [2045565,62888] 0 2024-03-29T04:45:26.788 INFO:tasks.workunit.client.1.smithi152.stdout:6/965: truncate d1/d3/d1a/d1b/f1e 397995 0 2024-03-29T04:45:26.796 INFO:tasks.workunit.client.1.smithi152.stdout:9/814: rename d5/d8/d31/c52 to d5/db/d92/c11a 0 2024-03-29T04:45:26.797 INFO:tasks.workunit.client.1.smithi152.stdout:6/966: unlink d1/d3/d9/d13/f26 0 2024-03-29T04:45:26.798 INFO:tasks.workunit.client.1.smithi152.stdout:9/815: rename d5/d8/d31/d75/db1/dcd to d5/d8/d31/d75/db1/dcd/d11b 22 2024-03-29T04:45:26.804 INFO:tasks.workunit.client.1.smithi152.stdout:9/816: dwrite d5/db/feb [4194304,4194304] 0 2024-03-29T04:45:26.807 INFO:tasks.workunit.client.1.smithi152.stdout:9/817: mknod d5/d8/d31/d75/db1/dcd/d6e/db4/c11c 0 2024-03-29T04:45:26.809 INFO:tasks.workunit.client.1.smithi152.stdout:9/818: symlink d5/d8/d31/d59/dc3/l11d 0 2024-03-29T04:45:26.816 INFO:tasks.workunit.client.1.smithi152.stdout:7/732: sync 2024-03-29T04:45:26.820 INFO:tasks.workunit.client.1.smithi152.stdout:7/733: dread d7/d8/d12/fb6 [0,4194304] 0 2024-03-29T04:45:26.824 INFO:tasks.workunit.client.1.smithi152.stdout:7/734: creat d7/d8/d12/d35/d68/dc3/fe0 x:0 0 0 2024-03-29T04:45:26.826 INFO:tasks.workunit.client.1.smithi152.stdout:7/735: fsync d7/d8/d93/d99/fad 0 2024-03-29T04:45:26.827 INFO:tasks.workunit.client.1.smithi152.stdout:7/736: creat d7/fe1 x:0 0 0 2024-03-29T04:45:26.830 INFO:tasks.workunit.client.1.smithi152.stdout:7/737: mkdir d7/d8/d12/de2 0 2024-03-29T04:45:26.834 INFO:tasks.workunit.client.1.smithi152.stdout:7/738: stat d7/d8/d12/d35/d68/la5 0 2024-03-29T04:45:26.834 INFO:tasks.workunit.client.1.smithi152.stdout:9/819: dread d5/d8/d31/d75/db1/dcd/f50 [0,4194304] 0 2024-03-29T04:45:26.834 INFO:tasks.workunit.client.1.smithi152.stdout:7/739: rmdir d7/dae/ddf 0 2024-03-29T04:45:26.836 INFO:tasks.workunit.client.1.smithi152.stdout:9/820: mknod d5/d8/d31/d75/db1/dcd/c11e 0 2024-03-29T04:45:26.841 INFO:tasks.workunit.client.1.smithi152.stdout:7/740: dwrite d7/d8/d93/d99/fbc [0,4194304] 0 2024-03-29T04:45:26.843 INFO:tasks.workunit.client.1.smithi152.stdout:7/741: mknod d7/d8/d12/d35/d68/d84/ce3 0 2024-03-29T04:45:26.844 INFO:tasks.workunit.client.1.smithi152.stdout:9/821: write d5/db/d1b/f37 [4585861,37259] 0 2024-03-29T04:45:26.844 INFO:tasks.workunit.client.1.smithi152.stdout:7/742: chown d7/d8/fa 311938667 1 2024-03-29T04:45:26.850 INFO:tasks.workunit.client.1.smithi152.stdout:7/743: unlink d7/d8/db/df/f2b 0 2024-03-29T04:45:26.852 INFO:tasks.workunit.client.1.smithi152.stdout:7/744: creat d7/d8/d12/d35/d68/dc3/fe4 x:0 0 0 2024-03-29T04:45:26.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.853+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1885749339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a0100440 msgr2=0x7f48a01008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:26.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.853+0000 7f48a5e4b700 1 --2- 172.21.15.142:0/1885749339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a0100440 0x7f48a01008a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4894009910 tx=0x7f4894009c20 comp rx=0 tx=0).stop 2024-03-29T04:45:26.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.853+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1885749339 shutdown_connections 2024-03-29T04:45:26.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.853+0000 7f48a5e4b700 1 --2- 172.21.15.142:0/1885749339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a0100440 0x7f48a01008a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:26.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.853+0000 7f48a5e4b700 1 --2- 172.21.15.142:0/1885749339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48a00ff240 0x7f48a00ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:26.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.853+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1885749339 >> 172.21.15.142:0/1885749339 conn(0x7f48a00fa7c0 msgr2=0x7f48a00fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:26.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.854+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1885749339 shutdown_connections 2024-03-29T04:45:26.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.854+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1885749339 wait complete. 2024-03-29T04:45:26.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.854+0000 7f48a5e4b700 1 Processor -- start 2024-03-29T04:45:26.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.855+0000 7f48a5e4b700 1 -- start start 2024-03-29T04:45:26.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.855+0000 7f48a5e4b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48a00ff240 0x7f48a019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:26.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.855+0000 7f48a5e4b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a0100440 0x7f48a019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:26.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.855+0000 7f48a5e4b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a019d4e0 con 0x7f48a0100440 2024-03-29T04:45:26.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.855+0000 7f48a5e4b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a019d620 con 0x7f48a00ff240 2024-03-29T04:45:26.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.855+0000 7f489effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a0100440 0x7f48a019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:26.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.855+0000 7f489effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a0100440 0x7f48a019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45298/0 (socket says 172.21.15.142:45298) 2024-03-29T04:45:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.855+0000 7f489effd700 1 -- 172.21.15.142:0/1207975438 learned_addr learned my addr 172.21.15.142:0/1207975438 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:45:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.855+0000 7f489f7fe700 1 --2- 172.21.15.142:0/1207975438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48a00ff240 0x7f48a019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.855+0000 7f489effd700 1 -- 172.21.15.142:0/1207975438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48a00ff240 msgr2=0x7f48a019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.855+0000 7f489effd700 1 --2- 172.21.15.142:0/1207975438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48a00ff240 0x7f48a019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.855+0000 7f489effd700 1 -- 172.21.15.142:0/1207975438 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48940095f0 con 0x7f48a0100440 2024-03-29T04:45:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.856+0000 7f489effd700 1 --2- 172.21.15.142:0/1207975438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a0100440 0x7f48a019cec0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4894000c00 tx=0x7f489400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.856+0000 7f489cff9700 1 -- 172.21.15.142:0/1207975438 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f489401d070 con 0x7f48a0100440 2024-03-29T04:45:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.856+0000 7f489cff9700 1 -- 172.21.15.142:0/1207975438 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4894005dc0 con 0x7f48a0100440 2024-03-29T04:45:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.856+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1207975438 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a01a2070 con 0x7f48a0100440 2024-03-29T04:45:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.856+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1207975438 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a01a2560 con 0x7f48a0100440 2024-03-29T04:45:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.856+0000 7f489cff9700 1 -- 172.21.15.142:0/1207975438 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48940226a0 con 0x7f48a0100440 2024-03-29T04:45:26.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.857+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1207975438 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a0104520 con 0x7f48a0100440 2024-03-29T04:45:26.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.858+0000 7f489cff9700 1 -- 172.21.15.142:0/1207975438 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f48940038f0 con 0x7f48a0100440 2024-03-29T04:45:26.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.858+0000 7f489cff9700 1 --2- 172.21.15.142:0/1207975438 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f488806c4e0 0x7f488806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:26.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.859+0000 7f489cff9700 1 -- 172.21.15.142:0/1207975438 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f489408cd60 con 0x7f48a0100440 2024-03-29T04:45:26.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.859+0000 7f489f7fe700 1 --2- 172.21.15.142:0/1207975438 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f488806c4e0 0x7f488806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:26.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.859+0000 7f489f7fe700 1 --2- 172.21.15.142:0/1207975438 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f488806c4e0 0x7f488806e9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f48a01002a0 tx=0x7f4890009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:26.859 INFO:tasks.workunit.client.1.smithi152.stdout:9/822: truncate d5/d8/d31/d75/db1/dcd/d6e/d73/d8e/fd4 166153 0 2024-03-29T04:45:26.861 INFO:tasks.workunit.client.1.smithi152.stdout:9/823: symlink d5/d8/d31/d59/d85/l11f 0 2024-03-29T04:45:26.864 INFO:tasks.workunit.client.1.smithi152.stdout:9/824: unlink d5/d12/f86 0 2024-03-29T04:45:26.864 INFO:tasks.workunit.client.1.smithi152.stdout:7/745: dwrite d7/d8/f8b [0,4194304] 0 2024-03-29T04:45:26.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:26.865+0000 7f489cff9700 1 -- 172.21.15.142:0/1207975438 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4894091070 con 0x7f48a0100440 2024-03-29T04:45:26.867 INFO:tasks.workunit.client.1.smithi152.stdout:9/825: write d5/d8/d31/d75/db1/dcd/d6e/d91/fc6 [2688308,129468] 0 2024-03-29T04:45:26.874 INFO:tasks.workunit.client.1.smithi152.stdout:7/746: dwrite d7/d8/d12/d35/dda/fd6 [0,4194304] 0 2024-03-29T04:45:26.878 INFO:tasks.workunit.client.1.smithi152.stdout:7/747: mkdir d7/d8/db/df/de5 0 2024-03-29T04:45:26.878 INFO:tasks.workunit.client.1.smithi152.stdout:9/826: dwrite d5/d8/ffa [0,4194304] 0 2024-03-29T04:45:26.880 INFO:tasks.workunit.client.1.smithi152.stdout:7/748: readlink d7/d8/d12/d1d/d41/l48 0 2024-03-29T04:45:26.881 INFO:tasks.workunit.client.1.smithi152.stdout:7/749: chown d7/d8/d37/c69 2010075 1 2024-03-29T04:45:26.884 INFO:tasks.workunit.client.1.smithi152.stdout:9/827: rename d5/db/feb to d5/d8/d31/d59/d6f/dbf/df4/f120 0 2024-03-29T04:45:26.888 INFO:tasks.workunit.client.1.smithi152.stdout:9/828: mknod d5/d8/d31/d75/db1/dcd/d6e/d91/de1/c121 0 2024-03-29T04:45:26.891 INFO:tasks.workunit.client.1.smithi152.stdout:7/750: dwrite d7/d8/d12/d35/d68/dc3/fe4 [0,4194304] 0 2024-03-29T04:45:26.894 INFO:tasks.workunit.client.1.smithi152.stdout:9/829: link d5/db/f65 d5/d8/d31/d75/db1/dcd/d6e/dea/f122 0 2024-03-29T04:45:26.895 INFO:tasks.workunit.client.1.smithi152.stdout:7/751: symlink d7/d8/dcd/le6 0 2024-03-29T04:45:26.896 INFO:tasks.workunit.client.1.smithi152.stdout:9/830: truncate d5/f5c 1340087 0 2024-03-29T04:45:26.898 INFO:tasks.workunit.client.1.smithi152.stdout:9/831: chown d5/d8/d31/d59/d6f/lc9 155146 1 2024-03-29T04:45:26.906 INFO:tasks.workunit.client.1.smithi152.stdout:9/832: dread d5/d8/d31/d75/db1/dcd/f76 [0,4194304] 0 2024-03-29T04:45:26.908 INFO:tasks.workunit.client.1.smithi152.stdout:7/752: write d7/d8/d12/d35/f1a [3529997,99255] 0 2024-03-29T04:45:26.910 INFO:tasks.workunit.client.1.smithi152.stdout:9/833: mknod d5/d8/d31/d59/d6f/d10b/c123 0 2024-03-29T04:45:26.910 INFO:tasks.workunit.client.1.smithi152.stdout:9/834: chown d5 4497 1 2024-03-29T04:45:26.911 INFO:tasks.workunit.client.1.smithi152.stdout:7/753: creat d7/d8/d12/de2/fe7 x:0 0 0 2024-03-29T04:45:26.912 INFO:tasks.workunit.client.1.smithi152.stdout:9/835: creat d5/d8/d31/d59/d85/f124 x:0 0 0 2024-03-29T04:45:26.913 INFO:tasks.workunit.client.1.smithi152.stdout:7/754: mknod d7/d8/db/df/de5/ce8 0 2024-03-29T04:45:26.914 INFO:tasks.workunit.client.1.smithi152.stdout:9/836: mkdir d5/d8/d31/d59/dc3/d125 0 2024-03-29T04:45:26.915 INFO:tasks.workunit.client.1.smithi152.stdout:9/837: chown d5/db/f60 1914890 1 2024-03-29T04:45:26.920 INFO:tasks.workunit.client.1.smithi152.stdout:9/838: write d5/d8/d31/d59/fb0 [4735169,80782] 0 2024-03-29T04:45:26.923 INFO:tasks.workunit.client.1.smithi152.stdout:9/839: truncate d5/d8/d31/d75/db1/fbc 604776 0 2024-03-29T04:45:26.927 INFO:tasks.workunit.client.1.smithi152.stdout:9/840: rename d5/d8/d31/d59/d85/l11f to d5/d8/d31/d75/db1/dcd/d6e/db4/l126 0 2024-03-29T04:45:26.928 INFO:tasks.workunit.client.1.smithi152.stdout:9/841: mknod d5/db/c127 0 2024-03-29T04:45:26.941 INFO:tasks.workunit.client.1.smithi152.stdout:9/842: dwrite d5/d8/d31/d75/db1/dcd/d6e/d73/d8e/f74 [0,4194304] 0 2024-03-29T04:45:26.942 INFO:tasks.workunit.client.1.smithi152.stdout:9/843: read - d5/db/d92/dd2/f114 zero size 2024-03-29T04:45:26.957 INFO:tasks.workunit.client.1.smithi152.stdout:9/844: dread d5/d12/ff3 [0,4194304] 0 2024-03-29T04:45:26.958 INFO:tasks.workunit.client.1.smithi152.stdout:9/845: write d5/db/d1b/f40 [1672724,118411] 0 2024-03-29T04:45:26.960 INFO:tasks.workunit.client.1.smithi152.stdout:9/846: dread d5/db/d92/fd3 [0,4194304] 0 2024-03-29T04:45:26.965 INFO:tasks.workunit.client.1.smithi152.stdout:9/847: link d5/d12/c4b d5/d8/d31/d75/db1/c128 0 2024-03-29T04:45:26.965 INFO:tasks.workunit.client.1.smithi152.stdout:9/848: chown d5/d12/db7 15029 1 2024-03-29T04:45:26.971 INFO:tasks.workunit.client.1.smithi152.stdout:9/849: unlink d5/db/c127 0 2024-03-29T04:45:26.973 INFO:tasks.workunit.client.1.smithi152.stdout:9/850: creat d5/d8/d31/d75/db1/dcd/d6e/dea/f129 x:0 0 0 2024-03-29T04:45:26.975 INFO:tasks.workunit.client.1.smithi152.stdout:9/851: symlink d5/d8/d31/d75/db1/dcd/dc4/l12a 0 2024-03-29T04:45:26.977 INFO:tasks.workunit.client.1.smithi152.stdout:9/852: creat d5/d8/d31/d75/db1/dcd/d6e/f12b x:0 0 0 2024-03-29T04:45:26.979 INFO:tasks.workunit.client.1.smithi152.stdout:9/853: unlink d5/f29 0 2024-03-29T04:45:26.980 INFO:tasks.workunit.client.1.smithi152.stdout:9/854: chown d5/db/d1b/fef 1298655801 1 2024-03-29T04:45:26.983 INFO:tasks.workunit.client.1.smithi152.stdout:9/855: link d5/d8/d31/d75/db1/dcd/f50 d5/db/d1b/f12c 0 2024-03-29T04:45:26.994 INFO:tasks.workunit.client.1.smithi152.stdout:9/856: symlink d5/d8/d31/d75/db1/dcd/d6e/d91/l12d 0 2024-03-29T04:45:26.996 INFO:tasks.workunit.client.1.smithi152.stdout:9/857: fsync d5/db/fd6 0 2024-03-29T04:45:26.997 INFO:tasks.workunit.client.1.smithi152.stdout:9/858: stat d5/d8/d31/d75/db1/dcd/d6e/d91/de1/c121 0 2024-03-29T04:45:27.006 INFO:tasks.workunit.client.1.smithi152.stdout:9/859: dwrite d5/db/d1b/f4f [0,4194304] 0 2024-03-29T04:45:27.010 INFO:tasks.workunit.client.1.smithi152.stdout:9/860: creat d5/d8/d31/d75/db1/dcd/d6e/db4/f12e x:0 0 0 2024-03-29T04:45:27.024 INFO:tasks.workunit.client.1.smithi152.stdout:9/861: dread d5/d8/d31/d75/db1/dcd/f3f [0,4194304] 0 2024-03-29T04:45:27.026 INFO:tasks.workunit.client.1.smithi152.stdout:9/862: creat d5/d8/d31/d59/d6f/d10b/f12f x:0 0 0 2024-03-29T04:45:27.049 INFO:tasks.workunit.client.1.smithi152.stdout:3/832: sync 2024-03-29T04:45:27.049 INFO:tasks.workunit.client.1.smithi152.stdout:0/767: sync 2024-03-29T04:45:27.050 INFO:tasks.workunit.client.1.smithi152.stdout:0/768: readlink d2/da/d17/dd8/db8/l8b 0 2024-03-29T04:45:27.051 INFO:tasks.workunit.client.1.smithi152.stdout:3/833: symlink d7/d19/d1b/d1e/da1/d41/d61/d99/l10d 0 2024-03-29T04:45:27.052 INFO:tasks.workunit.client.1.smithi152.stdout:3/834: write d7/d19/d1b/d1e/da1/d41/d61/d63/ff1 [17505,80763] 0 2024-03-29T04:45:27.053 INFO:tasks.workunit.client.1.smithi152.stdout:3/835: chown d7/d19/db5 46803425 1 2024-03-29T04:45:27.055 INFO:tasks.workunit.client.1.smithi152.stdout:0/769: write d2/da/d17/db5/fd1 [211629,59081] 0 2024-03-29T04:45:27.059 INFO:tasks.workunit.client.1.smithi152.stdout:0/770: creat d2/da/d17/f110 x:0 0 0 2024-03-29T04:45:27.060 INFO:tasks.workunit.client.1.smithi152.stdout:0/771: chown d2/da/d3a/d61/l95 237 1 2024-03-29T04:45:27.061 INFO:tasks.workunit.client.1.smithi152.stdout:0/772: write d2/da/df/fd5 [703549,69353] 0 2024-03-29T04:45:27.064 INFO:tasks.workunit.client.1.smithi152.stdout:0/773: mknod d2/da/d17/d57/dc5/d10a/c111 0 2024-03-29T04:45:27.069 INFO:tasks.workunit.client.1.smithi152.stdout:0/774: write d2/da/d3a/f68 [1043897,14036] 0 2024-03-29T04:45:27.074 INFO:tasks.workunit.client.1.smithi152.stdout:0/775: truncate d2/da/d17/d50/f103 382076 0 2024-03-29T04:45:27.078 INFO:tasks.workunit.client.1.smithi152.stdout:0/776: getdents d2/da/d17/d50 0 2024-03-29T04:45:27.081 INFO:tasks.workunit.client.1.smithi152.stdout:0/777: creat d2/da/d17/d57/d9c/f112 x:0 0 0 2024-03-29T04:45:27.086 INFO:tasks.workunit.client.1.smithi152.stdout:0/778: dwrite d2/da/df/f5b [0,4194304] 0 2024-03-29T04:45:27.093 INFO:tasks.workunit.client.1.smithi152.stdout:0/779: write d2/da/d17/f6d [654340,7214] 0 2024-03-29T04:45:27.098 INFO:tasks.workunit.client.1.smithi152.stdout:0/780: truncate d2/da/d3a/d61/f9a 2136354 0 2024-03-29T04:45:27.102 INFO:tasks.workunit.client.1.smithi152.stdout:0/781: creat d2/f113 x:0 0 0 2024-03-29T04:45:27.105 INFO:tasks.workunit.client.1.smithi152.stdout:0/782: rmdir d2/da/d17/dd8/db8/daf 39 2024-03-29T04:45:27.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:26 smithi142 ceph-mon[29027]: pgmap v279: 65 pgs: 65 active+clean; 3.2 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 73 MiB/s rd, 201 MiB/s wr, 442 op/s 2024-03-29T04:45:27.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.135+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1207975438 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f48a00611d0 con 0x7f488806c4e0 2024-03-29T04:45:27.136 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.136+0000 7f489cff9700 1 -- 172.21.15.142:0/1207975438 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f48a00611d0 con 0x7f488806c4e0 2024-03-29T04:45:27.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.138+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1207975438 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f488806c4e0 msgr2=0x7f488806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:27.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.138+0000 7f48a5e4b700 1 --2- 172.21.15.142:0/1207975438 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f488806c4e0 0x7f488806e9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f48a01002a0 tx=0x7f4890009260 comp rx=0 tx=0).stop 2024-03-29T04:45:27.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.138+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1207975438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a0100440 msgr2=0x7f48a019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:27.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.138+0000 7f48a5e4b700 1 --2- 172.21.15.142:0/1207975438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a0100440 0x7f48a019cec0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4894000c00 tx=0x7f489400f690 comp rx=0 tx=0).stop 2024-03-29T04:45:27.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.139+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1207975438 shutdown_connections 2024-03-29T04:45:27.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.139+0000 7f48a5e4b700 1 --2- 172.21.15.142:0/1207975438 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f488806c4e0 0x7f488806e9a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:27.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.139+0000 7f48a5e4b700 1 --2- 172.21.15.142:0/1207975438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48a00ff240 0x7f48a019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:27.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.139+0000 7f48a5e4b700 1 --2- 172.21.15.142:0/1207975438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a0100440 0x7f48a019cec0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:27.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.139+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1207975438 >> 172.21.15.142:0/1207975438 conn(0x7f48a00fa7c0 msgr2=0x7f48a0101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:27.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.139+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1207975438 shutdown_connections 2024-03-29T04:45:27.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.139+0000 7f48a5e4b700 1 -- 172.21.15.142:0/1207975438 wait complete. 2024-03-29T04:45:27.150 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:45:27.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.251+0000 7f24bf8b8700 1 -- 172.21.15.142:0/661153958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b8101db0 msgr2=0x7f24b8104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:27.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.251+0000 7f24bf8b8700 1 --2- 172.21.15.142:0/661153958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b8101db0 0x7f24b8104240 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009910 tx=0x7f24a8009c20 comp rx=0 tx=0).stop 2024-03-29T04:45:27.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.251+0000 7f24bf8b8700 1 -- 172.21.15.142:0/661153958 shutdown_connections 2024-03-29T04:45:27.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.251+0000 7f24bf8b8700 1 --2- 172.21.15.142:0/661153958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b8101db0 0x7f24b8104240 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:27.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.251+0000 7f24bf8b8700 1 --2- 172.21.15.142:0/661153958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24b80ff440 0x7f24b8101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:27.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.251+0000 7f24bf8b8700 1 -- 172.21.15.142:0/661153958 >> 172.21.15.142:0/661153958 conn(0x7f24b80fb060 msgr2=0x7f24b80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:27.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.251+0000 7f24bf8b8700 1 -- 172.21.15.142:0/661153958 shutdown_connections 2024-03-29T04:45:27.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.251+0000 7f24bf8b8700 1 -- 172.21.15.142:0/661153958 wait complete. 2024-03-29T04:45:27.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.252+0000 7f24bf8b8700 1 Processor -- start 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.252+0000 7f24bf8b8700 1 -- start start 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.252+0000 7f24bf8b8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b80ff440 0x7f24b819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.252+0000 7f24bf8b8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24b8101db0 0x7f24b819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.252+0000 7f24bf8b8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b819d550 con 0x7f24b80ff440 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.252+0000 7f24bf8b8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b819d690 con 0x7f24b8101db0 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.252+0000 7f24bd654700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b80ff440 0x7f24b819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.252+0000 7f24bd654700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b80ff440 0x7f24b819c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45328/0 (socket says 172.21.15.142:45328) 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.252+0000 7f24bd654700 1 -- 172.21.15.142:0/4078196526 learned_addr learned my addr 172.21.15.142:0/4078196526 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.252+0000 7f24bce53700 1 --2- 172.21.15.142:0/4078196526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24b8101db0 0x7f24b819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.253+0000 7f24bd654700 1 -- 172.21.15.142:0/4078196526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24b8101db0 msgr2=0x7f24b819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.253+0000 7f24bd654700 1 --2- 172.21.15.142:0/4078196526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24b8101db0 0x7f24b819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.253+0000 7f24bd654700 1 -- 172.21.15.142:0/4078196526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a80095f0 con 0x7f24b80ff440 2024-03-29T04:45:27.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.253+0000 7f24bce53700 1 --2- 172.21.15.142:0/4078196526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24b8101db0 0x7f24b819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:45:27.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.253+0000 7f24bd654700 1 --2- 172.21.15.142:0/4078196526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b80ff440 0x7f24b819c9f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f24b400b4b0 tx=0x7f24b400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:27.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.253+0000 7f24ae7fc700 1 -- 172.21.15.142:0/4078196526 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24b4011800 con 0x7f24b80ff440 2024-03-29T04:45:27.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.253+0000 7f24ae7fc700 1 -- 172.21.15.142:0/4078196526 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f24b4011e10 con 0x7f24b80ff440 2024-03-29T04:45:27.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.253+0000 7f24bf8b8700 1 -- 172.21.15.142:0/4078196526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b81a20f0 con 0x7f24b80ff440 2024-03-29T04:45:27.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.253+0000 7f24ae7fc700 1 -- 172.21.15.142:0/4078196526 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24b400f2e0 con 0x7f24b80ff440 2024-03-29T04:45:27.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.253+0000 7f24bf8b8700 1 -- 172.21.15.142:0/4078196526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b81a26c0 con 0x7f24b80ff440 2024-03-29T04:45:27.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.254+0000 7f24bf8b8700 1 -- 172.21.15.142:0/4078196526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b8196ba0 con 0x7f24b80ff440 2024-03-29T04:45:27.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.255+0000 7f24ae7fc700 1 -- 172.21.15.142:0/4078196526 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f24b4010380 con 0x7f24b80ff440 2024-03-29T04:45:27.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.255+0000 7f24ae7fc700 1 --2- 172.21.15.142:0/4078196526 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f24a406c4e0 0x7f24a406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:27.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.255+0000 7f24bce53700 1 --2- 172.21.15.142:0/4078196526 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f24a406c4e0 0x7f24a406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:27.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.255+0000 7f24ae7fc700 1 -- 172.21.15.142:0/4078196526 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f24b408b3f0 con 0x7f24b80ff440 2024-03-29T04:45:27.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.256+0000 7f24bce53700 1 --2- 172.21.15.142:0/4078196526 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f24a406c4e0 0x7f24a406e9a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24a800b5c0 tx=0x7f24a8005e20 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:27.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.260+0000 7f24ae7fc700 1 -- 172.21.15.142:0/4078196526 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f24b4059730 con 0x7f24b80ff440 2024-03-29T04:45:27.344 INFO:tasks.workunit.client.1.smithi152.stdout:4/709: sync 2024-03-29T04:45:27.351 INFO:tasks.workunit.client.1.smithi152.stdout:4/710: unlink d0/d7/d40/d23/d55/d6c/cd3 0 2024-03-29T04:45:27.354 INFO:tasks.workunit.client.1.smithi152.stdout:4/711: symlink d0/d7/d25/le1 0 2024-03-29T04:45:27.356 INFO:tasks.workunit.client.1.smithi152.stdout:4/712: symlink d0/d7/d40/d62/le2 0 2024-03-29T04:45:27.362 INFO:tasks.workunit.client.1.smithi152.stdout:4/713: dwrite d0/d7/d40/f67 [0,4194304] 0 2024-03-29T04:45:27.363 INFO:tasks.workunit.client.1.smithi152.stdout:4/714: readlink d0/d7/d25/d17/l32 0 2024-03-29T04:45:27.377 INFO:tasks.workunit.client.1.smithi152.stdout:4/715: dread d0/d7/d25/f3f [0,4194304] 0 2024-03-29T04:45:27.380 INFO:tasks.workunit.client.1.smithi152.stdout:4/716: mkdir d0/d7/d40/d23/dda/de3 0 2024-03-29T04:45:27.381 INFO:tasks.workunit.client.1.smithi152.stdout:4/717: dread - d0/d7/d40/d23/d6a/fa5 zero size 2024-03-29T04:45:27.386 INFO:tasks.workunit.client.1.smithi152.stdout:4/718: creat d0/d27/d6e/d8c/fe4 x:0 0 0 2024-03-29T04:45:27.393 INFO:tasks.workunit.client.1.smithi152.stdout:4/719: getdents d0/d7/d25/d17/d5e/d60/d74 0 2024-03-29T04:45:27.395 INFO:tasks.workunit.client.1.smithi152.stdout:4/720: dread - d0/d7/d25/d17/d5e/d60/d74/da2/fcf zero size 2024-03-29T04:45:27.408 INFO:tasks.workunit.client.1.smithi152.stdout:4/721: unlink d0/d7/d25/d17/d37/d77/cdb 0 2024-03-29T04:45:27.411 INFO:tasks.workunit.client.1.smithi152.stdout:5/941: sync 2024-03-29T04:45:27.411 INFO:tasks.workunit.client.1.smithi152.stdout:6/967: sync 2024-03-29T04:45:27.411 INFO:tasks.workunit.client.1.smithi152.stdout:2/767: sync 2024-03-29T04:45:27.411 INFO:tasks.workunit.client.1.smithi152.stdout:7/755: sync 2024-03-29T04:45:27.411 INFO:tasks.workunit.client.1.smithi152.stdout:8/840: sync 2024-03-29T04:45:27.411 INFO:tasks.workunit.client.1.smithi152.stdout:9/863: sync 2024-03-29T04:45:27.411 INFO:tasks.workunit.client.1.smithi152.stdout:0/783: sync 2024-03-29T04:45:27.411 INFO:tasks.workunit.client.1.smithi152.stdout:3/836: sync 2024-03-29T04:45:27.412 INFO:tasks.workunit.client.1.smithi152.stdout:4/722: mknod d0/d7/d40/d62/ce5 0 2024-03-29T04:45:27.414 INFO:tasks.workunit.client.1.smithi152.stdout:2/768: write d0/d3/f42 [6069414,96290] 0 2024-03-29T04:45:27.417 INFO:tasks.workunit.client.1.smithi152.stdout:7/756: chown d7/d8/d93/fbf 118242495 1 2024-03-29T04:45:27.419 INFO:tasks.workunit.client.1.smithi152.stdout:4/723: dread - d0/d7/d40/d62/f9a zero size 2024-03-29T04:45:27.424 INFO:tasks.workunit.client.1.smithi152.stdout:8/841: write d1/d8a/d91/f104 [151245,13810] 0 2024-03-29T04:45:27.426 INFO:tasks.workunit.client.1.smithi152.stdout:7/757: write d7/d8/d12/fbd [4901905,2451] 0 2024-03-29T04:45:27.427 INFO:tasks.workunit.client.1.smithi152.stdout:3/837: symlink d7/d19/d1b/d1e/da1/d41/d61/d63/l10e 0 2024-03-29T04:45:27.431 INFO:tasks.workunit.client.1.smithi152.stdout:3/838: truncate d7/fa4 1004493 0 2024-03-29T04:45:27.432 INFO:tasks.workunit.client.1.smithi152.stdout:7/758: dread - d7/d8/d12/d1d/d41/d45/f6a zero size 2024-03-29T04:45:27.434 INFO:tasks.workunit.client.1.smithi152.stdout:5/942: rename d6/d1d/d30/d6a/d108/f7a to d6/d16/f14b 0 2024-03-29T04:45:27.435 INFO:tasks.workunit.client.1.smithi152.stdout:5/943: fsync d6/d16/d109/f11f 0 2024-03-29T04:45:27.436 INFO:tasks.workunit.client.1.smithi152.stdout:9/864: dwrite d5/d8/d31/d75/db1/dcd/f54 [0,4194304] 0 2024-03-29T04:45:27.444 INFO:tasks.workunit.client.1.smithi152.stdout:7/759: truncate d7/d8/db/df/f58 516852 0 2024-03-29T04:45:27.449 INFO:tasks.workunit.client.1.smithi152.stdout:9/865: rename d5/db/f58 to d5/d12/db7/f130 0 2024-03-29T04:45:27.451 INFO:tasks.workunit.client.1.smithi152.stdout:3/839: dwrite d7/fa3 [0,4194304] 0 2024-03-29T04:45:27.451 INFO:tasks.workunit.client.1.smithi152.stdout:8/842: link d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/f88 d1/d2/d1d/dc6/f133 0 2024-03-29T04:45:27.453 INFO:tasks.workunit.client.1.smithi152.stdout:9/866: creat d5/d8/d31/d75/db1/dcd/d6e/db4/f131 x:0 0 0 2024-03-29T04:45:27.454 INFO:tasks.workunit.client.1.smithi152.stdout:9/867: dread - d5/d8/fff zero size 2024-03-29T04:45:27.456 INFO:tasks.workunit.client.1.smithi152.stdout:7/760: unlink d7/d8/db3/lc0 0 2024-03-29T04:45:27.458 INFO:tasks.workunit.client.1.smithi152.stdout:7/761: fdatasync d7/d8/d12/d35/dda/fd6 0 2024-03-29T04:45:27.458 INFO:tasks.workunit.client.1.smithi152.stdout:7/762: fsync d7/d8/db3/fc9 0 2024-03-29T04:45:27.460 INFO:tasks.workunit.client.1.smithi152.stdout:9/868: fdatasync d5/db/d1b/d68/dca/fc8 0 2024-03-29T04:45:27.467 INFO:tasks.workunit.client.1.smithi152.stdout:7/763: truncate d7/d8/d12/f20 2198887 0 2024-03-29T04:45:27.468 INFO:tasks.workunit.client.1.smithi152.stdout:9/869: read d5/db/d1b/f8b [648152,19845] 0 2024-03-29T04:45:27.470 INFO:tasks.workunit.client.1.smithi152.stdout:3/840: creat d7/d19/d1b/d1e/da1/d41/f10f x:0 0 0 2024-03-29T04:45:27.477 INFO:tasks.workunit.client.1.smithi152.stdout:7/764: rename d7/d8/db3/l54 to d7/d8/dcd/le9 0 2024-03-29T04:45:27.477 INFO:tasks.workunit.client.1.smithi152.stdout:6/968: write d1/d3/d1a/d1b/d29/dce/fdd [1183355,83986] 0 2024-03-29T04:45:27.477 INFO:tasks.workunit.client.1.smithi152.stdout:2/769: write d0/d3/df/d2c/f69 [1222099,38558] 0 2024-03-29T04:45:27.481 INFO:tasks.workunit.client.1.smithi152.stdout:0/784: truncate d2/da/d17/d57/fcc 2817192 0 2024-03-29T04:45:27.482 INFO:tasks.workunit.client.1.smithi152.stdout:0/785: fsync d2/da/d17/f6d 0 2024-03-29T04:45:27.484 INFO:tasks.workunit.client.1.smithi152.stdout:4/724: truncate d0/d27/f59 2301631 0 2024-03-29T04:45:27.487 INFO:tasks.workunit.client.1.smithi152.stdout:0/786: dread d2/da/df/f5b [0,4194304] 0 2024-03-29T04:45:27.488 INFO:tasks.workunit.client.1.smithi152.stdout:7/765: dread d7/d8/db3/f97 [0,4194304] 0 2024-03-29T04:45:27.489 INFO:tasks.workunit.client.1.smithi152.stdout:5/944: write d6/d1d/d30/d6a/d108/f89 [581582,86756] 0 2024-03-29T04:45:27.496 INFO:tasks.workunit.client.1.smithi152.stdout:0/787: dwrite d2/d31/f36 [4194304,4194304] 0 2024-03-29T04:45:27.499 INFO:tasks.workunit.client.1.smithi152.stdout:4/725: dread d0/d7/d25/d17/d5e/d60/d74/d57/f47 [0,4194304] 0 2024-03-29T04:45:27.500 INFO:tasks.workunit.client.1.smithi152.stdout:8/843: dwrite d1/d2/d1d/dc6/fc9 [0,4194304] 0 2024-03-29T04:45:27.504 INFO:tasks.workunit.client.1.smithi152.stdout:8/844: dread d1/d8a/d91/f104 [0,4194304] 0 2024-03-29T04:45:27.509 INFO:tasks.workunit.client.1.smithi152.stdout:4/726: dread d0/d7/d40/d23/d55/d6c/f9e [0,4194304] 0 2024-03-29T04:45:27.527 INFO:tasks.workunit.client.1.smithi152.stdout:3/841: fsync d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/f7a 0 2024-03-29T04:45:27.532 INFO:tasks.workunit.client.1.smithi152.stdout:5/945: write d6/d1d/d30/d6a/fa7 [4092835,122153] 0 2024-03-29T04:45:27.539 INFO:tasks.workunit.client.1.smithi152.stdout:5/946: dwrite d6/d16/d25/d2e/d9d/d10c/f14a [0,4194304] 0 2024-03-29T04:45:27.542 INFO:tasks.workunit.client.1.smithi152.stdout:9/870: creat d5/f132 x:0 0 0 2024-03-29T04:45:27.547 INFO:tasks.workunit.client.1.smithi152.stdout:2/770: symlink d0/d3/df/d9e/dce/lea 0 2024-03-29T04:45:27.552 INFO:tasks.workunit.client.1.smithi152.stdout:7/766: link d7/d8/db3/f76 d7/d8/d93/fea 0 2024-03-29T04:45:27.553 INFO:tasks.workunit.client.1.smithi152.stdout:0/788: symlink d2/da/d17/dd8/db8/daf/l114 0 2024-03-29T04:45:27.555 INFO:tasks.workunit.client.1.smithi152.stdout:0/789: write d2/da/df/dbb/f10c [121179,62382] 0 2024-03-29T04:45:27.560 INFO:tasks.workunit.client.1.smithi152.stdout:9/871: symlink d5/db/d1b/d68/l133 0 2024-03-29T04:45:27.562 INFO:tasks.workunit.client.1.smithi152.stdout:2/771: rename d0/d3/d64/d85/le6 to d0/d3/d30/d15/d53/db4/dc6/leb 0 2024-03-29T04:45:27.565 INFO:tasks.workunit.client.1.smithi152.stdout:8/845: link d1/d2/d3/c27 d1/d2/d1d/dd3/db0/d40/d47/c134 0 2024-03-29T04:45:27.566 INFO:tasks.workunit.client.1.smithi152.stdout:8/846: write d1/d2/d1d/f83 [1178079,59256] 0 2024-03-29T04:45:27.567 INFO:tasks.workunit.client.1.smithi152.stdout:8/847: truncate d1/d2/d3/f10c 1632231 0 2024-03-29T04:45:27.568 INFO:tasks.workunit.client.1.smithi152.stdout:7/767: rename d7/d8/db3/f3e to d7/d8/d93/d99/feb 0 2024-03-29T04:45:27.569 INFO:tasks.workunit.client.1.smithi152.stdout:8/848: chown d1/d2/d1d/d36/d85/fb2 115515167 1 2024-03-29T04:45:27.569 INFO:tasks.workunit.client.1.smithi152.stdout:3/842: link d7/d19/d1b/d1e/da1/d41/d6a/dcc/c10a d7/d19/db5/df8/c110 0 2024-03-29T04:45:27.571 INFO:tasks.workunit.client.1.smithi152.stdout:3/843: chown d7/d19/d1b/d1e/da1/cea 145895 1 2024-03-29T04:45:27.571 INFO:tasks.workunit.client.1.smithi152.stdout:0/790: truncate d2/da/d17/d50/fb7 1723813 0 2024-03-29T04:45:27.573 INFO:tasks.workunit.client.1.smithi152.stdout:3/844: readlink d7/d19/d1b/d1e/da1/d41/d61/d85/lae 0 2024-03-29T04:45:27.577 INFO:tasks.workunit.client.1.smithi152.stdout:4/727: link d0/d7/d25/d17/d5e/cc7 d0/d7/d40/d23/d55/d6c/ce6 0 2024-03-29T04:45:27.579 INFO:tasks.workunit.client.1.smithi152.stdout:0/791: rename d2/f4 to d2/da/d17/d57/def/f115 0 2024-03-29T04:45:27.581 INFO:tasks.workunit.client.1.smithi152.stdout:9/872: dread d5/d12/db7/f130 [0,4194304] 0 2024-03-29T04:45:27.581 INFO:tasks.workunit.client.1.smithi152.stdout:7/768: mknod d7/d8/d12/d1d/cec 0 2024-03-29T04:45:27.585 INFO:tasks.workunit.client.1.smithi152.stdout:9/873: write d5/d8/d31/d59/d85/f124 [599415,12723] 0 2024-03-29T04:45:27.589 INFO:tasks.workunit.client.1.smithi152.stdout:4/728: dwrite d0/d7/d25/d17/d37/d77/fd2 [0,4194304] 0 2024-03-29T04:45:27.590 INFO:tasks.workunit.client.1.smithi152.stdout:2/772: dread d0/d3/d30/d15/d22/d74/dbd/f95 [0,4194304] 0 2024-03-29T04:45:27.592 INFO:tasks.workunit.client.1.smithi152.stdout:4/729: truncate d0/d7/fab 1357219 0 2024-03-29T04:45:27.593 INFO:tasks.workunit.client.1.smithi152.stdout:9/874: symlink d5/d8/l134 0 2024-03-29T04:45:27.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.595+0000 7f24bf8b8700 1 -- 172.21.15.142:0/4078196526 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24b80611d0 con 0x7f24a406c4e0 2024-03-29T04:45:27.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.597+0000 7f24ae7fc700 1 -- 172.21.15.142:0/4078196526 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f24b80611d0 con 0x7f24a406c4e0 2024-03-29T04:45:27.596 INFO:tasks.workunit.client.1.smithi152.stdout:4/730: stat d0/d7/d25/d17/l4e 0 2024-03-29T04:45:27.596 INFO:tasks.workunit.client.1.smithi152.stdout:3/845: link d7/cac d7/d19/d1b/d1e/da1/d41/d61/c111 0 2024-03-29T04:45:27.597 INFO:tasks.workunit.client.1.smithi152.stdout:0/792: dread d2/da/d3a/d61/f9a [0,4194304] 0 2024-03-29T04:45:27.598 INFO:tasks.workunit.client.1.smithi152.stdout:7/769: link d7/d8/d12/de2/fe7 d7/d8/d12/d1d/fed 0 2024-03-29T04:45:27.598 INFO:tasks.workunit.client.1.smithi152.stdout:8/849: dread d1/d2/d3/f29 [0,4194304] 0 2024-03-29T04:45:27.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.600+0000 7f24bf8b8700 1 -- 172.21.15.142:0/4078196526 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f24a406c4e0 msgr2=0x7f24a406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:27.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.600+0000 7f24bf8b8700 1 --2- 172.21.15.142:0/4078196526 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f24a406c4e0 0x7f24a406e9a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24a800b5c0 tx=0x7f24a8005e20 comp rx=0 tx=0).stop 2024-03-29T04:45:27.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.600+0000 7f24bf8b8700 1 -- 172.21.15.142:0/4078196526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b80ff440 msgr2=0x7f24b819c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:27.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.600+0000 7f24bf8b8700 1 --2- 172.21.15.142:0/4078196526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b80ff440 0x7f24b819c9f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f24b400b4b0 tx=0x7f24b400b870 comp rx=0 tx=0).stop 2024-03-29T04:45:27.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.600+0000 7f24bf8b8700 1 -- 172.21.15.142:0/4078196526 shutdown_connections 2024-03-29T04:45:27.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.600+0000 7f24bf8b8700 1 --2- 172.21.15.142:0/4078196526 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f24a406c4e0 0x7f24a406e9a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:27.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.600+0000 7f24bf8b8700 1 --2- 172.21.15.142:0/4078196526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b80ff440 0x7f24b819c9f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:27.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.600+0000 7f24bf8b8700 1 --2- 172.21.15.142:0/4078196526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24b8101db0 0x7f24b819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:27.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.600+0000 7f24bf8b8700 1 -- 172.21.15.142:0/4078196526 >> 172.21.15.142:0/4078196526 conn(0x7f24b80fb060 msgr2=0x7f24b80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:27.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.600+0000 7f24bf8b8700 1 -- 172.21.15.142:0/4078196526 shutdown_connections 2024-03-29T04:45:27.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.600+0000 7f24bf8b8700 1 -- 172.21.15.142:0/4078196526 wait complete. 2024-03-29T04:45:27.602 INFO:tasks.workunit.client.1.smithi152.stdout:0/793: dread d2/da/df/dbb/f10c [0,4194304] 0 2024-03-29T04:45:27.631 INFO:tasks.workunit.client.1.smithi152.stdout:9/875: mknod d5/d12/c135 0 2024-03-29T04:45:27.633 INFO:tasks.workunit.client.1.smithi152.stdout:6/969: dwrite f0 [0,4194304] 0 2024-03-29T04:45:27.633 INFO:tasks.workunit.client.1.smithi152.stdout:7/770: mkdir d7/d8/db3/d72/dee 0 2024-03-29T04:45:27.634 INFO:tasks.workunit.client.1.smithi152.stdout:0/794: chown d2/da/d17/fec 344900859 1 2024-03-29T04:45:27.635 INFO:tasks.workunit.client.1.smithi152.stdout:5/947: write d6/d1b/fcb [509746,73593] 0 2024-03-29T04:45:27.638 INFO:tasks.workunit.client.1.smithi152.stdout:7/771: readlink d7/d8/d12/d1d/l26 0 2024-03-29T04:45:27.639 INFO:tasks.workunit.client.1.smithi152.stdout:2/773: link d0/d3/d30/d15/d53/d54/dad/dd5/fb2 d0/d3/d30/d15/d53/db4/fec 0 2024-03-29T04:45:27.639 INFO:tasks.workunit.client.1.smithi152.stdout:9/876: mkdir d5/db/d92/d136 0 2024-03-29T04:45:27.641 INFO:tasks.workunit.client.1.smithi152.stdout:3/846: write d7/d19/d1b/d1e/da1/d41/f107 [521211,110564] 0 2024-03-29T04:45:27.642 INFO:tasks.workunit.client.1.smithi152.stdout:8/850: write d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/fa8 [4881791,50787] 0 2024-03-29T04:45:27.644 INFO:tasks.workunit.client.1.smithi152.stdout:3/847: chown d7/fa4 2414652 1 2024-03-29T04:45:27.645 INFO:tasks.workunit.client.1.smithi152.stdout:7/772: unlink d7/la0 0 2024-03-29T04:45:27.650 INFO:tasks.workunit.client.1.smithi152.stdout:3/848: readlink d7/d19/d1b/d1e/da1/d41/d61/d63/l10e 0 2024-03-29T04:45:27.652 INFO:tasks.workunit.client.1.smithi152.stdout:9/877: rename d5/d8/d31/d75 to d5/db/d92/dd2/d137 0 2024-03-29T04:45:27.654 INFO:tasks.workunit.client.1.smithi152.stdout:2/774: dread d0/d3/d30/d15/d22/d4c/d5a/f6b [0,4194304] 0 2024-03-29T04:45:27.657 INFO:tasks.workunit.client.1.smithi152.stdout:5/948: creat d6/d16/d25/d2e/da0/d92/dc0/f14c x:0 0 0 2024-03-29T04:45:27.659 INFO:tasks.workunit.client.1.smithi152.stdout:6/970: rename d1/d3/d9/d13/dc0/dc3/dfb/d112/f132 to d1/d3/df/d84/da1/f147 0 2024-03-29T04:45:27.659 INFO:tasks.workunit.client.1.smithi152.stdout:8/851: dwrite d1/d2/d1d/dd3/db0/f90 [4194304,4194304] 0 2024-03-29T04:45:27.662 INFO:tasks.workunit.client.1.smithi152.stdout:3/849: mknod d7/d19/d1b/d1e/da1/d41/d61/d85/c112 0 2024-03-29T04:45:27.662 INFO:tasks.workunit.client.1.smithi152.stdout:6/971: write d1/d3/d1a/d1b/d29/dce/fdd [1537242,44881] 0 2024-03-29T04:45:27.666 INFO:tasks.workunit.client.1.smithi152.stdout:7/773: link d7/d8/d12/f91 d7/d8/db/df/de5/fef 0 2024-03-29T04:45:27.670 INFO:tasks.workunit.client.1.smithi152.stdout:8/852: mknod d1/d2/d1d/dd3/db0/d40/d6a/c135 0 2024-03-29T04:45:27.672 INFO:tasks.workunit.client.1.smithi152.stdout:0/795: dread d2/da/d3a/d6f/f85 [0,4194304] 0 2024-03-29T04:45:27.672 INFO:tasks.workunit.client.1.smithi152.stdout:6/972: rename d1/d3/d9/l40 to d1/d3/d9/d13/dc0/dea/l148 0 2024-03-29T04:45:27.673 INFO:tasks.workunit.client.1.smithi152.stdout:5/949: dwrite d6/d7/fa4 [0,4194304] 0 2024-03-29T04:45:27.683 INFO:tasks.workunit.client.1.smithi152.stdout:5/950: fdatasync d6/d16/d25/d2e/da0/d92/dc0/dec/f120 0 2024-03-29T04:45:27.686 INFO:tasks.workunit.client.1.smithi152.stdout:7/774: dwrite d7/d8/d93/d99/fbc [0,4194304] 0 2024-03-29T04:45:27.689 INFO:tasks.workunit.client.1.smithi152.stdout:0/796: chown d2/da/d17/d50/c75 39608 1 2024-03-29T04:45:27.690 INFO:tasks.workunit.client.1.smithi152.stdout:2/775: dread d0/d3/d64/fe2 [4194304,4194304] 0 2024-03-29T04:45:27.690 INFO:tasks.workunit.client.1.smithi152.stdout:9/878: dwrite d5/db/d1b/f8b [0,4194304] 0 2024-03-29T04:45:27.691 INFO:tasks.workunit.client.1.smithi152.stdout:8/853: write d1/d2/d1d/d36/d5d/fb5 [547452,39325] 0 2024-03-29T04:45:27.692 INFO:tasks.workunit.client.1.smithi152.stdout:6/973: write d1/d3/d1a/f1d [4141945,113714] 0 2024-03-29T04:45:27.694 INFO:tasks.workunit.client.1.smithi152.stdout:0/797: creat d2/d29/f116 x:0 0 0 2024-03-29T04:45:27.695 INFO:tasks.workunit.client.1.smithi152.stdout:7/775: chown d7/d8/d93/d99/feb 1347603 1 2024-03-29T04:45:27.701 INFO:tasks.workunit.client.1.smithi152.stdout:7/776: chown d7/d8/db3/f8d 152816111 1 2024-03-29T04:45:27.702 INFO:tasks.workunit.client.1.smithi152.stdout:3/850: dread d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/f49 [0,4194304] 0 2024-03-29T04:45:27.702 INFO:tasks.workunit.client.1.smithi152.stdout:5/951: rename d6/d16/f5b to d6/d1d/daa/db7/d56/dc1/f14d 0 2024-03-29T04:45:27.704 INFO:tasks.workunit.client.1.smithi152.stdout:3/851: chown d7/d19/f29 0 1 2024-03-29T04:45:27.704 INFO:tasks.workunit.client.1.smithi152.stdout:6/974: creat d1/d3/d1a/d1b/d29/dce/f149 x:0 0 0 2024-03-29T04:45:27.705 INFO:tasks.workunit.client.1.smithi152.stdout:0/798: creat d2/da/d17/d57/d9c/df9/f117 x:0 0 0 2024-03-29T04:45:27.706 INFO:tasks.workunit.client.1.smithi152.stdout:3/852: write d7/fa3 [536590,86951] 0 2024-03-29T04:45:27.707 INFO:tasks.workunit.client.1.smithi152.stdout:8/854: mkdir d1/d2/d1d/dd3/db0/d40/d58/d136 0 2024-03-29T04:45:27.709 INFO:tasks.workunit.client.1.smithi152.stdout:6/975: creat d1/d3/df/d84/da1/f14a x:0 0 0 2024-03-29T04:45:27.710 INFO:tasks.workunit.client.1.smithi152.stdout:7/777: dwrite d7/d8/d12/d35/dda/fd6 [0,4194304] 0 2024-03-29T04:45:27.713 INFO:tasks.workunit.client.1.smithi152.stdout:5/952: rename d6/d1d/d3b to d6/d16/d25/d2e/da0/d92/d14e 0 2024-03-29T04:45:27.714 INFO:tasks.workunit.client.1.smithi152.stdout:8/855: truncate d1/d2/d1d/dd3/db0/d40/ddd/f120 711541 0 2024-03-29T04:45:27.714 INFO:tasks.workunit.client.1.smithi152.stdout:7/778: stat d7/d8/d12/d35/d68/dc3/fe4 0 2024-03-29T04:45:27.720 INFO:tasks.workunit.client.1.smithi152.stdout:5/953: mknod d6/d16/d25/d2e/dc7/c14f 0 2024-03-29T04:45:27.723 INFO:tasks.workunit.client.1.smithi152.stdout:3/853: rename d7/lc3 to d7/d19/d1b/d1e/da1/d41/d6a/l113 0 2024-03-29T04:45:27.729 INFO:tasks.workunit.client.1.smithi152.stdout:8/856: dwrite d1/d8a/f8c [0,4194304] 0 2024-03-29T04:45:27.731 INFO:tasks.workunit.client.1.smithi152.stdout:2/776: dwrite d0/d3/df/f76 [0,4194304] 0 2024-03-29T04:45:27.738 INFO:tasks.workunit.client.1.smithi152.stdout:7/779: rename d7/d8/d93 to d7/d8/d12/d35/dba/df0 0 2024-03-29T04:45:27.738 INFO:tasks.workunit.client.1.smithi152.stdout:6/976: dread d1/d3/df/f2e [0,4194304] 0 2024-03-29T04:45:27.743 INFO:tasks.workunit.client.1.smithi152.stdout:3/854: mkdir d7/d19/d1b/d1e/da1/d41/d61/d99/d114 0 2024-03-29T04:45:27.744 INFO:tasks.workunit.client.1.smithi152.stdout:9/879: dwrite d5/db/d1b/f43 [0,4194304] 0 2024-03-29T04:45:27.745 INFO:tasks.workunit.client.1.smithi152.stdout:6/977: read d1/d3/d1a/f20 [1663559,31797] 0 2024-03-29T04:45:27.745 INFO:tasks.workunit.client.1.smithi152.stdout:7/780: creat d7/d8/dcd/ff1 x:0 0 0 2024-03-29T04:45:27.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.749+0000 7f009b65f700 1 -- 172.21.15.142:0/2079341445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0094074d60 msgr2=0x7f00940731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:27.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.749+0000 7f009b65f700 1 --2- 172.21.15.142:0/2079341445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0094074d60 0x7f00940731c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0090009910 tx=0x7f0090009c20 comp rx=0 tx=0).stop 2024-03-29T04:45:27.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.750+0000 7f009b65f700 1 -- 172.21.15.142:0/2079341445 shutdown_connections 2024-03-29T04:45:27.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.750+0000 7f009b65f700 1 --2- 172.21.15.142:0/2079341445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0094073790 0x7f0094073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:27.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.750+0000 7f009b65f700 1 --2- 172.21.15.142:0/2079341445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0094074d60 0x7f00940731c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:27.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.750+0000 7f009b65f700 1 -- 172.21.15.142:0/2079341445 >> 172.21.15.142:0/2079341445 conn(0x7f00940fc110 msgr2=0x7f00940fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:27.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.750+0000 7f009b65f700 1 -- 172.21.15.142:0/2079341445 shutdown_connections 2024-03-29T04:45:27.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.750+0000 7f009b65f700 1 -- 172.21.15.142:0/2079341445 wait complete. 2024-03-29T04:45:27.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.751+0000 7f009b65f700 1 Processor -- start 2024-03-29T04:45:27.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.751+0000 7f009b65f700 1 -- start start 2024-03-29T04:45:27.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.751+0000 7f009b65f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0094073790 0x7f0094198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:27.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.751+0000 7f009b65f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0094074d60 0x7f0094198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:27.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.751+0000 7f009b65f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00941990d0 con 0x7f0094074d60 2024-03-29T04:45:27.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.751+0000 7f009b65f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0094199210 con 0x7f0094073790 2024-03-29T04:45:27.752 INFO:tasks.workunit.client.1.smithi152.stdout:0/799: dwrite d2/da/df/f4b [4194304,4194304] 0 2024-03-29T04:45:27.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.752+0000 7f0098bfa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0094074d60 0x7f0094198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:27.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.752+0000 7f0098bfa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0094074d60 0x7f0094198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45340/0 (socket says 172.21.15.142:45340) 2024-03-29T04:45:27.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.753+0000 7f0098bfa700 1 -- 172.21.15.142:0/2032932032 learned_addr learned my addr 172.21.15.142:0/2032932032 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:45:27.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.753+0000 7f00993fb700 1 --2- 172.21.15.142:0/2032932032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0094073790 0x7f0094198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:27.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.753+0000 7f0098bfa700 1 -- 172.21.15.142:0/2032932032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0094073790 msgr2=0x7f0094198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:27.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.753+0000 7f0098bfa700 1 --2- 172.21.15.142:0/2032932032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0094073790 0x7f0094198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:27.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.753+0000 7f0098bfa700 1 -- 172.21.15.142:0/2032932032 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00900095f0 con 0x7f0094074d60 2024-03-29T04:45:27.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.754+0000 7f0098bfa700 1 --2- 172.21.15.142:0/2032932032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0094074d60 0x7f0094198ab0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f008400e920 tx=0x7f008400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:27.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.754+0000 7f008a7fc700 1 -- 172.21.15.142:0/2032932032 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0084009790 con 0x7f0094074d60 2024-03-29T04:45:27.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.754+0000 7f008a7fc700 1 -- 172.21.15.142:0/2032932032 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f008400cba0 con 0x7f0094074d60 2024-03-29T04:45:27.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.754+0000 7f009b65f700 1 -- 172.21.15.142:0/2032932032 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009406a950 con 0x7f0094074d60 2024-03-29T04:45:27.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.754+0000 7f008a7fc700 1 -- 172.21.15.142:0/2032932032 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0084018630 con 0x7f0094074d60 2024-03-29T04:45:27.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.754+0000 7f009b65f700 1 -- 172.21.15.142:0/2032932032 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009406aea0 con 0x7f0094074d60 2024-03-29T04:45:27.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.756+0000 7f009b65f700 1 -- 172.21.15.142:0/2032932032 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009410b900 con 0x7f0094074d60 2024-03-29T04:45:27.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.756+0000 7f008a7fc700 1 -- 172.21.15.142:0/2032932032 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f008400cd10 con 0x7f0094074d60 2024-03-29T04:45:27.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.757+0000 7f008a7fc700 1 --2- 172.21.15.142:0/2032932032 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f008006c4e0 0x7f008006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:27.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.757+0000 7f008a7fc700 1 -- 172.21.15.142:0/2032932032 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0084014070 con 0x7f0094074d60 2024-03-29T04:45:27.757 INFO:tasks.workunit.client.1.smithi152.stdout:5/954: dwrite d6/d16/d25/d2e/d9d/d10c/f132 [8388608,4194304] 0 2024-03-29T04:45:27.757 INFO:tasks.workunit.client.1.smithi152.stdout:3/855: creat d7/d19/d1b/d1e/df0/f115 x:0 0 0 2024-03-29T04:45:27.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.757+0000 7f00993fb700 1 --2- 172.21.15.142:0/2032932032 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f008006c4e0 0x7f008006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:27.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.758+0000 7f00993fb700 1 --2- 172.21.15.142:0/2032932032 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f008006c4e0 0x7f008006e9a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f00900059d0 tx=0x7f0090005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:27.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:27.763+0000 7f008a7fc700 1 -- 172.21.15.142:0/2032932032 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f00840520e0 con 0x7f0094074d60 2024-03-29T04:45:27.766 INFO:tasks.workunit.client.1.smithi152.stdout:6/978: truncate d1/d3/d1a/d1b/f1e 111309 0 2024-03-29T04:45:27.766 INFO:tasks.workunit.client.1.smithi152.stdout:0/800: creat d2/da/d17/d57/f118 x:0 0 0 2024-03-29T04:45:27.767 INFO:tasks.workunit.client.1.smithi152.stdout:5/955: fdatasync d6/d1d/d30/fa2 0 2024-03-29T04:45:27.767 INFO:tasks.workunit.client.1.smithi152.stdout:3/856: symlink d7/d19/d1b/d1e/da1/d41/d61/d99/l116 0 2024-03-29T04:45:27.768 INFO:tasks.workunit.client.1.smithi152.stdout:8/857: dwrite d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/fb9 [0,4194304] 0 2024-03-29T04:45:27.768 INFO:tasks.workunit.client.1.smithi152.stdout:3/857: chown d7/d19 1334 1 2024-03-29T04:45:27.771 INFO:tasks.workunit.client.1.smithi152.stdout:7/781: dread d7/d8/d12/d35/dda/f9b [0,4194304] 0 2024-03-29T04:45:27.772 INFO:tasks.workunit.client.1.smithi152.stdout:8/858: write d1/dc5/fec [415768,114930] 0 2024-03-29T04:45:27.773 INFO:tasks.workunit.client.1.smithi152.stdout:3/858: rmdir d7/d19/d1b/d1e/da1/d41/d61/d85 39 2024-03-29T04:45:27.773 INFO:tasks.workunit.client.1.smithi152.stdout:7/782: dread - d7/d8/db/df/fb8 zero size 2024-03-29T04:45:27.775 INFO:tasks.workunit.client.1.smithi152.stdout:2/777: dread d0/d3/d64/fe2 [0,4194304] 0 2024-03-29T04:45:27.776 INFO:tasks.workunit.client.1.smithi152.stdout:8/859: write d1/d8a/f8c [773585,2718] 0 2024-03-29T04:45:27.778 INFO:tasks.workunit.client.1.smithi152.stdout:7/783: symlink d7/d8/d12/d35/dda/lf2 0 2024-03-29T04:45:27.780 INFO:tasks.workunit.client.1.smithi152.stdout:2/778: write d0/d3/d30/d15/d22/d4c/d5a/fc5 [276277,42157] 0 2024-03-29T04:45:27.785 INFO:tasks.workunit.client.1.smithi152.stdout:8/860: truncate d1/d2/d1d/dd3/fbb 259670 0 2024-03-29T04:45:27.785 INFO:tasks.workunit.client.1.smithi152.stdout:3/859: rename d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/l5b to d7/d19/d1b/d1e/da1/d41/d61/l117 0 2024-03-29T04:45:27.785 INFO:tasks.workunit.client.1.smithi152.stdout:0/801: dwrite d2/da/d17/d50/dc6/fe4 [0,4194304] 0 2024-03-29T04:45:27.785 INFO:tasks.workunit.client.1.smithi152.stdout:7/784: read - d7/d8/db/df/de5/fef zero size 2024-03-29T04:45:27.789 INFO:tasks.workunit.client.1.smithi152.stdout:8/861: write d1/dc5/fec [2644065,126474] 0 2024-03-29T04:45:27.789 INFO:tasks.workunit.client.1.smithi152.stdout:6/979: write d1/d3/d91/f13a [29676,114471] 0 2024-03-29T04:45:27.790 INFO:tasks.workunit.client.1.smithi152.stdout:8/862: chown d1/d2/l38 13 1 2024-03-29T04:45:27.791 INFO:tasks.workunit.client.1.smithi152.stdout:2/779: symlink d0/d3/d30/d15/d22/d74/dbd/led 0 2024-03-29T04:45:27.793 INFO:tasks.workunit.client.1.smithi152.stdout:2/780: dread - d0/d3/d30/d15/da5/fe9 zero size 2024-03-29T04:45:27.795 INFO:tasks.workunit.client.1.smithi152.stdout:9/880: dread d5/db/d1b/f95 [0,4194304] 0 2024-03-29T04:45:27.796 INFO:tasks.workunit.client.1.smithi152.stdout:3/860: symlink d7/d19/db5/df8/l118 0 2024-03-29T04:45:27.797 INFO:tasks.workunit.client.1.smithi152.stdout:7/785: dread d7/d8/d12/f87 [0,4194304] 0 2024-03-29T04:45:27.801 INFO:tasks.workunit.client.1.smithi152.stdout:0/802: write d2/da/d3a/fd2 [493841,52657] 0 2024-03-29T04:45:27.802 INFO:tasks.workunit.client.1.smithi152.stdout:9/881: stat d5/d8/d31/ccc 0 2024-03-29T04:45:27.802 INFO:tasks.workunit.client.1.smithi152.stdout:0/803: fdatasync d2/da/d17/d57/f118 0 2024-03-29T04:45:27.803 INFO:tasks.workunit.client.1.smithi152.stdout:8/863: unlink d1/d2/d1d/d36/d85/lf5 0 2024-03-29T04:45:27.824 INFO:tasks.workunit.client.1.smithi152.stdout:3/861: symlink d7/d19/d1b/d1e/df0/l119 0 2024-03-29T04:45:27.825 INFO:tasks.workunit.client.1.smithi152.stdout:8/864: mknod d1/d2/d1d/d36/d5d/c137 0 2024-03-29T04:45:27.827 INFO:tasks.workunit.client.1.smithi152.stdout:7/786: link d7/d8/d12/d35/dba/df0/d99/fbc d7/d8/d12/d35/d68/d84/ff3 0 2024-03-29T04:45:27.828 INFO:tasks.workunit.client.1.smithi152.stdout:8/865: truncate d1/d2/d3c/d76/fe9 762529 0 2024-03-29T04:45:27.829 INFO:tasks.workunit.client.1.smithi152.stdout:9/882: creat d5/d8/d31/d59/dc3/d125/f138 x:0 0 0 2024-03-29T04:45:27.829 INFO:tasks.workunit.client.1.smithi152.stdout:6/980: getdents d1/d3/d9/d13/dc0/dea/def 0 2024-03-29T04:45:27.832 INFO:tasks.workunit.client.1.smithi152.stdout:3/862: chown d7/c9 85 1 2024-03-29T04:45:27.834 INFO:tasks.workunit.client.1.smithi152.stdout:0/804: dwrite d2/da/d17/fec [0,4194304] 0 2024-03-29T04:45:27.834 INFO:tasks.workunit.client.1.smithi152.stdout:6/981: fdatasync d1/d3/d9/f11 0 2024-03-29T04:45:27.834 INFO:tasks.workunit.client.1.smithi152.stdout:9/883: write d5/d8/d31/d59/d85/f10d [95983,14090] 0 2024-03-29T04:45:27.835 INFO:tasks.workunit.client.1.smithi152.stdout:7/787: rename d7/d8/d12/d35/dba to d7/d8/d12/d35/dba/df0/d99/dc8/df4 22 2024-03-29T04:45:27.835 INFO:tasks.workunit.client.1.smithi152.stdout:8/866: creat d1/d2/d1d/dd3/db0/d40/d47/f138 x:0 0 0 2024-03-29T04:45:27.839 INFO:tasks.workunit.client.1.smithi152.stdout:3/863: chown d7/d19/d1b/d1e/da1/d41/d61/d63/f67 85361 1 2024-03-29T04:45:27.839 INFO:tasks.workunit.client.1.smithi152.stdout:6/982: chown d1/d3/d9/d13/d28/d45/c74 146827 1 2024-03-29T04:45:27.840 INFO:tasks.workunit.client.1.smithi152.stdout:9/884: creat d5/d8/d31/d59/d6f/f139 x:0 0 0 2024-03-29T04:45:27.840 INFO:tasks.workunit.client.1.smithi152.stdout:0/805: mknod d2/da/d17/d50/dac/c119 0 2024-03-29T04:45:27.842 INFO:tasks.workunit.client.1.smithi152.stdout:6/983: chown d1/d3/d9/d99/de5 509643137 1 2024-03-29T04:45:27.843 INFO:tasks.workunit.client.1.smithi152.stdout:9/885: write d5/d8/f1a [796277,105584] 0 2024-03-29T04:45:27.844 INFO:tasks.workunit.client.1.smithi152.stdout:6/984: chown d1/d3/d9/d13/dc0 173480472 1 2024-03-29T04:45:27.844 INFO:tasks.workunit.client.1.smithi152.stdout:8/867: rename d1/d2/d1d/dd3/db0/d40/d6a/c9d to d1/d2/d3c/d76/d9a/c139 0 2024-03-29T04:45:27.845 INFO:tasks.workunit.client.1.smithi152.stdout:7/788: getdents d7/d8/d12/de2 0 2024-03-29T04:45:27.847 INFO:tasks.workunit.client.1.smithi152.stdout:8/868: read d1/d2/d1d/d36/d85/fb4 [2315218,35166] 0 2024-03-29T04:45:27.859 INFO:tasks.workunit.client.1.smithi152.stdout:8/869: write d1/d8a/d91/dee/f63 [831442,24806] 0 2024-03-29T04:45:27.860 INFO:tasks.workunit.client.1.smithi152.stdout:9/886: link d5/d8/d31/d59/d6f/fce d5/db/d92/dd2/d137/db1/dcd/d6e/db4/d110/f13a 0 2024-03-29T04:45:27.860 INFO:tasks.workunit.client.1.smithi152.stdout:8/870: chown d1/d2/d1d/d36/d4b/d51/lb1 804510 1 2024-03-29T04:45:27.861 INFO:tasks.workunit.client.1.smithi152.stdout:9/887: chown d5/d8/l14 31 1 2024-03-29T04:45:27.861 INFO:tasks.workunit.client.1.smithi152.stdout:7/789: write d7/d8/d12/f6f [723088,85216] 0 2024-03-29T04:45:27.862 INFO:tasks.workunit.client.1.smithi152.stdout:8/871: readlink d1/d2/d1d/d36/d4b/d51/ddc/l125 0 2024-03-29T04:45:27.863 INFO:tasks.workunit.client.1.smithi152.stdout:6/985: rmdir d1/d3/d9/d13/dc0/dc3/dfb/d112 0 2024-03-29T04:45:27.863 INFO:tasks.workunit.client.1.smithi152.stdout:8/872: write d1/d2/d3c/fac [539297,92167] 0 2024-03-29T04:45:27.864 INFO:tasks.workunit.client.1.smithi152.stdout:0/806: getdents d2/da/df/dbb 0 2024-03-29T04:45:27.865 INFO:tasks.workunit.client.1.smithi152.stdout:9/888: symlink d5/d8/d31/d59/dc3/l13b 0 2024-03-29T04:45:27.867 INFO:tasks.workunit.client.1.smithi152.stdout:6/986: fdatasync d1/fd0 0 2024-03-29T04:45:27.868 INFO:tasks.workunit.client.1.smithi152.stdout:6/987: fdatasync d1/d3/d9/d13/f31 0 2024-03-29T04:45:27.868 INFO:tasks.workunit.client.1.smithi152.stdout:0/807: rename d2/da/d17/db5 to d2/da/d3a/d61/db4/d11a 0 2024-03-29T04:45:27.869 INFO:tasks.workunit.client.1.smithi152.stdout:9/889: rmdir d5/db/d92 39 2024-03-29T04:45:27.877 INFO:tasks.workunit.client.1.smithi152.stdout:7/790: dwrite d7/d8/f9c [0,4194304] 0 2024-03-29T04:45:27.877 INFO:tasks.workunit.client.1.smithi152.stdout:6/988: write d1/d3/d1a/fbd [2676592,81741] 0 2024-03-29T04:45:27.883 INFO:tasks.workunit.client.1.smithi152.stdout:6/989: fdatasync d1/d3/d1a/d1b/d29/fcc 0 2024-03-29T04:45:27.883 INFO:tasks.workunit.client.1.smithi152.stdout:8/873: link d1/d2/d1d/dd3/db0/da9/d105/c10d d1/d2/d1d/dd3/db0/d7f/dd8/c13a 0 2024-03-29T04:45:27.884 INFO:tasks.workunit.client.1.smithi152.stdout:0/808: rename d2/da/df/f4a to d2/da/d17/dd8/f11b 0 2024-03-29T04:45:27.888 INFO:tasks.workunit.client.1.smithi152.stdout:6/990: chown d1/d3/df/d8a/d98/c146 856915240 1 2024-03-29T04:45:27.889 INFO:tasks.workunit.client.1.smithi152.stdout:8/874: rmdir d1/d8a 39 2024-03-29T04:45:27.889 INFO:tasks.workunit.client.1.smithi152.stdout:7/791: dwrite d7/d8/db/df/fd8 [0,4194304] 0 2024-03-29T04:45:27.892 INFO:tasks.workunit.client.1.smithi152.stdout:9/890: dwrite d5/f2f [0,4194304] 0 2024-03-29T04:45:27.892 INFO:tasks.workunit.client.1.smithi152.stdout:7/792: write d7/d8/d12/f43 [3609026,2021] 0 2024-03-29T04:45:27.892 INFO:tasks.workunit.client.1.smithi152.stdout:8/875: dread - d1/d2/d1d/d36/d4b/d51/ddc/f10a zero size 2024-03-29T04:45:27.893 INFO:tasks.workunit.client.1.smithi152.stdout:0/809: unlink d2/d29/c46 0 2024-03-29T04:45:27.896 INFO:tasks.workunit.client.1.smithi152.stdout:9/891: mkdir d5/db/d1b/d13c 0 2024-03-29T04:45:27.900 INFO:tasks.workunit.client.1.smithi152.stdout:6/991: creat d1/d3/d9/d13/d28/d45/f14b x:0 0 0 2024-03-29T04:45:27.902 INFO:tasks.workunit.client.1.smithi152.stdout:9/892: creat d5/d12/db7/f13d x:0 0 0 2024-03-29T04:45:27.904 INFO:tasks.workunit.client.1.smithi152.stdout:8/876: write d1/f7b [3099734,128025] 0 2024-03-29T04:45:27.905 INFO:tasks.workunit.client.1.smithi152.stdout:6/992: rename d1/d3/d1a/d1b/d29/d6f/d13b to d1/d3/d9/d99/d14c 0 2024-03-29T04:45:27.909 INFO:tasks.workunit.client.1.smithi152.stdout:7/793: dwrite d7/d8/d12/d35/f63 [0,4194304] 0 2024-03-29T04:45:27.909 INFO:tasks.workunit.client.1.smithi152.stdout:8/877: unlink d1/d2/d1d/dd3/db0/d8b/l131 0 2024-03-29T04:45:27.911 INFO:tasks.workunit.client.1.smithi152.stdout:9/893: truncate d5/db/d92/dd2/d137/db1/dcd/d6e/fdd 4864313 0 2024-03-29T04:45:27.912 INFO:tasks.workunit.client.1.smithi152.stdout:9/894: chown d5/db/d1b/d13c 0 1 2024-03-29T04:45:27.914 INFO:tasks.workunit.client.1.smithi152.stdout:7/794: unlink d7/d8/d12/d35/dda/l36 0 2024-03-29T04:45:27.916 INFO:tasks.workunit.client.1.smithi152.stdout:6/993: rename d1/d3/d9/d13/d28/d45/c92 to d1/d3/d9/d13/c14d 0 2024-03-29T04:45:27.917 INFO:tasks.workunit.client.1.smithi152.stdout:9/895: creat d5/db/d1b/d68/dca/f13e x:0 0 0 2024-03-29T04:45:27.921 INFO:tasks.workunit.client.1.smithi152.stdout:6/994: creat d1/d3/d9/d13/dc0/dc3/dfb/d142/f14e x:0 0 0 2024-03-29T04:45:27.924 INFO:tasks.workunit.client.1.smithi152.stdout:8/878: dread d1/d2/d1d/d36/d4b/d51/f5e [0,4194304] 0 2024-03-29T04:45:27.926 INFO:tasks.workunit.client.1.smithi152.stdout:8/879: read - d1/d2/d3c/fcb zero size 2024-03-29T04:45:27.926 INFO:tasks.workunit.client.1.smithi152.stdout:6/995: write d1/d3/d9/d13/dc0/dc3/dfb/d142/f14e [307758,49747] 0 2024-03-29T04:45:27.927 INFO:tasks.workunit.client.1.smithi152.stdout:9/896: dwrite d5/d8/f9 [0,4194304] 0 2024-03-29T04:45:27.929 INFO:tasks.workunit.client.1.smithi152.stdout:8/880: write d1/d2/d1d/d36/d4b/f11d [1131494,65117] 0 2024-03-29T04:45:27.932 INFO:tasks.workunit.client.1.smithi152.stdout:9/897: mkdir d5/db/d92/dd2/d137/db1/dcd/d6e/db4/d13f 0 2024-03-29T04:45:27.933 INFO:tasks.workunit.client.1.smithi152.stdout:8/881: mknod d1/d2/d1d/dd3/db0/d40/d6a/c13b 0 2024-03-29T04:45:27.934 INFO:tasks.workunit.client.1.smithi152.stdout:8/882: chown d1/d2/d1d/d36/d5d/lff 7 1 2024-03-29T04:45:27.942 INFO:tasks.workunit.client.1.smithi152.stdout:8/883: dwrite d1/d2/d1d/dd3/db0/f90 [4194304,4194304] 0 2024-03-29T04:45:27.943 INFO:tasks.workunit.client.1.smithi152.stdout:8/884: stat d1/d2/d3/ffb 0 2024-03-29T04:45:27.954 INFO:tasks.workunit.client.1.smithi152.stdout:8/885: truncate d1/dc5/fec 606889 0 2024-03-29T04:45:27.955 INFO:tasks.workunit.client.1.smithi152.stdout:6/996: dread d1/d3/d9/f11 [0,4194304] 0 2024-03-29T04:45:27.957 INFO:tasks.workunit.client.1.smithi152.stdout:8/886: readlink d1/d8a/d91/dee/l4c 0 2024-03-29T04:45:27.958 INFO:tasks.workunit.client.1.smithi152.stdout:6/997: mkdir d1/d3/d9/d13/d3e/d54/d14f 0 2024-03-29T04:45:27.959 INFO:tasks.workunit.client.1.smithi152.stdout:7/795: dread d7/d8/db3/f76 [0,4194304] 0 2024-03-29T04:45:27.961 INFO:tasks.workunit.client.1.smithi152.stdout:7/796: creat d7/ff5 x:0 0 0 2024-03-29T04:45:27.963 INFO:tasks.workunit.client.1.smithi152.stdout:8/887: getdents d1/d2/d6d 0 2024-03-29T04:45:27.965 INFO:tasks.workunit.client.1.smithi152.stdout:7/797: unlink d7/d8/db/df/fb8 0 2024-03-29T04:45:27.972 INFO:tasks.workunit.client.1.smithi152.stdout:7/798: rename d7/d8/d12/f20 to d7/dae/ff6 0 2024-03-29T04:45:27.973 INFO:tasks.workunit.client.1.smithi152.stdout:6/998: write d1/d3/df/f81 [1072398,96789] 0 2024-03-29T04:45:27.973 INFO:tasks.workunit.client.1.smithi152.stdout:7/799: write d7/d8/db/f42 [2717647,2936] 0 2024-03-29T04:45:27.977 INFO:tasks.workunit.client.1.smithi152.stdout:7/800: readlink d7/d8/d12/d1d/d41/l4c 0 2024-03-29T04:45:27.978 INFO:tasks.workunit.client.1.smithi152.stdout:6/999: creat d1/d3/d1a/d1b/d29/d6f/dee/d10f/f150 x:0 0 0 2024-03-29T04:45:27.982 INFO:tasks.workunit.client.1.smithi152.stdout:8/888: dread d1/d2/d3/f96 [0,4194304] 0 2024-03-29T04:45:27.983 INFO:tasks.workunit.client.1.smithi152.stdout:7/801: dwrite d7/d8/d12/d35/f32 [0,4194304] 0 2024-03-29T04:45:27.986 INFO:tasks.workunit.client.1.smithi152.stdout:7/802: creat d7/d8/d12/d1d/d41/d83/ff7 x:0 0 0 2024-03-29T04:45:27.989 INFO:tasks.workunit.client.1.smithi152.stdout:8/889: link d1/d8a/d91/dee/l4c d1/d8a/d91/l13c 0 2024-03-29T04:45:27.991 INFO:tasks.workunit.client.1.smithi152.stdout:7/803: symlink d7/d8/d12/d35/lf8 0 2024-03-29T04:45:27.992 INFO:tasks.workunit.client.1.smithi152.stdout:8/890: symlink d1/d2/d1d/dd3/db0/d40/d58/l13d 0 2024-03-29T04:45:27.992 INFO:tasks.workunit.client.1.smithi152.stdout:8/891: chown d1/d8a/d91/dee/c8d 46 1 2024-03-29T04:45:27.994 INFO:tasks.workunit.client.1.smithi152.stdout:8/892: fdatasync d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/fb9 0 2024-03-29T04:45:27.995 INFO:tasks.workunit.client.1.smithi152.stdout:8/893: chown d1/d2/d1d/d36/d85/lfd 1441700 1 2024-03-29T04:45:27.997 INFO:tasks.workunit.client.1.smithi152.stdout:8/894: symlink d1/d2/d1d/d36/l13e 0 2024-03-29T04:45:28.008 INFO:tasks.workunit.client.1.smithi152.stdout:7/804: dwrite d7/d8/d12/d35/dba/df0/f80 [0,4194304] 0 2024-03-29T04:45:28.010 INFO:tasks.workunit.client.1.smithi152.stdout:8/895: dread d1/d2/d1d/f4f [0,4194304] 0 2024-03-29T04:45:28.015 INFO:tasks.workunit.client.1.smithi152.stdout:8/896: write d1/d2/d1d/d36/d85/fb4 [1560232,55216] 0 2024-03-29T04:45:28.018 INFO:tasks.workunit.client.1.smithi152.stdout:8/897: creat d1/d8a/d91/dee/f13f x:0 0 0 2024-03-29T04:45:28.022 INFO:tasks.workunit.client.1.smithi152.stdout:8/898: creat d1/d2/d1d/dd3/db0/d119/f140 x:0 0 0 2024-03-29T04:45:28.028 INFO:tasks.workunit.client.1.smithi152.stdout:8/899: dwrite d1/d8a/d91/dee/d12d/f130 [0,4194304] 0 2024-03-29T04:45:28.033 INFO:tasks.workunit.client.1.smithi152.stdout:7/805: dread d7/d8/d12/d35/dba/df0/d99/fad [0,4194304] 0 2024-03-29T04:45:28.035 INFO:tasks.workunit.client.1.smithi152.stdout:7/806: write d7/d8/d12/d35/f1a [2531983,27209] 0 2024-03-29T04:45:28.035 INFO:tasks.workunit.client.1.smithi152.stdout:8/900: getdents d1/d2/d1d/dd3/db0/da9/d105 0 2024-03-29T04:45:28.039 INFO:tasks.workunit.client.1.smithi152.stdout:7/807: truncate d7/d8/d12/d1d/f34 1053722 0 2024-03-29T04:45:28.043 INFO:tasks.workunit.client.1.smithi152.stdout:8/901: dwrite d1/d2/d6d/f103 [0,4194304] 0 2024-03-29T04:45:28.043 INFO:tasks.workunit.client.1.smithi152.stdout:7/808: mkdir d7/d8/d12/d35/dba/df0/d99/df9 0 2024-03-29T04:45:28.045 INFO:tasks.workunit.client.1.smithi152.stdout:8/902: fdatasync d1/d2/d1d/dd3/db0/f90 0 2024-03-29T04:45:28.048 INFO:tasks.workunit.client.1.smithi152.stdout:7/809: rmdir d7/d8/d12/d1d 39 2024-03-29T04:45:28.053 INFO:tasks.workunit.client.1.smithi152.stdout:8/903: dwrite d1/d8a/d91/dee/fea [0,4194304] 0 2024-03-29T04:45:28.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.064+0000 7f009b65f700 1 -- 172.21.15.142:0/2032932032 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0094108110 con 0x7f008006c4e0 2024-03-29T04:45:28.071 INFO:tasks.workunit.client.1.smithi152.stdout:7/810: dwrite d7/d8/d12/d1d/d41/d83/ff7 [0,4194304] 0 2024-03-29T04:45:28.072 INFO:tasks.workunit.client.1.smithi152.stdout:8/904: creat d1/d2/d1d/dd3/f141 x:0 0 0 2024-03-29T04:45:28.073 INFO:tasks.workunit.client.1.smithi152.stdout:7/811: write d7/d8/db3/fc9 [1013495,77575] 0 2024-03-29T04:45:28.075 INFO:tasks.workunit.client.1.smithi152.stdout:7/812: write d7/d8/d12/f61 [795374,109622] 0 2024-03-29T04:45:28.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.076+0000 7f008a7fc700 1 -- 172.21.15.142:0/2032932032 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f0094108110 con 0x7f008006c4e0 2024-03-29T04:45:28.077 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:45:28.077 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (7m) 2m ago 8m 25.1M - 0.25.0 c8568f914cd2 d4a3e12f4047 2024-03-29T04:45:28.077 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (9m) 2m ago 9m 7512k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:45:28.077 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (8m) 2m ago 8m 7432k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:45:28.077 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (9m) 2m ago 9m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:45:28.077 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (7m) 2m ago 7m 6660k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:45:28.077 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (7m) 2m ago 8m 85.8M - 9.4.7 954c08fa6188 aba58e552b7c 2024-03-29T04:45:28.077 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (3m) 2m ago 3m 17.7M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:45:28.077 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (3m) 2m ago 3m 12.8M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:45:28.077 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (3m) 2m ago 3m 17.3M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:45:28.077 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (3m) 2m ago 3m 17.4M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:45:28.077 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:9283,8765,8443 running (10m) 2m ago 10m 489M - 18.2.1 5be31c24972a a5f21a9cfe24 2024-03-29T04:45:28.078 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (7m) 2m ago 7m 441M - 18.2.1 5be31c24972a c6124d88151c 2024-03-29T04:45:28.078 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (10m) 2m ago 10m 46.5M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:45:28.078 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (7m) 2m ago 7m 35.2M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:45:28.078 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (8m) 2m ago 8m 16.2M - 1.5.0 0da6a335fe13 91b95c929db2 2024-03-29T04:45:28.078 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (7m) 2m ago 7m 15.0M - 1.5.0 0da6a335fe13 97e90e5b337b 2024-03-29T04:45:28.078 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (6m) 2m ago 6m 48.8M 3946M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:45:28.078 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (6m) 2m ago 6m 51.1M 3946M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:45:28.078 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (6m) 2m ago 6m 49.1M 3946M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:45:28.078 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (5m) 2m ago 5m 45.8M 4970M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:45:28.078 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (5m) 2m ago 5m 44.2M 4970M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:45:28.078 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (4m) 2m ago 4m 43.8M 4970M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:45:28.078 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (7m) 2m ago 8m 58.7M - 2.43.0 a07b618ecd1d c6c39d3c30f6 2024-03-29T04:45:28.080 INFO:tasks.workunit.client.1.smithi152.stdout:8/905: dwrite d1/d2/d1d/dd3/db0/d40/ddd/f120 [0,4194304] 0 2024-03-29T04:45:28.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.081+0000 7f009b65f700 1 -- 172.21.15.142:0/2032932032 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f008006c4e0 msgr2=0x7f008006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:28.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.081+0000 7f009b65f700 1 --2- 172.21.15.142:0/2032932032 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f008006c4e0 0x7f008006e9a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f00900059d0 tx=0x7f0090005940 comp rx=0 tx=0).stop 2024-03-29T04:45:28.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.081+0000 7f009b65f700 1 -- 172.21.15.142:0/2032932032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0094074d60 msgr2=0x7f0094198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:28.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.081+0000 7f009b65f700 1 --2- 172.21.15.142:0/2032932032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0094074d60 0x7f0094198ab0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f008400e920 tx=0x7f008400ece0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.081+0000 7f009b65f700 1 -- 172.21.15.142:0/2032932032 shutdown_connections 2024-03-29T04:45:28.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.081+0000 7f009b65f700 1 --2- 172.21.15.142:0/2032932032 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f008006c4e0 0x7f008006e9a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.081+0000 7f009b65f700 1 --2- 172.21.15.142:0/2032932032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0094073790 0x7f0094198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.081+0000 7f009b65f700 1 --2- 172.21.15.142:0/2032932032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0094074d60 0x7f0094198ab0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.081+0000 7f009b65f700 1 -- 172.21.15.142:0/2032932032 >> 172.21.15.142:0/2032932032 conn(0x7f00940fc110 msgr2=0x7f00941069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:28.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.082+0000 7f009b65f700 1 -- 172.21.15.142:0/2032932032 shutdown_connections 2024-03-29T04:45:28.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.082+0000 7f009b65f700 1 -- 172.21.15.142:0/2032932032 wait complete. 2024-03-29T04:45:28.082 INFO:tasks.workunit.client.1.smithi152.stdout:7/813: link d7/d8/db3/f8d d7/d8/db3/d72/dee/ffa 0 2024-03-29T04:45:28.101 INFO:tasks.workunit.client.1.smithi152.stdout:8/906: dwrite d1/d2/d1d/dd3/db0/da9/f84 [0,4194304] 0 2024-03-29T04:45:28.101 INFO:tasks.workunit.client.1.smithi152.stdout:7/814: dwrite d7/d8/d12/d1d/d41/d45/f6a [0,4194304] 0 2024-03-29T04:45:28.102 INFO:tasks.workunit.client.1.smithi152.stdout:8/907: fdatasync d1/d8a/fb3 0 2024-03-29T04:45:28.110 INFO:tasks.workunit.client.1.smithi152.stdout:7/815: dwrite d7/d8/f9c [0,4194304] 0 2024-03-29T04:45:28.111 INFO:tasks.workunit.client.1.smithi152.stdout:8/908: link d1/fab d1/d2/d1d/dd3/db0/d40/ddd/f142 0 2024-03-29T04:45:28.120 INFO:tasks.workunit.client.1.smithi152.stdout:7/816: dwrite d7/d8/db/f4b [4194304,4194304] 0 2024-03-29T04:45:28.124 INFO:tasks.workunit.client.1.smithi152.stdout:7/817: rename d7/d8/d37/cab to d7/d8/d12/d35/dba/df0/d99/df9/cfb 0 2024-03-29T04:45:28.129 INFO:tasks.workunit.client.1.smithi152.stdout:7/818: rename d7/d8/db/df/fd8 to d7/d8/d12/d1d/d41/ffc 0 2024-03-29T04:45:28.132 INFO:tasks.workunit.client.1.smithi152.stdout:8/909: write d1/d8a/fdf [240625,11514] 0 2024-03-29T04:45:28.133 INFO:tasks.workunit.client.1.smithi152.stdout:7/819: getdents d7/d8/db/df/de5 0 2024-03-29T04:45:28.137 INFO:tasks.workunit.client.1.smithi152.stdout:8/910: creat d1/d2/d1d/dd3/f143 x:0 0 0 2024-03-29T04:45:28.139 INFO:tasks.workunit.client.1.smithi152.stdout:7/820: dread d7/d8/d12/d35/dda/fd6 [0,4194304] 0 2024-03-29T04:45:28.139 INFO:tasks.workunit.client.1.smithi152.stdout:8/911: read d1/f7c [930907,119685] 0 2024-03-29T04:45:28.142 INFO:tasks.workunit.client.1.smithi152.stdout:8/912: readlink d1/d2/d1d/dd3/db0/d7f/lf4 0 2024-03-29T04:45:28.145 INFO:tasks.workunit.client.1.smithi152.stdout:8/913: symlink d1/d8a/d91/dee/d11b/l144 0 2024-03-29T04:45:28.147 INFO:tasks.workunit.client.1.smithi152.stdout:7/821: dwrite d7/d8/d12/d35/dda/f9b [0,4194304] 0 2024-03-29T04:45:28.152 INFO:tasks.workunit.client.1.smithi152.stdout:8/914: mknod d1/d2/d1d/d36/d5d/c145 0 2024-03-29T04:45:28.153 INFO:tasks.workunit.client.1.smithi152.stdout:7/822: read d7/d8/db3/f31 [111127,41114] 0 2024-03-29T04:45:28.155 INFO:tasks.workunit.client.1.smithi152.stdout:8/915: creat d1/d2/d3c/d76/d127/f146 x:0 0 0 2024-03-29T04:45:28.156 INFO:tasks.workunit.client.1.smithi152.stdout:7/823: mkdir d7/d8/d12/d35/dba/dfd 0 2024-03-29T04:45:28.157 INFO:tasks.workunit.client.1.smithi152.stdout:8/916: creat d1/d2/d3/f147 x:0 0 0 2024-03-29T04:45:28.158 INFO:tasks.workunit.client.1.smithi152.stdout:7/824: rmdir d7/d8/db 39 2024-03-29T04:45:28.158 INFO:tasks.workunit.client.1.smithi152.stdout:7/825: stat d7/d8/d12/d35/d68/d84/ce3 0 2024-03-29T04:45:28.159 INFO:tasks.workunit.client.1.smithi152.stdout:8/917: chown d1/d2/d1d/dd3/db0/d40/d47/c134 10 1 2024-03-29T04:45:28.161 INFO:tasks.workunit.client.1.smithi152.stdout:8/918: mknod d1/d8a/d91/dee/d12d/c148 0 2024-03-29T04:45:28.166 INFO:tasks.workunit.client.1.smithi152.stdout:7/826: getdents d7/d8/d12/d35/dba 0 2024-03-29T04:45:28.168 INFO:tasks.workunit.client.1.smithi152.stdout:7/827: mknod d7/d8/d37/cfe 0 2024-03-29T04:45:28.168 INFO:tasks.workunit.client.1.smithi152.stdout:8/919: truncate d1/d2/d3/ffb 2684612 0 2024-03-29T04:45:28.169 INFO:tasks.workunit.client.1.smithi152.stdout:7/828: write d7/d8/d12/d35/d68/dc3/fe4 [3998045,14106] 0 2024-03-29T04:45:28.172 INFO:tasks.workunit.client.1.smithi152.stdout:8/920: rename d1/d8a/d91 to d1/dc5/d149 0 2024-03-29T04:45:28.172 INFO:tasks.workunit.client.1.smithi152.stdout:8/921: stat d1/d2/c1e 0 2024-03-29T04:45:28.179 INFO:tasks.workunit.client.1.smithi152.stdout:8/922: write d1/d2/d1d/d36/d4b/f68 [848259,79713] 0 2024-03-29T04:45:28.181 INFO:tasks.workunit.client.1.smithi152.stdout:8/923: write d1/d2/d1d/f30 [4581541,42687] 0 2024-03-29T04:45:28.186 INFO:tasks.workunit.client.1.smithi152.stdout:8/924: rename d1/d2/d1d/d36/d85 to d1/d2/d1d/d36/d107/d14a 0 2024-03-29T04:45:28.186 INFO:tasks.workunit.client.1.smithi152.stdout:8/925: readlink d1/d2/d1d/d36/d4b/l7e 0 2024-03-29T04:45:28.187 INFO:tasks.workunit.client.1.smithi152.stdout:8/926: fdatasync d1/dc5/fcd 0 2024-03-29T04:45:28.189 INFO:tasks.workunit.client.1.smithi152.stdout:8/927: mknod d1/d2/d1d/d36/d111/c14b 0 2024-03-29T04:45:28.193 INFO:tasks.workunit.client.1.smithi152.stdout:8/928: getdents d1/d2/d3c/d76/d9a 0 2024-03-29T04:45:28.194 INFO:tasks.workunit.client.1.smithi152.stdout:8/929: rmdir d1/d2/d1d/dd3/db0/d40/d47 39 2024-03-29T04:45:28.195 INFO:tasks.workunit.client.1.smithi152.stdout:8/930: fsync d1/d2/d6d/f106 0 2024-03-29T04:45:28.200 INFO:tasks.workunit.client.1.smithi152.stdout:8/931: link d1/d2/d3c/d76/d9a/fad d1/d2/d1d/dd3/db0/d40/ddd/f14c 0 2024-03-29T04:45:28.201 INFO:tasks.workunit.client.1.smithi152.stdout:8/932: truncate d1/d8a/fdf 1063054 0 2024-03-29T04:45:28.202 INFO:tasks.workunit.client.1.smithi152.stdout:8/933: dread - d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/f117 zero size 2024-03-29T04:45:28.206 INFO:tasks.workunit.client.1.smithi152.stdout:8/934: symlink d1/d2/d1d/dd3/db0/d40/d47/dca/l14d 0 2024-03-29T04:45:28.213 INFO:tasks.workunit.client.1.smithi152.stdout:8/935: rename d1/d2/d1d/dd3/db0/d40/d47/dca to d1/d2/d3c/d76/d9a/def/d14e 0 2024-03-29T04:45:28.215 INFO:tasks.workunit.client.1.smithi152.stdout:8/936: rmdir d1/d2/d1d/d36/d111 39 2024-03-29T04:45:28.220 INFO:tasks.workunit.client.1.smithi152.stdout:8/937: dwrite d1/d2/d3/f96 [0,4194304] 0 2024-03-29T04:45:28.226 INFO:tasks.workunit.client.1.smithi152.stdout:8/938: write d1/d2/d1d/d36/d107/d14a/fb2 [4006951,31124] 0 2024-03-29T04:45:28.229 INFO:tasks.workunit.client.1.smithi152.stdout:8/939: getdents d1/d2/d1d/dd3/db0/da9/d105 0 2024-03-29T04:45:28.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.232+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2170655308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80ffab0 msgr2=0x7fc8e80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:28.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.232+0000 7fc8edc2c700 1 --2- 172.21.15.142:0/2170655308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80ffab0 0x7fc8e80ffed0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009910 tx=0x7fc8d8009c20 comp rx=0 tx=0).stop 2024-03-29T04:45:28.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.232+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2170655308 shutdown_connections 2024-03-29T04:45:28.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.232+0000 7fc8edc2c700 1 --2- 172.21.15.142:0/2170655308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e8100cb0 0x7fc8e8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.232+0000 7fc8edc2c700 1 --2- 172.21.15.142:0/2170655308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80ffab0 0x7fc8e80ffed0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.232+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2170655308 >> 172.21.15.142:0/2170655308 conn(0x7fc8e80fb050 msgr2=0x7fc8e80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:28.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.233+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2170655308 shutdown_connections 2024-03-29T04:45:28.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.233+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2170655308 wait complete. 2024-03-29T04:45:28.233 INFO:tasks.workunit.client.1.smithi152.stdout:8/940: unlink d1/d2/d3/c1b 0 2024-03-29T04:45:28.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.233+0000 7fc8edc2c700 1 Processor -- start 2024-03-29T04:45:28.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.234+0000 7fc8edc2c700 1 -- start start 2024-03-29T04:45:28.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.234+0000 7fc8edc2c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80ffab0 0x7fc8e8195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:28.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.234+0000 7fc8edc2c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e8100cb0 0x7fc8e81957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:28.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.234+0000 7fc8edc2c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e8195de0 con 0x7fc8e80ffab0 2024-03-29T04:45:28.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.234+0000 7fc8edc2c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e8195f20 con 0x7fc8e8100cb0 2024-03-29T04:45:28.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.234+0000 7fc8e77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80ffab0 0x7fc8e8195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:28.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.234+0000 7fc8e77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80ffab0 0x7fc8e8195280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45350/0 (socket says 172.21.15.142:45350) 2024-03-29T04:45:28.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.234+0000 7fc8e77fe700 1 -- 172.21.15.142:0/2333159016 learned_addr learned my addr 172.21.15.142:0/2333159016 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:45:28.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.234+0000 7fc8e6ffd700 1 --2- 172.21.15.142:0/2333159016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e8100cb0 0x7fc8e81957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:28.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.234+0000 7fc8e77fe700 1 -- 172.21.15.142:0/2333159016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e8100cb0 msgr2=0x7fc8e81957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:28.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.234+0000 7fc8e77fe700 1 --2- 172.21.15.142:0/2333159016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e8100cb0 0x7fc8e81957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.234+0000 7fc8e77fe700 1 -- 172.21.15.142:0/2333159016 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d80095f0 con 0x7fc8e80ffab0 2024-03-29T04:45:28.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.235+0000 7fc8e77fe700 1 --2- 172.21.15.142:0/2333159016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80ffab0 0x7fc8e8195280 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800bde0 tx=0x7fc8d800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:28.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.235+0000 7fc8e4ff9700 1 -- 172.21.15.142:0/2333159016 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8d801f070 con 0x7fc8e80ffab0 2024-03-29T04:45:28.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.235+0000 7fc8e4ff9700 1 -- 172.21.15.142:0/2333159016 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc8d8019610 con 0x7fc8e80ffab0 2024-03-29T04:45:28.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.235+0000 7fc8e4ff9700 1 -- 172.21.15.142:0/2333159016 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8d8024820 con 0x7fc8e80ffab0 2024-03-29T04:45:28.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.235+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2333159016 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e8068ca0 con 0x7fc8e80ffab0 2024-03-29T04:45:28.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.235+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2333159016 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e8069190 con 0x7fc8e80ffab0 2024-03-29T04:45:28.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.236+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2333159016 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e8108610 con 0x7fc8e80ffab0 2024-03-29T04:45:28.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.237+0000 7fc8e4ff9700 1 -- 172.21.15.142:0/2333159016 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc8d8019780 con 0x7fc8e80ffab0 2024-03-29T04:45:28.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.237+0000 7fc8e4ff9700 1 --2- 172.21.15.142:0/2333159016 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc8d006c4e0 0x7fc8d006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:28.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.237+0000 7fc8e4ff9700 1 -- 172.21.15.142:0/2333159016 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc8d808f130 con 0x7fc8e80ffab0 2024-03-29T04:45:28.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.237+0000 7fc8e6ffd700 1 --2- 172.21.15.142:0/2333159016 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc8d006c4e0 0x7fc8d006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:28.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.238+0000 7fc8e6ffd700 1 --2- 172.21.15.142:0/2333159016 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc8d006c4e0 0x7fc8d006e9a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005950 tx=0x7fc8dc00a300 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:28.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.242+0000 7fc8e4ff9700 1 -- 172.21.15.142:0/2333159016 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc8d805d3c0 con 0x7fc8e80ffab0 2024-03-29T04:45:28.244 INFO:tasks.workunit.client.1.smithi152.stdout:8/941: dwrite d1/d2/d1d/dd3/db0/d40/f6c [0,4194304] 0 2024-03-29T04:45:28.247 INFO:tasks.workunit.client.1.smithi152.stdout:8/942: mknod d1/d2/d1d/d36/d107/d14a/d129/c14f 0 2024-03-29T04:45:28.248 INFO:tasks.workunit.client.1.smithi152.stdout:8/943: chown d1/d2/d1d/dd3/db0/d40/d58 333788778 1 2024-03-29T04:45:28.253 INFO:tasks.workunit.client.1.smithi152.stdout:8/944: rename d1/d2/d1d/dd3/db0/d40/ddd to d1/d2/d3/d150 0 2024-03-29T04:45:28.561 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.561+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2333159016 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc8e819a970 con 0x7fc8e80ffab0 2024-03-29T04:45:28.562 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.562+0000 7fc8e4ff9700 1 -- 172.21.15.142:0/2333159016 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fc8d805cf50 con 0x7fc8e80ffab0 2024-03-29T04:45:28.562 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:45:28.562 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:45:28.562 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:45:28.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:45:28.562 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:45:28.562 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:45:28.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:45:28.563 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:45:28.563 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:45:28.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:45:28.563 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:45:28.563 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:45:28.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:45:28.563 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:45:28.563 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-03-29T04:45:28.563 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:45:28.563 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:45:28.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.564+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2333159016 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc8d006c4e0 msgr2=0x7fc8d006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:28.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.564+0000 7fc8edc2c700 1 --2- 172.21.15.142:0/2333159016 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc8d006c4e0 0x7fc8d006e9a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005950 tx=0x7fc8dc00a300 comp rx=0 tx=0).stop 2024-03-29T04:45:28.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.564+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2333159016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80ffab0 msgr2=0x7fc8e8195280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:28.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.564+0000 7fc8edc2c700 1 --2- 172.21.15.142:0/2333159016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80ffab0 0x7fc8e8195280 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800bde0 tx=0x7fc8d800be10 comp rx=0 tx=0).stop 2024-03-29T04:45:28.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.564+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2333159016 shutdown_connections 2024-03-29T04:45:28.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.564+0000 7fc8edc2c700 1 --2- 172.21.15.142:0/2333159016 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7fc8d006c4e0 0x7fc8d006e9a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.564+0000 7fc8edc2c700 1 --2- 172.21.15.142:0/2333159016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80ffab0 0x7fc8e8195280 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.564+0000 7fc8edc2c700 1 --2- 172.21.15.142:0/2333159016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e8100cb0 0x7fc8e81957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.564+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2333159016 >> 172.21.15.142:0/2333159016 conn(0x7fc8e80fb050 msgr2=0x7fc8e8103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:28.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.565+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2333159016 shutdown_connections 2024-03-29T04:45:28.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.565+0000 7fc8edc2c700 1 -- 172.21.15.142:0/2333159016 wait complete. 2024-03-29T04:45:28.648 INFO:tasks.workunit.client.1.smithi152.stdout:5/956: sync 2024-03-29T04:45:28.648 INFO:tasks.workunit.client.1.smithi152.stdout:4/731: sync 2024-03-29T04:45:28.650 INFO:tasks.workunit.client.1.smithi152.stdout:4/732: write d0/d7/d25/d17/d5e/d60/f8b [1189513,45295] 0 2024-03-29T04:45:28.651 INFO:tasks.workunit.client.1.smithi152.stdout:4/733: chown d0/d7/d25/d17/d65 13134 1 2024-03-29T04:45:28.652 INFO:tasks.workunit.client.1.smithi152.stdout:5/957: rename d6/d16/d25/d2e/da0/d92/d14e to d6/d16/df8/d150 0 2024-03-29T04:45:28.652 INFO:tasks.workunit.client.1.smithi152.stdout:4/734: readlink d0/d7/d25/l49 0 2024-03-29T04:45:28.653 INFO:tasks.workunit.client.1.smithi152.stdout:5/958: read d6/d1b/fcb [245486,106587] 0 2024-03-29T04:45:28.664 INFO:tasks.workunit.client.1.smithi152.stdout:4/735: dread d0/d27/f59 [0,4194304] 0 2024-03-29T04:45:28.665 INFO:tasks.workunit.client.1.smithi152.stdout:4/736: chown d0/d7/d40/d62/f9a 20 1 2024-03-29T04:45:28.669 INFO:tasks.workunit.client.1.smithi152.stdout:4/737: mkdir d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/de7 0 2024-03-29T04:45:28.675 INFO:tasks.workunit.client.1.smithi152.stdout:4/738: getdents d0/d7/d25/d17/d5e/d60/d74/da2/da6 0 2024-03-29T04:45:28.675 INFO:tasks.workunit.client.1.smithi152.stdout:4/739: stat d0/d7/d40/l18 0 2024-03-29T04:45:28.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.680+0000 7f81cc95e700 1 -- 172.21.15.142:0/3091802371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c80ffab0 msgr2=0x7f81c80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:28.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.680+0000 7f81cc95e700 1 --2- 172.21.15.142:0/3091802371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c80ffab0 0x7f81c80ffed0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009910 tx=0x7f81b8009c20 comp rx=0 tx=0).stop 2024-03-29T04:45:28.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.680+0000 7f81cc95e700 1 -- 172.21.15.142:0/3091802371 shutdown_connections 2024-03-29T04:45:28.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.680+0000 7f81cc95e700 1 --2- 172.21.15.142:0/3091802371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c8100cb0 0x7f81c8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.680+0000 7f81cc95e700 1 --2- 172.21.15.142:0/3091802371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c80ffab0 0x7f81c80ffed0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.680+0000 7f81cc95e700 1 -- 172.21.15.142:0/3091802371 >> 172.21.15.142:0/3091802371 conn(0x7f81c80fb050 msgr2=0x7f81c80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:28.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.681+0000 7f81cc95e700 1 -- 172.21.15.142:0/3091802371 shutdown_connections 2024-03-29T04:45:28.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.681+0000 7f81cc95e700 1 -- 172.21.15.142:0/3091802371 wait complete. 2024-03-29T04:45:28.682 INFO:tasks.workunit.client.1.smithi152.stdout:5/959: write d6/d16/d25/d2e/da0/f146 [496332,122439] 0 2024-03-29T04:45:28.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.681+0000 7f81cc95e700 1 Processor -- start 2024-03-29T04:45:28.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.681+0000 7f81cc95e700 1 -- start start 2024-03-29T04:45:28.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.682+0000 7f81cc95e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c80ffab0 0x7f81c8195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:28.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.682+0000 7f81cc95e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c8100cb0 0x7f81c81957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:28.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.682+0000 7f81cc95e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c8195de0 con 0x7f81c8100cb0 2024-03-29T04:45:28.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.682+0000 7f81cc95e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c8195f20 con 0x7f81c80ffab0 2024-03-29T04:45:28.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.682+0000 7f81bffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c8100cb0 0x7f81c81957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:28.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.682+0000 7f81bffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c8100cb0 0x7f81c81957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35016/0 (socket says 172.21.15.142:35016) 2024-03-29T04:45:28.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.682+0000 7f81bffff700 1 -- 172.21.15.142:0/3068448876 learned_addr learned my addr 172.21.15.142:0/3068448876 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:45:28.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.682+0000 7f81c659c700 1 --2- 172.21.15.142:0/3068448876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c80ffab0 0x7f81c8195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:28.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.682+0000 7f81bffff700 1 -- 172.21.15.142:0/3068448876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c80ffab0 msgr2=0x7f81c8195280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:28.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.682+0000 7f81bffff700 1 --2- 172.21.15.142:0/3068448876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c80ffab0 0x7f81c8195280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:28.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.682+0000 7f81bffff700 1 -- 172.21.15.142:0/3068448876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b80095f0 con 0x7f81c8100cb0 2024-03-29T04:45:28.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.683+0000 7f81bffff700 1 --2- 172.21.15.142:0/3068448876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c8100cb0 0x7f81c81957c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81b000e920 tx=0x7f81b000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:28.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.683+0000 7f81bf7fe700 1 -- 172.21.15.142:0/3068448876 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81b0009790 con 0x7f81c8100cb0 2024-03-29T04:45:28.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.683+0000 7f81bf7fe700 1 -- 172.21.15.142:0/3068448876 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f81b000cba0 con 0x7f81c8100cb0 2024-03-29T04:45:28.685 INFO:tasks.workunit.client.1.smithi152.stdout:4/740: dread d0/d7/d25/d17/d5e/d60/f8b [0,4194304] 0 2024-03-29T04:45:28.685 INFO:tasks.workunit.client.1.smithi152.stdout:5/960: creat d6/d16/df8/f151 x:0 0 0 2024-03-29T04:45:28.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.684+0000 7f81bf7fe700 1 -- 172.21.15.142:0/3068448876 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81b0019850 con 0x7f81c8100cb0 2024-03-29T04:45:28.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.684+0000 7f81cc95e700 1 -- 172.21.15.142:0/3068448876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c8068d00 con 0x7f81c8100cb0 2024-03-29T04:45:28.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.684+0000 7f81cc95e700 1 -- 172.21.15.142:0/3068448876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c80691d0 con 0x7f81c8100cb0 2024-03-29T04:45:28.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.685+0000 7f81cc95e700 1 -- 172.21.15.142:0/3068448876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c8108610 con 0x7f81c8100cb0 2024-03-29T04:45:28.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.686+0000 7f81bf7fe700 1 -- 172.21.15.142:0/3068448876 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f81b000cd10 con 0x7f81c8100cb0 2024-03-29T04:45:28.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.686+0000 7f81bf7fe700 1 --2- 172.21.15.142:0/3068448876 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f81b406c2d0 0x7f81b406e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:28.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.686+0000 7f81bf7fe700 1 -- 172.21.15.142:0/3068448876 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f81b008d460 con 0x7f81c8100cb0 2024-03-29T04:45:28.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.687+0000 7f81c659c700 1 --2- 172.21.15.142:0/3068448876 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f81b406c2d0 0x7f81b406e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:28.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.687+0000 7f81c659c700 1 --2- 172.21.15.142:0/3068448876 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f81b406c2d0 0x7f81b406e790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f81b800b760 tx=0x7f81b8009f70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:28.687 INFO:tasks.workunit.client.1.smithi152.stdout:5/961: read - d6/d16/f104 zero size 2024-03-29T04:45:28.687 INFO:tasks.workunit.client.1.smithi152.stdout:5/962: creat d6/d1d/daa/db7/f152 x:0 0 0 2024-03-29T04:45:28.688 INFO:tasks.workunit.client.1.smithi152.stdout:4/741: getdents d0/d7/d40/d23/d55/d70/d96 0 2024-03-29T04:45:28.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:28.692+0000 7f81bf7fe700 1 -- 172.21.15.142:0/3068448876 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f81b005b460 con 0x7f81c8100cb0 2024-03-29T04:45:28.694 INFO:tasks.workunit.client.1.smithi152.stdout:4/742: creat d0/d7/d40/d23/d55/d70/d96/fe8 x:0 0 0 2024-03-29T04:45:28.697 INFO:tasks.workunit.client.1.smithi152.stdout:4/743: mknod d0/d7/d40/d23/dda/ce9 0 2024-03-29T04:45:28.699 INFO:tasks.workunit.client.1.smithi152.stdout:5/963: dwrite d6/d1b/fcb [0,4194304] 0 2024-03-29T04:45:28.700 INFO:tasks.workunit.client.1.smithi152.stdout:4/744: creat d0/d7/d40/d2a/fea x:0 0 0 2024-03-29T04:45:28.704 INFO:tasks.workunit.client.1.smithi152.stdout:4/745: creat d0/d7/d25/d17/d5e/d60/feb x:0 0 0 2024-03-29T04:45:28.712 INFO:tasks.workunit.client.1.smithi152.stdout:4/746: dwrite d0/d7/f8d [0,4194304] 0 2024-03-29T04:45:28.714 INFO:tasks.workunit.client.1.smithi152.stdout:2/781: sync 2024-03-29T04:45:28.714 INFO:tasks.workunit.client.1.smithi152.stdout:0/810: sync 2024-03-29T04:45:28.714 INFO:tasks.workunit.client.1.smithi152.stdout:7/829: sync 2024-03-29T04:45:28.714 INFO:tasks.workunit.client.1.smithi152.stdout:3/864: sync 2024-03-29T04:45:28.715 INFO:tasks.workunit.client.1.smithi152.stdout:4/747: mknod d0/d7/d25/d17/cec 0 2024-03-29T04:45:28.716 INFO:tasks.workunit.client.1.smithi152.stdout:7/830: chown d7/d8/d12/f6f 474440 1 2024-03-29T04:45:28.716 INFO:tasks.workunit.client.1.smithi152.stdout:9/898: sync 2024-03-29T04:45:28.716 INFO:tasks.workunit.client.1.smithi152.stdout:8/945: sync 2024-03-29T04:45:28.721 INFO:tasks.workunit.client.1.smithi152.stdout:3/865: fdatasync d7/d19/d1b/d1e/d4e/fad 0 2024-03-29T04:45:28.722 INFO:tasks.workunit.client.1.smithi152.stdout:2/782: write d0/d3/d30/d15/d53/d54/fcf [1102192,124925] 0 2024-03-29T04:45:28.726 INFO:tasks.workunit.client.1.smithi152.stdout:5/964: dread d6/d16/d25/d2e/f54 [0,4194304] 0 2024-03-29T04:45:28.726 INFO:tasks.workunit.client.1.smithi152.stdout:7/831: rmdir d7/d8/d12/d35/dba/dfd 0 2024-03-29T04:45:28.727 INFO:tasks.workunit.client.1.smithi152.stdout:0/811: getdents d2/da/d17/de7 0 2024-03-29T04:45:28.729 INFO:tasks.workunit.client.1.smithi152.stdout:2/783: dread d0/d3/d30/d15/d22/f91 [0,4194304] 0 2024-03-29T04:45:28.730 INFO:tasks.workunit.client.1.smithi152.stdout:4/748: write d0/d7/d40/d2a/f42 [139363,79132] 0 2024-03-29T04:45:28.731 INFO:tasks.workunit.client.1.smithi152.stdout:0/812: truncate d2/da/df/fb3 1006479 0 2024-03-29T04:45:28.732 INFO:tasks.workunit.client.1.smithi152.stdout:4/749: fdatasync d0/d27/d6e/fad 0 2024-03-29T04:45:28.732 INFO:tasks.workunit.client.1.smithi152.stdout:5/965: mknod d6/d1d/daa/c153 0 2024-03-29T04:45:28.732 INFO:tasks.workunit.client.1.smithi152.stdout:2/784: chown d0/d3/df/d2c/l50 490 1 2024-03-29T04:45:28.736 INFO:tasks.workunit.client.1.smithi152.stdout:9/899: truncate d5/db/d92/dd2/d137/db1/dcd/f5f 3638982 0 2024-03-29T04:45:28.740 INFO:tasks.workunit.client.1.smithi152.stdout:0/813: dwrite d2/da/d17/dd8/db8/f7b [0,4194304] 0 2024-03-29T04:45:28.741 INFO:tasks.workunit.client.1.smithi152.stdout:4/750: rename d0/d7/d25/d17/d5e/d60/f88 to d0/d7/d25/d17/d37/fed 0 2024-03-29T04:45:28.743 INFO:tasks.workunit.client.1.smithi152.stdout:7/832: creat d7/d8/db/fff x:0 0 0 2024-03-29T04:45:28.746 INFO:tasks.workunit.client.1.smithi152.stdout:4/751: creat d0/d7/d25/d17/d37/d77/fee x:0 0 0 2024-03-29T04:45:28.746 INFO:tasks.workunit.client.1.smithi152.stdout:8/946: dread d1/dc5/d149/dee/f92 [0,4194304] 0 2024-03-29T04:45:28.749 INFO:tasks.workunit.client.1.smithi152.stdout:3/866: truncate d7/d19/d1b/d1e/d4e/d8f/fc0 2604862 0 2024-03-29T04:45:28.750 INFO:tasks.workunit.client.1.smithi152.stdout:5/966: link l4 d6/d1d/d30/d6a/l154 0 2024-03-29T04:45:28.751 INFO:tasks.workunit.client.1.smithi152.stdout:8/947: write d1/d2/d1d/dd3/db0/d119/f140 [734348,60273] 0 2024-03-29T04:45:28.751 INFO:tasks.workunit.client.1.smithi152.stdout:7/833: fdatasync d7/d8/d12/fb6 0 2024-03-29T04:45:28.753 INFO:tasks.workunit.client.1.smithi152.stdout:4/752: creat d0/d7/d25/d17/d5e/fef x:0 0 0 2024-03-29T04:45:28.754 INFO:tasks.workunit.client.1.smithi152.stdout:0/814: creat d2/f11c x:0 0 0 2024-03-29T04:45:28.755 INFO:tasks.workunit.client.1.smithi152.stdout:4/753: stat d0/d7/d40/d75/f7d 0 2024-03-29T04:45:28.756 INFO:tasks.workunit.client.1.smithi152.stdout:7/834: fdatasync d7/d8/d12/d35/dba/df0/fa9 0 2024-03-29T04:45:28.756 INFO:tasks.workunit.client.1.smithi152.stdout:5/967: symlink d6/d16/d109/l155 0 2024-03-29T04:45:28.761 INFO:tasks.workunit.client.1.smithi152.stdout:9/900: dwrite d5/db/d1b/f12c [0,4194304] 0 2024-03-29T04:45:28.763 INFO:tasks.workunit.client.1.smithi152.stdout:7/835: creat d7/dae/f100 x:0 0 0 2024-03-29T04:45:28.764 INFO:tasks.workunit.client.1.smithi152.stdout:9/901: write d5/db/d1b/f43 [4643473,63019] 0 2024-03-29T04:45:28.764 INFO:tasks.workunit.client.1.smithi152.stdout:5/968: rename d6/d1d/d30/d6a/d108/l19 to d6/d1d/daa/db7/d56/d63/db9/l156 0 2024-03-29T04:45:28.768 INFO:tasks.workunit.client.1.smithi152.stdout:8/948: creat d1/d2/d1d/dd3/f151 x:0 0 0 2024-03-29T04:45:28.768 INFO:tasks.workunit.client.1.smithi152.stdout:2/785: dread d0/f3b [0,4194304] 0 2024-03-29T04:45:28.768 INFO:tasks.workunit.client.1.smithi152.stdout:2/786: readlink d0/d3/d5f/lc4 0 2024-03-29T04:45:28.769 INFO:tasks.workunit.client.1.smithi152.stdout:3/867: write d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/fab [425688,121713] 0 2024-03-29T04:45:28.772 INFO:tasks.workunit.client.1.smithi152.stdout:5/969: mknod d6/d1d/d30/d6a/d108/d13d/c157 0 2024-03-29T04:45:28.772 INFO:tasks.workunit.client.1.smithi152.stdout:5/970: chown d6/d1b 102219685 1 2024-03-29T04:45:28.779 INFO:tasks.workunit.client.1.smithi152.stdout:5/971: mkdir d6/d1d/d30/dc5/d158 0 2024-03-29T04:45:28.781 INFO:tasks.workunit.client.1.smithi152.stdout:8/949: mkdir d1/d2/d1d/d36/d152 0 2024-03-29T04:45:28.781 INFO:tasks.workunit.client.1.smithi152.stdout:4/754: dwrite d0/d7/f84 [0,4194304] 0 2024-03-29T04:45:28.782 INFO:tasks.workunit.client.1.smithi152.stdout:0/815: dread d2/f6b [0,4194304] 0 2024-03-29T04:45:28.784 INFO:tasks.workunit.client.1.smithi152.stdout:3/868: mknod d7/d19/d1b/d59/dec/c11a 0 2024-03-29T04:45:28.787 INFO:tasks.workunit.client.1.smithi152.stdout:9/902: rename d5/db/d1b/f8b to d5/db/d92/dd2/d137/db1/dcd/d6e/f140 0 2024-03-29T04:45:28.794 INFO:tasks.workunit.client.1.smithi152.stdout:0/816: mkdir d2/da/d3a/d61/db4/d11d 0 2024-03-29T04:45:28.795 INFO:tasks.workunit.client.1.smithi152.stdout:8/950: dread d1/dc5/d149/dee/f63 [0,4194304] 0 2024-03-29T04:45:28.800 INFO:tasks.workunit.client.1.smithi152.stdout:5/972: rename d6/d1d/d30/d6a/d108/f17 to d6/d16/db2/f159 0 2024-03-29T04:45:28.801 INFO:tasks.workunit.client.1.smithi152.stdout:7/836: dwrite d7/d8/f2d [0,4194304] 0 2024-03-29T04:45:28.803 INFO:tasks.workunit.client.1.smithi152.stdout:7/837: write d7/d8/f6e [2393753,44875] 0 2024-03-29T04:45:28.804 INFO:tasks.workunit.client.1.smithi152.stdout:0/817: unlink d2/da/d3a/cc1 0 2024-03-29T04:45:28.810 INFO:tasks.workunit.client.1.smithi152.stdout:2/787: dwrite d0/d3/d5f/f92 [0,4194304] 0 2024-03-29T04:45:28.811 INFO:tasks.workunit.client.1.smithi152.stdout:7/838: truncate d7/d8/d12/d35/d68/dc3/fe0 314875 0 2024-03-29T04:45:28.813 INFO:tasks.workunit.client.1.smithi152.stdout:7/839: stat d7/d8/db3/l5b 0 2024-03-29T04:45:28.818 INFO:tasks.workunit.client.1.smithi152.stdout:0/818: write d2/da/d17/fbd [1859847,114796] 0 2024-03-29T04:45:28.821 INFO:tasks.workunit.client.1.smithi152.stdout:3/869: getdents d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4 0 2024-03-29T04:45:28.821 INFO:tasks.workunit.client.1.smithi152.stdout:8/951: write d1/d2/d1d/d36/d4b/d51/f5e [494812,49668] 0 2024-03-29T04:45:28.821 INFO:tasks.workunit.client.1.smithi152.stdout:9/903: creat d5/d12/f141 x:0 0 0 2024-03-29T04:45:28.823 INFO:tasks.workunit.client.1.smithi152.stdout:4/755: dwrite d0/d7/d25/d17/f71 [0,4194304] 0 2024-03-29T04:45:28.825 INFO:tasks.workunit.client.1.smithi152.stdout:0/819: fdatasync d2/da/d17/d57/f118 0 2024-03-29T04:45:28.825 INFO:tasks.workunit.client.1.smithi152.stdout:3/870: dread - d7/d6c/fd7 zero size 2024-03-29T04:45:28.827 INFO:tasks.workunit.client.1.smithi152.stdout:4/756: read - d0/d7/d40/d6d/fd0 zero size 2024-03-29T04:45:28.831 INFO:tasks.workunit.client.1.smithi152.stdout:4/757: write d0/d27/d6e/f97 [5560334,69746] 0 2024-03-29T04:45:28.831 INFO:tasks.workunit.client.1.smithi152.stdout:8/952: mkdir d1/d2/d1d/d36/d5d/d153 0 2024-03-29T04:45:28.835 INFO:tasks.workunit.client.1.smithi152.stdout:5/973: dwrite d6/d16/d25/d2e/da0/dab/f10b [0,4194304] 0 2024-03-29T04:45:28.835 INFO:tasks.workunit.client.1.smithi152.stdout:4/758: chown d0/d7/d40/d75/c9c 17804 1 2024-03-29T04:45:28.836 INFO:tasks.workunit.client.1.smithi152.stdout:3/871: truncate d7/d19/d1b/f2f 6509180 0 2024-03-29T04:45:28.838 INFO:tasks.workunit.client.1.smithi152.stdout:8/953: symlink d1/d2/d1d/d36/d107/d14a/d129/l154 0 2024-03-29T04:45:28.841 INFO:tasks.workunit.client.1.smithi152.stdout:3/872: rmdir d7/d19/d1b/d1e/db9 39 2024-03-29T04:45:28.842 INFO:tasks.workunit.client.1.smithi152.stdout:2/788: write d0/d3/d30/d15/d53/d54/d57/fd2 [4577613,129300] 0 2024-03-29T04:45:28.843 INFO:tasks.workunit.client.1.smithi152.stdout:9/904: getdents d5/d8/d31/d59/dc3/d125 0 2024-03-29T04:45:28.846 INFO:tasks.workunit.client.1.smithi152.stdout:7/840: dwrite d7/f59 [0,4194304] 0 2024-03-29T04:45:28.852 INFO:tasks.workunit.client.1.smithi152.stdout:2/789: symlink d0/d3/df/d9e/lee 0 2024-03-29T04:45:28.854 INFO:tasks.workunit.client.1.smithi152.stdout:2/790: chown d0/d3/d30/d15/d22/d74 49144579 1 2024-03-29T04:45:28.855 INFO:tasks.workunit.client.1.smithi152.stdout:3/873: creat d7/d19/d1b/d1e/da1/d41/d61/d85/f11b x:0 0 0 2024-03-29T04:45:28.856 INFO:tasks.workunit.client.1.smithi152.stdout:3/874: read - d7/d19/d1b/d1e/df0/ff4 zero size 2024-03-29T04:45:28.856 INFO:tasks.workunit.client.1.smithi152.stdout:4/759: write d0/d7/d25/d17/d5e/d60/d74/f51 [947669,11065] 0 2024-03-29T04:45:28.863 INFO:tasks.workunit.client.1.smithi152.stdout:3/875: creat d7/d6c/f11c x:0 0 0 2024-03-29T04:45:28.865 INFO:tasks.workunit.client.1.smithi152.stdout:5/974: dread d6/d7/fa4 [0,4194304] 0 2024-03-29T04:45:28.865 INFO:tasks.workunit.client.1.smithi152.stdout:3/876: mknod d7/d19/d1b/d1e/d22/c11d 0 2024-03-29T04:45:28.867 INFO:tasks.workunit.client.1.smithi152.stdout:3/877: chown d7/d19/d1b/d1e/da1/f51 2021008002 1 2024-03-29T04:45:28.870 INFO:tasks.workunit.client.1.smithi152.stdout:5/975: truncate d6/d1d/daa/db7/f128 4886600 0 2024-03-29T04:45:28.872 INFO:tasks.workunit.client.1.smithi152.stdout:2/791: write d0/d3/d30/d15/d22/f70 [9077866,125325] 0 2024-03-29T04:45:28.874 INFO:tasks.workunit.client.1.smithi152.stdout:9/905: truncate d5/d8/d31/d59/d6f/dbf/df4/f120 7855728 0 2024-03-29T04:45:28.875 INFO:tasks.workunit.client.1.smithi152.stdout:7/841: truncate d7/d8/db3/f76 279494 0 2024-03-29T04:45:28.877 INFO:tasks.workunit.client.1.smithi152.stdout:5/976: write d6/d1d/daa/de5/fdb [4059555,19364] 0 2024-03-29T04:45:28.878 INFO:tasks.workunit.client.1.smithi152.stdout:9/906: readlink d5/db/l33 0 2024-03-29T04:45:28.882 INFO:tasks.workunit.client.1.smithi152.stdout:2/792: stat d0/d3/d30/d15/d22/d74/dbd/f79 0 2024-03-29T04:45:28.884 INFO:tasks.workunit.client.1.smithi152.stdout:3/878: dwrite d7/d19/d1b/d59/fd8 [0,4194304] 0 2024-03-29T04:45:28.885 INFO:tasks.workunit.client.1.smithi152.stdout:8/954: dwrite d1/d2/d3c/d76/d9a/def/d14e/f123 [0,4194304] 0 2024-03-29T04:45:28.887 INFO:tasks.workunit.client.1.smithi152.stdout:2/793: readlink d0/d3/d30/d15/d53/d54/d57/l61 0 2024-03-29T04:45:28.889 INFO:tasks.workunit.client.1.smithi152.stdout:7/842: fsync d7/d8/d12/d35/dba/df0/d99/fbc 0 2024-03-29T04:45:28.890 INFO:tasks.workunit.client.1.smithi152.stdout:5/977: fsync d6/d16/d25/f116 0 2024-03-29T04:45:28.892 INFO:tasks.workunit.client.1.smithi152.stdout:8/955: rmdir d1/d2/d1d/dd3 39 2024-03-29T04:45:28.898 INFO:tasks.workunit.client.1.smithi152.stdout:7/843: symlink d7/d8/d12/d35/d68/dc3/l101 0 2024-03-29T04:45:28.898 INFO:tasks.workunit.client.1.smithi152.stdout:7/844: fdatasync d7/d8/f3b 0 2024-03-29T04:45:28.901 INFO:tasks.workunit.client.1.smithi152.stdout:8/956: mkdir d1/d2/d1d/d36/d107/d14a/d155 0 2024-03-29T04:45:28.904 INFO:tasks.workunit.client.1.smithi152.stdout:7/845: rename d7/d8/d12/d1d/d41/lb1 to d7/d8/d12/d1d/d41/l102 0 2024-03-29T04:45:28.905 INFO:tasks.workunit.client.1.smithi152.stdout:8/957: mkdir d1/d2/d1d/d36/d4b/d156 0 2024-03-29T04:45:28.905 INFO:tasks.workunit.client.1.smithi152.stdout:9/907: link d5/d8/d31/d59/d85/le7 d5/d12/db7/l142 0 2024-03-29T04:45:28.907 INFO:tasks.workunit.client.1.smithi152.stdout:3/879: rename d7/d19/d1b/d1e/da1/d41/d73/f84 to d7/d19/d1b/d1e/da1/f11e 0 2024-03-29T04:45:28.909 INFO:tasks.workunit.client.1.smithi152.stdout:3/880: fsync d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/fc6 0 2024-03-29T04:45:28.909 INFO:tasks.workunit.client.1.smithi152.stdout:8/958: readlink d1/dc5/d149/l13c 0 2024-03-29T04:45:28.910 INFO:tasks.workunit.client.1.smithi152.stdout:3/881: stat d7/d19/db5/df8/c106 0 2024-03-29T04:45:28.911 INFO:tasks.workunit.client.1.smithi152.stdout:9/908: creat d5/db/d92/dd2/d137/db1/f143 x:0 0 0 2024-03-29T04:45:28.912 INFO:tasks.workunit.client.1.smithi152.stdout:2/794: link d0/d3/d30/d15/c62 d0/d3/cef 0 2024-03-29T04:45:28.914 INFO:tasks.workunit.client.1.smithi152.stdout:9/909: write d5/db/d1b/f4f [4979411,4960] 0 2024-03-29T04:45:28.915 INFO:tasks.workunit.client.1.smithi152.stdout:3/882: creat d7/d19/d1b/d1e/da1/d41/d73/f11f x:0 0 0 2024-03-29T04:45:28.919 INFO:tasks.workunit.client.1.smithi152.stdout:5/978: dwrite d6/d16/db3/dfd/f10e [0,4194304] 0 2024-03-29T04:45:28.919 INFO:tasks.workunit.client.1.smithi152.stdout:3/883: dread - d7/d19/d1b/d1e/da1/f9c zero size 2024-03-29T04:45:28.921 INFO:tasks.workunit.client.1.smithi152.stdout:5/979: write d6/d1d/daa/f133 [301537,23480] 0 2024-03-29T04:45:28.921 INFO:tasks.workunit.client.1.smithi152.stdout:3/884: chown d7/d19/d1b/d1e/da1/d41/d6a/cb1 72800 1 2024-03-29T04:45:28.924 INFO:tasks.workunit.client.1.smithi152.stdout:7/846: dread d7/d8/d12/d1d/f49 [0,4194304] 0 2024-03-29T04:45:28.926 INFO:tasks.workunit.client.1.smithi152.stdout:5/980: creat d6/d1d/d30/dc5/f15a x:0 0 0 2024-03-29T04:45:28.929 INFO:tasks.workunit.client.1.smithi152.stdout:9/910: rename d5/db/d92/c11a to d5/db/d92/dd2/d137/db1/dcd/d6e/d73/c144 0 2024-03-29T04:45:28.930 INFO:tasks.workunit.client.1.smithi152.stdout:5/981: creat d6/d1d/daa/db7/f15b x:0 0 0 2024-03-29T04:45:28.931 INFO:tasks.workunit.client.1.smithi152.stdout:9/911: read d5/d8/d31/d59/d85/fe9 [31239,85696] 0 2024-03-29T04:45:28.931 INFO:tasks.workunit.client.1.smithi152.stdout:7/847: mknod d7/d8/d12/d1d/c103 0 2024-03-29T04:45:28.932 INFO:tasks.workunit.client.1.smithi152.stdout:5/982: truncate d6/d16/df8/d150/f95 5659931 0 2024-03-29T04:45:28.933 INFO:tasks.workunit.client.1.smithi152.stdout:5/983: chown d6/d1d/d30/dc5/l101 6402899 1 2024-03-29T04:45:28.934 INFO:tasks.workunit.client.1.smithi152.stdout:7/848: mknod d7/d8/db/df/c104 0 2024-03-29T04:45:28.938 INFO:tasks.workunit.client.1.smithi152.stdout:7/849: symlink d7/d8/d12/d35/dba/df0/d99/dc8/l105 0 2024-03-29T04:45:28.939 INFO:tasks.workunit.client.1.smithi152.stdout:5/984: getdents d6/d1d/da5 0 2024-03-29T04:45:28.941 INFO:tasks.workunit.client.1.smithi152.stdout:7/850: creat d7/d8/d12/d35/dba/df0/f106 x:0 0 0 2024-03-29T04:45:28.945 INFO:tasks.workunit.client.1.smithi152.stdout:7/851: rename d7/d8/db3/d72/f79 to d7/d8/db3/d72/dee/f107 0 2024-03-29T04:45:28.950 INFO:tasks.workunit.client.1.smithi152.stdout:5/985: write d6/d1b/f91 [2481066,94311] 0 2024-03-29T04:45:28.954 INFO:tasks.workunit.client.1.smithi152.stdout:3/885: dread d7/d19/f29 [0,4194304] 0 2024-03-29T04:45:28.956 INFO:tasks.workunit.client.1.smithi152.stdout:7/852: dread d7/dae/ff6 [0,4194304] 0 2024-03-29T04:45:28.957 INFO:tasks.workunit.client.1.smithi152.stdout:5/986: write d6/d16/d25/f8e [1640651,26903] 0 2024-03-29T04:45:28.959 INFO:tasks.workunit.client.1.smithi152.stdout:7/853: creat d7/d8/d12/d35/dba/df0/d99/f108 x:0 0 0 2024-03-29T04:45:28.960 INFO:tasks.workunit.client.1.smithi152.stdout:5/987: rmdir d6/d16/db2 39 2024-03-29T04:45:28.960 INFO:tasks.workunit.client.1.smithi152.stdout:7/854: fsync d7/d8/d12/f6f 0 2024-03-29T04:45:28.962 INFO:tasks.workunit.client.1.smithi152.stdout:5/988: mknod d6/d1d/da5/c15c 0 2024-03-29T04:45:28.962 INFO:tasks.workunit.client.1.smithi152.stdout:5/989: stat d6/d1d/daa/db7 0 2024-03-29T04:45:28.964 INFO:tasks.workunit.client.1.smithi152.stdout:7/855: getdents d7/d8/db/df 0 2024-03-29T04:45:28.965 INFO:tasks.workunit.client.1.smithi152.stdout:5/990: write d6/d16/df8/d150/f39 [2284350,99225] 0 2024-03-29T04:45:28.968 INFO:tasks.workunit.client.1.smithi152.stdout:5/991: write d6/d16/db2/f159 [1018080,5923] 0 2024-03-29T04:45:28.970 INFO:tasks.workunit.client.1.smithi152.stdout:7/856: getdents d7/d8/d12/d35/dba/df0/d99 0 2024-03-29T04:45:28.971 INFO:tasks.workunit.client.1.smithi152.stdout:7/857: write d7/d8/db3/d72/fdc [5236973,22567] 0 2024-03-29T04:45:28.972 INFO:tasks.workunit.client.1.smithi152.stdout:5/992: mkdir d6/d16/d25/d15d 0 2024-03-29T04:45:28.973 INFO:tasks.workunit.client.1.smithi152.stdout:3/886: dread d7/d19/d1b/d1e/da1/f47 [0,4194304] 0 2024-03-29T04:45:28.974 INFO:tasks.workunit.client.1.smithi152.stdout:7/858: creat d7/d8/db3/d72/f109 x:0 0 0 2024-03-29T04:45:28.975 INFO:tasks.workunit.client.1.smithi152.stdout:7/859: fsync d7/d8/d37/f7f 0 2024-03-29T04:45:28.976 INFO:tasks.workunit.client.1.smithi152.stdout:5/993: symlink d6/d16/d25/d2e/da0/d92/dc0/dec/l15e 0 2024-03-29T04:45:28.976 INFO:tasks.workunit.client.1.smithi152.stdout:3/887: dread d7/d19/d1b/d1e/d4e/f5a [0,4194304] 0 2024-03-29T04:45:28.979 INFO:tasks.workunit.client.1.smithi152.stdout:5/994: rename d6/d1d/d30/dc5/ff9 to d6/d7/d75/f15f 0 2024-03-29T04:45:28.983 INFO:tasks.workunit.client.1.smithi152.stdout:7/860: dwrite d7/d8/d12/d35/dda/fd6 [0,4194304] 0 2024-03-29T04:45:28.983 INFO:tasks.workunit.client.1.smithi152.stdout:3/888: rename d7/d19/d1b/d59/ff3 to d7/d19/d1b/f120 0 2024-03-29T04:45:28.987 INFO:tasks.workunit.client.1.smithi152.stdout:3/889: read d7/f87 [3653934,29089] 0 2024-03-29T04:45:28.994 INFO:tasks.workunit.client.1.smithi152.stdout:5/995: write d6/d1d/daa/db7/f128 [659244,79580] 0 2024-03-29T04:45:28.996 INFO:tasks.workunit.client.1.smithi152.stdout:5/996: write d6/d1d/d30/d6a/d108/f66 [2925835,28377] 0 2024-03-29T04:45:29.001 INFO:tasks.workunit.client.1.smithi152.stdout:7/861: write d7/d8/d12/d35/dba/df0/fea [134489,74136] 0 2024-03-29T04:45:29.003 INFO:tasks.workunit.client.1.smithi152.stdout:7/862: creat d7/d8/d37/f10a x:0 0 0 2024-03-29T04:45:29.004 INFO:tasks.workunit.client.1.smithi152.stdout:5/997: truncate d6/d1d/d30/d6a/dfb/f10f 158027 0 2024-03-29T04:45:29.005 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:28 smithi152 ceph-mon[37177]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:45:29.006 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:28 smithi152 ceph-mon[37177]: from='client.14630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:45:29.006 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:28 smithi152 ceph-mon[37177]: pgmap v280: 65 pgs: 65 active+clean; 3.1 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 78 MiB/s rd, 201 MiB/s wr, 479 op/s 2024-03-29T04:45:29.006 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:28 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2333159016' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:45:29.008 INFO:tasks.workunit.client.1.smithi152.stdout:7/863: getdents d7/d8/d12/d35/dba/df0/d99/dc8 0 2024-03-29T04:45:29.010 INFO:tasks.workunit.client.1.smithi152.stdout:7/864: symlink d7/d8/d12/d35/dba/l10b 0 2024-03-29T04:45:29.012 INFO:tasks.workunit.client.1.smithi152.stdout:3/890: dread d7/d6c/fe6 [0,4194304] 0 2024-03-29T04:45:29.013 INFO:tasks.workunit.client.1.smithi152.stdout:5/998: rename d6/d1d/d30/d6a to d6/d1d/daa/db7/d56/d160 0 2024-03-29T04:45:29.014 INFO:tasks.workunit.client.1.smithi152.stdout:7/865: mkdir d7/d8/d12/d35/dba/d10c 0 2024-03-29T04:45:29.017 INFO:tasks.workunit.client.1.smithi152.stdout:7/866: fdatasync d7/d8/db3/f50 0 2024-03-29T04:45:29.018 INFO:tasks.workunit.client.1.smithi152.stdout:3/891: write d7/f10 [598064,18206] 0 2024-03-29T04:45:29.022 INFO:tasks.workunit.client.1.smithi152.stdout:3/892: write d7/d19/d1b/d1e/da1/f54 [2790310,65459] 0 2024-03-29T04:45:29.023 INFO:tasks.workunit.client.1.smithi152.stdout:3/893: stat d7/d19/d1b/d1e/d4e/f70 0 2024-03-29T04:45:29.026 INFO:tasks.workunit.client.1.smithi152.stdout:3/894: chown d7/d19/d1b/d1e/da1/d41/d6a/dcc 128848079 1 2024-03-29T04:45:29.026 INFO:tasks.workunit.client.1.smithi152.stdout:7/867: write d7/d8/d12/d1d/f34 [1160561,86394] 0 2024-03-29T04:45:29.031 INFO:tasks.workunit.client.1.smithi152.stdout:3/895: symlink d7/d19/d1b/d1e/da1/d41/d61/d75/dc7/l121 0 2024-03-29T04:45:29.031 INFO:tasks.workunit.client.1.smithi152.stdout:5/999: dwrite d6/d1b/f5a [0,4194304] 0 2024-03-29T04:45:29.032 INFO:tasks.workunit.client.1.smithi152.stdout:7/868: getdents d7/d8/db3/d72 0 2024-03-29T04:45:29.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.033+0000 7f81cc95e700 1 -- 172.21.15.142:0/3068448876 --> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f81c8105600 con 0x7f81b406c2d0 2024-03-29T04:45:29.033 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:28 smithi142 ceph-mon[29027]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:45:29.033 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:28 smithi142 ceph-mon[29027]: from='client.14630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:45:29.033 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:28 smithi142 ceph-mon[29027]: pgmap v280: 65 pgs: 65 active+clean; 3.1 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 78 MiB/s rd, 201 MiB/s wr, 479 op/s 2024-03-29T04:45:29.033 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:28 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2333159016' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:45:29.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.034+0000 7f81bf7fe700 1 -- 172.21.15.142:0/3068448876 <== mgr.14215 v2:172.21.15.142:6800/2395947042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f81c8105600 con 0x7f81b406c2d0 2024-03-29T04:45:29.034 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:45:29.034 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:45:29.034 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:45:29.034 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T04:45:29.034 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [], 2024-03-29T04:45:29.034 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "0/2 daemons upgraded", 2024-03-29T04:45:29.034 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mgr daemons", 2024-03-29T04:45:29.034 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:45:29.034 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:45:29.035 INFO:tasks.workunit.client.1.smithi152.stdout:7/869: chown d7/d8/d12/d35/d68/d84/ce3 8 1 2024-03-29T04:45:29.035 INFO:tasks.workunit.client.1.smithi152.stdout:3/896: write d7/d19/d1b/d1e/da1/d41/d73/f77 [154263,80778] 0 2024-03-29T04:45:29.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.038+0000 7f81cc95e700 1 -- 172.21.15.142:0/3068448876 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f81b406c2d0 msgr2=0x7f81b406e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:29.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.038+0000 7f81cc95e700 1 --2- 172.21.15.142:0/3068448876 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f81b406c2d0 0x7f81b406e790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f81b800b760 tx=0x7f81b8009f70 comp rx=0 tx=0).stop 2024-03-29T04:45:29.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.038+0000 7f81cc95e700 1 -- 172.21.15.142:0/3068448876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c8100cb0 msgr2=0x7f81c81957c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:29.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.038+0000 7f81cc95e700 1 --2- 172.21.15.142:0/3068448876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c8100cb0 0x7f81c81957c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81b000e920 tx=0x7f81b000ece0 comp rx=0 tx=0).stop 2024-03-29T04:45:29.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.038+0000 7f81cc95e700 1 -- 172.21.15.142:0/3068448876 shutdown_connections 2024-03-29T04:45:29.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.038+0000 7f81cc95e700 1 --2- 172.21.15.142:0/3068448876 >> [v2:172.21.15.142:6800/2395947042,v1:172.21.15.142:6801/2395947042] conn(0x7f81b406c2d0 0x7f81b406e790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:29.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.038+0000 7f81cc95e700 1 --2- 172.21.15.142:0/3068448876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81c80ffab0 0x7f81c8195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:29.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.038+0000 7f81cc95e700 1 --2- 172.21.15.142:0/3068448876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81c8100cb0 0x7f81c81957c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:29.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.038+0000 7f81cc95e700 1 -- 172.21.15.142:0/3068448876 >> 172.21.15.142:0/3068448876 conn(0x7f81c80fb050 msgr2=0x7f81c8103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:29.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.038+0000 7f81cc95e700 1 -- 172.21.15.142:0/3068448876 shutdown_connections 2024-03-29T04:45:29.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:29.038+0000 7f81cc95e700 1 -- 172.21.15.142:0/3068448876 wait complete. 2024-03-29T04:45:29.039 INFO:tasks.workunit.client.1.smithi152.stdout:7/870: link d7/d8/f2d d7/dae/f10d 0 2024-03-29T04:45:29.040 INFO:tasks.workunit.client.1.smithi152.stdout:7/871: mkdir d7/d10e 0 2024-03-29T04:45:29.042 INFO:tasks.workunit.client.1.smithi152.stdout:3/897: getdents d7/d19/d1b/d1e/da1/d41/d61/d85 0 2024-03-29T04:45:29.044 INFO:tasks.workunit.client.1.smithi152.stdout:7/872: rename d7/d8/d12/d35/d68/faa to d7/d8/d12/d35/dda/f10f 0 2024-03-29T04:45:29.047 INFO:tasks.workunit.client.1.smithi152.stdout:7/873: mknod d7/d8/d12/d1d/d41/d45/c110 0 2024-03-29T04:45:29.048 INFO:tasks.workunit.client.1.smithi152.stdout:3/898: creat d7/d19/d1b/d1e/f122 x:0 0 0 2024-03-29T04:45:29.049 INFO:tasks.workunit.client.1.smithi152.stdout:7/874: stat d7/d8/dcd/le9 0 2024-03-29T04:45:29.049 INFO:tasks.workunit.client.1.smithi152.stdout:3/899: fsync d7/d46/f6d 0 2024-03-29T04:45:29.051 INFO:tasks.workunit.client.1.smithi152.stdout:7/875: creat d7/d8/db/df/de5/f111 x:0 0 0 2024-03-29T04:45:29.053 INFO:tasks.workunit.client.1.smithi152.stdout:3/900: creat d7/d19/d1b/d59/f123 x:0 0 0 2024-03-29T04:45:29.055 INFO:tasks.workunit.client.1.smithi152.stdout:3/901: fdatasync d7/d19/d1b/d59/fcb 0 2024-03-29T04:45:29.056 INFO:tasks.workunit.client.1.smithi152.stdout:3/902: write d7/d19/d1b/f7e [726327,908] 0 2024-03-29T04:45:29.057 INFO:tasks.workunit.client.1.smithi152.stdout:3/903: write d7/d6c/f11c [730925,37573] 0 2024-03-29T04:45:29.060 INFO:tasks.workunit.client.1.smithi152.stdout:3/904: mkdir d7/d19/d1b/d1e/da1/d41/d124 0 2024-03-29T04:45:29.065 INFO:tasks.workunit.client.1.smithi152.stdout:3/905: getdents d7/d19/d1b/d1e/db9 0 2024-03-29T04:45:29.068 INFO:tasks.workunit.client.1.smithi152.stdout:7/876: dread d7/d8/d12/d35/dda/fb0 [0,4194304] 0 2024-03-29T04:45:29.069 INFO:tasks.workunit.client.1.smithi152.stdout:7/877: chown d7/d8/d12/d1d/f98 12352 1 2024-03-29T04:45:29.071 INFO:tasks.workunit.client.1.smithi152.stdout:7/878: unlink d7/d8/db3/f8d 0 2024-03-29T04:45:29.075 INFO:tasks.workunit.client.1.smithi152.stdout:7/879: dread d7/d8/d12/d35/f32 [0,4194304] 0 2024-03-29T04:45:29.076 INFO:tasks.workunit.client.1.smithi152.stdout:7/880: write d7/fe1 [708987,71009] 0 2024-03-29T04:45:29.082 INFO:tasks.workunit.client.1.smithi152.stdout:7/881: truncate d7/d8/d12/d35/dba/df0/f80 95965 0 2024-03-29T04:45:29.089 INFO:tasks.workunit.client.1.smithi152.stdout:7/882: dwrite d7/d8/d12/d35/dda/fd6 [0,4194304] 0 2024-03-29T04:45:29.098 INFO:tasks.workunit.client.1.smithi152.stdout:7/883: dwrite d7/d8/d12/d35/d68/dc3/fe0 [0,4194304] 0 2024-03-29T04:45:29.101 INFO:tasks.workunit.client.1.smithi152.stdout:7/884: write d7/d8/db/df/f3a [4839981,103203] 0 2024-03-29T04:45:29.109 INFO:tasks.workunit.client.1.smithi152.stdout:7/885: truncate d7/d8/d12/f7b 880596 0 2024-03-29T04:45:29.111 INFO:tasks.workunit.client.1.smithi152.stdout:7/886: creat d7/d10e/f112 x:0 0 0 2024-03-29T04:45:29.113 INFO:tasks.workunit.client.1.smithi152.stdout:7/887: mkdir d7/d8/db3/d72/d113 0 2024-03-29T04:45:29.117 INFO:tasks.workunit.client.1.smithi152.stdout:7/888: rename d7/d8/d12/fbd to d7/d8/d12/d35/dba/df0/d99/f114 0 2024-03-29T04:45:29.119 INFO:tasks.workunit.client.1.smithi152.stdout:7/889: truncate d7/d8/db3/f50 959074 0 2024-03-29T04:45:29.132 INFO:tasks.workunit.client.1.smithi152.stdout:7/890: dwrite d7/d8/d12/d35/dba/df0/d99/feb [0,4194304] 0 2024-03-29T04:45:29.136 INFO:tasks.workunit.client.1.smithi152.stdout:7/891: symlink d7/d8/db3/d72/l115 0 2024-03-29T04:45:29.137 INFO:tasks.workunit.client.1.smithi152.stdout:7/892: symlink d7/d10e/l116 0 2024-03-29T04:45:29.137 INFO:tasks.workunit.client.1.smithi152.stdout:7/893: stat d7/d8/db/ca3 0 2024-03-29T04:45:29.139 INFO:tasks.workunit.client.1.smithi152.stdout:7/894: write d7/d8/d12/d1d/fde [943739,62911] 0 2024-03-29T04:45:29.140 INFO:tasks.workunit.client.1.smithi152.stdout:7/895: fdatasync d7/d8/d12/d35/dba/df0/fea 0 2024-03-29T04:45:29.141 INFO:tasks.workunit.client.1.smithi152.stdout:7/896: chown d7/f92 0 1 2024-03-29T04:45:29.143 INFO:tasks.workunit.client.1.smithi152.stdout:7/897: creat d7/d8/d12/d1d/d41/d45/f117 x:0 0 0 2024-03-29T04:45:29.146 INFO:tasks.workunit.client.1.smithi152.stdout:7/898: truncate d7/d8/d12/f7b 1319836 0 2024-03-29T04:45:29.292 INFO:tasks.workunit.client.1.smithi152.stdout:0/820: sync 2024-03-29T04:45:29.322 INFO:tasks.workunit.client.1.smithi152.stdout:0/821: mkdir d2/da/d17/dd8/db8/daf/ddb/d11e 0 2024-03-29T04:45:29.322 INFO:tasks.workunit.client.1.smithi152.stdout:0/822: stat d2/da/d17/dd8/db8/daf/fd7 0 2024-03-29T04:45:29.322 INFO:tasks.workunit.client.1.smithi152.stdout:0/823: write d2/da/d17/d57/f8c [8279485,46540] 0 2024-03-29T04:45:29.323 INFO:tasks.workunit.client.1.smithi152.stdout:0/824: symlink d2/da/df/dbb/l11f 0 2024-03-29T04:45:29.323 INFO:tasks.workunit.client.1.smithi152.stdout:0/825: rename d2/da/l12 to d2/da/d3a/d61/l120 0 2024-03-29T04:45:29.323 INFO:tasks.workunit.client.1.smithi152.stdout:0/826: mkdir d2/da/d17/d57/def/df2/d121 0 2024-03-29T04:45:29.448 INFO:tasks.workunit.client.1.smithi152.stdout:3/906: sync 2024-03-29T04:45:29.448 INFO:tasks.workunit.client.1.smithi152.stdout:2/795: sync 2024-03-29T04:45:29.448 INFO:tasks.workunit.client.1.smithi152.stdout:4/760: sync 2024-03-29T04:45:29.448 INFO:tasks.workunit.client.1.smithi152.stdout:8/959: sync 2024-03-29T04:45:29.449 INFO:tasks.workunit.client.1.smithi152.stdout:7/899: sync 2024-03-29T04:45:29.449 INFO:tasks.workunit.client.1.smithi152.stdout:9/912: sync 2024-03-29T04:45:29.452 INFO:tasks.workunit.client.1.smithi152.stdout:8/960: write d1/d2/d1d/dd3/db0/d40/f6c [2305264,92784] 0 2024-03-29T04:45:29.455 INFO:tasks.workunit.client.1.smithi152.stdout:8/961: stat d1/d2/d3c/l12e 0 2024-03-29T04:45:29.459 INFO:tasks.workunit.client.1.smithi152.stdout:2/796: dwrite d0/d3/d30/d15/d53/d54/dad/fba [0,4194304] 0 2024-03-29T04:45:29.460 INFO:tasks.workunit.client.1.smithi152.stdout:3/907: write d7/d46/f9e [396027,105710] 0 2024-03-29T04:45:29.461 INFO:tasks.workunit.client.1.smithi152.stdout:9/913: write d5/d8/d31/d59/d6f/dbf/df4/f120 [2609592,3346] 0 2024-03-29T04:45:29.461 INFO:tasks.workunit.client.1.smithi152.stdout:8/962: dread d1/d2/d1d/dd3/db0/da9/fbc [0,4194304] 0 2024-03-29T04:45:29.462 INFO:tasks.workunit.client.1.smithi152.stdout:2/797: truncate d0/d3/d64/d85/fe4 712114 0 2024-03-29T04:45:29.462 INFO:tasks.workunit.client.1.smithi152.stdout:7/900: truncate d7/d8/d12/f43 4600314 0 2024-03-29T04:45:29.464 INFO:tasks.workunit.client.1.smithi152.stdout:4/761: truncate d0/d7/d40/d23/d55/d70/d96/f9f 802838 0 2024-03-29T04:45:29.466 INFO:tasks.workunit.client.1.smithi152.stdout:3/908: dread d7/d46/f9e [0,4194304] 0 2024-03-29T04:45:29.466 INFO:tasks.workunit.client.1.smithi152.stdout:4/762: chown d0/d7/d40/d2a/f94 5 1 2024-03-29T04:45:29.467 INFO:tasks.workunit.client.1.smithi152.stdout:2/798: dread d0/d3/d30/d15/d53/db4/fec [0,4194304] 0 2024-03-29T04:45:29.467 INFO:tasks.workunit.client.1.smithi152.stdout:8/963: mknod d1/d2/d1d/dd3/db0/d8b/c157 0 2024-03-29T04:45:29.469 INFO:tasks.workunit.client.1.smithi152.stdout:9/914: creat d5/db/d92/dd2/d137/db1/dcd/d6e/dea/f145 x:0 0 0 2024-03-29T04:45:29.470 INFO:tasks.workunit.client.1.smithi152.stdout:8/964: chown d1/d2/c1e 19 1 2024-03-29T04:45:29.472 INFO:tasks.workunit.client.1.smithi152.stdout:2/799: read d0/d3/f10 [3118850,63801] 0 2024-03-29T04:45:29.473 INFO:tasks.workunit.client.1.smithi152.stdout:3/909: creat d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/f125 x:0 0 0 2024-03-29T04:45:29.474 INFO:tasks.workunit.client.1.smithi152.stdout:4/763: symlink d0/d7/d25/d17/d5e/d60/d74/da2/da6/dac/lf0 0 2024-03-29T04:45:29.475 INFO:tasks.workunit.client.1.smithi152.stdout:7/901: dwrite d7/d8/d12/d1d/d41/d45/f117 [0,4194304] 0 2024-03-29T04:45:29.475 INFO:tasks.workunit.client.1.smithi152.stdout:8/965: mknod d1/d2/d3c/c158 0 2024-03-29T04:45:29.477 INFO:tasks.workunit.client.1.smithi152.stdout:2/800: fsync d0/d3/d30/d15/da5/fe9 0 2024-03-29T04:45:29.478 INFO:tasks.workunit.client.1.smithi152.stdout:9/915: rename d5/d8/d31/d59/d6f/dbf to d5/db/d92/d136/d146 0 2024-03-29T04:45:29.485 INFO:tasks.workunit.client.1.smithi152.stdout:0/827: sync 2024-03-29T04:45:29.487 INFO:tasks.workunit.client.1.smithi152.stdout:4/764: creat d0/d27/d6e/d78/ff1 x:0 0 0 2024-03-29T04:45:29.488 INFO:tasks.workunit.client.1.smithi152.stdout:7/902: rmdir d7/d8/db/df/de5 39 2024-03-29T04:45:29.489 INFO:tasks.workunit.client.1.smithi152.stdout:3/910: truncate d7/d19/d1b/f58 90501 0 2024-03-29T04:45:29.490 INFO:tasks.workunit.client.1.smithi152.stdout:9/916: write d5/d8/d31/d59/d85/fdf [50695,122962] 0 2024-03-29T04:45:29.495 INFO:tasks.workunit.client.1.smithi152.stdout:9/917: write d5/db/d92/dd2/d137/db1/dcd/f78 [2603363,101942] 0 2024-03-29T04:45:29.496 INFO:tasks.workunit.client.1.smithi152.stdout:3/911: read - d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/f125 zero size 2024-03-29T04:45:29.496 INFO:tasks.workunit.client.1.smithi152.stdout:3/912: readlink d7/d19/d1b/d1e/da1/d41/d61/le5 0 2024-03-29T04:45:29.510 INFO:tasks.workunit.client.1.smithi152.stdout:3/913: dread d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/f49 [0,4194304] 0 2024-03-29T04:45:29.522 INFO:tasks.workunit.client.1.smithi152.stdout:8/966: getdents d1/d8a/d12b 0 2024-03-29T04:45:29.522 INFO:tasks.workunit.client.1.smithi152.stdout:2/801: fsync d0/d3/d30/d15/f2b 0 2024-03-29T04:45:29.523 INFO:tasks.workunit.client.1.smithi152.stdout:4/765: mknod d0/d7/db2/cf2 0 2024-03-29T04:45:29.524 INFO:tasks.workunit.client.1.smithi152.stdout:0/828: dread - d2/da/d17/dd8/f11b zero size 2024-03-29T04:45:29.526 INFO:tasks.workunit.client.1.smithi152.stdout:2/802: chown d0/d3/df/d8c/ca1 78728801 1 2024-03-29T04:45:29.527 INFO:tasks.workunit.client.1.smithi152.stdout:4/766: readlink d0/d7/d40/d62/lc6 0 2024-03-29T04:45:29.528 INFO:tasks.workunit.client.1.smithi152.stdout:9/918: write d5/db/d1b/d68/fe8 [828246,91338] 0 2024-03-29T04:45:29.532 INFO:tasks.workunit.client.1.smithi152.stdout:8/967: dwrite d1/d2/d3c/d76/d9a/def/d14e/f123 [0,4194304] 0 2024-03-29T04:45:29.533 INFO:tasks.workunit.client.1.smithi152.stdout:0/829: read d2/da/d3a/d61/fde [511754,10909] 0 2024-03-29T04:45:29.533 INFO:tasks.workunit.client.1.smithi152.stdout:3/914: creat d7/d6c/d10c/f126 x:0 0 0 2024-03-29T04:45:29.534 INFO:tasks.workunit.client.1.smithi152.stdout:2/803: creat d0/d3/d64/ff0 x:0 0 0 2024-03-29T04:45:29.537 INFO:tasks.workunit.client.1.smithi152.stdout:9/919: dread d5/db/d1b/f12c [0,4194304] 0 2024-03-29T04:45:29.539 INFO:tasks.workunit.client.1.smithi152.stdout:8/968: mknod d1/d2/d3/d150/c159 0 2024-03-29T04:45:29.541 INFO:tasks.workunit.client.1.smithi152.stdout:3/915: fdatasync d7/d19/d1b/d1e/da1/f47 0 2024-03-29T04:45:29.543 INFO:tasks.workunit.client.1.smithi152.stdout:2/804: write d0/d3/d30/d15/d22/d74/dbd/f95 [1831298,99023] 0 2024-03-29T04:45:29.547 INFO:tasks.workunit.client.1.smithi152.stdout:8/969: fdatasync d1/d2/d3/f29 0 2024-03-29T04:45:29.548 INFO:tasks.workunit.client.1.smithi152.stdout:3/916: mknod d7/d19/d1b/d1e/da1/d41/d61/d75/c127 0 2024-03-29T04:45:29.553 INFO:tasks.workunit.client.1.smithi152.stdout:8/970: truncate d1/d2/d1d/dc6/f133 382688 0 2024-03-29T04:45:29.553 INFO:tasks.workunit.client.1.smithi152.stdout:3/917: getdents d7/d19/d1b/d1e/db9 0 2024-03-29T04:45:29.555 INFO:tasks.workunit.client.1.smithi152.stdout:3/918: fsync d7/d19/d1b/f7e 0 2024-03-29T04:45:29.555 INFO:tasks.workunit.client.1.smithi152.stdout:8/971: read d1/d2/d1d/dd3/db0/f78 [306514,109070] 0 2024-03-29T04:45:29.557 INFO:tasks.workunit.client.1.smithi152.stdout:2/805: getdents d0/d3/df/d8c/d97 0 2024-03-29T04:45:29.558 INFO:tasks.workunit.client.1.smithi152.stdout:2/806: stat d0/d3/d30/d15/d53/d54/dad/dd5 0 2024-03-29T04:45:29.558 INFO:tasks.workunit.client.1.smithi152.stdout:3/919: mkdir d7/d19/d1b/d1e/d4e/d8f/d128 0 2024-03-29T04:45:29.560 INFO:tasks.workunit.client.1.smithi152.stdout:3/920: chown d7/d19/d1b/d1e/da1/d41/d6a/dcc 1617489 1 2024-03-29T04:45:29.560 INFO:tasks.workunit.client.1.smithi152.stdout:8/972: mknod d1/d2/d1d/dd3/db0/d40/d58/d136/c15a 0 2024-03-29T04:45:29.562 INFO:tasks.workunit.client.1.smithi152.stdout:8/973: chown d1/d2/d1d/dd3/db0/f24 2017345 1 2024-03-29T04:45:29.562 INFO:tasks.workunit.client.1.smithi152.stdout:2/807: dread d0/d3/d30/d15/d22/d4c/f90 [0,4194304] 0 2024-03-29T04:45:29.562 INFO:tasks.workunit.client.1.smithi152.stdout:3/921: mkdir d7/d19/d1b/d1e/da1/d41/d6a/d78/d129 0 2024-03-29T04:45:29.565 INFO:tasks.workunit.client.1.smithi152.stdout:8/974: mknod d1/c15b 0 2024-03-29T04:45:29.566 INFO:tasks.workunit.client.1.smithi152.stdout:3/922: rmdir d7/d19/d1b/d1e/da1/d41/d61/d63 39 2024-03-29T04:45:29.566 INFO:tasks.workunit.client.1.smithi152.stdout:8/975: chown d1/dc5/d149/fd6 14508 1 2024-03-29T04:45:29.568 INFO:tasks.workunit.client.1.smithi152.stdout:8/976: truncate d1/d2/d1d/d36/d107/d14a/f100 175417 0 2024-03-29T04:45:29.570 INFO:tasks.workunit.client.1.smithi152.stdout:3/923: link d7/d6c/ceb d7/c12a 0 2024-03-29T04:45:29.575 INFO:tasks.workunit.client.1.smithi152.stdout:4/767: read d0/d7/d40/d23/d55/d70/d96/f9f [750997,870] 0 2024-03-29T04:45:29.578 INFO:tasks.workunit.client.1.smithi152.stdout:0/830: write d2/f8 [4039143,69320] 0 2024-03-29T04:45:29.579 INFO:tasks.workunit.client.1.smithi152.stdout:7/903: dwrite d7/d8/d12/d35/d68/f88 [0,4194304] 0 2024-03-29T04:45:29.581 INFO:tasks.workunit.client.1.smithi152.stdout:4/768: symlink d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/de7/lf3 0 2024-03-29T04:45:29.584 INFO:tasks.workunit.client.1.smithi152.stdout:0/831: creat d2/da/d17/d57/d9c/f122 x:0 0 0 2024-03-29T04:45:29.584 INFO:tasks.workunit.client.1.smithi152.stdout:2/808: write d0/d3/d30/d15/da5/fd7 [1047104,69324] 0 2024-03-29T04:45:29.586 INFO:tasks.workunit.client.1.smithi152.stdout:8/977: write d1/d2/d1d/f35 [663271,89717] 0 2024-03-29T04:45:29.587 INFO:tasks.workunit.client.1.smithi152.stdout:7/904: mknod d7/d8/d12/d35/dda/c118 0 2024-03-29T04:45:29.588 INFO:tasks.workunit.client.1.smithi152.stdout:2/809: fdatasync d0/d3/df/db8/fc0 0 2024-03-29T04:45:29.588 INFO:tasks.workunit.client.1.smithi152.stdout:9/920: dwrite d5/db/d1b/f49 [0,4194304] 0 2024-03-29T04:45:29.591 INFO:tasks.workunit.client.1.smithi152.stdout:7/905: chown d7/d8/d12/d35/dba/df0 115 1 2024-03-29T04:45:29.593 INFO:tasks.workunit.client.1.smithi152.stdout:0/832: dread d2/da/d17/d57/f73 [0,4194304] 0 2024-03-29T04:45:29.593 INFO:tasks.workunit.client.1.smithi152.stdout:7/906: write d7/d8/db/df/f3a [1012046,81863] 0 2024-03-29T04:45:29.595 INFO:tasks.workunit.client.1.smithi152.stdout:9/921: dread - d5/db/d92/dd2/d137/db1/dcd/d6e/db4/f131 zero size 2024-03-29T04:45:29.601 INFO:tasks.workunit.client.1.smithi152.stdout:7/907: link d7/d8/d12/d35/dda/fd6 d7/d8/d12/d35/dba/df0/f119 0 2024-03-29T04:45:29.605 INFO:tasks.workunit.client.1.smithi152.stdout:2/810: write d0/d3/d5f/f63 [2039359,82810] 0 2024-03-29T04:45:29.606 INFO:tasks.workunit.client.1.smithi152.stdout:0/833: dwrite d2/da/d3a/fd2 [0,4194304] 0 2024-03-29T04:45:29.608 INFO:tasks.workunit.client.1.smithi152.stdout:9/922: dwrite d5/db/d92/d136/d146/df4/f120 [4194304,4194304] 0 2024-03-29T04:45:29.609 INFO:tasks.workunit.client.1.smithi152.stdout:7/908: write d7/d8/d12/d35/dba/df0/d99/fad [2013640,66166] 0 2024-03-29T04:45:29.610 INFO:tasks.workunit.client.1.smithi152.stdout:9/923: readlink d5/l104 0 2024-03-29T04:45:29.613 INFO:tasks.workunit.client.1.smithi152.stdout:2/811: creat d0/d3/df/d2c/d45/ff1 x:0 0 0 2024-03-29T04:45:29.613 INFO:tasks.workunit.client.1.smithi152.stdout:4/769: dwrite d0/d7/d25/f21 [0,4194304] 0 2024-03-29T04:45:29.616 INFO:tasks.workunit.client.1.smithi152.stdout:0/834: symlink d2/da/d17/dd8/db8/daf/ddb/l123 0 2024-03-29T04:45:29.616 INFO:tasks.workunit.client.1.smithi152.stdout:4/770: write d0/d7/d25/d17/d5e/fef [332584,83461] 0 2024-03-29T04:45:29.619 INFO:tasks.workunit.client.1.smithi152.stdout:9/924: unlink d5/db/d92/dd2/d137/db1/dcd/f50 0 2024-03-29T04:45:29.620 INFO:tasks.workunit.client.1.smithi152.stdout:9/925: chown d5/db/d1b/d68 132432 1 2024-03-29T04:45:29.622 INFO:tasks.workunit.client.1.smithi152.stdout:4/771: creat d0/d7/d40/d23/dda/ff4 x:0 0 0 2024-03-29T04:45:29.622 INFO:tasks.workunit.client.1.smithi152.stdout:9/926: write d5/d8/ff9 [722045,45546] 0 2024-03-29T04:45:29.623 INFO:tasks.workunit.client.1.smithi152.stdout:2/812: mknod d0/d3/df/cf2 0 2024-03-29T04:45:29.629 INFO:tasks.workunit.client.1.smithi152.stdout:4/772: rename d0/d7/d40/d23/d55/d6c/d91/d93/fd4 to d0/d7/db2/ff5 0 2024-03-29T04:45:29.631 INFO:tasks.workunit.client.1.smithi152.stdout:2/813: creat d0/d3/d30/d15/d22/d4c/d5a/dd6/ff3 x:0 0 0 2024-03-29T04:45:29.633 INFO:tasks.workunit.client.1.smithi152.stdout:4/773: link d0/d7/d25/d17/d5e/d60/d74/fdf d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/ff6 0 2024-03-29T04:45:29.638 INFO:tasks.workunit.client.1.smithi152.stdout:4/774: creat d0/d7/d40/d23/d55/d70/d96/dc0/ff7 x:0 0 0 2024-03-29T04:45:29.639 INFO:tasks.workunit.client.1.smithi152.stdout:2/814: link d0/d3/d64/d85/db3/lda d0/d3/d30/d15/lf4 0 2024-03-29T04:45:29.640 INFO:tasks.workunit.client.1.smithi152.stdout:0/835: write d2/da/ff7 [121952,95994] 0 2024-03-29T04:45:29.642 INFO:tasks.workunit.client.1.smithi152.stdout:7/909: dwrite d7/d8/db3/f97 [0,4194304] 0 2024-03-29T04:45:29.645 INFO:tasks.workunit.client.1.smithi152.stdout:9/927: write d5/db/f4e [2409632,103423] 0 2024-03-29T04:45:29.648 INFO:tasks.workunit.client.1.smithi152.stdout:4/775: rename d0/d7/d40/d75 to d0/d7/d40/d23/d55/d70/d96/dc0/df8 0 2024-03-29T04:45:29.650 INFO:tasks.workunit.client.1.smithi152.stdout:0/836: creat d2/d29/dcf/f124 x:0 0 0 2024-03-29T04:45:29.653 INFO:tasks.workunit.client.1.smithi152.stdout:0/837: dread - d2/da/d17/d57/def/f10e zero size 2024-03-29T04:45:29.654 INFO:tasks.workunit.client.1.smithi152.stdout:2/815: dwrite d0/d3/d30/d15/d22/d4c/f51 [4194304,4194304] 0 2024-03-29T04:45:29.655 INFO:tasks.workunit.client.1.smithi152.stdout:4/776: dread - d0/d7/d25/d17/d5e/d60/f9b zero size 2024-03-29T04:45:29.658 INFO:tasks.workunit.client.1.smithi152.stdout:9/928: symlink d5/db/d92/dd2/d137/db1/dcd/d6e/d91/dd1/l147 0 2024-03-29T04:45:29.659 INFO:tasks.workunit.client.1.smithi152.stdout:0/838: read d2/feb [3010422,46486] 0 2024-03-29T04:45:29.662 INFO:tasks.workunit.client.1.smithi152.stdout:2/816: mkdir d0/d3/d30/d15/da5/df5 0 2024-03-29T04:45:29.663 INFO:tasks.workunit.client.1.smithi152.stdout:4/777: dread d0/d7/d40/d23/d55/d70/d96/fde [0,4194304] 0 2024-03-29T04:45:29.665 INFO:tasks.workunit.client.1.smithi152.stdout:7/910: dread d7/d8/d12/d35/dda/f3c [0,4194304] 0 2024-03-29T04:45:29.667 INFO:tasks.workunit.client.1.smithi152.stdout:0/839: fdatasync d2/da/d3a/d61/db4/d11a/ff8 0 2024-03-29T04:45:29.668 INFO:tasks.workunit.client.1.smithi152.stdout:7/911: creat d7/f11a x:0 0 0 2024-03-29T04:45:29.668 INFO:tasks.workunit.client.1.smithi152.stdout:9/929: getdents d5/db/d92/dd2/d137/db1/dcd/d6e/d91/de1 0 2024-03-29T04:45:29.669 INFO:tasks.workunit.client.1.smithi152.stdout:4/778: write d0/d7/d25/d17/d5e/d60/f8b [1130894,98139] 0 2024-03-29T04:45:29.670 INFO:tasks.workunit.client.1.smithi152.stdout:2/817: read d0/d3/d30/d15/f26 [1821,116268] 0 2024-03-29T04:45:29.671 INFO:tasks.workunit.client.1.smithi152.stdout:0/840: mknod d2/da/c125 0 2024-03-29T04:45:29.671 INFO:tasks.workunit.client.1.smithi152.stdout:7/912: write d7/d8/d12/fb6 [822776,29673] 0 2024-03-29T04:45:29.674 INFO:tasks.workunit.client.1.smithi152.stdout:2/818: readlink d0/d3/df/d2c/l50 0 2024-03-29T04:45:29.676 INFO:tasks.workunit.client.1.smithi152.stdout:2/819: readlink d0/d3/d30/d15/d22/d74/dbd/l7d 0 2024-03-29T04:45:29.676 INFO:tasks.workunit.client.1.smithi152.stdout:7/913: mkdir d7/d8/db/df/de5/d11b 0 2024-03-29T04:45:29.679 INFO:tasks.workunit.client.1.smithi152.stdout:9/930: link d5/db/d92/dd2/d137/db1/dcd/d6e/d73/d8e/l81 d5/db/d92/dd2/d137/db1/dcd/d6e/db4/d110/l148 0 2024-03-29T04:45:29.682 INFO:tasks.workunit.client.1.smithi152.stdout:0/841: dwrite d2/da/d17/dd8/fba [0,4194304] 0 2024-03-29T04:45:29.685 INFO:tasks.workunit.client.1.smithi152.stdout:7/914: link d7/d8/d12/d1d/d41/d83/ff7 d7/d8/d12/d35/dba/d10c/f11c 0 2024-03-29T04:45:29.688 INFO:tasks.workunit.client.1.smithi152.stdout:2/820: write d0/d3/d30/f1a [1366662,50674] 0 2024-03-29T04:45:29.688 INFO:tasks.workunit.client.1.smithi152.stdout:7/915: dread - d7/d8/db3/fa6 zero size 2024-03-29T04:45:29.689 INFO:tasks.workunit.client.1.smithi152.stdout:0/842: unlink d2/da/l108 0 2024-03-29T04:45:29.690 INFO:tasks.workunit.client.1.smithi152.stdout:9/931: link d5/db/d92/dd2/d137/db1/dcd/d6e/d91/de1/cf8 d5/db/d92/dd2/d137/db1/dcd/d6e/d91/dd1/c149 0 2024-03-29T04:45:29.692 INFO:tasks.workunit.client.1.smithi152.stdout:4/779: dwrite d0/f15 [0,4194304] 0 2024-03-29T04:45:29.692 INFO:tasks.workunit.client.1.smithi152.stdout:7/916: write d7/d8/d12/d35/dda/f9b [4153652,51318] 0 2024-03-29T04:45:29.693 INFO:tasks.workunit.client.1.smithi152.stdout:2/821: mkdir d0/d3/d30/d15/d22/d4c/d5a/df6 0 2024-03-29T04:45:29.695 INFO:tasks.workunit.client.1.smithi152.stdout:0/843: creat d2/da/d17/d57/d9c/f126 x:0 0 0 2024-03-29T04:45:29.697 INFO:tasks.workunit.client.1.smithi152.stdout:4/780: mknod d0/d7/d40/d23/d6a/cf9 0 2024-03-29T04:45:29.697 INFO:tasks.workunit.client.1.smithi152.stdout:7/917: chown d7/d8/d12/d1d/lbb 52 1 2024-03-29T04:45:29.697 INFO:tasks.workunit.client.1.smithi152.stdout:2/822: readlink d0/d3/d30/d15/d22/d74/l78 0 2024-03-29T04:45:29.702 INFO:tasks.workunit.client.1.smithi152.stdout:9/932: rmdir d5/db/d1b/d68/daa 39 2024-03-29T04:45:29.702 INFO:tasks.workunit.client.1.smithi152.stdout:4/781: chown d0/d7/d40/d23/d55/d70/d96/fe8 1230837274 1 2024-03-29T04:45:29.704 INFO:tasks.workunit.client.1.smithi152.stdout:9/933: read d5/f5c [117408,124144] 0 2024-03-29T04:45:29.705 INFO:tasks.workunit.client.1.smithi152.stdout:9/934: write d5/db/d92/dd2/d137/db1/dcd/f100 [853573,12632] 0 2024-03-29T04:45:29.706 INFO:tasks.workunit.client.1.smithi152.stdout:7/918: rename d7/d8/d12/d35/dba/df0/d99/f108 to d7/d8/db3/d72/f11d 0 2024-03-29T04:45:29.707 INFO:tasks.workunit.client.1.smithi152.stdout:4/782: rmdir d0/d7 39 2024-03-29T04:45:29.711 INFO:tasks.workunit.client.1.smithi152.stdout:7/919: mknod d7/d8/d12/d1d/d41/c11e 0 2024-03-29T04:45:29.713 INFO:tasks.workunit.client.1.smithi152.stdout:9/935: getdents d5/db/d92 0 2024-03-29T04:45:29.714 INFO:tasks.workunit.client.1.smithi152.stdout:7/920: creat d7/d8/d37/f11f x:0 0 0 2024-03-29T04:45:29.716 INFO:tasks.workunit.client.1.smithi152.stdout:2/823: dwrite d0/d3/df/f23 [0,4194304] 0 2024-03-29T04:45:29.716 INFO:tasks.workunit.client.1.smithi152.stdout:7/921: dread - d7/d8/d37/f10a zero size 2024-03-29T04:45:29.718 INFO:tasks.workunit.client.1.smithi152.stdout:2/824: stat d0/d3/d30/d15/d22/d74/dbd/led 0 2024-03-29T04:45:29.721 INFO:tasks.workunit.client.1.smithi152.stdout:9/936: mkdir d5/d8/d31/d14a 0 2024-03-29T04:45:29.722 INFO:tasks.workunit.client.1.smithi152.stdout:7/922: fsync d7/d8/d12/f7b 0 2024-03-29T04:45:29.734 INFO:tasks.workunit.client.1.smithi152.stdout:9/937: dread d5/db/d92/dd2/d137/db1/dcd/f3f [0,4194304] 0 2024-03-29T04:45:29.735 INFO:tasks.workunit.client.1.smithi152.stdout:4/783: dwrite d0/d7/d25/d17/d5e/d60/d74/da2/da6/fa9 [0,4194304] 0 2024-03-29T04:45:29.737 INFO:tasks.workunit.client.1.smithi152.stdout:9/938: chown d5/d12/fd9 13 1 2024-03-29T04:45:29.741 INFO:tasks.workunit.client.1.smithi152.stdout:7/923: dwrite d7/ff5 [0,4194304] 0 2024-03-29T04:45:29.741 INFO:tasks.workunit.client.1.smithi152.stdout:4/784: dread - d0/d27/d6e/d8c/fe4 zero size 2024-03-29T04:45:29.746 INFO:tasks.workunit.client.1.smithi152.stdout:2/825: dread d0/d3/f10 [0,4194304] 0 2024-03-29T04:45:29.747 INFO:tasks.workunit.client.1.smithi152.stdout:7/924: mkdir d7/d8/d12/de2/d120 0 2024-03-29T04:45:29.748 INFO:tasks.workunit.client.1.smithi152.stdout:9/939: dread d5/db/d92/dd2/d137/db1/fbc [0,4194304] 0 2024-03-29T04:45:29.748 INFO:tasks.workunit.client.1.smithi152.stdout:7/925: stat d7/d8/d12/d35/f6d 0 2024-03-29T04:45:29.749 INFO:tasks.workunit.client.1.smithi152.stdout:4/785: symlink d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/de7/lfa 0 2024-03-29T04:45:29.751 INFO:tasks.workunit.client.1.smithi152.stdout:7/926: write d7/d8/d12/d1d/d41/d45/f117 [2621375,99149] 0 2024-03-29T04:45:29.753 INFO:tasks.workunit.client.1.smithi152.stdout:7/927: chown d7/d8/d12/d35/dba/df0/fa9 55049 1 2024-03-29T04:45:29.754 INFO:tasks.workunit.client.1.smithi152.stdout:4/786: creat d0/d7/d40/dce/ffb x:0 0 0 2024-03-29T04:45:29.757 INFO:tasks.workunit.client.1.smithi152.stdout:7/928: mkdir d7/d8/db3/d121 0 2024-03-29T04:45:29.760 INFO:tasks.workunit.client.1.smithi152.stdout:7/929: mknod d7/d8/d12/d35/dba/df0/d99/c122 0 2024-03-29T04:45:29.761 INFO:tasks.workunit.client.1.smithi152.stdout:7/930: rmdir d7/d8/d12/d35/d68/d84 39 2024-03-29T04:45:29.764 INFO:tasks.workunit.client.1.smithi152.stdout:7/931: truncate d7/d8/d12/d35/d68/dc3/fe4 4668605 0 2024-03-29T04:45:29.768 INFO:tasks.workunit.client.1.smithi152.stdout:7/932: symlink d7/d8/d12/d35/dba/df0/d99/df9/l123 0 2024-03-29T04:45:29.768 INFO:tasks.workunit.client.1.smithi152.stdout:9/940: dwrite d5/db/fd6 [0,4194304] 0 2024-03-29T04:45:29.776 INFO:tasks.workunit.client.1.smithi152.stdout:7/933: truncate d7/d8/d12/d35/d68/d84/ff3 4800612 0 2024-03-29T04:45:29.777 INFO:tasks.workunit.client.1.smithi152.stdout:9/941: truncate d5/d8/d31/d59/d85/fe6 845572 0 2024-03-29T04:45:29.778 INFO:tasks.workunit.client.1.smithi152.stdout:7/934: write d7/d8/d37/f7d [490212,119302] 0 2024-03-29T04:45:29.780 INFO:tasks.workunit.client.1.smithi152.stdout:7/935: creat d7/d8/d12/de2/f124 x:0 0 0 2024-03-29T04:45:29.788 INFO:tasks.workunit.client.1.smithi152.stdout:7/936: write d7/d8/d12/d35/dba/df0/fb4 [121695,24421] 0 2024-03-29T04:45:29.791 INFO:tasks.workunit.client.1.smithi152.stdout:7/937: unlink d7/d8/d12/d1d/d41/ld9 0 2024-03-29T04:45:29.796 INFO:tasks.workunit.client.1.smithi152.stdout:7/938: link d7/d8/f6e d7/d8/d12/d35/d68/d84/f125 0 2024-03-29T04:45:29.799 INFO:tasks.workunit.client.1.smithi152.stdout:9/942: dread d5/db/f6c [0,4194304] 0 2024-03-29T04:45:29.807 INFO:tasks.workunit.client.1.smithi152.stdout:9/943: dwrite d5/db/fd6 [0,4194304] 0 2024-03-29T04:45:29.811 INFO:tasks.workunit.client.1.smithi152.stdout:7/939: dread d7/d8/d12/d35/dda/f9b [0,4194304] 0 2024-03-29T04:45:29.814 INFO:tasks.workunit.client.1.smithi152.stdout:9/944: dread d5/f20 [0,4194304] 0 2024-03-29T04:45:29.820 INFO:tasks.workunit.client.1.smithi152.stdout:7/940: write d7/d8/d12/d1d/d41/d83/ff7 [2443233,120386] 0 2024-03-29T04:45:29.822 INFO:tasks.workunit.client.1.smithi152.stdout:7/941: write d7/d8/d12/de2/f124 [185145,94524] 0 2024-03-29T04:45:29.825 INFO:tasks.workunit.client.1.smithi152.stdout:9/945: dread d5/db/d92/dd2/d137/db1/dcd/d6e/d73/d8e/fd4 [0,4194304] 0 2024-03-29T04:45:29.825 INFO:tasks.workunit.client.1.smithi152.stdout:7/942: mkdir d7/d8/d12/d35/dba/df0/d99/df9/d126 0 2024-03-29T04:45:29.830 INFO:tasks.workunit.client.1.smithi152.stdout:9/946: write d5/db/d92/dd2/d137/db1/dcd/d6e/d73/d8e/f74 [2603774,60176] 0 2024-03-29T04:45:29.831 INFO:tasks.workunit.client.1.smithi152.stdout:7/943: rename d7/d8/db3/f31 to d7/d8/d12/d35/dba/df0/d99/f127 0 2024-03-29T04:45:29.835 INFO:tasks.workunit.client.1.smithi152.stdout:9/947: truncate d5/d8/d31/d59/d6f/fce 618407 0 2024-03-29T04:45:29.838 INFO:tasks.workunit.client.1.smithi152.stdout:9/948: creat d5/db/d92/dd2/d137/f14b x:0 0 0 2024-03-29T04:45:29.841 INFO:tasks.workunit.client.1.smithi152.stdout:9/949: chown d5/d12/c4b 6 1 2024-03-29T04:45:29.843 INFO:tasks.workunit.client.1.smithi152.stdout:9/950: fsync d5/db/d92/dd2/d137/db1/dcd/f63 0 2024-03-29T04:45:29.847 INFO:tasks.workunit.client.1.smithi152.stdout:9/951: symlink d5/db/d92/dd2/d137/db1/dcd/d6e/d73/d8e/l14c 0 2024-03-29T04:45:29.850 INFO:tasks.workunit.client.1.smithi152.stdout:9/952: mknod d5/db/d92/dd2/d137/db1/dcd/d6e/db4/df2/c14d 0 2024-03-29T04:45:29.853 INFO:tasks.workunit.client.1.smithi152.stdout:9/953: mkdir d5/db/d14e 0 2024-03-29T04:45:29.860 INFO:tasks.workunit.client.1.smithi152.stdout:7/944: dread d7/d8/f9c [0,4194304] 0 2024-03-29T04:45:29.861 INFO:tasks.workunit.client.1.smithi152.stdout:9/954: dwrite d5/d8/f9 [0,4194304] 0 2024-03-29T04:45:29.862 INFO:tasks.workunit.client.1.smithi152.stdout:7/945: write d7/f92 [1858647,93017] 0 2024-03-29T04:45:29.867 INFO:tasks.workunit.client.1.smithi152.stdout:7/946: creat d7/d8/db/f128 x:0 0 0 2024-03-29T04:45:29.872 INFO:tasks.workunit.client.1.smithi152.stdout:9/955: dwrite d5/db/d92/dd2/d137/db1/dcd/f78 [0,4194304] 0 2024-03-29T04:45:29.875 INFO:tasks.workunit.client.1.smithi152.stdout:9/956: symlink d5/d8/d31/d59/d6f/d10b/l14f 0 2024-03-29T04:45:29.883 INFO:tasks.workunit.client.1.smithi152.stdout:7/947: write d7/d8/fa [5013561,123214] 0 2024-03-29T04:45:29.888 INFO:tasks.workunit.client.1.smithi152.stdout:7/948: rename d7/d8/db/df/f58 to d7/d8/d12/de2/f129 0 2024-03-29T04:45:29.889 INFO:tasks.workunit.client.1.smithi152.stdout:7/949: chown d7/d8/d12/d1d/d41/lcf 866541 1 2024-03-29T04:45:29.895 INFO:tasks.workunit.client.1.smithi152.stdout:7/950: link d7/d8/d12/f43 d7/d8/d12/d35/dda/f12a 0 2024-03-29T04:45:29.899 INFO:tasks.workunit.client.1.smithi152.stdout:9/957: dread d5/db/f4e [0,4194304] 0 2024-03-29T04:45:29.900 INFO:tasks.workunit.client.1.smithi152.stdout:7/951: link d7/d8/db3/d72/f109 d7/d8/db3/f12b 0 2024-03-29T04:45:29.901 INFO:tasks.workunit.client.1.smithi152.stdout:7/952: readlink d7/d8/dcd/le9 0 2024-03-29T04:45:29.902 INFO:tasks.workunit.client.1.smithi152.stdout:9/958: read - d5/db/d92/dd2/d137/db1/dcd/d6e/dea/f145 zero size 2024-03-29T04:45:29.907 INFO:tasks.workunit.client.1.smithi152.stdout:9/959: rename d5/db/ca9 to d5/d8/d31/d59/dc3/d125/c150 0 2024-03-29T04:45:29.908 INFO:tasks.workunit.client.1.smithi152.stdout:9/960: chown d5/db/cc0 1789 1 2024-03-29T04:45:29.909 INFO:tasks.workunit.client.1.smithi152.stdout:9/961: fsync d5/d8/d31/d59/d85/fe9 0 2024-03-29T04:45:29.913 INFO:tasks.workunit.client.1.smithi152.stdout:7/953: write d7/d8/db3/f90 [939144,65119] 0 2024-03-29T04:45:29.916 INFO:tasks.workunit.client.1.smithi152.stdout:7/954: rmdir d7/d8 39 2024-03-29T04:45:29.919 INFO:tasks.workunit.client.1.smithi152.stdout:7/955: rmdir d7/d8/d12/d35/dda 39 2024-03-29T04:45:29.921 INFO:tasks.workunit.client.1.smithi152.stdout:7/956: readlink d7/d8/d12/d1d/d41/lc2 0 2024-03-29T04:45:29.921 INFO:tasks.workunit.client.1.smithi152.stdout:9/962: write d5/f7 [1210717,86080] 0 2024-03-29T04:45:29.925 INFO:tasks.workunit.client.1.smithi152.stdout:7/957: link d7/d8/d12/d35/lf8 d7/d8/d12/d35/d68/l12c 0 2024-03-29T04:45:29.928 INFO:tasks.workunit.client.1.smithi152.stdout:7/958: symlink d7/d8/d12/d35/dba/df0/l12d 0 2024-03-29T04:45:29.929 INFO:tasks.workunit.client.1.smithi152.stdout:9/963: dread d5/db/d1b/f95 [0,4194304] 0 2024-03-29T04:45:29.929 INFO:tasks.workunit.client.1.smithi152.stdout:9/964: stat d5/d12 0 2024-03-29T04:45:29.931 INFO:tasks.workunit.client.1.smithi152.stdout:7/959: creat d7/d8/d12/d35/dda/f12e x:0 0 0 2024-03-29T04:45:29.931 INFO:tasks.workunit.client.1.smithi152.stdout:9/965: write d5/db/d1b/fef [8641,5509] 0 2024-03-29T04:45:29.933 INFO:tasks.workunit.client.1.smithi152.stdout:7/960: unlink d7/d8/d12/f33 0 2024-03-29T04:45:29.935 INFO:tasks.workunit.client.1.smithi152.stdout:9/966: symlink d5/l151 0 2024-03-29T04:45:29.936 INFO:tasks.workunit.client.1.smithi152.stdout:7/961: creat d7/d8/d12/d35/dba/df0/d99/f12f x:0 0 0 2024-03-29T04:45:29.936 INFO:tasks.workunit.client.1.smithi152.stdout:9/967: creat d5/db/d1b/d68/dca/f152 x:0 0 0 2024-03-29T04:45:29.937 INFO:tasks.workunit.client.1.smithi152.stdout:9/968: write d5/f7 [5220263,63023] 0 2024-03-29T04:45:29.938 INFO:tasks.workunit.client.1.smithi152.stdout:7/962: rmdir d7/d8/d12/de2 39 2024-03-29T04:45:29.940 INFO:tasks.workunit.client.1.smithi152.stdout:7/963: fsync d7/d8/d12/d1d/d41/d45/f117 0 2024-03-29T04:45:29.946 INFO:tasks.workunit.client.1.smithi152.stdout:9/969: write d5/db/d92/dd2/d137/db1/dcd/d6e/d73/d7a/fac [1498230,117451] 0 2024-03-29T04:45:29.947 INFO:tasks.workunit.client.1.smithi152.stdout:7/964: dwrite d7/d8/db/df/f3a [4194304,4194304] 0 2024-03-29T04:45:29.947 INFO:tasks.workunit.client.1.smithi152.stdout:9/970: stat d5/d12/f141 0 2024-03-29T04:45:29.948 INFO:tasks.workunit.client.1.smithi152.stdout:7/965: dread - d7/d8/d12/d1d/f8c zero size 2024-03-29T04:45:29.951 INFO:tasks.workunit.client.1.smithi152.stdout:9/971: mkdir d5/db/d92/dd2/d137/db1/d153 0 2024-03-29T04:45:29.952 INFO:tasks.workunit.client.1.smithi152.stdout:7/966: write d7/d8/d12/f87 [1678841,99794] 0 2024-03-29T04:45:29.955 INFO:tasks.workunit.client.1.smithi152.stdout:7/967: unlink d7/d8/d12/d35/dba/l10b 0 2024-03-29T04:45:29.959 INFO:tasks.workunit.client.1.smithi152.stdout:7/968: write d7/d8/d12/d35/dba/df0/d99/fad [2606593,79083] 0 2024-03-29T04:45:29.960 INFO:tasks.workunit.client.1.smithi152.stdout:9/972: dwrite d5/db/d1b/d68/daa/fb6 [0,4194304] 0 2024-03-29T04:45:29.962 INFO:tasks.workunit.client.1.smithi152.stdout:7/969: chown d7/d8/d12/d35/dda/c7a 985925 1 2024-03-29T04:45:29.964 INFO:tasks.workunit.client.1.smithi152.stdout:9/973: mkdir d5/d8/d31/d59/d85/d154 0 2024-03-29T04:45:29.966 INFO:tasks.workunit.client.1.smithi152.stdout:9/974: truncate d5/db/d92/dd2/d137/db1/f143 495018 0 2024-03-29T04:45:29.971 INFO:tasks.workunit.client.1.smithi152.stdout:9/975: dwrite d5/d8/f9 [0,4194304] 0 2024-03-29T04:45:29.973 INFO:tasks.workunit.client.1.smithi152.stdout:9/976: chown d5/d8/d31/d59/d6f/f71 197029 1 2024-03-29T04:45:29.981 INFO:tasks.workunit.client.1.smithi152.stdout:9/977: link d5/d8/d31/d59/dc3/d125/c150 d5/db/d1b/d68/c155 0 2024-03-29T04:45:29.983 INFO:tasks.workunit.client.1.smithi152.stdout:9/978: mknod d5/db/d92/dd2/d137/db1/dcd/dc4/c156 0 2024-03-29T04:45:29.984 INFO:tasks.workunit.client.1.smithi152.stdout:9/979: chown d5/db/f60 902823 1 2024-03-29T04:45:29.986 INFO:tasks.workunit.client.1.smithi152.stdout:9/980: creat d5/db/d92/d136/d146/f157 x:0 0 0 2024-03-29T04:45:29.990 INFO:tasks.workunit.client.1.smithi152.stdout:7/970: dread d7/d8/d12/d1d/d41/d45/f6a [0,4194304] 0 2024-03-29T04:45:29.992 INFO:tasks.workunit.client.1.smithi152.stdout:9/981: dwrite d5/d8/d31/d59/d6f/f139 [0,4194304] 0 2024-03-29T04:45:29.993 INFO:tasks.workunit.client.1.smithi152.stdout:7/971: stat d7/d8/db/c52 0 2024-03-29T04:45:29.995 INFO:tasks.workunit.client.1.smithi152.stdout:7/972: dread d7/f9d [0,4194304] 0 2024-03-29T04:45:29.995 INFO:tasks.workunit.client.1.smithi152.stdout:9/982: chown d5/d8/d31/d59/d6f/d10b/f12f 304 1 2024-03-29T04:45:30.000 INFO:tasks.workunit.client.1.smithi152.stdout:9/983: creat d5/db/d92/f158 x:0 0 0 2024-03-29T04:45:30.003 INFO:tasks.workunit.client.1.smithi152.stdout:9/984: mknod d5/d8/d31/d59/d6f/c159 0 2024-03-29T04:45:30.005 INFO:tasks.workunit.client.1.smithi152.stdout:9/985: mkdir d5/d15a 0 2024-03-29T04:45:30.008 INFO:tasks.workunit.client.1.smithi152.stdout:7/973: dwrite d7/d8/d12/d1d/fed [0,4194304] 0 2024-03-29T04:45:30.012 INFO:tasks.workunit.client.1.smithi152.stdout:7/974: truncate d7/d8/d12/d1d/d41/d45/f117 4691628 0 2024-03-29T04:45:30.015 INFO:tasks.workunit.client.1.smithi152.stdout:7/975: dread - d7/d8/d12/f9f zero size 2024-03-29T04:45:30.015 INFO:tasks.workunit.client.1.smithi152.stdout:9/986: dwrite d5/db/d1b/d68/dca/f13e [0,4194304] 0 2024-03-29T04:45:30.019 INFO:tasks.workunit.client.1.smithi152.stdout:9/987: truncate d5/db/d92/dd2/d137/db1/dcd/f78 4500299 0 2024-03-29T04:45:30.020 INFO:tasks.workunit.client.1.smithi152.stdout:9/988: truncate d5/db/d1b/d68/fe8 971709 0 2024-03-29T04:45:30.022 INFO:tasks.workunit.client.1.smithi152.stdout:7/976: link d7/d8/d12/d1d/d41/d45/f117 d7/d8/d12/de2/d120/f130 0 2024-03-29T04:45:30.022 INFO:tasks.workunit.client.1.smithi152.stdout:9/989: write d5/db/d92/dd2/d137/db1/dcd/d6e/d73/d8e/f74 [3856346,3080] 0 2024-03-29T04:45:30.027 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:29 smithi152 ceph-mon[37177]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:45:30.027 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:29 smithi152 ceph-mon[37177]: from='client.14642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:45:30.031 INFO:tasks.workunit.client.1.smithi152.stdout:9/990: dwrite d5/fb5 [0,4194304] 0 2024-03-29T04:45:30.036 INFO:tasks.workunit.client.1.smithi152.stdout:7/977: dwrite d7/d8/d12/d1d/d41/f4d [0,4194304] 0 2024-03-29T04:45:30.039 INFO:tasks.workunit.client.1.smithi152.stdout:7/978: fsync d7/d8/db/df/de5/fef 0 2024-03-29T04:45:30.046 INFO:tasks.workunit.client.1.smithi152.stdout:7/979: dwrite d7/d8/d12/d35/dba/df0/d99/feb [0,4194304] 0 2024-03-29T04:45:30.051 INFO:tasks.workunit.client.1.smithi152.stdout:7/980: dread d7/d8/db3/f97 [0,4194304] 0 2024-03-29T04:45:30.061 INFO:tasks.workunit.client.1.smithi152.stdout:7/981: dwrite d7/d8/d12/d1d/d41/d45/f117 [0,4194304] 0 2024-03-29T04:45:30.062 INFO:tasks.workunit.client.1.smithi152.stdout:7/982: mkdir d7/d10e/d131 0 2024-03-29T04:45:30.070 INFO:tasks.workunit.client.1.smithi152.stdout:7/983: dwrite d7/d8/db/f4b [8388608,4194304] 0 2024-03-29T04:45:30.073 INFO:tasks.workunit.client.1.smithi152.stdout:7/984: creat d7/f132 x:0 0 0 2024-03-29T04:45:30.075 INFO:tasks.workunit.client.1.smithi152.stdout:7/985: fsync d7/d8/d12/d35/dba/df0/d99/f114 0 2024-03-29T04:45:30.079 INFO:tasks.workunit.client.1.smithi152.stdout:3/924: sync 2024-03-29T04:45:30.080 INFO:tasks.workunit.client.1.smithi152.stdout:3/925: stat d7/d19/d1b/d1e/d4e/f70 0 2024-03-29T04:45:30.082 INFO:tasks.workunit.client.1.smithi152.stdout:3/926: rmdir d7/d19/d1b/d1e/da1/d41/d61/d99 39 2024-03-29T04:45:30.083 INFO:tasks.workunit.client.1.smithi152.stdout:7/986: dread d7/d8/d12/d1d/f62 [0,4194304] 0 2024-03-29T04:45:30.084 INFO:tasks.workunit.client.1.smithi152.stdout:7/987: dread - d7/f11a zero size 2024-03-29T04:45:30.086 INFO:tasks.workunit.client.1.smithi152.stdout:7/988: fsync d7/d8/d12/d1d/f62 0 2024-03-29T04:45:30.087 INFO:tasks.workunit.client.1.smithi152.stdout:7/989: creat d7/d8/d12/de2/f133 x:0 0 0 2024-03-29T04:45:30.088 INFO:tasks.workunit.client.1.smithi152.stdout:7/990: fsync d7/d8/d12/fd7 0 2024-03-29T04:45:30.115 INFO:tasks.workunit.client.1.smithi152.stdout:2/826: sync 2024-03-29T04:45:30.115 INFO:tasks.workunit.client.1.smithi152.stdout:4/787: sync 2024-03-29T04:45:30.115 INFO:tasks.workunit.client.1.smithi152.stdout:8/978: sync 2024-03-29T04:45:30.115 INFO:tasks.workunit.client.1.smithi152.stdout:0/844: sync 2024-03-29T04:45:30.115 INFO:tasks.workunit.client.1.smithi152.stdout:7/991: rename d7/d8/d12/d35/dba/df0/fb4 to d7/d8/d12/d35/dba/df0/d99/dc8/f134 0 2024-03-29T04:45:30.117 INFO:tasks.workunit.client.1.smithi152.stdout:3/927: truncate d7/fa3 908510 0 2024-03-29T04:45:30.117 INFO:tasks.workunit.client.1.smithi152.stdout:8/979: write d1/d2/d1d/f35 [1197535,86854] 0 2024-03-29T04:45:30.117 INFO:tasks.workunit.client.1.smithi152.stdout:3/928: stat d7/d19/d1b/d1e 0 2024-03-29T04:45:30.118 INFO:tasks.workunit.client.1.smithi152.stdout:8/980: write d1/d8a/fdf [722357,28173] 0 2024-03-29T04:45:30.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:29 smithi142 ceph-mon[29027]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:45:30.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:29 smithi142 ceph-mon[29027]: from='client.14642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:45:30.121 INFO:tasks.workunit.client.1.smithi152.stdout:7/992: fdatasync d7/d8/db3/d72/f109 0 2024-03-29T04:45:30.123 INFO:tasks.workunit.client.1.smithi152.stdout:8/981: symlink d1/d2/d1d/d36/d4b/d51/ddc/l15c 0 2024-03-29T04:45:30.125 INFO:tasks.workunit.client.1.smithi152.stdout:3/929: unlink d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/fc6 0 2024-03-29T04:45:30.127 INFO:tasks.workunit.client.1.smithi152.stdout:2/827: creat d0/d3/d30/d15/d53/d54/ff7 x:0 0 0 2024-03-29T04:45:30.131 INFO:tasks.workunit.client.1.smithi152.stdout:3/930: mknod d7/d19/d1b/d1e/da1/d41/d61/d75/c12b 0 2024-03-29T04:45:30.135 INFO:tasks.workunit.client.1.smithi152.stdout:4/788: dwrite d0/d7/d40/fbb [0,4194304] 0 2024-03-29T04:45:30.135 INFO:tasks.workunit.client.1.smithi152.stdout:8/982: mknod d1/d2/d1d/dd3/db0/d7f/dd8/c15d 0 2024-03-29T04:45:30.138 INFO:tasks.workunit.client.1.smithi152.stdout:7/993: link d7/d8/d12/d35/dda/f95 d7/d8/d12/de2/f135 0 2024-03-29T04:45:30.140 INFO:tasks.workunit.client.1.smithi152.stdout:7/994: readlink d7/d8/db/lca 0 2024-03-29T04:45:30.142 INFO:tasks.workunit.client.1.smithi152.stdout:7/995: read - d7/d8/d37/f11f zero size 2024-03-29T04:45:30.143 INFO:tasks.workunit.client.1.smithi152.stdout:2/828: write d0/d3/d30/f19 [6805602,69972] 0 2024-03-29T04:45:30.144 INFO:tasks.workunit.client.1.smithi152.stdout:4/789: dwrite d0/d7/fab [0,4194304] 0 2024-03-29T04:45:30.146 INFO:tasks.workunit.client.1.smithi152.stdout:2/829: fsync d0/d3/d30/d15/d22/d4c/d5a/dd6/ff3 0 2024-03-29T04:45:30.146 INFO:tasks.workunit.client.1.smithi152.stdout:3/931: write d7/d19/f29 [4852547,1954] 0 2024-03-29T04:45:30.148 INFO:tasks.workunit.client.1.smithi152.stdout:4/790: rmdir d0/d7/db2 39 2024-03-29T04:45:30.149 INFO:tasks.workunit.client.1.smithi152.stdout:2/830: mknod d0/d3/df/d8c/d97/cf8 0 2024-03-29T04:45:30.151 INFO:tasks.workunit.client.1.smithi152.stdout:4/791: rmdir d0/d7/d25 39 2024-03-29T04:45:30.152 INFO:tasks.workunit.client.1.smithi152.stdout:2/831: readlink d0/d3/d30/d15/d53/d54/dad/dd5/le0 0 2024-03-29T04:45:30.154 INFO:tasks.workunit.client.1.smithi152.stdout:2/832: stat d0/d3/d30/d15/d53/d54/d57/le3 0 2024-03-29T04:45:30.155 INFO:tasks.workunit.client.1.smithi152.stdout:2/833: chown d0/d3/d30/d15/f25 517767128 1 2024-03-29T04:45:30.155 INFO:tasks.workunit.client.1.smithi152.stdout:4/792: read - d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/ff6 zero size 2024-03-29T04:45:30.159 INFO:tasks.workunit.client.1.smithi152.stdout:7/996: dread d7/d8/d12/d35/dda/f12a [0,4194304] 0 2024-03-29T04:45:30.161 INFO:tasks.workunit.client.1.smithi152.stdout:4/793: read d0/d7/d40/d6d/fa4 [4672,126502] 0 2024-03-29T04:45:30.162 INFO:tasks.workunit.client.1.smithi152.stdout:2/834: dread d0/d3/df/da7/faf [0,4194304] 0 2024-03-29T04:45:30.164 INFO:tasks.workunit.client.1.smithi152.stdout:4/794: write d0/d7/d40/f67 [519252,32090] 0 2024-03-29T04:45:30.174 INFO:tasks.workunit.client.1.smithi152.stdout:2/835: dread d0/d3/d30/d15/d53/d54/dad/fba [0,4194304] 0 2024-03-29T04:45:30.176 INFO:tasks.workunit.client.1.smithi152.stdout:3/932: dread d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/faa [0,4194304] 0 2024-03-29T04:45:30.177 INFO:tasks.workunit.client.1.smithi152.stdout:7/997: dwrite d7/d8/d12/f91 [0,4194304] 0 2024-03-29T04:45:30.177 INFO:tasks.workunit.client.1.smithi152.stdout:4/795: dwrite d0/d7/d40/d23/d55/d70/d96/dc0/ff7 [0,4194304] 0 2024-03-29T04:45:30.181 INFO:tasks.workunit.client.1.smithi152.stdout:4/796: chown d0/d7/d25/d17/d5e/d60/d74/da2/da6/dac/laf 250241 1 2024-03-29T04:45:30.183 INFO:tasks.workunit.client.1.smithi152.stdout:7/998: rmdir d7/d8 39 2024-03-29T04:45:30.183 INFO:tasks.workunit.client.1.smithi152.stdout:4/797: chown d0/d7/d40/d23/d55/d6c/d91/d93/c95 137 1 2024-03-29T04:45:30.186 INFO:tasks.workunit.client.1.smithi152.stdout:3/933: dread d7/f87 [4194304,4194304] 0 2024-03-29T04:45:30.189 INFO:tasks.workunit.client.1.smithi152.stdout:2/836: write d0/d3/d30/d15/d53/db4/fec [824888,50139] 0 2024-03-29T04:45:30.190 INFO:tasks.workunit.client.1.smithi152.stdout:7/999: fdatasync d7/d8/d12/d35/dba/df0/f119 0 2024-03-29T04:45:30.191 INFO:tasks.workunit.client.1.smithi152.stdout:2/837: fdatasync d0/d3/d30/d15/d22/d4c/d5a/dd6/ff3 0 2024-03-29T04:45:30.193 INFO:tasks.workunit.client.1.smithi152.stdout:2/838: chown d0/d3/d30/d15/l83 300129458 1 2024-03-29T04:45:30.195 INFO:tasks.workunit.client.1.smithi152.stdout:2/839: dread - d0/d3/d30/d15/d53/d54/ff7 zero size 2024-03-29T04:45:30.196 INFO:tasks.workunit.client.1.smithi152.stdout:3/934: dwrite d7/d19/d1b/d59/fd8 [0,4194304] 0 2024-03-29T04:45:30.198 INFO:tasks.workunit.client.1.smithi152.stdout:2/840: rmdir d0/d3/d30/d15/d22/d74 39 2024-03-29T04:45:30.199 INFO:tasks.workunit.client.1.smithi152.stdout:3/935: read d7/d19/d1b/d1e/da1/d41/f107 [481472,59135] 0 2024-03-29T04:45:30.202 INFO:tasks.workunit.client.1.smithi152.stdout:4/798: truncate d0/f11 241559 0 2024-03-29T04:45:30.202 INFO:tasks.workunit.client.1.smithi152.stdout:3/936: fsync d7/d19/d1b/d1e/d4e/fad 0 2024-03-29T04:45:30.202 INFO:tasks.workunit.client.1.smithi152.stdout:2/841: mknod d0/d3/d30/d15/d22/d4c/d5a/cf9 0 2024-03-29T04:45:30.203 INFO:tasks.workunit.client.1.smithi152.stdout:4/799: dread - d0/d7/d25/d17/d5e/d60/d74/da2/fcf zero size 2024-03-29T04:45:30.208 INFO:tasks.workunit.client.1.smithi152.stdout:3/937: mknod d7/d19/d1b/d1e/da1/c12c 0 2024-03-29T04:45:30.210 INFO:tasks.workunit.client.1.smithi152.stdout:3/938: mknod d7/d19/d1b/d59/dec/c12d 0 2024-03-29T04:45:30.212 INFO:tasks.workunit.client.1.smithi152.stdout:3/939: mknod d7/d19/d1b/d1e/da1/d41/d61/d75/c12e 0 2024-03-29T04:45:30.215 INFO:tasks.workunit.client.1.smithi152.stdout:4/800: write d0/d27/f59 [577987,64628] 0 2024-03-29T04:45:30.216 INFO:tasks.workunit.client.1.smithi152.stdout:3/940: mkdir d7/d19/d1b/d1e/da1/d41/d61/d63/d12f 0 2024-03-29T04:45:30.219 INFO:tasks.workunit.client.1.smithi152.stdout:3/941: fsync d7/d19/d1b/d1e/da1/f11e 0 2024-03-29T04:45:30.220 INFO:tasks.workunit.client.1.smithi152.stdout:2/842: dwrite d0/f16 [0,4194304] 0 2024-03-29T04:45:30.221 INFO:tasks.workunit.client.1.smithi152.stdout:2/843: stat d0/d3/df/d2c/d45/c6a 0 2024-03-29T04:45:30.226 INFO:tasks.workunit.client.1.smithi152.stdout:2/844: rename d0/d3/df/db9/cdb to d0/d3/df/d9e/cfa 0 2024-03-29T04:45:30.227 INFO:tasks.workunit.client.1.smithi152.stdout:3/942: write d7/fd5 [1059892,10381] 0 2024-03-29T04:45:30.228 INFO:tasks.workunit.client.1.smithi152.stdout:2/845: chown d0/d3/df/d2c/c34 1 1 2024-03-29T04:45:30.229 INFO:tasks.workunit.client.1.smithi152.stdout:3/943: fdatasync d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/f6e 0 2024-03-29T04:45:30.230 INFO:tasks.workunit.client.1.smithi152.stdout:3/944: dread - d7/d19/d1b/d1e/da1/d41/d61/d85/f11b zero size 2024-03-29T04:45:30.232 INFO:tasks.workunit.client.1.smithi152.stdout:2/846: unlink d0/d3/d30/d15/d22/d4c/f89 0 2024-03-29T04:45:30.233 INFO:tasks.workunit.client.1.smithi152.stdout:3/945: fdatasync d7/d19/d1b/f7b 0 2024-03-29T04:45:30.236 INFO:tasks.workunit.client.1.smithi152.stdout:3/946: creat d7/d19/d1b/d1e/da1/d41/d61/f130 x:0 0 0 2024-03-29T04:45:30.238 INFO:tasks.workunit.client.1.smithi152.stdout:3/947: symlink d7/d19/db5/df8/l131 0 2024-03-29T04:45:30.242 INFO:tasks.workunit.client.1.smithi152.stdout:3/948: getdents d7/d19/d1b/d1e 0 2024-03-29T04:45:30.244 INFO:tasks.workunit.client.1.smithi152.stdout:3/949: symlink d7/d19/d1b/d1e/d4e/d8f/l132 0 2024-03-29T04:45:30.252 INFO:tasks.workunit.client.1.smithi152.stdout:3/950: dwrite d7/d19/d1b/d1e/da1/d41/f50 [0,4194304] 0 2024-03-29T04:45:30.254 INFO:tasks.workunit.client.1.smithi152.stdout:3/951: chown d7/d19/d1b/d1e/da1/d41/d61/d75/fb7 5261971 1 2024-03-29T04:45:30.257 INFO:tasks.workunit.client.1.smithi152.stdout:3/952: read d7/fa3 [425324,111707] 0 2024-03-29T04:45:30.259 INFO:tasks.workunit.client.1.smithi152.stdout:3/953: write d7/d19/d1b/d1e/f82 [2173354,69616] 0 2024-03-29T04:45:30.262 INFO:tasks.workunit.client.1.smithi152.stdout:3/954: mknod d7/d19/d1b/d1e/da1/d41/d6a/dcc/de4/c133 0 2024-03-29T04:45:30.263 INFO:tasks.workunit.client.1.smithi152.stdout:3/955: readlink d7/d19/d1b/d1e/d4e/d8f/l132 0 2024-03-29T04:45:30.637 INFO:tasks.workunit.client.1.smithi152.stdout:9/991: sync 2024-03-29T04:45:30.726 INFO:tasks.workunit.client.1.smithi152.stdout:9/992: rename d5/db/d92/dd2/d137/db1/dcd/c3a to d5/db/d1b/d68/c15b 0 2024-03-29T04:45:30.727 INFO:tasks.workunit.client.1.smithi152.stdout:8/983: sync 2024-03-29T04:45:30.727 INFO:tasks.workunit.client.1.smithi152.stdout:4/801: sync 2024-03-29T04:45:30.728 INFO:tasks.workunit.client.1.smithi152.stdout:0/845: sync 2024-03-29T04:45:30.728 INFO:tasks.workunit.client.1.smithi152.stdout:2/847: sync 2024-03-29T04:45:30.728 INFO:tasks.workunit.client.1.smithi152.stdout:3/956: sync 2024-03-29T04:45:30.728 INFO:tasks.workunit.client.1.smithi152.stdout:9/993: dread d5/db/f6c [0,4194304] 0 2024-03-29T04:45:30.728 INFO:tasks.workunit.client.1.smithi152.stdout:8/984: fsync d1/dc5/fcd 0 2024-03-29T04:45:30.731 INFO:tasks.workunit.client.1.smithi152.stdout:3/957: mknod d7/d19/d1b/d59/dec/c134 0 2024-03-29T04:45:30.731 INFO:tasks.workunit.client.1.smithi152.stdout:4/802: stat d0/d7/d25/d17/d5e/d60/d74/da2/da6 0 2024-03-29T04:45:30.733 INFO:tasks.workunit.client.1.smithi152.stdout:4/803: stat d0/d27 0 2024-03-29T04:45:30.734 INFO:tasks.workunit.client.1.smithi152.stdout:3/958: read - d7/d19/d1b/d1e/df0/ff4 zero size 2024-03-29T04:45:30.736 INFO:tasks.workunit.client.1.smithi152.stdout:2/848: link d0/d3/d30/d15/da5/fe9 d0/d3/d30/dd0/ffb 0 2024-03-29T04:45:30.740 INFO:tasks.workunit.client.1.smithi152.stdout:8/985: link d1/d2/d1d/dd3/db0/f2a d1/d2/d3c/d76/f15e 0 2024-03-29T04:45:30.741 INFO:tasks.workunit.client.1.smithi152.stdout:2/849: mkdir d0/d3/d30/dd0/dfc 0 2024-03-29T04:45:30.743 INFO:tasks.workunit.client.1.smithi152.stdout:0/846: dwrite d2/da/d17/dd8/db8/daf/f107 [0,4194304] 0 2024-03-29T04:45:30.750 INFO:tasks.workunit.client.1.smithi152.stdout:2/850: read d0/d3/d30/d15/d22/d4c/f51 [2540147,32577] 0 2024-03-29T04:45:30.754 INFO:tasks.workunit.client.1.smithi152.stdout:9/994: dwrite d5/db/f65 [0,4194304] 0 2024-03-29T04:45:30.755 INFO:tasks.workunit.client.1.smithi152.stdout:3/959: dread d7/d19/d1b/d59/fd8 [0,4194304] 0 2024-03-29T04:45:30.761 INFO:tasks.workunit.client.1.smithi152.stdout:4/804: dwrite d0/d7/d25/d17/d5e/d60/f9b [0,4194304] 0 2024-03-29T04:45:30.764 INFO:tasks.workunit.client.1.smithi152.stdout:3/960: dread - d7/d19/d1b/d1e/da1/f9c zero size 2024-03-29T04:45:30.766 INFO:tasks.workunit.client.1.smithi152.stdout:8/986: dwrite d1/f7b [0,4194304] 0 2024-03-29T04:45:30.766 INFO:tasks.workunit.client.1.smithi152.stdout:0/847: link d2/da/c125 d2/da/d17/d57/dc5/c127 0 2024-03-29T04:45:30.768 INFO:tasks.workunit.client.1.smithi152.stdout:2/851: write d0/d3/d30/d15/d53/d54/d57/f72 [1752292,121681] 0 2024-03-29T04:45:30.769 INFO:tasks.workunit.client.1.smithi152.stdout:4/805: chown d0/d7/d40/d23/d55/d70/d96/dc0/ff7 5601742 1 2024-03-29T04:45:30.769 INFO:tasks.workunit.client.1.smithi152.stdout:9/995: mknod d5/d8/d31/d14a/c15c 0 2024-03-29T04:45:30.774 INFO:tasks.workunit.client.1.smithi152.stdout:0/848: mkdir d2/da/d17/dd8/db8/d128 0 2024-03-29T04:45:30.774 INFO:tasks.workunit.client.1.smithi152.stdout:3/961: dread - d7/d19/d1b/d1e/da1/fe0 zero size 2024-03-29T04:45:30.776 INFO:tasks.workunit.client.1.smithi152.stdout:9/996: rmdir d5/db/d1b/d68 39 2024-03-29T04:45:30.779 INFO:tasks.workunit.client.1.smithi152.stdout:3/962: truncate d7/d19/d1b/d1e/d4e/f5a 1815524 0 2024-03-29T04:45:30.779 INFO:tasks.workunit.client.1.smithi152.stdout:2/852: mknod d0/dcc/cfd 0 2024-03-29T04:45:30.780 INFO:tasks.workunit.client.1.smithi152.stdout:4/806: dread d0/f11 [0,4194304] 0 2024-03-29T04:45:30.781 INFO:tasks.workunit.client.1.smithi152.stdout:8/987: truncate d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/ff9 397038 0 2024-03-29T04:45:30.782 INFO:tasks.workunit.client.1.smithi152.stdout:9/997: creat d5/db/d1b/d68/daa/f15d x:0 0 0 2024-03-29T04:45:30.784 INFO:tasks.workunit.client.1.smithi152.stdout:9/998: dread - d5/d8/d31/d59/dc3/d125/f138 zero size 2024-03-29T04:45:30.784 INFO:tasks.workunit.client.1.smithi152.stdout:4/807: mknod d0/d27/d6e/cfc 0 2024-03-29T04:45:30.785 INFO:tasks.workunit.client.1.smithi152.stdout:9/999: dread - d5/d12/db7/f13d zero size 2024-03-29T04:45:30.788 INFO:tasks.workunit.client.1.smithi152.stdout:8/988: dread d1/d2/d1d/dd3/db0/d119/f140 [0,4194304] 0 2024-03-29T04:45:30.789 INFO:tasks.workunit.client.1.smithi152.stdout:4/808: rename d0/d27/c99 to d0/d7/d25/d17/d5e/d60/d74/da2/da6/dac/cfd 0 2024-03-29T04:45:30.792 INFO:tasks.workunit.client.1.smithi152.stdout:4/809: write d0/d27/f59 [926070,66988] 0 2024-03-29T04:45:30.793 INFO:tasks.workunit.client.1.smithi152.stdout:8/989: mkdir d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/d15f 0 2024-03-29T04:45:30.795 INFO:tasks.workunit.client.1.smithi152.stdout:3/963: write d7/d19/d1b/d1e/da1/d41/d61/d63/f67 [3743529,84279] 0 2024-03-29T04:45:30.797 INFO:tasks.workunit.client.1.smithi152.stdout:8/990: chown d1/d2/d1d/dd3/db0/d40/d6a/dd5 26 1 2024-03-29T04:45:30.798 INFO:tasks.workunit.client.1.smithi152.stdout:8/991: chown d1/d2/d6d/f106 797362 1 2024-03-29T04:45:30.802 INFO:tasks.workunit.client.1.smithi152.stdout:8/992: truncate d1/d2/d1d/dd3/db0/da9/f84 4376471 0 2024-03-29T04:45:30.805 INFO:tasks.workunit.client.1.smithi152.stdout:3/964: write d7/d19/d1b/d1e/da1/d41/f107 [1155886,64421] 0 2024-03-29T04:45:30.805 INFO:tasks.workunit.client.1.smithi152.stdout:2/853: dread d0/d3/d30/d15/d53/d54/d57/fd2 [0,4194304] 0 2024-03-29T04:45:30.806 INFO:tasks.workunit.client.1.smithi152.stdout:8/993: dread - d1/d2/d1d/dd3/db0/d40/d6a/dd5/df2/f117 zero size 2024-03-29T04:45:30.808 INFO:tasks.workunit.client.1.smithi152.stdout:3/965: readlink d7/d19/d1b/d1e/d4e/d8f/l132 0 2024-03-29T04:45:30.808 INFO:tasks.workunit.client.1.smithi152.stdout:2/854: dread - d0/d3/d30/d15/d22/d4c/d5a/dd6/ff3 zero size 2024-03-29T04:45:30.813 INFO:tasks.workunit.client.1.smithi152.stdout:3/966: mknod d7/d19/d1b/d1e/da1/d41/c135 0 2024-03-29T04:45:30.815 INFO:tasks.workunit.client.1.smithi152.stdout:3/967: mkdir d7/d19/d1b/d1e/d4e/d8f/d128/d136 0 2024-03-29T04:45:30.816 INFO:tasks.workunit.client.1.smithi152.stdout:4/810: dread d0/d7/d40/d23/d55/d70/d96/dc0/df8/f7d [0,4194304] 0 2024-03-29T04:45:30.819 INFO:tasks.workunit.client.1.smithi152.stdout:4/811: mkdir d0/d7/d40/d23/d6a/dfe 0 2024-03-29T04:45:30.821 INFO:tasks.workunit.client.1.smithi152.stdout:4/812: write d0/d7/d40/d6d/fcc [3053278,96183] 0 2024-03-29T04:45:30.825 INFO:tasks.workunit.client.1.smithi152.stdout:4/813: mknod d0/d7/d25/d17/d5e/d60/d74/da2/da6/dac/cff 0 2024-03-29T04:45:30.827 INFO:tasks.workunit.client.1.smithi152.stdout:4/814: symlink d0/d7/d40/d23/d55/l100 0 2024-03-29T04:45:30.831 INFO:tasks.workunit.client.1.smithi152.stdout:3/968: dread d7/d19/d1b/d1e/da1/d41/d73/f77 [0,4194304] 0 2024-03-29T04:45:30.834 INFO:tasks.workunit.client.1.smithi152.stdout:3/969: fsync d7/d19/fb6 0 2024-03-29T04:45:30.835 INFO:tasks.workunit.client.1.smithi152.stdout:4/815: link d0/d7/d25/d17/f54 d0/d7/d40/d23/dda/de3/f101 0 2024-03-29T04:45:30.856 INFO:tasks.workunit.client.1.smithi152.stdout:3/970: write d7/d19/d1b/d1e/da1/d41/d6a/d78/fde [954799,88821] 0 2024-03-29T04:45:30.861 INFO:tasks.workunit.client.1.smithi152.stdout:3/971: getdents d7/d19/d1b/d1e/da1/d41/d73 0 2024-03-29T04:45:30.872 INFO:tasks.workunit.client.1.smithi152.stdout:3/972: dwrite d7/d19/db5/ff2 [0,4194304] 0 2024-03-29T04:45:30.885 INFO:tasks.workunit.client.1.smithi152.stdout:3/973: write d7/d19/d1b/d1e/da1/f11e [221834,97846] 0 2024-03-29T04:45:30.889 INFO:tasks.workunit.client.1.smithi152.stdout:3/974: mkdir d7/d19/d1b/d1e/da1/d41/d137 0 2024-03-29T04:45:30.894 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:30 smithi152 ceph-mon[37177]: pgmap v281: 65 pgs: 65 active+clean; 2.9 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 82 MiB/s rd, 195 MiB/s wr, 523 op/s 2024-03-29T04:45:30.896 INFO:tasks.workunit.client.1.smithi152.stdout:3/975: write d7/d19/d1b/d1e/d4e/d8f/f100 [827328,14800] 0 2024-03-29T04:45:30.901 INFO:tasks.workunit.client.1.smithi152.stdout:3/976: mkdir d7/d19/d1b/d59/d138 0 2024-03-29T04:45:30.911 INFO:tasks.workunit.client.1.smithi152.stdout:3/977: write d7/d19/d1b/d1e/d4e/d8f/fd6 [361774,104305] 0 2024-03-29T04:45:30.917 INFO:tasks.workunit.client.1.smithi152.stdout:3/978: dread d7/d19/d1b/d1e/d4e/f5a [0,4194304] 0 2024-03-29T04:45:30.940 INFO:tasks.workunit.client.1.smithi152.stdout:3/979: dread d7/d19/f29 [0,4194304] 0 2024-03-29T04:45:30.942 INFO:tasks.workunit.client.1.smithi152.stdout:3/980: rename d7/d19/d1b/d1e/da1/d41/d61/d75/c12e to d7/d19/db5/c139 0 2024-03-29T04:45:30.946 INFO:tasks.workunit.client.1.smithi152.stdout:3/981: rename d7/d19/d1b/d1e/da1/fe0 to d7/d19/d1b/d1e/da1/d41/d6a/d78/f13a 0 2024-03-29T04:45:30.949 INFO:tasks.workunit.client.1.smithi152.stdout:3/982: read d7/d19/d1b/d1e/da1/f47 [2922621,19289] 0 2024-03-29T04:45:30.951 INFO:tasks.workunit.client.1.smithi152.stdout:3/983: truncate d7/d6c/fca 815261 0 2024-03-29T04:45:30.956 INFO:tasks.workunit.client.1.smithi152.stdout:3/984: link c4 d7/d19/d1b/d1e/da1/d41/d61/c13b 0 2024-03-29T04:45:30.961 INFO:tasks.workunit.client.1.smithi152.stdout:3/985: write d7/fa3 [79043,74582] 0 2024-03-29T04:45:30.964 INFO:tasks.workunit.client.1.smithi152.stdout:3/986: mknod d7/d19/d1b/d1e/da1/d41/d6a/dcc/c13c 0 2024-03-29T04:45:30.966 INFO:tasks.workunit.client.1.smithi152.stdout:3/987: read - d7/d19/d1b/d59/f123 zero size 2024-03-29T04:45:30.971 INFO:tasks.workunit.client.1.smithi152.stdout:3/988: creat d7/d19/d1b/d1e/da1/d41/d124/f13d x:0 0 0 2024-03-29T04:45:30.975 INFO:tasks.workunit.client.1.smithi152.stdout:3/989: chown d7/d19/d1b/f58 18 1 2024-03-29T04:45:30.979 INFO:tasks.workunit.client.1.smithi152.stdout:3/990: rename d7/d19/d1b/d1e/da1/f51 to d7/d19/d1b/d1e/d4e/f13e 0 2024-03-29T04:45:30.980 INFO:tasks.workunit.client.1.smithi152.stdout:3/991: stat d7/d19/d1b/d1e/c10b 0 2024-03-29T04:45:30.982 INFO:tasks.workunit.client.1.smithi152.stdout:3/992: truncate d7/d19/d1b/d1e/fb3 351548 0 2024-03-29T04:45:30.983 INFO:tasks.workunit.client.1.smithi152.stdout:3/993: readlink d7/d19/d1b/d1e/da1/d41/d61/d75/d8c/l9d 0 2024-03-29T04:45:30.983 INFO:tasks.workunit.client.1.smithi152.stdout:3/994: readlink d7/d46/l74 0 2024-03-29T04:45:30.985 INFO:tasks.workunit.client.1.smithi152.stdout:3/995: write d7/d19/d1b/d1e/d4e/f70 [2083303,103943] 0 2024-03-29T04:45:30.992 INFO:tasks.workunit.client.1.smithi152.stdout:3/996: dwrite d7/d19/d1b/d1e/ffb [4194304,4194304] 0 2024-03-29T04:45:30.998 INFO:tasks.workunit.client.1.smithi152.stdout:3/997: write d7/d19/d1b/d1e/d4e/f5a [1107028,21511] 0 2024-03-29T04:45:31.000 INFO:tasks.workunit.client.1.smithi152.stdout:3/998: write d7/d19/d1b/d1e/d4e/d8f/fd6 [12263,47135] 0 2024-03-29T04:45:31.003 INFO:tasks.workunit.client.1.smithi152.stdout:3/999: mkdir d7/d19/d1b/d1e/da1/d13f 0 2024-03-29T04:45:31.096 INFO:tasks.workunit.client.1.smithi152.stdout:0/849: sync 2024-03-29T04:45:31.098 INFO:tasks.workunit.client.1.smithi152.stdout:0/850: creat d2/da/d3a/d61/db4/f129 x:0 0 0 2024-03-29T04:45:31.100 INFO:tasks.workunit.client.1.smithi152.stdout:0/851: mknod d2/da/d3a/d61/db4/d11d/c12a 0 2024-03-29T04:45:31.102 INFO:tasks.workunit.client.1.smithi152.stdout:0/852: mknod d2/da/d17/d57/dc5/c12b 0 2024-03-29T04:45:31.115 INFO:tasks.workunit.client.1.smithi152.stdout:0/853: dread d2/da/d17/d57/fcc [0,4194304] 0 2024-03-29T04:45:31.120 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:30 smithi142 ceph-mon[29027]: pgmap v281: 65 pgs: 65 active+clean; 2.9 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 82 MiB/s rd, 195 MiB/s wr, 523 op/s 2024-03-29T04:45:31.126 INFO:tasks.workunit.client.1.smithi152.stdout:0/854: dwrite d2/da/d17/d50/fb7 [0,4194304] 0 2024-03-29T04:45:31.130 INFO:tasks.workunit.client.1.smithi152.stdout:0/855: write d2/da/d3a/f81 [1079393,7922] 0 2024-03-29T04:45:31.133 INFO:tasks.workunit.client.1.smithi152.stdout:0/856: rmdir d2/da/d3a/d61/db4/d11d 39 2024-03-29T04:45:31.136 INFO:tasks.workunit.client.1.smithi152.stdout:0/857: mkdir d2/da/d17/d50/dc6/d12c 0 2024-03-29T04:45:31.153 INFO:tasks.workunit.client.1.smithi152.stdout:0/858: dread d2/da/df/f5e [0,4194304] 0 2024-03-29T04:45:31.165 INFO:tasks.workunit.client.1.smithi152.stdout:0/859: dread d2/da/d17/dd8/fba [0,4194304] 0 2024-03-29T04:45:31.167 INFO:tasks.workunit.client.1.smithi152.stdout:0/860: dread d2/da/d17/f77 [0,4194304] 0 2024-03-29T04:45:31.169 INFO:tasks.workunit.client.1.smithi152.stdout:0/861: creat d2/da/d17/d50/dc6/d12c/f12d x:0 0 0 2024-03-29T04:45:31.176 INFO:tasks.workunit.client.1.smithi152.stdout:0/862: dwrite d2/da/d17/dd8/fba [0,4194304] 0 2024-03-29T04:45:31.177 INFO:tasks.workunit.client.1.smithi152.stdout:0/863: chown d2/da/d17/f74 23572 1 2024-03-29T04:45:31.180 INFO:tasks.workunit.client.1.smithi152.stdout:0/864: mkdir d2/da/d17/d50/dc6/d12e 0 2024-03-29T04:45:31.183 INFO:tasks.workunit.client.1.smithi152.stdout:0/865: symlink d2/da/d17/d50/dc6/d12e/l12f 0 2024-03-29T04:45:31.184 INFO:tasks.workunit.client.1.smithi152.stdout:0/866: stat d2/da/c28 0 2024-03-29T04:45:31.186 INFO:tasks.workunit.client.1.smithi152.stdout:4/816: sync 2024-03-29T04:45:31.186 INFO:tasks.workunit.client.1.smithi152.stdout:8/994: sync 2024-03-29T04:45:31.186 INFO:tasks.workunit.client.1.smithi152.stdout:2/855: sync 2024-03-29T04:45:31.190 INFO:tasks.workunit.client.1.smithi152.stdout:8/995: unlink d1/dc5/d149/dee/f13f 0 2024-03-29T04:45:31.190 INFO:tasks.workunit.client.1.smithi152.stdout:2/856: dread d0/d3/f10 [0,4194304] 0 2024-03-29T04:45:31.191 INFO:tasks.workunit.client.1.smithi152.stdout:8/996: stat d1/d2/d1d/f4f 0 2024-03-29T04:45:31.193 INFO:tasks.workunit.client.1.smithi152.stdout:2/857: creat d0/d3/d64/ffe x:0 0 0 2024-03-29T04:45:31.194 INFO:tasks.workunit.client.1.smithi152.stdout:8/997: creat d1/d2/d1d/d36/d107/f160 x:0 0 0 2024-03-29T04:45:31.195 INFO:tasks.workunit.client.1.smithi152.stdout:2/858: mkdir d0/d3/d30/d15/d22/d4c/d5a/dff 0 2024-03-29T04:45:31.197 INFO:tasks.workunit.client.1.smithi152.stdout:2/859: rename d0/d3/d30/d15/d53 to d0/d3/d30/d15/d53/d100 22 2024-03-29T04:45:31.198 INFO:tasks.workunit.client.1.smithi152.stdout:4/817: dread d0/d7/d25/d17/d5e/d60/d74/f51 [0,4194304] 0 2024-03-29T04:45:31.198 INFO:tasks.workunit.client.1.smithi152.stdout:8/998: getdents d1/dc5 0 2024-03-29T04:45:31.200 INFO:tasks.workunit.client.1.smithi152.stdout:2/860: dread - d0/d3/d30/d15/d53/d54/ff7 zero size 2024-03-29T04:45:31.200 INFO:tasks.workunit.client.1.smithi152.stdout:4/818: dread - d0/fba zero size 2024-03-29T04:45:31.203 INFO:tasks.workunit.client.1.smithi152.stdout:8/999: creat d1/d2/d1d/f161 x:0 0 0 2024-03-29T04:45:31.205 INFO:tasks.workunit.client.1.smithi152.stdout:0/867: dwrite d2/da/df/f8e [0,4194304] 0 2024-03-29T04:45:31.207 INFO:tasks.workunit.client.1.smithi152.stdout:2/861: unlink d0/d3/d30/d15/d53/db4/fec 0 2024-03-29T04:45:31.210 INFO:tasks.workunit.client.1.smithi152.stdout:2/862: creat d0/d3/d64/f101 x:0 0 0 2024-03-29T04:45:31.211 INFO:tasks.workunit.client.1.smithi152.stdout:4/819: write d0/f11 [756481,112410] 0 2024-03-29T04:45:31.212 INFO:tasks.workunit.client.1.smithi152.stdout:2/863: creat d0/dcc/f102 x:0 0 0 2024-03-29T04:45:31.216 INFO:tasks.workunit.client.1.smithi152.stdout:2/864: mkdir d0/d3/d30/d15/da5/d103 0 2024-03-29T04:45:31.217 INFO:tasks.workunit.client.1.smithi152.stdout:0/868: write d2/da/d3a/d61/fd0 [632447,60064] 0 2024-03-29T04:45:31.218 INFO:tasks.workunit.client.1.smithi152.stdout:2/865: dread - d0/d3/df/d8c/d97/fc7 zero size 2024-03-29T04:45:31.218 INFO:tasks.workunit.client.1.smithi152.stdout:0/869: creat d2/d29/dcf/f130 x:0 0 0 2024-03-29T04:45:31.219 INFO:tasks.workunit.client.1.smithi152.stdout:2/866: chown d0/d3/df/d2c/la9 400743 1 2024-03-29T04:45:31.221 INFO:tasks.workunit.client.1.smithi152.stdout:4/820: write d0/d7/d25/d17/f5b [825499,18343] 0 2024-03-29T04:45:31.221 INFO:tasks.workunit.client.1.smithi152.stdout:0/870: read d2/da/d17/f74 [3366896,28808] 0 2024-03-29T04:45:31.222 INFO:tasks.workunit.client.1.smithi152.stdout:4/821: write d0/d7/f84 [2961519,95046] 0 2024-03-29T04:45:31.225 INFO:tasks.workunit.client.1.smithi152.stdout:0/871: creat d2/da/d17/d57/d9c/dbe/f131 x:0 0 0 2024-03-29T04:45:31.226 INFO:tasks.workunit.client.1.smithi152.stdout:4/822: creat d0/d27/f102 x:0 0 0 2024-03-29T04:45:31.228 INFO:tasks.workunit.client.1.smithi152.stdout:4/823: read d0/d7/d25/f36 [532511,13525] 0 2024-03-29T04:45:31.233 INFO:tasks.workunit.client.1.smithi152.stdout:2/867: dwrite d0/d3/d30/d15/f2b [0,4194304] 0 2024-03-29T04:45:31.234 INFO:tasks.workunit.client.1.smithi152.stdout:4/824: unlink d0/d7/d40/d23/d55/d70/c92 0 2024-03-29T04:45:31.237 INFO:tasks.workunit.client.1.smithi152.stdout:2/868: chown d0/d3/c75 1 1 2024-03-29T04:45:31.237 INFO:tasks.workunit.client.1.smithi152.stdout:0/872: dwrite d2/da/d17/d50/dc6/d12c/f12d [0,4194304] 0 2024-03-29T04:45:31.238 INFO:tasks.workunit.client.1.smithi152.stdout:0/873: stat d2/da/d17/fe0 0 2024-03-29T04:45:31.241 INFO:tasks.workunit.client.1.smithi152.stdout:2/869: creat d0/d3/d30/d15/d53/f104 x:0 0 0 2024-03-29T04:45:31.242 INFO:tasks.workunit.client.1.smithi152.stdout:2/870: dread - d0/d3/d64/ffe zero size 2024-03-29T04:45:31.245 INFO:tasks.workunit.client.1.smithi152.stdout:0/874: creat d2/da/d3a/d61/db4/f132 x:0 0 0 2024-03-29T04:45:31.249 INFO:tasks.workunit.client.1.smithi152.stdout:2/871: rename d0/d3/d30/d15/da5/fe9 to d0/d3/d30/d15/d53/d54/dad/dd5/f105 0 2024-03-29T04:45:31.250 INFO:tasks.workunit.client.1.smithi152.stdout:2/872: chown d0/d3/d30/d15/d53/d54/d57/le3 28 1 2024-03-29T04:45:31.252 INFO:tasks.workunit.client.1.smithi152.stdout:0/875: rename d2/d29/f116 to d2/da/d3a/d6f/f133 0 2024-03-29T04:45:31.254 INFO:tasks.workunit.client.1.smithi152.stdout:0/876: symlink d2/da/d17/d57/d9c/l134 0 2024-03-29T04:45:31.256 INFO:tasks.workunit.client.1.smithi152.stdout:2/873: rmdir d0/d3/d30/dd0/dfc 0 2024-03-29T04:45:31.257 INFO:tasks.workunit.client.1.smithi152.stdout:0/877: creat d2/da/d17/d57/d9c/dbe/df3/f135 x:0 0 0 2024-03-29T04:45:31.259 INFO:tasks.workunit.client.1.smithi152.stdout:2/874: creat d0/d3/d5f/f106 x:0 0 0 2024-03-29T04:45:31.260 INFO:tasks.workunit.client.1.smithi152.stdout:2/875: chown d0/d3/d30/d15/d22/d4c/d5a/dd6/ff3 1303 1 2024-03-29T04:45:31.260 INFO:tasks.workunit.client.1.smithi152.stdout:0/878: mkdir d2/da/d17/d57/d136 0 2024-03-29T04:45:31.263 INFO:tasks.workunit.client.1.smithi152.stdout:0/879: write d2/da/d17/d57/d9c/f112 [564130,93571] 0 2024-03-29T04:45:31.264 INFO:tasks.workunit.client.1.smithi152.stdout:2/876: mkdir d0/d3/d30/d15/da5/d103/d107 0 2024-03-29T04:45:31.271 INFO:tasks.workunit.client.1.smithi152.stdout:4/825: dread d0/d27/d6e/f97 [0,4194304] 0 2024-03-29T04:45:31.273 INFO:tasks.workunit.client.1.smithi152.stdout:0/880: dwrite d2/da/d3a/d61/f7f [0,4194304] 0 2024-03-29T04:45:31.275 INFO:tasks.workunit.client.1.smithi152.stdout:2/877: dwrite d0/d3/d30/d15/da5/fd7 [0,4194304] 0 2024-03-29T04:45:31.282 INFO:tasks.workunit.client.1.smithi152.stdout:4/826: symlink d0/d7/d40/d23/d55/d70/d96/l103 0 2024-03-29T04:45:31.284 INFO:tasks.workunit.client.1.smithi152.stdout:2/878: creat d0/d3/df/db9/f108 x:0 0 0 2024-03-29T04:45:31.287 INFO:tasks.workunit.client.1.smithi152.stdout:2/879: truncate d0/d3/f31 4058404 0 2024-03-29T04:45:31.289 INFO:tasks.workunit.client.1.smithi152.stdout:4/827: creat d0/d7/d40/d23/f104 x:0 0 0 2024-03-29T04:45:31.289 INFO:tasks.workunit.client.1.smithi152.stdout:2/880: write d0/d3/d30/f33 [6762297,25346] 0 2024-03-29T04:45:31.295 INFO:tasks.workunit.client.1.smithi152.stdout:4/828: rename d0/d7/d40/d23/d55/d6c/l7f to d0/d7/d40/d23/d55/d70/d96/l105 0 2024-03-29T04:45:31.297 INFO:tasks.workunit.client.1.smithi152.stdout:4/829: mknod d0/d7/d40/d23/d55/d70/d96/c106 0 2024-03-29T04:45:31.301 INFO:tasks.workunit.client.1.smithi152.stdout:2/881: write d0/d3/d30/d15/d22/f99 [690115,110501] 0 2024-03-29T04:45:31.309 INFO:tasks.workunit.client.1.smithi152.stdout:2/882: truncate d0/d3/d30/d15/f25 3054238 0 2024-03-29T04:45:31.310 INFO:tasks.workunit.client.1.smithi152.stdout:2/883: write d0/f16 [3709083,31879] 0 2024-03-29T04:45:31.311 INFO:tasks.workunit.client.1.smithi152.stdout:0/881: dread d2/da/d17/dd8/f9e [0,4194304] 0 2024-03-29T04:45:31.318 INFO:tasks.workunit.client.1.smithi152.stdout:0/882: write d2/da/d17/f74 [2592640,104538] 0 2024-03-29T04:45:31.324 INFO:tasks.workunit.client.1.smithi152.stdout:2/884: dwrite d0/d3/f1d [0,4194304] 0 2024-03-29T04:45:31.325 INFO:tasks.workunit.client.1.smithi152.stdout:0/883: read d2/da/d3a/d61/db4/ffb [3455392,130814] 0 2024-03-29T04:45:31.326 INFO:tasks.workunit.client.1.smithi152.stdout:2/885: chown d0/d3/d30/d15/da5 26231188 1 2024-03-29T04:45:31.327 INFO:tasks.workunit.client.1.smithi152.stdout:2/886: chown d0/d3/c75 6191485 1 2024-03-29T04:45:31.329 INFO:tasks.workunit.client.1.smithi152.stdout:0/884: creat d2/da/d17/d57/dc5/f137 x:0 0 0 2024-03-29T04:45:31.337 INFO:tasks.workunit.client.1.smithi152.stdout:2/887: dwrite d0/d3/d30/d15/d22/f37 [8388608,4194304] 0 2024-03-29T04:45:31.342 INFO:tasks.workunit.client.1.smithi152.stdout:0/885: dwrite d2/da/d17/d57/d9c/f112 [0,4194304] 0 2024-03-29T04:45:31.343 INFO:tasks.workunit.client.1.smithi152.stdout:2/888: dread d0/f16 [0,4194304] 0 2024-03-29T04:45:31.344 INFO:tasks.workunit.client.1.smithi152.stdout:2/889: readlink d0/l71 0 2024-03-29T04:45:31.348 INFO:tasks.workunit.client.1.smithi152.stdout:0/886: mknod d2/da/d17/d57/d9c/c138 0 2024-03-29T04:45:31.351 INFO:tasks.workunit.client.1.smithi152.stdout:2/890: dwrite d0/d3/d30/d15/d22/f99 [0,4194304] 0 2024-03-29T04:45:31.351 INFO:tasks.workunit.client.1.smithi152.stdout:0/887: mkdir d2/da/d17/d139 0 2024-03-29T04:45:31.353 INFO:tasks.workunit.client.1.smithi152.stdout:2/891: chown d0/d3/d30/dd0/ffb 21 1 2024-03-29T04:45:31.360 INFO:tasks.workunit.client.1.smithi152.stdout:0/888: rename d2/da/d3a/fd2 to d2/f13a 0 2024-03-29T04:45:31.363 INFO:tasks.workunit.client.1.smithi152.stdout:2/892: unlink d0/d3/l46 0 2024-03-29T04:45:31.365 INFO:tasks.workunit.client.1.smithi152.stdout:0/889: rename d2/da/d17/d57/d9c/f126 to d2/da/d17/d57/dc5/d10a/f13b 0 2024-03-29T04:45:31.367 INFO:tasks.workunit.client.1.smithi152.stdout:0/890: write d2/da/d17/dd8/db8/daf/f107 [3008965,62259] 0 2024-03-29T04:45:31.369 INFO:tasks.workunit.client.1.smithi152.stdout:0/891: write d2/da/d17/d57/d9c/dbe/df3/f10b [676166,11055] 0 2024-03-29T04:45:31.377 INFO:tasks.workunit.client.1.smithi152.stdout:0/892: creat d2/da/d3a/d61/db4/f13c x:0 0 0 2024-03-29T04:45:31.380 INFO:tasks.workunit.client.1.smithi152.stdout:0/893: symlink d2/da/d17/d57/d9c/df0/l13d 0 2024-03-29T04:45:31.381 INFO:tasks.workunit.client.1.smithi152.stdout:2/893: dwrite d0/faa [4194304,4194304] 0 2024-03-29T04:45:31.386 INFO:tasks.workunit.client.1.smithi152.stdout:0/894: rename d2/da/d17/d57/def to d2/da/d17/d57/d9c/d13e 0 2024-03-29T04:45:31.388 INFO:tasks.workunit.client.1.smithi152.stdout:2/894: symlink d0/d3/d30/d15/d22/d4c/l109 0 2024-03-29T04:45:31.391 INFO:tasks.workunit.client.1.smithi152.stdout:2/895: creat d0/d3/d64/d85/f10a x:0 0 0 2024-03-29T04:45:31.395 INFO:tasks.workunit.client.1.smithi152.stdout:2/896: creat d0/d3/d30/d15/d53/d54/f10b x:0 0 0 2024-03-29T04:45:31.401 INFO:tasks.workunit.client.1.smithi152.stdout:0/895: truncate d2/da/d17/d57/f64 105222 0 2024-03-29T04:45:31.401 INFO:tasks.workunit.client.1.smithi152.stdout:2/897: write d0/d3/f10 [1541453,61350] 0 2024-03-29T04:45:31.403 INFO:tasks.workunit.client.1.smithi152.stdout:2/898: fsync d0/d3/d30/d15/d22/d4c/d5a/dd6/ff3 0 2024-03-29T04:45:31.409 INFO:tasks.workunit.client.1.smithi152.stdout:2/899: write d0/d3/d30/d15/da5/fca [781381,129025] 0 2024-03-29T04:45:31.413 INFO:tasks.workunit.client.1.smithi152.stdout:0/896: dwrite d2/f5 [0,4194304] 0 2024-03-29T04:45:31.429 INFO:tasks.workunit.client.1.smithi152.stdout:2/900: dread d0/d3/df/f76 [0,4194304] 0 2024-03-29T04:45:31.432 INFO:tasks.workunit.client.1.smithi152.stdout:2/901: creat d0/d3/df/f10c x:0 0 0 2024-03-29T04:45:31.434 INFO:tasks.workunit.client.1.smithi152.stdout:2/902: getdents d0/d3/d30/d15/d22/d4c/d5a/dff 0 2024-03-29T04:45:31.434 INFO:tasks.workunit.client.1.smithi152.stdout:0/897: dread d2/da/d17/d50/dc6/fe4 [0,4194304] 0 2024-03-29T04:45:31.435 INFO:tasks.workunit.client.1.smithi152.stdout:0/898: write d2/f38 [4830238,66677] 0 2024-03-29T04:45:31.436 INFO:tasks.workunit.client.1.smithi152.stdout:2/903: rmdir d0/d3/df/d2c/d45 39 2024-03-29T04:45:31.440 INFO:tasks.workunit.client.1.smithi152.stdout:0/899: write d2/da/d3a/d6f/f85 [3117181,83597] 0 2024-03-29T04:45:31.443 INFO:tasks.workunit.client.1.smithi152.stdout:0/900: creat d2/da/d17/de7/f13f x:0 0 0 2024-03-29T04:45:31.444 INFO:tasks.workunit.client.1.smithi152.stdout:2/904: dwrite d0/d3/df/f43 [0,4194304] 0 2024-03-29T04:45:31.445 INFO:tasks.workunit.client.1.smithi152.stdout:0/901: mknod d2/da/d17/d57/d9c/c140 0 2024-03-29T04:45:31.450 INFO:tasks.workunit.client.1.smithi152.stdout:2/905: rename d0/c8b to d0/d3/df/da7/c10d 0 2024-03-29T04:45:31.455 INFO:tasks.workunit.client.1.smithi152.stdout:2/906: dwrite d0/d3/df/fa2 [0,4194304] 0 2024-03-29T04:45:31.457 INFO:tasks.workunit.client.1.smithi152.stdout:0/902: write d2/da/d3a/f97 [117997,57519] 0 2024-03-29T04:45:31.458 INFO:tasks.workunit.client.1.smithi152.stdout:2/907: write d0/d3/d64/ffe [925318,78352] 0 2024-03-29T04:45:31.459 INFO:tasks.workunit.client.1.smithi152.stdout:2/908: fsync d0/d3/d64/fe1 0 2024-03-29T04:45:31.461 INFO:tasks.workunit.client.1.smithi152.stdout:2/909: chown d0/d3/d30/dd0 129280 1 2024-03-29T04:45:31.462 INFO:tasks.workunit.client.1.smithi152.stdout:0/903: rename d2/d31/f36 to d2/da/d17/dd8/db8/daf/ddb/f141 0 2024-03-29T04:45:31.464 INFO:tasks.workunit.client.1.smithi152.stdout:2/910: truncate d0/d3/df/fbf 603943 0 2024-03-29T04:45:31.465 INFO:tasks.workunit.client.1.smithi152.stdout:0/904: creat d2/da/d17/d57/d9c/dbe/df3/f142 x:0 0 0 2024-03-29T04:45:31.468 INFO:tasks.workunit.client.1.smithi152.stdout:0/905: creat d2/da/df/f143 x:0 0 0 2024-03-29T04:45:31.469 INFO:tasks.workunit.client.1.smithi152.stdout:2/911: symlink d0/d3/d30/d15/d53/d54/d57/l10e 0 2024-03-29T04:45:31.469 INFO:tasks.workunit.client.1.smithi152.stdout:0/906: chown d2/da/d17/d50/l58 61 1 2024-03-29T04:45:31.472 INFO:tasks.workunit.client.1.smithi152.stdout:2/912: chown d0/d3/d30/d15/d22/d74/l78 87 1 2024-03-29T04:45:31.480 INFO:tasks.workunit.client.1.smithi152.stdout:0/907: dread d2/da/d3a/d6f/f85 [0,4194304] 0 2024-03-29T04:45:31.482 INFO:tasks.workunit.client.1.smithi152.stdout:0/908: symlink d2/da/d17/d139/l144 0 2024-03-29T04:45:31.483 INFO:tasks.workunit.client.1.smithi152.stdout:2/913: dwrite d0/d3/d30/d15/d53/d54/dad/dd5/f105 [0,4194304] 0 2024-03-29T04:45:31.484 INFO:tasks.workunit.client.1.smithi152.stdout:2/914: stat d0/d3/d30/dd0 0 2024-03-29T04:45:31.489 INFO:tasks.workunit.client.1.smithi152.stdout:2/915: rename d0/d3/d30/d15/d22/d74/dbd/l7d to d0/d3/d30/d15/d53/d54/dad/dd5/l10f 0 2024-03-29T04:45:31.498 INFO:tasks.workunit.client.1.smithi152.stdout:0/909: dwrite d2/da/d17/d50/dac/fb2 [4194304,4194304] 0 2024-03-29T04:45:31.529 INFO:tasks.workunit.client.1.smithi152.stdout:0/910: creat d2/da/d3a/d61/db4/d11a/f145 x:0 0 0 2024-03-29T04:45:31.530 INFO:tasks.workunit.client.1.smithi152.stdout:2/916: unlink d0/d3/d30/d15/d53/d54/dad/dd5/ld8 0 2024-03-29T04:45:31.532 INFO:tasks.workunit.client.1.smithi152.stdout:0/911: truncate d2/da/d17/d57/d9c/dbe/df3/f142 200834 0 2024-03-29T04:45:31.532 INFO:tasks.workunit.client.1.smithi152.stdout:2/917: symlink d0/d3/d64/d85/l110 0 2024-03-29T04:45:31.536 INFO:tasks.workunit.client.1.smithi152.stdout:0/912: link d2/d29/dcf/f130 d2/f146 0 2024-03-29T04:45:31.538 INFO:tasks.workunit.client.1.smithi152.stdout:0/913: write d2/da/d3a/d61/db4/d11a/f145 [892049,77533] 0 2024-03-29T04:45:31.539 INFO:tasks.workunit.client.1.smithi152.stdout:0/914: write d2/da/d3a/d61/db4/f129 [922132,56224] 0 2024-03-29T04:45:31.541 INFO:tasks.workunit.client.1.smithi152.stdout:0/915: fsync d2/d31/fda 0 2024-03-29T04:45:31.546 INFO:tasks.workunit.client.1.smithi152.stdout:0/916: mknod d2/da/d17/d50/dac/c147 0 2024-03-29T04:45:31.547 INFO:tasks.workunit.client.1.smithi152.stdout:0/917: mknod d2/da/d17/d57/d9c/df0/c148 0 2024-03-29T04:45:31.547 INFO:tasks.workunit.client.1.smithi152.stdout:0/918: chown d2/da/lc 283 1 2024-03-29T04:45:31.548 INFO:tasks.workunit.client.1.smithi152.stdout:0/919: truncate d2/da/d3a/d61/db4/f13c 1010538 0 2024-03-29T04:45:31.549 INFO:tasks.workunit.client.1.smithi152.stdout:0/920: write d2/da/d3a/d61/fd0 [1293932,125345] 0 2024-03-29T04:45:31.554 INFO:tasks.workunit.client.1.smithi152.stdout:2/918: dwrite d0/f3b [0,4194304] 0 2024-03-29T04:45:31.556 INFO:tasks.workunit.client.1.smithi152.stdout:0/921: rename d2/da/d3a/d6f/c90 to d2/da/d17/dd8/c149 0 2024-03-29T04:45:31.558 INFO:tasks.workunit.client.1.smithi152.stdout:2/919: creat d0/d3/df/d2c/f111 x:0 0 0 2024-03-29T04:45:31.565 INFO:tasks.workunit.client.1.smithi152.stdout:2/920: rename d0/d3/df/d2c/f2d to d0/d3/d30/d15/d22/d4c/f112 0 2024-03-29T04:45:31.566 INFO:tasks.workunit.client.1.smithi152.stdout:0/922: write d2/d29/f6c [295902,114614] 0 2024-03-29T04:45:31.575 INFO:tasks.workunit.client.1.smithi152.stdout:0/923: fdatasync d2/d29/f6c 0 2024-03-29T04:45:31.576 INFO:tasks.workunit.client.1.smithi152.stdout:2/921: dread d0/d3/d30/d15/d53/d54/fcf [0,4194304] 0 2024-03-29T04:45:31.579 INFO:tasks.workunit.client.1.smithi152.stdout:2/922: mknod d0/d3/d30/d15/d22/d4c/c113 0 2024-03-29T04:45:31.580 INFO:tasks.workunit.client.1.smithi152.stdout:2/923: fsync d0/d3/d64/f101 0 2024-03-29T04:45:31.581 INFO:tasks.workunit.client.1.smithi152.stdout:0/924: dwrite d2/da/d17/d50/fb7 [0,4194304] 0 2024-03-29T04:45:31.582 INFO:tasks.workunit.client.1.smithi152.stdout:2/924: creat d0/d3/df/db9/f114 x:0 0 0 2024-03-29T04:45:31.589 INFO:tasks.workunit.client.1.smithi152.stdout:0/925: rename d2/da/d3a/d61/f9a to d2/da/d3a/d61/db4/d11d/f14a 0 2024-03-29T04:45:31.590 INFO:tasks.workunit.client.1.smithi152.stdout:0/926: fdatasync d2/da/d3a/d61/db4/d11a/f145 0 2024-03-29T04:45:31.592 INFO:tasks.workunit.client.1.smithi152.stdout:0/927: readlink d2/da/d3a/l79 0 2024-03-29T04:45:31.594 INFO:tasks.workunit.client.1.smithi152.stdout:2/925: truncate d0/d3/d30/d15/d53/d54/d57/f72 386845 0 2024-03-29T04:45:31.596 INFO:tasks.workunit.client.1.smithi152.stdout:2/926: write d0/d3/d30/d15/d22/d74/dbd/f95 [4279516,109376] 0 2024-03-29T04:45:31.597 INFO:tasks.workunit.client.1.smithi152.stdout:2/927: write d0/d3/d30/f33 [1038988,2057] 0 2024-03-29T04:45:31.599 INFO:tasks.workunit.client.1.smithi152.stdout:2/928: fdatasync d0/d3/d30/f87 0 2024-03-29T04:45:31.605 INFO:tasks.workunit.client.1.smithi152.stdout:2/929: dwrite d0/dcc/f102 [0,4194304] 0 2024-03-29T04:45:31.606 INFO:tasks.workunit.client.1.smithi152.stdout:2/930: readlink d0/d3/df/d9e/lee 0 2024-03-29T04:45:31.613 INFO:tasks.workunit.client.1.smithi152.stdout:2/931: write d0/d3/df/da7/faf [1126399,77661] 0 2024-03-29T04:45:31.616 INFO:tasks.workunit.client.1.smithi152.stdout:2/932: mknod d0/d3/d30/d15/d22/c115 0 2024-03-29T04:45:31.619 INFO:tasks.workunit.client.1.smithi152.stdout:2/933: read d0/d3/d30/d15/d22/d4c/f112 [238698,75651] 0 2024-03-29T04:45:31.622 INFO:tasks.workunit.client.1.smithi152.stdout:2/934: rename d0/d3/f10 to d0/d3/d30/d15/d53/db4/f116 0 2024-03-29T04:45:31.625 INFO:tasks.workunit.client.1.smithi152.stdout:2/935: creat d0/d3/d30/d15/f117 x:0 0 0 2024-03-29T04:45:31.626 INFO:tasks.workunit.client.1.smithi152.stdout:2/936: rmdir d0/d3/d64/d85 39 2024-03-29T04:45:31.629 INFO:tasks.workunit.client.1.smithi152.stdout:2/937: symlink d0/d3/df/d8c/l118 0 2024-03-29T04:45:31.633 INFO:tasks.workunit.client.1.smithi152.stdout:2/938: unlink d0/d3/d30/d15/d53/d54/dad/dd5/fb2 0 2024-03-29T04:45:31.638 INFO:tasks.workunit.client.1.smithi152.stdout:2/939: rmdir d0/d3/d30/d15/d53/d54/dad/dd5 39 2024-03-29T04:45:31.638 INFO:tasks.workunit.client.1.smithi152.stdout:2/940: chown d0/dcc/f102 10 1 2024-03-29T04:45:31.641 INFO:tasks.workunit.client.1.smithi152.stdout:2/941: creat d0/d3/d30/d15/da5/df5/f119 x:0 0 0 2024-03-29T04:45:31.645 INFO:tasks.workunit.client.1.smithi152.stdout:2/942: getdents d0/d3/df 0 2024-03-29T04:45:31.656 INFO:tasks.workunit.client.1.smithi152.stdout:2/943: dread d0/d3/d30/d15/d22/d4c/f112 [0,4194304] 0 2024-03-29T04:45:31.658 INFO:tasks.workunit.client.1.smithi152.stdout:2/944: write d0/d3/df/d8c/fb1 [2460160,104878] 0 2024-03-29T04:45:31.660 INFO:tasks.workunit.client.1.smithi152.stdout:2/945: creat d0/d3/d30/d15/da5/d103/f11a x:0 0 0 2024-03-29T04:45:31.662 INFO:tasks.workunit.client.1.smithi152.stdout:2/946: creat d0/f11b x:0 0 0 2024-03-29T04:45:31.663 INFO:tasks.workunit.client.1.smithi152.stdout:2/947: write d0/d3/df/f35 [1041646,70389] 0 2024-03-29T04:45:31.666 INFO:tasks.workunit.client.1.smithi152.stdout:4/830: sync 2024-03-29T04:45:31.666 INFO:tasks.workunit.client.1.smithi152.stdout:4/831: fsync d0/d27/f59 0 2024-03-29T04:45:31.667 INFO:tasks.workunit.client.1.smithi152.stdout:2/948: getdents d0/d3/d30/d15/da5/df5 0 2024-03-29T04:45:31.670 INFO:tasks.workunit.client.1.smithi152.stdout:4/832: creat d0/f107 x:0 0 0 2024-03-29T04:45:31.671 INFO:tasks.workunit.client.1.smithi152.stdout:2/949: dread d0/d3/d30/d15/d53/d54/dad/fba [0,4194304] 0 2024-03-29T04:45:31.674 INFO:tasks.workunit.client.1.smithi152.stdout:2/950: creat d0/d3/d30/d15/d22/d4c/d5a/f11c x:0 0 0 2024-03-29T04:45:31.682 INFO:tasks.workunit.client.1.smithi152.stdout:2/951: truncate d0/d3/d30/dd0/ffb 3790982 0 2024-03-29T04:45:31.684 INFO:tasks.workunit.client.1.smithi152.stdout:2/952: mkdir d0/d3/d30/d15/d22/d4c/d5a/df6/d11d 0 2024-03-29T04:45:31.685 INFO:tasks.workunit.client.1.smithi152.stdout:4/833: dread d0/d7/d40/f80 [0,4194304] 0 2024-03-29T04:45:31.686 INFO:tasks.workunit.client.1.smithi152.stdout:4/834: dread - d0/f107 zero size 2024-03-29T04:45:31.686 INFO:tasks.workunit.client.1.smithi152.stdout:2/953: chown d0/d3/cef 46 1 2024-03-29T04:45:31.687 INFO:tasks.workunit.client.1.smithi152.stdout:4/835: dread - d0/d27/d6e/d78/ff1 zero size 2024-03-29T04:45:31.688 INFO:tasks.workunit.client.1.smithi152.stdout:2/954: fdatasync d0/d3/d30/d15/f26 0 2024-03-29T04:45:31.690 INFO:tasks.workunit.client.1.smithi152.stdout:4/836: mknod d0/d7/d25/d17/d5e/d60/d74/d57/c108 0 2024-03-29T04:45:31.697 INFO:tasks.workunit.client.1.smithi152.stdout:4/837: truncate d0/d7/d25/d17/d5e/d60/d74/da2/da6/fa9 2476093 0 2024-03-29T04:45:31.701 INFO:tasks.workunit.client.1.smithi152.stdout:2/955: dwrite d0/d3/d30/fd3 [0,4194304] 0 2024-03-29T04:45:31.706 INFO:tasks.workunit.client.1.smithi152.stdout:4/838: truncate d0/d7/d25/d17/f3b 316679 0 2024-03-29T04:45:31.708 INFO:tasks.workunit.client.1.smithi152.stdout:2/956: write d0/d3/d30/d15/d22/d4c/d5a/fa4 [655381,98993] 0 2024-03-29T04:45:31.710 INFO:tasks.workunit.client.1.smithi152.stdout:2/957: chown d0/d3/d30/d15/d22/d74/dbd/f95 3 1 2024-03-29T04:45:31.712 INFO:tasks.workunit.client.1.smithi152.stdout:2/958: symlink d0/d3/d30/d15/d22/d4c/d5a/l11e 0 2024-03-29T04:45:31.715 INFO:tasks.workunit.client.1.smithi152.stdout:2/959: rename d0/d3/d30/d15/d22/d4c/d5a to d0/d3/d30/d15/d22/d4c/d5a/d11f 22 2024-03-29T04:45:31.717 INFO:tasks.workunit.client.1.smithi152.stdout:2/960: dread d0/d3/df/f43 [0,4194304] 0 2024-03-29T04:45:31.720 INFO:tasks.workunit.client.1.smithi152.stdout:2/961: rename d0/d3/d30/d15/d53/d54/d57/lcd to d0/d3/d30/d15/d22/d4c/l120 0 2024-03-29T04:45:31.721 INFO:tasks.workunit.client.1.smithi152.stdout:2/962: chown d0/d3/d30/d15/d53/l66 85 1 2024-03-29T04:45:31.722 INFO:tasks.workunit.client.1.smithi152.stdout:2/963: readlink d0/d3/d30/d15/d22/d4c/la8 0 2024-03-29T04:45:31.727 INFO:tasks.workunit.client.1.smithi152.stdout:2/964: dwrite d0/d3/f18 [4194304,4194304] 0 2024-03-29T04:45:31.741 INFO:tasks.workunit.client.1.smithi152.stdout:2/965: dwrite d0/d3/d30/d15/f29 [0,4194304] 0 2024-03-29T04:45:31.746 INFO:tasks.workunit.client.1.smithi152.stdout:2/966: creat d0/d3/d30/d15/d22/d4c/f121 x:0 0 0 2024-03-29T04:45:31.748 INFO:tasks.workunit.client.1.smithi152.stdout:2/967: chown d0/d3/d30/d15/d22/d74/dbd/c65 2045 1 2024-03-29T04:45:31.763 INFO:tasks.workunit.client.1.smithi152.stdout:2/968: dwrite d0/d3/d30/f3f [0,4194304] 0 2024-03-29T04:45:31.771 INFO:tasks.workunit.client.1.smithi152.stdout:2/969: dwrite d0/f16 [0,4194304] 0 2024-03-29T04:45:31.773 INFO:tasks.workunit.client.1.smithi152.stdout:2/970: write d0/d3/d30/d15/da5/d103/f11a [529413,85187] 0 2024-03-29T04:45:31.780 INFO:tasks.workunit.client.1.smithi152.stdout:2/971: link d0/d3/d30/d15/d22/f37 d0/d3/d30/d15/d22/f122 0 2024-03-29T04:45:31.785 INFO:tasks.workunit.client.1.smithi152.stdout:2/972: link d0/d3/d30/d15/d22/d4c/f112 d0/d3/d30/d15/d53/d54/f123 0 2024-03-29T04:45:31.788 INFO:tasks.workunit.client.1.smithi152.stdout:2/973: rename d0/d3/d30/d15/d22/d4c/d5a/l11e to d0/d3/d30/d15/d22/d4c/d5a/df6/d11d/l124 0 2024-03-29T04:45:31.794 INFO:tasks.workunit.client.1.smithi152.stdout:2/974: write d0/d3/d30/d15/d53/fc3 [1467884,80064] 0 2024-03-29T04:45:31.797 INFO:tasks.workunit.client.1.smithi152.stdout:2/975: creat d0/d3/df/d9e/f125 x:0 0 0 2024-03-29T04:45:31.798 INFO:tasks.workunit.client.1.smithi152.stdout:2/976: rmdir d0/d3/d30/d15/da5/df5 39 2024-03-29T04:45:31.805 INFO:tasks.workunit.client.1.smithi152.stdout:2/977: dwrite d0/d3/d30/d15/d22/f91 [0,4194304] 0 2024-03-29T04:45:31.811 INFO:tasks.workunit.client.1.smithi152.stdout:2/978: rename d0/d3/d30/d15/d22/d4c/l120 to d0/d3/df/da7/l126 0 2024-03-29T04:45:31.815 INFO:tasks.workunit.client.1.smithi152.stdout:2/979: rename d0/d3/d94/cc9 to d0/d3/d30/d15/da5/df5/c127 0 2024-03-29T04:45:31.817 INFO:tasks.workunit.client.1.smithi152.stdout:2/980: write d0/d3/d64/d85/fe4 [1588578,108466] 0 2024-03-29T04:45:31.818 INFO:tasks.workunit.client.1.smithi152.stdout:2/981: stat d0/d3/d64/d85/f10a 0 2024-03-29T04:45:31.823 INFO:tasks.workunit.client.1.smithi152.stdout:2/982: rename d0/d3/df/d8c/fb1 to d0/d3/d30/d15/d22/d4c/f128 0 2024-03-29T04:45:31.826 INFO:tasks.workunit.client.1.smithi152.stdout:2/983: mknod d0/d3/d30/d15/d53/d54/dad/c129 0 2024-03-29T04:45:31.826 INFO:tasks.workunit.client.1.smithi152.stdout:2/984: stat d0/d3/d64/d85/f10a 0 2024-03-29T04:45:31.829 INFO:tasks.workunit.client.1.smithi152.stdout:2/985: rename d0/d3/df/da7/le7 to d0/dcc/l12a 0 2024-03-29T04:45:31.831 INFO:tasks.workunit.client.1.smithi152.stdout:2/986: fdatasync d0/d3/d30/d15/d22/d74/dbd/f79 0 2024-03-29T04:45:31.833 INFO:tasks.workunit.client.1.smithi152.stdout:2/987: mknod d0/d3/d30/d15/da5/d103/c12b 0 2024-03-29T04:45:31.837 INFO:tasks.workunit.client.1.smithi152.stdout:2/988: getdents d0/d3/df/d2c 0 2024-03-29T04:45:31.840 INFO:tasks.workunit.client.1.smithi152.stdout:2/989: creat d0/d3/d30/d15/d53/d54/f12c x:0 0 0 2024-03-29T04:45:31.841 INFO:tasks.workunit.client.1.smithi152.stdout:2/990: chown d0/d3/d30/d15/d22/d4c/f4d 22855 1 2024-03-29T04:45:31.842 INFO:tasks.workunit.client.1.smithi152.stdout:2/991: fsync d0/d3/f81 0 2024-03-29T04:45:31.848 INFO:tasks.workunit.client.1.smithi152.stdout:2/992: dwrite d0/d3/d30/f19 [4194304,4194304] 0 2024-03-29T04:45:31.874 INFO:tasks.workunit.client.1.smithi152.stdout:2/993: write d0/d3/d30/d15/d22/d4c/f4d [3787873,68602] 0 2024-03-29T04:45:31.877 INFO:tasks.workunit.client.1.smithi152.stdout:2/994: read d0/d3/df/d2c/fa6 [2301378,32973] 0 2024-03-29T04:45:31.886 INFO:tasks.workunit.client.1.smithi152.stdout:2/995: dwrite d0/d3/df/f76 [4194304,4194304] 0 2024-03-29T04:45:31.888 INFO:tasks.workunit.client.1.smithi152.stdout:2/996: chown d0/d3/d30/c3d 59730079 1 2024-03-29T04:45:31.889 INFO:tasks.workunit.client.1.smithi152.stdout:2/997: rmdir d0/d3/df/d9e/dce 39 2024-03-29T04:45:31.893 INFO:tasks.workunit.client.1.smithi152.stdout:2/998: mkdir d0/d3/d30/d15/d22/d12d 0 2024-03-29T04:45:31.901 INFO:tasks.workunit.client.1.smithi152.stdout:2/999: unlink d0/d3/d30/d15/d22/d4c/f90 0 2024-03-29T04:45:32.109 INFO:tasks.workunit.client.1.smithi152.stdout:4/839: sync 2024-03-29T04:45:32.109 INFO:tasks.workunit.client.1.smithi152.stdout:0/928: sync 2024-03-29T04:45:32.112 INFO:tasks.workunit.client.1.smithi152.stdout:4/840: creat d0/f109 x:0 0 0 2024-03-29T04:45:32.113 INFO:tasks.workunit.client.1.smithi152.stdout:4/841: chown d0/d7/d25/d17/d5e/l76 1803 1 2024-03-29T04:45:32.115 INFO:tasks.workunit.client.1.smithi152.stdout:0/929: truncate d2/f13a 2413942 0 2024-03-29T04:45:32.116 INFO:tasks.workunit.client.1.smithi152.stdout:0/930: read d2/da/df/dbb/f10c [125128,15709] 0 2024-03-29T04:45:32.117 INFO:tasks.workunit.client.1.smithi152.stdout:4/842: dread d0/d7/d40/f67 [0,4194304] 0 2024-03-29T04:45:32.120 INFO:tasks.workunit.client.1.smithi152.stdout:4/843: creat d0/d7/d40/d23/d6a/dfe/f10a x:0 0 0 2024-03-29T04:45:32.121 INFO:tasks.workunit.client.1.smithi152.stdout:0/931: write d2/d31/fda [1913273,26251] 0 2024-03-29T04:45:32.122 INFO:tasks.workunit.client.1.smithi152.stdout:4/844: mkdir d0/d7/d40/dce/d10b 0 2024-03-29T04:45:32.486 INFO:tasks.workunit.client.1.smithi152.stdout:0/932: sync 2024-03-29T04:45:32.487 INFO:tasks.workunit.client.1.smithi152.stdout:0/933: link d2/da/d17/d50/l7d d2/da/d17/d57/d9c/d13e/l14b 0 2024-03-29T04:45:32.489 INFO:tasks.workunit.client.1.smithi152.stdout:0/934: mknod d2/da/d3a/c14c 0 2024-03-29T04:45:32.494 INFO:tasks.workunit.client.1.smithi152.stdout:0/935: getdents d2/da/d17/d57/d9c/dbe/df3 0 2024-03-29T04:45:32.500 INFO:tasks.workunit.client.1.smithi152.stdout:0/936: mknod d2/da/d17/d57/d136/c14d 0 2024-03-29T04:45:32.500 INFO:tasks.workunit.client.1.smithi152.stdout:0/937: chown d2/da/d3a/d6f/fc9 903344 1 2024-03-29T04:45:32.505 INFO:tasks.workunit.client.1.smithi152.stdout:0/938: dwrite d2/da/df/f4b [4194304,4194304] 0 2024-03-29T04:45:32.542 INFO:tasks.workunit.client.1.smithi152.stdout:4/845: sync 2024-03-29T04:45:32.548 INFO:tasks.workunit.client.1.smithi152.stdout:0/939: creat d2/f14e x:0 0 0 2024-03-29T04:45:32.548 INFO:tasks.workunit.client.1.smithi152.stdout:4/846: dwrite d0/d27/d6e/d8c/fe4 [0,4194304] 0 2024-03-29T04:45:32.554 INFO:tasks.workunit.client.1.smithi152.stdout:4/847: creat d0/d7/d40/dce/d10b/f10c x:0 0 0 2024-03-29T04:45:32.573 INFO:tasks.workunit.client.1.smithi152.stdout:0/940: write d2/da/d17/f77 [499864,37618] 0 2024-03-29T04:45:32.576 INFO:tasks.workunit.client.1.smithi152.stdout:0/941: dread d2/da/d3a/d61/db4/f13c [0,4194304] 0 2024-03-29T04:45:32.577 INFO:tasks.workunit.client.1.smithi152.stdout:4/848: dwrite d0/d27/f2b [0,4194304] 0 2024-03-29T04:45:32.581 INFO:tasks.workunit.client.1.smithi152.stdout:0/942: dread - d2/da/d3a/d61/fd6 zero size 2024-03-29T04:45:32.582 INFO:tasks.workunit.client.1.smithi152.stdout:4/849: fsync d0/d7/d40/f80 0 2024-03-29T04:45:32.585 INFO:tasks.workunit.client.1.smithi152.stdout:0/943: rmdir d2/da/d17/dd8 39 2024-03-29T04:45:32.587 INFO:tasks.workunit.client.1.smithi152.stdout:4/850: symlink d0/d7/d25/d17/d5e/d60/d74/d57/l10d 0 2024-03-29T04:45:32.589 INFO:tasks.workunit.client.1.smithi152.stdout:0/944: truncate d2/da/d17/f6d 1586547 0 2024-03-29T04:45:32.599 INFO:tasks.workunit.client.1.smithi152.stdout:0/945: write d2/da/df/f106 [768258,76971] 0 2024-03-29T04:45:32.601 INFO:tasks.workunit.client.1.smithi152.stdout:0/946: chown d2/da/d17/de7 1649 1 2024-03-29T04:45:32.603 INFO:tasks.workunit.client.1.smithi152.stdout:4/851: dwrite d0/d7/d40/d23/d6a/fa5 [0,4194304] 0 2024-03-29T04:45:32.604 INFO:tasks.workunit.client.1.smithi152.stdout:4/852: chown d0/d7/d40/d23/d55/d70/d96/la7 239211 1 2024-03-29T04:45:32.605 INFO:tasks.workunit.client.1.smithi152.stdout:0/947: unlink d2/f8 0 2024-03-29T04:45:32.606 INFO:tasks.workunit.client.1.smithi152.stdout:0/948: dread - d2/f11c zero size 2024-03-29T04:45:32.608 INFO:tasks.workunit.client.1.smithi152.stdout:0/949: rmdir d2/da/d17/d57/d136 39 2024-03-29T04:45:32.613 INFO:tasks.workunit.client.1.smithi152.stdout:4/853: getdents d0 0 2024-03-29T04:45:32.615 INFO:tasks.workunit.client.1.smithi152.stdout:4/854: unlink d0/d7/d25/d17/d5e/d60/d74/d57/f47 0 2024-03-29T04:45:32.618 INFO:tasks.workunit.client.1.smithi152.stdout:4/855: symlink d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/l10e 0 2024-03-29T04:45:32.621 INFO:tasks.workunit.client.1.smithi152.stdout:0/950: write d2/da/d17/dd8/f9e [243135,117978] 0 2024-03-29T04:45:32.624 INFO:tasks.workunit.client.1.smithi152.stdout:0/951: creat d2/da/d17/dd8/db8/f14f x:0 0 0 2024-03-29T04:45:32.627 INFO:tasks.workunit.client.1.smithi152.stdout:0/952: symlink d2/da/d17/dd8/db8/daf/ddb/d11e/l150 0 2024-03-29T04:45:32.628 INFO:tasks.workunit.client.1.smithi152.stdout:0/953: dread - d2/da/d17/d57/dc5/d10a/f13b zero size 2024-03-29T04:45:32.628 INFO:tasks.workunit.client.1.smithi152.stdout:4/856: truncate d0/d27/f2b 718329 0 2024-03-29T04:45:32.629 INFO:tasks.workunit.client.1.smithi152.stdout:0/954: chown d2/da/d17/d50/dac/c119 7068008 1 2024-03-29T04:45:32.631 INFO:tasks.workunit.client.1.smithi152.stdout:0/955: rmdir d2/da/d17/d57/d9c/d13e/df2 39 2024-03-29T04:45:32.640 INFO:tasks.workunit.client.1.smithi152.stdout:4/857: dwrite d0/d7/d25/d17/d5e/fd9 [0,4194304] 0 2024-03-29T04:45:32.643 INFO:tasks.workunit.client.1.smithi152.stdout:0/956: dwrite d2/da/d17/d57/d9c/fad [0,4194304] 0 2024-03-29T04:45:32.648 INFO:tasks.workunit.client.1.smithi152.stdout:4/858: rename d0/d7/d40/d23/d55/d70/d96/la7 to d0/d7/d40/dce/l10f 0 2024-03-29T04:45:32.652 INFO:tasks.workunit.client.1.smithi152.stdout:4/859: creat d0/d27/d6e/d8c/f110 x:0 0 0 2024-03-29T04:45:32.655 INFO:tasks.workunit.client.1.smithi152.stdout:4/860: mkdir d0/d7/d25/d17/d65/d111 0 2024-03-29T04:45:32.659 INFO:tasks.workunit.client.1.smithi152.stdout:4/861: fsync d0/d7/d25/f36 0 2024-03-29T04:45:32.661 INFO:tasks.workunit.client.1.smithi152.stdout:4/862: mknod d0/d27/d6e/d78/c112 0 2024-03-29T04:45:32.663 INFO:tasks.workunit.client.1.smithi152.stdout:4/863: mkdir d0/d7/d25/d17/d65/d111/d113 0 2024-03-29T04:45:32.667 INFO:tasks.workunit.client.1.smithi152.stdout:4/864: dread d0/d7/db2/ff5 [0,4194304] 0 2024-03-29T04:45:32.672 INFO:tasks.workunit.client.1.smithi152.stdout:4/865: link d0/d7/d25/d17/d5e/d60/f9b d0/d7/d25/d17/d5e/f114 0 2024-03-29T04:45:32.676 INFO:tasks.workunit.client.1.smithi152.stdout:4/866: rmdir d0/d7/d25/d17/d37/ddd 0 2024-03-29T04:45:32.678 INFO:tasks.workunit.client.1.smithi152.stdout:4/867: write d0/d7/d25/d17/d5e/d60/f9b [2375692,31144] 0 2024-03-29T04:45:32.679 INFO:tasks.workunit.client.1.smithi152.stdout:4/868: fsync d0/d7/d25/d17/f71 0 2024-03-29T04:45:32.687 INFO:tasks.workunit.client.1.smithi152.stdout:4/869: dread d0/d27/f59 [0,4194304] 0 2024-03-29T04:45:32.688 INFO:tasks.workunit.client.1.smithi152.stdout:4/870: write d0/d7/d25/d17/d5e/fd9 [3751013,114615] 0 2024-03-29T04:45:32.696 INFO:tasks.workunit.client.1.smithi152.stdout:4/871: link d0/d7/d40/d2a/f94 d0/d7/d25/d17/d5e/d60/d74/da2/da6/f115 0 2024-03-29T04:45:32.699 INFO:tasks.workunit.client.1.smithi152.stdout:4/872: mknod d0/d27/d6e/d78/c116 0 2024-03-29T04:45:32.710 INFO:tasks.workunit.client.1.smithi152.stdout:4/873: truncate d0/d7/d40/d23/d55/d70/d96/dc0/df8/f7d 835244 0 2024-03-29T04:45:32.712 INFO:tasks.workunit.client.1.smithi152.stdout:4/874: dread d0/d27/f59 [0,4194304] 0 2024-03-29T04:45:32.712 INFO:tasks.workunit.client.1.smithi152.stdout:4/875: chown d0/d27/d6e/d8c 505824563 1 2024-03-29T04:45:32.713 INFO:tasks.workunit.client.1.smithi152.stdout:4/876: chown d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd 64128436 1 2024-03-29T04:45:32.714 INFO:tasks.workunit.client.1.smithi152.stdout:4/877: readlink d0/d27/ld8 0 2024-03-29T04:45:32.716 INFO:tasks.workunit.client.1.smithi152.stdout:4/878: write d0/d7/d40/d23/d6a/dfe/f10a [129667,114156] 0 2024-03-29T04:45:32.791 INFO:tasks.workunit.client.1.smithi152.stdout:0/957: sync 2024-03-29T04:45:32.792 INFO:tasks.workunit.client.1.smithi152.stdout:0/958: dread - d2/da/d3a/d61/fd6 zero size 2024-03-29T04:45:32.797 INFO:tasks.workunit.client.1.smithi152.stdout:0/959: link d2/f13a d2/da/d17/d57/d9c/dbe/f151 0 2024-03-29T04:45:32.809 INFO:tasks.workunit.client.1.smithi152.stdout:0/960: dread d2/da/d17/d57/f62 [0,4194304] 0 2024-03-29T04:45:32.812 INFO:tasks.workunit.client.1.smithi152.stdout:0/961: mkdir d2/da/df/d152 0 2024-03-29T04:45:32.814 INFO:tasks.workunit.client.1.smithi152.stdout:0/962: symlink d2/da/d17/dd8/db8/daf/l153 0 2024-03-29T04:45:32.821 INFO:tasks.workunit.client.1.smithi152.stdout:0/963: write d2/f20 [2408168,57201] 0 2024-03-29T04:45:32.827 INFO:tasks.workunit.client.1.smithi152.stdout:0/964: write d2/da/d3a/d61/db4/ffb [4297950,111729] 0 2024-03-29T04:45:32.828 INFO:tasks.workunit.client.1.smithi152.stdout:0/965: chown d2/da/d17/d57/cd9 1 1 2024-03-29T04:45:32.836 INFO:tasks.workunit.client.1.smithi152.stdout:0/966: getdents d2/da/d17/d50/dc6 0 2024-03-29T04:45:32.843 INFO:tasks.workunit.client.1.smithi152.stdout:0/967: dwrite d2/d29/dcf/f124 [0,4194304] 0 2024-03-29T04:45:32.846 INFO:tasks.workunit.client.1.smithi152.stdout:0/968: fdatasync d2/da/d17/d57/d9c/d13e/f10e 0 2024-03-29T04:45:32.849 INFO:tasks.workunit.client.1.smithi152.stdout:0/969: read d2/da/d17/d57/d9c/dbe/df3/f10b [599570,101046] 0 2024-03-29T04:45:32.858 INFO:tasks.workunit.client.1.smithi152.stdout:0/970: link c0 d2/da/d17/d50/dac/c154 0 2024-03-29T04:45:32.862 INFO:tasks.workunit.client.1.smithi152.stdout:0/971: dread d2/da/d17/d57/f62 [0,4194304] 0 2024-03-29T04:45:32.863 INFO:tasks.workunit.client.1.smithi152.stdout:0/972: dread - d2/f11c zero size 2024-03-29T04:45:32.864 INFO:tasks.workunit.client.1.smithi152.stdout:0/973: read d2/da/d3a/d61/db4/d11a/f145 [344302,34162] 0 2024-03-29T04:45:32.865 INFO:tasks.workunit.client.1.smithi152.stdout:0/974: fsync d2/da/d3a/d61/db4/f132 0 2024-03-29T04:45:32.867 INFO:tasks.workunit.client.1.smithi152.stdout:0/975: mknod d2/d29/c155 0 2024-03-29T04:45:32.878 INFO:tasks.workunit.client.1.smithi152.stdout:0/976: write d2/da/df/fd5 [196696,19791] 0 2024-03-29T04:45:32.886 INFO:tasks.workunit.client.1.smithi152.stdout:0/977: link d2/da/d17/d57/d9c/f9f d2/da/d17/d50/dac/f156 0 2024-03-29T04:45:32.889 INFO:tasks.workunit.client.1.smithi152.stdout:0/978: creat d2/da/d17/de7/f157 x:0 0 0 2024-03-29T04:45:32.892 INFO:tasks.workunit.client.1.smithi152.stdout:0/979: mkdir d2/da/d17/d57/d9c/df0/d158 0 2024-03-29T04:45:32.910 INFO:tasks.workunit.client.1.smithi152.stdout:4/879: sync 2024-03-29T04:45:32.912 INFO:tasks.workunit.client.1.smithi152.stdout:0/980: getdents d2/da/d17/d57/d9c/df0 0 2024-03-29T04:45:32.913 INFO:tasks.workunit.client.1.smithi152.stdout:0/981: readlink d2/da/d17/dd8/db8/daf/ddb/d11e/l150 0 2024-03-29T04:45:32.913 INFO:tasks.workunit.client.1.smithi152.stdout:0/982: chown d2/da/df/fd5 1990992 1 2024-03-29T04:45:32.914 INFO:tasks.workunit.client.1.smithi152.stdout:4/880: rename d0/d7/d40/d62 to d0/d27/d6e/d78/d117 0 2024-03-29T04:45:32.915 INFO:tasks.workunit.client.1.smithi152.stdout:0/983: chown d2/da/d17/d50/dc6/d12e/l12f 1 1 2024-03-29T04:45:32.918 INFO:tasks.workunit.client.1.smithi152.stdout:0/984: fdatasync d2/f7a 0 2024-03-29T04:45:32.922 INFO:tasks.workunit.client.1.smithi152.stdout:0/985: rename d2/da/d17/d50/dc6/d12c/f12d to d2/da/d17/d50/dc6/d12e/f159 0 2024-03-29T04:45:32.927 INFO:tasks.workunit.client.1.smithi152.stdout:4/881: dread d0/d7/f8d [4194304,4194304] 0 2024-03-29T04:45:32.929 INFO:tasks.workunit.client.1.smithi152.stdout:4/882: symlink d0/d7/d40/d23/d55/d70/d96/dc0/l118 0 2024-03-29T04:45:32.931 INFO:tasks.workunit.client.1.smithi152.stdout:4/883: mkdir d0/d7/d25/d17/d5e/d119 0 2024-03-29T04:45:32.935 INFO:tasks.workunit.client.1.smithi152.stdout:4/884: dread d0/d7/d25/f3f [0,4194304] 0 2024-03-29T04:45:32.935 INFO:tasks.workunit.client.1.smithi152.stdout:0/986: dread d2/da/d17/dd8/db8/daf/ddb/f141 [4194304,4194304] 0 2024-03-29T04:45:32.938 INFO:tasks.workunit.client.1.smithi152.stdout:0/987: creat d2/da/d17/d50/dc6/d12e/f15a x:0 0 0 2024-03-29T04:45:32.942 INFO:tasks.workunit.client.1.smithi152.stdout:4/885: link d0/d7/d40/c20 d0/d7/d25/d17/d65/d111/d113/c11a 0 2024-03-29T04:45:32.944 INFO:tasks.workunit.client.1.smithi152.stdout:4/886: rmdir d0/d7/d40/d23/dda 39 2024-03-29T04:45:32.945 INFO:tasks.workunit.client.1.smithi152.stdout:0/988: link d2/da/d17/d50/dc6/lf5 d2/da/d17/d57/dc5/d10a/l15b 0 2024-03-29T04:45:32.947 INFO:tasks.workunit.client.1.smithi152.stdout:4/887: truncate d0/d7/d40/f8a 1153850 0 2024-03-29T04:45:32.949 INFO:tasks.workunit.client.1.smithi152.stdout:0/989: rename d2/da/d17/dd8/db8/daf/ddb/d11e/l150 to d2/da/d17/d50/dac/l15c 0 2024-03-29T04:45:32.952 INFO:tasks.workunit.client.1.smithi152.stdout:4/888: truncate d0/d7/d25/f3f 105421 0 2024-03-29T04:45:32.954 INFO:tasks.workunit.client.1.smithi152.stdout:0/990: rename d2/da/d17/de7/f157 to d2/da/d3a/d61/db4/d11d/f15d 0 2024-03-29T04:45:32.956 INFO:tasks.workunit.client.1.smithi152.stdout:4/889: rename d0/d7/d40/d23/f104 to d0/d7/d25/d17/d37/f11b 0 2024-03-29T04:45:32.957 INFO:tasks.workunit.client.1.smithi152.stdout:4/890: read d0/d7/d40/d23/d55/d70/d96/f9f [318708,43338] 0 2024-03-29T04:45:32.960 INFO:tasks.workunit.client.1.smithi152.stdout:4/891: write d0/d7/d40/d23/dda/ff4 [748691,32247] 0 2024-03-29T04:45:32.965 INFO:tasks.workunit.client.1.smithi152.stdout:4/892: creat d0/d7/d40/d23/d55/d70/d96/dc0/df8/f11c x:0 0 0 2024-03-29T04:45:32.967 INFO:tasks.workunit.client.1.smithi152.stdout:4/893: creat d0/d7/d40/d23/dda/f11d x:0 0 0 2024-03-29T04:45:32.970 INFO:tasks.workunit.client.1.smithi152.stdout:0/991: dwrite d2/da/d17/d57/d9c/d13e/f115 [4194304,4194304] 0 2024-03-29T04:45:32.977 INFO:tasks.workunit.client.1.smithi152.stdout:4/894: rename d0/d7/d25/d17/d5e/d60/d74/d57/c61 to d0/d27/d6e/c11e 0 2024-03-29T04:45:32.982 INFO:tasks.workunit.client.1.smithi152.stdout:0/992: dwrite d2/da/d17/d50/dc6/d12e/f15a [0,4194304] 0 2024-03-29T04:45:32.983 INFO:tasks.workunit.client.1.smithi152.stdout:4/895: dread - d0/d7/d40/d23/dda/fdc zero size 2024-03-29T04:45:32.983 INFO:tasks.workunit.client.1.smithi152.stdout:4/896: mkdir d0/d27/d11f 0 2024-03-29T04:45:32.984 INFO:tasks.workunit.client.1.smithi152.stdout:4/897: fdatasync d0/d7/d25/d17/d5e/d60/d74/da2/fcf 0 2024-03-29T04:45:32.985 INFO:tasks.workunit.client.1.smithi152.stdout:0/993: mknod d2/da/d3a/c15e 0 2024-03-29T04:45:32.988 INFO:tasks.workunit.client.1.smithi152.stdout:4/898: rename d0/d7/d25/d17/d5e/l76 to d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/de7/l120 0 2024-03-29T04:45:32.989 INFO:tasks.workunit.client.1.smithi152.stdout:4/899: chown d0/d7/d25/d17/d5e/d60/d74/fdf 35317 1 2024-03-29T04:45:32.990 INFO:tasks.workunit.client.1.smithi152.stdout:0/994: creat d2/da/d17/d57/d9c/df0/f15f x:0 0 0 2024-03-29T04:45:32.990 INFO:tasks.workunit.client.1.smithi152.stdout:0/995: fsync d2/da/d17/de7/f13f 0 2024-03-29T04:45:32.993 INFO:tasks.workunit.client.1.smithi152.stdout:4/900: dread d0/d27/d6e/f97 [0,4194304] 0 2024-03-29T04:45:32.994 INFO:tasks.workunit.client.1.smithi152.stdout:0/996: symlink d2/da/d17/d57/l160 0 2024-03-29T04:45:32.997 INFO:tasks.workunit.client.1.smithi152.stdout:0/997: symlink d2/da/d17/d57/dc5/d10a/l161 0 2024-03-29T04:45:32.997 INFO:tasks.workunit.client.1.smithi152.stdout:0/998: dread - d2/f7a zero size 2024-03-29T04:45:32.998 INFO:tasks.workunit.client.1.smithi152.stdout:0/999: chown d2/da/l66 1867 1 2024-03-29T04:45:33.002 INFO:tasks.workunit.client.1.smithi152.stdout:4/901: write d0/d7/d40/d23/dda/de3/f101 [3622594,119882] 0 2024-03-29T04:45:33.003 INFO:tasks.workunit.client.1.smithi152.stdout:4/902: chown d0/d7/d40/d23/d6a/fc3 97965 1 2024-03-29T04:45:33.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:32 smithi152 ceph-mon[37177]: pgmap v282: 65 pgs: 65 active+clean; 2.9 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 64 MiB/s rd, 155 MiB/s wr, 405 op/s 2024-03-29T04:45:33.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:32 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:45:33.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:32 smithi142 ceph-mon[29027]: pgmap v282: 65 pgs: 65 active+clean; 2.9 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 64 MiB/s rd, 155 MiB/s wr, 405 op/s 2024-03-29T04:45:33.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:32 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:45:33.228 INFO:tasks.workunit.client.1.smithi152.stdout:4/903: sync 2024-03-29T04:45:33.231 INFO:tasks.workunit.client.1.smithi152.stdout:4/904: mknod d0/d7/d40/dce/d10b/c121 0 2024-03-29T04:45:33.234 INFO:tasks.workunit.client.1.smithi152.stdout:4/905: rmdir d0/d7/d25/d17/d37/d77 39 2024-03-29T04:45:33.238 INFO:tasks.workunit.client.1.smithi152.stdout:4/906: rename d0/d7/d40/d6d/c90 to d0/d7/d40/d23/d55/c122 0 2024-03-29T04:45:33.243 INFO:tasks.workunit.client.1.smithi152.stdout:4/907: rename d0/d27/d6e/c11e to d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/de7/c123 0 2024-03-29T04:45:33.247 INFO:tasks.workunit.client.1.smithi152.stdout:4/908: rmdir d0/d7/d40/d23/d55/d70/d96/dc0/df8 39 2024-03-29T04:45:33.258 INFO:tasks.workunit.client.1.smithi152.stdout:4/909: dwrite d0/d7/d25/d17/d5e/d60/d74/da2/da6/f115 [0,4194304] 0 2024-03-29T04:45:33.261 INFO:tasks.workunit.client.1.smithi152.stdout:4/910: dread - d0/d7/d25/d17/d5e/d60/d74/da2/fcf zero size 2024-03-29T04:45:33.261 INFO:tasks.workunit.client.1.smithi152.stdout:4/911: chown d0/d7/d40/d23/d55/d70 14958113 1 2024-03-29T04:45:33.266 INFO:tasks.workunit.client.1.smithi152.stdout:4/912: rename d0/d7/d40/d23/d55/d70/d96/dc0/df8/fc2 to d0/d7/d40/d23/dda/f124 0 2024-03-29T04:45:33.273 INFO:tasks.workunit.client.1.smithi152.stdout:4/913: dwrite d0/d7/d40/d23/f28 [4194304,4194304] 0 2024-03-29T04:45:33.289 INFO:tasks.workunit.client.1.smithi152.stdout:4/914: write d0/d7/d40/d23/dda/f124 [4806155,42685] 0 2024-03-29T04:45:33.293 INFO:tasks.workunit.client.1.smithi152.stdout:4/915: mknod d0/d7/d40/d23/d55/d6c/d91/d93/c125 0 2024-03-29T04:45:33.293 INFO:tasks.workunit.client.1.smithi152.stdout:4/916: chown d0/d7/d40/d2a/faa 32441245 1 2024-03-29T04:45:33.305 INFO:tasks.workunit.client.1.smithi152.stdout:4/917: write d0/d7/d25/d17/d5e/d60/d74/f50 [142429,86661] 0 2024-03-29T04:45:33.313 INFO:tasks.workunit.client.1.smithi152.stdout:4/918: dwrite d0/d7/d40/d23/d55/d70/d96/fcb [0,4194304] 0 2024-03-29T04:45:33.314 INFO:tasks.workunit.client.1.smithi152.stdout:4/919: stat d0/d27/d6e/d8c/lb6 0 2024-03-29T04:45:33.316 INFO:tasks.workunit.client.1.smithi152.stdout:4/920: mkdir d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/d126 0 2024-03-29T04:45:33.318 INFO:tasks.workunit.client.1.smithi152.stdout:4/921: write d0/d7/d40/d2a/f42 [840313,72216] 0 2024-03-29T04:45:33.338 INFO:tasks.workunit.client.1.smithi152.stdout:4/922: dread d0/d7/d25/d17/d65/f79 [0,4194304] 0 2024-03-29T04:45:33.339 INFO:tasks.workunit.client.1.smithi152.stdout:4/923: stat d0/d7/d25 0 2024-03-29T04:45:33.343 INFO:tasks.workunit.client.1.smithi152.stdout:4/924: getdents d0/d7/d40/d23/d6a 0 2024-03-29T04:45:33.344 INFO:tasks.workunit.client.1.smithi152.stdout:4/925: readlink d0/d7/d25/d17/d5e/d60/d74/d57/lb0 0 2024-03-29T04:45:33.347 INFO:tasks.workunit.client.1.smithi152.stdout:4/926: write d0/d7/d25/d17/d37/d77/fee [880919,121166] 0 2024-03-29T04:45:33.360 INFO:tasks.workunit.client.1.smithi152.stdout:4/927: dread d0/d7/fab [0,4194304] 0 2024-03-29T04:45:33.362 INFO:tasks.workunit.client.1.smithi152.stdout:4/928: truncate d0/d7/d25/d17/d37/d77/fd2 4500022 0 2024-03-29T04:45:33.368 INFO:tasks.workunit.client.1.smithi152.stdout:4/929: dwrite d0/d27/f102 [0,4194304] 0 2024-03-29T04:45:33.371 INFO:tasks.workunit.client.1.smithi152.stdout:4/930: rename d0/d7/d40/d23/d6a/fc3 to d0/d7/d25/d17/d5e/d60/d74/f127 0 2024-03-29T04:45:33.372 INFO:tasks.workunit.client.1.smithi152.stdout:4/931: chown d0/d7/d25/d17/d5e/cbf 0 1 2024-03-29T04:45:33.373 INFO:tasks.workunit.client.1.smithi152.stdout:4/932: chown d0/d7/d40/d23/d55/d6c/f9e 130 1 2024-03-29T04:45:33.375 INFO:tasks.workunit.client.1.smithi152.stdout:4/933: write d0/d27/f59 [1176797,2674] 0 2024-03-29T04:45:33.377 INFO:tasks.workunit.client.1.smithi152.stdout:4/934: stat d0/d7/d40/d23/d55/d70/f7e 0 2024-03-29T04:45:33.377 INFO:tasks.workunit.client.1.smithi152.stdout:4/935: fsync d0/f15 0 2024-03-29T04:45:33.380 INFO:tasks.workunit.client.1.smithi152.stdout:4/936: write d0/d7/d40/d6d/fa4 [219476,38206] 0 2024-03-29T04:45:33.385 INFO:tasks.workunit.client.1.smithi152.stdout:4/937: fsync d0/d7/d25/d17/d5e/d60/d74/d57/f7a 0 2024-03-29T04:45:33.387 INFO:tasks.workunit.client.1.smithi152.stdout:4/938: mknod d0/d27/d6e/c128 0 2024-03-29T04:45:33.389 INFO:tasks.workunit.client.1.smithi152.stdout:4/939: chown d0/d7/d40/ff 6 1 2024-03-29T04:45:33.498 INFO:tasks.workunit.client.1.smithi152.stdout:4/940: sync 2024-03-29T04:45:33.505 INFO:tasks.workunit.client.1.smithi152.stdout:4/941: fdatasync d0/d7/d25/d17/d5e/d60/d74/f51 0 2024-03-29T04:45:33.579 INFO:tasks.workunit.client.1.smithi152.stdout:4/942: write d0/d7/d25/d17/d65/f79 [982232,33621] 0 2024-03-29T04:45:33.582 INFO:tasks.workunit.client.1.smithi152.stdout:4/943: mkdir d0/d27/d6e/d129 0 2024-03-29T04:45:33.588 INFO:tasks.workunit.client.1.smithi152.stdout:4/944: link d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/de7/lfa d0/d7/d25/d17/d5e/l12a 0 2024-03-29T04:45:33.593 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-29T04:45:33.593 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-29T04:45:33.594 INFO:tasks.workunit.client.1.smithi152.stdout:4/945: dread d0/d7/d40/d23/f28 [4194304,4194304] 0 2024-03-29T04:45:33.598 INFO:tasks.workunit.client.1.smithi152.stdout:4/946: creat d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/de7/f12b x:0 0 0 2024-03-29T04:45:33.698 INFO:tasks.workunit.client.1.smithi152.stdout:4/947: sync 2024-03-29T04:45:33.701 INFO:tasks.workunit.client.1.smithi152.stdout:4/948: truncate d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/de7/f12b 1015428 0 2024-03-29T04:45:33.702 INFO:tasks.workunit.client.1.smithi152.stdout:4/949: stat d0/d7/d25/d17/d5e/d60/c85 0 2024-03-29T04:45:33.703 INFO:tasks.workunit.client.1.smithi152.stdout:4/950: fdatasync d0/d7/d40/d23/d55/d70/d96/fcb 0 2024-03-29T04:45:33.713 INFO:tasks.workunit.client.1.smithi152.stdout:4/951: dwrite d0/d7/d40/fd7 [0,4194304] 0 2024-03-29T04:45:33.716 INFO:tasks.workunit.client.1.smithi152.stdout:4/952: truncate d0/d7/d25/d17/d37/f11b 476511 0 2024-03-29T04:45:33.722 INFO:tasks.workunit.client.1.smithi152.stdout:4/953: getdents d0/d27/d6e/d78/d117 0 2024-03-29T04:45:33.727 INFO:tasks.workunit.client.1.smithi152.stdout:4/954: creat d0/d7/d25/d17/d5e/d119/f12c x:0 0 0 2024-03-29T04:45:33.730 INFO:tasks.workunit.client.1.smithi152.stdout:4/955: creat d0/d7/d40/d23/d55/d6c/d91/f12d x:0 0 0 2024-03-29T04:45:33.736 INFO:tasks.workunit.client.1.smithi152.stdout:4/956: dwrite d0/d7/d40/d23/d55/d70/d96/dc0/ff7 [0,4194304] 0 2024-03-29T04:45:33.738 INFO:tasks.workunit.client.1.smithi152.stdout:4/957: fdatasync d0/d7/d25/d17/d37/f11b 0 2024-03-29T04:45:33.745 INFO:tasks.workunit.client.1.smithi152.stdout:4/958: dwrite d0/d7/d25/f21 [0,4194304] 0 2024-03-29T04:45:33.755 INFO:tasks.workunit.client.1.smithi152.stdout:4/959: dwrite d0/d7/d25/d17/d37/f11b [0,4194304] 0 2024-03-29T04:45:33.762 INFO:tasks.workunit.client.1.smithi152.stdout:4/960: dwrite d0/d7/d40/d23/dda/de3/f101 [0,4194304] 0 2024-03-29T04:45:33.765 INFO:tasks.workunit.client.1.smithi152.stdout:4/961: truncate d0/d7/d40/d23/d55/d6c/d91/f12d 757752 0 2024-03-29T04:45:33.766 INFO:tasks.workunit.client.1.smithi152.stdout:4/962: chown d0/d7/d40/d23/d55/d6c/d91/d93 443398940 1 2024-03-29T04:45:33.777 INFO:tasks.workunit.client.1.smithi152.stdout:4/963: dwrite d0/d7/d40/d23/d6a/dfe/f10a [0,4194304] 0 2024-03-29T04:45:33.784 INFO:tasks.workunit.client.1.smithi152.stdout:4/964: unlink d0/d7/d25/d17/d5e/d60/d74/d57/l41 0 2024-03-29T04:45:33.788 INFO:tasks.workunit.client.1.smithi152.stdout:4/965: creat d0/d7/d40/d23/dda/de3/f12e x:0 0 0 2024-03-29T04:45:33.790 INFO:tasks.workunit.client.1.smithi152.stdout:4/966: chown d0/d7/d40/f8a 1 1 2024-03-29T04:45:33.791 INFO:tasks.workunit.client.1.smithi152.stdout:4/967: write d0/d7/d40/f67 [1559058,88355] 0 2024-03-29T04:45:33.793 INFO:tasks.workunit.client.1.smithi152.stdout:4/968: write d0/d7/d40/d2a/f94 [1178661,97161] 0 2024-03-29T04:45:33.797 INFO:tasks.workunit.client.1.smithi152.stdout:4/969: write d0/d7/d25/d17/d5e/fef [827963,94921] 0 2024-03-29T04:45:33.880 INFO:tasks.workunit.client.1.smithi152.stdout:4/970: sync 2024-03-29T04:45:33.883 INFO:tasks.workunit.client.1.smithi152.stdout:4/971: readlink d0/d7/d25/d17/d5e/d60/lc9 0 2024-03-29T04:45:33.888 INFO:tasks.workunit.client.1.smithi152.stdout:4/972: mkdir d0/d7/d40/d12f 0 2024-03-29T04:45:33.890 INFO:tasks.workunit.client.1.smithi152.stdout:4/973: symlink d0/d27/l130 0 2024-03-29T04:45:33.907 INFO:tasks.workunit.client.1.smithi152.stdout:4/974: truncate d0/d7/d25/d17/f5b 624908 0 2024-03-29T04:45:33.923 INFO:tasks.workunit.client.1.smithi152.stdout:4/975: write d0/d7/d40/f82 [217322,85989] 0 2024-03-29T04:45:33.938 INFO:tasks.workunit.client.1.smithi152.stdout:4/976: truncate d0/d7/d40/d23/d55/d6c/d91/f12d 723256 0 2024-03-29T04:45:33.940 INFO:tasks.workunit.client.1.smithi152.stdout:4/977: rmdir d0/d27/d6e 39 2024-03-29T04:45:33.944 INFO:tasks.workunit.client.1.smithi152.stdout:4/978: unlink d0/d27/d6e/d78/d117/fb5 0 2024-03-29T04:45:33.945 INFO:tasks.workunit.client.1.smithi152.stdout:4/979: write d0/d7/d40/d6d/fd0 [522531,88575] 0 2024-03-29T04:45:33.946 INFO:tasks.workunit.client.1.smithi152.stdout:4/980: read d0/d7/db2/ff5 [241813,26251] 0 2024-03-29T04:45:33.951 INFO:tasks.workunit.client.1.smithi152.stdout:4/981: dread d0/d7/d40/d23/dda/de3/f101 [0,4194304] 0 2024-03-29T04:45:33.955 INFO:tasks.workunit.client.1.smithi152.stdout:4/982: link d0/d7/d40/d23/d55/fb8 d0/d7/d40/d23/d55/d6c/d91/f131 0 2024-03-29T04:45:33.961 INFO:tasks.workunit.client.1.smithi152.stdout:4/983: link d0/c3d d0/d7/d40/d23/d55/d6c/d91/c132 0 2024-03-29T04:45:33.964 INFO:tasks.workunit.client.1.smithi152.stdout:4/984: creat d0/d7/d40/d12f/f133 x:0 0 0 2024-03-29T04:45:33.964 INFO:tasks.workunit.client.1.smithi152.stdout:4/985: stat d0/d7/d25/d17/d5e/d60/f9b 0 2024-03-29T04:45:33.967 INFO:tasks.workunit.client.1.smithi152.stdout:4/986: mkdir d0/d7/d25/d17/d5e/d119/d134 0 2024-03-29T04:45:33.969 INFO:tasks.workunit.client.1.smithi152.stdout:4/987: mkdir d0/d7/d40/d23/d55/d135 0 2024-03-29T04:45:33.972 INFO:tasks.workunit.client.1.smithi152.stdout:4/988: creat d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/f136 x:0 0 0 2024-03-29T04:45:34.116 INFO:tasks.workunit.client.1.smithi152.stdout:4/989: sync 2024-03-29T04:45:34.117 INFO:tasks.workunit.client.1.smithi152.stdout:4/990: chown d0/d7/d25/d17/d5e/d119/f12c 461 1 2024-03-29T04:45:34.118 INFO:tasks.workunit.client.1.smithi152.stdout:4/991: dread - d0/d7/d40/d2a/faa zero size 2024-03-29T04:45:34.120 INFO:tasks.workunit.client.1.smithi152.stdout:4/992: chown d0/d7/d40/d23/dda/f124 50 1 2024-03-29T04:45:34.122 INFO:tasks.workunit.client.1.smithi152.stdout:4/993: creat d0/d7/d25/d17/d5e/d60/d74/da2/da6/dbd/f137 x:0 0 0 2024-03-29T04:45:34.123 INFO:tasks.workunit.client.1.smithi152.stdout:4/994: chown d0/d7/d25/d17/d5e/d60/d74 7722 1 2024-03-29T04:45:34.124 INFO:tasks.workunit.client.1.smithi152.stdout:4/995: stat d0/d7/d25/d17/d5e/f114 0 2024-03-29T04:45:34.125 INFO:tasks.workunit.client.1.smithi152.stdout:4/996: dread - d0/d27/d6e/d78/d117/f9a zero size 2024-03-29T04:45:34.128 INFO:tasks.workunit.client.1.smithi152.stdout:4/997: unlink d0/d7/d25/d17/d5e/d60/f9b 0 2024-03-29T04:45:34.133 INFO:tasks.workunit.client.1.smithi152.stdout:4/998: dwrite d0/f15 [0,4194304] 0 2024-03-29T04:45:34.135 INFO:tasks.workunit.client.1.smithi152.stdout:4/999: rmdir d0/d7/d25/d17/d65 39 2024-03-29T04:45:34.143 INFO:tasks.workunit.client.1.smithi152.stderr:+ rm -rf -- ./tmp.MnuWEfH4w7 2024-03-29T04:45:34.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:34 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:34.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:34 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:34.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:34 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:45:34.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:34 smithi152 ceph-mon[37177]: pgmap v283: 65 pgs: 65 active+clean; 2.8 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 60 MiB/s rd, 144 MiB/s wr, 393 op/s 2024-03-29T04:45:34.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:34 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:34.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:34 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:34.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:34 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:45:34.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:34 smithi142 ceph-mon[29027]: pgmap v283: 65 pgs: 65 active+clean; 2.8 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 60 MiB/s rd, 144 MiB/s wr, 393 op/s 2024-03-29T04:45:37.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:36 smithi152 ceph-mon[37177]: pgmap v284: 65 pgs: 65 active+clean; 2.8 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 54 MiB/s rd, 123 MiB/s wr, 368 op/s 2024-03-29T04:45:37.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:36 smithi142 ceph-mon[29027]: pgmap v284: 65 pgs: 65 active+clean; 2.8 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 54 MiB/s rd, 123 MiB/s wr, 368 op/s 2024-03-29T04:45:38.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:37 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:38.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:37 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:38.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:37 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:38.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:37 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:39.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:38 smithi152 ceph-mon[37177]: pgmap v285: 65 pgs: 65 active+clean; 2.4 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 51 MiB/s rd, 119 MiB/s wr, 397 op/s 2024-03-29T04:45:39.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:39.041 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:38 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:39.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:38 smithi142 ceph-mon[29027]: pgmap v285: 65 pgs: 65 active+clean; 2.4 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 51 MiB/s rd, 119 MiB/s wr, 397 op/s 2024-03-29T04:45:39.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:39.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:38 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:41.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:40 smithi142 ceph-mon[29027]: pgmap v286: 65 pgs: 65 active+clean; 2.1 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 40 MiB/s rd, 96 MiB/s wr, 332 op/s 2024-03-29T04:45:41.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:40 smithi152 ceph-mon[37177]: pgmap v286: 65 pgs: 65 active+clean; 2.1 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 40 MiB/s rd, 96 MiB/s wr, 332 op/s 2024-03-29T04:45:43.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: pgmap v287: 65 pgs: 65 active+clean; 2.0 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 21 MiB/s rd, 52 MiB/s wr, 254 op/s 2024-03-29T04:45:43.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:43.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:43.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:45:43.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:45:43.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:43.061 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:45:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr fail", "who": "smithi142.swesrg"}]: dispatch 2024-03-29T04:45:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: osdmap e48: 6 total, 6 up, 6 in 2024-03-29T04:45:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "mgr fail", "who": "smithi142.swesrg"}]': finished 2024-03-29T04:45:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: mgrmap e20: smithi152.usoolh(active, starting, since 0.0105965s) 2024-03-29T04:45:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: Active manager daemon smithi152.usoolh restarted 2024-03-29T04:45:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: Activating manager daemon smithi152.usoolh 2024-03-29T04:45:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/crt"}]: dispatch 2024-03-29T04:45:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: osdmap e49: 6 total, 6 up, 6 in 2024-03-29T04:45:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: mgrmap e21: smithi152.usoolh(active, starting, since 0.0108339s) 2024-03-29T04:45:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/crt"}]: dispatch 2024-03-29T04:45:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:45:43.062 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mgr metadata", "who": "smithi152.usoolh", "id": "smithi152.usoolh"}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T04:45:43.063 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T04:45:43.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T04:45:43.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:45:43.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/key"}]: dispatch 2024-03-29T04:45:43.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:45:43.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: Manager daemon smithi152.usoolh is now available 2024-03-29T04:45:43.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:45:43.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:45:43.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi152.usoolh/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:45:43.064 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi152.usoolh/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:45:43.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi152.usoolh/trash_purge_schedule"}]: dispatch 2024-03-29T04:45:43.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:42 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi152.usoolh/trash_purge_schedule"}]: dispatch 2024-03-29T04:45:43.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: pgmap v287: 65 pgs: 65 active+clean; 2.0 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 21 MiB/s rd, 52 MiB/s wr, 254 op/s 2024-03-29T04:45:43.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:43.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:43.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:45:43.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:45:43.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' 2024-03-29T04:45:43.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:45:43.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr fail", "who": "smithi142.swesrg"}]: dispatch 2024-03-29T04:45:43.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: osdmap e48: 6 total, 6 up, 6 in 2024-03-29T04:45:43.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.14215 172.21.15.142:0/4023083253' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "mgr fail", "who": "smithi142.swesrg"}]': finished 2024-03-29T04:45:43.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: mgrmap e20: smithi152.usoolh(active, starting, since 0.0105965s) 2024-03-29T04:45:43.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: Active manager daemon smithi152.usoolh restarted 2024-03-29T04:45:43.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: Activating manager daemon smithi152.usoolh 2024-03-29T04:45:43.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/crt"}]: dispatch 2024-03-29T04:45:43.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: osdmap e49: 6 total, 6 up, 6 in 2024-03-29T04:45:43.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: mgrmap e21: smithi152.usoolh(active, starting, since 0.0108339s) 2024-03-29T04:45:43.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/crt"}]: dispatch 2024-03-29T04:45:43.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:45:43.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:45:43.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T04:45:43.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:45:43.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:45:43.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T04:45:43.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mgr metadata", "who": "smithi152.usoolh", "id": "smithi152.usoolh"}]: dispatch 2024-03-29T04:45:43.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:45:43.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:45:43.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:45:43.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:45:43.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:45:43.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:45:43.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T04:45:43.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T04:45:43.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T04:45:43.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:45:43.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/key"}]: dispatch 2024-03-29T04:45:43.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:45:43.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: Manager daemon smithi152.usoolh is now available 2024-03-29T04:45:43.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:45:43.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:45:43.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi152.usoolh/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:45:43.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi152.usoolh/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:45:43.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi152.usoolh/trash_purge_schedule"}]: dispatch 2024-03-29T04:45:43.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:42 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi152.usoolh/trash_purge_schedule"}]: dispatch 2024-03-29T04:45:44.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:43 smithi142 ceph-mon[29027]: Deploying cephadm binary to smithi152 2024-03-29T04:45:44.068 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:43 smithi142 ceph-mon[29027]: Deploying cephadm binary to smithi142 2024-03-29T04:45:44.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:43 smithi142 ceph-mon[29027]: mgrmap e22: smithi152.usoolh(active, since 1.01266s) 2024-03-29T04:45:44.257 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:43 smithi152 ceph-mon[37177]: Deploying cephadm binary to smithi152 2024-03-29T04:45:44.257 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:43 smithi152 ceph-mon[37177]: Deploying cephadm binary to smithi142 2024-03-29T04:45:44.257 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:43 smithi152 ceph-mon[37177]: mgrmap e22: smithi152.usoolh(active, since 1.01266s) 2024-03-29T04:45:45.099 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:44 smithi142 ceph-mon[29027]: pgmap v3: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail 2024-03-29T04:45:45.100 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:44 smithi142 ceph-mon[29027]: [29/Mar/2024:04:45:43] ENGINE Bus STARTING 2024-03-29T04:45:45.100 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:44 smithi142 ceph-mon[29027]: [29/Mar/2024:04:45:43] ENGINE Serving on http://172.21.15.152:8765 2024-03-29T04:45:45.100 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:44 smithi142 ceph-mon[29027]: [29/Mar/2024:04:45:43] ENGINE Serving on https://172.21.15.152:7150 2024-03-29T04:45:45.100 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:44 smithi142 ceph-mon[29027]: [29/Mar/2024:04:45:43] ENGINE Bus STARTED 2024-03-29T04:45:45.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:44 smithi152 ceph-mon[37177]: pgmap v3: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail 2024-03-29T04:45:45.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:44 smithi152 ceph-mon[37177]: [29/Mar/2024:04:45:43] ENGINE Bus STARTING 2024-03-29T04:45:45.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:44 smithi152 ceph-mon[37177]: [29/Mar/2024:04:45:43] ENGINE Serving on http://172.21.15.152:8765 2024-03-29T04:45:45.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:44 smithi152 ceph-mon[37177]: [29/Mar/2024:04:45:43] ENGINE Serving on https://172.21.15.152:7150 2024-03-29T04:45:45.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:44 smithi152 ceph-mon[37177]: [29/Mar/2024:04:45:43] ENGINE Bus STARTED 2024-03-29T04:45:46.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:45 smithi142 ceph-mon[29027]: pgmap v4: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail 2024-03-29T04:45:46.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:45 smithi142 ceph-mon[29027]: mgrmap e23: smithi152.usoolh(active, since 2s) 2024-03-29T04:45:46.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:45 smithi152 ceph-mon[37177]: pgmap v4: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail 2024-03-29T04:45:46.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:45 smithi152 ceph-mon[37177]: mgrmap e23: smithi152.usoolh(active, since 2s) 2024-03-29T04:45:48.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:47 smithi142 ceph-mon[29027]: pgmap v5: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail 2024-03-29T04:45:48.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:47 smithi142 ceph-mon[29027]: mgrmap e24: smithi152.usoolh(active, since 4s) 2024-03-29T04:45:48.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:47 smithi152 ceph-mon[37177]: pgmap v5: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail 2024-03-29T04:45:48.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:47 smithi152 ceph-mon[37177]: mgrmap e24: smithi152.usoolh(active, since 4s) 2024-03-29T04:45:50.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:49 smithi152 ceph-mon[37177]: pgmap v6: 65 pgs: 65 active+clean; 1.4 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-03-29T04:45:50.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:49 smithi142 ceph-mon[29027]: pgmap v6: 65 pgs: 65 active+clean; 1.4 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-03-29T04:45:50.902 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:50 smithi142 ceph-mon[29027]: Standby manager daemon smithi142.swesrg started 2024-03-29T04:45:50.902 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:50 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.142:0/388909001' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.swesrg/crt"}]: dispatch 2024-03-29T04:45:50.902 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:50 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.142:0/388909001' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:45:50.902 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:50 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.142:0/388909001' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.swesrg/key"}]: dispatch 2024-03-29T04:45:50.902 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:50 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.142:0/388909001' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:45:51.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:50 smithi152 ceph-mon[37177]: Standby manager daemon smithi142.swesrg started 2024-03-29T04:45:51.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:50 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.142:0/388909001' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.swesrg/crt"}]: dispatch 2024-03-29T04:45:51.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:50 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.142:0/388909001' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:45:51.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:50 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.142:0/388909001' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.swesrg/key"}]: dispatch 2024-03-29T04:45:51.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:50 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.142:0/388909001' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:45:52.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:51 smithi142 ceph-mon[29027]: pgmap v7: 65 pgs: 65 active+clean; 904 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-03-29T04:45:52.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:51 smithi142 ceph-mon[29027]: mgrmap e25: smithi152.usoolh(active, since 8s), standbys: smithi142.swesrg 2024-03-29T04:45:52.121 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:51 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mgr metadata", "who": "smithi142.swesrg", "id": "smithi142.swesrg"}]: dispatch 2024-03-29T04:45:52.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:51 smithi152 ceph-mon[37177]: pgmap v7: 65 pgs: 65 active+clean; 904 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-03-29T04:45:52.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:51 smithi152 ceph-mon[37177]: mgrmap e25: smithi152.usoolh(active, since 8s), standbys: smithi142.swesrg 2024-03-29T04:45:52.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:51 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mgr metadata", "who": "smithi142.swesrg", "id": "smithi142.swesrg"}]: dispatch 2024-03-29T04:45:52.752 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-29T04:45:52.752 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-29T04:45:54.083 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:53 smithi142 ceph-mon[29027]: pgmap v8: 65 pgs: 65 active+clean; 904 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 944 KiB/s rd, 900 KiB/s wr, 201 op/s 2024-03-29T04:45:54.083 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:53 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:54.083 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:53 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:54.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:53 smithi152 ceph-mon[37177]: pgmap v8: 65 pgs: 65 active+clean; 904 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 944 KiB/s rd, 900 KiB/s wr, 201 op/s 2024-03-29T04:45:54.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:53 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:54.290 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:53 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:56.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:55 smithi152 ceph-mon[37177]: pgmap v9: 65 pgs: 65 active+clean; 527 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 317 op/s 2024-03-29T04:45:56.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:55 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:56.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:55 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:56.040 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:55 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:56.041 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:55 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:56.371 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:55 smithi142 ceph-mon[29027]: pgmap v9: 65 pgs: 65 active+clean; 527 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 317 op/s 2024-03-29T04:45:56.371 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:55 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:56.371 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:55 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:56.371 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:55 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:56.371 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:55 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:57.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:57 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:57.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:57 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:57.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:57 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:45:57.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:57 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:45:57.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:57 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:45:57.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:57 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:45:57.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:57 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:45:57.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:57 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:45:57.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:57 smithi142 ceph-mon[29027]: Adjusting osd_memory_target on smithi142 to 1216M 2024-03-29T04:45:57.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:57 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:57.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:57 smithi142 ceph-mon[29027]: pgmap v10: 65 pgs: 65 active+clean; 527 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 290 op/s 2024-03-29T04:45:57.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:57 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:57.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:57 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:57.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:57 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:45:57.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:57 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:45:57.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:57 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:45:57.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:57 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:45:57.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:57 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:45:57.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:57 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:45:57.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:57 smithi152 ceph-mon[37177]: Adjusting osd_memory_target on smithi142 to 1216M 2024-03-29T04:45:57.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:57 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:45:57.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:57 smithi152 ceph-mon[37177]: pgmap v10: 65 pgs: 65 active+clean; 527 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 290 op/s 2024-03-29T04:45:58.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:58 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:45:58.787 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:58 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:45:59.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.161+0000 7f2293837700 1 -- 172.21.15.142:0/1650428651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f228c101660 msgr2=0x7f228c103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:59.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.161+0000 7f2293837700 1 --2- 172.21.15.142:0/1650428651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f228c101660 0x7f228c103a50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2288009910 tx=0x7f2288009c20 comp rx=0 tx=0).stop 2024-03-29T04:45:59.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.161+0000 7f2293837700 1 -- 172.21.15.142:0/1650428651 shutdown_connections 2024-03-29T04:45:59.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.161+0000 7f2293837700 1 --2- 172.21.15.142:0/1650428651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c103f90 0x7f228c106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:59.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.161+0000 7f2293837700 1 --2- 172.21.15.142:0/1650428651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f228c101660 0x7f228c103a50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:59.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.161+0000 7f2293837700 1 -- 172.21.15.142:0/1650428651 >> 172.21.15.142:0/1650428651 conn(0x7f228c0fb050 msgr2=0x7f228c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:59.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.161+0000 7f2293837700 1 -- 172.21.15.142:0/1650428651 shutdown_connections 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.161+0000 7f2293837700 1 -- 172.21.15.142:0/1650428651 wait complete. 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.161+0000 7f2293837700 1 Processor -- start 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.161+0000 7f2293837700 1 -- start start 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f2293837700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f228c101660 0x7f228c196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f2293837700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c103f90 0x7f228c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f2293837700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c196ed0 con 0x7f228c101660 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f2293837700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c197010 con 0x7f228c103f90 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f22915d3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f228c101660 0x7f228c196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f22915d3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f228c101660 0x7f228c196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53918/0 (socket says 172.21.15.142:53918) 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f22915d3700 1 -- 172.21.15.142:0/450526011 learned_addr learned my addr 172.21.15.142:0/450526011 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f2290dd2700 1 --2- 172.21.15.142:0/450526011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c103f90 0x7f228c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f2290dd2700 1 -- 172.21.15.142:0/450526011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f228c101660 msgr2=0x7f228c196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:59.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f2290dd2700 1 --2- 172.21.15.142:0/450526011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f228c101660 0x7f228c196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:59.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f2290dd2700 1 -- 172.21.15.142:0/450526011 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22880095f0 con 0x7f228c103f90 2024-03-29T04:45:59.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f22915d3700 1 --2- 172.21.15.142:0/450526011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f228c101660 0x7f228c196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:45:59.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.162+0000 7f2290dd2700 1 --2- 172.21.15.142:0/450526011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c103f90 0x7f228c1968b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f227c00e970 tx=0x7f227c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:59.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.163+0000 7f22827fc700 1 -- 172.21.15.142:0/450526011 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f227c00c9d0 con 0x7f228c103f90 2024-03-29T04:45:59.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.163+0000 7f22827fc700 1 -- 172.21.15.142:0/450526011 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f227c010430 con 0x7f228c103f90 2024-03-29T04:45:59.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.163+0000 7f2293837700 1 -- 172.21.15.142:0/450526011 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c19bac0 con 0x7f228c103f90 2024-03-29T04:45:59.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.163+0000 7f22827fc700 1 -- 172.21.15.142:0/450526011 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f227c018650 con 0x7f228c103f90 2024-03-29T04:45:59.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.163+0000 7f2293837700 1 -- 172.21.15.142:0/450526011 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c19c010 con 0x7f228c103f90 2024-03-29T04:45:59.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.164+0000 7f2293837700 1 -- 172.21.15.142:0/450526011 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c190540 con 0x7f228c103f90 2024-03-29T04:45:59.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.164+0000 7f22827fc700 1 -- 172.21.15.142:0/450526011 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) v1 ==== 91117+0+0 (secure 0 0 0) 0x7f227c0105a0 con 0x7f228c103f90 2024-03-29T04:45:59.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.165+0000 7f22827fc700 1 --2- 172.21.15.142:0/450526011 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f227806d0a0 0x7f227806f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:59.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.165+0000 7f22827fc700 1 -- 172.21.15.142:0/450526011 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f227c014070 con 0x7f228c103f90 2024-03-29T04:45:59.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.165+0000 7f22915d3700 1 --2- 172.21.15.142:0/450526011 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f227806d0a0 0x7f227806f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:59.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.166+0000 7f22915d3700 1 --2- 172.21.15.142:0/450526011 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f227806d0a0 0x7f227806f560 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2288009fd0 tx=0x7f2288005940 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:59.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.173+0000 7f22827fc700 1 -- 172.21.15.142:0/450526011 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f227c05a0b0 con 0x7f228c103f90 2024-03-29T04:45:59.442 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.441+0000 7f2293837700 1 -- 172.21.15.142:0/450526011 --> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f228c0611d0 con 0x7f227806d0a0 2024-03-29T04:45:59.442 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.442+0000 7f22827fc700 1 -- 172.21.15.142:0/450526011 <== mgr.24395 v2:172.21.15.152:6828/3130987667 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f228c0611d0 con 0x7f227806d0a0 2024-03-29T04:45:59.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.444+0000 7f2293837700 1 -- 172.21.15.142:0/450526011 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f227806d0a0 msgr2=0x7f227806f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:59.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.444+0000 7f2293837700 1 --2- 172.21.15.142:0/450526011 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f227806d0a0 0x7f227806f560 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2288009fd0 tx=0x7f2288005940 comp rx=0 tx=0).stop 2024-03-29T04:45:59.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.444+0000 7f2293837700 1 -- 172.21.15.142:0/450526011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c103f90 msgr2=0x7f228c1968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:59.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.444+0000 7f2293837700 1 --2- 172.21.15.142:0/450526011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c103f90 0x7f228c1968b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f227c00e970 tx=0x7f227c00ec80 comp rx=0 tx=0).stop 2024-03-29T04:45:59.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.445+0000 7f2293837700 1 -- 172.21.15.142:0/450526011 shutdown_connections 2024-03-29T04:45:59.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.445+0000 7f2293837700 1 --2- 172.21.15.142:0/450526011 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f227806d0a0 0x7f227806f560 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:59.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.445+0000 7f2293837700 1 --2- 172.21.15.142:0/450526011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f228c101660 0x7f228c196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:59.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.445+0000 7f2293837700 1 --2- 172.21.15.142:0/450526011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c103f90 0x7f228c1968b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:59.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.445+0000 7f2293837700 1 -- 172.21.15.142:0/450526011 >> 172.21.15.142:0/450526011 conn(0x7f228c0fb050 msgr2=0x7f228c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:59.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.445+0000 7f2293837700 1 -- 172.21.15.142:0/450526011 shutdown_connections 2024-03-29T04:45:59.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.445+0000 7f2293837700 1 -- 172.21.15.142:0/450526011 wait complete. 2024-03-29T04:45:59.456 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:45:59.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.584+0000 7fe23123a700 1 -- 172.21.15.142:0/652027554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c107220 msgr2=0x7fe22c107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:59.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.584+0000 7fe23123a700 1 --2- 172.21.15.142:0/652027554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c107220 0x7fe22c107640 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe220009910 tx=0x7fe220009c20 comp rx=0 tx=0).stop 2024-03-29T04:45:59.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.585+0000 7fe23123a700 1 -- 172.21.15.142:0/652027554 shutdown_connections 2024-03-29T04:45:59.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.585+0000 7fe23123a700 1 --2- 172.21.15.142:0/652027554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe22c108420 0x7fe22c108880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:59.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.585+0000 7fe23123a700 1 --2- 172.21.15.142:0/652027554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c107220 0x7fe22c107640 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:59.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.585+0000 7fe23123a700 1 -- 172.21.15.142:0/652027554 >> 172.21.15.142:0/652027554 conn(0x7fe22c076200 msgr2=0x7fe22c078660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:45:59.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.585+0000 7fe23123a700 1 -- 172.21.15.142:0/652027554 shutdown_connections 2024-03-29T04:45:59.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.585+0000 7fe23123a700 1 -- 172.21.15.142:0/652027554 wait complete. 2024-03-29T04:45:59.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.586+0000 7fe23123a700 1 Processor -- start 2024-03-29T04:45:59.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.586+0000 7fe23123a700 1 -- start start 2024-03-29T04:45:59.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.586+0000 7fe23123a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe22c107220 0x7fe22c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:59.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.586+0000 7fe23123a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108420 0x7fe22c072290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:59.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.586+0000 7fe23123a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c0728b0 con 0x7fe22c108420 2024-03-29T04:45:59.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.586+0000 7fe23123a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1aa7a0 con 0x7fe22c107220 2024-03-29T04:45:59.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.587+0000 7fe22a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108420 0x7fe22c072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:59.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.587+0000 7fe22a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108420 0x7fe22c072290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53950/0 (socket says 172.21.15.142:53950) 2024-03-29T04:45:59.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.587+0000 7fe22a59c700 1 -- 172.21.15.142:0/3144251512 learned_addr learned my addr 172.21.15.142:0/3144251512 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:45:59.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.587+0000 7fe22ad9d700 1 --2- 172.21.15.142:0/3144251512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe22c107220 0x7fe22c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:59.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.588+0000 7fe22a59c700 1 -- 172.21.15.142:0/3144251512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe22c107220 msgr2=0x7fe22c071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:45:59.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.588+0000 7fe22a59c700 1 --2- 172.21.15.142:0/3144251512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe22c107220 0x7fe22c071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:45:59.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.588+0000 7fe22a59c700 1 -- 172.21.15.142:0/3144251512 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2200095f0 con 0x7fe22c108420 2024-03-29T04:45:59.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.589+0000 7fe22a59c700 1 --2- 172.21.15.142:0/3144251512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108420 0x7fe22c072290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00d8d0 tx=0x7fe21c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:59.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.589+0000 7fe213fff700 1 -- 172.21.15.142:0/3144251512 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe21c0098a0 con 0x7fe22c108420 2024-03-29T04:45:59.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.589+0000 7fe213fff700 1 -- 172.21.15.142:0/3144251512 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe21c010460 con 0x7fe22c108420 2024-03-29T04:45:59.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.589+0000 7fe23123a700 1 -- 172.21.15.142:0/3144251512 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22c1aa940 con 0x7fe22c108420 2024-03-29T04:45:59.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.589+0000 7fe23123a700 1 -- 172.21.15.142:0/3144251512 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c1aae10 con 0x7fe22c108420 2024-03-29T04:45:59.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.589+0000 7fe213fff700 1 -- 172.21.15.142:0/3144251512 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe21c00f5f0 con 0x7fe22c108420 2024-03-29T04:45:59.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.591+0000 7fe23123a700 1 -- 172.21.15.142:0/3144251512 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe218005320 con 0x7fe22c108420 2024-03-29T04:45:59.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.592+0000 7fe213fff700 1 -- 172.21.15.142:0/3144251512 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 91117+0+0 (secure 0 0 0) 0x7fe21c009a00 con 0x7fe22c108420 2024-03-29T04:45:59.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.592+0000 7fe213fff700 1 --2- 172.21.15.142:0/3144251512 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fe21406d0d0 0x7fe21406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:45:59.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.593+0000 7fe22ad9d700 1 --2- 172.21.15.142:0/3144251512 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fe21406d0d0 0x7fe21406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:45:59.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.593+0000 7fe213fff700 1 -- 172.21.15.142:0/3144251512 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe21c05a510 con 0x7fe22c108420 2024-03-29T04:45:59.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.595+0000 7fe22ad9d700 1 --2- 172.21.15.142:0/3144251512 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fe21406d0d0 0x7fe21406f590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe220005090 tx=0x7fe22000b560 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:45:59.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:45:59.601+0000 7fe213fff700 1 -- 172.21.15.142:0/3144251512 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fe21c056840 con 0x7fe22c108420 2024-03-29T04:45:59.785 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:45:59 smithi152 ceph-mon[37177]: pgmap v11: 65 pgs: 65 active+clean; 483 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 344 op/s 2024-03-29T04:45:59.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:45:59 smithi142 ceph-mon[29027]: pgmap v11: 65 pgs: 65 active+clean; 483 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 344 op/s 2024-03-29T04:46:00.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.047+0000 7fe23123a700 1 -- 172.21.15.142:0/3144251512 --> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe218000bf0 con 0x7fe21406d0d0 2024-03-29T04:46:00.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.048+0000 7fe213fff700 1 -- 172.21.15.142:0/3144251512 <== mgr.24395 v2:172.21.15.152:6828/3130987667 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fe218000bf0 con 0x7fe21406d0d0 2024-03-29T04:46:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.051+0000 7fe23123a700 1 -- 172.21.15.142:0/3144251512 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fe21406d0d0 msgr2=0x7fe21406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.051+0000 7fe23123a700 1 --2- 172.21.15.142:0/3144251512 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fe21406d0d0 0x7fe21406f590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe220005090 tx=0x7fe22000b560 comp rx=0 tx=0).stop 2024-03-29T04:46:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.051+0000 7fe23123a700 1 -- 172.21.15.142:0/3144251512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108420 msgr2=0x7fe22c072290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.051+0000 7fe23123a700 1 --2- 172.21.15.142:0/3144251512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108420 0x7fe22c072290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00d8d0 tx=0x7fe21c00dbe0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.051+0000 7fe23123a700 1 -- 172.21.15.142:0/3144251512 shutdown_connections 2024-03-29T04:46:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.051+0000 7fe23123a700 1 --2- 172.21.15.142:0/3144251512 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fe21406d0d0 0x7fe21406f590 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.051+0000 7fe23123a700 1 --2- 172.21.15.142:0/3144251512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe22c107220 0x7fe22c071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.051+0000 7fe23123a700 1 --2- 172.21.15.142:0/3144251512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108420 0x7fe22c072290 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.051+0000 7fe23123a700 1 -- 172.21.15.142:0/3144251512 >> 172.21.15.142:0/3144251512 conn(0x7fe22c076200 msgr2=0x7fe22c10b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:00.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.051+0000 7fe23123a700 1 -- 172.21.15.142:0/3144251512 shutdown_connections 2024-03-29T04:46:00.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.051+0000 7fe23123a700 1 -- 172.21.15.142:0/3144251512 wait complete. 2024-03-29T04:46:00.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.193+0000 7faac67b1700 1 -- 172.21.15.142:0/182286671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac00fec20 msgr2=0x7faac00ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:00.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.193+0000 7faac67b1700 1 --2- 172.21.15.142:0/182286671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac00fec20 0x7faac00ff080 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faab4009910 tx=0x7faab4009c20 comp rx=0 tx=0).stop 2024-03-29T04:46:00.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.193+0000 7faac67b1700 1 -- 172.21.15.142:0/182286671 shutdown_connections 2024-03-29T04:46:00.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.193+0000 7faac67b1700 1 --2- 172.21.15.142:0/182286671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac00fec20 0x7faac00ff080 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.193+0000 7faac67b1700 1 --2- 172.21.15.142:0/182286671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faac01021a0 0x7faac01025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.193+0000 7faac67b1700 1 -- 172.21.15.142:0/182286671 >> 172.21.15.142:0/182286671 conn(0x7faac00fa7c0 msgr2=0x7faac00fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:00.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.193+0000 7faac67b1700 1 -- 172.21.15.142:0/182286671 shutdown_connections 2024-03-29T04:46:00.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.193+0000 7faac67b1700 1 -- 172.21.15.142:0/182286671 wait complete. 2024-03-29T04:46:00.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.194+0000 7faac67b1700 1 Processor -- start 2024-03-29T04:46:00.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.194+0000 7faac67b1700 1 -- start start 2024-03-29T04:46:00.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.194+0000 7faac67b1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac00fec20 0x7faac0196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:00.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.194+0000 7faac67b1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faac01021a0 0x7faac0196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:00.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.194+0000 7faac67b1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac0196f60 con 0x7faac00fec20 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.194+0000 7faac67b1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01970a0 con 0x7faac01021a0 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.194+0000 7faabffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac00fec20 0x7faac0196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.194+0000 7faabffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac00fec20 0x7faac0196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53972/0 (socket says 172.21.15.142:53972) 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.194+0000 7faabffff700 1 -- 172.21.15.142:0/4283775490 learned_addr learned my addr 172.21.15.142:0/4283775490 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.194+0000 7faabf7fe700 1 --2- 172.21.15.142:0/4283775490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faac01021a0 0x7faac0196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.195+0000 7faabffff700 1 -- 172.21.15.142:0/4283775490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faac01021a0 msgr2=0x7faac0196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.195+0000 7faabffff700 1 --2- 172.21.15.142:0/4283775490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faac01021a0 0x7faac0196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.195+0000 7faabffff700 1 -- 172.21.15.142:0/4283775490 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab40095f0 con 0x7faac00fec20 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.195+0000 7faabf7fe700 1 --2- 172.21.15.142:0/4283775490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faac01021a0 0x7faac0196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.195+0000 7faabffff700 1 --2- 172.21.15.142:0/4283775490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac00fec20 0x7faac0196400 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faab000e8f0 tx=0x7faab000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.195+0000 7faabd7fa700 1 -- 172.21.15.142:0/4283775490 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faab0009790 con 0x7faac00fec20 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.195+0000 7faabd7fa700 1 -- 172.21.15.142:0/4283775490 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faab000c810 con 0x7faac00fec20 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.195+0000 7faac67b1700 1 -- 172.21.15.142:0/4283775490 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac019bb50 con 0x7faac00fec20 2024-03-29T04:46:00.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.196+0000 7faabd7fa700 1 -- 172.21.15.142:0/4283775490 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faab0003d20 con 0x7faac00fec20 2024-03-29T04:46:00.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.196+0000 7faac67b1700 1 -- 172.21.15.142:0/4283775490 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac019c0a0 con 0x7faac00fec20 2024-03-29T04:46:00.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.197+0000 7faac67b1700 1 -- 172.21.15.142:0/4283775490 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac0109730 con 0x7faac00fec20 2024-03-29T04:46:00.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.197+0000 7faabd7fa700 1 -- 172.21.15.142:0/4283775490 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 91117+0+0 (secure 0 0 0) 0x7faab00041f0 con 0x7faac00fec20 2024-03-29T04:46:00.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.198+0000 7faabd7fa700 1 --2- 172.21.15.142:0/4283775490 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7faaa806d0a0 0x7faaa806f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:00.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.198+0000 7faabd7fa700 1 -- 172.21.15.142:0/4283775490 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7faab0014070 con 0x7faac00fec20 2024-03-29T04:46:00.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.198+0000 7faabf7fe700 1 --2- 172.21.15.142:0/4283775490 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7faaa806d0a0 0x7faaa806f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:00.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.199+0000 7faabf7fe700 1 --2- 172.21.15.142:0/4283775490 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7faaa806d0a0 0x7faaa806f560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faab4009fd0 tx=0x7faab4005940 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:00.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.207+0000 7faabd7fa700 1 -- 172.21.15.142:0/4283775490 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7faab005a0c0 con 0x7faac00fec20 2024-03-29T04:46:00.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.434+0000 7faac67b1700 1 -- 172.21.15.142:0/4283775490 --> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faac0106890 con 0x7faaa806d0a0 2024-03-29T04:46:00.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.454+0000 7faabd7fa700 1 -- 172.21.15.142:0/4283775490 <== mgr.24395 v2:172.21.15.152:6828/3130987667 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7faac0106890 con 0x7faaa806d0a0 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (7m) 5s ago 9m 25.1M - 0.25.0 c8568f914cd2 d4a3e12f4047 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (9m) 5s ago 9m 7856k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (8m) 7s ago 8m 7721k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (9m) 5s ago 9m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (8m) 7s ago 8m 6660k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (7m) 5s ago 9m 85.8M - 9.4.7 954c08fa6188 aba58e552b7c 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (3m) 5s ago 3m 176M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (3m) 5s ago 3m 95.3M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (3m) 7s ago 3m 89.4M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (3m) 7s ago 3m 149M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:9283,8765,8443 running (10m) 5s ago 10m 421M - 18.2.1 5be31c24972a a5f21a9cfe24 2024-03-29T04:46:00.455 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (28s) 7s ago 8m 489M - 19.0.0-1959-g8da23624 54112f57da44 2aeb29a22d55 2024-03-29T04:46:00.456 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (10m) 5s ago 10m 51.0M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:46:00.456 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (8m) 7s ago 8m 39.4M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:46:00.456 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (9m) 5s ago 9m 15.7M - 1.5.0 0da6a335fe13 91b95c929db2 2024-03-29T04:46:00.456 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (8m) 7s ago 8m 14.9M - 1.5.0 0da6a335fe13 97e90e5b337b 2024-03-29T04:46:00.456 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (7m) 5s ago 7m 202M 1216M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:46:00.456 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (7m) 5s ago 7m 218M 1216M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:46:00.456 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (6m) 5s ago 6m 176M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:46:00.456 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (6m) 7s ago 6m 255M 4970M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:46:00.456 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (5m) 7s ago 5m 218M 4970M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:46:00.456 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (5m) 7s ago 5m 208M 4970M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:46:00.456 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (7m) 5s ago 8m 62.7M - 2.43.0 a07b618ecd1d c6c39d3c30f6 2024-03-29T04:46:00.457 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.457+0000 7faac67b1700 1 -- 172.21.15.142:0/4283775490 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7faaa806d0a0 msgr2=0x7faaa806f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:00.457 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.457+0000 7faac67b1700 1 --2- 172.21.15.142:0/4283775490 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7faaa806d0a0 0x7faaa806f560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faab4009fd0 tx=0x7faab4005940 comp rx=0 tx=0).stop 2024-03-29T04:46:00.457 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.457+0000 7faac67b1700 1 -- 172.21.15.142:0/4283775490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac00fec20 msgr2=0x7faac0196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:00.457 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.457+0000 7faac67b1700 1 --2- 172.21.15.142:0/4283775490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac00fec20 0x7faac0196400 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faab000e8f0 tx=0x7faab000ec00 comp rx=0 tx=0).stop 2024-03-29T04:46:00.457 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.457+0000 7faac67b1700 1 -- 172.21.15.142:0/4283775490 shutdown_connections 2024-03-29T04:46:00.457 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.457+0000 7faac67b1700 1 --2- 172.21.15.142:0/4283775490 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7faaa806d0a0 0x7faaa806f560 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.457 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.457+0000 7faac67b1700 1 --2- 172.21.15.142:0/4283775490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac00fec20 0x7faac0196400 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.457+0000 7faac67b1700 1 --2- 172.21.15.142:0/4283775490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faac01021a0 0x7faac0196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.457+0000 7faac67b1700 1 -- 172.21.15.142:0/4283775490 >> 172.21.15.142:0/4283775490 conn(0x7faac00fa7c0 msgr2=0x7faac0105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:00.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.457+0000 7faac67b1700 1 -- 172.21.15.142:0/4283775490 shutdown_connections 2024-03-29T04:46:00.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.457+0000 7faac67b1700 1 -- 172.21.15.142:0/4283775490 wait complete. 2024-03-29T04:46:00.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.575+0000 7ffa0199c700 1 -- 172.21.15.142:0/4254822563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc1016a0 msgr2=0x7ff9fc103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:00.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.575+0000 7ffa0199c700 1 --2- 172.21.15.142:0/4254822563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc1016a0 0x7ff9fc103a90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009910 tx=0x7ff9ec009c20 comp rx=0 tx=0).stop 2024-03-29T04:46:00.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.576+0000 7ffa0199c700 1 -- 172.21.15.142:0/4254822563 shutdown_connections 2024-03-29T04:46:00.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.576+0000 7ffa0199c700 1 --2- 172.21.15.142:0/4254822563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9fc103fd0 0x7ff9fc1063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.576+0000 7ffa0199c700 1 --2- 172.21.15.142:0/4254822563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc1016a0 0x7ff9fc103a90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.576+0000 7ffa0199c700 1 -- 172.21.15.142:0/4254822563 >> 172.21.15.142:0/4254822563 conn(0x7ff9fc0fb030 msgr2=0x7ff9fc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:00.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.576+0000 7ffa0199c700 1 -- 172.21.15.142:0/4254822563 shutdown_connections 2024-03-29T04:46:00.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.576+0000 7ffa0199c700 1 -- 172.21.15.142:0/4254822563 wait complete. 2024-03-29T04:46:00.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.576+0000 7ffa0199c700 1 Processor -- start 2024-03-29T04:46:00.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.576+0000 7ffa0199c700 1 -- start start 2024-03-29T04:46:00.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ffa0199c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9fc1016a0 0x7ff9fc196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:00.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ffa0199c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc103fd0 0x7ff9fc196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:00.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ffa0199c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc196e70 con 0x7ff9fc103fd0 2024-03-29T04:46:00.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ffa0199c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc196fb0 con 0x7ff9fc1016a0 2024-03-29T04:46:00.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ff9fa7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc103fd0 0x7ff9fc196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:00.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ff9fa7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc103fd0 0x7ff9fc196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53992/0 (socket says 172.21.15.142:53992) 2024-03-29T04:46:00.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ff9fa7fc700 1 -- 172.21.15.142:0/1236716027 learned_addr learned my addr 172.21.15.142:0/1236716027 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:46:00.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ff9faffd700 1 --2- 172.21.15.142:0/1236716027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9fc1016a0 0x7ff9fc196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:00.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ff9fa7fc700 1 -- 172.21.15.142:0/1236716027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9fc1016a0 msgr2=0x7ff9fc196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:00.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ff9fa7fc700 1 --2- 172.21.15.142:0/1236716027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9fc1016a0 0x7ff9fc196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:00.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ff9fa7fc700 1 -- 172.21.15.142:0/1236716027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0095f0 con 0x7ff9fc103fd0 2024-03-29T04:46:00.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ff9faffd700 1 --2- 172.21.15.142:0/1236716027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9fc1016a0 0x7ff9fc196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:46:00.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.577+0000 7ff9fa7fc700 1 --2- 172.21.15.142:0/1236716027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc103fd0 0x7ff9fc196850 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000e920 tx=0x7ff9f000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:00.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.578+0000 7ffa0099a700 1 -- 172.21.15.142:0/1236716027 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9f000c9d0 con 0x7ff9fc103fd0 2024-03-29T04:46:00.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.578+0000 7ffa0199c700 1 -- 172.21.15.142:0/1236716027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc19ba60 con 0x7ff9fc103fd0 2024-03-29T04:46:00.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.578+0000 7ffa0099a700 1 -- 172.21.15.142:0/1236716027 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff9f0010430 con 0x7ff9fc103fd0 2024-03-29T04:46:00.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.578+0000 7ffa0099a700 1 -- 172.21.15.142:0/1236716027 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9f0018650 con 0x7ff9fc103fd0 2024-03-29T04:46:00.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.578+0000 7ffa0199c700 1 -- 172.21.15.142:0/1236716027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc19bfb0 con 0x7ff9fc103fd0 2024-03-29T04:46:00.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.579+0000 7ffa0199c700 1 -- 172.21.15.142:0/1236716027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc190520 con 0x7ff9fc103fd0 2024-03-29T04:46:00.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.579+0000 7ffa0099a700 1 -- 172.21.15.142:0/1236716027 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 91117+0+0 (secure 0 0 0) 0x7ff9f00105a0 con 0x7ff9fc103fd0 2024-03-29T04:46:00.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.580+0000 7ffa0099a700 1 --2- 172.21.15.142:0/1236716027 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7ff9e406d0a0 0x7ff9e406f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:00.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.580+0000 7ffa0099a700 1 -- 172.21.15.142:0/1236716027 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff9f0014070 con 0x7ff9fc103fd0 2024-03-29T04:46:00.582 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.580+0000 7ff9faffd700 1 --2- 172.21.15.142:0/1236716027 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7ff9e406d0a0 0x7ff9e406f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:00.582 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.581+0000 7ff9faffd700 1 --2- 172.21.15.142:0/1236716027 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7ff9e406d0a0 0x7ff9e406f560 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009fd0 tx=0x7ff9ec005940 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:00.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.585+0000 7ffa0099a700 1 -- 172.21.15.142:0/1236716027 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7ff9f0052fc0 con 0x7ff9fc103fd0 2024-03-29T04:46:00.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:00 smithi152 ceph-mon[37177]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:00.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:00 smithi152 ceph-mon[37177]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:00.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:00 smithi142 ceph-mon[29027]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:00.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:00 smithi142 ceph-mon[29027]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:00.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.996+0000 7ffa0199c700 1 -- 172.21.15.142:0/1236716027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff9fc02cc70 con 0x7ff9fc103fd0 2024-03-29T04:46:00.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.996+0000 7ffa0099a700 1 -- 172.21.15.142:0/1236716027 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7ff9f005a120 con 0x7ff9fc103fd0 2024-03-29T04:46:00.997 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:46:00.997 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:46:00.998 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:46:00.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:46:00.998 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:46:00.998 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-29T04:46:00.998 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 1 2024-03-29T04:46:00.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:46:00.999 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:46:00.999 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:46:00.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:46:00.999 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:46:00.999 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:46:00.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:46:01.000 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:46:01.000 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-03-29T04:46:01.000 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 1 2024-03-29T04:46:01.000 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:46:01.000 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:46:01.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.999+0000 7ffa0199c700 1 -- 172.21.15.142:0/1236716027 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7ff9e406d0a0 msgr2=0x7ff9e406f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:01.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.999+0000 7ffa0199c700 1 --2- 172.21.15.142:0/1236716027 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7ff9e406d0a0 0x7ff9e406f560 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009fd0 tx=0x7ff9ec005940 comp rx=0 tx=0).stop 2024-03-29T04:46:01.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.999+0000 7ffa0199c700 1 -- 172.21.15.142:0/1236716027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc103fd0 msgr2=0x7ff9fc196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:01.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.999+0000 7ffa0199c700 1 --2- 172.21.15.142:0/1236716027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc103fd0 0x7ff9fc196850 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000e920 tx=0x7ff9f000ec30 comp rx=0 tx=0).stop 2024-03-29T04:46:01.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.999+0000 7ffa0199c700 1 -- 172.21.15.142:0/1236716027 shutdown_connections 2024-03-29T04:46:01.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.999+0000 7ffa0199c700 1 --2- 172.21.15.142:0/1236716027 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7ff9e406d0a0 0x7ff9e406f560 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:01.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.999+0000 7ffa0199c700 1 --2- 172.21.15.142:0/1236716027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9fc1016a0 0x7ff9fc196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:01.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.999+0000 7ffa0199c700 1 --2- 172.21.15.142:0/1236716027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc103fd0 0x7ff9fc196850 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:01.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.999+0000 7ffa0199c700 1 -- 172.21.15.142:0/1236716027 >> 172.21.15.142:0/1236716027 conn(0x7ff9fc0fb030 msgr2=0x7ff9fc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:01.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.999+0000 7ffa0199c700 1 -- 172.21.15.142:0/1236716027 shutdown_connections 2024-03-29T04:46:01.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:00.999+0000 7ffa0199c700 1 -- 172.21.15.142:0/1236716027 wait complete. 2024-03-29T04:46:01.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.101+0000 7fc6b1024700 1 -- 172.21.15.142:0/2004033664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac101eb0 msgr2=0x7fc6ac102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:01.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.101+0000 7fc6b1024700 1 --2- 172.21.15.142:0/2004033664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac101eb0 0x7fc6ac102310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009910 tx=0x7fc6a0009c20 comp rx=0 tx=0).stop 2024-03-29T04:46:01.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.101+0000 7fc6b1024700 1 -- 172.21.15.142:0/2004033664 shutdown_connections 2024-03-29T04:46:01.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.101+0000 7fc6b1024700 1 --2- 172.21.15.142:0/2004033664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac101eb0 0x7fc6ac102310 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:01.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.101+0000 7fc6b1024700 1 --2- 172.21.15.142:0/2004033664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6ac0feed0 0x7fc6ac0ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:01.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.101+0000 7fc6b1024700 1 -- 172.21.15.142:0/2004033664 >> 172.21.15.142:0/2004033664 conn(0x7fc6ac0fa810 msgr2=0x7fc6ac0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:01.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.101+0000 7fc6b1024700 1 -- 172.21.15.142:0/2004033664 shutdown_connections 2024-03-29T04:46:01.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.101+0000 7fc6b1024700 1 -- 172.21.15.142:0/2004033664 wait complete. 2024-03-29T04:46:01.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.102+0000 7fc6b1024700 1 Processor -- start 2024-03-29T04:46:01.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.102+0000 7fc6b1024700 1 -- start start 2024-03-29T04:46:01.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.102+0000 7fc6b1024700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac0feed0 0x7fc6ac1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:01.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.102+0000 7fc6b1024700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6ac101eb0 0x7fc6ac196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:01.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.102+0000 7fc6b1024700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac196f40 con 0x7fc6ac0feed0 2024-03-29T04:46:01.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.102+0000 7fc6b1024700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac197080 con 0x7fc6ac101eb0 2024-03-29T04:46:01.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc6aad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac0feed0 0x7fc6ac1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:01.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc6aad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac0feed0 0x7fc6ac1963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54008/0 (socket says 172.21.15.142:54008) 2024-03-29T04:46:01.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc6aad9d700 1 -- 172.21.15.142:0/2334692052 learned_addr learned my addr 172.21.15.142:0/2334692052 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:46:01.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc6aa59c700 1 --2- 172.21.15.142:0/2334692052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6ac101eb0 0x7fc6ac196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:01.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc6aad9d700 1 -- 172.21.15.142:0/2334692052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6ac101eb0 msgr2=0x7fc6ac196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:01.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc6aad9d700 1 --2- 172.21.15.142:0/2334692052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6ac101eb0 0x7fc6ac196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:01.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc6aad9d700 1 -- 172.21.15.142:0/2334692052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a00095f0 con 0x7fc6ac0feed0 2024-03-29T04:46:01.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc6aa59c700 1 --2- 172.21.15.142:0/2334692052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6ac101eb0 0x7fc6ac196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:46:01.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc6aad9d700 1 --2- 172.21.15.142:0/2334692052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac0feed0 0x7fc6ac1963e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00e8f0 tx=0x7fc69c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:01.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc693fff700 1 -- 172.21.15.142:0/2334692052 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc69c009790 con 0x7fc6ac0feed0 2024-03-29T04:46:01.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc693fff700 1 -- 172.21.15.142:0/2334692052 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc69c00c810 con 0x7fc6ac0feed0 2024-03-29T04:46:01.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc6b1024700 1 -- 172.21.15.142:0/2334692052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac19bb30 con 0x7fc6ac0feed0 2024-03-29T04:46:01.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc693fff700 1 -- 172.21.15.142:0/2334692052 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc69c003d20 con 0x7fc6ac0feed0 2024-03-29T04:46:01.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.103+0000 7fc6b1024700 1 -- 172.21.15.142:0/2334692052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac19c080 con 0x7fc6ac0feed0 2024-03-29T04:46:01.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.104+0000 7fc6b1024700 1 -- 172.21.15.142:0/2334692052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ac109770 con 0x7fc6ac0feed0 2024-03-29T04:46:01.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.105+0000 7fc693fff700 1 -- 172.21.15.142:0/2334692052 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 91117+0+0 (secure 0 0 0) 0x7fc69c018430 con 0x7fc6ac0feed0 2024-03-29T04:46:01.107 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.105+0000 7fc693fff700 1 --2- 172.21.15.142:0/2334692052 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fc69406d0a0 0x7fc69406f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:01.107 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.105+0000 7fc693fff700 1 -- 172.21.15.142:0/2334692052 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc69c014070 con 0x7fc6ac0feed0 2024-03-29T04:46:01.107 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.105+0000 7fc6aa59c700 1 --2- 172.21.15.142:0/2334692052 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fc69406d0a0 0x7fc69406f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:01.107 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.106+0000 7fc6aa59c700 1 --2- 172.21.15.142:0/2334692052 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fc69406d0a0 0x7fc69406f560 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00059d0 tx=0x7fc6a0005940 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:01.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.113+0000 7fc693fff700 1 -- 172.21.15.142:0/2334692052 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fc69c05a2d0 con 0x7fc6ac0feed0 2024-03-29T04:46:01.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.374+0000 7fc6b1024700 1 -- 172.21.15.142:0/2334692052 --> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6ac106800 con 0x7fc69406d0a0 2024-03-29T04:46:01.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.375+0000 7fc693fff700 1 -- 172.21.15.142:0/2334692052 <== mgr.24395 v2:172.21.15.152:6828/3130987667 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fc6ac106800 con 0x7fc69406d0a0 2024-03-29T04:46:01.375 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:46:01.375 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:46:01.375 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:46:01.375 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T04:46:01.375 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [], 2024-03-29T04:46:01.376 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "1/2 daemons upgraded", 2024-03-29T04:46:01.376 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "", 2024-03-29T04:46:01.376 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:46:01.376 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:46:01.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.378+0000 7fc6b1024700 1 -- 172.21.15.142:0/2334692052 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fc69406d0a0 msgr2=0x7fc69406f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:01.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.378+0000 7fc6b1024700 1 --2- 172.21.15.142:0/2334692052 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fc69406d0a0 0x7fc69406f560 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00059d0 tx=0x7fc6a0005940 comp rx=0 tx=0).stop 2024-03-29T04:46:01.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.378+0000 7fc6b1024700 1 -- 172.21.15.142:0/2334692052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac0feed0 msgr2=0x7fc6ac1963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:01.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.378+0000 7fc6b1024700 1 --2- 172.21.15.142:0/2334692052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac0feed0 0x7fc6ac1963e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00e8f0 tx=0x7fc69c00ec00 comp rx=0 tx=0).stop 2024-03-29T04:46:01.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.378+0000 7fc6b1024700 1 -- 172.21.15.142:0/2334692052 shutdown_connections 2024-03-29T04:46:01.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.378+0000 7fc6b1024700 1 --2- 172.21.15.142:0/2334692052 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fc69406d0a0 0x7fc69406f560 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:01.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.378+0000 7fc6b1024700 1 --2- 172.21.15.142:0/2334692052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6ac0feed0 0x7fc6ac1963e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:01.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.378+0000 7fc6b1024700 1 --2- 172.21.15.142:0/2334692052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6ac101eb0 0x7fc6ac196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:01.379 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.378+0000 7fc6b1024700 1 -- 172.21.15.142:0/2334692052 >> 172.21.15.142:0/2334692052 conn(0x7fc6ac0fa810 msgr2=0x7fc6ac1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:01.379 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.378+0000 7fc6b1024700 1 -- 172.21.15.142:0/2334692052 shutdown_connections 2024-03-29T04:46:01.379 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:01.378+0000 7fc6b1024700 1 -- 172.21.15.142:0/2334692052 wait complete. 2024-03-29T04:46:01.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:01 smithi152 ceph-mon[37177]: pgmap v12: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 433 op/s 2024-03-29T04:46:01.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:01 smithi152 ceph-mon[37177]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:01.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:01 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/1236716027' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:46:01.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:01 smithi142 ceph-mon[29027]: pgmap v12: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 433 op/s 2024-03-29T04:46:01.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:01 smithi142 ceph-mon[29027]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:01.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:01 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/1236716027' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:46:02.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:02 smithi152 ceph-mon[37177]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:02.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:02 smithi142 ceph-mon[29027]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:03.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:03 smithi152 ceph-mon[37177]: pgmap v13: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 343 op/s 2024-03-29T04:46:03.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:03 smithi142 ceph-mon[29027]: pgmap v13: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 343 op/s 2024-03-29T04:46:05.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:05 smithi152 ceph-mon[37177]: pgmap v14: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 542 op/s 2024-03-29T04:46:05.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:05 smithi142 ceph-mon[29027]: pgmap v14: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 542 op/s 2024-03-29T04:46:07.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:07 smithi152 ceph-mon[37177]: pgmap v15: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 402 op/s 2024-03-29T04:46:07.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:07 smithi142 ceph-mon[29027]: pgmap v15: 65 pgs: 65 active+clean; 294 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 402 op/s 2024-03-29T04:46:09.790 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:09 smithi152 ceph-mon[37177]: pgmap v16: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 431 op/s 2024-03-29T04:46:09.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:09 smithi142 ceph-mon[29027]: pgmap v16: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 431 op/s 2024-03-29T04:46:11.717 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:11 smithi152 ceph-mon[37177]: pgmap v17: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 521 op/s 2024-03-29T04:46:11.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:11 smithi142 ceph-mon[29027]: pgmap v17: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 521 op/s 2024-03-29T04:46:13.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:13.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:13.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:46:13.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:46:13.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:46:13.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:46:13.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:46:13.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:46:13.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: Adjusting osd_memory_target on smithi152 to 2240M 2024-03-29T04:46:13.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:13.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:46:13.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:46:13.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-29T04:46:13.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-29T04:46:13.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:13 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:46:13.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:46:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:46:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:46:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:46:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:46:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-29T04:46:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: Adjusting osd_memory_target on smithi152 to 2240M 2024-03-29T04:46:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:46:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:46:13.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-29T04:46:13.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-29T04:46:13.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:13 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:46:14.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:14 smithi142 ceph-mon[29027]: pgmap v18: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 371 op/s 2024-03-29T04:46:14.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:14 smithi142 ceph-mon[29027]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:46:14.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:14 smithi142 ceph-mon[29027]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:46:14.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:14 smithi142 ceph-mon[29027]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:46:14.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:14 smithi142 ceph-mon[29027]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:46:14.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:14 smithi142 ceph-mon[29027]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:46:14.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:14 smithi142 ceph-mon[29027]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:46:14.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:14 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:14.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:14 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:14.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:14 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:14.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:14 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:14.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:14 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:14.540 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:14 smithi152 ceph-mon[37177]: pgmap v18: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 371 op/s 2024-03-29T04:46:14.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:14 smithi152 ceph-mon[37177]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:46:14.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:14 smithi152 ceph-mon[37177]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:46:14.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:14 smithi152 ceph-mon[37177]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:46:14.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:14 smithi152 ceph-mon[37177]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:46:14.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:14 smithi152 ceph-mon[37177]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:46:14.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:14 smithi152 ceph-mon[37177]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:46:14.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:14 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:14.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:14 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:14.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:14 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:14.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:14 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:14.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:14 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:15.540 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:15 smithi152 ceph-mon[37177]: Reconfiguring prometheus.smithi142 (dependencies changed)... 2024-03-29T04:46:15.540 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:15 smithi152 ceph-mon[37177]: Reconfiguring daemon prometheus.smithi142 on smithi142 2024-03-29T04:46:15.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:15 smithi142 ceph-mon[29027]: Reconfiguring prometheus.smithi142 (dependencies changed)... 2024-03-29T04:46:15.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:15 smithi142 ceph-mon[29027]: Reconfiguring daemon prometheus.smithi142 on smithi142 2024-03-29T04:46:16.540 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:16 smithi152 ceph-mon[37177]: pgmap v19: 65 pgs: 65 active+clean; 295 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 595 op/s 2024-03-29T04:46:16.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:16 smithi142 ceph-mon[29027]: pgmap v19: 65 pgs: 65 active+clean; 295 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 595 op/s 2024-03-29T04:46:18.450 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:18 smithi152 ceph-mon[37177]: pgmap v20: 65 pgs: 65 active+clean; 295 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 397 op/s 2024-03-29T04:46:18.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:18 smithi142 ceph-mon[29027]: pgmap v20: 65 pgs: 65 active+clean; 295 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 397 op/s 2024-03-29T04:46:20.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:20 smithi152 ceph-mon[37177]: pgmap v21: 65 pgs: 65 active+clean; 298 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 443 op/s 2024-03-29T04:46:20.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:20 smithi142 ceph-mon[29027]: pgmap v21: 65 pgs: 65 active+clean; 298 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 443 op/s 2024-03-29T04:46:22.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:22 smithi152 ceph-mon[37177]: pgmap v22: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 610 op/s 2024-03-29T04:46:22.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:22 smithi142 ceph-mon[29027]: pgmap v22: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 610 op/s 2024-03-29T04:46:24.540 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:24 smithi152 ceph-mon[37177]: pgmap v23: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 466 op/s 2024-03-29T04:46:24.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:24 smithi142 ceph-mon[29027]: pgmap v23: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 466 op/s 2024-03-29T04:46:26.447 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:26 smithi142 ceph-mon[29027]: pgmap v24: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 651 op/s 2024-03-29T04:46:26.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:26 smithi152 ceph-mon[37177]: pgmap v24: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 651 op/s 2024-03-29T04:46:28.303 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:28 smithi142 ceph-mon[29027]: pgmap v25: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 427 op/s 2024-03-29T04:46:28.303 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:28 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:46:28.507 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:28 smithi152 ceph-mon[37177]: pgmap v25: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 427 op/s 2024-03-29T04:46:28.507 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:28 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:46:30.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:30 smithi152 ceph-mon[37177]: pgmap v26: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 453 op/s 2024-03-29T04:46:30.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:30 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:30.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:30 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:30.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:30 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:46:30.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:30 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:46:30.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:30 smithi142 ceph-mon[29027]: pgmap v26: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 453 op/s 2024-03-29T04:46:30.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:30 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:30.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:30 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:30.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:30 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:46:30.555 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:30 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:46:30.557 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-29T04:46:30.557 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-29T04:46:30.914 DEBUG:teuthology.parallel:result is None 2024-03-29T04:46:31.534 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.533+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1217707257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c04101f20 msgr2=0x7f5c04102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:31.534 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.533+0000 7f5c0c23f700 1 --2- 172.21.15.142:0/1217707257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c04101f20 0x7f5c04102380 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009910 tx=0x7f5bf4009c20 comp rx=0 tx=0).stop 2024-03-29T04:46:31.535 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:31 smithi142 ceph-mon[29027]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:46:31.535 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:31 smithi142 ceph-mon[29027]: Upgrade: Need to upgrade myself (mgr.smithi152.usoolh) 2024-03-29T04:46:31.535 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:31 smithi142 ceph-mon[29027]: Upgrade: Need to upgrade myself (mgr.smithi152.usoolh) 2024-03-29T04:46:31.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.533+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1217707257 shutdown_connections 2024-03-29T04:46:31.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.533+0000 7f5c0c23f700 1 --2- 172.21.15.142:0/1217707257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c04101f20 0x7f5c04102380 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:31.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.533+0000 7f5c0c23f700 1 --2- 172.21.15.142:0/1217707257 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c040feea0 0x7f5c040ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:31.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.533+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1217707257 >> 172.21.15.142:0/1217707257 conn(0x7f5c040fa7c0 msgr2=0x7f5c040fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:31.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.533+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1217707257 shutdown_connections 2024-03-29T04:46:31.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.533+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1217707257 wait complete. 2024-03-29T04:46:31.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.534+0000 7f5c0c23f700 1 Processor -- start 2024-03-29T04:46:31.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.534+0000 7f5c0c23f700 1 -- start start 2024-03-29T04:46:31.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.534+0000 7f5c0c23f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c040feea0 0x7f5c04198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:31.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.534+0000 7f5c0c23f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c04101f20 0x7f5c04198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:31.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.534+0000 7f5c0c23f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c041990e0 con 0x7f5c04101f20 2024-03-29T04:46:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.534+0000 7f5c0c23f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c04199220 con 0x7f5c040feea0 2024-03-29T04:46:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.534+0000 7f5c097da700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c04101f20 0x7f5c04198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.534+0000 7f5c097da700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c04101f20 0x7f5c04198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58548/0 (socket says 172.21.15.142:58548) 2024-03-29T04:46:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.534+0000 7f5c097da700 1 -- 172.21.15.142:0/1473690510 learned_addr learned my addr 172.21.15.142:0/1473690510 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:46:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.534+0000 7f5c09fdb700 1 --2- 172.21.15.142:0/1473690510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c040feea0 0x7f5c04198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.535+0000 7f5c097da700 1 -- 172.21.15.142:0/1473690510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c040feea0 msgr2=0x7f5c04198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.535+0000 7f5c097da700 1 --2- 172.21.15.142:0/1473690510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c040feea0 0x7f5c04198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.535+0000 7f5c097da700 1 -- 172.21.15.142:0/1473690510 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf40095f0 con 0x7f5c04101f20 2024-03-29T04:46:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.535+0000 7f5c097da700 1 --2- 172.21.15.142:0/1473690510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c04101f20 0x7f5c04198ac0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4000c00 tx=0x7f5bf400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.535+0000 7f5bfaffd700 1 -- 172.21.15.142:0/1473690510 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bf401d070 con 0x7f5c04101f20 2024-03-29T04:46:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.535+0000 7f5bfaffd700 1 -- 172.21.15.142:0/1473690510 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5bf4005dc0 con 0x7f5c04101f20 2024-03-29T04:46:31.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.535+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1473690510 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0419dc70 con 0x7f5c04101f20 2024-03-29T04:46:31.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.536+0000 7f5bfaffd700 1 -- 172.21.15.142:0/1473690510 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bf40226a0 con 0x7f5c04101f20 2024-03-29T04:46:31.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.536+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1473690510 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0419e160 con 0x7f5c04101f20 2024-03-29T04:46:31.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.538+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1473690510 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c04106000 con 0x7f5c04101f20 2024-03-29T04:46:31.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.538+0000 7f5bfaffd700 1 -- 172.21.15.142:0/1473690510 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 91117+0+0 (secure 0 0 0) 0x7f5bf40038f0 con 0x7f5c04101f20 2024-03-29T04:46:31.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.538+0000 7f5bfaffd700 1 --2- 172.21.15.142:0/1473690510 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f5bf006d0a0 0x7f5bf006f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:31.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.538+0000 7f5bfaffd700 1 -- 172.21.15.142:0/1473690510 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f5bf408dbd0 con 0x7f5c04101f20 2024-03-29T04:46:31.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.539+0000 7f5c09fdb700 1 --2- 172.21.15.142:0/1473690510 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f5bf006d0a0 0x7f5bf006f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:31.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.539+0000 7f5c09fdb700 1 --2- 172.21.15.142:0/1473690510 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f5bf006d0a0 0x7f5bf006f560 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5c04068150 tx=0x7f5c00009260 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:31.540 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:31 smithi152 ceph-mon[37177]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:46:31.540 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:31 smithi152 ceph-mon[37177]: Upgrade: Need to upgrade myself (mgr.smithi152.usoolh) 2024-03-29T04:46:31.540 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:31 smithi152 ceph-mon[37177]: Upgrade: Need to upgrade myself (mgr.smithi152.usoolh) 2024-03-29T04:46:31.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.548+0000 7f5bfaffd700 1 -- 172.21.15.142:0/1473690510 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f5bf405b1d0 con 0x7f5c04101f20 2024-03-29T04:46:31.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.797+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1473690510 --> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c040611d0 con 0x7f5bf006d0a0 2024-03-29T04:46:31.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.798+0000 7f5bfaffd700 1 -- 172.21.15.142:0/1473690510 <== mgr.24395 v2:172.21.15.152:6828/3130987667 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f5c040611d0 con 0x7f5bf006d0a0 2024-03-29T04:46:31.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.801+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1473690510 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f5bf006d0a0 msgr2=0x7f5bf006f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:31.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.801+0000 7f5c0c23f700 1 --2- 172.21.15.142:0/1473690510 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f5bf006d0a0 0x7f5bf006f560 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5c04068150 tx=0x7f5c00009260 comp rx=0 tx=0).stop 2024-03-29T04:46:31.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.801+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1473690510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c04101f20 msgr2=0x7f5c04198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:31.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.801+0000 7f5c0c23f700 1 --2- 172.21.15.142:0/1473690510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c04101f20 0x7f5c04198ac0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4000c00 tx=0x7f5bf400f690 comp rx=0 tx=0).stop 2024-03-29T04:46:31.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.801+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1473690510 shutdown_connections 2024-03-29T04:46:31.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.801+0000 7f5c0c23f700 1 --2- 172.21.15.142:0/1473690510 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f5bf006d0a0 0x7f5bf006f560 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:31.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.801+0000 7f5c0c23f700 1 --2- 172.21.15.142:0/1473690510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c040feea0 0x7f5c04198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:31.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.801+0000 7f5c0c23f700 1 --2- 172.21.15.142:0/1473690510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c04101f20 0x7f5c04198ac0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:31.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.801+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1473690510 >> 172.21.15.142:0/1473690510 conn(0x7f5c040fa7c0 msgr2=0x7f5c04103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:31.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.801+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1473690510 shutdown_connections 2024-03-29T04:46:31.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.801+0000 7f5c0c23f700 1 -- 172.21.15.142:0/1473690510 wait complete. 2024-03-29T04:46:31.813 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:46:31.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.926+0000 7f6f931b1700 1 -- 172.21.15.142:0/1256231372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c103f70 msgr2=0x7f6f8c106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:31.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.926+0000 7f6f931b1700 1 --2- 172.21.15.142:0/1256231372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c103f70 0x7f6f8c106360 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009910 tx=0x7f6f78009c20 comp rx=0 tx=0).stop 2024-03-29T04:46:31.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.926+0000 7f6f931b1700 1 -- 172.21.15.142:0/1256231372 shutdown_connections 2024-03-29T04:46:31.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.926+0000 7f6f931b1700 1 --2- 172.21.15.142:0/1256231372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c103f70 0x7f6f8c106360 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:31.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.926+0000 7f6f931b1700 1 --2- 172.21.15.142:0/1256231372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f8c101640 0x7f6f8c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:31.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.926+0000 7f6f931b1700 1 -- 172.21.15.142:0/1256231372 >> 172.21.15.142:0/1256231372 conn(0x7f6f8c0fb030 msgr2=0x7f6f8c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:31.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.927+0000 7f6f931b1700 1 -- 172.21.15.142:0/1256231372 shutdown_connections 2024-03-29T04:46:31.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.927+0000 7f6f931b1700 1 -- 172.21.15.142:0/1256231372 wait complete. 2024-03-29T04:46:31.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.927+0000 7f6f931b1700 1 Processor -- start 2024-03-29T04:46:31.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.927+0000 7f6f931b1700 1 -- start start 2024-03-29T04:46:31.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.927+0000 7f6f931b1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f8c101640 0x7f6f8c196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:31.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.927+0000 7f6f931b1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c103f70 0x7f6f8c196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:31.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.927+0000 7f6f931b1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c196e90 con 0x7f6f8c103f70 2024-03-29T04:46:31.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.927+0000 7f6f931b1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8c196fd0 con 0x7f6f8c101640 2024-03-29T04:46:31.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.927+0000 7f6f8bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c103f70 0x7f6f8c196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:31.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.928+0000 7f6f90f4d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f8c101640 0x7f6f8c196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:31.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.928+0000 7f6f8bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c103f70 0x7f6f8c196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58562/0 (socket says 172.21.15.142:58562) 2024-03-29T04:46:31.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.928+0000 7f6f8bfff700 1 -- 172.21.15.142:0/608310602 learned_addr learned my addr 172.21.15.142:0/608310602 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:46:31.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.928+0000 7f6f90f4d700 1 -- 172.21.15.142:0/608310602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c103f70 msgr2=0x7f6f8c196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:31.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.928+0000 7f6f90f4d700 1 --2- 172.21.15.142:0/608310602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c103f70 0x7f6f8c196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:31.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.928+0000 7f6f90f4d700 1 -- 172.21.15.142:0/608310602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f780095f0 con 0x7f6f8c101640 2024-03-29T04:46:31.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.928+0000 7f6f90f4d700 1 --2- 172.21.15.142:0/608310602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f8c101640 0x7f6f8c196330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000e970 tx=0x7f6f8000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:31.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.928+0000 7f6f89ffb700 1 -- 172.21.15.142:0/608310602 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f8000c9d0 con 0x7f6f8c101640 2024-03-29T04:46:31.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.928+0000 7f6f931b1700 1 -- 172.21.15.142:0/608310602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c19ba80 con 0x7f6f8c101640 2024-03-29T04:46:31.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.928+0000 7f6f89ffb700 1 -- 172.21.15.142:0/608310602 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6f80010430 con 0x7f6f8c101640 2024-03-29T04:46:31.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.928+0000 7f6f89ffb700 1 -- 172.21.15.142:0/608310602 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f80018650 con 0x7f6f8c101640 2024-03-29T04:46:31.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.929+0000 7f6f931b1700 1 -- 172.21.15.142:0/608310602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8c19bfd0 con 0x7f6f8c101640 2024-03-29T04:46:31.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.930+0000 7f6f931b1700 1 -- 172.21.15.142:0/608310602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8c190520 con 0x7f6f8c101640 2024-03-29T04:46:31.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.930+0000 7f6f89ffb700 1 -- 172.21.15.142:0/608310602 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) v1 ==== 91117+0+0 (secure 0 0 0) 0x7f6f800105a0 con 0x7f6f8c101640 2024-03-29T04:46:31.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.930+0000 7f6f89ffb700 1 --2- 172.21.15.142:0/608310602 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f6f7c06d0a0 0x7f6f7c06f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:31.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.931+0000 7f6f89ffb700 1 -- 172.21.15.142:0/608310602 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f6f80014070 con 0x7f6f8c101640 2024-03-29T04:46:31.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.931+0000 7f6f8bfff700 1 --2- 172.21.15.142:0/608310602 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f6f7c06d0a0 0x7f6f7c06f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:31.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.931+0000 7f6f8bfff700 1 --2- 172.21.15.142:0/608310602 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f6f7c06d0a0 0x7f6f7c06f560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009fd0 tx=0x7f6f78005940 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:31.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:31.936+0000 7f6f89ffb700 1 -- 172.21.15.142:0/608310602 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f6f80052fc0 con 0x7f6f8c101640 2024-03-29T04:46:32.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.204+0000 7f6f931b1700 1 -- 172.21.15.142:0/608310602 --> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f8c0611d0 con 0x7f6f7c06d0a0 2024-03-29T04:46:32.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.205+0000 7f6f89ffb700 1 -- 172.21.15.142:0/608310602 <== mgr.24395 v2:172.21.15.152:6828/3130987667 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f6f8c0611d0 con 0x7f6f7c06d0a0 2024-03-29T04:46:32.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.208+0000 7f6f931b1700 1 -- 172.21.15.142:0/608310602 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f6f7c06d0a0 msgr2=0x7f6f7c06f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:32.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.208+0000 7f6f931b1700 1 --2- 172.21.15.142:0/608310602 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f6f7c06d0a0 0x7f6f7c06f560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009fd0 tx=0x7f6f78005940 comp rx=0 tx=0).stop 2024-03-29T04:46:32.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.208+0000 7f6f931b1700 1 -- 172.21.15.142:0/608310602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f8c101640 msgr2=0x7f6f8c196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:32.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.208+0000 7f6f931b1700 1 --2- 172.21.15.142:0/608310602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f8c101640 0x7f6f8c196330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000e970 tx=0x7f6f8000ec80 comp rx=0 tx=0).stop 2024-03-29T04:46:32.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.208+0000 7f6f931b1700 1 -- 172.21.15.142:0/608310602 shutdown_connections 2024-03-29T04:46:32.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.208+0000 7f6f931b1700 1 --2- 172.21.15.142:0/608310602 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f6f7c06d0a0 0x7f6f7c06f560 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:32.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.208+0000 7f6f931b1700 1 --2- 172.21.15.142:0/608310602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f8c101640 0x7f6f8c196330 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:32.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.208+0000 7f6f931b1700 1 --2- 172.21.15.142:0/608310602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f8c103f70 0x7f6f8c196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:32.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.208+0000 7f6f931b1700 1 -- 172.21.15.142:0/608310602 >> 172.21.15.142:0/608310602 conn(0x7f6f8c0fb030 msgr2=0x7f6f8c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:32.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.208+0000 7f6f931b1700 1 -- 172.21.15.142:0/608310602 shutdown_connections 2024-03-29T04:46:32.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.208+0000 7f6f931b1700 1 -- 172.21.15.142:0/608310602 wait complete. 2024-03-29T04:46:32.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.325+0000 7f7d6f169700 1 -- 172.21.15.142:0/1495844974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d680ffab0 msgr2=0x7f7d680ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:32.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.325+0000 7f7d6f169700 1 --2- 172.21.15.142:0/1495844974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d680ffab0 0x7f7d680ffed0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009910 tx=0x7f7d5c009c20 comp rx=0 tx=0).stop 2024-03-29T04:46:32.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.325+0000 7f7d6f169700 1 -- 172.21.15.142:0/1495844974 shutdown_connections 2024-03-29T04:46:32.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.325+0000 7f7d6f169700 1 --2- 172.21.15.142:0/1495844974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d68100cb0 0x7f7d68101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:32.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.325+0000 7f7d6f169700 1 --2- 172.21.15.142:0/1495844974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d680ffab0 0x7f7d680ffed0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:32.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.325+0000 7f7d6f169700 1 -- 172.21.15.142:0/1495844974 >> 172.21.15.142:0/1495844974 conn(0x7f7d680fb030 msgr2=0x7f7d680fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:32.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.325+0000 7f7d6f169700 1 -- 172.21.15.142:0/1495844974 shutdown_connections 2024-03-29T04:46:32.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.325+0000 7f7d6f169700 1 -- 172.21.15.142:0/1495844974 wait complete. 2024-03-29T04:46:32.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.326+0000 7f7d6f169700 1 Processor -- start 2024-03-29T04:46:32.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.326+0000 7f7d6f169700 1 -- start start 2024-03-29T04:46:32.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.326+0000 7f7d6f169700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d680ffab0 0x7f7d68103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:32.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.326+0000 7f7d6f169700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d68100cb0 0x7f7d68102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:32.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.326+0000 7f7d6f169700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d681025d0 con 0x7f7d68100cb0 2024-03-29T04:46:32.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.326+0000 7f7d6f169700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d68102710 con 0x7f7d680ffab0 2024-03-29T04:46:32.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.326+0000 7f7d67fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d68100cb0 0x7f7d68102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:32.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.326+0000 7f7d67fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d68100cb0 0x7f7d68102090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58576/0 (socket says 172.21.15.142:58576) 2024-03-29T04:46:32.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.326+0000 7f7d67fff700 1 -- 172.21.15.142:0/144280243 learned_addr learned my addr 172.21.15.142:0/144280243 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:46:32.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.326+0000 7f7d6cf05700 1 --2- 172.21.15.142:0/144280243 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d680ffab0 0x7f7d68103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:32.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.327+0000 7f7d67fff700 1 -- 172.21.15.142:0/144280243 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d680ffab0 msgr2=0x7f7d68103a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:32.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.327+0000 7f7d67fff700 1 --2- 172.21.15.142:0/144280243 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d680ffab0 0x7f7d68103a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:32.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.327+0000 7f7d67fff700 1 -- 172.21.15.142:0/144280243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c0095f0 con 0x7f7d68100cb0 2024-03-29T04:46:32.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.327+0000 7f7d6cf05700 1 --2- 172.21.15.142:0/144280243 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d680ffab0 0x7f7d68103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:46:32.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.327+0000 7f7d67fff700 1 --2- 172.21.15.142:0/144280243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d68100cb0 0x7f7d68102090 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400e920 tx=0x7f7d5400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:32.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.327+0000 7f7d65ffb700 1 -- 172.21.15.142:0/144280243 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d5400c9d0 con 0x7f7d68100cb0 2024-03-29T04:46:32.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.327+0000 7f7d6f169700 1 -- 172.21.15.142:0/144280243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d681029f0 con 0x7f7d68100cb0 2024-03-29T04:46:32.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.327+0000 7f7d65ffb700 1 -- 172.21.15.142:0/144280243 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7d54010430 con 0x7f7d68100cb0 2024-03-29T04:46:32.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.327+0000 7f7d65ffb700 1 -- 172.21.15.142:0/144280243 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d54018650 con 0x7f7d68100cb0 2024-03-29T04:46:32.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.327+0000 7f7d6f169700 1 -- 172.21.15.142:0/144280243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6819e380 con 0x7f7d68100cb0 2024-03-29T04:46:32.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.328+0000 7f7d6f169700 1 -- 172.21.15.142:0/144280243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d68108610 con 0x7f7d68100cb0 2024-03-29T04:46:32.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.329+0000 7f7d65ffb700 1 -- 172.21.15.142:0/144280243 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 91117+0+0 (secure 0 0 0) 0x7f7d540105a0 con 0x7f7d68100cb0 2024-03-29T04:46:32.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.329+0000 7f7d65ffb700 1 --2- 172.21.15.142:0/144280243 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f7d5806d0a0 0x7f7d5806f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:32.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.329+0000 7f7d65ffb700 1 -- 172.21.15.142:0/144280243 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f7d54014070 con 0x7f7d68100cb0 2024-03-29T04:46:32.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.329+0000 7f7d6cf05700 1 --2- 172.21.15.142:0/144280243 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f7d5806d0a0 0x7f7d5806f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:32.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.330+0000 7f7d6cf05700 1 --2- 172.21.15.142:0/144280243 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f7d5806d0a0 0x7f7d5806f560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00b6f0 tx=0x7f7d5c005940 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:32.334 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.334+0000 7f7d65ffb700 1 -- 172.21.15.142:0/144280243 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f7d54052fc0 con 0x7f7d68100cb0 2024-03-29T04:46:32.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:32 smithi152 ceph-mon[37177]: pgmap v27: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 555 op/s 2024-03-29T04:46:32.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.612+0000 7f7d6f169700 1 -- 172.21.15.142:0/144280243 --> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d68068ae0 con 0x7f7d5806d0a0 2024-03-29T04:46:32.613 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:32 smithi142 ceph-mon[29027]: pgmap v27: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 555 op/s 2024-03-29T04:46:32.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.629+0000 7f7d65ffb700 1 -- 172.21.15.142:0/144280243 <== mgr.24395 v2:172.21.15.152:6828/3130987667 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7d68068ae0 con 0x7f7d5806d0a0 2024-03-29T04:46:32.630 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:46:32.630 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (8m) 37s ago 9m 25.1M - 0.25.0 c8568f914cd2 d4a3e12f4047 2024-03-29T04:46:32.630 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (10m) 37s ago 10m 7856k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:46:32.630 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (9m) 39s ago 9m 7721k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:46:32.630 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (10m) 37s ago 10m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:46:32.630 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (9m) 39s ago 9m 6660k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (8m) 37s ago 9m 85.8M - 9.4.7 954c08fa6188 aba58e552b7c 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (4m) 37s ago 4m 176M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (4m) 37s ago 4m 95.3M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (4m) 39s ago 4m 89.4M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (4m) 39s ago 4m 149M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:9283,8765,8443 running (11m) 37s ago 11m 421M - 18.2.1 5be31c24972a a5f21a9cfe24 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (61s) 39s ago 8m 489M - 19.0.0-1959-g8da23624 54112f57da44 2aeb29a22d55 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (11m) 37s ago 11m 51.0M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (8m) 39s ago 8m 39.4M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (10m) 37s ago 10m 15.7M - 1.5.0 0da6a335fe13 91b95c929db2 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (8m) 39s ago 8m 14.9M - 1.5.0 0da6a335fe13 97e90e5b337b 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (8m) 37s ago 8m 202M 1216M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (7m) 37s ago 7m 218M 1216M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:46:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (7m) 37s ago 7m 176M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:46:32.632 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (6m) 39s ago 6m 255M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:46:32.632 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (6m) 39s ago 6m 218M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:46:32.632 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (5m) 39s ago 5m 208M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:46:32.632 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 starting - - - - 2024-03-29T04:46:32.633 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.632+0000 7f7d6f169700 1 -- 172.21.15.142:0/144280243 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f7d5806d0a0 msgr2=0x7f7d5806f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:32.633 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.632+0000 7f7d6f169700 1 --2- 172.21.15.142:0/144280243 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f7d5806d0a0 0x7f7d5806f560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00b6f0 tx=0x7f7d5c005940 comp rx=0 tx=0).stop 2024-03-29T04:46:32.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.632+0000 7f7d6f169700 1 -- 172.21.15.142:0/144280243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d68100cb0 msgr2=0x7f7d68102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:32.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.632+0000 7f7d6f169700 1 --2- 172.21.15.142:0/144280243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d68100cb0 0x7f7d68102090 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400e920 tx=0x7f7d5400ec30 comp rx=0 tx=0).stop 2024-03-29T04:46:32.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.633+0000 7f7d6f169700 1 -- 172.21.15.142:0/144280243 shutdown_connections 2024-03-29T04:46:32.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.633+0000 7f7d6f169700 1 --2- 172.21.15.142:0/144280243 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7f7d5806d0a0 0x7f7d5806f560 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:32.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.633+0000 7f7d6f169700 1 --2- 172.21.15.142:0/144280243 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d680ffab0 0x7f7d68103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:32.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.633+0000 7f7d6f169700 1 --2- 172.21.15.142:0/144280243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d68100cb0 0x7f7d68102090 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:32.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.633+0000 7f7d6f169700 1 -- 172.21.15.142:0/144280243 >> 172.21.15.142:0/144280243 conn(0x7f7d680fb030 msgr2=0x7f7d680683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:32.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.633+0000 7f7d6f169700 1 -- 172.21.15.142:0/144280243 shutdown_connections 2024-03-29T04:46:32.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.633+0000 7f7d6f169700 1 -- 172.21.15.142:0/144280243 wait complete. 2024-03-29T04:46:32.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.771+0000 7fc62ff2c700 1 -- 172.21.15.142:0/2962998573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc628101df0 msgr2=0x7fc628102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:32.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.771+0000 7fc62ff2c700 1 --2- 172.21.15.142:0/2962998573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc628101df0 0x7fc628102250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc618009910 tx=0x7fc618009c20 comp rx=0 tx=0).stop 2024-03-29T04:46:32.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.771+0000 7fc62ff2c700 1 -- 172.21.15.142:0/2962998573 shutdown_connections 2024-03-29T04:46:32.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.771+0000 7fc62ff2c700 1 --2- 172.21.15.142:0/2962998573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc628101df0 0x7fc628102250 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:32.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.771+0000 7fc62ff2c700 1 --2- 172.21.15.142:0/2962998573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc628100bf0 0x7fc628101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:32.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.771+0000 7fc62ff2c700 1 -- 172.21.15.142:0/2962998573 >> 172.21.15.142:0/2962998573 conn(0x7fc6280fc190 msgr2=0x7fc6280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62ff2c700 1 -- 172.21.15.142:0/2962998573 shutdown_connections 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62ff2c700 1 -- 172.21.15.142:0/2962998573 wait complete. 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62ff2c700 1 Processor -- start 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62ff2c700 1 -- start start 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62ff2c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc628100bf0 0x7fc628074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62ff2c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc628101df0 0x7fc628073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62ff2c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6280736c0 con 0x7fc628101df0 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62ff2c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc628073800 con 0x7fc628100bf0 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62d4c7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc628101df0 0x7fc628073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62dcc8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc628100bf0 0x7fc628074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62dcc8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc628100bf0 0x7fc628074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.142:37104/0 (socket says 172.21.15.142:37104) 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62d4c7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc628101df0 0x7fc628073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58598/0 (socket says 172.21.15.142:58598) 2024-03-29T04:46:32.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.772+0000 7fc62dcc8700 1 -- 172.21.15.142:0/3944098909 learned_addr learned my addr 172.21.15.142:0/3944098909 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:46:32.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.773+0000 7fc62dcc8700 1 -- 172.21.15.142:0/3944098909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc628101df0 msgr2=0x7fc628073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:32.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.773+0000 7fc62dcc8700 1 --2- 172.21.15.142:0/3944098909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc628101df0 0x7fc628073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:32.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.773+0000 7fc62dcc8700 1 -- 172.21.15.142:0/3944098909 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6180095f0 con 0x7fc628100bf0 2024-03-29T04:46:32.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.773+0000 7fc62dcc8700 1 --2- 172.21.15.142:0/3944098909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc628100bf0 0x7fc628074b30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc62400e8f0 tx=0x7fc62400ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:32.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.773+0000 7fc61effd700 1 -- 172.21.15.142:0/3944098909 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc624009790 con 0x7fc628100bf0 2024-03-29T04:46:32.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.773+0000 7fc61effd700 1 -- 172.21.15.142:0/3944098909 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc62400c810 con 0x7fc628100bf0 2024-03-29T04:46:32.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.773+0000 7fc62ff2c700 1 -- 172.21.15.142:0/3944098909 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc628073ae0 con 0x7fc628100bf0 2024-03-29T04:46:32.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.773+0000 7fc61effd700 1 -- 172.21.15.142:0/3944098909 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc624003d20 con 0x7fc628100bf0 2024-03-29T04:46:32.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.773+0000 7fc62ff2c700 1 -- 172.21.15.142:0/3944098909 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc628073fd0 con 0x7fc628100bf0 2024-03-29T04:46:32.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.775+0000 7fc62ff2c700 1 -- 172.21.15.142:0/3944098909 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc628109750 con 0x7fc628100bf0 2024-03-29T04:46:32.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.775+0000 7fc61effd700 1 -- 172.21.15.142:0/3944098909 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) v1 ==== 91117+0+0 (secure 0 0 0) 0x7fc6240041f0 con 0x7fc628100bf0 2024-03-29T04:46:32.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.775+0000 7fc61effd700 1 --2- 172.21.15.142:0/3944098909 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fc61406d0a0 0x7fc61406f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:32.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.776+0000 7fc61effd700 1 -- 172.21.15.142:0/3944098909 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc624014070 con 0x7fc628100bf0 2024-03-29T04:46:32.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.776+0000 7fc62d4c7700 1 --2- 172.21.15.142:0/3944098909 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fc61406d0a0 0x7fc61406f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:32.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.776+0000 7fc62d4c7700 1 --2- 172.21.15.142:0/3944098909 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fc61406d0a0 0x7fc61406f560 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc618009fd0 tx=0x7fc618005940 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:32.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:32.780+0000 7fc61effd700 1 -- 172.21.15.142:0/3944098909 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fc62405a130 con 0x7fc628100bf0 2024-03-29T04:46:33.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.132+0000 7fc62ff2c700 1 -- 172.21.15.142:0/3944098909 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc62819e8c0 con 0x7fc628100bf0 2024-03-29T04:46:33.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.133+0000 7fc61effd700 1 -- 172.21.15.142:0/3944098909 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7fc624059cc0 con 0x7fc628100bf0 2024-03-29T04:46:33.134 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:46:33.134 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:46:33.134 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:46:33.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:46:33.134 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:46:33.134 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-29T04:46:33.134 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 1 2024-03-29T04:46:33.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:46:33.134 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:46:33.134 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:46:33.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:46:33.135 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:46:33.135 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:46:33.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:46:33.135 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:46:33.135 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-03-29T04:46:33.135 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 1 2024-03-29T04:46:33.135 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:46:33.135 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:46:33.136 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.135+0000 7fc62ff2c700 1 -- 172.21.15.142:0/3944098909 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fc61406d0a0 msgr2=0x7fc61406f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:33.136 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.135+0000 7fc62ff2c700 1 --2- 172.21.15.142:0/3944098909 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fc61406d0a0 0x7fc61406f560 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc618009fd0 tx=0x7fc618005940 comp rx=0 tx=0).stop 2024-03-29T04:46:33.136 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.135+0000 7fc62ff2c700 1 -- 172.21.15.142:0/3944098909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc628100bf0 msgr2=0x7fc628074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:33.136 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.135+0000 7fc62ff2c700 1 --2- 172.21.15.142:0/3944098909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc628100bf0 0x7fc628074b30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc62400e8f0 tx=0x7fc62400ec00 comp rx=0 tx=0).stop 2024-03-29T04:46:33.136 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.135+0000 7fc62ff2c700 1 -- 172.21.15.142:0/3944098909 shutdown_connections 2024-03-29T04:46:33.136 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.135+0000 7fc62ff2c700 1 --2- 172.21.15.142:0/3944098909 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fc61406d0a0 0x7fc61406f560 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:33.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.135+0000 7fc62ff2c700 1 --2- 172.21.15.142:0/3944098909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc628100bf0 0x7fc628074b30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:33.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.135+0000 7fc62ff2c700 1 --2- 172.21.15.142:0/3944098909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc628101df0 0x7fc628073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:33.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.135+0000 7fc62ff2c700 1 -- 172.21.15.142:0/3944098909 >> 172.21.15.142:0/3944098909 conn(0x7fc6280fc190 msgr2=0x7fc628105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:33.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.135+0000 7fc62ff2c700 1 -- 172.21.15.142:0/3944098909 shutdown_connections 2024-03-29T04:46:33.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.135+0000 7fc62ff2c700 1 -- 172.21.15.142:0/3944098909 wait complete. 2024-03-29T04:46:33.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.253+0000 7fbece231700 1 -- 172.21.15.142:0/1172831115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8103f70 msgr2=0x7fbec8106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:33.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.253+0000 7fbece231700 1 --2- 172.21.15.142:0/1172831115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8103f70 0x7fbec8106360 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009910 tx=0x7fbebc009c20 comp rx=0 tx=0).stop 2024-03-29T04:46:33.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.253+0000 7fbece231700 1 -- 172.21.15.142:0/1172831115 shutdown_connections 2024-03-29T04:46:33.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.253+0000 7fbece231700 1 --2- 172.21.15.142:0/1172831115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8103f70 0x7fbec8106360 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:33.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.253+0000 7fbece231700 1 --2- 172.21.15.142:0/1172831115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbec8101640 0x7fbec8103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:33.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.253+0000 7fbece231700 1 -- 172.21.15.142:0/1172831115 >> 172.21.15.142:0/1172831115 conn(0x7fbec80fb050 msgr2=0x7fbec80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:33.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.253+0000 7fbece231700 1 -- 172.21.15.142:0/1172831115 shutdown_connections 2024-03-29T04:46:33.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.253+0000 7fbece231700 1 -- 172.21.15.142:0/1172831115 wait complete. 2024-03-29T04:46:33.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbece231700 1 Processor -- start 2024-03-29T04:46:33.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbece231700 1 -- start start 2024-03-29T04:46:33.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbece231700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbec8101640 0x7fbec8196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:33.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbece231700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8103f70 0x7fbec8196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:33.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbece231700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec8196e80 con 0x7fbec8103f70 2024-03-29T04:46:33.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbece231700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec8196fc0 con 0x7fbec8101640 2024-03-29T04:46:33.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbec6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8103f70 0x7fbec8196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:33.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbec6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8103f70 0x7fbec8196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58612/0 (socket says 172.21.15.142:58612) 2024-03-29T04:46:33.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbec6ffd700 1 -- 172.21.15.142:0/420595918 learned_addr learned my addr 172.21.15.142:0/420595918 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:46:33.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbec77fe700 1 --2- 172.21.15.142:0/420595918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbec8101640 0x7fbec8196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:33.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbec6ffd700 1 -- 172.21.15.142:0/420595918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbec8101640 msgr2=0x7fbec8196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:33.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbec6ffd700 1 --2- 172.21.15.142:0/420595918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbec8101640 0x7fbec8196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:33.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbec6ffd700 1 -- 172.21.15.142:0/420595918 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc0095f0 con 0x7fbec8103f70 2024-03-29T04:46:33.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.254+0000 7fbec77fe700 1 --2- 172.21.15.142:0/420595918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbec8101640 0x7fbec8196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:46:33.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.255+0000 7fbec6ffd700 1 --2- 172.21.15.142:0/420595918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8103f70 0x7fbec8196860 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0092e0 tx=0x7fbebc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:33.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.255+0000 7fbec4ff9700 1 -- 172.21.15.142:0/420595918 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbebc01d070 con 0x7fbec8103f70 2024-03-29T04:46:33.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.255+0000 7fbec4ff9700 1 -- 172.21.15.142:0/420595918 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbebc005dc0 con 0x7fbec8103f70 2024-03-29T04:46:33.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.255+0000 7fbec4ff9700 1 -- 172.21.15.142:0/420595918 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbebc017630 con 0x7fbec8103f70 2024-03-29T04:46:33.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.255+0000 7fbece231700 1 -- 172.21.15.142:0/420595918 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec819ba10 con 0x7fbec8103f70 2024-03-29T04:46:33.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.255+0000 7fbece231700 1 -- 172.21.15.142:0/420595918 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec819bf00 con 0x7fbec8103f70 2024-03-29T04:46:33.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.256+0000 7fbece231700 1 -- 172.21.15.142:0/420595918 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec8190520 con 0x7fbec8103f70 2024-03-29T04:46:33.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.256+0000 7fbec4ff9700 1 -- 172.21.15.142:0/420595918 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 91117+0+0 (secure 0 0 0) 0x7fbebc00faf0 con 0x7fbec8103f70 2024-03-29T04:46:33.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.257+0000 7fbec4ff9700 1 --2- 172.21.15.142:0/420595918 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fbeb006d0a0 0x7fbeb006f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:46:33.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.257+0000 7fbec4ff9700 1 -- 172.21.15.142:0/420595918 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fbebc08dde0 con 0x7fbec8103f70 2024-03-29T04:46:33.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.257+0000 7fbec77fe700 1 --2- 172.21.15.142:0/420595918 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fbeb006d0a0 0x7fbeb006f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:46:33.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.257+0000 7fbec77fe700 1 --2- 172.21.15.142:0/420595918 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fbeb006d0a0 0x7fbeb006f560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8011490 tx=0x7fbeb8009260 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:46:33.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.262+0000 7fbec4ff9700 1 -- 172.21.15.142:0/420595918 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fbebc05b2b0 con 0x7fbec8103f70 2024-03-29T04:46:33.540 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:33 smithi152 ceph-mon[37177]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:33.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:33 smithi152 ceph-mon[37177]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:33.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:33 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3944098909' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:46:33.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.580+0000 7fbece231700 1 -- 172.21.15.142:0/420595918 --> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbec80611d0 con 0x7fbeb006d0a0 2024-03-29T04:46:33.581 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:33 smithi142 ceph-mon[29027]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:33.581 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:33 smithi142 ceph-mon[29027]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:33.581 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:33 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3944098909' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:46:33.582 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.582+0000 7fbec4ff9700 1 -- 172.21.15.142:0/420595918 <== mgr.24395 v2:172.21.15.152:6828/3130987667 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fbec80611d0 con 0x7fbeb006d0a0 2024-03-29T04:46:33.582 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:46:33.582 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:46:33.583 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:46:33.583 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T04:46:33.583 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [], 2024-03-29T04:46:33.583 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "1/2 daemons upgraded", 2024-03-29T04:46:33.583 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mgr daemons", 2024-03-29T04:46:33.583 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:46:33.583 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:46:33.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.584+0000 7fbece231700 1 -- 172.21.15.142:0/420595918 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fbeb006d0a0 msgr2=0x7fbeb006f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:33.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.584+0000 7fbece231700 1 --2- 172.21.15.142:0/420595918 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fbeb006d0a0 0x7fbeb006f560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8011490 tx=0x7fbeb8009260 comp rx=0 tx=0).stop 2024-03-29T04:46:33.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.584+0000 7fbece231700 1 -- 172.21.15.142:0/420595918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8103f70 msgr2=0x7fbec8196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:46:33.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.584+0000 7fbece231700 1 --2- 172.21.15.142:0/420595918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8103f70 0x7fbec8196860 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0092e0 tx=0x7fbebc00f690 comp rx=0 tx=0).stop 2024-03-29T04:46:33.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.584+0000 7fbece231700 1 -- 172.21.15.142:0/420595918 shutdown_connections 2024-03-29T04:46:33.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.584+0000 7fbece231700 1 --2- 172.21.15.142:0/420595918 >> [v2:172.21.15.152:6828/3130987667,v1:172.21.15.152:6829/3130987667] conn(0x7fbeb006d0a0 0x7fbeb006f560 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:33.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.584+0000 7fbece231700 1 --2- 172.21.15.142:0/420595918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbec8101640 0x7fbec8196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:33.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.584+0000 7fbece231700 1 --2- 172.21.15.142:0/420595918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8103f70 0x7fbec8196860 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:46:33.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.584+0000 7fbece231700 1 -- 172.21.15.142:0/420595918 >> 172.21.15.142:0/420595918 conn(0x7fbec80fb050 msgr2=0x7fbec80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:46:33.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.585+0000 7fbece231700 1 -- 172.21.15.142:0/420595918 shutdown_connections 2024-03-29T04:46:33.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:46:33.585+0000 7fbece231700 1 -- 172.21.15.142:0/420595918 wait complete. 2024-03-29T04:46:34.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:34 smithi152 ceph-mon[37177]: pgmap v28: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 359 op/s 2024-03-29T04:46:34.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:34 smithi152 ceph-mon[37177]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:34.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:34 smithi142 ceph-mon[29027]: pgmap v28: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 359 op/s 2024-03-29T04:46:34.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:34 smithi142 ceph-mon[29027]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:35.345 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:35 smithi142 ceph-mon[29027]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:35.345 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:35 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:35.345 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:35 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.swesrg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:46:35.345 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:35 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.swesrg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:46:35.345 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:35 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:46:35.345 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:35 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:46:35.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:35 smithi152 ceph-mon[37177]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:46:35.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:35 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:35.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:35 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.swesrg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:46:35.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:35 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.swesrg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:46:35.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:35 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:46:35.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:35 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:46:36.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:36 smithi152 ceph-mon[37177]: Upgrade: Updating mgr.smithi142.swesrg 2024-03-29T04:46:36.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:36 smithi152 ceph-mon[37177]: Deploying daemon mgr.smithi142.swesrg on smithi142 2024-03-29T04:46:36.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:36 smithi152 ceph-mon[37177]: pgmap v29: 65 pgs: 65 active+clean; 280 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 555 op/s 2024-03-29T04:46:36.621 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:36 smithi142 ceph-mon[29027]: Upgrade: Updating mgr.smithi142.swesrg 2024-03-29T04:46:36.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:36 smithi142 ceph-mon[29027]: Deploying daemon mgr.smithi142.swesrg on smithi142 2024-03-29T04:46:36.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:36 smithi142 ceph-mon[29027]: pgmap v29: 65 pgs: 65 active+clean; 280 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 555 op/s 2024-03-29T04:46:38.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:38 smithi142 ceph-mon[29027]: pgmap v30: 65 pgs: 65 active+clean; 280 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 370 op/s 2024-03-29T04:46:38.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:38 smithi152 ceph-mon[37177]: pgmap v30: 65 pgs: 65 active+clean; 280 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 370 op/s 2024-03-29T04:46:40.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:40 smithi152 ceph-mon[37177]: pgmap v31: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 384 op/s 2024-03-29T04:46:40.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:40 smithi142 ceph-mon[29027]: pgmap v31: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 384 op/s 2024-03-29T04:46:42.510 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:42 smithi152 ceph-mon[37177]: pgmap v32: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 467 op/s 2024-03-29T04:46:42.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:42 smithi142 ceph-mon[29027]: pgmap v32: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 467 op/s 2024-03-29T04:46:43.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:43 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:46:43.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:43 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:43.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:43 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:43.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:43 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:46:43.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:43 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:46:43.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:43 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:43.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:43 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:43.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:43 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:46:43.930 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-29T04:46:43.931 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-29T04:46:44.220 DEBUG:teuthology.parallel:result is None 2024-03-29T04:46:44.221 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-29T04:46:44.257 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-29T04:46:44.257 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-29T04:46:44.294 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-29T04:46:44.294 DEBUG:teuthology.parallel:result is None 2024-03-29T04:46:44.492 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:44 smithi142 ceph-mon[29027]: pgmap v33: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 932 KiB/s rd, 980 KiB/s wr, 319 op/s 2024-03-29T04:46:44.492 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:44 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:44.492 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:44 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:44.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:44 smithi152 ceph-mon[37177]: pgmap v33: 65 pgs: 65 active+clean; 276 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 932 KiB/s rd, 980 KiB/s wr, 319 op/s 2024-03-29T04:46:44.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:44 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:44.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:44 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:46.455 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:46 smithi142 ceph-mon[29027]: pgmap v34: 65 pgs: 65 active+clean; 275 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 427 op/s 2024-03-29T04:46:46.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:46 smithi152 ceph-mon[37177]: pgmap v34: 65 pgs: 65 active+clean; 275 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 427 op/s 2024-03-29T04:46:48.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:48 smithi152 ceph-mon[37177]: pgmap v35: 65 pgs: 65 active+clean; 275 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 789 KiB/s rd, 796 KiB/s wr, 231 op/s 2024-03-29T04:46:48.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:48 smithi142 ceph-mon[29027]: pgmap v35: 65 pgs: 65 active+clean; 275 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 789 KiB/s rd, 796 KiB/s wr, 231 op/s 2024-03-29T04:46:50.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:50 smithi152 ceph-mon[37177]: pgmap v36: 65 pgs: 65 active+clean; 274 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 886 KiB/s rd, 879 KiB/s wr, 260 op/s 2024-03-29T04:46:50.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:50 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:50.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:50 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:50.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:50 smithi152 ceph-mon[37177]: Standby manager daemon smithi142.swesrg restarted 2024-03-29T04:46:50.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:50 smithi152 ceph-mon[37177]: Standby manager daemon smithi142.swesrg started 2024-03-29T04:46:50.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:50 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.swesrg/crt"}]: dispatch 2024-03-29T04:46:50.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:50 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:46:50.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:50 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.swesrg/key"}]: dispatch 2024-03-29T04:46:50.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:50 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:46:50.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:50 smithi142 ceph-mon[29027]: pgmap v36: 65 pgs: 65 active+clean; 274 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 886 KiB/s rd, 879 KiB/s wr, 260 op/s 2024-03-29T04:46:50.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:50 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:50.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:50 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:50.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:50 smithi142 ceph-mon[29027]: Standby manager daemon smithi142.swesrg restarted 2024-03-29T04:46:50.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:50 smithi142 ceph-mon[29027]: Standby manager daemon smithi142.swesrg started 2024-03-29T04:46:50.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:50 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.swesrg/crt"}]: dispatch 2024-03-29T04:46:50.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:50 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:46:50.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:50 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi142.swesrg/key"}]: dispatch 2024-03-29T04:46:50.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:50 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:46:51.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:51 smithi152 ceph-mon[37177]: pgmap v37: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 893 KiB/s rd, 887 KiB/s wr, 281 op/s 2024-03-29T04:46:51.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:51 smithi152 ceph-mon[37177]: mgrmap e26: smithi152.usoolh(active, since 68s), standbys: smithi142.swesrg 2024-03-29T04:46:51.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:51 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:51.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:51 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:51.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:51 smithi142 ceph-mon[29027]: pgmap v37: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 893 KiB/s rd, 887 KiB/s wr, 281 op/s 2024-03-29T04:46:51.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:51 smithi142 ceph-mon[29027]: mgrmap e26: smithi152.usoolh(active, since 68s), standbys: smithi142.swesrg 2024-03-29T04:46:51.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:51 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:51.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:51 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:53.591 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:53 smithi142 ceph-mon[29027]: pgmap v38: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 503 KiB/s rd, 492 KiB/s wr, 173 op/s 2024-03-29T04:46:53.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:53 smithi152 ceph-mon[37177]: pgmap v38: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 503 KiB/s rd, 492 KiB/s wr, 173 op/s 2024-03-29T04:46:55.042 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:55.042 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:55.042 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:46:55.042 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:46:55.042 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:55.042 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mgr fail", "who": "smithi152.usoolh"}]: dispatch 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mgr fail", "who": "smithi152.usoolh"}]: dispatch 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: osdmap e50: 6 total, 6 up, 6 in 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd='[{"prefix": "mgr fail", "who": "smithi152.usoolh"}]': finished 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: mgrmap e27: smithi142.swesrg(active, starting, since 0.0054118s) 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi142.swesrg", "id": "smithi142.swesrg"}]: dispatch 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:46:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:46:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:46:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:46:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:46:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:46:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T04:46:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T04:46:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T04:46:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: Manager daemon smithi142.swesrg is now available 2024-03-29T04:46:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:46:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:46:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:46:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:55 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/trash_purge_schedule"}]: dispatch 2024-03-29T04:46:55.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:55.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:55.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:46:55.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:46:55.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' 2024-03-29T04:46:55.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:46:55.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24395 172.21.15.152:0/3758177669' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mgr fail", "who": "smithi152.usoolh"}]: dispatch 2024-03-29T04:46:55.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "mgr fail", "who": "smithi152.usoolh"}]: dispatch 2024-03-29T04:46:55.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: osdmap e50: 6 total, 6 up, 6 in 2024-03-29T04:46:55.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24395 ' entity='mgr.smithi152.usoolh' cmd='[{"prefix": "mgr fail", "who": "smithi152.usoolh"}]': finished 2024-03-29T04:46:55.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: mgrmap e27: smithi142.swesrg(active, starting, since 0.0054118s) 2024-03-29T04:46:55.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:46:55.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:46:55.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T04:46:55.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:46:55.314 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi142.swesrg", "id": "smithi142.swesrg"}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: Manager daemon smithi142.swesrg is now available 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:46:55.315 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:46:55.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:46:55.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:55 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/trash_purge_schedule"}]: dispatch 2024-03-29T04:46:56.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:56 smithi142 ceph-mon[29027]: [29/Mar/2024:04:46:54] ENGINE Bus STARTING 2024-03-29T04:46:56.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:56 smithi142 ceph-mon[29027]: [29/Mar/2024:04:46:54] ENGINE Serving on https://172.21.15.142:7150 2024-03-29T04:46:56.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:56 smithi142 ceph-mon[29027]: [29/Mar/2024:04:46:54] ENGINE Serving on http://172.21.15.142:8765 2024-03-29T04:46:56.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:56 smithi142 ceph-mon[29027]: [29/Mar/2024:04:46:54] ENGINE Bus STARTED 2024-03-29T04:46:56.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:56 smithi142 ceph-mon[29027]: mgrmap e28: smithi142.swesrg(active, since 1.00892s) 2024-03-29T04:46:56.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:56 smithi142 ceph-mon[29027]: pgmap v3: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:46:56.481 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:56 smithi152 ceph-mon[37177]: [29/Mar/2024:04:46:54] ENGINE Bus STARTING 2024-03-29T04:46:56.482 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:56 smithi152 ceph-mon[37177]: [29/Mar/2024:04:46:54] ENGINE Serving on https://172.21.15.142:7150 2024-03-29T04:46:56.482 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:56 smithi152 ceph-mon[37177]: [29/Mar/2024:04:46:54] ENGINE Serving on http://172.21.15.142:8765 2024-03-29T04:46:56.482 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:56 smithi152 ceph-mon[37177]: [29/Mar/2024:04:46:54] ENGINE Bus STARTED 2024-03-29T04:46:56.482 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:56 smithi152 ceph-mon[37177]: mgrmap e28: smithi142.swesrg(active, since 1.00892s) 2024-03-29T04:46:56.482 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:56 smithi152 ceph-mon[37177]: pgmap v3: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:46:57.482 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:57 smithi142 ceph-mon[29027]: pgmap v4: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:46:57.483 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:57 smithi142 ceph-mon[29027]: mgrmap e29: smithi142.swesrg(active, since 2s) 2024-03-29T04:46:57.497 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:57 smithi152 ceph-mon[37177]: pgmap v4: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:46:57.497 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:57 smithi152 ceph-mon[37177]: mgrmap e29: smithi142.swesrg(active, since 2s) 2024-03-29T04:46:59.275 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:59 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:46:59.275 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:59 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:46:59.275 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:59 smithi142 ceph-mon[29027]: pgmap v5: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:46:59.275 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:46:59 smithi142 ceph-mon[29027]: mgrmap e30: smithi142.swesrg(active, since 4s) 2024-03-29T04:46:59.381 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:59 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:46:59.381 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:59 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:46:59.381 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:59 smithi152 ceph-mon[37177]: pgmap v5: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:46:59.381 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:46:59 smithi152 ceph-mon[37177]: mgrmap e30: smithi142.swesrg(active, since 4s) 2024-03-29T04:47:00.445 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:00 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:00.446 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:00 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:00.446 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:00 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:00.446 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:00 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:00.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:00 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:00.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:00 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:00.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:00 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:00.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:00 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:01.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:01 smithi152 ceph-mon[37177]: pgmap v6: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:47:01.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:01.291 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:01.468 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:01 smithi142 ceph-mon[29027]: pgmap v6: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:47:01.468 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:01.469 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:02.472 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:02 smithi142 ceph-mon[29027]: Standby manager daemon smithi152.usoolh started 2024-03-29T04:47:02.472 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:02 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/2541955400' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/crt"}]: dispatch 2024-03-29T04:47:02.472 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:02 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/2541955400' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:47:02.472 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:02 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/2541955400' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/key"}]: dispatch 2024-03-29T04:47:02.472 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:02 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/2541955400' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:47:02.510 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:02 smithi152 ceph-mon[37177]: Standby manager daemon smithi152.usoolh started 2024-03-29T04:47:02.511 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:02 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/2541955400' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/crt"}]: dispatch 2024-03-29T04:47:02.511 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:02 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/2541955400' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:47:02.511 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:02 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/2541955400' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/key"}]: dispatch 2024-03-29T04:47:02.511 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:02 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/2541955400' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:47:03.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:03 smithi152 ceph-mon[37177]: pgmap v7: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-29T04:47:03.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:03 smithi152 ceph-mon[37177]: mgrmap e31: smithi142.swesrg(active, since 8s), standbys: smithi152.usoolh 2024-03-29T04:47:03.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:03 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi152.usoolh", "id": "smithi152.usoolh"}]: dispatch 2024-03-29T04:47:03.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:03 smithi142 ceph-mon[29027]: pgmap v7: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-29T04:47:03.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:03 smithi142 ceph-mon[29027]: mgrmap e31: smithi142.swesrg(active, since 8s), standbys: smithi152.usoolh 2024-03-29T04:47:03.562 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:03 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi152.usoolh", "id": "smithi152.usoolh"}]: dispatch 2024-03-29T04:47:03.694 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.693+0000 7ff3f3549700 1 -- 172.21.15.142:0/2684021793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3ec101e00 msgr2=0x7ff3ec102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:03.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.693+0000 7ff3f3549700 1 --2- 172.21.15.142:0/2684021793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3ec101e00 0x7ff3ec102260 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009910 tx=0x7ff3dc009c20 comp rx=0 tx=0).stop 2024-03-29T04:47:03.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.693+0000 7ff3f3549700 1 -- 172.21.15.142:0/2684021793 shutdown_connections 2024-03-29T04:47:03.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.693+0000 7ff3f3549700 1 --2- 172.21.15.142:0/2684021793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3ec101e00 0x7ff3ec102260 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:03.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.693+0000 7ff3f3549700 1 --2- 172.21.15.142:0/2684021793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3ec100c00 0x7ff3ec101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:03.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.693+0000 7ff3f3549700 1 -- 172.21.15.142:0/2684021793 >> 172.21.15.142:0/2684021793 conn(0x7ff3ec0fc180 msgr2=0x7ff3ec0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:03.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.693+0000 7ff3f3549700 1 -- 172.21.15.142:0/2684021793 shutdown_connections 2024-03-29T04:47:03.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.693+0000 7ff3f3549700 1 -- 172.21.15.142:0/2684021793 wait complete. 2024-03-29T04:47:03.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f3549700 1 Processor -- start 2024-03-29T04:47:03.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f3549700 1 -- start start 2024-03-29T04:47:03.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f3549700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3ec100c00 0x7ff3ec196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:03.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f3549700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3ec101e00 0x7ff3ec196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:03.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f3549700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec196f80 con 0x7ff3ec101e00 2024-03-29T04:47:03.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f3549700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec1970c0 con 0x7ff3ec100c00 2024-03-29T04:47:03.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f0ae4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3ec101e00 0x7ff3ec196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:03.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f0ae4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3ec101e00 0x7ff3ec196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40706/0 (socket says 172.21.15.142:40706) 2024-03-29T04:47:03.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f0ae4700 1 -- 172.21.15.142:0/262428004 learned_addr learned my addr 172.21.15.142:0/262428004 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:47:03.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f0ae4700 1 -- 172.21.15.142:0/262428004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3ec100c00 msgr2=0x7ff3ec196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:03.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f12e5700 1 --2- 172.21.15.142:0/262428004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3ec100c00 0x7ff3ec196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:03.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f0ae4700 1 --2- 172.21.15.142:0/262428004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3ec100c00 0x7ff3ec196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:03.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.694+0000 7ff3f0ae4700 1 -- 172.21.15.142:0/262428004 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc0095f0 con 0x7ff3ec101e00 2024-03-29T04:47:03.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.695+0000 7ff3f0ae4700 1 --2- 172.21.15.142:0/262428004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3ec101e00 0x7ff3ec196960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc006010 tx=0x7ff3dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:03.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.695+0000 7ff3e27fc700 1 -- 172.21.15.142:0/262428004 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3dc01d070 con 0x7ff3ec101e00 2024-03-29T04:47:03.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.695+0000 7ff3e27fc700 1 -- 172.21.15.142:0/262428004 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff3dc00f460 con 0x7ff3ec101e00 2024-03-29T04:47:03.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.695+0000 7ff3f3549700 1 -- 172.21.15.142:0/262428004 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec19bb10 con 0x7ff3ec101e00 2024-03-29T04:47:03.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.695+0000 7ff3e27fc700 1 -- 172.21.15.142:0/262428004 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3dc021640 con 0x7ff3ec101e00 2024-03-29T04:47:03.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.695+0000 7ff3f3549700 1 -- 172.21.15.142:0/262428004 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec073040 con 0x7ff3ec101e00 2024-03-29T04:47:03.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.696+0000 7ff3f3549700 1 -- 172.21.15.142:0/262428004 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec109800 con 0x7ff3ec101e00 2024-03-29T04:47:03.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.696+0000 7ff3e27fc700 1 -- 172.21.15.142:0/262428004 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 31) v1 ==== 91818+0+0 (secure 0 0 0) 0x7ff3dc00fab0 con 0x7ff3ec101e00 2024-03-29T04:47:03.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.697+0000 7ff3e27fc700 1 --2- 172.21.15.142:0/262428004 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7ff3d8071f30 0x7ff3d80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:03.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.697+0000 7ff3e27fc700 1 -- 172.21.15.142:0/262428004 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff3dc08ed50 con 0x7ff3ec101e00 2024-03-29T04:47:03.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.697+0000 7ff3f12e5700 1 --2- 172.21.15.142:0/262428004 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7ff3d8071f30 0x7ff3d80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:03.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.697+0000 7ff3f12e5700 1 --2- 172.21.15.142:0/262428004 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7ff3d8071f30 0x7ff3d80743f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8006fd0 tx=0x7ff3e8008040 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:03.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.701+0000 7ff3e27fc700 1 -- 172.21.15.142:0/262428004 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7ff3dc05c0d0 con 0x7ff3ec101e00 2024-03-29T04:47:03.920 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.918+0000 7ff3f3549700 1 -- 172.21.15.142:0/262428004 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3ec106750 con 0x7ff3d8071f30 2024-03-29T04:47:03.921 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.920+0000 7ff3e27fc700 1 -- 172.21.15.142:0/262428004 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7ff3ec106750 con 0x7ff3d8071f30 2024-03-29T04:47:03.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.922+0000 7ff3f3549700 1 -- 172.21.15.142:0/262428004 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7ff3d8071f30 msgr2=0x7ff3d80743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:03.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.922+0000 7ff3f3549700 1 --2- 172.21.15.142:0/262428004 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7ff3d8071f30 0x7ff3d80743f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8006fd0 tx=0x7ff3e8008040 comp rx=0 tx=0).stop 2024-03-29T04:47:03.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.922+0000 7ff3f3549700 1 -- 172.21.15.142:0/262428004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3ec101e00 msgr2=0x7ff3ec196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:03.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.922+0000 7ff3f3549700 1 --2- 172.21.15.142:0/262428004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3ec101e00 0x7ff3ec196960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc006010 tx=0x7ff3dc00be20 comp rx=0 tx=0).stop 2024-03-29T04:47:03.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.922+0000 7ff3f3549700 1 -- 172.21.15.142:0/262428004 shutdown_connections 2024-03-29T04:47:03.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.922+0000 7ff3f3549700 1 --2- 172.21.15.142:0/262428004 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7ff3d8071f30 0x7ff3d80743f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:03.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.922+0000 7ff3f3549700 1 --2- 172.21.15.142:0/262428004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3ec100c00 0x7ff3ec196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:03.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.922+0000 7ff3f3549700 1 --2- 172.21.15.142:0/262428004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3ec101e00 0x7ff3ec196960 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:03.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.922+0000 7ff3f3549700 1 -- 172.21.15.142:0/262428004 >> 172.21.15.142:0/262428004 conn(0x7ff3ec0fc180 msgr2=0x7ff3ec105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:03.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.922+0000 7ff3f3549700 1 -- 172.21.15.142:0/262428004 shutdown_connections 2024-03-29T04:47:03.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:03.922+0000 7ff3f3549700 1 -- 172.21.15.142:0/262428004 wait complete. 2024-03-29T04:47:03.933 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:47:04.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.027+0000 7f5b318ce700 1 -- 172.21.15.142:0/199980038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b2c1016a0 msgr2=0x7f5b2c103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.027+0000 7f5b318ce700 1 --2- 172.21.15.142:0/199980038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b2c1016a0 0x7f5b2c103a90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009910 tx=0x7f5b1c009c20 comp rx=0 tx=0).stop 2024-03-29T04:47:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.028+0000 7f5b318ce700 1 -- 172.21.15.142:0/199980038 shutdown_connections 2024-03-29T04:47:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.028+0000 7f5b318ce700 1 --2- 172.21.15.142:0/199980038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b2c103fd0 0x7f5b2c1063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.028+0000 7f5b318ce700 1 --2- 172.21.15.142:0/199980038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b2c1016a0 0x7f5b2c103a90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.028+0000 7f5b318ce700 1 -- 172.21.15.142:0/199980038 >> 172.21.15.142:0/199980038 conn(0x7f5b2c0fb030 msgr2=0x7f5b2c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.028+0000 7f5b318ce700 1 -- 172.21.15.142:0/199980038 shutdown_connections 2024-03-29T04:47:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.028+0000 7f5b318ce700 1 -- 172.21.15.142:0/199980038 wait complete. 2024-03-29T04:47:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.028+0000 7f5b318ce700 1 Processor -- start 2024-03-29T04:47:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.028+0000 7f5b318ce700 1 -- start start 2024-03-29T04:47:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b318ce700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b2c1016a0 0x7f5b2c196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b318ce700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b2c103fd0 0x7f5b2c1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:04.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b318ce700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c196ee0 con 0x7f5b2c103fd0 2024-03-29T04:47:04.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b318ce700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c197020 con 0x7f5b2c1016a0 2024-03-29T04:47:04.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b2a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b2c103fd0 0x7f5b2c1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:04.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b2a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b2c103fd0 0x7f5b2c1968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40718/0 (socket says 172.21.15.142:40718) 2024-03-29T04:47:04.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b2a7fc700 1 -- 172.21.15.142:0/3295000865 learned_addr learned my addr 172.21.15.142:0/3295000865 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:47:04.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b2affd700 1 --2- 172.21.15.142:0/3295000865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b2c1016a0 0x7f5b2c196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:04.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b2a7fc700 1 -- 172.21.15.142:0/3295000865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b2c1016a0 msgr2=0x7f5b2c196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:04.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b2a7fc700 1 --2- 172.21.15.142:0/3295000865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b2c1016a0 0x7f5b2c196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b2a7fc700 1 -- 172.21.15.142:0/3295000865 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c0095f0 con 0x7f5b2c103fd0 2024-03-29T04:47:04.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b2a7fc700 1 --2- 172.21.15.142:0/3295000865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b2c103fd0 0x7f5b2c1968c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000b4b0 tx=0x7f5b2000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:04.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b308cc700 1 -- 172.21.15.142:0/3295000865 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b20011800 con 0x7f5b2c103fd0 2024-03-29T04:47:04.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b318ce700 1 -- 172.21.15.142:0/3295000865 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c19bad0 con 0x7f5b2c103fd0 2024-03-29T04:47:04.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b308cc700 1 -- 172.21.15.142:0/3295000865 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5b20011e10 con 0x7f5b2c103fd0 2024-03-29T04:47:04.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.029+0000 7f5b308cc700 1 -- 172.21.15.142:0/3295000865 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b2000f2e0 con 0x7f5b2c103fd0 2024-03-29T04:47:04.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.030+0000 7f5b318ce700 1 -- 172.21.15.142:0/3295000865 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c19c0a0 con 0x7f5b2c103fd0 2024-03-29T04:47:04.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.031+0000 7f5b318ce700 1 -- 172.21.15.142:0/3295000865 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2c1905a0 con 0x7f5b2c103fd0 2024-03-29T04:47:04.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.031+0000 7f5b308cc700 1 -- 172.21.15.142:0/3295000865 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 31) v1 ==== 91818+0+0 (secure 0 0 0) 0x7f5b20010380 con 0x7f5b2c103fd0 2024-03-29T04:47:04.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.031+0000 7f5b308cc700 1 --2- 172.21.15.142:0/3295000865 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f5b1406db20 0x7f5b1406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:04.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.031+0000 7f5b308cc700 1 -- 172.21.15.142:0/3295000865 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5b2008d3b0 con 0x7f5b2c103fd0 2024-03-29T04:47:04.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.031+0000 7f5b2affd700 1 --2- 172.21.15.142:0/3295000865 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f5b1406db20 0x7f5b1406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:04.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.032+0000 7f5b2affd700 1 --2- 172.21.15.142:0/3295000865 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f5b1406db20 0x7f5b1406ffe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0059d0 tx=0x7f5b1c005920 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:04.037 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.036+0000 7f5b308cc700 1 -- 172.21.15.142:0/3295000865 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f5b2005a840 con 0x7f5b2c103fd0 2024-03-29T04:47:04.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.251+0000 7f5b318ce700 1 -- 172.21.15.142:0/3295000865 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b2c0611d0 con 0x7f5b1406db20 2024-03-29T04:47:04.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.252+0000 7f5b308cc700 1 -- 172.21.15.142:0/3295000865 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f5b2c0611d0 con 0x7f5b1406db20 2024-03-29T04:47:04.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.254+0000 7f5b318ce700 1 -- 172.21.15.142:0/3295000865 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f5b1406db20 msgr2=0x7f5b1406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:04.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.254+0000 7f5b318ce700 1 --2- 172.21.15.142:0/3295000865 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f5b1406db20 0x7f5b1406ffe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0059d0 tx=0x7f5b1c005920 comp rx=0 tx=0).stop 2024-03-29T04:47:04.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.254+0000 7f5b318ce700 1 -- 172.21.15.142:0/3295000865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b2c103fd0 msgr2=0x7f5b2c1968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:04.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.254+0000 7f5b318ce700 1 --2- 172.21.15.142:0/3295000865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b2c103fd0 0x7f5b2c1968c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000b4b0 tx=0x7f5b2000b870 comp rx=0 tx=0).stop 2024-03-29T04:47:04.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.254+0000 7f5b318ce700 1 -- 172.21.15.142:0/3295000865 shutdown_connections 2024-03-29T04:47:04.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.254+0000 7f5b318ce700 1 --2- 172.21.15.142:0/3295000865 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f5b1406db20 0x7f5b1406ffe0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.254+0000 7f5b318ce700 1 --2- 172.21.15.142:0/3295000865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b2c1016a0 0x7f5b2c196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.254+0000 7f5b318ce700 1 --2- 172.21.15.142:0/3295000865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b2c103fd0 0x7f5b2c1968c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.254+0000 7f5b318ce700 1 -- 172.21.15.142:0/3295000865 >> 172.21.15.142:0/3295000865 conn(0x7f5b2c0fb030 msgr2=0x7f5b2c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:04.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.254+0000 7f5b318ce700 1 -- 172.21.15.142:0/3295000865 shutdown_connections 2024-03-29T04:47:04.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.254+0000 7f5b318ce700 1 -- 172.21.15.142:0/3295000865 wait complete. 2024-03-29T04:47:04.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 -- 172.21.15.142:0/2893569565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f167c103f70 msgr2=0x7f167c106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:04.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 --2- 172.21.15.142:0/2893569565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f167c103f70 0x7f167c106360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f166c009910 tx=0x7f166c009c20 comp rx=0 tx=0).stop 2024-03-29T04:47:04.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 -- 172.21.15.142:0/2893569565 shutdown_connections 2024-03-29T04:47:04.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 --2- 172.21.15.142:0/2893569565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f167c103f70 0x7f167c106360 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 --2- 172.21.15.142:0/2893569565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f167c101640 0x7f167c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 -- 172.21.15.142:0/2893569565 >> 172.21.15.142:0/2893569565 conn(0x7f167c0fb030 msgr2=0x7f167c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:04.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 -- 172.21.15.142:0/2893569565 shutdown_connections 2024-03-29T04:47:04.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 -- 172.21.15.142:0/2893569565 wait complete. 2024-03-29T04:47:04.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 Processor -- start 2024-03-29T04:47:04.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 -- start start 2024-03-29T04:47:04.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f167c101640 0x7f167c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:04.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f167c103f70 0x7f167c196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c196e80 con 0x7f167c103f70 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.362+0000 7f1683e18700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c196fc0 con 0x7f167c101640 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.363+0000 7f16813b3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f167c103f70 0x7f167c196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.363+0000 7f16813b3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f167c103f70 0x7f167c196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40726/0 (socket says 172.21.15.142:40726) 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.363+0000 7f16813b3700 1 -- 172.21.15.142:0/4031025738 learned_addr learned my addr 172.21.15.142:0/4031025738 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.363+0000 7f16813b3700 1 -- 172.21.15.142:0/4031025738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f167c101640 msgr2=0x7f167c196320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.363+0000 7f16813b3700 1 --2- 172.21.15.142:0/4031025738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f167c101640 0x7f167c196320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.363+0000 7f16813b3700 1 -- 172.21.15.142:0/4031025738 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c0095f0 con 0x7f167c103f70 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.363+0000 7f16813b3700 1 --2- 172.21.15.142:0/4031025738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f167c103f70 0x7f167c196860 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f166c006010 tx=0x7f166c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.363+0000 7f1672ffd700 1 -- 172.21.15.142:0/4031025738 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f166c01d070 con 0x7f167c103f70 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.363+0000 7f1683e18700 1 -- 172.21.15.142:0/4031025738 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167c19ba10 con 0x7f167c103f70 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.363+0000 7f1672ffd700 1 -- 172.21.15.142:0/4031025738 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f166c00f460 con 0x7f167c103f70 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.363+0000 7f1672ffd700 1 -- 172.21.15.142:0/4031025738 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f166c017630 con 0x7f167c103f70 2024-03-29T04:47:04.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.363+0000 7f1683e18700 1 -- 172.21.15.142:0/4031025738 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167c19bf00 con 0x7f167c103f70 2024-03-29T04:47:04.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.364+0000 7f1683e18700 1 -- 172.21.15.142:0/4031025738 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167c190540 con 0x7f167c103f70 2024-03-29T04:47:04.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.365+0000 7f1672ffd700 1 -- 172.21.15.142:0/4031025738 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 31) v1 ==== 91818+0+0 (secure 0 0 0) 0x7f166c003a60 con 0x7f167c103f70 2024-03-29T04:47:04.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.365+0000 7f1672ffd700 1 --2- 172.21.15.142:0/4031025738 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f166806db20 0x7f166806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:04.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.365+0000 7f1672ffd700 1 -- 172.21.15.142:0/4031025738 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f166c08ecb0 con 0x7f167c103f70 2024-03-29T04:47:04.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.365+0000 7f1681bb4700 1 --2- 172.21.15.142:0/4031025738 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f166806db20 0x7f166806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:04.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.366+0000 7f1681bb4700 1 --2- 172.21.15.142:0/4031025738 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f166806db20 0x7f166806ffe0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1678009510 tx=0x7f1678009300 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:04.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.370+0000 7f1672ffd700 1 -- 172.21.15.142:0/4031025738 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f166c05c0b0 con 0x7f167c103f70 2024-03-29T04:47:04.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.591+0000 7f1683e18700 1 -- 172.21.15.142:0/4031025738 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f167c0611d0 con 0x7f166806db20 2024-03-29T04:47:04.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.602+0000 7f1672ffd700 1 -- 172.21.15.142:0/4031025738 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f167c0611d0 con 0x7f166806db20 2024-03-29T04:47:04.603 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:47:04.603 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (8m) 4s ago 10m 25.1M - 0.25.0 c8568f914cd2 d4a3e12f4047 2024-03-29T04:47:04.603 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (10m) 4s ago 10m 7944k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:47:04.603 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (9m) 6s ago 9m 7810k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (10m) 4s ago 10m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (9m) 6s ago 9m 6660k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (8m) 4s ago 10m 86.6M - 9.4.7 954c08fa6188 aba58e552b7c 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (4m) 4s ago 4m 137M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (4m) 4s ago 4m 63.4M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (4m) 6s ago 4m 65.0M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (4m) 6s ago 4m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (25s) 4s ago 11m 472M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (93s) 6s ago 9m 69.9M - 19.0.0-1959-g8da23624 54112f57da44 2aeb29a22d55 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (11m) 4s ago 11m 54.2M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (9m) 6s ago 9m 39.9M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (10m) 4s ago 10m 16.2M - 1.5.0 0da6a335fe13 91b95c929db2 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (9m) 6s ago 9m 15.1M - 1.5.0 0da6a335fe13 97e90e5b337b 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (8m) 4s ago 8m 217M 1216M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:47:04.604 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (8m) 4s ago 8m 239M 1216M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:47:04.605 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (7m) 4s ago 7m 197M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:47:04.605 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (7m) 6s ago 7m 273M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:47:04.605 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (6m) 6s ago 6m 233M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:47:04.605 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (6m) 6s ago 6m 222M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:47:04.605 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (35s) 4s ago 9m 42.5M - 2.43.0 a07b618ecd1d 77b443116644 2024-03-29T04:47:04.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.604+0000 7f1683e18700 1 -- 172.21.15.142:0/4031025738 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f166806db20 msgr2=0x7f166806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:04.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.604+0000 7f1683e18700 1 --2- 172.21.15.142:0/4031025738 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f166806db20 0x7f166806ffe0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1678009510 tx=0x7f1678009300 comp rx=0 tx=0).stop 2024-03-29T04:47:04.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.604+0000 7f1683e18700 1 -- 172.21.15.142:0/4031025738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f167c103f70 msgr2=0x7f167c196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:04.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.604+0000 7f1683e18700 1 --2- 172.21.15.142:0/4031025738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f167c103f70 0x7f167c196860 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f166c006010 tx=0x7f166c00be20 comp rx=0 tx=0).stop 2024-03-29T04:47:04.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.605+0000 7f1683e18700 1 -- 172.21.15.142:0/4031025738 shutdown_connections 2024-03-29T04:47:04.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.605+0000 7f1683e18700 1 --2- 172.21.15.142:0/4031025738 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f166806db20 0x7f166806ffe0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.605+0000 7f1683e18700 1 --2- 172.21.15.142:0/4031025738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f167c101640 0x7f167c196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.605+0000 7f1683e18700 1 --2- 172.21.15.142:0/4031025738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f167c103f70 0x7f167c196860 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.605+0000 7f1683e18700 1 -- 172.21.15.142:0/4031025738 >> 172.21.15.142:0/4031025738 conn(0x7f167c0fb030 msgr2=0x7f167c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:04.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.605+0000 7f1683e18700 1 -- 172.21.15.142:0/4031025738 shutdown_connections 2024-03-29T04:47:04.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.605+0000 7f1683e18700 1 -- 172.21.15.142:0/4031025738 wait complete. 2024-03-29T04:47:04.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.734+0000 7f214ec27700 1 -- 172.21.15.142:0/1201152458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148100bc0 msgr2=0x7f2148100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:04.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.734+0000 7f214ec27700 1 --2- 172.21.15.142:0/1201152458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148100bc0 0x7f2148100fe0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f214000b0a0 tx=0x7f214000b3b0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.734+0000 7f214ec27700 1 -- 172.21.15.142:0/1201152458 shutdown_connections 2024-03-29T04:47:04.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.734+0000 7f214ec27700 1 --2- 172.21.15.142:0/1201152458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2148101dc0 0x7f2148102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.734+0000 7f214ec27700 1 --2- 172.21.15.142:0/1201152458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148100bc0 0x7f2148100fe0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.734+0000 7f214ec27700 1 -- 172.21.15.142:0/1201152458 >> 172.21.15.142:0/1201152458 conn(0x7f21480fc140 msgr2=0x7f21480fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:04.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.734+0000 7f214ec27700 1 -- 172.21.15.142:0/1201152458 shutdown_connections 2024-03-29T04:47:04.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.734+0000 7f214ec27700 1 -- 172.21.15.142:0/1201152458 wait complete. 2024-03-29T04:47:04.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.734+0000 7f214ec27700 1 Processor -- start 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f214ec27700 1 -- start start 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f214ec27700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2148100bc0 0x7f2148196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f214ec27700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148101dc0 0x7f2148196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f214ec27700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2148196f60 con 0x7f2148101dc0 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f214ec27700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21481970a0 con 0x7f2148100bc0 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f2147fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148101dc0 0x7f2148196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f2147fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148101dc0 0x7f2148196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40736/0 (socket says 172.21.15.142:40736) 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f2147fff700 1 -- 172.21.15.142:0/3695363174 learned_addr learned my addr 172.21.15.142:0/3695363174 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f214c9c3700 1 --2- 172.21.15.142:0/3695363174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2148100bc0 0x7f2148196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f2147fff700 1 -- 172.21.15.142:0/3695363174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2148100bc0 msgr2=0x7f2148196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f2147fff700 1 --2- 172.21.15.142:0/3695363174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2148100bc0 0x7f2148196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f2147fff700 1 -- 172.21.15.142:0/3695363174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2140009d00 con 0x7f2148101dc0 2024-03-29T04:47:04.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.735+0000 7f2147fff700 1 --2- 172.21.15.142:0/3695363174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148101dc0 0x7f2148196940 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f213c00e920 tx=0x7f213c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:04.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.736+0000 7f2145ffb700 1 -- 172.21.15.142:0/3695363174 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f213c00c9d0 con 0x7f2148101dc0 2024-03-29T04:47:04.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.736+0000 7f214ec27700 1 -- 172.21.15.142:0/3695363174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214819bb50 con 0x7f2148101dc0 2024-03-29T04:47:04.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.736+0000 7f2145ffb700 1 -- 172.21.15.142:0/3695363174 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f213c010430 con 0x7f2148101dc0 2024-03-29T04:47:04.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.736+0000 7f2145ffb700 1 -- 172.21.15.142:0/3695363174 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f213c018650 con 0x7f2148101dc0 2024-03-29T04:47:04.740 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.736+0000 7f214ec27700 1 -- 172.21.15.142:0/3695363174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214819c120 con 0x7f2148101dc0 2024-03-29T04:47:04.740 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.737+0000 7f214ec27700 1 -- 172.21.15.142:0/3695363174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2148109720 con 0x7f2148101dc0 2024-03-29T04:47:04.740 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.737+0000 7f2145ffb700 1 -- 172.21.15.142:0/3695363174 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 31) v1 ==== 91818+0+0 (secure 0 0 0) 0x7f213c0105a0 con 0x7f2148101dc0 2024-03-29T04:47:04.740 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.738+0000 7f2145ffb700 1 --2- 172.21.15.142:0/3695363174 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f213006db10 0x7f213006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:04.740 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.738+0000 7f2145ffb700 1 -- 172.21.15.142:0/3695363174 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f213c014070 con 0x7f2148101dc0 2024-03-29T04:47:04.740 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.738+0000 7f214c9c3700 1 --2- 172.21.15.142:0/3695363174 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f213006db10 0x7f213006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:04.740 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.739+0000 7f214c9c3700 1 --2- 172.21.15.142:0/3695363174 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f213006db10 0x7f213006ffd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2140009fd0 tx=0x7f2140006000 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:04.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:04.743+0000 7f2145ffb700 1 -- 172.21.15.142:0/3695363174 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f213c052fc0 con 0x7f2148101dc0 2024-03-29T04:47:04.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:04 smithi152 ceph-mon[37177]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:04.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:04 smithi152 ceph-mon[37177]: pgmap v8: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-03-29T04:47:04.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:04 smithi152 ceph-mon[37177]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:04.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:04 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:04.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:04 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:04.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:04 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:47:04.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:04 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:47:04.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:04 smithi142 ceph-mon[29027]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:04.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:04 smithi142 ceph-mon[29027]: pgmap v8: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-03-29T04:47:04.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:04 smithi142 ceph-mon[29027]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:04.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:04 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:04.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:04 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:04.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:04 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:47:04.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:04 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:47:05.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.056+0000 7f214ec27700 1 -- 172.21.15.142:0/3695363174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f214819c730 con 0x7f2148101dc0 2024-03-29T04:47:05.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.056+0000 7f2145ffb700 1 -- 172.21.15.142:0/3695363174 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f213c05ae90 con 0x7f2148101dc0 2024-03-29T04:47:05.058 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:47:05.058 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:47:05.058 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:47:05.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:47:05.058 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:47:05.058 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:47:05.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:47:05.058 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:47:05.058 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:47:05.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:47:05.059 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:47:05.059 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:47:05.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:47:05.059 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:47:05.059 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-29T04:47:05.059 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:47:05.059 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:47:05.059 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:47:05.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.059+0000 7f214ec27700 1 -- 172.21.15.142:0/3695363174 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f213006db10 msgr2=0x7f213006ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:05.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.059+0000 7f214ec27700 1 --2- 172.21.15.142:0/3695363174 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f213006db10 0x7f213006ffd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2140009fd0 tx=0x7f2140006000 comp rx=0 tx=0).stop 2024-03-29T04:47:05.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.059+0000 7f214ec27700 1 -- 172.21.15.142:0/3695363174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148101dc0 msgr2=0x7f2148196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:05.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.059+0000 7f214ec27700 1 --2- 172.21.15.142:0/3695363174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148101dc0 0x7f2148196940 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f213c00e920 tx=0x7f213c00ec30 comp rx=0 tx=0).stop 2024-03-29T04:47:05.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.059+0000 7f214ec27700 1 -- 172.21.15.142:0/3695363174 shutdown_connections 2024-03-29T04:47:05.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.059+0000 7f214ec27700 1 --2- 172.21.15.142:0/3695363174 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f213006db10 0x7f213006ffd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:05.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.059+0000 7f214ec27700 1 --2- 172.21.15.142:0/3695363174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2148100bc0 0x7f2148196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:05.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.059+0000 7f214ec27700 1 --2- 172.21.15.142:0/3695363174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2148101dc0 0x7f2148196940 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:05.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.059+0000 7f214ec27700 1 -- 172.21.15.142:0/3695363174 >> 172.21.15.142:0/3695363174 conn(0x7f21480fc140 msgr2=0x7f2148104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:05.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.060+0000 7f214ec27700 1 -- 172.21.15.142:0/3695363174 shutdown_connections 2024-03-29T04:47:05.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.060+0000 7f214ec27700 1 -- 172.21.15.142:0/3695363174 wait complete. 2024-03-29T04:47:05.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.170+0000 7f163ad35700 1 -- 172.21.15.142:0/3456825746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16340fee80 msgr2=0x7f16340ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:05.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.170+0000 7f163ad35700 1 --2- 172.21.15.142:0/3456825746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16340fee80 0x7f16340ff2a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1628009910 tx=0x7f1628009c20 comp rx=0 tx=0).stop 2024-03-29T04:47:05.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.170+0000 7f163ad35700 1 -- 172.21.15.142:0/3456825746 shutdown_connections 2024-03-29T04:47:05.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.170+0000 7f163ad35700 1 --2- 172.21.15.142:0/3456825746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1634101e60 0x7f16341022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:05.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.170+0000 7f163ad35700 1 --2- 172.21.15.142:0/3456825746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16340fee80 0x7f16340ff2a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:05.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.170+0000 7f163ad35700 1 -- 172.21.15.142:0/3456825746 >> 172.21.15.142:0/3456825746 conn(0x7f16340fa7c0 msgr2=0x7f16340fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:05.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.170+0000 7f163ad35700 1 -- 172.21.15.142:0/3456825746 shutdown_connections 2024-03-29T04:47:05.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.170+0000 7f163ad35700 1 -- 172.21.15.142:0/3456825746 wait complete. 2024-03-29T04:47:05.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.171+0000 7f163ad35700 1 Processor -- start 2024-03-29T04:47:05.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.171+0000 7f163ad35700 1 -- start start 2024-03-29T04:47:05.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.171+0000 7f163ad35700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16340fee80 0x7f16341963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:05.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.171+0000 7f163ad35700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1634101e60 0x7f16341968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:05.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.171+0000 7f163ad35700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1634196f00 con 0x7f1634101e60 2024-03-29T04:47:05.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.171+0000 7f163ad35700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1634197040 con 0x7f16340fee80 2024-03-29T04:47:05.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.171+0000 7f1633fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1634101e60 0x7f16341968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:05.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.171+0000 7f1633fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1634101e60 0x7f16341968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40758/0 (socket says 172.21.15.142:40758) 2024-03-29T04:47:05.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.171+0000 7f1633fff700 1 -- 172.21.15.142:0/3245174702 learned_addr learned my addr 172.21.15.142:0/3245174702 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:47:05.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.171+0000 7f1638ad1700 1 --2- 172.21.15.142:0/3245174702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16340fee80 0x7f16341963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:05.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.172+0000 7f1633fff700 1 -- 172.21.15.142:0/3245174702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16340fee80 msgr2=0x7f16341963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:05.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.172+0000 7f1633fff700 1 --2- 172.21.15.142:0/3245174702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16340fee80 0x7f16341963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:05.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.172+0000 7f1633fff700 1 -- 172.21.15.142:0/3245174702 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16280095f0 con 0x7f1634101e60 2024-03-29T04:47:05.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.172+0000 7f1633fff700 1 --2- 172.21.15.142:0/3245174702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1634101e60 0x7f16341968e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f162000d900 tx=0x7f162000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:05.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.172+0000 7f1631ffb700 1 -- 172.21.15.142:0/3245174702 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16200041f0 con 0x7f1634101e60 2024-03-29T04:47:05.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.172+0000 7f163ad35700 1 -- 172.21.15.142:0/3245174702 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163419baf0 con 0x7f1634101e60 2024-03-29T04:47:05.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.172+0000 7f1631ffb700 1 -- 172.21.15.142:0/3245174702 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1620004ce0 con 0x7f1634101e60 2024-03-29T04:47:05.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.172+0000 7f1631ffb700 1 -- 172.21.15.142:0/3245174702 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f162000b750 con 0x7f1634101e60 2024-03-29T04:47:05.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.172+0000 7f163ad35700 1 -- 172.21.15.142:0/3245174702 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163419c040 con 0x7f1634101e60 2024-03-29T04:47:05.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.173+0000 7f163ad35700 1 -- 172.21.15.142:0/3245174702 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1634109720 con 0x7f1634101e60 2024-03-29T04:47:05.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.174+0000 7f1631ffb700 1 -- 172.21.15.142:0/3245174702 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 31) v1 ==== 91818+0+0 (secure 0 0 0) 0x7f1620004830 con 0x7f1634101e60 2024-03-29T04:47:05.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.174+0000 7f1631ffb700 1 --2- 172.21.15.142:0/3245174702 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f162406db20 0x7f162406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:05.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.174+0000 7f1631ffb700 1 -- 172.21.15.142:0/3245174702 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f162008d7e0 con 0x7f1634101e60 2024-03-29T04:47:05.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.174+0000 7f1638ad1700 1 --2- 172.21.15.142:0/3245174702 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f162406db20 0x7f162406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:05.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.175+0000 7f1638ad1700 1 --2- 172.21.15.142:0/3245174702 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f162406db20 0x7f162406ffe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1628006010 tx=0x7f1628005ec0 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:05.180 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.179+0000 7f1631ffb700 1 -- 172.21.15.142:0/3245174702 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f162005ab60 con 0x7f1634101e60 2024-03-29T04:47:05.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.409+0000 7f163ad35700 1 -- 172.21.15.142:0/3245174702 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f16341067b0 con 0x7f162406db20 2024-03-29T04:47:05.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.410+0000 7f1631ffb700 1 -- 172.21.15.142:0/3245174702 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f16341067b0 con 0x7f162406db20 2024-03-29T04:47:05.412 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:47:05.412 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:47:05.412 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:47:05.412 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T04:47:05.412 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:47:05.412 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr" 2024-03-29T04:47:05.412 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:47:05.412 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "2/2 daemons upgraded", 2024-03-29T04:47:05.412 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "", 2024-03-29T04:47:05.412 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:47:05.412 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:47:05.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.413+0000 7f163ad35700 1 -- 172.21.15.142:0/3245174702 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f162406db20 msgr2=0x7f162406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:05.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.413+0000 7f163ad35700 1 --2- 172.21.15.142:0/3245174702 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f162406db20 0x7f162406ffe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1628006010 tx=0x7f1628005ec0 comp rx=0 tx=0).stop 2024-03-29T04:47:05.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.413+0000 7f163ad35700 1 -- 172.21.15.142:0/3245174702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1634101e60 msgr2=0x7f16341968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:05.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.413+0000 7f163ad35700 1 --2- 172.21.15.142:0/3245174702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1634101e60 0x7f16341968e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f162000d900 tx=0x7f162000dc10 comp rx=0 tx=0).stop 2024-03-29T04:47:05.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.413+0000 7f163ad35700 1 -- 172.21.15.142:0/3245174702 shutdown_connections 2024-03-29T04:47:05.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.413+0000 7f163ad35700 1 --2- 172.21.15.142:0/3245174702 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f162406db20 0x7f162406ffe0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:05.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.413+0000 7f163ad35700 1 --2- 172.21.15.142:0/3245174702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16340fee80 0x7f16341963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:05.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.413+0000 7f163ad35700 1 --2- 172.21.15.142:0/3245174702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1634101e60 0x7f16341968e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:05.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.413+0000 7f163ad35700 1 -- 172.21.15.142:0/3245174702 >> 172.21.15.142:0/3245174702 conn(0x7f16340fa7c0 msgr2=0x7f1634105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:05.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.413+0000 7f163ad35700 1 -- 172.21.15.142:0/3245174702 shutdown_connections 2024-03-29T04:47:05.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:05.413+0000 7f163ad35700 1 -- 172.21.15.142:0/3245174702 wait complete. 2024-03-29T04:47:05.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-29T04:47:05.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-29T04:47:05.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:47:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:47:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:47:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:47:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3695363174' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:47:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:47:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:05 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:05.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-29T04:47:05.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-29T04:47:05.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:05.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:47:05.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:47:05.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:47:05.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:47:05.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3695363174' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:05.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:47:05.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:47:05.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:05.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:05.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:05.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:05.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:05 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:06.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:06 smithi152 ceph-mon[37177]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:06.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:06 smithi152 ceph-mon[37177]: Reconfiguring prometheus.smithi142 (dependencies changed)... 2024-03-29T04:47:06.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:06 smithi152 ceph-mon[37177]: Reconfiguring daemon prometheus.smithi142 on smithi142 2024-03-29T04:47:06.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:06 smithi152 ceph-mon[37177]: pgmap v9: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-29T04:47:06.849 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:06 smithi142 ceph-mon[29027]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:06.849 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:06 smithi142 ceph-mon[29027]: Reconfiguring prometheus.smithi142 (dependencies changed)... 2024-03-29T04:47:06.849 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:06 smithi142 ceph-mon[29027]: Reconfiguring daemon prometheus.smithi142 on smithi142 2024-03-29T04:47:06.850 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:06 smithi142 ceph-mon[29027]: pgmap v9: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-29T04:47:09.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:09 smithi142 ceph-mon[29027]: pgmap v10: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-29T04:47:09.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:09 smithi152 ceph-mon[37177]: pgmap v10: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-29T04:47:10.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:10 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:47:10.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:10 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:47:11.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:11 smithi152 ceph-mon[37177]: pgmap v11: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-29T04:47:11.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:11 smithi142 ceph-mon[29027]: pgmap v11: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-29T04:47:12.730 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:12 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:12.730 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:12 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:12.730 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:12 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:47:12.730 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:12 smithi152 ceph-mon[37177]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:47:12.730 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:12 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:47:12.730 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:12 smithi152 ceph-mon[37177]: pgmap v12: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-29T04:47:12.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:12 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:12.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:12 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:12.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:12 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:47:12.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:12 smithi142 ceph-mon[29027]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:47:12.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:12 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:47:12.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:12 smithi142 ceph-mon[29027]: pgmap v12: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-29T04:47:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:13 smithi142 ceph-mon[29027]: Upgrade: Updating mgr.smithi152.usoolh 2024-03-29T04:47:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:13 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi152.usoolh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:47:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:13 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:47:13.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:13 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:47:13.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:13 smithi142 ceph-mon[29027]: Deploying daemon mgr.smithi152.usoolh on smithi152 2024-03-29T04:47:13.668 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:13 smithi152 ceph-mon[37177]: Upgrade: Updating mgr.smithi152.usoolh 2024-03-29T04:47:13.668 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:13 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi152.usoolh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:47:13.668 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:13 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:47:13.668 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:13 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:47:13.668 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:13 smithi152 ceph-mon[37177]: Deploying daemon mgr.smithi152.usoolh on smithi152 2024-03-29T04:47:14.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:14 smithi152 ceph-mon[37177]: pgmap v13: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:14.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:14 smithi142 ceph-mon[29027]: pgmap v13: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:17.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:17 smithi152 ceph-mon[37177]: pgmap v14: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:17.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:17 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:17.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:17 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:17.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:17 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:47:17.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:17 smithi142 ceph-mon[29027]: pgmap v14: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:17.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:17 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:17.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:17 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:17.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:17 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:47:19.290 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:19 smithi142 ceph-mon[29027]: pgmap v15: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:19.515 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:19 smithi152 ceph-mon[37177]: pgmap v15: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:21.137 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:21 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:21.137 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:21 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:21.137 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:21 smithi152 ceph-mon[37177]: pgmap v16: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:21.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:21 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:21.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:21 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:21.372 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:21 smithi142 ceph-mon[29027]: pgmap v16: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:22.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:22 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:22.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:22 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:22.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:22 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:22.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:22 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:22.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:22 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:22.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:22 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:22.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:22 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:22.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:22 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:23.428 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:23 smithi152 ceph-mon[37177]: pgmap v17: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:23.622 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:23 smithi142 ceph-mon[29027]: pgmap v17: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:24.782 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:24 smithi152 ceph-mon[37177]: Standby manager daemon smithi152.usoolh restarted 2024-03-29T04:47:24.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:24 smithi152 ceph-mon[37177]: Standby manager daemon smithi152.usoolh started 2024-03-29T04:47:24.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:24 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/144443352' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/crt"}]: dispatch 2024-03-29T04:47:24.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:24 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/144443352' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:47:24.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:24 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/144443352' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/key"}]: dispatch 2024-03-29T04:47:24.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:24 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/144443352' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:47:24.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:24 smithi152 ceph-mon[37177]: pgmap v18: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:24.783 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:24 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:47:24.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:24 smithi142 ceph-mon[29027]: Standby manager daemon smithi152.usoolh restarted 2024-03-29T04:47:24.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:24 smithi142 ceph-mon[29027]: Standby manager daemon smithi152.usoolh started 2024-03-29T04:47:24.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:24 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/144443352' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/crt"}]: dispatch 2024-03-29T04:47:24.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:24 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/144443352' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:47:24.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:24 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/144443352' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/key"}]: dispatch 2024-03-29T04:47:24.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:24 smithi142 ceph-mon[29027]: from='mgr.? 172.21.15.152:0/144443352' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:47:24.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:24 smithi142 ceph-mon[29027]: pgmap v18: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:24.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:24 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:47:25.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: mgrmap e32: smithi142.swesrg(active, since 30s), standbys: smithi152.usoolh 2024-03-29T04:47:25.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:47:25.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:47:25.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:47:25.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: Upgrade: Setting container_image for all mgr 2024-03-29T04:47:25.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi142.swesrg"}]: dispatch 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi142.swesrg"}]': finished 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi152.usoolh"}]: dispatch 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi152.usoolh"}]': finished 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: Upgrade: Setting container_image for all rgw 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: Upgrade: Setting container_image for all rbd-mirror 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-29T04:47:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: Upgrade: Setting container_image for all iscsi 2024-03-29T04:47:25.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: Upgrade: Setting container_image for all nfs 2024-03-29T04:47:25.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: Upgrade: Setting container_image for all nvmeof 2024-03-29T04:47:25.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:25 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.854 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: mgrmap e32: smithi142.swesrg(active, since 30s), standbys: smithi152.usoolh 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: Upgrade: Setting container_image for all mgr 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi142.swesrg"}]: dispatch 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi142.swesrg"}]': finished 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi152.usoolh"}]: dispatch 2024-03-29T04:47:25.855 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi152.usoolh"}]': finished 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: Upgrade: Setting container_image for all rgw 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: Upgrade: Setting container_image for all rbd-mirror 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: Upgrade: Setting container_image for all iscsi 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.856 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: Upgrade: Setting container_image for all nfs 2024-03-29T04:47:25.857 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:25.857 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:25.857 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: Upgrade: Setting container_image for all nvmeof 2024-03-29T04:47:25.857 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:25 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:26.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:26 smithi152 ceph-mon[37177]: pgmap v19: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:26.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:26 smithi142 ceph-mon[29027]: pgmap v19: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:27.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:27 smithi142 ceph-mon[29027]: Upgrade: Updating node-exporter.smithi142 (1/2) 2024-03-29T04:47:27.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:27 smithi142 ceph-mon[29027]: Deploying daemon node-exporter.smithi142 on smithi142 2024-03-29T04:47:27.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:27 smithi152 ceph-mon[37177]: Upgrade: Updating node-exporter.smithi142 (1/2) 2024-03-29T04:47:27.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:27 smithi152 ceph-mon[37177]: Deploying daemon node-exporter.smithi142 on smithi142 2024-03-29T04:47:28.657 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:28 smithi142 ceph-mon[29027]: pgmap v20: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:28.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:28 smithi152 ceph-mon[37177]: pgmap v20: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:30.791 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:30 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:30.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:30 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:30.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:30 smithi152 ceph-mon[37177]: pgmap v21: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:30.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:30 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:30.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:30 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:30.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:30 smithi142 ceph-mon[29027]: pgmap v21: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:31.755 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:31 smithi152 ceph-mon[37177]: Upgrade: Updating node-exporter.smithi152 (2/2) 2024-03-29T04:47:31.755 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:31 smithi152 ceph-mon[37177]: Deploying daemon node-exporter.smithi152 on smithi152 2024-03-29T04:47:31.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:31 smithi142 ceph-mon[29027]: Upgrade: Updating node-exporter.smithi152 (2/2) 2024-03-29T04:47:31.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:31 smithi142 ceph-mon[29027]: Deploying daemon node-exporter.smithi152 on smithi152 2024-03-29T04:47:32.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:32 smithi152 ceph-mon[37177]: pgmap v22: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:32.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:32 smithi142 ceph-mon[29027]: pgmap v22: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:34.872 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:34 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:34.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:34 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:34.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:34 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:47:34.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:34 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:34.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:34 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:34.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:34 smithi142 ceph-mon[29027]: pgmap v23: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:34.877 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:34 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:34.878 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:34 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:34.878 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:34 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:47:34.878 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:34 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:34.878 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:34 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:34.878 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:34 smithi152 ceph-mon[37177]: pgmap v23: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:35.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.521+0000 7f8db97e6700 1 -- 172.21.15.142:0/1531086005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db4100cb0 msgr2=0x7f8db4101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:35.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.521+0000 7f8db97e6700 1 --2- 172.21.15.142:0/1531086005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db4100cb0 0x7f8db4101110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009910 tx=0x7f8da8009c20 comp rx=0 tx=0).stop 2024-03-29T04:47:35.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.522+0000 7f8db97e6700 1 -- 172.21.15.142:0/1531086005 shutdown_connections 2024-03-29T04:47:35.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.522+0000 7f8db97e6700 1 --2- 172.21.15.142:0/1531086005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db4100cb0 0x7f8db4101110 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:35.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.522+0000 7f8db97e6700 1 --2- 172.21.15.142:0/1531086005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8db40ffab0 0x7f8db40ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:35.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.522+0000 7f8db97e6700 1 -- 172.21.15.142:0/1531086005 >> 172.21.15.142:0/1531086005 conn(0x7f8db40fb030 msgr2=0x7f8db40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:35.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.522+0000 7f8db97e6700 1 -- 172.21.15.142:0/1531086005 shutdown_connections 2024-03-29T04:47:35.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.522+0000 7f8db97e6700 1 -- 172.21.15.142:0/1531086005 wait complete. 2024-03-29T04:47:35.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.522+0000 7f8db97e6700 1 Processor -- start 2024-03-29T04:47:35.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db97e6700 1 -- start start 2024-03-29T04:47:35.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db97e6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8db40ffab0 0x7f8db4068f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:35.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db97e6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db4100cb0 0x7f8db419b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:35.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db97e6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db419b670 con 0x7f8db4100cb0 2024-03-29T04:47:35.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db97e6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db419b7e0 con 0x7f8db40ffab0 2024-03-29T04:47:35.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db27fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db4100cb0 0x7f8db419b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:35.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db27fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db4100cb0 0x7f8db419b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52826/0 (socket says 172.21.15.142:52826) 2024-03-29T04:47:35.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db27fc700 1 -- 172.21.15.142:0/3350671568 learned_addr learned my addr 172.21.15.142:0/3350671568 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:47:35.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db2ffd700 1 --2- 172.21.15.142:0/3350671568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8db40ffab0 0x7f8db4068f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:35.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db27fc700 1 -- 172.21.15.142:0/3350671568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8db40ffab0 msgr2=0x7f8db4068f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:35.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db27fc700 1 --2- 172.21.15.142:0/3350671568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8db40ffab0 0x7f8db4068f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:35.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db27fc700 1 -- 172.21.15.142:0/3350671568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da80095f0 con 0x7f8db4100cb0 2024-03-29T04:47:35.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db27fc700 1 --2- 172.21.15.142:0/3350671568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db4100cb0 0x7f8db419b0a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8da8000c00 tx=0x7f8da800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:35.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8d9bfff700 1 -- 172.21.15.142:0/3350671568 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8da801d070 con 0x7f8db4100cb0 2024-03-29T04:47:35.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8d9bfff700 1 -- 172.21.15.142:0/3350671568 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8da8005dc0 con 0x7f8db4100cb0 2024-03-29T04:47:35.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8d9bfff700 1 -- 172.21.15.142:0/3350671568 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8da80226a0 con 0x7f8db4100cb0 2024-03-29T04:47:35.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.523+0000 7f8db97e6700 1 -- 172.21.15.142:0/3350671568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db419ba60 con 0x7f8db4100cb0 2024-03-29T04:47:35.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.524+0000 7f8db97e6700 1 -- 172.21.15.142:0/3350671568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db419bf50 con 0x7f8db4100cb0 2024-03-29T04:47:35.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.525+0000 7f8db97e6700 1 -- 172.21.15.142:0/3350671568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db4108610 con 0x7f8db4100cb0 2024-03-29T04:47:35.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.525+0000 7f8d9bfff700 1 -- 172.21.15.142:0/3350671568 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 91818+0+0 (secure 0 0 0) 0x7f8da80038f0 con 0x7f8db4100cb0 2024-03-29T04:47:35.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.525+0000 7f8d9bfff700 1 --2- 172.21.15.142:0/3350671568 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8d9c06db20 0x7f8d9c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:35.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.525+0000 7f8d9bfff700 1 -- 172.21.15.142:0/3350671568 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8da808ee30 con 0x7f8db4100cb0 2024-03-29T04:47:35.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.526+0000 7f8db2ffd700 1 --2- 172.21.15.142:0/3350671568 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8d9c06db20 0x7f8d9c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:35.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.526+0000 7f8db2ffd700 1 --2- 172.21.15.142:0/3350671568 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8d9c06db20 0x7f8d9c06ffe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8db4100b10 tx=0x7f8da4009260 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:35.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.530+0000 7f8d9bfff700 1 -- 172.21.15.142:0/3350671568 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f8da805c110 con 0x7f8db4100cb0 2024-03-29T04:47:35.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.770+0000 7f8db97e6700 1 -- 172.21.15.142:0/3350671568 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8db4105600 con 0x7f8d9c06db20 2024-03-29T04:47:35.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.771+0000 7f8d9bfff700 1 -- 172.21.15.142:0/3350671568 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f8db4105600 con 0x7f8d9c06db20 2024-03-29T04:47:35.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.774+0000 7f8db97e6700 1 -- 172.21.15.142:0/3350671568 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8d9c06db20 msgr2=0x7f8d9c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:35.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.774+0000 7f8db97e6700 1 --2- 172.21.15.142:0/3350671568 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8d9c06db20 0x7f8d9c06ffe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8db4100b10 tx=0x7f8da4009260 comp rx=0 tx=0).stop 2024-03-29T04:47:35.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.774+0000 7f8db97e6700 1 -- 172.21.15.142:0/3350671568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db4100cb0 msgr2=0x7f8db419b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:35.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.774+0000 7f8db97e6700 1 --2- 172.21.15.142:0/3350671568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db4100cb0 0x7f8db419b0a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8da8000c00 tx=0x7f8da800f690 comp rx=0 tx=0).stop 2024-03-29T04:47:35.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.774+0000 7f8db97e6700 1 -- 172.21.15.142:0/3350671568 shutdown_connections 2024-03-29T04:47:35.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.774+0000 7f8db97e6700 1 --2- 172.21.15.142:0/3350671568 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8d9c06db20 0x7f8d9c06ffe0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:35.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.774+0000 7f8db97e6700 1 --2- 172.21.15.142:0/3350671568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8db40ffab0 0x7f8db4068f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:35.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.774+0000 7f8db97e6700 1 --2- 172.21.15.142:0/3350671568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8db4100cb0 0x7f8db419b0a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:35.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.774+0000 7f8db97e6700 1 -- 172.21.15.142:0/3350671568 >> 172.21.15.142:0/3350671568 conn(0x7f8db40fb030 msgr2=0x7f8db4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:35.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.774+0000 7f8db97e6700 1 -- 172.21.15.142:0/3350671568 shutdown_connections 2024-03-29T04:47:35.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.774+0000 7f8db97e6700 1 -- 172.21.15.142:0/3350671568 wait complete. 2024-03-29T04:47:35.785 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:47:35.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.879+0000 7f2e898b5700 1 -- 172.21.15.142:0/2022717034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84101710 msgr2=0x7f2e84103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:35.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.879+0000 7f2e898b5700 1 --2- 172.21.15.142:0/2022717034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84101710 0x7f2e84103b00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00b0a0 tx=0x7f2e7c00b3b0 comp rx=0 tx=0).stop 2024-03-29T04:47:35.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.879+0000 7f2e898b5700 1 -- 172.21.15.142:0/2022717034 shutdown_connections 2024-03-29T04:47:35.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.879+0000 7f2e898b5700 1 --2- 172.21.15.142:0/2022717034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e84104040 0x7f2e84106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:35.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.879+0000 7f2e898b5700 1 --2- 172.21.15.142:0/2022717034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84101710 0x7f2e84103b00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:35.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.879+0000 7f2e898b5700 1 -- 172.21.15.142:0/2022717034 >> 172.21.15.142:0/2022717034 conn(0x7f2e840fb080 msgr2=0x7f2e840fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:35.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.879+0000 7f2e898b5700 1 -- 172.21.15.142:0/2022717034 shutdown_connections 2024-03-29T04:47:35.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.879+0000 7f2e898b5700 1 -- 172.21.15.142:0/2022717034 wait complete. 2024-03-29T04:47:35.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.879+0000 7f2e898b5700 1 Processor -- start 2024-03-29T04:47:35.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e898b5700 1 -- start start 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e898b5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84101710 0x7f2e84196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e898b5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e84104040 0x7f2e841969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e898b5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84196fe0 con 0x7f2e84101710 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e898b5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84197120 con 0x7f2e84104040 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e82ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84101710 0x7f2e84196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e82ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84101710 0x7f2e84196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52848/0 (socket says 172.21.15.142:52848) 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e82ffd700 1 -- 172.21.15.142:0/1398699153 learned_addr learned my addr 172.21.15.142:0/1398699153 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e827fc700 1 --2- 172.21.15.142:0/1398699153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e84104040 0x7f2e841969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e82ffd700 1 -- 172.21.15.142:0/1398699153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e84104040 msgr2=0x7f2e841969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e82ffd700 1 --2- 172.21.15.142:0/1398699153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e84104040 0x7f2e841969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e82ffd700 1 -- 172.21.15.142:0/1398699153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c009d00 con 0x7f2e84101710 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e82ffd700 1 --2- 172.21.15.142:0/1398699153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84101710 0x7f2e84196480 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0077b0 tx=0x7f2e7c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:35.883 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e888b3700 1 -- 172.21.15.142:0/1398699153 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e7c00e040 con 0x7f2e84101710 2024-03-29T04:47:35.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e898b5700 1 -- 172.21.15.142:0/1398699153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8419bb70 con 0x7f2e84101710 2024-03-29T04:47:35.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.880+0000 7f2e888b3700 1 -- 172.21.15.142:0/1398699153 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2e7c00ce60 con 0x7f2e84101710 2024-03-29T04:47:35.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.881+0000 7f2e888b3700 1 -- 172.21.15.142:0/1398699153 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e7c01daf0 con 0x7f2e84101710 2024-03-29T04:47:35.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.881+0000 7f2e898b5700 1 -- 172.21.15.142:0/1398699153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8419c060 con 0x7f2e84101710 2024-03-29T04:47:35.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.882+0000 7f2e898b5700 1 -- 172.21.15.142:0/1398699153 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e841905e0 con 0x7f2e84101710 2024-03-29T04:47:35.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.882+0000 7f2e888b3700 1 -- 172.21.15.142:0/1398699153 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 91818+0+0 (secure 0 0 0) 0x7f2e7c004250 con 0x7f2e84101710 2024-03-29T04:47:35.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.882+0000 7f2e888b3700 1 --2- 172.21.15.142:0/1398699153 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f2e7006db20 0x7f2e7006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:35.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.883+0000 7f2e888b3700 1 -- 172.21.15.142:0/1398699153 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2e7c08f310 con 0x7f2e84101710 2024-03-29T04:47:35.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.883+0000 7f2e827fc700 1 --2- 172.21.15.142:0/1398699153 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f2e7006db20 0x7f2e7006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:35.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.883+0000 7f2e827fc700 1 --2- 172.21.15.142:0/1398699153 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f2e7006db20 0x7f2e7006ffe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c006fd0 tx=0x7f2e6c008040 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:35.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:35.887+0000 7f2e888b3700 1 -- 172.21.15.142:0/1398699153 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f2e7c05c690 con 0x7f2e84101710 2024-03-29T04:47:36.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.113+0000 7f2e898b5700 1 -- 172.21.15.142:0/1398699153 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e840611d0 con 0x7f2e7006db20 2024-03-29T04:47:36.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.114+0000 7f2e888b3700 1 -- 172.21.15.142:0/1398699153 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f2e840611d0 con 0x7f2e7006db20 2024-03-29T04:47:36.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.116+0000 7f2e898b5700 1 -- 172.21.15.142:0/1398699153 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f2e7006db20 msgr2=0x7f2e7006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:36.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.116+0000 7f2e898b5700 1 --2- 172.21.15.142:0/1398699153 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f2e7006db20 0x7f2e7006ffe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c006fd0 tx=0x7f2e6c008040 comp rx=0 tx=0).stop 2024-03-29T04:47:36.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.116+0000 7f2e898b5700 1 -- 172.21.15.142:0/1398699153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84101710 msgr2=0x7f2e84196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:36.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.116+0000 7f2e898b5700 1 --2- 172.21.15.142:0/1398699153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84101710 0x7f2e84196480 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0077b0 tx=0x7f2e7c003ce0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.116+0000 7f2e898b5700 1 -- 172.21.15.142:0/1398699153 shutdown_connections 2024-03-29T04:47:36.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.116+0000 7f2e898b5700 1 --2- 172.21.15.142:0/1398699153 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f2e7006db20 0x7f2e7006ffe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.116+0000 7f2e898b5700 1 --2- 172.21.15.142:0/1398699153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84101710 0x7f2e84196480 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.116+0000 7f2e898b5700 1 --2- 172.21.15.142:0/1398699153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e84104040 0x7f2e841969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.116+0000 7f2e898b5700 1 -- 172.21.15.142:0/1398699153 >> 172.21.15.142:0/1398699153 conn(0x7f2e840fb080 msgr2=0x7f2e840fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:36.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.116+0000 7f2e898b5700 1 -- 172.21.15.142:0/1398699153 shutdown_connections 2024-03-29T04:47:36.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.116+0000 7f2e898b5700 1 -- 172.21.15.142:0/1398699153 wait complete. 2024-03-29T04:47:36.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.220+0000 7f9225741700 1 -- 172.21.15.142:0/889794959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9220100be0 msgr2=0x7f9220101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:36.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.220+0000 7f9225741700 1 --2- 172.21.15.142:0/889794959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9220100be0 0x7f9220101000 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9214009910 tx=0x7f9214009c20 comp rx=0 tx=0).stop 2024-03-29T04:47:36.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.220+0000 7f9225741700 1 -- 172.21.15.142:0/889794959 shutdown_connections 2024-03-29T04:47:36.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.220+0000 7f9225741700 1 --2- 172.21.15.142:0/889794959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9220101de0 0x7f9220102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.220+0000 7f9225741700 1 --2- 172.21.15.142:0/889794959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9220100be0 0x7f9220101000 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.220+0000 7f9225741700 1 -- 172.21.15.142:0/889794959 >> 172.21.15.142:0/889794959 conn(0x7f92200fc140 msgr2=0x7f92200fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:36.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.220+0000 7f9225741700 1 -- 172.21.15.142:0/889794959 shutdown_connections 2024-03-29T04:47:36.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.220+0000 7f9225741700 1 -- 172.21.15.142:0/889794959 wait complete. 2024-03-29T04:47:36.224 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f9225741700 1 Processor -- start 2024-03-29T04:47:36.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f9225741700 1 -- start start 2024-03-29T04:47:36.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f9225741700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9220100be0 0x7f9220196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:36.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f9225741700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9220101de0 0x7f9220196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:36.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f9225741700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9220196f70 con 0x7f9220101de0 2024-03-29T04:47:36.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f9225741700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92201970b0 con 0x7f9220100be0 2024-03-29T04:47:36.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f921e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9220101de0 0x7f9220196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:36.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f921e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9220101de0 0x7f9220196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52866/0 (socket says 172.21.15.142:52866) 2024-03-29T04:47:36.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f921e7fc700 1 -- 172.21.15.142:0/2896164201 learned_addr learned my addr 172.21.15.142:0/2896164201 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:47:36.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f921effd700 1 --2- 172.21.15.142:0/2896164201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9220100be0 0x7f9220196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:36.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f921e7fc700 1 -- 172.21.15.142:0/2896164201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9220100be0 msgr2=0x7f9220196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:36.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f921e7fc700 1 --2- 172.21.15.142:0/2896164201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9220100be0 0x7f9220196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f921e7fc700 1 -- 172.21.15.142:0/2896164201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92140095f0 con 0x7f9220101de0 2024-03-29T04:47:36.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.221+0000 7f921effd700 1 --2- 172.21.15.142:0/2896164201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9220100be0 0x7f9220196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:47:36.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.222+0000 7f921e7fc700 1 --2- 172.21.15.142:0/2896164201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9220101de0 0x7f9220196950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f921000b4b0 tx=0x7f921000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:36.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.222+0000 7f9207fff700 1 -- 172.21.15.142:0/2896164201 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9210011800 con 0x7f9220101de0 2024-03-29T04:47:36.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.222+0000 7f9225741700 1 -- 172.21.15.142:0/2896164201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922019bb60 con 0x7f9220101de0 2024-03-29T04:47:36.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.222+0000 7f9207fff700 1 -- 172.21.15.142:0/2896164201 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9210011e10 con 0x7f9220101de0 2024-03-29T04:47:36.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.222+0000 7f9207fff700 1 -- 172.21.15.142:0/2896164201 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f921000f2e0 con 0x7f9220101de0 2024-03-29T04:47:36.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.222+0000 7f9225741700 1 -- 172.21.15.142:0/2896164201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922019c130 con 0x7f9220101de0 2024-03-29T04:47:36.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.223+0000 7f9225741700 1 -- 172.21.15.142:0/2896164201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92201097c0 con 0x7f9220101de0 2024-03-29T04:47:36.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.223+0000 7f9207fff700 1 -- 172.21.15.142:0/2896164201 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 91818+0+0 (secure 0 0 0) 0x7f9210010380 con 0x7f9220101de0 2024-03-29T04:47:36.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.224+0000 7f9207fff700 1 --2- 172.21.15.142:0/2896164201 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f920806db20 0x7f920806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:36.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.224+0000 7f9207fff700 1 -- 172.21.15.142:0/2896164201 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f921008d3c0 con 0x7f9220101de0 2024-03-29T04:47:36.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.224+0000 7f921effd700 1 --2- 172.21.15.142:0/2896164201 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f920806db20 0x7f920806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:36.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.224+0000 7f921effd700 1 --2- 172.21.15.142:0/2896164201 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f920806db20 0x7f920806ffe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9214006010 tx=0x7f9214005920 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:36.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.228+0000 7f9207fff700 1 -- 172.21.15.142:0/2896164201 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f921005a850 con 0x7f9220101de0 2024-03-29T04:47:36.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.443+0000 7f9225741700 1 -- 172.21.15.142:0/2896164201 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9220106730 con 0x7f920806db20 2024-03-29T04:47:36.456 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.455+0000 7f9207fff700 1 -- 172.21.15.142:0/2896164201 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9220106730 con 0x7f920806db20 2024-03-29T04:47:36.456 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (9m) 14s ago 10m 25.1M - 0.25.0 c8568f914cd2 d4a3e12f4047 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (11m) 14s ago 11m 7972k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (10m) 16s ago 10m 7872k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (11m) 14s ago 11m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (10m) 16s ago 10m 6660k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (9m) 14s ago 10m 86.6M - 9.4.7 954c08fa6188 aba58e552b7c 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (5m) 14s ago 5m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (5m) 14s ago 5m 63.2M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (5m) 16s ago 5m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (5m) 16s ago 5m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (57s) 14s ago 12m 491M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (20s) 16s ago 9m 65.5M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:47:36.457 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (12m) 14s ago 12m 56.8M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:47:36.458 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (9m) 16s ago 9m 42.9M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:47:36.458 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 starting - - - - 2024-03-29T04:47:36.458 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 starting - - - - 2024-03-29T04:47:36.458 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (9m) 14s ago 9m 216M 1216M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:47:36.458 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (8m) 14s ago 8m 239M 1216M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:47:36.458 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (8m) 14s ago 8m 197M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:47:36.458 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (7m) 16s ago 7m 274M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:47:36.458 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (7m) 16s ago 7m 233M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:47:36.458 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (6m) 16s ago 6m 221M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:47:36.458 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (25s) 14s ago 10m 44.1M - 2.43.0 a07b618ecd1d 76126aade185 2024-03-29T04:47:36.459 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.457+0000 7f9225741700 1 -- 172.21.15.142:0/2896164201 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f920806db20 msgr2=0x7f920806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:36.459 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.457+0000 7f9225741700 1 --2- 172.21.15.142:0/2896164201 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f920806db20 0x7f920806ffe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9214006010 tx=0x7f9214005920 comp rx=0 tx=0).stop 2024-03-29T04:47:36.459 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.457+0000 7f9225741700 1 -- 172.21.15.142:0/2896164201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9220101de0 msgr2=0x7f9220196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:36.459 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.457+0000 7f9225741700 1 --2- 172.21.15.142:0/2896164201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9220101de0 0x7f9220196950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f921000b4b0 tx=0x7f921000b870 comp rx=0 tx=0).stop 2024-03-29T04:47:36.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.457+0000 7f9225741700 1 -- 172.21.15.142:0/2896164201 shutdown_connections 2024-03-29T04:47:36.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.457+0000 7f9225741700 1 --2- 172.21.15.142:0/2896164201 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f920806db20 0x7f920806ffe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.457+0000 7f9225741700 1 --2- 172.21.15.142:0/2896164201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9220100be0 0x7f9220196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.457+0000 7f9225741700 1 --2- 172.21.15.142:0/2896164201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9220101de0 0x7f9220196950 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.457+0000 7f9225741700 1 -- 172.21.15.142:0/2896164201 >> 172.21.15.142:0/2896164201 conn(0x7f92200fc140 msgr2=0x7f9220105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:36.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.457+0000 7f9225741700 1 -- 172.21.15.142:0/2896164201 shutdown_connections 2024-03-29T04:47:36.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.457+0000 7f9225741700 1 -- 172.21.15.142:0/2896164201 wait complete. 2024-03-29T04:47:36.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.574+0000 7f791e22b700 1 -- 172.21.15.142:0/1858644052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7918100bc0 msgr2=0x7f7918100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:36.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.574+0000 7f791e22b700 1 --2- 172.21.15.142:0/1858644052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7918100bc0 0x7f7918100fe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7908009910 tx=0x7f7908009c20 comp rx=0 tx=0).stop 2024-03-29T04:47:36.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.574+0000 7f791e22b700 1 -- 172.21.15.142:0/1858644052 shutdown_connections 2024-03-29T04:47:36.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.574+0000 7f791e22b700 1 --2- 172.21.15.142:0/1858644052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7918101dc0 0x7f7918102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.574+0000 7f791e22b700 1 --2- 172.21.15.142:0/1858644052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7918100bc0 0x7f7918100fe0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.574+0000 7f791e22b700 1 -- 172.21.15.142:0/1858644052 >> 172.21.15.142:0/1858644052 conn(0x7f79180fc140 msgr2=0x7f79180fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:36.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.574+0000 7f791e22b700 1 -- 172.21.15.142:0/1858644052 shutdown_connections 2024-03-29T04:47:36.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.574+0000 7f791e22b700 1 -- 172.21.15.142:0/1858644052 wait complete. 2024-03-29T04:47:36.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.575+0000 7f791e22b700 1 Processor -- start 2024-03-29T04:47:36.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.575+0000 7f791e22b700 1 -- start start 2024-03-29T04:47:36.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.575+0000 7f791e22b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7918100bc0 0x7f7918074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:36.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.575+0000 7f791e22b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7918101dc0 0x7f7918073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:36.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.575+0000 7f791e22b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79180736c0 con 0x7f7918100bc0 2024-03-29T04:47:36.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.575+0000 7f791e22b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7918073800 con 0x7f7918101dc0 2024-03-29T04:47:36.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.575+0000 7f79177fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7918100bc0 0x7f7918074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:36.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.575+0000 7f79177fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7918100bc0 0x7f7918074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52884/0 (socket says 172.21.15.142:52884) 2024-03-29T04:47:36.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.575+0000 7f79177fe700 1 -- 172.21.15.142:0/3104581787 learned_addr learned my addr 172.21.15.142:0/3104581787 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:47:36.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.575+0000 7f7916ffd700 1 --2- 172.21.15.142:0/3104581787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7918101dc0 0x7f7918073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:36.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.576+0000 7f79177fe700 1 -- 172.21.15.142:0/3104581787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7918101dc0 msgr2=0x7f7918073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:36.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.576+0000 7f79177fe700 1 --2- 172.21.15.142:0/3104581787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7918101dc0 0x7f7918073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.576+0000 7f79177fe700 1 -- 172.21.15.142:0/3104581787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79080095f0 con 0x7f7918100bc0 2024-03-29T04:47:36.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.576+0000 7f79177fe700 1 --2- 172.21.15.142:0/3104581787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7918100bc0 0x7f7918074b30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7908000c00 tx=0x7f790800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:36.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.576+0000 7f7914ff9700 1 -- 172.21.15.142:0/3104581787 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f790801d070 con 0x7f7918100bc0 2024-03-29T04:47:36.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.576+0000 7f791e22b700 1 -- 172.21.15.142:0/3104581787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7918073a80 con 0x7f7918100bc0 2024-03-29T04:47:36.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.576+0000 7f7914ff9700 1 -- 172.21.15.142:0/3104581787 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7908005dc0 con 0x7f7918100bc0 2024-03-29T04:47:36.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.576+0000 7f7914ff9700 1 -- 172.21.15.142:0/3104581787 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79080226a0 con 0x7f7918100bc0 2024-03-29T04:47:36.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.576+0000 7f791e22b700 1 -- 172.21.15.142:0/3104581787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7918073f70 con 0x7f7918100bc0 2024-03-29T04:47:36.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.577+0000 7f791e22b700 1 -- 172.21.15.142:0/3104581787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7918109720 con 0x7f7918100bc0 2024-03-29T04:47:36.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.577+0000 7f7914ff9700 1 -- 172.21.15.142:0/3104581787 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 91818+0+0 (secure 0 0 0) 0x7f7908003910 con 0x7f7918100bc0 2024-03-29T04:47:36.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.578+0000 7f7914ff9700 1 --2- 172.21.15.142:0/3104581787 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f790006db20 0x7f790006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:36.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.578+0000 7f7914ff9700 1 -- 172.21.15.142:0/3104581787 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f790808eed0 con 0x7f7918100bc0 2024-03-29T04:47:36.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.578+0000 7f7916ffd700 1 --2- 172.21.15.142:0/3104581787 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f790006db20 0x7f790006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:36.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.578+0000 7f7916ffd700 1 --2- 172.21.15.142:0/3104581787 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f790006db20 0x7f790006ffe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f790c009bf0 tx=0x7f790c009260 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:36.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.583+0000 7f7914ff9700 1 -- 172.21.15.142:0/3104581787 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f790805c250 con 0x7f7918100bc0 2024-03-29T04:47:36.877 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.875+0000 7f791e22b700 1 -- 172.21.15.142:0/3104581787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f791819e810 con 0x7f7918100bc0 2024-03-29T04:47:36.877 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.876+0000 7f7914ff9700 1 -- 172.21.15.142:0/3104581787 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f790805bde0 con 0x7f7918100bc0 2024-03-29T04:47:36.877 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:47:36.877 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:47:36.877 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-29T04:47:36.878 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:47:36.879 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:47:36.879 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:47:36.879 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.878+0000 7f791e22b700 1 -- 172.21.15.142:0/3104581787 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f790006db20 msgr2=0x7f790006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:36.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.878+0000 7f791e22b700 1 --2- 172.21.15.142:0/3104581787 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f790006db20 0x7f790006ffe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f790c009bf0 tx=0x7f790c009260 comp rx=0 tx=0).stop 2024-03-29T04:47:36.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.878+0000 7f791e22b700 1 -- 172.21.15.142:0/3104581787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7918100bc0 msgr2=0x7f7918074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:36.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.878+0000 7f791e22b700 1 --2- 172.21.15.142:0/3104581787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7918100bc0 0x7f7918074b30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7908000c00 tx=0x7f790800f740 comp rx=0 tx=0).stop 2024-03-29T04:47:36.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.878+0000 7f791e22b700 1 -- 172.21.15.142:0/3104581787 shutdown_connections 2024-03-29T04:47:36.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.878+0000 7f791e22b700 1 --2- 172.21.15.142:0/3104581787 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f790006db20 0x7f790006ffe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.878+0000 7f791e22b700 1 --2- 172.21.15.142:0/3104581787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7918100bc0 0x7f7918074b30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.878+0000 7f791e22b700 1 --2- 172.21.15.142:0/3104581787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7918101dc0 0x7f7918073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.878+0000 7f791e22b700 1 -- 172.21.15.142:0/3104581787 >> 172.21.15.142:0/3104581787 conn(0x7f79180fc140 msgr2=0x7f7918104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:36.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.878+0000 7f791e22b700 1 -- 172.21.15.142:0/3104581787 shutdown_connections 2024-03-29T04:47:36.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.878+0000 7f791e22b700 1 -- 172.21.15.142:0/3104581787 wait complete. 2024-03-29T04:47:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.982+0000 7faca0720700 1 -- 172.21.15.142:0/1545484436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac98100cc0 msgr2=0x7fac98101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.982+0000 7faca0720700 1 --2- 172.21.15.142:0/1545484436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac98100cc0 0x7fac98101120 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fac88009910 tx=0x7fac88009c20 comp rx=0 tx=0).stop 2024-03-29T04:47:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.982+0000 7faca0720700 1 -- 172.21.15.142:0/1545484436 shutdown_connections 2024-03-29T04:47:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.982+0000 7faca0720700 1 --2- 172.21.15.142:0/1545484436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac98100cc0 0x7fac98101120 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.982+0000 7faca0720700 1 --2- 172.21.15.142:0/1545484436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac980ffac0 0x7fac980ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.982+0000 7faca0720700 1 -- 172.21.15.142:0/1545484436 >> 172.21.15.142:0/1545484436 conn(0x7fac980fb060 msgr2=0x7fac980fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.982+0000 7faca0720700 1 -- 172.21.15.142:0/1545484436 shutdown_connections 2024-03-29T04:47:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.982+0000 7faca0720700 1 -- 172.21.15.142:0/1545484436 wait complete. 2024-03-29T04:47:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.983+0000 7faca0720700 1 Processor -- start 2024-03-29T04:47:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.983+0000 7faca0720700 1 -- start start 2024-03-29T04:47:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.983+0000 7faca0720700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac980ffac0 0x7fac98103a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:36.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.983+0000 7faca0720700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac98100cc0 0x7fac981020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:36.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.983+0000 7faca0720700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac981025e0 con 0x7fac98100cc0 2024-03-29T04:47:36.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.983+0000 7faca0720700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac98102720 con 0x7fac980ffac0 2024-03-29T04:47:36.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.983+0000 7fac9dcbb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac98100cc0 0x7fac981020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:36.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.983+0000 7fac9dcbb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac98100cc0 0x7fac981020a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52888/0 (socket says 172.21.15.142:52888) 2024-03-29T04:47:36.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.983+0000 7fac9dcbb700 1 -- 172.21.15.142:0/3650676183 learned_addr learned my addr 172.21.15.142:0/3650676183 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:47:36.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.983+0000 7fac9e4bc700 1 --2- 172.21.15.142:0/3650676183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac980ffac0 0x7fac98103a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.983+0000 7fac9dcbb700 1 -- 172.21.15.142:0/3650676183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac980ffac0 msgr2=0x7fac98103a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.983+0000 7fac9dcbb700 1 --2- 172.21.15.142:0/3650676183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac980ffac0 0x7fac98103a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.984+0000 7fac9dcbb700 1 -- 172.21.15.142:0/3650676183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac880095f0 con 0x7fac98100cc0 2024-03-29T04:47:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.984+0000 7fac9dcbb700 1 --2- 172.21.15.142:0/3650676183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac98100cc0 0x7fac981020a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fac88000c00 tx=0x7fac8800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.984+0000 7fac8f7fe700 1 -- 172.21.15.142:0/3650676183 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac8801d070 con 0x7fac98100cc0 2024-03-29T04:47:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.984+0000 7faca0720700 1 -- 172.21.15.142:0/3650676183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac981029a0 con 0x7fac98100cc0 2024-03-29T04:47:36.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.984+0000 7fac8f7fe700 1 -- 172.21.15.142:0/3650676183 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fac88005dc0 con 0x7fac98100cc0 2024-03-29T04:47:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.984+0000 7fac8f7fe700 1 -- 172.21.15.142:0/3650676183 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac880226a0 con 0x7fac98100cc0 2024-03-29T04:47:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.984+0000 7faca0720700 1 -- 172.21.15.142:0/3650676183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac98102e90 con 0x7fac98100cc0 2024-03-29T04:47:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.985+0000 7faca0720700 1 -- 172.21.15.142:0/3650676183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac98108620 con 0x7fac98100cc0 2024-03-29T04:47:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.986+0000 7fac8f7fe700 1 -- 172.21.15.142:0/3650676183 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 91818+0+0 (secure 0 0 0) 0x7fac880038f0 con 0x7fac98100cc0 2024-03-29T04:47:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.986+0000 7fac8f7fe700 1 --2- 172.21.15.142:0/3650676183 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fac8406db20 0x7fac8406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:47:36.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.986+0000 7fac9e4bc700 1 --2- 172.21.15.142:0/3650676183 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fac8406db20 0x7fac8406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:47:36.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.986+0000 7fac8f7fe700 1 -- 172.21.15.142:0/3650676183 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fac8808ed90 con 0x7fac98100cc0 2024-03-29T04:47:36.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.987+0000 7fac9e4bc700 1 --2- 172.21.15.142:0/3650676183 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fac8406db20 0x7fac8406ffe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac98100b20 tx=0x7fac94009260 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:47:36.992 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:36.991+0000 7fac8f7fe700 1 -- 172.21.15.142:0/3650676183 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fac8805c110 con 0x7fac98100cc0 2024-03-29T04:47:37.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.231+0000 7faca0720700 1 -- 172.21.15.142:0/3650676183 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac980694c0 con 0x7fac8406db20 2024-03-29T04:47:37.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.233+0000 7fac8f7fe700 1 -- 172.21.15.142:0/3650676183 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fac980694c0 con 0x7fac8406db20 2024-03-29T04:47:37.234 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:47:37.234 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:47:37.234 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:47:37.234 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T04:47:37.235 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:47:37.235 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr" 2024-03-29T04:47:37.235 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:47:37.235 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "2/2 daemons upgraded", 2024-03-29T04:47:37.235 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading node-exporter daemons", 2024-03-29T04:47:37.235 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:47:37.235 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:47:37.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.235+0000 7faca0720700 1 -- 172.21.15.142:0/3650676183 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fac8406db20 msgr2=0x7fac8406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:37.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.235+0000 7faca0720700 1 --2- 172.21.15.142:0/3650676183 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fac8406db20 0x7fac8406ffe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac98100b20 tx=0x7fac94009260 comp rx=0 tx=0).stop 2024-03-29T04:47:37.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.235+0000 7faca0720700 1 -- 172.21.15.142:0/3650676183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac98100cc0 msgr2=0x7fac981020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:47:37.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.235+0000 7faca0720700 1 --2- 172.21.15.142:0/3650676183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac98100cc0 0x7fac981020a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fac88000c00 tx=0x7fac8800f690 comp rx=0 tx=0).stop 2024-03-29T04:47:37.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.235+0000 7faca0720700 1 -- 172.21.15.142:0/3650676183 shutdown_connections 2024-03-29T04:47:37.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.235+0000 7faca0720700 1 --2- 172.21.15.142:0/3650676183 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fac8406db20 0x7fac8406ffe0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:37.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.235+0000 7faca0720700 1 --2- 172.21.15.142:0/3650676183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac980ffac0 0x7fac98103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:37.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.235+0000 7faca0720700 1 --2- 172.21.15.142:0/3650676183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac98100cc0 0x7fac981020a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:47:37.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.235+0000 7faca0720700 1 -- 172.21.15.142:0/3650676183 >> 172.21.15.142:0/3650676183 conn(0x7fac980fb060 msgr2=0x7fac98068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:47:37.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.235+0000 7faca0720700 1 -- 172.21.15.142:0/3650676183 shutdown_connections 2024-03-29T04:47:37.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:47:37.235+0000 7faca0720700 1 -- 172.21.15.142:0/3650676183 wait complete. 2024-03-29T04:47:37.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:37 smithi142 ceph-mon[29027]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:37.510 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:37 smithi142 ceph-mon[29027]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:37.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:37 smithi142 ceph-mon[29027]: pgmap v24: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:37.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:37 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3104581787' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:37.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:37 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:37.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:37 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:37.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:37 smithi152 ceph-mon[37177]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:37.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:37 smithi152 ceph-mon[37177]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:37.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:37 smithi152 ceph-mon[37177]: pgmap v24: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:37.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:37 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3104581787' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:37.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:37 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:37.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:37 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:38.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:38 smithi142 ceph-mon[29027]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:38.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:38 smithi142 ceph-mon[29027]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:38.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:38 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:38.417 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:38 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:38.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:38 smithi152 ceph-mon[37177]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:38.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:38 smithi152 ceph-mon[37177]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:47:38.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:38 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:38.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:38 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:39.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:39 smithi152 ceph-mon[37177]: pgmap v25: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:39.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:39 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:39.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:39 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:39.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:39 smithi142 ceph-mon[29027]: pgmap v25: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:39.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:39 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:39.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:39 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:40.541 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:40 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:47:40.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:40 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:40.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:40 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:40.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:40 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:47:40.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:40 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:40.565 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:40 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:41.355 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:41 smithi152 ceph-mon[37177]: pgmap v26: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:41.590 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:41 smithi142 ceph-mon[29027]: pgmap v26: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:42.766 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:42.766 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:42.767 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:42 smithi142 ceph-mon[29027]: pgmap v27: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:42.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:42.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:42.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:42 smithi152 ceph-mon[37177]: pgmap v27: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:44.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:44.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:44 smithi152 ceph-mon[37177]: pgmap v28: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:44.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:44.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:44.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:47:44.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:47:44.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:44.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:47:44.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:47:44.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:44 smithi142 ceph-mon[29027]: pgmap v28: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:45.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:45 smithi152 ceph-mon[37177]: Upgrade: Updating prometheus.smithi142 2024-03-29T04:47:45.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:45 smithi142 ceph-mon[29027]: Upgrade: Updating prometheus.smithi142 2024-03-29T04:47:46.858 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:46 smithi142 ceph-mon[29027]: Deploying daemon prometheus.smithi142 on smithi142 2024-03-29T04:47:46.858 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:46 smithi142 ceph-mon[29027]: pgmap v29: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:47.042 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:46 smithi152 ceph-mon[37177]: Deploying daemon prometheus.smithi142 on smithi142 2024-03-29T04:47:47.042 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:46 smithi152 ceph-mon[37177]: pgmap v29: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:49.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:49 smithi152 ceph-mon[37177]: pgmap v30: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:49.608 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:49 smithi142 ceph-mon[29027]: pgmap v30: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:51.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:51 smithi152 ceph-mon[37177]: pgmap v31: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:51.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:51 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:51.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:51 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:51.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:51 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:47:51.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:51 smithi142 ceph-mon[29027]: pgmap v31: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:51.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:51 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:51.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:51 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:51.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:51 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:47:53.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:53 smithi152 ceph-mon[37177]: pgmap v32: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:53.605 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:53 smithi142 ceph-mon[29027]: pgmap v32: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:54.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:54 smithi142 ceph-mon[29027]: pgmap v33: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:54.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:54 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:47:54.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:54 smithi152 ceph-mon[37177]: pgmap v33: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:54.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:54 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:47:57.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:57 smithi152 ceph-mon[37177]: pgmap v34: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:57.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:57 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:57.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:57 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:57.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:57 smithi142 ceph-mon[29027]: pgmap v34: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:47:57.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:57 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:57.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:57 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:58.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:58 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:58.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:58 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:58.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:58 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:58.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:58 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:47:59.445 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:47:59 smithi142 ceph-mon[29027]: pgmap v35: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:47:59.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:47:59 smithi152 ceph-mon[37177]: pgmap v35: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:01.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: pgmap v36: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:01.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:01 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: pgmap v36: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:01.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:01.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:01.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:01 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:02.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:02 smithi142 ceph-mon[29027]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:48:02.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:02 smithi152 ceph-mon[37177]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T04:48:03.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:03 smithi152 ceph-mon[37177]: pgmap v37: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:03.542 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:03 smithi142 ceph-mon[29027]: pgmap v37: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:04.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:04 smithi152 ceph-mon[37177]: Upgrade: Updating alertmanager.smithi142 2024-03-29T04:48:04.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:04 smithi152 ceph-mon[37177]: Deploying daemon alertmanager.smithi142 on smithi142 2024-03-29T04:48:04.545 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:04 smithi142 ceph-mon[29027]: Upgrade: Updating alertmanager.smithi142 2024-03-29T04:48:04.545 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:04 smithi142 ceph-mon[29027]: Deploying daemon alertmanager.smithi142 on smithi142 2024-03-29T04:48:05.422 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:05 smithi142 ceph-mon[29027]: pgmap v38: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:05.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:05 smithi152 ceph-mon[37177]: pgmap v38: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:07.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.349+0000 7f7d1e964700 1 -- 172.21.15.142:0/3684190686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d18101dc0 msgr2=0x7f7d18102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:07.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.349+0000 7f7d1e964700 1 --2- 172.21.15.142:0/3684190686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d18101dc0 0x7f7d18102220 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009910 tx=0x7f7d0c009c20 comp rx=0 tx=0).stop 2024-03-29T04:48:07.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.349+0000 7f7d1e964700 1 -- 172.21.15.142:0/3684190686 shutdown_connections 2024-03-29T04:48:07.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.349+0000 7f7d1e964700 1 --2- 172.21.15.142:0/3684190686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d18101dc0 0x7f7d18102220 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.349+0000 7f7d1e964700 1 --2- 172.21.15.142:0/3684190686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d18100bc0 0x7f7d18100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.349+0000 7f7d1e964700 1 -- 172.21.15.142:0/3684190686 >> 172.21.15.142:0/3684190686 conn(0x7f7d180fc140 msgr2=0x7f7d180fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:07.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.349+0000 7f7d1e964700 1 -- 172.21.15.142:0/3684190686 shutdown_connections 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.349+0000 7f7d1e964700 1 -- 172.21.15.142:0/3684190686 wait complete. 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.349+0000 7f7d1e964700 1 Processor -- start 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.349+0000 7f7d1e964700 1 -- start start 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d1e964700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d18100bc0 0x7f7d18074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d1e964700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d18101dc0 0x7f7d18073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d1e964700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180736c0 con 0x7f7d18101dc0 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d1e964700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18073800 con 0x7f7d18100bc0 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d177fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d18101dc0 0x7f7d18073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d177fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d18101dc0 0x7f7d18073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:48988/0 (socket says 172.21.15.142:48988) 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d177fe700 1 -- 172.21.15.142:0/2530133282 learned_addr learned my addr 172.21.15.142:0/2530133282 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d17fff700 1 --2- 172.21.15.142:0/2530133282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d18100bc0 0x7f7d18074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d177fe700 1 -- 172.21.15.142:0/2530133282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d18100bc0 msgr2=0x7f7d18074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:07.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d177fe700 1 --2- 172.21.15.142:0/2530133282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d18100bc0 0x7f7d18074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d177fe700 1 -- 172.21.15.142:0/2530133282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c0095f0 con 0x7f7d18101dc0 2024-03-29T04:48:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d177fe700 1 --2- 172.21.15.142:0/2530133282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d18101dc0 0x7f7d18073180 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c006010 tx=0x7f7d0c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d157fa700 1 -- 172.21.15.142:0/2530133282 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d0c01d070 con 0x7f7d18101dc0 2024-03-29T04:48:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d157fa700 1 -- 172.21.15.142:0/2530133282 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7d0c00f460 con 0x7f7d18101dc0 2024-03-29T04:48:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d1e964700 1 -- 172.21.15.142:0/2530133282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d18073a80 con 0x7f7d18101dc0 2024-03-29T04:48:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.350+0000 7f7d157fa700 1 -- 172.21.15.142:0/2530133282 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d0c021640 con 0x7f7d18101dc0 2024-03-29T04:48:07.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.351+0000 7f7d1e964700 1 -- 172.21.15.142:0/2530133282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d18073f70 con 0x7f7d18101dc0 2024-03-29T04:48:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.351+0000 7f7d1e964700 1 -- 172.21.15.142:0/2530133282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d18109720 con 0x7f7d18101dc0 2024-03-29T04:48:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.352+0000 7f7d157fa700 1 -- 172.21.15.142:0/2530133282 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 91818+0+0 (secure 0 0 0) 0x7f7d0c00fab0 con 0x7f7d18101dc0 2024-03-29T04:48:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.352+0000 7f7d157fa700 1 --2- 172.21.15.142:0/2530133282 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f7d0006db20 0x7f7d0006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.352+0000 7f7d157fa700 1 -- 172.21.15.142:0/2530133282 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7d0c08eb30 con 0x7f7d18101dc0 2024-03-29T04:48:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.352+0000 7f7d17fff700 1 --2- 172.21.15.142:0/2530133282 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f7d0006db20 0x7f7d0006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:07.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.353+0000 7f7d17fff700 1 --2- 172.21.15.142:0/2530133282 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f7d0006db20 0x7f7d0006ffe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7d18101c20 tx=0x7f7d0800b410 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:07.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.357+0000 7f7d157fa700 1 -- 172.21.15.142:0/2530133282 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f7d0c05beb0 con 0x7f7d18101dc0 2024-03-29T04:48:07.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.588+0000 7f7d1e964700 1 -- 172.21.15.142:0/2530133282 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d18106710 con 0x7f7d0006db20 2024-03-29T04:48:07.592 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:07 smithi142 ceph-mon[29027]: pgmap v39: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:07.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.590+0000 7f7d157fa700 1 -- 172.21.15.142:0/2530133282 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f7d18106710 con 0x7f7d0006db20 2024-03-29T04:48:07.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.592+0000 7f7d1e964700 1 -- 172.21.15.142:0/2530133282 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f7d0006db20 msgr2=0x7f7d0006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:07.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.592+0000 7f7d1e964700 1 --2- 172.21.15.142:0/2530133282 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f7d0006db20 0x7f7d0006ffe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7d18101c20 tx=0x7f7d0800b410 comp rx=0 tx=0).stop 2024-03-29T04:48:07.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.592+0000 7f7d1e964700 1 -- 172.21.15.142:0/2530133282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d18101dc0 msgr2=0x7f7d18073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:07.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.592+0000 7f7d1e964700 1 --2- 172.21.15.142:0/2530133282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d18101dc0 0x7f7d18073180 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c006010 tx=0x7f7d0c00bd70 comp rx=0 tx=0).stop 2024-03-29T04:48:07.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.592+0000 7f7d1e964700 1 -- 172.21.15.142:0/2530133282 shutdown_connections 2024-03-29T04:48:07.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.592+0000 7f7d1e964700 1 --2- 172.21.15.142:0/2530133282 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f7d0006db20 0x7f7d0006ffe0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.592+0000 7f7d1e964700 1 --2- 172.21.15.142:0/2530133282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d18100bc0 0x7f7d18074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.592+0000 7f7d1e964700 1 --2- 172.21.15.142:0/2530133282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d18101dc0 0x7f7d18073180 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.592+0000 7f7d1e964700 1 -- 172.21.15.142:0/2530133282 >> 172.21.15.142:0/2530133282 conn(0x7f7d180fc140 msgr2=0x7f7d18104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:07.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.593+0000 7f7d1e964700 1 -- 172.21.15.142:0/2530133282 shutdown_connections 2024-03-29T04:48:07.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.593+0000 7f7d1e964700 1 -- 172.21.15.142:0/2530133282 wait complete. 2024-03-29T04:48:07.606 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:48:07.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.702+0000 7faaed817700 1 -- 172.21.15.142:0/3803935863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae8101dc0 msgr2=0x7faae8102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:07.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.702+0000 7faaed817700 1 --2- 172.21.15.142:0/3803935863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae8101dc0 0x7faae8102220 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faadc009910 tx=0x7faadc009c20 comp rx=0 tx=0).stop 2024-03-29T04:48:07.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.702+0000 7faaed817700 1 -- 172.21.15.142:0/3803935863 shutdown_connections 2024-03-29T04:48:07.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.702+0000 7faaed817700 1 --2- 172.21.15.142:0/3803935863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae8101dc0 0x7faae8102220 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.702+0000 7faaed817700 1 --2- 172.21.15.142:0/3803935863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae8100bc0 0x7faae8100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.702+0000 7faaed817700 1 -- 172.21.15.142:0/3803935863 >> 172.21.15.142:0/3803935863 conn(0x7faae80fc140 msgr2=0x7faae80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:07.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.702+0000 7faaed817700 1 -- 172.21.15.142:0/3803935863 shutdown_connections 2024-03-29T04:48:07.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.702+0000 7faaed817700 1 -- 172.21.15.142:0/3803935863 wait complete. 2024-03-29T04:48:07.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faaed817700 1 Processor -- start 2024-03-29T04:48:07.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faaed817700 1 -- start start 2024-03-29T04:48:07.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faaed817700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae8100bc0 0x7faae8196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:07.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faaed817700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae8101dc0 0x7faae81968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:07.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faaed817700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae8196ed0 con 0x7faae8101dc0 2024-03-29T04:48:07.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faaed817700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae8197010 con 0x7faae8100bc0 2024-03-29T04:48:07.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faae67fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae8101dc0 0x7faae81968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:07.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faae67fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae8101dc0 0x7faae81968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:48992/0 (socket says 172.21.15.142:48992) 2024-03-29T04:48:07.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faae67fc700 1 -- 172.21.15.142:0/749333253 learned_addr learned my addr 172.21.15.142:0/749333253 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:48:07.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faae6ffd700 1 --2- 172.21.15.142:0/749333253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae8100bc0 0x7faae8196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:07.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faae67fc700 1 -- 172.21.15.142:0/749333253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae8100bc0 msgr2=0x7faae8196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:07.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faae67fc700 1 --2- 172.21.15.142:0/749333253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae8100bc0 0x7faae8196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.703+0000 7faae67fc700 1 -- 172.21.15.142:0/749333253 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc0095f0 con 0x7faae8101dc0 2024-03-29T04:48:07.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.704+0000 7faae67fc700 1 --2- 172.21.15.142:0/749333253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae8101dc0 0x7faae81968b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faadc006010 tx=0x7faadc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:07.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.704+0000 7faaec815700 1 -- 172.21.15.142:0/749333253 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faadc01d070 con 0x7faae8101dc0 2024-03-29T04:48:07.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.704+0000 7faaed817700 1 -- 172.21.15.142:0/749333253 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae819ba60 con 0x7faae8101dc0 2024-03-29T04:48:07.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.704+0000 7faaec815700 1 -- 172.21.15.142:0/749333253 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faadc00f460 con 0x7faae8101dc0 2024-03-29T04:48:07.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.704+0000 7faaec815700 1 -- 172.21.15.142:0/749333253 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faadc017630 con 0x7faae8101dc0 2024-03-29T04:48:07.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.704+0000 7faaed817700 1 -- 172.21.15.142:0/749333253 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae819bf50 con 0x7faae8101dc0 2024-03-29T04:48:07.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.705+0000 7faaed817700 1 -- 172.21.15.142:0/749333253 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae8109720 con 0x7faae8101dc0 2024-03-29T04:48:07.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.705+0000 7faaec815700 1 -- 172.21.15.142:0/749333253 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 91818+0+0 (secure 0 0 0) 0x7faadc003a60 con 0x7faae8101dc0 2024-03-29T04:48:07.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.706+0000 7faaec815700 1 --2- 172.21.15.142:0/749333253 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7faad006db20 0x7faad006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:07.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.706+0000 7faaec815700 1 -- 172.21.15.142:0/749333253 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faadc08eda0 con 0x7faae8101dc0 2024-03-29T04:48:07.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.706+0000 7faae6ffd700 1 --2- 172.21.15.142:0/749333253 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7faad006db20 0x7faad006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:07.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.706+0000 7faae6ffd700 1 --2- 172.21.15.142:0/749333253 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7faad006db20 0x7faad006ffe0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faad8005950 tx=0x7faad80058e0 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:07.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.710+0000 7faaec815700 1 -- 172.21.15.142:0/749333253 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7faadc05c120 con 0x7faae8101dc0 2024-03-29T04:48:07.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:07 smithi152 ceph-mon[37177]: pgmap v39: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:07.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.945+0000 7faaed817700 1 -- 172.21.15.142:0/749333253 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faae819c6a0 con 0x7faad006db20 2024-03-29T04:48:07.949 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.947+0000 7faaec815700 1 -- 172.21.15.142:0/749333253 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7faae819c6a0 con 0x7faad006db20 2024-03-29T04:48:07.951 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.949+0000 7faaed817700 1 -- 172.21.15.142:0/749333253 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7faad006db20 msgr2=0x7faad006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:07.951 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.949+0000 7faaed817700 1 --2- 172.21.15.142:0/749333253 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7faad006db20 0x7faad006ffe0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faad8005950 tx=0x7faad80058e0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.951 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.949+0000 7faaed817700 1 -- 172.21.15.142:0/749333253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae8101dc0 msgr2=0x7faae81968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:07.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.949+0000 7faaed817700 1 --2- 172.21.15.142:0/749333253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae8101dc0 0x7faae81968b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faadc006010 tx=0x7faadc00be20 comp rx=0 tx=0).stop 2024-03-29T04:48:07.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.950+0000 7faaed817700 1 -- 172.21.15.142:0/749333253 shutdown_connections 2024-03-29T04:48:07.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.950+0000 7faaed817700 1 --2- 172.21.15.142:0/749333253 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7faad006db20 0x7faad006ffe0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.950+0000 7faaed817700 1 --2- 172.21.15.142:0/749333253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae8100bc0 0x7faae8196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.950+0000 7faaed817700 1 --2- 172.21.15.142:0/749333253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faae8101dc0 0x7faae81968b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:07.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.950+0000 7faaed817700 1 -- 172.21.15.142:0/749333253 >> 172.21.15.142:0/749333253 conn(0x7faae80fc140 msgr2=0x7faae8104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:07.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.950+0000 7faaed817700 1 -- 172.21.15.142:0/749333253 shutdown_connections 2024-03-29T04:48:07.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:07.950+0000 7faaed817700 1 -- 172.21.15.142:0/749333253 wait complete. 2024-03-29T04:48:08.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.069+0000 7fa0a8750700 1 -- 172.21.15.142:0/3662678613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0a00fec50 msgr2=0x7fa0a00ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:08.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.069+0000 7fa0a8750700 1 --2- 172.21.15.142:0/3662678613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0a00fec50 0x7fa0a00ff0d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa090009910 tx=0x7fa090009c20 comp rx=0 tx=0).stop 2024-03-29T04:48:08.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.070+0000 7fa0a8750700 1 -- 172.21.15.142:0/3662678613 shutdown_connections 2024-03-29T04:48:08.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.070+0000 7fa0a8750700 1 --2- 172.21.15.142:0/3662678613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0a00fec50 0x7fa0a00ff0d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.070+0000 7fa0a8750700 1 --2- 172.21.15.142:0/3662678613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a01021d0 0x7fa0a01025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.070+0000 7fa0a8750700 1 -- 172.21.15.142:0/3662678613 >> 172.21.15.142:0/3662678613 conn(0x7fa0a00fa7f0 msgr2=0x7fa0a00fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:08.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.070+0000 7fa0a8750700 1 -- 172.21.15.142:0/3662678613 shutdown_connections 2024-03-29T04:48:08.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.070+0000 7fa0a8750700 1 -- 172.21.15.142:0/3662678613 wait complete. 2024-03-29T04:48:08.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.070+0000 7fa0a8750700 1 Processor -- start 2024-03-29T04:48:08.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.070+0000 7fa0a8750700 1 -- start start 2024-03-29T04:48:08.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.070+0000 7fa0a8750700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a00fec50 0x7fa0a01018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:08.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.070+0000 7fa0a8750700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0a01021d0 0x7fa0a00fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.070+0000 7fa0a8750700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a0100460 con 0x7fa0a01021d0 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.070+0000 7fa0a8750700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a01005a0 con 0x7fa0a00fec50 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0a5ceb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0a01021d0 0x7fa0a00fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0a5ceb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0a01021d0 0x7fa0a00fff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:49012/0 (socket says 172.21.15.142:49012) 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0a5ceb700 1 -- 172.21.15.142:0/2541515040 learned_addr learned my addr 172.21.15.142:0/2541515040 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0a64ec700 1 --2- 172.21.15.142:0/2541515040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a00fec50 0x7fa0a01018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0a5ceb700 1 -- 172.21.15.142:0/2541515040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a00fec50 msgr2=0x7fa0a01018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0a5ceb700 1 --2- 172.21.15.142:0/2541515040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a00fec50 0x7fa0a01018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0a5ceb700 1 -- 172.21.15.142:0/2541515040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0900095f0 con 0x7fa0a01021d0 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0a5ceb700 1 --2- 172.21.15.142:0/2541515040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0a01021d0 0x7fa0a00fff20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa09000bda0 tx=0x7fa09000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0977fe700 1 -- 172.21.15.142:0/2541515040 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa09001d070 con 0x7fa0a01021d0 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0977fe700 1 -- 172.21.15.142:0/2541515040 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa09000f460 con 0x7fa0a01021d0 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0a8750700 1 -- 172.21.15.142:0/2541515040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a0100820 con 0x7fa0a01021d0 2024-03-29T04:48:08.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0977fe700 1 -- 172.21.15.142:0/2541515040 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa090021620 con 0x7fa0a01021d0 2024-03-29T04:48:08.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.071+0000 7fa0a8750700 1 -- 172.21.15.142:0/2541515040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a0100d10 con 0x7fa0a01021d0 2024-03-29T04:48:08.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.072+0000 7fa0a8750700 1 -- 172.21.15.142:0/2541515040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a0109750 con 0x7fa0a01021d0 2024-03-29T04:48:08.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.073+0000 7fa0977fe700 1 -- 172.21.15.142:0/2541515040 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 91818+0+0 (secure 0 0 0) 0x7fa09000f5d0 con 0x7fa0a01021d0 2024-03-29T04:48:08.077 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.073+0000 7fa0977fe700 1 --2- 172.21.15.142:0/2541515040 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fa08c06db20 0x7fa08c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:08.077 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.073+0000 7fa0977fe700 1 -- 172.21.15.142:0/2541515040 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa09008eb70 con 0x7fa0a01021d0 2024-03-29T04:48:08.077 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.073+0000 7fa0a64ec700 1 --2- 172.21.15.142:0/2541515040 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fa08c06db20 0x7fa08c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:08.077 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.074+0000 7fa0a64ec700 1 --2- 172.21.15.142:0/2541515040 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fa08c06db20 0x7fa08c06ffe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0068b10 tx=0x7fa09c00b410 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:08.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.078+0000 7fa0977fe700 1 -- 172.21.15.142:0/2541515040 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fa09005bef0 con 0x7fa0a01021d0 2024-03-29T04:48:08.287 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.285+0000 7fa0a8750700 1 -- 172.21.15.142:0/2541515040 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0a01068c0 con 0x7fa08c06db20 2024-03-29T04:48:08.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.296+0000 7fa0977fe700 1 -- 172.21.15.142:0/2541515040 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa0a01068c0 con 0x7fa08c06db20 2024-03-29T04:48:08.298 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:48:08.298 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (10m) 12s ago 11m 25.1M - 0.25.0 c8568f914cd2 d4a3e12f4047 2024-03-29T04:48:08.298 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (11m) 12s ago 11m 8019k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:48:08.298 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (10m) 31s ago 10m 7902k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:48:08.298 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (11m) 12s ago 11m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:48:08.298 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (10m) 31s ago 10m 6665k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (9m) 12s ago 11m 84.8M - 9.4.7 954c08fa6188 aba58e552b7c 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (5m) 12s ago 5m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (5m) 12s ago 5m 63.4M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (5m) 31s ago 5m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (5m) 31s ago 5m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (89s) 12s ago 12m 496M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (52s) 31s ago 10m 447M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (12m) 12s ago 13m 58.7M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (10m) 31s ago 10m 44.0M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (39s) 12s ago 11m 8488k - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (35s) 31s ago 10m 8052k - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (9m) 12s ago 9m 217M 1216M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:48:08.299 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (9m) 12s ago 9m 240M 1216M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:48:08.300 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (8m) 12s ago 8m 197M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:48:08.300 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (8m) 31s ago 8m 274M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:48:08.300 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (7m) 31s ago 7m 233M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:48:08.300 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (7m) 31s ago 7m 222M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:48:08.300 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (17s) 12s ago 10m 37.0M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:48:08.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.298+0000 7fa0a8750700 1 -- 172.21.15.142:0/2541515040 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fa08c06db20 msgr2=0x7fa08c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:08.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.298+0000 7fa0a8750700 1 --2- 172.21.15.142:0/2541515040 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fa08c06db20 0x7fa08c06ffe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0068b10 tx=0x7fa09c00b410 comp rx=0 tx=0).stop 2024-03-29T04:48:08.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.299+0000 7fa0a8750700 1 -- 172.21.15.142:0/2541515040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0a01021d0 msgr2=0x7fa0a00fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:08.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.299+0000 7fa0a8750700 1 --2- 172.21.15.142:0/2541515040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0a01021d0 0x7fa0a00fff20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa09000bda0 tx=0x7fa09000bdd0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.299+0000 7fa0a8750700 1 -- 172.21.15.142:0/2541515040 shutdown_connections 2024-03-29T04:48:08.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.299+0000 7fa0a8750700 1 --2- 172.21.15.142:0/2541515040 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fa08c06db20 0x7fa08c06ffe0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.299+0000 7fa0a8750700 1 --2- 172.21.15.142:0/2541515040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a00fec50 0x7fa0a01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.299+0000 7fa0a8750700 1 --2- 172.21.15.142:0/2541515040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0a01021d0 0x7fa0a00fff20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.299+0000 7fa0a8750700 1 -- 172.21.15.142:0/2541515040 >> 172.21.15.142:0/2541515040 conn(0x7fa0a00fa7f0 msgr2=0x7fa0a01051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:08.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.299+0000 7fa0a8750700 1 -- 172.21.15.142:0/2541515040 shutdown_connections 2024-03-29T04:48:08.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.299+0000 7fa0a8750700 1 -- 172.21.15.142:0/2541515040 wait complete. 2024-03-29T04:48:08.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.406+0000 7f7f2d546700 1 -- 172.21.15.142:0/1559940262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f28103ff0 msgr2=0x7f7f281063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:08.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.406+0000 7f7f2d546700 1 --2- 172.21.15.142:0/1559940262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f28103ff0 0x7f7f281063e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009910 tx=0x7f7f1c009c20 comp rx=0 tx=0).stop 2024-03-29T04:48:08.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.406+0000 7f7f2d546700 1 -- 172.21.15.142:0/1559940262 shutdown_connections 2024-03-29T04:48:08.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.406+0000 7f7f2d546700 1 --2- 172.21.15.142:0/1559940262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f28103ff0 0x7f7f281063e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.406+0000 7f7f2d546700 1 --2- 172.21.15.142:0/1559940262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f281016c0 0x7f7f28103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.406+0000 7f7f2d546700 1 -- 172.21.15.142:0/1559940262 >> 172.21.15.142:0/1559940262 conn(0x7f7f280fb030 msgr2=0x7f7f280fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:08.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.406+0000 7f7f2d546700 1 -- 172.21.15.142:0/1559940262 shutdown_connections 2024-03-29T04:48:08.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.406+0000 7f7f2d546700 1 -- 172.21.15.142:0/1559940262 wait complete. 2024-03-29T04:48:08.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.406+0000 7f7f2d546700 1 Processor -- start 2024-03-29T04:48:08.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f2d546700 1 -- start start 2024-03-29T04:48:08.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f2d546700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f281016c0 0x7f7f28196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:08.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f2d546700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f28103ff0 0x7f7f281968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:08.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f2d546700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f28196ef0 con 0x7f7f281016c0 2024-03-29T04:48:08.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f2d546700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f28197030 con 0x7f7f28103ff0 2024-03-29T04:48:08.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f26ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f281016c0 0x7f7f28196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:08.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f26ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f281016c0 0x7f7f28196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:49028/0 (socket says 172.21.15.142:49028) 2024-03-29T04:48:08.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f26ffd700 1 -- 172.21.15.142:0/3784518829 learned_addr learned my addr 172.21.15.142:0/3784518829 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:48:08.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f267fc700 1 --2- 172.21.15.142:0/3784518829 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f28103ff0 0x7f7f281968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:08.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f26ffd700 1 -- 172.21.15.142:0/3784518829 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f28103ff0 msgr2=0x7f7f281968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:08.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f26ffd700 1 --2- 172.21.15.142:0/3784518829 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f28103ff0 0x7f7f281968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f26ffd700 1 -- 172.21.15.142:0/3784518829 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c0095f0 con 0x7f7f281016c0 2024-03-29T04:48:08.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f26ffd700 1 --2- 172.21.15.142:0/3784518829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f281016c0 0x7f7f28196390 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800b4b0 tx=0x7f7f1800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:08.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.407+0000 7f7f0ffff700 1 -- 172.21.15.142:0/3784518829 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f18011800 con 0x7f7f281016c0 2024-03-29T04:48:08.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.408+0000 7f7f0ffff700 1 -- 172.21.15.142:0/3784518829 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7f18011e10 con 0x7f7f281016c0 2024-03-29T04:48:08.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.408+0000 7f7f2d546700 1 -- 172.21.15.142:0/3784518829 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2819bae0 con 0x7f7f281016c0 2024-03-29T04:48:08.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.408+0000 7f7f0ffff700 1 -- 172.21.15.142:0/3784518829 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f1800f2e0 con 0x7f7f281016c0 2024-03-29T04:48:08.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.408+0000 7f7f2d546700 1 -- 172.21.15.142:0/3784518829 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2819c030 con 0x7f7f281016c0 2024-03-29T04:48:08.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.409+0000 7f7f2d546700 1 -- 172.21.15.142:0/3784518829 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f281905a0 con 0x7f7f281016c0 2024-03-29T04:48:08.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.409+0000 7f7f0ffff700 1 -- 172.21.15.142:0/3784518829 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 91818+0+0 (secure 0 0 0) 0x7f7f18010380 con 0x7f7f281016c0 2024-03-29T04:48:08.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.410+0000 7f7f0ffff700 1 --2- 172.21.15.142:0/3784518829 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f7f1006db20 0x7f7f1006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:08.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.410+0000 7f7f0ffff700 1 -- 172.21.15.142:0/3784518829 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7f1808d250 con 0x7f7f281016c0 2024-03-29T04:48:08.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.410+0000 7f7f267fc700 1 --2- 172.21.15.142:0/3784518829 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f7f1006db20 0x7f7f1006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:08.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.410+0000 7f7f267fc700 1 --2- 172.21.15.142:0/3784518829 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f7f1006db20 0x7f7f1006ffe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006010 tx=0x7f7f1c00b580 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:08.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.414+0000 7f7f0ffff700 1 -- 172.21.15.142:0/3784518829 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f7f1805a760 con 0x7f7f281016c0 2024-03-29T04:48:08.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.703+0000 7f7f2d546700 1 -- 172.21.15.142:0/3784518829 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7f2802cc70 con 0x7f7f281016c0 2024-03-29T04:48:08.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.703+0000 7f7f0ffff700 1 -- 172.21.15.142:0/3784518829 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f7f1805a2f0 con 0x7f7f281016c0 2024-03-29T04:48:08.705 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:48:08.706 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-29T04:48:08.707 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:48:08.707 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:48:08.707 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:48:08.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.705+0000 7f7f2d546700 1 -- 172.21.15.142:0/3784518829 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f7f1006db20 msgr2=0x7f7f1006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:08.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.705+0000 7f7f2d546700 1 --2- 172.21.15.142:0/3784518829 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f7f1006db20 0x7f7f1006ffe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006010 tx=0x7f7f1c00b580 comp rx=0 tx=0).stop 2024-03-29T04:48:08.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.705+0000 7f7f2d546700 1 -- 172.21.15.142:0/3784518829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f281016c0 msgr2=0x7f7f28196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:08.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.705+0000 7f7f2d546700 1 --2- 172.21.15.142:0/3784518829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f281016c0 0x7f7f28196390 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800b4b0 tx=0x7f7f1800b870 comp rx=0 tx=0).stop 2024-03-29T04:48:08.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.705+0000 7f7f2d546700 1 -- 172.21.15.142:0/3784518829 shutdown_connections 2024-03-29T04:48:08.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.705+0000 7f7f2d546700 1 --2- 172.21.15.142:0/3784518829 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f7f1006db20 0x7f7f1006ffe0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.705+0000 7f7f2d546700 1 --2- 172.21.15.142:0/3784518829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f281016c0 0x7f7f28196390 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.705+0000 7f7f2d546700 1 --2- 172.21.15.142:0/3784518829 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f28103ff0 0x7f7f281968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.705+0000 7f7f2d546700 1 -- 172.21.15.142:0/3784518829 >> 172.21.15.142:0/3784518829 conn(0x7f7f280fb030 msgr2=0x7f7f280fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:08.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.706+0000 7f7f2d546700 1 -- 172.21.15.142:0/3784518829 shutdown_connections 2024-03-29T04:48:08.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.706+0000 7f7f2d546700 1 -- 172.21.15.142:0/3784518829 wait complete. 2024-03-29T04:48:08.818 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.816+0000 7f47a65ec700 1 -- 172.21.15.142:0/1396843601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0101dc0 msgr2=0x7f47a0102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:08.818 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.816+0000 7f47a65ec700 1 --2- 172.21.15.142:0/1396843601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0101dc0 0x7f47a0102220 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4794009910 tx=0x7f4794009c20 comp rx=0 tx=0).stop 2024-03-29T04:48:08.818 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.816+0000 7f47a65ec700 1 -- 172.21.15.142:0/1396843601 shutdown_connections 2024-03-29T04:48:08.818 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.816+0000 7f47a65ec700 1 --2- 172.21.15.142:0/1396843601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0101dc0 0x7f47a0102220 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.818 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.816+0000 7f47a65ec700 1 --2- 172.21.15.142:0/1396843601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a0100bc0 0x7f47a0100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.816+0000 7f47a65ec700 1 -- 172.21.15.142:0/1396843601 >> 172.21.15.142:0/1396843601 conn(0x7f47a00fc140 msgr2=0x7f47a00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:08.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.816+0000 7f47a65ec700 1 -- 172.21.15.142:0/1396843601 shutdown_connections 2024-03-29T04:48:08.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.816+0000 7f47a65ec700 1 -- 172.21.15.142:0/1396843601 wait complete. 2024-03-29T04:48:08.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.817+0000 7f47a65ec700 1 Processor -- start 2024-03-29T04:48:08.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.817+0000 7f47a65ec700 1 -- start start 2024-03-29T04:48:08.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.817+0000 7f47a65ec700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a0100bc0 0x7f47a0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:08.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.817+0000 7f47a65ec700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0101dc0 0x7f47a0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:08.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.817+0000 7f47a65ec700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a01990c0 con 0x7f47a0101dc0 2024-03-29T04:48:08.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.817+0000 7f47a65ec700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a0199200 con 0x7f47a0100bc0 2024-03-29T04:48:08.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.817+0000 7f479f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0101dc0 0x7f47a0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:08.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.817+0000 7f479f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0101dc0 0x7f47a0198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45432/0 (socket says 172.21.15.142:45432) 2024-03-29T04:48:08.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.817+0000 7f479f7fe700 1 -- 172.21.15.142:0/3460027656 learned_addr learned my addr 172.21.15.142:0/3460027656 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:48:08.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.817+0000 7f479ffff700 1 --2- 172.21.15.142:0/3460027656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a0100bc0 0x7f47a0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:08.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.818+0000 7f479f7fe700 1 -- 172.21.15.142:0/3460027656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a0100bc0 msgr2=0x7f47a0198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:08.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.818+0000 7f479f7fe700 1 --2- 172.21.15.142:0/3460027656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a0100bc0 0x7f47a0198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:08.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.818+0000 7f479f7fe700 1 -- 172.21.15.142:0/3460027656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47940095f0 con 0x7f47a0101dc0 2024-03-29T04:48:08.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.818+0000 7f479f7fe700 1 --2- 172.21.15.142:0/3460027656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0101dc0 0x7f47a0198aa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4794006010 tx=0x7f479400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:08.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.818+0000 7f479d7fa700 1 -- 172.21.15.142:0/3460027656 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f479401d070 con 0x7f47a0101dc0 2024-03-29T04:48:08.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.818+0000 7f47a65ec700 1 -- 172.21.15.142:0/3460027656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a019dc50 con 0x7f47a0101dc0 2024-03-29T04:48:08.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.818+0000 7f479d7fa700 1 -- 172.21.15.142:0/3460027656 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f479400f460 con 0x7f47a0101dc0 2024-03-29T04:48:08.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.818+0000 7f479d7fa700 1 -- 172.21.15.142:0/3460027656 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4794021640 con 0x7f47a0101dc0 2024-03-29T04:48:08.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.818+0000 7f47a65ec700 1 -- 172.21.15.142:0/3460027656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a019e140 con 0x7f47a0101dc0 2024-03-29T04:48:08.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.819+0000 7f47a65ec700 1 -- 172.21.15.142:0/3460027656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a0105ea0 con 0x7f47a0101dc0 2024-03-29T04:48:08.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.820+0000 7f479d7fa700 1 -- 172.21.15.142:0/3460027656 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 32) v1 ==== 91818+0+0 (secure 0 0 0) 0x7f479400fab0 con 0x7f47a0101dc0 2024-03-29T04:48:08.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.820+0000 7f479d7fa700 1 --2- 172.21.15.142:0/3460027656 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f478806db20 0x7f478806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:08.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.820+0000 7f479d7fa700 1 -- 172.21.15.142:0/3460027656 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f479408eb50 con 0x7f47a0101dc0 2024-03-29T04:48:08.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.821+0000 7f479ffff700 1 --2- 172.21.15.142:0/3460027656 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f478806db20 0x7f478806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:08.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.821+0000 7f479ffff700 1 --2- 172.21.15.142:0/3460027656 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f478806db20 0x7f478806ffe0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4798009550 tx=0x7f47980091b0 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:08.827 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:08.825+0000 7f479d7fa700 1 -- 172.21.15.142:0/3460027656 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f479405bed0 con 0x7f47a0101dc0 2024-03-29T04:48:09.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.048+0000 7f47a65ec700 1 -- 172.21.15.142:0/3460027656 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47a00611d0 con 0x7f478806db20 2024-03-29T04:48:09.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.049+0000 7f479d7fa700 1 -- 172.21.15.142:0/3460027656 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f47a00611d0 con 0x7f478806db20 2024-03-29T04:48:09.051 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:48:09.052 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:48:09.052 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:48:09.052 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T04:48:09.052 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:48:09.052 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr" 2024-03-29T04:48:09.052 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:48:09.052 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "2/2 daemons upgraded", 2024-03-29T04:48:09.052 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading alertmanager daemons", 2024-03-29T04:48:09.052 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:48:09.052 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:48:09.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.052+0000 7f47a65ec700 1 -- 172.21.15.142:0/3460027656 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f478806db20 msgr2=0x7f478806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:09.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.052+0000 7f47a65ec700 1 --2- 172.21.15.142:0/3460027656 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f478806db20 0x7f478806ffe0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4798009550 tx=0x7f47980091b0 comp rx=0 tx=0).stop 2024-03-29T04:48:09.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.052+0000 7f47a65ec700 1 -- 172.21.15.142:0/3460027656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0101dc0 msgr2=0x7f47a0198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:09.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.052+0000 7f47a65ec700 1 --2- 172.21.15.142:0/3460027656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0101dc0 0x7f47a0198aa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4794006010 tx=0x7f479400bd70 comp rx=0 tx=0).stop 2024-03-29T04:48:09.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.052+0000 7f47a65ec700 1 -- 172.21.15.142:0/3460027656 shutdown_connections 2024-03-29T04:48:09.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.052+0000 7f47a65ec700 1 --2- 172.21.15.142:0/3460027656 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f478806db20 0x7f478806ffe0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:09.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.052+0000 7f47a65ec700 1 --2- 172.21.15.142:0/3460027656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a0100bc0 0x7f47a0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:09.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.052+0000 7f47a65ec700 1 --2- 172.21.15.142:0/3460027656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0101dc0 0x7f47a0198aa0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:09.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.052+0000 7f47a65ec700 1 -- 172.21.15.142:0/3460027656 >> 172.21.15.142:0/3460027656 conn(0x7f47a00fc140 msgr2=0x7f47a0102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:09.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.052+0000 7f47a65ec700 1 -- 172.21.15.142:0/3460027656 shutdown_connections 2024-03-29T04:48:09.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:09.052+0000 7f47a65ec700 1 -- 172.21.15.142:0/3460027656 wait complete. 2024-03-29T04:48:09.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:09 smithi142 ceph-mon[29027]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:09.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:09 smithi142 ceph-mon[29027]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:09.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:09 smithi142 ceph-mon[29027]: pgmap v40: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:09.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:09 smithi142 ceph-mon[29027]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:09.609 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:09 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3784518829' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:09.610 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:09 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:09.610 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:09 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:09.610 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:09 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:09.610 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:09 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:48:09.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:09 smithi152 ceph-mon[37177]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:09.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:09 smithi152 ceph-mon[37177]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:09.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:09 smithi152 ceph-mon[37177]: pgmap v40: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:09.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:09 smithi152 ceph-mon[37177]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:09.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:09 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3784518829' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:09.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:09 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:09.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:09 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:09.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:09 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:09.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:09 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:48:10.582 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:10 smithi142 ceph-mon[29027]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:10.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:10 smithi152 ceph-mon[37177]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:11.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:11 smithi142 ceph-mon[29027]: pgmap v41: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:11.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:11 smithi152 ceph-mon[37177]: pgmap v41: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:13.613 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:13 smithi142 ceph-mon[29027]: pgmap v42: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:13.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:13 smithi152 ceph-mon[37177]: pgmap v42: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:15.526 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:15 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:15.527 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:15 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:15.527 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:15 smithi142 ceph-mon[29027]: pgmap v43: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:15.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:15 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:15.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:15 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:15.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:15 smithi152 ceph-mon[37177]: pgmap v43: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:16.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:16 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:16.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:16 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:16.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:16 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:16.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:16 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:17.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:17 smithi152 ceph-mon[37177]: pgmap v44: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:17.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:17 smithi142 ceph-mon[29027]: pgmap v44: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:19.340 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: pgmap v45: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.341 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.342 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.342 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:19 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: pgmap v45: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:19.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:19.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:19.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:48:19.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:48:19.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:19.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T04:48:19.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:19.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:19.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:19 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:20.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:20 smithi152 ceph-mon[37177]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T04:48:20.581 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:20 smithi142 ceph-mon[29027]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T04:48:21.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:21 smithi152 ceph-mon[37177]: pgmap v46: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:21.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:21 smithi142 ceph-mon[29027]: pgmap v46: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:22.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:22 smithi152 ceph-mon[37177]: Upgrade: Updating grafana.smithi142 2024-03-29T04:48:22.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:22 smithi152 ceph-mon[37177]: Deploying daemon grafana.smithi142 on smithi142 2024-03-29T04:48:22.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:22 smithi142 ceph-mon[29027]: Upgrade: Updating grafana.smithi142 2024-03-29T04:48:22.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:22 smithi142 ceph-mon[29027]: Deploying daemon grafana.smithi142 on smithi142 2024-03-29T04:48:23.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:23 smithi142 ceph-mon[29027]: pgmap v47: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:23.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:23 smithi152 ceph-mon[37177]: pgmap v47: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:24.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:24 smithi152 ceph-mon[37177]: pgmap v48: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:24.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:24 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:48:24.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:24 smithi142 ceph-mon[29027]: pgmap v48: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:24.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:24 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:48:27.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:27 smithi152 ceph-mon[37177]: mgrmap e33: smithi142.swesrg(active, since 91s), standbys: smithi152.usoolh 2024-03-29T04:48:27.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:27 smithi152 ceph-mon[37177]: pgmap v49: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-29T04:48:27.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:27 smithi142 ceph-mon[29027]: mgrmap e33: smithi142.swesrg(active, since 91s), standbys: smithi152.usoolh 2024-03-29T04:48:27.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:27 smithi142 ceph-mon[29027]: pgmap v49: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-29T04:48:29.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:29 smithi152 ceph-mon[37177]: pgmap v50: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-29T04:48:29.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:29 smithi142 ceph-mon[29027]: pgmap v50: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-29T04:48:31.503 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:31 smithi142 ceph-mon[29027]: pgmap v51: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-29T04:48:31.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:31 smithi152 ceph-mon[37177]: pgmap v51: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-29T04:48:33.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:32 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:33.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:32 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:33.292 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:32 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:33.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:32 smithi152 ceph-mon[37177]: pgmap v52: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-29T04:48:33.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:32 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:33.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:32 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:33.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:32 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:33.373 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:32 smithi142 ceph-mon[29027]: pgmap v52: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-29T04:48:34.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:34 smithi152 ceph-mon[37177]: pgmap v53: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-29T04:48:34.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:34 smithi142 ceph-mon[29027]: pgmap v53: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-29T04:48:37.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:37 smithi152 ceph-mon[37177]: pgmap v54: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-29T04:48:37.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:37 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:37.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:37 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:37.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:37 smithi142 ceph-mon[29027]: pgmap v54: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-29T04:48:37.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:37 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:37.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:37 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:39.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.161+0000 7ffa538d7700 1 -- 172.21.15.142:0/2767040889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c100be0 msgr2=0x7ffa4c101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:39.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.161+0000 7ffa538d7700 1 --2- 172.21.15.142:0/2767040889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c100be0 0x7ffa4c101000 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009910 tx=0x7ffa48009c20 comp rx=0 tx=0).stop 2024-03-29T04:48:39.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.161+0000 7ffa538d7700 1 -- 172.21.15.142:0/2767040889 shutdown_connections 2024-03-29T04:48:39.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.161+0000 7ffa538d7700 1 --2- 172.21.15.142:0/2767040889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c101de0 0x7ffa4c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.161+0000 7ffa538d7700 1 --2- 172.21.15.142:0/2767040889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c100be0 0x7ffa4c101000 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.161+0000 7ffa538d7700 1 -- 172.21.15.142:0/2767040889 >> 172.21.15.142:0/2767040889 conn(0x7ffa4c0fc140 msgr2=0x7ffa4c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:39.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.161+0000 7ffa538d7700 1 -- 172.21.15.142:0/2767040889 shutdown_connections 2024-03-29T04:48:39.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.161+0000 7ffa538d7700 1 -- 172.21.15.142:0/2767040889 wait complete. 2024-03-29T04:48:39.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa538d7700 1 Processor -- start 2024-03-29T04:48:39.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa538d7700 1 -- start start 2024-03-29T04:48:39.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa538d7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c100be0 0x7ffa4c196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:39.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa538d7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c101de0 0x7ffa4c1968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:39.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa538d7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c196e60 con 0x7ffa4c100be0 2024-03-29T04:48:39.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa538d7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c196fa0 con 0x7ffa4c101de0 2024-03-29T04:48:39.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa51673700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c100be0 0x7ffa4c196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:39.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa51673700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c100be0 0x7ffa4c196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59276/0 (socket says 172.21.15.142:59276) 2024-03-29T04:48:39.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa51673700 1 -- 172.21.15.142:0/794549670 learned_addr learned my addr 172.21.15.142:0/794549670 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:48:39.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa50e72700 1 --2- 172.21.15.142:0/794549670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c101de0 0x7ffa4c1968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:39.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa51673700 1 -- 172.21.15.142:0/794549670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c101de0 msgr2=0x7ffa4c1968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:39.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa51673700 1 --2- 172.21.15.142:0/794549670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c101de0 0x7ffa4c1968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa51673700 1 -- 172.21.15.142:0/794549670 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480095f0 con 0x7ffa4c100be0 2024-03-29T04:48:39.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa51673700 1 --2- 172.21.15.142:0/794549670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c100be0 0x7ffa4c196390 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800bda0 tx=0x7ffa4800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:39.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.162+0000 7ffa427fc700 1 -- 172.21.15.142:0/794549670 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa4801d070 con 0x7ffa4c100be0 2024-03-29T04:48:39.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.163+0000 7ffa427fc700 1 -- 172.21.15.142:0/794549670 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffa48004c50 con 0x7ffa4c100be0 2024-03-29T04:48:39.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.163+0000 7ffa538d7700 1 -- 172.21.15.142:0/794549670 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c19ba00 con 0x7ffa4c100be0 2024-03-29T04:48:39.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.163+0000 7ffa427fc700 1 -- 172.21.15.142:0/794549670 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa4800f690 con 0x7ffa4c100be0 2024-03-29T04:48:39.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.163+0000 7ffa538d7700 1 -- 172.21.15.142:0/794549670 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c19bec0 con 0x7ffa4c100be0 2024-03-29T04:48:39.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.164+0000 7ffa538d7700 1 -- 172.21.15.142:0/794549670 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c109740 con 0x7ffa4c100be0 2024-03-29T04:48:39.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.164+0000 7ffa427fc700 1 -- 172.21.15.142:0/794549670 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7ffa48022a20 con 0x7ffa4c100be0 2024-03-29T04:48:39.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.164+0000 7ffa427fc700 1 --2- 172.21.15.142:0/794549670 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7ffa3806d9e0 0x7ffa3806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:39.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.165+0000 7ffa50e72700 1 --2- 172.21.15.142:0/794549670 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7ffa3806d9e0 0x7ffa3806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:39.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.165+0000 7ffa427fc700 1 -- 172.21.15.142:0/794549670 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ffa4808ec50 con 0x7ffa4c100be0 2024-03-29T04:48:39.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.165+0000 7ffa50e72700 1 --2- 172.21.15.142:0/794549670 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7ffa3806d9e0 0x7ffa3806fea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c005ea0 tx=0x7ffa3c005e30 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:39.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.169+0000 7ffa427fc700 1 -- 172.21.15.142:0/794549670 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7ffa4805c440 con 0x7ffa4c100be0 2024-03-29T04:48:39.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.384+0000 7ffa538d7700 1 -- 172.21.15.142:0/794549670 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa4c106730 con 0x7ffa3806d9e0 2024-03-29T04:48:39.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.385+0000 7ffa427fc700 1 -- 172.21.15.142:0/794549670 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7ffa4c106730 con 0x7ffa3806d9e0 2024-03-29T04:48:39.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.387+0000 7ffa538d7700 1 -- 172.21.15.142:0/794549670 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7ffa3806d9e0 msgr2=0x7ffa3806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:39.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.387+0000 7ffa538d7700 1 --2- 172.21.15.142:0/794549670 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7ffa3806d9e0 0x7ffa3806fea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c005ea0 tx=0x7ffa3c005e30 comp rx=0 tx=0).stop 2024-03-29T04:48:39.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.387+0000 7ffa538d7700 1 -- 172.21.15.142:0/794549670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c100be0 msgr2=0x7ffa4c196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:39.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.387+0000 7ffa538d7700 1 --2- 172.21.15.142:0/794549670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c100be0 0x7ffa4c196390 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800bda0 tx=0x7ffa4800be80 comp rx=0 tx=0).stop 2024-03-29T04:48:39.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.387+0000 7ffa538d7700 1 -- 172.21.15.142:0/794549670 shutdown_connections 2024-03-29T04:48:39.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.387+0000 7ffa538d7700 1 --2- 172.21.15.142:0/794549670 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7ffa3806d9e0 0x7ffa3806fea0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.387+0000 7ffa538d7700 1 --2- 172.21.15.142:0/794549670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c100be0 0x7ffa4c196390 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.387+0000 7ffa538d7700 1 --2- 172.21.15.142:0/794549670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c101de0 0x7ffa4c1968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.387+0000 7ffa538d7700 1 -- 172.21.15.142:0/794549670 >> 172.21.15.142:0/794549670 conn(0x7ffa4c0fc140 msgr2=0x7ffa4c105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:39.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.387+0000 7ffa538d7700 1 -- 172.21.15.142:0/794549670 shutdown_connections 2024-03-29T04:48:39.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.387+0000 7ffa538d7700 1 -- 172.21.15.142:0/794549670 wait complete. 2024-03-29T04:48:39.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:39 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:39.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:39 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:39.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:39 smithi142 ceph-mon[29027]: pgmap v55: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:39.401 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:48:39.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.493+0000 7fd3d898c700 1 -- 172.21.15.142:0/350149867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d4100cd0 msgr2=0x7fd3d4101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:39.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.493+0000 7fd3d898c700 1 --2- 172.21.15.142:0/350149867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d4100cd0 0x7fd3d4101130 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009910 tx=0x7fd3c8009c20 comp rx=0 tx=0).stop 2024-03-29T04:48:39.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.493+0000 7fd3d898c700 1 -- 172.21.15.142:0/350149867 shutdown_connections 2024-03-29T04:48:39.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.493+0000 7fd3d898c700 1 --2- 172.21.15.142:0/350149867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d4100cd0 0x7fd3d4101130 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.493+0000 7fd3d898c700 1 --2- 172.21.15.142:0/350149867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3d40ffad0 0x7fd3d40ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.493+0000 7fd3d898c700 1 -- 172.21.15.142:0/350149867 >> 172.21.15.142:0/350149867 conn(0x7fd3d40fb050 msgr2=0x7fd3d40fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:39.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.493+0000 7fd3d898c700 1 -- 172.21.15.142:0/350149867 shutdown_connections 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.493+0000 7fd3d898c700 1 -- 172.21.15.142:0/350149867 wait complete. 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d898c700 1 Processor -- start 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d898c700 1 -- start start 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d898c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3d40ffad0 0x7fd3d40698f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d898c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d4100cd0 0x7fd3d419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d898c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d4196ff0 con 0x7fd3d4100cd0 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d898c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d419dac0 con 0x7fd3d40ffad0 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d4100cd0 0x7fd3d419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d4100cd0 0x7fd3d419d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59302/0 (socket says 172.21.15.142:59302) 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d1d9b700 1 -- 172.21.15.142:0/677483238 learned_addr learned my addr 172.21.15.142:0/677483238 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d1d9b700 1 -- 172.21.15.142:0/677483238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3d40ffad0 msgr2=0x7fd3d40698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d1d9b700 1 --2- 172.21.15.142:0/677483238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3d40ffad0 0x7fd3d40698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d1d9b700 1 -- 172.21.15.142:0/677483238 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c80095f0 con 0x7fd3d4100cd0 2024-03-29T04:48:39.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.494+0000 7fd3d1d9b700 1 --2- 172.21.15.142:0/677483238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d4100cd0 0x7fd3d419d580 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800bd70 tx=0x7fd3c800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:39.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.495+0000 7fd3c37fe700 1 -- 172.21.15.142:0/677483238 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3c801d070 con 0x7fd3d4100cd0 2024-03-29T04:48:39.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.495+0000 7fd3c37fe700 1 -- 172.21.15.142:0/677483238 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd3c800f460 con 0x7fd3d4100cd0 2024-03-29T04:48:39.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.495+0000 7fd3d898c700 1 -- 172.21.15.142:0/677483238 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d419dd10 con 0x7fd3d4100cd0 2024-03-29T04:48:39.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.495+0000 7fd3c37fe700 1 -- 172.21.15.142:0/677483238 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3c8021640 con 0x7fd3d4100cd0 2024-03-29T04:48:39.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.495+0000 7fd3d898c700 1 -- 172.21.15.142:0/677483238 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d419e280 con 0x7fd3d4100cd0 2024-03-29T04:48:39.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.496+0000 7fd3d898c700 1 -- 172.21.15.142:0/677483238 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d41086d0 con 0x7fd3d4100cd0 2024-03-29T04:48:39.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.496+0000 7fd3c37fe700 1 -- 172.21.15.142:0/677483238 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7fd3c800fab0 con 0x7fd3d4100cd0 2024-03-29T04:48:39.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.497+0000 7fd3c37fe700 1 --2- 172.21.15.142:0/677483238 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fd3bc06d9e0 0x7fd3bc06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:39.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.497+0000 7fd3c37fe700 1 -- 172.21.15.142:0/677483238 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd3c808ea20 con 0x7fd3d4100cd0 2024-03-29T04:48:39.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.497+0000 7fd3d259c700 1 --2- 172.21.15.142:0/677483238 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fd3bc06d9e0 0x7fd3bc06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:39.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.497+0000 7fd3d259c700 1 --2- 172.21.15.142:0/677483238 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fd3bc06d9e0 0x7fd3bc06fea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4100b30 tx=0x7fd3c4006d20 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:39.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.501+0000 7fd3c37fe700 1 -- 172.21.15.142:0/677483238 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fd3c805c040 con 0x7fd3d4100cd0 2024-03-29T04:48:39.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:39 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:39.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:39 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:39.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:39 smithi152 ceph-mon[37177]: pgmap v55: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:39.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.719+0000 7fd3d898c700 1 -- 172.21.15.142:0/677483238 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3d4105620 con 0x7fd3bc06d9e0 2024-03-29T04:48:39.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.721+0000 7fd3c37fe700 1 -- 172.21.15.142:0/677483238 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fd3d4105620 con 0x7fd3bc06d9e0 2024-03-29T04:48:39.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.723+0000 7fd3d898c700 1 -- 172.21.15.142:0/677483238 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fd3bc06d9e0 msgr2=0x7fd3bc06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:39.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.723+0000 7fd3d898c700 1 --2- 172.21.15.142:0/677483238 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fd3bc06d9e0 0x7fd3bc06fea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4100b30 tx=0x7fd3c4006d20 comp rx=0 tx=0).stop 2024-03-29T04:48:39.726 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.723+0000 7fd3d898c700 1 -- 172.21.15.142:0/677483238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d4100cd0 msgr2=0x7fd3d419d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:39.726 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.723+0000 7fd3d898c700 1 --2- 172.21.15.142:0/677483238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d4100cd0 0x7fd3d419d580 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800bd70 tx=0x7fd3c800be50 comp rx=0 tx=0).stop 2024-03-29T04:48:39.726 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.723+0000 7fd3d898c700 1 -- 172.21.15.142:0/677483238 shutdown_connections 2024-03-29T04:48:39.726 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.723+0000 7fd3d898c700 1 --2- 172.21.15.142:0/677483238 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fd3bc06d9e0 0x7fd3bc06fea0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.726 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.723+0000 7fd3d898c700 1 --2- 172.21.15.142:0/677483238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3d40ffad0 0x7fd3d40698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.726 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.723+0000 7fd3d898c700 1 --2- 172.21.15.142:0/677483238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3d4100cd0 0x7fd3d419d580 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.726 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.723+0000 7fd3d898c700 1 -- 172.21.15.142:0/677483238 >> 172.21.15.142:0/677483238 conn(0x7fd3d40fb050 msgr2=0x7fd3d4103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:39.726 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.723+0000 7fd3d898c700 1 -- 172.21.15.142:0/677483238 shutdown_connections 2024-03-29T04:48:39.727 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.723+0000 7fd3d898c700 1 -- 172.21.15.142:0/677483238 wait complete. 2024-03-29T04:48:39.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.828+0000 7fdcdce9e700 1 -- 172.21.15.142:0/3924764510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcd8101640 msgr2=0x7fdcd8103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:39.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.828+0000 7fdcdce9e700 1 --2- 172.21.15.142:0/3924764510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcd8101640 0x7fdcd8103a30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009910 tx=0x7fdcc8009c20 comp rx=0 tx=0).stop 2024-03-29T04:48:39.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.828+0000 7fdcdce9e700 1 -- 172.21.15.142:0/3924764510 shutdown_connections 2024-03-29T04:48:39.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.828+0000 7fdcdce9e700 1 --2- 172.21.15.142:0/3924764510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcd8103f70 0x7fdcd8106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.828+0000 7fdcdce9e700 1 --2- 172.21.15.142:0/3924764510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcd8101640 0x7fdcd8103a30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.828+0000 7fdcdce9e700 1 -- 172.21.15.142:0/3924764510 >> 172.21.15.142:0/3924764510 conn(0x7fdcd80fb050 msgr2=0x7fdcd80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.828+0000 7fdcdce9e700 1 -- 172.21.15.142:0/3924764510 shutdown_connections 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.828+0000 7fdcdce9e700 1 -- 172.21.15.142:0/3924764510 wait complete. 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.829+0000 7fdcdce9e700 1 Processor -- start 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.829+0000 7fdcdce9e700 1 -- start start 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.829+0000 7fdcdce9e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcd8101640 0x7fdcd8196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.829+0000 7fdcdce9e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcd8103f70 0x7fdcd8196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.829+0000 7fdcdce9e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd8196e70 con 0x7fdcd8103f70 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.829+0000 7fdcdce9e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd8196fb0 con 0x7fdcd8101640 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.829+0000 7fdcd5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcd8103f70 0x7fdcd8196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.829+0000 7fdcd5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcd8103f70 0x7fdcd8196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59326/0 (socket says 172.21.15.142:59326) 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.829+0000 7fdcd5d9b700 1 -- 172.21.15.142:0/1159012392 learned_addr learned my addr 172.21.15.142:0/1159012392 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.829+0000 7fdcd659c700 1 --2- 172.21.15.142:0/1159012392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcd8101640 0x7fdcd8196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.829+0000 7fdcd5d9b700 1 -- 172.21.15.142:0/1159012392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcd8101640 msgr2=0x7fdcd8196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:39.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.829+0000 7fdcd5d9b700 1 --2- 172.21.15.142:0/1159012392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcd8101640 0x7fdcd8196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:39.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.830+0000 7fdcd5d9b700 1 -- 172.21.15.142:0/1159012392 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc80095f0 con 0x7fdcd8103f70 2024-03-29T04:48:39.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.830+0000 7fdcd5d9b700 1 --2- 172.21.15.142:0/1159012392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcd8103f70 0x7fdcd8196850 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00d8d0 tx=0x7fdccc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:39.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.830+0000 7fdcc77fe700 1 -- 172.21.15.142:0/1159012392 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdccc0098a0 con 0x7fdcd8103f70 2024-03-29T04:48:39.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.830+0000 7fdcdce9e700 1 -- 172.21.15.142:0/1159012392 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd819ba60 con 0x7fdcd8103f70 2024-03-29T04:48:39.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.830+0000 7fdcc77fe700 1 -- 172.21.15.142:0/1159012392 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdccc010460 con 0x7fdcd8103f70 2024-03-29T04:48:39.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.830+0000 7fdcc77fe700 1 -- 172.21.15.142:0/1159012392 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdccc00f5f0 con 0x7fdcd8103f70 2024-03-29T04:48:39.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.830+0000 7fdcdce9e700 1 -- 172.21.15.142:0/1159012392 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd819bfb0 con 0x7fdcd8103f70 2024-03-29T04:48:39.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.831+0000 7fdcdce9e700 1 -- 172.21.15.142:0/1159012392 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd8190520 con 0x7fdcd8103f70 2024-03-29T04:48:39.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.832+0000 7fdcc77fe700 1 -- 172.21.15.142:0/1159012392 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7fdccc009a00 con 0x7fdcd8103f70 2024-03-29T04:48:39.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.832+0000 7fdcc77fe700 1 --2- 172.21.15.142:0/1159012392 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fdcc006d9e0 0x7fdcc006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:39.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.832+0000 7fdcc77fe700 1 -- 172.21.15.142:0/1159012392 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fdccc08d460 con 0x7fdcd8103f70 2024-03-29T04:48:39.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.832+0000 7fdcd659c700 1 --2- 172.21.15.142:0/1159012392 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fdcc006d9e0 0x7fdcc006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:39.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.833+0000 7fdcd659c700 1 --2- 172.21.15.142:0/1159012392 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fdcc006d9e0 0x7fdcc006fea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8006010 tx=0x7fdcc800b560 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:39.843 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:39.841+0000 7fdcc77fe700 1 -- 172.21.15.142:0/1159012392 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fdccc05aa00 con 0x7fdcd8103f70 2024-03-29T04:48:40.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.073+0000 7fdcdce9e700 1 -- 172.21.15.142:0/1159012392 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcd80611d0 con 0x7fdcc006d9e0 2024-03-29T04:48:40.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.083+0000 7fdcc77fe700 1 -- 172.21.15.142:0/1159012392 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fdcd80611d0 con 0x7fdcc006d9e0 2024-03-29T04:48:40.086 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:48:40.086 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (32s) 2s ago 11m 16.8M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:48:40.086 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (12m) 2s ago 12m 8070k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:48:40.086 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (11m) 63s ago 11m 7902k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:48:40.086 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (12m) 2s ago 12m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:48:40.086 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (11m) 63s ago 11m 6665k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:48:40.086 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (8s) 2s ago 11m 51.7M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:48:40.086 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (6m) 2s ago 6m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:48:40.087 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (6m) 2s ago 6m 63.3M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:48:40.087 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (6m) 63s ago 6m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:48:40.087 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (6m) 63s ago 6m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:48:40.087 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (2m) 2s ago 13m 504M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:48:40.087 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (83s) 63s ago 11m 447M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:48:40.087 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (13m) 2s ago 13m 60.7M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:48:40.087 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (10m) 63s ago 10m 44.0M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:48:40.087 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (71s) 2s ago 12m 8526k - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:48:40.088 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (66s) 63s ago 11m 8052k - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:48:40.088 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (10m) 2s ago 10m 218M 1216M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:48:40.088 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (9m) 2s ago 9m 240M 1216M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:48:40.088 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (9m) 2s ago 9m 198M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:48:40.088 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (8m) 63s ago 8m 274M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:48:40.088 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (8m) 63s ago 8m 233M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:48:40.088 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (8m) 63s ago 8m 222M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:48:40.088 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (49s) 2s ago 11m 44.9M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:48:40.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.085+0000 7fdcdce9e700 1 -- 172.21.15.142:0/1159012392 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fdcc006d9e0 msgr2=0x7fdcc006fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:40.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.086+0000 7fdcdce9e700 1 --2- 172.21.15.142:0/1159012392 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fdcc006d9e0 0x7fdcc006fea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8006010 tx=0x7fdcc800b560 comp rx=0 tx=0).stop 2024-03-29T04:48:40.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.086+0000 7fdcdce9e700 1 -- 172.21.15.142:0/1159012392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcd8103f70 msgr2=0x7fdcd8196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:40.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.086+0000 7fdcdce9e700 1 --2- 172.21.15.142:0/1159012392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcd8103f70 0x7fdcd8196850 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00d8d0 tx=0x7fdccc00dbe0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.086+0000 7fdcdce9e700 1 -- 172.21.15.142:0/1159012392 shutdown_connections 2024-03-29T04:48:40.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.086+0000 7fdcdce9e700 1 --2- 172.21.15.142:0/1159012392 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fdcc006d9e0 0x7fdcc006fea0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.086+0000 7fdcdce9e700 1 --2- 172.21.15.142:0/1159012392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcd8101640 0x7fdcd8196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.086+0000 7fdcdce9e700 1 --2- 172.21.15.142:0/1159012392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcd8103f70 0x7fdcd8196850 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.086+0000 7fdcdce9e700 1 -- 172.21.15.142:0/1159012392 >> 172.21.15.142:0/1159012392 conn(0x7fdcd80fb050 msgr2=0x7fdcd80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:40.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.086+0000 7fdcdce9e700 1 -- 172.21.15.142:0/1159012392 shutdown_connections 2024-03-29T04:48:40.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.086+0000 7fdcdce9e700 1 -- 172.21.15.142:0/1159012392 wait complete. 2024-03-29T04:48:40.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.191+0000 7fa0219d2700 1 -- 172.21.15.142:0/1900995291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c0ffab0 msgr2=0x7fa01c0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:40.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.191+0000 7fa0219d2700 1 --2- 172.21.15.142:0/1900995291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c0ffab0 0x7fa01c0ffed0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009910 tx=0x7fa00c009c20 comp rx=0 tx=0).stop 2024-03-29T04:48:40.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.191+0000 7fa0219d2700 1 -- 172.21.15.142:0/1900995291 shutdown_connections 2024-03-29T04:48:40.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.191+0000 7fa0219d2700 1 --2- 172.21.15.142:0/1900995291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa01c100cb0 0x7fa01c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.191+0000 7fa0219d2700 1 --2- 172.21.15.142:0/1900995291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c0ffab0 0x7fa01c0ffed0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.191+0000 7fa0219d2700 1 -- 172.21.15.142:0/1900995291 >> 172.21.15.142:0/1900995291 conn(0x7fa01c0fb030 msgr2=0x7fa01c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:40.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.191+0000 7fa0219d2700 1 -- 172.21.15.142:0/1900995291 shutdown_connections 2024-03-29T04:48:40.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.191+0000 7fa0219d2700 1 -- 172.21.15.142:0/1900995291 wait complete. 2024-03-29T04:48:40.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.192+0000 7fa0219d2700 1 Processor -- start 2024-03-29T04:48:40.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.192+0000 7fa0219d2700 1 -- start start 2024-03-29T04:48:40.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.192+0000 7fa0219d2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c0ffab0 0x7fa01c103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:40.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.192+0000 7fa0219d2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa01c100cb0 0x7fa01c102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:40.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.192+0000 7fa0219d2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c1025d0 con 0x7fa01c0ffab0 2024-03-29T04:48:40.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.192+0000 7fa0219d2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c102710 con 0x7fa01c100cb0 2024-03-29T04:48:40.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.192+0000 7fa01affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c0ffab0 0x7fa01c103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:40.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.192+0000 7fa01affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c0ffab0 0x7fa01c103a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59340/0 (socket says 172.21.15.142:59340) 2024-03-29T04:48:40.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.192+0000 7fa01affd700 1 -- 172.21.15.142:0/2559672397 learned_addr learned my addr 172.21.15.142:0/2559672397 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:48:40.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.193+0000 7fa01a7fc700 1 --2- 172.21.15.142:0/2559672397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa01c100cb0 0x7fa01c102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:40.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.193+0000 7fa01affd700 1 -- 172.21.15.142:0/2559672397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa01c100cb0 msgr2=0x7fa01c102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:40.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.193+0000 7fa01affd700 1 --2- 172.21.15.142:0/2559672397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa01c100cb0 0x7fa01c102090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.193+0000 7fa01affd700 1 -- 172.21.15.142:0/2559672397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c0095f0 con 0x7fa01c0ffab0 2024-03-29T04:48:40.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.193+0000 7fa01affd700 1 --2- 172.21.15.142:0/2559672397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c0ffab0 0x7fa01c103a40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00bda0 tx=0x7fa00c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:40.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.193+0000 7fa0209d0700 1 -- 172.21.15.142:0/2559672397 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa00c01d070 con 0x7fa01c0ffab0 2024-03-29T04:48:40.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.193+0000 7fa0219d2700 1 -- 172.21.15.142:0/2559672397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c102990 con 0x7fa01c0ffab0 2024-03-29T04:48:40.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.193+0000 7fa0209d0700 1 -- 172.21.15.142:0/2559672397 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa00c022470 con 0x7fa01c0ffab0 2024-03-29T04:48:40.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.193+0000 7fa0209d0700 1 -- 172.21.15.142:0/2559672397 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa00c00f650 con 0x7fa01c0ffab0 2024-03-29T04:48:40.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.193+0000 7fa0219d2700 1 -- 172.21.15.142:0/2559672397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c102e80 con 0x7fa01c0ffab0 2024-03-29T04:48:40.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.194+0000 7fa0219d2700 1 -- 172.21.15.142:0/2559672397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01c108610 con 0x7fa01c0ffab0 2024-03-29T04:48:40.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.195+0000 7fa0209d0700 1 -- 172.21.15.142:0/2559672397 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7fa00c004af0 con 0x7fa01c0ffab0 2024-03-29T04:48:40.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.195+0000 7fa0209d0700 1 --2- 172.21.15.142:0/2559672397 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fa00406d9e0 0x7fa00406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:40.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.196+0000 7fa01a7fc700 1 --2- 172.21.15.142:0/2559672397 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fa00406d9e0 0x7fa00406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:40.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.196+0000 7fa0209d0700 1 -- 172.21.15.142:0/2559672397 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa00c08eda0 con 0x7fa01c0ffab0 2024-03-29T04:48:40.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.196+0000 7fa01a7fc700 1 --2- 172.21.15.142:0/2559672397 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fa00406d9e0 0x7fa00406fea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa010005950 tx=0x7fa01000a300 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:40.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.200+0000 7fa0209d0700 1 -- 172.21.15.142:0/2559672397 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fa00c05c510 con 0x7fa01c0ffab0 2024-03-29T04:48:40.308 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:40 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:48:40.486 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.484+0000 7fa0219d2700 1 -- 172.21.15.142:0/2559672397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa01c108800 con 0x7fa01c0ffab0 2024-03-29T04:48:40.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.485+0000 7fa0209d0700 1 -- 172.21.15.142:0/2559672397 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fa00c0272c0 con 0x7fa01c0ffab0 2024-03-29T04:48:40.487 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:48:40.487 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:48:40.487 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:48:40.488 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:48:40.489 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:48:40.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.487+0000 7fa0219d2700 1 -- 172.21.15.142:0/2559672397 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fa00406d9e0 msgr2=0x7fa00406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:40.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.487+0000 7fa0219d2700 1 --2- 172.21.15.142:0/2559672397 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fa00406d9e0 0x7fa00406fea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa010005950 tx=0x7fa01000a300 comp rx=0 tx=0).stop 2024-03-29T04:48:40.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.487+0000 7fa0219d2700 1 -- 172.21.15.142:0/2559672397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c0ffab0 msgr2=0x7fa01c103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:40.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.487+0000 7fa0219d2700 1 --2- 172.21.15.142:0/2559672397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c0ffab0 0x7fa01c103a40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00bda0 tx=0x7fa00c00bdd0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.487+0000 7fa0219d2700 1 -- 172.21.15.142:0/2559672397 shutdown_connections 2024-03-29T04:48:40.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.487+0000 7fa0219d2700 1 --2- 172.21.15.142:0/2559672397 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fa00406d9e0 0x7fa00406fea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.487+0000 7fa0219d2700 1 --2- 172.21.15.142:0/2559672397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa01c0ffab0 0x7fa01c103a40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.487+0000 7fa0219d2700 1 --2- 172.21.15.142:0/2559672397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa01c100cb0 0x7fa01c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.487+0000 7fa0219d2700 1 -- 172.21.15.142:0/2559672397 >> 172.21.15.142:0/2559672397 conn(0x7fa01c0fb030 msgr2=0x7fa01c068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:40.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.487+0000 7fa0219d2700 1 -- 172.21.15.142:0/2559672397 shutdown_connections 2024-03-29T04:48:40.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.487+0000 7fa0219d2700 1 -- 172.21.15.142:0/2559672397 wait complete. 2024-03-29T04:48:40.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:40 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:48:40.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.595+0000 7efc2a962700 1 -- 172.21.15.142:0/96690694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc241016e0 msgr2=0x7efc24103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:40.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.595+0000 7efc2a962700 1 --2- 172.21.15.142:0/96690694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc241016e0 0x7efc24103ad0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efc14009910 tx=0x7efc14009c20 comp rx=0 tx=0).stop 2024-03-29T04:48:40.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.595+0000 7efc2a962700 1 -- 172.21.15.142:0/96690694 shutdown_connections 2024-03-29T04:48:40.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.595+0000 7efc2a962700 1 --2- 172.21.15.142:0/96690694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc24104010 0x7efc24106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.595+0000 7efc2a962700 1 --2- 172.21.15.142:0/96690694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc241016e0 0x7efc24103ad0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.595+0000 7efc2a962700 1 -- 172.21.15.142:0/96690694 >> 172.21.15.142:0/96690694 conn(0x7efc240fb050 msgr2=0x7efc240fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:40.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.595+0000 7efc2a962700 1 -- 172.21.15.142:0/96690694 shutdown_connections 2024-03-29T04:48:40.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.595+0000 7efc2a962700 1 -- 172.21.15.142:0/96690694 wait complete. 2024-03-29T04:48:40.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.595+0000 7efc2a962700 1 Processor -- start 2024-03-29T04:48:40.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.595+0000 7efc2a962700 1 -- start start 2024-03-29T04:48:40.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc2a962700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc241016e0 0x7efc24196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc2a962700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc24104010 0x7efc24196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc2a962700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc24196f60 con 0x7efc241016e0 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc2a962700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc241970a0 con 0x7efc24104010 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc23fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc241016e0 0x7efc24196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc23fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc241016e0 0x7efc24196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59356/0 (socket says 172.21.15.142:59356) 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc23fff700 1 -- 172.21.15.142:0/2513118892 learned_addr learned my addr 172.21.15.142:0/2513118892 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc237fe700 1 --2- 172.21.15.142:0/2513118892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc24104010 0x7efc24196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc23fff700 1 -- 172.21.15.142:0/2513118892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc24104010 msgr2=0x7efc24196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc23fff700 1 --2- 172.21.15.142:0/2513118892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc24104010 0x7efc24196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc23fff700 1 -- 172.21.15.142:0/2513118892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc140095f0 con 0x7efc241016e0 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc237fe700 1 --2- 172.21.15.142:0/2513118892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc24104010 0x7efc24196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.596+0000 7efc23fff700 1 --2- 172.21.15.142:0/2513118892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc241016e0 0x7efc24196400 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efc1400bd70 tx=0x7efc1400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:40.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.597+0000 7efc217fa700 1 -- 172.21.15.142:0/2513118892 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc1401d070 con 0x7efc241016e0 2024-03-29T04:48:40.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.597+0000 7efc217fa700 1 -- 172.21.15.142:0/2513118892 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efc14022470 con 0x7efc241016e0 2024-03-29T04:48:40.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.597+0000 7efc2a962700 1 -- 172.21.15.142:0/2513118892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2419baf0 con 0x7efc241016e0 2024-03-29T04:48:40.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.597+0000 7efc2a962700 1 -- 172.21.15.142:0/2513118892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2419c060 con 0x7efc241016e0 2024-03-29T04:48:40.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.597+0000 7efc217fa700 1 -- 172.21.15.142:0/2513118892 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc1400f670 con 0x7efc241016e0 2024-03-29T04:48:40.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.598+0000 7efc2a962700 1 -- 172.21.15.142:0/2513118892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc241905c0 con 0x7efc241016e0 2024-03-29T04:48:40.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.598+0000 7efc217fa700 1 -- 172.21.15.142:0/2513118892 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7efc14004b70 con 0x7efc241016e0 2024-03-29T04:48:40.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.599+0000 7efc217fa700 1 --2- 172.21.15.142:0/2513118892 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7efc0c06d9e0 0x7efc0c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:48:40.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.599+0000 7efc217fa700 1 -- 172.21.15.142:0/2513118892 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7efc1408ebf0 con 0x7efc241016e0 2024-03-29T04:48:40.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.599+0000 7efc237fe700 1 --2- 172.21.15.142:0/2513118892 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7efc0c06d9e0 0x7efc0c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:48:40.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.599+0000 7efc237fe700 1 --2- 172.21.15.142:0/2513118892 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7efc0c06d9e0 0x7efc0c06fea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efc18007900 tx=0x7efc18008040 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:48:40.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.603+0000 7efc217fa700 1 -- 172.21.15.142:0/2513118892 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7efc1405c2c0 con 0x7efc241016e0 2024-03-29T04:48:40.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.838+0000 7efc2a962700 1 -- 172.21.15.142:0/2513118892 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc240611d0 con 0x7efc0c06d9e0 2024-03-29T04:48:40.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.839+0000 7efc217fa700 1 -- 172.21.15.142:0/2513118892 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7efc240611d0 con 0x7efc0c06d9e0 2024-03-29T04:48:40.841 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:48:40.841 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:48:40.841 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:48:40.841 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T04:48:40.841 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:48:40.842 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr" 2024-03-29T04:48:40.842 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:48:40.842 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "2/2 daemons upgraded", 2024-03-29T04:48:40.842 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading grafana daemons", 2024-03-29T04:48:40.842 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:48:40.842 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:48:40.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.841+0000 7efc2a962700 1 -- 172.21.15.142:0/2513118892 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7efc0c06d9e0 msgr2=0x7efc0c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:40.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.841+0000 7efc2a962700 1 --2- 172.21.15.142:0/2513118892 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7efc0c06d9e0 0x7efc0c06fea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efc18007900 tx=0x7efc18008040 comp rx=0 tx=0).stop 2024-03-29T04:48:40.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.841+0000 7efc2a962700 1 -- 172.21.15.142:0/2513118892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc241016e0 msgr2=0x7efc24196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:48:40.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.841+0000 7efc2a962700 1 --2- 172.21.15.142:0/2513118892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc241016e0 0x7efc24196400 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efc1400bd70 tx=0x7efc1400be50 comp rx=0 tx=0).stop 2024-03-29T04:48:40.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.841+0000 7efc2a962700 1 -- 172.21.15.142:0/2513118892 shutdown_connections 2024-03-29T04:48:40.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.841+0000 7efc2a962700 1 --2- 172.21.15.142:0/2513118892 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7efc0c06d9e0 0x7efc0c06fea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.841+0000 7efc2a962700 1 --2- 172.21.15.142:0/2513118892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc241016e0 0x7efc24196400 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.841+0000 7efc2a962700 1 --2- 172.21.15.142:0/2513118892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc24104010 0x7efc24196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:48:40.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.841+0000 7efc2a962700 1 -- 172.21.15.142:0/2513118892 >> 172.21.15.142:0/2513118892 conn(0x7efc240fb050 msgr2=0x7efc240fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:48:40.844 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.842+0000 7efc2a962700 1 -- 172.21.15.142:0/2513118892 shutdown_connections 2024-03-29T04:48:40.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:48:40.842+0000 7efc2a962700 1 -- 172.21.15.142:0/2513118892 wait complete. 2024-03-29T04:48:41.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:41 smithi142 ceph-mon[29027]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:41.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:41 smithi142 ceph-mon[29027]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:41.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:41 smithi142 ceph-mon[29027]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:41.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:41 smithi142 ceph-mon[29027]: pgmap v56: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:41.304 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:41 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2559672397' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:41.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:41 smithi152 ceph-mon[37177]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:41.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:41 smithi152 ceph-mon[37177]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:41.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:41 smithi152 ceph-mon[37177]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:41.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:41 smithi152 ceph-mon[37177]: pgmap v56: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:41.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:41 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2559672397' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:42.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:42.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:42.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:48:42.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:48:42.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-29T04:48:42.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-29T04:48:42.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-29T04:48:42.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:42.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:48:42.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:42.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:42.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:48:42.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:48:42.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:42.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T04:48:42.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-29T04:48:42.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-29T04:48:42.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-29T04:48:42.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-29T04:48:42.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-29T04:48:42.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-29T04:48:42.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-29T04:48:42.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-29T04:48:42.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T04:48:42.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-29T04:48:42.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-29T04:48:42.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:43.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:43 smithi152 ceph-mon[37177]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T04:48:43.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:43 smithi152 ceph-mon[37177]: Upgrade: Finalizing container_image settings 2024-03-29T04:48:43.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:43 smithi152 ceph-mon[37177]: Upgrade: Complete! 2024-03-29T04:48:43.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:43 smithi152 ceph-mon[37177]: pgmap v57: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:43.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:43 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:48:43.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:43 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:48:43.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:43 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:48:43.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:43 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:43.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:43 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:43.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:43 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:48:43.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:43 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:48:43.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:43 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:43.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:43 smithi142 ceph-mon[29027]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T04:48:43.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:43 smithi142 ceph-mon[29027]: Upgrade: Finalizing container_image settings 2024-03-29T04:48:43.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:43 smithi142 ceph-mon[29027]: Upgrade: Complete! 2024-03-29T04:48:43.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:43 smithi142 ceph-mon[29027]: pgmap v57: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:43.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:43 smithi142 ceph-mon[29027]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:48:43.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:43 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:48:43.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:43 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:48:43.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:43 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:43.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:43 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:48:43.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:43 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:48:43.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:43 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:48:43.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:43 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:45.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:45 smithi142 ceph-mon[29027]: pgmap v58: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:45.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:45 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:45.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:45 smithi152 ceph-mon[37177]: pgmap v58: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:45.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:45 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:48:47.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:47 smithi142 ceph-mon[29027]: pgmap v59: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:47.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:47 smithi152 ceph-mon[37177]: pgmap v59: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:49.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:49 smithi142 ceph-mon[29027]: pgmap v60: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:49.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:49 smithi152 ceph-mon[37177]: pgmap v60: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:51.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:51 smithi142 ceph-mon[29027]: pgmap v61: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:51.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:51 smithi152 ceph-mon[37177]: pgmap v61: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:53.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:53 smithi142 ceph-mon[29027]: pgmap v62: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:53.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:53 smithi152 ceph-mon[37177]: pgmap v62: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:54.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:54 smithi142 ceph-mon[29027]: pgmap v63: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:54.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:54 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:48:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:54 smithi152 ceph-mon[37177]: pgmap v63: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:55.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:54 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:48:57.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:57 smithi152 ceph-mon[37177]: pgmap v64: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:57.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:57 smithi142 ceph-mon[29027]: pgmap v64: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:48:59.542 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:48:59 smithi152 ceph-mon[37177]: pgmap v65: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:48:59.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:48:59 smithi142 ceph-mon[29027]: pgmap v65: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:01.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:01 smithi142 ceph-mon[29027]: pgmap v66: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:01.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:01 smithi152 ceph-mon[37177]: pgmap v66: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:02.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:02 smithi142 ceph-mon[29027]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:49:02.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:02 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:49:03.623 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:03 smithi142 ceph-mon[29027]: pgmap v67: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:03.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:03 smithi152 ceph-mon[37177]: pgmap v67: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:04.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:04 smithi142 ceph-mon[29027]: pgmap v68: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:05.042 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:04 smithi152 ceph-mon[37177]: pgmap v68: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:07.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:07 smithi152 ceph-mon[37177]: pgmap v69: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:07.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:07 smithi142 ceph-mon[29027]: pgmap v69: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:09.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:09 smithi152 ceph-mon[37177]: pgmap v70: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:09.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:09 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:49:09.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:09 smithi142 ceph-mon[29027]: pgmap v70: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:09.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:09 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:49:10.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.962+0000 7f901577d700 1 -- 172.21.15.142:0/3966856405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010100be0 msgr2=0x7f9010101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:10.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.962+0000 7f901577d700 1 --2- 172.21.15.142:0/3966856405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010100be0 0x7f9010101000 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9000009910 tx=0x7f9000009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:10.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.962+0000 7f901577d700 1 -- 172.21.15.142:0/3966856405 shutdown_connections 2024-03-29T04:49:10.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.962+0000 7f901577d700 1 --2- 172.21.15.142:0/3966856405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9010101de0 0x7f9010102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:10.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.962+0000 7f901577d700 1 --2- 172.21.15.142:0/3966856405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010100be0 0x7f9010101000 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:10.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.962+0000 7f901577d700 1 -- 172.21.15.142:0/3966856405 >> 172.21.15.142:0/3966856405 conn(0x7f90100fc140 msgr2=0x7f90100fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:10.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.962+0000 7f901577d700 1 -- 172.21.15.142:0/3966856405 shutdown_connections 2024-03-29T04:49:10.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.962+0000 7f901577d700 1 -- 172.21.15.142:0/3966856405 wait complete. 2024-03-29T04:49:10.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.962+0000 7f901577d700 1 Processor -- start 2024-03-29T04:49:10.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.962+0000 7f901577d700 1 -- start start 2024-03-29T04:49:10.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f901577d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9010100be0 0x7f9010196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:10.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f901577d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010101de0 0x7f90101969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:10.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f901577d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9010196ff0 con 0x7f9010101de0 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f901577d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9010197130 con 0x7f9010100be0 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f900e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010101de0 0x7f90101969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f900e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010101de0 0x7f90101969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54870/0 (socket says 172.21.15.142:54870) 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f900e7fc700 1 -- 172.21.15.142:0/2006162806 learned_addr learned my addr 172.21.15.142:0/2006162806 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f900effd700 1 --2- 172.21.15.142:0/2006162806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9010100be0 0x7f9010196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f900e7fc700 1 -- 172.21.15.142:0/2006162806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9010100be0 msgr2=0x7f9010196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f900e7fc700 1 --2- 172.21.15.142:0/2006162806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9010100be0 0x7f9010196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f900e7fc700 1 -- 172.21.15.142:0/2006162806 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90000095f0 con 0x7f9010101de0 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f900effd700 1 --2- 172.21.15.142:0/2006162806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9010100be0 0x7f9010196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f900e7fc700 1 --2- 172.21.15.142:0/2006162806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010101de0 0x7f90101969d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f900400b4b0 tx=0x7f900400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f8ff7fff700 1 -- 172.21.15.142:0/2006162806 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9004011800 con 0x7f9010101de0 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f8ff7fff700 1 -- 172.21.15.142:0/2006162806 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9004011e20 con 0x7f9010101de0 2024-03-29T04:49:10.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f901577d700 1 -- 172.21.15.142:0/2006162806 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901019bbe0 con 0x7f9010101de0 2024-03-29T04:49:10.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.963+0000 7f8ff7fff700 1 -- 172.21.15.142:0/2006162806 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f900400f2e0 con 0x7f9010101de0 2024-03-29T04:49:10.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.964+0000 7f901577d700 1 -- 172.21.15.142:0/2006162806 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901019c1b0 con 0x7f9010101de0 2024-03-29T04:49:10.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.964+0000 7f901577d700 1 -- 172.21.15.142:0/2006162806 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90101097c0 con 0x7f9010101de0 2024-03-29T04:49:10.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.965+0000 7f8ff7fff700 1 -- 172.21.15.142:0/2006162806 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7f9004010380 con 0x7f9010101de0 2024-03-29T04:49:10.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.965+0000 7f8ff7fff700 1 --2- 172.21.15.142:0/2006162806 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8ff806d9e0 0x7f8ff806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:10.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.965+0000 7f8ff7fff700 1 -- 172.21.15.142:0/2006162806 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f900408d200 con 0x7f9010101de0 2024-03-29T04:49:10.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.966+0000 7f900effd700 1 --2- 172.21.15.142:0/2006162806 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8ff806d9e0 0x7f8ff806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:10.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.966+0000 7f900effd700 1 --2- 172.21.15.142:0/2006162806 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8ff806d9e0 0x7f8ff806fea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9000006010 tx=0x7f9000005960 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:10.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:10.970+0000 7f8ff7fff700 1 -- 172.21.15.142:0/2006162806 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f900405a7a0 con 0x7f9010101de0 2024-03-29T04:49:11.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.191+0000 7f901577d700 1 -- 172.21.15.142:0/2006162806 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9010106730 con 0x7f8ff806d9e0 2024-03-29T04:49:11.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.192+0000 7f8ff7fff700 1 -- 172.21.15.142:0/2006162806 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f9010106730 con 0x7f8ff806d9e0 2024-03-29T04:49:11.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.194+0000 7f901577d700 1 -- 172.21.15.142:0/2006162806 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8ff806d9e0 msgr2=0x7f8ff806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:11.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.194+0000 7f901577d700 1 --2- 172.21.15.142:0/2006162806 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8ff806d9e0 0x7f8ff806fea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9000006010 tx=0x7f9000005960 comp rx=0 tx=0).stop 2024-03-29T04:49:11.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.194+0000 7f901577d700 1 -- 172.21.15.142:0/2006162806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010101de0 msgr2=0x7f90101969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:11.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.194+0000 7f901577d700 1 --2- 172.21.15.142:0/2006162806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010101de0 0x7f90101969d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f900400b4b0 tx=0x7f900400b870 comp rx=0 tx=0).stop 2024-03-29T04:49:11.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.194+0000 7f901577d700 1 -- 172.21.15.142:0/2006162806 shutdown_connections 2024-03-29T04:49:11.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.194+0000 7f901577d700 1 --2- 172.21.15.142:0/2006162806 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8ff806d9e0 0x7f8ff806fea0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:11.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.194+0000 7f901577d700 1 --2- 172.21.15.142:0/2006162806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9010100be0 0x7f9010196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:11.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.194+0000 7f901577d700 1 --2- 172.21.15.142:0/2006162806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9010101de0 0x7f90101969d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:11.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.194+0000 7f901577d700 1 -- 172.21.15.142:0/2006162806 >> 172.21.15.142:0/2006162806 conn(0x7f90100fc140 msgr2=0x7f9010105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:11.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.195+0000 7f901577d700 1 -- 172.21.15.142:0/2006162806 shutdown_connections 2024-03-29T04:49:11.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:11.195+0000 7f901577d700 1 -- 172.21.15.142:0/2006162806 wait complete. 2024-03-29T04:49:11.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:11 smithi142 ceph-mon[29027]: pgmap v71: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:11.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:11 smithi152 ceph-mon[37177]: pgmap v71: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:11.968 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-03-29T04:49:12.181 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:49:12.556 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:12 smithi142 ceph-mon[29027]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:12.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:12 smithi152 ceph-mon[37177]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:13.564 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:13 smithi142 ceph-mon[29027]: pgmap v72: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:13.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.778+0000 7fddf8b96700 1 -- 172.21.15.142:0/1806561327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddf4105a40 msgr2=0x7fddf4105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:13.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.778+0000 7fddf8b96700 1 --2- 172.21.15.142:0/1806561327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddf4105a40 0x7fddf4105e20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009910 tx=0x7fdde8009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:13.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.778+0000 7fddf8b96700 1 -- 172.21.15.142:0/1806561327 shutdown_connections 2024-03-29T04:49:13.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.778+0000 7fddf8b96700 1 --2- 172.21.15.142:0/1806561327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddf40ffbd0 0x7fddf4100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:13.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.778+0000 7fddf8b96700 1 --2- 172.21.15.142:0/1806561327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddf4105a40 0x7fddf4105e20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:13.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.778+0000 7fddf8b96700 1 -- 172.21.15.142:0/1806561327 >> 172.21.15.142:0/1806561327 conn(0x7fddf40755a0 msgr2=0x7fddf40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:13.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.778+0000 7fddf8b96700 1 -- 172.21.15.142:0/1806561327 shutdown_connections 2024-03-29T04:49:13.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.778+0000 7fddf8b96700 1 -- 172.21.15.142:0/1806561327 wait complete. 2024-03-29T04:49:13.784 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.779+0000 7fddf8b96700 1 Processor -- start 2024-03-29T04:49:13.784 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.779+0000 7fddf8b96700 1 -- start start 2024-03-29T04:49:13.784 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.779+0000 7fddf8b96700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddf40ffbd0 0x7fddf4197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:13.784 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.779+0000 7fddf8b96700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddf4105a40 0x7fddf4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:13.784 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.779+0000 7fddf8b96700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf41980f0 con 0x7fddf4105a40 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.779+0000 7fddf8b96700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4198260 con 0x7fddf40ffbd0 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.779+0000 7fddf1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddf4105a40 0x7fddf4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.779+0000 7fddf1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddf4105a40 0x7fddf4197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54882/0 (socket says 172.21.15.142:54882) 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.779+0000 7fddf1d9b700 1 -- 172.21.15.142:0/518108856 learned_addr learned my addr 172.21.15.142:0/518108856 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.780+0000 7fddf259c700 1 --2- 172.21.15.142:0/518108856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddf40ffbd0 0x7fddf4197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.780+0000 7fddf1d9b700 1 -- 172.21.15.142:0/518108856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddf40ffbd0 msgr2=0x7fddf4197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.780+0000 7fddf1d9b700 1 --2- 172.21.15.142:0/518108856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddf40ffbd0 0x7fddf4197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.780+0000 7fddf1d9b700 1 -- 172.21.15.142:0/518108856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde80095f0 con 0x7fddf4105a40 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.780+0000 7fddf1d9b700 1 --2- 172.21.15.142:0/518108856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddf4105a40 0x7fddf4197ad0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdde400d900 tx=0x7fdde400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.780+0000 7fdde37fe700 1 -- 172.21.15.142:0/518108856 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde40041f0 con 0x7fddf4105a40 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.780+0000 7fdde37fe700 1 -- 172.21.15.142:0/518108856 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdde4004cf0 con 0x7fddf4105a40 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.780+0000 7fddf8b96700 1 -- 172.21.15.142:0/518108856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf41a0600 con 0x7fddf4105a40 2024-03-29T04:49:13.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.780+0000 7fdde37fe700 1 -- 172.21.15.142:0/518108856 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde400b750 con 0x7fddf4105a40 2024-03-29T04:49:13.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.780+0000 7fddf8b96700 1 -- 172.21.15.142:0/518108856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf41a0af0 con 0x7fddf4105a40 2024-03-29T04:49:13.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.781+0000 7fddf8b96700 1 -- 172.21.15.142:0/518108856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf4108040 con 0x7fddf4105a40 2024-03-29T04:49:13.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.782+0000 7fdde37fe700 1 -- 172.21.15.142:0/518108856 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7fdde4004750 con 0x7fddf4105a40 2024-03-29T04:49:13.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.782+0000 7fdde37fe700 1 --2- 172.21.15.142:0/518108856 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fdddc06d9e0 0x7fdddc06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:13.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.782+0000 7fdde37fe700 1 -- 172.21.15.142:0/518108856 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fdde408d490 con 0x7fddf4105a40 2024-03-29T04:49:13.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.782+0000 7fddf259c700 1 --2- 172.21.15.142:0/518108856 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fdddc06d9e0 0x7fdddc06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:13.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.782+0000 7fddf259c700 1 --2- 172.21.15.142:0/518108856 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fdddc06d9e0 0x7fdddc06fea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdde8006010 tx=0x7fdde801a040 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:13.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:13.787+0000 7fdde37fe700 1 -- 172.21.15.142:0/518108856 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fdde4056f00 con 0x7fddf4105a40 2024-03-29T04:49:13.792 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:13 smithi152 ceph-mon[37177]: pgmap v72: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:14.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.067+0000 7fddf8b96700 1 -- 172.21.15.142:0/518108856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fddf404ea90 con 0x7fddf4105a40 2024-03-29T04:49:14.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.067+0000 7fdde37fe700 1 -- 172.21.15.142:0/518108856 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fdde405a5c0 con 0x7fddf4105a40 2024-03-29T04:49:14.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.070+0000 7fddf8b96700 1 -- 172.21.15.142:0/518108856 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fdddc06d9e0 msgr2=0x7fdddc06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:14.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.070+0000 7fddf8b96700 1 --2- 172.21.15.142:0/518108856 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fdddc06d9e0 0x7fdddc06fea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdde8006010 tx=0x7fdde801a040 comp rx=0 tx=0).stop 2024-03-29T04:49:14.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.070+0000 7fddf8b96700 1 -- 172.21.15.142:0/518108856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddf4105a40 msgr2=0x7fddf4197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:14.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.070+0000 7fddf8b96700 1 --2- 172.21.15.142:0/518108856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddf4105a40 0x7fddf4197ad0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdde400d900 tx=0x7fdde400dc10 comp rx=0 tx=0).stop 2024-03-29T04:49:14.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.070+0000 7fddf8b96700 1 -- 172.21.15.142:0/518108856 shutdown_connections 2024-03-29T04:49:14.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.070+0000 7fddf8b96700 1 --2- 172.21.15.142:0/518108856 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fdddc06d9e0 0x7fdddc06fea0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:14.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.070+0000 7fddf8b96700 1 --2- 172.21.15.142:0/518108856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddf40ffbd0 0x7fddf4197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:14.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.070+0000 7fddf8b96700 1 --2- 172.21.15.142:0/518108856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddf4105a40 0x7fddf4197ad0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:14.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.070+0000 7fddf8b96700 1 -- 172.21.15.142:0/518108856 >> 172.21.15.142:0/518108856 conn(0x7fddf40755a0 msgr2=0x7fddf40fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:14.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.070+0000 7fddf8b96700 1 -- 172.21.15.142:0/518108856 shutdown_connections 2024-03-29T04:49:14.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:14.070+0000 7fddf8b96700 1 -- 172.21.15.142:0/518108856 wait complete. 2024-03-29T04:49:14.083 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:49:14.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:14 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/518108856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:49:14.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:14 smithi142 ceph-mon[29027]: pgmap v73: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:14.773 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-03-29T04:49:14.991 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:49:15.042 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:14 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/518108856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:49:15.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:14 smithi152 ceph-mon[37177]: pgmap v73: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:16.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.709+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3191877934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e94100da0 msgr2=0x7f1e94101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:16.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.709+0000 7f1e9c79b700 1 --2- 172.21.15.142:0/3191877934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e94100da0 0x7f1e94101180 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009910 tx=0x7f1e84009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:16.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3191877934 shutdown_connections 2024-03-29T04:49:16.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e9c79b700 1 --2- 172.21.15.142:0/3191877934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e94101750 0x7f1e941057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e9c79b700 1 --2- 172.21.15.142:0/3191877934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e94100da0 0x7f1e94101180 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3191877934 >> 172.21.15.142:0/3191877934 conn(0x7f1e940fc630 msgr2=0x7f1e940fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3191877934 shutdown_connections 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3191877934 wait complete. 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e9c79b700 1 Processor -- start 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e9c79b700 1 -- start start 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e9c79b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e94100da0 0x7f1e9419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e9c79b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e94101750 0x7f1e9419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e9c79b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9419d210 con 0x7f1e94101750 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e9c79b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941966c0 con 0x7f1e94100da0 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.710+0000 7f1e99d36700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e94101750 0x7f1e9419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e99d36700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e94101750 0x7f1e9419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54900/0 (socket says 172.21.15.142:54900) 2024-03-29T04:49:16.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e99d36700 1 -- 172.21.15.142:0/3088188321 learned_addr learned my addr 172.21.15.142:0/3088188321 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:16.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e9a537700 1 --2- 172.21.15.142:0/3088188321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e94100da0 0x7f1e9419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:16.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e99d36700 1 -- 172.21.15.142:0/3088188321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e94100da0 msgr2=0x7f1e9419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:16.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e99d36700 1 --2- 172.21.15.142:0/3088188321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e94100da0 0x7f1e9419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:16.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e99d36700 1 -- 172.21.15.142:0/3088188321 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e840095f0 con 0x7f1e94101750 2024-03-29T04:49:16.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e9a537700 1 --2- 172.21.15.142:0/3088188321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e94100da0 0x7f1e9419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:49:16.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e99d36700 1 --2- 172.21.15.142:0/3088188321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e94101750 0x7f1e9419cb80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000b480 tx=0x7f1e9000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:16.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e8b7fe700 1 -- 172.21.15.142:0/3088188321 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e900118c0 con 0x7f1e94101750 2024-03-29T04:49:16.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3088188321 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e941969a0 con 0x7f1e94101750 2024-03-29T04:49:16.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e8b7fe700 1 -- 172.21.15.142:0/3088188321 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e90010380 con 0x7f1e94101750 2024-03-29T04:49:16.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e8b7fe700 1 -- 172.21.15.142:0/3088188321 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e9000f4f0 con 0x7f1e94101750 2024-03-29T04:49:16.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.711+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3088188321 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e94196ef0 con 0x7f1e94101750 2024-03-29T04:49:16.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.712+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3088188321 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e94109140 con 0x7f1e94101750 2024-03-29T04:49:16.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.713+0000 7f1e8b7fe700 1 -- 172.21.15.142:0/3088188321 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7f1e900104f0 con 0x7f1e94101750 2024-03-29T04:49:16.718 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.713+0000 7f1e8b7fe700 1 --2- 172.21.15.142:0/3088188321 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f1e8006d9e0 0x7f1e8006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:16.718 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.713+0000 7f1e9a537700 1 --2- 172.21.15.142:0/3088188321 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f1e8006d9e0 0x7f1e8006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:16.718 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.713+0000 7f1e8b7fe700 1 -- 172.21.15.142:0/3088188321 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1e9008d210 con 0x7f1e94101750 2024-03-29T04:49:16.718 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.714+0000 7f1e9a537700 1 --2- 172.21.15.142:0/3088188321 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f1e8006d9e0 0x7f1e8006fea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400b5c0 tx=0x7f1e84005480 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:16.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.718+0000 7f1e8b7fe700 1 -- 172.21.15.142:0/3088188321 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f1e9005a7b0 con 0x7f1e94101750 2024-03-29T04:49:16.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.991+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3088188321 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1e9404ea90 con 0x7f1e94101750 2024-03-29T04:49:16.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.991+0000 7f1e8b7fe700 1 -- 172.21.15.142:0/3088188321 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f1e9005a340 con 0x7f1e94101750 2024-03-29T04:49:16.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.994+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3088188321 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f1e8006d9e0 msgr2=0x7f1e8006fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:16.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.994+0000 7f1e9c79b700 1 --2- 172.21.15.142:0/3088188321 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f1e8006d9e0 0x7f1e8006fea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400b5c0 tx=0x7f1e84005480 comp rx=0 tx=0).stop 2024-03-29T04:49:16.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.994+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3088188321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e94101750 msgr2=0x7f1e9419cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:16.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.994+0000 7f1e9c79b700 1 --2- 172.21.15.142:0/3088188321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e94101750 0x7f1e9419cb80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000b480 tx=0x7f1e9000b840 comp rx=0 tx=0).stop 2024-03-29T04:49:16.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.994+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3088188321 shutdown_connections 2024-03-29T04:49:16.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.994+0000 7f1e9c79b700 1 --2- 172.21.15.142:0/3088188321 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f1e8006d9e0 0x7f1e8006fea0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:16.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.994+0000 7f1e9c79b700 1 --2- 172.21.15.142:0/3088188321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e94100da0 0x7f1e9419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:16.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.994+0000 7f1e9c79b700 1 --2- 172.21.15.142:0/3088188321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e94101750 0x7f1e9419cb80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:16.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.994+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3088188321 >> 172.21.15.142:0/3088188321 conn(0x7f1e940fc630 msgr2=0x7f1e940fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:16.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.994+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3088188321 shutdown_connections 2024-03-29T04:49:16.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:16.994+0000 7f1e9c79b700 1 -- 172.21.15.142:0/3088188321 wait complete. 2024-03-29T04:49:17.007 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)" 2024-03-29T04:49:17.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:17 smithi152 ceph-mon[37177]: pgmap v74: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:17.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:17 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/3088188321' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:49:17.599 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:17 smithi142 ceph-mon[29027]: pgmap v74: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:17.599 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:17 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/3088188321' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:49:17.620 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-03-29T04:49:17.830 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:49:19.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.508+0000 7f85629d3700 1 -- 172.21.15.142:0/65250648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c101780 msgr2=0x7f855c1057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:19.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.508+0000 7f85629d3700 1 --2- 172.21.15.142:0/65250648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c101780 0x7f855c1057d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f854c009910 tx=0x7f854c009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:19.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.508+0000 7f85629d3700 1 -- 172.21.15.142:0/65250648 shutdown_connections 2024-03-29T04:49:19.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.508+0000 7f85629d3700 1 --2- 172.21.15.142:0/65250648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c101780 0x7f855c1057d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:19.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.508+0000 7f85629d3700 1 --2- 172.21.15.142:0/65250648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f855c100dd0 0x7f855c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:19.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.508+0000 7f85629d3700 1 -- 172.21.15.142:0/65250648 >> 172.21.15.142:0/65250648 conn(0x7f855c0fc640 msgr2=0x7f855c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:19.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.508+0000 7f85629d3700 1 -- 172.21.15.142:0/65250648 shutdown_connections 2024-03-29T04:49:19.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.508+0000 7f85629d3700 1 -- 172.21.15.142:0/65250648 wait complete. 2024-03-29T04:49:19.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.509+0000 7f85629d3700 1 Processor -- start 2024-03-29T04:49:19.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.509+0000 7f85629d3700 1 -- start start 2024-03-29T04:49:19.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.509+0000 7f85629d3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c100dd0 0x7f855c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:19.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.509+0000 7f85629d3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f855c101780 0x7f855c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:19.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.509+0000 7f85629d3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c19d240 con 0x7f855c100dd0 2024-03-29T04:49:19.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.509+0000 7f85629d3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c1966f0 con 0x7f855c101780 2024-03-29T04:49:19.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.509+0000 7f855bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c100dd0 0x7f855c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:19.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.509+0000 7f855bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c100dd0 0x7f855c19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37788/0 (socket says 172.21.15.142:37788) 2024-03-29T04:49:19.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.509+0000 7f855bfff700 1 -- 172.21.15.142:0/2373026990 learned_addr learned my addr 172.21.15.142:0/2373026990 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:19.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.509+0000 7f855b7fe700 1 --2- 172.21.15.142:0/2373026990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f855c101780 0x7f855c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:19.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.510+0000 7f855bfff700 1 -- 172.21.15.142:0/2373026990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f855c101780 msgr2=0x7f855c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:19.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.510+0000 7f855bfff700 1 --2- 172.21.15.142:0/2373026990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f855c101780 0x7f855c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:19.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.510+0000 7f855bfff700 1 -- 172.21.15.142:0/2373026990 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c0095f0 con 0x7f855c100dd0 2024-03-29T04:49:19.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.510+0000 7f855b7fe700 1 --2- 172.21.15.142:0/2373026990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f855c101780 0x7f855c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:49:19.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.510+0000 7f855bfff700 1 --2- 172.21.15.142:0/2373026990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c100dd0 0x7f855c19c670 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f855000b4b0 tx=0x7f855000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:19.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.510+0000 7f85597fa700 1 -- 172.21.15.142:0/2373026990 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8550011800 con 0x7f855c100dd0 2024-03-29T04:49:19.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.510+0000 7f85597fa700 1 -- 172.21.15.142:0/2373026990 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8550011e20 con 0x7f855c100dd0 2024-03-29T04:49:19.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.510+0000 7f85629d3700 1 -- 172.21.15.142:0/2373026990 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855c1969d0 con 0x7f855c100dd0 2024-03-29T04:49:19.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.510+0000 7f85597fa700 1 -- 172.21.15.142:0/2373026990 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f855000f2e0 con 0x7f855c100dd0 2024-03-29T04:49:19.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.510+0000 7f85629d3700 1 -- 172.21.15.142:0/2373026990 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c196f20 con 0x7f855c100dd0 2024-03-29T04:49:19.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.511+0000 7f85629d3700 1 -- 172.21.15.142:0/2373026990 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855c109100 con 0x7f855c100dd0 2024-03-29T04:49:19.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.511+0000 7f85597fa700 1 -- 172.21.15.142:0/2373026990 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7f8550010380 con 0x7f855c100dd0 2024-03-29T04:49:19.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.512+0000 7f85597fa700 1 --2- 172.21.15.142:0/2373026990 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f854406d9e0 0x7f854406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:19.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.512+0000 7f85597fa700 1 -- 172.21.15.142:0/2373026990 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f855008d0f0 con 0x7f855c100dd0 2024-03-29T04:49:19.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.512+0000 7f855b7fe700 1 --2- 172.21.15.142:0/2373026990 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f854406d9e0 0x7f854406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:19.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.512+0000 7f855b7fe700 1 --2- 172.21.15.142:0/2373026990 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f854406d9e0 0x7f854406fea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f854c006010 tx=0x7f854c005960 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:19.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.517+0000 7f85597fa700 1 -- 172.21.15.142:0/2373026990 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f855005a7c0 con 0x7f855c100dd0 2024-03-29T04:49:19.523 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:19 smithi142 ceph-mon[29027]: pgmap v75: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:19.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:19 smithi152 ceph-mon[37177]: pgmap v75: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:19.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.794+0000 7f85629d3700 1 -- 172.21.15.142:0/2373026990 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f855c197220 con 0x7f855c100dd0 2024-03-29T04:49:19.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.795+0000 7f85597fa700 1 -- 172.21.15.142:0/2373026990 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f855005a350 con 0x7f855c100dd0 2024-03-29T04:49:19.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.797+0000 7f85629d3700 1 -- 172.21.15.142:0/2373026990 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f854406d9e0 msgr2=0x7f854406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:19.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.797+0000 7f85629d3700 1 --2- 172.21.15.142:0/2373026990 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f854406d9e0 0x7f854406fea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f854c006010 tx=0x7f854c005960 comp rx=0 tx=0).stop 2024-03-29T04:49:19.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.797+0000 7f85629d3700 1 -- 172.21.15.142:0/2373026990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c100dd0 msgr2=0x7f855c19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:19.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.797+0000 7f85629d3700 1 --2- 172.21.15.142:0/2373026990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c100dd0 0x7f855c19c670 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f855000b4b0 tx=0x7f855000b870 comp rx=0 tx=0).stop 2024-03-29T04:49:19.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.797+0000 7f85629d3700 1 -- 172.21.15.142:0/2373026990 shutdown_connections 2024-03-29T04:49:19.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.797+0000 7f85629d3700 1 --2- 172.21.15.142:0/2373026990 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f854406d9e0 0x7f854406fea0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:19.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.797+0000 7f85629d3700 1 --2- 172.21.15.142:0/2373026990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c100dd0 0x7f855c19c670 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:19.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.797+0000 7f85629d3700 1 --2- 172.21.15.142:0/2373026990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f855c101780 0x7f855c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:19.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.797+0000 7f85629d3700 1 -- 172.21.15.142:0/2373026990 >> 172.21.15.142:0/2373026990 conn(0x7f855c0fc640 msgr2=0x7f855c0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:19.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.797+0000 7f85629d3700 1 -- 172.21.15.142:0/2373026990 shutdown_connections 2024-03-29T04:49:19.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:19.797+0000 7f85629d3700 1 -- 172.21.15.142:0/2373026990 wait complete. 2024-03-29T04:49:19.810 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:49:20.464 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:20 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2373026990' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:49:20.489 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-03-29T04:49:20.698 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:49:20.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:20 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2373026990' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:49:21.508 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:21 smithi142 ceph-mon[29027]: pgmap v76: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:21.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:21 smithi152 ceph-mon[37177]: pgmap v76: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:22.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.430+0000 7f2252a21700 1 -- 172.21.15.142:0/481258336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c105a30 msgr2=0x7f224c105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:22.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.430+0000 7f2252a21700 1 --2- 172.21.15.142:0/481258336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c105a30 0x7f224c105e10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f223c009910 tx=0x7f223c009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:22.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.430+0000 7f2252a21700 1 -- 172.21.15.142:0/481258336 shutdown_connections 2024-03-29T04:49:22.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.430+0000 7f2252a21700 1 --2- 172.21.15.142:0/481258336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f224c0ffa30 0x7f224c0ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:22.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.430+0000 7f2252a21700 1 --2- 172.21.15.142:0/481258336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c105a30 0x7f224c105e10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:22.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.430+0000 7f2252a21700 1 -- 172.21.15.142:0/481258336 >> 172.21.15.142:0/481258336 conn(0x7f224c0fb540 msgr2=0x7f224c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:22.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.430+0000 7f2252a21700 1 -- 172.21.15.142:0/481258336 shutdown_connections 2024-03-29T04:49:22.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.430+0000 7f2252a21700 1 -- 172.21.15.142:0/481258336 wait complete. 2024-03-29T04:49:22.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.431+0000 7f2252a21700 1 Processor -- start 2024-03-29T04:49:22.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.431+0000 7f2252a21700 1 -- start start 2024-03-29T04:49:22.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.431+0000 7f2252a21700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f224c0ffa30 0x7f224c0fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:22.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.431+0000 7f2252a21700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c105a30 0x7f224c0fc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:22.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.431+0000 7f2252a21700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c0fc930 con 0x7f224c105a30 2024-03-29T04:49:22.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.431+0000 7f2252a21700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c0fca70 con 0x7f224c0ffa30 2024-03-29T04:49:22.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.431+0000 7f224b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c105a30 0x7f224c0fc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:22.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.431+0000 7f224b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c105a30 0x7f224c0fc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37806/0 (socket says 172.21.15.142:37806) 2024-03-29T04:49:22.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.431+0000 7f224b7fe700 1 -- 172.21.15.142:0/974637704 learned_addr learned my addr 172.21.15.142:0/974637704 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:22.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.431+0000 7f224b7fe700 1 -- 172.21.15.142:0/974637704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f224c0ffa30 msgr2=0x7f224c0fbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:22.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.431+0000 7f224b7fe700 1 --2- 172.21.15.142:0/974637704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f224c0ffa30 0x7f224c0fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:22.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.431+0000 7f224b7fe700 1 -- 172.21.15.142:0/974637704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c0095f0 con 0x7f224c105a30 2024-03-29T04:49:22.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.432+0000 7f224b7fe700 1 --2- 172.21.15.142:0/974637704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c105a30 0x7f224c0fc3f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f224000b4b0 tx=0x7f224000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:22.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.432+0000 7f22497fa700 1 -- 172.21.15.142:0/974637704 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2240011800 con 0x7f224c105a30 2024-03-29T04:49:22.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.432+0000 7f2252a21700 1 -- 172.21.15.142:0/974637704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224c1a29e0 con 0x7f224c105a30 2024-03-29T04:49:22.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.432+0000 7f22497fa700 1 -- 172.21.15.142:0/974637704 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2240011e20 con 0x7f224c105a30 2024-03-29T04:49:22.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.432+0000 7f22497fa700 1 -- 172.21.15.142:0/974637704 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f224000f2e0 con 0x7f224c105a30 2024-03-29T04:49:22.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.432+0000 7f2252a21700 1 -- 172.21.15.142:0/974637704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224c1a2df0 con 0x7f224c105a30 2024-03-29T04:49:22.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.433+0000 7f2252a21700 1 -- 172.21.15.142:0/974637704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224c109040 con 0x7f224c105a30 2024-03-29T04:49:22.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.433+0000 7f22497fa700 1 -- 172.21.15.142:0/974637704 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7f2240010380 con 0x7f224c105a30 2024-03-29T04:49:22.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.434+0000 7f22497fa700 1 --2- 172.21.15.142:0/974637704 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f223406d9e0 0x7f223406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:22.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.434+0000 7f22497fa700 1 -- 172.21.15.142:0/974637704 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f224008d0f0 con 0x7f224c105a30 2024-03-29T04:49:22.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.434+0000 7f224bfff700 1 --2- 172.21.15.142:0/974637704 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f223406d9e0 0x7f223406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:22.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.434+0000 7f224bfff700 1 --2- 172.21.15.142:0/974637704 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f223406d9e0 0x7f223406fea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f223c00b5c0 tx=0x7f223c005960 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:22.441 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.438+0000 7f22497fa700 1 -- 172.21.15.142:0/974637704 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f2240056c90 con 0x7f224c105a30 2024-03-29T04:49:22.651 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:22.648+0000 7f2252a21700 1 -- 172.21.15.142:0/974637704 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba", "target": ["mon-mgr", ""]}) v1 -- 0x7f224c02cd30 con 0x7f223406d9e0 2024-03-29T04:49:23.571 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:23 smithi142 ceph-mon[29027]: pgmap v77: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:23.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:23 smithi152 ceph-mon[37177]: pgmap v77: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:24.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:24 smithi152 ceph-mon[37177]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:24.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:24 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:49:24.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:24 smithi142 ceph-mon[29027]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:24.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:24 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:49:24.721 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:24.718+0000 7f22497fa700 1 -- 172.21.15.142:0/974637704 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5391 (secure 0 0 0) 0x7f224c02cd30 con 0x7f223406d9e0 2024-03-29T04:49:24.724 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:24.721+0000 7f2252a21700 1 -- 172.21.15.142:0/974637704 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f223406d9e0 msgr2=0x7f223406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:24.724 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:24.721+0000 7f2252a21700 1 --2- 172.21.15.142:0/974637704 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f223406d9e0 0x7f223406fea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f223c00b5c0 tx=0x7f223c005960 comp rx=0 tx=0).stop 2024-03-29T04:49:24.724 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:24.721+0000 7f2252a21700 1 -- 172.21.15.142:0/974637704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c105a30 msgr2=0x7f224c0fc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:24.724 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:24.721+0000 7f2252a21700 1 --2- 172.21.15.142:0/974637704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c105a30 0x7f224c0fc3f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f224000b4b0 tx=0x7f224000b870 comp rx=0 tx=0).stop 2024-03-29T04:49:24.724 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:24.721+0000 7f2252a21700 1 -- 172.21.15.142:0/974637704 shutdown_connections 2024-03-29T04:49:24.724 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:24.721+0000 7f2252a21700 1 --2- 172.21.15.142:0/974637704 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f223406d9e0 0x7f223406fea0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:24.724 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:24.721+0000 7f2252a21700 1 --2- 172.21.15.142:0/974637704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f224c0ffa30 0x7f224c0fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:24.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:24.721+0000 7f2252a21700 1 --2- 172.21.15.142:0/974637704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f224c105a30 0x7f224c0fc3f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:24.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:24.721+0000 7f2252a21700 1 -- 172.21.15.142:0/974637704 >> 172.21.15.142:0/974637704 conn(0x7f224c0fb540 msgr2=0x7f224c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:24.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:24.721+0000 7f2252a21700 1 -- 172.21.15.142:0/974637704 shutdown_connections 2024-03-29T04:49:24.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:24.721+0000 7f2252a21700 1 -- 172.21.15.142:0/974637704 wait complete. 2024-03-29T04:49:24.734 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:49:25.395 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph orch ps' 2024-03-29T04:49:25.603 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:49:25.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:25 smithi142 ceph-mon[29027]: pgmap v78: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:25.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:25 smithi152 ceph-mon[37177]: pgmap v78: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:27.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.202+0000 7f45e4801700 1 -- 172.21.15.142:0/467744491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45e0105a40 msgr2=0x7f45e0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:27.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.202+0000 7f45e4801700 1 --2- 172.21.15.142:0/467744491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45e0105a40 0x7f45e0105e20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009910 tx=0x7f45d4009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:27.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.202+0000 7f45e4801700 1 -- 172.21.15.142:0/467744491 shutdown_connections 2024-03-29T04:49:27.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.202+0000 7f45e4801700 1 --2- 172.21.15.142:0/467744491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45e00ffbd0 0x7f45e0100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:27.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.202+0000 7f45e4801700 1 --2- 172.21.15.142:0/467744491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45e0105a40 0x7f45e0105e20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:27.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.202+0000 7f45e4801700 1 -- 172.21.15.142:0/467744491 >> 172.21.15.142:0/467744491 conn(0x7f45e00755a0 msgr2=0x7f45e00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:27.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.202+0000 7f45e4801700 1 -- 172.21.15.142:0/467744491 shutdown_connections 2024-03-29T04:49:27.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.202+0000 7f45e4801700 1 -- 172.21.15.142:0/467744491 wait complete. 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.203+0000 7f45e4801700 1 Processor -- start 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.203+0000 7f45e4801700 1 -- start start 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.203+0000 7f45e4801700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45e00ffbd0 0x7f45e019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.203+0000 7f45e4801700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45e0105a40 0x7f45e019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.203+0000 7f45e4801700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e019f3d0 con 0x7f45e00ffbd0 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.203+0000 7f45e4801700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e0198880 con 0x7f45e0105a40 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.203+0000 7f45de59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45e00ffbd0 0x7f45e019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.203+0000 7f45de59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45e00ffbd0 0x7f45e019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37824/0 (socket says 172.21.15.142:37824) 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.203+0000 7f45de59c700 1 -- 172.21.15.142:0/2777501420 learned_addr learned my addr 172.21.15.142:0/2777501420 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.204+0000 7f45ddd9b700 1 --2- 172.21.15.142:0/2777501420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45e0105a40 0x7f45e019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.204+0000 7f45de59c700 1 -- 172.21.15.142:0/2777501420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45e0105a40 msgr2=0x7f45e019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.204+0000 7f45de59c700 1 --2- 172.21.15.142:0/2777501420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45e0105a40 0x7f45e019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.204+0000 7f45de59c700 1 -- 172.21.15.142:0/2777501420 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d40095f0 con 0x7f45e00ffbd0 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.204+0000 7f45ddd9b700 1 --2- 172.21.15.142:0/2777501420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45e0105a40 0x7f45e019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:49:27.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.204+0000 7f45de59c700 1 --2- 172.21.15.142:0/2777501420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45e00ffbd0 0x7f45e019e800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f45d4006010 tx=0x7f45d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:27.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.204+0000 7f45cf7fe700 1 -- 172.21.15.142:0/2777501420 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d401d070 con 0x7f45e00ffbd0 2024-03-29T04:49:27.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.204+0000 7f45cf7fe700 1 -- 172.21.15.142:0/2777501420 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45d4004fb0 con 0x7f45e00ffbd0 2024-03-29T04:49:27.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.204+0000 7f45e4801700 1 -- 172.21.15.142:0/2777501420 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e0198b00 con 0x7f45e00ffbd0 2024-03-29T04:49:27.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.204+0000 7f45cf7fe700 1 -- 172.21.15.142:0/2777501420 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d4022ea0 con 0x7f45e00ffbd0 2024-03-29T04:49:27.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.204+0000 7f45e4801700 1 -- 172.21.15.142:0/2777501420 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e0198ff0 con 0x7f45e00ffbd0 2024-03-29T04:49:27.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.205+0000 7f45e4801700 1 -- 172.21.15.142:0/2777501420 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e004ea90 con 0x7f45e00ffbd0 2024-03-29T04:49:27.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.206+0000 7f45cf7fe700 1 -- 172.21.15.142:0/2777501420 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7f45d4022470 con 0x7f45e00ffbd0 2024-03-29T04:49:27.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.206+0000 7f45cf7fe700 1 --2- 172.21.15.142:0/2777501420 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f45c806d9e0 0x7f45c806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:27.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.206+0000 7f45ddd9b700 1 --2- 172.21.15.142:0/2777501420 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f45c806d9e0 0x7f45c806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:27.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.206+0000 7f45cf7fe700 1 -- 172.21.15.142:0/2777501420 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f45d408eb50 con 0x7f45e00ffbd0 2024-03-29T04:49:27.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.207+0000 7f45ddd9b700 1 --2- 172.21.15.142:0/2777501420 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f45c806d9e0 0x7f45c806fea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f45e0104df0 tx=0x7f45d0008040 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:27.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.211+0000 7f45cf7fe700 1 -- 172.21.15.142:0/2777501420 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f45d405c1a0 con 0x7f45e00ffbd0 2024-03-29T04:49:27.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.417+0000 7f45e4801700 1 -- 172.21.15.142:0/2777501420 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f45e0199440 con 0x7f45c806d9e0 2024-03-29T04:49:27.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.427+0000 7f45cf7fe700 1 -- 172.21.15.142:0/2777501420 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f45e0199440 con 0x7f45c806d9e0 2024-03-29T04:49:27.430 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (79s) 50s ago 12m 16.8M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (13m) 50s ago 13m 8070k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (12m) 110s ago 11m 7902k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (13m) 50s ago 13m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (11m) 110s ago 11m 6665k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (56s) 50s ago 12m 51.7M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (7m) 50s ago 7m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (7m) 50s ago 7m 63.3M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (7m) 110s ago 7m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (7m) 110s ago 7m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (2m) 50s ago 14m 504M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (2m) 110s ago 11m 447M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:49:27.431 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (14m) 50s ago 14m 60.7M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:49:27.432 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (11m) 110s ago 11m 44.0M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:49:27.432 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (118s) 50s ago 12m 8526k - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:49:27.432 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (114s) 110s ago 11m 8052k - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:49:27.432 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (10m) 50s ago 10m 218M 1216M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:49:27.432 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (10m) 50s ago 10m 240M 1216M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:49:27.432 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (10m) 50s ago 10m 198M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:49:27.432 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (9m) 110s ago 9m 274M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:49:27.432 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (9m) 110s ago 9m 233M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:49:27.432 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (8m) 110s ago 8m 222M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:49:27.432 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (96s) 50s ago 12m 44.9M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:49:27.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.430+0000 7f45e4801700 1 -- 172.21.15.142:0/2777501420 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f45c806d9e0 msgr2=0x7f45c806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:27.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.430+0000 7f45e4801700 1 --2- 172.21.15.142:0/2777501420 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f45c806d9e0 0x7f45c806fea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f45e0104df0 tx=0x7f45d0008040 comp rx=0 tx=0).stop 2024-03-29T04:49:27.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.430+0000 7f45e4801700 1 -- 172.21.15.142:0/2777501420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45e00ffbd0 msgr2=0x7f45e019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:27.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.430+0000 7f45e4801700 1 --2- 172.21.15.142:0/2777501420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45e00ffbd0 0x7f45e019e800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f45d4006010 tx=0x7f45d400be20 comp rx=0 tx=0).stop 2024-03-29T04:49:27.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.430+0000 7f45e4801700 1 -- 172.21.15.142:0/2777501420 shutdown_connections 2024-03-29T04:49:27.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.430+0000 7f45e4801700 1 --2- 172.21.15.142:0/2777501420 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f45c806d9e0 0x7f45c806fea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:27.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.430+0000 7f45e4801700 1 --2- 172.21.15.142:0/2777501420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45e00ffbd0 0x7f45e019e800 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:27.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.430+0000 7f45e4801700 1 --2- 172.21.15.142:0/2777501420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45e0105a40 0x7f45e019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:27.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.430+0000 7f45e4801700 1 -- 172.21.15.142:0/2777501420 >> 172.21.15.142:0/2777501420 conn(0x7f45e00755a0 msgr2=0x7f45e0068510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:27.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.430+0000 7f45e4801700 1 -- 172.21.15.142:0/2777501420 shutdown_connections 2024-03-29T04:49:27.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:27.430+0000 7f45e4801700 1 -- 172.21.15.142:0/2777501420 wait complete. 2024-03-29T04:49:27.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:27 smithi142 ceph-mon[29027]: pgmap v79: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:27.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:27 smithi152 ceph-mon[37177]: pgmap v79: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:28.122 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-29T04:49:28.122 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T04:49:28.123 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-03-29T04:49:28.344 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:49:29.439 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:29 smithi142 ceph-mon[29027]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:29.439 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:29 smithi142 ceph-mon[29027]: pgmap v80: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:29.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:29 smithi152 ceph-mon[37177]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:29.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:29 smithi152 ceph-mon[37177]: pgmap v80: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:29.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.993+0000 7fb4051c4700 1 -- 172.21.15.142:0/83048225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb400105a40 msgr2=0x7fb400105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:29.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.993+0000 7fb4051c4700 1 --2- 172.21.15.142:0/83048225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb400105a40 0x7fb400105e20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009910 tx=0x7fb3f4009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:29.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.993+0000 7fb4051c4700 1 -- 172.21.15.142:0/83048225 shutdown_connections 2024-03-29T04:49:29.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.993+0000 7fb4051c4700 1 --2- 172.21.15.142:0/83048225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4000ffa40 0x7fb4000ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:29.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.993+0000 7fb4051c4700 1 --2- 172.21.15.142:0/83048225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb400105a40 0x7fb400105e20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.993+0000 7fb4051c4700 1 -- 172.21.15.142:0/83048225 >> 172.21.15.142:0/83048225 conn(0x7fb4000fb550 msgr2=0x7fb4000fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.993+0000 7fb4051c4700 1 -- 172.21.15.142:0/83048225 shutdown_connections 2024-03-29T04:49:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.993+0000 7fb4051c4700 1 -- 172.21.15.142:0/83048225 wait complete. 2024-03-29T04:49:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.994+0000 7fb4051c4700 1 Processor -- start 2024-03-29T04:49:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.994+0000 7fb4051c4700 1 -- start start 2024-03-29T04:49:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.994+0000 7fb4051c4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4000ffa40 0x7fb40019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.994+0000 7fb4051c4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb400105a40 0x7fb40019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.994+0000 7fb4051c4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40019d150 con 0x7fb400105a40 2024-03-29T04:49:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.994+0000 7fb4051c4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb400196600 con 0x7fb4000ffa40 2024-03-29T04:49:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.994+0000 7fb3fe59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb400105a40 0x7fb40019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.994+0000 7fb3fe59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb400105a40 0x7fb40019cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58978/0 (socket says 172.21.15.142:58978) 2024-03-29T04:49:29.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.994+0000 7fb3fe59c700 1 -- 172.21.15.142:0/18044699 learned_addr learned my addr 172.21.15.142:0/18044699 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:30.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.995+0000 7fb3fed9d700 1 --2- 172.21.15.142:0/18044699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4000ffa40 0x7fb40019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:30.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.995+0000 7fb3fe59c700 1 -- 172.21.15.142:0/18044699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4000ffa40 msgr2=0x7fb40019c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:30.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.995+0000 7fb3fe59c700 1 --2- 172.21.15.142:0/18044699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4000ffa40 0x7fb40019c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:30.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.995+0000 7fb3fe59c700 1 -- 172.21.15.142:0/18044699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f40095f0 con 0x7fb400105a40 2024-03-29T04:49:30.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.995+0000 7fb3fed9d700 1 --2- 172.21.15.142:0/18044699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4000ffa40 0x7fb40019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:49:30.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.995+0000 7fb3fe59c700 1 --2- 172.21.15.142:0/18044699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb400105a40 0x7fb40019cac0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000d8d0 tx=0x7fb3f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:30.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.995+0000 7fb3e7fff700 1 -- 172.21.15.142:0/18044699 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3f0009940 con 0x7fb400105a40 2024-03-29T04:49:30.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.995+0000 7fb4051c4700 1 -- 172.21.15.142:0/18044699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4001968e0 con 0x7fb400105a40 2024-03-29T04:49:30.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.995+0000 7fb3e7fff700 1 -- 172.21.15.142:0/18044699 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3f0010460 con 0x7fb400105a40 2024-03-29T04:49:30.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.995+0000 7fb3e7fff700 1 -- 172.21.15.142:0/18044699 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3f000f5f0 con 0x7fb400105a40 2024-03-29T04:49:30.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.995+0000 7fb4051c4700 1 -- 172.21.15.142:0/18044699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb400196e30 con 0x7fb400105a40 2024-03-29T04:49:30.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.996+0000 7fb4051c4700 1 -- 172.21.15.142:0/18044699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb400109050 con 0x7fb400105a40 2024-03-29T04:49:30.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.997+0000 7fb3e7fff700 1 -- 172.21.15.142:0/18044699 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7fb3f00105d0 con 0x7fb400105a40 2024-03-29T04:49:30.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.997+0000 7fb3e7fff700 1 --2- 172.21.15.142:0/18044699 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fb3e806d9e0 0x7fb3e806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:30.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.997+0000 7fb3e7fff700 1 -- 172.21.15.142:0/18044699 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb3f008d4a0 con 0x7fb400105a40 2024-03-29T04:49:30.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.997+0000 7fb3fed9d700 1 --2- 172.21.15.142:0/18044699 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fb3e806d9e0 0x7fb3e806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:30.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:29.998+0000 7fb3fed9d700 1 --2- 172.21.15.142:0/18044699 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fb3e806d9e0 0x7fb3e806fea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4005b40 tx=0x7fb3f401a040 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:30.005 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.002+0000 7fb3e7fff700 1 -- 172.21.15.142:0/18044699 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fb3f005aac0 con 0x7fb400105a40 2024-03-29T04:49:30.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.196+0000 7fb4051c4700 1 -- 172.21.15.142:0/18044699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fb4001970a0 con 0x7fb400105a40 2024-03-29T04:49:30.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.201+0000 7fb3e7fff700 1 -- 172.21.15.142:0/18044699 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v44)=0 v44) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb3f005a650 con 0x7fb400105a40 2024-03-29T04:49:30.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.203+0000 7fb4051c4700 1 -- 172.21.15.142:0/18044699 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fb3e806d9e0 msgr2=0x7fb3e806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:30.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.203+0000 7fb4051c4700 1 --2- 172.21.15.142:0/18044699 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fb3e806d9e0 0x7fb3e806fea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4005b40 tx=0x7fb3f401a040 comp rx=0 tx=0).stop 2024-03-29T04:49:30.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.203+0000 7fb4051c4700 1 -- 172.21.15.142:0/18044699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb400105a40 msgr2=0x7fb40019cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:30.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.203+0000 7fb4051c4700 1 --2- 172.21.15.142:0/18044699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb400105a40 0x7fb40019cac0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000d8d0 tx=0x7fb3f000dc90 comp rx=0 tx=0).stop 2024-03-29T04:49:30.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.203+0000 7fb4051c4700 1 -- 172.21.15.142:0/18044699 shutdown_connections 2024-03-29T04:49:30.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.203+0000 7fb4051c4700 1 --2- 172.21.15.142:0/18044699 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fb3e806d9e0 0x7fb3e806fea0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:30.206 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.203+0000 7fb4051c4700 1 --2- 172.21.15.142:0/18044699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4000ffa40 0x7fb40019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:30.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.203+0000 7fb4051c4700 1 --2- 172.21.15.142:0/18044699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb400105a40 0x7fb40019cac0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:30.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.203+0000 7fb4051c4700 1 -- 172.21.15.142:0/18044699 >> 172.21.15.142:0/18044699 conn(0x7fb4000fb550 msgr2=0x7fb400069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:30.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.203+0000 7fb4051c4700 1 -- 172.21.15.142:0/18044699 shutdown_connections 2024-03-29T04:49:30.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:30.203+0000 7fb4051c4700 1 -- 172.21.15.142:0/18044699 wait complete. 2024-03-29T04:49:30.894 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-29T04:49:30.894 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T04:49:30.895 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-29T04:49:31.104 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:49:31.467 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:31 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/18044699' entity='client.admin' 2024-03-29T04:49:31.467 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:31 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:49:31.467 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:31 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:49:31.467 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:31 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:49:31.467 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:31 smithi142 ceph-mon[29027]: pgmap v81: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:31.467 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:31 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:49:31.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:31 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/18044699' entity='client.admin' 2024-03-29T04:49:31.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:31 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:49:31.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:31 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:49:31.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:31 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:49:31.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:31 smithi152 ceph-mon[37177]: pgmap v81: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:31.543 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:31 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:49:32.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.813+0000 7f4cb6d09700 1 -- 172.21.15.142:0/3564886505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb0100de0 msgr2=0x7f4cb01011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:32.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.813+0000 7f4cb6d09700 1 --2- 172.21.15.142:0/3564886505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb0100de0 0x7f4cb01011c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009910 tx=0x7f4ca4009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:32.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.813+0000 7f4cb6d09700 1 -- 172.21.15.142:0/3564886505 shutdown_connections 2024-03-29T04:49:32.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.813+0000 7f4cb6d09700 1 --2- 172.21.15.142:0/3564886505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cb0101790 0x7f4cb01057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:32.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.813+0000 7f4cb6d09700 1 --2- 172.21.15.142:0/3564886505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb0100de0 0x7f4cb01011c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:32.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.813+0000 7f4cb6d09700 1 -- 172.21.15.142:0/3564886505 >> 172.21.15.142:0/3564886505 conn(0x7f4cb00fc670 msgr2=0x7f4cb00fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:32.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.813+0000 7f4cb6d09700 1 -- 172.21.15.142:0/3564886505 shutdown_connections 2024-03-29T04:49:32.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.813+0000 7f4cb6d09700 1 -- 172.21.15.142:0/3564886505 wait complete. 2024-03-29T04:49:32.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.814+0000 7f4cb6d09700 1 Processor -- start 2024-03-29T04:49:32.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.814+0000 7f4cb6d09700 1 -- start start 2024-03-29T04:49:32.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.814+0000 7f4cb6d09700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cb0100de0 0x7f4cb0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:32.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.814+0000 7f4cb6d09700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb0101790 0x7f4cb00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:32.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.814+0000 7f4cb6d09700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb0073b10 con 0x7f4cb0101790 2024-03-29T04:49:32.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.814+0000 7f4cb6d09700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb0073c50 con 0x7f4cb0100de0 2024-03-29T04:49:32.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4caffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb0101790 0x7f4cb00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:32.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4caffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb0101790 0x7f4cb00735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59002/0 (socket says 172.21.15.142:59002) 2024-03-29T04:49:32.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4caffff700 1 -- 172.21.15.142:0/379738764 learned_addr learned my addr 172.21.15.142:0/379738764 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:32.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4cb4aa5700 1 --2- 172.21.15.142:0/379738764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cb0100de0 0x7f4cb0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:32.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4caffff700 1 -- 172.21.15.142:0/379738764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cb0100de0 msgr2=0x7f4cb0073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:32.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4caffff700 1 --2- 172.21.15.142:0/379738764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cb0100de0 0x7f4cb0073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:32.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4caffff700 1 -- 172.21.15.142:0/379738764 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca40095f0 con 0x7f4cb0101790 2024-03-29T04:49:32.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4caffff700 1 --2- 172.21.15.142:0/379738764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb0101790 0x7f4cb00735d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00c960 tx=0x7f4c9c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:32.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4cadffb700 1 -- 172.21.15.142:0/379738764 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9c007a70 con 0x7f4cb0101790 2024-03-29T04:49:32.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4cadffb700 1 -- 172.21.15.142:0/379738764 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c9c007bd0 con 0x7f4cb0101790 2024-03-29T04:49:32.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4cb6d09700 1 -- 172.21.15.142:0/379738764 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb01a2930 con 0x7f4cb0101790 2024-03-29T04:49:32.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4cadffb700 1 -- 172.21.15.142:0/379738764 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9c0186c0 con 0x7f4cb0101790 2024-03-29T04:49:32.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.815+0000 7f4cb6d09700 1 -- 172.21.15.142:0/379738764 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb01a2c90 con 0x7f4cb0101790 2024-03-29T04:49:32.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.816+0000 7f4cb6d09700 1 -- 172.21.15.142:0/379738764 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb0109180 con 0x7f4cb0101790 2024-03-29T04:49:32.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.817+0000 7f4cadffb700 1 -- 172.21.15.142:0/379738764 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7f4c9c00faa0 con 0x7f4cb0101790 2024-03-29T04:49:32.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.817+0000 7f4cadffb700 1 --2- 172.21.15.142:0/379738764 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f4ca006d9e0 0x7f4ca006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:32.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.817+0000 7f4cadffb700 1 -- 172.21.15.142:0/379738764 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4c9c08d600 con 0x7f4cb0101790 2024-03-29T04:49:32.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.817+0000 7f4cb4aa5700 1 --2- 172.21.15.142:0/379738764 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f4ca006d9e0 0x7f4ca006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:32.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.818+0000 7f4cb4aa5700 1 --2- 172.21.15.142:0/379738764 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f4ca006d9e0 0x7f4ca006fea0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4006010 tx=0x7f4ca401a040 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:32.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:32.822+0000 7f4cadffb700 1 -- 172.21.15.142:0/379738764 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f4c9c05aba0 con 0x7f4cb0101790 2024-03-29T04:49:33.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.016+0000 7f4cb6d09700 1 -- 172.21.15.142:0/379738764 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f4cb004ea90 con 0x7f4cb0101790 2024-03-29T04:49:33.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.016+0000 7f4cadffb700 1 -- 172.21.15.142:0/379738764 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v44)=0 v44) v1 ==== 155+0+0 (secure 0 0 0) 0x7f4c9c05a730 con 0x7f4cb0101790 2024-03-29T04:49:33.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.018+0000 7f4cb6d09700 1 -- 172.21.15.142:0/379738764 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f4ca006d9e0 msgr2=0x7f4ca006fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:33.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.018+0000 7f4cb6d09700 1 --2- 172.21.15.142:0/379738764 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f4ca006d9e0 0x7f4ca006fea0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4006010 tx=0x7f4ca401a040 comp rx=0 tx=0).stop 2024-03-29T04:49:33.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.018+0000 7f4cb6d09700 1 -- 172.21.15.142:0/379738764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb0101790 msgr2=0x7f4cb00735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:33.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.018+0000 7f4cb6d09700 1 --2- 172.21.15.142:0/379738764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb0101790 0x7f4cb00735d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00c960 tx=0x7f4c9c00cd20 comp rx=0 tx=0).stop 2024-03-29T04:49:33.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.018+0000 7f4cb6d09700 1 -- 172.21.15.142:0/379738764 shutdown_connections 2024-03-29T04:49:33.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.018+0000 7f4cb6d09700 1 --2- 172.21.15.142:0/379738764 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f4ca006d9e0 0x7f4ca006fea0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:33.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.018+0000 7f4cb6d09700 1 --2- 172.21.15.142:0/379738764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cb0100de0 0x7f4cb0073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:33.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.018+0000 7f4cb6d09700 1 --2- 172.21.15.142:0/379738764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cb0101790 0x7f4cb00735d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:33.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.018+0000 7f4cb6d09700 1 -- 172.21.15.142:0/379738764 >> 172.21.15.142:0/379738764 conn(0x7f4cb00fc670 msgr2=0x7f4cb00fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:33.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.019+0000 7f4cb6d09700 1 -- 172.21.15.142:0/379738764 shutdown_connections 2024-03-29T04:49:33.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:33.019+0000 7f4cb6d09700 1 -- 172.21.15.142:0/379738764 wait complete. 2024-03-29T04:49:33.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:33 smithi142 ceph-mon[29027]: pgmap v82: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:33.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-29T04:49:33.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:33 smithi152 ceph-mon[37177]: pgmap v82: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:33.896 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:49:34.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:34 smithi142 ceph-mon[29027]: pgmap v83: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:35.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:34 smithi152 ceph-mon[37177]: pgmap v83: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:35.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.545+0000 7f536a02d700 1 -- 172.21.15.142:0/479755722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5364100dd0 msgr2=0x7f53641011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:35.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.545+0000 7f536a02d700 1 --2- 172.21.15.142:0/479755722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5364100dd0 0x7f53641011b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5354009910 tx=0x7f5354009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:35.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.545+0000 7f536a02d700 1 -- 172.21.15.142:0/479755722 shutdown_connections 2024-03-29T04:49:35.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.545+0000 7f536a02d700 1 --2- 172.21.15.142:0/479755722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5364101780 0x7f53641057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:35.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.545+0000 7f536a02d700 1 --2- 172.21.15.142:0/479755722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5364100dd0 0x7f53641011b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:35.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.545+0000 7f536a02d700 1 -- 172.21.15.142:0/479755722 >> 172.21.15.142:0/479755722 conn(0x7f53640fc640 msgr2=0x7f53640fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:35.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.545+0000 7f536a02d700 1 -- 172.21.15.142:0/479755722 shutdown_connections 2024-03-29T04:49:35.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.545+0000 7f536a02d700 1 -- 172.21.15.142:0/479755722 wait complete. 2024-03-29T04:49:35.550 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.546+0000 7f536a02d700 1 Processor -- start 2024-03-29T04:49:35.550 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.546+0000 7f536a02d700 1 -- start start 2024-03-29T04:49:35.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.546+0000 7f536a02d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5364100dd0 0x7f536419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:35.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.546+0000 7f536a02d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5364101780 0x7f536419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:35.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.546+0000 7f536a02d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536419d1c0 con 0x7f5364100dd0 2024-03-29T04:49:35.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.546+0000 7f536a02d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5364196670 con 0x7f5364101780 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.547+0000 7f53637fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5364100dd0 0x7f536419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.547+0000 7f53637fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5364100dd0 0x7f536419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59010/0 (socket says 172.21.15.142:59010) 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.547+0000 7f53637fe700 1 -- 172.21.15.142:0/676346287 learned_addr learned my addr 172.21.15.142:0/676346287 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.547+0000 7f535affd700 1 --2- 172.21.15.142:0/676346287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5364101780 0x7f536419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.547+0000 7f53637fe700 1 -- 172.21.15.142:0/676346287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5364101780 msgr2=0x7f536419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.547+0000 7f53637fe700 1 --2- 172.21.15.142:0/676346287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5364101780 0x7f536419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.547+0000 7f53637fe700 1 -- 172.21.15.142:0/676346287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53540095f0 con 0x7f5364100dd0 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.547+0000 7f535affd700 1 --2- 172.21.15.142:0/676346287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5364101780 0x7f536419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.547+0000 7f53637fe700 1 --2- 172.21.15.142:0/676346287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5364100dd0 0x7f536419c5f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5354006010 tx=0x7f535400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.547+0000 7f53617fa700 1 -- 172.21.15.142:0/676346287 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f535401d070 con 0x7f5364100dd0 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.547+0000 7f53617fa700 1 -- 172.21.15.142:0/676346287 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5354004fb0 con 0x7f5364100dd0 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.547+0000 7f536a02d700 1 -- 172.21.15.142:0/676346287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53641968f0 con 0x7f5364100dd0 2024-03-29T04:49:35.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.548+0000 7f53617fa700 1 -- 172.21.15.142:0/676346287 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5354022ea0 con 0x7f5364100dd0 2024-03-29T04:49:35.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.548+0000 7f536a02d700 1 -- 172.21.15.142:0/676346287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5364196de0 con 0x7f5364100dd0 2024-03-29T04:49:35.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.549+0000 7f53617fa700 1 -- 172.21.15.142:0/676346287 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7f5354004af0 con 0x7f5364100dd0 2024-03-29T04:49:35.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.549+0000 7f536a02d700 1 -- 172.21.15.142:0/676346287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5364109170 con 0x7f5364100dd0 2024-03-29T04:49:35.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.549+0000 7f53617fa700 1 --2- 172.21.15.142:0/676346287 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f535006d7d0 0x7f535006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:35.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.549+0000 7f53617fa700 1 -- 172.21.15.142:0/676346287 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f535408ebe0 con 0x7f5364100dd0 2024-03-29T04:49:35.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.550+0000 7f535affd700 1 --2- 172.21.15.142:0/676346287 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f535006d7d0 0x7f535006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:35.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.550+0000 7f535affd700 1 --2- 172.21.15.142:0/676346287 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f535006d7d0 0x7f535006fc90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f53641059c0 tx=0x7f534c008040 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:35.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.554+0000 7f53617fa700 1 -- 172.21.15.142:0/676346287 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f535405c180 con 0x7f5364100dd0 2024-03-29T04:49:35.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.749+0000 7f536a02d700 1 -- 172.21.15.142:0/676346287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f5364108b50 con 0x7f5364100dd0 2024-03-29T04:49:35.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.749+0000 7f53617fa700 1 -- 172.21.15.142:0/676346287 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v44)=0 v44) v1 ==== 163+0+0 (secure 0 0 0) 0x7f535405bd10 con 0x7f5364100dd0 2024-03-29T04:49:35.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.752+0000 7f536a02d700 1 -- 172.21.15.142:0/676346287 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f535006d7d0 msgr2=0x7f535006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:35.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.752+0000 7f536a02d700 1 --2- 172.21.15.142:0/676346287 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f535006d7d0 0x7f535006fc90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f53641059c0 tx=0x7f534c008040 comp rx=0 tx=0).stop 2024-03-29T04:49:35.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.752+0000 7f536a02d700 1 -- 172.21.15.142:0/676346287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5364100dd0 msgr2=0x7f536419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:35.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.752+0000 7f536a02d700 1 --2- 172.21.15.142:0/676346287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5364100dd0 0x7f536419c5f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5354006010 tx=0x7f535400be20 comp rx=0 tx=0).stop 2024-03-29T04:49:35.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.752+0000 7f536a02d700 1 -- 172.21.15.142:0/676346287 shutdown_connections 2024-03-29T04:49:35.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.752+0000 7f536a02d700 1 --2- 172.21.15.142:0/676346287 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f535006d7d0 0x7f535006fc90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:35.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.752+0000 7f536a02d700 1 --2- 172.21.15.142:0/676346287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5364100dd0 0x7f536419c5f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:35.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.752+0000 7f536a02d700 1 --2- 172.21.15.142:0/676346287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5364101780 0x7f536419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:35.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.752+0000 7f536a02d700 1 -- 172.21.15.142:0/676346287 >> 172.21.15.142:0/676346287 conn(0x7f53640fc640 msgr2=0x7f53640fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:35.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.752+0000 7f536a02d700 1 -- 172.21.15.142:0/676346287 shutdown_connections 2024-03-29T04:49:35.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:35.752+0000 7f536a02d700 1 -- 172.21.15.142:0/676346287 wait complete. 2024-03-29T04:49:36.447 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-29T04:49:36.656 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:49:37.374 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:37 smithi142 ceph-mon[29027]: pgmap v84: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:37.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:37 smithi152 ceph-mon[37177]: pgmap v84: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:38.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.366+0000 7fafe1556700 1 -- 172.21.15.142:0/153728236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafdc0ffa50 msgr2=0x7fafdc0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:38.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.366+0000 7fafe1556700 1 --2- 172.21.15.142:0/153728236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafdc0ffa50 0x7fafdc0ffed0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009910 tx=0x7fafc4009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:38.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.366+0000 7fafe1556700 1 -- 172.21.15.142:0/153728236 shutdown_connections 2024-03-29T04:49:38.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.366+0000 7fafe1556700 1 --2- 172.21.15.142:0/153728236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafdc0ffa50 0x7fafdc0ffed0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:38.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.366+0000 7fafe1556700 1 --2- 172.21.15.142:0/153728236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafdc105a50 0x7fafdc105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:38.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.366+0000 7fafe1556700 1 -- 172.21.15.142:0/153728236 >> 172.21.15.142:0/153728236 conn(0x7fafdc0fb540 msgr2=0x7fafdc0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:38.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.366+0000 7fafe1556700 1 -- 172.21.15.142:0/153728236 shutdown_connections 2024-03-29T04:49:38.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.366+0000 7fafe1556700 1 -- 172.21.15.142:0/153728236 wait complete. 2024-03-29T04:49:38.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.367+0000 7fafe1556700 1 Processor -- start 2024-03-29T04:49:38.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.367+0000 7fafe1556700 1 -- start start 2024-03-29T04:49:38.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.367+0000 7fafe1556700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafdc0ffa50 0x7fafdc195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.367+0000 7fafe1556700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafdc105a50 0x7fafdc195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.367+0000 7fafe1556700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc196280 con 0x7fafdc105a50 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.367+0000 7fafe1556700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc19a010 con 0x7fafdc0ffa50 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.367+0000 7fafda7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafdc105a50 0x7fafdc195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.367+0000 7fafda7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafdc105a50 0x7fafdc195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59024/0 (socket says 172.21.15.142:59024) 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.367+0000 7fafda7fc700 1 -- 172.21.15.142:0/3026583053 learned_addr learned my addr 172.21.15.142:0/3026583053 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.367+0000 7fafdaffd700 1 --2- 172.21.15.142:0/3026583053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafdc0ffa50 0x7fafdc195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.368+0000 7fafda7fc700 1 -- 172.21.15.142:0/3026583053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafdc0ffa50 msgr2=0x7fafdc195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.368+0000 7fafda7fc700 1 --2- 172.21.15.142:0/3026583053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafdc0ffa50 0x7fafdc195660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.368+0000 7fafda7fc700 1 -- 172.21.15.142:0/3026583053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40095f0 con 0x7fafdc105a50 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.368+0000 7fafdaffd700 1 --2- 172.21.15.142:0/3026583053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafdc0ffa50 0x7fafdc195660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.368+0000 7fafda7fc700 1 --2- 172.21.15.142:0/3026583053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafdc105a50 0x7fafdc195ba0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fafc4000c00 tx=0x7fafc400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:38.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.368+0000 7fafd3fff700 1 -- 172.21.15.142:0/3026583053 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafc401d070 con 0x7fafdc105a50 2024-03-29T04:49:38.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.368+0000 7fafd3fff700 1 -- 172.21.15.142:0/3026583053 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafc4005dc0 con 0x7fafdc105a50 2024-03-29T04:49:38.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.368+0000 7fafe1556700 1 -- 172.21.15.142:0/3026583053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc19a290 con 0x7fafdc105a50 2024-03-29T04:49:38.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.368+0000 7fafd3fff700 1 -- 172.21.15.142:0/3026583053 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafc4017400 con 0x7fafdc105a50 2024-03-29T04:49:38.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.368+0000 7fafe1556700 1 -- 172.21.15.142:0/3026583053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc19a780 con 0x7fafdc105a50 2024-03-29T04:49:38.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.369+0000 7fafe1556700 1 -- 172.21.15.142:0/3026583053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc107fe0 con 0x7fafdc105a50 2024-03-29T04:49:38.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.369+0000 7fafd3fff700 1 -- 172.21.15.142:0/3026583053 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7fafc4003a60 con 0x7fafdc105a50 2024-03-29T04:49:38.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.370+0000 7fafd3fff700 1 --2- 172.21.15.142:0/3026583053 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fafc806d9e0 0x7fafc806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:38.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.370+0000 7fafd3fff700 1 -- 172.21.15.142:0/3026583053 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fafc408eb50 con 0x7fafdc105a50 2024-03-29T04:49:38.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.370+0000 7fafdaffd700 1 --2- 172.21.15.142:0/3026583053 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fafc806d9e0 0x7fafc806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:38.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.370+0000 7fafdaffd700 1 --2- 172.21.15.142:0/3026583053 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fafc806d9e0 0x7fafc806fea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009dd0 tx=0x7fafcc009040 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:38.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.374+0000 7fafd3fff700 1 -- 172.21.15.142:0/3026583053 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fafc40585c0 con 0x7fafdc105a50 2024-03-29T04:49:38.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.567+0000 7fafe1556700 1 -- 172.21.15.142:0/3026583053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fafdc04ea90 con 0x7fafdc105a50 2024-03-29T04:49:38.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.567+0000 7fafd3fff700 1 -- 172.21.15.142:0/3026583053 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v44)=0 v44) v1 ==== 135+0+0 (secure 0 0 0) 0x7fafc405bc80 con 0x7fafdc105a50 2024-03-29T04:49:38.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.569+0000 7fafe1556700 1 -- 172.21.15.142:0/3026583053 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fafc806d9e0 msgr2=0x7fafc806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:38.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.569+0000 7fafe1556700 1 --2- 172.21.15.142:0/3026583053 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fafc806d9e0 0x7fafc806fea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009dd0 tx=0x7fafcc009040 comp rx=0 tx=0).stop 2024-03-29T04:49:38.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.569+0000 7fafe1556700 1 -- 172.21.15.142:0/3026583053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafdc105a50 msgr2=0x7fafdc195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:38.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.569+0000 7fafe1556700 1 --2- 172.21.15.142:0/3026583053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafdc105a50 0x7fafdc195ba0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fafc4000c00 tx=0x7fafc400be20 comp rx=0 tx=0).stop 2024-03-29T04:49:38.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.570+0000 7fafe1556700 1 -- 172.21.15.142:0/3026583053 shutdown_connections 2024-03-29T04:49:38.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.570+0000 7fafe1556700 1 --2- 172.21.15.142:0/3026583053 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fafc806d9e0 0x7fafc806fea0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:38.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.570+0000 7fafe1556700 1 --2- 172.21.15.142:0/3026583053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafdc0ffa50 0x7fafdc195660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:38.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.570+0000 7fafe1556700 1 --2- 172.21.15.142:0/3026583053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafdc105a50 0x7fafdc195ba0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:38.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.570+0000 7fafe1556700 1 -- 172.21.15.142:0/3026583053 >> 172.21.15.142:0/3026583053 conn(0x7fafdc0fb540 msgr2=0x7fafdc0fbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:38.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.570+0000 7fafe1556700 1 -- 172.21.15.142:0/3026583053 shutdown_connections 2024-03-29T04:49:38.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:38.570+0000 7fafe1556700 1 -- 172.21.15.142:0/3026583053 wait complete. 2024-03-29T04:49:39.231 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-29T04:49:39.442 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:49:39.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:39 smithi142 ceph-mon[29027]: pgmap v85: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:39.560 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:39 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:49:39.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:39 smithi152 ceph-mon[37177]: pgmap v85: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:39.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:39 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:49:41.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.095+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/1939630814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880ffee0 msgr2=0x7f8d88105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:41.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.095+0000 7f8d8f2ed700 1 --2- 172.21.15.142:0/1939630814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880ffee0 0x7f8d88105810 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009910 tx=0x7f8d84009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:41.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.095+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/1939630814 shutdown_connections 2024-03-29T04:49:41.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.095+0000 7f8d8f2ed700 1 --2- 172.21.15.142:0/1939630814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880ffee0 0x7f8d88105810 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:41.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.095+0000 7f8d8f2ed700 1 --2- 172.21.15.142:0/1939630814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d880ff530 0x7f8d880ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:41.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.095+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/1939630814 >> 172.21.15.142:0/1939630814 conn(0x7f8d88074d50 msgr2=0x7f8d88075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:41.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.095+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/1939630814 shutdown_connections 2024-03-29T04:49:41.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.095+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/1939630814 wait complete. 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.095+0000 7f8d8f2ed700 1 Processor -- start 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.095+0000 7f8d8f2ed700 1 -- start start 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8f2ed700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880ff530 0x7f8d8819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8f2ed700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d880ffee0 0x7f8d8819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8f2ed700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8819d160 con 0x7f8d880ff530 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8f2ed700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d88196610 con 0x7f8d880ffee0 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8d089700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880ff530 0x7f8d8819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8d089700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880ff530 0x7f8d8819c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55576/0 (socket says 172.21.15.142:55576) 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8d089700 1 -- 172.21.15.142:0/2364996857 learned_addr learned my addr 172.21.15.142:0/2364996857 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8c888700 1 --2- 172.21.15.142:0/2364996857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d880ffee0 0x7f8d8819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8d089700 1 -- 172.21.15.142:0/2364996857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d880ffee0 msgr2=0x7f8d8819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8d089700 1 --2- 172.21.15.142:0/2364996857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d880ffee0 0x7f8d8819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:41.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8d089700 1 -- 172.21.15.142:0/2364996857 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840095f0 con 0x7f8d880ff530 2024-03-29T04:49:41.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8d089700 1 --2- 172.21.15.142:0/2364996857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880ff530 0x7f8d8819c590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800c8a0 tx=0x7f8d7800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:41.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d7e7fc700 1 -- 172.21.15.142:0/2364996857 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d7800cea0 con 0x7f8d880ff530 2024-03-29T04:49:41.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d7e7fc700 1 -- 172.21.15.142:0/2364996857 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d78004830 con 0x7f8d880ff530 2024-03-29T04:49:41.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d7e7fc700 1 -- 172.21.15.142:0/2364996857 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d780055b0 con 0x7f8d880ff530 2024-03-29T04:49:41.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/2364996857 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d881968f0 con 0x7f8d880ff530 2024-03-29T04:49:41.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.096+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/2364996857 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d88196de0 con 0x7f8d880ff530 2024-03-29T04:49:41.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.097+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/2364996857 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d881091a0 con 0x7f8d880ff530 2024-03-29T04:49:41.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.098+0000 7f8d7e7fc700 1 -- 172.21.15.142:0/2364996857 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7f8d780049a0 con 0x7f8d880ff530 2024-03-29T04:49:41.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.098+0000 7f8d7e7fc700 1 --2- 172.21.15.142:0/2364996857 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8d7406d9e0 0x7f8d7406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:41.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.098+0000 7f8d7e7fc700 1 -- 172.21.15.142:0/2364996857 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8d7808d1b0 con 0x7f8d880ff530 2024-03-29T04:49:41.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.099+0000 7f8d8c888700 1 --2- 172.21.15.142:0/2364996857 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8d7406d9e0 0x7f8d7406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:41.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.099+0000 7f8d8c888700 1 --2- 172.21.15.142:0/2364996857 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8d7406d9e0 0x7f8d7406fea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d84006010 tx=0x7f8d84005960 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:41.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.103+0000 7f8d7e7fc700 1 -- 172.21.15.142:0/2364996857 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f8d7805a7d0 con 0x7f8d880ff530 2024-03-29T04:49:41.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.315+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/2364996857 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d88197190 con 0x7f8d7406d9e0 2024-03-29T04:49:41.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.320+0000 7f8d7e7fc700 1 -- 172.21.15.142:0/2364996857 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f8d88197190 con 0x7f8d7406d9e0 2024-03-29T04:49:41.323 INFO:teuthology.orchestra.run.smithi142.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:49:41.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.323+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/2364996857 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8d7406d9e0 msgr2=0x7f8d7406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:41.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.323+0000 7f8d8f2ed700 1 --2- 172.21.15.142:0/2364996857 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8d7406d9e0 0x7f8d7406fea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d84006010 tx=0x7f8d84005960 comp rx=0 tx=0).stop 2024-03-29T04:49:41.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.323+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/2364996857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880ff530 msgr2=0x7f8d8819c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:41.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.323+0000 7f8d8f2ed700 1 --2- 172.21.15.142:0/2364996857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880ff530 0x7f8d8819c590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800c8a0 tx=0x7f8d7800cc60 comp rx=0 tx=0).stop 2024-03-29T04:49:41.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.323+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/2364996857 shutdown_connections 2024-03-29T04:49:41.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.323+0000 7f8d8f2ed700 1 --2- 172.21.15.142:0/2364996857 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f8d7406d9e0 0x7f8d7406fea0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:41.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.323+0000 7f8d8f2ed700 1 --2- 172.21.15.142:0/2364996857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880ff530 0x7f8d8819c590 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:41.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.323+0000 7f8d8f2ed700 1 --2- 172.21.15.142:0/2364996857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d880ffee0 0x7f8d8819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:41.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.323+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/2364996857 >> 172.21.15.142:0/2364996857 conn(0x7f8d88074d50 msgr2=0x7f8d880fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:41.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.323+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/2364996857 shutdown_connections 2024-03-29T04:49:41.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:41.323+0000 7f8d8f2ed700 1 -- 172.21.15.142:0/2364996857 wait complete. 2024-03-29T04:49:41.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:41 smithi142 ceph-mon[29027]: pgmap v86: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:41.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:41 smithi152 ceph-mon[37177]: pgmap v86: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:41.975 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-29T04:49:41.975 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T04:49:41.976 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-29T04:49:42.193 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T04:49:42.575 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:42 smithi142 ceph-mon[29027]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:42.575 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:42 smithi142 ceph-mon[29027]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:49:42.576 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:49:42.576 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:49:42.576 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:49:42.576 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:49:42.576 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:42 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:49:42.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:42 smithi152 ceph-mon[37177]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:42.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:42 smithi152 ceph-mon[37177]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:49:42.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:49:42.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:49:42.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:49:42.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:49:42.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:42 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:49:43.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:43 smithi142 ceph-mon[29027]: pgmap v87: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:43.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:43 smithi142 ceph-mon[29027]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:49:43.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:43 smithi152 ceph-mon[37177]: pgmap v87: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:43.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:43 smithi152 ceph-mon[37177]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8da23624bc17fe0b1f797227e72a0db0ce7821ba 2024-03-29T04:49:44.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.325+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1384912126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd54105a40 msgr2=0x7fbd54105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:44.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.325+0000 7fbd5b7e3700 1 --2- 172.21.15.142:0/1384912126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd54105a40 0x7fbd54105e20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009910 tx=0x7fbd50009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.325+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1384912126 shutdown_connections 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.325+0000 7fbd5b7e3700 1 --2- 172.21.15.142:0/1384912126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd540ffa40 0x7fbd540ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.325+0000 7fbd5b7e3700 1 --2- 172.21.15.142:0/1384912126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd54105a40 0x7fbd54105e20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.325+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1384912126 >> 172.21.15.142:0/1384912126 conn(0x7fbd540fb570 msgr2=0x7fbd540fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.325+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1384912126 shutdown_connections 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.325+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1384912126 wait complete. 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.325+0000 7fbd5b7e3700 1 Processor -- start 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.325+0000 7fbd5b7e3700 1 -- start start 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5b7e3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd540ffa40 0x7fbd541975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5b7e3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd54105a40 0x7fbd54197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5b7e3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54198090 con 0x7fbd540ffa40 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5b7e3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd541981d0 con 0x7fbd54105a40 2024-03-29T04:49:44.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5957f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd540ffa40 0x7fbd541975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:44.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5957f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd540ffa40 0x7fbd541975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55594/0 (socket says 172.21.15.142:55594) 2024-03-29T04:49:44.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5957f700 1 -- 172.21.15.142:0/1287553771 learned_addr learned my addr 172.21.15.142:0/1287553771 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:44.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd58d7e700 1 --2- 172.21.15.142:0/1287553771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd54105a40 0x7fbd54197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:44.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5957f700 1 -- 172.21.15.142:0/1287553771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd54105a40 msgr2=0x7fbd54197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:44.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5957f700 1 --2- 172.21.15.142:0/1287553771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd54105a40 0x7fbd54197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:44.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5957f700 1 -- 172.21.15.142:0/1287553771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd500095f0 con 0x7fbd540ffa40 2024-03-29T04:49:44.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5957f700 1 --2- 172.21.15.142:0/1287553771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd540ffa40 0x7fbd541975c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbd50006010 tx=0x7fbd5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:44.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd4a7fc700 1 -- 172.21.15.142:0/1287553771 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd5001d070 con 0x7fbd540ffa40 2024-03-29T04:49:44.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd4a7fc700 1 -- 172.21.15.142:0/1287553771 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd50004fb0 con 0x7fbd540ffa40 2024-03-29T04:49:44.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1287553771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd541a0630 con 0x7fbd540ffa40 2024-03-29T04:49:44.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd4a7fc700 1 -- 172.21.15.142:0/1287553771 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd50022ea0 con 0x7fbd540ffa40 2024-03-29T04:49:44.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.326+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1287553771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd541a09e0 con 0x7fbd540ffa40 2024-03-29T04:49:44.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.327+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1287553771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd54108040 con 0x7fbd540ffa40 2024-03-29T04:49:44.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.328+0000 7fbd4a7fc700 1 -- 172.21.15.142:0/1287553771 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7fbd5000f6b0 con 0x7fbd540ffa40 2024-03-29T04:49:44.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.328+0000 7fbd4a7fc700 1 --2- 172.21.15.142:0/1287553771 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fbd4006d9e0 0x7fbd4006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:44.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.328+0000 7fbd4a7fc700 1 -- 172.21.15.142:0/1287553771 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbd5008eb60 con 0x7fbd540ffa40 2024-03-29T04:49:44.333 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.328+0000 7fbd58d7e700 1 --2- 172.21.15.142:0/1287553771 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fbd4006d9e0 0x7fbd4006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:44.333 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.329+0000 7fbd58d7e700 1 --2- 172.21.15.142:0/1287553771 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fbd4006d9e0 0x7fbd4006fea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd54069550 tx=0x7fbd44005d70 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:44.336 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.333+0000 7fbd4a7fc700 1 -- 172.21.15.142:0/1287553771 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fbd500586f0 con 0x7fbd540ffa40 2024-03-29T04:49:44.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.553+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1287553771 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd5402cd60 con 0x7fbd4006d9e0 2024-03-29T04:49:44.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.555+0000 7fbd4a7fc700 1 -- 172.21.15.142:0/1287553771 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fbd5402cd60 con 0x7fbd4006d9e0 2024-03-29T04:49:44.560 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.557+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1287553771 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fbd4006d9e0 msgr2=0x7fbd4006fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:44.560 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.557+0000 7fbd5b7e3700 1 --2- 172.21.15.142:0/1287553771 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fbd4006d9e0 0x7fbd4006fea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd54069550 tx=0x7fbd44005d70 comp rx=0 tx=0).stop 2024-03-29T04:49:44.560 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.557+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1287553771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd540ffa40 msgr2=0x7fbd541975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:44.560 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.557+0000 7fbd5b7e3700 1 --2- 172.21.15.142:0/1287553771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd540ffa40 0x7fbd541975c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbd50006010 tx=0x7fbd5000be20 comp rx=0 tx=0).stop 2024-03-29T04:49:44.560 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.557+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1287553771 shutdown_connections 2024-03-29T04:49:44.560 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.557+0000 7fbd5b7e3700 1 --2- 172.21.15.142:0/1287553771 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fbd4006d9e0 0x7fbd4006fea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:44.560 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.557+0000 7fbd5b7e3700 1 --2- 172.21.15.142:0/1287553771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd540ffa40 0x7fbd541975c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:44.560 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.557+0000 7fbd5b7e3700 1 --2- 172.21.15.142:0/1287553771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd54105a40 0x7fbd54197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:44.561 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.557+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1287553771 >> 172.21.15.142:0/1287553771 conn(0x7fbd540fb570 msgr2=0x7fbd540fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:44.561 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.557+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1287553771 shutdown_connections 2024-03-29T04:49:44.561 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.557+0000 7fbd5b7e3700 1 -- 172.21.15.142:0/1287553771 wait complete. 2024-03-29T04:49:44.571 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:49:44.598 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:44 smithi142 ceph-mon[29027]: pgmap v88: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:44.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.667+0000 7fc1c8873700 1 -- 172.21.15.142:0/1812750693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c4105a40 msgr2=0x7fc1c4105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:44.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.667+0000 7fc1c8873700 1 --2- 172.21.15.142:0/1812750693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c4105a40 0x7fc1c4105e20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009910 tx=0x7fc1b4009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:44.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.667+0000 7fc1c8873700 1 -- 172.21.15.142:0/1812750693 shutdown_connections 2024-03-29T04:49:44.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.667+0000 7fc1c8873700 1 --2- 172.21.15.142:0/1812750693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1c40ffbd0 0x7fc1c4100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:44.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.667+0000 7fc1c8873700 1 --2- 172.21.15.142:0/1812750693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c4105a40 0x7fc1c4105e20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:44.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.667+0000 7fc1c8873700 1 -- 172.21.15.142:0/1812750693 >> 172.21.15.142:0/1812750693 conn(0x7fc1c40755a0 msgr2=0x7fc1c40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:44.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.668+0000 7fc1c8873700 1 -- 172.21.15.142:0/1812750693 shutdown_connections 2024-03-29T04:49:44.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.668+0000 7fc1c8873700 1 -- 172.21.15.142:0/1812750693 wait complete. 2024-03-29T04:49:44.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c8873700 1 Processor -- start 2024-03-29T04:49:44.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c8873700 1 -- start start 2024-03-29T04:49:44.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c8873700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c40ffbd0 0x7fc1c419b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:44.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c8873700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1c4105a40 0x7fc1c419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:44.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c8873700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c41955c0 con 0x7fc1c40ffbd0 2024-03-29T04:49:44.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c8873700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c4195730 con 0x7fc1c4105a40 2024-03-29T04:49:44.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c40ffbd0 0x7fc1c419b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:44.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c40ffbd0 0x7fc1c419b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55606/0 (socket says 172.21.15.142:55606) 2024-03-29T04:49:44.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c259c700 1 -- 172.21.15.142:0/1477317228 learned_addr learned my addr 172.21.15.142:0/1477317228 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:44.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c1d9b700 1 --2- 172.21.15.142:0/1477317228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1c4105a40 0x7fc1c419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:44.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c259c700 1 -- 172.21.15.142:0/1477317228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1c4105a40 msgr2=0x7fc1c419bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:44.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c259c700 1 --2- 172.21.15.142:0/1477317228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1c4105a40 0x7fc1c419bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:44.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c259c700 1 -- 172.21.15.142:0/1477317228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b40095f0 con 0x7fc1c40ffbd0 2024-03-29T04:49:44.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.669+0000 7fc1c1d9b700 1 --2- 172.21.15.142:0/1477317228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1c4105a40 0x7fc1c419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:49:44.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.670+0000 7fc1c259c700 1 --2- 172.21.15.142:0/1477317228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c40ffbd0 0x7fc1c419b5d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4006010 tx=0x7fc1b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:44.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.670+0000 7fc1b37fe700 1 -- 172.21.15.142:0/1477317228 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1b401d070 con 0x7fc1c40ffbd0 2024-03-29T04:49:44.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.670+0000 7fc1c8873700 1 -- 172.21.15.142:0/1477317228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c41959b0 con 0x7fc1c40ffbd0 2024-03-29T04:49:44.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.670+0000 7fc1b37fe700 1 -- 172.21.15.142:0/1477317228 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1b4004fb0 con 0x7fc1c40ffbd0 2024-03-29T04:49:44.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.670+0000 7fc1b37fe700 1 -- 172.21.15.142:0/1477317228 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1b4022ea0 con 0x7fc1c40ffbd0 2024-03-29T04:49:44.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.670+0000 7fc1c8873700 1 -- 172.21.15.142:0/1477317228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c4195ea0 con 0x7fc1c40ffbd0 2024-03-29T04:49:44.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.671+0000 7fc1c8873700 1 -- 172.21.15.142:0/1477317228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c404ea90 con 0x7fc1c40ffbd0 2024-03-29T04:49:44.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.672+0000 7fc1b37fe700 1 -- 172.21.15.142:0/1477317228 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7fc1b400f690 con 0x7fc1c40ffbd0 2024-03-29T04:49:44.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.672+0000 7fc1b37fe700 1 --2- 172.21.15.142:0/1477317228 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fc1ac06d9e0 0x7fc1ac06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:44.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.672+0000 7fc1b37fe700 1 -- 172.21.15.142:0/1477317228 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc1b408eb50 con 0x7fc1c40ffbd0 2024-03-29T04:49:44.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.672+0000 7fc1c1d9b700 1 --2- 172.21.15.142:0/1477317228 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fc1ac06d9e0 0x7fc1ac06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:44.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.673+0000 7fc1c1d9b700 1 --2- 172.21.15.142:0/1477317228 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fc1ac06d9e0 0x7fc1ac06fea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4196e60 tx=0x7fc1b8008040 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:44.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.677+0000 7fc1b37fe700 1 -- 172.21.15.142:0/1477317228 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fc1b4058670 con 0x7fc1c40ffbd0 2024-03-29T04:49:44.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:44 smithi152 ceph-mon[37177]: pgmap v88: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:44.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.912+0000 7fc1c8873700 1 -- 172.21.15.142:0/1477317228 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1c4196160 con 0x7fc1ac06d9e0 2024-03-29T04:49:44.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.913+0000 7fc1b37fe700 1 -- 172.21.15.142:0/1477317228 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fc1c4196160 con 0x7fc1ac06d9e0 2024-03-29T04:49:44.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.915+0000 7fc1c8873700 1 -- 172.21.15.142:0/1477317228 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fc1ac06d9e0 msgr2=0x7fc1ac06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:44.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.915+0000 7fc1c8873700 1 --2- 172.21.15.142:0/1477317228 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fc1ac06d9e0 0x7fc1ac06fea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4196e60 tx=0x7fc1b8008040 comp rx=0 tx=0).stop 2024-03-29T04:49:44.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.915+0000 7fc1c8873700 1 -- 172.21.15.142:0/1477317228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c40ffbd0 msgr2=0x7fc1c419b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:44.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.915+0000 7fc1c8873700 1 --2- 172.21.15.142:0/1477317228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c40ffbd0 0x7fc1c419b5d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4006010 tx=0x7fc1b400be20 comp rx=0 tx=0).stop 2024-03-29T04:49:44.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.915+0000 7fc1c8873700 1 -- 172.21.15.142:0/1477317228 shutdown_connections 2024-03-29T04:49:44.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.915+0000 7fc1c8873700 1 --2- 172.21.15.142:0/1477317228 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fc1ac06d9e0 0x7fc1ac06fea0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:44.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.915+0000 7fc1c8873700 1 --2- 172.21.15.142:0/1477317228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c40ffbd0 0x7fc1c419b5d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:44.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.915+0000 7fc1c8873700 1 --2- 172.21.15.142:0/1477317228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1c4105a40 0x7fc1c419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:44.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.915+0000 7fc1c8873700 1 -- 172.21.15.142:0/1477317228 >> 172.21.15.142:0/1477317228 conn(0x7fc1c40755a0 msgr2=0x7fc1c40fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:44.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.915+0000 7fc1c8873700 1 -- 172.21.15.142:0/1477317228 shutdown_connections 2024-03-29T04:49:44.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:44.915+0000 7fc1c8873700 1 -- 172.21.15.142:0/1477317228 wait complete. 2024-03-29T04:49:45.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.023+0000 7fd55a959700 1 -- 172.21.15.142:0/1701834068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd554100de0 msgr2=0x7fd5541011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:45.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.023+0000 7fd55a959700 1 --2- 172.21.15.142:0/1701834068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd554100de0 0x7fd5541011c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd544009910 tx=0x7fd544009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:45.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.023+0000 7fd55a959700 1 -- 172.21.15.142:0/1701834068 shutdown_connections 2024-03-29T04:49:45.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.023+0000 7fd55a959700 1 --2- 172.21.15.142:0/1701834068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd554101790 0x7fd5541057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.023+0000 7fd55a959700 1 --2- 172.21.15.142:0/1701834068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd554100de0 0x7fd5541011c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.023+0000 7fd55a959700 1 -- 172.21.15.142:0/1701834068 >> 172.21.15.142:0/1701834068 conn(0x7fd5540fc6b0 msgr2=0x7fd5540fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:45.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.023+0000 7fd55a959700 1 -- 172.21.15.142:0/1701834068 shutdown_connections 2024-03-29T04:49:45.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.023+0000 7fd55a959700 1 -- 172.21.15.142:0/1701834068 wait complete. 2024-03-29T04:49:45.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.024+0000 7fd55a959700 1 Processor -- start 2024-03-29T04:49:45.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.024+0000 7fd55a959700 1 -- start start 2024-03-29T04:49:45.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.024+0000 7fd55a959700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd554100de0 0x7fd55419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:45.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.024+0000 7fd55a959700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd554101790 0x7fd55419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:45.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.024+0000 7fd55a959700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55419d190 con 0x7fd554101790 2024-03-29T04:49:45.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.024+0000 7fd55a959700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd554196640 con 0x7fd554100de0 2024-03-29T04:49:45.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd5537fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd554101790 0x7fd55419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:45.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd5537fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd554101790 0x7fd55419cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55628/0 (socket says 172.21.15.142:55628) 2024-03-29T04:49:45.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd5537fe700 1 -- 172.21.15.142:0/4165139578 learned_addr learned my addr 172.21.15.142:0/4165139578 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:45.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd553fff700 1 --2- 172.21.15.142:0/4165139578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd554100de0 0x7fd55419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:45.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd5537fe700 1 -- 172.21.15.142:0/4165139578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd554100de0 msgr2=0x7fd55419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:45.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd5537fe700 1 --2- 172.21.15.142:0/4165139578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd554100de0 0x7fd55419c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd5537fe700 1 -- 172.21.15.142:0/4165139578 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440095f0 con 0x7fd554101790 2024-03-29T04:49:45.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd5537fe700 1 --2- 172.21.15.142:0/4165139578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd554101790 0x7fd55419cb00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd54800d900 tx=0x7fd54800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:45.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd5517fa700 1 -- 172.21.15.142:0/4165139578 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd548009900 con 0x7fd554101790 2024-03-29T04:49:45.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd55a959700 1 -- 172.21.15.142:0/4165139578 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd554196920 con 0x7fd554101790 2024-03-29T04:49:45.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd5517fa700 1 -- 172.21.15.142:0/4165139578 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd548010460 con 0x7fd554101790 2024-03-29T04:49:45.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd5517fa700 1 -- 172.21.15.142:0/4165139578 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd54800f5f0 con 0x7fd554101790 2024-03-29T04:49:45.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.025+0000 7fd55a959700 1 -- 172.21.15.142:0/4165139578 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd554196e70 con 0x7fd554101790 2024-03-29T04:49:45.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.026+0000 7fd55a959700 1 -- 172.21.15.142:0/4165139578 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd554109180 con 0x7fd554101790 2024-03-29T04:49:45.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.027+0000 7fd5517fa700 1 -- 172.21.15.142:0/4165139578 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7fd5480105d0 con 0x7fd554101790 2024-03-29T04:49:45.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.027+0000 7fd5517fa700 1 --2- 172.21.15.142:0/4165139578 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fd53c06d9e0 0x7fd53c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:45.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.027+0000 7fd5517fa700 1 -- 172.21.15.142:0/4165139578 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd54808d190 con 0x7fd554101790 2024-03-29T04:49:45.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.027+0000 7fd553fff700 1 --2- 172.21.15.142:0/4165139578 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fd53c06d9e0 0x7fd53c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:45.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.028+0000 7fd553fff700 1 --2- 172.21.15.142:0/4165139578 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fd53c06d9e0 0x7fd53c06fea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd544006010 tx=0x7fd54401a040 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:45.035 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.032+0000 7fd5517fa700 1 -- 172.21.15.142:0/4165139578 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7fd54805a730 con 0x7fd554101790 2024-03-29T04:49:45.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.248+0000 7fd55a959700 1 -- 172.21.15.142:0/4165139578 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd554197120 con 0x7fd53c06d9e0 2024-03-29T04:49:45.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.259+0000 7fd5517fa700 1 -- 172.21.15.142:0/4165139578 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd554197120 con 0x7fd53c06d9e0 2024-03-29T04:49:45.262 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (97s) 68s ago 13m 16.8M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (13m) 68s ago 13m 8070k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (12m) 2m ago 12m 7902k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (13m) 68s ago 13m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (12m) 2m ago 12m 6665k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (73s) 68s ago 12m 51.7M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (7m) 68s ago 7m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (7m) 68s ago 7m 63.3M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (7m) 2m ago 7m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (7m) 2m ago 7m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (3m) 68s ago 14m 504M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (2m) 2m ago 12m 447M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:49:45.263 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (14m) 68s ago 14m 60.7M 2048M 18.2.1 5be31c24972a 32c014de5479 2024-03-29T04:49:45.264 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (12m) 2m ago 12m 44.0M 2048M 18.2.1 5be31c24972a defcc70499d9 2024-03-29T04:49:45.264 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (2m) 68s ago 13m 8526k - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:49:45.264 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (2m) 2m ago 12m 8052k - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:49:45.264 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (11m) 68s ago 11m 218M 1216M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:49:45.264 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (10m) 68s ago 10m 240M 1216M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:49:45.264 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (10m) 68s ago 10m 198M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:49:45.264 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (9m) 2m ago 9m 274M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:49:45.264 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (9m) 2m ago 9m 233M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:49:45.264 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (9m) 2m ago 9m 222M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:49:45.264 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (114s) 68s ago 12m 44.9M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:49:45.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.261+0000 7fd55a959700 1 -- 172.21.15.142:0/4165139578 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fd53c06d9e0 msgr2=0x7fd53c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:45.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.261+0000 7fd55a959700 1 --2- 172.21.15.142:0/4165139578 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fd53c06d9e0 0x7fd53c06fea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd544006010 tx=0x7fd54401a040 comp rx=0 tx=0).stop 2024-03-29T04:49:45.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.261+0000 7fd55a959700 1 -- 172.21.15.142:0/4165139578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd554101790 msgr2=0x7fd55419cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:45.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.261+0000 7fd55a959700 1 --2- 172.21.15.142:0/4165139578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd554101790 0x7fd55419cb00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd54800d900 tx=0x7fd54800dc10 comp rx=0 tx=0).stop 2024-03-29T04:49:45.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.262+0000 7fd55a959700 1 -- 172.21.15.142:0/4165139578 shutdown_connections 2024-03-29T04:49:45.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.262+0000 7fd55a959700 1 --2- 172.21.15.142:0/4165139578 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7fd53c06d9e0 0x7fd53c06fea0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.262+0000 7fd55a959700 1 --2- 172.21.15.142:0/4165139578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd554100de0 0x7fd55419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.262+0000 7fd55a959700 1 --2- 172.21.15.142:0/4165139578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd554101790 0x7fd55419cb00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.262+0000 7fd55a959700 1 -- 172.21.15.142:0/4165139578 >> 172.21.15.142:0/4165139578 conn(0x7fd5540fc6b0 msgr2=0x7fd5540fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:45.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.262+0000 7fd55a959700 1 -- 172.21.15.142:0/4165139578 shutdown_connections 2024-03-29T04:49:45.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.262+0000 7fd55a959700 1 -- 172.21.15.142:0/4165139578 wait complete. 2024-03-29T04:49:45.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.366+0000 7feb37caf700 1 -- 172.21.15.142:0/1863638894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb300ffdb0 msgr2=0x7feb30103e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:45.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.366+0000 7feb37caf700 1 --2- 172.21.15.142:0/1863638894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb300ffdb0 0x7feb30103e00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feb20009910 tx=0x7feb20009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:45.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.366+0000 7feb37caf700 1 -- 172.21.15.142:0/1863638894 shutdown_connections 2024-03-29T04:49:45.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.366+0000 7feb37caf700 1 --2- 172.21.15.142:0/1863638894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb300ffdb0 0x7feb30103e00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.366+0000 7feb37caf700 1 --2- 172.21.15.142:0/1863638894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb300ff400 0x7feb300ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.366+0000 7feb37caf700 1 -- 172.21.15.142:0/1863638894 >> 172.21.15.142:0/1863638894 conn(0x7feb30075e30 msgr2=0x7feb300fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:45.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.366+0000 7feb37caf700 1 -- 172.21.15.142:0/1863638894 shutdown_connections 2024-03-29T04:49:45.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.366+0000 7feb37caf700 1 -- 172.21.15.142:0/1863638894 wait complete. 2024-03-29T04:49:45.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.367+0000 7feb37caf700 1 Processor -- start 2024-03-29T04:49:45.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.367+0000 7feb37caf700 1 -- start start 2024-03-29T04:49:45.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.367+0000 7feb37caf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb300ff400 0x7feb30196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:45.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.367+0000 7feb37caf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb300ffdb0 0x7feb30196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:45.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.367+0000 7feb37caf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb301972e0 con 0x7feb300ff400 2024-03-29T04:49:45.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.367+0000 7feb37caf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3019a010 con 0x7feb300ffdb0 2024-03-29T04:49:45.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.367+0000 7feb35a4b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb300ff400 0x7feb30196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:45.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.367+0000 7feb35a4b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb300ff400 0x7feb30196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55648/0 (socket says 172.21.15.142:55648) 2024-03-29T04:49:45.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.367+0000 7feb35a4b700 1 -- 172.21.15.142:0/810870210 learned_addr learned my addr 172.21.15.142:0/810870210 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:45.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.368+0000 7feb35a4b700 1 -- 172.21.15.142:0/810870210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb300ffdb0 msgr2=0x7feb30196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:45.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.368+0000 7feb35a4b700 1 --2- 172.21.15.142:0/810870210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb300ffdb0 0x7feb30196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.368+0000 7feb35a4b700 1 -- 172.21.15.142:0/810870210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb200095f0 con 0x7feb300ff400 2024-03-29T04:49:45.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.368+0000 7feb35a4b700 1 --2- 172.21.15.142:0/810870210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb300ff400 0x7feb30196710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00d8d0 tx=0x7feb2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:45.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.368+0000 7feb26ffd700 1 -- 172.21.15.142:0/810870210 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb2c009940 con 0x7feb300ff400 2024-03-29T04:49:45.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.368+0000 7feb26ffd700 1 -- 172.21.15.142:0/810870210 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb2c010460 con 0x7feb300ff400 2024-03-29T04:49:45.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.368+0000 7feb37caf700 1 -- 172.21.15.142:0/810870210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3019a2f0 con 0x7feb300ff400 2024-03-29T04:49:45.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.368+0000 7feb26ffd700 1 -- 172.21.15.142:0/810870210 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb2c00f5f0 con 0x7feb300ff400 2024-03-29T04:49:45.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.368+0000 7feb37caf700 1 -- 172.21.15.142:0/810870210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3019a840 con 0x7feb300ff400 2024-03-29T04:49:45.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.369+0000 7feb37caf700 1 -- 172.21.15.142:0/810870210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3004ea90 con 0x7feb300ff400 2024-03-29T04:49:45.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.370+0000 7feb26ffd700 1 -- 172.21.15.142:0/810870210 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7feb2c0105d0 con 0x7feb300ff400 2024-03-29T04:49:45.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.370+0000 7feb26ffd700 1 --2- 172.21.15.142:0/810870210 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7feb1c06d9e0 0x7feb1c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:45.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.370+0000 7feb26ffd700 1 -- 172.21.15.142:0/810870210 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7feb2c08d470 con 0x7feb300ff400 2024-03-29T04:49:45.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.370+0000 7feb3524a700 1 --2- 172.21.15.142:0/810870210 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7feb1c06d9e0 0x7feb1c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:45.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.371+0000 7feb3524a700 1 --2- 172.21.15.142:0/810870210 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7feb1c06d9e0 0x7feb1c06fea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feb200059d0 tx=0x7feb20005960 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:45.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.374+0000 7feb26ffd700 1 -- 172.21.15.142:0/810870210 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7feb2c0570b0 con 0x7feb300ff400 2024-03-29T04:49:45.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.669+0000 7feb37caf700 1 -- 172.21.15.142:0/810870210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feb30197a70 con 0x7feb300ff400 2024-03-29T04:49:45.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.670+0000 7feb26ffd700 1 -- 172.21.15.142:0/810870210 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7feb2c05a770 con 0x7feb300ff400 2024-03-29T04:49:45.673 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:49:45.673 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:49:45.673 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:49:45.674 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:49:45.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.672+0000 7feb37caf700 1 -- 172.21.15.142:0/810870210 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7feb1c06d9e0 msgr2=0x7feb1c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:45.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.672+0000 7feb37caf700 1 --2- 172.21.15.142:0/810870210 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7feb1c06d9e0 0x7feb1c06fea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feb200059d0 tx=0x7feb20005960 comp rx=0 tx=0).stop 2024-03-29T04:49:45.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.672+0000 7feb37caf700 1 -- 172.21.15.142:0/810870210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb300ff400 msgr2=0x7feb30196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:45.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.672+0000 7feb37caf700 1 --2- 172.21.15.142:0/810870210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb300ff400 0x7feb30196710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00d8d0 tx=0x7feb2c00dc90 comp rx=0 tx=0).stop 2024-03-29T04:49:45.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.672+0000 7feb37caf700 1 -- 172.21.15.142:0/810870210 shutdown_connections 2024-03-29T04:49:45.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.672+0000 7feb37caf700 1 --2- 172.21.15.142:0/810870210 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7feb1c06d9e0 0x7feb1c06fea0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.672+0000 7feb37caf700 1 --2- 172.21.15.142:0/810870210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb300ff400 0x7feb30196710 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.672+0000 7feb37caf700 1 --2- 172.21.15.142:0/810870210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb300ffdb0 0x7feb30196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.672+0000 7feb37caf700 1 -- 172.21.15.142:0/810870210 >> 172.21.15.142:0/810870210 conn(0x7feb30075e30 msgr2=0x7feb300fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:45.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.672+0000 7feb37caf700 1 -- 172.21.15.142:0/810870210 shutdown_connections 2024-03-29T04:49:45.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.672+0000 7feb37caf700 1 -- 172.21.15.142:0/810870210 wait complete. 2024-03-29T04:49:45.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.779+0000 7f049201f700 1 -- 172.21.15.142:0/3060985359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f048c101750 msgr2=0x7f048c1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:45.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.779+0000 7f049201f700 1 --2- 172.21.15.142:0/3060985359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f048c101750 0x7f048c1057a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0480009910 tx=0x7f0480009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:45.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.779+0000 7f049201f700 1 -- 172.21.15.142:0/3060985359 shutdown_connections 2024-03-29T04:49:45.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.779+0000 7f049201f700 1 --2- 172.21.15.142:0/3060985359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f048c101750 0x7f048c1057a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.779+0000 7f049201f700 1 --2- 172.21.15.142:0/3060985359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f048c100da0 0x7f048c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.779+0000 7f049201f700 1 -- 172.21.15.142:0/3060985359 >> 172.21.15.142:0/3060985359 conn(0x7f048c0fc650 msgr2=0x7f048c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:45.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.779+0000 7f049201f700 1 -- 172.21.15.142:0/3060985359 shutdown_connections 2024-03-29T04:49:45.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.779+0000 7f049201f700 1 -- 172.21.15.142:0/3060985359 wait complete. 2024-03-29T04:49:45.784 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:45 smithi142 ceph-mon[29027]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:45.784 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:45 smithi142 ceph-mon[29027]: from='client.14876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:45.784 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:45 smithi142 ceph-mon[29027]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:45.784 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:45 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:49:45.784 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:45 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:49:45.784 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:45 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:49:45.784 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:45 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:49:45.784 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:45 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-29T04:49:45.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.780+0000 7f049201f700 1 Processor -- start 2024-03-29T04:49:45.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.780+0000 7f049201f700 1 -- start start 2024-03-29T04:49:45.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.780+0000 7f049201f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f048c100da0 0x7f048c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:45.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.780+0000 7f049201f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f048c101750 0x7f048c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:45.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.780+0000 7f049201f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c19d1f0 con 0x7f048c100da0 2024-03-29T04:49:45.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.780+0000 7f049201f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c1966a0 con 0x7f048c101750 2024-03-29T04:49:45.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f048b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f048c100da0 0x7f048c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:45.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f048b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f048c100da0 0x7f048c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55662/0 (socket says 172.21.15.142:55662) 2024-03-29T04:49:45.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f048b7fe700 1 -- 172.21.15.142:0/2431460876 learned_addr learned my addr 172.21.15.142:0/2431460876 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:45.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f048affd700 1 --2- 172.21.15.142:0/2431460876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f048c101750 0x7f048c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:45.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f048b7fe700 1 -- 172.21.15.142:0/2431460876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f048c101750 msgr2=0x7f048c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:45.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f048b7fe700 1 --2- 172.21.15.142:0/2431460876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f048c101750 0x7f048c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:45.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f048b7fe700 1 -- 172.21.15.142:0/2431460876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04800095f0 con 0x7f048c100da0 2024-03-29T04:49:45.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f048b7fe700 1 --2- 172.21.15.142:0/2431460876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f048c100da0 0x7f048c19c620 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f047c00b4b0 tx=0x7f047c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:45.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f0488ff9700 1 -- 172.21.15.142:0/2431460876 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f047c011800 con 0x7f048c100da0 2024-03-29T04:49:45.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f049201f700 1 -- 172.21.15.142:0/2431460876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c196980 con 0x7f048c100da0 2024-03-29T04:49:45.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f0488ff9700 1 -- 172.21.15.142:0/2431460876 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f047c011e20 con 0x7f048c100da0 2024-03-29T04:49:45.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f0488ff9700 1 -- 172.21.15.142:0/2431460876 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f047c00f2e0 con 0x7f048c100da0 2024-03-29T04:49:45.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.781+0000 7f049201f700 1 -- 172.21.15.142:0/2431460876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c196ed0 con 0x7f048c100da0 2024-03-29T04:49:45.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.783+0000 7f049201f700 1 -- 172.21.15.142:0/2431460876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c1090d0 con 0x7f048c100da0 2024-03-29T04:49:45.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.783+0000 7f0488ff9700 1 -- 172.21.15.142:0/2431460876 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7f047c010380 con 0x7f048c100da0 2024-03-29T04:49:45.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.784+0000 7f0488ff9700 1 --2- 172.21.15.142:0/2431460876 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f047406d9e0 0x7f047406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:45.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.784+0000 7f0488ff9700 1 -- 172.21.15.142:0/2431460876 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f047c08d0f0 con 0x7f048c100da0 2024-03-29T04:49:45.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.784+0000 7f048affd700 1 --2- 172.21.15.142:0/2431460876 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f047406d9e0 0x7f047406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:45.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.784+0000 7f048affd700 1 --2- 172.21.15.142:0/2431460876 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f047406d9e0 0x7f047406fea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f048c0fcf70 tx=0x7f048001a040 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:45.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:45.788+0000 7f0488ff9700 1 -- 172.21.15.142:0/2431460876 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f047c05a690 con 0x7f048c100da0 2024-03-29T04:49:46.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.035+0000 7f049201f700 1 -- 172.21.15.142:0/2431460876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f048c197150 con 0x7f048c100da0 2024-03-29T04:49:46.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.036+0000 7f0488ff9700 1 -- 172.21.15.142:0/2431460876 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1853 (secure 0 0 0) 0x7f047c05a220 con 0x7f048c100da0 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:e21 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 21 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:49:01.288637+0000 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:49:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 47 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:up {0=14494,1=24275} 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:49:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:49:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:49:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:49:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:49:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:49:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{0:14494} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:49:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{0:14506} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:49:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{1:24275} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:49:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{1:24287} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:49:46.042 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:49:46.042 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:49:46.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.038+0000 7f049201f700 1 -- 172.21.15.142:0/2431460876 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f047406d9e0 msgr2=0x7f047406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:46.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.038+0000 7f049201f700 1 --2- 172.21.15.142:0/2431460876 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f047406d9e0 0x7f047406fea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f048c0fcf70 tx=0x7f048001a040 comp rx=0 tx=0).stop 2024-03-29T04:49:46.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.038+0000 7f049201f700 1 -- 172.21.15.142:0/2431460876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f048c100da0 msgr2=0x7f048c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:46.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.038+0000 7f049201f700 1 --2- 172.21.15.142:0/2431460876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f048c100da0 0x7f048c19c620 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f047c00b4b0 tx=0x7f047c00b7c0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.038+0000 7f049201f700 1 -- 172.21.15.142:0/2431460876 shutdown_connections 2024-03-29T04:49:46.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.038+0000 7f049201f700 1 --2- 172.21.15.142:0/2431460876 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f047406d9e0 0x7f047406fea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.038+0000 7f049201f700 1 --2- 172.21.15.142:0/2431460876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f048c100da0 0x7f048c19c620 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.038+0000 7f049201f700 1 --2- 172.21.15.142:0/2431460876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f048c101750 0x7f048c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.038+0000 7f049201f700 1 -- 172.21.15.142:0/2431460876 >> 172.21.15.142:0/2431460876 conn(0x7f048c0fc650 msgr2=0x7f048c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:46.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.038+0000 7f049201f700 1 -- 172.21.15.142:0/2431460876 shutdown_connections 2024-03-29T04:49:46.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.038+0000 7f049201f700 1 -- 172.21.15.142:0/2431460876 wait complete. 2024-03-29T04:49:46.044 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 21 2024-03-29T04:49:46.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:45 smithi152 ceph-mon[37177]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:46.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:45 smithi152 ceph-mon[37177]: from='client.14876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:46.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:45 smithi152 ceph-mon[37177]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:46.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:45 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:49:46.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:45 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:49:46.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:45 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:49:46.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:45 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:49:46.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:45 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-29T04:49:46.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.144+0000 7f60a8e51700 1 -- 172.21.15.142:0/3629139382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a40ffa30 msgr2=0x7f60a40ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:46.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.144+0000 7f60a8e51700 1 --2- 172.21.15.142:0/3629139382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a40ffa30 0x7f60a40ffeb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6098009910 tx=0x7f6098009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:46.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.144+0000 7f60a8e51700 1 -- 172.21.15.142:0/3629139382 shutdown_connections 2024-03-29T04:49:46.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.144+0000 7f60a8e51700 1 --2- 172.21.15.142:0/3629139382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a40ffa30 0x7f60a40ffeb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.144+0000 7f60a8e51700 1 --2- 172.21.15.142:0/3629139382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60a4105a30 0x7f60a4105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.144+0000 7f60a8e51700 1 -- 172.21.15.142:0/3629139382 >> 172.21.15.142:0/3629139382 conn(0x7f60a40fb540 msgr2=0x7f60a40fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:46.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.144+0000 7f60a8e51700 1 -- 172.21.15.142:0/3629139382 shutdown_connections 2024-03-29T04:49:46.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.144+0000 7f60a8e51700 1 -- 172.21.15.142:0/3629139382 wait complete. 2024-03-29T04:49:46.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.145+0000 7f60a8e51700 1 Processor -- start 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.145+0000 7f60a8e51700 1 -- start start 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.145+0000 7f60a8e51700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60a40ffa30 0x7f60a41956a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.145+0000 7f60a8e51700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a4105a30 0x7f60a4195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.145+0000 7f60a8e51700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a4196270 con 0x7f60a4105a30 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.145+0000 7f60a8e51700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a4198fa0 con 0x7f60a40ffa30 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.145+0000 7f60a1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a4105a30 0x7f60a4195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.145+0000 7f60a1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a4105a30 0x7f60a4195be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55680/0 (socket says 172.21.15.142:55680) 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.145+0000 7f60a1d9b700 1 -- 172.21.15.142:0/3635075509 learned_addr learned my addr 172.21.15.142:0/3635075509 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.145+0000 7f60a259c700 1 --2- 172.21.15.142:0/3635075509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60a40ffa30 0x7f60a41956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.146+0000 7f60a1d9b700 1 -- 172.21.15.142:0/3635075509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60a40ffa30 msgr2=0x7f60a41956a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.146+0000 7f60a1d9b700 1 --2- 172.21.15.142:0/3635075509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60a40ffa30 0x7f60a41956a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.146+0000 7f60a1d9b700 1 -- 172.21.15.142:0/3635075509 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980095f0 con 0x7f60a4105a30 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.146+0000 7f60a259c700 1 --2- 172.21.15.142:0/3635075509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60a40ffa30 0x7f60a41956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:49:46.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.146+0000 7f60a1d9b700 1 --2- 172.21.15.142:0/3635075509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a4105a30 0x7f60a4195be0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6098006010 tx=0x7f609800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:46.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.146+0000 7f60937fe700 1 -- 172.21.15.142:0/3635075509 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f609801f070 con 0x7f60a4105a30 2024-03-29T04:49:46.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.146+0000 7f60937fe700 1 -- 172.21.15.142:0/3635075509 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6098019610 con 0x7f60a4105a30 2024-03-29T04:49:46.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.146+0000 7f60937fe700 1 -- 172.21.15.142:0/3635075509 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f609800f690 con 0x7f60a4105a30 2024-03-29T04:49:46.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.146+0000 7f60a8e51700 1 -- 172.21.15.142:0/3635075509 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a4199220 con 0x7f60a4105a30 2024-03-29T04:49:46.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.146+0000 7f60a8e51700 1 -- 172.21.15.142:0/3635075509 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a4199710 con 0x7f60a4105a30 2024-03-29T04:49:46.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.147+0000 7f60a8e51700 1 -- 172.21.15.142:0/3635075509 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a404ea90 con 0x7f60a4105a30 2024-03-29T04:49:46.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.147+0000 7f60937fe700 1 -- 172.21.15.142:0/3635075509 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7f6098003a60 con 0x7f60a4105a30 2024-03-29T04:49:46.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.148+0000 7f60937fe700 1 --2- 172.21.15.142:0/3635075509 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f608c06d9e0 0x7f608c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:46.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.148+0000 7f60937fe700 1 -- 172.21.15.142:0/3635075509 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6098090cd0 con 0x7f60a4105a30 2024-03-29T04:49:46.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.148+0000 7f60a259c700 1 --2- 172.21.15.142:0/3635075509 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f608c06d9e0 0x7f608c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:46.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.148+0000 7f60a259c700 1 --2- 172.21.15.142:0/3635075509 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f608c06d9e0 0x7f608c06fea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6094005950 tx=0x7f60940058e0 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:46.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.152+0000 7f60937fe700 1 -- 172.21.15.142:0/3635075509 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f609805a7c0 con 0x7f60a4105a30 2024-03-29T04:49:46.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.370+0000 7f60a8e51700 1 -- 172.21.15.142:0/3635075509 --> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60a41999f0 con 0x7f608c06d9e0 2024-03-29T04:49:46.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.371+0000 7f60937fe700 1 -- 172.21.15.142:0/3635075509 <== mgr.24465 v2:172.21.15.142:6800/1771239042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f60a41999f0 con 0x7f608c06d9e0 2024-03-29T04:49:46.374 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:49:46.375 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:49:46.375 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:49:46.375 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:49:46.375 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:49:46.375 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr" 2024-03-29T04:49:46.375 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:49:46.375 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "2/23 daemons upgraded", 2024-03-29T04:49:46.375 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mon daemons", 2024-03-29T04:49:46.375 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:49:46.375 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:49:46.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.373+0000 7f60a8e51700 1 -- 172.21.15.142:0/3635075509 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f608c06d9e0 msgr2=0x7f608c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:46.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.373+0000 7f60a8e51700 1 --2- 172.21.15.142:0/3635075509 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f608c06d9e0 0x7f608c06fea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6094005950 tx=0x7f60940058e0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.373+0000 7f60a8e51700 1 -- 172.21.15.142:0/3635075509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a4105a30 msgr2=0x7f60a4195be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:46.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.373+0000 7f60a8e51700 1 --2- 172.21.15.142:0/3635075509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a4105a30 0x7f60a4195be0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6098006010 tx=0x7f609800be20 comp rx=0 tx=0).stop 2024-03-29T04:49:46.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.373+0000 7f60a8e51700 1 -- 172.21.15.142:0/3635075509 shutdown_connections 2024-03-29T04:49:46.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.373+0000 7f60a8e51700 1 --2- 172.21.15.142:0/3635075509 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f608c06d9e0 0x7f608c06fea0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.373+0000 7f60a8e51700 1 --2- 172.21.15.142:0/3635075509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60a40ffa30 0x7f60a41956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.373+0000 7f60a8e51700 1 --2- 172.21.15.142:0/3635075509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a4105a30 0x7f60a4195be0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.373+0000 7f60a8e51700 1 -- 172.21.15.142:0/3635075509 >> 172.21.15.142:0/3635075509 conn(0x7f60a40fb540 msgr2=0x7f60a40fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:46.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.374+0000 7f60a8e51700 1 -- 172.21.15.142:0/3635075509 shutdown_connections 2024-03-29T04:49:46.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.374+0000 7f60a8e51700 1 -- 172.21.15.142:0/3635075509 wait complete. 2024-03-29T04:49:46.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.482+0000 7f93e9c98700 1 -- 172.21.15.142:0/3803777958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4105a40 msgr2=0x7f93e4105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:46.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.482+0000 7f93e9c98700 1 --2- 172.21.15.142:0/3803777958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4105a40 0x7f93e4105e20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009910 tx=0x7f93d8009c20 comp rx=0 tx=0).stop 2024-03-29T04:49:46.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.482+0000 7f93e9c98700 1 -- 172.21.15.142:0/3803777958 shutdown_connections 2024-03-29T04:49:46.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.482+0000 7f93e9c98700 1 --2- 172.21.15.142:0/3803777958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e40ffbd0 0x7f93e4100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.482+0000 7f93e9c98700 1 --2- 172.21.15.142:0/3803777958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4105a40 0x7f93e4105e20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.482+0000 7f93e9c98700 1 -- 172.21.15.142:0/3803777958 >> 172.21.15.142:0/3803777958 conn(0x7f93e40755a0 msgr2=0x7f93e40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:46.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.482+0000 7f93e9c98700 1 -- 172.21.15.142:0/3803777958 shutdown_connections 2024-03-29T04:49:46.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.482+0000 7f93e9c98700 1 -- 172.21.15.142:0/3803777958 wait complete. 2024-03-29T04:49:46.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.483+0000 7f93e9c98700 1 Processor -- start 2024-03-29T04:49:46.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.483+0000 7f93e9c98700 1 -- start start 2024-03-29T04:49:46.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.483+0000 7f93e9c98700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e40ffbd0 0x7f93e4196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:46.487 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.483+0000 7f93e9c98700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e4105a40 0x7f93e4196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:46.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.483+0000 7f93e9c98700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41972e0 con 0x7f93e40ffbd0 2024-03-29T04:49:46.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.483+0000 7f93e9c98700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e419a010 con 0x7f93e4105a40 2024-03-29T04:49:46.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e40ffbd0 0x7f93e4196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:46.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e40ffbd0 0x7f93e4196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55692/0 (socket says 172.21.15.142:55692) 2024-03-29T04:49:46.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e37fe700 1 -- 172.21.15.142:0/908966288 learned_addr learned my addr 172.21.15.142:0/908966288 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:49:46.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e4105a40 0x7f93e4196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:46.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e37fe700 1 -- 172.21.15.142:0/908966288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e4105a40 msgr2=0x7f93e4196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:46.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e37fe700 1 --2- 172.21.15.142:0/908966288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e4105a40 0x7f93e4196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e37fe700 1 -- 172.21.15.142:0/908966288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d80095f0 con 0x7f93e40ffbd0 2024-03-29T04:49:46.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e37fe700 1 --2- 172.21.15.142:0/908966288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e40ffbd0 0x7f93e4196710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f93d8006010 tx=0x7f93d800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:46.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e0ff9700 1 -- 172.21.15.142:0/908966288 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d801d070 con 0x7f93e40ffbd0 2024-03-29T04:49:46.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e0ff9700 1 -- 172.21.15.142:0/908966288 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93d8005dc0 con 0x7f93e40ffbd0 2024-03-29T04:49:46.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e9c98700 1 -- 172.21.15.142:0/908966288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e419a290 con 0x7f93e40ffbd0 2024-03-29T04:49:46.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e0ff9700 1 -- 172.21.15.142:0/908966288 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d8017400 con 0x7f93e40ffbd0 2024-03-29T04:49:46.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.484+0000 7f93e9c98700 1 -- 172.21.15.142:0/908966288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e419a780 con 0x7f93e40ffbd0 2024-03-29T04:49:46.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.485+0000 7f93e9c98700 1 -- 172.21.15.142:0/908966288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e404ea90 con 0x7f93e40ffbd0 2024-03-29T04:49:46.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.486+0000 7f93e0ff9700 1 -- 172.21.15.142:0/908966288 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 33) v1 ==== 91716+0+0 (secure 0 0 0) 0x7f93d8003a60 con 0x7f93e40ffbd0 2024-03-29T04:49:46.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.486+0000 7f93e0ff9700 1 --2- 172.21.15.142:0/908966288 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f93cc06d9e0 0x7f93cc06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:49:46.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.486+0000 7f93e0ff9700 1 -- 172.21.15.142:0/908966288 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f93d808ea50 con 0x7f93e40ffbd0 2024-03-29T04:49:46.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.486+0000 7f93e2ffd700 1 --2- 172.21.15.142:0/908966288 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f93cc06d9e0 0x7f93cc06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:49:46.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.487+0000 7f93e2ffd700 1 --2- 172.21.15.142:0/908966288 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f93cc06d9e0 0x7f93cc06fea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93e4197d30 tx=0x7f93d400b410 comp rx=0 tx=0).ready entity=mgr.24465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:49:46.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.491+0000 7f93e0ff9700 1 -- 172.21.15.142:0/908966288 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184237 (secure 0 0 0) 0x7f93d80584c0 con 0x7f93e40ffbd0 2024-03-29T04:49:46.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.782+0000 7f93e9c98700 1 -- 172.21.15.142:0/908966288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f93e419aa90 con 0x7f93e40ffbd0 2024-03-29T04:49:46.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.783+0000 7f93e0ff9700 1 -- 172.21.15.142:0/908966288 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f93d805bb80 con 0x7f93e40ffbd0 2024-03-29T04:49:46.787 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_OK 2024-03-29T04:49:46.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.785+0000 7f93e9c98700 1 -- 172.21.15.142:0/908966288 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f93cc06d9e0 msgr2=0x7f93cc06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:46.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.785+0000 7f93e9c98700 1 --2- 172.21.15.142:0/908966288 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f93cc06d9e0 0x7f93cc06fea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93e4197d30 tx=0x7f93d400b410 comp rx=0 tx=0).stop 2024-03-29T04:49:46.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.785+0000 7f93e9c98700 1 -- 172.21.15.142:0/908966288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e40ffbd0 msgr2=0x7f93e4196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:49:46.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.785+0000 7f93e9c98700 1 --2- 172.21.15.142:0/908966288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e40ffbd0 0x7f93e4196710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f93d8006010 tx=0x7f93d800bd70 comp rx=0 tx=0).stop 2024-03-29T04:49:46.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.785+0000 7f93e9c98700 1 -- 172.21.15.142:0/908966288 shutdown_connections 2024-03-29T04:49:46.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.785+0000 7f93e9c98700 1 --2- 172.21.15.142:0/908966288 >> [v2:172.21.15.142:6800/1771239042,v1:172.21.15.142:6801/1771239042] conn(0x7f93cc06d9e0 0x7f93cc06fea0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.785+0000 7f93e9c98700 1 --2- 172.21.15.142:0/908966288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e40ffbd0 0x7f93e4196710 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.785+0000 7f93e9c98700 1 --2- 172.21.15.142:0/908966288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e4105a40 0x7f93e4196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:49:46.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.785+0000 7f93e9c98700 1 -- 172.21.15.142:0/908966288 >> 172.21.15.142:0/908966288 conn(0x7f93e40755a0 msgr2=0x7f93e41037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:49:46.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.785+0000 7f93e9c98700 1 -- 172.21.15.142:0/908966288 shutdown_connections 2024-03-29T04:49:46.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:49:46.786+0000 7f93e9c98700 1 -- 172.21.15.142:0/908966288 wait complete. 2024-03-29T04:49:46.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:46 smithi142 ceph-mon[29027]: Upgrade: Target is version 19.0.0-1959-g8da23624 (squid) 2024-03-29T04:49:46.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:46 smithi142 ceph-mon[29027]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28'] 2024-03-29T04:49:46.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:46 smithi142 ceph-mon[29027]: Upgrade: Setting container_image for all mgr 2024-03-29T04:49:46.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:46 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/810870210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:49:46.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:46 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/2431460876' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:49:46.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:46 smithi142 ceph-mon[29027]: pgmap v89: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:47.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:46 smithi152 ceph-mon[37177]: Upgrade: Target is version 19.0.0-1959-g8da23624 (squid) 2024-03-29T04:49:47.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:46 smithi152 ceph-mon[37177]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28'] 2024-03-29T04:49:47.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:46 smithi152 ceph-mon[37177]: Upgrade: Setting container_image for all mgr 2024-03-29T04:49:47.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:46 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/810870210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:49:47.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:46 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/2431460876' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:49:47.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:46 smithi152 ceph-mon[37177]: pgmap v89: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:47.866 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:47 smithi142 ceph-mon[29027]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:47.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:47 smithi142 ceph-mon[29027]: from='client.? 172.21.15.142:0/908966288' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:49:47.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:47 smithi142 ceph-mon[29027]: Upgrade: Updating mon.smithi142 2024-03-29T04:49:47.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:47 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:49:47.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:47 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:49:47.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:47 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T04:49:47.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:47 smithi142 ceph-mon[29027]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:49:47.867 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:47 smithi142 ceph-mon[29027]: Deploying daemon mon.smithi142 on smithi142 2024-03-29T04:49:48.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:47 smithi152 ceph-mon[37177]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:49:48.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:47 smithi152 ceph-mon[37177]: from='client.? 172.21.15.142:0/908966288' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:49:48.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:47 smithi152 ceph-mon[37177]: Upgrade: Updating mon.smithi142 2024-03-29T04:49:48.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:47 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' 2024-03-29T04:49:48.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:47 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:49:48.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:47 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T04:49:48.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:47 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:49:48.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:47 smithi152 ceph-mon[37177]: Deploying daemon mon.smithi142 on smithi142 2024-03-29T04:49:48.822 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:48 smithi142 ceph-mon[29027]: pgmap v90: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:49.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:48 smithi152 ceph-mon[37177]: pgmap v90: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:49.361 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:49 smithi142 systemd[1]: Stopping Ceph mon.smithi142 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:49:49.361 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:49 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[29023]: 2024-03-29T04:49:49.348+0000 7f1cb727e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi142 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T04:49:49.361 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:49 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[29023]: 2024-03-29T04:49:49.348+0000 7f1cb727e700 -1 mon.smithi142@0(leader) e2 *** Got Signal Terminated *** 2024-03-29T04:49:49.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:49 smithi142 podman[112308]: 2024-03-29 04:49:49.584109014 +0000 UTC m=+0.336381661 container died 32c014de54798a63a71db248997c8b52a5423a4f5793bf0b1e5fca634ceb7fad (image=quay.io/ceph/ceph:v18.2.1, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:49:50.211 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:49 smithi142 podman[112308]: 2024-03-29 04:49:49.99908974 +0000 UTC m=+0.751362397 container cleanup 32c014de54798a63a71db248997c8b52a5423a4f5793bf0b1e5fca634ceb7fad (image=quay.io/ceph/ceph:v18.2.1, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, org.label-schema.build-date=20240222, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, GIT_CLEAN=True, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:49:50.211 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:49 smithi142 bash[112308]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142 2024-03-29T04:49:50.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:50 smithi142 podman[112333]: 2024-03-29 04:49:50.20857093 +0000 UTC m=+0.625247763 container remove 32c014de54798a63a71db248997c8b52a5423a4f5793bf0b1e5fca634ceb7fad (image=quay.io/ceph/ceph:v18.2.1, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD) 2024-03-29T04:49:50.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:50 smithi142 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi142.service: Deactivated successfully. 2024-03-29T04:49:50.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:50 smithi142 systemd[1]: Stopped Ceph mon.smithi142 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:49:50.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:50 smithi142 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi142.service: Consumed 14.135s CPU time. 2024-03-29T04:49:50.986 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:50 smithi142 systemd[1]: Starting Ceph mon.smithi142 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:49:50.987 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:50 smithi142 podman[112456]: 2024-03-29 04:49:50.769711293 +0000 UTC m=+0.021407775 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:49:50.987 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:50 smithi142 podman[112456]: 2024-03-29 04:49:50.876293166 +0000 UTC m=+0.127989626 container create 434f713a3fd23a14db6943020b23e47e6aba693f3c53a4cc888153bd3e00945c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-29T04:49:51.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 podman[112456]: 2024-03-29 04:49:51.086981293 +0000 UTC m=+0.338677763 container init 434f713a3fd23a14db6943020b23e47e6aba693f3c53a4cc888153bd3e00945c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.build-date=20240326, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-29T04:49:51.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 podman[112456]: 2024-03-29 04:49:51.090065201 +0000 UTC m=+0.341761670 container start 434f713a3fd23a14db6943020b23e47e6aba693f3c53a4cc888153bd3e00945c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-03-29T04:49:51.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: set uid:gid to 167:167 (ceph:ceph) 2024-03-29T04:49:51.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev), process ceph-mon, pid 2 2024-03-29T04:49:51.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: pidfile_write: ignore empty --pid-file 2024-03-29T04:49:51.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: load: jerasure load: lrc 2024-03-29T04:49:51.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: RocksDB version: 7.9.2 2024-03-29T04:49:51.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Git sha 0 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Compile date 2024-03-28 15:35:57 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: DB SUMMARY 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: DB Session ID: U9EICSO3TPZBNM0AMMZ9 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: CURRENT file: CURRENT 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: IDENTITY file: IDENTITY 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1517 Bytes 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi142/store.db dir, Total Num: 1, files: 000032.sst 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi142/store.db: 000030.log size: 5204036 ; 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.error_if_exists: 0 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.create_if_missing: 0 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.paranoid_checks: 1 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-29T04:49:51.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-29T04:49:51.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.env: 0x556f2bc74c60 2024-03-29T04:49:51.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.fs: PosixFileSystem 2024-03-29T04:49:51.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.info_log: 0x556f2e4c79a0 2024-03-29T04:49:51.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_file_opening_threads: 16 2024-03-29T04:49:51.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.statistics: (nil) 2024-03-29T04:49:51.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.use_fsync: 0 2024-03-29T04:49:51.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_log_file_size: 0 2024-03-29T04:49:51.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-29T04:49:51.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-29T04:49:51.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.keep_log_file_num: 1000 2024-03-29T04:49:51.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.recycle_log_file_num: 0 2024-03-29T04:49:51.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.allow_fallocate: 1 2024-03-29T04:49:51.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.allow_mmap_reads: 0 2024-03-29T04:49:51.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.allow_mmap_writes: 0 2024-03-29T04:49:51.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.use_direct_reads: 0 2024-03-29T04:49:51.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-29T04:49:51.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.create_missing_column_families: 0 2024-03-29T04:49:51.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.db_log_dir: 2024-03-29T04:49:51.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.wal_dir: 2024-03-29T04:49:51.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-29T04:49:51.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-29T04:49:51.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-29T04:49:51.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-29T04:49:51.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-29T04:49:51.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-29T04:49:51.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.advise_random_on_open: 1 2024-03-29T04:49:51.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.db_write_buffer_size: 0 2024-03-29T04:49:51.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.write_buffer_manager: 0x556f2e49a1e0 2024-03-29T04:49:51.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-29T04:49:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-29T04:49:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-29T04:49:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.rate_limiter: (nil) 2024-03-29T04:49:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-29T04:49:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.wal_recovery_mode: 2 2024-03-29T04:49:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.enable_thread_tracking: 0 2024-03-29T04:49:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.enable_pipelined_write: 0 2024-03-29T04:49:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.unordered_write: 0 2024-03-29T04:49:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-29T04:49:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-29T04:49:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-29T04:49:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.row_cache: None 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.wal_filter: None 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.allow_ingest_behind: 0 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.two_write_queues: 0 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.manual_wal_flush: 0 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.wal_compression: 0 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.atomic_flush: 0 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.log_readahead_size: 0 2024-03-29T04:49:51.382 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.best_efforts_recovery: 0 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.allow_data_in_errors: 0 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.db_host_id: __hostname__ 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_background_jobs: 2 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_background_compactions: -1 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_subcompactions: 1 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_total_wal_size: 0 2024-03-29T04:49:51.383 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_open_files: -1 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bytes_per_sync: 0 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_readahead_size: 0 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_background_flushes: -1 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Compression algorithms supported: 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: kZSTD supported: 0 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: kXpressCompression supported: 0 2024-03-29T04:49:51.384 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: kLZ4HCCompression supported: 1 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: kZlibCompression supported: 1 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: kSnappyCompression supported: 1 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: kLZ4Compression supported: 1 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: kBZip2Compression supported: 0 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi142/store.db/MANIFEST-000015 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.merge_operator: 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_filter: None 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_filter_factory: None 2024-03-29T04:49:51.385 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.sst_partitioner_factory: None 2024-03-29T04:49:51.386 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-29T04:49:51.386 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-29T04:49:51.386 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556f2e4c7aa0) 2024-03-29T04:49:51.386 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: cache_index_and_filter_blocks: 1 2024-03-29T04:49:51.386 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-29T04:49:51.386 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-29T04:49:51.386 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: pin_top_level_index_and_filter: 1 2024-03-29T04:49:51.386 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: index_type: 0 2024-03-29T04:49:51.386 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: data_block_index_type: 0 2024-03-29T04:49:51.386 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: index_shortening: 1 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: checksum: 4 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: no_block_cache: 0 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache: 0x556f2e4cb090 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache_name: BinnedLRUCache 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache_options: 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: capacity : 536870912 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: num_shard_bits : 4 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: strict_capacity_limit : 0 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: high_pri_pool_ratio: 0.000 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_cache_compressed: (nil) 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: persistent_cache: (nil) 2024-03-29T04:49:51.387 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_size: 4096 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_size_deviation: 10 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_restart_interval: 16 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: index_block_restart_interval: 1 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: metadata_block_size: 4096 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: partition_filters: 0 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: use_delta_encoding: 1 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: filter_policy: bloomfilter 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: whole_key_filtering: 1 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: verify_compression: 0 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: read_amp_bytes_per_bit: 0 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: format_version: 5 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: enable_index_compression: 1 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: block_align: 0 2024-03-29T04:49:51.388 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: max_auto_readahead_size: 262144 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: prepopulate_block_cache: 0 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: initial_auto_readahead_size: 8192 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.write_buffer_size: 33554432 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_write_buffer_number: 2 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compression: NoCompression 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bottommost_compression: Disabled 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.prefix_extractor: nullptr 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.num_levels: 7 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-29T04:49:51.389 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compression_opts.level: 32767 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compression_opts.strategy: 0 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-29T04:49:51.390 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compression_opts.enabled: false 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.target_file_size_base: 67108864 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-29T04:49:51.391 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.arena_block_size: 1048576 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.disable_auto_compactions: 0 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-29T04:49:51.392 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.table_properties_collectors: 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.inplace_update_support: 0 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-29T04:49:51.393 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.bloom_locality: 0 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.max_successive_merges: 0 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.paranoid_file_checks: 0 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.force_consistency_checks: 1 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.report_bg_io_stats: 0 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.ttl: 2592000 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.enable_blob_files: false 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.min_blob_size: 0 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.blob_file_size: 268435456 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-29T04:49:51.394 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.blob_file_starting_level: 0 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi142/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12305, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 63462697-da50-4f71-bff3-8fd7e1db5fe3 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711687791119847, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711687791137908, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 4601299, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12306, "largest_seqno": 15777, "table_properties": {"data_size": 4587744, "index_size": 8630, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3909, "raw_key_size": 34153, "raw_average_key_size": 21, "raw_value_size": 4557028, "raw_average_value_size": 2934, "num_data_blocks": 392, "num_entries": 1553, "num_filter_entries": 1553, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711687791, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "63462697-da50-4f71-bff3-8fd7e1db5fe3", "db_session_id": "U9EICSO3TPZBNM0AMMZ9", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711687791138063, "job": 1, "event": "recovery_finished"} 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-03-29T04:49:51.395 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi142/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556f2e5bc000 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: DB pointer 0x556f2e5a6000 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** DB Stats ** 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-29T04:49:51.396 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** Compaction Stats [default] ** 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: L0 1/0 4.39 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 534.4 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: L6 1/0 4.87 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Sum 2/0 9.26 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 534.4 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 534.4 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** Compaction Stats [default] ** 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 534.4 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:49:51.397 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Cumulative compaction: 0.00 GB write, 210.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Interval compaction: 0.00 GB write, 210.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Block cache BinnedLRUCache@0x556f2e4cb090#2 capacity: 512.00 MB usage: 13.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,9.08 KB,0.00173151%) Misc(1,0.00 KB,0%) 2024-03-29T04:49:51.398 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: starting mon.smithi142 rank 0 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi142 fsid 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: mon.smithi142@-1(???) e2 preinit fsid 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: mon.smithi142@-1(???).mds e21 new map 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: mon.smithi142@-1(???).mds e21 print_map 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: e21 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: legacy client fscid: 1 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: Filesystem 'cephfs' (1) 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: fs_name cephfs 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: epoch 21 2024-03-29T04:49:51.399 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: modified 2024-03-29T04:49:01.288637+0000 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: tableserver 0 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: root 0 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: session_timeout 60 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: session_autoclose 300 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: max_file_size 1099511627776 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: max_xattr_size 65536 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: required_client_features {} 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: last_failure 0 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: last_failure_osd_epoch 47 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: max_mds 2 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: in 0,1 2024-03-29T04:49:51.400 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: up {0=14494,1=24275} 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: failed 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: damaged 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: stopped 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: data_pools [3] 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: metadata_pool 2 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: inline_data disabled 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: balancer 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: bal_rank_mask -1 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: standby_count_wanted 1 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: qdb_cluster leader: 0 members: 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: [mds.cephfs.smithi142.bmkjzf{0:14494} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: [mds.cephfs.smithi142.wgmndr{0:14506} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: [mds.cephfs.smithi152.uhebsi{1:24275} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: [mds.cephfs.smithi152.hiilum{1:24287} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:49:51.401 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:49:51.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout: 2024-03-29T04:49:51.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: mon.smithi142@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-29T04:49:51.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: mon.smithi142@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-29T04:49:51.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: mon.smithi142@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-29T04:49:51.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: mon.smithi142@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-29T04:49:51.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 ceph-mon[112475]: mon.smithi142@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-03-29T04:49:51.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 bash[112456]: 434f713a3fd23a14db6943020b23e47e6aba693f3c53a4cc888153bd3e00945c 2024-03-29T04:49:51.402 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:51 smithi142 systemd[1]: Started Ceph mon.smithi142 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:49:52.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: pgmap v91: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: pgmap v92: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: mon.smithi142 calling monitor election 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: mon.smithi142 is new leader, mons smithi142,smithi152 in quorum (ranks 0,1) 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: monmap epoch 2 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: fsid 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: last_changed 2024-03-29T04:37:44.323694+0000 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: created 2024-03-29T04:35:04.327053+0000 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: min_mon_release 18 (reef) 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: election_strategy: 1 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: 0: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.smithi142 2024-03-29T04:49:52.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: 1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.smithi152 2024-03-29T04:49:52.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:49:52.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: osdmap e50: 6 total, 6 up, 6 in 2024-03-29T04:49:52.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: mgrmap e33: smithi142.swesrg(active, since 2m), standbys: smithi152.usoolh 2024-03-29T04:49:52.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: overall HEALTH_OK 2024-03-29T04:49:52.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: from='mgr.24465 ' entity='' 2024-03-29T04:49:52.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:52 smithi142 ceph-mon[112475]: mgrmap e34: smithi142.swesrg(active, since 2m), standbys: smithi152.usoolh 2024-03-29T04:49:53.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:49:53.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: from='mgr.24465 172.21.15.142:0/1748266435' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:49:53.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: pgmap v91: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:49:53.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: pgmap v92: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:49:53.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: mon.smithi142 calling monitor election 2024-03-29T04:49:53.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: mon.smithi142 is new leader, mons smithi142,smithi152 in quorum (ranks 0,1) 2024-03-29T04:49:53.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: monmap epoch 2 2024-03-29T04:49:53.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: fsid 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:49:53.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: last_changed 2024-03-29T04:37:44.323694+0000 2024-03-29T04:49:53.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: created 2024-03-29T04:35:04.327053+0000 2024-03-29T04:49:53.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: min_mon_release 18 (reef) 2024-03-29T04:49:53.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: election_strategy: 1 2024-03-29T04:49:53.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: 0: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.smithi142 2024-03-29T04:49:53.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: 1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.smithi152 2024-03-29T04:49:53.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:49:53.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: osdmap e50: 6 total, 6 up, 6 in 2024-03-29T04:49:53.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: mgrmap e33: smithi142.swesrg(active, since 2m), standbys: smithi152.usoolh 2024-03-29T04:49:53.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: overall HEALTH_OK 2024-03-29T04:49:53.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: from='mgr.24465 ' entity='' 2024-03-29T04:49:53.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:52 smithi152 ceph-mon[37177]: mgrmap e34: smithi142.swesrg(active, since 2m), standbys: smithi152.usoolh 2024-03-29T04:49:59.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:59 smithi142 ceph-mon[112475]: Standby manager daemon smithi152.usoolh restarted 2024-03-29T04:49:59.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:59 smithi142 ceph-mon[112475]: Standby manager daemon smithi152.usoolh started 2024-03-29T04:49:59.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:59 smithi142 ceph-mon[112475]: from='mgr.? 172.21.15.152:0/169222942' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/crt"}]: dispatch 2024-03-29T04:49:59.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:59 smithi142 ceph-mon[112475]: from='mgr.? 172.21.15.152:0/169222942' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:49:59.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:59 smithi142 ceph-mon[112475]: from='mgr.? 172.21.15.152:0/169222942' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/key"}]: dispatch 2024-03-29T04:49:59.506 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:49:59 smithi142 ceph-mon[112475]: from='mgr.? 172.21.15.152:0/169222942' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:49:59.616 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:59 smithi152 ceph-mon[37177]: Standby manager daemon smithi152.usoolh restarted 2024-03-29T04:49:59.617 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:59 smithi152 ceph-mon[37177]: Standby manager daemon smithi152.usoolh started 2024-03-29T04:49:59.617 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:59 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/169222942' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/crt"}]: dispatch 2024-03-29T04:49:59.617 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:59 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/169222942' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T04:49:59.617 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:59 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/169222942' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi152.usoolh/key"}]: dispatch 2024-03-29T04:49:59.617 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:49:59 smithi152 ceph-mon[37177]: from='mgr.? 172.21.15.152:0/169222942' entity='mgr.smithi152.usoolh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T04:50:00.662 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: mgrmap e35: smithi142.swesrg(active, since 3m), standbys: smithi152.usoolh 2024-03-29T04:50:00.662 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: Active manager daemon smithi142.swesrg restarted 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: Activating manager daemon smithi142.swesrg 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: osdmap e51: 6 total, 6 up, 6 in 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: mgrmap e36: smithi142.swesrg(active, starting, since 0.0058994s), standbys: smithi152.usoolh 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi142.swesrg", "id": "smithi142.swesrg"}]: dispatch 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi152.usoolh", "id": "smithi152.usoolh"}]: dispatch 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:50:00.663 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: Manager daemon smithi142.swesrg is now available 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/trash_purge_schedule"}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/trash_purge_schedule"}]: dispatch 2024-03-29T04:50:00.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:00 smithi152 ceph-mon[37177]: overall HEALTH_OK 2024-03-29T04:50:00.703 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: mgrmap e35: smithi142.swesrg(active, since 3m), standbys: smithi152.usoolh 2024-03-29T04:50:00.703 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: Active manager daemon smithi142.swesrg restarted 2024-03-29T04:50:00.703 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: Activating manager daemon smithi142.swesrg 2024-03-29T04:50:00.703 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: osdmap e51: 6 total, 6 up, 6 in 2024-03-29T04:50:00.703 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: mgrmap e36: smithi142.swesrg(active, starting, since 0.0058994s), standbys: smithi152.usoolh 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi142.swesrg", "id": "smithi142.swesrg"}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr metadata", "who": "smithi152.usoolh", "id": "smithi152.usoolh"}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:50:00.704 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:50:00.705 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T04:50:00.705 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T04:50:00.705 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T04:50:00.705 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: Manager daemon smithi142.swesrg is now available 2024-03-29T04:50:00.705 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:50:00.705 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:50:00.705 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:50:00.705 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/mirror_snapshot_schedule"}]: dispatch 2024-03-29T04:50:00.705 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/trash_purge_schedule"}]: dispatch 2024-03-29T04:50:00.705 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi142.swesrg/trash_purge_schedule"}]: dispatch 2024-03-29T04:50:00.705 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:00 smithi142 ceph-mon[112475]: overall HEALTH_OK 2024-03-29T04:50:01.704 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:01 smithi152 ceph-mon[37177]: mgrmap e37: smithi142.swesrg(active, since 1.0115s), standbys: smithi152.usoolh 2024-03-29T04:50:01.704 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:01 smithi152 ceph-mon[37177]: pgmap v3: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:50:01.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:01 smithi142 ceph-mon[112475]: mgrmap e37: smithi142.swesrg(active, since 1.0115s), standbys: smithi152.usoolh 2024-03-29T04:50:01.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:01 smithi142 ceph-mon[112475]: pgmap v3: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:50:02.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:02 smithi152 ceph-mon[37177]: [29/Mar/2024:04:50:01] ENGINE Bus STARTING 2024-03-29T04:50:02.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:02 smithi152 ceph-mon[37177]: [29/Mar/2024:04:50:01] ENGINE Serving on https://172.21.15.142:7150 2024-03-29T04:50:02.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:02 smithi152 ceph-mon[37177]: [29/Mar/2024:04:50:01] ENGINE Serving on http://172.21.15.142:8765 2024-03-29T04:50:02.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:02 smithi152 ceph-mon[37177]: [29/Mar/2024:04:50:01] ENGINE Bus STARTED 2024-03-29T04:50:02.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:02 smithi152 ceph-mon[37177]: pgmap v4: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:50:02.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:02 smithi152 ceph-mon[37177]: mgrmap e38: smithi142.swesrg(active, since 2s), standbys: smithi152.usoolh 2024-03-29T04:50:02.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:02 smithi142 ceph-mon[112475]: [29/Mar/2024:04:50:01] ENGINE Bus STARTING 2024-03-29T04:50:02.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:02 smithi142 ceph-mon[112475]: [29/Mar/2024:04:50:01] ENGINE Serving on https://172.21.15.142:7150 2024-03-29T04:50:02.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:02 smithi142 ceph-mon[112475]: [29/Mar/2024:04:50:01] ENGINE Serving on http://172.21.15.142:8765 2024-03-29T04:50:02.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:02 smithi142 ceph-mon[112475]: [29/Mar/2024:04:50:01] ENGINE Bus STARTED 2024-03-29T04:50:02.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:02 smithi142 ceph-mon[112475]: pgmap v4: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:50:02.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:02 smithi142 ceph-mon[112475]: mgrmap e38: smithi142.swesrg(active, since 2s), standbys: smithi152.usoolh 2024-03-29T04:50:04.124 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:03 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:04.124 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:03 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:04.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:03 smithi142 ceph-mon[112475]: pgmap v5: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:50:04.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:03 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:04.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:03 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:04.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:03 smithi152 ceph-mon[37177]: pgmap v5: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:50:05.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:04 smithi152 ceph-mon[37177]: mgrmap e39: smithi142.swesrg(active, since 4s), standbys: smithi152.usoolh 2024-03-29T04:50:05.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:04 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:05.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:04 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:05.124 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:04 smithi142 ceph-mon[112475]: mgrmap e39: smithi142.swesrg(active, since 4s), standbys: smithi152.usoolh 2024-03-29T04:50:05.124 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:04 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:05.124 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:04 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:06.573 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:06 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:06.574 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:06 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:06.574 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:06 smithi152 ceph-mon[37177]: pgmap v6: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 0 B/s wr, 26 op/s 2024-03-29T04:50:06.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:06 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:06.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:06 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:06.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:06 smithi142 ceph-mon[112475]: pgmap v6: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 0 B/s wr, 26 op/s 2024-03-29T04:50:07.443 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:07 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:07.443 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:07 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:07.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:07 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:07.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:07 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:08.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:08 smithi152 ceph-mon[37177]: Detected new or changed devices on smithi152 2024-03-29T04:50:08.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:08 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:08.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:08 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:08.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:08 smithi152 ceph-mon[37177]: pgmap v7: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-29T04:50:08.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:08 smithi142 ceph-mon[112475]: Detected new or changed devices on smithi152 2024-03-29T04:50:08.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:08 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:08.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:08 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:08.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:08 smithi142 ceph-mon[112475]: pgmap v7: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-29T04:50:10.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:10 smithi142 ceph-mon[112475]: pgmap v8: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-03-29T04:50:10.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:10 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:10.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:10 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:10.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:10.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:50:11.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:10 smithi152 ceph-mon[37177]: pgmap v8: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-03-29T04:50:11.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:10 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:11.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:10 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:11.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:10 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:11.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:10 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:50:11.632 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: Detected new or changed devices on smithi142 2024-03-29T04:50:11.632 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-29T04:50:11.632 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-29T04:50:11.632 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:50:11.632 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:50:11.632 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:50:11.633 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:50:11.633 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:50:11.633 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:50:11.633 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:11.633 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:11.633 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:11.633 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:11.633 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:11.633 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:50:11.633 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:50:11.633 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:11 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-29T04:50:11.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: Detected new or changed devices on smithi142 2024-03-29T04:50:11.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-29T04:50:11.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-29T04:50:11.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:50:11.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-29T04:50:11.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:50:11.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-03-29T04:50:11.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: Updating smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:50:11.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: Updating smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-29T04:50:11.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:11.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:11.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:11.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:11.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:11.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:50:11.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:50:11.792 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:11 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-29T04:50:12.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:12 smithi152 ceph-mon[37177]: pgmap v9: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-29T04:50:12.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:12 smithi152 ceph-mon[37177]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:12.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:12 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:50:12.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:12 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T04:50:12.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:12 smithi152 ceph-mon[37177]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:12.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:12 smithi142 ceph-mon[112475]: pgmap v9: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-29T04:50:12.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:12 smithi142 ceph-mon[112475]: from='mgr.34104 ' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:12.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:12 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:50:12.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:12 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T04:50:12.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:12 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:13.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:13 smithi142 ceph-mon[112475]: Upgrade: Updating mon.smithi152 2024-03-29T04:50:13.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:13 smithi142 ceph-mon[112475]: Deploying daemon mon.smithi152 on smithi152 2024-03-29T04:50:13.928 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:13 smithi152 ceph-mon[37177]: Upgrade: Updating mon.smithi152 2024-03-29T04:50:13.928 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:13 smithi152 ceph-mon[37177]: Deploying daemon mon.smithi152 on smithi152 2024-03-29T04:50:14.492 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:14 smithi152 systemd[1]: Stopping Ceph mon.smithi152 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:50:14.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:14 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152[37171]: 2024-03-29T04:50:14.487+0000 7f649f0a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi152 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T04:50:14.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:14 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152[37171]: 2024-03-29T04:50:14.487+0000 7f649f0a1700 -1 mon.smithi152@1(peon) e2 *** Got Signal Terminated *** 2024-03-29T04:50:14.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:14 smithi152 podman[88025]: 2024-03-29 04:50:14.733297978 +0000 UTC m=+0.376867080 container died defcc70499d901aaa1b677159d548a85d392b6552f960844aa4202bae16020e3 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-29T04:50:15.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:15 smithi152 podman[88025]: 2024-03-29 04:50:15.138172453 +0000 UTC m=+0.781741555 container cleanup defcc70499d901aaa1b677159d548a85d392b6552f960844aa4202bae16020e3 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152, org.label-schema.build-date=20240222, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-03-29T04:50:15.293 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:15 smithi152 bash[88025]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152 2024-03-29T04:50:15.691 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:15 smithi152 podman[88039]: 2024-03-29 04:50:15.373020818 +0000 UTC m=+0.638805478 container remove defcc70499d901aaa1b677159d548a85d392b6552f960844aa4202bae16020e3 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, RELEASE=HEAD) 2024-03-29T04:50:15.692 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:15 smithi152 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi152.service: Deactivated successfully. 2024-03-29T04:50:15.692 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:15 smithi152 systemd[1]: Stopped Ceph mon.smithi152 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:50:15.692 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:15 smithi152 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi152.service: Consumed 8.060s CPU time. 2024-03-29T04:50:16.030 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:15 smithi152 systemd[1]: Starting Ceph mon.smithi152 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:50:16.030 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:15 smithi152 podman[88167]: 2024-03-29 04:50:15.805376385 +0000 UTC m=+0.017360598 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:50:16.030 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:15 smithi152 podman[88167]: 2024-03-29 04:50:15.916154577 +0000 UTC m=+0.128138783 container create 4ad4708e8405d5b4877453023a469ee756500171220edd9bfc6a176829a9244e (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, RELEASE=squid-8da2362, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-29T04:50:16.214 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 podman[88167]: 2024-03-29 04:50:16.177561584 +0000 UTC m=+0.389545804 container init 4ad4708e8405d5b4877453023a469ee756500171220edd9bfc6a176829a9244e (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8da2362, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-29T04:50:16.215 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 podman[88167]: 2024-03-29 04:50:16.180935751 +0000 UTC m=+0.392919966 container start 4ad4708e8405d5b4877453023a469ee756500171220edd9bfc6a176829a9244e (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=squid-8da2362, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True) 2024-03-29T04:50:16.215 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: set uid:gid to 167:167 (ceph:ceph) 2024-03-29T04:50:16.215 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev), process ceph-mon, pid 2 2024-03-29T04:50:16.215 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: pidfile_write: ignore empty --pid-file 2024-03-29T04:50:16.215 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: load: jerasure load: lrc 2024-03-29T04:50:16.215 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: RocksDB version: 7.9.2 2024-03-29T04:50:16.215 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Git sha 0 2024-03-29T04:50:16.215 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Compile date 2024-03-28 15:35:57 2024-03-29T04:50:16.215 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: DB SUMMARY 2024-03-29T04:50:16.215 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: DB Session ID: RRLOW78HS6L1CLXMJTPV 2024-03-29T04:50:16.215 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: CURRENT file: CURRENT 2024-03-29T04:50:16.215 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: IDENTITY file: IDENTITY 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1411 Bytes 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi152/store.db dir, Total Num: 1, files: 000027.sst 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi152/store.db: 000025.log size: 10788535 ; 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.error_if_exists: 0 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.create_if_missing: 0 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.paranoid_checks: 1 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.env: 0x55947e72ec60 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.fs: PosixFileSystem 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.info_log: 0x559480c619a0 2024-03-29T04:50:16.216 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_file_opening_threads: 16 2024-03-29T04:50:16.217 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.statistics: (nil) 2024-03-29T04:50:16.217 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.use_fsync: 0 2024-03-29T04:50:16.217 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_log_file_size: 0 2024-03-29T04:50:16.217 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.keep_log_file_num: 1000 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.recycle_log_file_num: 0 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.allow_fallocate: 1 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.allow_mmap_reads: 0 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.allow_mmap_writes: 0 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.use_direct_reads: 0 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.create_missing_column_families: 0 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.db_log_dir: 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.wal_dir: 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-29T04:50:16.218 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.advise_random_on_open: 1 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.db_write_buffer_size: 0 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.write_buffer_manager: 0x559480c341e0 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.rate_limiter: (nil) 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-29T04:50:16.219 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.wal_recovery_mode: 2 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.enable_thread_tracking: 0 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.enable_pipelined_write: 0 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.unordered_write: 0 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.row_cache: None 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.wal_filter: None 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.allow_ingest_behind: 0 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.two_write_queues: 0 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.manual_wal_flush: 0 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.wal_compression: 0 2024-03-29T04:50:16.220 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.atomic_flush: 0 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.log_readahead_size: 0 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.best_efforts_recovery: 0 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.allow_data_in_errors: 0 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.db_host_id: __hostname__ 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_background_jobs: 2 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_background_compactions: -1 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_subcompactions: 1 2024-03-29T04:50:16.221 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_total_wal_size: 0 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_open_files: -1 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bytes_per_sync: 0 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_readahead_size: 0 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_background_flushes: -1 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Compression algorithms supported: 2024-03-29T04:50:16.222 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: kZSTD supported: 0 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: kXpressCompression supported: 0 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: kLZ4HCCompression supported: 1 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: kZlibCompression supported: 1 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: kSnappyCompression supported: 1 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: kLZ4Compression supported: 1 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: kBZip2Compression supported: 0 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi152/store.db/MANIFEST-000010 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.merge_operator: 2024-03-29T04:50:16.223 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_filter: None 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_filter_factory: None 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.sst_partitioner_factory: None 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559480c61aa0) 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: cache_index_and_filter_blocks: 1 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: pin_top_level_index_and_filter: 1 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: index_type: 0 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: data_block_index_type: 0 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: index_shortening: 1 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-29T04:50:16.224 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: checksum: 4 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: no_block_cache: 0 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: block_cache: 0x559480c65090 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: block_cache_name: BinnedLRUCache 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: block_cache_options: 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: capacity : 536870912 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: num_shard_bits : 4 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: strict_capacity_limit : 0 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: high_pri_pool_ratio: 0.000 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: block_cache_compressed: (nil) 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: persistent_cache: (nil) 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: block_size: 4096 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: block_size_deviation: 10 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: block_restart_interval: 16 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: index_block_restart_interval: 1 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: metadata_block_size: 4096 2024-03-29T04:50:16.225 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: partition_filters: 0 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: use_delta_encoding: 1 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: filter_policy: bloomfilter 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: whole_key_filtering: 1 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: verify_compression: 0 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: read_amp_bytes_per_bit: 0 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: format_version: 5 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: enable_index_compression: 1 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: block_align: 0 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: max_auto_readahead_size: 262144 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: prepopulate_block_cache: 0 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: initial_auto_readahead_size: 8192 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: num_file_reads_for_auto_readahead: 2 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.write_buffer_size: 33554432 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_write_buffer_number: 2 2024-03-29T04:50:16.226 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compression: NoCompression 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bottommost_compression: Disabled 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.prefix_extractor: nullptr 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.num_levels: 7 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-29T04:50:16.227 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compression_opts.level: 32767 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compression_opts.strategy: 0 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compression_opts.enabled: false 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.target_file_size_base: 67108864 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-29T04:50:16.228 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-29T04:50:16.229 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-29T04:50:16.229 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-29T04:50:16.229 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-29T04:50:16.229 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-29T04:50:16.229 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-29T04:50:16.229 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-29T04:50:16.229 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-29T04:50:16.229 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.arena_block_size: 1048576 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.disable_auto_compactions: 0 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-29T04:50:16.230 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.table_properties_collectors: 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.inplace_update_support: 0 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.bloom_locality: 0 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.max_successive_merges: 0 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.paranoid_file_checks: 0 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.force_consistency_checks: 1 2024-03-29T04:50:16.231 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.report_bg_io_stats: 0 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.ttl: 2592000 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.enable_blob_files: false 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.min_blob_size: 0 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.blob_file_size: 268435456 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.blob_file_starting_level: 0 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-29T04:50:16.232 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-29T04:50:16.233 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi152/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12370, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-03-29T04:50:16.233 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-03-29T04:50:16.233 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 13f07d13-2fc7-4da2-9de0-9c9736da580c 2024-03-29T04:50:16.233 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711687816210496, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-03-29T04:50:16.233 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-03-29T04:50:16.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711687816237182, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 6366573, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12375, "largest_seqno": 16728, "table_properties": {"data_size": 6350884, "index_size": 10252, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4421, "raw_key_size": 39079, "raw_average_key_size": 22, "raw_value_size": 6315686, "raw_average_value_size": 3578, "num_data_blocks": 461, "num_entries": 1765, "num_filter_entries": 1765, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711687816, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "13f07d13-2fc7-4da2-9de0-9c9736da580c", "db_session_id": "RRLOW78HS6L1CLXMJTPV", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-03-29T04:50:16.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711687816237325, "job": 1, "event": "recovery_finished"} 2024-03-29T04:50:16.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-03-29T04:50:16.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-29T04:50:16.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi152/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-29T04:50:16.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559480d56000 2024-03-29T04:50:16.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: DB pointer 0x559480d40000 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: ** DB Stats ** 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: ** Compaction Stats [default] ** 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-29T04:50:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: L0 1/0 6.07 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 593.8 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: L6 1/0 4.87 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Sum 2/0 10.95 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 593.8 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 593.8 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: ** Compaction Stats [default] ** 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 593.8 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T04:50:16.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Flush(GB): cumulative 0.006, interval 0.006 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Cumulative compaction: 0.01 GB write, 204.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Interval compaction: 0.01 GB write, 204.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Block cache BinnedLRUCache@0x559480c65090#2 capacity: 512.00 MB usage: 16.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.5e-05 secs_since: 0 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,12.08 KB,0.00230372%) Misc(1,0.00 KB,0%) 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: starting mon.smithi152 rank 1 at public addrs [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] at bind addrs [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi152 fsid 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: mon.smithi152@-1(???) e2 preinit fsid 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:50:16.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: mon.smithi152@-1(???).mds e21 new map 2024-03-29T04:50:16.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: mon.smithi152@-1(???).mds e21 print_map 2024-03-29T04:50:16.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: e21 2024-03-29T04:50:16.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:50:16.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:50:16.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: legacy client fscid: 1 2024-03-29T04:50:16.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: 2024-03-29T04:50:16.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: Filesystem 'cephfs' (1) 2024-03-29T04:50:16.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: fs_name cephfs 2024-03-29T04:50:16.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: epoch 21 2024-03-29T04:50:16.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: modified 2024-03-29T04:49:01.288637+0000 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: tableserver 0 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: root 0 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: session_timeout 60 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: session_autoclose 300 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: max_file_size 1099511627776 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: max_xattr_size 65536 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: required_client_features {} 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: last_failure 0 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: last_failure_osd_epoch 47 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: max_mds 2 2024-03-29T04:50:16.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: in 0,1 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: up {0=14494,1=24275} 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: failed 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: damaged 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: stopped 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: data_pools [3] 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: metadata_pool 2 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: inline_data disabled 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: balancer 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: bal_rank_mask -1 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: standby_count_wanted 1 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: qdb_cluster leader: 0 members: 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: [mds.cephfs.smithi142.bmkjzf{0:14494} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:50:16.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: [mds.cephfs.smithi142.wgmndr{0:14506} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:50:16.551 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: [mds.cephfs.smithi152.uhebsi{1:24275} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:50:16.551 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: [mds.cephfs.smithi152.hiilum{1:24287} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:50:16.551 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: 2024-03-29T04:50:16.551 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout: 2024-03-29T04:50:16.551 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: mon.smithi152@-1(???).osd e51 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-29T04:50:16.551 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: mon.smithi152@-1(???).osd e51 crush map has features 288514051259236352, adjusting msgr requires 2024-03-29T04:50:16.551 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: mon.smithi152@-1(???).osd e51 crush map has features 288514051259236352, adjusting msgr requires 2024-03-29T04:50:16.551 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: mon.smithi152@-1(???).osd e51 crush map has features 288514051259236352, adjusting msgr requires 2024-03-29T04:50:16.551 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 ceph-mon[88186]: mon.smithi152@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-03-29T04:50:16.551 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 bash[88167]: 4ad4708e8405d5b4877453023a469ee756500171220edd9bfc6a176829a9244e 2024-03-29T04:50:16.551 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:16 smithi152 systemd[1]: Started Ceph mon.smithi152 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:50:16.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.909+0000 7f39b359e700 1 -- 172.21.15.142:0/2523012847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39b4101b60 msgr2=0x7f39b4101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:16.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.909+0000 7f39b359e700 1 --2- 172.21.15.142:0/2523012847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39b4101b60 0x7f39b4101f40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009910 tx=0x7f39a8009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:16.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.910+0000 7f39b359e700 1 -- 172.21.15.142:0/2523012847 shutdown_connections 2024-03-29T04:50:16.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.910+0000 7f39b359e700 1 --2- 172.21.15.142:0/2523012847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39b4102480 0x7f39b410a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:16.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.910+0000 7f39b359e700 1 --2- 172.21.15.142:0/2523012847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39b4101b60 0x7f39b4101f40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:16.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.910+0000 7f39b359e700 1 -- 172.21.15.142:0/2523012847 >> 172.21.15.142:0/2523012847 conn(0x7f39b40fb390 msgr2=0x7f39b40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:16.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.910+0000 7f39b359e700 1 -- 172.21.15.142:0/2523012847 shutdown_connections 2024-03-29T04:50:16.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.910+0000 7f39b359e700 1 -- 172.21.15.142:0/2523012847 wait complete. 2024-03-29T04:50:16.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b359e700 1 Processor -- start 2024-03-29T04:50:16.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b359e700 1 -- start start 2024-03-29T04:50:16.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b359e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39b4101b60 0x7f39b4196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:16.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b359e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39b4102480 0x7f39b4196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:16.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b359e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4197100 con 0x7f39b4102480 2024-03-29T04:50:16.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b359e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4197240 con 0x7f39b4101b60 2024-03-29T04:50:16.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39b4102480 0x7f39b4196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:16.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39b4102480 0x7f39b4196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52140/0 (socket says 172.21.15.142:52140) 2024-03-29T04:50:16.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b1d9b700 1 -- 172.21.15.142:0/1344348696 learned_addr learned my addr 172.21.15.142:0/1344348696 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:16.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b259c700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39b4101b60 0x7f39b4196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b259c700 1 -- 172.21.15.142:0/1344348696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39b4101b60 msgr2=0x7f39b4196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b259c700 1 -- 172.21.15.142:0/1344348696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39b4101b60 msgr2=0x7f39b4196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b259c700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39b4101b60 0x7f39b4196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b259c700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39b4101b60 0x7f39b4196570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b1d9b700 1 -- 172.21.15.142:0/1344348696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39b4101b60 msgr2=0x7f39b4196570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b1d9b700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39b4101b60 0x7f39b4196570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.911+0000 7f39b1d9b700 1 -- 172.21.15.142:0/1344348696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80095f0 con 0x7f39b4102480 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.912+0000 7f39b1d9b700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39b4102480 0x7f39b4196ab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c8f0 tx=0x7f39a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.912+0000 7f39a37fe700 1 -- 172.21.15.142:0/1344348696 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39a4007ad0 con 0x7f39b4102480 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.912+0000 7f39b359e700 1 -- 172.21.15.142:0/1344348696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b419b090 con 0x7f39b4102480 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.912+0000 7f39a37fe700 1 -- 172.21.15.142:0/1344348696 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39a400ce90 con 0x7f39b4102480 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.912+0000 7f39a37fe700 1 -- 172.21.15.142:0/1344348696 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39a4018630 con 0x7f39b4102480 2024-03-29T04:50:16.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.912+0000 7f39b359e700 1 -- 172.21.15.142:0/1344348696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b419b5e0 con 0x7f39b4102480 2024-03-29T04:50:16.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.913+0000 7f39b359e700 1 -- 172.21.15.142:0/1344348696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b4108070 con 0x7f39b4102480 2024-03-29T04:50:16.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.913+0000 7f39a37fe700 1 -- 172.21.15.142:0/1344348696 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7f39a400f450 con 0x7f39b4102480 2024-03-29T04:50:16.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.914+0000 7f39a37fe700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f399c06de90 0x7f399c070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:16.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.914+0000 7f39a37fe700 1 -- 172.21.15.142:0/1344348696 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f39a408dc30 con 0x7f39b4102480 2024-03-29T04:50:16.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.914+0000 7f39b259c700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f399c06de90 0x7f399c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:16.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.914+0000 7f39b259c700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f399c06de90 0x7f399c070350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f39a8006010 tx=0x7f39a801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:16.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:16.919+0000 7f39a37fe700 1 -- 172.21.15.142:0/1344348696 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f39a4057080 con 0x7f39b4102480 2024-03-29T04:50:17.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.134+0000 7f39b359e700 1 -- 172.21.15.142:0/1344348696 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f39b402cd60 con 0x7f399c06de90 2024-03-29T04:50:17.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.136+0000 7f39a37fe700 1 -- 172.21.15.142:0/1344348696 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f39b402cd60 con 0x7f399c06de90 2024-03-29T04:50:17.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.138+0000 7f39b359e700 1 -- 172.21.15.142:0/1344348696 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f399c06de90 msgr2=0x7f399c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:17.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.138+0000 7f39b359e700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f399c06de90 0x7f399c070350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f39a8006010 tx=0x7f39a801a040 comp rx=0 tx=0).stop 2024-03-29T04:50:17.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.138+0000 7f39b359e700 1 -- 172.21.15.142:0/1344348696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39b4102480 msgr2=0x7f39b4196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:17.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.138+0000 7f39b359e700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39b4102480 0x7f39b4196ab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c8f0 tx=0x7f39a400ccb0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.138+0000 7f39b359e700 1 -- 172.21.15.142:0/1344348696 shutdown_connections 2024-03-29T04:50:17.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.138+0000 7f39b359e700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f399c06de90 0x7f399c070350 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.138+0000 7f39b359e700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39b4101b60 0x7f39b4196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.138+0000 7f39b359e700 1 --2- 172.21.15.142:0/1344348696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39b4102480 0x7f39b4196ab0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.138+0000 7f39b359e700 1 -- 172.21.15.142:0/1344348696 >> 172.21.15.142:0/1344348696 conn(0x7f39b40fb390 msgr2=0x7f39b41051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:17.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.138+0000 7f39b359e700 1 -- 172.21.15.142:0/1344348696 shutdown_connections 2024-03-29T04:50:17.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.138+0000 7f39b359e700 1 -- 172.21.15.142:0/1344348696 wait complete. 2024-03-29T04:50:17.152 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:50:17.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.242+0000 7f91442b7700 1 -- 172.21.15.142:0/2297661472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f913c1017a0 msgr2=0x7f913c1057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.242+0000 7f91442b7700 1 --2- 172.21.15.142:0/2297661472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f913c1017a0 0x7f913c1057f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9138009910 tx=0x7f9138009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.242+0000 7f91442b7700 1 -- 172.21.15.142:0/2297661472 shutdown_connections 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.242+0000 7f91442b7700 1 --2- 172.21.15.142:0/2297661472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f913c1017a0 0x7f913c1057f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.242+0000 7f91442b7700 1 --2- 172.21.15.142:0/2297661472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f913c100df0 0x7f913c1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.242+0000 7f91442b7700 1 -- 172.21.15.142:0/2297661472 >> 172.21.15.142:0/2297661472 conn(0x7f913c0fc6a0 msgr2=0x7f913c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.242+0000 7f91442b7700 1 -- 172.21.15.142:0/2297661472 shutdown_connections 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.242+0000 7f91442b7700 1 -- 172.21.15.142:0/2297661472 wait complete. 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f91442b7700 1 Processor -- start 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f91442b7700 1 -- start start 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f91442b7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f913c100df0 0x7f913c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f91442b7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f913c1017a0 0x7f913c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f91442b7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c19f400 con 0x7f913c1017a0 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f91442b7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c1988b0 con 0x7f913c100df0 2024-03-29T04:50:17.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f9141852700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f913c1017a0 0x7f913c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:17.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f9141852700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f913c1017a0 0x7f913c19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52164/0 (socket says 172.21.15.142:52164) 2024-03-29T04:50:17.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f9141852700 1 -- 172.21.15.142:0/4037878567 learned_addr learned my addr 172.21.15.142:0/4037878567 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:17.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f9141852700 1 -- 172.21.15.142:0/4037878567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f913c100df0 msgr2=0x7f913c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:17.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f9142053700 1 --2- 172.21.15.142:0/4037878567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f913c100df0 0x7f913c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:17.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f9141852700 1 --2- 172.21.15.142:0/4037878567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f913c100df0 0x7f913c19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.243+0000 7f9141852700 1 -- 172.21.15.142:0/4037878567 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91380095f0 con 0x7f913c1017a0 2024-03-29T04:50:17.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.244+0000 7f9141852700 1 --2- 172.21.15.142:0/4037878567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f913c1017a0 0x7f913c19ed70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91380092e0 tx=0x7f913800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:17.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.244+0000 7f91337fe700 1 -- 172.21.15.142:0/4037878567 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f913801d070 con 0x7f913c1017a0 2024-03-29T04:50:17.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.244+0000 7f91337fe700 1 -- 172.21.15.142:0/4037878567 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f913800f460 con 0x7f913c1017a0 2024-03-29T04:50:17.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.244+0000 7f91442b7700 1 -- 172.21.15.142:0/4037878567 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c198b30 con 0x7f913c1017a0 2024-03-29T04:50:17.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.244+0000 7f91337fe700 1 -- 172.21.15.142:0/4037878567 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91380175f0 con 0x7f913c1017a0 2024-03-29T04:50:17.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.244+0000 7f91442b7700 1 -- 172.21.15.142:0/4037878567 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c199020 con 0x7f913c1017a0 2024-03-29T04:50:17.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.245+0000 7f91442b7700 1 -- 172.21.15.142:0/4037878567 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c104f70 con 0x7f913c1017a0 2024-03-29T04:50:17.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.245+0000 7f91337fe700 1 -- 172.21.15.142:0/4037878567 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7f913800f5d0 con 0x7f913c1017a0 2024-03-29T04:50:17.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.245+0000 7f91337fe700 1 --2- 172.21.15.142:0/4037878567 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f912806dea0 0x7f9128070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:17.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.246+0000 7f91337fe700 1 -- 172.21.15.142:0/4037878567 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f913808f1e0 con 0x7f913c1017a0 2024-03-29T04:50:17.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.246+0000 7f9142053700 1 --2- 172.21.15.142:0/4037878567 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f912806dea0 0x7f9128070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:17.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.246+0000 7f9142053700 1 --2- 172.21.15.142:0/4037878567 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f912806dea0 0x7f9128070360 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f912c006fd0 tx=0x7f912c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:17.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.250+0000 7f91337fe700 1 -- 172.21.15.142:0/4037878567 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f913805c160 con 0x7f913c1017a0 2024-03-29T04:50:17.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.463+0000 7f91442b7700 1 -- 172.21.15.142:0/4037878567 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f913c02cd30 con 0x7f912806dea0 2024-03-29T04:50:17.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.464+0000 7f91337fe700 1 -- 172.21.15.142:0/4037878567 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f913c02cd30 con 0x7f912806dea0 2024-03-29T04:50:17.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.466+0000 7f91442b7700 1 -- 172.21.15.142:0/4037878567 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f912806dea0 msgr2=0x7f9128070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:17.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.466+0000 7f91442b7700 1 --2- 172.21.15.142:0/4037878567 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f912806dea0 0x7f9128070360 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f912c006fd0 tx=0x7f912c008040 comp rx=0 tx=0).stop 2024-03-29T04:50:17.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.466+0000 7f91442b7700 1 -- 172.21.15.142:0/4037878567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f913c1017a0 msgr2=0x7f913c19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:17.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.466+0000 7f91442b7700 1 --2- 172.21.15.142:0/4037878567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f913c1017a0 0x7f913c19ed70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91380092e0 tx=0x7f913800be20 comp rx=0 tx=0).stop 2024-03-29T04:50:17.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.466+0000 7f91442b7700 1 -- 172.21.15.142:0/4037878567 shutdown_connections 2024-03-29T04:50:17.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.466+0000 7f91442b7700 1 --2- 172.21.15.142:0/4037878567 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f912806dea0 0x7f9128070360 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.466+0000 7f91442b7700 1 --2- 172.21.15.142:0/4037878567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f913c100df0 0x7f913c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.466+0000 7f91442b7700 1 --2- 172.21.15.142:0/4037878567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f913c1017a0 0x7f913c19ed70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.466+0000 7f91442b7700 1 -- 172.21.15.142:0/4037878567 >> 172.21.15.142:0/4037878567 conn(0x7f913c0fc6a0 msgr2=0x7f913c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:17.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.466+0000 7f91442b7700 1 -- 172.21.15.142:0/4037878567 shutdown_connections 2024-03-29T04:50:17.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.466+0000 7f91442b7700 1 -- 172.21.15.142:0/4037878567 wait complete. 2024-03-29T04:50:17.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.584+0000 7fed41d5a700 1 -- 172.21.15.142:0/1760566296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c101790 msgr2=0x7fed3c1057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:17.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.584+0000 7fed41d5a700 1 --2- 172.21.15.142:0/1760566296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c101790 0x7fed3c1057e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fed30009910 tx=0x7fed30009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:17.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.584+0000 7fed41d5a700 1 -- 172.21.15.142:0/1760566296 shutdown_connections 2024-03-29T04:50:17.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.584+0000 7fed41d5a700 1 --2- 172.21.15.142:0/1760566296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c101790 0x7fed3c1057e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.584+0000 7fed41d5a700 1 --2- 172.21.15.142:0/1760566296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c100de0 0x7fed3c1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.584+0000 7fed41d5a700 1 -- 172.21.15.142:0/1760566296 >> 172.21.15.142:0/1760566296 conn(0x7fed3c0fc6b0 msgr2=0x7fed3c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:17.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.585+0000 7fed41d5a700 1 -- 172.21.15.142:0/1760566296 shutdown_connections 2024-03-29T04:50:17.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.585+0000 7fed41d5a700 1 -- 172.21.15.142:0/1760566296 wait complete. 2024-03-29T04:50:17.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.585+0000 7fed41d5a700 1 Processor -- start 2024-03-29T04:50:17.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.585+0000 7fed41d5a700 1 -- start start 2024-03-29T04:50:17.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed41d5a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c100de0 0x7fed3c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed41d5a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 0x7fed3c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed41d5a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c19d0f0 con 0x7fed3c100de0 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed41d5a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c1965a0 con 0x7fed3c101790 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed3b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c100de0 0x7fed3c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed3b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c100de0 0x7fed3c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52184/0 (socket says 172.21.15.142:52184) 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed3b7fe700 1 -- 172.21.15.142:0/97837173 learned_addr learned my addr 172.21.15.142:0/97837173 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed3b7fe700 1 -- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c100de0 msgr2=0x7fed3c19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed3b7fe700 1 -- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c100de0 msgr2=0x7fed3c19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed3b7fe700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c100de0 0x7fed3c19c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed3b7fe700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c100de0 0x7fed3c19c520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed3affd700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 0x7fed3c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed3affd700 1 -- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 msgr2=0x7fed3c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-29T04:50:17.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed3affd700 1 -- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 msgr2=0x7fed3c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-29T04:50:17.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed3affd700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 0x7fed3c19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:50:17.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.586+0000 7fed3affd700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 0x7fed3c19ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:50:17.790 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.787+0000 7fed3b7fe700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c100de0 0x7fed3c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:17.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.787+0000 7fed3affd700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 0x7fed3c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:17.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:17 smithi142 ceph-mon[112475]: pgmap v10: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-29T04:50:17.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:17 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:50:17.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:17 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:50:17.791 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:17 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:50:17.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.787+0000 7fed3affd700 1 -- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 msgr2=0x7fed3c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-29T04:50:17.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.787+0000 7fed3affd700 1 -- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 msgr2=0x7fed3c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-29T04:50:17.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.787+0000 7fed3affd700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 0x7fed3c19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:50:17.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.787+0000 7fed3affd700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 0x7fed3c19ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:50:17.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.787+0000 7fed3b7fe700 1 -- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 msgr2=0x7fed3c19ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:50:17.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.787+0000 7fed3b7fe700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 0x7fed3c19ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:17.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.787+0000 7fed3b7fe700 1 -- 172.21.15.142:0/97837173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed300095f0 con 0x7fed3c100de0 2024-03-29T04:50:17.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.787+0000 7fed3b7fe700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c100de0 0x7fed3c19c520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fed2c003fd0 tx=0x7fed2c00f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:17.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.787+0000 7fed38ff9700 1 -- 172.21.15.142:0/97837173 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed2c0118e0 con 0x7fed3c100de0 2024-03-29T04:50:17.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.788+0000 7fed38ff9700 1 -- 172.21.15.142:0/97837173 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed2c010430 con 0x7fed3c100de0 2024-03-29T04:50:17.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.788+0000 7fed41d5a700 1 -- 172.21.15.142:0/97837173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c196880 con 0x7fed3c100de0 2024-03-29T04:50:17.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.788+0000 7fed38ff9700 1 -- 172.21.15.142:0/97837173 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed2c019610 con 0x7fed3c100de0 2024-03-29T04:50:17.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.788+0000 7fed41d5a700 1 -- 172.21.15.142:0/97837173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c196da0 con 0x7fed3c100de0 2024-03-29T04:50:17.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.789+0000 7fed41d5a700 1 -- 172.21.15.142:0/97837173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3c1091e0 con 0x7fed3c100de0 2024-03-29T04:50:17.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.789+0000 7fed38ff9700 1 -- 172.21.15.142:0/97837173 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7fed2c011a40 con 0x7fed3c100de0 2024-03-29T04:50:17.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.790+0000 7fed38ff9700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fed2406dea0 0x7fed24070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:17.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.790+0000 7fed38ff9700 1 -- 172.21.15.142:0/97837173 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fed2c08e250 con 0x7fed3c100de0 2024-03-29T04:50:17.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.790+0000 7fed3affd700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fed2406dea0 0x7fed24070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:17.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.790+0000 7fed3affd700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fed2406dea0 0x7fed24070360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fed3000b5c0 tx=0x7fed3001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:17.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.795+0000 7fed38ff9700 1 -- 172.21.15.142:0/97837173 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fed2c05b130 con 0x7fed3c100de0 2024-03-29T04:50:18.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:17.998+0000 7fed41d5a700 1 -- 172.21.15.142:0/97837173 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fed3c197010 con 0x7fed2406dea0 2024-03-29T04:50:18.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.009+0000 7fed38ff9700 1 -- 172.21.15.142:0/97837173 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fed3c197010 con 0x7fed2406dea0 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (2m) 12s ago 13m 16.9M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (13m) 12s ago 13m 8186k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (12m) 15s ago 12m 8094k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (13m) 12s ago 13m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (12m) 15s ago 12m 6665k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (106s) 12s ago 13m 75.9M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (7m) 12s ago 7m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (8m) 12s ago 8m 63.2M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (7m) 15s ago 7m 65.3M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (8m) 15s ago 8m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (3m) 12s ago 15m 471M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:50:18.013 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (3m) 15s ago 12m 446M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:50:18.014 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (27s) 12s ago 15m 29.0M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:50:18.014 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 starting - - - 2048M 2024-03-29T04:50:18.014 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (2m) 12s ago 13m 15.0M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:50:18.014 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (2m) 15s ago 12m 16.2M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:50:18.014 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (11m) 12s ago 11m 219M 1216M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:50:18.014 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (11m) 12s ago 11m 241M 1216M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:50:18.014 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (10m) 12s ago 10m 198M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:50:18.014 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (10m) 15s ago 10m 276M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:50:18.014 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (10m) 15s ago 10m 236M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:50:18.014 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (9m) 15s ago 9m 225M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:50:18.014 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (2m) 12s ago 13m 53.5M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:50:18.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.011+0000 7fed41d5a700 1 -- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fed2406dea0 msgr2=0x7fed24070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:18.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.011+0000 7fed41d5a700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fed2406dea0 0x7fed24070360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fed3000b5c0 tx=0x7fed3001a040 comp rx=0 tx=0).stop 2024-03-29T04:50:18.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.011+0000 7fed41d5a700 1 -- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c100de0 msgr2=0x7fed3c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:18.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.011+0000 7fed41d5a700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c100de0 0x7fed3c19c520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fed2c003fd0 tx=0x7fed2c00f9e0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.011+0000 7fed41d5a700 1 -- 172.21.15.142:0/97837173 shutdown_connections 2024-03-29T04:50:18.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.011+0000 7fed41d5a700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fed2406dea0 0x7fed24070360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.011+0000 7fed41d5a700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed3c100de0 0x7fed3c19c520 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.011+0000 7fed41d5a700 1 --2- 172.21.15.142:0/97837173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed3c101790 0x7fed3c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.011+0000 7fed41d5a700 1 -- 172.21.15.142:0/97837173 >> 172.21.15.142:0/97837173 conn(0x7fed3c0fc6b0 msgr2=0x7fed3c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:18.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.011+0000 7fed41d5a700 1 -- 172.21.15.142:0/97837173 shutdown_connections 2024-03-29T04:50:18.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.011+0000 7fed41d5a700 1 -- 172.21.15.142:0/97837173 wait complete. 2024-03-29T04:50:18.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:17 smithi152 ceph-mon[88186]: pgmap v10: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-29T04:50:18.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:17 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:50:18.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:17 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:50:18.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:17 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:50:18.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.113+0000 7fca723be700 1 -- 172.21.15.142:0/2461066462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6c101c70 msgr2=0x7fca6c102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:18.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.113+0000 7fca723be700 1 --2- 172.21.15.142:0/2461066462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6c101c70 0x7fca6c102050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009910 tx=0x7fca5c009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:18.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.113+0000 7fca723be700 1 -- 172.21.15.142:0/2461066462 shutdown_connections 2024-03-29T04:50:18.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.113+0000 7fca723be700 1 --2- 172.21.15.142:0/2461066462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca6c102620 0x7fca6c10ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.113+0000 7fca723be700 1 --2- 172.21.15.142:0/2461066462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6c101c70 0x7fca6c102050 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.113+0000 7fca723be700 1 -- 172.21.15.142:0/2461066462 >> 172.21.15.142:0/2461066462 conn(0x7fca6c0fb530 msgr2=0x7fca6c0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:18.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.113+0000 7fca723be700 1 -- 172.21.15.142:0/2461066462 shutdown_connections 2024-03-29T04:50:18.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.113+0000 7fca723be700 1 -- 172.21.15.142:0/2461066462 wait complete. 2024-03-29T04:50:18.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.114+0000 7fca723be700 1 Processor -- start 2024-03-29T04:50:18.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.114+0000 7fca723be700 1 -- start start 2024-03-29T04:50:18.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.114+0000 7fca723be700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca6c101c70 0x7fca6c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:18.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca723be700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6c102620 0x7fca6c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:18.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca723be700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c19d0c0 con 0x7fca6c102620 2024-03-29T04:50:18.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca723be700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c196570 con 0x7fca6c101c70 2024-03-29T04:50:18.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca6b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6c102620 0x7fca6c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca6b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6c102620 0x7fca6c19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52208/0 (socket says 172.21.15.142:52208) 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca6b7fe700 1 -- 172.21.15.142:0/3546015102 learned_addr learned my addr 172.21.15.142:0/3546015102 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca6bfff700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca6c101c70 0x7fca6c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca6bfff700 1 -- 172.21.15.142:0/3546015102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca6c101c70 msgr2=0x7fca6c19c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca6bfff700 1 -- 172.21.15.142:0/3546015102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca6c101c70 msgr2=0x7fca6c19c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca6bfff700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca6c101c70 0x7fca6c19c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca6bfff700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca6c101c70 0x7fca6c19c4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca6b7fe700 1 -- 172.21.15.142:0/3546015102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca6c101c70 msgr2=0x7fca6c19c4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca6b7fe700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca6c101c70 0x7fca6c19c4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca6b7fe700 1 -- 172.21.15.142:0/3546015102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c0095f0 con 0x7fca6c102620 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca6b7fe700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6c102620 0x7fca6c19ca30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fca6000d900 tx=0x7fca6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca697fa700 1 -- 172.21.15.142:0/3546015102 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca60009900 con 0x7fca6c102620 2024-03-29T04:50:18.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca697fa700 1 -- 172.21.15.142:0/3546015102 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca60010460 con 0x7fca6c102620 2024-03-29T04:50:18.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.115+0000 7fca697fa700 1 -- 172.21.15.142:0/3546015102 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca6000f5f0 con 0x7fca6c102620 2024-03-29T04:50:18.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.116+0000 7fca723be700 1 -- 172.21.15.142:0/3546015102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6c196850 con 0x7fca6c102620 2024-03-29T04:50:18.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.116+0000 7fca723be700 1 -- 172.21.15.142:0/3546015102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6c196da0 con 0x7fca6c102620 2024-03-29T04:50:18.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.117+0000 7fca723be700 1 -- 172.21.15.142:0/3546015102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6c108210 con 0x7fca6c102620 2024-03-29T04:50:18.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.117+0000 7fca697fa700 1 -- 172.21.15.142:0/3546015102 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7fca600105d0 con 0x7fca6c102620 2024-03-29T04:50:18.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.118+0000 7fca697fa700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fca5406dea0 0x7fca54070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:18.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.118+0000 7fca697fa700 1 -- 172.21.15.142:0/3546015102 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fca6008db00 con 0x7fca6c102620 2024-03-29T04:50:18.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.118+0000 7fca6bfff700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fca5406dea0 0x7fca54070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:18.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.118+0000 7fca6bfff700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fca5406dea0 0x7fca54070360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00b5c0 tx=0x7fca5c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:18.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.123+0000 7fca697fa700 1 -- 172.21.15.142:0/3546015102 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fca6005aa80 con 0x7fca6c102620 2024-03-29T04:50:18.400 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.397+0000 7fca723be700 1 -- 172.21.15.142:0/3546015102 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fca6c04ea90 con 0x7fca6c102620 2024-03-29T04:50:18.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.397+0000 7fca697fa700 1 -- 172.21.15.142:0/3546015102 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7fca6005a610 con 0x7fca6c102620 2024-03-29T04:50:18.401 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:50:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:50:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:50:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:50:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:50:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:50:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:50:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:50:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:50:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:50:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:50:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:50:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:50:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:50:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-03-29T04:50:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T04:50:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:50:18.402 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:50:18.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.399+0000 7fca723be700 1 -- 172.21.15.142:0/3546015102 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fca5406dea0 msgr2=0x7fca54070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:18.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.399+0000 7fca723be700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fca5406dea0 0x7fca54070360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00b5c0 tx=0x7fca5c01a040 comp rx=0 tx=0).stop 2024-03-29T04:50:18.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.399+0000 7fca723be700 1 -- 172.21.15.142:0/3546015102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6c102620 msgr2=0x7fca6c19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:18.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.399+0000 7fca723be700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6c102620 0x7fca6c19ca30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fca6000d900 tx=0x7fca6000dcc0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.400+0000 7fca723be700 1 -- 172.21.15.142:0/3546015102 shutdown_connections 2024-03-29T04:50:18.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.400+0000 7fca723be700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fca5406dea0 0x7fca54070360 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.400+0000 7fca723be700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca6c101c70 0x7fca6c19c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.400+0000 7fca723be700 1 --2- 172.21.15.142:0/3546015102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca6c102620 0x7fca6c19ca30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.400+0000 7fca723be700 1 -- 172.21.15.142:0/3546015102 >> 172.21.15.142:0/3546015102 conn(0x7fca6c0fb530 msgr2=0x7fca6c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:18.404 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.400+0000 7fca723be700 1 -- 172.21.15.142:0/3546015102 shutdown_connections 2024-03-29T04:50:18.404 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.400+0000 7fca723be700 1 -- 172.21.15.142:0/3546015102 wait complete. 2024-03-29T04:50:18.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.503+0000 7f03f9d56700 1 -- 172.21.15.142:0/3560641293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03f4102610 msgr2=0x7f03f410ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:18.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.503+0000 7f03f9d56700 1 --2- 172.21.15.142:0/3560641293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03f4102610 0x7f03f410ab00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009910 tx=0x7f03e8009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:18.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.503+0000 7f03f9d56700 1 -- 172.21.15.142:0/3560641293 shutdown_connections 2024-03-29T04:50:18.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.503+0000 7f03f9d56700 1 --2- 172.21.15.142:0/3560641293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03f4102610 0x7f03f410ab00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.503+0000 7f03f9d56700 1 --2- 172.21.15.142:0/3560641293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f4101c60 0x7f03f4102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.503+0000 7f03f9d56700 1 -- 172.21.15.142:0/3560641293 >> 172.21.15.142:0/3560641293 conn(0x7f03f40fb520 msgr2=0x7f03f40fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:18.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.503+0000 7f03f9d56700 1 -- 172.21.15.142:0/3560641293 shutdown_connections 2024-03-29T04:50:18.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.503+0000 7f03f9d56700 1 -- 172.21.15.142:0/3560641293 wait complete. 2024-03-29T04:50:18.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.503+0000 7f03f9d56700 1 Processor -- start 2024-03-29T04:50:18.508 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.503+0000 7f03f9d56700 1 -- start start 2024-03-29T04:50:18.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f9d56700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f4101c60 0x7f03f419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:18.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f9d56700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03f4102610 0x7f03f419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:18.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f9d56700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f419d0e0 con 0x7f03f4102610 2024-03-29T04:50:18.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f9d56700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f4196590 con 0x7f03f4101c60 2024-03-29T04:50:18.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03f4102610 0x7f03f419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:18.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03f4102610 0x7f03f419ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:48136/0 (socket says 172.21.15.142:48136) 2024-03-29T04:50:18.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f2ffd700 1 -- 172.21.15.142:0/2328140456 learned_addr learned my addr 172.21.15.142:0/2328140456 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:18.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f37fe700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f4101c60 0x7f03f419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:18.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f37fe700 1 -- 172.21.15.142:0/2328140456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f4101c60 msgr2=0x7f03f419c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-29T04:50:18.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f37fe700 1 -- 172.21.15.142:0/2328140456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f4101c60 msgr2=0x7f03f419c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-29T04:50:18.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f37fe700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f4101c60 0x7f03f419c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:50:18.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f2ffd700 1 -- 172.21.15.142:0/2328140456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f4101c60 msgr2=0x7f03f419c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:18.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f37fe700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f4101c60 0x7f03f419c510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:50:18.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f2ffd700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f4101c60 0x7f03f419c510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f2ffd700 1 -- 172.21.15.142:0/2328140456 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e80095f0 con 0x7f03f4102610 2024-03-29T04:50:18.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f2ffd700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03f4102610 0x7f03f419ca50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f03e80092e0 tx=0x7f03e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:18.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f0ff9700 1 -- 172.21.15.142:0/2328140456 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e801d070 con 0x7f03f4102610 2024-03-29T04:50:18.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f0ff9700 1 -- 172.21.15.142:0/2328140456 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03e8005dc0 con 0x7f03f4102610 2024-03-29T04:50:18.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f9d56700 1 -- 172.21.15.142:0/2328140456 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f4196810 con 0x7f03f4102610 2024-03-29T04:50:18.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f0ff9700 1 -- 172.21.15.142:0/2328140456 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e8017400 con 0x7f03f4102610 2024-03-29T04:50:18.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.504+0000 7f03f9d56700 1 -- 172.21.15.142:0/2328140456 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f4196d00 con 0x7f03f4102610 2024-03-29T04:50:18.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.505+0000 7f03f9d56700 1 -- 172.21.15.142:0/2328140456 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f4108200 con 0x7f03f4102610 2024-03-29T04:50:18.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.506+0000 7f03f0ff9700 1 -- 172.21.15.142:0/2328140456 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7f03e800f460 con 0x7f03f4102610 2024-03-29T04:50:18.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.506+0000 7f03f0ff9700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f03dc06dea0 0x7f03dc070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:18.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.506+0000 7f03f0ff9700 1 -- 172.21.15.142:0/2328140456 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f03e808f260 con 0x7f03f4102610 2024-03-29T04:50:18.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.506+0000 7f03f37fe700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f03dc06dea0 0x7f03dc070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:18.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.507+0000 7f03f37fe700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f03dc06dea0 0x7f03dc070360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f03e4005920 tx=0x7f03e400b2c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:18.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.511+0000 7f03f0ff9700 1 -- 172.21.15.142:0/2328140456 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f03e805c160 con 0x7f03f4102610 2024-03-29T04:50:18.609 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:18.609 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:18.609 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: pgmap v12: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:50:18.609 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:50:18.609 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: osdmap e51: 6 total, 6 up, 6 in 2024-03-29T04:50:18.609 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: mgrmap e39: smithi142.swesrg(active, since 18s), standbys: smithi152.usoolh 2024-03-29T04:50:18.609 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: overall HEALTH_OK 2024-03-29T04:50:18.609 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:50:18.609 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:50:18.609 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: mon.smithi142 calling monitor election 2024-03-29T04:50:18.609 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: mon.smithi152 calling monitor election 2024-03-29T04:50:18.609 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: mon.smithi142 is new leader, mons smithi142,smithi152 in quorum (ranks 0,1) 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: monmap epoch 3 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: fsid 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: last_changed 2024-03-29T04:50:17.571402+0000 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: created 2024-03-29T04:35:04.327053+0000 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: min_mon_release 19 (squid) 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: election_strategy: 1 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: 0: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.smithi142 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: 1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.smithi152 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: osdmap e51: 6 total, 6 up, 6 in 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: mgrmap e39: smithi142.swesrg(active, since 18s), standbys: smithi152.usoolh 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: overall HEALTH_OK 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:18.610 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:18.611 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:50:18.611 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:18 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3546015102' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:50:18.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.754+0000 7f03f9d56700 1 -- 172.21.15.142:0/2328140456 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f03f4197aa0 con 0x7f03f4102610 2024-03-29T04:50:18.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.755+0000 7f03f0ff9700 1 -- 172.21.15.142:0/2328140456 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1906 (secure 0 0 0) 0x7f03e805bcf0 con 0x7f03f4102610 2024-03-29T04:50:18.759 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:18.759 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:18.759 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: pgmap v12: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:50:18.759 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:50:18.759 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: osdmap e51: 6 total, 6 up, 6 in 2024-03-29T04:50:18.759 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: mgrmap e39: smithi142.swesrg(active, since 18s), standbys: smithi152.usoolh 2024-03-29T04:50:18.759 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: overall HEALTH_OK 2024-03-29T04:50:18.759 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi142"}]: dispatch 2024-03-29T04:50:18.759 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mon metadata", "id": "smithi152"}]: dispatch 2024-03-29T04:50:18.759 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: mon.smithi142 calling monitor election 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: mon.smithi152 calling monitor election 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: mon.smithi142 is new leader, mons smithi142,smithi152 in quorum (ranks 0,1) 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: monmap epoch 3 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: fsid 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: last_changed 2024-03-29T04:50:17.571402+0000 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: created 2024-03-29T04:35:04.327053+0000 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: min_mon_release 19 (squid) 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: election_strategy: 1 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: 0: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.smithi142 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: 1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.smithi152 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2 up:standby-replay 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: osdmap e51: 6 total, 6 up, 6 in 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: mgrmap e39: smithi142.swesrg(active, since 18s), standbys: smithi152.usoolh 2024-03-29T04:50:18.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: overall HEALTH_OK 2024-03-29T04:50:18.761 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:18.761 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:18.761 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:50:18.761 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:18 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3546015102' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:50:18.761 INFO:teuthology.orchestra.run.smithi142.stdout:e21 2024-03-29T04:50:18.761 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:50:18.761 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:50:18.761 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:50:18.761 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:50:18.761 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 21 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:49:01.288637+0000 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:50:18.762 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 47 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:up {0=14494,1=24275} 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:50:18.763 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:50:18.764 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{0:14494} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:50:18.764 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{0:14506} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:50:18.764 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{1:24275} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:50:18.764 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{1:24287} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:50:18.764 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:50:18.764 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:50:18.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.757+0000 7f03f9d56700 1 -- 172.21.15.142:0/2328140456 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f03dc06dea0 msgr2=0x7f03dc070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:18.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.757+0000 7f03f9d56700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f03dc06dea0 0x7f03dc070360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f03e4005920 tx=0x7f03e400b2c0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.757+0000 7f03f9d56700 1 -- 172.21.15.142:0/2328140456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03f4102610 msgr2=0x7f03f419ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:18.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.757+0000 7f03f9d56700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03f4102610 0x7f03f419ca50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f03e80092e0 tx=0x7f03e800be20 comp rx=0 tx=0).stop 2024-03-29T04:50:18.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.757+0000 7f03f9d56700 1 -- 172.21.15.142:0/2328140456 shutdown_connections 2024-03-29T04:50:18.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.757+0000 7f03f9d56700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f03dc06dea0 0x7f03dc070360 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.757+0000 7f03f9d56700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03f4101c60 0x7f03f419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.757+0000 7f03f9d56700 1 --2- 172.21.15.142:0/2328140456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03f4102610 0x7f03f419ca50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.757+0000 7f03f9d56700 1 -- 172.21.15.142:0/2328140456 >> 172.21.15.142:0/2328140456 conn(0x7f03f40fb520 msgr2=0x7f03f4105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:18.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.758+0000 7f03f9d56700 1 -- 172.21.15.142:0/2328140456 shutdown_connections 2024-03-29T04:50:18.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.758+0000 7f03f9d56700 1 -- 172.21.15.142:0/2328140456 wait complete. 2024-03-29T04:50:18.765 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 21 2024-03-29T04:50:18.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.861+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3628584062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3dc0ffa40 msgr2=0x7fc3dc0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:18.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.861+0000 7fc3e10b1700 1 --2- 172.21.15.142:0/3628584062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3dc0ffa40 0x7fc3dc0ffec0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009910 tx=0x7fc3d0009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:18.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.861+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3628584062 shutdown_connections 2024-03-29T04:50:18.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.861+0000 7fc3e10b1700 1 --2- 172.21.15.142:0/3628584062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3dc0ffa40 0x7fc3dc0ffec0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.861+0000 7fc3e10b1700 1 --2- 172.21.15.142:0/3628584062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3dc105a40 0x7fc3dc105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.861+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3628584062 >> 172.21.15.142:0/3628584062 conn(0x7fc3dc0fb570 msgr2=0x7fc3dc0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:18.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.861+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3628584062 shutdown_connections 2024-03-29T04:50:18.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.861+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3628584062 wait complete. 2024-03-29T04:50:18.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.862+0000 7fc3e10b1700 1 Processor -- start 2024-03-29T04:50:18.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.862+0000 7fc3e10b1700 1 -- start start 2024-03-29T04:50:18.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3e10b1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3dc0ffa40 0x7fc3dc101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:18.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3e10b1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3dc105a40 0x7fc3dc101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:18.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3e10b1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc102090 con 0x7fc3dc0ffa40 2024-03-29T04:50:18.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3e10b1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc1021d0 con 0x7fc3dc105a40 2024-03-29T04:50:18.871 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3dad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3dc0ffa40 0x7fc3dc101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:18.871 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3dad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3dc0ffa40 0x7fc3dc101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:48150/0 (socket says 172.21.15.142:48150) 2024-03-29T04:50:18.871 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3dad9d700 1 -- 172.21.15.142:0/3748734414 learned_addr learned my addr 172.21.15.142:0/3748734414 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:18.871 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3da59c700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3dc105a40 0x7fc3dc101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:18.871 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3da59c700 1 -- 172.21.15.142:0/3748734414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3dc105a40 msgr2=0x7fc3dc101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-29T04:50:18.871 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3da59c700 1 -- 172.21.15.142:0/3748734414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3dc105a40 msgr2=0x7fc3dc101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-29T04:50:18.871 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3da59c700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3dc105a40 0x7fc3dc101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:50:18.871 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3da59c700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3dc105a40 0x7fc3dc101b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:50:18.872 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3dad9d700 1 -- 172.21.15.142:0/3748734414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3dc105a40 msgr2=0x7fc3dc101b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:50:18.872 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3dad9d700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3dc105a40 0x7fc3dc101b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:18.872 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3dad9d700 1 -- 172.21.15.142:0/3748734414 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d00095f0 con 0x7fc3dc0ffa40 2024-03-29T04:50:18.872 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3dad9d700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3dc0ffa40 0x7fc3dc101610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00d8d0 tx=0x7fc3cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:18.872 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3c3fff700 1 -- 172.21.15.142:0/3748734414 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3cc004a00 con 0x7fc3dc0ffa40 2024-03-29T04:50:18.872 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3748734414 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3dc1a28f0 con 0x7fc3dc0ffa40 2024-03-29T04:50:18.872 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3c3fff700 1 -- 172.21.15.142:0/3748734414 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3cc0054e0 con 0x7fc3dc0ffa40 2024-03-29T04:50:18.873 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3c3fff700 1 -- 172.21.15.142:0/3748734414 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3cc009d60 con 0x7fc3dc0ffa40 2024-03-29T04:50:18.873 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.863+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3748734414 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3dc1a2d00 con 0x7fc3dc0ffa40 2024-03-29T04:50:18.873 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.864+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3748734414 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3dc04ea90 con 0x7fc3dc0ffa40 2024-03-29T04:50:18.873 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.865+0000 7fc3c3fff700 1 -- 172.21.15.142:0/3748734414 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7fc3cc005020 con 0x7fc3dc0ffa40 2024-03-29T04:50:18.873 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.865+0000 7fc3c3fff700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc3c406dea0 0x7fc3c4070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:18.873 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.865+0000 7fc3c3fff700 1 -- 172.21.15.142:0/3748734414 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc3cc08d990 con 0x7fc3dc0ffa40 2024-03-29T04:50:18.873 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.865+0000 7fc3da59c700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc3c406dea0 0x7fc3c4070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:18.874 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.866+0000 7fc3da59c700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc3c406dea0 0x7fc3c4070360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0005b40 tx=0x7fc3d001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:18.874 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:18.870+0000 7fc3c3fff700 1 -- 172.21.15.142:0/3748734414 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc3cc056d60 con 0x7fc3dc0ffa40 2024-03-29T04:50:19.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.083+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3748734414 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3dc0695a0 con 0x7fc3c406dea0 2024-03-29T04:50:19.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.085+0000 7fc3c3fff700 1 -- 172.21.15.142:0/3748734414 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc3dc0695a0 con 0x7fc3c406dea0 2024-03-29T04:50:19.089 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:50:19.089 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:50:19.089 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:50:19.089 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:50:19.090 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:50:19.090 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr" 2024-03-29T04:50:19.090 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:50:19.090 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "3/23 daemons upgraded", 2024-03-29T04:50:19.090 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mon daemons", 2024-03-29T04:50:19.090 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:50:19.091 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:50:19.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.087+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3748734414 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc3c406dea0 msgr2=0x7fc3c4070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:19.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.087+0000 7fc3e10b1700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc3c406dea0 0x7fc3c4070360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0005b40 tx=0x7fc3d001a040 comp rx=0 tx=0).stop 2024-03-29T04:50:19.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.087+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3748734414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3dc0ffa40 msgr2=0x7fc3dc101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:19.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.087+0000 7fc3e10b1700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3dc0ffa40 0x7fc3dc101610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00d8d0 tx=0x7fc3cc00dc90 comp rx=0 tx=0).stop 2024-03-29T04:50:19.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.087+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3748734414 shutdown_connections 2024-03-29T04:50:19.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.087+0000 7fc3e10b1700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc3c406dea0 0x7fc3c4070360 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:19.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.087+0000 7fc3e10b1700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3dc0ffa40 0x7fc3dc101610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:19.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.087+0000 7fc3e10b1700 1 --2- 172.21.15.142:0/3748734414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3dc105a40 0x7fc3dc101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:19.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.087+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3748734414 >> 172.21.15.142:0/3748734414 conn(0x7fc3dc0fb570 msgr2=0x7fc3dc068e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:19.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.087+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3748734414 shutdown_connections 2024-03-29T04:50:19.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.087+0000 7fc3e10b1700 1 -- 172.21.15.142:0/3748734414 wait complete. 2024-03-29T04:50:19.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.191+0000 7f9849d4d700 1 -- 172.21.15.142:0/1013041378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9844069020 msgr2=0x7f9844105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:19.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.191+0000 7f9849d4d700 1 --2- 172.21.15.142:0/1013041378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9844069020 0x7f9844105790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f982c009910 tx=0x7f982c009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:19.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.191+0000 7f9849d4d700 1 -- 172.21.15.142:0/1013041378 shutdown_connections 2024-03-29T04:50:19.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.191+0000 7f9849d4d700 1 --2- 172.21.15.142:0/1013041378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9844069020 0x7f9844105790 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:19.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.191+0000 7f9849d4d700 1 --2- 172.21.15.142:0/1013041378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9844068670 0x7f9844068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:19.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.191+0000 7f9849d4d700 1 -- 172.21.15.142:0/1013041378 >> 172.21.15.142:0/1013041378 conn(0x7f98440755a0 msgr2=0x7f98440759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:19.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.191+0000 7f9849d4d700 1 -- 172.21.15.142:0/1013041378 shutdown_connections 2024-03-29T04:50:19.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.191+0000 7f9849d4d700 1 -- 172.21.15.142:0/1013041378 wait complete. 2024-03-29T04:50:19.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f9849d4d700 1 Processor -- start 2024-03-29T04:50:19.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f9849d4d700 1 -- start start 2024-03-29T04:50:19.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f9849d4d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9844068670 0x7f9844196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:19.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f9849d4d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9844069020 0x7f9844196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:19.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f9849d4d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98441972d0 con 0x7f9844069020 2024-03-29T04:50:19.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f9849d4d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984419a000 con 0x7f9844068670 2024-03-29T04:50:19.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f983affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9844069020 0x7f9844196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:19.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f983affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9844069020 0x7f9844196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:48168/0 (socket says 172.21.15.142:48168) 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f983affd700 1 -- 172.21.15.142:0/3192257234 learned_addr learned my addr 172.21.15.142:0/3192257234 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f98437fe700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9844068670 0x7f9844196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f98437fe700 1 -- 172.21.15.142:0/3192257234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9844068670 msgr2=0x7f9844196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f98437fe700 1 -- 172.21.15.142:0/3192257234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9844068670 msgr2=0x7f9844196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f98437fe700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9844068670 0x7f9844196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f98437fe700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9844068670 0x7f9844196700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f983affd700 1 -- 172.21.15.142:0/3192257234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9844068670 msgr2=0x7f9844196700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f983affd700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9844068670 0x7f9844196700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.192+0000 7f983affd700 1 -- 172.21.15.142:0/3192257234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c0095f0 con 0x7f9844069020 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.193+0000 7f983affd700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9844069020 0x7f9844196c40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f982c0092e0 tx=0x7f982c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.193+0000 7f98417fa700 1 -- 172.21.15.142:0/3192257234 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f982c01d070 con 0x7f9844069020 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.193+0000 7f98417fa700 1 -- 172.21.15.142:0/3192257234 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f982c005dc0 con 0x7f9844069020 2024-03-29T04:50:19.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.193+0000 7f9849d4d700 1 -- 172.21.15.142:0/3192257234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984419a280 con 0x7f9844069020 2024-03-29T04:50:19.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.193+0000 7f98417fa700 1 -- 172.21.15.142:0/3192257234 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f982c017400 con 0x7f9844069020 2024-03-29T04:50:19.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.193+0000 7f9849d4d700 1 -- 172.21.15.142:0/3192257234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984419a770 con 0x7f9844069020 2024-03-29T04:50:19.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.194+0000 7f9849d4d700 1 -- 172.21.15.142:0/3192257234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98441090a0 con 0x7f9844069020 2024-03-29T04:50:19.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.194+0000 7f98417fa700 1 -- 172.21.15.142:0/3192257234 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7f982c00f460 con 0x7f9844069020 2024-03-29T04:50:19.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.195+0000 7f98417fa700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f983006dea0 0x7f9830070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:19.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.195+0000 7f98417fa700 1 -- 172.21.15.142:0/3192257234 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f982c08f260 con 0x7f9844069020 2024-03-29T04:50:19.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.195+0000 7f98437fe700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f983006dea0 0x7f9830070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:19.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.195+0000 7f98437fe700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f983006dea0 0x7f9830070360 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9834005920 tx=0x7f983400b2c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:19.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.199+0000 7f98417fa700 1 -- 172.21.15.142:0/3192257234 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f982c058630 con 0x7f9844069020 2024-03-29T04:50:19.479 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.475+0000 7f9849d4d700 1 -- 172.21.15.142:0/3192257234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f984404ea90 con 0x7f9844069020 2024-03-29T04:50:19.479 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.476+0000 7f98417fa700 1 -- 172.21.15.142:0/3192257234 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f982c05bcf0 con 0x7f9844069020 2024-03-29T04:50:19.480 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_OK 2024-03-29T04:50:19.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.478+0000 7f9849d4d700 1 -- 172.21.15.142:0/3192257234 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f983006dea0 msgr2=0x7f9830070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:19.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.478+0000 7f9849d4d700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f983006dea0 0x7f9830070360 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9834005920 tx=0x7f983400b2c0 comp rx=0 tx=0).stop 2024-03-29T04:50:19.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.478+0000 7f9849d4d700 1 -- 172.21.15.142:0/3192257234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9844069020 msgr2=0x7f9844196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:19.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.478+0000 7f9849d4d700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9844069020 0x7f9844196c40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f982c0092e0 tx=0x7f982c00be20 comp rx=0 tx=0).stop 2024-03-29T04:50:19.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.478+0000 7f9849d4d700 1 -- 172.21.15.142:0/3192257234 shutdown_connections 2024-03-29T04:50:19.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.478+0000 7f9849d4d700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f983006dea0 0x7f9830070360 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:19.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.478+0000 7f9849d4d700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9844068670 0x7f9844196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:19.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.478+0000 7f9849d4d700 1 --2- 172.21.15.142:0/3192257234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9844069020 0x7f9844196c40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:19.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.478+0000 7f9849d4d700 1 -- 172.21.15.142:0/3192257234 >> 172.21.15.142:0/3192257234 conn(0x7f98440755a0 msgr2=0x7f9844105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:19.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.478+0000 7f9849d4d700 1 -- 172.21.15.142:0/3192257234 shutdown_connections 2024-03-29T04:50:19.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:19.478+0000 7f9849d4d700 1 -- 172.21.15.142:0/3192257234 wait complete. 2024-03-29T04:50:19.841 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:19 smithi152 ceph-mon[88186]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:19.841 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:19 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2328140456' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:50:19.841 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:19 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3192257234' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:50:19.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:19 smithi142 ceph-mon[112475]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:19.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:19 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2328140456' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:50:19.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:19 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3192257234' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:50:20.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:20 smithi142 ceph-mon[112475]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:20.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:20 smithi142 ceph-mon[112475]: pgmap v13: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:21.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:20 smithi152 ceph-mon[88186]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:21.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:20 smithi152 ceph-mon[88186]: pgmap v13: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:21.874 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:21 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:21.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:21 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:21.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:21 smithi142 ceph-mon[112475]: pgmap v14: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:22.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:21 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:22.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:21 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:22.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:21 smithi152 ceph-mon[88186]: pgmap v14: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:23.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:22 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:23.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:22 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:23.124 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:22 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:23.124 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:22 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:24.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:23 smithi152 ceph-mon[88186]: pgmap v15: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:24.124 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:23 smithi142 ceph-mon[112475]: pgmap v15: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:26.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:26 smithi142 ceph-mon[112475]: pgmap v16: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:26.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:26.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:26.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:26.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:50:26.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:26.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:50:26.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T04:50:26.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:27.043 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:26 smithi152 ceph-mon[88186]: pgmap v16: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:27.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:27.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:27.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:27.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:50:27.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:27.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:50:27.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T04:50:27.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:27.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:27 smithi142 ceph-mon[112475]: Reconfiguring mon.smithi142 (monmap changed)... 2024-03-29T04:50:27.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:27 smithi142 ceph-mon[112475]: Reconfiguring daemon mon.smithi142 on smithi142 2024-03-29T04:50:27.812 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:27 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:27.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:27 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:27.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:27 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.swesrg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:50:27.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:27 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:50:27.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:27 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:28.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:27 smithi152 ceph-mon[88186]: Reconfiguring mon.smithi142 (monmap changed)... 2024-03-29T04:50:28.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:27 smithi152 ceph-mon[88186]: Reconfiguring daemon mon.smithi142 on smithi142 2024-03-29T04:50:28.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:27 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:28.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:27 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:28.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:27 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi142.swesrg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:50:28.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:27 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:50:28.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:27 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:28.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:28 smithi142 ceph-mon[112475]: Reconfiguring mgr.smithi142.swesrg (monmap changed)... 2024-03-29T04:50:28.760 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:28 smithi142 ceph-mon[112475]: Reconfiguring daemon mgr.smithi142.swesrg on smithi142 2024-03-29T04:50:28.761 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:28 smithi142 ceph-mon[112475]: pgmap v17: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:29.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:28 smithi152 ceph-mon[88186]: Reconfiguring mgr.smithi142.swesrg (monmap changed)... 2024-03-29T04:50:29.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:28 smithi152 ceph-mon[88186]: Reconfiguring daemon mgr.smithi142.swesrg on smithi142 2024-03-29T04:50:29.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:28 smithi152 ceph-mon[88186]: pgmap v17: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:30.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:30.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:30.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:29 smithi142 ceph-mon[112475]: Reconfiguring ceph-exporter.smithi142 (monmap changed)... 2024-03-29T04:50:30.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:50:30.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:50:30.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-29T04:50:30.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:29 smithi142 ceph-mon[112475]: Unable to update caps for client.ceph-exporter.smithi142 2024-03-29T04:50:30.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi142"}]: dispatch 2024-03-29T04:50:30.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:30.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:29 smithi142 ceph-mon[112475]: Reconfiguring daemon ceph-exporter.smithi142 on smithi142 2024-03-29T04:50:30.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:29 smithi142 ceph-mon[112475]: pgmap v18: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:30.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:50:30.273 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:30.274 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:30.274 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:29 smithi152 ceph-mon[88186]: Reconfiguring ceph-exporter.smithi142 (monmap changed)... 2024-03-29T04:50:30.274 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:50:30.274 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:50:30.274 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-29T04:50:30.274 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:29 smithi152 ceph-mon[88186]: Unable to update caps for client.ceph-exporter.smithi142 2024-03-29T04:50:30.274 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi142"}]: dispatch 2024-03-29T04:50:30.274 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:30.274 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:29 smithi152 ceph-mon[88186]: Reconfiguring daemon ceph-exporter.smithi142 on smithi142 2024-03-29T04:50:30.274 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:29 smithi152 ceph-mon[88186]: pgmap v18: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:30.274 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:50:31.624 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:31 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:31.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:31 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:31.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:31 smithi142 ceph-mon[112475]: Reconfiguring crash.smithi142 (monmap changed)... 2024-03-29T04:50:31.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:31 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:50:31.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:31 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:31.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:31 smithi142 ceph-mon[112475]: Reconfiguring daemon crash.smithi142 on smithi142 2024-03-29T04:50:31.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:31 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:31.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:31 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:31.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:31 smithi152 ceph-mon[88186]: Reconfiguring crash.smithi142 (monmap changed)... 2024-03-29T04:50:31.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:31 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:50:31.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:31 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:31.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:31 smithi152 ceph-mon[88186]: Reconfiguring daemon crash.smithi142 on smithi142 2024-03-29T04:50:32.790 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:32 smithi142 ceph-mon[112475]: pgmap v19: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:32.790 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:32.790 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:32.790 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-29T04:50:32.790 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:32.793 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:32 smithi152 ceph-mon[88186]: pgmap v19: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:32.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:32.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:32.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-29T04:50:32.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:33.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:33 smithi152 ceph-mon[88186]: Reconfiguring osd.0 (monmap changed)... 2024-03-29T04:50:33.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:33 smithi152 ceph-mon[88186]: Reconfiguring daemon osd.0 on smithi142 2024-03-29T04:50:33.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:33 smithi142 ceph-mon[112475]: Reconfiguring osd.0 (monmap changed)... 2024-03-29T04:50:33.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:33 smithi142 ceph-mon[112475]: Reconfiguring daemon osd.0 on smithi142 2024-03-29T04:50:34.619 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:34 smithi142 ceph-mon[112475]: pgmap v20: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:34.620 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:34 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:34.620 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:34 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:34.620 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:34 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-29T04:50:34.620 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:34 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:34.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:34 smithi152 ceph-mon[88186]: pgmap v20: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:34.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:34 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:34.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:34 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:34.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:34 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-29T04:50:34.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:34 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:35.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:35 smithi142 ceph-mon[112475]: Reconfiguring osd.1 (monmap changed)... 2024-03-29T04:50:35.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:35 smithi142 ceph-mon[112475]: Reconfiguring daemon osd.1 on smithi142 2024-03-29T04:50:35.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:35.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:35.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-29T04:50:35.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:35.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:35 smithi152 ceph-mon[88186]: Reconfiguring osd.1 (monmap changed)... 2024-03-29T04:50:35.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:35 smithi152 ceph-mon[88186]: Reconfiguring daemon osd.1 on smithi142 2024-03-29T04:50:35.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:35.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:35.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-29T04:50:35.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:36.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:36 smithi142 ceph-mon[112475]: Reconfiguring osd.2 (monmap changed)... 2024-03-29T04:50:36.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:36 smithi142 ceph-mon[112475]: Reconfiguring daemon osd.2 on smithi142 2024-03-29T04:50:36.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:36 smithi142 ceph-mon[112475]: pgmap v21: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:36.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:36 smithi152 ceph-mon[88186]: Reconfiguring osd.2 (monmap changed)... 2024-03-29T04:50:36.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:36 smithi152 ceph-mon[88186]: Reconfiguring daemon osd.2 on smithi142 2024-03-29T04:50:36.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:36 smithi152 ceph-mon[88186]: pgmap v21: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:38.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:38 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:38.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:38 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:38.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:38 smithi142 ceph-mon[112475]: Reconfiguring mds.cephfs.smithi142.wgmndr (monmap changed)... 2024-03-29T04:50:38.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:38 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.wgmndr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:50:38.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:38 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:38.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:38 smithi142 ceph-mon[112475]: Reconfiguring daemon mds.cephfs.smithi142.wgmndr on smithi142 2024-03-29T04:50:38.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:38 smithi142 ceph-mon[112475]: pgmap v22: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:38.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:38 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:38.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:38 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:38.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:38 smithi152 ceph-mon[88186]: Reconfiguring mds.cephfs.smithi142.wgmndr (monmap changed)... 2024-03-29T04:50:38.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:38 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.wgmndr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:50:38.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:38 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:38.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:38 smithi152 ceph-mon[88186]: Reconfiguring daemon mds.cephfs.smithi142.wgmndr on smithi142 2024-03-29T04:50:38.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:38 smithi152 ceph-mon[88186]: pgmap v22: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:40.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:39 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:40.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:39 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:40.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:39 smithi142 ceph-mon[112475]: Reconfiguring mds.cephfs.smithi142.bmkjzf (monmap changed)... 2024-03-29T04:50:40.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:39 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.bmkjzf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:50:40.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:39 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:40.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:39 smithi142 ceph-mon[112475]: Reconfiguring daemon mds.cephfs.smithi142.bmkjzf on smithi142 2024-03-29T04:50:40.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:39 smithi142 ceph-mon[112475]: pgmap v23: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:40.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:39 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:40.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:39 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:40.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:39 smithi152 ceph-mon[88186]: Reconfiguring mds.cephfs.smithi142.bmkjzf (monmap changed)... 2024-03-29T04:50:40.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:39 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.bmkjzf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:50:40.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:39 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:40.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:39 smithi152 ceph-mon[88186]: Reconfiguring daemon mds.cephfs.smithi142.bmkjzf on smithi142 2024-03-29T04:50:40.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:39 smithi152 ceph-mon[88186]: pgmap v23: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:41.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:41 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:41.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:41 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:41.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:41 smithi152 ceph-mon[88186]: Reconfiguring ceph-exporter.smithi152 (monmap changed)... 2024-03-29T04:50:41.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:41 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi152", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:50:41.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:41 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi152", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:50:41.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:41 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi152", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-29T04:50:41.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:41 smithi152 ceph-mon[88186]: Unable to update caps for client.ceph-exporter.smithi152 2024-03-29T04:50:41.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:41 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi152"}]: dispatch 2024-03-29T04:50:41.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:41 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:41.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:41 smithi152 ceph-mon[88186]: Reconfiguring daemon ceph-exporter.smithi152 on smithi152 2024-03-29T04:50:41.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:41 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:41.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:41 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:41.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:41 smithi142 ceph-mon[112475]: Reconfiguring ceph-exporter.smithi152 (monmap changed)... 2024-03-29T04:50:41.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:41 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi152", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:50:41.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:41 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi152", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T04:50:41.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:41 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi152", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-29T04:50:41.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:41 smithi142 ceph-mon[112475]: Unable to update caps for client.ceph-exporter.smithi152 2024-03-29T04:50:41.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:41 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi152"}]: dispatch 2024-03-29T04:50:41.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:41 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:41.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:41 smithi142 ceph-mon[112475]: Reconfiguring daemon ceph-exporter.smithi152 on smithi152 2024-03-29T04:50:42.712 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:42 smithi152 ceph-mon[88186]: pgmap v24: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:42.712 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:42 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:42.712 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:42 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:42.712 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:42 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi152", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:50:42.712 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:42 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:42.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:42 smithi142 ceph-mon[112475]: pgmap v24: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:42.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:42 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:42.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:42 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:42.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:42 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi152", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:50:42.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:42 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:43.691 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:43 smithi152 ceph-mon[88186]: Reconfiguring crash.smithi152 (monmap changed)... 2024-03-29T04:50:43.692 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:43 smithi152 ceph-mon[88186]: Reconfiguring daemon crash.smithi152 on smithi152 2024-03-29T04:50:43.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:43 smithi142 ceph-mon[112475]: Reconfiguring crash.smithi152 (monmap changed)... 2024-03-29T04:50:43.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:43 smithi142 ceph-mon[112475]: Reconfiguring daemon crash.smithi152 on smithi152 2024-03-29T04:50:44.654 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:44 smithi152 ceph-mon[88186]: pgmap v25: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:44.654 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:44.654 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:44.654 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi152.usoolh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:50:44.654 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:50:44.654 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:44.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:44 smithi142 ceph-mon[112475]: pgmap v25: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:44.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:44.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:44.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi152.usoolh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T04:50:44.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T04:50:44.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:45.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:45 smithi152 ceph-mon[88186]: Reconfiguring mgr.smithi152.usoolh (monmap changed)... 2024-03-29T04:50:45.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:45 smithi152 ceph-mon[88186]: Reconfiguring daemon mgr.smithi152.usoolh on smithi152 2024-03-29T04:50:45.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:50:45.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:45.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:45.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:50:45.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T04:50:45.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:45.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:45 smithi142 ceph-mon[112475]: Reconfiguring mgr.smithi152.usoolh (monmap changed)... 2024-03-29T04:50:45.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:45 smithi142 ceph-mon[112475]: Reconfiguring daemon mgr.smithi152.usoolh on smithi152 2024-03-29T04:50:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:50:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T04:50:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T04:50:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:46.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:46 smithi152 ceph-mon[88186]: Reconfiguring mon.smithi152 (monmap changed)... 2024-03-29T04:50:46.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:46 smithi152 ceph-mon[88186]: Reconfiguring daemon mon.smithi152 on smithi152 2024-03-29T04:50:46.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:46 smithi152 ceph-mon[88186]: pgmap v26: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:46.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:46 smithi142 ceph-mon[112475]: Reconfiguring mon.smithi152 (monmap changed)... 2024-03-29T04:50:46.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:46 smithi142 ceph-mon[112475]: Reconfiguring daemon mon.smithi152 on smithi152 2024-03-29T04:50:46.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:46 smithi142 ceph-mon[112475]: pgmap v26: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:48.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:48 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:48.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:48 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:48.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:48 smithi152 ceph-mon[88186]: Reconfiguring osd.3 (monmap changed)... 2024-03-29T04:50:48.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:48 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-29T04:50:48.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:48 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:48.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:48 smithi152 ceph-mon[88186]: Reconfiguring daemon osd.3 on smithi152 2024-03-29T04:50:48.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:48 smithi152 ceph-mon[88186]: pgmap v27: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:48.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:48 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:48.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:48 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:48.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:48 smithi142 ceph-mon[112475]: Reconfiguring osd.3 (monmap changed)... 2024-03-29T04:50:48.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:48 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-29T04:50:48.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:48 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:48.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:48 smithi142 ceph-mon[112475]: Reconfiguring daemon osd.3 on smithi152 2024-03-29T04:50:48.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:48 smithi142 ceph-mon[112475]: pgmap v27: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:49.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.599+0000 7f6ce4060700 1 -- 172.21.15.142:0/3187892622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cdc100dc0 msgr2=0x7f6cdc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:49.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.599+0000 7f6ce4060700 1 --2- 172.21.15.142:0/3187892622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cdc100dc0 0x7f6cdc1011a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009910 tx=0x7f6cd8009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:49.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.599+0000 7f6ce4060700 1 -- 172.21.15.142:0/3187892622 shutdown_connections 2024-03-29T04:50:49.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.599+0000 7f6ce4060700 1 --2- 172.21.15.142:0/3187892622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cdc101770 0x7f6cdc1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:49.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.599+0000 7f6ce4060700 1 --2- 172.21.15.142:0/3187892622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cdc100dc0 0x7f6cdc1011a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:49.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.599+0000 7f6ce4060700 1 -- 172.21.15.142:0/3187892622 >> 172.21.15.142:0/3187892622 conn(0x7f6cdc0fc650 msgr2=0x7f6cdc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:49.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.599+0000 7f6ce4060700 1 -- 172.21.15.142:0/3187892622 shutdown_connections 2024-03-29T04:50:49.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.599+0000 7f6ce4060700 1 -- 172.21.15.142:0/3187892622 wait complete. 2024-03-29T04:50:49.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.599+0000 7f6ce4060700 1 Processor -- start 2024-03-29T04:50:49.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.599+0000 7f6ce4060700 1 -- start start 2024-03-29T04:50:49.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce4060700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cdc100dc0 0x7f6cdc19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:49.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce4060700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cdc101770 0x7f6cdc19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:49.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce4060700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc19d0d0 con 0x7f6cdc100dc0 2024-03-29T04:50:49.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce4060700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc196580 con 0x7f6cdc101770 2024-03-29T04:50:49.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce1dfc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cdc100dc0 0x7f6cdc19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce1dfc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cdc100dc0 0x7f6cdc19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53572/0 (socket says 172.21.15.142:53572) 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce1dfc700 1 -- 172.21.15.142:0/3751778460 learned_addr learned my addr 172.21.15.142:0/3751778460 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce15fb700 1 --2- 172.21.15.142:0/3751778460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cdc101770 0x7f6cdc19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce1dfc700 1 -- 172.21.15.142:0/3751778460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cdc101770 msgr2=0x7f6cdc19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce1dfc700 1 --2- 172.21.15.142:0/3751778460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cdc101770 0x7f6cdc19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce1dfc700 1 -- 172.21.15.142:0/3751778460 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd80095f0 con 0x7f6cdc100dc0 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce1dfc700 1 --2- 172.21.15.142:0/3751778460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cdc100dc0 0x7f6cdc19c500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800f690 tx=0x7f6cd800f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6cd2ffd700 1 -- 172.21.15.142:0/3751778460 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cd801d070 con 0x7f6cdc100dc0 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6ce4060700 1 -- 172.21.15.142:0/3751778460 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc196800 con 0x7f6cdc100dc0 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6cd2ffd700 1 -- 172.21.15.142:0/3751778460 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cd80050d0 con 0x7f6cdc100dc0 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.600+0000 7f6cd2ffd700 1 -- 172.21.15.142:0/3751778460 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cd8022470 con 0x7f6cdc100dc0 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.601+0000 7f6ce4060700 1 -- 172.21.15.142:0/3751778460 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc196cf0 con 0x7f6cdc100dc0 2024-03-29T04:50:49.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.602+0000 7f6ce4060700 1 -- 172.21.15.142:0/3751778460 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cdc109160 con 0x7f6cdc100dc0 2024-03-29T04:50:49.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.602+0000 7f6cd2ffd700 1 -- 172.21.15.142:0/3751778460 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7f6cd8003780 con 0x7f6cdc100dc0 2024-03-29T04:50:49.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.602+0000 7f6cd2ffd700 1 --2- 172.21.15.142:0/3751778460 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6cc806de90 0x7f6cc8070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:49.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.603+0000 7f6cd2ffd700 1 -- 172.21.15.142:0/3751778460 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6cd808f140 con 0x7f6cdc100dc0 2024-03-29T04:50:49.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.603+0000 7f6ce15fb700 1 --2- 172.21.15.142:0/3751778460 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6cc806de90 0x7f6cc8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:49.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.603+0000 7f6ce15fb700 1 --2- 172.21.15.142:0/3751778460 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6cc806de90 0x7f6cc8070350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc1059b0 tx=0x7f6ccc009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:49.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.607+0000 7f6cd2ffd700 1 -- 172.21.15.142:0/3751778460 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6cd805c0f0 con 0x7f6cdc100dc0 2024-03-29T04:50:49.827 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.822+0000 7f6ce4060700 1 -- 172.21.15.142:0/3751778460 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cdc02ccc0 con 0x7f6cc806de90 2024-03-29T04:50:49.828 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.824+0000 7f6cd2ffd700 1 -- 172.21.15.142:0/3751778460 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f6cdc02ccc0 con 0x7f6cc806de90 2024-03-29T04:50:49.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.826+0000 7f6ce4060700 1 -- 172.21.15.142:0/3751778460 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6cc806de90 msgr2=0x7f6cc8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:49.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.826+0000 7f6ce4060700 1 --2- 172.21.15.142:0/3751778460 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6cc806de90 0x7f6cc8070350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc1059b0 tx=0x7f6ccc009260 comp rx=0 tx=0).stop 2024-03-29T04:50:49.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.826+0000 7f6ce4060700 1 -- 172.21.15.142:0/3751778460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cdc100dc0 msgr2=0x7f6cdc19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:49.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.826+0000 7f6ce4060700 1 --2- 172.21.15.142:0/3751778460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cdc100dc0 0x7f6cdc19c500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800f690 tx=0x7f6cd800f6c0 comp rx=0 tx=0).stop 2024-03-29T04:50:49.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.826+0000 7f6ce4060700 1 -- 172.21.15.142:0/3751778460 shutdown_connections 2024-03-29T04:50:49.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.826+0000 7f6ce4060700 1 --2- 172.21.15.142:0/3751778460 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6cc806de90 0x7f6cc8070350 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:49.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.826+0000 7f6ce4060700 1 --2- 172.21.15.142:0/3751778460 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cdc100dc0 0x7f6cdc19c500 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:49.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.826+0000 7f6ce4060700 1 --2- 172.21.15.142:0/3751778460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cdc101770 0x7f6cdc19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:49.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.826+0000 7f6ce4060700 1 -- 172.21.15.142:0/3751778460 >> 172.21.15.142:0/3751778460 conn(0x7f6cdc0fc650 msgr2=0x7f6cdc0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:49.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.826+0000 7f6ce4060700 1 -- 172.21.15.142:0/3751778460 shutdown_connections 2024-03-29T04:50:49.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.826+0000 7f6ce4060700 1 -- 172.21.15.142:0/3751778460 wait complete. 2024-03-29T04:50:49.840 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:50:49.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:49 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:49.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:49 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:49.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:49 smithi142 ceph-mon[112475]: Reconfiguring osd.4 (monmap changed)... 2024-03-29T04:50:49.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:49 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-29T04:50:49.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:49 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:49.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:49 smithi142 ceph-mon[112475]: Reconfiguring daemon osd.4 on smithi152 2024-03-29T04:50:49.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:49 smithi142 ceph-mon[112475]: pgmap v28: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:49.931 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:49 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:49.931 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:49 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:49.931 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:49 smithi152 ceph-mon[88186]: Reconfiguring osd.4 (monmap changed)... 2024-03-29T04:50:49.931 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:49 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-29T04:50:49.931 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:49 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:49.931 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:49 smithi152 ceph-mon[88186]: Reconfiguring daemon osd.4 on smithi152 2024-03-29T04:50:49.931 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:49 smithi152 ceph-mon[88186]: pgmap v28: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:49.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.933+0000 7fed09640700 1 -- 172.21.15.142:0/1974822167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04100da0 msgr2=0x7fed04101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:49.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.933+0000 7fed09640700 1 --2- 172.21.15.142:0/1974822167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04100da0 0x7fed04101180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009910 tx=0x7fecf4009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:49.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.933+0000 7fed09640700 1 -- 172.21.15.142:0/1974822167 shutdown_connections 2024-03-29T04:50:49.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.933+0000 7fed09640700 1 --2- 172.21.15.142:0/1974822167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed04101750 0x7fed041057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:49.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.933+0000 7fed09640700 1 --2- 172.21.15.142:0/1974822167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04100da0 0x7fed04101180 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:49.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.933+0000 7fed09640700 1 -- 172.21.15.142:0/1974822167 >> 172.21.15.142:0/1974822167 conn(0x7fed040fc630 msgr2=0x7fed040fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:49.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.933+0000 7fed09640700 1 -- 172.21.15.142:0/1974822167 shutdown_connections 2024-03-29T04:50:49.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.933+0000 7fed09640700 1 -- 172.21.15.142:0/1974822167 wait complete. 2024-03-29T04:50:49.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed09640700 1 Processor -- start 2024-03-29T04:50:49.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed09640700 1 -- start start 2024-03-29T04:50:49.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed09640700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed04100da0 0x7fed04073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:49.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed09640700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04101750 0x7fed040735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed09640700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed04073b10 con 0x7fed04101750 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed09640700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed04073c50 con 0x7fed04100da0 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed027fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04101750 0x7fed040735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed02ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed04100da0 0x7fed04073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed027fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04101750 0x7fed040735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53590/0 (socket says 172.21.15.142:53590) 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed027fc700 1 -- 172.21.15.142:0/1040866996 learned_addr learned my addr 172.21.15.142:0/1040866996 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed02ffd700 1 -- 172.21.15.142:0/1040866996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04101750 msgr2=0x7fed040735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed02ffd700 1 --2- 172.21.15.142:0/1040866996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04101750 0x7fed040735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed02ffd700 1 -- 172.21.15.142:0/1040866996 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf40095f0 con 0x7fed04100da0 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.934+0000 7fed027fc700 1 --2- 172.21.15.142:0/1040866996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04101750 0x7fed040735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.935+0000 7fed02ffd700 1 --2- 172.21.15.142:0/1040866996 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed04100da0 0x7fed04073090 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009fd0 tx=0x7fecf400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.935+0000 7fecebfff700 1 -- 172.21.15.142:0/1040866996 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecf401d070 con 0x7fed04100da0 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.935+0000 7fecebfff700 1 -- 172.21.15.142:0/1040866996 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecf400b610 con 0x7fed04100da0 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.935+0000 7fecebfff700 1 -- 172.21.15.142:0/1040866996 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecf4022470 con 0x7fed04100da0 2024-03-29T04:50:49.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.935+0000 7fed09640700 1 -- 172.21.15.142:0/1040866996 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed04073ed0 con 0x7fed04100da0 2024-03-29T04:50:49.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.935+0000 7fed09640700 1 -- 172.21.15.142:0/1040866996 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed041a2cd0 con 0x7fed04100da0 2024-03-29T04:50:49.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.936+0000 7fed09640700 1 -- 172.21.15.142:0/1040866996 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed04109140 con 0x7fed04100da0 2024-03-29T04:50:49.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.936+0000 7fecebfff700 1 -- 172.21.15.142:0/1040866996 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7fecf40050d0 con 0x7fed04100da0 2024-03-29T04:50:49.942 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.937+0000 7fecebfff700 1 --2- 172.21.15.142:0/1040866996 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fecec06dea0 0x7fecec070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:49.942 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.937+0000 7fecebfff700 1 -- 172.21.15.142:0/1040866996 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fecf408f1b0 con 0x7fed04100da0 2024-03-29T04:50:49.942 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.937+0000 7fed027fc700 1 --2- 172.21.15.142:0/1040866996 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fecec06dea0 0x7fecec070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:49.942 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.937+0000 7fed027fc700 1 --2- 172.21.15.142:0/1040866996 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fecec06dea0 0x7fecec070360 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed04074880 tx=0x7fecf8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:49.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:49.942+0000 7fecebfff700 1 -- 172.21.15.142:0/1040866996 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fecf4058580 con 0x7fed04100da0 2024-03-29T04:50:50.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.152+0000 7fed09640700 1 -- 172.21.15.142:0/1040866996 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed0402cce0 con 0x7fecec06dea0 2024-03-29T04:50:50.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.154+0000 7fecebfff700 1 -- 172.21.15.142:0/1040866996 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fed0402cce0 con 0x7fecec06dea0 2024-03-29T04:50:50.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.156+0000 7fed09640700 1 -- 172.21.15.142:0/1040866996 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fecec06dea0 msgr2=0x7fecec070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:50.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.156+0000 7fed09640700 1 --2- 172.21.15.142:0/1040866996 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fecec06dea0 0x7fecec070360 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed04074880 tx=0x7fecf8009260 comp rx=0 tx=0).stop 2024-03-29T04:50:50.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.156+0000 7fed09640700 1 -- 172.21.15.142:0/1040866996 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed04100da0 msgr2=0x7fed04073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:50.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.156+0000 7fed09640700 1 --2- 172.21.15.142:0/1040866996 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed04100da0 0x7fed04073090 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009fd0 tx=0x7fecf400f690 comp rx=0 tx=0).stop 2024-03-29T04:50:50.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.156+0000 7fed09640700 1 -- 172.21.15.142:0/1040866996 shutdown_connections 2024-03-29T04:50:50.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.156+0000 7fed09640700 1 --2- 172.21.15.142:0/1040866996 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fecec06dea0 0x7fecec070360 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.156+0000 7fed09640700 1 --2- 172.21.15.142:0/1040866996 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed04100da0 0x7fed04073090 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.156+0000 7fed09640700 1 --2- 172.21.15.142:0/1040866996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04101750 0x7fed040735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.156+0000 7fed09640700 1 -- 172.21.15.142:0/1040866996 >> 172.21.15.142:0/1040866996 conn(0x7fed040fc630 msgr2=0x7fed040fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:50.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.156+0000 7fed09640700 1 -- 172.21.15.142:0/1040866996 shutdown_connections 2024-03-29T04:50:50.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.156+0000 7fed09640700 1 -- 172.21.15.142:0/1040866996 wait complete. 2024-03-29T04:50:50.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.260+0000 7fdc28887700 1 -- 172.21.15.142:0/3588270822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc240ffee0 msgr2=0x7fdc24105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:50.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.260+0000 7fdc28887700 1 --2- 172.21.15.142:0/3588270822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc240ffee0 0x7fdc24105870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009910 tx=0x7fdc0c009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:50.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.260+0000 7fdc28887700 1 -- 172.21.15.142:0/3588270822 shutdown_connections 2024-03-29T04:50:50.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.260+0000 7fdc28887700 1 --2- 172.21.15.142:0/3588270822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc240ffee0 0x7fdc24105870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.260+0000 7fdc28887700 1 --2- 172.21.15.142:0/3588270822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc240ff530 0x7fdc240ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.260+0000 7fdc28887700 1 -- 172.21.15.142:0/3588270822 >> 172.21.15.142:0/3588270822 conn(0x7fdc24074df0 msgr2=0x7fdc24075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:50.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.260+0000 7fdc28887700 1 -- 172.21.15.142:0/3588270822 shutdown_connections 2024-03-29T04:50:50.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.260+0000 7fdc28887700 1 -- 172.21.15.142:0/3588270822 wait complete. 2024-03-29T04:50:50.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc28887700 1 Processor -- start 2024-03-29T04:50:50.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc28887700 1 -- start start 2024-03-29T04:50:50.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc28887700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc240ff530 0x7fdc2419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:50.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc28887700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc240ffee0 0x7fdc2419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc28887700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2419d170 con 0x7fdc240ffee0 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc28887700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc24196620 con 0x7fdc240ff530 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc21d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc240ffee0 0x7fdc2419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc21d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc240ffee0 0x7fdc2419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53608/0 (socket says 172.21.15.142:53608) 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc21d9b700 1 -- 172.21.15.142:0/2772636474 learned_addr learned my addr 172.21.15.142:0/2772636474 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc2259c700 1 --2- 172.21.15.142:0/2772636474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc240ff530 0x7fdc2419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc21d9b700 1 -- 172.21.15.142:0/2772636474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc240ff530 msgr2=0x7fdc2419c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc21d9b700 1 --2- 172.21.15.142:0/2772636474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc240ff530 0x7fdc2419c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc21d9b700 1 -- 172.21.15.142:0/2772636474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c0095f0 con 0x7fdc240ffee0 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.261+0000 7fdc21d9b700 1 --2- 172.21.15.142:0/2772636474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc240ffee0 0x7fdc2419cae0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0059d0 tx=0x7fdc0c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.262+0000 7fdc1b7fe700 1 -- 172.21.15.142:0/2772636474 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc0c01d070 con 0x7fdc240ffee0 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.262+0000 7fdc28887700 1 -- 172.21.15.142:0/2772636474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc241968a0 con 0x7fdc240ffee0 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.262+0000 7fdc1b7fe700 1 -- 172.21.15.142:0/2772636474 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc0c00b610 con 0x7fdc240ffee0 2024-03-29T04:50:50.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.262+0000 7fdc1b7fe700 1 -- 172.21.15.142:0/2772636474 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc0c022470 con 0x7fdc240ffee0 2024-03-29T04:50:50.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.262+0000 7fdc28887700 1 -- 172.21.15.142:0/2772636474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc24196d90 con 0x7fdc240ffee0 2024-03-29T04:50:50.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.263+0000 7fdc28887700 1 -- 172.21.15.142:0/2772636474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc24109150 con 0x7fdc240ffee0 2024-03-29T04:50:50.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.263+0000 7fdc1b7fe700 1 -- 172.21.15.142:0/2772636474 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7fdc0c00fac0 con 0x7fdc240ffee0 2024-03-29T04:50:50.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.263+0000 7fdc1b7fe700 1 --2- 172.21.15.142:0/2772636474 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdc1006dea0 0x7fdc10070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:50.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.264+0000 7fdc1b7fe700 1 -- 172.21.15.142:0/2772636474 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fdc0c08f1e0 con 0x7fdc240ffee0 2024-03-29T04:50:50.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.264+0000 7fdc2259c700 1 --2- 172.21.15.142:0/2772636474 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdc1006dea0 0x7fdc10070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:50.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.264+0000 7fdc2259c700 1 --2- 172.21.15.142:0/2772636474 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdc1006dea0 0x7fdc10070360 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009c60 tx=0x7fdc1c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:50.272 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.268+0000 7fdc1b7fe700 1 -- 172.21.15.142:0/2772636474 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdc0c05c190 con 0x7fdc240ffee0 2024-03-29T04:50:50.479 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.475+0000 7fdc28887700 1 -- 172.21.15.142:0/2772636474 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc24197070 con 0x7fdc1006dea0 2024-03-29T04:50:50.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.486+0000 7fdc1b7fe700 1 -- 172.21.15.142:0/2772636474 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fdc24197070 con 0x7fdc1006dea0 2024-03-29T04:50:50.490 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:50:50.490 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (2m) 45s ago 14m 16.9M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:50:50.490 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (14m) 45s ago 14m 8186k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:50:50.490 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (13m) 29s ago 13m 8116k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:50:50.490 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (14m) 45s ago 14m 6665k - 18.2.1 5be31c24972a f4b954af2aa8 2024-03-29T04:50:50.490 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (13m) 29s ago 13m 6665k - 18.2.1 5be31c24972a 739f4e009370 2024-03-29T04:50:50.490 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (2m) 45s ago 13m 75.9M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (8m) 45s ago 8m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (8m) 45s ago 8m 63.2M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (8m) 29s ago 8m 65.6M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (8m) 29s ago 8m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (4m) 45s ago 15m 471M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (3m) 29s ago 13m 447M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (59s) 45s ago 15m 29.0M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (34s) 29s ago 13m 30.3M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (3m) 45s ago 14m 15.0M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (3m) 29s ago 13m 16.0M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (12m) 45s ago 12m 219M 1216M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (11m) 45s ago 11m 241M 1216M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (11m) 45s ago 11m 198M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (11m) 29s ago 11m 276M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:50:50.491 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (10m) 29s ago 10m 236M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:50:50.492 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (10m) 29s ago 10m 225M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:50:50.492 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (2m) 45s ago 13m 53.5M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:50:50.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.488+0000 7fdc28887700 1 -- 172.21.15.142:0/2772636474 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdc1006dea0 msgr2=0x7fdc10070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:50.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.488+0000 7fdc28887700 1 --2- 172.21.15.142:0/2772636474 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdc1006dea0 0x7fdc10070360 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009c60 tx=0x7fdc1c009260 comp rx=0 tx=0).stop 2024-03-29T04:50:50.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.488+0000 7fdc28887700 1 -- 172.21.15.142:0/2772636474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc240ffee0 msgr2=0x7fdc2419cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:50.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.488+0000 7fdc28887700 1 --2- 172.21.15.142:0/2772636474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc240ffee0 0x7fdc2419cae0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0059d0 tx=0x7fdc0c00f690 comp rx=0 tx=0).stop 2024-03-29T04:50:50.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.488+0000 7fdc28887700 1 -- 172.21.15.142:0/2772636474 shutdown_connections 2024-03-29T04:50:50.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.488+0000 7fdc28887700 1 --2- 172.21.15.142:0/2772636474 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdc1006dea0 0x7fdc10070360 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.488+0000 7fdc28887700 1 --2- 172.21.15.142:0/2772636474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc240ff530 0x7fdc2419c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.488+0000 7fdc28887700 1 --2- 172.21.15.142:0/2772636474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc240ffee0 0x7fdc2419cae0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.488+0000 7fdc28887700 1 -- 172.21.15.142:0/2772636474 >> 172.21.15.142:0/2772636474 conn(0x7fdc24074df0 msgr2=0x7fdc241050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:50.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.488+0000 7fdc28887700 1 -- 172.21.15.142:0/2772636474 shutdown_connections 2024-03-29T04:50:50.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.488+0000 7fdc28887700 1 -- 172.21.15.142:0/2772636474 wait complete. 2024-03-29T04:50:50.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.591+0000 7fac8e8ef700 1 -- 172.21.15.142:0/2778750820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac88100dc0 msgr2=0x7fac881011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:50.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.591+0000 7fac8e8ef700 1 --2- 172.21.15.142:0/2778750820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac88100dc0 0x7fac881011a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fac78009910 tx=0x7fac78009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:50.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.591+0000 7fac8e8ef700 1 -- 172.21.15.142:0/2778750820 shutdown_connections 2024-03-29T04:50:50.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.591+0000 7fac8e8ef700 1 --2- 172.21.15.142:0/2778750820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac88101770 0x7fac881057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.591+0000 7fac8e8ef700 1 --2- 172.21.15.142:0/2778750820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac88100dc0 0x7fac881011a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.591+0000 7fac8e8ef700 1 -- 172.21.15.142:0/2778750820 >> 172.21.15.142:0/2778750820 conn(0x7fac880fc650 msgr2=0x7fac880fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:50.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.591+0000 7fac8e8ef700 1 -- 172.21.15.142:0/2778750820 shutdown_connections 2024-03-29T04:50:50.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.591+0000 7fac8e8ef700 1 -- 172.21.15.142:0/2778750820 wait complete. 2024-03-29T04:50:50.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.592+0000 7fac8e8ef700 1 Processor -- start 2024-03-29T04:50:50.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.592+0000 7fac8e8ef700 1 -- start start 2024-03-29T04:50:50.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.592+0000 7fac8e8ef700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac88100dc0 0x7fac88073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:50.597 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.592+0000 7fac8e8ef700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac88101770 0x7fac880735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.592+0000 7fac8e8ef700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac88073b10 con 0x7fac88101770 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.592+0000 7fac8e8ef700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac88073c50 con 0x7fac88100dc0 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac877fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac88101770 0x7fac880735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac877fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac88101770 0x7fac880735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53618/0 (socket says 172.21.15.142:53618) 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac877fe700 1 -- 172.21.15.142:0/240906540 learned_addr learned my addr 172.21.15.142:0/240906540 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac87fff700 1 --2- 172.21.15.142:0/240906540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac88100dc0 0x7fac88073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac877fe700 1 -- 172.21.15.142:0/240906540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac88100dc0 msgr2=0x7fac88073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac877fe700 1 --2- 172.21.15.142:0/240906540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac88100dc0 0x7fac88073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac877fe700 1 -- 172.21.15.142:0/240906540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac780095f0 con 0x7fac88101770 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac877fe700 1 --2- 172.21.15.142:0/240906540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac88101770 0x7fac880735d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00d8d0 tx=0x7fac7c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac857fa700 1 -- 172.21.15.142:0/240906540 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac7c0041f0 con 0x7fac88101770 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac857fa700 1 -- 172.21.15.142:0/240906540 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac7c007620 con 0x7fac88101770 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac8e8ef700 1 -- 172.21.15.142:0/240906540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac881a2940 con 0x7fac88101770 2024-03-29T04:50:50.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac857fa700 1 -- 172.21.15.142:0/240906540 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac7c0056b0 con 0x7fac88101770 2024-03-29T04:50:50.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.593+0000 7fac8e8ef700 1 -- 172.21.15.142:0/240906540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac881a2d50 con 0x7fac88101770 2024-03-29T04:50:50.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.594+0000 7fac8e8ef700 1 -- 172.21.15.142:0/240906540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac88109160 con 0x7fac88101770 2024-03-29T04:50:50.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.595+0000 7fac857fa700 1 -- 172.21.15.142:0/240906540 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7fac7c00f7d0 con 0x7fac88101770 2024-03-29T04:50:50.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.595+0000 7fac857fa700 1 --2- 172.21.15.142:0/240906540 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fac7006dea0 0x7fac70070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:50.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.596+0000 7fac87fff700 1 --2- 172.21.15.142:0/240906540 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fac7006dea0 0x7fac70070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:50.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.596+0000 7fac857fa700 1 -- 172.21.15.142:0/240906540 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fac7c08db90 con 0x7fac88101770 2024-03-29T04:50:50.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.596+0000 7fac87fff700 1 --2- 172.21.15.142:0/240906540 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fac7006dea0 0x7fac70070360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac78006010 tx=0x7fac7801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:50.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.600+0000 7fac857fa700 1 -- 172.21.15.142:0/240906540 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fac7c056f60 con 0x7fac88101770 2024-03-29T04:50:50.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.883+0000 7fac8e8ef700 1 -- 172.21.15.142:0/240906540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fac8802cc70 con 0x7fac88101770 2024-03-29T04:50:50.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.883+0000 7fac857fa700 1 -- 172.21.15.142:0/240906540 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7fac7c05a620 con 0x7fac88101770 2024-03-29T04:50:50.887 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:50:50.887 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:50:50.888 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:50:50.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:50:50.888 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:50:50.888 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:50:50.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:50:50.888 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:50:50.888 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-29T04:50:50.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:50:50.888 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:50:50.888 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:50:50.888 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:50:50.888 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:50:50.889 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-03-29T04:50:50.889 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T04:50:50.889 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:50:50.889 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:50:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.885+0000 7fac8e8ef700 1 -- 172.21.15.142:0/240906540 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fac7006dea0 msgr2=0x7fac70070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.885+0000 7fac8e8ef700 1 --2- 172.21.15.142:0/240906540 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fac7006dea0 0x7fac70070360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac78006010 tx=0x7fac7801a040 comp rx=0 tx=0).stop 2024-03-29T04:50:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.885+0000 7fac8e8ef700 1 -- 172.21.15.142:0/240906540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac88101770 msgr2=0x7fac880735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.885+0000 7fac8e8ef700 1 --2- 172.21.15.142:0/240906540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac88101770 0x7fac880735d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00d8d0 tx=0x7fac7c00dbe0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.886+0000 7fac8e8ef700 1 -- 172.21.15.142:0/240906540 shutdown_connections 2024-03-29T04:50:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.886+0000 7fac8e8ef700 1 --2- 172.21.15.142:0/240906540 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fac7006dea0 0x7fac70070360 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.886+0000 7fac8e8ef700 1 --2- 172.21.15.142:0/240906540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac88100dc0 0x7fac88073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.886+0000 7fac8e8ef700 1 --2- 172.21.15.142:0/240906540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac88101770 0x7fac880735d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.886+0000 7fac8e8ef700 1 -- 172.21.15.142:0/240906540 >> 172.21.15.142:0/240906540 conn(0x7fac880fc650 msgr2=0x7fac880fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.886+0000 7fac8e8ef700 1 -- 172.21.15.142:0/240906540 shutdown_connections 2024-03-29T04:50:50.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.886+0000 7fac8e8ef700 1 -- 172.21.15.142:0/240906540 wait complete. 2024-03-29T04:50:50.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.989+0000 7f9ab07ab700 1 -- 172.21.15.142:0/622610700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa8105a40 msgr2=0x7f9aa8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:50.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.989+0000 7f9ab07ab700 1 --2- 172.21.15.142:0/622610700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa8105a40 0x7f9aa8105e20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009910 tx=0x7f9aa4009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:50.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.990+0000 7f9ab07ab700 1 -- 172.21.15.142:0/622610700 shutdown_connections 2024-03-29T04:50:50.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.990+0000 7f9ab07ab700 1 --2- 172.21.15.142:0/622610700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa80ffa40 0x7f9aa80ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.990+0000 7f9ab07ab700 1 --2- 172.21.15.142:0/622610700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa8105a40 0x7f9aa8105e20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.990+0000 7f9ab07ab700 1 -- 172.21.15.142:0/622610700 >> 172.21.15.142:0/622610700 conn(0x7f9aa80fb570 msgr2=0x7f9aa80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:50.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.990+0000 7f9ab07ab700 1 -- 172.21.15.142:0/622610700 shutdown_connections 2024-03-29T04:50:50.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.990+0000 7f9ab07ab700 1 -- 172.21.15.142:0/622610700 wait complete. 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.990+0000 7f9ab07ab700 1 Processor -- start 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.990+0000 7f9ab07ab700 1 -- start start 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.990+0000 7f9ab07ab700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa80ffa40 0x7f9aa81975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.990+0000 7f9ab07ab700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa8105a40 0x7f9aa8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.990+0000 7f9ab07ab700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa8198110 con 0x7f9aa8105a40 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.990+0000 7f9ab07ab700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa8198250 con 0x7f9aa80ffa40 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9aadd46700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa8105a40 0x7f9aa8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9aadd46700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa8105a40 0x7f9aa8197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53640/0 (socket says 172.21.15.142:53640) 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9aadd46700 1 -- 172.21.15.142:0/90072437 learned_addr learned my addr 172.21.15.142:0/90072437 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9aae547700 1 --2- 172.21.15.142:0/90072437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa80ffa40 0x7f9aa81975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9aadd46700 1 -- 172.21.15.142:0/90072437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa80ffa40 msgr2=0x7f9aa81975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9aadd46700 1 --2- 172.21.15.142:0/90072437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa80ffa40 0x7f9aa81975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9aadd46700 1 -- 172.21.15.142:0/90072437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa40095f0 con 0x7f9aa8105a40 2024-03-29T04:50:50.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9aadd46700 1 --2- 172.21.15.142:0/90072437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa8105a40 0x7f9aa8197af0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800d8d0 tx=0x7f9a9800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:50.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9a9f7fe700 1 -- 172.21.15.142:0/90072437 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a980098a0 con 0x7f9aa8105a40 2024-03-29T04:50:50.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9a9f7fe700 1 -- 172.21.15.142:0/90072437 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a98010460 con 0x7f9aa8105a40 2024-03-29T04:50:50.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9a9f7fe700 1 -- 172.21.15.142:0/90072437 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a9800f5f0 con 0x7f9aa8105a40 2024-03-29T04:50:50.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9ab07ab700 1 -- 172.21.15.142:0/90072437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa81a0620 con 0x7f9aa8105a40 2024-03-29T04:50:50.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.991+0000 7f9ab07ab700 1 -- 172.21.15.142:0/90072437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa81a0ae0 con 0x7f9aa8105a40 2024-03-29T04:50:50.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.992+0000 7f9ab07ab700 1 -- 172.21.15.142:0/90072437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa8108040 con 0x7f9aa8105a40 2024-03-29T04:50:50.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.993+0000 7f9a9f7fe700 1 -- 172.21.15.142:0/90072437 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7f9a98009a00 con 0x7f9aa8105a40 2024-03-29T04:50:50.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.993+0000 7f9a9f7fe700 1 --2- 172.21.15.142:0/90072437 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9a9406dea0 0x7f9a94070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:50.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.993+0000 7f9a9f7fe700 1 -- 172.21.15.142:0/90072437 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9a9808dbf0 con 0x7f9aa8105a40 2024-03-29T04:50:50.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.993+0000 7f9aae547700 1 --2- 172.21.15.142:0/90072437 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9a9406dea0 0x7f9a94070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:50.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.994+0000 7f9aae547700 1 --2- 172.21.15.142:0/90072437 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9a9406dea0 0x7f9a94070360 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4006010 tx=0x7f9aa401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:51.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:50.998+0000 7f9a9f7fe700 1 -- 172.21.15.142:0/90072437 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9a98056fc0 con 0x7f9aa8105a40 2024-03-29T04:50:51.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.233+0000 7f9ab07ab700 1 -- 172.21.15.142:0/90072437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9aa804ea90 con 0x7f9aa8105a40 2024-03-29T04:50:51.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.233+0000 7f9a9f7fe700 1 -- 172.21.15.142:0/90072437 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1906 (secure 0 0 0) 0x7f9a9805a680 con 0x7f9aa8105a40 2024-03-29T04:50:51.238 INFO:teuthology.orchestra.run.smithi142.stdout:e21 2024-03-29T04:50:51.238 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:50:51.238 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:50:51.238 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:50:51.238 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:50:51.238 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:50:51.238 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:50:51.238 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 21 2024-03-29T04:50:51.238 INFO:teuthology.orchestra.run.smithi142.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T04:50:51.238 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:50:51.238 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:49:01.288637+0000 2024-03-29T04:50:51.238 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 47 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:up {0=14494,1=24275} 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T04:50:51.239 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{0:14494} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{0:14506} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{1:24275} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{1:24287} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:50:51.240 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:50:51.241 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:50:51.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.236+0000 7f9ab07ab700 1 -- 172.21.15.142:0/90072437 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9a9406dea0 msgr2=0x7f9a94070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:51.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.236+0000 7f9ab07ab700 1 --2- 172.21.15.142:0/90072437 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9a9406dea0 0x7f9a94070360 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4006010 tx=0x7f9aa401a040 comp rx=0 tx=0).stop 2024-03-29T04:50:51.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.236+0000 7f9ab07ab700 1 -- 172.21.15.142:0/90072437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa8105a40 msgr2=0x7f9aa8197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:51.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.236+0000 7f9ab07ab700 1 --2- 172.21.15.142:0/90072437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa8105a40 0x7f9aa8197af0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800d8d0 tx=0x7f9a9800dbe0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.236+0000 7f9ab07ab700 1 -- 172.21.15.142:0/90072437 shutdown_connections 2024-03-29T04:50:51.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.236+0000 7f9ab07ab700 1 --2- 172.21.15.142:0/90072437 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9a9406dea0 0x7f9a94070360 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.236+0000 7f9ab07ab700 1 --2- 172.21.15.142:0/90072437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa80ffa40 0x7f9aa81975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.236+0000 7f9ab07ab700 1 --2- 172.21.15.142:0/90072437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9aa8105a40 0x7f9aa8197af0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.236+0000 7f9ab07ab700 1 -- 172.21.15.142:0/90072437 >> 172.21.15.142:0/90072437 conn(0x7f9aa80fb570 msgr2=0x7f9aa80fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:51.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.237+0000 7f9ab07ab700 1 -- 172.21.15.142:0/90072437 shutdown_connections 2024-03-29T04:50:51.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.237+0000 7f9ab07ab700 1 -- 172.21.15.142:0/90072437 wait complete. 2024-03-29T04:50:51.244 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 21 2024-03-29T04:50:51.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.342+0000 7fb994b61700 1 -- 172.21.15.142:0/1715129099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990101770 msgr2=0x7fb9901057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:51.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.342+0000 7fb994b61700 1 --2- 172.21.15.142:0/1715129099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990101770 0x7fb9901057c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb980009910 tx=0x7fb980009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:51.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.342+0000 7fb994b61700 1 -- 172.21.15.142:0/1715129099 shutdown_connections 2024-03-29T04:50:51.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.342+0000 7fb994b61700 1 --2- 172.21.15.142:0/1715129099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990101770 0x7fb9901057c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.342+0000 7fb994b61700 1 --2- 172.21.15.142:0/1715129099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb990100dc0 0x7fb9901011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.342+0000 7fb994b61700 1 -- 172.21.15.142:0/1715129099 >> 172.21.15.142:0/1715129099 conn(0x7fb9900fc650 msgr2=0x7fb9900fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:51.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.342+0000 7fb994b61700 1 -- 172.21.15.142:0/1715129099 shutdown_connections 2024-03-29T04:50:51.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.342+0000 7fb994b61700 1 -- 172.21.15.142:0/1715129099 wait complete. 2024-03-29T04:50:51.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.343+0000 7fb994b61700 1 Processor -- start 2024-03-29T04:50:51.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.343+0000 7fb994b61700 1 -- start start 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.343+0000 7fb994b61700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990100dc0 0x7fb99019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.343+0000 7fb994b61700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb990101770 0x7fb99019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.343+0000 7fb994b61700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99019d130 con 0x7fb990100dc0 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.343+0000 7fb994b61700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9901965e0 con 0x7fb990101770 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.343+0000 7fb98e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990100dc0 0x7fb99019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.343+0000 7fb98e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990100dc0 0x7fb99019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53664/0 (socket says 172.21.15.142:53664) 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.343+0000 7fb98e59c700 1 -- 172.21.15.142:0/2457394533 learned_addr learned my addr 172.21.15.142:0/2457394533 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.343+0000 7fb98dd9b700 1 --2- 172.21.15.142:0/2457394533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb990101770 0x7fb99019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.344+0000 7fb98e59c700 1 -- 172.21.15.142:0/2457394533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb990101770 msgr2=0x7fb99019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.344+0000 7fb98e59c700 1 --2- 172.21.15.142:0/2457394533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb990101770 0x7fb99019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.344+0000 7fb98e59c700 1 -- 172.21.15.142:0/2457394533 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9800095f0 con 0x7fb990100dc0 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.344+0000 7fb98e59c700 1 --2- 172.21.15.142:0/2457394533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990100dc0 0x7fb99019c560 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb98400d8d0 tx=0x7fb98400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:51.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.344+0000 7fb97f7fe700 1 -- 172.21.15.142:0/2457394533 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9840098a0 con 0x7fb990100dc0 2024-03-29T04:50:51.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.344+0000 7fb97f7fe700 1 -- 172.21.15.142:0/2457394533 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb984010460 con 0x7fb990100dc0 2024-03-29T04:50:51.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.344+0000 7fb994b61700 1 -- 172.21.15.142:0/2457394533 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9901968c0 con 0x7fb990100dc0 2024-03-29T04:50:51.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.344+0000 7fb97f7fe700 1 -- 172.21.15.142:0/2457394533 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb98400f5f0 con 0x7fb990100dc0 2024-03-29T04:50:51.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.344+0000 7fb994b61700 1 -- 172.21.15.142:0/2457394533 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb990196e10 con 0x7fb990100dc0 2024-03-29T04:50:51.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.345+0000 7fb994b61700 1 -- 172.21.15.142:0/2457394533 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9901091c0 con 0x7fb990100dc0 2024-03-29T04:50:51.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.346+0000 7fb97f7fe700 1 -- 172.21.15.142:0/2457394533 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7fb984009a00 con 0x7fb990100dc0 2024-03-29T04:50:51.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.346+0000 7fb97f7fe700 1 --2- 172.21.15.142:0/2457394533 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb97806dea0 0x7fb978070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:51.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.346+0000 7fb97f7fe700 1 -- 172.21.15.142:0/2457394533 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb98408dbf0 con 0x7fb990100dc0 2024-03-29T04:50:51.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.346+0000 7fb98dd9b700 1 --2- 172.21.15.142:0/2457394533 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb97806dea0 0x7fb978070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:51.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.346+0000 7fb98dd9b700 1 --2- 172.21.15.142:0/2457394533 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb97806dea0 0x7fb978070360 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9900fcea0 tx=0x7fb98001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:51.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.351+0000 7fb97f7fe700 1 -- 172.21.15.142:0/2457394533 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb98405aaf0 con 0x7fb990100dc0 2024-03-29T04:50:51.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:51 smithi142 ceph-mon[112475]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:51.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:51 smithi142 ceph-mon[112475]: from='client.44105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:51.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:51 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:51.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:51 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:51.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:51 smithi142 ceph-mon[112475]: Reconfiguring osd.5 (monmap changed)... 2024-03-29T04:50:51.512 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:51 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-29T04:50:51.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:51 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:51.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:51 smithi142 ceph-mon[112475]: Reconfiguring daemon osd.5 on smithi152 2024-03-29T04:50:51.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:51 smithi142 ceph-mon[112475]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:51.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:51 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/240906540' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:50:51.513 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:51 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/90072437' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:50:51.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:51 smithi152 ceph-mon[88186]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:51.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:51 smithi152 ceph-mon[88186]: from='client.44105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:51.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:51 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:51.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:51 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:51.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:51 smithi152 ceph-mon[88186]: Reconfiguring osd.5 (monmap changed)... 2024-03-29T04:50:51.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:51 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-29T04:50:51.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:51 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:51.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:51 smithi152 ceph-mon[88186]: Reconfiguring daemon osd.5 on smithi152 2024-03-29T04:50:51.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:51 smithi152 ceph-mon[88186]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:51.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:51 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/240906540' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:50:51.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:51 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/90072437' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:50:51.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.564+0000 7fb994b61700 1 -- 172.21.15.142:0/2457394533 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9901970b0 con 0x7fb97806dea0 2024-03-29T04:50:51.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.565+0000 7fb97f7fe700 1 -- 172.21.15.142:0/2457394533 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fb9901970b0 con 0x7fb97806dea0 2024-03-29T04:50:51.569 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:50:51.569 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:50:51.570 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:50:51.570 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:50:51.570 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:50:51.570 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:50:51.570 INFO:teuthology.orchestra.run.smithi142.stdout: "mon" 2024-03-29T04:50:51.570 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:50:51.570 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "4/23 daemons upgraded", 2024-03-29T04:50:51.570 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mon daemons", 2024-03-29T04:50:51.571 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:50:51.571 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:50:51.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.568+0000 7fb994b61700 1 -- 172.21.15.142:0/2457394533 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb97806dea0 msgr2=0x7fb978070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:51.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.568+0000 7fb994b61700 1 --2- 172.21.15.142:0/2457394533 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb97806dea0 0x7fb978070360 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9900fcea0 tx=0x7fb98001a040 comp rx=0 tx=0).stop 2024-03-29T04:50:51.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.568+0000 7fb994b61700 1 -- 172.21.15.142:0/2457394533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990100dc0 msgr2=0x7fb99019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:51.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.568+0000 7fb994b61700 1 --2- 172.21.15.142:0/2457394533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990100dc0 0x7fb99019c560 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb98400d8d0 tx=0x7fb98400dbe0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.568+0000 7fb994b61700 1 -- 172.21.15.142:0/2457394533 shutdown_connections 2024-03-29T04:50:51.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.568+0000 7fb994b61700 1 --2- 172.21.15.142:0/2457394533 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb97806dea0 0x7fb978070360 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.568+0000 7fb994b61700 1 --2- 172.21.15.142:0/2457394533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990100dc0 0x7fb99019c560 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.568+0000 7fb994b61700 1 --2- 172.21.15.142:0/2457394533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb990101770 0x7fb99019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.568+0000 7fb994b61700 1 -- 172.21.15.142:0/2457394533 >> 172.21.15.142:0/2457394533 conn(0x7fb9900fc650 msgr2=0x7fb9900fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:51.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.568+0000 7fb994b61700 1 -- 172.21.15.142:0/2457394533 shutdown_connections 2024-03-29T04:50:51.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.568+0000 7fb994b61700 1 -- 172.21.15.142:0/2457394533 wait complete. 2024-03-29T04:50:51.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.672+0000 7f1c29eaf700 1 -- 172.21.15.142:0/741967140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c24068670 msgr2=0x7f1c24068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:51.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.672+0000 7f1c29eaf700 1 --2- 172.21.15.142:0/741967140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c24068670 0x7f1c24068a50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009910 tx=0x7f1c14009c20 comp rx=0 tx=0).stop 2024-03-29T04:50:51.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.672+0000 7f1c29eaf700 1 -- 172.21.15.142:0/741967140 shutdown_connections 2024-03-29T04:50:51.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.672+0000 7f1c29eaf700 1 --2- 172.21.15.142:0/741967140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c24069020 0x7f1c24105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.672+0000 7f1c29eaf700 1 --2- 172.21.15.142:0/741967140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c24068670 0x7f1c24068a50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.672+0000 7f1c29eaf700 1 -- 172.21.15.142:0/741967140 >> 172.21.15.142:0/741967140 conn(0x7f1c240755a0 msgr2=0x7f1c240759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:51.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.672+0000 7f1c29eaf700 1 -- 172.21.15.142:0/741967140 shutdown_connections 2024-03-29T04:50:51.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.672+0000 7f1c29eaf700 1 -- 172.21.15.142:0/741967140 wait complete. 2024-03-29T04:50:51.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.673+0000 7f1c29eaf700 1 Processor -- start 2024-03-29T04:50:51.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.673+0000 7f1c29eaf700 1 -- start start 2024-03-29T04:50:51.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.673+0000 7f1c29eaf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c24068670 0x7f1c2419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:51.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.673+0000 7f1c29eaf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c24069020 0x7f1c2419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:51.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.673+0000 7f1c29eaf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2419d0e0 con 0x7f1c24068670 2024-03-29T04:50:51.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.673+0000 7f1c29eaf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24196590 con 0x7f1c24069020 2024-03-29T04:50:51.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.673+0000 7f1c237fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c24068670 0x7f1c2419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:51.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.673+0000 7f1c237fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c24068670 0x7f1c2419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53680/0 (socket says 172.21.15.142:53680) 2024-03-29T04:50:51.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.673+0000 7f1c237fe700 1 -- 172.21.15.142:0/1263987556 learned_addr learned my addr 172.21.15.142:0/1263987556 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:50:51.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.673+0000 7f1c22ffd700 1 --2- 172.21.15.142:0/1263987556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c24069020 0x7f1c2419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:51.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.674+0000 7f1c237fe700 1 -- 172.21.15.142:0/1263987556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c24069020 msgr2=0x7f1c2419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:51.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.674+0000 7f1c237fe700 1 --2- 172.21.15.142:0/1263987556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c24069020 0x7f1c2419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.674+0000 7f1c237fe700 1 -- 172.21.15.142:0/1263987556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c140095f0 con 0x7f1c24068670 2024-03-29T04:50:51.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.674+0000 7f1c22ffd700 1 --2- 172.21.15.142:0/1263987556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c24069020 0x7f1c2419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:50:51.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.674+0000 7f1c237fe700 1 --2- 172.21.15.142:0/1263987556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c24068670 0x7f1c2419c510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c14005f50 tx=0x7f1c1400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:51.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.674+0000 7f1c20ff9700 1 -- 172.21.15.142:0/1263987556 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c1401d070 con 0x7f1c24068670 2024-03-29T04:50:51.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.674+0000 7f1c20ff9700 1 -- 172.21.15.142:0/1263987556 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c1400b610 con 0x7f1c24068670 2024-03-29T04:50:51.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.674+0000 7f1c20ff9700 1 -- 172.21.15.142:0/1263987556 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c14022470 con 0x7f1c24068670 2024-03-29T04:50:51.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.674+0000 7f1c29eaf700 1 -- 172.21.15.142:0/1263987556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c24196810 con 0x7f1c24068670 2024-03-29T04:50:51.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.674+0000 7f1c29eaf700 1 -- 172.21.15.142:0/1263987556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c24196d00 con 0x7f1c24068670 2024-03-29T04:50:51.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.675+0000 7f1c29eaf700 1 -- 172.21.15.142:0/1263987556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c24109130 con 0x7f1c24068670 2024-03-29T04:50:51.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.675+0000 7f1c20ff9700 1 -- 172.21.15.142:0/1263987556 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7f1c14003780 con 0x7f1c24068670 2024-03-29T04:50:51.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.676+0000 7f1c20ff9700 1 --2- 172.21.15.142:0/1263987556 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1c0c06dea0 0x7f1c0c070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:50:51.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.676+0000 7f1c20ff9700 1 -- 172.21.15.142:0/1263987556 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1c1408f0b0 con 0x7f1c24068670 2024-03-29T04:50:51.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.676+0000 7f1c22ffd700 1 --2- 172.21.15.142:0/1263987556 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1c0c06dea0 0x7f1c0c070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:50:51.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.676+0000 7f1c22ffd700 1 --2- 172.21.15.142:0/1263987556 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1c0c06dea0 0x7f1c0c070360 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1c24107750 tx=0x7f1c18009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:50:51.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.680+0000 7f1c20ff9700 1 -- 172.21.15.142:0/1263987556 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1c1405bfd0 con 0x7f1c24068670 2024-03-29T04:50:51.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.957+0000 7f1c29eaf700 1 -- 172.21.15.142:0/1263987556 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1c2404ea90 con 0x7f1c24068670 2024-03-29T04:50:51.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.957+0000 7f1c20ff9700 1 -- 172.21.15.142:0/1263987556 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f1c1405bb60 con 0x7f1c24068670 2024-03-29T04:50:51.961 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_OK 2024-03-29T04:50:51.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.959+0000 7f1c29eaf700 1 -- 172.21.15.142:0/1263987556 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1c0c06dea0 msgr2=0x7f1c0c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:51.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.959+0000 7f1c29eaf700 1 --2- 172.21.15.142:0/1263987556 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1c0c06dea0 0x7f1c0c070360 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1c24107750 tx=0x7f1c18009260 comp rx=0 tx=0).stop 2024-03-29T04:50:51.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.959+0000 7f1c29eaf700 1 -- 172.21.15.142:0/1263987556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c24068670 msgr2=0x7f1c2419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:50:51.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.959+0000 7f1c29eaf700 1 --2- 172.21.15.142:0/1263987556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c24068670 0x7f1c2419c510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c14005f50 tx=0x7f1c1400f690 comp rx=0 tx=0).stop 2024-03-29T04:50:51.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.959+0000 7f1c29eaf700 1 -- 172.21.15.142:0/1263987556 shutdown_connections 2024-03-29T04:50:51.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.959+0000 7f1c29eaf700 1 --2- 172.21.15.142:0/1263987556 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1c0c06dea0 0x7f1c0c070360 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.959+0000 7f1c29eaf700 1 --2- 172.21.15.142:0/1263987556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c24068670 0x7f1c2419c510 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.959+0000 7f1c29eaf700 1 --2- 172.21.15.142:0/1263987556 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c24069020 0x7f1c2419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:50:51.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.959+0000 7f1c29eaf700 1 -- 172.21.15.142:0/1263987556 >> 172.21.15.142:0/1263987556 conn(0x7f1c240755a0 msgr2=0x7f1c240fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:50:51.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.960+0000 7f1c29eaf700 1 -- 172.21.15.142:0/1263987556 shutdown_connections 2024-03-29T04:50:51.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:50:51.960+0000 7f1c29eaf700 1 -- 172.21.15.142:0/1263987556 wait complete. 2024-03-29T04:50:52.623 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:52 smithi152 ceph-mon[88186]: pgmap v29: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:52.624 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:52 smithi152 ceph-mon[88186]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:52.624 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:52 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1263987556' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:50:52.624 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:52.624 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:52.624 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.uhebsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:50:52.624 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:52.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:52 smithi142 ceph-mon[112475]: pgmap v29: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:52.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:52 smithi142 ceph-mon[112475]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:50:52.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:52 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1263987556' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:50:52.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:52.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:52.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.uhebsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:50:52.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:53.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:53 smithi142 ceph-mon[112475]: Reconfiguring mds.cephfs.smithi152.uhebsi (monmap changed)... 2024-03-29T04:50:53.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:53 smithi142 ceph-mon[112475]: Reconfiguring daemon mds.cephfs.smithi152.uhebsi on smithi152 2024-03-29T04:50:53.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:53 smithi152 ceph-mon[88186]: Reconfiguring mds.cephfs.smithi152.uhebsi (monmap changed)... 2024-03-29T04:50:53.664 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:53 smithi152 ceph-mon[88186]: Reconfiguring daemon mds.cephfs.smithi152.uhebsi on smithi152 2024-03-29T04:50:54.559 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:54 smithi152 ceph-mon[88186]: pgmap v30: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:54.559 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:54.559 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:54.559 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.hiilum", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:50:54.559 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:54.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:54 smithi142 ceph-mon[112475]: pgmap v30: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:54.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:54.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:54.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.hiilum", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:50:54.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:55.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:55 smithi142 ceph-mon[112475]: Reconfiguring mds.cephfs.smithi152.hiilum (monmap changed)... 2024-03-29T04:50:55.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:55 smithi142 ceph-mon[112475]: Reconfiguring daemon mds.cephfs.smithi152.hiilum on smithi152 2024-03-29T04:50:55.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:55 smithi152 ceph-mon[88186]: Reconfiguring mds.cephfs.smithi152.hiilum (monmap changed)... 2024-03-29T04:50:55.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:55 smithi152 ceph-mon[88186]: Reconfiguring daemon mds.cephfs.smithi152.hiilum on smithi152 2024-03-29T04:50:56.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:56 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:56.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:56 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:56.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:56 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:50:56.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:56 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:50:56.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:56 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:50:56.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:56 smithi142 ceph-mon[112475]: Upgrade: Setting container_image for all mon 2024-03-29T04:50:56.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:56 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:56.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:56 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi142"}]: dispatch 2024-03-29T04:50:56.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:56 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi142"}]': finished 2024-03-29T04:50:56.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:56 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi152"}]: dispatch 2024-03-29T04:50:56.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:56 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi152"}]': finished 2024-03-29T04:50:56.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:56 smithi142 ceph-mon[112475]: pgmap v31: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:56.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:56 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:56.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:56 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:56.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:56 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:50:56.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:56 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:50:56.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:56 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:50:56.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:56 smithi152 ceph-mon[88186]: Upgrade: Setting container_image for all mon 2024-03-29T04:50:56.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:56 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:56.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:56 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi142"}]: dispatch 2024-03-29T04:50:56.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:56 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi142"}]': finished 2024-03-29T04:50:56.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:56 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi152"}]: dispatch 2024-03-29T04:50:56.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:56 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi152"}]': finished 2024-03-29T04:50:56.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:56 smithi152 ceph-mon[88186]: pgmap v31: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:50:58.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:58 smithi142 ceph-mon[112475]: Upgrade: Updating crash.smithi142 (1/2) 2024-03-29T04:50:58.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:58.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:50:58.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:58.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:58 smithi142 ceph-mon[112475]: Deploying daemon crash.smithi142 on smithi142 2024-03-29T04:50:58.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:58 smithi142 ceph-mon[112475]: pgmap v32: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:58.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:58 smithi152 ceph-mon[88186]: Upgrade: Updating crash.smithi142 (1/2) 2024-03-29T04:50:58.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:50:58.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi142", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:50:58.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:50:58.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:58 smithi152 ceph-mon[88186]: Deploying daemon crash.smithi142 on smithi142 2024-03-29T04:50:58.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:58 smithi152 ceph-mon[88186]: pgmap v32: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:50:59.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:50:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:51:00.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:50:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:51:00.871 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:00 smithi142 ceph-mon[112475]: pgmap v33: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:01.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:00 smithi152 ceph-mon[88186]: pgmap v33: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:02.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:01 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:02.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:01 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:02.294 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:01 smithi152 ceph-mon[88186]: pgmap v34: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:02.327 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:01 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:02.327 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:01 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:02.327 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:01 smithi142 ceph-mon[112475]: pgmap v34: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:03.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:03 smithi152 ceph-mon[88186]: Upgrade: Updating crash.smithi152 (2/2) 2024-03-29T04:51:03.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:03 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:03.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:03 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi152", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:51:03.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:03 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:51:03.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:03 smithi152 ceph-mon[88186]: Deploying daemon crash.smithi152 on smithi152 2024-03-29T04:51:03.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:03 smithi152 ceph-mon[88186]: pgmap v35: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:04.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:03 smithi142 ceph-mon[112475]: Upgrade: Updating crash.smithi152 (2/2) 2024-03-29T04:51:04.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:03 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:04.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:03 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi152", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T04:51:04.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:03 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:51:04.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:03 smithi142 ceph-mon[112475]: Deploying daemon crash.smithi152 on smithi152 2024-03-29T04:51:04.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:03 smithi142 ceph-mon[112475]: pgmap v35: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:06.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:06 smithi142 ceph-mon[112475]: pgmap v36: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:06.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:06 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:06.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:06 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:06.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:06 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:51:07.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:06 smithi152 ceph-mon[88186]: pgmap v36: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:07.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:06 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:07.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:06 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:07.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:06 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:51:08.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:08 smithi142 ceph-mon[112475]: pgmap v37: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:08.900 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:08 smithi152 ceph-mon[88186]: pgmap v37: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:10.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:10.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:10.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:10 smithi152 ceph-mon[88186]: pgmap v38: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:10.827 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:10.827 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:10.827 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:10 smithi142 ceph-mon[112475]: pgmap v38: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:11.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:11 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:11.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:11 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:11.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:11 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:11.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:11 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:12.672 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:12 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:12.672 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:12 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:12.672 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:12 smithi152 ceph-mon[88186]: pgmap v39: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:12.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:12 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:12.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:12 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:12.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:12 smithi142 ceph-mon[112475]: pgmap v39: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:13.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:13 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:13.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:13 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:13.751 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:13 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:13.752 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:13 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:14.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:14 smithi142 ceph-mon[112475]: pgmap v40: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:14.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:14.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:15.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:14 smithi152 ceph-mon[88186]: pgmap v40: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:15.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:15.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:15.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:51:16.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:51:16.838 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: pgmap v41: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:16.838 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:16.838 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:16.838 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:51:16.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:51:16.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:16.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:51:16.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:16.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:16.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:16.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:16.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi142"}]: dispatch 2024-03-29T04:51:16.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi142"}]': finished 2024-03-29T04:51:16.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi152"}]: dispatch 2024-03-29T04:51:16.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi152"}]': finished 2024-03-29T04:51:16.839 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-29T04:51:17.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: pgmap v41: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:17.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi142"}]: dispatch 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi142"}]': finished 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi152"}]: dispatch 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi152"}]': finished 2024-03-29T04:51:17.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-29T04:51:17.743 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:17 smithi142 ceph-mon[112475]: Upgrade: Setting container_image for all crash 2024-03-29T04:51:17.743 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:17 smithi142 ceph-mon[112475]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-29T04:51:17.744 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:17 smithi142 ceph-mon[112475]: Upgrade: osd.0 is safe to restart 2024-03-29T04:51:18.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:17 smithi152 ceph-mon[88186]: Upgrade: Setting container_image for all crash 2024-03-29T04:51:18.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:17 smithi152 ceph-mon[88186]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-29T04:51:18.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:17 smithi152 ceph-mon[88186]: Upgrade: osd.0 is safe to restart 2024-03-29T04:51:18.605 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:18 smithi142 ceph-mon[112475]: pgmap v42: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:18.605 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:18 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:18.606 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:18 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-29T04:51:18.606 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:18 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:51:18.852 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:18 smithi152 ceph-mon[88186]: pgmap v42: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:18.852 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:18 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:18.852 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:18 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-29T04:51:18.852 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:18 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:51:19.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:19 smithi142 ceph-mon[112475]: Upgrade: Updating osd.0 2024-03-29T04:51:19.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:19 smithi142 ceph-mon[112475]: Deploying daemon osd.0 on smithi142 2024-03-29T04:51:20.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:19 smithi152 ceph-mon[88186]: Upgrade: Updating osd.0 2024-03-29T04:51:20.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:19 smithi152 ceph-mon[88186]: Deploying daemon osd.0 on smithi142 2024-03-29T04:51:20.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:20 smithi152 ceph-mon[88186]: pgmap v43: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:20.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:20 smithi142 ceph-mon[112475]: pgmap v43: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:21.181 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:20 smithi142 systemd[1]: Stopping Ceph osd.0 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:51:21.509 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:21 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0[48057]: 2024-03-29T04:51:21.176+0000 7f70df63d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T04:51:21.510 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:21 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0[48057]: 2024-03-29T04:51:21.176+0000 7f70df63d700 -1 osd.0 51 *** Got signal Terminated *** 2024-03-29T04:51:21.510 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:21 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0[48057]: 2024-03-29T04:51:21.176+0000 7f70df63d700 -1 osd.0 51 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T04:51:21.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:21 smithi142 ceph-mon[112475]: osd.0 marked itself down and dead 2024-03-29T04:51:22.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:21 smithi152 ceph-mon[88186]: osd.0 marked itself down and dead 2024-03-29T04:51:22.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.073+0000 7f874ea23700 1 -- 172.21.15.142:0/4105399238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87480ff530 msgr2=0x7f87480ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:22.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.073+0000 7f874ea23700 1 --2- 172.21.15.142:0/4105399238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87480ff530 0x7f87480ff910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8738009910 tx=0x7f8738009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:22.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.073+0000 7f874ea23700 1 -- 172.21.15.142:0/4105399238 shutdown_connections 2024-03-29T04:51:22.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.073+0000 7f874ea23700 1 --2- 172.21.15.142:0/4105399238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87480ffee0 0x7f8748105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.073+0000 7f874ea23700 1 --2- 172.21.15.142:0/4105399238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87480ff530 0x7f87480ff910 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.073+0000 7f874ea23700 1 -- 172.21.15.142:0/4105399238 >> 172.21.15.142:0/4105399238 conn(0x7f8748074df0 msgr2=0x7f8748075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:22.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.073+0000 7f874ea23700 1 -- 172.21.15.142:0/4105399238 shutdown_connections 2024-03-29T04:51:22.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.073+0000 7f874ea23700 1 -- 172.21.15.142:0/4105399238 wait complete. 2024-03-29T04:51:22.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.073+0000 7f874ea23700 1 Processor -- start 2024-03-29T04:51:22.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.073+0000 7f874ea23700 1 -- start start 2024-03-29T04:51:22.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.074+0000 7f874ea23700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87480ff530 0x7f874819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:22.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.074+0000 7f874ea23700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87480ffee0 0x7f874819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:22.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.074+0000 7f874ea23700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874819d160 con 0x7f87480ffee0 2024-03-29T04:51:22.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.074+0000 7f874ea23700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8748196610 con 0x7f87480ff530 2024-03-29T04:51:22.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.074+0000 7f87477fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87480ffee0 0x7f874819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:22.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.074+0000 7f87477fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87480ffee0 0x7f874819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57148/0 (socket says 172.21.15.142:57148) 2024-03-29T04:51:22.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.074+0000 7f87477fe700 1 -- 172.21.15.142:0/4130451416 learned_addr learned my addr 172.21.15.142:0/4130451416 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:22.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.074+0000 7f8747fff700 1 --2- 172.21.15.142:0/4130451416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87480ff530 0x7f874819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:22.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.074+0000 7f87477fe700 1 -- 172.21.15.142:0/4130451416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87480ff530 msgr2=0x7f874819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:22.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.074+0000 7f87477fe700 1 --2- 172.21.15.142:0/4130451416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87480ff530 0x7f874819c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.074+0000 7f87477fe700 1 -- 172.21.15.142:0/4130451416 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87380095f0 con 0x7f87480ffee0 2024-03-29T04:51:22.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.075+0000 7f87477fe700 1 --2- 172.21.15.142:0/4130451416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87480ffee0 0x7f874819cad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f873c00d900 tx=0x7f873c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:22.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.075+0000 7f87457fa700 1 -- 172.21.15.142:0/4130451416 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f873c009900 con 0x7f87480ffee0 2024-03-29T04:51:22.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.075+0000 7f87457fa700 1 -- 172.21.15.142:0/4130451416 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f873c010460 con 0x7f87480ffee0 2024-03-29T04:51:22.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.075+0000 7f874ea23700 1 -- 172.21.15.142:0/4130451416 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87481968f0 con 0x7f87480ffee0 2024-03-29T04:51:22.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.075+0000 7f87457fa700 1 -- 172.21.15.142:0/4130451416 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f873c00f5f0 con 0x7f87480ffee0 2024-03-29T04:51:22.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.075+0000 7f874ea23700 1 -- 172.21.15.142:0/4130451416 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8748196e40 con 0x7f87480ffee0 2024-03-29T04:51:22.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.076+0000 7f874ea23700 1 -- 172.21.15.142:0/4130451416 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87481091e0 con 0x7f87480ffee0 2024-03-29T04:51:22.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.076+0000 7f87457fa700 1 -- 172.21.15.142:0/4130451416 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7f873c0105d0 con 0x7f87480ffee0 2024-03-29T04:51:22.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.077+0000 7f87457fa700 1 --2- 172.21.15.142:0/4130451416 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f873006dea0 0x7f8730070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:22.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.077+0000 7f87457fa700 1 -- 172.21.15.142:0/4130451416 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f873c08dbf0 con 0x7f87480ffee0 2024-03-29T04:51:22.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.077+0000 7f8747fff700 1 --2- 172.21.15.142:0/4130451416 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f873006dea0 0x7f8730070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:22.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.077+0000 7f8747fff700 1 --2- 172.21.15.142:0/4130451416 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f873006dea0 0x7f8730070360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f873800b5c0 tx=0x7f873801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:22.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.082+0000 7f87457fa700 1 -- 172.21.15.142:0/4130451416 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f873c05aaf0 con 0x7f87480ffee0 2024-03-29T04:51:22.265 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:21 smithi142 podman[123603]: 2024-03-29 04:51:21.97419945 +0000 UTC m=+0.920667569 container died 712d71175343015924f587ba55f05512d2a95ce39bf0ab1b4fdef79e627b47bf (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-03-29T04:51:22.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.299+0000 7f874ea23700 1 -- 172.21.15.142:0/4130451416 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8748197060 con 0x7f873006dea0 2024-03-29T04:51:22.304 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.300+0000 7f87457fa700 1 -- 172.21.15.142:0/4130451416 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8748197060 con 0x7f873006dea0 2024-03-29T04:51:22.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.302+0000 7f874ea23700 1 -- 172.21.15.142:0/4130451416 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f873006dea0 msgr2=0x7f8730070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:22.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.302+0000 7f874ea23700 1 --2- 172.21.15.142:0/4130451416 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f873006dea0 0x7f8730070360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f873800b5c0 tx=0x7f873801a040 comp rx=0 tx=0).stop 2024-03-29T04:51:22.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.302+0000 7f874ea23700 1 -- 172.21.15.142:0/4130451416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87480ffee0 msgr2=0x7f874819cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:22.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.302+0000 7f874ea23700 1 --2- 172.21.15.142:0/4130451416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87480ffee0 0x7f874819cad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f873c00d900 tx=0x7f873c00dcc0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.303+0000 7f874ea23700 1 -- 172.21.15.142:0/4130451416 shutdown_connections 2024-03-29T04:51:22.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.303+0000 7f874ea23700 1 --2- 172.21.15.142:0/4130451416 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f873006dea0 0x7f8730070360 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.303+0000 7f874ea23700 1 --2- 172.21.15.142:0/4130451416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87480ff530 0x7f874819c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.303+0000 7f874ea23700 1 --2- 172.21.15.142:0/4130451416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87480ffee0 0x7f874819cad0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.303+0000 7f874ea23700 1 -- 172.21.15.142:0/4130451416 >> 172.21.15.142:0/4130451416 conn(0x7f8748074df0 msgr2=0x7f87480feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:22.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.303+0000 7f874ea23700 1 -- 172.21.15.142:0/4130451416 shutdown_connections 2024-03-29T04:51:22.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.303+0000 7f874ea23700 1 -- 172.21.15.142:0/4130451416 wait complete. 2024-03-29T04:51:22.318 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:51:22.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.408+0000 7f58c0212700 1 -- 172.21.15.142:0/2904196596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b8100dc0 msgr2=0x7f58b81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:22.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.408+0000 7f58c0212700 1 --2- 172.21.15.142:0/2904196596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b8100dc0 0x7f58b81011a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009910 tx=0x7f58b4009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:22.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.408+0000 7f58c0212700 1 -- 172.21.15.142:0/2904196596 shutdown_connections 2024-03-29T04:51:22.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.408+0000 7f58c0212700 1 --2- 172.21.15.142:0/2904196596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8101770 0x7f58b81057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.408+0000 7f58c0212700 1 --2- 172.21.15.142:0/2904196596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b8100dc0 0x7f58b81011a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.408+0000 7f58c0212700 1 -- 172.21.15.142:0/2904196596 >> 172.21.15.142:0/2904196596 conn(0x7f58b80fc650 msgr2=0x7f58b80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.409+0000 7f58c0212700 1 -- 172.21.15.142:0/2904196596 shutdown_connections 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.409+0000 7f58c0212700 1 -- 172.21.15.142:0/2904196596 wait complete. 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.409+0000 7f58c0212700 1 Processor -- start 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.409+0000 7f58c0212700 1 -- start start 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.409+0000 7f58c0212700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b8100dc0 0x7f58b8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.409+0000 7f58c0212700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8101770 0x7f58b80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.409+0000 7f58c0212700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8073b10 con 0x7f58b8100dc0 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.409+0000 7f58c0212700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8073c50 con 0x7f58b8101770 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.409+0000 7f58bdfae700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b8100dc0 0x7f58b8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.409+0000 7f58bdfae700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b8100dc0 0x7f58b8073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57168/0 (socket says 172.21.15.142:57168) 2024-03-29T04:51:22.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.409+0000 7f58bdfae700 1 -- 172.21.15.142:0/2800788805 learned_addr learned my addr 172.21.15.142:0/2800788805 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:22.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.410+0000 7f58bd7ad700 1 --2- 172.21.15.142:0/2800788805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8101770 0x7f58b80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:22.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.410+0000 7f58bdfae700 1 -- 172.21.15.142:0/2800788805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8101770 msgr2=0x7f58b80735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:22.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.410+0000 7f58bdfae700 1 --2- 172.21.15.142:0/2800788805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8101770 0x7f58b80735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.410+0000 7f58bdfae700 1 -- 172.21.15.142:0/2800788805 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40095f0 con 0x7f58b8100dc0 2024-03-29T04:51:22.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.410+0000 7f58bdfae700 1 --2- 172.21.15.142:0/2800788805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b8100dc0 0x7f58b8073090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f58b40092e0 tx=0x7f58b400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:22.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.410+0000 7f58aeffd700 1 -- 172.21.15.142:0/2800788805 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58b401d070 con 0x7f58b8100dc0 2024-03-29T04:51:22.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.410+0000 7f58aeffd700 1 -- 172.21.15.142:0/2800788805 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58b4005e00 con 0x7f58b8100dc0 2024-03-29T04:51:22.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.410+0000 7f58c0212700 1 -- 172.21.15.142:0/2800788805 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b8073ed0 con 0x7f58b8100dc0 2024-03-29T04:51:22.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.410+0000 7f58aeffd700 1 -- 172.21.15.142:0/2800788805 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58b4017400 con 0x7f58b8100dc0 2024-03-29T04:51:22.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.410+0000 7f58c0212700 1 -- 172.21.15.142:0/2800788805 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b81a2c70 con 0x7f58b8100dc0 2024-03-29T04:51:22.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.411+0000 7f58c0212700 1 -- 172.21.15.142:0/2800788805 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b8109160 con 0x7f58b8100dc0 2024-03-29T04:51:22.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.412+0000 7f58aeffd700 1 -- 172.21.15.142:0/2800788805 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7f58b400f6f0 con 0x7f58b8100dc0 2024-03-29T04:51:22.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.412+0000 7f58aeffd700 1 --2- 172.21.15.142:0/2800788805 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f58a406de90 0x7f58a4070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:22.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.412+0000 7f58aeffd700 1 -- 172.21.15.142:0/2800788805 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f58b408f200 con 0x7f58b8100dc0 2024-03-29T04:51:22.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.412+0000 7f58bd7ad700 1 --2- 172.21.15.142:0/2800788805 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f58a406de90 0x7f58a4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:22.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.413+0000 7f58bd7ad700 1 --2- 172.21.15.142:0/2800788805 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f58a406de90 0x7f58a4070350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f58b8074880 tx=0x7f58a800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:22.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.417+0000 7f58aeffd700 1 -- 172.21.15.142:0/2800788805 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f58b40585d0 con 0x7f58b8100dc0 2024-03-29T04:51:22.534 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:22 smithi142 podman[123603]: 2024-03-29 04:51:22.478153277 +0000 UTC m=+1.424621372 container cleanup 712d71175343015924f587ba55f05512d2a95ce39bf0ab1b4fdef79e627b47bf (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=HEAD, GIT_BRANCH=HEAD) 2024-03-29T04:51:22.534 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:22 smithi142 bash[123603]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0 2024-03-29T04:51:22.694 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.689+0000 7f58c0212700 1 -- 172.21.15.142:0/2800788805 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58b802cd30 con 0x7f58a406de90 2024-03-29T04:51:22.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.691+0000 7f58aeffd700 1 -- 172.21.15.142:0/2800788805 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f58b802cd30 con 0x7f58a406de90 2024-03-29T04:51:22.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.693+0000 7f58c0212700 1 -- 172.21.15.142:0/2800788805 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f58a406de90 msgr2=0x7f58a4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:22.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.693+0000 7f58c0212700 1 --2- 172.21.15.142:0/2800788805 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f58a406de90 0x7f58a4070350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f58b8074880 tx=0x7f58a800b410 comp rx=0 tx=0).stop 2024-03-29T04:51:22.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.693+0000 7f58c0212700 1 -- 172.21.15.142:0/2800788805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b8100dc0 msgr2=0x7f58b8073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:22.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.693+0000 7f58c0212700 1 --2- 172.21.15.142:0/2800788805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b8100dc0 0x7f58b8073090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f58b40092e0 tx=0x7f58b400bd70 comp rx=0 tx=0).stop 2024-03-29T04:51:22.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.693+0000 7f58c0212700 1 -- 172.21.15.142:0/2800788805 shutdown_connections 2024-03-29T04:51:22.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.693+0000 7f58c0212700 1 --2- 172.21.15.142:0/2800788805 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f58a406de90 0x7f58a4070350 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.693+0000 7f58c0212700 1 --2- 172.21.15.142:0/2800788805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58b8100dc0 0x7f58b8073090 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.693+0000 7f58c0212700 1 --2- 172.21.15.142:0/2800788805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8101770 0x7f58b80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.693+0000 7f58c0212700 1 -- 172.21.15.142:0/2800788805 >> 172.21.15.142:0/2800788805 conn(0x7f58b80fc650 msgr2=0x7f58b80fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:22.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.693+0000 7f58c0212700 1 -- 172.21.15.142:0/2800788805 shutdown_connections 2024-03-29T04:51:22.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.693+0000 7f58c0212700 1 -- 172.21.15.142:0/2800788805 wait complete. 2024-03-29T04:51:22.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.800+0000 7f2b0f8db700 1 -- 172.21.15.142:0/3193381414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b080ffa40 msgr2=0x7f2b080ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:22.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.800+0000 7f2b0f8db700 1 --2- 172.21.15.142:0/3193381414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b080ffa40 0x7f2b080ffec0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009910 tx=0x7f2af8009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:22.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.800+0000 7f2b0f8db700 1 -- 172.21.15.142:0/3193381414 shutdown_connections 2024-03-29T04:51:22.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.800+0000 7f2b0f8db700 1 --2- 172.21.15.142:0/3193381414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b080ffa40 0x7f2b080ffec0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.800+0000 7f2b0f8db700 1 --2- 172.21.15.142:0/3193381414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b08105a40 0x7f2b08105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.800+0000 7f2b0f8db700 1 -- 172.21.15.142:0/3193381414 >> 172.21.15.142:0/3193381414 conn(0x7f2b080fb570 msgr2=0x7f2b080fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:22.805 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:22 smithi142 podman[123622]: 2024-03-29 04:51:22.755374254 +0000 UTC m=+0.780360295 container remove 712d71175343015924f587ba55f05512d2a95ce39bf0ab1b4fdef79e627b47bf (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1) 2024-03-29T04:51:22.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:22 smithi142 ceph-mon[112475]: pgmap v44: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:22.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:22 smithi142 ceph-mon[112475]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T04:51:22.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:22 smithi142 ceph-mon[112475]: osdmap e52: 6 total, 5 up, 6 in 2024-03-29T04:51:22.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.800+0000 7f2b0f8db700 1 -- 172.21.15.142:0/3193381414 shutdown_connections 2024-03-29T04:51:22.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.800+0000 7f2b0f8db700 1 -- 172.21.15.142:0/3193381414 wait complete. 2024-03-29T04:51:22.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.801+0000 7f2b0f8db700 1 Processor -- start 2024-03-29T04:51:22.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.801+0000 7f2b0f8db700 1 -- start start 2024-03-29T04:51:22.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.801+0000 7f2b0f8db700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b080ffa40 0x7f2b08197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:22.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.801+0000 7f2b0f8db700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b08105a40 0x7f2b08197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:22.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.801+0000 7f2b0f8db700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b081980f0 con 0x7f2b080ffa40 2024-03-29T04:51:22.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.801+0000 7f2b0f8db700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b08198230 con 0x7f2b08105a40 2024-03-29T04:51:22.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.801+0000 7f2b0d677700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b080ffa40 0x7f2b08197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:22.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.801+0000 7f2b0d677700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b080ffa40 0x7f2b08197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57184/0 (socket says 172.21.15.142:57184) 2024-03-29T04:51:22.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.801+0000 7f2b0d677700 1 -- 172.21.15.142:0/4091906081 learned_addr learned my addr 172.21.15.142:0/4091906081 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:22.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.801+0000 7f2b0ce76700 1 --2- 172.21.15.142:0/4091906081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b08105a40 0x7f2b08197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:22.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.802+0000 7f2b0d677700 1 -- 172.21.15.142:0/4091906081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b08105a40 msgr2=0x7f2b08197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:22.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.802+0000 7f2b0d677700 1 --2- 172.21.15.142:0/4091906081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b08105a40 0x7f2b08197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:22.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.802+0000 7f2b0d677700 1 -- 172.21.15.142:0/4091906081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80095f0 con 0x7f2b080ffa40 2024-03-29T04:51:22.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.802+0000 7f2b0d677700 1 --2- 172.21.15.142:0/4091906081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b080ffa40 0x7f2b08197590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400c8a0 tx=0x7f2b0400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:22.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.802+0000 7f2afe7fc700 1 -- 172.21.15.142:0/4091906081 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b040078e0 con 0x7f2b080ffa40 2024-03-29T04:51:22.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.802+0000 7f2afe7fc700 1 -- 172.21.15.142:0/4091906081 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b0400f450 con 0x7f2b080ffa40 2024-03-29T04:51:22.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.802+0000 7f2b0f8db700 1 -- 172.21.15.142:0/4091906081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b081a0600 con 0x7f2b080ffa40 2024-03-29T04:51:22.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.802+0000 7f2afe7fc700 1 -- 172.21.15.142:0/4091906081 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b0400e5e0 con 0x7f2b080ffa40 2024-03-29T04:51:22.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.802+0000 7f2b0f8db700 1 -- 172.21.15.142:0/4091906081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b081a0ac0 con 0x7f2b080ffa40 2024-03-29T04:51:22.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.803+0000 7f2b0f8db700 1 -- 172.21.15.142:0/4091906081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b08107fd0 con 0x7f2b080ffa40 2024-03-29T04:51:22.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.804+0000 7f2afe7fc700 1 -- 172.21.15.142:0/4091906081 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7f2b0400f5c0 con 0x7f2b080ffa40 2024-03-29T04:51:22.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.804+0000 7f2afe7fc700 1 --2- 172.21.15.142:0/4091906081 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2af406de90 0x7f2af4070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:22.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.804+0000 7f2afe7fc700 1 -- 172.21.15.142:0/4091906081 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2b0408dc40 con 0x7f2b080ffa40 2024-03-29T04:51:22.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.804+0000 7f2b0ce76700 1 --2- 172.21.15.142:0/4091906081 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2af406de90 0x7f2af4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:22.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.805+0000 7f2b0ce76700 1 --2- 172.21.15.142:0/4091906081 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2af406de90 0x7f2af4070350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2af8006010 tx=0x7f2af801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:22.813 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:22.809+0000 7f2afe7fc700 1 -- 172.21.15.142:0/4091906081 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2b04057090 con 0x7f2b080ffa40 2024-03-29T04:51:23.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:22 smithi152 ceph-mon[88186]: pgmap v44: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:23.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:22 smithi152 ceph-mon[88186]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T04:51:23.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:22 smithi152 ceph-mon[88186]: osdmap e52: 6 total, 5 up, 6 in 2024-03-29T04:51:23.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.047+0000 7f2b0f8db700 1 -- 172.21.15.142:0/4091906081 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b0802cd80 con 0x7f2af406de90 2024-03-29T04:51:23.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.058+0000 7f2afe7fc700 1 -- 172.21.15.142:0/4091906081 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2b0802cd80 con 0x7f2af406de90 2024-03-29T04:51:23.062 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:51:23.062 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (3m) 11s ago 14m 26.5M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:51:23.062 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (15m) 11s ago 14m 8270k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:51:23.062 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (13m) 13s ago 13m 8182k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:51:23.062 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (22s) 11s ago 14m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:51:23.062 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (17s) 13s ago 13m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:51:23.063 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (2m) 11s ago 14m 76.9M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:51:23.063 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (9m) 11s ago 9m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:51:23.063 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (9m) 11s ago 9m 63.2M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:51:23.063 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (9m) 13s ago 8m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:51:23.063 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (9m) 13s ago 9m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:51:23.063 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (4m) 11s ago 16m 498M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:51:23.063 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (4m) 13s ago 13m 449M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:51:23.063 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (92s) 11s ago 16m 44.2M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:51:23.063 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (67s) 13s ago 13m 41.1M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:51:23.063 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (3m) 11s ago 14m 15.0M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:51:23.063 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (3m) 13s ago 13m 16.0M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:51:23.063 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (12m) 11s ago 12m 220M 1216M 18.2.1 5be31c24972a 712d71175343 2024-03-29T04:51:23.064 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (12m) 11s ago 12m 241M 1216M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:51:23.064 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (12m) 11s ago 11m 198M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:51:23.064 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (11m) 13s ago 11m 276M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:51:23.064 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (11m) 13s ago 11m 236M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:51:23.064 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (10m) 13s ago 10m 225M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:51:23.064 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (3m) 11s ago 14m 63.3M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:51:23.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.060+0000 7f2b0f8db700 1 -- 172.21.15.142:0/4091906081 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2af406de90 msgr2=0x7f2af4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:23.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.060+0000 7f2b0f8db700 1 --2- 172.21.15.142:0/4091906081 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2af406de90 0x7f2af4070350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2af8006010 tx=0x7f2af801a040 comp rx=0 tx=0).stop 2024-03-29T04:51:23.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.060+0000 7f2b0f8db700 1 -- 172.21.15.142:0/4091906081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b080ffa40 msgr2=0x7f2b08197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:23.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.060+0000 7f2b0f8db700 1 --2- 172.21.15.142:0/4091906081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b080ffa40 0x7f2b08197590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400c8a0 tx=0x7f2b0400cbb0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.060+0000 7f2b0f8db700 1 -- 172.21.15.142:0/4091906081 shutdown_connections 2024-03-29T04:51:23.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.060+0000 7f2b0f8db700 1 --2- 172.21.15.142:0/4091906081 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2af406de90 0x7f2af4070350 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.060+0000 7f2b0f8db700 1 --2- 172.21.15.142:0/4091906081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b080ffa40 0x7f2b08197590 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.060+0000 7f2b0f8db700 1 --2- 172.21.15.142:0/4091906081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b08105a40 0x7f2b08197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.060+0000 7f2b0f8db700 1 -- 172.21.15.142:0/4091906081 >> 172.21.15.142:0/4091906081 conn(0x7f2b080fb570 msgr2=0x7f2b080fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:23.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.060+0000 7f2b0f8db700 1 -- 172.21.15.142:0/4091906081 shutdown_connections 2024-03-29T04:51:23.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.060+0000 7f2b0f8db700 1 -- 172.21.15.142:0/4091906081 wait complete. 2024-03-29T04:51:23.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.165+0000 7f3940f4a700 1 -- 172.21.15.142:0/1845679391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f393c105a40 msgr2=0x7f393c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:23.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.165+0000 7f3940f4a700 1 --2- 172.21.15.142:0/1845679391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f393c105a40 0x7f393c105e20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f392c009910 tx=0x7f392c009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:23.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.165+0000 7f3940f4a700 1 -- 172.21.15.142:0/1845679391 shutdown_connections 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.165+0000 7f3940f4a700 1 --2- 172.21.15.142:0/1845679391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f393c0ffa40 0x7f393c0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.165+0000 7f3940f4a700 1 --2- 172.21.15.142:0/1845679391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f393c105a40 0x7f393c105e20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.165+0000 7f3940f4a700 1 -- 172.21.15.142:0/1845679391 >> 172.21.15.142:0/1845679391 conn(0x7f393c0fb570 msgr2=0x7f393c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.165+0000 7f3940f4a700 1 -- 172.21.15.142:0/1845679391 shutdown_connections 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.165+0000 7f3940f4a700 1 -- 172.21.15.142:0/1845679391 wait complete. 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3940f4a700 1 Processor -- start 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3940f4a700 1 -- start start 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3940f4a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f393c0ffa40 0x7f393c0fbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3940f4a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f393c105a40 0x7f393c0fc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3940f4a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0fc8b0 con 0x7f393c105a40 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3940f4a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0fc9f0 con 0x7f393c0ffa40 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3939d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f393c105a40 0x7f393c0fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3939d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f393c105a40 0x7f393c0fc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57202/0 (socket says 172.21.15.142:57202) 2024-03-29T04:51:23.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3939d9b700 1 -- 172.21.15.142:0/749975354 learned_addr learned my addr 172.21.15.142:0/749975354 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:23.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f393a59c700 1 --2- 172.21.15.142:0/749975354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f393c0ffa40 0x7f393c0fbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:23.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3939d9b700 1 -- 172.21.15.142:0/749975354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f393c0ffa40 msgr2=0x7f393c0fbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:23.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3939d9b700 1 --2- 172.21.15.142:0/749975354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f393c0ffa40 0x7f393c0fbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3939d9b700 1 -- 172.21.15.142:0/749975354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c0095f0 con 0x7f393c105a40 2024-03-29T04:51:23.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.166+0000 7f3939d9b700 1 --2- 172.21.15.142:0/749975354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f393c105a40 0x7f393c0fc370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f393000c930 tx=0x7f393000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:23.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.167+0000 7f392b7fe700 1 -- 172.21.15.142:0/749975354 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3930007ad0 con 0x7f393c105a40 2024-03-29T04:51:23.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.167+0000 7f392b7fe700 1 -- 172.21.15.142:0/749975354 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f393000ce80 con 0x7f393c105a40 2024-03-29T04:51:23.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.167+0000 7f3940f4a700 1 -- 172.21.15.142:0/749975354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393c1a2960 con 0x7f393c105a40 2024-03-29T04:51:23.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.167+0000 7f392b7fe700 1 -- 172.21.15.142:0/749975354 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3930018630 con 0x7f393c105a40 2024-03-29T04:51:23.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.167+0000 7f3940f4a700 1 -- 172.21.15.142:0/749975354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c1a2cc0 con 0x7f393c105a40 2024-03-29T04:51:23.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.168+0000 7f3940f4a700 1 -- 172.21.15.142:0/749975354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393c109050 con 0x7f393c105a40 2024-03-29T04:51:23.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.168+0000 7f392b7fe700 1 -- 172.21.15.142:0/749975354 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7f393000f450 con 0x7f393c105a40 2024-03-29T04:51:23.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.169+0000 7f392b7fe700 1 --2- 172.21.15.142:0/749975354 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f392406dea0 0x7f3924070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:23.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.169+0000 7f392b7fe700 1 -- 172.21.15.142:0/749975354 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f393008dd10 con 0x7f393c105a40 2024-03-29T04:51:23.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.169+0000 7f393a59c700 1 --2- 172.21.15.142:0/749975354 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f392406dea0 0x7f3924070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:23.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.169+0000 7f393a59c700 1 --2- 172.21.15.142:0/749975354 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f392406dea0 0x7f3924070360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f392c006010 tx=0x7f392c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:23.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.173+0000 7f392b7fe700 1 -- 172.21.15.142:0/749975354 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3930057210 con 0x7f393c105a40 2024-03-29T04:51:23.339 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:23 smithi142 podman[123784]: 2024-03-29 04:51:22.995287566 +0000 UTC m=+0.018014660 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:51:23.339 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:23 smithi142 podman[123784]: 2024-03-29 04:51:23.112563334 +0000 UTC m=+0.135290411 container create 763c534c639d6cd0119f71a42607829828f0eb1f88f53f21810619d39ad0f7df (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True, RELEASE=squid-8da2362, org.label-schema.vendor=CentOS) 2024-03-29T04:51:23.339 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:23 smithi142 podman[123784]: 2024-03-29 04:51:23.334758128 +0000 UTC m=+0.357485217 container init 763c534c639d6cd0119f71a42607829828f0eb1f88f53f21810619d39ad0f7df (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8da2362, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, maintainer=Guillaume Abrioux ) 2024-03-29T04:51:23.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.461+0000 7f3940f4a700 1 -- 172.21.15.142:0/749975354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f393c0fcbb0 con 0x7f393c105a40 2024-03-29T04:51:23.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.462+0000 7f392b7fe700 1 -- 172.21.15.142:0/749975354 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+687 (secure 0 0 0) 0x7f393005a8d0 con 0x7f393c105a40 2024-03-29T04:51:23.466 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:51:23.466 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:51:23.466 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:51:23.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:51:23.466 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:51:23.466 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:51:23.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:51:23.467 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:51:23.467 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5 2024-03-29T04:51:23.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:51:23.467 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:51:23.467 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:51:23.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:51:23.467 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:51:23.467 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-03-29T04:51:23.467 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T04:51:23.467 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:51:23.467 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:51:23.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.465+0000 7f3940f4a700 1 -- 172.21.15.142:0/749975354 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f392406dea0 msgr2=0x7f3924070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:23.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.465+0000 7f3940f4a700 1 --2- 172.21.15.142:0/749975354 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f392406dea0 0x7f3924070360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f392c006010 tx=0x7f392c01a040 comp rx=0 tx=0).stop 2024-03-29T04:51:23.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.465+0000 7f3940f4a700 1 -- 172.21.15.142:0/749975354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f393c105a40 msgr2=0x7f393c0fc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:23.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.465+0000 7f3940f4a700 1 --2- 172.21.15.142:0/749975354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f393c105a40 0x7f393c0fc370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f393000c930 tx=0x7f393000cc40 comp rx=0 tx=0).stop 2024-03-29T04:51:23.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.466+0000 7f3940f4a700 1 -- 172.21.15.142:0/749975354 shutdown_connections 2024-03-29T04:51:23.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.466+0000 7f3940f4a700 1 --2- 172.21.15.142:0/749975354 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f392406dea0 0x7f3924070360 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.466+0000 7f3940f4a700 1 --2- 172.21.15.142:0/749975354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f393c0ffa40 0x7f393c0fbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.466+0000 7f3940f4a700 1 --2- 172.21.15.142:0/749975354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f393c105a40 0x7f393c0fc370 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.466+0000 7f3940f4a700 1 -- 172.21.15.142:0/749975354 >> 172.21.15.142:0/749975354 conn(0x7f393c0fb570 msgr2=0x7f393c068f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:23.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.466+0000 7f3940f4a700 1 -- 172.21.15.142:0/749975354 shutdown_connections 2024-03-29T04:51:23.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.466+0000 7f3940f4a700 1 -- 172.21.15.142:0/749975354 wait complete. 2024-03-29T04:51:23.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.576+0000 7fd7a302f700 1 -- 172.21.15.142:0/2333320987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd79c105a70 msgr2=0x7fd79c105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:23.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.576+0000 7fd7a302f700 1 --2- 172.21.15.142:0/2333320987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd79c105a70 0x7fd79c105e50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd790009910 tx=0x7fd790009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:23.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.576+0000 7fd7a302f700 1 -- 172.21.15.142:0/2333320987 shutdown_connections 2024-03-29T04:51:23.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.576+0000 7fd7a302f700 1 --2- 172.21.15.142:0/2333320987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd79c0ffa70 0x7fd79c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.576+0000 7fd7a302f700 1 --2- 172.21.15.142:0/2333320987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd79c105a70 0x7fd79c105e50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.576+0000 7fd7a302f700 1 -- 172.21.15.142:0/2333320987 >> 172.21.15.142:0/2333320987 conn(0x7fd79c0fb560 msgr2=0x7fd79c0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:23.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.576+0000 7fd7a302f700 1 -- 172.21.15.142:0/2333320987 shutdown_connections 2024-03-29T04:51:23.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.576+0000 7fd7a302f700 1 -- 172.21.15.142:0/2333320987 wait complete. 2024-03-29T04:51:23.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.576+0000 7fd7a302f700 1 Processor -- start 2024-03-29T04:51:23.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.576+0000 7fd7a302f700 1 -- start start 2024-03-29T04:51:23.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd7a302f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd79c0ffa70 0x7fd79c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:23.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd7a302f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd79c105a70 0x7fd79c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:23.582 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd7a302f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c19f2e0 con 0x7fd79c0ffa70 2024-03-29T04:51:23.582 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd7a302f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c198790 con 0x7fd79c105a70 2024-03-29T04:51:23.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd7a0dcb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd79c0ffa70 0x7fd79c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:23.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd7a0dcb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd79c0ffa70 0x7fd79c19e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57216/0 (socket says 172.21.15.142:57216) 2024-03-29T04:51:23.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd7a0dcb700 1 -- 172.21.15.142:0/1359219729 learned_addr learned my addr 172.21.15.142:0/1359219729 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:23.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd79bfff700 1 --2- 172.21.15.142:0/1359219729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd79c105a70 0x7fd79c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:23.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd7a0dcb700 1 -- 172.21.15.142:0/1359219729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd79c105a70 msgr2=0x7fd79c19ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:23.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd7a0dcb700 1 --2- 172.21.15.142:0/1359219729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd79c105a70 0x7fd79c19ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd7a0dcb700 1 -- 172.21.15.142:0/1359219729 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7900095f0 con 0x7fd79c0ffa70 2024-03-29T04:51:23.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd7a0dcb700 1 --2- 172.21.15.142:0/1359219729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd79c0ffa70 0x7fd79c19e710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd7900092e0 tx=0x7fd79000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:23.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd799ffb700 1 -- 172.21.15.142:0/1359219729 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd79001d070 con 0x7fd79c0ffa70 2024-03-29T04:51:23.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd799ffb700 1 -- 172.21.15.142:0/1359219729 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd790005e00 con 0x7fd79c0ffa70 2024-03-29T04:51:23.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.577+0000 7fd7a302f700 1 -- 172.21.15.142:0/1359219729 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79c198a10 con 0x7fd79c0ffa70 2024-03-29T04:51:23.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.578+0000 7fd799ffb700 1 -- 172.21.15.142:0/1359219729 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd790017400 con 0x7fd79c0ffa70 2024-03-29T04:51:23.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.578+0000 7fd7a302f700 1 -- 172.21.15.142:0/1359219729 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79c198f00 con 0x7fd79c0ffa70 2024-03-29T04:51:23.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.579+0000 7fd7a302f700 1 -- 172.21.15.142:0/1359219729 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79c04ea90 con 0x7fd79c0ffa70 2024-03-29T04:51:23.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.579+0000 7fd799ffb700 1 -- 172.21.15.142:0/1359219729 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7fd79000f6f0 con 0x7fd79c0ffa70 2024-03-29T04:51:23.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.580+0000 7fd799ffb700 1 --2- 172.21.15.142:0/1359219729 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd78c06de90 0x7fd78c070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:23.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.580+0000 7fd799ffb700 1 -- 172.21.15.142:0/1359219729 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd79008f1e0 con 0x7fd79c0ffa70 2024-03-29T04:51:23.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.580+0000 7fd79bfff700 1 --2- 172.21.15.142:0/1359219729 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd78c06de90 0x7fd78c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:23.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.580+0000 7fd79bfff700 1 --2- 172.21.15.142:0/1359219729 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd78c06de90 0x7fd78c070350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd79c199c00 tx=0x7fd78800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:23.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.584+0000 7fd799ffb700 1 -- 172.21.15.142:0/1359219729 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd79005c0e0 con 0x7fd79c0ffa70 2024-03-29T04:51:23.589 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:23 smithi142 podman[123784]: 2024-03-29 04:51:23.339671668 +0000 UTC m=+0.362398753 container start 763c534c639d6cd0119f71a42607829828f0eb1f88f53f21810619d39ad0f7df (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-29T04:51:23.589 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:23 smithi142 podman[123784]: 2024-03-29 04:51:23.459742459 +0000 UTC m=+0.482469552 container attach 763c534c639d6cd0119f71a42607829828f0eb1f88f53f21810619d39ad0f7df (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-29T04:51:23.844 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:23 smithi142 ceph-mon[112475]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:23.844 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:23 smithi142 ceph-mon[112475]: osdmap e53: 6 total, 5 up, 6 in 2024-03-29T04:51:23.844 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:23 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/749975354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:23.844 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:23 smithi142 podman[123784]: 2024-03-29 04:51:23.65421737 +0000 UTC m=+0.676944448 container died 763c534c639d6cd0119f71a42607829828f0eb1f88f53f21810619d39ad0f7df (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-8da2362, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-29T04:51:23.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.839+0000 7fd7a302f700 1 -- 172.21.15.142:0/1359219729 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd79c199ca0 con 0x7fd79c0ffa70 2024-03-29T04:51:23.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.840+0000 7fd799ffb700 1 -- 172.21.15.142:0/1359219729 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1906 (secure 0 0 0) 0x7fd79005bc70 con 0x7fd79c0ffa70 2024-03-29T04:51:23.845 INFO:teuthology.orchestra.run.smithi142.stdout:e21 2024-03-29T04:51:23.845 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:51:23.845 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:51:23.845 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:51:23.845 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:51:23.845 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:51:23.845 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 21 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:49:01.288637+0000 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:51:23.846 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 47 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:up {0=14494,1=24275} 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:51:23.847 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:51:23.848 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{0:14494} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:51:23.848 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{0:14506} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:51:23.848 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{1:24275} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:51:23.848 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{1:24287} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:51:23.848 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:51:23.848 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:51:23.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.842+0000 7fd7a302f700 1 -- 172.21.15.142:0/1359219729 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd78c06de90 msgr2=0x7fd78c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:23.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.842+0000 7fd7a302f700 1 --2- 172.21.15.142:0/1359219729 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd78c06de90 0x7fd78c070350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd79c199c00 tx=0x7fd78800b410 comp rx=0 tx=0).stop 2024-03-29T04:51:23.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.842+0000 7fd7a302f700 1 -- 172.21.15.142:0/1359219729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd79c0ffa70 msgr2=0x7fd79c19e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:23.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.842+0000 7fd7a302f700 1 --2- 172.21.15.142:0/1359219729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd79c0ffa70 0x7fd79c19e710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd7900092e0 tx=0x7fd79000bd70 comp rx=0 tx=0).stop 2024-03-29T04:51:23.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.842+0000 7fd7a302f700 1 -- 172.21.15.142:0/1359219729 shutdown_connections 2024-03-29T04:51:23.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.842+0000 7fd7a302f700 1 --2- 172.21.15.142:0/1359219729 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd78c06de90 0x7fd78c070350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.842+0000 7fd7a302f700 1 --2- 172.21.15.142:0/1359219729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd79c0ffa70 0x7fd79c19e710 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.842+0000 7fd7a302f700 1 --2- 172.21.15.142:0/1359219729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd79c105a70 0x7fd79c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.842+0000 7fd7a302f700 1 -- 172.21.15.142:0/1359219729 >> 172.21.15.142:0/1359219729 conn(0x7fd79c0fb560 msgr2=0x7fd79c068e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:23.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.842+0000 7fd7a302f700 1 -- 172.21.15.142:0/1359219729 shutdown_connections 2024-03-29T04:51:23.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.842+0000 7fd7a302f700 1 -- 172.21.15.142:0/1359219729 wait complete. 2024-03-29T04:51:23.849 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 21 2024-03-29T04:51:23.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.949+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/2212861207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4acc0ffbd0 msgr2=0x7f4acc100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:23.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.949+0000 7f4ad1e7e700 1 --2- 172.21.15.142:0/2212861207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4acc0ffbd0 0x7f4acc100050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009910 tx=0x7f4ac0009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:23.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.949+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/2212861207 shutdown_connections 2024-03-29T04:51:23.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.949+0000 7f4ad1e7e700 1 --2- 172.21.15.142:0/2212861207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4acc0ffbd0 0x7f4acc100050 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.949+0000 7f4ad1e7e700 1 --2- 172.21.15.142:0/2212861207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4acc105a40 0x7f4acc105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.949+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/2212861207 >> 172.21.15.142:0/2212861207 conn(0x7f4acc0755a0 msgr2=0x7f4acc0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:23.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.949+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/2212861207 shutdown_connections 2024-03-29T04:51:23.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.949+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/2212861207 wait complete. 2024-03-29T04:51:23.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.950+0000 7f4ad1e7e700 1 Processor -- start 2024-03-29T04:51:23.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.950+0000 7f4ad1e7e700 1 -- start start 2024-03-29T04:51:23.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.950+0000 7f4ad1e7e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4acc0ffbd0 0x7f4acc19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:23.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.950+0000 7f4ad1e7e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4acc105a40 0x7f4acc19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:23.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.950+0000 7f4ad1e7e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc19d0d0 con 0x7f4acc105a40 2024-03-29T04:51:23.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.950+0000 7f4ad1e7e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc196580 con 0x7f4acc0ffbd0 2024-03-29T04:51:23.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.950+0000 7f4acaffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4acc105a40 0x7f4acc19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.951+0000 7f4acaffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4acc105a40 0x7f4acc19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57230/0 (socket says 172.21.15.142:57230) 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.951+0000 7f4acaffd700 1 -- 172.21.15.142:0/3767745427 learned_addr learned my addr 172.21.15.142:0/3767745427 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.951+0000 7f4acb7fe700 1 --2- 172.21.15.142:0/3767745427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4acc0ffbd0 0x7f4acc19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.951+0000 7f4acaffd700 1 -- 172.21.15.142:0/3767745427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4acc0ffbd0 msgr2=0x7f4acc19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.951+0000 7f4acaffd700 1 --2- 172.21.15.142:0/3767745427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4acc0ffbd0 0x7f4acc19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.951+0000 7f4acaffd700 1 -- 172.21.15.142:0/3767745427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac00095f0 con 0x7f4acc105a40 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.951+0000 7f4acaffd700 1 --2- 172.21.15.142:0/3767745427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4acc105a40 0x7f4acc19ca40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006010 tx=0x7f4ac000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.951+0000 7f4ac8ff9700 1 -- 172.21.15.142:0/3767745427 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac001d070 con 0x7f4acc105a40 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.951+0000 7f4ac8ff9700 1 -- 172.21.15.142:0/3767745427 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ac000f460 con 0x7f4acc105a40 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.951+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/3767745427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4acc196800 con 0x7f4acc105a40 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.951+0000 7f4ac8ff9700 1 -- 172.21.15.142:0/3767745427 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac0017630 con 0x7f4acc105a40 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.951+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/3767745427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4acc196cf0 con 0x7f4acc105a40 2024-03-29T04:51:23.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.952+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/3767745427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4acc108fe0 con 0x7f4acc105a40 2024-03-29T04:51:23.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.953+0000 7f4ac8ff9700 1 -- 172.21.15.142:0/3767745427 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7f4ac000f5d0 con 0x7f4acc105a40 2024-03-29T04:51:23.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.953+0000 7f4ac8ff9700 1 --2- 172.21.15.142:0/3767745427 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4ab406dea0 0x7f4ab4070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:23.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.953+0000 7f4ac8ff9700 1 -- 172.21.15.142:0/3767745427 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4ac008f290 con 0x7f4acc105a40 2024-03-29T04:51:23.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.953+0000 7f4acb7fe700 1 --2- 172.21.15.142:0/3767745427 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4ab406dea0 0x7f4ab4070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:23.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.953+0000 7f4acb7fe700 1 --2- 172.21.15.142:0/3767745427 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4ab406dea0 0x7f4ab4070360 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005950 tx=0x7f4abc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:23.962 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:23.958+0000 7f4ac8ff9700 1 -- 172.21.15.142:0/3767745427 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4ac005c0f0 con 0x7f4acc105a40 2024-03-29T04:51:24.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:23 smithi152 ceph-mon[88186]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:24.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:23 smithi152 ceph-mon[88186]: osdmap e53: 6 total, 5 up, 6 in 2024-03-29T04:51:24.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:23 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/749975354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:24.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.174+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/3767745427 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4acc02ccf0 con 0x7f4ab406dea0 2024-03-29T04:51:24.179 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.175+0000 7f4ac8ff9700 1 -- 172.21.15.142:0/3767745427 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4acc02ccf0 con 0x7f4ab406dea0 2024-03-29T04:51:24.179 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:51:24.179 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:51:24.179 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:51:24.180 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:51:24.180 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:51:24.180 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:51:24.180 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:51:24.180 INFO:teuthology.orchestra.run.smithi142.stdout: "mon" 2024-03-29T04:51:24.180 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:51:24.180 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "6/23 daemons upgraded", 2024-03-29T04:51:24.180 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading osd daemons", 2024-03-29T04:51:24.180 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:51:24.180 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:51:24.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.177+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/3767745427 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4ab406dea0 msgr2=0x7f4ab4070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:24.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.177+0000 7f4ad1e7e700 1 --2- 172.21.15.142:0/3767745427 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4ab406dea0 0x7f4ab4070360 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005950 tx=0x7f4abc0058e0 comp rx=0 tx=0).stop 2024-03-29T04:51:24.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.177+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/3767745427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4acc105a40 msgr2=0x7f4acc19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:24.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.177+0000 7f4ad1e7e700 1 --2- 172.21.15.142:0/3767745427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4acc105a40 0x7f4acc19ca40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006010 tx=0x7f4ac000be20 comp rx=0 tx=0).stop 2024-03-29T04:51:24.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.178+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/3767745427 shutdown_connections 2024-03-29T04:51:24.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.178+0000 7f4ad1e7e700 1 --2- 172.21.15.142:0/3767745427 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4ab406dea0 0x7f4ab4070360 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:24.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.178+0000 7f4ad1e7e700 1 --2- 172.21.15.142:0/3767745427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4acc0ffbd0 0x7f4acc19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:24.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.178+0000 7f4ad1e7e700 1 --2- 172.21.15.142:0/3767745427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4acc105a40 0x7f4acc19ca40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:24.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.178+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/3767745427 >> 172.21.15.142:0/3767745427 conn(0x7f4acc0755a0 msgr2=0x7f4acc068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:24.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.178+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/3767745427 shutdown_connections 2024-03-29T04:51:24.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.178+0000 7f4ad1e7e700 1 -- 172.21.15.142:0/3767745427 wait complete. 2024-03-29T04:51:24.288 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.284+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/4144148385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf04068670 msgr2=0x7fbf04068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:24.288 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.284+0000 7fbf0bd2e700 1 --2- 172.21.15.142:0/4144148385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf04068670 0x7fbf04068a50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009910 tx=0x7fbef4009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:24.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.284+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/4144148385 shutdown_connections 2024-03-29T04:51:24.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.284+0000 7fbf0bd2e700 1 --2- 172.21.15.142:0/4144148385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf04069020 0x7fbf04105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:24.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.284+0000 7fbf0bd2e700 1 --2- 172.21.15.142:0/4144148385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf04068670 0x7fbf04068a50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:24.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.284+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/4144148385 >> 172.21.15.142:0/4144148385 conn(0x7fbf040755a0 msgr2=0x7fbf040759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:24.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.284+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/4144148385 shutdown_connections 2024-03-29T04:51:24.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.284+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/4144148385 wait complete. 2024-03-29T04:51:24.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.285+0000 7fbf0bd2e700 1 Processor -- start 2024-03-29T04:51:24.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.285+0000 7fbf0bd2e700 1 -- start start 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.285+0000 7fbf0bd2e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf04068670 0x7fbf0419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.285+0000 7fbf0bd2e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf04069020 0x7fbf0419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.285+0000 7fbf0bd2e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0419d050 con 0x7fbf04069020 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.285+0000 7fbf0bd2e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf04196590 con 0x7fbf04068670 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.285+0000 7fbf092c9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf04069020 0x7fbf0419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.285+0000 7fbf092c9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf04069020 0x7fbf0419ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57246/0 (socket says 172.21.15.142:57246) 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.285+0000 7fbf092c9700 1 -- 172.21.15.142:0/3397891333 learned_addr learned my addr 172.21.15.142:0/3397891333 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.285+0000 7fbf092c9700 1 -- 172.21.15.142:0/3397891333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf04068670 msgr2=0x7fbf0419c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.285+0000 7fbf092c9700 1 --2- 172.21.15.142:0/3397891333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf04068670 0x7fbf0419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.285+0000 7fbf092c9700 1 -- 172.21.15.142:0/3397891333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40095f0 con 0x7fbf04069020 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.286+0000 7fbf092c9700 1 --2- 172.21.15.142:0/3397891333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf04069020 0x7fbf0419ca50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000c960 tx=0x7fbf0000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.286+0000 7fbefaffd700 1 -- 172.21.15.142:0/3397891333 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf00007a70 con 0x7fbf04069020 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.286+0000 7fbefaffd700 1 -- 172.21.15.142:0/3397891333 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf00007bd0 con 0x7fbf04069020 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.286+0000 7fbefaffd700 1 -- 172.21.15.142:0/3397891333 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf000186c0 con 0x7fbf04069020 2024-03-29T04:51:24.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.286+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/3397891333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf04196870 con 0x7fbf04069020 2024-03-29T04:51:24.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.286+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/3397891333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf04196dc0 con 0x7fbf04069020 2024-03-29T04:51:24.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.287+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/3397891333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf04109130 con 0x7fbf04069020 2024-03-29T04:51:24.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.287+0000 7fbefaffd700 1 -- 172.21.15.142:0/3397891333 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 39) v1 ==== 91965+0+0 (secure 0 0 0) 0x7fbf0000faa0 con 0x7fbf04069020 2024-03-29T04:51:24.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.288+0000 7fbefaffd700 1 --2- 172.21.15.142:0/3397891333 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbef006de90 0x7fbef0070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:24.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.288+0000 7fbefaffd700 1 -- 172.21.15.142:0/3397891333 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbf0008dce0 con 0x7fbf04069020 2024-03-29T04:51:24.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.288+0000 7fbf09aca700 1 --2- 172.21.15.142:0/3397891333 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbef006de90 0x7fbef0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:24.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.288+0000 7fbf09aca700 1 --2- 172.21.15.142:0/3397891333 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbef006de90 0x7fbef0070350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbef400b5c0 tx=0x7fbef4005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:24.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.292+0000 7fbefaffd700 1 -- 172.21.15.142:0/3397891333 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbf0005a9d0 con 0x7fbf04069020 2024-03-29T04:51:24.581 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:24 smithi142 podman[123856]: 2024-03-29 04:51:24.322385755 +0000 UTC m=+0.658869987 container remove 763c534c639d6cd0119f71a42607829828f0eb1f88f53f21810619d39ad0f7df (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-03-29T04:51:24.581 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:24 smithi142 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-29T04:51:24.581 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:24 smithi142 systemd[1]: Stopped Ceph osd.0 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:51:24.581 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:24 smithi142 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.0.service: Consumed 1min 1.136s CPU time. 2024-03-29T04:51:24.629 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_WARN 1 osds down 2024-03-29T04:51:24.630 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-29T04:51:24.630 INFO:teuthology.orchestra.run.smithi142.stdout: osd.0 (root=default,host=smithi142) is down 2024-03-29T04:51:24.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.624+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/3397891333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbf0404ea90 con 0x7fbf04069020 2024-03-29T04:51:24.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.625+0000 7fbefaffd700 1 -- 172.21.15.142:0/3397891333 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7fbf00061070 con 0x7fbf04069020 2024-03-29T04:51:24.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.627+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/3397891333 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbef006de90 msgr2=0x7fbef0070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:24.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.627+0000 7fbf0bd2e700 1 --2- 172.21.15.142:0/3397891333 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbef006de90 0x7fbef0070350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbef400b5c0 tx=0x7fbef4005fd0 comp rx=0 tx=0).stop 2024-03-29T04:51:24.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.627+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/3397891333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf04069020 msgr2=0x7fbf0419ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:24.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.627+0000 7fbf0bd2e700 1 --2- 172.21.15.142:0/3397891333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf04069020 0x7fbf0419ca50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000c960 tx=0x7fbf0000cd20 comp rx=0 tx=0).stop 2024-03-29T04:51:24.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.627+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/3397891333 shutdown_connections 2024-03-29T04:51:24.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.627+0000 7fbf0bd2e700 1 --2- 172.21.15.142:0/3397891333 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbef006de90 0x7fbef0070350 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:24.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.627+0000 7fbf0bd2e700 1 --2- 172.21.15.142:0/3397891333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf04068670 0x7fbf0419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:24.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.627+0000 7fbf0bd2e700 1 --2- 172.21.15.142:0/3397891333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf04069020 0x7fbf0419ca50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:24.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.627+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/3397891333 >> 172.21.15.142:0/3397891333 conn(0x7fbf040755a0 msgr2=0x7fbf040fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:24.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.627+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/3397891333 shutdown_connections 2024-03-29T04:51:24.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:24.628+0000 7fbf0bd2e700 1 -- 172.21.15.142:0/3397891333 wait complete. 2024-03-29T04:51:24.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:24 smithi142 ceph-mon[112475]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:24.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:24 smithi142 ceph-mon[112475]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:24.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:24 smithi142 ceph-mon[112475]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-29T04:51:24.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:24 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1359219729' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:51:24.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:24 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3397891333' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:51:24.876 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:24 smithi142 systemd[1]: Starting Ceph osd.0 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:51:24.876 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:24 smithi142 podman[124008]: 2024-03-29 04:51:24.702735578 +0000 UTC m=+0.018686222 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:51:24.876 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:24 smithi142 podman[124008]: 2024-03-29 04:51:24.80718118 +0000 UTC m=+0.123131804 container create c0ae128ed1b7304ae2b8ce2f0ed1b4cafa7dbcc9ce4a1357a9e4e242f82c3316 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362, GIT_CLEAN=True) 2024-03-29T04:51:25.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:24 smithi152 ceph-mon[88186]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:25.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:24 smithi152 ceph-mon[88186]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:25.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:24 smithi152 ceph-mon[88186]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-29T04:51:25.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:24 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1359219729' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:51:25.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:24 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3397891333' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:51:25.375 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 podman[124008]: 2024-03-29 04:51:25.059959776 +0000 UTC m=+0.375910401 container init c0ae128ed1b7304ae2b8ce2f0ed1b4cafa7dbcc9ce4a1357a9e4e242f82c3316 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6) 2024-03-29T04:51:25.375 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 podman[124008]: 2024-03-29 04:51:25.063128684 +0000 UTC m=+0.379079312 container start c0ae128ed1b7304ae2b8ce2f0ed1b4cafa7dbcc9ce4a1357a9e4e242f82c3316 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate, RELEASE=squid-8da2362, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6) 2024-03-29T04:51:25.375 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 podman[124008]: 2024-03-29 04:51:25.14331865 +0000 UTC m=+0.459269279 container attach c0ae128ed1b7304ae2b8ce2f0ed1b4cafa7dbcc9ce4a1357a9e4e242f82c3316 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362) 2024-03-29T04:51:25.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:25 smithi152 ceph-mon[88186]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:25.850 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:25 smithi142 ceph-mon[112475]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:25.851 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate[124023]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-29T04:51:25.851 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate[124023]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-29T04:51:25.851 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 bash[124008]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-29T04:51:25.852 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 bash[124008]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-29T04:51:25.852 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 bash[124008]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-29T04:51:25.852 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 bash[124008]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-29T04:51:25.852 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 bash[124008]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-29T04:51:25.852 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 bash[124008]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-29T04:51:25.852 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 bash[124008]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-29T04:51:25.852 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate[124023]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-29T04:51:25.853 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate[124023]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-29T04:51:25.853 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate[124023]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-29T04:51:25.853 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate[124023]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-29T04:51:25.853 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate[124023]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-29T04:51:25.853 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:25 smithi142 podman[124008]: 2024-03-29 04:51:25.532832327 +0000 UTC m=+0.848782958 container died c0ae128ed1b7304ae2b8ce2f0ed1b4cafa7dbcc9ce4a1357a9e4e242f82c3316 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, RELEASE=squid-8da2362, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-29T04:51:26.375 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:26 smithi142 podman[124163]: 2024-03-29 04:51:26.114089583 +0000 UTC m=+0.575947734 container remove c0ae128ed1b7304ae2b8ce2f0ed1b4cafa7dbcc9ce4a1357a9e4e242f82c3316 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8da2362, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:51:26.375 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:26 smithi142 podman[124228]: 2024-03-29 04:51:26.231500835 +0000 UTC m=+0.018072490 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:51:26.375 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:26 smithi142 podman[124228]: 2024-03-29 04:51:26.348356367 +0000 UTC m=+0.134928062 container create 5c47cbf77b860de266dfc47f6c8fd807c632d329bd2c2a7de52986c0931d53a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-03-29T04:51:26.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:26 smithi142 ceph-mon[112475]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 65/333 objects degraded (19.520%) 2024-03-29T04:51:26.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:26 smithi142 ceph-mon[112475]: Health check failed: Degraded data redundancy: 65/333 objects degraded (19.520%), 19 pgs degraded (PG_DEGRADED) 2024-03-29T04:51:26.875 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:26 smithi142 podman[124228]: 2024-03-29 04:51:26.550844024 +0000 UTC m=+0.337415688 container init 5c47cbf77b860de266dfc47f6c8fd807c632d329bd2c2a7de52986c0931d53a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326, RELEASE=squid-8da2362, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-29T04:51:26.876 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:26 smithi142 podman[124228]: 2024-03-29 04:51:26.554091798 +0000 UTC m=+0.340663451 container start 5c47cbf77b860de266dfc47f6c8fd807c632d329bd2c2a7de52986c0931d53a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.schema-version=1.0, RELEASE=squid-8da2362, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD) 2024-03-29T04:51:26.876 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:26 smithi142 ceph-osd[124247]: -- 172.21.15.142:0/2193788041 <== mon.0 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558c49831680 con 0x558c497fac00 2024-03-29T04:51:26.876 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:26 smithi142 bash[124228]: 5c47cbf77b860de266dfc47f6c8fd807c632d329bd2c2a7de52986c0931d53a1 2024-03-29T04:51:26.876 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:26 smithi142 systemd[1]: Started Ceph osd.0 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:51:27.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:26 smithi152 ceph-mon[88186]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 65/333 objects degraded (19.520%) 2024-03-29T04:51:27.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:26 smithi152 ceph-mon[88186]: Health check failed: Degraded data redundancy: 65/333 objects degraded (19.520%), 19 pgs degraded (PG_DEGRADED) 2024-03-29T04:51:27.375 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:27 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0[124243]: 2024-03-29T04:51:27.104+0000 7fe6357e67c0 -1 Falling back to public interface 2024-03-29T04:51:28.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:27 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:28.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:27 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:28.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:27 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:51:28.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:27 smithi152 ceph-mon[88186]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 65/333 objects degraded (19.520%) 2024-03-29T04:51:28.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:27 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:28.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:27 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:28.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:27 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:51:28.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:27 smithi142 ceph-mon[112475]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 65/333 objects degraded (19.520%) 2024-03-29T04:51:30.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:30 smithi152 ceph-mon[88186]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 65/333 objects degraded (19.520%) 2024-03-29T04:51:30.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:30.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:51:30.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:30 smithi142 ceph-mon[112475]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 65/333 objects degraded (19.520%) 2024-03-29T04:51:30.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:30.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:51:31.525 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:31 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0[124243]: 2024-03-29T04:51:31.326+0000 7fe6357e67c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-29T04:51:31.875 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:31 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0[124243]: 2024-03-29T04:51:31.751+0000 7fe6357e67c0 -1 osd.0 51 log_to_monitors true 2024-03-29T04:51:32.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:32 smithi142 ceph-mon[112475]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 516 B/s wr, 2 op/s; 65/333 objects degraded (19.520%) 2024-03-29T04:51:32.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:32 smithi142 ceph-mon[112475]: mgrmap e40: smithi142.swesrg(active, since 92s), standbys: smithi152.usoolh 2024-03-29T04:51:32.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:32 smithi142 ceph-mon[112475]: from='osd.0 [v2:172.21.15.142:6802/1587216149,v1:172.21.15.142:6803/1587216149]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-29T04:51:32.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:32.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:33.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:32 smithi152 ceph-mon[88186]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 516 B/s wr, 2 op/s; 65/333 objects degraded (19.520%) 2024-03-29T04:51:33.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:32 smithi152 ceph-mon[88186]: mgrmap e40: smithi142.swesrg(active, since 92s), standbys: smithi152.usoolh 2024-03-29T04:51:33.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:32 smithi152 ceph-mon[88186]: from='osd.0 [v2:172.21.15.142:6802/1587216149,v1:172.21.15.142:6803/1587216149]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-29T04:51:33.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:33.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:33.795 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 04:51:33 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0[124243]: 2024-03-29T04:51:33.406+0000 7fe622c36700 -1 osd.0 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:51:33.795 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:33 smithi142 ceph-mon[112475]: from='osd.0 [v2:172.21.15.142:6802/1587216149,v1:172.21.15.142:6803/1587216149]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-29T04:51:33.795 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:33 smithi142 ceph-mon[112475]: osdmap e54: 6 total, 5 up, 6 in 2024-03-29T04:51:33.795 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:33 smithi142 ceph-mon[112475]: from='osd.0 [v2:172.21.15.142:6802/1587216149,v1:172.21.15.142:6803/1587216149]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-29T04:51:33.796 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:33.796 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:33.796 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:33 smithi142 ceph-mon[112475]: from='osd.0 [v2:172.21.15.142:6802/1587216149,v1:172.21.15.142:6803/1587216149]' entity='osd.0' 2024-03-29T04:51:34.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:33 smithi152 ceph-mon[88186]: from='osd.0 [v2:172.21.15.142:6802/1587216149,v1:172.21.15.142:6803/1587216149]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-29T04:51:34.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:33 smithi152 ceph-mon[88186]: osdmap e54: 6 total, 5 up, 6 in 2024-03-29T04:51:34.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:33 smithi152 ceph-mon[88186]: from='osd.0 [v2:172.21.15.142:6802/1587216149,v1:172.21.15.142:6803/1587216149]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-29T04:51:34.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:34.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:34.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:33 smithi152 ceph-mon[88186]: from='osd.0 [v2:172.21.15.142:6802/1587216149,v1:172.21.15.142:6803/1587216149]' entity='osd.0' 2024-03-29T04:51:34.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:34 smithi142 ceph-mon[112475]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 511 B/s wr, 2 op/s; 65/333 objects degraded (19.520%) 2024-03-29T04:51:34.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:34 smithi142 ceph-mon[112475]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T04:51:34.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:34 smithi142 ceph-mon[112475]: osd.0 [v2:172.21.15.142:6802/1587216149,v1:172.21.15.142:6803/1587216149] boot 2024-03-29T04:51:34.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:34 smithi142 ceph-mon[112475]: osdmap e55: 6 total, 6 up, 6 in 2024-03-29T04:51:34.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:34 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:51:35.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:34 smithi152 ceph-mon[88186]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 511 B/s wr, 2 op/s; 65/333 objects degraded (19.520%) 2024-03-29T04:51:35.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:34 smithi152 ceph-mon[88186]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T04:51:35.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:34 smithi152 ceph-mon[88186]: osd.0 [v2:172.21.15.142:6802/1587216149,v1:172.21.15.142:6803/1587216149] boot 2024-03-29T04:51:35.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:34 smithi152 ceph-mon[88186]: osdmap e55: 6 total, 6 up, 6 in 2024-03-29T04:51:35.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:34 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T04:51:36.784 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:36 smithi142 ceph-mon[112475]: osdmap e56: 6 total, 6 up, 6 in 2024-03-29T04:51:36.784 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:36 smithi142 ceph-mon[112475]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s; 11/333 objects degraded (3.303%) 2024-03-29T04:51:36.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:36 smithi152 ceph-mon[88186]: osdmap e56: 6 total, 6 up, 6 in 2024-03-29T04:51:36.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:36 smithi152 ceph-mon[88186]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s; 11/333 objects degraded (3.303%) 2024-03-29T04:51:37.699 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:37 smithi142 ceph-mon[112475]: Health check update: Degraded data redundancy: 11/333 objects degraded (3.303%), 3 pgs degraded (PG_DEGRADED) 2024-03-29T04:51:37.699 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:37.699 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:37.699 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:51:37.699 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:51:37.699 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:37.699 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:51:37.699 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:37.699 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:37.699 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:37.699 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T04:51:37.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:37 smithi152 ceph-mon[88186]: Health check update: Degraded data redundancy: 11/333 objects degraded (3.303%), 3 pgs degraded (PG_DEGRADED) 2024-03-29T04:51:37.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:37.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:37.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:51:37.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:51:37.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:37.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:51:37.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:37.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:37.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:37.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T04:51:38.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:38 smithi152 ceph-mon[88186]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T04:51:38.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:38 smithi152 ceph-mon[88186]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-03-29T04:51:38.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:38 smithi152 ceph-mon[88186]: pgmap v57: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 11/333 objects degraded (3.303%) 2024-03-29T04:51:38.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:38 smithi142 ceph-mon[112475]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T04:51:38.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:38 smithi142 ceph-mon[112475]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-03-29T04:51:38.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:38 smithi142 ceph-mon[112475]: pgmap v57: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 11/333 objects degraded (3.303%) 2024-03-29T04:51:40.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:39 smithi152 ceph-mon[88186]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/333 objects degraded (3.303%), 3 pgs degraded) 2024-03-29T04:51:40.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:39 smithi152 ceph-mon[88186]: Cluster is now healthy 2024-03-29T04:51:40.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:39 smithi142 ceph-mon[112475]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/333 objects degraded (3.303%), 3 pgs degraded) 2024-03-29T04:51:40.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:39 smithi142 ceph-mon[112475]: Cluster is now healthy 2024-03-29T04:51:41.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:40 smithi152 ceph-mon[88186]: pgmap v58: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-03-29T04:51:41.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:40 smithi142 ceph-mon[112475]: pgmap v58: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-03-29T04:51:41.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:41 smithi142 ceph-mon[112475]: pgmap v59: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:42.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:41 smithi152 ceph-mon[88186]: pgmap v59: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:44.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:44 smithi142 ceph-mon[112475]: pgmap v60: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-29T04:51:44.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:44.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:51:44.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:45.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:44 smithi152 ceph-mon[88186]: pgmap v60: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-29T04:51:45.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:45.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:51:45.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:46.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:46 smithi142 ceph-mon[112475]: pgmap v61: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-29T04:51:47.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:46 smithi152 ceph-mon[88186]: pgmap v61: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-29T04:51:48.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:48 smithi142 ceph-mon[112475]: pgmap v62: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:49.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:48 smithi152 ceph-mon[88186]: pgmap v62: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:50.865 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:50 smithi152 ceph-mon[88186]: pgmap v63: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:50.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:50 smithi142 ceph-mon[112475]: pgmap v63: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:52.857 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:52 smithi142 ceph-mon[112475]: pgmap v64: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:52.858 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T04:51:53.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:52 smithi152 ceph-mon[88186]: pgmap v64: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:53.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T04:51:53.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:53 smithi142 ceph-mon[112475]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T04:51:53.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:53 smithi142 ceph-mon[112475]: Upgrade: osd.1 is safe to restart 2024-03-29T04:51:54.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:53 smithi152 ceph-mon[88186]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T04:51:54.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:53 smithi152 ceph-mon[88186]: Upgrade: osd.1 is safe to restart 2024-03-29T04:51:54.746 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.740+0000 7f885eaae700 1 -- 172.21.15.142:0/1269424829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8858105a40 msgr2=0x7f8858105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:54.746 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.740+0000 7f885eaae700 1 --2- 172.21.15.142:0/1269424829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8858105a40 0x7f8858105e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f884c009910 tx=0x7f884c009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.740+0000 7f885eaae700 1 -- 172.21.15.142:0/1269424829 shutdown_connections 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.740+0000 7f885eaae700 1 --2- 172.21.15.142:0/1269424829 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88580ffbd0 0x7f8858100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.740+0000 7f885eaae700 1 --2- 172.21.15.142:0/1269424829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8858105a40 0x7f8858105e20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.740+0000 7f885eaae700 1 -- 172.21.15.142:0/1269424829 >> 172.21.15.142:0/1269424829 conn(0x7f88580755a0 msgr2=0x7f88580759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.740+0000 7f885eaae700 1 -- 172.21.15.142:0/1269424829 shutdown_connections 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.740+0000 7f885eaae700 1 -- 172.21.15.142:0/1269424829 wait complete. 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.741+0000 7f885eaae700 1 Processor -- start 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.741+0000 7f885eaae700 1 -- start start 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.741+0000 7f885eaae700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88580ffbd0 0x7f885819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.741+0000 7f885eaae700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8858105a40 0x7f885819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.741+0000 7f885eaae700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885819d060 con 0x7f88580ffbd0 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.741+0000 7f885eaae700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8858196520 con 0x7f8858105a40 2024-03-29T04:51:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.741+0000 7f885c84a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88580ffbd0 0x7f885819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:54.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.741+0000 7f885c84a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88580ffbd0 0x7f885819c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47536/0 (socket says 172.21.15.142:47536) 2024-03-29T04:51:54.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.741+0000 7f885c84a700 1 -- 172.21.15.142:0/3844004307 learned_addr learned my addr 172.21.15.142:0/3844004307 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:54.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.741+0000 7f885c84a700 1 -- 172.21.15.142:0/3844004307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8858105a40 msgr2=0x7f885819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:54.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.741+0000 7f885c84a700 1 --2- 172.21.15.142:0/3844004307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8858105a40 0x7f885819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:54.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.741+0000 7f885c84a700 1 -- 172.21.15.142:0/3844004307 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c0095f0 con 0x7f88580ffbd0 2024-03-29T04:51:54.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.742+0000 7f885c84a700 1 --2- 172.21.15.142:0/3844004307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88580ffbd0 0x7f885819c440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f884c006010 tx=0x7f884c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:54.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.742+0000 7f8855ffb700 1 -- 172.21.15.142:0/3844004307 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f884c01d070 con 0x7f88580ffbd0 2024-03-29T04:51:54.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.742+0000 7f8855ffb700 1 -- 172.21.15.142:0/3844004307 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f884c00f460 con 0x7f88580ffbd0 2024-03-29T04:51:54.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.742+0000 7f8855ffb700 1 -- 172.21.15.142:0/3844004307 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f884c017630 con 0x7f88580ffbd0 2024-03-29T04:51:54.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.742+0000 7f885eaae700 1 -- 172.21.15.142:0/3844004307 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88581967a0 con 0x7f88580ffbd0 2024-03-29T04:51:54.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.742+0000 7f885eaae700 1 -- 172.21.15.142:0/3844004307 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8858196c90 con 0x7f88580ffbd0 2024-03-29T04:51:54.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.743+0000 7f885eaae700 1 -- 172.21.15.142:0/3844004307 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8858109050 con 0x7f88580ffbd0 2024-03-29T04:51:54.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.744+0000 7f8855ffb700 1 -- 172.21.15.142:0/3844004307 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f884c00f5d0 con 0x7f88580ffbd0 2024-03-29T04:51:54.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.744+0000 7f8855ffb700 1 --2- 172.21.15.142:0/3844004307 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f884806dd60 0x7f8848070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.744+0000 7f8855ffb700 1 -- 172.21.15.142:0/3844004307 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f884c08eef0 con 0x7f88580ffbd0 2024-03-29T04:51:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.744+0000 7f8857fff700 1 --2- 172.21.15.142:0/3844004307 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f884806dd60 0x7f8848070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.744+0000 7f8857fff700 1 --2- 172.21.15.142:0/3844004307 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f884806dd60 0x7f8848070220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8844005950 tx=0x7f88440058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:54.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.749+0000 7f8855ffb700 1 -- 172.21.15.142:0/3844004307 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f884c05c090 con 0x7f88580ffbd0 2024-03-29T04:51:54.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:54 smithi142 ceph-mon[112475]: pgmap v65: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:54.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:54.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-29T04:51:54.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:51:54.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.969+0000 7f885eaae700 1 -- 172.21.15.142:0/3844004307 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8858196f70 con 0x7f884806dd60 2024-03-29T04:51:54.975 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.971+0000 7f8855ffb700 1 -- 172.21.15.142:0/3844004307 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8858196f70 con 0x7f884806dd60 2024-03-29T04:51:54.977 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.973+0000 7f885eaae700 1 -- 172.21.15.142:0/3844004307 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f884806dd60 msgr2=0x7f8848070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:54.977 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.973+0000 7f885eaae700 1 --2- 172.21.15.142:0/3844004307 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f884806dd60 0x7f8848070220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8844005950 tx=0x7f88440058e0 comp rx=0 tx=0).stop 2024-03-29T04:51:54.977 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.973+0000 7f885eaae700 1 -- 172.21.15.142:0/3844004307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88580ffbd0 msgr2=0x7f885819c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:54.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.973+0000 7f885eaae700 1 --2- 172.21.15.142:0/3844004307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88580ffbd0 0x7f885819c440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f884c006010 tx=0x7f884c00be20 comp rx=0 tx=0).stop 2024-03-29T04:51:54.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.973+0000 7f885eaae700 1 -- 172.21.15.142:0/3844004307 shutdown_connections 2024-03-29T04:51:54.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.973+0000 7f885eaae700 1 --2- 172.21.15.142:0/3844004307 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f884806dd60 0x7f8848070220 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:54.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.973+0000 7f885eaae700 1 --2- 172.21.15.142:0/3844004307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88580ffbd0 0x7f885819c440 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:54.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.973+0000 7f885eaae700 1 --2- 172.21.15.142:0/3844004307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8858105a40 0x7f885819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:54.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.973+0000 7f885eaae700 1 -- 172.21.15.142:0/3844004307 >> 172.21.15.142:0/3844004307 conn(0x7f88580755a0 msgr2=0x7f88580685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:54.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.973+0000 7f885eaae700 1 -- 172.21.15.142:0/3844004307 shutdown_connections 2024-03-29T04:51:54.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:54.973+0000 7f885eaae700 1 -- 172.21.15.142:0/3844004307 wait complete. 2024-03-29T04:51:54.988 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:51:55.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:54 smithi152 ceph-mon[88186]: pgmap v65: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:55.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:55.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-29T04:51:55.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:51:55.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.078+0000 7f98bb5df700 1 -- 172.21.15.142:0/2123987369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98b4100dd0 msgr2=0x7f98b41011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:55.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.078+0000 7f98bb5df700 1 --2- 172.21.15.142:0/2123987369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98b4100dd0 0x7f98b41011b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009910 tx=0x7f98a4009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:55.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.078+0000 7f98bb5df700 1 -- 172.21.15.142:0/2123987369 shutdown_connections 2024-03-29T04:51:55.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.078+0000 7f98bb5df700 1 --2- 172.21.15.142:0/2123987369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98b4101780 0x7f98b41057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.078+0000 7f98bb5df700 1 --2- 172.21.15.142:0/2123987369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98b4100dd0 0x7f98b41011b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.078+0000 7f98bb5df700 1 -- 172.21.15.142:0/2123987369 >> 172.21.15.142:0/2123987369 conn(0x7f98b40fc640 msgr2=0x7f98b40fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:55.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.078+0000 7f98bb5df700 1 -- 172.21.15.142:0/2123987369 shutdown_connections 2024-03-29T04:51:55.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.078+0000 7f98bb5df700 1 -- 172.21.15.142:0/2123987369 wait complete. 2024-03-29T04:51:55.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98bb5df700 1 Processor -- start 2024-03-29T04:51:55.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98bb5df700 1 -- start start 2024-03-29T04:51:55.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98bb5df700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98b4100dd0 0x7f98b4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:55.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98bb5df700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98b4101780 0x7f98b40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:55.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98bb5df700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b4073b10 con 0x7f98b4100dd0 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98bb5df700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b4073c50 con 0x7f98b4101780 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98b937b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98b4100dd0 0x7f98b4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98b937b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98b4100dd0 0x7f98b4073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47540/0 (socket says 172.21.15.142:47540) 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98b937b700 1 -- 172.21.15.142:0/1531051522 learned_addr learned my addr 172.21.15.142:0/1531051522 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98b8b7a700 1 --2- 172.21.15.142:0/1531051522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98b4101780 0x7f98b40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98b937b700 1 -- 172.21.15.142:0/1531051522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98b4101780 msgr2=0x7f98b40735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98b937b700 1 --2- 172.21.15.142:0/1531051522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98b4101780 0x7f98b40735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.079+0000 7f98b937b700 1 -- 172.21.15.142:0/1531051522 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a40095f0 con 0x7f98b4100dd0 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.080+0000 7f98b937b700 1 --2- 172.21.15.142:0/1531051522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98b4100dd0 0x7f98b4073090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f98a4000c00 tx=0x7f98a400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.080+0000 7f98aa7fc700 1 -- 172.21.15.142:0/1531051522 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98a401d070 con 0x7f98b4100dd0 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.080+0000 7f98bb5df700 1 -- 172.21.15.142:0/1531051522 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b4073ed0 con 0x7f98b4100dd0 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.080+0000 7f98aa7fc700 1 -- 172.21.15.142:0/1531051522 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98a4005dc0 con 0x7f98b4100dd0 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.080+0000 7f98aa7fc700 1 -- 172.21.15.142:0/1531051522 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98a4017400 con 0x7f98b4100dd0 2024-03-29T04:51:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.080+0000 7f98bb5df700 1 -- 172.21.15.142:0/1531051522 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b41a2c40 con 0x7f98b4100dd0 2024-03-29T04:51:55.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.081+0000 7f98bb5df700 1 -- 172.21.15.142:0/1531051522 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b4109170 con 0x7f98b4100dd0 2024-03-29T04:51:55.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.081+0000 7f98aa7fc700 1 -- 172.21.15.142:0/1531051522 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f98a400f460 con 0x7f98b4100dd0 2024-03-29T04:51:55.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.082+0000 7f98aa7fc700 1 --2- 172.21.15.142:0/1531051522 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f98a006dd60 0x7f98a0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:55.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.082+0000 7f98aa7fc700 1 -- 172.21.15.142:0/1531051522 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f98a408ef30 con 0x7f98b4100dd0 2024-03-29T04:51:55.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.082+0000 7f98b8b7a700 1 --2- 172.21.15.142:0/1531051522 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f98a006dd60 0x7f98a0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:55.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.082+0000 7f98b8b7a700 1 --2- 172.21.15.142:0/1531051522 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f98a006dd60 0x7f98a0070220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f98b4074880 tx=0x7f98b000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:55.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.086+0000 7f98aa7fc700 1 -- 172.21.15.142:0/1531051522 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f98a40c3280 con 0x7f98b4100dd0 2024-03-29T04:51:55.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.305+0000 7f98bb5df700 1 -- 172.21.15.142:0/1531051522 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f98b40611d0 con 0x7f98a006dd60 2024-03-29T04:51:55.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.306+0000 7f98aa7fc700 1 -- 172.21.15.142:0/1531051522 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f98b40611d0 con 0x7f98a006dd60 2024-03-29T04:51:55.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.308+0000 7f98bb5df700 1 -- 172.21.15.142:0/1531051522 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f98a006dd60 msgr2=0x7f98a0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:55.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.308+0000 7f98bb5df700 1 --2- 172.21.15.142:0/1531051522 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f98a006dd60 0x7f98a0070220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f98b4074880 tx=0x7f98b000b410 comp rx=0 tx=0).stop 2024-03-29T04:51:55.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.308+0000 7f98bb5df700 1 -- 172.21.15.142:0/1531051522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98b4100dd0 msgr2=0x7f98b4073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:55.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.308+0000 7f98bb5df700 1 --2- 172.21.15.142:0/1531051522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98b4100dd0 0x7f98b4073090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f98a4000c00 tx=0x7f98a400bd70 comp rx=0 tx=0).stop 2024-03-29T04:51:55.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.308+0000 7f98bb5df700 1 -- 172.21.15.142:0/1531051522 shutdown_connections 2024-03-29T04:51:55.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.308+0000 7f98bb5df700 1 --2- 172.21.15.142:0/1531051522 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f98a006dd60 0x7f98a0070220 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.308+0000 7f98bb5df700 1 --2- 172.21.15.142:0/1531051522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98b4100dd0 0x7f98b4073090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.308+0000 7f98bb5df700 1 --2- 172.21.15.142:0/1531051522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98b4101780 0x7f98b40735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.308+0000 7f98bb5df700 1 -- 172.21.15.142:0/1531051522 >> 172.21.15.142:0/1531051522 conn(0x7f98b40fc640 msgr2=0x7f98b40fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:55.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.308+0000 7f98bb5df700 1 -- 172.21.15.142:0/1531051522 shutdown_connections 2024-03-29T04:51:55.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.308+0000 7f98bb5df700 1 -- 172.21.15.142:0/1531051522 wait complete. 2024-03-29T04:51:55.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.417+0000 7f646a0d3700 1 -- 172.21.15.142:0/4280827134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6464068670 msgr2=0x7f6464068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:55.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.417+0000 7f646a0d3700 1 --2- 172.21.15.142:0/4280827134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6464068670 0x7f6464068a50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6454009910 tx=0x7f6454009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:55.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.417+0000 7f646a0d3700 1 -- 172.21.15.142:0/4280827134 shutdown_connections 2024-03-29T04:51:55.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.417+0000 7f646a0d3700 1 --2- 172.21.15.142:0/4280827134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6464069020 0x7f6464105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.417+0000 7f646a0d3700 1 --2- 172.21.15.142:0/4280827134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6464068670 0x7f6464068a50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.417+0000 7f646a0d3700 1 -- 172.21.15.142:0/4280827134 >> 172.21.15.142:0/4280827134 conn(0x7f64640755a0 msgr2=0x7f64640759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:55.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.417+0000 7f646a0d3700 1 -- 172.21.15.142:0/4280827134 shutdown_connections 2024-03-29T04:51:55.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.417+0000 7f646a0d3700 1 -- 172.21.15.142:0/4280827134 wait complete. 2024-03-29T04:51:55.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.417+0000 7f646a0d3700 1 Processor -- start 2024-03-29T04:51:55.424 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.418+0000 7f646a0d3700 1 -- start start 2024-03-29T04:51:55.424 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.418+0000 7f646a0d3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6464068670 0x7f646419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:55.424 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.418+0000 7f646a0d3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6464069020 0x7f646419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:55.424 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.418+0000 7f646a0d3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646419d290 con 0x7f6464068670 2024-03-29T04:51:55.424 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.418+0000 7f646a0d3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6464196740 con 0x7f6464069020 2024-03-29T04:51:55.424 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.418+0000 7f64637fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6464068670 0x7f646419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:55.424 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.418+0000 7f64637fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6464068670 0x7f646419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47554/0 (socket says 172.21.15.142:47554) 2024-03-29T04:51:55.424 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.418+0000 7f64637fe700 1 -- 172.21.15.142:0/343155715 learned_addr learned my addr 172.21.15.142:0/343155715 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:55.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.418+0000 7f64637fe700 1 -- 172.21.15.142:0/343155715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6464069020 msgr2=0x7f646419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:55.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.418+0000 7f64637fe700 1 --2- 172.21.15.142:0/343155715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6464069020 0x7f646419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.418+0000 7f64637fe700 1 -- 172.21.15.142:0/343155715 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64540095f0 con 0x7f6464068670 2024-03-29T04:51:55.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.418+0000 7f64637fe700 1 --2- 172.21.15.142:0/343155715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6464068670 0x7f646419c6c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6454006010 tx=0x7f645400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:55.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.419+0000 7f6460ff9700 1 -- 172.21.15.142:0/343155715 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f645401d070 con 0x7f6464068670 2024-03-29T04:51:55.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.419+0000 7f6460ff9700 1 -- 172.21.15.142:0/343155715 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6454004fb0 con 0x7f6464068670 2024-03-29T04:51:55.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.419+0000 7f646a0d3700 1 -- 172.21.15.142:0/343155715 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64641969c0 con 0x7f6464068670 2024-03-29T04:51:55.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.419+0000 7f6460ff9700 1 -- 172.21.15.142:0/343155715 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6454022ea0 con 0x7f6464068670 2024-03-29T04:51:55.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.419+0000 7f646a0d3700 1 -- 172.21.15.142:0/343155715 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6464196eb0 con 0x7f6464068670 2024-03-29T04:51:55.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.420+0000 7f646a0d3700 1 -- 172.21.15.142:0/343155715 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6464109130 con 0x7f6464068670 2024-03-29T04:51:55.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.420+0000 7f6460ff9700 1 -- 172.21.15.142:0/343155715 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6454022470 con 0x7f6464068670 2024-03-29T04:51:55.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.421+0000 7f6460ff9700 1 --2- 172.21.15.142:0/343155715 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f644c06dd60 0x7f644c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:55.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.421+0000 7f6460ff9700 1 -- 172.21.15.142:0/343155715 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f645408f060 con 0x7f6464068670 2024-03-29T04:51:55.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.421+0000 7f6462ffd700 1 --2- 172.21.15.142:0/343155715 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f644c06dd60 0x7f644c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:55.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.421+0000 7f6462ffd700 1 --2- 172.21.15.142:0/343155715 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f644c06dd60 0x7f644c070220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6464197bb0 tx=0x7f6458006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:55.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.425+0000 7f6460ff9700 1 -- 172.21.15.142:0/343155715 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f645405c230 con 0x7f6464068670 2024-03-29T04:51:55.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.643+0000 7f646a0d3700 1 -- 172.21.15.142:0/343155715 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f646402cce0 con 0x7f644c06dd60 2024-03-29T04:51:55.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.653+0000 7f6460ff9700 1 -- 172.21.15.142:0/343155715 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f646402cce0 con 0x7f644c06dd60 2024-03-29T04:51:55.658 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:51:55.658 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (3m) 23s ago 15m 26.5M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:51:55.658 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (15m) 23s ago 15m 8296k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:51:55.658 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (14m) 46s ago 14m 8182k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:51:55.658 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (55s) 23s ago 15m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:51:55.658 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (49s) 46s ago 14m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (3m) 23s ago 15m 78.4M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (9m) 23s ago 9m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (9m) 23s ago 9m 63.5M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (9m) 46s ago 9m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (9m) 46s ago 9m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (5m) 23s ago 16m 500M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (4m) 46s ago 14m 449M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (2m) 23s ago 16m 45.7M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (99s) 46s ago 14m 41.1M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (4m) 23s ago 15m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (4m) 46s ago 14m 16.0M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (29s) 23s ago 13m 15.1M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (13m) 23s ago 12m 240M 1216M 18.2.1 5be31c24972a 539c2cbf33ac 2024-03-29T04:51:55.659 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (12m) 23s ago 12m 199M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:51:55.660 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (12m) 46s ago 12m 276M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:51:55.660 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (11m) 46s ago 11m 236M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:51:55.660 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (11m) 46s ago 11m 225M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:51:55.660 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (4m) 23s ago 14m 63.3M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:51:55.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.656+0000 7f646a0d3700 1 -- 172.21.15.142:0/343155715 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f644c06dd60 msgr2=0x7f644c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:55.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.656+0000 7f646a0d3700 1 --2- 172.21.15.142:0/343155715 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f644c06dd60 0x7f644c070220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6464197bb0 tx=0x7f6458006d20 comp rx=0 tx=0).stop 2024-03-29T04:51:55.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.656+0000 7f646a0d3700 1 -- 172.21.15.142:0/343155715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6464068670 msgr2=0x7f646419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:55.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.656+0000 7f646a0d3700 1 --2- 172.21.15.142:0/343155715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6464068670 0x7f646419c6c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6454006010 tx=0x7f645400be20 comp rx=0 tx=0).stop 2024-03-29T04:51:55.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.656+0000 7f646a0d3700 1 -- 172.21.15.142:0/343155715 shutdown_connections 2024-03-29T04:51:55.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.656+0000 7f646a0d3700 1 --2- 172.21.15.142:0/343155715 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f644c06dd60 0x7f644c070220 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.656+0000 7f646a0d3700 1 --2- 172.21.15.142:0/343155715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6464068670 0x7f646419c6c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.656+0000 7f646a0d3700 1 --2- 172.21.15.142:0/343155715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6464069020 0x7f646419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.656+0000 7f646a0d3700 1 -- 172.21.15.142:0/343155715 >> 172.21.15.142:0/343155715 conn(0x7f64640755a0 msgr2=0x7f64640fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:55.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.656+0000 7f646a0d3700 1 -- 172.21.15.142:0/343155715 shutdown_connections 2024-03-29T04:51:55.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.656+0000 7f646a0d3700 1 -- 172.21.15.142:0/343155715 wait complete. 2024-03-29T04:51:55.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.765+0000 7f43df3c2700 1 -- 172.21.15.142:0/2823546148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d8101790 msgr2=0x7f43d81057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:55.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.765+0000 7f43df3c2700 1 --2- 172.21.15.142:0/2823546148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d8101790 0x7f43d81057e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009910 tx=0x7f43c8009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:55.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.765+0000 7f43df3c2700 1 -- 172.21.15.142:0/2823546148 shutdown_connections 2024-03-29T04:51:55.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.765+0000 7f43df3c2700 1 --2- 172.21.15.142:0/2823546148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d8101790 0x7f43d81057e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.765+0000 7f43df3c2700 1 --2- 172.21.15.142:0/2823546148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43d8100de0 0x7f43d81011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.765+0000 7f43df3c2700 1 -- 172.21.15.142:0/2823546148 >> 172.21.15.142:0/2823546148 conn(0x7f43d80fc670 msgr2=0x7f43d80fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:55.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.765+0000 7f43df3c2700 1 -- 172.21.15.142:0/2823546148 shutdown_connections 2024-03-29T04:51:55.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.765+0000 7f43df3c2700 1 -- 172.21.15.142:0/2823546148 wait complete. 2024-03-29T04:51:55.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.766+0000 7f43df3c2700 1 Processor -- start 2024-03-29T04:51:55.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.766+0000 7f43df3c2700 1 -- start start 2024-03-29T04:51:55.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.766+0000 7f43df3c2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d8100de0 0x7f43d819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:55.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.766+0000 7f43df3c2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43d8101790 0x7f43d819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:55.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.766+0000 7f43df3c2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d819d130 con 0x7f43d8100de0 2024-03-29T04:51:55.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.766+0000 7f43df3c2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d81965e0 con 0x7f43d8101790 2024-03-29T04:51:55.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.766+0000 7f43dd15e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d8100de0 0x7f43d819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:55.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.766+0000 7f43dd15e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d8100de0 0x7f43d819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47578/0 (socket says 172.21.15.142:47578) 2024-03-29T04:51:55.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.766+0000 7f43dd15e700 1 -- 172.21.15.142:0/673473347 learned_addr learned my addr 172.21.15.142:0/673473347 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:55.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.766+0000 7f43dc95d700 1 --2- 172.21.15.142:0/673473347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43d8101790 0x7f43d819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:55.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.767+0000 7f43dd15e700 1 -- 172.21.15.142:0/673473347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43d8101790 msgr2=0x7f43d819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:55.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.767+0000 7f43dd15e700 1 --2- 172.21.15.142:0/673473347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43d8101790 0x7f43d819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:55.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.767+0000 7f43dd15e700 1 -- 172.21.15.142:0/673473347 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c80095f0 con 0x7f43d8100de0 2024-03-29T04:51:55.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.767+0000 7f43dd15e700 1 --2- 172.21.15.142:0/673473347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d8100de0 0x7f43d819c560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f43d400e920 tx=0x7f43d400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:55.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.767+0000 7f43ce7fc700 1 -- 172.21.15.142:0/673473347 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43d400c9d0 con 0x7f43d8100de0 2024-03-29T04:51:55.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.767+0000 7f43ce7fc700 1 -- 172.21.15.142:0/673473347 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43d4010430 con 0x7f43d8100de0 2024-03-29T04:51:55.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.767+0000 7f43df3c2700 1 -- 172.21.15.142:0/673473347 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d81968c0 con 0x7f43d8100de0 2024-03-29T04:51:55.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.767+0000 7f43ce7fc700 1 -- 172.21.15.142:0/673473347 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43d4018630 con 0x7f43d8100de0 2024-03-29T04:51:55.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.767+0000 7f43df3c2700 1 -- 172.21.15.142:0/673473347 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d8196de0 con 0x7f43d8100de0 2024-03-29T04:51:55.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.768+0000 7f43df3c2700 1 -- 172.21.15.142:0/673473347 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d81091e0 con 0x7f43d8100de0 2024-03-29T04:51:55.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.769+0000 7f43ce7fc700 1 -- 172.21.15.142:0/673473347 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f43d40105a0 con 0x7f43d8100de0 2024-03-29T04:51:55.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.769+0000 7f43ce7fc700 1 --2- 172.21.15.142:0/673473347 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f43c406dd60 0x7f43c4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:55.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.769+0000 7f43ce7fc700 1 -- 172.21.15.142:0/673473347 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f43d4014070 con 0x7f43d8100de0 2024-03-29T04:51:55.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.769+0000 7f43dc95d700 1 --2- 172.21.15.142:0/673473347 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f43c406dd60 0x7f43c4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:55.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.770+0000 7f43dc95d700 1 --2- 172.21.15.142:0/673473347 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f43c406dd60 0x7f43c4070220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43c80059d0 tx=0x7f43c8005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:55.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:55.774+0000 7f43ce7fc700 1 -- 172.21.15.142:0/673473347 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f43d4053280 con 0x7f43d8100de0 2024-03-29T04:51:55.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:55 smithi142 ceph-mon[112475]: Upgrade: Updating osd.1 2024-03-29T04:51:55.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:55 smithi142 ceph-mon[112475]: Deploying daemon osd.1 on smithi142 2024-03-29T04:51:56.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:55 smithi152 ceph-mon[88186]: Upgrade: Updating osd.1 2024-03-29T04:51:56.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:55 smithi152 ceph-mon[88186]: Deploying daemon osd.1 on smithi142 2024-03-29T04:51:56.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.061+0000 7f43df3c2700 1 -- 172.21.15.142:0/673473347 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f43d804ea90 con 0x7f43d8100de0 2024-03-29T04:51:56.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.062+0000 7f43ce7fc700 1 -- 172.21.15.142:0/673473347 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f43d405aeb0 con 0x7f43d8100de0 2024-03-29T04:51:56.066 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:51:56.066 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:51:56.066 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 1 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:51:56.067 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-03-29T04:51:56.068 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 5 2024-03-29T04:51:56.068 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:51:56.068 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:51:56.068 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.064+0000 7f43df3c2700 1 -- 172.21.15.142:0/673473347 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f43c406dd60 msgr2=0x7f43c4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:56.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.064+0000 7f43df3c2700 1 --2- 172.21.15.142:0/673473347 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f43c406dd60 0x7f43c4070220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43c80059d0 tx=0x7f43c8005940 comp rx=0 tx=0).stop 2024-03-29T04:51:56.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.064+0000 7f43df3c2700 1 -- 172.21.15.142:0/673473347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d8100de0 msgr2=0x7f43d819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:56.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.064+0000 7f43df3c2700 1 --2- 172.21.15.142:0/673473347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d8100de0 0x7f43d819c560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f43d400e920 tx=0x7f43d400ec30 comp rx=0 tx=0).stop 2024-03-29T04:51:56.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.064+0000 7f43df3c2700 1 -- 172.21.15.142:0/673473347 shutdown_connections 2024-03-29T04:51:56.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.064+0000 7f43df3c2700 1 --2- 172.21.15.142:0/673473347 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f43c406dd60 0x7f43c4070220 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.064+0000 7f43df3c2700 1 --2- 172.21.15.142:0/673473347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43d8100de0 0x7f43d819c560 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.064+0000 7f43df3c2700 1 --2- 172.21.15.142:0/673473347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43d8101790 0x7f43d819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.064+0000 7f43df3c2700 1 -- 172.21.15.142:0/673473347 >> 172.21.15.142:0/673473347 conn(0x7f43d80fc670 msgr2=0x7f43d80fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:56.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.064+0000 7f43df3c2700 1 -- 172.21.15.142:0/673473347 shutdown_connections 2024-03-29T04:51:56.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.064+0000 7f43df3c2700 1 -- 172.21.15.142:0/673473347 wait complete. 2024-03-29T04:51:56.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.169+0000 7f6b90bda700 1 -- 172.21.15.142:0/1506713392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b8c101770 msgr2=0x7f6b8c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:56.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.169+0000 7f6b90bda700 1 --2- 172.21.15.142:0/1506713392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b8c101770 0x7f6b8c1057c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009910 tx=0x7f6b7c009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:56.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.169+0000 7f6b90bda700 1 -- 172.21.15.142:0/1506713392 shutdown_connections 2024-03-29T04:51:56.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.169+0000 7f6b90bda700 1 --2- 172.21.15.142:0/1506713392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b8c101770 0x7f6b8c1057c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.169+0000 7f6b90bda700 1 --2- 172.21.15.142:0/1506713392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b8c100dc0 0x7f6b8c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.169+0000 7f6b90bda700 1 -- 172.21.15.142:0/1506713392 >> 172.21.15.142:0/1506713392 conn(0x7f6b8c0fc650 msgr2=0x7f6b8c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:56.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.169+0000 7f6b90bda700 1 -- 172.21.15.142:0/1506713392 shutdown_connections 2024-03-29T04:51:56.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.169+0000 7f6b90bda700 1 -- 172.21.15.142:0/1506713392 wait complete. 2024-03-29T04:51:56.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.170+0000 7f6b90bda700 1 Processor -- start 2024-03-29T04:51:56.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.170+0000 7f6b90bda700 1 -- start start 2024-03-29T04:51:56.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.170+0000 7f6b90bda700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b8c100dc0 0x7f6b8c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:56.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.170+0000 7f6b90bda700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b8c101770 0x7f6b8c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:56.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.170+0000 7f6b90bda700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c19d130 con 0x7f6b8c101770 2024-03-29T04:51:56.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.170+0000 7f6b90bda700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c1965e0 con 0x7f6b8c100dc0 2024-03-29T04:51:56.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b89d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b8c101770 0x7f6b8c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:56.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b89d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b8c101770 0x7f6b8c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47596/0 (socket says 172.21.15.142:47596) 2024-03-29T04:51:56.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b89d9b700 1 -- 172.21.15.142:0/3235228699 learned_addr learned my addr 172.21.15.142:0/3235228699 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:56.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b8a59c700 1 --2- 172.21.15.142:0/3235228699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b8c100dc0 0x7f6b8c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:56.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b89d9b700 1 -- 172.21.15.142:0/3235228699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b8c100dc0 msgr2=0x7f6b8c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:56.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b89d9b700 1 --2- 172.21.15.142:0/3235228699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b8c100dc0 0x7f6b8c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b89d9b700 1 -- 172.21.15.142:0/3235228699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c0095f0 con 0x7f6b8c101770 2024-03-29T04:51:56.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b89d9b700 1 --2- 172.21.15.142:0/3235228699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b8c101770 0x7f6b8c19caa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00bd70 tx=0x7f6b7c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:56.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b7b7fe700 1 -- 172.21.15.142:0/3235228699 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b7c01d070 con 0x7f6b8c101770 2024-03-29T04:51:56.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b7b7fe700 1 -- 172.21.15.142:0/3235228699 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b7c005dc0 con 0x7f6b8c101770 2024-03-29T04:51:56.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b7b7fe700 1 -- 172.21.15.142:0/3235228699 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b7c017400 con 0x7f6b8c101770 2024-03-29T04:51:56.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b90bda700 1 -- 172.21.15.142:0/3235228699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8c196860 con 0x7f6b8c101770 2024-03-29T04:51:56.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.171+0000 7f6b90bda700 1 -- 172.21.15.142:0/3235228699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8c196d50 con 0x7f6b8c101770 2024-03-29T04:51:56.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.172+0000 7f6b90bda700 1 -- 172.21.15.142:0/3235228699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8c1091c0 con 0x7f6b8c101770 2024-03-29T04:51:56.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.173+0000 7f6b7b7fe700 1 -- 172.21.15.142:0/3235228699 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6b7c00f460 con 0x7f6b8c101770 2024-03-29T04:51:56.179 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.173+0000 7f6b7b7fe700 1 --2- 172.21.15.142:0/3235228699 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b7406dd60 0x7f6b74070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:56.179 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.173+0000 7f6b7b7fe700 1 -- 172.21.15.142:0/3235228699 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6b7c08ef40 con 0x7f6b8c101770 2024-03-29T04:51:56.179 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.173+0000 7f6b8a59c700 1 --2- 172.21.15.142:0/3235228699 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b7406dd60 0x7f6b74070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:56.180 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.174+0000 7f6b8a59c700 1 --2- 172.21.15.142:0/3235228699 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b7406dd60 0x7f6b74070220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b80005950 tx=0x7f6b8000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:56.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.178+0000 7f6b7b7fe700 1 -- 172.21.15.142:0/3235228699 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6b7c0c3280 con 0x7f6b8c101770 2024-03-29T04:51:56.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.425+0000 7f6b90bda700 1 -- 172.21.15.142:0/3235228699 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6b8c04ea90 con 0x7f6b8c101770 2024-03-29T04:51:56.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.425+0000 7f6b7b7fe700 1 -- 172.21.15.142:0/3235228699 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1906 (secure 0 0 0) 0x7f6b7c05c060 con 0x7f6b8c101770 2024-03-29T04:51:56.430 INFO:teuthology.orchestra.run.smithi142.stdout:e21 2024-03-29T04:51:56.430 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:51:56.430 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:51:56.430 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:51:56.430 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:51:56.430 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:51:56.430 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:51:56.430 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 21 2024-03-29T04:51:56.430 INFO:teuthology.orchestra.run.smithi142.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T04:51:56.431 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:51:56.431 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:49:01.288637+0000 2024-03-29T04:51:56.431 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:51:56.431 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:51:56.431 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:51:56.431 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:51:56.431 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:51:56.431 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:51:56.431 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:51:56.431 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:51:56.431 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 47 2024-03-29T04:51:56.431 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:51:56.432 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:51:56.432 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:51:56.432 INFO:teuthology.orchestra.run.smithi142.stdout:up {0=14494,1=24275} 2024-03-29T04:51:56.432 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T04:51:56.432 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:51:56.432 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:51:56.432 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:51:56.432 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:51:56.432 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:51:56.432 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:51:56.432 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:51:56.432 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:51:56.433 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:51:56.433 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{0:14494} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:51:56.433 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{0:14506} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:51:56.433 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{1:24275} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:51:56.433 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{1:24287} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:51:56.433 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:51:56.433 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:51:56.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.427+0000 7f6b90bda700 1 -- 172.21.15.142:0/3235228699 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b7406dd60 msgr2=0x7f6b74070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:56.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.427+0000 7f6b90bda700 1 --2- 172.21.15.142:0/3235228699 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b7406dd60 0x7f6b74070220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b80005950 tx=0x7f6b8000b410 comp rx=0 tx=0).stop 2024-03-29T04:51:56.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.427+0000 7f6b90bda700 1 -- 172.21.15.142:0/3235228699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b8c101770 msgr2=0x7f6b8c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:56.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.427+0000 7f6b90bda700 1 --2- 172.21.15.142:0/3235228699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b8c101770 0x7f6b8c19caa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00bd70 tx=0x7f6b7c00bda0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.427+0000 7f6b90bda700 1 -- 172.21.15.142:0/3235228699 shutdown_connections 2024-03-29T04:51:56.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.427+0000 7f6b90bda700 1 --2- 172.21.15.142:0/3235228699 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b7406dd60 0x7f6b74070220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.427+0000 7f6b90bda700 1 --2- 172.21.15.142:0/3235228699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b8c100dc0 0x7f6b8c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.427+0000 7f6b90bda700 1 --2- 172.21.15.142:0/3235228699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b8c101770 0x7f6b8c19caa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.427+0000 7f6b90bda700 1 -- 172.21.15.142:0/3235228699 >> 172.21.15.142:0/3235228699 conn(0x7f6b8c0fc650 msgr2=0x7f6b8c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:56.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.427+0000 7f6b90bda700 1 -- 172.21.15.142:0/3235228699 shutdown_connections 2024-03-29T04:51:56.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.427+0000 7f6b90bda700 1 -- 172.21.15.142:0/3235228699 wait complete. 2024-03-29T04:51:56.435 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 21 2024-03-29T04:51:56.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.534+0000 7f06db64c700 1 -- 172.21.15.142:0/4095058656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d40fecf0 msgr2=0x7f06d40ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:56.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.534+0000 7f06db64c700 1 --2- 172.21.15.142:0/4095058656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d40fecf0 0x7f06d40ff170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009910 tx=0x7f06c4009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:56.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.534+0000 7f06db64c700 1 -- 172.21.15.142:0/4095058656 shutdown_connections 2024-03-29T04:51:56.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.534+0000 7f06db64c700 1 --2- 172.21.15.142:0/4095058656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d40fecf0 0x7f06d40ff170 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.534+0000 7f06db64c700 1 --2- 172.21.15.142:0/4095058656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06d4106d80 0x7f06d4107160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.534+0000 7f06db64c700 1 -- 172.21.15.142:0/4095058656 >> 172.21.15.142:0/4095058656 conn(0x7f06d4075dd0 msgr2=0x7f06d40761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:56.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.534+0000 7f06db64c700 1 -- 172.21.15.142:0/4095058656 shutdown_connections 2024-03-29T04:51:56.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.534+0000 7f06db64c700 1 -- 172.21.15.142:0/4095058656 wait complete. 2024-03-29T04:51:56.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06db64c700 1 Processor -- start 2024-03-29T04:51:56.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06db64c700 1 -- start start 2024-03-29T04:51:56.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06db64c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06d40fecf0 0x7f06d41944a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:56.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06db64c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d4106d80 0x7f06d41949e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:56.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06db64c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d41950c0 con 0x7f06d4106d80 2024-03-29T04:51:56.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06db64c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d4198e50 con 0x7f06d40fecf0 2024-03-29T04:51:56.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06d8be7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d4106d80 0x7f06d41949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:56.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06d8be7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d4106d80 0x7f06d41949e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47616/0 (socket says 172.21.15.142:47616) 2024-03-29T04:51:56.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06d8be7700 1 -- 172.21.15.142:0/4048441516 learned_addr learned my addr 172.21.15.142:0/4048441516 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:56.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06d93e8700 1 --2- 172.21.15.142:0/4048441516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06d40fecf0 0x7f06d41944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:56.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06d8be7700 1 -- 172.21.15.142:0/4048441516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06d40fecf0 msgr2=0x7f06d41944a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:56.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06d8be7700 1 --2- 172.21.15.142:0/4048441516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06d40fecf0 0x7f06d41944a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06d8be7700 1 -- 172.21.15.142:0/4048441516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c40095f0 con 0x7f06d4106d80 2024-03-29T04:51:56.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06d8be7700 1 --2- 172.21.15.142:0/4048441516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d4106d80 0x7f06d41949e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f06c400bd70 tx=0x7f06c400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:56.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.535+0000 7f06ca7fc700 1 -- 172.21.15.142:0/4048441516 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06c4010ac0 con 0x7f06d4106d80 2024-03-29T04:51:56.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.536+0000 7f06ca7fc700 1 -- 172.21.15.142:0/4048441516 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06c4010c20 con 0x7f06d4106d80 2024-03-29T04:51:56.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.536+0000 7f06db64c700 1 -- 172.21.15.142:0/4048441516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d41990d0 con 0x7f06d4106d80 2024-03-29T04:51:56.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.536+0000 7f06ca7fc700 1 -- 172.21.15.142:0/4048441516 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06c4018810 con 0x7f06d4106d80 2024-03-29T04:51:56.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.536+0000 7f06db64c700 1 -- 172.21.15.142:0/4048441516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d41996a0 con 0x7f06d4106d80 2024-03-29T04:51:56.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.537+0000 7f06db64c700 1 -- 172.21.15.142:0/4048441516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d404ea90 con 0x7f06d4106d80 2024-03-29T04:51:56.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.537+0000 7f06ca7fc700 1 -- 172.21.15.142:0/4048441516 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f06c4003c40 con 0x7f06d4106d80 2024-03-29T04:51:56.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.537+0000 7f06ca7fc700 1 --2- 172.21.15.142:0/4048441516 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f06c006dd60 0x7f06c0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:56.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.537+0000 7f06ca7fc700 1 -- 172.21.15.142:0/4048441516 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f06c40935c0 con 0x7f06d4106d80 2024-03-29T04:51:56.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.537+0000 7f06d93e8700 1 --2- 172.21.15.142:0/4048441516 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f06c006dd60 0x7f06c0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:56.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.538+0000 7f06d93e8700 1 --2- 172.21.15.142:0/4048441516 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f06c006dd60 0x7f06c0070220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f06d0005950 tx=0x7f06d000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:56.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.542+0000 7f06ca7fc700 1 -- 172.21.15.142:0/4048441516 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f06c405cbb0 con 0x7f06d4106d80 2024-03-29T04:51:56.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.769+0000 7f06db64c700 1 -- 172.21.15.142:0/4048441516 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06d4199a60 con 0x7f06c006dd60 2024-03-29T04:51:56.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.771+0000 7f06ca7fc700 1 -- 172.21.15.142:0/4048441516 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f06d4199a60 con 0x7f06c006dd60 2024-03-29T04:51:56.775 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:51:56.775 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:51:56.776 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:51:56.776 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:51:56.776 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:51:56.776 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:51:56.776 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:51:56.776 INFO:teuthology.orchestra.run.smithi142.stdout: "mon" 2024-03-29T04:51:56.776 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:51:56.776 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "7/23 daemons upgraded", 2024-03-29T04:51:56.776 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading osd daemons", 2024-03-29T04:51:56.776 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:51:56.776 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:51:56.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.773+0000 7f06db64c700 1 -- 172.21.15.142:0/4048441516 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f06c006dd60 msgr2=0x7f06c0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:56.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.773+0000 7f06db64c700 1 --2- 172.21.15.142:0/4048441516 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f06c006dd60 0x7f06c0070220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f06d0005950 tx=0x7f06d000b410 comp rx=0 tx=0).stop 2024-03-29T04:51:56.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.773+0000 7f06db64c700 1 -- 172.21.15.142:0/4048441516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d4106d80 msgr2=0x7f06d41949e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:56.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.773+0000 7f06db64c700 1 --2- 172.21.15.142:0/4048441516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d4106d80 0x7f06d41949e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f06c400bd70 tx=0x7f06c400bda0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.773+0000 7f06db64c700 1 -- 172.21.15.142:0/4048441516 shutdown_connections 2024-03-29T04:51:56.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.773+0000 7f06db64c700 1 --2- 172.21.15.142:0/4048441516 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f06c006dd60 0x7f06c0070220 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.773+0000 7f06db64c700 1 --2- 172.21.15.142:0/4048441516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06d40fecf0 0x7f06d41944a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.773+0000 7f06db64c700 1 --2- 172.21.15.142:0/4048441516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06d4106d80 0x7f06d41949e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.773+0000 7f06db64c700 1 -- 172.21.15.142:0/4048441516 >> 172.21.15.142:0/4048441516 conn(0x7f06d4075dd0 msgr2=0x7f06d40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:56.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.773+0000 7f06db64c700 1 -- 172.21.15.142:0/4048441516 shutdown_connections 2024-03-29T04:51:56.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.773+0000 7f06db64c700 1 -- 172.21.15.142:0/4048441516 wait complete. 2024-03-29T04:51:56.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:56 smithi142 ceph-mon[112475]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:56.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:56 smithi142 ceph-mon[112475]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:56.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:56 smithi142 ceph-mon[112475]: pgmap v66: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:56.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:56 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/673473347' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:56.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:56 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3235228699' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:51:56.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.885+0000 7f621bc12700 1 -- 172.21.15.142:0/1066756628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62140ffa60 msgr2=0x7f62140ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:56.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.885+0000 7f621bc12700 1 --2- 172.21.15.142:0/1066756628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62140ffa60 0x7f62140ffee0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6204009910 tx=0x7f6204009c20 comp rx=0 tx=0).stop 2024-03-29T04:51:56.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.885+0000 7f621bc12700 1 -- 172.21.15.142:0/1066756628 shutdown_connections 2024-03-29T04:51:56.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.885+0000 7f621bc12700 1 --2- 172.21.15.142:0/1066756628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62140ffa60 0x7f62140ffee0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.885+0000 7f621bc12700 1 --2- 172.21.15.142:0/1066756628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6214105a60 0x7f6214105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.885+0000 7f621bc12700 1 -- 172.21.15.142:0/1066756628 >> 172.21.15.142:0/1066756628 conn(0x7f62140fb550 msgr2=0x7f62140fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:56.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.885+0000 7f621bc12700 1 -- 172.21.15.142:0/1066756628 shutdown_connections 2024-03-29T04:51:56.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.885+0000 7f621bc12700 1 -- 172.21.15.142:0/1066756628 wait complete. 2024-03-29T04:51:56.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f621bc12700 1 Processor -- start 2024-03-29T04:51:56.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f621bc12700 1 -- start start 2024-03-29T04:51:56.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f621bc12700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62140ffa60 0x7f6214197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:56.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f621bc12700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6214105a60 0x7f6214197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:56.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f621bc12700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6214198190 con 0x7f62140ffa60 2024-03-29T04:51:56.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f621bc12700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62141982d0 con 0x7f6214105a60 2024-03-29T04:51:56.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f62199ae700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62140ffa60 0x7f6214197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:56.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f62199ae700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62140ffa60 0x7f6214197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47634/0 (socket says 172.21.15.142:47634) 2024-03-29T04:51:56.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f62199ae700 1 -- 172.21.15.142:0/2297797357 learned_addr learned my addr 172.21.15.142:0/2297797357 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:51:56.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f62191ad700 1 --2- 172.21.15.142:0/2297797357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6214105a60 0x7f6214197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:56.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f62199ae700 1 -- 172.21.15.142:0/2297797357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6214105a60 msgr2=0x7f6214197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:56.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f62199ae700 1 --2- 172.21.15.142:0/2297797357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6214105a60 0x7f6214197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:56.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.886+0000 7f62199ae700 1 -- 172.21.15.142:0/2297797357 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62040095f0 con 0x7f62140ffa60 2024-03-29T04:51:56.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.887+0000 7f62199ae700 1 --2- 172.21.15.142:0/2297797357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62140ffa60 0x7f6214197570 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f621000d8d0 tx=0x7f621000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:56.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.887+0000 7f620affd700 1 -- 172.21.15.142:0/2297797357 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6210009940 con 0x7f62140ffa60 2024-03-29T04:51:56.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.887+0000 7f621bc12700 1 -- 172.21.15.142:0/2297797357 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62141a05e0 con 0x7f62140ffa60 2024-03-29T04:51:56.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.887+0000 7f620affd700 1 -- 172.21.15.142:0/2297797357 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6210010460 con 0x7f62140ffa60 2024-03-29T04:51:56.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.887+0000 7f620affd700 1 -- 172.21.15.142:0/2297797357 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f621000f5f0 con 0x7f62140ffa60 2024-03-29T04:51:56.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.887+0000 7f621bc12700 1 -- 172.21.15.142:0/2297797357 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62141a0b00 con 0x7f62140ffa60 2024-03-29T04:51:56.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.888+0000 7f621bc12700 1 -- 172.21.15.142:0/2297797357 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6214107ff0 con 0x7f62140ffa60 2024-03-29T04:51:56.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.888+0000 7f620affd700 1 -- 172.21.15.142:0/2297797357 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f62100105d0 con 0x7f62140ffa60 2024-03-29T04:51:56.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.889+0000 7f620affd700 1 --2- 172.21.15.142:0/2297797357 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f620006dd60 0x7f6200070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:51:56.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.889+0000 7f620affd700 1 -- 172.21.15.142:0/2297797357 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f621008db00 con 0x7f62140ffa60 2024-03-29T04:51:56.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.889+0000 7f62191ad700 1 --2- 172.21.15.142:0/2297797357 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f620006dd60 0x7f6200070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:51:56.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.889+0000 7f62191ad700 1 --2- 172.21.15.142:0/2297797357 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f620006dd60 0x7f6200070220 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f620400b5c0 tx=0x7f6204005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:51:56.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:56.898+0000 7f620affd700 1 -- 172.21.15.142:0/2297797357 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f62100570f0 con 0x7f62140ffa60 2024-03-29T04:51:57.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:56 smithi152 ceph-mon[88186]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:57.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:56 smithi152 ceph-mon[88186]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:57.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:56 smithi152 ceph-mon[88186]: pgmap v66: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:51:57.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:56 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/673473347' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:51:57.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:56 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3235228699' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:51:57.228 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:57 smithi142 systemd[1]: Stopping Ceph osd.1 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:51:57.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.223+0000 7f621bc12700 1 -- 172.21.15.142:0/2297797357 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f621402cd10 con 0x7f62140ffa60 2024-03-29T04:51:57.229 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_OK 2024-03-29T04:51:57.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.223+0000 7f620affd700 1 -- 172.21.15.142:0/2297797357 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f621005a7b0 con 0x7f62140ffa60 2024-03-29T04:51:57.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.226+0000 7f621bc12700 1 -- 172.21.15.142:0/2297797357 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f620006dd60 msgr2=0x7f6200070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:57.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.226+0000 7f621bc12700 1 --2- 172.21.15.142:0/2297797357 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f620006dd60 0x7f6200070220 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f620400b5c0 tx=0x7f6204005960 comp rx=0 tx=0).stop 2024-03-29T04:51:57.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.226+0000 7f621bc12700 1 -- 172.21.15.142:0/2297797357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62140ffa60 msgr2=0x7f6214197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:51:57.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.226+0000 7f621bc12700 1 --2- 172.21.15.142:0/2297797357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62140ffa60 0x7f6214197570 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f621000d8d0 tx=0x7f621000dc90 comp rx=0 tx=0).stop 2024-03-29T04:51:57.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.226+0000 7f621bc12700 1 -- 172.21.15.142:0/2297797357 shutdown_connections 2024-03-29T04:51:57.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.226+0000 7f621bc12700 1 --2- 172.21.15.142:0/2297797357 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f620006dd60 0x7f6200070220 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:57.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.226+0000 7f621bc12700 1 --2- 172.21.15.142:0/2297797357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62140ffa60 0x7f6214197570 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:57.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.226+0000 7f621bc12700 1 --2- 172.21.15.142:0/2297797357 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6214105a60 0x7f6214197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:51:57.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.226+0000 7f621bc12700 1 -- 172.21.15.142:0/2297797357 >> 172.21.15.142:0/2297797357 conn(0x7f62140fb550 msgr2=0x7f62140fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:51:57.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.226+0000 7f621bc12700 1 -- 172.21.15.142:0/2297797357 shutdown_connections 2024-03-29T04:51:57.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:51:57.226+0000 7f621bc12700 1 -- 172.21.15.142:0/2297797357 wait complete. 2024-03-29T04:51:57.624 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:57 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[55551]: 2024-03-29T04:51:57.354+0000 7fd435aec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T04:51:57.625 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:57 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[55551]: 2024-03-29T04:51:57.354+0000 7fd435aec700 -1 osd.1 56 *** Got signal Terminated *** 2024-03-29T04:51:57.625 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:57 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[55551]: 2024-03-29T04:51:57.354+0000 7fd435aec700 -1 osd.1 56 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T04:51:57.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:57 smithi142 ceph-mon[112475]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:57.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:57 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2297797357' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:51:57.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:57 smithi142 ceph-mon[112475]: osd.1 marked itself down and dead 2024-03-29T04:51:58.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:57 smithi152 ceph-mon[88186]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:58.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:57 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2297797357' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:51:58.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:57 smithi152 ceph-mon[88186]: osd.1 marked itself down and dead 2024-03-29T04:51:58.240 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:58 smithi142 podman[129241]: 2024-03-29 04:51:58.161415277 +0000 UTC m=+0.964250001 container died 539c2cbf33ac775967c4353e1b93f737082d1297da220ced7f83208445735c79 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-29T04:51:58.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:58 smithi142 ceph-mon[112475]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:58.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:58 smithi142 ceph-mon[112475]: pgmap v67: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:58.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:58 smithi142 ceph-mon[112475]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T04:51:58.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:58 smithi142 ceph-mon[112475]: osdmap e57: 6 total, 5 up, 6 in 2024-03-29T04:51:58.876 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:58 smithi142 podman[129241]: 2024-03-29 04:51:58.570709322 +0000 UTC m=+1.373544052 container cleanup 539c2cbf33ac775967c4353e1b93f737082d1297da220ced7f83208445735c79 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-03-29T04:51:58.876 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:58 smithi142 bash[129241]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1 2024-03-29T04:51:58.876 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:58 smithi142 podman[129261]: 2024-03-29 04:51:58.814271824 +0000 UTC m=+0.655122364 container remove 539c2cbf33ac775967c4353e1b93f737082d1297da220ced7f83208445735c79 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1, org.label-schema.build-date=20240222, GIT_CLEAN=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True) 2024-03-29T04:51:59.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:58 smithi152 ceph-mon[88186]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:51:59.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:58 smithi152 ceph-mon[88186]: pgmap v67: 65 pgs: 65 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:51:59.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:58 smithi152 ceph-mon[88186]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T04:51:59.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:58 smithi152 ceph-mon[88186]: osdmap e57: 6 total, 5 up, 6 in 2024-03-29T04:51:59.624 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:59 smithi142 podman[129364]: 2024-03-29 04:51:59.18435676 +0000 UTC m=+0.018331130 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:51:59.624 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:59 smithi142 podman[129364]: 2024-03-29 04:51:59.37561967 +0000 UTC m=+0.209594028 container create 1c94f07a3d6e23b98d9ea7128daa0bd26b2900f83e7aacd9664370cca600fe2f (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-03-29T04:51:59.875 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:59 smithi142 podman[129364]: 2024-03-29 04:51:59.751827731 +0000 UTC m=+0.585802080 container init 1c94f07a3d6e23b98d9ea7128daa0bd26b2900f83e7aacd9664370cca600fe2f (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-8da2362, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-29T04:51:59.876 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:59 smithi142 podman[129364]: 2024-03-29 04:51:59.756417803 +0000 UTC m=+0.590392159 container start 1c94f07a3d6e23b98d9ea7128daa0bd26b2900f83e7aacd9664370cca600fe2f (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8da2362, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326) 2024-03-29T04:51:59.876 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:51:59 smithi142 podman[129364]: 2024-03-29 04:51:59.851887011 +0000 UTC m=+0.685861381 container attach 1c94f07a3d6e23b98d9ea7128daa0bd26b2900f83e7aacd9664370cca600fe2f (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-29T04:51:59.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: osdmap e58: 6 total, 5 up, 6 in 2024-03-29T04:51:59.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [0, 2]}]: dispatch 2024-03-29T04:51:59.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 2]}]: dispatch 2024-03-29T04:51:59.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [3, 1]}]: dispatch 2024-03-29T04:51:59.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 1]}]: dispatch 2024-03-29T04:51:59.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 1]}]: dispatch 2024-03-29T04:51:59.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]: dispatch 2024-03-29T04:51:59.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [3, 1]}]: dispatch 2024-03-29T04:51:59.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [3, 1]}]: dispatch 2024-03-29T04:51:59.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 0]}]: dispatch 2024-03-29T04:51:59.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 2]}]: dispatch 2024-03-29T04:51:59.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:51:59.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:51:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:52:00.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: osdmap e58: 6 total, 5 up, 6 in 2024-03-29T04:52:00.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [0, 2]}]: dispatch 2024-03-29T04:52:00.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 2]}]: dispatch 2024-03-29T04:52:00.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [3, 1]}]: dispatch 2024-03-29T04:52:00.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 1]}]: dispatch 2024-03-29T04:52:00.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 1]}]: dispatch 2024-03-29T04:52:00.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]: dispatch 2024-03-29T04:52:00.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [3, 1]}]: dispatch 2024-03-29T04:52:00.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [3, 1]}]: dispatch 2024-03-29T04:52:00.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 0]}]: dispatch 2024-03-29T04:52:00.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 2]}]: dispatch 2024-03-29T04:52:00.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:00.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:51:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:52:00.375 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:00 smithi142 podman[129364]: 2024-03-29 04:52:00.047655414 +0000 UTC m=+0.881629772 container died 1c94f07a3d6e23b98d9ea7128daa0bd26b2900f83e7aacd9664370cca600fe2f (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_BRANCH=HEAD, RELEASE=squid-8da2362, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-29T04:52:00.642 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:00 smithi142 podman[129390]: 2024-03-29 04:52:00.605635345 +0000 UTC m=+0.552823873 container remove 1c94f07a3d6e23b98d9ea7128daa0bd26b2900f83e7aacd9664370cca600fe2f (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-8da2362, ceph=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-29T04:52:00.643 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:00 smithi142 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-29T04:52:00.643 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:00 smithi142 systemd[1]: Stopped Ceph osd.1 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:52:00.643 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:00 smithi142 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.1.service: Consumed 1min 10.642s CPU time. 2024-03-29T04:52:01.044 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:00 smithi142 systemd[1]: Starting Ceph osd.1 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:52:01.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:00 smithi142 ceph-mon[112475]: pgmap v70: 65 pgs: 12 stale+active+clean, 53 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-29T04:52:01.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [0, 2]}]': finished 2024-03-29T04:52:01.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 2]}]': finished 2024-03-29T04:52:01.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [3, 1]}]': finished 2024-03-29T04:52:01.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 1]}]': finished 2024-03-29T04:52:01.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 1]}]': finished 2024-03-29T04:52:01.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]': finished 2024-03-29T04:52:01.045 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [3, 1]}]': finished 2024-03-29T04:52:01.045 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [3, 1]}]': finished 2024-03-29T04:52:01.045 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 0]}]': finished 2024-03-29T04:52:01.045 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 2]}]': finished 2024-03-29T04:52:01.045 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:00 smithi142 ceph-mon[112475]: osdmap e59: 6 total, 5 up, 6 in 2024-03-29T04:52:01.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:00 smithi152 ceph-mon[88186]: pgmap v70: 65 pgs: 12 stale+active+clean, 53 active+clean; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-29T04:52:01.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [0, 2]}]': finished 2024-03-29T04:52:01.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [0, 2]}]': finished 2024-03-29T04:52:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [3, 1]}]': finished 2024-03-29T04:52:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 1]}]': finished 2024-03-29T04:52:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 1]}]': finished 2024-03-29T04:52:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]': finished 2024-03-29T04:52:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.f", "id": [3, 1]}]': finished 2024-03-29T04:52:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [3, 1]}]': finished 2024-03-29T04:52:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 0]}]': finished 2024-03-29T04:52:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 2]}]': finished 2024-03-29T04:52:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:00 smithi152 ceph-mon[88186]: osdmap e59: 6 total, 5 up, 6 in 2024-03-29T04:52:01.330 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 podman[129488]: 2024-03-29 04:52:01.040238375 +0000 UTC m=+0.096066347 container create 1635a80be196326f6110e08f3a9b13507d3648865054e061e95a50327124f8bc (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate, RELEASE=squid-8da2362, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-29T04:52:01.330 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 podman[129488]: 2024-03-29 04:52:00.962136246 +0000 UTC m=+0.017964225 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:52:01.330 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 podman[129488]: 2024-03-29 04:52:01.251771069 +0000 UTC m=+0.307599055 container init 1635a80be196326f6110e08f3a9b13507d3648865054e061e95a50327124f8bc (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate, ceph=True, RELEASE=squid-8da2362, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-29T04:52:01.330 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 podman[129488]: 2024-03-29 04:52:01.255103927 +0000 UTC m=+0.310931904 container start 1635a80be196326f6110e08f3a9b13507d3648865054e061e95a50327124f8bc (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-29T04:52:01.330 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 podman[129488]: 2024-03-29 04:52:01.326179331 +0000 UTC m=+0.382007317 container attach 1635a80be196326f6110e08f3a9b13507d3648865054e061e95a50327124f8bc (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-29T04:52:02.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:01 smithi152 ceph-mon[88186]: osdmap e60: 6 total, 5 up, 6 in 2024-03-29T04:52:02.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:01 smithi152 ceph-mon[88186]: pgmap v73: 65 pgs: 5 unknown, 4 peering, 13 active+undersized+degraded, 2 stale+active+clean, 13 active+undersized, 28 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41/327 objects degraded (12.538%) 2024-03-29T04:52:02.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:01 smithi152 ceph-mon[88186]: osdmap e61: 6 total, 5 up, 6 in 2024-03-29T04:52:02.058 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:01 smithi142 ceph-mon[112475]: osdmap e60: 6 total, 5 up, 6 in 2024-03-29T04:52:02.058 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:01 smithi142 ceph-mon[112475]: pgmap v73: 65 pgs: 5 unknown, 4 peering, 13 active+undersized+degraded, 2 stale+active+clean, 13 active+undersized, 28 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41/327 objects degraded (12.538%) 2024-03-29T04:52:02.058 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:01 smithi142 ceph-mon[112475]: osdmap e61: 6 total, 5 up, 6 in 2024-03-29T04:52:02.059 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate[129503]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-29T04:52:02.059 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate[129503]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-29T04:52:02.059 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 bash[129488]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-29T04:52:02.059 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 bash[129488]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-29T04:52:02.059 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 bash[129488]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-29T04:52:02.059 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 bash[129488]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-29T04:52:02.059 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 bash[129488]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-29T04:52:02.059 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 bash[129488]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-29T04:52:02.059 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 bash[129488]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-29T04:52:02.059 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate[129503]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-29T04:52:02.060 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate[129503]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-29T04:52:02.060 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate[129503]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-29T04:52:02.060 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate[129503]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-29T04:52:02.060 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate[129503]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-29T04:52:02.060 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:01 smithi142 podman[129488]: 2024-03-29 04:52:01.751829826 +0000 UTC m=+0.807657804 container died 1635a80be196326f6110e08f3a9b13507d3648865054e061e95a50327124f8bc (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate, io.buildah.version=1.33.5, RELEASE=squid-8da2362, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:52:02.837 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:02 smithi142 ceph-mon[112475]: Health check failed: Degraded data redundancy: 41/327 objects degraded (12.538%), 13 pgs degraded (PG_DEGRADED) 2024-03-29T04:52:02.838 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:02 smithi142 podman[129632]: 2024-03-29 04:52:02.581570601 +0000 UTC m=+0.824695719 container remove 1635a80be196326f6110e08f3a9b13507d3648865054e061e95a50327124f8bc (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-8da2362, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.schema-version=1.0) 2024-03-29T04:52:02.838 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:02 smithi142 podman[129693]: 2024-03-29 04:52:02.711392348 +0000 UTC m=+0.021530064 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:52:02.838 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:02 smithi142 podman[129693]: 2024-03-29 04:52:02.832810552 +0000 UTC m=+0.142948228 container create fadc1526c831c6a43697478b628fd39fc116361a09aeb388a9dd8ac14c1dc3fd (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=squid-8da2362, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6) 2024-03-29T04:52:03.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:02 smithi152 ceph-mon[88186]: Health check failed: Degraded data redundancy: 41/327 objects degraded (12.538%), 13 pgs degraded (PG_DEGRADED) 2024-03-29T04:52:03.107 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:03 smithi142 podman[129693]: 2024-03-29 04:52:03.077413732 +0000 UTC m=+0.387551428 container init fadc1526c831c6a43697478b628fd39fc116361a09aeb388a9dd8ac14c1dc3fd (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:52:03.107 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:03 smithi142 podman[129693]: 2024-03-29 04:52:03.080887173 +0000 UTC m=+0.391024861 container start fadc1526c831c6a43697478b628fd39fc116361a09aeb388a9dd8ac14c1dc3fd (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, RELEASE=squid-8da2362, GIT_BRANCH=HEAD) 2024-03-29T04:52:03.375 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:03 smithi142 ceph-osd[129711]: -- 172.21.15.142:0/3864965820 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x5643c32abba0 con 0x5643c327cc00 2024-03-29T04:52:03.376 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:03 smithi142 ceph-osd[129711]: -- 172.21.15.142:0/3864965820 <== mon.0 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5643c32b3680 con 0x5643c327cc00 2024-03-29T04:52:03.376 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:03 smithi142 bash[129693]: fadc1526c831c6a43697478b628fd39fc116361a09aeb388a9dd8ac14c1dc3fd 2024-03-29T04:52:03.376 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:03 smithi142 systemd[1]: Started Ceph osd.1 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:52:04.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:03 smithi152 ceph-mon[88186]: osdmap e62: 6 total, 5 up, 6 in 2024-03-29T04:52:04.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:03 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:04.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:03 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:04.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:03 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:52:04.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:03 smithi152 ceph-mon[88186]: pgmap v76: 65 pgs: 26 active+clean, 16 active+undersized, 1 active+undersized+remapped, 14 active+undersized+degraded, 4 peering, 4 unknown; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/333 objects degraded (13.514%); 2/333 objects misplaced (0.601%); 2.0 MiB/s, 0 objects/s recovering 2024-03-29T04:52:04.126 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:03 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[129707]: 2024-03-29T04:52:03.629+0000 7fd21a9297c0 -1 Falling back to public interface 2024-03-29T04:52:04.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:03 smithi142 ceph-mon[112475]: osdmap e62: 6 total, 5 up, 6 in 2024-03-29T04:52:04.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:03 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:04.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:03 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:04.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:03 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:52:04.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:03 smithi142 ceph-mon[112475]: pgmap v76: 65 pgs: 26 active+clean, 16 active+undersized, 1 active+undersized+remapped, 14 active+undersized+degraded, 4 peering, 4 unknown; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/333 objects degraded (13.514%); 2/333 objects misplaced (0.601%); 2.0 MiB/s, 0 objects/s recovering 2024-03-29T04:52:05.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:04 smithi152 ceph-mon[88186]: osdmap e63: 6 total, 5 up, 6 in 2024-03-29T04:52:05.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:04 smithi152 ceph-mon[88186]: osdmap e64: 6 total, 5 up, 6 in 2024-03-29T04:52:05.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:04 smithi142 ceph-mon[112475]: osdmap e63: 6 total, 5 up, 6 in 2024-03-29T04:52:05.072 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:04 smithi142 ceph-mon[112475]: osdmap e64: 6 total, 5 up, 6 in 2024-03-29T04:52:05.982 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:05 smithi142 ceph-mon[112475]: pgmap v79: 65 pgs: 1 remapped+peering, 8 active+clean+remapped, 26 active+clean, 16 active+undersized, 14 active+undersized+degraded; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 131 op/s; 45/333 objects degraded (13.514%); 26/333 objects misplaced (7.808%); 7.0 MiB/s, 181 objects/s recovering 2024-03-29T04:52:06.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:05 smithi152 ceph-mon[88186]: pgmap v79: 65 pgs: 1 remapped+peering, 8 active+clean+remapped, 26 active+clean, 16 active+undersized, 14 active+undersized+degraded; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 131 op/s; 45/333 objects degraded (13.514%); 26/333 objects misplaced (7.808%); 7.0 MiB/s, 181 objects/s recovering 2024-03-29T04:52:06.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:06 smithi142 ceph-mon[112475]: osdmap e65: 6 total, 5 up, 6 in 2024-03-29T04:52:07.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:06 smithi152 ceph-mon[88186]: osdmap e65: 6 total, 5 up, 6 in 2024-03-29T04:52:08.020 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:07 smithi142 ceph-mon[112475]: pgmap v81: 65 pgs: 1 remapped+peering, 8 active+clean+remapped, 26 active+clean, 16 active+undersized, 14 active+undersized+degraded; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s; 45/333 objects degraded (13.514%); 26/333 objects misplaced (7.808%); 4.1 MiB/s, 148 objects/s recovering 2024-03-29T04:52:08.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:07 smithi152 ceph-mon[88186]: pgmap v81: 65 pgs: 1 remapped+peering, 8 active+clean+remapped, 26 active+clean, 16 active+undersized, 14 active+undersized+degraded; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s; 45/333 objects degraded (13.514%); 26/333 objects misplaced (7.808%); 4.1 MiB/s, 148 objects/s recovering 2024-03-29T04:52:08.376 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:08 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[129707]: 2024-03-29T04:52:08.174+0000 7fd21a9297c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-29T04:52:08.964 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:08 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[129707]: 2024-03-29T04:52:08.678+0000 7fd21a9297c0 -1 osd.1 56 log_to_monitors true 2024-03-29T04:52:08.964 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:08 smithi142 ceph-mon[112475]: from='osd.1 [v2:172.21.15.142:6810/296003948,v1:172.21.15.142:6811/296003948]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-29T04:52:09.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:08 smithi152 ceph-mon[88186]: from='osd.1 [v2:172.21.15.142:6810/296003948,v1:172.21.15.142:6811/296003948]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-29T04:52:09.876 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 04:52:09 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[129707]: 2024-03-29T04:52:09.549+0000 7fd207d79700 -1 osd.1 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:52:09.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:09 smithi142 ceph-mon[112475]: from='osd.1 [v2:172.21.15.142:6810/296003948,v1:172.21.15.142:6811/296003948]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-29T04:52:09.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:09 smithi142 ceph-mon[112475]: osdmap e66: 6 total, 5 up, 6 in 2024-03-29T04:52:09.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:09 smithi142 ceph-mon[112475]: from='osd.1 [v2:172.21.15.142:6810/296003948,v1:172.21.15.142:6811/296003948]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-29T04:52:09.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:09 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:09.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:09 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:09.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:09 smithi142 ceph-mon[112475]: pgmap v83: 65 pgs: 1 remapped+peering, 9 active+clean+remapped, 25 active+clean, 16 active+undersized, 14 active+undersized+degraded; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 45/333 objects degraded (13.514%); 26/333 objects misplaced (7.808%); 3.4 MiB/s, 241 objects/s recovering 2024-03-29T04:52:09.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:09 smithi142 ceph-mon[112475]: from='osd.1 [v2:172.21.15.142:6810/296003948,v1:172.21.15.142:6811/296003948]' entity='osd.1' 2024-03-29T04:52:10.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:09 smithi152 ceph-mon[88186]: from='osd.1 [v2:172.21.15.142:6810/296003948,v1:172.21.15.142:6811/296003948]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-29T04:52:10.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:09 smithi152 ceph-mon[88186]: osdmap e66: 6 total, 5 up, 6 in 2024-03-29T04:52:10.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:09 smithi152 ceph-mon[88186]: from='osd.1 [v2:172.21.15.142:6810/296003948,v1:172.21.15.142:6811/296003948]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-29T04:52:10.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:09 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:10.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:09 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:10.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:09 smithi152 ceph-mon[88186]: pgmap v83: 65 pgs: 1 remapped+peering, 9 active+clean+remapped, 25 active+clean, 16 active+undersized, 14 active+undersized+degraded; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 45/333 objects degraded (13.514%); 26/333 objects misplaced (7.808%); 3.4 MiB/s, 241 objects/s recovering 2024-03-29T04:52:10.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:09 smithi152 ceph-mon[88186]: from='osd.1 [v2:172.21.15.142:6810/296003948,v1:172.21.15.142:6811/296003948]' entity='osd.1' 2024-03-29T04:52:11.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:11 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:11.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:11 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:11.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:11 smithi142 ceph-mon[112475]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T04:52:11.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:11 smithi142 ceph-mon[112475]: osd.1 [v2:172.21.15.142:6810/296003948,v1:172.21.15.142:6811/296003948] boot 2024-03-29T04:52:11.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:11 smithi142 ceph-mon[112475]: osdmap e67: 6 total, 6 up, 6 in 2024-03-29T04:52:11.376 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:11 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:52:11.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:11 smithi142 ceph-mon[112475]: Health check update: Degraded data redundancy: 45/333 objects degraded (13.514%), 14 pgs degraded (PG_DEGRADED) 2024-03-29T04:52:11.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:11 smithi142 ceph-mon[112475]: osdmap e68: 6 total, 6 up, 6 in 2024-03-29T04:52:11.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:11 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:11.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:11 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:11.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:11 smithi152 ceph-mon[88186]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T04:52:11.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:11 smithi152 ceph-mon[88186]: osd.1 [v2:172.21.15.142:6810/296003948,v1:172.21.15.142:6811/296003948] boot 2024-03-29T04:52:11.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:11 smithi152 ceph-mon[88186]: osdmap e67: 6 total, 6 up, 6 in 2024-03-29T04:52:11.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:11 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T04:52:11.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:11 smithi152 ceph-mon[88186]: Health check update: Degraded data redundancy: 45/333 objects degraded (13.514%), 14 pgs degraded (PG_DEGRADED) 2024-03-29T04:52:11.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:11 smithi152 ceph-mon[88186]: osdmap e68: 6 total, 6 up, 6 in 2024-03-29T04:52:12.544 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:12 smithi152 ceph-mon[88186]: pgmap v86: 65 pgs: 10 active+clean+remapped, 25 active+clean, 16 active+undersized, 14 active+undersized+degraded; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 45/333 objects degraded (13.514%); 26/333 objects misplaced (7.808%); 0 B/s, 118 objects/s recovering 2024-03-29T04:52:12.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:12 smithi152 ceph-mon[88186]: osdmap e69: 6 total, 6 up, 6 in 2024-03-29T04:52:12.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:12 smithi142 ceph-mon[112475]: pgmap v86: 65 pgs: 10 active+clean+remapped, 25 active+clean, 16 active+undersized, 14 active+undersized+degraded; 270 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 45/333 objects degraded (13.514%); 26/333 objects misplaced (7.808%); 0 B/s, 118 objects/s recovering 2024-03-29T04:52:12.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:12 smithi142 ceph-mon[112475]: osdmap e69: 6 total, 6 up, 6 in 2024-03-29T04:52:14.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:14 smithi142 ceph-mon[112475]: osdmap e70: 6 total, 6 up, 6 in 2024-03-29T04:52:14.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:14 smithi142 ceph-mon[112475]: pgmap v89: 65 pgs: 9 active+clean+remapped, 28 active+clean, 14 active+undersized, 14 active+undersized+degraded; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 45/333 objects degraded (13.514%); 26/333 objects misplaced (7.808%) 2024-03-29T04:52:14.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:14.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:14.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:52:14.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:52:14.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:14.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:52:14.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:14.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:14.487 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:14.488 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T04:52:14.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:14 smithi152 ceph-mon[88186]: osdmap e70: 6 total, 6 up, 6 in 2024-03-29T04:52:14.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:14 smithi152 ceph-mon[88186]: pgmap v89: 65 pgs: 9 active+clean+remapped, 28 active+clean, 14 active+undersized, 14 active+undersized+degraded; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 45/333 objects degraded (13.514%); 26/333 objects misplaced (7.808%) 2024-03-29T04:52:14.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:14.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:14.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:52:14.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:52:14.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:14.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:52:14.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:14.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:14.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:14.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T04:52:15.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:15 smithi152 ceph-mon[88186]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T04:52:15.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:15 smithi152 ceph-mon[88186]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-29T04:52:15.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:15 smithi152 ceph-mon[88186]: osdmap e71: 6 total, 6 up, 6 in 2024-03-29T04:52:15.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:15.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:52:15.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:15 smithi142 ceph-mon[112475]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T04:52:15.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:15 smithi142 ceph-mon[112475]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-29T04:52:15.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:15 smithi142 ceph-mon[112475]: osdmap e71: 6 total, 6 up, 6 in 2024-03-29T04:52:15.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:15.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:52:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:16 smithi152 ceph-mon[88186]: osdmap e72: 6 total, 6 up, 6 in 2024-03-29T04:52:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:16 smithi152 ceph-mon[88186]: pgmap v92: 65 pgs: 3 peering, 1 active+clean+remapped, 61 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 12 MiB/s, 185 objects/s recovering 2024-03-29T04:52:16.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:16 smithi152 ceph-mon[88186]: osdmap e73: 6 total, 6 up, 6 in 2024-03-29T04:52:16.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:16 smithi142 ceph-mon[112475]: osdmap e72: 6 total, 6 up, 6 in 2024-03-29T04:52:16.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:16 smithi142 ceph-mon[112475]: pgmap v92: 65 pgs: 3 peering, 1 active+clean+remapped, 61 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 12 MiB/s, 185 objects/s recovering 2024-03-29T04:52:16.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:16 smithi142 ceph-mon[112475]: osdmap e73: 6 total, 6 up, 6 in 2024-03-29T04:52:17.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:17 smithi152 ceph-mon[88186]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/333 objects degraded (13.514%), 14 pgs degraded) 2024-03-29T04:52:17.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:17 smithi152 ceph-mon[88186]: Cluster is now healthy 2024-03-29T04:52:17.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:17 smithi142 ceph-mon[112475]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/333 objects degraded (13.514%), 14 pgs degraded) 2024-03-29T04:52:17.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:17 smithi142 ceph-mon[112475]: Cluster is now healthy 2024-03-29T04:52:18.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:18 smithi152 ceph-mon[88186]: pgmap v94: 65 pgs: 3 peering, 1 active+clean+remapped, 61 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 473 B/s rd, 1 op/s; 11 MiB/s, 171 objects/s recovering 2024-03-29T04:52:18.625 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:18 smithi142 ceph-mon[112475]: pgmap v94: 65 pgs: 3 peering, 1 active+clean+remapped, 61 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 473 B/s rd, 1 op/s; 11 MiB/s, 171 objects/s recovering 2024-03-29T04:52:20.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:20 smithi142 ceph-mon[112475]: pgmap v95: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 8.0 MiB/s, 239 objects/s recovering 2024-03-29T04:52:21.025 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:20 smithi152 ceph-mon[88186]: pgmap v95: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 8.0 MiB/s, 239 objects/s recovering 2024-03-29T04:52:22.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:22 smithi142 ceph-mon[112475]: pgmap v96: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s; 559 KiB/s, 194 objects/s recovering 2024-03-29T04:52:23.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:22 smithi152 ceph-mon[88186]: pgmap v96: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s; 559 KiB/s, 194 objects/s recovering 2024-03-29T04:52:24.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:24 smithi142 ceph-mon[112475]: pgmap v97: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 3 op/s; 492 KiB/s, 171 objects/s recovering 2024-03-29T04:52:24.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:25.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:24 smithi152 ceph-mon[88186]: pgmap v97: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 3 op/s; 492 KiB/s, 171 objects/s recovering 2024-03-29T04:52:25.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:26.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:26 smithi142 ceph-mon[112475]: pgmap v98: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 0 B/s, 69 objects/s recovering 2024-03-29T04:52:27.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:26 smithi152 ceph-mon[88186]: pgmap v98: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 0 B/s, 69 objects/s recovering 2024-03-29T04:52:27.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.350+0000 7f8d35dec700 1 -- 172.21.15.142:0/826469443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d30105a40 msgr2=0x7f8d30105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:27.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.350+0000 7f8d35dec700 1 --2- 172.21.15.142:0/826469443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d30105a40 0x7f8d30105e20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009910 tx=0x7f8d20009c20 comp rx=0 tx=0).stop 2024-03-29T04:52:27.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.350+0000 7f8d35dec700 1 -- 172.21.15.142:0/826469443 shutdown_connections 2024-03-29T04:52:27.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.350+0000 7f8d35dec700 1 --2- 172.21.15.142:0/826469443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d300ffa40 0x7f8d300ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.350+0000 7f8d35dec700 1 --2- 172.21.15.142:0/826469443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d30105a40 0x7f8d30105e20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.350+0000 7f8d35dec700 1 -- 172.21.15.142:0/826469443 >> 172.21.15.142:0/826469443 conn(0x7f8d300fb570 msgr2=0x7f8d300fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:27.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.350+0000 7f8d35dec700 1 -- 172.21.15.142:0/826469443 shutdown_connections 2024-03-29T04:52:27.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.350+0000 7f8d35dec700 1 -- 172.21.15.142:0/826469443 wait complete. 2024-03-29T04:52:27.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.351+0000 7f8d35dec700 1 Processor -- start 2024-03-29T04:52:27.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.351+0000 7f8d35dec700 1 -- start start 2024-03-29T04:52:27.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.351+0000 7f8d35dec700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d300ffa40 0x7f8d301975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:27.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.351+0000 7f8d35dec700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d30105a40 0x7f8d30197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:27.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.351+0000 7f8d35dec700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30198110 con 0x7f8d30105a40 2024-03-29T04:52:27.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.351+0000 7f8d35dec700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30198250 con 0x7f8d300ffa40 2024-03-29T04:52:27.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.351+0000 7f8d2effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d30105a40 0x7f8d30197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:27.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.351+0000 7f8d2effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d30105a40 0x7f8d30197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53048/0 (socket says 172.21.15.142:53048) 2024-03-29T04:52:27.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.351+0000 7f8d2effd700 1 -- 172.21.15.142:0/14515548 learned_addr learned my addr 172.21.15.142:0/14515548 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:52:27.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.351+0000 7f8d2f7fe700 1 --2- 172.21.15.142:0/14515548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d300ffa40 0x7f8d301975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:27.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.351+0000 7f8d2effd700 1 -- 172.21.15.142:0/14515548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d300ffa40 msgr2=0x7f8d301975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:27.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.351+0000 7f8d2effd700 1 --2- 172.21.15.142:0/14515548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d300ffa40 0x7f8d301975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.352+0000 7f8d2effd700 1 -- 172.21.15.142:0/14515548 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200095f0 con 0x7f8d30105a40 2024-03-29T04:52:27.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.352+0000 7f8d2effd700 1 --2- 172.21.15.142:0/14515548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d30105a40 0x7f8d30197af0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400d8d0 tx=0x7f8d2400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:27.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.352+0000 7f8d2cff9700 1 -- 172.21.15.142:0/14515548 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d240098a0 con 0x7f8d30105a40 2024-03-29T04:52:27.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.352+0000 7f8d2cff9700 1 -- 172.21.15.142:0/14515548 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d24010460 con 0x7f8d30105a40 2024-03-29T04:52:27.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.352+0000 7f8d35dec700 1 -- 172.21.15.142:0/14515548 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d301a0620 con 0x7f8d30105a40 2024-03-29T04:52:27.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.352+0000 7f8d2cff9700 1 -- 172.21.15.142:0/14515548 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d2400f5f0 con 0x7f8d30105a40 2024-03-29T04:52:27.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.352+0000 7f8d35dec700 1 -- 172.21.15.142:0/14515548 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d301a0ae0 con 0x7f8d30105a40 2024-03-29T04:52:27.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.353+0000 7f8d35dec700 1 -- 172.21.15.142:0/14515548 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d30108040 con 0x7f8d30105a40 2024-03-29T04:52:27.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.354+0000 7f8d2cff9700 1 -- 172.21.15.142:0/14515548 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f8d24009a00 con 0x7f8d30105a40 2024-03-29T04:52:27.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.354+0000 7f8d2cff9700 1 --2- 172.21.15.142:0/14515548 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8d1806dd60 0x7f8d18070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:27.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.354+0000 7f8d2f7fe700 1 --2- 172.21.15.142:0/14515548 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8d1806dd60 0x7f8d18070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:27.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.354+0000 7f8d2cff9700 1 -- 172.21.15.142:0/14515548 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f8d2408da90 con 0x7f8d30105a40 2024-03-29T04:52:27.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.355+0000 7f8d2f7fe700 1 --2- 172.21.15.142:0/14515548 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8d1806dd60 0x7f8d18070220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d20006010 tx=0x7f8d2001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:27.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.359+0000 7f8d2cff9700 1 -- 172.21.15.142:0/14515548 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8d24056f60 con 0x7f8d30105a40 2024-03-29T04:52:27.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.575+0000 7f8d35dec700 1 -- 172.21.15.142:0/14515548 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d3002cd50 con 0x7f8d1806dd60 2024-03-29T04:52:27.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.577+0000 7f8d2cff9700 1 -- 172.21.15.142:0/14515548 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8d3002cd50 con 0x7f8d1806dd60 2024-03-29T04:52:27.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.579+0000 7f8d35dec700 1 -- 172.21.15.142:0/14515548 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8d1806dd60 msgr2=0x7f8d18070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:27.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.579+0000 7f8d35dec700 1 --2- 172.21.15.142:0/14515548 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8d1806dd60 0x7f8d18070220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d20006010 tx=0x7f8d2001a040 comp rx=0 tx=0).stop 2024-03-29T04:52:27.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.579+0000 7f8d35dec700 1 -- 172.21.15.142:0/14515548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d30105a40 msgr2=0x7f8d30197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:27.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.579+0000 7f8d35dec700 1 --2- 172.21.15.142:0/14515548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d30105a40 0x7f8d30197af0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400d8d0 tx=0x7f8d2400dbe0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.579+0000 7f8d35dec700 1 -- 172.21.15.142:0/14515548 shutdown_connections 2024-03-29T04:52:27.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.579+0000 7f8d35dec700 1 --2- 172.21.15.142:0/14515548 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8d1806dd60 0x7f8d18070220 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.579+0000 7f8d35dec700 1 --2- 172.21.15.142:0/14515548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d300ffa40 0x7f8d301975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.579+0000 7f8d35dec700 1 --2- 172.21.15.142:0/14515548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d30105a40 0x7f8d30197af0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.579+0000 7f8d35dec700 1 -- 172.21.15.142:0/14515548 >> 172.21.15.142:0/14515548 conn(0x7f8d300fb570 msgr2=0x7f8d300fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:27.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.579+0000 7f8d35dec700 1 -- 172.21.15.142:0/14515548 shutdown_connections 2024-03-29T04:52:27.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.579+0000 7f8d35dec700 1 -- 172.21.15.142:0/14515548 wait complete. 2024-03-29T04:52:27.594 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:52:27.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.690+0000 7fd0d8545700 1 -- 172.21.15.142:0/2535155120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0d0068670 msgr2=0x7fd0d0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:27.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.690+0000 7fd0d8545700 1 --2- 172.21.15.142:0/2535155120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0d0068670 0x7fd0d0068a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009910 tx=0x7fd0cc009c20 comp rx=0 tx=0).stop 2024-03-29T04:52:27.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.690+0000 7fd0d8545700 1 -- 172.21.15.142:0/2535155120 shutdown_connections 2024-03-29T04:52:27.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.690+0000 7fd0d8545700 1 --2- 172.21.15.142:0/2535155120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0d0069020 0x7fd0d0105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.690+0000 7fd0d8545700 1 --2- 172.21.15.142:0/2535155120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0d0068670 0x7fd0d0068a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.690+0000 7fd0d8545700 1 -- 172.21.15.142:0/2535155120 >> 172.21.15.142:0/2535155120 conn(0x7fd0d00755a0 msgr2=0x7fd0d00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:27.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.690+0000 7fd0d8545700 1 -- 172.21.15.142:0/2535155120 shutdown_connections 2024-03-29T04:52:27.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.690+0000 7fd0d8545700 1 -- 172.21.15.142:0/2535155120 wait complete. 2024-03-29T04:52:27.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d8545700 1 Processor -- start 2024-03-29T04:52:27.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d8545700 1 -- start start 2024-03-29T04:52:27.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d8545700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0d0068670 0x7fd0d00ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:27.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d8545700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0d0069020 0x7fd0d01001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:27.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d8545700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d0100720 con 0x7fd0d0068670 2024-03-29T04:52:27.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d8545700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d0100860 con 0x7fd0d0069020 2024-03-29T04:52:27.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d62e1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0d0068670 0x7fd0d00ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:27.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d62e1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0d0068670 0x7fd0d00ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53080/0 (socket says 172.21.15.142:53080) 2024-03-29T04:52:27.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d62e1700 1 -- 172.21.15.142:0/3215025541 learned_addr learned my addr 172.21.15.142:0/3215025541 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:52:27.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d5ae0700 1 --2- 172.21.15.142:0/3215025541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0d0069020 0x7fd0d01001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:27.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d62e1700 1 -- 172.21.15.142:0/3215025541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0d0069020 msgr2=0x7fd0d01001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:27.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d62e1700 1 --2- 172.21.15.142:0/3215025541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0d0069020 0x7fd0d01001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.691+0000 7fd0d62e1700 1 -- 172.21.15.142:0/3215025541 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc0095f0 con 0x7fd0d0068670 2024-03-29T04:52:27.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.692+0000 7fd0d62e1700 1 --2- 172.21.15.142:0/3215025541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0d0068670 0x7fd0d00ffca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc006010 tx=0x7fd0cc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:27.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.692+0000 7fd0c77fe700 1 -- 172.21.15.142:0/3215025541 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0cc01d070 con 0x7fd0d0068670 2024-03-29T04:52:27.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.692+0000 7fd0c77fe700 1 -- 172.21.15.142:0/3215025541 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0cc005dc0 con 0x7fd0d0068670 2024-03-29T04:52:27.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.692+0000 7fd0d8545700 1 -- 172.21.15.142:0/3215025541 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d0100ae0 con 0x7fd0d0068670 2024-03-29T04:52:27.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.692+0000 7fd0c77fe700 1 -- 172.21.15.142:0/3215025541 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0cc017400 con 0x7fd0d0068670 2024-03-29T04:52:27.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.692+0000 7fd0d8545700 1 -- 172.21.15.142:0/3215025541 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d01a2c40 con 0x7fd0d0068670 2024-03-29T04:52:27.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.693+0000 7fd0d8545700 1 -- 172.21.15.142:0/3215025541 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d0109130 con 0x7fd0d0068670 2024-03-29T04:52:27.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.693+0000 7fd0c77fe700 1 -- 172.21.15.142:0/3215025541 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fd0cc003a60 con 0x7fd0d0068670 2024-03-29T04:52:27.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.694+0000 7fd0c77fe700 1 --2- 172.21.15.142:0/3215025541 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd0bc06dd60 0x7fd0bc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:27.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.694+0000 7fd0c77fe700 1 -- 172.21.15.142:0/3215025541 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fd0cc08f010 con 0x7fd0d0068670 2024-03-29T04:52:27.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.694+0000 7fd0d5ae0700 1 --2- 172.21.15.142:0/3215025541 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd0bc06dd60 0x7fd0bc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:27.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.694+0000 7fd0d5ae0700 1 --2- 172.21.15.142:0/3215025541 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd0bc06dd60 0x7fd0bc070220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0101490 tx=0x7fd0c000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:27.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.703+0000 7fd0c77fe700 1 -- 172.21.15.142:0/3215025541 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd0cc0584e0 con 0x7fd0d0068670 2024-03-29T04:52:27.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.927+0000 7fd0d8545700 1 -- 172.21.15.142:0/3215025541 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0d002cce0 con 0x7fd0bc06dd60 2024-03-29T04:52:27.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.928+0000 7fd0c77fe700 1 -- 172.21.15.142:0/3215025541 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd0d002cce0 con 0x7fd0bc06dd60 2024-03-29T04:52:27.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.930+0000 7fd0d8545700 1 -- 172.21.15.142:0/3215025541 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd0bc06dd60 msgr2=0x7fd0bc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:27.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.930+0000 7fd0d8545700 1 --2- 172.21.15.142:0/3215025541 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd0bc06dd60 0x7fd0bc070220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0101490 tx=0x7fd0c000b410 comp rx=0 tx=0).stop 2024-03-29T04:52:27.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.930+0000 7fd0d8545700 1 -- 172.21.15.142:0/3215025541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0d0068670 msgr2=0x7fd0d00ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:27.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.930+0000 7fd0d8545700 1 --2- 172.21.15.142:0/3215025541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0d0068670 0x7fd0d00ffca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc006010 tx=0x7fd0cc00bd70 comp rx=0 tx=0).stop 2024-03-29T04:52:27.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.930+0000 7fd0d8545700 1 -- 172.21.15.142:0/3215025541 shutdown_connections 2024-03-29T04:52:27.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.930+0000 7fd0d8545700 1 --2- 172.21.15.142:0/3215025541 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd0bc06dd60 0x7fd0bc070220 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.930+0000 7fd0d8545700 1 --2- 172.21.15.142:0/3215025541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0d0068670 0x7fd0d00ffca0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.930+0000 7fd0d8545700 1 --2- 172.21.15.142:0/3215025541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0d0069020 0x7fd0d01001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:27.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.930+0000 7fd0d8545700 1 -- 172.21.15.142:0/3215025541 >> 172.21.15.142:0/3215025541 conn(0x7fd0d00755a0 msgr2=0x7fd0d00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:27.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.931+0000 7fd0d8545700 1 -- 172.21.15.142:0/3215025541 shutdown_connections 2024-03-29T04:52:27.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:27.931+0000 7fd0d8545700 1 -- 172.21.15.142:0/3215025541 wait complete. 2024-03-29T04:52:28.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.039+0000 7f3659347700 1 -- 172.21.15.142:0/883513970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3654101790 msgr2=0x7f36541057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:28.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.039+0000 7f3659347700 1 --2- 172.21.15.142:0/883513970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3654101790 0x7f36541057e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3648009910 tx=0x7f3648009c20 comp rx=0 tx=0).stop 2024-03-29T04:52:28.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.039+0000 7f3659347700 1 -- 172.21.15.142:0/883513970 shutdown_connections 2024-03-29T04:52:28.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.039+0000 7f3659347700 1 --2- 172.21.15.142:0/883513970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3654101790 0x7f36541057e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.039+0000 7f3659347700 1 --2- 172.21.15.142:0/883513970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3654100de0 0x7f36541011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.039+0000 7f3659347700 1 -- 172.21.15.142:0/883513970 >> 172.21.15.142:0/883513970 conn(0x7f36540fc670 msgr2=0x7f36540fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:28.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.039+0000 7f3659347700 1 -- 172.21.15.142:0/883513970 shutdown_connections 2024-03-29T04:52:28.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.039+0000 7f3659347700 1 -- 172.21.15.142:0/883513970 wait complete. 2024-03-29T04:52:28.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.040+0000 7f3659347700 1 Processor -- start 2024-03-29T04:52:28.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.040+0000 7f3659347700 1 -- start start 2024-03-29T04:52:28.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.040+0000 7f3659347700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3654100de0 0x7f3654073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:28.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.040+0000 7f3659347700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3654101790 0x7f36540735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:28.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.040+0000 7f3659347700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3654073b10 con 0x7f3654100de0 2024-03-29T04:52:28.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.040+0000 7f3659347700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3654073c50 con 0x7f3654101790 2024-03-29T04:52:28.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f3652ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3654100de0 0x7f3654073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:28.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f3652ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3654100de0 0x7f3654073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53086/0 (socket says 172.21.15.142:53086) 2024-03-29T04:52:28.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f3652ffd700 1 -- 172.21.15.142:0/1500762412 learned_addr learned my addr 172.21.15.142:0/1500762412 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:52:28.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f36527fc700 1 --2- 172.21.15.142:0/1500762412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3654101790 0x7f36540735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:28.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f3652ffd700 1 -- 172.21.15.142:0/1500762412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3654101790 msgr2=0x7f36540735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:28.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f3652ffd700 1 --2- 172.21.15.142:0/1500762412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3654101790 0x7f36540735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f3652ffd700 1 -- 172.21.15.142:0/1500762412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480095f0 con 0x7f3654100de0 2024-03-29T04:52:28.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f3652ffd700 1 --2- 172.21.15.142:0/1500762412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3654100de0 0x7f3654073090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f364400d8d0 tx=0x7f364400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:28.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f363bfff700 1 -- 172.21.15.142:0/1500762412 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3644009940 con 0x7f3654100de0 2024-03-29T04:52:28.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f3659347700 1 -- 172.21.15.142:0/1500762412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36541a2960 con 0x7f3654100de0 2024-03-29T04:52:28.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f3659347700 1 -- 172.21.15.142:0/1500762412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36541a2d70 con 0x7f3654100de0 2024-03-29T04:52:28.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f363bfff700 1 -- 172.21.15.142:0/1500762412 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3644010460 con 0x7f3654100de0 2024-03-29T04:52:28.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.041+0000 7f363bfff700 1 -- 172.21.15.142:0/1500762412 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f364400f5f0 con 0x7f3654100de0 2024-03-29T04:52:28.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.042+0000 7f3659347700 1 -- 172.21.15.142:0/1500762412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3640005320 con 0x7f3654100de0 2024-03-29T04:52:28.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.043+0000 7f363bfff700 1 -- 172.21.15.142:0/1500762412 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f3644009aa0 con 0x7f3654100de0 2024-03-29T04:52:28.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.043+0000 7f363bfff700 1 --2- 172.21.15.142:0/1500762412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f363c06dd60 0x7f363c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:28.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.043+0000 7f363bfff700 1 -- 172.21.15.142:0/1500762412 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f364408da60 con 0x7f3654100de0 2024-03-29T04:52:28.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.043+0000 7f36527fc700 1 --2- 172.21.15.142:0/1500762412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f363c06dd60 0x7f363c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:28.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.044+0000 7f36527fc700 1 --2- 172.21.15.142:0/1500762412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f363c06dd60 0x7f363c070220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3654074880 tx=0x7f364801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:28.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.051+0000 7f363bfff700 1 -- 172.21.15.142:0/1500762412 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f364405ac50 con 0x7f3654100de0 2024-03-29T04:52:28.277 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.272+0000 7f3659347700 1 -- 172.21.15.142:0/1500762412 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3640000bf0 con 0x7f363c06dd60 2024-03-29T04:52:28.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.292+0000 7f363bfff700 1 -- 172.21.15.142:0/1500762412 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3640000bf0 con 0x7f363c06dd60 2024-03-29T04:52:28.297 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:52:28.297 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (4m) 19s ago 15m 26.5M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:52:28.297 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (16m) 19s ago 16m 8548k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:52:28.297 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (15m) 78s ago 15m 8182k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:52:28.297 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (87s) 19s ago 16m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:52:28.297 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (82s) 78s ago 14m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:52:28.297 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (3m) 19s ago 15m 80.1M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:52:28.297 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (10m) 19s ago 10m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:52:28.297 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (10m) 19s ago 10m 63.5M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:52:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (10m) 78s ago 10m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:52:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (10m) 78s ago 10m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:52:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (5m) 19s ago 17m 505M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:52:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (5m) 78s ago 14m 449M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:52:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (2m) 19s ago 17m 47.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:52:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (2m) 78s ago 14m 41.1M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:52:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (4m) 19s ago 15m 14.9M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:52:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (4m) 78s ago 14m 16.0M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:52:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (62s) 19s ago 13m 107M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:52:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (25s) 19s ago 13m 12.1M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:52:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (13m) 19s ago 13m 238M 1216M 18.2.1 5be31c24972a 75e480e18a87 2024-03-29T04:52:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (12m) 78s ago 12m 276M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:52:28.299 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (12m) 78s ago 12m 236M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:52:28.299 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (11m) 78s ago 11m 225M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:52:28.299 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (4m) 19s ago 15m 63.5M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:52:28.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.294+0000 7f3659347700 1 -- 172.21.15.142:0/1500762412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f363c06dd60 msgr2=0x7f363c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:28.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.294+0000 7f3659347700 1 --2- 172.21.15.142:0/1500762412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f363c06dd60 0x7f363c070220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3654074880 tx=0x7f364801a040 comp rx=0 tx=0).stop 2024-03-29T04:52:28.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.294+0000 7f3659347700 1 -- 172.21.15.142:0/1500762412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3654100de0 msgr2=0x7f3654073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:28.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.294+0000 7f3659347700 1 --2- 172.21.15.142:0/1500762412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3654100de0 0x7f3654073090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f364400d8d0 tx=0x7f364400dc90 comp rx=0 tx=0).stop 2024-03-29T04:52:28.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.294+0000 7f3659347700 1 -- 172.21.15.142:0/1500762412 shutdown_connections 2024-03-29T04:52:28.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.294+0000 7f3659347700 1 --2- 172.21.15.142:0/1500762412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f363c06dd60 0x7f363c070220 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.294+0000 7f3659347700 1 --2- 172.21.15.142:0/1500762412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3654100de0 0x7f3654073090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.294+0000 7f3659347700 1 --2- 172.21.15.142:0/1500762412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3654101790 0x7f36540735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.294+0000 7f3659347700 1 -- 172.21.15.142:0/1500762412 >> 172.21.15.142:0/1500762412 conn(0x7f36540fc670 msgr2=0x7f36540fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:28.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.294+0000 7f3659347700 1 -- 172.21.15.142:0/1500762412 shutdown_connections 2024-03-29T04:52:28.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.294+0000 7f3659347700 1 -- 172.21.15.142:0/1500762412 wait complete. 2024-03-29T04:52:28.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.401+0000 7f0d84b4c700 1 -- 172.21.15.142:0/1934318185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d80105a40 msgr2=0x7f0d80105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:28.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.401+0000 7f0d84b4c700 1 --2- 172.21.15.142:0/1934318185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d80105a40 0x7f0d80105e20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009910 tx=0x7f0d70009c20 comp rx=0 tx=0).stop 2024-03-29T04:52:28.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.401+0000 7f0d84b4c700 1 -- 172.21.15.142:0/1934318185 shutdown_connections 2024-03-29T04:52:28.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.401+0000 7f0d84b4c700 1 --2- 172.21.15.142:0/1934318185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d800ffa40 0x7f0d800ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.401+0000 7f0d84b4c700 1 --2- 172.21.15.142:0/1934318185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d80105a40 0x7f0d80105e20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.401+0000 7f0d84b4c700 1 -- 172.21.15.142:0/1934318185 >> 172.21.15.142:0/1934318185 conn(0x7f0d800fb570 msgr2=0x7f0d800fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:28.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.401+0000 7f0d84b4c700 1 -- 172.21.15.142:0/1934318185 shutdown_connections 2024-03-29T04:52:28.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.401+0000 7f0d84b4c700 1 -- 172.21.15.142:0/1934318185 wait complete. 2024-03-29T04:52:28.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.402+0000 7f0d84b4c700 1 Processor -- start 2024-03-29T04:52:28.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.402+0000 7f0d84b4c700 1 -- start start 2024-03-29T04:52:28.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.402+0000 7f0d84b4c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d800ffa40 0x7f0d801953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:28.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.402+0000 7f0d84b4c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d80105a40 0x7f0d801958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.402+0000 7f0d84b4c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d80195e30 con 0x7f0d80105a40 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.402+0000 7f0d84b4c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d80195fa0 con 0x7f0d800ffa40 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d7dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d80105a40 0x7f0d801958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d7dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d80105a40 0x7f0d801958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53102/0 (socket says 172.21.15.142:53102) 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d7dd9b700 1 -- 172.21.15.142:0/3921653845 learned_addr learned my addr 172.21.15.142:0/3921653845 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d7e59c700 1 --2- 172.21.15.142:0/3921653845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d800ffa40 0x7f0d801953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d7dd9b700 1 -- 172.21.15.142:0/3921653845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d800ffa40 msgr2=0x7f0d801953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d7dd9b700 1 --2- 172.21.15.142:0/3921653845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d800ffa40 0x7f0d801953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d7dd9b700 1 -- 172.21.15.142:0/3921653845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d700095f0 con 0x7f0d80105a40 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d7dd9b700 1 --2- 172.21.15.142:0/3921653845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d80105a40 0x7f0d801958f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400d8d0 tx=0x7f0d7400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d6f7fe700 1 -- 172.21.15.142:0/3921653845 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d740098a0 con 0x7f0d80105a40 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d84b4c700 1 -- 172.21.15.142:0/3921653845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d801a2a20 con 0x7f0d80105a40 2024-03-29T04:52:28.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d6f7fe700 1 -- 172.21.15.142:0/3921653845 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d74010460 con 0x7f0d80105a40 2024-03-29T04:52:28.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d6f7fe700 1 -- 172.21.15.142:0/3921653845 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d7400f5f0 con 0x7f0d80105a40 2024-03-29T04:52:28.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.403+0000 7f0d84b4c700 1 -- 172.21.15.142:0/3921653845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d801a2d80 con 0x7f0d80105a40 2024-03-29T04:52:28.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.404+0000 7f0d84b4c700 1 -- 172.21.15.142:0/3921653845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d80108040 con 0x7f0d80105a40 2024-03-29T04:52:28.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.405+0000 7f0d6f7fe700 1 -- 172.21.15.142:0/3921653845 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f0d74009a00 con 0x7f0d80105a40 2024-03-29T04:52:28.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.405+0000 7f0d6f7fe700 1 --2- 172.21.15.142:0/3921653845 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0d6806dd60 0x7f0d68070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:28.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.406+0000 7f0d7e59c700 1 --2- 172.21.15.142:0/3921653845 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0d6806dd60 0x7f0d68070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:28.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.406+0000 7f0d6f7fe700 1 -- 172.21.15.142:0/3921653845 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f0d7408da80 con 0x7f0d80105a40 2024-03-29T04:52:28.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.406+0000 7f0d7e59c700 1 --2- 172.21.15.142:0/3921653845 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0d6806dd60 0x7f0d68070220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b5c0 tx=0x7f0d7001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:28.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.410+0000 7f0d6f7fe700 1 -- 172.21.15.142:0/3921653845 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0d7405aa80 con 0x7f0d80105a40 2024-03-29T04:52:28.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.701+0000 7f0d84b4c700 1 -- 172.21.15.142:0/3921653845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0d8004ea90 con 0x7f0d80105a40 2024-03-29T04:52:28.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.702+0000 7f0d6f7fe700 1 -- 172.21.15.142:0/3921653845 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f0d7405a610 con 0x7f0d80105a40 2024-03-29T04:52:28.707 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:52:28.707 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:52:28.707 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:52:28.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:52:28.707 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:52:28.707 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:52:28.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:52:28.707 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:52:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-03-29T04:52:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:52:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:52:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:52:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:52:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:52:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:52:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-03-29T04:52:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T04:52:28.708 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:52:28.708 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:52:28.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.704+0000 7f0d84b4c700 1 -- 172.21.15.142:0/3921653845 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0d6806dd60 msgr2=0x7f0d68070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:28.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.704+0000 7f0d84b4c700 1 --2- 172.21.15.142:0/3921653845 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0d6806dd60 0x7f0d68070220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b5c0 tx=0x7f0d7001a040 comp rx=0 tx=0).stop 2024-03-29T04:52:28.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.704+0000 7f0d84b4c700 1 -- 172.21.15.142:0/3921653845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d80105a40 msgr2=0x7f0d801958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:28.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.704+0000 7f0d84b4c700 1 --2- 172.21.15.142:0/3921653845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d80105a40 0x7f0d801958f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400d8d0 tx=0x7f0d7400dbe0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.704+0000 7f0d84b4c700 1 -- 172.21.15.142:0/3921653845 shutdown_connections 2024-03-29T04:52:28.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.704+0000 7f0d84b4c700 1 --2- 172.21.15.142:0/3921653845 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0d6806dd60 0x7f0d68070220 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.704+0000 7f0d84b4c700 1 --2- 172.21.15.142:0/3921653845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d800ffa40 0x7f0d801953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.704+0000 7f0d84b4c700 1 --2- 172.21.15.142:0/3921653845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d80105a40 0x7f0d801958f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.704+0000 7f0d84b4c700 1 -- 172.21.15.142:0/3921653845 >> 172.21.15.142:0/3921653845 conn(0x7f0d800fb570 msgr2=0x7f0d800fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:28.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.704+0000 7f0d84b4c700 1 -- 172.21.15.142:0/3921653845 shutdown_connections 2024-03-29T04:52:28.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.704+0000 7f0d84b4c700 1 -- 172.21.15.142:0/3921653845 wait complete. 2024-03-29T04:52:28.816 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.811+0000 7fb20b790700 1 -- 172.21.15.142:0/228229790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2040ffbd0 msgr2=0x7fb204100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:28.816 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.811+0000 7fb20b790700 1 --2- 172.21.15.142:0/228229790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2040ffbd0 0x7fb204100050 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009910 tx=0x7fb1f4009c20 comp rx=0 tx=0).stop 2024-03-29T04:52:28.816 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.811+0000 7fb20b790700 1 -- 172.21.15.142:0/228229790 shutdown_connections 2024-03-29T04:52:28.816 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.811+0000 7fb20b790700 1 --2- 172.21.15.142:0/228229790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2040ffbd0 0x7fb204100050 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.816 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.811+0000 7fb20b790700 1 --2- 172.21.15.142:0/228229790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb204105a40 0x7fb204105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.816 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.811+0000 7fb20b790700 1 -- 172.21.15.142:0/228229790 >> 172.21.15.142:0/228229790 conn(0x7fb2040755a0 msgr2=0x7fb2040759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:28.816 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.811+0000 7fb20b790700 1 -- 172.21.15.142:0/228229790 shutdown_connections 2024-03-29T04:52:28.816 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.811+0000 7fb20b790700 1 -- 172.21.15.142:0/228229790 wait complete. 2024-03-29T04:52:28.818 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb20b790700 1 Processor -- start 2024-03-29T04:52:28.818 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb20b790700 1 -- start start 2024-03-29T04:52:28.818 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb20b790700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2040ffbd0 0x7fb20419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:28.818 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb20b790700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb204105a40 0x7fb20419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:28.818 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb20b790700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20419f2c0 con 0x7fb204105a40 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb20b790700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb204198770 con 0x7fb2040ffbd0 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb208d2b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb204105a40 0x7fb20419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb208d2b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb204105a40 0x7fb20419ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:32914/0 (socket says 172.21.15.142:32914) 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb208d2b700 1 -- 172.21.15.142:0/2688326237 learned_addr learned my addr 172.21.15.142:0/2688326237 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb20952c700 1 --2- 172.21.15.142:0/2688326237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2040ffbd0 0x7fb20419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb208d2b700 1 -- 172.21.15.142:0/2688326237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2040ffbd0 msgr2=0x7fb20419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb208d2b700 1 --2- 172.21.15.142:0/2688326237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2040ffbd0 0x7fb20419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb208d2b700 1 -- 172.21.15.142:0/2688326237 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f40095f0 con 0x7fb204105a40 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb20952c700 1 --2- 172.21.15.142:0/2688326237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2040ffbd0 0x7fb20419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.812+0000 7fb208d2b700 1 --2- 172.21.15.142:0/2688326237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb204105a40 0x7fb20419ec30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4006010 tx=0x7fb1f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.813+0000 7fb1fa7fc700 1 -- 172.21.15.142:0/2688326237 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1f401d070 con 0x7fb204105a40 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.813+0000 7fb20b790700 1 -- 172.21.15.142:0/2688326237 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2041989f0 con 0x7fb204105a40 2024-03-29T04:52:28.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.813+0000 7fb1fa7fc700 1 -- 172.21.15.142:0/2688326237 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1f4004fb0 con 0x7fb204105a40 2024-03-29T04:52:28.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.813+0000 7fb1fa7fc700 1 -- 172.21.15.142:0/2688326237 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1f4022ea0 con 0x7fb204105a40 2024-03-29T04:52:28.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.813+0000 7fb20b790700 1 -- 172.21.15.142:0/2688326237 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb204198ee0 con 0x7fb204105a40 2024-03-29T04:52:28.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.814+0000 7fb20b790700 1 -- 172.21.15.142:0/2688326237 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e8005320 con 0x7fb204105a40 2024-03-29T04:52:28.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.814+0000 7fb1fa7fc700 1 -- 172.21.15.142:0/2688326237 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fb1f4004af0 con 0x7fb204105a40 2024-03-29T04:52:28.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.815+0000 7fb1fa7fc700 1 --2- 172.21.15.142:0/2688326237 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb1f006dd60 0x7fb1f0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:28.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.815+0000 7fb1fa7fc700 1 -- 172.21.15.142:0/2688326237 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fb1f408f560 con 0x7fb204105a40 2024-03-29T04:52:28.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.815+0000 7fb20952c700 1 --2- 172.21.15.142:0/2688326237 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb1f006dd60 0x7fb1f0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:28.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.815+0000 7fb20952c700 1 --2- 172.21.15.142:0/2688326237 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb1f006dd60 0x7fb1f0070220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb200005950 tx=0x7fb200005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:28.827 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:28.823+0000 7fb1fa7fc700 1 -- 172.21.15.142:0/2688326237 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb1f405c690 con 0x7fb204105a40 2024-03-29T04:52:28.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:28 smithi142 ceph-mon[112475]: pgmap v99: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s; 0 B/s, 61 objects/s recovering 2024-03-29T04:52:28.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:28 smithi142 ceph-mon[112475]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:52:29.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:28 smithi152 ceph-mon[88186]: pgmap v99: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s; 0 B/s, 61 objects/s recovering 2024-03-29T04:52:29.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:28 smithi152 ceph-mon[88186]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:52:29.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.082+0000 7fb20b790700 1 -- 172.21.15.142:0/2688326237 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb1e8005cc0 con 0x7fb204105a40 2024-03-29T04:52:29.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.083+0000 7fb1fa7fc700 1 -- 172.21.15.142:0/2688326237 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1906 (secure 0 0 0) 0x7fb1f4031790 con 0x7fb204105a40 2024-03-29T04:52:29.088 INFO:teuthology.orchestra.run.smithi142.stdout:e21 2024-03-29T04:52:29.088 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:52:29.088 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:52:29.088 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:52:29.088 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:52:29.088 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:52:29.088 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:52:29.088 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 21 2024-03-29T04:52:29.088 INFO:teuthology.orchestra.run.smithi142.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T04:52:29.088 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:49:01.288637+0000 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 47 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:52:29.089 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:up {0=14494,1=24275} 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{0:14494} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{0:14506} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:52:29.090 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{1:24275} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:52:29.091 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{1:24287} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:52:29.091 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:52:29.091 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:52:29.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.085+0000 7fb20b790700 1 -- 172.21.15.142:0/2688326237 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb1f006dd60 msgr2=0x7fb1f0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:29.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.085+0000 7fb20b790700 1 --2- 172.21.15.142:0/2688326237 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb1f006dd60 0x7fb1f0070220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb200005950 tx=0x7fb200005f50 comp rx=0 tx=0).stop 2024-03-29T04:52:29.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.085+0000 7fb20b790700 1 -- 172.21.15.142:0/2688326237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb204105a40 msgr2=0x7fb20419ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:29.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.085+0000 7fb20b790700 1 --2- 172.21.15.142:0/2688326237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb204105a40 0x7fb20419ec30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4006010 tx=0x7fb1f400be20 comp rx=0 tx=0).stop 2024-03-29T04:52:29.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.085+0000 7fb20b790700 1 -- 172.21.15.142:0/2688326237 shutdown_connections 2024-03-29T04:52:29.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.085+0000 7fb20b790700 1 --2- 172.21.15.142:0/2688326237 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb1f006dd60 0x7fb1f0070220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.085+0000 7fb20b790700 1 --2- 172.21.15.142:0/2688326237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2040ffbd0 0x7fb20419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.085+0000 7fb20b790700 1 --2- 172.21.15.142:0/2688326237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb204105a40 0x7fb20419ec30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.085+0000 7fb20b790700 1 -- 172.21.15.142:0/2688326237 >> 172.21.15.142:0/2688326237 conn(0x7fb2040755a0 msgr2=0x7fb204068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:29.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.085+0000 7fb20b790700 1 -- 172.21.15.142:0/2688326237 shutdown_connections 2024-03-29T04:52:29.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.085+0000 7fb20b790700 1 -- 172.21.15.142:0/2688326237 wait complete. 2024-03-29T04:52:29.092 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 21 2024-03-29T04:52:29.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.194+0000 7fa6b6a1a700 1 -- 172.21.15.142:0/4071130548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b0101750 msgr2=0x7fa6b01057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:29.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.194+0000 7fa6b6a1a700 1 --2- 172.21.15.142:0/4071130548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b0101750 0x7fa6b01057a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009910 tx=0x7fa6a4009c20 comp rx=0 tx=0).stop 2024-03-29T04:52:29.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.194+0000 7fa6b6a1a700 1 -- 172.21.15.142:0/4071130548 shutdown_connections 2024-03-29T04:52:29.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.194+0000 7fa6b6a1a700 1 --2- 172.21.15.142:0/4071130548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b0101750 0x7fa6b01057a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.194+0000 7fa6b6a1a700 1 --2- 172.21.15.142:0/4071130548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0100da0 0x7fa6b0101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.194+0000 7fa6b6a1a700 1 -- 172.21.15.142:0/4071130548 >> 172.21.15.142:0/4071130548 conn(0x7fa6b00fc630 msgr2=0x7fa6b00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:29.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.195+0000 7fa6b6a1a700 1 -- 172.21.15.142:0/4071130548 shutdown_connections 2024-03-29T04:52:29.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.195+0000 7fa6b6a1a700 1 -- 172.21.15.142:0/4071130548 wait complete. 2024-03-29T04:52:29.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.195+0000 7fa6b6a1a700 1 Processor -- start 2024-03-29T04:52:29.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.195+0000 7fa6b6a1a700 1 -- start start 2024-03-29T04:52:29.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.196+0000 7fa6b6a1a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b0100da0 0x7fa6b0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:29.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.196+0000 7fa6b6a1a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0101750 0x7fa6b00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:29.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.196+0000 7fa6b6a1a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0073b10 con 0x7fa6b0100da0 2024-03-29T04:52:29.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.196+0000 7fa6b6a1a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0073c50 con 0x7fa6b0101750 2024-03-29T04:52:29.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.196+0000 7fa6affff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b0100da0 0x7fa6b0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:29.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.196+0000 7fa6affff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b0100da0 0x7fa6b0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:32940/0 (socket says 172.21.15.142:32940) 2024-03-29T04:52:29.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.196+0000 7fa6affff700 1 -- 172.21.15.142:0/1806264979 learned_addr learned my addr 172.21.15.142:0/1806264979 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:52:29.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.196+0000 7fa6af7fe700 1 --2- 172.21.15.142:0/1806264979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0101750 0x7fa6b00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:29.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.196+0000 7fa6affff700 1 -- 172.21.15.142:0/1806264979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0101750 msgr2=0x7fa6b00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:29.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.196+0000 7fa6affff700 1 --2- 172.21.15.142:0/1806264979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0101750 0x7fa6b00735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.196+0000 7fa6affff700 1 -- 172.21.15.142:0/1806264979 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a40095f0 con 0x7fa6b0100da0 2024-03-29T04:52:29.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.196+0000 7fa6affff700 1 --2- 172.21.15.142:0/1806264979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b0100da0 0x7fa6b0073090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000b4b0 tx=0x7fa6a000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:29.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.197+0000 7fa6ad7fa700 1 -- 172.21.15.142:0/1806264979 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6a0011800 con 0x7fa6b0100da0 2024-03-29T04:52:29.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.197+0000 7fa6b6a1a700 1 -- 172.21.15.142:0/1806264979 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b01a2aa0 con 0x7fa6b0100da0 2024-03-29T04:52:29.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.197+0000 7fa6ad7fa700 1 -- 172.21.15.142:0/1806264979 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6a0011e20 con 0x7fa6b0100da0 2024-03-29T04:52:29.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.197+0000 7fa6ad7fa700 1 -- 172.21.15.142:0/1806264979 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6a000f2e0 con 0x7fa6b0100da0 2024-03-29T04:52:29.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.197+0000 7fa6b6a1a700 1 -- 172.21.15.142:0/1806264979 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b01a2eb0 con 0x7fa6b0100da0 2024-03-29T04:52:29.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.198+0000 7fa696ffd700 1 -- 172.21.15.142:0/1806264979 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6900052f0 con 0x7fa6b0100da0 2024-03-29T04:52:29.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.198+0000 7fa6ad7fa700 1 -- 172.21.15.142:0/1806264979 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fa6a0010380 con 0x7fa6b0100da0 2024-03-29T04:52:29.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.199+0000 7fa6ad7fa700 1 --2- 172.21.15.142:0/1806264979 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa69806dd60 0x7fa698070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:29.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.199+0000 7fa6ad7fa700 1 -- 172.21.15.142:0/1806264979 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fa6a008d760 con 0x7fa6b0100da0 2024-03-29T04:52:29.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.199+0000 7fa6af7fe700 1 --2- 172.21.15.142:0/1806264979 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa69806dd60 0x7fa698070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:29.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.200+0000 7fa6af7fe700 1 --2- 172.21.15.142:0/1806264979 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa69806dd60 0x7fa698070220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0074880 tx=0x7fa6a401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:29.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.204+0000 7fa6ad7fa700 1 -- 172.21.15.142:0/1806264979 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa6a0056c30 con 0x7fa6b0100da0 2024-03-29T04:52:29.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.423+0000 7fa696ffd700 1 -- 172.21.15.142:0/1806264979 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa690000bc0 con 0x7fa69806dd60 2024-03-29T04:52:29.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.424+0000 7fa6ad7fa700 1 -- 172.21.15.142:0/1806264979 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa690000bc0 con 0x7fa69806dd60 2024-03-29T04:52:29.429 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:52:29.429 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:52:29.429 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:52:29.429 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:52:29.429 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:52:29.429 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:52:29.429 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:52:29.429 INFO:teuthology.orchestra.run.smithi142.stdout: "mon" 2024-03-29T04:52:29.429 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:52:29.429 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "8/23 daemons upgraded", 2024-03-29T04:52:29.430 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading osd daemons", 2024-03-29T04:52:29.430 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:52:29.430 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:52:29.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.426+0000 7fa696ffd700 1 -- 172.21.15.142:0/1806264979 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa69806dd60 msgr2=0x7fa698070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:29.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.426+0000 7fa696ffd700 1 --2- 172.21.15.142:0/1806264979 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa69806dd60 0x7fa698070220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0074880 tx=0x7fa6a401a040 comp rx=0 tx=0).stop 2024-03-29T04:52:29.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.426+0000 7fa696ffd700 1 -- 172.21.15.142:0/1806264979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b0100da0 msgr2=0x7fa6b0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:29.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.426+0000 7fa696ffd700 1 --2- 172.21.15.142:0/1806264979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b0100da0 0x7fa6b0073090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000b4b0 tx=0x7fa6a000b7c0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.426+0000 7fa696ffd700 1 -- 172.21.15.142:0/1806264979 shutdown_connections 2024-03-29T04:52:29.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.426+0000 7fa696ffd700 1 --2- 172.21.15.142:0/1806264979 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa69806dd60 0x7fa698070220 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.426+0000 7fa696ffd700 1 --2- 172.21.15.142:0/1806264979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b0100da0 0x7fa6b0073090 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.426+0000 7fa696ffd700 1 --2- 172.21.15.142:0/1806264979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6b0101750 0x7fa6b00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.426+0000 7fa696ffd700 1 -- 172.21.15.142:0/1806264979 >> 172.21.15.142:0/1806264979 conn(0x7fa6b00fc630 msgr2=0x7fa6b00fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:29.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.427+0000 7fa696ffd700 1 -- 172.21.15.142:0/1806264979 shutdown_connections 2024-03-29T04:52:29.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.427+0000 7fa696ffd700 1 -- 172.21.15.142:0/1806264979 wait complete. 2024-03-29T04:52:29.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.543+0000 7fc827745700 1 -- 172.21.15.142:0/3524323857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8200ff510 msgr2=0x7fc8200ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:29.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.543+0000 7fc827745700 1 --2- 172.21.15.142:0/3524323857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8200ff510 0x7fc8200ff8f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009910 tx=0x7fc81c009c20 comp rx=0 tx=0).stop 2024-03-29T04:52:29.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.543+0000 7fc827745700 1 -- 172.21.15.142:0/3524323857 shutdown_connections 2024-03-29T04:52:29.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.543+0000 7fc827745700 1 --2- 172.21.15.142:0/3524323857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8200ffec0 0x7fc8201057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.543+0000 7fc827745700 1 --2- 172.21.15.142:0/3524323857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8200ff510 0x7fc8200ff8f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.543+0000 7fc827745700 1 -- 172.21.15.142:0/3524323857 >> 172.21.15.142:0/3524323857 conn(0x7fc820074e00 msgr2=0x7fc820075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:29.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.544+0000 7fc827745700 1 -- 172.21.15.142:0/3524323857 shutdown_connections 2024-03-29T04:52:29.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.544+0000 7fc827745700 1 -- 172.21.15.142:0/3524323857 wait complete. 2024-03-29T04:52:29.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.544+0000 7fc827745700 1 Processor -- start 2024-03-29T04:52:29.550 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.544+0000 7fc827745700 1 -- start start 2024-03-29T04:52:29.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.544+0000 7fc827745700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8200ff510 0x7fc820100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:29.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.544+0000 7fc827745700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8200ffec0 0x7fc8201009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:29.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.544+0000 7fc827745700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc820100f10 con 0x7fc8200ffec0 2024-03-29T04:52:29.552 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.544+0000 7fc827745700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc820101050 con 0x7fc8200ff510 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc824ce0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8200ffec0 0x7fc8201009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc824ce0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8200ffec0 0x7fc8201009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:32954/0 (socket says 172.21.15.142:32954) 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc824ce0700 1 -- 172.21.15.142:0/1953464301 learned_addr learned my addr 172.21.15.142:0/1953464301 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc8254e1700 1 --2- 172.21.15.142:0/1953464301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8200ff510 0x7fc820100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc824ce0700 1 -- 172.21.15.142:0/1953464301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8200ff510 msgr2=0x7fc820100490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc824ce0700 1 --2- 172.21.15.142:0/1953464301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8200ff510 0x7fc820100490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc824ce0700 1 -- 172.21.15.142:0/1953464301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c0095f0 con 0x7fc8200ffec0 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc824ce0700 1 --2- 172.21.15.142:0/1953464301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8200ffec0 0x7fc8201009d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc81000d900 tx=0x7fc81000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc8167fc700 1 -- 172.21.15.142:0/1953464301 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc810009900 con 0x7fc8200ffec0 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc8167fc700 1 -- 172.21.15.142:0/1953464301 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc810010460 con 0x7fc8200ffec0 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc8167fc700 1 -- 172.21.15.142:0/1953464301 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc81000f5f0 con 0x7fc8200ffec0 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc827745700 1 -- 172.21.15.142:0/1953464301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8201a29b0 con 0x7fc8200ffec0 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.545+0000 7fc827745700 1 -- 172.21.15.142:0/1953464301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8201a2dc0 con 0x7fc8200ffec0 2024-03-29T04:52:29.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.546+0000 7fc827745700 1 -- 172.21.15.142:0/1953464301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8201091e0 con 0x7fc8200ffec0 2024-03-29T04:52:29.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.547+0000 7fc8167fc700 1 -- 172.21.15.142:0/1953464301 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fc8100105d0 con 0x7fc8200ffec0 2024-03-29T04:52:29.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.548+0000 7fc8167fc700 1 --2- 172.21.15.142:0/1953464301 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc80c06dd60 0x7fc80c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:29.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.548+0000 7fc8167fc700 1 -- 172.21.15.142:0/1953464301 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fc81008d7c0 con 0x7fc8200ffec0 2024-03-29T04:52:29.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.548+0000 7fc8254e1700 1 --2- 172.21.15.142:0/1953464301 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc80c06dd60 0x7fc80c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:29.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.548+0000 7fc8254e1700 1 --2- 172.21.15.142:0/1953464301 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc80c06dd60 0x7fc80c070220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc81c006010 tx=0x7fc81c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:29.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.552+0000 7fc8167fc700 1 -- 172.21.15.142:0/1953464301 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc810056c90 con 0x7fc8200ffec0 2024-03-29T04:52:29.803 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:29 smithi142 ceph-mon[112475]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:52:29.803 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:29 smithi142 ceph-mon[112475]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:52:29.803 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:29 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3921653845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:29.803 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T04:52:29.804 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:29 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2688326237' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:52:29.804 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:29.804 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:52:29.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.839+0000 7fc827745700 1 -- 172.21.15.142:0/1953464301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc82004ea90 con 0x7fc8200ffec0 2024-03-29T04:52:29.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.840+0000 7fc8167fc700 1 -- 172.21.15.142:0/1953464301 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc81005a350 con 0x7fc8200ffec0 2024-03-29T04:52:29.845 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_OK 2024-03-29T04:52:29.847 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.842+0000 7fc827745700 1 -- 172.21.15.142:0/1953464301 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc80c06dd60 msgr2=0x7fc80c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:29.847 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.842+0000 7fc827745700 1 --2- 172.21.15.142:0/1953464301 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc80c06dd60 0x7fc80c070220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc81c006010 tx=0x7fc81c01a040 comp rx=0 tx=0).stop 2024-03-29T04:52:29.847 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.842+0000 7fc827745700 1 -- 172.21.15.142:0/1953464301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8200ffec0 msgr2=0x7fc8201009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:29.847 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.842+0000 7fc827745700 1 --2- 172.21.15.142:0/1953464301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8200ffec0 0x7fc8201009d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc81000d900 tx=0x7fc81000dc10 comp rx=0 tx=0).stop 2024-03-29T04:52:29.847 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.842+0000 7fc827745700 1 -- 172.21.15.142:0/1953464301 shutdown_connections 2024-03-29T04:52:29.847 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.842+0000 7fc827745700 1 --2- 172.21.15.142:0/1953464301 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc80c06dd60 0x7fc80c070220 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.847 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.842+0000 7fc827745700 1 --2- 172.21.15.142:0/1953464301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8200ff510 0x7fc820100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.842+0000 7fc827745700 1 --2- 172.21.15.142:0/1953464301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8200ffec0 0x7fc8201009d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:29.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.842+0000 7fc827745700 1 -- 172.21.15.142:0/1953464301 >> 172.21.15.142:0/1953464301 conn(0x7fc820074e00 msgr2=0x7fc8200fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:29.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.842+0000 7fc827745700 1 -- 172.21.15.142:0/1953464301 shutdown_connections 2024-03-29T04:52:29.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:29.842+0000 7fc827745700 1 -- 172.21.15.142:0/1953464301 wait complete. 2024-03-29T04:52:30.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:29 smithi152 ceph-mon[88186]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:52:30.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:29 smithi152 ceph-mon[88186]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:52:30.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:29 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3921653845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:30.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T04:52:30.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:29 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2688326237' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:52:30.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:30.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:52:31.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:30 smithi152 ceph-mon[88186]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T04:52:31.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:30 smithi152 ceph-mon[88186]: Upgrade: osd.2 is safe to restart 2024-03-29T04:52:31.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:30 smithi152 ceph-mon[88186]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:52:31.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:30 smithi152 ceph-mon[88186]: pgmap v100: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s; 0 B/s, 58 objects/s recovering 2024-03-29T04:52:31.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:30 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1953464301' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:52:31.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:30 smithi142 ceph-mon[112475]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T04:52:31.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:30 smithi142 ceph-mon[112475]: Upgrade: osd.2 is safe to restart 2024-03-29T04:52:31.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:30 smithi142 ceph-mon[112475]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:52:31.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:30 smithi142 ceph-mon[112475]: pgmap v100: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s; 0 B/s, 58 objects/s recovering 2024-03-29T04:52:31.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:30 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1953464301' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:52:31.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:31 smithi142 ceph-mon[112475]: Upgrade: Updating osd.2 2024-03-29T04:52:31.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:31 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:31.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:31 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-29T04:52:31.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:31 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:52:31.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:31 smithi142 ceph-mon[112475]: Deploying daemon osd.2 on smithi142 2024-03-29T04:52:31.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:31 smithi142 ceph-mon[112475]: pgmap v101: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:52:32.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:31 smithi152 ceph-mon[88186]: Upgrade: Updating osd.2 2024-03-29T04:52:32.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:31 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:32.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:31 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-29T04:52:32.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:31 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:52:32.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:31 smithi152 ceph-mon[88186]: Deploying daemon osd.2 on smithi142 2024-03-29T04:52:32.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:31 smithi152 ceph-mon[88186]: pgmap v101: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:52:34.327 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:34 smithi142 systemd[1]: Stopping Ceph osd.2 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:52:34.605 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:34 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2[63214]: 2024-03-29T04:52:34.320+0000 7fc4210cb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T04:52:34.605 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:34 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2[63214]: 2024-03-29T04:52:34.320+0000 7fc4210cb700 -1 osd.2 73 *** Got signal Terminated *** 2024-03-29T04:52:34.605 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:34 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2[63214]: 2024-03-29T04:52:34.320+0000 7fc4210cb700 -1 osd.2 73 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T04:52:34.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:34 smithi142 ceph-mon[112475]: pgmap v102: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:52:34.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:34 smithi142 ceph-mon[112475]: osd.2 marked itself down and dead 2024-03-29T04:52:35.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:34 smithi152 ceph-mon[88186]: pgmap v102: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:52:35.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:34 smithi152 ceph-mon[88186]: osd.2 marked itself down and dead 2024-03-29T04:52:35.226 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:34 smithi142 podman[134688]: 2024-03-29 04:52:34.952947325 +0000 UTC m=+0.743350057 container died 75e480e18a87e561c0898d43634b2ae84acd6f1ca70a4050d4a24d9060331d43 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:52:35.606 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:35 smithi142 podman[134688]: 2024-03-29 04:52:35.32068704 +0000 UTC m=+1.111089759 container cleanup 75e480e18a87e561c0898d43634b2ae84acd6f1ca70a4050d4a24d9060331d43 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1) 2024-03-29T04:52:35.606 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:35 smithi142 bash[134688]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2 2024-03-29T04:52:35.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:35 smithi142 ceph-mon[112475]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T04:52:35.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:35 smithi142 ceph-mon[112475]: osdmap e74: 6 total, 5 up, 6 in 2024-03-29T04:52:35.876 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:35 smithi142 podman[134707]: 2024-03-29 04:52:35.630770998 +0000 UTC m=+0.671857575 container remove 75e480e18a87e561c0898d43634b2ae84acd6f1ca70a4050d4a24d9060331d43 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_CLEAN=True) 2024-03-29T04:52:36.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:35 smithi152 ceph-mon[88186]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T04:52:36.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:35 smithi152 ceph-mon[88186]: osdmap e74: 6 total, 5 up, 6 in 2024-03-29T04:52:36.280 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:35 smithi142 podman[134800]: 2024-03-29 04:52:35.891029129 +0000 UTC m=+0.018213251 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:52:36.280 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:35 smithi142 podman[134800]: 2024-03-29 04:52:35.99010367 +0000 UTC m=+0.117287778 container create 5bdcf0fbb8645b41031ba35a82dc7434fb11c0e937283febb0c2d58af23815ba (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, RELEASE=squid-8da2362, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:52:36.280 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:36 smithi142 podman[134800]: 2024-03-29 04:52:36.201279431 +0000 UTC m=+0.328463545 container init 5bdcf0fbb8645b41031ba35a82dc7434fb11c0e937283febb0c2d58af23815ba (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, RELEASE=squid-8da2362, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-29T04:52:36.280 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:36 smithi142 podman[134800]: 2024-03-29 04:52:36.20475663 +0000 UTC m=+0.331940742 container start 5bdcf0fbb8645b41031ba35a82dc7434fb11c0e937283febb0c2d58af23815ba (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-8da2362, org.label-schema.schema-version=1.0, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-03-29T04:52:36.280 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:36 smithi142 podman[134800]: 2024-03-29 04:52:36.275726752 +0000 UTC m=+0.402910875 container attach 5bdcf0fbb8645b41031ba35a82dc7434fb11c0e937283febb0c2d58af23815ba (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, RELEASE=squid-8da2362) 2024-03-29T04:52:36.608 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:36 smithi142 podman[134800]: 2024-03-29 04:52:36.503351216 +0000 UTC m=+0.630535327 container died 5bdcf0fbb8645b41031ba35a82dc7434fb11c0e937283febb0c2d58af23815ba (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-deactivate, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-29T04:52:36.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:36 smithi142 ceph-mon[112475]: pgmap v104: 65 pgs: 13 peering, 7 stale+active+clean, 45 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-29T04:52:36.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:36 smithi142 ceph-mon[112475]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-29T04:52:36.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:36 smithi142 ceph-mon[112475]: osdmap e75: 6 total, 5 up, 6 in 2024-03-29T04:52:37.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:36 smithi152 ceph-mon[88186]: pgmap v104: 65 pgs: 13 peering, 7 stale+active+clean, 45 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-29T04:52:37.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:36 smithi152 ceph-mon[88186]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-29T04:52:37.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:36 smithi152 ceph-mon[88186]: osdmap e75: 6 total, 5 up, 6 in 2024-03-29T04:52:37.369 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:37 smithi142 podman[134826]: 2024-03-29 04:52:37.129547875 +0000 UTC m=+0.616532570 container remove 5bdcf0fbb8645b41031ba35a82dc7434fb11c0e937283febb0c2d58af23815ba (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8da2362, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True) 2024-03-29T04:52:37.369 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:37 smithi142 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-29T04:52:37.369 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:37 smithi142 systemd[1]: Stopped Ceph osd.2 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:52:37.369 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:37 smithi142 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.2.service: Consumed 58.184s CPU time. 2024-03-29T04:52:37.626 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:37 smithi142 systemd[1]: Starting Ceph osd.2 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:52:37.626 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:37 smithi142 podman[134928]: 2024-03-29 04:52:37.484988194 +0000 UTC m=+0.018106654 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:52:37.626 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:37 smithi142 podman[134928]: 2024-03-29 04:52:37.61449437 +0000 UTC m=+0.147612811 container create 30ba89dcdb4e36c32bc0c040bec50c6a27a2ecd9ccff8bd7e987b3e9f72921d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8da2362, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-29T04:52:38.126 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:37 smithi142 podman[134928]: 2024-03-29 04:52:37.817468918 +0000 UTC m=+0.350587362 container init 30ba89dcdb4e36c32bc0c040bec50c6a27a2ecd9ccff8bd7e987b3e9f72921d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8da2362) 2024-03-29T04:52:38.126 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:37 smithi142 podman[134928]: 2024-03-29 04:52:37.820877404 +0000 UTC m=+0.353995850 container start 30ba89dcdb4e36c32bc0c040bec50c6a27a2ecd9ccff8bd7e987b3e9f72921d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8da2362) 2024-03-29T04:52:38.126 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:37 smithi142 podman[134928]: 2024-03-29 04:52:37.908869875 +0000 UTC m=+0.441988351 container attach 30ba89dcdb4e36c32bc0c040bec50c6a27a2ecd9ccff8bd7e987b3e9f72921d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-03-29T04:52:38.611 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate[134944]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-29T04:52:38.611 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate[134944]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-29T04:52:38.611 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 bash[134928]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-29T04:52:38.611 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 bash[134928]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-29T04:52:38.611 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 bash[134928]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-29T04:52:38.612 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 bash[134928]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-29T04:52:38.612 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 bash[134928]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-29T04:52:38.612 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 bash[134928]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-29T04:52:38.612 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 bash[134928]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-29T04:52:38.612 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate[134944]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-29T04:52:38.612 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate[134944]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-29T04:52:38.612 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate[134944]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-29T04:52:38.612 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate[134944]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-29T04:52:38.612 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate[134944]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-29T04:52:38.612 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 podman[134928]: 2024-03-29 04:52:38.291804009 +0000 UTC m=+0.824922444 container died 30ba89dcdb4e36c32bc0c040bec50c6a27a2ecd9ccff8bd7e987b3e9f72921d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-8da2362, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.build-date=20240326) 2024-03-29T04:52:38.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:38 smithi142 ceph-mon[112475]: pgmap v106: 65 pgs: 13 peering, 7 stale+active+clean, 45 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 2 op/s 2024-03-29T04:52:39.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:38 smithi152 ceph-mon[88186]: pgmap v106: 65 pgs: 13 peering, 7 stale+active+clean, 45 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 2 op/s 2024-03-29T04:52:39.281 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:38 smithi142 podman[135084]: 2024-03-29 04:52:38.946497458 +0000 UTC m=+0.645179814 container remove 30ba89dcdb4e36c32bc0c040bec50c6a27a2ecd9ccff8bd7e987b3e9f72921d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-activate, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-29T04:52:39.281 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:39 smithi142 podman[135152]: 2024-03-29 04:52:39.060339304 +0000 UTC m=+0.017855947 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:52:39.281 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:39 smithi142 podman[135152]: 2024-03-29 04:52:39.172451626 +0000 UTC m=+0.129968292 container create ce765131932542e8de854311b3bf5b71cedd9f40e73ff065e3b64ccb6a925edb (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, RELEASE=squid-8da2362, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-29T04:52:39.876 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:39 smithi142 podman[135152]: 2024-03-29 04:52:39.391511361 +0000 UTC m=+0.349027983 container init ce765131932542e8de854311b3bf5b71cedd9f40e73ff065e3b64ccb6a925edb (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, maintainer=Guillaume Abrioux , ceph=True) 2024-03-29T04:52:39.876 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:39 smithi142 podman[135152]: 2024-03-29 04:52:39.394592434 +0000 UTC m=+0.352109055 container start ce765131932542e8de854311b3bf5b71cedd9f40e73ff065e3b64ccb6a925edb (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362) 2024-03-29T04:52:39.877 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:39 smithi142 ceph-osd[135172]: -- 172.21.15.142:0/1054718066 <== mon.0 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561496e6f680 con 0x561496e39000 2024-03-29T04:52:39.877 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:39 smithi142 bash[135152]: ce765131932542e8de854311b3bf5b71cedd9f40e73ff065e3b64ccb6a925edb 2024-03-29T04:52:39.877 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:39 smithi142 systemd[1]: Started Ceph osd.2 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:52:40.359 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:39 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2[135168]: 2024-03-29T04:52:39.935+0000 7fc58249e7c0 -1 Falling back to public interface 2024-03-29T04:52:40.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:40 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:40.615 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:40 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:40.616 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:40 smithi142 ceph-mon[112475]: pgmap v107: 65 pgs: 9 active+undersized, 8 active+undersized+degraded, 13 peering, 35 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28/333 objects degraded (8.408%) 2024-03-29T04:52:40.616 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:40 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:52:40.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:40 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:40.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:40 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:40.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:40 smithi152 ceph-mon[88186]: pgmap v107: 65 pgs: 9 active+undersized, 8 active+undersized+degraded, 13 peering, 35 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28/333 objects degraded (8.408%) 2024-03-29T04:52:40.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:40 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:52:41.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:41 smithi152 ceph-mon[88186]: Health check failed: Degraded data redundancy: 28/333 objects degraded (8.408%), 8 pgs degraded (PG_DEGRADED) 2024-03-29T04:52:41.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:41 smithi142 ceph-mon[112475]: Health check failed: Degraded data redundancy: 28/333 objects degraded (8.408%), 8 pgs degraded (PG_DEGRADED) 2024-03-29T04:52:42.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:42 smithi152 ceph-mon[88186]: pgmap v108: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 48/333 objects degraded (14.414%) 2024-03-29T04:52:42.808 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:42 smithi142 ceph-mon[112475]: pgmap v108: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 48/333 objects degraded (14.414%) 2024-03-29T04:52:43.581 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:43 smithi142 ceph-mon[112475]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-03-29T04:52:43.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:43 smithi152 ceph-mon[88186]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-03-29T04:52:44.549 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:44 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2[135168]: 2024-03-29T04:52:44.356+0000 7fc58249e7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-29T04:52:44.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:44 smithi142 ceph-mon[112475]: pgmap v109: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 48/333 objects degraded (14.414%) 2024-03-29T04:52:45.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:44 smithi152 ceph-mon[88186]: pgmap v109: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 48/333 objects degraded (14.414%) 2024-03-29T04:52:45.376 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:44 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2[135168]: 2024-03-29T04:52:44.925+0000 7fc58249e7c0 -1 osd.2 73 log_to_monitors true 2024-03-29T04:52:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:52:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:45 smithi142 ceph-mon[112475]: from='osd.2 [v2:172.21.15.142:6818/658903196,v1:172.21.15.142:6819/658903196]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-29T04:52:46.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:46.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:52:46.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:46.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:46.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:45 smithi152 ceph-mon[88186]: from='osd.2 [v2:172.21.15.142:6818/658903196,v1:172.21.15.142:6819/658903196]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-29T04:52:46.376 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 04:52:45 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2[135168]: 2024-03-29T04:52:45.918+0000 7fc5738f6700 -1 osd.2 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:52:46.846 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:46 smithi142 ceph-mon[112475]: pgmap v110: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 48/333 objects degraded (14.414%) 2024-03-29T04:52:46.846 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:46 smithi142 ceph-mon[112475]: from='osd.2 [v2:172.21.15.142:6818/658903196,v1:172.21.15.142:6819/658903196]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-29T04:52:46.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:46 smithi142 ceph-mon[112475]: osdmap e76: 6 total, 5 up, 6 in 2024-03-29T04:52:46.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:46 smithi142 ceph-mon[112475]: from='osd.2 [v2:172.21.15.142:6818/658903196,v1:172.21.15.142:6819/658903196]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-29T04:52:46.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:46 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:46.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:46 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:46.847 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:46 smithi142 ceph-mon[112475]: Health check update: Degraded data redundancy: 48/333 objects degraded (14.414%), 14 pgs degraded (PG_DEGRADED) 2024-03-29T04:52:47.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:46 smithi152 ceph-mon[88186]: pgmap v110: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 48/333 objects degraded (14.414%) 2024-03-29T04:52:47.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:46 smithi152 ceph-mon[88186]: from='osd.2 [v2:172.21.15.142:6818/658903196,v1:172.21.15.142:6819/658903196]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-29T04:52:47.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:46 smithi152 ceph-mon[88186]: osdmap e76: 6 total, 5 up, 6 in 2024-03-29T04:52:47.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:46 smithi152 ceph-mon[88186]: from='osd.2 [v2:172.21.15.142:6818/658903196,v1:172.21.15.142:6819/658903196]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-29T04:52:47.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:46 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:47.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:46 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:47.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:46 smithi152 ceph-mon[88186]: Health check update: Degraded data redundancy: 48/333 objects degraded (14.414%), 14 pgs degraded (PG_DEGRADED) 2024-03-29T04:52:47.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:47 smithi142 ceph-mon[112475]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T04:52:47.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:47 smithi142 ceph-mon[112475]: osd.2 [v2:172.21.15.142:6818/658903196,v1:172.21.15.142:6819/658903196] boot 2024-03-29T04:52:47.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:47 smithi142 ceph-mon[112475]: osdmap e77: 6 total, 6 up, 6 in 2024-03-29T04:52:47.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:47 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:52:48.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:47 smithi152 ceph-mon[88186]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T04:52:48.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:47 smithi152 ceph-mon[88186]: osd.2 [v2:172.21.15.142:6818/658903196,v1:172.21.15.142:6819/658903196] boot 2024-03-29T04:52:48.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:47 smithi152 ceph-mon[88186]: osdmap e77: 6 total, 6 up, 6 in 2024-03-29T04:52:48.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:47 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T04:52:48.769 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:48 smithi142 ceph-mon[112475]: pgmap v113: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s; 48/333 objects degraded (14.414%) 2024-03-29T04:52:48.769 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:48 smithi142 ceph-mon[112475]: osdmap e78: 6 total, 6 up, 6 in 2024-03-29T04:52:49.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:48 smithi152 ceph-mon[88186]: pgmap v113: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s; 48/333 objects degraded (14.414%) 2024-03-29T04:52:49.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:48 smithi152 ceph-mon[88186]: osdmap e78: 6 total, 6 up, 6 in 2024-03-29T04:52:50.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:50.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:50.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:52:50.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:52:50.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:50.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: pgmap v115: 65 pgs: 6 peering, 10 active+undersized, 10 active+undersized+degraded, 39 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 35/333 objects degraded (10.511%) 2024-03-29T04:52:50.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:52:50.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:50.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:50.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:50.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T04:52:50.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T04:52:50.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:50 smithi152 ceph-mon[88186]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-03-29T04:52:50.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:50.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:50.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:52:50.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:52:50.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:52:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: pgmap v115: 65 pgs: 6 peering, 10 active+undersized, 10 active+undersized+degraded, 39 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 35/333 objects degraded (10.511%) 2024-03-29T04:52:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:52:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:52:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T04:52:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T04:52:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:50 smithi142 ceph-mon[112475]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-03-29T04:52:51.697 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:51 smithi152 ceph-mon[88186]: Health check update: Degraded data redundancy: 35/333 objects degraded (10.511%), 10 pgs degraded (PG_DEGRADED) 2024-03-29T04:52:51.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:51 smithi142 ceph-mon[112475]: Health check update: Degraded data redundancy: 35/333 objects degraded (10.511%), 10 pgs degraded (PG_DEGRADED) 2024-03-29T04:52:52.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:52 smithi152 ceph-mon[88186]: pgmap v116: 65 pgs: 6 peering, 59 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-29T04:52:52.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:52 smithi142 ceph-mon[112475]: pgmap v116: 65 pgs: 6 peering, 59 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-29T04:52:53.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:53 smithi152 ceph-mon[88186]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 35/333 objects degraded (10.511%), 10 pgs degraded) 2024-03-29T04:52:53.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:53 smithi152 ceph-mon[88186]: Cluster is now healthy 2024-03-29T04:52:53.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:53 smithi142 ceph-mon[112475]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 35/333 objects degraded (10.511%), 10 pgs degraded) 2024-03-29T04:52:53.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:53 smithi142 ceph-mon[112475]: Cluster is now healthy 2024-03-29T04:52:54.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:54 smithi152 ceph-mon[88186]: pgmap v117: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-03-29T04:52:54.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:54 smithi142 ceph-mon[112475]: pgmap v117: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-03-29T04:52:56.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:56 smithi152 ceph-mon[88186]: pgmap v118: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:52:56.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:56 smithi142 ceph-mon[112475]: pgmap v118: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:52:58.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:52:58 smithi142 ceph-mon[112475]: pgmap v119: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-29T04:52:59.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:52:58 smithi152 ceph-mon[88186]: pgmap v119: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-29T04:52:59.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.958+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2117050998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf6c0ffeb0 msgr2=0x7fdf6c105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:59.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.958+0000 7fdf72e2e700 1 --2- 172.21.15.142:0/2117050998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf6c0ffeb0 0x7fdf6c105840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009910 tx=0x7fdf5c009c20 comp rx=0 tx=0).stop 2024-03-29T04:52:59.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.958+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2117050998 shutdown_connections 2024-03-29T04:52:59.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.958+0000 7fdf72e2e700 1 --2- 172.21.15.142:0/2117050998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf6c0ffeb0 0x7fdf6c105840 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:59.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.958+0000 7fdf72e2e700 1 --2- 172.21.15.142:0/2117050998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf6c0ff500 0x7fdf6c0ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:59.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.958+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2117050998 >> 172.21.15.142:0/2117050998 conn(0x7fdf6c074d80 msgr2=0x7fdf6c075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:52:59.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.959+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2117050998 shutdown_connections 2024-03-29T04:52:59.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.959+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2117050998 wait complete. 2024-03-29T04:52:59.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.959+0000 7fdf72e2e700 1 Processor -- start 2024-03-29T04:52:59.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.959+0000 7fdf72e2e700 1 -- start start 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.959+0000 7fdf72e2e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf6c0ff500 0x7fdf6c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf72e2e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf6c0ffeb0 0x7fdf6c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf72e2e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c19d100 con 0x7fdf6c0ffeb0 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf72e2e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c1965b0 con 0x7fdf6c0ff500 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf6bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf6c0ffeb0 0x7fdf6c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf6bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf6c0ffeb0 0x7fdf6c19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33774/0 (socket says 172.21.15.142:33774) 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf6bfff700 1 -- 172.21.15.142:0/2486505580 learned_addr learned my addr 172.21.15.142:0/2486505580 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf70bca700 1 --2- 172.21.15.142:0/2486505580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf6c0ff500 0x7fdf6c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf6bfff700 1 -- 172.21.15.142:0/2486505580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf6c0ff500 msgr2=0x7fdf6c19c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf6bfff700 1 --2- 172.21.15.142:0/2486505580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf6c0ff500 0x7fdf6c19c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf6bfff700 1 -- 172.21.15.142:0/2486505580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c0095f0 con 0x7fdf6c0ffeb0 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf70bca700 1 --2- 172.21.15.142:0/2486505580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf6c0ff500 0x7fdf6c19c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf6bfff700 1 --2- 172.21.15.142:0/2486505580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf6c0ffeb0 0x7fdf6c19ca70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0059d0 tx=0x7fdf5c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:59.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf69ffb700 1 -- 172.21.15.142:0/2486505580 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf5c01d070 con 0x7fdf6c0ffeb0 2024-03-29T04:52:59.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf69ffb700 1 -- 172.21.15.142:0/2486505580 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf5c00b610 con 0x7fdf6c0ffeb0 2024-03-29T04:52:59.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2486505580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6c196830 con 0x7fdf6c0ffeb0 2024-03-29T04:52:59.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.960+0000 7fdf69ffb700 1 -- 172.21.15.142:0/2486505580 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf5c022470 con 0x7fdf6c0ffeb0 2024-03-29T04:52:59.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.961+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2486505580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6c196d20 con 0x7fdf6c0ffeb0 2024-03-29T04:52:59.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.962+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2486505580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6c1090e0 con 0x7fdf6c0ffeb0 2024-03-29T04:52:59.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.962+0000 7fdf69ffb700 1 -- 172.21.15.142:0/2486505580 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fdf5c003710 con 0x7fdf6c0ffeb0 2024-03-29T04:52:59.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.963+0000 7fdf69ffb700 1 --2- 172.21.15.142:0/2486505580 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdf5806dd60 0x7fdf58070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:52:59.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.963+0000 7fdf69ffb700 1 -- 172.21.15.142:0/2486505580 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fdf5c08f0b0 con 0x7fdf6c0ffeb0 2024-03-29T04:52:59.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.963+0000 7fdf70bca700 1 --2- 172.21.15.142:0/2486505580 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdf5806dd60 0x7fdf58070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:52:59.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.963+0000 7fdf70bca700 1 --2- 172.21.15.142:0/2486505580 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdf5806dd60 0x7fdf58070220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdf54011490 tx=0x7fdf54009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:52:59.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:52:59.968+0000 7fdf69ffb700 1 -- 172.21.15.142:0/2486505580 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdf5c05c0b0 con 0x7fdf6c0ffeb0 2024-03-29T04:53:00.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.181+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2486505580 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf6c02ccc0 con 0x7fdf5806dd60 2024-03-29T04:53:00.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.183+0000 7fdf69ffb700 1 -- 172.21.15.142:0/2486505580 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdf6c02ccc0 con 0x7fdf5806dd60 2024-03-29T04:53:00.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.185+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2486505580 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdf5806dd60 msgr2=0x7fdf58070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:00.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.185+0000 7fdf72e2e700 1 --2- 172.21.15.142:0/2486505580 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdf5806dd60 0x7fdf58070220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdf54011490 tx=0x7fdf54009260 comp rx=0 tx=0).stop 2024-03-29T04:53:00.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.185+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2486505580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf6c0ffeb0 msgr2=0x7fdf6c19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:00.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.185+0000 7fdf72e2e700 1 --2- 172.21.15.142:0/2486505580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf6c0ffeb0 0x7fdf6c19ca70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0059d0 tx=0x7fdf5c00f690 comp rx=0 tx=0).stop 2024-03-29T04:53:00.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.185+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2486505580 shutdown_connections 2024-03-29T04:53:00.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.185+0000 7fdf72e2e700 1 --2- 172.21.15.142:0/2486505580 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdf5806dd60 0x7fdf58070220 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.185+0000 7fdf72e2e700 1 --2- 172.21.15.142:0/2486505580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdf6c0ff500 0x7fdf6c19c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.185+0000 7fdf72e2e700 1 --2- 172.21.15.142:0/2486505580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf6c0ffeb0 0x7fdf6c19ca70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.185+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2486505580 >> 172.21.15.142:0/2486505580 conn(0x7fdf6c074d80 msgr2=0x7fdf6c1050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:00.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.185+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2486505580 shutdown_connections 2024-03-29T04:53:00.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.185+0000 7fdf72e2e700 1 -- 172.21.15.142:0/2486505580 wait complete. 2024-03-29T04:53:00.201 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:53:00.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.290+0000 7f3665bd0700 1 -- 172.21.15.142:0/2746343970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36600ff8a0 msgr2=0x7f36600ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:00.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.290+0000 7f3665bd0700 1 --2- 172.21.15.142:0/2746343970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36600ff8a0 0x7f36600ffc80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3650009910 tx=0x7f3650009c20 comp rx=0 tx=0).stop 2024-03-29T04:53:00.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.290+0000 7f3665bd0700 1 -- 172.21.15.142:0/2746343970 shutdown_connections 2024-03-29T04:53:00.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.290+0000 7f3665bd0700 1 --2- 172.21.15.142:0/2746343970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3660100250 0x7f366010cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.290+0000 7f3665bd0700 1 --2- 172.21.15.142:0/2746343970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36600ff8a0 0x7f36600ffc80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.290+0000 7f3665bd0700 1 -- 172.21.15.142:0/2746343970 >> 172.21.15.142:0/2746343970 conn(0x7f36600fb540 msgr2=0x7f36600fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:00.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.290+0000 7f3665bd0700 1 -- 172.21.15.142:0/2746343970 shutdown_connections 2024-03-29T04:53:00.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.290+0000 7f3665bd0700 1 -- 172.21.15.142:0/2746343970 wait complete. 2024-03-29T04:53:00.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f3665bd0700 1 Processor -- start 2024-03-29T04:53:00.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f3665bd0700 1 -- start start 2024-03-29T04:53:00.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f3665bd0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36600ff8a0 0x7f366019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:00.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f3665bd0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3660100250 0x7f366019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:00.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f3665bd0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366019f230 con 0x7f36600ff8a0 2024-03-29T04:53:00.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f3665bd0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3660198770 con 0x7f3660100250 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f365f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36600ff8a0 0x7f366019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f365f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36600ff8a0 0x7f366019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33804/0 (socket says 172.21.15.142:33804) 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f365f7fe700 1 -- 172.21.15.142:0/2753434025 learned_addr learned my addr 172.21.15.142:0/2753434025 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f365effd700 1 --2- 172.21.15.142:0/2753434025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3660100250 0x7f366019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f365f7fe700 1 -- 172.21.15.142:0/2753434025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3660100250 msgr2=0x7f366019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f365f7fe700 1 --2- 172.21.15.142:0/2753434025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3660100250 0x7f366019ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.291+0000 7f365f7fe700 1 -- 172.21.15.142:0/2753434025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36500095f0 con 0x7f36600ff8a0 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.292+0000 7f365f7fe700 1 --2- 172.21.15.142:0/2753434025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36600ff8a0 0x7f366019e6f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3650006010 tx=0x7f36500056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.292+0000 7f365cff9700 1 -- 172.21.15.142:0/2753434025 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365001d070 con 0x7f36600ff8a0 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.292+0000 7f365cff9700 1 -- 172.21.15.142:0/2753434025 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f365000f460 con 0x7f36600ff8a0 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.292+0000 7f3665bd0700 1 -- 172.21.15.142:0/2753434025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36601989f0 con 0x7f36600ff8a0 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.292+0000 7f365cff9700 1 -- 172.21.15.142:0/2753434025 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3650021640 con 0x7f36600ff8a0 2024-03-29T04:53:00.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.292+0000 7f3665bd0700 1 -- 172.21.15.142:0/2753434025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3660198f40 con 0x7f36600ff8a0 2024-03-29T04:53:00.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.293+0000 7f3665bd0700 1 -- 172.21.15.142:0/2753434025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366010a680 con 0x7f36600ff8a0 2024-03-29T04:53:00.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.293+0000 7f365cff9700 1 -- 172.21.15.142:0/2753434025 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f365000fad0 con 0x7f36600ff8a0 2024-03-29T04:53:00.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.294+0000 7f365cff9700 1 --2- 172.21.15.142:0/2753434025 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f364806dd60 0x7f3648070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:00.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.294+0000 7f365cff9700 1 -- 172.21.15.142:0/2753434025 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f365008ee30 con 0x7f36600ff8a0 2024-03-29T04:53:00.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.294+0000 7f365effd700 1 --2- 172.21.15.142:0/2753434025 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f364806dd60 0x7f3648070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:00.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.294+0000 7f365effd700 1 --2- 172.21.15.142:0/2753434025 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f364806dd60 0x7f3648070220 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3660199be0 tx=0x7f3654005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:00.304 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.299+0000 7f365cff9700 1 -- 172.21.15.142:0/2753434025 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f365005beb0 con 0x7f36600ff8a0 2024-03-29T04:53:00.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.510+0000 7f3665bd0700 1 -- 172.21.15.142:0/2753434025 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3660199d90 con 0x7f364806dd60 2024-03-29T04:53:00.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.511+0000 7f365cff9700 1 -- 172.21.15.142:0/2753434025 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3660199d90 con 0x7f364806dd60 2024-03-29T04:53:00.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.513+0000 7f3665bd0700 1 -- 172.21.15.142:0/2753434025 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f364806dd60 msgr2=0x7f3648070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:00.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.513+0000 7f3665bd0700 1 --2- 172.21.15.142:0/2753434025 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f364806dd60 0x7f3648070220 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3660199be0 tx=0x7f3654005d50 comp rx=0 tx=0).stop 2024-03-29T04:53:00.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.513+0000 7f3665bd0700 1 -- 172.21.15.142:0/2753434025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36600ff8a0 msgr2=0x7f366019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:00.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.513+0000 7f3665bd0700 1 --2- 172.21.15.142:0/2753434025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36600ff8a0 0x7f366019e6f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3650006010 tx=0x7f36500056a0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.514+0000 7f3665bd0700 1 -- 172.21.15.142:0/2753434025 shutdown_connections 2024-03-29T04:53:00.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.514+0000 7f3665bd0700 1 --2- 172.21.15.142:0/2753434025 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f364806dd60 0x7f3648070220 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.514+0000 7f3665bd0700 1 --2- 172.21.15.142:0/2753434025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36600ff8a0 0x7f366019e6f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.514+0000 7f3665bd0700 1 --2- 172.21.15.142:0/2753434025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3660100250 0x7f366019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.514+0000 7f3665bd0700 1 -- 172.21.15.142:0/2753434025 >> 172.21.15.142:0/2753434025 conn(0x7f36600fb540 msgr2=0x7f36600fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:00.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.514+0000 7f3665bd0700 1 -- 172.21.15.142:0/2753434025 shutdown_connections 2024-03-29T04:53:00.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.514+0000 7f3665bd0700 1 -- 172.21.15.142:0/2753434025 wait complete. 2024-03-29T04:53:00.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.618+0000 7fb52a8a1700 1 -- 172.21.15.142:0/3008609796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5240ffa40 msgr2=0x7fb5240ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:00.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.618+0000 7fb52a8a1700 1 --2- 172.21.15.142:0/3008609796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5240ffa40 0x7fb5240ffec0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb518009910 tx=0x7fb518009c20 comp rx=0 tx=0).stop 2024-03-29T04:53:00.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.619+0000 7fb52a8a1700 1 -- 172.21.15.142:0/3008609796 shutdown_connections 2024-03-29T04:53:00.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.619+0000 7fb52a8a1700 1 --2- 172.21.15.142:0/3008609796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5240ffa40 0x7fb5240ffec0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.619+0000 7fb52a8a1700 1 --2- 172.21.15.142:0/3008609796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb524105a40 0x7fb524105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.619+0000 7fb52a8a1700 1 -- 172.21.15.142:0/3008609796 >> 172.21.15.142:0/3008609796 conn(0x7fb5240fb550 msgr2=0x7fb5240fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:00.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.619+0000 7fb52a8a1700 1 -- 172.21.15.142:0/3008609796 shutdown_connections 2024-03-29T04:53:00.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.619+0000 7fb52a8a1700 1 -- 172.21.15.142:0/3008609796 wait complete. 2024-03-29T04:53:00.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.619+0000 7fb52a8a1700 1 Processor -- start 2024-03-29T04:53:00.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.619+0000 7fb52a8a1700 1 -- start start 2024-03-29T04:53:00.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb52a8a1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5240ffa40 0x7fb524198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:00.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb52a8a1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb524105a40 0x7fb524198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:00.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb52a8a1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb524199540 con 0x7fb5240ffa40 2024-03-29T04:53:00.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb52a8a1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52419d350 con 0x7fb524105a40 2024-03-29T04:53:00.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb523fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5240ffa40 0x7fb524198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:00.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb523fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5240ffa40 0x7fb524198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33826/0 (socket says 172.21.15.142:33826) 2024-03-29T04:53:00.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb523fff700 1 -- 172.21.15.142:0/2163307789 learned_addr learned my addr 172.21.15.142:0/2163307789 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:00.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb523fff700 1 -- 172.21.15.142:0/2163307789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb524105a40 msgr2=0x7fb524198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:00.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb523fff700 1 --2- 172.21.15.142:0/2163307789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb524105a40 0x7fb524198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb523fff700 1 -- 172.21.15.142:0/2163307789 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180095f0 con 0x7fb5240ffa40 2024-03-29T04:53:00.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb523fff700 1 --2- 172.21.15.142:0/2163307789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5240ffa40 0x7fb524198970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb51400d8d0 tx=0x7fb51400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:00.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb5217fa700 1 -- 172.21.15.142:0/2163307789 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb514009940 con 0x7fb5240ffa40 2024-03-29T04:53:00.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb5217fa700 1 -- 172.21.15.142:0/2163307789 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb514010460 con 0x7fb5240ffa40 2024-03-29T04:53:00.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb52a8a1700 1 -- 172.21.15.142:0/2163307789 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52419d630 con 0x7fb5240ffa40 2024-03-29T04:53:00.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb5217fa700 1 -- 172.21.15.142:0/2163307789 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb51400f5f0 con 0x7fb5240ffa40 2024-03-29T04:53:00.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.620+0000 7fb52a8a1700 1 -- 172.21.15.142:0/2163307789 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52419db80 con 0x7fb5240ffa40 2024-03-29T04:53:00.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.621+0000 7fb52a8a1700 1 -- 172.21.15.142:0/2163307789 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb524107fd0 con 0x7fb5240ffa40 2024-03-29T04:53:00.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.622+0000 7fb5217fa700 1 -- 172.21.15.142:0/2163307789 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fb5140105d0 con 0x7fb5240ffa40 2024-03-29T04:53:00.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.622+0000 7fb5217fa700 1 --2- 172.21.15.142:0/2163307789 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb50c06dd60 0x7fb50c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:00.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.622+0000 7fb5217fa700 1 -- 172.21.15.142:0/2163307789 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fb51408dc20 con 0x7fb5240ffa40 2024-03-29T04:53:00.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.622+0000 7fb5237fe700 1 --2- 172.21.15.142:0/2163307789 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb50c06dd60 0x7fb50c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:00.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.623+0000 7fb5237fe700 1 --2- 172.21.15.142:0/2163307789 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb50c06dd60 0x7fb50c070220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb518009fd0 tx=0x7fb51800b560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:00.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.627+0000 7fb5217fa700 1 -- 172.21.15.142:0/2163307789 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb51405ac20 con 0x7fb5240ffa40 2024-03-29T04:53:00.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.829+0000 7fb52a8a1700 1 -- 172.21.15.142:0/2163307789 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb52419de60 con 0x7fb50c06dd60 2024-03-29T04:53:00.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:00 smithi142 ceph-mon[112475]: pgmap v120: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-29T04:53:00.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:00.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:53:00.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:00.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.840+0000 7fb5217fa700 1 -- 172.21.15.142:0/2163307789 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb52419de60 con 0x7fb50c06dd60 2024-03-29T04:53:00.846 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:53:00.846 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (4m) 15s ago 16m 25.5M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:53:00.846 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (16m) 15s ago 16m 8762k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:53:00.846 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (15m) 111s ago 15m 8182k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:53:00.846 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (2m) 15s ago 16m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:53:00.846 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (114s) 111s ago 15m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:53:00.846 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (4m) 15s ago 16m 80.3M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:53:00.846 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (10m) 15s ago 10m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:53:00.846 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (10m) 15s ago 10m 63.5M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:53:00.846 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (10m) 111s ago 10m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:53:00.846 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (10m) 111s ago 10m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (6m) 15s ago 17m 506M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (5m) 111s ago 15m 449M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (3m) 15s ago 17m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (2m) 111s ago 15m 41.1M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (5m) 15s ago 16m 14.9M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (5m) 111s ago 15m 16.0M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (94s) 15s ago 14m 105M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (58s) 15s ago 14m 142M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (21s) 15s ago 13m 12.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (13m) 111s ago 13m 276M 2240M 18.2.1 5be31c24972a 575eedc75115 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (12m) 111s ago 12m 236M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (12m) 111s ago 12m 225M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:53:00.847 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (5m) 15s ago 15m 63.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:53:00.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.843+0000 7fb52a8a1700 1 -- 172.21.15.142:0/2163307789 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb50c06dd60 msgr2=0x7fb50c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:00.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.843+0000 7fb52a8a1700 1 --2- 172.21.15.142:0/2163307789 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb50c06dd60 0x7fb50c070220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb518009fd0 tx=0x7fb51800b560 comp rx=0 tx=0).stop 2024-03-29T04:53:00.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.843+0000 7fb52a8a1700 1 -- 172.21.15.142:0/2163307789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5240ffa40 msgr2=0x7fb524198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:00.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.843+0000 7fb52a8a1700 1 --2- 172.21.15.142:0/2163307789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5240ffa40 0x7fb524198970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb51400d8d0 tx=0x7fb51400dc90 comp rx=0 tx=0).stop 2024-03-29T04:53:00.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.843+0000 7fb52a8a1700 1 -- 172.21.15.142:0/2163307789 shutdown_connections 2024-03-29T04:53:00.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.843+0000 7fb52a8a1700 1 --2- 172.21.15.142:0/2163307789 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb50c06dd60 0x7fb50c070220 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.843+0000 7fb52a8a1700 1 --2- 172.21.15.142:0/2163307789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5240ffa40 0x7fb524198970 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.843+0000 7fb52a8a1700 1 --2- 172.21.15.142:0/2163307789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb524105a40 0x7fb524198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.843+0000 7fb52a8a1700 1 -- 172.21.15.142:0/2163307789 >> 172.21.15.142:0/2163307789 conn(0x7fb5240fb550 msgr2=0x7fb5240fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:00.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.843+0000 7fb52a8a1700 1 -- 172.21.15.142:0/2163307789 shutdown_connections 2024-03-29T04:53:00.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.843+0000 7fb52a8a1700 1 -- 172.21.15.142:0/2163307789 wait complete. 2024-03-29T04:53:00.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.946+0000 7f4baf870700 1 -- 172.21.15.142:0/1647629755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba8102620 msgr2=0x7f4ba810ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:00.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.946+0000 7f4baf870700 1 --2- 172.21.15.142:0/1647629755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba8102620 0x7f4ba810ab10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009910 tx=0x7f4b98009c20 comp rx=0 tx=0).stop 2024-03-29T04:53:00.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.946+0000 7f4baf870700 1 -- 172.21.15.142:0/1647629755 shutdown_connections 2024-03-29T04:53:00.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.946+0000 7f4baf870700 1 --2- 172.21.15.142:0/1647629755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba8102620 0x7f4ba810ab10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.946+0000 7f4baf870700 1 --2- 172.21.15.142:0/1647629755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ba8101c70 0x7f4ba8102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.946+0000 7f4baf870700 1 -- 172.21.15.142:0/1647629755 >> 172.21.15.142:0/1647629755 conn(0x7f4ba80fb570 msgr2=0x7f4ba80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:00.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.946+0000 7f4baf870700 1 -- 172.21.15.142:0/1647629755 shutdown_connections 2024-03-29T04:53:00.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.946+0000 7f4baf870700 1 -- 172.21.15.142:0/1647629755 wait complete. 2024-03-29T04:53:00.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.946+0000 7f4baf870700 1 Processor -- start 2024-03-29T04:53:00.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.946+0000 7f4baf870700 1 -- start start 2024-03-29T04:53:00.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4baf870700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ba8101c70 0x7f4ba80ff960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:00.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4baf870700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba8102620 0x7f4ba80ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:00.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4baf870700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba81003e0 con 0x7f4ba8102620 2024-03-29T04:53:00.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4baf870700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8100520 con 0x7f4ba8101c70 2024-03-29T04:53:00.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4bace0b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba8102620 0x7f4ba80ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:00.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4bace0b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba8102620 0x7f4ba80ffea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33844/0 (socket says 172.21.15.142:33844) 2024-03-29T04:53:00.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4bace0b700 1 -- 172.21.15.142:0/706190211 learned_addr learned my addr 172.21.15.142:0/706190211 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:00.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4bad60c700 1 --2- 172.21.15.142:0/706190211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ba8101c70 0x7f4ba80ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:00.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4bace0b700 1 -- 172.21.15.142:0/706190211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ba8101c70 msgr2=0x7f4ba80ff960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:00.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4bace0b700 1 --2- 172.21.15.142:0/706190211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ba8101c70 0x7f4ba80ff960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:00.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4bace0b700 1 -- 172.21.15.142:0/706190211 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b980095f0 con 0x7f4ba8102620 2024-03-29T04:53:00.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4bace0b700 1 --2- 172.21.15.142:0/706190211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba8102620 0x7f4ba80ffea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b98005f50 tx=0x7f4b9800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:00.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4b9e7fc700 1 -- 172.21.15.142:0/706190211 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b9801d070 con 0x7f4ba8102620 2024-03-29T04:53:00.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4b9e7fc700 1 -- 172.21.15.142:0/706190211 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b9800f460 con 0x7f4ba8102620 2024-03-29T04:53:00.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.947+0000 7f4baf870700 1 -- 172.21.15.142:0/706190211 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba81007a0 con 0x7f4ba8102620 2024-03-29T04:53:00.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.948+0000 7f4b9e7fc700 1 -- 172.21.15.142:0/706190211 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b98017630 con 0x7f4ba8102620 2024-03-29T04:53:00.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.948+0000 7f4baf870700 1 -- 172.21.15.142:0/706190211 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba81a2d50 con 0x7f4ba8102620 2024-03-29T04:53:00.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.949+0000 7f4baf870700 1 -- 172.21.15.142:0/706190211 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba8108210 con 0x7f4ba8102620 2024-03-29T04:53:00.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.949+0000 7f4b9e7fc700 1 -- 172.21.15.142:0/706190211 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f4b9800f5d0 con 0x7f4ba8102620 2024-03-29T04:53:00.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.949+0000 7f4b9e7fc700 1 --2- 172.21.15.142:0/706190211 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b9406dd60 0x7f4b94070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:00.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.950+0000 7f4b9e7fc700 1 -- 172.21.15.142:0/706190211 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f4b9808f060 con 0x7f4ba8102620 2024-03-29T04:53:00.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.950+0000 7f4bad60c700 1 --2- 172.21.15.142:0/706190211 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b9406dd60 0x7f4b94070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:00.957 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.950+0000 7f4bad60c700 1 --2- 172.21.15.142:0/706190211 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b9406dd60 0x7f4b94070220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009a10 tx=0x7f4ba4009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:00.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:00.954+0000 7f4b9e7fc700 1 -- 172.21.15.142:0/706190211 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4b980585b0 con 0x7f4ba8102620 2024-03-29T04:53:01.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:00 smithi152 ceph-mon[88186]: pgmap v120: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-29T04:53:01.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:01.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:53:01.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:01.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.232+0000 7f4baf870700 1 -- 172.21.15.142:0/706190211 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4ba804ea90 con 0x7f4ba8102620 2024-03-29T04:53:01.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.232+0000 7f4b9e7fc700 1 -- 172.21.15.142:0/706190211 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f4b9805bc70 con 0x7f4ba8102620 2024-03-29T04:53:01.237 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:53:01.237 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:53:01.237 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:53:01.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:53:01.238 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:53:01.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:53:01.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:53:01.238 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:53:01.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-03-29T04:53:01.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 3 2024-03-29T04:53:01.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:53:01.238 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:53:01.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:53:01.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:53:01.239 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:53:01.239 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-03-29T04:53:01.239 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 7 2024-03-29T04:53:01.239 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:53:01.239 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:53:01.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.234+0000 7f4baf870700 1 -- 172.21.15.142:0/706190211 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b9406dd60 msgr2=0x7f4b94070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:01.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.234+0000 7f4baf870700 1 --2- 172.21.15.142:0/706190211 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b9406dd60 0x7f4b94070220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009a10 tx=0x7f4ba4009190 comp rx=0 tx=0).stop 2024-03-29T04:53:01.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.234+0000 7f4baf870700 1 -- 172.21.15.142:0/706190211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba8102620 msgr2=0x7f4ba80ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:01.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.234+0000 7f4baf870700 1 --2- 172.21.15.142:0/706190211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba8102620 0x7f4ba80ffea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b98005f50 tx=0x7f4b9800bd70 comp rx=0 tx=0).stop 2024-03-29T04:53:01.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.234+0000 7f4baf870700 1 -- 172.21.15.142:0/706190211 shutdown_connections 2024-03-29T04:53:01.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.234+0000 7f4baf870700 1 --2- 172.21.15.142:0/706190211 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b9406dd60 0x7f4b94070220 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.234+0000 7f4baf870700 1 --2- 172.21.15.142:0/706190211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ba8101c70 0x7f4ba80ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.234+0000 7f4baf870700 1 --2- 172.21.15.142:0/706190211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba8102620 0x7f4ba80ffea0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.234+0000 7f4baf870700 1 -- 172.21.15.142:0/706190211 >> 172.21.15.142:0/706190211 conn(0x7f4ba80fb570 msgr2=0x7f4ba8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:01.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.234+0000 7f4baf870700 1 -- 172.21.15.142:0/706190211 shutdown_connections 2024-03-29T04:53:01.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.234+0000 7f4baf870700 1 -- 172.21.15.142:0/706190211 wait complete. 2024-03-29T04:53:01.342 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.337+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/232463617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d00ff400 msgr2=0x7ff3d00ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:01.342 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.337+0000 7ff3d4fa8700 1 --2- 172.21.15.142:0/232463617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d00ff400 0x7ff3d00ff7e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800c210 tx=0x7ff3c800c520 comp rx=0 tx=0).stop 2024-03-29T04:53:01.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.337+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/232463617 shutdown_connections 2024-03-29T04:53:01.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.337+0000 7ff3d4fa8700 1 --2- 172.21.15.142:0/232463617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3d00ffdb0 0x7ff3d0103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.337+0000 7ff3d4fa8700 1 --2- 172.21.15.142:0/232463617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d00ff400 0x7ff3d00ff7e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.337+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/232463617 >> 172.21.15.142:0/232463617 conn(0x7ff3d0075e30 msgr2=0x7ff3d00fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:01.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.337+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/232463617 shutdown_connections 2024-03-29T04:53:01.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.337+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/232463617 wait complete. 2024-03-29T04:53:01.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.338+0000 7ff3d4fa8700 1 Processor -- start 2024-03-29T04:53:01.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.338+0000 7ff3d4fa8700 1 -- start start 2024-03-29T04:53:01.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.338+0000 7ff3d4fa8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3d00ff400 0x7ff3d019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:01.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.338+0000 7ff3d4fa8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d00ffdb0 0x7ff3d019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:01.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.338+0000 7ff3d4fa8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d019d0f0 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.338+0000 7ff3d4fa8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d01965a0 con 0x7ff3d00ff400 2024-03-29T04:53:01.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.338+0000 7ff3cdd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d00ffdb0 0x7ff3d019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:01.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.338+0000 7ff3cdd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d00ffdb0 0x7ff3d019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33856/0 (socket says 172.21.15.142:33856) 2024-03-29T04:53:01.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.338+0000 7ff3cdd9b700 1 -- 172.21.15.142:0/1503793045 learned_addr learned my addr 172.21.15.142:0/1503793045 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:01.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.338+0000 7ff3ce59c700 1 --2- 172.21.15.142:0/1503793045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3d00ff400 0x7ff3d019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:01.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.339+0000 7ff3cdd9b700 1 -- 172.21.15.142:0/1503793045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3d00ff400 msgr2=0x7ff3d019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:01.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.339+0000 7ff3cdd9b700 1 --2- 172.21.15.142:0/1503793045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3d00ff400 0x7ff3d019c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.339+0000 7ff3cdd9b700 1 -- 172.21.15.142:0/1503793045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c800ae10 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.339+0000 7ff3cdd9b700 1 --2- 172.21.15.142:0/1503793045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d00ffdb0 0x7ff3d019ca60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400c8f0 tx=0x7ff3c400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:01.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.339+0000 7ff3bf7fe700 1 -- 172.21.15.142:0/1503793045 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3c40078a0 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.339+0000 7ff3bf7fe700 1 -- 172.21.15.142:0/1503793045 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3c400f450 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.339+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/1503793045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d0196880 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.339+0000 7ff3bf7fe700 1 -- 172.21.15.142:0/1503793045 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3c4018610 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.339+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/1503793045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d0196dd0 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.340+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/1503793045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d01097b0 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.341+0000 7ff3bf7fe700 1 -- 172.21.15.142:0/1503793045 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff3c400f5c0 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.341+0000 7ff3bf7fe700 1 --2- 172.21.15.142:0/1503793045 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff3b806dd60 0x7ff3b8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:01.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.341+0000 7ff3bf7fe700 1 -- 172.21.15.142:0/1503793045 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6704+0+0 (secure 0 0 0) 0x7ff3c408daf0 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.341+0000 7ff3ce59c700 1 --2- 172.21.15.142:0/1503793045 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff3b806dd60 0x7ff3b8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:01.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.341+0000 7ff3ce59c700 1 --2- 172.21.15.142:0/1503793045 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff3b806dd60 0x7ff3b8070220 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8006130 tx=0x7ff3c80060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:01.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.346+0000 7ff3bf7fe700 1 -- 172.21.15.142:0/1503793045 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff3c405aaf0 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.584+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/1503793045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff3d0197080 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.584+0000 7ff3bf7fe700 1 -- 172.21.15.142:0/1503793045 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1906 (secure 0 0 0) 0x7ff3c405a680 con 0x7ff3d00ffdb0 2024-03-29T04:53:01.590 INFO:teuthology.orchestra.run.smithi142.stdout:e21 2024-03-29T04:53:01.590 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:53:01.590 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:53:01.590 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:53:01.590 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:53:01.590 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:53:01.590 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:53:01.590 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 21 2024-03-29T04:53:01.590 INFO:teuthology.orchestra.run.smithi142.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T04:53:01.590 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:53:01.590 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:49:01.288637+0000 2024-03-29T04:53:01.590 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:53:01.591 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:53:01.591 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:53:01.591 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:53:01.591 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:53:01.591 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:53:01.591 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:53:01.591 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:53:01.591 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 47 2024-03-29T04:53:01.591 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:53:01.591 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:53:01.591 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:53:01.591 INFO:teuthology.orchestra.run.smithi142.stdout:up {0=14494,1=24275} 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{0:14494} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{0:14506} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{1:24275} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:53:01.592 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{1:24287} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:53:01.593 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:53:01.593 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:53:01.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.586+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/1503793045 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff3b806dd60 msgr2=0x7ff3b8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:01.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.586+0000 7ff3d4fa8700 1 --2- 172.21.15.142:0/1503793045 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff3b806dd60 0x7ff3b8070220 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8006130 tx=0x7ff3c80060c0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.586+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/1503793045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d00ffdb0 msgr2=0x7ff3d019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:01.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.586+0000 7ff3d4fa8700 1 --2- 172.21.15.142:0/1503793045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d00ffdb0 0x7ff3d019ca60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400c8f0 tx=0x7ff3c400cc00 comp rx=0 tx=0).stop 2024-03-29T04:53:01.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.586+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/1503793045 shutdown_connections 2024-03-29T04:53:01.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.586+0000 7ff3d4fa8700 1 --2- 172.21.15.142:0/1503793045 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff3b806dd60 0x7ff3b8070220 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.586+0000 7ff3d4fa8700 1 --2- 172.21.15.142:0/1503793045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3d00ff400 0x7ff3d019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.586+0000 7ff3d4fa8700 1 --2- 172.21.15.142:0/1503793045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d00ffdb0 0x7ff3d019ca60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.586+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/1503793045 >> 172.21.15.142:0/1503793045 conn(0x7ff3d0075e30 msgr2=0x7ff3d00fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:01.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.587+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/1503793045 shutdown_connections 2024-03-29T04:53:01.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.587+0000 7ff3d4fa8700 1 -- 172.21.15.142:0/1503793045 wait complete. 2024-03-29T04:53:01.594 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 21 2024-03-29T04:53:01.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.692+0000 7f29cde1e700 1 -- 172.21.15.142:0/424936347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c8100410 msgr2=0x7f29c81048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:01.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.692+0000 7f29cde1e700 1 --2- 172.21.15.142:0/424936347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c8100410 0x7f29c81048e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009910 tx=0x7f29bc009c20 comp rx=0 tx=0).stop 2024-03-29T04:53:01.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.692+0000 7f29cde1e700 1 -- 172.21.15.142:0/424936347 shutdown_connections 2024-03-29T04:53:01.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.692+0000 7f29cde1e700 1 --2- 172.21.15.142:0/424936347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c8100410 0x7f29c81048e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.692+0000 7f29cde1e700 1 --2- 172.21.15.142:0/424936347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c80ffa60 0x7f29c80ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.692+0000 7f29cde1e700 1 -- 172.21.15.142:0/424936347 >> 172.21.15.142:0/424936347 conn(0x7f29c80fb570 msgr2=0x7f29c80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:01.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.692+0000 7f29cde1e700 1 -- 172.21.15.142:0/424936347 shutdown_connections 2024-03-29T04:53:01.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.692+0000 7f29cde1e700 1 -- 172.21.15.142:0/424936347 wait complete. 2024-03-29T04:53:01.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.692+0000 7f29cde1e700 1 Processor -- start 2024-03-29T04:53:01.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29cde1e700 1 -- start start 2024-03-29T04:53:01.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29cde1e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c80ffa60 0x7f29c819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:01.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29cde1e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c8100410 0x7f29c819d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:01.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29cde1e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c819da60 con 0x7f29c80ffa60 2024-03-29T04:53:01.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29cde1e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c81a17f0 con 0x7f29c8100410 2024-03-29T04:53:01.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29c77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c80ffa60 0x7f29c819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:01.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29c77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c80ffa60 0x7f29c819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33874/0 (socket says 172.21.15.142:33874) 2024-03-29T04:53:01.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29c77fe700 1 -- 172.21.15.142:0/3359622760 learned_addr learned my addr 172.21.15.142:0/3359622760 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:01.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29c6ffd700 1 --2- 172.21.15.142:0/3359622760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c8100410 0x7f29c819d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:01.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29c77fe700 1 -- 172.21.15.142:0/3359622760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c8100410 msgr2=0x7f29c819d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:01.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29c77fe700 1 --2- 172.21.15.142:0/3359622760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c8100410 0x7f29c819d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29c77fe700 1 -- 172.21.15.142:0/3359622760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc0095f0 con 0x7f29c80ffa60 2024-03-29T04:53:01.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29c77fe700 1 --2- 172.21.15.142:0/3359622760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c80ffa60 0x7f29c819ce40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f29b800b4b0 tx=0x7f29b800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:01.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29c4ff9700 1 -- 172.21.15.142:0/3359622760 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29b8011800 con 0x7f29c80ffa60 2024-03-29T04:53:01.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29c4ff9700 1 -- 172.21.15.142:0/3359622760 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29b8011e20 con 0x7f29c80ffa60 2024-03-29T04:53:01.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29cde1e700 1 -- 172.21.15.142:0/3359622760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c81a1ad0 con 0x7f29c80ffa60 2024-03-29T04:53:01.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29c4ff9700 1 -- 172.21.15.142:0/3359622760 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29b800f2e0 con 0x7f29c80ffa60 2024-03-29T04:53:01.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.693+0000 7f29cde1e700 1 -- 172.21.15.142:0/3359622760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c81a2020 con 0x7f29c80ffa60 2024-03-29T04:53:01.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.694+0000 7f29cde1e700 1 -- 172.21.15.142:0/3359622760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c8102040 con 0x7f29c80ffa60 2024-03-29T04:53:01.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.695+0000 7f29c4ff9700 1 -- 172.21.15.142:0/3359622760 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f29b8010380 con 0x7f29c80ffa60 2024-03-29T04:53:01.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.695+0000 7f29c4ff9700 1 --2- 172.21.15.142:0/3359622760 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f29b006dd60 0x7f29b0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:01.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.696+0000 7f29c6ffd700 1 --2- 172.21.15.142:0/3359622760 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f29b006dd60 0x7f29b0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:01.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.696+0000 7f29c4ff9700 1 -- 172.21.15.142:0/3359622760 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f29b808d7e0 con 0x7f29c80ffa60 2024-03-29T04:53:01.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.696+0000 7f29c6ffd700 1 --2- 172.21.15.142:0/3359622760 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f29b006dd60 0x7f29b0070220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f29c819e460 tx=0x7f29bc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:01.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.700+0000 7f29c4ff9700 1 -- 172.21.15.142:0/3359622760 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f29b8056cb0 con 0x7f29c80ffa60 2024-03-29T04:53:01.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:01 smithi152 ceph-mon[88186]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:01.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:01 smithi152 ceph-mon[88186]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:01.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:01 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/706190211' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:01.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:01 smithi142 ceph-mon[112475]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:01.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:01 smithi142 ceph-mon[112475]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:01.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:01 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/706190211' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:01.921 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.916+0000 7f29cde1e700 1 -- 172.21.15.142:0/3359622760 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f29c81a2370 con 0x7f29b006dd60 2024-03-29T04:53:01.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.918+0000 7f29c4ff9700 1 -- 172.21.15.142:0/3359622760 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f29c81a2370 con 0x7f29b006dd60 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout: "mon" 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "9/23 daemons upgraded", 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading osd daemons", 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:53:01.923 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:53:01.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.920+0000 7f29cde1e700 1 -- 172.21.15.142:0/3359622760 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f29b006dd60 msgr2=0x7f29b0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:01.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.920+0000 7f29cde1e700 1 --2- 172.21.15.142:0/3359622760 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f29b006dd60 0x7f29b0070220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f29c819e460 tx=0x7f29bc01a040 comp rx=0 tx=0).stop 2024-03-29T04:53:01.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.920+0000 7f29cde1e700 1 -- 172.21.15.142:0/3359622760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c80ffa60 msgr2=0x7f29c819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:01.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.920+0000 7f29cde1e700 1 --2- 172.21.15.142:0/3359622760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c80ffa60 0x7f29c819ce40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f29b800b4b0 tx=0x7f29b800b870 comp rx=0 tx=0).stop 2024-03-29T04:53:01.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.920+0000 7f29cde1e700 1 -- 172.21.15.142:0/3359622760 shutdown_connections 2024-03-29T04:53:01.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.920+0000 7f29cde1e700 1 --2- 172.21.15.142:0/3359622760 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f29b006dd60 0x7f29b0070220 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.920+0000 7f29cde1e700 1 --2- 172.21.15.142:0/3359622760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29c80ffa60 0x7f29c819ce40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.920+0000 7f29cde1e700 1 --2- 172.21.15.142:0/3359622760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29c8100410 0x7f29c819d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:01.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.920+0000 7f29cde1e700 1 -- 172.21.15.142:0/3359622760 >> 172.21.15.142:0/3359622760 conn(0x7f29c80fb570 msgr2=0x7f29c8110f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:01.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.920+0000 7f29cde1e700 1 -- 172.21.15.142:0/3359622760 shutdown_connections 2024-03-29T04:53:01.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:01.920+0000 7f29cde1e700 1 -- 172.21.15.142:0/3359622760 wait complete. 2024-03-29T04:53:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.024+0000 7fa01c779700 1 -- 172.21.15.142:0/3641574554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa014102680 msgr2=0x7fa01410ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.024+0000 7fa01c779700 1 --2- 172.21.15.142:0/3641574554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa014102680 0x7fa01410ab70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa004009940 tx=0x7fa004009c50 comp rx=0 tx=0).stop 2024-03-29T04:53:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.024+0000 7fa01c779700 1 -- 172.21.15.142:0/3641574554 shutdown_connections 2024-03-29T04:53:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.024+0000 7fa01c779700 1 --2- 172.21.15.142:0/3641574554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa014102680 0x7fa01410ab70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.024+0000 7fa01c779700 1 --2- 172.21.15.142:0/3641574554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa014101d60 0x7fa014102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.024+0000 7fa01c779700 1 -- 172.21.15.142:0/3641574554 >> 172.21.15.142:0/3641574554 conn(0x7fa0140fb570 msgr2=0x7fa0140fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.024+0000 7fa01c779700 1 -- 172.21.15.142:0/3641574554 shutdown_connections 2024-03-29T04:53:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.024+0000 7fa01c779700 1 -- 172.21.15.142:0/3641574554 wait complete. 2024-03-29T04:53:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01c779700 1 Processor -- start 2024-03-29T04:53:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01c779700 1 -- start start 2024-03-29T04:53:02.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01c779700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa014101d60 0x7fa01419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01c779700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa014102680 0x7fa01419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01c779700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01419f2b0 con 0x7fa014101d60 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01c779700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa014198760 con 0x7fa014102680 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01a515700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa014101d60 0x7fa01419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01a515700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa014101d60 0x7fa01419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33890/0 (socket says 172.21.15.142:33890) 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01a515700 1 -- 172.21.15.142:0/1848437707 learned_addr learned my addr 172.21.15.142:0/1848437707 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa019d14700 1 --2- 172.21.15.142:0/1848437707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa014102680 0x7fa01419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01a515700 1 -- 172.21.15.142:0/1848437707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa014102680 msgr2=0x7fa01419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01a515700 1 --2- 172.21.15.142:0/1848437707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa014102680 0x7fa01419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01a515700 1 -- 172.21.15.142:0/1848437707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0040095f0 con 0x7fa014101d60 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa01a515700 1 --2- 172.21.15.142:0/1848437707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa014101d60 0x7fa01419e6e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0060b0 tx=0x7fa00c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.025+0000 7fa00b7fe700 1 -- 172.21.15.142:0/1848437707 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa00c00dda0 con 0x7fa014101d60 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.026+0000 7fa00b7fe700 1 -- 172.21.15.142:0/1848437707 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa00c00f040 con 0x7fa014101d60 2024-03-29T04:53:02.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.026+0000 7fa01c779700 1 -- 172.21.15.142:0/1848437707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa014198a40 con 0x7fa014101d60 2024-03-29T04:53:02.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.026+0000 7fa00b7fe700 1 -- 172.21.15.142:0/1848437707 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa00c014790 con 0x7fa014101d60 2024-03-29T04:53:02.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.026+0000 7fa01c779700 1 -- 172.21.15.142:0/1848437707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa014198f60 con 0x7fa014101d60 2024-03-29T04:53:02.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.026+0000 7fa01c779700 1 -- 172.21.15.142:0/1848437707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01410a6d0 con 0x7fa014101d60 2024-03-29T04:53:02.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.027+0000 7fa00b7fe700 1 -- 172.21.15.142:0/1848437707 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fa00c014980 con 0x7fa014101d60 2024-03-29T04:53:02.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.027+0000 7fa00b7fe700 1 --2- 172.21.15.142:0/1848437707 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa00006dd60 0x7fa000070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:02.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.028+0000 7fa019d14700 1 --2- 172.21.15.142:0/1848437707 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa00006dd60 0x7fa000070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:02.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.028+0000 7fa00b7fe700 1 -- 172.21.15.142:0/1848437707 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fa00c08d670 con 0x7fa014101d60 2024-03-29T04:53:02.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.028+0000 7fa019d14700 1 --2- 172.21.15.142:0/1848437707 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa00006dd60 0x7fa000070220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa004006010 tx=0x7fa004005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:02.037 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.032+0000 7fa00b7fe700 1 -- 172.21.15.142:0/1848437707 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa00c05a720 con 0x7fa014101d60 2024-03-29T04:53:02.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.305+0000 7fa01c779700 1 -- 172.21.15.142:0/1848437707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa01404ea90 con 0x7fa014101d60 2024-03-29T04:53:02.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.306+0000 7fa00b7fe700 1 -- 172.21.15.142:0/1848437707 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa00c05a2b0 con 0x7fa014101d60 2024-03-29T04:53:02.311 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_OK 2024-03-29T04:53:02.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.309+0000 7fa01c779700 1 -- 172.21.15.142:0/1848437707 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa00006dd60 msgr2=0x7fa000070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:02.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.309+0000 7fa01c779700 1 --2- 172.21.15.142:0/1848437707 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa00006dd60 0x7fa000070220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa004006010 tx=0x7fa004005a90 comp rx=0 tx=0).stop 2024-03-29T04:53:02.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.309+0000 7fa01c779700 1 -- 172.21.15.142:0/1848437707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa014101d60 msgr2=0x7fa01419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:02.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.309+0000 7fa01c779700 1 --2- 172.21.15.142:0/1848437707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa014101d60 0x7fa01419e6e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0060b0 tx=0x7fa00c00d3b0 comp rx=0 tx=0).stop 2024-03-29T04:53:02.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.309+0000 7fa01c779700 1 -- 172.21.15.142:0/1848437707 shutdown_connections 2024-03-29T04:53:02.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.309+0000 7fa01c779700 1 --2- 172.21.15.142:0/1848437707 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa00006dd60 0x7fa000070220 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:02.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.309+0000 7fa01c779700 1 --2- 172.21.15.142:0/1848437707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa014101d60 0x7fa01419e6e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:02.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.309+0000 7fa01c779700 1 --2- 172.21.15.142:0/1848437707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa014102680 0x7fa01419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:02.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.309+0000 7fa01c779700 1 -- 172.21.15.142:0/1848437707 >> 172.21.15.142:0/1848437707 conn(0x7fa0140fb570 msgr2=0x7fa014100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:02.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.309+0000 7fa01c779700 1 -- 172.21.15.142:0/1848437707 shutdown_connections 2024-03-29T04:53:02.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:02.309+0000 7fa01c779700 1 -- 172.21.15.142:0/1848437707 wait complete. 2024-03-29T04:53:02.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:02 smithi142 ceph-mon[112475]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:02.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:02 smithi142 ceph-mon[112475]: pgmap v121: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-29T04:53:02.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:02 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1503793045' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:53:02.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:02 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1848437707' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:53:03.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:02 smithi152 ceph-mon[88186]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:03.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:02 smithi152 ceph-mon[88186]: pgmap v121: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-29T04:53:03.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:02 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1503793045' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:53:03.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:02 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1848437707' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:53:03.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:03 smithi142 ceph-mon[112475]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:04.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:03 smithi152 ceph-mon[88186]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:04.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:04 smithi142 ceph-mon[112475]: pgmap v122: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:53:05.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:04 smithi152 ceph-mon[88186]: pgmap v122: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:53:05.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T04:53:05.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:05 smithi152 ceph-mon[88186]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T04:53:05.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:05 smithi152 ceph-mon[88186]: Upgrade: osd.3 is safe to restart 2024-03-29T04:53:05.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T04:53:05.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:05 smithi142 ceph-mon[112475]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T04:53:05.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:05 smithi142 ceph-mon[112475]: Upgrade: osd.3 is safe to restart 2024-03-29T04:53:06.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:06 smithi152 ceph-mon[88186]: pgmap v123: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:53:06.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:06 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:06.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:06 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-29T04:53:06.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:06 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:53:06.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:06 smithi142 ceph-mon[112475]: pgmap v123: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:53:06.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:06 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:06.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:06 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-29T04:53:06.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:06 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:53:07.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:07 smithi142 ceph-mon[112475]: Upgrade: Updating osd.3 2024-03-29T04:53:07.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:07 smithi142 ceph-mon[112475]: Deploying daemon osd.3 on smithi152 2024-03-29T04:53:07.883 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:07 smithi152 ceph-mon[88186]: Upgrade: Updating osd.3 2024-03-29T04:53:07.883 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:07 smithi152 ceph-mon[88186]: Deploying daemon osd.3 on smithi152 2024-03-29T04:53:08.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:08 smithi142 ceph-mon[112475]: pgmap v124: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:53:08.921 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:08 smithi152 ceph-mon[88186]: pgmap v124: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:53:09.795 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:09 smithi152 systemd[1]: Stopping Ceph osd.3 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:53:10.295 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:09 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3[43139]: 2024-03-29T04:53:09.833+0000 7fc8a4175700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T04:53:10.295 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:09 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3[43139]: 2024-03-29T04:53:09.833+0000 7fc8a4175700 -1 osd.3 78 *** Got signal Terminated *** 2024-03-29T04:53:10.295 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:09 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3[43139]: 2024-03-29T04:53:09.833+0000 7fc8a4175700 -1 osd.3 78 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T04:53:10.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:10 smithi142 ceph-mon[112475]: pgmap v125: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:53:10.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:10 smithi142 ceph-mon[112475]: osd.3 marked itself down and dead 2024-03-29T04:53:11.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:10 smithi152 ceph-mon[88186]: pgmap v125: 65 pgs: 65 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:53:11.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:10 smithi152 ceph-mon[88186]: osd.3 marked itself down and dead 2024-03-29T04:53:11.463 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:11 smithi152 podman[97177]: 2024-03-29 04:53:11.16707399 +0000 UTC m=+1.471617542 container died 575eedc751156c4468446c2095418af0f76465deed1f0afc83519027456a6f20 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222) 2024-03-29T04:53:11.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:11 smithi152 ceph-mon[88186]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T04:53:11.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:11 smithi152 ceph-mon[88186]: osdmap e79: 6 total, 5 up, 6 in 2024-03-29T04:53:11.795 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:11 smithi152 podman[97177]: 2024-03-29 04:53:11.550495197 +0000 UTC m=+1.855038748 container cleanup 575eedc751156c4468446c2095418af0f76465deed1f0afc83519027456a6f20 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-29T04:53:11.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:11 smithi152 bash[97177]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3 2024-03-29T04:53:11.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:11 smithi142 ceph-mon[112475]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T04:53:11.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:11 smithi142 ceph-mon[112475]: osdmap e79: 6 total, 5 up, 6 in 2024-03-29T04:53:12.192 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:11 smithi152 podman[97205]: 2024-03-29 04:53:11.827720365 +0000 UTC m=+0.663008989 container remove 575eedc751156c4468446c2095418af0f76465deed1f0afc83519027456a6f20 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD) 2024-03-29T04:53:12.444 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:12 smithi152 podman[97297]: 2024-03-29 04:53:12.091026229 +0000 UTC m=+0.017917533 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:53:12.444 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:12 smithi152 podman[97297]: 2024-03-29 04:53:12.211984066 +0000 UTC m=+0.138875361 container create 3c15fbc80586df34c2a4f30b163070ab3988effa83e77a02e92e37363aeaf2aa (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8da2362, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-29T04:53:12.444 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:12 smithi152 podman[97297]: 2024-03-29 04:53:12.439718037 +0000 UTC m=+0.366609331 container init 3c15fbc80586df34c2a4f30b163070ab3988effa83e77a02e92e37363aeaf2aa (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8da2362, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-29T04:53:12.742 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:12 smithi152 ceph-mon[88186]: pgmap v127: 65 pgs: 4 peering, 11 stale+active+clean, 50 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-29T04:53:12.742 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:12 smithi152 ceph-mon[88186]: osdmap e80: 6 total, 5 up, 6 in 2024-03-29T04:53:12.742 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:12 smithi152 ceph-mon[88186]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-29T04:53:12.743 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:12 smithi152 podman[97297]: 2024-03-29 04:53:12.443021288 +0000 UTC m=+0.369912584 container start 3c15fbc80586df34c2a4f30b163070ab3988effa83e77a02e92e37363aeaf2aa (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-03-29T04:53:12.743 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:12 smithi152 podman[97297]: 2024-03-29 04:53:12.531383021 +0000 UTC m=+0.458274343 container attach 3c15fbc80586df34c2a4f30b163070ab3988effa83e77a02e92e37363aeaf2aa (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8da2362, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-29T04:53:13.030 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:12 smithi152 podman[97297]: 2024-03-29 04:53:12.738709024 +0000 UTC m=+0.665600324 container died 3c15fbc80586df34c2a4f30b163070ab3988effa83e77a02e92e37363aeaf2aa (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-deactivate, RELEASE=squid-8da2362, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-03-29T04:53:13.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:12 smithi142 ceph-mon[112475]: pgmap v127: 65 pgs: 4 peering, 11 stale+active+clean, 50 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-29T04:53:13.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:12 smithi142 ceph-mon[112475]: osdmap e80: 6 total, 5 up, 6 in 2024-03-29T04:53:13.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:12 smithi142 ceph-mon[112475]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-29T04:53:13.684 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:13 smithi152 podman[97323]: 2024-03-29 04:53:13.335542179 +0000 UTC m=+0.591253779 container remove 3c15fbc80586df34c2a4f30b163070ab3988effa83e77a02e92e37363aeaf2aa (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-29T04:53:13.684 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:13 smithi152 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-29T04:53:13.684 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:13 smithi152 systemd[1]: Stopped Ceph osd.3 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:53:13.684 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:13 smithi152 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.3.service: Consumed 1min 18.574s CPU time. 2024-03-29T04:53:13.684 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:13 smithi152 systemd[1]: Starting Ceph osd.3 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:53:13.950 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:13 smithi152 ceph-mon[88186]: pgmap v129: 65 pgs: 4 active+undersized, 3 active+undersized+degraded, 4 peering, 8 stale+active+clean, 46 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 7/333 objects degraded (2.102%) 2024-03-29T04:53:13.950 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:13 smithi152 ceph-mon[88186]: Health check failed: Degraded data redundancy: 7/333 objects degraded (2.102%), 3 pgs degraded (PG_DEGRADED) 2024-03-29T04:53:13.951 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:13 smithi152 podman[97423]: 2024-03-29 04:53:13.682020801 +0000 UTC m=+0.018055080 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:53:13.951 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:13 smithi152 podman[97423]: 2024-03-29 04:53:13.828968047 +0000 UTC m=+0.165002299 container create ead256df1b329aa16657be67c4e278b2e37b44371562ecc57530eb4f02c1bb32 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-8da2362, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326) 2024-03-29T04:53:14.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:13 smithi142 ceph-mon[112475]: pgmap v129: 65 pgs: 4 active+undersized, 3 active+undersized+degraded, 4 peering, 8 stale+active+clean, 46 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 7/333 objects degraded (2.102%) 2024-03-29T04:53:14.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:13 smithi142 ceph-mon[112475]: Health check failed: Degraded data redundancy: 7/333 objects degraded (2.102%), 3 pgs degraded (PG_DEGRADED) 2024-03-29T04:53:14.295 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 podman[97423]: 2024-03-29 04:53:14.073744191 +0000 UTC m=+0.409778457 container init ead256df1b329aa16657be67c4e278b2e37b44371562ecc57530eb4f02c1bb32 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8da2362, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-29T04:53:14.295 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 podman[97423]: 2024-03-29 04:53:14.076954194 +0000 UTC m=+0.412988447 container start ead256df1b329aa16657be67c4e278b2e37b44371562ecc57530eb4f02c1bb32 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8da2362, org.label-schema.build-date=20240326, ceph=True) 2024-03-29T04:53:14.295 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 podman[97423]: 2024-03-29 04:53:14.156575045 +0000 UTC m=+0.492609317 container attach ead256df1b329aa16657be67c4e278b2e37b44371562ecc57530eb4f02c1bb32 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, org.label-schema.license=GPLv2) 2024-03-29T04:53:14.795 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate[97438]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-29T04:53:14.795 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate[97438]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 bash[97423]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 bash[97423]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 bash[97423]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 bash[97423]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 bash[97423]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 bash[97423]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 bash[97423]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate[97438]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate[97438]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate[97438]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate[97438]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate[97438]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-29T04:53:14.796 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:14 smithi152 podman[97423]: 2024-03-29 04:53:14.552089376 +0000 UTC m=+0.888123645 container died ead256df1b329aa16657be67c4e278b2e37b44371562ecc57530eb4f02c1bb32 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8da2362, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-29T04:53:15.295 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:15 smithi152 podman[97563]: 2024-03-29 04:53:15.085925966 +0000 UTC m=+0.529065759 container remove ead256df1b329aa16657be67c4e278b2e37b44371562ecc57530eb4f02c1bb32 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-activate, org.label-schema.license=GPLv2, RELEASE=squid-8da2362, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-03-29T04:53:15.568 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:15 smithi152 podman[97628]: 2024-03-29 04:53:15.202992657 +0000 UTC m=+0.019139276 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:53:15.568 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:15 smithi152 podman[97628]: 2024-03-29 04:53:15.32867122 +0000 UTC m=+0.144817864 container create c41086b4f6bc25351259a93f8fd5101ed1c927fe0098eaee399c7ee8584b68f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3, GIT_BRANCH=HEAD, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6) 2024-03-29T04:53:15.569 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:15 smithi152 podman[97628]: 2024-03-29 04:53:15.564374173 +0000 UTC m=+0.380520781 container init c41086b4f6bc25351259a93f8fd5101ed1c927fe0098eaee399c7ee8584b68f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:53:15.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:15.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:53:16.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:16.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:53:16.046 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:15 smithi152 podman[97628]: 2024-03-29 04:53:15.56752947 +0000 UTC m=+0.383676069 container start c41086b4f6bc25351259a93f8fd5101ed1c927fe0098eaee399c7ee8584b68f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326) 2024-03-29T04:53:16.046 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:15 smithi152 ceph-osd[97647]: -- 172.21.15.152:0/4125600050 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d50ede7680 con 0x55d50edb0c00 2024-03-29T04:53:16.046 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:15 smithi152 bash[97628]: c41086b4f6bc25351259a93f8fd5101ed1c927fe0098eaee399c7ee8584b68f9 2024-03-29T04:53:16.046 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:15 smithi152 systemd[1]: Started Ceph osd.3 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:53:16.490 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:16 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3[97643]: 2024-03-29T04:53:16.117+0000 7f7fef04d7c0 -1 Falling back to public interface 2024-03-29T04:53:16.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:16 smithi152 ceph-mon[88186]: pgmap v130: 65 pgs: 14 active+undersized, 15 active+undersized+degraded, 4 peering, 32 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/333 objects degraded (14.715%) 2024-03-29T04:53:16.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:16.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:16.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:16 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:53:16.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:16 smithi142 ceph-mon[112475]: pgmap v130: 65 pgs: 14 active+undersized, 15 active+undersized+degraded, 4 peering, 32 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/333 objects degraded (14.715%) 2024-03-29T04:53:16.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:16.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:16.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:16 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:53:17.911 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:17 smithi152 ceph-mon[88186]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-29T04:53:18.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:17 smithi142 ceph-mon[112475]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-29T04:53:19.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:18 smithi152 ceph-mon[88186]: pgmap v131: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-03-29T04:53:19.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:18 smithi142 ceph-mon[112475]: pgmap v131: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-03-29T04:53:20.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:19 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:20.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:19 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:20.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:19 smithi152 ceph-mon[88186]: pgmap v132: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-03-29T04:53:20.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:19 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:20.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:19 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:20.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:19 smithi142 ceph-mon[112475]: pgmap v132: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-03-29T04:53:21.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:20 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:21.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:20 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:21.046 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:20 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3[97643]: 2024-03-29T04:53:20.732+0000 7f7fef04d7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-29T04:53:21.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:20 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:21.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:20 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:21.475 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:21 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3[97643]: 2024-03-29T04:53:21.205+0000 7f7fef04d7c0 -1 osd.3 78 log_to_monitors true 2024-03-29T04:53:22.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:21 smithi152 ceph-mon[88186]: Health check update: Degraded data redundancy: 52/333 objects degraded (15.616%), 16 pgs degraded (PG_DEGRADED) 2024-03-29T04:53:22.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:21 smithi152 ceph-mon[88186]: from='osd.3 [v2:172.21.15.152:6800/1538535999,v1:172.21.15.152:6801/1538535999]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T04:53:22.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:21 smithi152 ceph-mon[88186]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T04:53:22.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:21 smithi152 ceph-mon[88186]: pgmap v133: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-03-29T04:53:22.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:21 smithi142 ceph-mon[112475]: Health check update: Degraded data redundancy: 52/333 objects degraded (15.616%), 16 pgs degraded (PG_DEGRADED) 2024-03-29T04:53:22.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:21 smithi142 ceph-mon[112475]: from='osd.3 [v2:172.21.15.152:6800/1538535999,v1:172.21.15.152:6801/1538535999]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T04:53:22.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:21 smithi142 ceph-mon[112475]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T04:53:22.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:21 smithi142 ceph-mon[112475]: pgmap v133: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 270 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-03-29T04:53:22.545 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 04:53:22 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3[97643]: 2024-03-29T04:53:22.219+0000 7f7fe04a5700 -1 osd.3 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:53:23.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:23 smithi152 ceph-mon[88186]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-29T04:53:23.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:23 smithi152 ceph-mon[88186]: from='osd.3 [v2:172.21.15.152:6800/1538535999,v1:172.21.15.152:6801/1538535999]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:53:23.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:23 smithi152 ceph-mon[88186]: osdmap e81: 6 total, 5 up, 6 in 2024-03-29T04:53:23.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:23 smithi152 ceph-mon[88186]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:53:23.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:23 smithi142 ceph-mon[112475]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-29T04:53:23.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:23 smithi142 ceph-mon[112475]: from='osd.3 [v2:172.21.15.152:6800/1538535999,v1:172.21.15.152:6801/1538535999]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:53:23.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:23 smithi142 ceph-mon[112475]: osdmap e81: 6 total, 5 up, 6 in 2024-03-29T04:53:23.561 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:23 smithi142 ceph-mon[112475]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:53:24.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: osd.3 [v2:172.21.15.152:6800/1538535999,v1:172.21.15.152:6801/1538535999] boot 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: osdmap e82: 6 total, 6 up, 6 in 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:24.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T04:53:24.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T04:53:24.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: Upgrade: unsafe to stop osd(s) at this time (14 PGs are or would become offline) 2024-03-29T04:53:24.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: pgmap v136: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 52/333 objects degraded (15.616%) 2024-03-29T04:53:24.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:24 smithi152 ceph-mon[88186]: osdmap e83: 6 total, 6 up, 6 in 2024-03-29T04:53:24.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: osd.3 [v2:172.21.15.152:6800/1538535999,v1:172.21.15.152:6801/1538535999] boot 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: osdmap e82: 6 total, 6 up, 6 in 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:24.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T04:53:24.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T04:53:24.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: Upgrade: unsafe to stop osd(s) at this time (14 PGs are or would become offline) 2024-03-29T04:53:24.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: pgmap v136: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 52/333 objects degraded (15.616%) 2024-03-29T04:53:24.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:24 smithi142 ceph-mon[112475]: osdmap e83: 6 total, 6 up, 6 in 2024-03-29T04:53:26.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:26 smithi142 ceph-mon[112475]: pgmap v138: 65 pgs: 7 peering, 10 active+undersized, 9 active+undersized+degraded, 39 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 30/333 objects degraded (9.009%) 2024-03-29T04:53:26.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:26 smithi142 ceph-mon[112475]: Health check update: Degraded data redundancy: 30/333 objects degraded (9.009%), 9 pgs degraded (PG_DEGRADED) 2024-03-29T04:53:27.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:26 smithi152 ceph-mon[88186]: pgmap v138: 65 pgs: 7 peering, 10 active+undersized, 9 active+undersized+degraded, 39 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 30/333 objects degraded (9.009%) 2024-03-29T04:53:27.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:26 smithi152 ceph-mon[88186]: Health check update: Degraded data redundancy: 30/333 objects degraded (9.009%), 9 pgs degraded (PG_DEGRADED) 2024-03-29T04:53:28.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:28 smithi142 ceph-mon[112475]: pgmap v139: 65 pgs: 7 peering, 3 active+undersized, 1 active+undersized+degraded, 54 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s; 3/333 objects degraded (0.901%) 2024-03-29T04:53:29.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:28 smithi152 ceph-mon[88186]: pgmap v139: 65 pgs: 7 peering, 3 active+undersized, 1 active+undersized+degraded, 54 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s; 3/333 objects degraded (0.901%) 2024-03-29T04:53:30.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:30 smithi142 ceph-mon[112475]: pgmap v140: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-03-29T04:53:30.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:30.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:53:30.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:30 smithi142 ceph-mon[112475]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/333 objects degraded (0.901%), 1 pg degraded) 2024-03-29T04:53:30.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:30 smithi142 ceph-mon[112475]: Cluster is now healthy 2024-03-29T04:53:31.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:30 smithi152 ceph-mon[88186]: pgmap v140: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-03-29T04:53:31.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:31.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:53:31.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:30 smithi152 ceph-mon[88186]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/333 objects degraded (0.901%), 1 pg degraded) 2024-03-29T04:53:31.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:30 smithi152 ceph-mon[88186]: Cluster is now healthy 2024-03-29T04:53:32.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.427+0000 7f6a486e8700 1 -- 172.21.15.142:0/1965800041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a400ffa40 msgr2=0x7f6a400ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:32.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.427+0000 7f6a486e8700 1 --2- 172.21.15.142:0/1965800041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a400ffa40 0x7f6a400ffec0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009910 tx=0x7f6a30009c20 comp rx=0 tx=0).stop 2024-03-29T04:53:32.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.427+0000 7f6a486e8700 1 -- 172.21.15.142:0/1965800041 shutdown_connections 2024-03-29T04:53:32.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.427+0000 7f6a486e8700 1 --2- 172.21.15.142:0/1965800041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a400ffa40 0x7f6a400ffec0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.427+0000 7f6a486e8700 1 --2- 172.21.15.142:0/1965800041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a40105a40 0x7f6a40105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.427+0000 7f6a486e8700 1 -- 172.21.15.142:0/1965800041 >> 172.21.15.142:0/1965800041 conn(0x7f6a400fb550 msgr2=0x7f6a400fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:32.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.427+0000 7f6a486e8700 1 -- 172.21.15.142:0/1965800041 shutdown_connections 2024-03-29T04:53:32.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.427+0000 7f6a486e8700 1 -- 172.21.15.142:0/1965800041 wait complete. 2024-03-29T04:53:32.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.427+0000 7f6a486e8700 1 Processor -- start 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.427+0000 7f6a486e8700 1 -- start start 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a486e8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a400ffa40 0x7f6a40197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a486e8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a40105a40 0x7f6a40197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a486e8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40198060 con 0x7f6a400ffa40 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a486e8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a401981d0 con 0x7f6a40105a40 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a46484700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a400ffa40 0x7f6a40197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a46484700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a400ffa40 0x7f6a40197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59530/0 (socket says 172.21.15.142:59530) 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a46484700 1 -- 172.21.15.142:0/2043750669 learned_addr learned my addr 172.21.15.142:0/2043750669 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a45c83700 1 --2- 172.21.15.142:0/2043750669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a40105a40 0x7f6a40197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a46484700 1 -- 172.21.15.142:0/2043750669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a40105a40 msgr2=0x7f6a40197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a46484700 1 --2- 172.21.15.142:0/2043750669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a40105a40 0x7f6a40197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a46484700 1 -- 172.21.15.142:0/2043750669 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a300095f0 con 0x7f6a400ffa40 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a46484700 1 --2- 172.21.15.142:0/2043750669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a400ffa40 0x7f6a40197590 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c930 tx=0x7f6a3c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:32.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a377fe700 1 -- 172.21.15.142:0/2043750669 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a3c007ad0 con 0x7f6a400ffa40 2024-03-29T04:53:32.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a486e8700 1 -- 172.21.15.142:0/2043750669 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a401a0600 con 0x7f6a400ffa40 2024-03-29T04:53:32.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a377fe700 1 -- 172.21.15.142:0/2043750669 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a3c007c30 con 0x7f6a400ffa40 2024-03-29T04:53:32.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.428+0000 7f6a377fe700 1 -- 172.21.15.142:0/2043750669 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a3c018610 con 0x7f6a400ffa40 2024-03-29T04:53:32.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.429+0000 7f6a486e8700 1 -- 172.21.15.142:0/2043750669 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a401a0a70 con 0x7f6a400ffa40 2024-03-29T04:53:32.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.429+0000 7f6a486e8700 1 -- 172.21.15.142:0/2043750669 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a40107fd0 con 0x7f6a400ffa40 2024-03-29T04:53:32.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.430+0000 7f6a377fe700 1 -- 172.21.15.142:0/2043750669 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6a3c00f450 con 0x7f6a400ffa40 2024-03-29T04:53:32.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.430+0000 7f6a377fe700 1 --2- 172.21.15.142:0/2043750669 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6a2c06dd60 0x7f6a2c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:32.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.430+0000 7f6a377fe700 1 -- 172.21.15.142:0/2043750669 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f6a3c08dde0 con 0x7f6a400ffa40 2024-03-29T04:53:32.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.431+0000 7f6a45c83700 1 --2- 172.21.15.142:0/2043750669 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6a2c06dd60 0x7f6a2c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:32.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.431+0000 7f6a45c83700 1 --2- 172.21.15.142:0/2043750669 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6a2c06dd60 0x7f6a2c070220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a40069680 tx=0x7f6a3001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:32.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.435+0000 7f6a377fe700 1 -- 172.21.15.142:0/2043750669 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6a3c0572b0 con 0x7f6a400ffa40 2024-03-29T04:53:32.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.649+0000 7f6a486e8700 1 -- 172.21.15.142:0/2043750669 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a4002cce0 con 0x7f6a2c06dd60 2024-03-29T04:53:32.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.651+0000 7f6a377fe700 1 -- 172.21.15.142:0/2043750669 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f6a4002cce0 con 0x7f6a2c06dd60 2024-03-29T04:53:32.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.653+0000 7f6a486e8700 1 -- 172.21.15.142:0/2043750669 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6a2c06dd60 msgr2=0x7f6a2c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:32.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.653+0000 7f6a486e8700 1 --2- 172.21.15.142:0/2043750669 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6a2c06dd60 0x7f6a2c070220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6a40069680 tx=0x7f6a3001a040 comp rx=0 tx=0).stop 2024-03-29T04:53:32.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.653+0000 7f6a486e8700 1 -- 172.21.15.142:0/2043750669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a400ffa40 msgr2=0x7f6a40197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:32.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.653+0000 7f6a486e8700 1 --2- 172.21.15.142:0/2043750669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a400ffa40 0x7f6a40197590 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c930 tx=0x7f6a3c00ccf0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.653+0000 7f6a486e8700 1 -- 172.21.15.142:0/2043750669 shutdown_connections 2024-03-29T04:53:32.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.653+0000 7f6a486e8700 1 --2- 172.21.15.142:0/2043750669 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6a2c06dd60 0x7f6a2c070220 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.653+0000 7f6a486e8700 1 --2- 172.21.15.142:0/2043750669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a400ffa40 0x7f6a40197590 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.653+0000 7f6a486e8700 1 --2- 172.21.15.142:0/2043750669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a40105a40 0x7f6a40197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.653+0000 7f6a486e8700 1 -- 172.21.15.142:0/2043750669 >> 172.21.15.142:0/2043750669 conn(0x7f6a400fb550 msgr2=0x7f6a400fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:32.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.653+0000 7f6a486e8700 1 -- 172.21.15.142:0/2043750669 shutdown_connections 2024-03-29T04:53:32.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.653+0000 7f6a486e8700 1 -- 172.21.15.142:0/2043750669 wait complete. 2024-03-29T04:53:32.669 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:53:32.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.757+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2169080427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3e0105a70 msgr2=0x7ff3e0105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:32.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.757+0000 7ff3e5f7b700 1 --2- 172.21.15.142:0/2169080427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3e0105a70 0x7ff3e0105e50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800b0a0 tx=0x7ff3d800b3b0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.757+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2169080427 shutdown_connections 2024-03-29T04:53:32.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.757+0000 7ff3e5f7b700 1 --2- 172.21.15.142:0/2169080427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e00ffa70 0x7ff3e00ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.757+0000 7ff3e5f7b700 1 --2- 172.21.15.142:0/2169080427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3e0105a70 0x7ff3e0105e50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.757+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2169080427 >> 172.21.15.142:0/2169080427 conn(0x7ff3e00fb5a0 msgr2=0x7ff3e00fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:32.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.757+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2169080427 shutdown_connections 2024-03-29T04:53:32.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.757+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2169080427 wait complete. 2024-03-29T04:53:32.763 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:32 smithi142 ceph-mon[112475]: pgmap v141: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 2 op/s 2024-03-29T04:53:32.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.758+0000 7ff3e5f7b700 1 Processor -- start 2024-03-29T04:53:32.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.758+0000 7ff3e5f7b700 1 -- start start 2024-03-29T04:53:32.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.758+0000 7ff3e5f7b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3e00ffa70 0x7ff3e019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:32.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.758+0000 7ff3e5f7b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e0105a70 0x7ff3e019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:32.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.758+0000 7ff3e5f7b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e019f3e0 con 0x7ff3e00ffa70 2024-03-29T04:53:32.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.758+0000 7ff3e5f7b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e0198890 con 0x7ff3e0105a70 2024-03-29T04:53:32.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.758+0000 7ff3df7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3e00ffa70 0x7ff3e019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.758+0000 7ff3df7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3e00ffa70 0x7ff3e019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59544/0 (socket says 172.21.15.142:59544) 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.758+0000 7ff3df7fe700 1 -- 172.21.15.142:0/2918123340 learned_addr learned my addr 172.21.15.142:0/2918123340 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.759+0000 7ff3deffd700 1 --2- 172.21.15.142:0/2918123340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e0105a70 0x7ff3e019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.759+0000 7ff3df7fe700 1 -- 172.21.15.142:0/2918123340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e0105a70 msgr2=0x7ff3e019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.759+0000 7ff3df7fe700 1 --2- 172.21.15.142:0/2918123340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e0105a70 0x7ff3e019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.759+0000 7ff3df7fe700 1 -- 172.21.15.142:0/2918123340 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d8009d00 con 0x7ff3e00ffa70 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.759+0000 7ff3deffd700 1 --2- 172.21.15.142:0/2918123340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e0105a70 0x7ff3e019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.759+0000 7ff3df7fe700 1 --2- 172.21.15.142:0/2918123340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3e00ffa70 0x7ff3e019e810 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8000f80 tx=0x7ff3d800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.759+0000 7ff3dcff9700 1 -- 172.21.15.142:0/2918123340 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3d800e040 con 0x7ff3e00ffa70 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.759+0000 7ff3dcff9700 1 -- 172.21.15.142:0/2918123340 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3d8003e60 con 0x7ff3e00ffa70 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.759+0000 7ff3dcff9700 1 -- 172.21.15.142:0/2918123340 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3d801d910 con 0x7ff3e00ffa70 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.759+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2918123340 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e0198b10 con 0x7ff3e00ffa70 2024-03-29T04:53:32.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.759+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2918123340 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e0199000 con 0x7ff3e00ffa70 2024-03-29T04:53:32.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.760+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2918123340 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e004ea90 con 0x7ff3e00ffa70 2024-03-29T04:53:32.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.761+0000 7ff3dcff9700 1 -- 172.21.15.142:0/2918123340 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff3d8004820 con 0x7ff3e00ffa70 2024-03-29T04:53:32.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.762+0000 7ff3dcff9700 1 --2- 172.21.15.142:0/2918123340 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff3cc06dd60 0x7ff3cc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:32.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.762+0000 7ff3dcff9700 1 -- 172.21.15.142:0/2918123340 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6704+0+0 (secure 0 0 0) 0x7ff3d808f470 con 0x7ff3e00ffa70 2024-03-29T04:53:32.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.762+0000 7ff3deffd700 1 --2- 172.21.15.142:0/2918123340 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff3cc06dd60 0x7ff3cc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:32.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.762+0000 7ff3deffd700 1 --2- 172.21.15.142:0/2918123340 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff3cc06dd60 0x7ff3cc070220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0104e90 tx=0x7ff3c8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:32.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.767+0000 7ff3dcff9700 1 -- 172.21.15.142:0/2918123340 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff3d805c4f0 con 0x7ff3e00ffa70 2024-03-29T04:53:32.985 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.980+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2918123340 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3e0199430 con 0x7ff3cc06dd60 2024-03-29T04:53:32.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.981+0000 7ff3dcff9700 1 -- 172.21.15.142:0/2918123340 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ff3e0199430 con 0x7ff3cc06dd60 2024-03-29T04:53:32.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.983+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2918123340 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff3cc06dd60 msgr2=0x7ff3cc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:32.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.983+0000 7ff3e5f7b700 1 --2- 172.21.15.142:0/2918123340 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff3cc06dd60 0x7ff3cc070220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0104e90 tx=0x7ff3c8008040 comp rx=0 tx=0).stop 2024-03-29T04:53:32.990 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.983+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2918123340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3e00ffa70 msgr2=0x7ff3e019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:32.990 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.983+0000 7ff3e5f7b700 1 --2- 172.21.15.142:0/2918123340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3e00ffa70 0x7ff3e019e810 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8000f80 tx=0x7ff3d800cdd0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.990 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.983+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2918123340 shutdown_connections 2024-03-29T04:53:32.990 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.983+0000 7ff3e5f7b700 1 --2- 172.21.15.142:0/2918123340 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff3cc06dd60 0x7ff3cc070220 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.990 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.983+0000 7ff3e5f7b700 1 --2- 172.21.15.142:0/2918123340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3e00ffa70 0x7ff3e019e810 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.984+0000 7ff3e5f7b700 1 --2- 172.21.15.142:0/2918123340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3e0105a70 0x7ff3e019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:32.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.984+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2918123340 >> 172.21.15.142:0/2918123340 conn(0x7ff3e00fb5a0 msgr2=0x7ff3e0068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:32.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.984+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2918123340 shutdown_connections 2024-03-29T04:53:32.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:32.984+0000 7ff3e5f7b700 1 -- 172.21.15.142:0/2918123340 wait complete. 2024-03-29T04:53:33.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:32 smithi152 ceph-mon[88186]: pgmap v141: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 2 op/s 2024-03-29T04:53:33.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.088+0000 7fc63ba86700 1 -- 172.21.15.142:0/3333542555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6340ffbd0 msgr2=0x7fc634100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:33.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.088+0000 7fc63ba86700 1 --2- 172.21.15.142:0/3333542555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6340ffbd0 0x7fc634100050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc628009910 tx=0x7fc628009c20 comp rx=0 tx=0).stop 2024-03-29T04:53:33.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.088+0000 7fc63ba86700 1 -- 172.21.15.142:0/3333542555 shutdown_connections 2024-03-29T04:53:33.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.088+0000 7fc63ba86700 1 --2- 172.21.15.142:0/3333542555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6340ffbd0 0x7fc634100050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.088+0000 7fc63ba86700 1 --2- 172.21.15.142:0/3333542555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc634105a40 0x7fc634105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.088+0000 7fc63ba86700 1 -- 172.21.15.142:0/3333542555 >> 172.21.15.142:0/3333542555 conn(0x7fc6340755a0 msgr2=0x7fc6340759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:33.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.088+0000 7fc63ba86700 1 -- 172.21.15.142:0/3333542555 shutdown_connections 2024-03-29T04:53:33.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.088+0000 7fc63ba86700 1 -- 172.21.15.142:0/3333542555 wait complete. 2024-03-29T04:53:33.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.089+0000 7fc63ba86700 1 Processor -- start 2024-03-29T04:53:33.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.089+0000 7fc63ba86700 1 -- start start 2024-03-29T04:53:33.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.089+0000 7fc63ba86700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6340ffbd0 0x7fc634197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:33.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.089+0000 7fc63ba86700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc634105a40 0x7fc634197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:33.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.089+0000 7fc63ba86700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc634198190 con 0x7fc634105a40 2024-03-29T04:53:33.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.089+0000 7fc63ba86700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc634198300 con 0x7fc6340ffbd0 2024-03-29T04:53:33.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.089+0000 7fc639021700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc634105a40 0x7fc634197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:33.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.089+0000 7fc639021700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc634105a40 0x7fc634197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59568/0 (socket says 172.21.15.142:59568) 2024-03-29T04:53:33.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.089+0000 7fc639021700 1 -- 172.21.15.142:0/2168895630 learned_addr learned my addr 172.21.15.142:0/2168895630 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:33.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.089+0000 7fc639822700 1 --2- 172.21.15.142:0/2168895630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6340ffbd0 0x7fc634197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:33.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.090+0000 7fc639021700 1 -- 172.21.15.142:0/2168895630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6340ffbd0 msgr2=0x7fc634197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:33.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.090+0000 7fc639021700 1 --2- 172.21.15.142:0/2168895630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6340ffbd0 0x7fc634197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.090+0000 7fc639021700 1 -- 172.21.15.142:0/2168895630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6280095f0 con 0x7fc634105a40 2024-03-29T04:53:33.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.090+0000 7fc639822700 1 --2- 172.21.15.142:0/2168895630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6340ffbd0 0x7fc634197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:53:33.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.090+0000 7fc639021700 1 --2- 172.21.15.142:0/2168895630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc634105a40 0x7fc634197ab0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc628005f50 tx=0x7fc62800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:33.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.090+0000 7fc632ffd700 1 -- 172.21.15.142:0/2168895630 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc62801d070 con 0x7fc634105a40 2024-03-29T04:53:33.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.090+0000 7fc632ffd700 1 -- 172.21.15.142:0/2168895630 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc62800b610 con 0x7fc634105a40 2024-03-29T04:53:33.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.090+0000 7fc63ba86700 1 -- 172.21.15.142:0/2168895630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6341a05e0 con 0x7fc634105a40 2024-03-29T04:53:33.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.090+0000 7fc632ffd700 1 -- 172.21.15.142:0/2168895630 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc628022470 con 0x7fc634105a40 2024-03-29T04:53:33.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.090+0000 7fc63ba86700 1 -- 172.21.15.142:0/2168895630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6341a0a80 con 0x7fc634105a40 2024-03-29T04:53:33.098 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.091+0000 7fc63ba86700 1 -- 172.21.15.142:0/2168895630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc634107fd0 con 0x7fc634105a40 2024-03-29T04:53:33.098 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.092+0000 7fc632ffd700 1 -- 172.21.15.142:0/2168895630 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fc62800fca0 con 0x7fc634105a40 2024-03-29T04:53:33.098 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.092+0000 7fc632ffd700 1 --2- 172.21.15.142:0/2168895630 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc61c06dd60 0x7fc61c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:33.098 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.092+0000 7fc632ffd700 1 -- 172.21.15.142:0/2168895630 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fc62808f040 con 0x7fc634105a40 2024-03-29T04:53:33.098 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.092+0000 7fc639822700 1 --2- 172.21.15.142:0/2168895630 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc61c06dd60 0x7fc61c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:33.098 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.093+0000 7fc639822700 1 --2- 172.21.15.142:0/2168895630 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc61c06dd60 0x7fc61c070220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc624009cb0 tx=0x7fc624009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:33.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.097+0000 7fc632ffd700 1 -- 172.21.15.142:0/2168895630 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc628058510 con 0x7fc634105a40 2024-03-29T04:53:33.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.300+0000 7fc63ba86700 1 -- 172.21.15.142:0/2168895630 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc63402cd30 con 0x7fc61c06dd60 2024-03-29T04:53:33.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.310+0000 7fc632ffd700 1 -- 172.21.15.142:0/2168895630 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc63402cd30 con 0x7fc61c06dd60 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (5m) 48s ago 16m 25.5M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (17m) 48s ago 17m 8762k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (16m) 14s ago 16m 8358k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (2m) 48s ago 17m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (2m) 14s ago 16m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (5m) 48s ago 16m 80.3M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (11m) 48s ago 11m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (11m) 48s ago 11m 63.5M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (11m) 14s ago 11m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (11m) 14s ago 11m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (6m) 48s ago 18m 506M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:53:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (6m) 14s ago 15m 451M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:53:33.317 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (3m) 48s ago 18m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:53:33.317 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (3m) 14s ago 15m 43.1M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:53:33.317 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (6m) 48s ago 17m 14.9M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:53:33.317 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (6m) 14s ago 15m 16.0M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:53:33.317 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (2m) 48s ago 15m 105M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:53:33.317 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (90s) 48s ago 14m 142M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:53:33.317 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (54s) 48s ago 14m 12.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:53:33.317 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (18s) 14s ago 13m 14.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T04:53:33.317 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (13m) 14s ago 13m 273M 2240M 18.2.1 5be31c24972a 5e0acbe50fe3 2024-03-29T04:53:33.317 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (12m) 14s ago 12m 275M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:53:33.317 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (5m) 48s ago 16m 63.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:53:33.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.312+0000 7fc63ba86700 1 -- 172.21.15.142:0/2168895630 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc61c06dd60 msgr2=0x7fc61c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:33.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.313+0000 7fc63ba86700 1 --2- 172.21.15.142:0/2168895630 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc61c06dd60 0x7fc61c070220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc624009cb0 tx=0x7fc624009260 comp rx=0 tx=0).stop 2024-03-29T04:53:33.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.313+0000 7fc63ba86700 1 -- 172.21.15.142:0/2168895630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc634105a40 msgr2=0x7fc634197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:33.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.313+0000 7fc63ba86700 1 --2- 172.21.15.142:0/2168895630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc634105a40 0x7fc634197ab0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc628005f50 tx=0x7fc62800f740 comp rx=0 tx=0).stop 2024-03-29T04:53:33.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.313+0000 7fc63ba86700 1 -- 172.21.15.142:0/2168895630 shutdown_connections 2024-03-29T04:53:33.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.313+0000 7fc63ba86700 1 --2- 172.21.15.142:0/2168895630 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc61c06dd60 0x7fc61c070220 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.313+0000 7fc63ba86700 1 --2- 172.21.15.142:0/2168895630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6340ffbd0 0x7fc634197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.313+0000 7fc63ba86700 1 --2- 172.21.15.142:0/2168895630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc634105a40 0x7fc634197ab0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.313+0000 7fc63ba86700 1 -- 172.21.15.142:0/2168895630 >> 172.21.15.142:0/2168895630 conn(0x7fc6340755a0 msgr2=0x7fc6340fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:33.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.313+0000 7fc63ba86700 1 -- 172.21.15.142:0/2168895630 shutdown_connections 2024-03-29T04:53:33.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.313+0000 7fc63ba86700 1 -- 172.21.15.142:0/2168895630 wait complete. 2024-03-29T04:53:33.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.417+0000 7f2a40f0a700 1 -- 172.21.15.142:0/2138509014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c105a40 msgr2=0x7f2a3c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:33.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.417+0000 7f2a40f0a700 1 --2- 172.21.15.142:0/2138509014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c105a40 0x7f2a3c105e20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009910 tx=0x7f2a2c009c20 comp rx=0 tx=0).stop 2024-03-29T04:53:33.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.417+0000 7f2a40f0a700 1 -- 172.21.15.142:0/2138509014 shutdown_connections 2024-03-29T04:53:33.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.417+0000 7f2a40f0a700 1 --2- 172.21.15.142:0/2138509014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3c0ffbd0 0x7f2a3c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.417+0000 7f2a40f0a700 1 --2- 172.21.15.142:0/2138509014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c105a40 0x7f2a3c105e20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.417+0000 7f2a40f0a700 1 -- 172.21.15.142:0/2138509014 >> 172.21.15.142:0/2138509014 conn(0x7f2a3c0755a0 msgr2=0x7f2a3c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:33.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.417+0000 7f2a40f0a700 1 -- 172.21.15.142:0/2138509014 shutdown_connections 2024-03-29T04:53:33.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.417+0000 7f2a40f0a700 1 -- 172.21.15.142:0/2138509014 wait complete. 2024-03-29T04:53:33.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a40f0a700 1 Processor -- start 2024-03-29T04:53:33.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a40f0a700 1 -- start start 2024-03-29T04:53:33.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a40f0a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3c0ffbd0 0x7f2a3c197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:33.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a40f0a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c105a40 0x7f2a3c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:33.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a40f0a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c198170 con 0x7f2a3c105a40 2024-03-29T04:53:33.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a40f0a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c1982b0 con 0x7f2a3c0ffbd0 2024-03-29T04:53:33.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a39d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c105a40 0x7f2a3c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a39d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c105a40 0x7f2a3c197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59580/0 (socket says 172.21.15.142:59580) 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a39d9b700 1 -- 172.21.15.142:0/491331972 learned_addr learned my addr 172.21.15.142:0/491331972 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a3a59c700 1 --2- 172.21.15.142:0/491331972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3c0ffbd0 0x7f2a3c197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a39d9b700 1 -- 172.21.15.142:0/491331972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3c0ffbd0 msgr2=0x7f2a3c197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a39d9b700 1 --2- 172.21.15.142:0/491331972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3c0ffbd0 0x7f2a3c197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.418+0000 7f2a39d9b700 1 -- 172.21.15.142:0/491331972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c0095f0 con 0x7f2a3c105a40 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.419+0000 7f2a39d9b700 1 --2- 172.21.15.142:0/491331972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c105a40 0x7f2a3c197a90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c8a0 tx=0x7f2a3000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.419+0000 7f2a2b7fe700 1 -- 172.21.15.142:0/491331972 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a300078e0 con 0x7f2a3c105a40 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.419+0000 7f2a40f0a700 1 -- 172.21.15.142:0/491331972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c1a05c0 con 0x7f2a3c105a40 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.419+0000 7f2a2b7fe700 1 -- 172.21.15.142:0/491331972 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a3000f450 con 0x7f2a3c105a40 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.419+0000 7f2a2b7fe700 1 -- 172.21.15.142:0/491331972 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3000e5e0 con 0x7f2a3c105a40 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.419+0000 7f2a40f0a700 1 -- 172.21.15.142:0/491331972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c1a0ae0 con 0x7f2a3c105a40 2024-03-29T04:53:33.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.420+0000 7f2a40f0a700 1 -- 172.21.15.142:0/491331972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c108040 con 0x7f2a3c105a40 2024-03-29T04:53:33.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.420+0000 7f2a2b7fe700 1 -- 172.21.15.142:0/491331972 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f2a3000f5c0 con 0x7f2a3c105a40 2024-03-29T04:53:33.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.421+0000 7f2a2b7fe700 1 --2- 172.21.15.142:0/491331972 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2a2406dd60 0x7f2a24070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:33.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.421+0000 7f2a2b7fe700 1 -- 172.21.15.142:0/491331972 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f2a3008dba0 con 0x7f2a3c105a40 2024-03-29T04:53:33.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.421+0000 7f2a3a59c700 1 --2- 172.21.15.142:0/491331972 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2a2406dd60 0x7f2a24070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:33.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.421+0000 7f2a3a59c700 1 --2- 172.21.15.142:0/491331972 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2a2406dd60 0x7f2a24070220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c006010 tx=0x7f2a2c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:33.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.425+0000 7f2a2b7fe700 1 -- 172.21.15.142:0/491331972 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2a30057070 con 0x7f2a3c105a40 2024-03-29T04:53:33.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.707+0000 7f2a40f0a700 1 -- 172.21.15.142:0/491331972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2a3c04ea90 con 0x7f2a3c105a40 2024-03-29T04:53:33.713 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:33 smithi142 ceph-mon[112475]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:33.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.708+0000 7f2a2b7fe700 1 -- 172.21.15.142:0/491331972 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f2a3005a730 con 0x7f2a3c105a40 2024-03-29T04:53:33.714 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:53:33.714 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:53:33.714 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:53:33.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:53:33.714 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:53:33.714 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:53:33.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:53:33.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:53:33.714 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-03-29T04:53:33.714 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T04:53:33.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:53:33.715 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:53:33.715 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:53:33.715 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:53:33.715 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:53:33.715 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-03-29T04:53:33.715 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 8 2024-03-29T04:53:33.715 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:53:33.715 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:53:33.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.710+0000 7f2a40f0a700 1 -- 172.21.15.142:0/491331972 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2a2406dd60 msgr2=0x7f2a24070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:33.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.710+0000 7f2a40f0a700 1 --2- 172.21.15.142:0/491331972 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2a2406dd60 0x7f2a24070220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c006010 tx=0x7f2a2c01a040 comp rx=0 tx=0).stop 2024-03-29T04:53:33.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.710+0000 7f2a40f0a700 1 -- 172.21.15.142:0/491331972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c105a40 msgr2=0x7f2a3c197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:33.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.710+0000 7f2a40f0a700 1 --2- 172.21.15.142:0/491331972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c105a40 0x7f2a3c197a90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c8a0 tx=0x7f2a3000cbb0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.710+0000 7f2a40f0a700 1 -- 172.21.15.142:0/491331972 shutdown_connections 2024-03-29T04:53:33.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.710+0000 7f2a40f0a700 1 --2- 172.21.15.142:0/491331972 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2a2406dd60 0x7f2a24070220 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.710+0000 7f2a40f0a700 1 --2- 172.21.15.142:0/491331972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a3c0ffbd0 0x7f2a3c197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.710+0000 7f2a40f0a700 1 --2- 172.21.15.142:0/491331972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c105a40 0x7f2a3c197a90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.710+0000 7f2a40f0a700 1 -- 172.21.15.142:0/491331972 >> 172.21.15.142:0/491331972 conn(0x7f2a3c0755a0 msgr2=0x7f2a3c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:33.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.710+0000 7f2a40f0a700 1 -- 172.21.15.142:0/491331972 shutdown_connections 2024-03-29T04:53:33.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.710+0000 7f2a40f0a700 1 -- 172.21.15.142:0/491331972 wait complete. 2024-03-29T04:53:33.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:33 smithi152 ceph-mon[88186]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:33.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.815+0000 7fbd22638700 1 -- 172.21.15.142:0/3825000574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c105a30 msgr2=0x7fbd1c105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:33.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.815+0000 7fbd22638700 1 --2- 172.21.15.142:0/3825000574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c105a30 0x7fbd1c105e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009910 tx=0x7fbd0c009c20 comp rx=0 tx=0).stop 2024-03-29T04:53:33.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.815+0000 7fbd22638700 1 -- 172.21.15.142:0/3825000574 shutdown_connections 2024-03-29T04:53:33.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.815+0000 7fbd22638700 1 --2- 172.21.15.142:0/3825000574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd1c0ffa30 0x7fbd1c0ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.815+0000 7fbd22638700 1 --2- 172.21.15.142:0/3825000574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c105a30 0x7fbd1c105e10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.815+0000 7fbd22638700 1 -- 172.21.15.142:0/3825000574 >> 172.21.15.142:0/3825000574 conn(0x7fbd1c0fb520 msgr2=0x7fbd1c0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:33.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.815+0000 7fbd22638700 1 -- 172.21.15.142:0/3825000574 shutdown_connections 2024-03-29T04:53:33.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.815+0000 7fbd22638700 1 -- 172.21.15.142:0/3825000574 wait complete. 2024-03-29T04:53:33.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.816+0000 7fbd22638700 1 Processor -- start 2024-03-29T04:53:33.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.816+0000 7fbd22638700 1 -- start start 2024-03-29T04:53:33.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.816+0000 7fbd22638700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c0ffa30 0x7fbd1c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:33.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.816+0000 7fbd22638700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd1c105a30 0x7fbd1c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:33.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.816+0000 7fbd22638700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c19d240 con 0x7fbd1c0ffa30 2024-03-29T04:53:33.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.816+0000 7fbd22638700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c1966f0 con 0x7fbd1c105a30 2024-03-29T04:53:33.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.816+0000 7fbd1bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c0ffa30 0x7fbd1c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:33.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.816+0000 7fbd1bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c0ffa30 0x7fbd1c19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59594/0 (socket says 172.21.15.142:59594) 2024-03-29T04:53:33.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.816+0000 7fbd1bfff700 1 -- 172.21.15.142:0/1245164937 learned_addr learned my addr 172.21.15.142:0/1245164937 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:33.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.817+0000 7fbd1b7fe700 1 --2- 172.21.15.142:0/1245164937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd1c105a30 0x7fbd1c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:33.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.817+0000 7fbd1bfff700 1 -- 172.21.15.142:0/1245164937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd1c105a30 msgr2=0x7fbd1c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:33.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.817+0000 7fbd1bfff700 1 --2- 172.21.15.142:0/1245164937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd1c105a30 0x7fbd1c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:33.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.817+0000 7fbd1bfff700 1 -- 172.21.15.142:0/1245164937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0095f0 con 0x7fbd1c0ffa30 2024-03-29T04:53:33.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.817+0000 7fbd1b7fe700 1 --2- 172.21.15.142:0/1245164937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd1c105a30 0x7fbd1c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:53:33.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.817+0000 7fbd1bfff700 1 --2- 172.21.15.142:0/1245164937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c0ffa30 0x7fbd1c19c670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c006010 tx=0x7fbd0c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:33.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.817+0000 7fbd197fa700 1 -- 172.21.15.142:0/1245164937 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd0c01d070 con 0x7fbd1c0ffa30 2024-03-29T04:53:33.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.817+0000 7fbd197fa700 1 -- 172.21.15.142:0/1245164937 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd0c004fb0 con 0x7fbd1c0ffa30 2024-03-29T04:53:33.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.817+0000 7fbd22638700 1 -- 172.21.15.142:0/1245164937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1c196970 con 0x7fbd1c0ffa30 2024-03-29T04:53:33.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.817+0000 7fbd197fa700 1 -- 172.21.15.142:0/1245164937 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd0c022ea0 con 0x7fbd1c0ffa30 2024-03-29T04:53:33.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.817+0000 7fbd22638700 1 -- 172.21.15.142:0/1245164937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1c196e60 con 0x7fbd1c0ffa30 2024-03-29T04:53:33.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.818+0000 7fbd22638700 1 -- 172.21.15.142:0/1245164937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1c108030 con 0x7fbd1c0ffa30 2024-03-29T04:53:33.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.818+0000 7fbd197fa700 1 -- 172.21.15.142:0/1245164937 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fbd0c022470 con 0x7fbd1c0ffa30 2024-03-29T04:53:33.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.819+0000 7fbd197fa700 1 --2- 172.21.15.142:0/1245164937 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbd0406dd60 0x7fbd04070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:33.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.819+0000 7fbd197fa700 1 -- 172.21.15.142:0/1245164937 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fbd0c08f1c0 con 0x7fbd1c0ffa30 2024-03-29T04:53:33.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.819+0000 7fbd1b7fe700 1 --2- 172.21.15.142:0/1245164937 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbd0406dd60 0x7fbd04070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:33.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.819+0000 7fbd1b7fe700 1 --2- 172.21.15.142:0/1245164937 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbd0406dd60 0x7fbd04070220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c197b60 tx=0x7fbd10008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:33.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:33.823+0000 7fbd197fa700 1 -- 172.21.15.142:0/1245164937 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbd0c05c230 con 0x7fbd1c0ffa30 2024-03-29T04:53:34.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.063+0000 7fbd22638700 1 -- 172.21.15.142:0/1245164937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbd1c04ea90 con 0x7fbd1c0ffa30 2024-03-29T04:53:34.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.064+0000 7fbd197fa700 1 -- 172.21.15.142:0/1245164937 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1906 (secure 0 0 0) 0x7fbd0c022720 con 0x7fbd1c0ffa30 2024-03-29T04:53:34.069 INFO:teuthology.orchestra.run.smithi142.stdout:e21 2024-03-29T04:53:34.069 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:53:34.069 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:53:34.069 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 21 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:49:01.288637+0000 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:53:34.070 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 47 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:up {0=14494,1=24275} 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:53:34.071 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:53:34.072 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:53:34.072 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:53:34.072 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:53:34.072 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:53:34.072 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{0:14494} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:53:34.072 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{0:14506} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:53:34.072 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{1:24275} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:53:34.072 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{1:24287} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:53:34.072 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:53:34.072 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:53:34.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.066+0000 7fbd22638700 1 -- 172.21.15.142:0/1245164937 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbd0406dd60 msgr2=0x7fbd04070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:34.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.066+0000 7fbd22638700 1 --2- 172.21.15.142:0/1245164937 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbd0406dd60 0x7fbd04070220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c197b60 tx=0x7fbd10008040 comp rx=0 tx=0).stop 2024-03-29T04:53:34.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.066+0000 7fbd22638700 1 -- 172.21.15.142:0/1245164937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c0ffa30 msgr2=0x7fbd1c19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:34.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.066+0000 7fbd22638700 1 --2- 172.21.15.142:0/1245164937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c0ffa30 0x7fbd1c19c670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c006010 tx=0x7fbd0c00be20 comp rx=0 tx=0).stop 2024-03-29T04:53:34.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.066+0000 7fbd22638700 1 -- 172.21.15.142:0/1245164937 shutdown_connections 2024-03-29T04:53:34.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.066+0000 7fbd22638700 1 --2- 172.21.15.142:0/1245164937 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbd0406dd60 0x7fbd04070220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.066+0000 7fbd22638700 1 --2- 172.21.15.142:0/1245164937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c0ffa30 0x7fbd1c19c670 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.066+0000 7fbd22638700 1 --2- 172.21.15.142:0/1245164937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd1c105a30 0x7fbd1c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.066+0000 7fbd22638700 1 -- 172.21.15.142:0/1245164937 >> 172.21.15.142:0/1245164937 conn(0x7fbd1c0fb520 msgr2=0x7fbd1c0fbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:34.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.066+0000 7fbd22638700 1 -- 172.21.15.142:0/1245164937 shutdown_connections 2024-03-29T04:53:34.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.066+0000 7fbd22638700 1 -- 172.21.15.142:0/1245164937 wait complete. 2024-03-29T04:53:34.074 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 21 2024-03-29T04:53:34.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.175+0000 7f0279dbe700 1 -- 172.21.15.142:0/1410764989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02740ff310 msgr2=0x7f02740ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:34.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.175+0000 7f0279dbe700 1 --2- 172.21.15.142:0/1410764989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02740ff310 0x7f02740ff770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0268009910 tx=0x7f0268009c20 comp rx=0 tx=0).stop 2024-03-29T04:53:34.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.175+0000 7f0279dbe700 1 -- 172.21.15.142:0/1410764989 shutdown_connections 2024-03-29T04:53:34.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.175+0000 7f0279dbe700 1 --2- 172.21.15.142:0/1410764989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02740ff310 0x7f02740ff770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.175+0000 7f0279dbe700 1 --2- 172.21.15.142:0/1410764989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0274106c10 0x7f0274106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.175+0000 7f0279dbe700 1 -- 172.21.15.142:0/1410764989 >> 172.21.15.142:0/1410764989 conn(0x7f0274074bf0 msgr2=0x7f0274075000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:34.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.175+0000 7f0279dbe700 1 -- 172.21.15.142:0/1410764989 shutdown_connections 2024-03-29T04:53:34.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.175+0000 7f0279dbe700 1 -- 172.21.15.142:0/1410764989 wait complete. 2024-03-29T04:53:34.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.175+0000 7f0279dbe700 1 Processor -- start 2024-03-29T04:53:34.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f0279dbe700 1 -- start start 2024-03-29T04:53:34.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f0279dbe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02740ff310 0x7f02741004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:34.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f0279dbe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0274106c10 0x7f0274100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:34.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f0279dbe700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0274100f70 con 0x7f02740ff310 2024-03-29T04:53:34.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f0279dbe700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02741010b0 con 0x7f0274106c10 2024-03-29T04:53:34.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f02737fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02740ff310 0x7f02741004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:34.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f02737fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02740ff310 0x7f02741004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59604/0 (socket says 172.21.15.142:59604) 2024-03-29T04:53:34.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f02737fe700 1 -- 172.21.15.142:0/1644071077 learned_addr learned my addr 172.21.15.142:0/1644071077 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:34.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f02737fe700 1 -- 172.21.15.142:0/1644071077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0274106c10 msgr2=0x7f0274100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:34.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f02737fe700 1 --2- 172.21.15.142:0/1644071077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0274106c10 0x7f0274100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f02737fe700 1 -- 172.21.15.142:0/1644071077 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680095f0 con 0x7f02740ff310 2024-03-29T04:53:34.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f02737fe700 1 --2- 172.21.15.142:0/1644071077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02740ff310 0x7f02741004f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f026400b4b0 tx=0x7f026400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:34.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f0270ff9700 1 -- 172.21.15.142:0/1644071077 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0264011800 con 0x7f02740ff310 2024-03-29T04:53:34.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f0270ff9700 1 -- 172.21.15.142:0/1644071077 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0264011e20 con 0x7f02740ff310 2024-03-29T04:53:34.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f0279dbe700 1 -- 172.21.15.142:0/1644071077 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02741a2a60 con 0x7f02740ff310 2024-03-29T04:53:34.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.176+0000 7f0270ff9700 1 -- 172.21.15.142:0/1644071077 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f026400f2e0 con 0x7f02740ff310 2024-03-29T04:53:34.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.177+0000 7f0279dbe700 1 -- 172.21.15.142:0/1644071077 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02741a2e10 con 0x7f02740ff310 2024-03-29T04:53:34.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.178+0000 7f0279dbe700 1 -- 172.21.15.142:0/1644071077 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0274109110 con 0x7f02740ff310 2024-03-29T04:53:34.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.178+0000 7f0270ff9700 1 -- 172.21.15.142:0/1644071077 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f0264010380 con 0x7f02740ff310 2024-03-29T04:53:34.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.178+0000 7f0270ff9700 1 --2- 172.21.15.142:0/1644071077 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f025c06dd60 0x7f025c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:34.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.179+0000 7f0270ff9700 1 -- 172.21.15.142:0/1644071077 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f026408d760 con 0x7f02740ff310 2024-03-29T04:53:34.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.179+0000 7f0272ffd700 1 --2- 172.21.15.142:0/1644071077 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f025c06dd60 0x7f025c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:34.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.179+0000 7f0272ffd700 1 --2- 172.21.15.142:0/1644071077 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f025c06dd60 0x7f025c070220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f026800b5c0 tx=0x7f0268005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:34.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.183+0000 7f0270ff9700 1 -- 172.21.15.142:0/1644071077 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0264056c30 con 0x7f02740ff310 2024-03-29T04:53:34.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.400+0000 7f0279dbe700 1 -- 172.21.15.142:0/1644071077 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f027402cd10 con 0x7f025c06dd60 2024-03-29T04:53:34.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.401+0000 7f0270ff9700 1 -- 172.21.15.142:0/1644071077 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f027402cd10 con 0x7f025c06dd60 2024-03-29T04:53:34.407 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:53:34.407 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:53:34.407 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:53:34.407 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:53:34.407 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:53:34.407 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:53:34.407 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:53:34.407 INFO:teuthology.orchestra.run.smithi142.stdout: "mon" 2024-03-29T04:53:34.407 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:53:34.407 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "10/23 daemons upgraded", 2024-03-29T04:53:34.407 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading osd daemons", 2024-03-29T04:53:34.407 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:53:34.408 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:53:34.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.404+0000 7f0279dbe700 1 -- 172.21.15.142:0/1644071077 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f025c06dd60 msgr2=0x7f025c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:34.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.404+0000 7f0279dbe700 1 --2- 172.21.15.142:0/1644071077 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f025c06dd60 0x7f025c070220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f026800b5c0 tx=0x7f0268005960 comp rx=0 tx=0).stop 2024-03-29T04:53:34.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.404+0000 7f0279dbe700 1 -- 172.21.15.142:0/1644071077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02740ff310 msgr2=0x7f02741004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:34.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.404+0000 7f0279dbe700 1 --2- 172.21.15.142:0/1644071077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02740ff310 0x7f02741004f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f026400b4b0 tx=0x7f026400b870 comp rx=0 tx=0).stop 2024-03-29T04:53:34.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.404+0000 7f0279dbe700 1 -- 172.21.15.142:0/1644071077 shutdown_connections 2024-03-29T04:53:34.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.404+0000 7f0279dbe700 1 --2- 172.21.15.142:0/1644071077 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f025c06dd60 0x7f025c070220 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.404+0000 7f0279dbe700 1 --2- 172.21.15.142:0/1644071077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02740ff310 0x7f02741004f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.404+0000 7f0279dbe700 1 --2- 172.21.15.142:0/1644071077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0274106c10 0x7f0274100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.404+0000 7f0279dbe700 1 -- 172.21.15.142:0/1644071077 >> 172.21.15.142:0/1644071077 conn(0x7f0274074bf0 msgr2=0x7f02740fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:34.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.404+0000 7f0279dbe700 1 -- 172.21.15.142:0/1644071077 shutdown_connections 2024-03-29T04:53:34.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.404+0000 7f0279dbe700 1 -- 172.21.15.142:0/1644071077 wait complete. 2024-03-29T04:53:34.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.507+0000 7f45da94b700 1 -- 172.21.15.142:0/1963192016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4101770 msgr2=0x7f45d41057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:34.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.507+0000 7f45da94b700 1 --2- 172.21.15.142:0/1963192016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4101770 0x7f45d41057c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009910 tx=0x7f45c8009c20 comp rx=0 tx=0).stop 2024-03-29T04:53:34.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.507+0000 7f45da94b700 1 -- 172.21.15.142:0/1963192016 shutdown_connections 2024-03-29T04:53:34.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.507+0000 7f45da94b700 1 --2- 172.21.15.142:0/1963192016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4101770 0x7f45d41057c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.507+0000 7f45da94b700 1 --2- 172.21.15.142:0/1963192016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45d4100dc0 0x7f45d41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.507+0000 7f45da94b700 1 -- 172.21.15.142:0/1963192016 >> 172.21.15.142:0/1963192016 conn(0x7f45d40fc650 msgr2=0x7f45d40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:34.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.508+0000 7f45da94b700 1 -- 172.21.15.142:0/1963192016 shutdown_connections 2024-03-29T04:53:34.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.508+0000 7f45da94b700 1 -- 172.21.15.142:0/1963192016 wait complete. 2024-03-29T04:53:34.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.508+0000 7f45da94b700 1 Processor -- start 2024-03-29T04:53:34.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.508+0000 7f45da94b700 1 -- start start 2024-03-29T04:53:34.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.508+0000 7f45da94b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4100dc0 0x7f45d4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:34.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.508+0000 7f45da94b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45d4101770 0x7f45d40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45da94b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d4073b10 con 0x7f45d4100dc0 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45da94b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d4073c50 con 0x7f45d4101770 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45d3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4100dc0 0x7f45d4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45d3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4100dc0 0x7f45d4073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59618/0 (socket says 172.21.15.142:59618) 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45d3fff700 1 -- 172.21.15.142:0/3202240810 learned_addr learned my addr 172.21.15.142:0/3202240810 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45d37fe700 1 --2- 172.21.15.142:0/3202240810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45d4101770 0x7f45d40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45d3fff700 1 -- 172.21.15.142:0/3202240810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45d4101770 msgr2=0x7f45d40735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45d3fff700 1 --2- 172.21.15.142:0/3202240810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45d4101770 0x7f45d40735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45d3fff700 1 -- 172.21.15.142:0/3202240810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c80095f0 con 0x7f45d4100dc0 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45d3fff700 1 --2- 172.21.15.142:0/3202240810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4100dc0 0x7f45d4073090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f45c400b4b0 tx=0x7f45c400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45d17fa700 1 -- 172.21.15.142:0/3202240810 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45c4011800 con 0x7f45d4100dc0 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45d17fa700 1 -- 172.21.15.142:0/3202240810 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45c4011e20 con 0x7f45d4100dc0 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45da94b700 1 -- 172.21.15.142:0/3202240810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d41a2a70 con 0x7f45d4100dc0 2024-03-29T04:53:34.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45d17fa700 1 -- 172.21.15.142:0/3202240810 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45c400f2e0 con 0x7f45d4100dc0 2024-03-29T04:53:34.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.509+0000 7f45da94b700 1 -- 172.21.15.142:0/3202240810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d41a2e80 con 0x7f45d4100dc0 2024-03-29T04:53:34.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.510+0000 7f45da94b700 1 -- 172.21.15.142:0/3202240810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d41090f0 con 0x7f45d4100dc0 2024-03-29T04:53:34.518 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.511+0000 7f45d17fa700 1 -- 172.21.15.142:0/3202240810 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f45c4011960 con 0x7f45d4100dc0 2024-03-29T04:53:34.518 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.511+0000 7f45d17fa700 1 --2- 172.21.15.142:0/3202240810 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45bc06dd60 0x7f45bc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:53:34.518 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.511+0000 7f45d17fa700 1 -- 172.21.15.142:0/3202240810 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f45c408d810 con 0x7f45d4100dc0 2024-03-29T04:53:34.518 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.511+0000 7f45d37fe700 1 --2- 172.21.15.142:0/3202240810 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45bc06dd60 0x7f45bc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:53:34.518 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.512+0000 7f45d37fe700 1 --2- 172.21.15.142:0/3202240810 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45bc06dd60 0x7f45bc070220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f45c80059d0 tx=0x7f45c8005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:53:34.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.516+0000 7f45d17fa700 1 -- 172.21.15.142:0/3202240810 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f45c4056ce0 con 0x7f45d4100dc0 2024-03-29T04:53:34.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.796+0000 7f45da94b700 1 -- 172.21.15.142:0/3202240810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f45d404ea90 con 0x7f45d4100dc0 2024-03-29T04:53:34.802 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:34 smithi142 ceph-mon[112475]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:34.802 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:34 smithi142 ceph-mon[112475]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:34.802 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:34 smithi142 ceph-mon[112475]: pgmap v142: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-03-29T04:53:34.802 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:34 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/491331972' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:34.803 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:34 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1245164937' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:53:34.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.797+0000 7f45d17fa700 1 -- 172.21.15.142:0/3202240810 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f45c405a3a0 con 0x7f45d4100dc0 2024-03-29T04:53:34.803 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_OK 2024-03-29T04:53:34.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.799+0000 7f45da94b700 1 -- 172.21.15.142:0/3202240810 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45bc06dd60 msgr2=0x7f45bc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:34.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.799+0000 7f45da94b700 1 --2- 172.21.15.142:0/3202240810 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45bc06dd60 0x7f45bc070220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f45c80059d0 tx=0x7f45c8005960 comp rx=0 tx=0).stop 2024-03-29T04:53:34.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.799+0000 7f45da94b700 1 -- 172.21.15.142:0/3202240810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4100dc0 msgr2=0x7f45d4073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:53:34.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.799+0000 7f45da94b700 1 --2- 172.21.15.142:0/3202240810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4100dc0 0x7f45d4073090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f45c400b4b0 tx=0x7f45c400b870 comp rx=0 tx=0).stop 2024-03-29T04:53:34.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.799+0000 7f45da94b700 1 -- 172.21.15.142:0/3202240810 shutdown_connections 2024-03-29T04:53:34.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.799+0000 7f45da94b700 1 --2- 172.21.15.142:0/3202240810 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45bc06dd60 0x7f45bc070220 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.799+0000 7f45da94b700 1 --2- 172.21.15.142:0/3202240810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45d4100dc0 0x7f45d4073090 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.799+0000 7f45da94b700 1 --2- 172.21.15.142:0/3202240810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45d4101770 0x7f45d40735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:53:34.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.799+0000 7f45da94b700 1 -- 172.21.15.142:0/3202240810 >> 172.21.15.142:0/3202240810 conn(0x7f45d40fc650 msgr2=0x7f45d40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:53:34.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.799+0000 7f45da94b700 1 -- 172.21.15.142:0/3202240810 shutdown_connections 2024-03-29T04:53:34.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:53:34.799+0000 7f45da94b700 1 -- 172.21.15.142:0/3202240810 wait complete. 2024-03-29T04:53:35.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:34 smithi152 ceph-mon[88186]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:35.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:34 smithi152 ceph-mon[88186]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:35.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:34 smithi152 ceph-mon[88186]: pgmap v142: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-03-29T04:53:35.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:34 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/491331972' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:35.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:34 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1245164937' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:53:36.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:35 smithi152 ceph-mon[88186]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:36.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:36.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:35 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3202240810' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:53:36.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:35 smithi142 ceph-mon[112475]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:53:36.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:36.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:35 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3202240810' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:53:37.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:36 smithi152 ceph-mon[88186]: pgmap v143: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:53:37.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:36 smithi142 ceph-mon[112475]: pgmap v143: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:53:39.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:38 smithi152 ceph-mon[88186]: pgmap v144: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-03-29T04:53:39.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:38 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T04:53:39.106 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:38 smithi142 ceph-mon[112475]: pgmap v144: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-03-29T04:53:39.106 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:38 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T04:53:39.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:39 smithi152 ceph-mon[88186]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T04:53:39.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:39 smithi152 ceph-mon[88186]: Upgrade: osd.4 is safe to restart 2024-03-29T04:53:40.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:39 smithi142 ceph-mon[112475]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T04:53:40.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:39 smithi142 ceph-mon[112475]: Upgrade: osd.4 is safe to restart 2024-03-29T04:53:40.962 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:40 smithi152 ceph-mon[88186]: pgmap v145: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-29T04:53:40.962 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:40 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:40.962 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:40 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-29T04:53:40.962 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:40 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:53:41.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:40 smithi142 ceph-mon[112475]: pgmap v145: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-29T04:53:41.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:40 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:41.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:40 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-29T04:53:41.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:40 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:53:41.658 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:41 smithi152 ceph-mon[88186]: Upgrade: Updating osd.4 2024-03-29T04:53:41.658 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:41 smithi152 ceph-mon[88186]: Deploying daemon osd.4 on smithi152 2024-03-29T04:53:41.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:41 smithi142 ceph-mon[112475]: Upgrade: Updating osd.4 2024-03-29T04:53:41.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:41 smithi142 ceph-mon[112475]: Deploying daemon osd.4 on smithi152 2024-03-29T04:53:42.984 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:42 smithi152 ceph-mon[88186]: pgmap v146: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:53:43.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:42 smithi142 ceph-mon[112475]: pgmap v146: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T04:53:43.622 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:43 smithi152 systemd[1]: Stopping Ceph osd.4 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:53:44.046 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:43 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[50068]: 2024-03-29T04:53:43.658+0000 7f36c2c0c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T04:53:44.046 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:43 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[50068]: 2024-03-29T04:53:43.658+0000 7f36c2c0c700 -1 osd.4 83 *** Got signal Terminated *** 2024-03-29T04:53:44.046 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:43 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[50068]: 2024-03-29T04:53:43.658+0000 7f36c2c0c700 -1 osd.4 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T04:53:45.012 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:44 smithi152 ceph-mon[88186]: pgmap v147: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:53:45.012 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:44 smithi152 ceph-mon[88186]: osd.4 marked itself down and dead 2024-03-29T04:53:45.012 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:45.013 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:53:45.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:44 smithi142 ceph-mon[112475]: pgmap v147: 65 pgs: 65 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:53:45.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:44 smithi142 ceph-mon[112475]: osd.4 marked itself down and dead 2024-03-29T04:53:45.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:45.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:53:45.295 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:45 smithi152 podman[102557]: 2024-03-29 04:53:45.008056734 +0000 UTC m=+1.489000118 container died 5e0acbe50fe3df4edb7d801d1855c179c99521c3e8a22425d4a587575ea534bf (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD) 2024-03-29T04:53:45.648 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:45 smithi152 podman[102557]: 2024-03-29 04:53:45.384396622 +0000 UTC m=+1.865340012 container cleanup 5e0acbe50fe3df4edb7d801d1855c179c99521c3e8a22425d4a587575ea534bf (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4, RELEASE=HEAD, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:53:45.648 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:45 smithi152 bash[102557]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4 2024-03-29T04:53:45.969 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:45 smithi152 podman[102575]: 2024-03-29 04:53:45.644135929 +0000 UTC m=+0.640056616 container remove 5e0acbe50fe3df4edb7d801d1855c179c99521c3e8a22425d4a587575ea534bf (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-03-29T04:53:45.969 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:45 smithi152 ceph-mon[88186]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T04:53:45.969 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:45 smithi152 ceph-mon[88186]: osdmap e84: 6 total, 5 up, 6 in 2024-03-29T04:53:46.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:45 smithi142 ceph-mon[112475]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T04:53:46.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:45 smithi142 ceph-mon[112475]: osdmap e84: 6 total, 5 up, 6 in 2024-03-29T04:53:46.295 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:45 smithi152 podman[102669]: 2024-03-29 04:53:45.867978767 +0000 UTC m=+0.017635463 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:53:46.295 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:45 smithi152 podman[102669]: 2024-03-29 04:53:45.986905662 +0000 UTC m=+0.136562352 container create c74a59ccb9c80e3694f70608c06fe35efa197dc36ee6113d7a6b5f01ed76ca3c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, RELEASE=squid-8da2362, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-03-29T04:53:46.647 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:46 smithi152 podman[102669]: 2024-03-29 04:53:46.348154086 +0000 UTC m=+0.497810787 container init c74a59ccb9c80e3694f70608c06fe35efa197dc36ee6113d7a6b5f01ed76ca3c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362) 2024-03-29T04:53:46.647 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:46 smithi152 podman[102669]: 2024-03-29 04:53:46.351371815 +0000 UTC m=+0.501028505 container start c74a59ccb9c80e3694f70608c06fe35efa197dc36ee6113d7a6b5f01ed76ca3c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=squid-8da2362, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:53:46.647 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:46 smithi152 podman[102669]: 2024-03-29 04:53:46.431421203 +0000 UTC m=+0.581077903 container attach c74a59ccb9c80e3694f70608c06fe35efa197dc36ee6113d7a6b5f01ed76ca3c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-29T04:53:46.939 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:46 smithi152 podman[102669]: 2024-03-29 04:53:46.642604699 +0000 UTC m=+0.792261388 container died c74a59ccb9c80e3694f70608c06fe35efa197dc36ee6113d7a6b5f01ed76ca3c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:53:46.940 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:46 smithi152 ceph-mon[88186]: pgmap v149: 65 pgs: 4 peering, 4 stale+active+clean, 57 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-03-29T04:53:46.940 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:46 smithi152 ceph-mon[88186]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-29T04:53:46.940 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:46 smithi152 ceph-mon[88186]: osdmap e85: 6 total, 5 up, 6 in 2024-03-29T04:53:47.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:46 smithi142 ceph-mon[112475]: pgmap v149: 65 pgs: 4 peering, 4 stale+active+clean, 57 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-03-29T04:53:47.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:46 smithi142 ceph-mon[112475]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-29T04:53:47.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:46 smithi142 ceph-mon[112475]: osdmap e85: 6 total, 5 up, 6 in 2024-03-29T04:53:47.546 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:47 smithi152 podman[102695]: 2024-03-29 04:53:47.210677191 +0000 UTC m=+0.559690808 container remove c74a59ccb9c80e3694f70608c06fe35efa197dc36ee6113d7a6b5f01ed76ca3c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8da2362, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-29T04:53:47.546 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:47 smithi152 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-29T04:53:47.546 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:47 smithi152 systemd[1]: Stopped Ceph osd.4 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:53:47.546 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:47 smithi152 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.4.service: Consumed 1min 7.140s CPU time. 2024-03-29T04:53:47.546 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:47 smithi152 systemd[1]: Starting Ceph osd.4 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:53:47.898 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:47 smithi152 ceph-mon[88186]: pgmap v151: 65 pgs: 6 active+undersized, 3 active+undersized+degraded, 4 peering, 2 stale+active+clean, 50 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 9/333 objects degraded (2.703%) 2024-03-29T04:53:47.898 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:47 smithi152 ceph-mon[88186]: Health check failed: Degraded data redundancy: 9/333 objects degraded (2.703%), 3 pgs degraded (PG_DEGRADED) 2024-03-29T04:53:47.898 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:47 smithi152 podman[102799]: 2024-03-29 04:53:47.545405023 +0000 UTC m=+0.018032375 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:53:47.898 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:47 smithi152 podman[102799]: 2024-03-29 04:53:47.654067233 +0000 UTC m=+0.126694584 container create e935c9039bfe704a6bff3bf97dc27cb526297764d8da4e209baff917d970e9a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate, ceph=True, RELEASE=squid-8da2362, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6) 2024-03-29T04:53:47.898 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:47 smithi152 podman[102799]: 2024-03-29 04:53:47.890325056 +0000 UTC m=+0.362952421 container init e935c9039bfe704a6bff3bf97dc27cb526297764d8da4e209baff917d970e9a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8da2362, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-29T04:53:47.898 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:47 smithi152 podman[102799]: 2024-03-29 04:53:47.893505496 +0000 UTC m=+0.366132858 container start e935c9039bfe704a6bff3bf97dc27cb526297764d8da4e209baff917d970e9a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate, RELEASE=squid-8da2362, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-29T04:53:48.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:47 smithi142 ceph-mon[112475]: pgmap v151: 65 pgs: 6 active+undersized, 3 active+undersized+degraded, 4 peering, 2 stale+active+clean, 50 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 9/333 objects degraded (2.703%) 2024-03-29T04:53:48.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:47 smithi142 ceph-mon[112475]: Health check failed: Degraded data redundancy: 9/333 objects degraded (2.703%), 3 pgs degraded (PG_DEGRADED) 2024-03-29T04:53:48.295 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:47 smithi152 podman[102799]: 2024-03-29 04:53:47.998794867 +0000 UTC m=+0.471422232 container attach e935c9039bfe704a6bff3bf97dc27cb526297764d8da4e209baff917d970e9a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, io.buildah.version=1.33.5) 2024-03-29T04:53:48.683 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate[102814]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-29T04:53:48.683 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate[102814]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-29T04:53:48.683 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 bash[102799]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-29T04:53:48.683 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 bash[102799]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-29T04:53:48.683 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 bash[102799]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-29T04:53:48.683 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 bash[102799]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-29T04:53:48.683 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 bash[102799]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-29T04:53:48.683 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 bash[102799]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-29T04:53:48.683 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 bash[102799]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-29T04:53:48.683 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate[102814]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-29T04:53:48.684 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate[102814]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-29T04:53:48.684 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate[102814]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-29T04:53:48.684 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate[102814]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-29T04:53:48.684 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate[102814]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-29T04:53:48.684 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:48 smithi152 podman[102799]: 2024-03-29 04:53:48.366773078 +0000 UTC m=+0.839400438 container died e935c9039bfe704a6bff3bf97dc27cb526297764d8da4e209baff917d970e9a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:53:49.295 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:49 smithi152 podman[102940]: 2024-03-29 04:53:49.003442462 +0000 UTC m=+0.628112794 container remove e935c9039bfe704a6bff3bf97dc27cb526297764d8da4e209baff917d970e9a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-29T04:53:49.296 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:49 smithi152 podman[103002]: 2024-03-29 04:53:49.115607509 +0000 UTC m=+0.018064542 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:53:49.296 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:49 smithi152 podman[103002]: 2024-03-29 04:53:49.246316543 +0000 UTC m=+0.148773550 container create eca640a33744335e0e28848261b5f5014ee010ded7484201dc7e3362b974607e (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8da2362, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-29T04:53:49.796 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:49 smithi152 podman[103002]: 2024-03-29 04:53:49.457322981 +0000 UTC m=+0.359780004 container init eca640a33744335e0e28848261b5f5014ee010ded7484201dc7e3362b974607e (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4, GIT_CLEAN=True, ceph=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, RELEASE=squid-8da2362, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-29T04:53:49.796 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:49 smithi152 podman[103002]: 2024-03-29 04:53:49.460335609 +0000 UTC m=+0.362792616 container start eca640a33744335e0e28848261b5f5014ee010ded7484201dc7e3362b974607e (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6) 2024-03-29T04:53:49.796 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:49 smithi152 ceph-osd[103021]: -- 172.21.15.152:0/3125999611 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560745af1680 con 0x560745abac00 2024-03-29T04:53:49.796 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:49 smithi152 bash[103002]: eca640a33744335e0e28848261b5f5014ee010ded7484201dc7e3362b974607e 2024-03-29T04:53:49.796 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:49 smithi152 systemd[1]: Started Ceph osd.4 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:53:50.295 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:50 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T04:53:50.011+0000 7f865c4fc7c0 -1 Falling back to public interface 2024-03-29T04:53:50.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:50 smithi152 ceph-mon[88186]: pgmap v152: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 4 peering, 33 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 50/333 objects degraded (15.015%) 2024-03-29T04:53:50.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:50.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:50.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:53:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:50 smithi142 ceph-mon[112475]: pgmap v152: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 4 peering, 33 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 50/333 objects degraded (15.015%) 2024-03-29T04:53:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:53:52.545 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:52 smithi152 ceph-mon[88186]: pgmap v153: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-03-29T04:53:52.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:52 smithi142 ceph-mon[112475]: pgmap v153: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-03-29T04:53:53.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:53 smithi152 ceph-mon[88186]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-29T04:53:53.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:53 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:53.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:53 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:53.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:53 smithi142 ceph-mon[112475]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-29T04:53:53.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:53 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:53.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:53 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:54.523 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:54 smithi152 ceph-mon[88186]: pgmap v154: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-03-29T04:53:54.523 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:54.523 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:54.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:54 smithi142 ceph-mon[112475]: pgmap v154: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 270 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-03-29T04:53:54.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:54.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:54.795 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:54 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T04:53:54.574+0000 7f865c4fc7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-29T04:53:55.421 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:55 smithi152 ceph-mon[88186]: from='osd.4 [v2:172.21.15.152:6808/3103638517,v1:172.21.15.152:6809/3103638517]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T04:53:55.421 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:55 smithi152 ceph-mon[88186]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T04:53:55.421 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:55 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T04:53:55.066+0000 7f865c4fc7c0 -1 osd.4 83 log_to_monitors true 2024-03-29T04:53:55.421 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 04:53:55 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T04:53:55.232+0000 7f864d954700 -1 osd.4 83 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:53:55.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:55 smithi142 ceph-mon[112475]: from='osd.4 [v2:172.21.15.152:6808/3103638517,v1:172.21.15.152:6809/3103638517]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T04:53:55.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:55 smithi142 ceph-mon[112475]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T04:53:56.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:56 smithi152 ceph-mon[88186]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-29T04:53:56.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:56 smithi152 ceph-mon[88186]: from='osd.4 [v2:172.21.15.152:6808/3103638517,v1:172.21.15.152:6809/3103638517]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:53:56.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:56 smithi152 ceph-mon[88186]: osdmap e86: 6 total, 5 up, 6 in 2024-03-29T04:53:56.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:56 smithi152 ceph-mon[88186]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:53:56.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:56 smithi152 ceph-mon[88186]: pgmap v156: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-03-29T04:53:56.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:56 smithi152 ceph-mon[88186]: Health check update: Degraded data redundancy: 55/333 objects degraded (16.517%), 16 pgs degraded (PG_DEGRADED) 2024-03-29T04:53:56.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:56 smithi142 ceph-mon[112475]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-29T04:53:56.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:56 smithi142 ceph-mon[112475]: from='osd.4 [v2:172.21.15.152:6808/3103638517,v1:172.21.15.152:6809/3103638517]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:53:56.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:56 smithi142 ceph-mon[112475]: osdmap e86: 6 total, 5 up, 6 in 2024-03-29T04:53:56.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:56 smithi142 ceph-mon[112475]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:53:56.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:56 smithi142 ceph-mon[112475]: pgmap v156: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-03-29T04:53:56.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:56 smithi142 ceph-mon[112475]: Health check update: Degraded data redundancy: 55/333 objects degraded (16.517%), 16 pgs degraded (PG_DEGRADED) 2024-03-29T04:53:57.422 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:57 smithi152 ceph-mon[88186]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T04:53:57.422 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:57 smithi152 ceph-mon[88186]: osd.4 [v2:172.21.15.152:6808/3103638517,v1:172.21.15.152:6809/3103638517] boot 2024-03-29T04:53:57.422 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:57 smithi152 ceph-mon[88186]: osdmap e87: 6 total, 6 up, 6 in 2024-03-29T04:53:57.422 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:57 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:53:57.423 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:57 smithi152 ceph-mon[88186]: osdmap e88: 6 total, 6 up, 6 in 2024-03-29T04:53:57.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:57 smithi142 ceph-mon[112475]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T04:53:57.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:57 smithi142 ceph-mon[112475]: osd.4 [v2:172.21.15.152:6808/3103638517,v1:172.21.15.152:6809/3103638517] boot 2024-03-29T04:53:57.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:57 smithi142 ceph-mon[112475]: osdmap e87: 6 total, 6 up, 6 in 2024-03-29T04:53:57.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:57 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T04:53:57.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:57 smithi142 ceph-mon[112475]: osdmap e88: 6 total, 6 up, 6 in 2024-03-29T04:53:58.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:58.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:58.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:53:58.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:53:58.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:58.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: pgmap v159: 65 pgs: 6 peering, 12 active+undersized, 14 active+undersized+degraded, 33 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 49/333 objects degraded (14.715%) 2024-03-29T04:53:58.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:53:58.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:58.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:58.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:58.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T04:53:58.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T04:53:58.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:53:58 smithi152 ceph-mon[88186]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-03-29T04:53:58.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:58.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:58.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:53:58.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:53:58.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:53:58.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: pgmap v159: 65 pgs: 6 peering, 12 active+undersized, 14 active+undersized+degraded, 33 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 49/333 objects degraded (14.715%) 2024-03-29T04:53:58.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:53:58.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:58.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:58.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:53:58.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T04:53:58.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T04:53:58.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:53:58 smithi142 ceph-mon[112475]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-03-29T04:54:00.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:00 smithi142 ceph-mon[112475]: pgmap v160: 65 pgs: 6 peering, 5 active+undersized, 4 active+undersized+degraded, 50 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s; 10/333 objects degraded (3.003%) 2024-03-29T04:54:00.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:00.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:54:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:00 smithi152 ceph-mon[88186]: pgmap v160: 65 pgs: 6 peering, 5 active+undersized, 4 active+undersized+degraded, 50 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s; 10/333 objects degraded (3.003%) 2024-03-29T04:54:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:54:01.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:01 smithi142 ceph-mon[112475]: Health check update: Degraded data redundancy: 10/333 objects degraded (3.003%), 4 pgs degraded (PG_DEGRADED) 2024-03-29T04:54:02.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:01 smithi152 ceph-mon[88186]: Health check update: Degraded data redundancy: 10/333 objects degraded (3.003%), 4 pgs degraded (PG_DEGRADED) 2024-03-29T04:54:02.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:02 smithi152 ceph-mon[88186]: pgmap v161: 65 pgs: 6 peering, 59 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-29T04:54:02.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:02 smithi152 ceph-mon[88186]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/333 objects degraded (3.003%), 4 pgs degraded) 2024-03-29T04:54:02.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:02 smithi152 ceph-mon[88186]: Cluster is now healthy 2024-03-29T04:54:02.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:02 smithi142 ceph-mon[112475]: pgmap v161: 65 pgs: 6 peering, 59 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-29T04:54:02.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:02 smithi142 ceph-mon[112475]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/333 objects degraded (3.003%), 4 pgs degraded) 2024-03-29T04:54:02.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:02 smithi142 ceph-mon[112475]: Cluster is now healthy 2024-03-29T04:54:04.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:04 smithi142 ceph-mon[112475]: pgmap v162: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-29T04:54:04.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.916+0000 7f175ef2e700 1 -- 172.21.15.142:0/619198036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1758105a40 msgr2=0x7f1758105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:04.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.916+0000 7f175ef2e700 1 --2- 172.21.15.142:0/619198036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1758105a40 0x7f1758105e20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f174c009910 tx=0x7f174c009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:04.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.916+0000 7f175ef2e700 1 -- 172.21.15.142:0/619198036 shutdown_connections 2024-03-29T04:54:04.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.916+0000 7f175ef2e700 1 --2- 172.21.15.142:0/619198036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17580ffa40 0x7f17580ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:04.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.916+0000 7f175ef2e700 1 --2- 172.21.15.142:0/619198036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1758105a40 0x7f1758105e20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:04.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.916+0000 7f175ef2e700 1 -- 172.21.15.142:0/619198036 >> 172.21.15.142:0/619198036 conn(0x7f17580fb570 msgr2=0x7f17580fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:04.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.916+0000 7f175ef2e700 1 -- 172.21.15.142:0/619198036 shutdown_connections 2024-03-29T04:54:04.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.916+0000 7f175ef2e700 1 -- 172.21.15.142:0/619198036 wait complete. 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f175ef2e700 1 Processor -- start 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f175ef2e700 1 -- start start 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f175ef2e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17580ffa40 0x7f17580fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f175ef2e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1758105a40 0x7f17580fc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f175ef2e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580fc960 con 0x7f1758105a40 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f175ef2e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580fcaa0 con 0x7f17580ffa40 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f1757fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1758105a40 0x7f17580fc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f1757fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1758105a40 0x7f17580fc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43534/0 (socket says 172.21.15.142:43534) 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f1757fff700 1 -- 172.21.15.142:0/4114229224 learned_addr learned my addr 172.21.15.142:0/4114229224 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f175ccca700 1 --2- 172.21.15.142:0/4114229224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17580ffa40 0x7f17580fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f1757fff700 1 -- 172.21.15.142:0/4114229224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17580ffa40 msgr2=0x7f17580fbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f1757fff700 1 --2- 172.21.15.142:0/4114229224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17580ffa40 0x7f17580fbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:04.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f1757fff700 1 -- 172.21.15.142:0/4114229224 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c0095f0 con 0x7f1758105a40 2024-03-29T04:54:04.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.917+0000 7f175ccca700 1 --2- 172.21.15.142:0/4114229224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17580ffa40 0x7f17580fbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:54:04.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.918+0000 7f1757fff700 1 --2- 172.21.15.142:0/4114229224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1758105a40 0x7f17580fc420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f174400b4b0 tx=0x7f174400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:04.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.918+0000 7f1755ffb700 1 -- 172.21.15.142:0/4114229224 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1744011800 con 0x7f1758105a40 2024-03-29T04:54:04.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.918+0000 7f175ef2e700 1 -- 172.21.15.142:0/4114229224 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17581a2a00 con 0x7f1758105a40 2024-03-29T04:54:04.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.918+0000 7f1755ffb700 1 -- 172.21.15.142:0/4114229224 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1744011e20 con 0x7f1758105a40 2024-03-29T04:54:04.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.918+0000 7f1755ffb700 1 -- 172.21.15.142:0/4114229224 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f174400f2e0 con 0x7f1758105a40 2024-03-29T04:54:04.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.918+0000 7f175ef2e700 1 -- 172.21.15.142:0/4114229224 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17581a2e10 con 0x7f1758105a40 2024-03-29T04:54:04.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.919+0000 7f175ef2e700 1 -- 172.21.15.142:0/4114229224 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1758109050 con 0x7f1758105a40 2024-03-29T04:54:04.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.920+0000 7f1755ffb700 1 -- 172.21.15.142:0/4114229224 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f1744010380 con 0x7f1758105a40 2024-03-29T04:54:04.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.920+0000 7f1755ffb700 1 --2- 172.21.15.142:0/4114229224 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f174806dd60 0x7f1748070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:04.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.920+0000 7f1755ffb700 1 -- 172.21.15.142:0/4114229224 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f174408d800 con 0x7f1758105a40 2024-03-29T04:54:04.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.920+0000 7f175ccca700 1 --2- 172.21.15.142:0/4114229224 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f174806dd60 0x7f1748070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:04.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.921+0000 7f175ccca700 1 --2- 172.21.15.142:0/4114229224 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f174806dd60 0x7f1748070220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f174c00b5c0 tx=0x7f174c005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:04.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:04.925+0000 7f1755ffb700 1 -- 172.21.15.142:0/4114229224 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1744056cd0 con 0x7f1758105a40 2024-03-29T04:54:05.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:04 smithi152 ceph-mon[88186]: pgmap v162: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-29T04:54:05.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.137+0000 7f175ef2e700 1 -- 172.21.15.142:0/4114229224 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f175802cd80 con 0x7f174806dd60 2024-03-29T04:54:05.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.138+0000 7f1755ffb700 1 -- 172.21.15.142:0/4114229224 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f175802cd80 con 0x7f174806dd60 2024-03-29T04:54:05.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.140+0000 7f175ef2e700 1 -- 172.21.15.142:0/4114229224 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f174806dd60 msgr2=0x7f1748070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:05.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.140+0000 7f175ef2e700 1 --2- 172.21.15.142:0/4114229224 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f174806dd60 0x7f1748070220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f174c00b5c0 tx=0x7f174c005480 comp rx=0 tx=0).stop 2024-03-29T04:54:05.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.140+0000 7f175ef2e700 1 -- 172.21.15.142:0/4114229224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1758105a40 msgr2=0x7f17580fc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:05.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.140+0000 7f175ef2e700 1 --2- 172.21.15.142:0/4114229224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1758105a40 0x7f17580fc420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f174400b4b0 tx=0x7f174400b870 comp rx=0 tx=0).stop 2024-03-29T04:54:05.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.140+0000 7f175ef2e700 1 -- 172.21.15.142:0/4114229224 shutdown_connections 2024-03-29T04:54:05.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.140+0000 7f175ef2e700 1 --2- 172.21.15.142:0/4114229224 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f174806dd60 0x7f1748070220 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.140+0000 7f175ef2e700 1 --2- 172.21.15.142:0/4114229224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17580ffa40 0x7f17580fbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.140+0000 7f175ef2e700 1 --2- 172.21.15.142:0/4114229224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1758105a40 0x7f17580fc420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.140+0000 7f175ef2e700 1 -- 172.21.15.142:0/4114229224 >> 172.21.15.142:0/4114229224 conn(0x7f17580fb570 msgr2=0x7f1758068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:05.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.141+0000 7f175ef2e700 1 -- 172.21.15.142:0/4114229224 shutdown_connections 2024-03-29T04:54:05.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.141+0000 7f175ef2e700 1 -- 172.21.15.142:0/4114229224 wait complete. 2024-03-29T04:54:05.156 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:54:05.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.246+0000 7f69aedbb700 1 -- 172.21.15.142:0/350269526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69a8100de0 msgr2=0x7f69a81011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:05.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.246+0000 7f69aedbb700 1 --2- 172.21.15.142:0/350269526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69a8100de0 0x7f69a81011c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f69a000b0a0 tx=0x7f69a000b3b0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.246+0000 7f69aedbb700 1 -- 172.21.15.142:0/350269526 shutdown_connections 2024-03-29T04:54:05.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.246+0000 7f69aedbb700 1 --2- 172.21.15.142:0/350269526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69a8101790 0x7f69a81057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.246+0000 7f69aedbb700 1 --2- 172.21.15.142:0/350269526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69a8100de0 0x7f69a81011c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.246+0000 7f69aedbb700 1 -- 172.21.15.142:0/350269526 >> 172.21.15.142:0/350269526 conn(0x7f69a80fc670 msgr2=0x7f69a80fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:05.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.246+0000 7f69aedbb700 1 -- 172.21.15.142:0/350269526 shutdown_connections 2024-03-29T04:54:05.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.246+0000 7f69aedbb700 1 -- 172.21.15.142:0/350269526 wait complete. 2024-03-29T04:54:05.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69aedbb700 1 Processor -- start 2024-03-29T04:54:05.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69aedbb700 1 -- start start 2024-03-29T04:54:05.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69aedbb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69a8100de0 0x7f69a819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:05.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69aedbb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69a8101790 0x7f69a819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:05.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69aedbb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a819d100 con 0x7f69a8100de0 2024-03-29T04:54:05.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69aedbb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a81965b0 con 0x7f69a8101790 2024-03-29T04:54:05.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69acb57700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69a8100de0 0x7f69a819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:05.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69acb57700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69a8100de0 0x7f69a819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43560/0 (socket says 172.21.15.142:43560) 2024-03-29T04:54:05.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69a7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69a8101790 0x7f69a819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:05.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69a7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69a8101790 0x7f69a819ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.142:45786/0 (socket says 172.21.15.142:45786) 2024-03-29T04:54:05.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69a7fff700 1 -- 172.21.15.142:0/3252792698 learned_addr learned my addr 172.21.15.142:0/3252792698 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:05.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69acb57700 1 -- 172.21.15.142:0/3252792698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69a8101790 msgr2=0x7f69a819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:05.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69acb57700 1 --2- 172.21.15.142:0/3252792698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69a8101790 0x7f69a819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69acb57700 1 -- 172.21.15.142:0/3252792698 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a0009d00 con 0x7f69a8100de0 2024-03-29T04:54:05.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.247+0000 7f69a7fff700 1 --2- 172.21.15.142:0/3252792698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69a8101790 0x7f69a819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:54:05.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.248+0000 7f69acb57700 1 --2- 172.21.15.142:0/3252792698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69a8100de0 0x7f69a819c530 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f69a0006130 tx=0x7f69a000bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:05.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.248+0000 7f69a5ffb700 1 -- 172.21.15.142:0/3252792698 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69a000e040 con 0x7f69a8100de0 2024-03-29T04:54:05.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.248+0000 7f69a5ffb700 1 -- 172.21.15.142:0/3252792698 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69a0003c10 con 0x7f69a8100de0 2024-03-29T04:54:05.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.248+0000 7f69a5ffb700 1 -- 172.21.15.142:0/3252792698 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69a001d6a0 con 0x7f69a8100de0 2024-03-29T04:54:05.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.248+0000 7f69aedbb700 1 -- 172.21.15.142:0/3252792698 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a8196830 con 0x7f69a8100de0 2024-03-29T04:54:05.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.248+0000 7f69aedbb700 1 -- 172.21.15.142:0/3252792698 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a8196ca0 con 0x7f69a8100de0 2024-03-29T04:54:05.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.249+0000 7f69aedbb700 1 -- 172.21.15.142:0/3252792698 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a8109180 con 0x7f69a8100de0 2024-03-29T04:54:05.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.250+0000 7f69a5ffb700 1 -- 172.21.15.142:0/3252792698 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f69a0019070 con 0x7f69a8100de0 2024-03-29T04:54:05.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.250+0000 7f69a5ffb700 1 --2- 172.21.15.142:0/3252792698 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f699406dd60 0x7f6994070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:05.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.250+0000 7f69a5ffb700 1 -- 172.21.15.142:0/3252792698 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f69a008eff0 con 0x7f69a8100de0 2024-03-29T04:54:05.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.250+0000 7f69a7fff700 1 --2- 172.21.15.142:0/3252792698 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f699406dd60 0x7f6994070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:05.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.251+0000 7f69a7fff700 1 --2- 172.21.15.142:0/3252792698 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f699406dd60 0x7f6994070220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f69a81059f0 tx=0x7f6998009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:05.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.255+0000 7f69a5ffb700 1 -- 172.21.15.142:0/3252792698 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f69a005c050 con 0x7f69a8100de0 2024-03-29T04:54:05.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.466+0000 7f69aedbb700 1 -- 172.21.15.142:0/3252792698 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f69a8196fd0 con 0x7f699406dd60 2024-03-29T04:54:05.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.467+0000 7f69a5ffb700 1 -- 172.21.15.142:0/3252792698 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f69a8196fd0 con 0x7f699406dd60 2024-03-29T04:54:05.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.469+0000 7f69aedbb700 1 -- 172.21.15.142:0/3252792698 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f699406dd60 msgr2=0x7f6994070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:05.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.469+0000 7f69aedbb700 1 --2- 172.21.15.142:0/3252792698 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f699406dd60 0x7f6994070220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f69a81059f0 tx=0x7f6998009260 comp rx=0 tx=0).stop 2024-03-29T04:54:05.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.469+0000 7f69aedbb700 1 -- 172.21.15.142:0/3252792698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69a8100de0 msgr2=0x7f69a819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:05.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.469+0000 7f69aedbb700 1 --2- 172.21.15.142:0/3252792698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69a8100de0 0x7f69a819c530 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f69a0006130 tx=0x7f69a000bc60 comp rx=0 tx=0).stop 2024-03-29T04:54:05.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.469+0000 7f69aedbb700 1 -- 172.21.15.142:0/3252792698 shutdown_connections 2024-03-29T04:54:05.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.469+0000 7f69aedbb700 1 --2- 172.21.15.142:0/3252792698 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f699406dd60 0x7f6994070220 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.469+0000 7f69aedbb700 1 --2- 172.21.15.142:0/3252792698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69a8100de0 0x7f69a819c530 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.469+0000 7f69aedbb700 1 --2- 172.21.15.142:0/3252792698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69a8101790 0x7f69a819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.475 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.469+0000 7f69aedbb700 1 -- 172.21.15.142:0/3252792698 >> 172.21.15.142:0/3252792698 conn(0x7f69a80fc670 msgr2=0x7f69a80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:05.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.469+0000 7f69aedbb700 1 -- 172.21.15.142:0/3252792698 shutdown_connections 2024-03-29T04:54:05.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.469+0000 7f69aedbb700 1 -- 172.21.15.142:0/3252792698 wait complete. 2024-03-29T04:54:05.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.575+0000 7f675567b700 1 -- 172.21.15.142:0/1743305060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750068670 msgr2=0x7f6750068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:05.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.575+0000 7f675567b700 1 --2- 172.21.15.142:0/1743305060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750068670 0x7f6750068a50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6740009910 tx=0x7f6740009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:05.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.575+0000 7f675567b700 1 -- 172.21.15.142:0/1743305060 shutdown_connections 2024-03-29T04:54:05.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.575+0000 7f675567b700 1 --2- 172.21.15.142:0/1743305060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6750069020 0x7f6750105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.575+0000 7f675567b700 1 --2- 172.21.15.142:0/1743305060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750068670 0x7f6750068a50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.575+0000 7f675567b700 1 -- 172.21.15.142:0/1743305060 >> 172.21.15.142:0/1743305060 conn(0x7f67500755a0 msgr2=0x7f67500759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:05.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.575+0000 7f675567b700 1 -- 172.21.15.142:0/1743305060 shutdown_connections 2024-03-29T04:54:05.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.575+0000 7f675567b700 1 -- 172.21.15.142:0/1743305060 wait complete. 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.575+0000 7f675567b700 1 Processor -- start 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f675567b700 1 -- start start 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f675567b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6750068670 0x7f675019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f675567b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750069020 0x7f675019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f675567b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675019d050 con 0x7f6750069020 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f675567b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6750196590 con 0x7f6750068670 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f67467fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750069020 0x7f675019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f67467fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750069020 0x7f675019ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43586/0 (socket says 172.21.15.142:43586) 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f67467fc700 1 -- 172.21.15.142:0/2173038277 learned_addr learned my addr 172.21.15.142:0/2173038277 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f674effd700 1 --2- 172.21.15.142:0/2173038277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6750068670 0x7f675019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f67467fc700 1 -- 172.21.15.142:0/2173038277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6750068670 msgr2=0x7f675019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f67467fc700 1 --2- 172.21.15.142:0/2173038277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6750068670 0x7f675019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f67467fc700 1 -- 172.21.15.142:0/2173038277 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67400095f0 con 0x7f6750069020 2024-03-29T04:54:05.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.576+0000 7f67467fc700 1 --2- 172.21.15.142:0/2173038277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750069020 0x7f675019ca50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f673800c960 tx=0x7f673800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:05.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.577+0000 7f674cff9700 1 -- 172.21.15.142:0/2173038277 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6738007a70 con 0x7f6750069020 2024-03-29T04:54:05.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.577+0000 7f674cff9700 1 -- 172.21.15.142:0/2173038277 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6738007bd0 con 0x7f6750069020 2024-03-29T04:54:05.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.577+0000 7f675567b700 1 -- 172.21.15.142:0/2173038277 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6750196870 con 0x7f6750069020 2024-03-29T04:54:05.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.577+0000 7f674cff9700 1 -- 172.21.15.142:0/2173038277 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6738018630 con 0x7f6750069020 2024-03-29T04:54:05.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.577+0000 7f675567b700 1 -- 172.21.15.142:0/2173038277 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6750196d60 con 0x7f6750069020 2024-03-29T04:54:05.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.578+0000 7f675567b700 1 -- 172.21.15.142:0/2173038277 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6750109130 con 0x7f6750069020 2024-03-29T04:54:05.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.578+0000 7f674cff9700 1 -- 172.21.15.142:0/2173038277 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f673800f450 con 0x7f6750069020 2024-03-29T04:54:05.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.579+0000 7f674cff9700 1 --2- 172.21.15.142:0/2173038277 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f673c06dd60 0x7f673c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:05.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.579+0000 7f674cff9700 1 -- 172.21.15.142:0/2173038277 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f673808da80 con 0x7f6750069020 2024-03-29T04:54:05.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.579+0000 7f674effd700 1 --2- 172.21.15.142:0/2173038277 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f673c06dd60 0x7f673c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:05.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.579+0000 7f674effd700 1 --2- 172.21.15.142:0/2173038277 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f673c06dd60 0x7f673c070220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6740006010 tx=0x7f674001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:05.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.584+0000 7f674cff9700 1 -- 172.21.15.142:0/2173038277 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f673805abb0 con 0x7f6750069020 2024-03-29T04:54:05.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.801+0000 7f675567b700 1 -- 172.21.15.142:0/2173038277 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6750197090 con 0x7f673c06dd60 2024-03-29T04:54:05.817 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.812+0000 7f674cff9700 1 -- 172.21.15.142:0/2173038277 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6750197090 con 0x7f673c06dd60 2024-03-29T04:54:05.817 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:54:05.817 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (5m) 80s ago 17m 25.5M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (17m) 80s ago 17m 8762k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (16m) 13s ago 16m 8782k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (3m) 80s ago 17m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (2m) 13s ago 16m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (5m) 80s ago 17m 80.3M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (11m) 80s ago 11m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (11m) 80s ago 11m 63.5M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (11m) 13s ago 11m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (11m) 13s ago 11m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (7m) 80s ago 18m 506M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (6m) 13s ago 16m 451M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (4m) 80s ago 18m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:54:05.818 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (3m) 13s ago 16m 43.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:54:05.819 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (6m) 80s ago 17m 14.9M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:54:05.819 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (6m) 13s ago 16m 16.4M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:54:05.819 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (2m) 80s ago 15m 105M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:54:05.819 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (2m) 80s ago 15m 142M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:54:05.819 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (86s) 80s ago 14m 12.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:54:05.819 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (50s) 13s ago 14m 98.2M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T04:54:05.819 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (16s) 13s ago 13m 13.5M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T04:54:05.819 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (13m) 13s ago 13m 276M 2240M 18.2.1 5be31c24972a 3e9c96d907fc 2024-03-29T04:54:05.819 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (6m) 80s ago 16m 63.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:54:05.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.814+0000 7f675567b700 1 -- 172.21.15.142:0/2173038277 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f673c06dd60 msgr2=0x7f673c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:05.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.814+0000 7f675567b700 1 --2- 172.21.15.142:0/2173038277 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f673c06dd60 0x7f673c070220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6740006010 tx=0x7f674001a040 comp rx=0 tx=0).stop 2024-03-29T04:54:05.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.814+0000 7f675567b700 1 -- 172.21.15.142:0/2173038277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750069020 msgr2=0x7f675019ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:05.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.814+0000 7f675567b700 1 --2- 172.21.15.142:0/2173038277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750069020 0x7f675019ca50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f673800c960 tx=0x7f673800cc70 comp rx=0 tx=0).stop 2024-03-29T04:54:05.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.814+0000 7f675567b700 1 -- 172.21.15.142:0/2173038277 shutdown_connections 2024-03-29T04:54:05.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.814+0000 7f675567b700 1 --2- 172.21.15.142:0/2173038277 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f673c06dd60 0x7f673c070220 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.814+0000 7f675567b700 1 --2- 172.21.15.142:0/2173038277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6750068670 0x7f675019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.814+0000 7f675567b700 1 --2- 172.21.15.142:0/2173038277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750069020 0x7f675019ca50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.814+0000 7f675567b700 1 -- 172.21.15.142:0/2173038277 >> 172.21.15.142:0/2173038277 conn(0x7f67500755a0 msgr2=0x7f67500fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:05.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.814+0000 7f675567b700 1 -- 172.21.15.142:0/2173038277 shutdown_connections 2024-03-29T04:54:05.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.814+0000 7f675567b700 1 -- 172.21.15.142:0/2173038277 wait complete. 2024-03-29T04:54:05.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.917+0000 7fa55a9f4700 1 -- 172.21.15.142:0/3269107001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101770 msgr2=0x7fa5541057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:05.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.917+0000 7fa55a9f4700 1 --2- 172.21.15.142:0/3269107001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101770 0x7fa5541057c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa544009910 tx=0x7fa544009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:05.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.917+0000 7fa55a9f4700 1 -- 172.21.15.142:0/3269107001 shutdown_connections 2024-03-29T04:54:05.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.917+0000 7fa55a9f4700 1 --2- 172.21.15.142:0/3269107001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101770 0x7fa5541057c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.917+0000 7fa55a9f4700 1 --2- 172.21.15.142:0/3269107001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa554100dc0 0x7fa5541011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.917+0000 7fa55a9f4700 1 -- 172.21.15.142:0/3269107001 >> 172.21.15.142:0/3269107001 conn(0x7fa5540fc650 msgr2=0x7fa5540fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:05.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.917+0000 7fa55a9f4700 1 -- 172.21.15.142:0/3269107001 shutdown_connections 2024-03-29T04:54:05.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.918+0000 7fa55a9f4700 1 -- 172.21.15.142:0/3269107001 wait complete. 2024-03-29T04:54:05.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.918+0000 7fa55a9f4700 1 Processor -- start 2024-03-29T04:54:05.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.918+0000 7fa55a9f4700 1 -- start start 2024-03-29T04:54:05.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.918+0000 7fa55a9f4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554100dc0 0x7fa55419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:05.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.918+0000 7fa55a9f4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa554101770 0x7fa55419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:05.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.918+0000 7fa55a9f4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55419d130 con 0x7fa554100dc0 2024-03-29T04:54:05.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.918+0000 7fa55a9f4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5541965e0 con 0x7fa554101770 2024-03-29T04:54:05.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa553fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554100dc0 0x7fa55419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:05.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa553fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554100dc0 0x7fa55419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43606/0 (socket says 172.21.15.142:43606) 2024-03-29T04:54:05.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa553fff700 1 -- 172.21.15.142:0/4009592302 learned_addr learned my addr 172.21.15.142:0/4009592302 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:05.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa5537fe700 1 --2- 172.21.15.142:0/4009592302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa554101770 0x7fa55419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:05.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa553fff700 1 -- 172.21.15.142:0/4009592302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa554101770 msgr2=0x7fa55419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:05.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa553fff700 1 --2- 172.21.15.142:0/4009592302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa554101770 0x7fa55419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:05.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa553fff700 1 -- 172.21.15.142:0/4009592302 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5440095f0 con 0x7fa554100dc0 2024-03-29T04:54:05.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa5537fe700 1 --2- 172.21.15.142:0/4009592302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa554101770 0x7fa55419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:54:05.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa553fff700 1 --2- 172.21.15.142:0/4009592302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554100dc0 0x7fa55419c560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa54800d8d0 tx=0x7fa54800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:05.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa5517fa700 1 -- 172.21.15.142:0/4009592302 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa548009940 con 0x7fa554100dc0 2024-03-29T04:54:05.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa5517fa700 1 -- 172.21.15.142:0/4009592302 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa548010460 con 0x7fa554100dc0 2024-03-29T04:54:05.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa55a9f4700 1 -- 172.21.15.142:0/4009592302 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5541968c0 con 0x7fa554100dc0 2024-03-29T04:54:05.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa5517fa700 1 -- 172.21.15.142:0/4009592302 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa54800f5f0 con 0x7fa554100dc0 2024-03-29T04:54:05.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.919+0000 7fa55a9f4700 1 -- 172.21.15.142:0/4009592302 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa554196e10 con 0x7fa554100dc0 2024-03-29T04:54:05.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.920+0000 7fa55a9f4700 1 -- 172.21.15.142:0/4009592302 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5541091c0 con 0x7fa554100dc0 2024-03-29T04:54:05.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.921+0000 7fa5517fa700 1 -- 172.21.15.142:0/4009592302 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fa5480105d0 con 0x7fa554100dc0 2024-03-29T04:54:05.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.921+0000 7fa5517fa700 1 --2- 172.21.15.142:0/4009592302 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa53c06dd60 0x7fa53c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:05.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.921+0000 7fa5517fa700 1 -- 172.21.15.142:0/4009592302 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fa54808dc20 con 0x7fa554100dc0 2024-03-29T04:54:05.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.921+0000 7fa5537fe700 1 --2- 172.21.15.142:0/4009592302 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa53c06dd60 0x7fa53c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:05.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.922+0000 7fa5537fe700 1 --2- 172.21.15.142:0/4009592302 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa53c06dd60 0x7fa53c070220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa54400b5c0 tx=0x7fa5440053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:05.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:05.926+0000 7fa5517fa700 1 -- 172.21.15.142:0/4009592302 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa54805ac20 con 0x7fa554100dc0 2024-03-29T04:54:06.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.202+0000 7fa55a9f4700 1 -- 172.21.15.142:0/4009592302 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa5541970b0 con 0x7fa554100dc0 2024-03-29T04:54:06.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.202+0000 7fa5517fa700 1 -- 172.21.15.142:0/4009592302 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fa54805a7b0 con 0x7fa554100dc0 2024-03-29T04:54:06.208 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:54:06.208 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:54:06.208 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:54:06.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:54:06.208 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:54:06.208 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:54:06.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:54:06.209 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:54:06.209 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-29T04:54:06.209 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 5 2024-03-29T04:54:06.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:54:06.209 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:54:06.209 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-29T04:54:06.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:54:06.209 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:54:06.209 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-03-29T04:54:06.209 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 9 2024-03-29T04:54:06.209 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:54:06.209 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:54:06.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.205+0000 7fa55a9f4700 1 -- 172.21.15.142:0/4009592302 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa53c06dd60 msgr2=0x7fa53c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:06.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.205+0000 7fa55a9f4700 1 --2- 172.21.15.142:0/4009592302 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa53c06dd60 0x7fa53c070220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa54400b5c0 tx=0x7fa5440053a0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.205+0000 7fa55a9f4700 1 -- 172.21.15.142:0/4009592302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554100dc0 msgr2=0x7fa55419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:06.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.205+0000 7fa55a9f4700 1 --2- 172.21.15.142:0/4009592302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554100dc0 0x7fa55419c560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa54800d8d0 tx=0x7fa54800dc90 comp rx=0 tx=0).stop 2024-03-29T04:54:06.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.205+0000 7fa55a9f4700 1 -- 172.21.15.142:0/4009592302 shutdown_connections 2024-03-29T04:54:06.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.205+0000 7fa55a9f4700 1 --2- 172.21.15.142:0/4009592302 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa53c06dd60 0x7fa53c070220 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.205+0000 7fa55a9f4700 1 --2- 172.21.15.142:0/4009592302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554100dc0 0x7fa55419c560 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.205+0000 7fa55a9f4700 1 --2- 172.21.15.142:0/4009592302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa554101770 0x7fa55419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.205+0000 7fa55a9f4700 1 -- 172.21.15.142:0/4009592302 >> 172.21.15.142:0/4009592302 conn(0x7fa5540fc650 msgr2=0x7fa5540fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:06.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.205+0000 7fa55a9f4700 1 -- 172.21.15.142:0/4009592302 shutdown_connections 2024-03-29T04:54:06.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.205+0000 7fa55a9f4700 1 -- 172.21.15.142:0/4009592302 wait complete. 2024-03-29T04:54:06.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.308+0000 7f28f5243700 1 -- 172.21.15.142:0/2217336444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f00ff530 msgr2=0x7f28f00ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:06.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.308+0000 7f28f5243700 1 --2- 172.21.15.142:0/2217336444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f00ff530 0x7f28f00ff910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009910 tx=0x7f28e0009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:06.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.308+0000 7f28f5243700 1 -- 172.21.15.142:0/2217336444 shutdown_connections 2024-03-29T04:54:06.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.308+0000 7f28f5243700 1 --2- 172.21.15.142:0/2217336444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f00ffee0 0x7f28f0105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.308+0000 7f28f5243700 1 --2- 172.21.15.142:0/2217336444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f00ff530 0x7f28f00ff910 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.308+0000 7f28f5243700 1 -- 172.21.15.142:0/2217336444 >> 172.21.15.142:0/2217336444 conn(0x7f28f0074df0 msgr2=0x7f28f0075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:06.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.308+0000 7f28f5243700 1 -- 172.21.15.142:0/2217336444 shutdown_connections 2024-03-29T04:54:06.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.308+0000 7f28f5243700 1 -- 172.21.15.142:0/2217336444 wait complete. 2024-03-29T04:54:06.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.309+0000 7f28f5243700 1 Processor -- start 2024-03-29T04:54:06.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.309+0000 7f28f5243700 1 -- start start 2024-03-29T04:54:06.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28f5243700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f00ff530 0x7f28f019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:06.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28f5243700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f00ffee0 0x7f28f019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:06.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28f5243700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f019d160 con 0x7f28f00ffee0 2024-03-29T04:54:06.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28f5243700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f0196610 con 0x7f28f00ff530 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28ee59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f00ffee0 0x7f28f019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28ee59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f00ffee0 0x7f28f019cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43622/0 (socket says 172.21.15.142:43622) 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28ee59c700 1 -- 172.21.15.142:0/3446571124 learned_addr learned my addr 172.21.15.142:0/3446571124 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28eed9d700 1 --2- 172.21.15.142:0/3446571124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f00ff530 0x7f28f019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28ee59c700 1 -- 172.21.15.142:0/3446571124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f00ff530 msgr2=0x7f28f019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28ee59c700 1 --2- 172.21.15.142:0/3446571124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f00ff530 0x7f28f019c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28ee59c700 1 -- 172.21.15.142:0/3446571124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e00095f0 con 0x7f28f00ffee0 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28ee59c700 1 --2- 172.21.15.142:0/3446571124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f00ffee0 0x7f28f019cad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28e400d900 tx=0x7f28e400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28d7fff700 1 -- 172.21.15.142:0/3446571124 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28e4009900 con 0x7f28f00ffee0 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28f5243700 1 -- 172.21.15.142:0/3446571124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f01968f0 con 0x7f28f00ffee0 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28d7fff700 1 -- 172.21.15.142:0/3446571124 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28e4010460 con 0x7f28f00ffee0 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28d7fff700 1 -- 172.21.15.142:0/3446571124 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28e400f5f0 con 0x7f28f00ffee0 2024-03-29T04:54:06.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.310+0000 7f28f5243700 1 -- 172.21.15.142:0/3446571124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f0196e40 con 0x7f28f00ffee0 2024-03-29T04:54:06.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.311+0000 7f28f5243700 1 -- 172.21.15.142:0/3446571124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f01091e0 con 0x7f28f00ffee0 2024-03-29T04:54:06.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.312+0000 7f28d7fff700 1 -- 172.21.15.142:0/3446571124 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f28e40105d0 con 0x7f28f00ffee0 2024-03-29T04:54:06.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.312+0000 7f28d7fff700 1 --2- 172.21.15.142:0/3446571124 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f28d806dd60 0x7f28d8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:06.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.313+0000 7f28eed9d700 1 --2- 172.21.15.142:0/3446571124 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f28d806dd60 0x7f28d8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:06.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.313+0000 7f28d7fff700 1 -- 172.21.15.142:0/3446571124 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f28e408d7c0 con 0x7f28f00ffee0 2024-03-29T04:54:06.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.313+0000 7f28eed9d700 1 --2- 172.21.15.142:0/3446571124 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f28d806dd60 0x7f28d8070220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f28e000b5c0 tx=0x7f28e001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:06.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.317+0000 7f28d7fff700 1 -- 172.21.15.142:0/3446571124 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f28e405a7c0 con 0x7f28f00ffee0 2024-03-29T04:54:06.559 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.553+0000 7f28f5243700 1 -- 172.21.15.142:0/3446571124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f28f004ea90 con 0x7f28f00ffee0 2024-03-29T04:54:06.559 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.554+0000 7f28d7fff700 1 -- 172.21.15.142:0/3446571124 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1906 (secure 0 0 0) 0x7f28e405a350 con 0x7f28f00ffee0 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:e21 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 21 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:49:01.288637+0000 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:54:06.560 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 47 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:up {0=14494,1=24275} 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T04:54:06.561 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:54:06.562 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:54:06.562 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:54:06.562 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:54:06.562 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:54:06.562 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:54:06.562 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:54:06.562 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:54:06.562 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:54:06.562 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{0:14494} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.142:6828/1941073852,v1:172.21.15.142:6829/1941073852] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:54:06.562 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{0:14506} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/1415100534,v1:172.21.15.142:6827/1415100534] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:54:06.563 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{1:24275} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.152:6824/832699894,v1:172.21.15.152:6825/832699894] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:54:06.563 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{1:24287} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.152:6826/3411697908,v1:172.21.15.152:6827/3411697908] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:54:06.563 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:54:06.563 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:54:06.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.556+0000 7f28f5243700 1 -- 172.21.15.142:0/3446571124 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f28d806dd60 msgr2=0x7f28d8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:06.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.556+0000 7f28f5243700 1 --2- 172.21.15.142:0/3446571124 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f28d806dd60 0x7f28d8070220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f28e000b5c0 tx=0x7f28e001a040 comp rx=0 tx=0).stop 2024-03-29T04:54:06.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.556+0000 7f28f5243700 1 -- 172.21.15.142:0/3446571124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f00ffee0 msgr2=0x7f28f019cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:06.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.556+0000 7f28f5243700 1 --2- 172.21.15.142:0/3446571124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f00ffee0 0x7f28f019cad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28e400d900 tx=0x7f28e400dc10 comp rx=0 tx=0).stop 2024-03-29T04:54:06.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.556+0000 7f28f5243700 1 -- 172.21.15.142:0/3446571124 shutdown_connections 2024-03-29T04:54:06.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.556+0000 7f28f5243700 1 --2- 172.21.15.142:0/3446571124 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f28d806dd60 0x7f28d8070220 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.556+0000 7f28f5243700 1 --2- 172.21.15.142:0/3446571124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f00ff530 0x7f28f019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.556+0000 7f28f5243700 1 --2- 172.21.15.142:0/3446571124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f00ffee0 0x7f28f019cad0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.556+0000 7f28f5243700 1 -- 172.21.15.142:0/3446571124 >> 172.21.15.142:0/3446571124 conn(0x7f28f0074df0 msgr2=0x7f28f00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:06.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.556+0000 7f28f5243700 1 -- 172.21.15.142:0/3446571124 shutdown_connections 2024-03-29T04:54:06.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.556+0000 7f28f5243700 1 -- 172.21.15.142:0/3446571124 wait complete. 2024-03-29T04:54:06.564 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 21 2024-03-29T04:54:06.669 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.662+0000 7f8e19cc3700 1 -- 172.21.15.142:0/503707355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e14101770 msgr2=0x7f8e141057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:06.669 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.662+0000 7f8e19cc3700 1 --2- 172.21.15.142:0/503707355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e14101770 0x7f8e141057c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009910 tx=0x7f8e08009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:06.669 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.662+0000 7f8e19cc3700 1 -- 172.21.15.142:0/503707355 shutdown_connections 2024-03-29T04:54:06.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.662+0000 7f8e19cc3700 1 --2- 172.21.15.142:0/503707355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e14101770 0x7f8e141057c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.662+0000 7f8e19cc3700 1 --2- 172.21.15.142:0/503707355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e14100dc0 0x7f8e141011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.662+0000 7f8e19cc3700 1 -- 172.21.15.142:0/503707355 >> 172.21.15.142:0/503707355 conn(0x7f8e140fc650 msgr2=0x7f8e140fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:06.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.662+0000 7f8e19cc3700 1 -- 172.21.15.142:0/503707355 shutdown_connections 2024-03-29T04:54:06.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.662+0000 7f8e19cc3700 1 -- 172.21.15.142:0/503707355 wait complete. 2024-03-29T04:54:06.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.662+0000 7f8e19cc3700 1 Processor -- start 2024-03-29T04:54:06.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e19cc3700 1 -- start start 2024-03-29T04:54:06.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e19cc3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e14100dc0 0x7f8e14073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:06.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e19cc3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e14101770 0x7f8e140735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:06.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e19cc3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e14073b10 con 0x7f8e14100dc0 2024-03-29T04:54:06.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e19cc3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e14073c50 con 0x7f8e14101770 2024-03-29T04:54:06.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e137fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e14100dc0 0x7f8e14073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e137fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e14100dc0 0x7f8e14073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43648/0 (socket says 172.21.15.142:43648) 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e137fe700 1 -- 172.21.15.142:0/695150371 learned_addr learned my addr 172.21.15.142:0/695150371 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e12ffd700 1 --2- 172.21.15.142:0/695150371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e14101770 0x7f8e140735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e137fe700 1 -- 172.21.15.142:0/695150371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e14101770 msgr2=0x7f8e140735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e137fe700 1 --2- 172.21.15.142:0/695150371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e14101770 0x7f8e140735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e137fe700 1 -- 172.21.15.142:0/695150371 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e080095f0 con 0x7f8e14100dc0 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e12ffd700 1 --2- 172.21.15.142:0/695150371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e14101770 0x7f8e140735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e137fe700 1 --2- 172.21.15.142:0/695150371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e14100dc0 0x7f8e14073090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400c8a0 tx=0x7f8e0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e10ff9700 1 -- 172.21.15.142:0/695150371 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e0400cea0 con 0x7f8e14100dc0 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e19cc3700 1 -- 172.21.15.142:0/695150371 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e141a2960 con 0x7f8e14100dc0 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e10ff9700 1 -- 172.21.15.142:0/695150371 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e04004830 con 0x7f8e14100dc0 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.663+0000 7f8e10ff9700 1 -- 172.21.15.142:0/695150371 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e040055b0 con 0x7f8e14100dc0 2024-03-29T04:54:06.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.664+0000 7f8e19cc3700 1 -- 172.21.15.142:0/695150371 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e141a2d70 con 0x7f8e14100dc0 2024-03-29T04:54:06.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.665+0000 7f8e19cc3700 1 -- 172.21.15.142:0/695150371 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e141091c0 con 0x7f8e14100dc0 2024-03-29T04:54:06.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.665+0000 7f8e10ff9700 1 -- 172.21.15.142:0/695150371 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f8e0400f450 con 0x7f8e14100dc0 2024-03-29T04:54:06.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.665+0000 7f8e10ff9700 1 --2- 172.21.15.142:0/695150371 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8dfc06dd60 0x7f8dfc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:06.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.665+0000 7f8e10ff9700 1 -- 172.21.15.142:0/695150371 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f8e0408d8b0 con 0x7f8e14100dc0 2024-03-29T04:54:06.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.665+0000 7f8e12ffd700 1 --2- 172.21.15.142:0/695150371 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8dfc06dd60 0x7f8dfc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:06.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.666+0000 7f8e12ffd700 1 --2- 172.21.15.142:0/695150371 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8dfc06dd60 0x7f8dfc070220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800b5c0 tx=0x7f8e08005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:06.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.670+0000 7f8e10ff9700 1 -- 172.21.15.142:0/695150371 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8e04056df0 con 0x7f8e14100dc0 2024-03-29T04:54:06.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:06 smithi142 ceph-mon[112475]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:06.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:06 smithi142 ceph-mon[112475]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:06.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:06 smithi142 ceph-mon[112475]: pgmap v163: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-03-29T04:54:06.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:06 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/4009592302' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:06.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:06 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3446571124' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:54:06.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.886+0000 7f8e19cc3700 1 -- 172.21.15.142:0/695150371 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e1402cdd0 con 0x7f8dfc06dd60 2024-03-29T04:54:06.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.887+0000 7f8e10ff9700 1 -- 172.21.15.142:0/695150371 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f8e1402cdd0 con 0x7f8dfc06dd60 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout: "mon" 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "11/23 daemons upgraded", 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading osd daemons", 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:54:06.893 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:54:06.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.889+0000 7f8e19cc3700 1 -- 172.21.15.142:0/695150371 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8dfc06dd60 msgr2=0x7f8dfc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:06.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.889+0000 7f8e19cc3700 1 --2- 172.21.15.142:0/695150371 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8dfc06dd60 0x7f8dfc070220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800b5c0 tx=0x7f8e08005480 comp rx=0 tx=0).stop 2024-03-29T04:54:06.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.889+0000 7f8e19cc3700 1 -- 172.21.15.142:0/695150371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e14100dc0 msgr2=0x7f8e14073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:06.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.889+0000 7f8e19cc3700 1 --2- 172.21.15.142:0/695150371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e14100dc0 0x7f8e14073090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400c8a0 tx=0x7f8e0400cc60 comp rx=0 tx=0).stop 2024-03-29T04:54:06.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.889+0000 7f8e19cc3700 1 -- 172.21.15.142:0/695150371 shutdown_connections 2024-03-29T04:54:06.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.889+0000 7f8e19cc3700 1 --2- 172.21.15.142:0/695150371 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8dfc06dd60 0x7f8dfc070220 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.889+0000 7f8e19cc3700 1 --2- 172.21.15.142:0/695150371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e14100dc0 0x7f8e14073090 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.889+0000 7f8e19cc3700 1 --2- 172.21.15.142:0/695150371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e14101770 0x7f8e140735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:06.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.889+0000 7f8e19cc3700 1 -- 172.21.15.142:0/695150371 >> 172.21.15.142:0/695150371 conn(0x7f8e140fc650 msgr2=0x7f8e140fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:06.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.890+0000 7f8e19cc3700 1 -- 172.21.15.142:0/695150371 shutdown_connections 2024-03-29T04:54:06.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.890+0000 7f8e19cc3700 1 -- 172.21.15.142:0/695150371 wait complete. 2024-03-29T04:54:07.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.993+0000 7f9d817f2700 1 -- 172.21.15.142:0/3258935563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c100e10 msgr2=0x7f9d7c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:07.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.993+0000 7f9d817f2700 1 --2- 172.21.15.142:0/3258935563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c100e10 0x7f9d7c1011f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009910 tx=0x7f9d6c009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:07.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.993+0000 7f9d817f2700 1 -- 172.21.15.142:0/3258935563 shutdown_connections 2024-03-29T04:54:07.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.993+0000 7f9d817f2700 1 --2- 172.21.15.142:0/3258935563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7c1017c0 0x7f9d7c105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:07.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.993+0000 7f9d817f2700 1 --2- 172.21.15.142:0/3258935563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c100e10 0x7f9d7c1011f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:07.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.993+0000 7f9d817f2700 1 -- 172.21.15.142:0/3258935563 >> 172.21.15.142:0/3258935563 conn(0x7f9d7c0fc6c0 msgr2=0x7f9d7c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:07.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.993+0000 7f9d817f2700 1 -- 172.21.15.142:0/3258935563 shutdown_connections 2024-03-29T04:54:07.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.993+0000 7f9d817f2700 1 -- 172.21.15.142:0/3258935563 wait complete. 2024-03-29T04:54:07.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.994+0000 7f9d817f2700 1 Processor -- start 2024-03-29T04:54:07.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.994+0000 7f9d817f2700 1 -- start start 2024-03-29T04:54:07.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.994+0000 7f9d817f2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c100e10 0x7f9d7c19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:07.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.994+0000 7f9d817f2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7c1017c0 0x7f9d7c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:07.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.994+0000 7f9d817f2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c19d190 con 0x7f9d7c100e10 2024-03-29T04:54:07.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.994+0000 7f9d817f2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c196640 con 0x7f9d7c1017c0 2024-03-29T04:54:07.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.994+0000 7f9d7affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c100e10 0x7f9d7c19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:07.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.994+0000 7f9d7affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c100e10 0x7f9d7c19c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43662/0 (socket says 172.21.15.142:43662) 2024-03-29T04:54:07.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.994+0000 7f9d7affd700 1 -- 172.21.15.142:0/3874553889 learned_addr learned my addr 172.21.15.142:0/3874553889 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:07.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.995+0000 7f9d7a7fc700 1 --2- 172.21.15.142:0/3874553889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7c1017c0 0x7f9d7c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:07.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.995+0000 7f9d7affd700 1 -- 172.21.15.142:0/3874553889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7c1017c0 msgr2=0x7f9d7c19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:07.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.995+0000 7f9d7affd700 1 --2- 172.21.15.142:0/3874553889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7c1017c0 0x7f9d7c19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:07.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.995+0000 7f9d7affd700 1 -- 172.21.15.142:0/3874553889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0095f0 con 0x7f9d7c100e10 2024-03-29T04:54:07.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.995+0000 7f9d7affd700 1 --2- 172.21.15.142:0/3874553889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c100e10 0x7f9d7c19c5c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c006010 tx=0x7f9d6c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:07.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.995+0000 7f9d63fff700 1 -- 172.21.15.142:0/3874553889 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d6c01d070 con 0x7f9d7c100e10 2024-03-29T04:54:07.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.995+0000 7f9d63fff700 1 -- 172.21.15.142:0/3874553889 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d6c00f460 con 0x7f9d7c100e10 2024-03-29T04:54:07.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.995+0000 7f9d817f2700 1 -- 172.21.15.142:0/3874553889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7c1968c0 con 0x7f9d7c100e10 2024-03-29T04:54:07.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.995+0000 7f9d63fff700 1 -- 172.21.15.142:0/3874553889 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d6c017630 con 0x7f9d7c100e10 2024-03-29T04:54:07.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.995+0000 7f9d817f2700 1 -- 172.21.15.142:0/3874553889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7c196db0 con 0x7f9d7c100e10 2024-03-29T04:54:07.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.996+0000 7f9d817f2700 1 -- 172.21.15.142:0/3874553889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7c1091b0 con 0x7f9d7c100e10 2024-03-29T04:54:07.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.996+0000 7f9d63fff700 1 -- 172.21.15.142:0/3874553889 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f9d6c00f5d0 con 0x7f9d7c100e10 2024-03-29T04:54:07.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.997+0000 7f9d63fff700 1 --2- 172.21.15.142:0/3874553889 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d6406dd60 0x7f9d64070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:07.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.997+0000 7f9d63fff700 1 -- 172.21.15.142:0/3874553889 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f9d6c08f010 con 0x7f9d7c100e10 2024-03-29T04:54:07.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.997+0000 7f9d7a7fc700 1 --2- 172.21.15.142:0/3874553889 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d6406dd60 0x7f9d64070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:07.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:06.997+0000 7f9d7a7fc700 1 --2- 172.21.15.142:0/3874553889 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d6406dd60 0x7f9d64070220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9d70005950 tx=0x7f9d700058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:07.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.002+0000 7f9d63fff700 1 -- 172.21.15.142:0/3874553889 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9d6c05c090 con 0x7f9d7c100e10 2024-03-29T04:54:07.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:06 smithi152 ceph-mon[88186]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:07.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:06 smithi152 ceph-mon[88186]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:07.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:06 smithi152 ceph-mon[88186]: pgmap v163: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-03-29T04:54:07.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:06 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/4009592302' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:07.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:06 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3446571124' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:54:07.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.274+0000 7f9d817f2700 1 -- 172.21.15.142:0/3874553889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9d7c04ea90 con 0x7f9d7c100e10 2024-03-29T04:54:07.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.274+0000 7f9d63fff700 1 -- 172.21.15.142:0/3874553889 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f9d6c05bc20 con 0x7f9d7c100e10 2024-03-29T04:54:07.280 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_OK 2024-03-29T04:54:07.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.276+0000 7f9d817f2700 1 -- 172.21.15.142:0/3874553889 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d6406dd60 msgr2=0x7f9d64070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:07.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.276+0000 7f9d817f2700 1 --2- 172.21.15.142:0/3874553889 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d6406dd60 0x7f9d64070220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9d70005950 tx=0x7f9d700058e0 comp rx=0 tx=0).stop 2024-03-29T04:54:07.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.276+0000 7f9d817f2700 1 -- 172.21.15.142:0/3874553889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c100e10 msgr2=0x7f9d7c19c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:07.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.276+0000 7f9d817f2700 1 --2- 172.21.15.142:0/3874553889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c100e10 0x7f9d7c19c5c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c006010 tx=0x7f9d6c00be20 comp rx=0 tx=0).stop 2024-03-29T04:54:07.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.277+0000 7f9d817f2700 1 -- 172.21.15.142:0/3874553889 shutdown_connections 2024-03-29T04:54:07.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.277+0000 7f9d817f2700 1 --2- 172.21.15.142:0/3874553889 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d6406dd60 0x7f9d64070220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:07.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.277+0000 7f9d817f2700 1 --2- 172.21.15.142:0/3874553889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c100e10 0x7f9d7c19c5c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:07.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.277+0000 7f9d817f2700 1 --2- 172.21.15.142:0/3874553889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d7c1017c0 0x7f9d7c19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:07.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.277+0000 7f9d817f2700 1 -- 172.21.15.142:0/3874553889 >> 172.21.15.142:0/3874553889 conn(0x7f9d7c0fc6c0 msgr2=0x7f9d7c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:07.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.277+0000 7f9d817f2700 1 -- 172.21.15.142:0/3874553889 shutdown_connections 2024-03-29T04:54:07.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:07.277+0000 7f9d817f2700 1 -- 172.21.15.142:0/3874553889 wait complete. 2024-03-29T04:54:07.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:07 smithi142 ceph-mon[112475]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:07.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:07 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3874553889' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:54:08.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:07 smithi152 ceph-mon[88186]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:08.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:07 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3874553889' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:54:08.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:08 smithi142 ceph-mon[112475]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:08.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:08 smithi142 ceph-mon[112475]: pgmap v164: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-03-29T04:54:09.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:08 smithi152 ceph-mon[88186]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:09.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:08 smithi152 ceph-mon[88186]: pgmap v164: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-03-29T04:54:11.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:10 smithi152 ceph-mon[88186]: pgmap v165: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-03-29T04:54:11.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:11.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:10 smithi142 ceph-mon[112475]: pgmap v165: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-03-29T04:54:11.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:13.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:12 smithi152 ceph-mon[88186]: pgmap v166: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:54:13.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:12 smithi142 ceph-mon[112475]: pgmap v166: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T04:54:14.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:13 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T04:54:14.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:13 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T04:54:14.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:14 smithi142 ceph-mon[112475]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T04:54:14.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:14 smithi142 ceph-mon[112475]: Upgrade: osd.5 is safe to restart 2024-03-29T04:54:14.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:14 smithi142 ceph-mon[112475]: pgmap v167: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:54:14.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:14.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-29T04:54:14.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:54:14.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:14.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:54:14.898 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:14 smithi152 ceph-mon[88186]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T04:54:14.899 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:14 smithi152 ceph-mon[88186]: Upgrade: osd.5 is safe to restart 2024-03-29T04:54:14.899 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:14 smithi152 ceph-mon[88186]: pgmap v167: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:54:14.899 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:14.899 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-29T04:54:14.899 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:54:14.899 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:14.899 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:54:15.758 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:15 smithi152 ceph-mon[88186]: Upgrade: Updating osd.5 2024-03-29T04:54:15.758 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:15 smithi152 ceph-mon[88186]: Deploying daemon osd.5 on smithi152 2024-03-29T04:54:16.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:15 smithi142 ceph-mon[112475]: Upgrade: Updating osd.5 2024-03-29T04:54:16.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:15 smithi142 ceph-mon[112475]: Deploying daemon osd.5 on smithi152 2024-03-29T04:54:17.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:16 smithi152 ceph-mon[88186]: pgmap v168: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:54:17.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:16 smithi142 ceph-mon[112475]: pgmap v168: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T04:54:17.863 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:17 smithi152 systemd[1]: Stopping Ceph osd.5 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:54:18.296 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:17 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[57008]: 2024-03-29T04:54:17.856+0000 7ff717b1b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T04:54:18.296 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:17 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[57008]: 2024-03-29T04:54:17.856+0000 7ff717b1b700 -1 osd.5 88 *** Got signal Terminated *** 2024-03-29T04:54:18.296 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:17 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[57008]: 2024-03-29T04:54:17.856+0000 7ff717b1b700 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T04:54:19.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:18 smithi152 ceph-mon[88186]: pgmap v169: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:54:19.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:18 smithi152 ceph-mon[88186]: osd.5 marked itself down and dead 2024-03-29T04:54:19.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:18 smithi142 ceph-mon[112475]: pgmap v169: 65 pgs: 65 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T04:54:19.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:18 smithi142 ceph-mon[112475]: osd.5 marked itself down and dead 2024-03-29T04:54:19.498 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:19 smithi152 podman[107742]: 2024-03-29 04:54:19.15113658 +0000 UTC m=+1.394187988 container died 3e9c96d907fcaf1129504584c246f090e71643da76724d3947afb765b79e3cb0 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=-18.2.1) 2024-03-29T04:54:19.796 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:19 smithi152 podman[107742]: 2024-03-29 04:54:19.592150305 +0000 UTC m=+1.835201720 container cleanup 3e9c96d907fcaf1129504584c246f090e71643da76724d3947afb765b79e3cb0 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, RELEASE=HEAD) 2024-03-29T04:54:19.796 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:19 smithi152 bash[107742]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5 2024-03-29T04:54:19.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:19 smithi152 ceph-mon[88186]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T04:54:19.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:19 smithi152 ceph-mon[88186]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-29T04:54:19.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:19 smithi152 ceph-mon[88186]: osdmap e89: 6 total, 5 up, 6 in 2024-03-29T04:54:19.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:19 smithi152 ceph-mon[88186]: osdmap e90: 6 total, 5 up, 6 in 2024-03-29T04:54:19.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:19 smithi142 ceph-mon[112475]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T04:54:19.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:19 smithi142 ceph-mon[112475]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-29T04:54:19.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:19 smithi142 ceph-mon[112475]: osdmap e89: 6 total, 5 up, 6 in 2024-03-29T04:54:19.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:19 smithi142 ceph-mon[112475]: osdmap e90: 6 total, 5 up, 6 in 2024-03-29T04:54:20.211 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:19 smithi152 podman[107770]: 2024-03-29 04:54:19.893636106 +0000 UTC m=+0.745863111 container remove 3e9c96d907fcaf1129504584c246f090e71643da76724d3947afb765b79e3cb0 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.29.1, RELEASE=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1) 2024-03-29T04:54:20.464 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:20 smithi152 podman[107864]: 2024-03-29 04:54:20.109441709 +0000 UTC m=+0.017530395 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:54:20.464 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:20 smithi152 podman[107864]: 2024-03-29 04:54:20.228054303 +0000 UTC m=+0.136142985 container create 023207cb4e8827ddfa4b5a2f738eea0e2cdc1b3d4e71e5320206ff0024dbe100 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-29T04:54:20.464 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:20 smithi152 podman[107864]: 2024-03-29 04:54:20.455771628 +0000 UTC m=+0.363860316 container init 023207cb4e8827ddfa4b5a2f738eea0e2cdc1b3d4e71e5320206ff0024dbe100 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-deactivate, RELEASE=squid-8da2362, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-29T04:54:20.464 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:20 smithi152 podman[107864]: 2024-03-29 04:54:20.459008171 +0000 UTC m=+0.367096850 container start 023207cb4e8827ddfa4b5a2f738eea0e2cdc1b3d4e71e5320206ff0024dbe100 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-8da2362, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-29T04:54:20.760 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:20 smithi152 ceph-mon[88186]: pgmap v171: 65 pgs: 8 peering, 11 stale+active+clean, 46 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-03-29T04:54:20.760 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:20 smithi152 ceph-mon[88186]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-29T04:54:20.760 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:20 smithi152 podman[107864]: 2024-03-29 04:54:20.564276975 +0000 UTC m=+0.472365658 container attach 023207cb4e8827ddfa4b5a2f738eea0e2cdc1b3d4e71e5320206ff0024dbe100 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8da2362, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-29T04:54:21.045 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:20 smithi152 podman[107864]: 2024-03-29 04:54:20.7559466 +0000 UTC m=+0.664035286 container died 023207cb4e8827ddfa4b5a2f738eea0e2cdc1b3d4e71e5320206ff0024dbe100 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6) 2024-03-29T04:54:21.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:20 smithi142 ceph-mon[112475]: pgmap v171: 65 pgs: 8 peering, 11 stale+active+clean, 46 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-03-29T04:54:21.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:20 smithi142 ceph-mon[112475]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-29T04:54:21.677 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:21 smithi152 podman[107890]: 2024-03-29 04:54:21.385174537 +0000 UTC m=+0.624194742 container remove 023207cb4e8827ddfa4b5a2f738eea0e2cdc1b3d4e71e5320206ff0024dbe100 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8da2362) 2024-03-29T04:54:21.677 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:21 smithi152 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-29T04:54:21.677 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:21 smithi152 systemd[1]: Stopped Ceph osd.5 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:54:21.677 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:21 smithi152 systemd[1]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.5.service: Consumed 1min 9.133s CPU time. 2024-03-29T04:54:21.678 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:21 smithi152 systemd[1]: Starting Ceph osd.5 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T04:54:21.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:21 smithi142 ceph-mon[112475]: pgmap v173: 65 pgs: 6 active+undersized, 6 active+undersized+degraded, 8 peering, 4 stale+active+clean, 41 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 14/333 objects degraded (4.204%) 2024-03-29T04:54:21.981 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:21 smithi152 ceph-mon[88186]: pgmap v173: 65 pgs: 6 active+undersized, 6 active+undersized+degraded, 8 peering, 4 stale+active+clean, 41 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 14/333 objects degraded (4.204%) 2024-03-29T04:54:21.982 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:21 smithi152 podman[107995]: 2024-03-29 04:54:21.725776775 +0000 UTC m=+0.017281327 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:54:21.982 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:21 smithi152 podman[107995]: 2024-03-29 04:54:21.845321713 +0000 UTC m=+0.136826331 container create 78f5d15f2ad3030b0cf4bd2542f5d61c8a4b509b974ff917aa5126b1a00f4cc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.schema-version=1.0, ceph=True) 2024-03-29T04:54:22.296 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 podman[107995]: 2024-03-29 04:54:22.107028142 +0000 UTC m=+0.398532695 container init 78f5d15f2ad3030b0cf4bd2542f5d61c8a4b509b974ff917aa5126b1a00f4cc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-8da2362, org.label-schema.build-date=20240326, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, io.buildah.version=1.33.5) 2024-03-29T04:54:22.296 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 podman[107995]: 2024-03-29 04:54:22.111531043 +0000 UTC m=+0.403035604 container start 78f5d15f2ad3030b0cf4bd2542f5d61c8a4b509b974ff917aa5126b1a00f4cc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-29T04:54:22.296 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 podman[107995]: 2024-03-29 04:54:22.214909271 +0000 UTC m=+0.506413836 container attach 78f5d15f2ad3030b0cf4bd2542f5d61c8a4b509b974ff917aa5126b1a00f4cc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-29T04:54:22.613 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate[108009]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-29T04:54:22.613 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate[108009]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-29T04:54:22.613 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 bash[107995]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-29T04:54:22.613 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 bash[107995]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-29T04:54:22.613 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 bash[107995]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-29T04:54:22.613 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 bash[107995]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-29T04:54:22.613 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 bash[107995]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-29T04:54:22.613 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 bash[107995]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-29T04:54:22.613 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 bash[107995]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-29T04:54:22.613 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate[108009]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-29T04:54:22.613 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate[108009]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-29T04:54:22.614 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate[108009]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-29T04:54:22.614 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate[108009]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-29T04:54:22.614 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate[108009]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-29T04:54:22.614 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:22 smithi152 podman[107995]: 2024-03-29 04:54:22.586224697 +0000 UTC m=+0.877729242 container died 78f5d15f2ad3030b0cf4bd2542f5d61c8a4b509b974ff917aa5126b1a00f4cc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_CLEAN=True, RELEASE=squid-8da2362, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:54:22.882 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:22 smithi152 ceph-mon[88186]: Health check failed: Degraded data redundancy: 14/333 objects degraded (4.204%), 6 pgs degraded (PG_DEGRADED) 2024-03-29T04:54:23.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:22 smithi142 ceph-mon[112475]: Health check failed: Degraded data redundancy: 14/333 objects degraded (4.204%), 6 pgs degraded (PG_DEGRADED) 2024-03-29T04:54:23.527 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:23 smithi152 podman[108145]: 2024-03-29 04:54:23.169325675 +0000 UTC m=+0.577844136 container remove 78f5d15f2ad3030b0cf4bd2542f5d61c8a4b509b974ff917aa5126b1a00f4cc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-activate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-29T04:54:23.527 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:23 smithi152 podman[108207]: 2024-03-29 04:54:23.284613894 +0000 UTC m=+0.018019454 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T04:54:23.527 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:23 smithi152 podman[108207]: 2024-03-29 04:54:23.40355606 +0000 UTC m=+0.136961601 container create dfaff5665341bf2d23dc830135c62ee4d3b2c8b6c5f81cc50d1e1db8d1628fac (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8da2362, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-29T04:54:23.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:23 smithi152 ceph-mon[88186]: pgmap v174: 65 pgs: 11 active+undersized, 9 active+undersized+degraded, 8 peering, 37 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 27/333 objects degraded (8.108%) 2024-03-29T04:54:23.796 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:23 smithi152 podman[108207]: 2024-03-29 04:54:23.639560915 +0000 UTC m=+0.372966456 container init dfaff5665341bf2d23dc830135c62ee4d3b2c8b6c5f81cc50d1e1db8d1628fac (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8da2362, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-29T04:54:23.796 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:23 smithi152 podman[108207]: 2024-03-29 04:54:23.642618923 +0000 UTC m=+0.376024465 container start dfaff5665341bf2d23dc830135c62ee4d3b2c8b6c5f81cc50d1e1db8d1628fac (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5, ceph=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T04:54:23.796 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:23 smithi152 ceph-osd[108226]: -- 172.21.15.152:0/3283137509 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5611d033b680 con 0x5611d0304c00 2024-03-29T04:54:23.796 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:23 smithi152 bash[108207]: dfaff5665341bf2d23dc830135c62ee4d3b2c8b6c5f81cc50d1e1db8d1628fac 2024-03-29T04:54:23.796 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:23 smithi152 systemd[1]: Started Ceph osd.5 for 9f96302c-ed85-11ee-95d0-87774f69a715. 2024-03-29T04:54:24.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:23 smithi142 ceph-mon[112475]: pgmap v174: 65 pgs: 11 active+undersized, 9 active+undersized+degraded, 8 peering, 37 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 27/333 objects degraded (8.108%) 2024-03-29T04:54:24.546 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:24 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T04:54:24.194+0000 7f8c94e987c0 -1 Falling back to public interface 2024-03-29T04:54:25.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:25.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:25.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:24 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:54:25.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:25.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:25.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:24 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:54:26.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:25 smithi152 ceph-mon[88186]: pgmap v175: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-03-29T04:54:26.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:25 smithi142 ceph-mon[112475]: pgmap v175: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-03-29T04:54:27.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:26 smithi142 ceph-mon[112475]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-03-29T04:54:27.295 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:26 smithi152 ceph-mon[88186]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-03-29T04:54:28.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:28 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:28.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:28 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:28.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:28 smithi152 ceph-mon[88186]: pgmap v176: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-03-29T04:54:28.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:28 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:28.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:28 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:28.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:28 smithi142 ceph-mon[112475]: pgmap v176: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 270 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-03-29T04:54:28.928 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:28 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T04:54:28.670+0000 7f8c94e987c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-29T04:54:29.296 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:29 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T04:54:29.114+0000 7f8c94e987c0 -1 osd.5 88 log_to_monitors true 2024-03-29T04:54:29.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:29.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:29.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:29 smithi152 ceph-mon[88186]: from='osd.5 [v2:172.21.15.152:6816/3741970037,v1:172.21.15.152:6817/3741970037]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T04:54:29.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:29 smithi152 ceph-mon[88186]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T04:54:29.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:29.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:29.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:29 smithi142 ceph-mon[112475]: from='osd.5 [v2:172.21.15.152:6816/3741970037,v1:172.21.15.152:6817/3741970037]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T04:54:29.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:29 smithi142 ceph-mon[112475]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T04:54:30.059 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 04:54:30 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T04:54:30.028+0000 7f8c822e8700 -1 osd.5 88 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:54:30.312 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:30 smithi152 ceph-mon[88186]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-29T04:54:30.313 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:30 smithi152 ceph-mon[88186]: osdmap e91: 6 total, 5 up, 6 in 2024-03-29T04:54:30.313 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:30 smithi152 ceph-mon[88186]: from='osd.5 [v2:172.21.15.152:6816/3741970037,v1:172.21.15.152:6817/3741970037]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:54:30.313 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:30 smithi152 ceph-mon[88186]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:54:30.313 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:30 smithi152 ceph-mon[88186]: pgmap v178: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-03-29T04:54:30.313 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:30.313 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:54:30.313 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:30 smithi152 ceph-mon[88186]: from='osd.5 ' entity='osd.5' 2024-03-29T04:54:30.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:30 smithi142 ceph-mon[112475]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-29T04:54:30.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:30 smithi142 ceph-mon[112475]: osdmap e91: 6 total, 5 up, 6 in 2024-03-29T04:54:30.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:30 smithi142 ceph-mon[112475]: from='osd.5 [v2:172.21.15.152:6816/3741970037,v1:172.21.15.152:6817/3741970037]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:54:30.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:30 smithi142 ceph-mon[112475]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-29T04:54:30.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:30 smithi142 ceph-mon[112475]: pgmap v178: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-03-29T04:54:30.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:30.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:54:30.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:30 smithi142 ceph-mon[112475]: from='osd.5 ' entity='osd.5' 2024-03-29T04:54:31.708 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:31 smithi152 ceph-mon[88186]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T04:54:31.709 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:31 smithi152 ceph-mon[88186]: osd.5 [v2:172.21.15.152:6816/3741970037,v1:172.21.15.152:6817/3741970037] boot 2024-03-29T04:54:31.709 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:31 smithi152 ceph-mon[88186]: osdmap e92: 6 total, 6 up, 6 in 2024-03-29T04:54:31.709 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:31 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:54:31.709 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:31 smithi152 ceph-mon[88186]: Health check update: Degraded data redundancy: 51/333 objects degraded (15.315%), 14 pgs degraded (PG_DEGRADED) 2024-03-29T04:54:31.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:31 smithi142 ceph-mon[112475]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T04:54:31.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:31 smithi142 ceph-mon[112475]: osd.5 [v2:172.21.15.152:6816/3741970037,v1:172.21.15.152:6817/3741970037] boot 2024-03-29T04:54:31.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:31 smithi142 ceph-mon[112475]: osdmap e92: 6 total, 6 up, 6 in 2024-03-29T04:54:31.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:31 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T04:54:31.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:31 smithi142 ceph-mon[112475]: Health check update: Degraded data redundancy: 51/333 objects degraded (15.315%), 14 pgs degraded (PG_DEGRADED) 2024-03-29T04:54:32.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: pgmap v180: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 51/333 objects degraded (15.315%) 2024-03-29T04:54:32.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:32.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-29T04:54:32.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: osdmap e93: 6 total, 6 up, 6 in 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:32 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-29T04:54:32.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: pgmap v180: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 51/333 objects degraded (15.315%) 2024-03-29T04:54:32.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-29T04:54:32.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: osdmap e93: 6 total, 6 up, 6 in 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:32 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-29T04:54:33.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[112471]: 2024-03-29T04:54:33.201+0000 7f6a4acaf700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:54:33.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: Upgrade: Setting container_image for all osd 2024-03-29T04:54:33.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: Upgrade: Setting require_osd_release to 19 squid 2024-03-29T04:54:33.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-29T04:54:33.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: osdmap e94: 6 total, 6 up, 6 in 2024-03-29T04:54:33.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-29T04:54:33.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:54:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-29T04:54:33.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-29T04:54:33.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: osdmap e95: 6 total, 6 up, 6 in 2024-03-29T04:54:33.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.152:6826/869112786,v1:172.21.15.152:6827/869112786] up:boot 2024-03-29T04:54:33.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.142:6826/3774187452,v1:172.21.15.142:6827/3774187452] up:boot 2024-03-29T04:54:33.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-29T04:54:33.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:54:33.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:33 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T04:54:34.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: Upgrade: Setting container_image for all osd 2024-03-29T04:54:34.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: Upgrade: Setting require_osd_release to 19 squid 2024-03-29T04:54:34.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: osdmap e94: 6 total, 6 up, 6 in 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: fsmap cephfs:2 {0=cephfs.smithi142.bmkjzf=up:active,1=cephfs.smithi152.uhebsi=up:active} 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-29T04:54:34.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-29T04:54:34.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:54:34.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-29T04:54:34.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-29T04:54:34.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: osdmap e95: 6 total, 6 up, 6 in 2024-03-29T04:54:34.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.152:6826/869112786,v1:172.21.15.152:6827/869112786] up:boot 2024-03-29T04:54:34.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.142:6826/3774187452,v1:172.21.15.142:6827/3774187452] up:boot 2024-03-29T04:54:34.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-29T04:54:34.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:54:34.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:33 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T04:54:34.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:34 smithi142 ceph-mon[112475]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-29T04:54:34.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:34 smithi142 ceph-mon[112475]: pgmap v184: 65 pgs: 12 active+undersized, 12 active+undersized+degraded, 41 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 42/333 objects degraded (12.613%) 2024-03-29T04:54:34.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:34 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.152:6824/924845457,v1:172.21.15.152:6825/924845457] up:boot 2024-03-29T04:54:34.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:34 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.142:6828/168150659,v1:172.21.15.142:6829/168150659] up:boot 2024-03-29T04:54:34.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:34 smithi142 ceph-mon[112475]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T04:54:34.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:34 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T04:54:34.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:34 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:54:35.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:34 smithi152 ceph-mon[88186]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-29T04:54:35.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:34 smithi152 ceph-mon[88186]: pgmap v184: 65 pgs: 12 active+undersized, 12 active+undersized+degraded, 41 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 42/333 objects degraded (12.613%) 2024-03-29T04:54:35.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:34 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.152:6824/924845457,v1:172.21.15.152:6825/924845457] up:boot 2024-03-29T04:54:35.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:34 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.142:6828/168150659,v1:172.21.15.142:6829/168150659] up:boot 2024-03-29T04:54:35.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:34 smithi152 ceph-mon[88186]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T04:54:35.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:34 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T04:54:35.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:34 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:54:36.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:35 smithi142 ceph-mon[112475]: Upgrade: Updating mds.cephfs.smithi142.wgmndr 2024-03-29T04:54:36.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:36.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.wgmndr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:54:36.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:54:36.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:35 smithi142 ceph-mon[112475]: Deploying daemon mds.cephfs.smithi142.wgmndr on smithi142 2024-03-29T04:54:36.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:35 smithi142 ceph-mon[112475]: pgmap v185: 65 pgs: 1 active+undersized, 1 active+undersized+degraded, 63 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2/333 objects degraded (0.601%) 2024-03-29T04:54:36.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:35 smithi152 ceph-mon[88186]: Upgrade: Updating mds.cephfs.smithi142.wgmndr 2024-03-29T04:54:36.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:36.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.wgmndr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:54:36.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:54:36.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:35 smithi152 ceph-mon[88186]: Deploying daemon mds.cephfs.smithi142.wgmndr on smithi142 2024-03-29T04:54:36.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:35 smithi152 ceph-mon[88186]: pgmap v185: 65 pgs: 1 active+undersized, 1 active+undersized+degraded, 63 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2/333 objects degraded (0.601%) 2024-03-29T04:54:37.093 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:36 smithi142 ceph-mon[112475]: Health check update: Degraded data redundancy: 2/333 objects degraded (0.601%), 1 pg degraded (PG_DEGRADED) 2024-03-29T04:54:37.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:36 smithi152 ceph-mon[88186]: Health check update: Degraded data redundancy: 2/333 objects degraded (0.601%), 1 pg degraded (PG_DEGRADED) 2024-03-29T04:54:37.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.386+0000 7f15fa264700 1 -- 172.21.15.142:0/662750884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4105a70 msgr2=0x7f15f4105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:37.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.386+0000 7f15fa264700 1 --2- 172.21.15.142:0/662750884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4105a70 0x7f15f4105e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009910 tx=0x7f15e4009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:37.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.386+0000 7f15fa264700 1 -- 172.21.15.142:0/662750884 shutdown_connections 2024-03-29T04:54:37.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.386+0000 7f15fa264700 1 --2- 172.21.15.142:0/662750884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f40ffa70 0x7f15f40ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:37.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.386+0000 7f15fa264700 1 --2- 172.21.15.142:0/662750884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4105a70 0x7f15f4105e50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:37.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.386+0000 7f15fa264700 1 -- 172.21.15.142:0/662750884 >> 172.21.15.142:0/662750884 conn(0x7f15f40fb560 msgr2=0x7f15f40fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:37.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.386+0000 7f15fa264700 1 -- 172.21.15.142:0/662750884 shutdown_connections 2024-03-29T04:54:37.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.386+0000 7f15fa264700 1 -- 172.21.15.142:0/662750884 wait complete. 2024-03-29T04:54:37.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.387+0000 7f15fa264700 1 Processor -- start 2024-03-29T04:54:37.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.387+0000 7f15fa264700 1 -- start start 2024-03-29T04:54:37.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15fa264700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f40ffa70 0x7f15f4195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:37.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15fa264700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4105a70 0x7f15f41958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:37.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15fa264700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4195e10 con 0x7f15f4105a70 2024-03-29T04:54:37.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15fa264700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4195f80 con 0x7f15f40ffa70 2024-03-29T04:54:37.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15f37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4105a70 0x7f15f41958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:37.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15f37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4105a70 0x7f15f41958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54510/0 (socket says 172.21.15.142:54510) 2024-03-29T04:54:37.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15f37fe700 1 -- 172.21.15.142:0/991786303 learned_addr learned my addr 172.21.15.142:0/991786303 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:37.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15f3fff700 1 --2- 172.21.15.142:0/991786303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f40ffa70 0x7f15f4195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:37.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15f37fe700 1 -- 172.21.15.142:0/991786303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f40ffa70 msgr2=0x7f15f4195390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:37.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15f37fe700 1 --2- 172.21.15.142:0/991786303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f40ffa70 0x7f15f4195390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:37.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15f37fe700 1 -- 172.21.15.142:0/991786303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e40095f0 con 0x7f15f4105a70 2024-03-29T04:54:37.400 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15f3fff700 1 --2- 172.21.15.142:0/991786303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f40ffa70 0x7f15f4195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:54:37.400 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15f37fe700 1 --2- 172.21.15.142:0/991786303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4105a70 0x7f15f41958d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00d8d0 tx=0x7f15dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:37.400 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15f17fa700 1 -- 172.21.15.142:0/991786303 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15dc009940 con 0x7f15f4105a70 2024-03-29T04:54:37.400 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15fa264700 1 -- 172.21.15.142:0/991786303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f41a2a00 con 0x7f15f4105a70 2024-03-29T04:54:37.400 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15f17fa700 1 -- 172.21.15.142:0/991786303 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15dc010460 con 0x7f15f4105a70 2024-03-29T04:54:37.400 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15f17fa700 1 -- 172.21.15.142:0/991786303 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15dc00f5f0 con 0x7f15f4105a70 2024-03-29T04:54:37.400 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.388+0000 7f15fa264700 1 -- 172.21.15.142:0/991786303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f41a2d60 con 0x7f15f4105a70 2024-03-29T04:54:37.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.390+0000 7f15fa264700 1 -- 172.21.15.142:0/991786303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f404ea90 con 0x7f15f4105a70 2024-03-29T04:54:37.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.390+0000 7f15f17fa700 1 -- 172.21.15.142:0/991786303 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f15dc009aa0 con 0x7f15f4105a70 2024-03-29T04:54:37.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.390+0000 7f15f17fa700 1 --2- 172.21.15.142:0/991786303 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f15e006dd60 0x7f15e0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:37.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.390+0000 7f15f17fa700 1 -- 172.21.15.142:0/991786303 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f15dc08dbf0 con 0x7f15f4105a70 2024-03-29T04:54:37.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.390+0000 7f15f3fff700 1 --2- 172.21.15.142:0/991786303 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f15e006dd60 0x7f15e0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:37.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.391+0000 7f15f3fff700 1 --2- 172.21.15.142:0/991786303 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f15e006dd60 0x7f15e0070220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15e4005b40 tx=0x7f15e401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:37.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.395+0000 7f15f17fa700 1 -- 172.21.15.142:0/991786303 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f15dc05ab40 con 0x7f15f4105a70 2024-03-29T04:54:37.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.609+0000 7f15fa264700 1 -- 172.21.15.142:0/991786303 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15f40fc5f0 con 0x7f15e006dd60 2024-03-29T04:54:37.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.611+0000 7f15f17fa700 1 -- 172.21.15.142:0/991786303 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f15f40fc5f0 con 0x7f15e006dd60 2024-03-29T04:54:37.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.613+0000 7f15fa264700 1 -- 172.21.15.142:0/991786303 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f15e006dd60 msgr2=0x7f15e0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:37.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.613+0000 7f15fa264700 1 --2- 172.21.15.142:0/991786303 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f15e006dd60 0x7f15e0070220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15e4005b40 tx=0x7f15e401a040 comp rx=0 tx=0).stop 2024-03-29T04:54:37.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.613+0000 7f15fa264700 1 -- 172.21.15.142:0/991786303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4105a70 msgr2=0x7f15f41958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:37.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.613+0000 7f15fa264700 1 --2- 172.21.15.142:0/991786303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4105a70 0x7f15f41958d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00d8d0 tx=0x7f15dc00dc90 comp rx=0 tx=0).stop 2024-03-29T04:54:37.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.613+0000 7f15fa264700 1 -- 172.21.15.142:0/991786303 shutdown_connections 2024-03-29T04:54:37.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.613+0000 7f15fa264700 1 --2- 172.21.15.142:0/991786303 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f15e006dd60 0x7f15e0070220 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:37.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.613+0000 7f15fa264700 1 --2- 172.21.15.142:0/991786303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f40ffa70 0x7f15f4195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:37.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.613+0000 7f15fa264700 1 --2- 172.21.15.142:0/991786303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15f4105a70 0x7f15f41958d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:37.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.613+0000 7f15fa264700 1 -- 172.21.15.142:0/991786303 >> 172.21.15.142:0/991786303 conn(0x7f15f40fb560 msgr2=0x7f15f40fbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:37.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.613+0000 7f15fa264700 1 -- 172.21.15.142:0/991786303 shutdown_connections 2024-03-29T04:54:37.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.613+0000 7f15fa264700 1 -- 172.21.15.142:0/991786303 wait complete. 2024-03-29T04:54:37.630 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:54:37.724 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.718+0000 7f6b398d5700 1 -- 172.21.15.142:0/4167783334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b34105a30 msgr2=0x7f6b34105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.718+0000 7f6b398d5700 1 --2- 172.21.15.142:0/4167783334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b34105a30 0x7f6b34105e10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009910 tx=0x7f6b24009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.718+0000 7f6b398d5700 1 -- 172.21.15.142:0/4167783334 shutdown_connections 2024-03-29T04:54:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.718+0000 7f6b398d5700 1 --2- 172.21.15.142:0/4167783334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b340ffa30 0x7f6b340ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.718+0000 7f6b398d5700 1 --2- 172.21.15.142:0/4167783334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b34105a30 0x7f6b34105e10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.718+0000 7f6b398d5700 1 -- 172.21.15.142:0/4167783334 >> 172.21.15.142:0/4167783334 conn(0x7f6b340fb520 msgr2=0x7f6b340fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.718+0000 7f6b398d5700 1 -- 172.21.15.142:0/4167783334 shutdown_connections 2024-03-29T04:54:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.718+0000 7f6b398d5700 1 -- 172.21.15.142:0/4167783334 wait complete. 2024-03-29T04:54:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.719+0000 7f6b398d5700 1 Processor -- start 2024-03-29T04:54:37.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.719+0000 7f6b398d5700 1 -- start start 2024-03-29T04:54:37.727 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.719+0000 7f6b398d5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b340ffa30 0x7f6b3419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:37.727 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.719+0000 7f6b398d5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b34105a30 0x7f6b3419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:37.727 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.719+0000 7f6b398d5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3419d050 con 0x7f6b34105a30 2024-03-29T04:54:37.727 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.719+0000 7f6b398d5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b34196510 con 0x7f6b340ffa30 2024-03-29T04:54:37.727 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.719+0000 7f6b327fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b34105a30 0x7f6b3419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:37.727 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.719+0000 7f6b327fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b34105a30 0x7f6b3419c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54518/0 (socket says 172.21.15.142:54518) 2024-03-29T04:54:37.727 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.719+0000 7f6b327fc700 1 -- 172.21.15.142:0/3467115020 learned_addr learned my addr 172.21.15.142:0/3467115020 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:37.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.720+0000 7f6b32ffd700 1 --2- 172.21.15.142:0/3467115020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b340ffa30 0x7f6b3419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:37.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.720+0000 7f6b327fc700 1 -- 172.21.15.142:0/3467115020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b340ffa30 msgr2=0x7f6b3419c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:37.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.720+0000 7f6b327fc700 1 --2- 172.21.15.142:0/3467115020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b340ffa30 0x7f6b3419c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:37.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.720+0000 7f6b327fc700 1 -- 172.21.15.142:0/3467115020 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b240095f0 con 0x7f6b34105a30 2024-03-29T04:54:37.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.720+0000 7f6b32ffd700 1 --2- 172.21.15.142:0/3467115020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b340ffa30 0x7f6b3419c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:54:37.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.720+0000 7f6b327fc700 1 --2- 172.21.15.142:0/3467115020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b34105a30 0x7f6b3419c970 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800d8d0 tx=0x7f6b2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:37.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.720+0000 7f6b388d3700 1 -- 172.21.15.142:0/3467115020 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b28009940 con 0x7f6b34105a30 2024-03-29T04:54:37.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.720+0000 7f6b388d3700 1 -- 172.21.15.142:0/3467115020 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b28010460 con 0x7f6b34105a30 2024-03-29T04:54:37.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.720+0000 7f6b398d5700 1 -- 172.21.15.142:0/3467115020 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b341967f0 con 0x7f6b34105a30 2024-03-29T04:54:37.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.720+0000 7f6b388d3700 1 -- 172.21.15.142:0/3467115020 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b2800f5f0 con 0x7f6b34105a30 2024-03-29T04:54:37.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.720+0000 7f6b398d5700 1 -- 172.21.15.142:0/3467115020 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b34196d40 con 0x7f6b34105a30 2024-03-29T04:54:37.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.721+0000 7f6b398d5700 1 -- 172.21.15.142:0/3467115020 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b34109040 con 0x7f6b34105a30 2024-03-29T04:54:37.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.722+0000 7f6b388d3700 1 -- 172.21.15.142:0/3467115020 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6b280105d0 con 0x7f6b34105a30 2024-03-29T04:54:37.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.722+0000 7f6b388d3700 1 --2- 172.21.15.142:0/3467115020 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b1c06dd60 0x7f6b1c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:37.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.722+0000 7f6b388d3700 1 -- 172.21.15.142:0/3467115020 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6b2808dbe0 con 0x7f6b34105a30 2024-03-29T04:54:37.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.722+0000 7f6b32ffd700 1 --2- 172.21.15.142:0/3467115020 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b1c06dd60 0x7f6b1c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:37.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.723+0000 7f6b32ffd700 1 --2- 172.21.15.142:0/3467115020 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b1c06dd60 0x7f6b1c070220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b24005b40 tx=0x7f6b2401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:37.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.727+0000 7f6b388d3700 1 -- 172.21.15.142:0/3467115020 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6b2805abb0 con 0x7f6b34105a30 2024-03-29T04:54:37.949 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.943+0000 7f6b398d5700 1 -- 172.21.15.142:0/3467115020 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b34197070 con 0x7f6b1c06dd60 2024-03-29T04:54:37.950 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.944+0000 7f6b388d3700 1 -- 172.21.15.142:0/3467115020 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6b34197070 con 0x7f6b1c06dd60 2024-03-29T04:54:37.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.947+0000 7f6b398d5700 1 -- 172.21.15.142:0/3467115020 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b1c06dd60 msgr2=0x7f6b1c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:37.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.947+0000 7f6b398d5700 1 --2- 172.21.15.142:0/3467115020 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b1c06dd60 0x7f6b1c070220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b24005b40 tx=0x7f6b2401a040 comp rx=0 tx=0).stop 2024-03-29T04:54:37.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.947+0000 7f6b398d5700 1 -- 172.21.15.142:0/3467115020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b34105a30 msgr2=0x7f6b3419c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:37.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.947+0000 7f6b398d5700 1 --2- 172.21.15.142:0/3467115020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b34105a30 0x7f6b3419c970 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800d8d0 tx=0x7f6b2800dc90 comp rx=0 tx=0).stop 2024-03-29T04:54:37.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.947+0000 7f6b398d5700 1 -- 172.21.15.142:0/3467115020 shutdown_connections 2024-03-29T04:54:37.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.947+0000 7f6b398d5700 1 --2- 172.21.15.142:0/3467115020 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b1c06dd60 0x7f6b1c070220 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:37.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.947+0000 7f6b398d5700 1 --2- 172.21.15.142:0/3467115020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b340ffa30 0x7f6b3419c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:37.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.947+0000 7f6b398d5700 1 --2- 172.21.15.142:0/3467115020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b34105a30 0x7f6b3419c970 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:37.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.947+0000 7f6b398d5700 1 -- 172.21.15.142:0/3467115020 >> 172.21.15.142:0/3467115020 conn(0x7f6b340fb520 msgr2=0x7f6b34068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:37.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.947+0000 7f6b398d5700 1 -- 172.21.15.142:0/3467115020 shutdown_connections 2024-03-29T04:54:37.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:37.947+0000 7f6b398d5700 1 -- 172.21.15.142:0/3467115020 wait complete. 2024-03-29T04:54:38.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.051+0000 7f8dff88a700 1 -- 172.21.15.142:0/3707512575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df80ffa40 msgr2=0x7f8df80ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:38.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.051+0000 7f8dff88a700 1 --2- 172.21.15.142:0/3707512575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df80ffa40 0x7f8df80ffec0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009910 tx=0x7f8df4009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:38.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.051+0000 7f8dff88a700 1 -- 172.21.15.142:0/3707512575 shutdown_connections 2024-03-29T04:54:38.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.051+0000 7f8dff88a700 1 --2- 172.21.15.142:0/3707512575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df80ffa40 0x7f8df80ffec0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.051+0000 7f8dff88a700 1 --2- 172.21.15.142:0/3707512575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8df8105a40 0x7f8df8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.051+0000 7f8dff88a700 1 -- 172.21.15.142:0/3707512575 >> 172.21.15.142:0/3707512575 conn(0x7f8df80fb570 msgr2=0x7f8df80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:38.058 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:37 smithi142 ceph-mon[112475]: pgmap v186: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:38.058 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:37 smithi142 ceph-mon[112475]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:38.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.051+0000 7f8dff88a700 1 -- 172.21.15.142:0/3707512575 shutdown_connections 2024-03-29T04:54:38.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.051+0000 7f8dff88a700 1 -- 172.21.15.142:0/3707512575 wait complete. 2024-03-29T04:54:38.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.052+0000 7f8dff88a700 1 Processor -- start 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.052+0000 7f8dff88a700 1 -- start start 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.052+0000 7f8dff88a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df80ffa40 0x7f8df8195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.052+0000 7f8dff88a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8df8105a40 0x7f8df8195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.052+0000 7f8dff88a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8196270 con 0x7f8df80ffa40 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.052+0000 7f8dff88a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df819a000 con 0x7f8df8105a40 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.052+0000 7f8dfd626700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df80ffa40 0x7f8df8195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.052+0000 7f8dfd626700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df80ffa40 0x7f8df8195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54530/0 (socket says 172.21.15.142:54530) 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.052+0000 7f8dfd626700 1 -- 172.21.15.142:0/2617694677 learned_addr learned my addr 172.21.15.142:0/2617694677 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.053+0000 7f8dfce25700 1 --2- 172.21.15.142:0/2617694677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8df8105a40 0x7f8df8195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.053+0000 7f8dfd626700 1 -- 172.21.15.142:0/2617694677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8df8105a40 msgr2=0x7f8df8195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.053+0000 7f8dfd626700 1 --2- 172.21.15.142:0/2617694677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8df8105a40 0x7f8df8195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.053+0000 7f8dfd626700 1 -- 172.21.15.142:0/2617694677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df40095f0 con 0x7f8df80ffa40 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.053+0000 7f8dfce25700 1 --2- 172.21.15.142:0/2617694677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8df8105a40 0x7f8df8195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:54:38.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.053+0000 7f8dfd626700 1 --2- 172.21.15.142:0/2617694677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df80ffa40 0x7f8df8195650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8de800d900 tx=0x7f8de800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:38.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.053+0000 7f8dee7fc700 1 -- 172.21.15.142:0/2617694677 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8de8004a00 con 0x7f8df80ffa40 2024-03-29T04:54:38.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.053+0000 7f8dff88a700 1 -- 172.21.15.142:0/2617694677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df819a2e0 con 0x7f8df80ffa40 2024-03-29T04:54:38.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.053+0000 7f8dee7fc700 1 -- 172.21.15.142:0/2617694677 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8de80054e0 con 0x7f8df80ffa40 2024-03-29T04:54:38.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.053+0000 7f8dee7fc700 1 -- 172.21.15.142:0/2617694677 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8de8009d60 con 0x7f8df80ffa40 2024-03-29T04:54:38.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.053+0000 7f8dff88a700 1 -- 172.21.15.142:0/2617694677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df819a830 con 0x7f8df80ffa40 2024-03-29T04:54:38.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.054+0000 7f8dff88a700 1 -- 172.21.15.142:0/2617694677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df8107fd0 con 0x7f8df80ffa40 2024-03-29T04:54:38.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.055+0000 7f8dee7fc700 1 -- 172.21.15.142:0/2617694677 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f8de8005020 con 0x7f8df80ffa40 2024-03-29T04:54:38.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.055+0000 7f8dee7fc700 1 --2- 172.21.15.142:0/2617694677 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8de406dd60 0x7f8de4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:38.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.055+0000 7f8dfce25700 1 --2- 172.21.15.142:0/2617694677 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8de406dd60 0x7f8de4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:38.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.055+0000 7f8dee7fc700 1 -- 172.21.15.142:0/2617694677 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f8de808df10 con 0x7f8df80ffa40 2024-03-29T04:54:38.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.056+0000 7f8dfce25700 1 --2- 172.21.15.142:0/2617694677 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8de406dd60 0x7f8de4070220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8df4005b40 tx=0x7f8df401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:38.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.060+0000 7f8dee7fc700 1 -- 172.21.15.142:0/2617694677 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8de80572d0 con 0x7f8df80ffa40 2024-03-29T04:54:38.272 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.267+0000 7f8dff88a700 1 -- 172.21.15.142:0/2617694677 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8df819ab10 con 0x7f8de406dd60 2024-03-29T04:54:38.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.277+0000 7f8dee7fc700 1 -- 172.21.15.142:0/2617694677 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f8df819ab10 con 0x7f8de406dd60 2024-03-29T04:54:38.283 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:54:38.283 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (6m) 113s ago 17m 25.5M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:54:38.283 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (18m) 113s ago 18m 8762k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:54:38.284 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (17m) 11s ago 17m 8870k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:54:38.284 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (3m) 113s ago 18m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:54:38.284 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (3m) 11s ago 17m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:54:38.284 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (6m) 113s ago 17m 80.3M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:54:38.284 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (12m) 113s ago 12m 136M - 18.2.1 5be31c24972a e74104651183 2024-03-29T04:54:38.284 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (12m) 113s ago 12m 63.5M - 18.2.1 5be31c24972a 7fe6e8084fbf 2024-03-29T04:54:38.284 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (12m) 11s ago 12m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:54:38.284 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (12m) 11s ago 12m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:54:38.284 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (7m) 113s ago 19m 506M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:54:38.284 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (7m) 11s ago 16m 451M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:54:38.284 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (4m) 113s ago 19m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:54:38.284 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (4m) 11s ago 16m 44.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:54:38.285 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (7m) 113s ago 18m 14.9M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:54:38.285 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (7m) 11s ago 17m 16.0M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:54:38.285 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (3m) 113s ago 16m 105M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:54:38.285 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (2m) 113s ago 15m 142M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:54:38.285 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (119s) 113s ago 15m 12.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:54:38.285 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (83s) 11s ago 14m 99.5M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T04:54:38.285 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (49s) 11s ago 14m 87.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T04:54:38.285 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (14s) 11s ago 13m 12.7M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T04:54:38.285 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (6m) 113s ago 17m 63.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:54:38.286 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.280+0000 7f8dff88a700 1 -- 172.21.15.142:0/2617694677 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8de406dd60 msgr2=0x7f8de4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:38.286 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.280+0000 7f8dff88a700 1 --2- 172.21.15.142:0/2617694677 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8de406dd60 0x7f8de4070220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8df4005b40 tx=0x7f8df401a040 comp rx=0 tx=0).stop 2024-03-29T04:54:38.286 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.280+0000 7f8dff88a700 1 -- 172.21.15.142:0/2617694677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df80ffa40 msgr2=0x7f8df8195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:38.286 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.280+0000 7f8dff88a700 1 --2- 172.21.15.142:0/2617694677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df80ffa40 0x7f8df8195650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8de800d900 tx=0x7f8de800dcc0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.286 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.280+0000 7f8dff88a700 1 -- 172.21.15.142:0/2617694677 shutdown_connections 2024-03-29T04:54:38.287 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.280+0000 7f8dff88a700 1 --2- 172.21.15.142:0/2617694677 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8de406dd60 0x7f8de4070220 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.287 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.280+0000 7f8dff88a700 1 --2- 172.21.15.142:0/2617694677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df80ffa40 0x7f8df8195650 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.287 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.280+0000 7f8dff88a700 1 --2- 172.21.15.142:0/2617694677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8df8105a40 0x7f8df8195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.287 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.280+0000 7f8dff88a700 1 -- 172.21.15.142:0/2617694677 >> 172.21.15.142:0/2617694677 conn(0x7f8df80fb570 msgr2=0x7f8df80fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:38.287 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.280+0000 7f8dff88a700 1 -- 172.21.15.142:0/2617694677 shutdown_connections 2024-03-29T04:54:38.287 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.280+0000 7f8dff88a700 1 -- 172.21.15.142:0/2617694677 wait complete. 2024-03-29T04:54:38.295 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:37 smithi152 ceph-mon[88186]: pgmap v186: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:38.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:37 smithi152 ceph-mon[88186]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:38.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.384+0000 7f823364e700 1 -- 172.21.15.142:0/3127377830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822c0ffeb0 msgr2=0x7f822c105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:38.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.384+0000 7f823364e700 1 --2- 172.21.15.142:0/3127377830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822c0ffeb0 0x7f822c105840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f821c009910 tx=0x7f821c009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:38.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.384+0000 7f823364e700 1 -- 172.21.15.142:0/3127377830 shutdown_connections 2024-03-29T04:54:38.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.384+0000 7f823364e700 1 --2- 172.21.15.142:0/3127377830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822c0ffeb0 0x7f822c105840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.384+0000 7f823364e700 1 --2- 172.21.15.142:0/3127377830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f822c0ff500 0x7f822c0ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.384+0000 7f823364e700 1 -- 172.21.15.142:0/3127377830 >> 172.21.15.142:0/3127377830 conn(0x7f822c074d80 msgr2=0x7f822c075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:38.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.384+0000 7f823364e700 1 -- 172.21.15.142:0/3127377830 shutdown_connections 2024-03-29T04:54:38.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.384+0000 7f823364e700 1 -- 172.21.15.142:0/3127377830 wait complete. 2024-03-29T04:54:38.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.385+0000 7f823364e700 1 Processor -- start 2024-03-29T04:54:38.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.385+0000 7f823364e700 1 -- start start 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.385+0000 7f823364e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822c0ff500 0x7f822c100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.385+0000 7f823364e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f822c0ffeb0 0x7f822c1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.385+0000 7f823364e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c100f00 con 0x7f822c0ff500 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.385+0000 7f823364e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c101040 con 0x7f822c0ffeb0 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.385+0000 7f82313ea700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822c0ff500 0x7f822c100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.385+0000 7f82313ea700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822c0ff500 0x7f822c100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54548/0 (socket says 172.21.15.142:54548) 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.385+0000 7f82313ea700 1 -- 172.21.15.142:0/1756451091 learned_addr learned my addr 172.21.15.142:0/1756451091 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.386+0000 7f8230be9700 1 --2- 172.21.15.142:0/1756451091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f822c0ffeb0 0x7f822c1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.386+0000 7f82313ea700 1 -- 172.21.15.142:0/1756451091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f822c0ffeb0 msgr2=0x7f822c1009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.386+0000 7f82313ea700 1 --2- 172.21.15.142:0/1756451091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f822c0ffeb0 0x7f822c1009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.386+0000 7f82313ea700 1 -- 172.21.15.142:0/1756451091 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0095f0 con 0x7f822c0ff500 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.386+0000 7f82313ea700 1 --2- 172.21.15.142:0/1756451091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822c0ff500 0x7f822c100480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f822800c8a0 tx=0x7f822800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:38.393 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.386+0000 7f82227fc700 1 -- 172.21.15.142:0/1756451091 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f822800cea0 con 0x7f822c0ff500 2024-03-29T04:54:38.394 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.386+0000 7f82227fc700 1 -- 172.21.15.142:0/1756451091 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8228004830 con 0x7f822c0ff500 2024-03-29T04:54:38.394 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.386+0000 7f823364e700 1 -- 172.21.15.142:0/1756451091 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822c1a2930 con 0x7f822c0ff500 2024-03-29T04:54:38.394 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.386+0000 7f82227fc700 1 -- 172.21.15.142:0/1756451091 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82280055b0 con 0x7f822c0ff500 2024-03-29T04:54:38.394 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.386+0000 7f823364e700 1 -- 172.21.15.142:0/1756451091 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c1a2d40 con 0x7f822c0ff500 2024-03-29T04:54:38.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.387+0000 7f823364e700 1 -- 172.21.15.142:0/1756451091 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822c1090e0 con 0x7f822c0ff500 2024-03-29T04:54:38.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.387+0000 7f82227fc700 1 -- 172.21.15.142:0/1756451091 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f82280049a0 con 0x7f822c0ff500 2024-03-29T04:54:38.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.388+0000 7f82227fc700 1 --2- 172.21.15.142:0/1756451091 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f821806dd60 0x7f8218070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:38.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.388+0000 7f82227fc700 1 -- 172.21.15.142:0/1756451091 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f822808d880 con 0x7f822c0ff500 2024-03-29T04:54:38.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.388+0000 7f8230be9700 1 --2- 172.21.15.142:0/1756451091 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f821806dd60 0x7f8218070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:38.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.388+0000 7f8230be9700 1 --2- 172.21.15.142:0/1756451091 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f821806dd60 0x7f8218070220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f821c0059d0 tx=0x7f821c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:38.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.392+0000 7f82227fc700 1 -- 172.21.15.142:0/1756451091 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8228056d20 con 0x7f822c0ff500 2024-03-29T04:54:38.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.674+0000 7f823364e700 1 -- 172.21.15.142:0/1756451091 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f822c02cdd0 con 0x7f822c0ff500 2024-03-29T04:54:38.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.675+0000 7f82227fc700 1 -- 172.21.15.142:0/1756451091 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+701 (secure 0 0 0) 0x7f8228004c50 con 0x7f822c0ff500 2024-03-29T04:54:38.681 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:54:38.681 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:54:38.681 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:54:38.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:54:38.681 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:54:38.681 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:54:38.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:54:38.681 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:54:38.681 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T04:54:38.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:54:38.681 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:54:38.681 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3 2024-03-29T04:54:38.682 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:54:38.682 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:54:38.682 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-03-29T04:54:38.682 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 10 2024-03-29T04:54:38.682 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:54:38.682 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:54:38.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.677+0000 7f823364e700 1 -- 172.21.15.142:0/1756451091 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f821806dd60 msgr2=0x7f8218070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:38.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.677+0000 7f823364e700 1 --2- 172.21.15.142:0/1756451091 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f821806dd60 0x7f8218070220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f821c0059d0 tx=0x7f821c005960 comp rx=0 tx=0).stop 2024-03-29T04:54:38.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.677+0000 7f823364e700 1 -- 172.21.15.142:0/1756451091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822c0ff500 msgr2=0x7f822c100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:38.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.677+0000 7f823364e700 1 --2- 172.21.15.142:0/1756451091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822c0ff500 0x7f822c100480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f822800c8a0 tx=0x7f822800cc60 comp rx=0 tx=0).stop 2024-03-29T04:54:38.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.677+0000 7f823364e700 1 -- 172.21.15.142:0/1756451091 shutdown_connections 2024-03-29T04:54:38.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.677+0000 7f823364e700 1 --2- 172.21.15.142:0/1756451091 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f821806dd60 0x7f8218070220 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.677+0000 7f823364e700 1 --2- 172.21.15.142:0/1756451091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822c0ff500 0x7f822c100480 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.677+0000 7f823364e700 1 --2- 172.21.15.142:0/1756451091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f822c0ffeb0 0x7f822c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.677+0000 7f823364e700 1 -- 172.21.15.142:0/1756451091 >> 172.21.15.142:0/1756451091 conn(0x7f822c074d80 msgr2=0x7f822c1050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:38.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.677+0000 7f823364e700 1 -- 172.21.15.142:0/1756451091 shutdown_connections 2024-03-29T04:54:38.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.677+0000 7f823364e700 1 -- 172.21.15.142:0/1756451091 wait complete. 2024-03-29T04:54:38.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.788+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/131251205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f68069020 msgr2=0x7f1f68105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:38.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.788+0000 7f1f6d2e2700 1 --2- 172.21.15.142:0/131251205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f68069020 0x7f1f68105790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009910 tx=0x7f1f50009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:38.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.788+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/131251205 shutdown_connections 2024-03-29T04:54:38.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.788+0000 7f1f6d2e2700 1 --2- 172.21.15.142:0/131251205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f68069020 0x7f1f68105790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.788+0000 7f1f6d2e2700 1 --2- 172.21.15.142:0/131251205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f68068670 0x7f1f68068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.788+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/131251205 >> 172.21.15.142:0/131251205 conn(0x7f1f680755a0 msgr2=0x7f1f680759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:38.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.788+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/131251205 shutdown_connections 2024-03-29T04:54:38.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.788+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/131251205 wait complete. 2024-03-29T04:54:38.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.788+0000 7f1f6d2e2700 1 Processor -- start 2024-03-29T04:54:38.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f6d2e2700 1 -- start start 2024-03-29T04:54:38.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f6d2e2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f68068670 0x7f1f68196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:38.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f6d2e2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f68069020 0x7f1f68196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:38.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f6d2e2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f681972d0 con 0x7f1f68069020 2024-03-29T04:54:38.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f6d2e2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6819a000 con 0x7f1f68068670 2024-03-29T04:54:38.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f5e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f68069020 0x7f1f68196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:38.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f5e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f68069020 0x7f1f68196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37264/0 (socket says 172.21.15.142:37264) 2024-03-29T04:54:38.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f5e7fc700 1 -- 172.21.15.142:0/996511539 learned_addr learned my addr 172.21.15.142:0/996511539 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:38.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f66ffd700 1 --2- 172.21.15.142:0/996511539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f68068670 0x7f1f68196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:38.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f5e7fc700 1 -- 172.21.15.142:0/996511539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f68068670 msgr2=0x7f1f68196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:38.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f5e7fc700 1 --2- 172.21.15.142:0/996511539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f68068670 0x7f1f68196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:38.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f5e7fc700 1 -- 172.21.15.142:0/996511539 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500095f0 con 0x7f1f68069020 2024-03-29T04:54:38.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f66ffd700 1 --2- 172.21.15.142:0/996511539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f68068670 0x7f1f68196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:54:38.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.789+0000 7f1f5e7fc700 1 --2- 172.21.15.142:0/996511539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f68069020 0x7f1f68196c40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f500092e0 tx=0x7f1f5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:38.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.790+0000 7f1f64ff9700 1 -- 172.21.15.142:0/996511539 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f5001d070 con 0x7f1f68069020 2024-03-29T04:54:38.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.790+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/996511539 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6819a280 con 0x7f1f68069020 2024-03-29T04:54:38.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.790+0000 7f1f64ff9700 1 -- 172.21.15.142:0/996511539 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f5000f460 con 0x7f1f68069020 2024-03-29T04:54:38.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.790+0000 7f1f64ff9700 1 -- 172.21.15.142:0/996511539 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f500175f0 con 0x7f1f68069020 2024-03-29T04:54:38.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.790+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/996511539 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6819a770 con 0x7f1f68069020 2024-03-29T04:54:38.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.791+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/996511539 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f681090a0 con 0x7f1f68069020 2024-03-29T04:54:38.801 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.791+0000 7f1f64ff9700 1 -- 172.21.15.142:0/996511539 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f1f5000f5d0 con 0x7f1f68069020 2024-03-29T04:54:38.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.792+0000 7f1f64ff9700 1 --2- 172.21.15.142:0/996511539 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1f5406dd60 0x7f1f54070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:38.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.792+0000 7f1f64ff9700 1 -- 172.21.15.142:0/996511539 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1f5008f210 con 0x7f1f68069020 2024-03-29T04:54:38.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.792+0000 7f1f66ffd700 1 --2- 172.21.15.142:0/996511539 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1f5406dd60 0x7f1f54070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:38.802 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.792+0000 7f1f66ffd700 1 --2- 172.21.15.142:0/996511539 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1f5406dd60 0x7f1f54070220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f58005fd0 tx=0x7f1f58005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:38.803 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:38.796+0000 7f1f64ff9700 1 -- 172.21.15.142:0/996511539 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1f50058630 con 0x7f1f68069020 2024-03-29T04:54:39.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.037+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/996511539 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1f6804ea90 con 0x7f1f68069020 2024-03-29T04:54:39.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:38 smithi142 ceph-mon[112475]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/333 objects degraded (0.601%), 1 pg degraded) 2024-03-29T04:54:39.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:38 smithi142 ceph-mon[112475]: osdmap e96: 6 total, 6 up, 6 in 2024-03-29T04:54:39.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:38 smithi142 ceph-mon[112475]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T04:54:39.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:38 smithi142 ceph-mon[112475]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:39.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:38 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:54:39.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:38 smithi142 ceph-mon[112475]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:39.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:38 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1756451091' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:39.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.037+0000 7f1f64ff9700 1 -- 172.21.15.142:0/996511539 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1693 (secure 0 0 0) 0x7f1f5005bcf0 con 0x7f1f68069020 2024-03-29T04:54:39.044 INFO:teuthology.orchestra.run.smithi142.stdout:e25 2024-03-29T04:54:39.044 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:54:39.044 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:54:39.044 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:54:39.044 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:54:39.044 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:54:39.044 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:54:39.044 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T04:54:39.044 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:54:39.045 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T04:54:39.046 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T04:54:39.046 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:54:39.046 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:54:39.046 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:54:39.046 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:54:39.046 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:54:39.046 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:54:39.046 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:54:39.046 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:54:39.046 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:54:39.046 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:54:39.046 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:54:39.047 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T04:54:39.047 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:54:39.047 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/168150659,v1:172.21.15.142:6829/168150659] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:54:39.047 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44255} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/869112786,v1:172.21.15.152:6827/869112786] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:54:39.047 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/924845457,v1:172.21.15.152:6825/924845457] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:54:39.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.040+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/996511539 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1f5406dd60 msgr2=0x7f1f54070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:39.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.040+0000 7f1f6d2e2700 1 --2- 172.21.15.142:0/996511539 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1f5406dd60 0x7f1f54070220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f58005fd0 tx=0x7f1f58005dc0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.040+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/996511539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f68069020 msgr2=0x7f1f68196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:39.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.040+0000 7f1f6d2e2700 1 --2- 172.21.15.142:0/996511539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f68069020 0x7f1f68196c40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f500092e0 tx=0x7f1f5000be20 comp rx=0 tx=0).stop 2024-03-29T04:54:39.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.040+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/996511539 shutdown_connections 2024-03-29T04:54:39.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.040+0000 7f1f6d2e2700 1 --2- 172.21.15.142:0/996511539 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1f5406dd60 0x7f1f54070220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.040+0000 7f1f6d2e2700 1 --2- 172.21.15.142:0/996511539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f68068670 0x7f1f68196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.040+0000 7f1f6d2e2700 1 --2- 172.21.15.142:0/996511539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f68069020 0x7f1f68196c40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.040+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/996511539 >> 172.21.15.142:0/996511539 conn(0x7f1f680755a0 msgr2=0x7f1f68105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:39.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.041+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/996511539 shutdown_connections 2024-03-29T04:54:39.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.041+0000 7f1f6d2e2700 1 -- 172.21.15.142:0/996511539 wait complete. 2024-03-29T04:54:39.049 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 25 2024-03-29T04:54:39.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.147+0000 7fa2f3b58700 1 -- 172.21.15.142:0/1238246332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec101790 msgr2=0x7fa2ec1057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:39.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.147+0000 7fa2f3b58700 1 --2- 172.21.15.142:0/1238246332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec101790 0x7fa2ec1057e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009910 tx=0x7fa2dc009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:39.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.147+0000 7fa2f3b58700 1 -- 172.21.15.142:0/1238246332 shutdown_connections 2024-03-29T04:54:39.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.147+0000 7fa2f3b58700 1 --2- 172.21.15.142:0/1238246332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec101790 0x7fa2ec1057e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.147+0000 7fa2f3b58700 1 --2- 172.21.15.142:0/1238246332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2ec100de0 0x7fa2ec1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.147+0000 7fa2f3b58700 1 -- 172.21.15.142:0/1238246332 >> 172.21.15.142:0/1238246332 conn(0x7fa2ec0fc6b0 msgr2=0x7fa2ec0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:39.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.147+0000 7fa2f3b58700 1 -- 172.21.15.142:0/1238246332 shutdown_connections 2024-03-29T04:54:39.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.147+0000 7fa2f3b58700 1 -- 172.21.15.142:0/1238246332 wait complete. 2024-03-29T04:54:39.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f3b58700 1 Processor -- start 2024-03-29T04:54:39.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f3b58700 1 -- start start 2024-03-29T04:54:39.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f3b58700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec100de0 0x7fa2ec19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:39.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f3b58700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2ec101790 0x7fa2ec19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:39.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f3b58700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec19d0f0 con 0x7fa2ec100de0 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f3b58700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec1965a0 con 0x7fa2ec101790 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f18f4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec100de0 0x7fa2ec19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f18f4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec100de0 0x7fa2ec19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37280/0 (socket says 172.21.15.142:37280) 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f18f4700 1 -- 172.21.15.142:0/2392395894 learned_addr learned my addr 172.21.15.142:0/2392395894 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f10f3700 1 --2- 172.21.15.142:0/2392395894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2ec101790 0x7fa2ec19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f18f4700 1 -- 172.21.15.142:0/2392395894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2ec101790 msgr2=0x7fa2ec19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f18f4700 1 --2- 172.21.15.142:0/2392395894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2ec101790 0x7fa2ec19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f18f4700 1 -- 172.21.15.142:0/2392395894 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc0095f0 con 0x7fa2ec100de0 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.148+0000 7fa2f18f4700 1 --2- 172.21.15.142:0/2392395894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec100de0 0x7fa2ec19c520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800d900 tx=0x7fa2e800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.149+0000 7fa2e2ffd700 1 -- 172.21.15.142:0/2392395894 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2e8004a00 con 0x7fa2ec100de0 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.149+0000 7fa2e2ffd700 1 -- 172.21.15.142:0/2392395894 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2e80054e0 con 0x7fa2ec100de0 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.149+0000 7fa2f3b58700 1 -- 172.21.15.142:0/2392395894 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec196880 con 0x7fa2ec100de0 2024-03-29T04:54:39.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.149+0000 7fa2e2ffd700 1 -- 172.21.15.142:0/2392395894 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2e8009d00 con 0x7fa2ec100de0 2024-03-29T04:54:39.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.149+0000 7fa2f3b58700 1 -- 172.21.15.142:0/2392395894 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec196dd0 con 0x7fa2ec100de0 2024-03-29T04:54:39.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.150+0000 7fa2f3b58700 1 -- 172.21.15.142:0/2392395894 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec1090f0 con 0x7fa2ec100de0 2024-03-29T04:54:39.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.150+0000 7fa2e2ffd700 1 -- 172.21.15.142:0/2392395894 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fa2e8005020 con 0x7fa2ec100de0 2024-03-29T04:54:39.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.150+0000 7fa2e2ffd700 1 --2- 172.21.15.142:0/2392395894 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa2d806dd60 0x7fa2d8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:39.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.151+0000 7fa2e2ffd700 1 -- 172.21.15.142:0/2392395894 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa2e808dbe0 con 0x7fa2ec100de0 2024-03-29T04:54:39.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.151+0000 7fa2f10f3700 1 --2- 172.21.15.142:0/2392395894 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa2d806dd60 0x7fa2d8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:39.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.151+0000 7fa2f10f3700 1 --2- 172.21.15.142:0/2392395894 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa2d806dd60 0x7fa2d8070220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0fdfb0 tx=0x7fa2dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:39.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.155+0000 7fa2e2ffd700 1 -- 172.21.15.142:0/2392395894 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa2e805ab30 con 0x7fa2ec100de0 2024-03-29T04:54:39.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:38 smithi152 ceph-mon[88186]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/333 objects degraded (0.601%), 1 pg degraded) 2024-03-29T04:54:39.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:38 smithi152 ceph-mon[88186]: osdmap e96: 6 total, 6 up, 6 in 2024-03-29T04:54:39.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:38 smithi152 ceph-mon[88186]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T04:54:39.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:38 smithi152 ceph-mon[88186]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:39.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:38 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:54:39.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:38 smithi152 ceph-mon[88186]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:39.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:38 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1756451091' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:39.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.366+0000 7fa2f3b58700 1 -- 172.21.15.142:0/2392395894 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2ec1970b0 con 0x7fa2d806dd60 2024-03-29T04:54:39.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.368+0000 7fa2e2ffd700 1 -- 172.21.15.142:0/2392395894 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa2ec1970b0 con 0x7fa2d806dd60 2024-03-29T04:54:39.374 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:54:39.374 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:54:39.374 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:54:39.374 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:54:39.374 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:54:39.374 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:54:39.374 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:54:39.375 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T04:54:39.375 INFO:teuthology.orchestra.run.smithi142.stdout: "mon" 2024-03-29T04:54:39.375 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:54:39.375 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "12/23 daemons upgraded", 2024-03-29T04:54:39.375 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T04:54:39.375 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:54:39.375 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:54:39.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.370+0000 7fa2f3b58700 1 -- 172.21.15.142:0/2392395894 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa2d806dd60 msgr2=0x7fa2d8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:39.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.370+0000 7fa2f3b58700 1 --2- 172.21.15.142:0/2392395894 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa2d806dd60 0x7fa2d8070220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0fdfb0 tx=0x7fa2dc01a040 comp rx=0 tx=0).stop 2024-03-29T04:54:39.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.370+0000 7fa2f3b58700 1 -- 172.21.15.142:0/2392395894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec100de0 msgr2=0x7fa2ec19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:39.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.370+0000 7fa2f3b58700 1 --2- 172.21.15.142:0/2392395894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec100de0 0x7fa2ec19c520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800d900 tx=0x7fa2e800dc10 comp rx=0 tx=0).stop 2024-03-29T04:54:39.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.370+0000 7fa2f3b58700 1 -- 172.21.15.142:0/2392395894 shutdown_connections 2024-03-29T04:54:39.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.370+0000 7fa2f3b58700 1 --2- 172.21.15.142:0/2392395894 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa2d806dd60 0x7fa2d8070220 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.370+0000 7fa2f3b58700 1 --2- 172.21.15.142:0/2392395894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ec100de0 0x7fa2ec19c520 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.370+0000 7fa2f3b58700 1 --2- 172.21.15.142:0/2392395894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2ec101790 0x7fa2ec19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.370+0000 7fa2f3b58700 1 -- 172.21.15.142:0/2392395894 >> 172.21.15.142:0/2392395894 conn(0x7fa2ec0fc6b0 msgr2=0x7fa2ec105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:39.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.370+0000 7fa2f3b58700 1 -- 172.21.15.142:0/2392395894 shutdown_connections 2024-03-29T04:54:39.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.370+0000 7fa2f3b58700 1 -- 172.21.15.142:0/2392395894 wait complete. 2024-03-29T04:54:39.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.474+0000 7fa81e86e700 1 -- 172.21.15.142:0/1118832925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa818100da0 msgr2=0x7fa818101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:39.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.474+0000 7fa81e86e700 1 --2- 172.21.15.142:0/1118832925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa818100da0 0x7fa818101180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa808009910 tx=0x7fa808009c20 comp rx=0 tx=0).stop 2024-03-29T04:54:39.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.474+0000 7fa81e86e700 1 -- 172.21.15.142:0/1118832925 shutdown_connections 2024-03-29T04:54:39.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.474+0000 7fa81e86e700 1 --2- 172.21.15.142:0/1118832925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa818101750 0x7fa8181057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.474+0000 7fa81e86e700 1 --2- 172.21.15.142:0/1118832925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa818100da0 0x7fa818101180 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.474+0000 7fa81e86e700 1 -- 172.21.15.142:0/1118832925 >> 172.21.15.142:0/1118832925 conn(0x7fa8180fc650 msgr2=0x7fa8180fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:39.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.474+0000 7fa81e86e700 1 -- 172.21.15.142:0/1118832925 shutdown_connections 2024-03-29T04:54:39.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.474+0000 7fa81e86e700 1 -- 172.21.15.142:0/1118832925 wait complete. 2024-03-29T04:54:39.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.474+0000 7fa81e86e700 1 Processor -- start 2024-03-29T04:54:39.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.474+0000 7fa81e86e700 1 -- start start 2024-03-29T04:54:39.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa81e86e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa818100da0 0x7fa818073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:39.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa81e86e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa818101750 0x7fa8180735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:39.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa81e86e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa818073b10 con 0x7fa818101750 2024-03-29T04:54:39.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa81e86e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa818073c50 con 0x7fa818100da0 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa8177fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa818101750 0x7fa8180735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa8177fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa818101750 0x7fa8180735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37288/0 (socket says 172.21.15.142:37288) 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa8177fe700 1 -- 172.21.15.142:0/3243173785 learned_addr learned my addr 172.21.15.142:0/3243173785 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa817fff700 1 --2- 172.21.15.142:0/3243173785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa818100da0 0x7fa818073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa8177fe700 1 -- 172.21.15.142:0/3243173785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa818100da0 msgr2=0x7fa818073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa8177fe700 1 --2- 172.21.15.142:0/3243173785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa818100da0 0x7fa818073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa8177fe700 1 -- 172.21.15.142:0/3243173785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8080095f0 con 0x7fa818101750 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa8177fe700 1 --2- 172.21.15.142:0/3243173785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa818101750 0x7fa8180735d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00d900 tx=0x7fa80c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa8157fa700 1 -- 172.21.15.142:0/3243173785 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa80c009900 con 0x7fa818101750 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa8157fa700 1 -- 172.21.15.142:0/3243173785 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa80c010460 con 0x7fa818101750 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa81e86e700 1 -- 172.21.15.142:0/3243173785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8181a2920 con 0x7fa818101750 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.475+0000 7fa8157fa700 1 -- 172.21.15.142:0/3243173785 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa80c00f5f0 con 0x7fa818101750 2024-03-29T04:54:39.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.476+0000 7fa81e86e700 1 -- 172.21.15.142:0/3243173785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8181a2d30 con 0x7fa818101750 2024-03-29T04:54:39.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.477+0000 7fa81e86e700 1 -- 172.21.15.142:0/3243173785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa818109140 con 0x7fa818101750 2024-03-29T04:54:39.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.477+0000 7fa8157fa700 1 -- 172.21.15.142:0/3243173785 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fa80c0105d0 con 0x7fa818101750 2024-03-29T04:54:39.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.477+0000 7fa8157fa700 1 --2- 172.21.15.142:0/3243173785 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa80006dd60 0x7fa800070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:54:39.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.477+0000 7fa8157fa700 1 -- 172.21.15.142:0/3243173785 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa80c08d750 con 0x7fa818101750 2024-03-29T04:54:39.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.478+0000 7fa817fff700 1 --2- 172.21.15.142:0/3243173785 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa80006dd60 0x7fa800070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:54:39.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.478+0000 7fa817fff700 1 --2- 172.21.15.142:0/3243173785 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa80006dd60 0x7fa800070220 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa808006010 tx=0x7fa80801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:54:39.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.482+0000 7fa8157fa700 1 -- 172.21.15.142:0/3243173785 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa80c056bf0 con 0x7fa818101750 2024-03-29T04:54:39.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.762+0000 7fa81e86e700 1 -- 172.21.15.142:0/3243173785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa81804ea90 con 0x7fa818101750 2024-03-29T04:54:39.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.762+0000 7fa8157fa700 1 -- 172.21.15.142:0/3243173785 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa80c05a2b0 con 0x7fa818101750 2024-03-29T04:54:39.769 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T04:54:39.769 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T04:54:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T04:54:39.769 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T04:54:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T04:54:39.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.764+0000 7fa81e86e700 1 -- 172.21.15.142:0/3243173785 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa80006dd60 msgr2=0x7fa800070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:39.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.764+0000 7fa81e86e700 1 --2- 172.21.15.142:0/3243173785 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa80006dd60 0x7fa800070220 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa808006010 tx=0x7fa80801a040 comp rx=0 tx=0).stop 2024-03-29T04:54:39.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.764+0000 7fa81e86e700 1 -- 172.21.15.142:0/3243173785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa818101750 msgr2=0x7fa8180735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:54:39.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.764+0000 7fa81e86e700 1 --2- 172.21.15.142:0/3243173785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa818101750 0x7fa8180735d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00d900 tx=0x7fa80c00dc10 comp rx=0 tx=0).stop 2024-03-29T04:54:39.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.764+0000 7fa81e86e700 1 -- 172.21.15.142:0/3243173785 shutdown_connections 2024-03-29T04:54:39.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.764+0000 7fa81e86e700 1 --2- 172.21.15.142:0/3243173785 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa80006dd60 0x7fa800070220 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.764+0000 7fa81e86e700 1 --2- 172.21.15.142:0/3243173785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa818100da0 0x7fa818073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.764+0000 7fa81e86e700 1 --2- 172.21.15.142:0/3243173785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa818101750 0x7fa8180735d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:54:39.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.764+0000 7fa81e86e700 1 -- 172.21.15.142:0/3243173785 >> 172.21.15.142:0/3243173785 conn(0x7fa8180fc650 msgr2=0x7fa8180fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:54:39.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.764+0000 7fa81e86e700 1 -- 172.21.15.142:0/3243173785 shutdown_connections 2024-03-29T04:54:39.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:54:39.764+0000 7fa81e86e700 1 -- 172.21.15.142:0/3243173785 wait complete. 2024-03-29T04:54:40.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:39 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/996511539' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:54:40.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:39 smithi142 ceph-mon[112475]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:40.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:39 smithi142 ceph-mon[112475]: pgmap v188: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:40.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:39 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3243173785' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:54:40.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:39 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/996511539' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:54:40.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:39 smithi152 ceph-mon[88186]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:54:40.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:39 smithi152 ceph-mon[88186]: pgmap v188: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:40.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:39 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3243173785' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:54:42.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:42 smithi142 ceph-mon[112475]: pgmap v189: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:43.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:42 smithi152 ceph-mon[88186]: pgmap v189: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:43.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:43 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:43.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:43 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:43.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:43 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:54:43.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:43 smithi142 ceph-mon[112475]: pgmap v190: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:43.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:43 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] up:boot 2024-03-29T04:54:43.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:43 smithi142 ceph-mon[112475]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T04:54:43.868 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:43 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:54:44.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:43 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:44.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:43 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:44.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:43 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:54:44.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:43 smithi152 ceph-mon[88186]: pgmap v190: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:44.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:43 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] up:boot 2024-03-29T04:54:44.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:43 smithi152 ceph-mon[88186]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T04:54:44.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:43 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T04:54:45.586 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:45.587 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:54:45.587 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:46.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:46.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:54:46.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:46.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:46 smithi142 ceph-mon[112475]: pgmap v191: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:47.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:46 smithi152 ceph-mon[88186]: pgmap v191: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:48.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:48 smithi142 ceph-mon[112475]: pgmap v192: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:48.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:48 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:48.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:48 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:49.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:48 smithi152 ceph-mon[88186]: pgmap v192: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:49.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:48 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:49.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:48 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:50.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:49 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:50.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:49 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:50.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:49 smithi152 ceph-mon[88186]: pgmap v193: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:50.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:49 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:50.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:49 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:50.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:49 smithi142 ceph-mon[112475]: pgmap v193: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:52.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:52 smithi142 ceph-mon[112475]: pgmap v194: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:52.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:52.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:52.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:54:52.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:54:52.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:52.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:54:52.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:52.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:52.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:52.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:52 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:53.000 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:52 smithi152 ceph-mon[88186]: pgmap v194: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:53.000 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:53.000 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:53.000 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:54:53.000 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:54:53.000 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:53.001 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:54:53.001 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:53.001 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:53.001 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:53.001 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:52 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:54:54.909 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:54 smithi142 ceph-mon[112475]: pgmap v195: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:54.909 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:54.909 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.bmkjzf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:54:54.909 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:54:55.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:54 smithi152 ceph-mon[88186]: pgmap v195: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:55.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:54:55.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi142.bmkjzf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:54:55.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:54:55.832 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:55 smithi142 ceph-mon[112475]: Upgrade: Updating mds.cephfs.smithi142.bmkjzf 2024-03-29T04:54:55.832 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:55 smithi142 ceph-mon[112475]: Deploying daemon mds.cephfs.smithi142.bmkjzf on smithi142 2024-03-29T04:54:56.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:55 smithi152 ceph-mon[88186]: Upgrade: Updating mds.cephfs.smithi142.bmkjzf 2024-03-29T04:54:56.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:55 smithi152 ceph-mon[88186]: Deploying daemon mds.cephfs.smithi142.bmkjzf on smithi142 2024-03-29T04:54:57.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:56 smithi152 ceph-mon[88186]: pgmap v196: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:57.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:56 smithi142 ceph-mon[112475]: pgmap v196: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:57.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:57 smithi142 ceph-mon[112475]: osdmap e97: 6 total, 6 up, 6 in 2024-03-29T04:54:57.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:57 smithi142 ceph-mon[112475]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T04:54:58.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:57 smithi152 ceph-mon[88186]: osdmap e97: 6 total, 6 up, 6 in 2024-03-29T04:54:58.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:57 smithi152 ceph-mon[88186]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T04:54:59.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:58 smithi152 ceph-mon[88186]: pgmap v198: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:54:59.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:58 smithi142 ceph-mon[112475]: pgmap v198: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:00.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:00.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:00.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:55:00.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:55:00.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:59 smithi142 ceph-mon[112475]: pgmap v199: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:00.043 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:59 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] up:boot 2024-03-29T04:55:00.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:59 smithi142 ceph-mon[112475]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T04:55:00.044 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:54:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T04:55:00.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:00.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:00.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:55:00.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:55:00.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:59 smithi152 ceph-mon[88186]: pgmap v199: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:00.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:59 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] up:boot 2024-03-29T04:55:00.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:59 smithi152 ceph-mon[88186]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T04:55:00.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:54:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T04:55:02.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:02 smithi142 ceph-mon[112475]: pgmap v200: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:03.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:02 smithi152 ceph-mon[88186]: pgmap v200: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:05.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:04 smithi152 ceph-mon[88186]: pgmap v201: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:05.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:05.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:05.105 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:04 smithi142 ceph-mon[112475]: pgmap v201: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:05.105 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:05.105 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:06.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:06 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:06.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:06 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:06.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:06 smithi142 ceph-mon[112475]: pgmap v202: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:06.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:06 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:06.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:06 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:06.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:06 smithi152 ceph-mon[88186]: pgmap v202: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:08.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:08 smithi142 ceph-mon[112475]: pgmap v203: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:08.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:08 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:08.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:08 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:08.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:08 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:55:08.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:08 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:55:08.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:08 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:09.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:08 smithi152 ceph-mon[88186]: pgmap v203: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:09.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:08 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:09.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:08 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:09.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:08 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:55:09.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:08 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:55:09.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:08 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:09.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.882+0000 7fde1dc91700 1 -- 172.21.15.142:0/3450169219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde18101770 msgr2=0x7fde181057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:09.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.882+0000 7fde1dc91700 1 --2- 172.21.15.142:0/3450169219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde18101770 0x7fde181057c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fde00009910 tx=0x7fde00009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:09.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.882+0000 7fde1dc91700 1 -- 172.21.15.142:0/3450169219 shutdown_connections 2024-03-29T04:55:09.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.882+0000 7fde1dc91700 1 --2- 172.21.15.142:0/3450169219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde18101770 0x7fde181057c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:09.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.882+0000 7fde1dc91700 1 --2- 172.21.15.142:0/3450169219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde18100dc0 0x7fde181011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:09.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.882+0000 7fde1dc91700 1 -- 172.21.15.142:0/3450169219 >> 172.21.15.142:0/3450169219 conn(0x7fde180fc650 msgr2=0x7fde180fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:09.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.882+0000 7fde1dc91700 1 -- 172.21.15.142:0/3450169219 shutdown_connections 2024-03-29T04:55:09.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.882+0000 7fde1dc91700 1 -- 172.21.15.142:0/3450169219 wait complete. 2024-03-29T04:55:09.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.883+0000 7fde1dc91700 1 Processor -- start 2024-03-29T04:55:09.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.883+0000 7fde1dc91700 1 -- start start 2024-03-29T04:55:09.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.883+0000 7fde1dc91700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde18100dc0 0x7fde18073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:09.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.883+0000 7fde1dc91700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde18101770 0x7fde180735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:09.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.883+0000 7fde1dc91700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde18073b10 con 0x7fde18101770 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.883+0000 7fde1dc91700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde18073c50 con 0x7fde18100dc0 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.883+0000 7fde0effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde18101770 0x7fde180735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.883+0000 7fde0effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde18101770 0x7fde180735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40488/0 (socket says 172.21.15.142:40488) 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.883+0000 7fde0effd700 1 -- 172.21.15.142:0/82082441 learned_addr learned my addr 172.21.15.142:0/82082441 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.883+0000 7fde177fe700 1 --2- 172.21.15.142:0/82082441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde18100dc0 0x7fde18073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.884+0000 7fde0effd700 1 -- 172.21.15.142:0/82082441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde18100dc0 msgr2=0x7fde18073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.884+0000 7fde0effd700 1 --2- 172.21.15.142:0/82082441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde18100dc0 0x7fde18073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.884+0000 7fde0effd700 1 -- 172.21.15.142:0/82082441 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000095f0 con 0x7fde18101770 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.884+0000 7fde0effd700 1 --2- 172.21.15.142:0/82082441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde18101770 0x7fde180735d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fde00009fd0 tx=0x7fde0000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.884+0000 7fde157fa700 1 -- 172.21.15.142:0/82082441 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde0001d070 con 0x7fde18101770 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.884+0000 7fde157fa700 1 -- 172.21.15.142:0/82082441 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde00022470 con 0x7fde18101770 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.884+0000 7fde1dc91700 1 -- 172.21.15.142:0/82082441 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde18073ed0 con 0x7fde18101770 2024-03-29T04:55:09.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.884+0000 7fde157fa700 1 -- 172.21.15.142:0/82082441 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde00017650 con 0x7fde18101770 2024-03-29T04:55:09.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.884+0000 7fde1dc91700 1 -- 172.21.15.142:0/82082441 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde181a2c30 con 0x7fde18101770 2024-03-29T04:55:09.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.885+0000 7fde1dc91700 1 -- 172.21.15.142:0/82082441 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde181091c0 con 0x7fde18101770 2024-03-29T04:55:09.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.886+0000 7fde157fa700 1 -- 172.21.15.142:0/82082441 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fde000225e0 con 0x7fde18101770 2024-03-29T04:55:09.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.886+0000 7fde157fa700 1 --2- 172.21.15.142:0/82082441 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fde0406dd60 0x7fde04070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:09.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.886+0000 7fde157fa700 1 -- 172.21.15.142:0/82082441 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fde0008f160 con 0x7fde18101770 2024-03-29T04:55:09.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.886+0000 7fde177fe700 1 --2- 172.21.15.142:0/82082441 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fde0406dd60 0x7fde04070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:09.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.886+0000 7fde177fe700 1 --2- 172.21.15.142:0/82082441 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fde0406dd60 0x7fde04070220 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fde08009c60 tx=0x7fde08009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:09.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:09.891+0000 7fde157fa700 1 -- 172.21.15.142:0/82082441 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fde00058580 con 0x7fde18101770 2024-03-29T04:55:10.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:09 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:55:10.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:09 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:10.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:09 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:10.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:09 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:10.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:09 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:10.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.104+0000 7fde1dc91700 1 -- 172.21.15.142:0/82082441 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde1802cce0 con 0x7fde0406dd60 2024-03-29T04:55:10.110 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:09 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:55:10.110 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:09 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:10.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:09 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:10.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:09 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:10.111 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:09 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:10.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.106+0000 7fde157fa700 1 -- 172.21.15.142:0/82082441 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fde1802cce0 con 0x7fde0406dd60 2024-03-29T04:55:10.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.108+0000 7fde1dc91700 1 -- 172.21.15.142:0/82082441 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fde0406dd60 msgr2=0x7fde04070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:10.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.108+0000 7fde1dc91700 1 --2- 172.21.15.142:0/82082441 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fde0406dd60 0x7fde04070220 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fde08009c60 tx=0x7fde08009260 comp rx=0 tx=0).stop 2024-03-29T04:55:10.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.108+0000 7fde1dc91700 1 -- 172.21.15.142:0/82082441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde18101770 msgr2=0x7fde180735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:10.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.108+0000 7fde1dc91700 1 --2- 172.21.15.142:0/82082441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde18101770 0x7fde180735d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fde00009fd0 tx=0x7fde0000f690 comp rx=0 tx=0).stop 2024-03-29T04:55:10.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.108+0000 7fde1dc91700 1 -- 172.21.15.142:0/82082441 shutdown_connections 2024-03-29T04:55:10.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.108+0000 7fde1dc91700 1 --2- 172.21.15.142:0/82082441 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fde0406dd60 0x7fde04070220 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.108+0000 7fde1dc91700 1 --2- 172.21.15.142:0/82082441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde18100dc0 0x7fde18073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.108+0000 7fde1dc91700 1 --2- 172.21.15.142:0/82082441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde18101770 0x7fde180735d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.108+0000 7fde1dc91700 1 -- 172.21.15.142:0/82082441 >> 172.21.15.142:0/82082441 conn(0x7fde180fc650 msgr2=0x7fde180fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:10.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.108+0000 7fde1dc91700 1 -- 172.21.15.142:0/82082441 shutdown_connections 2024-03-29T04:55:10.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.108+0000 7fde1dc91700 1 -- 172.21.15.142:0/82082441 wait complete. 2024-03-29T04:55:10.125 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:55:10.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.223+0000 7f4b81a39700 1 -- 172.21.15.142:0/106551543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7c102610 msgr2=0x7f4b7c10ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:10.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.223+0000 7f4b81a39700 1 --2- 172.21.15.142:0/106551543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7c102610 0x7f4b7c10ab00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009910 tx=0x7f4b70009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:10.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.223+0000 7f4b81a39700 1 -- 172.21.15.142:0/106551543 shutdown_connections 2024-03-29T04:55:10.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.223+0000 7f4b81a39700 1 --2- 172.21.15.142:0/106551543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7c102610 0x7f4b7c10ab00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.223+0000 7f4b81a39700 1 --2- 172.21.15.142:0/106551543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b7c101c60 0x7f4b7c102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.223+0000 7f4b81a39700 1 -- 172.21.15.142:0/106551543 >> 172.21.15.142:0/106551543 conn(0x7f4b7c0fb540 msgr2=0x7f4b7c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:10.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.223+0000 7f4b81a39700 1 -- 172.21.15.142:0/106551543 shutdown_connections 2024-03-29T04:55:10.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.223+0000 7f4b81a39700 1 -- 172.21.15.142:0/106551543 wait complete. 2024-03-29T04:55:10.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.223+0000 7f4b81a39700 1 Processor -- start 2024-03-29T04:55:10.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b81a39700 1 -- start start 2024-03-29T04:55:10.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b81a39700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7c101c60 0x7f4b7c0ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:10.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b81a39700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b7c102610 0x7f4b7c0ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:10.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b81a39700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c1003d0 con 0x7f4b7c101c60 2024-03-29T04:55:10.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b81a39700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c100510 con 0x7f4b7c102610 2024-03-29T04:55:10.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b7affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7c101c60 0x7f4b7c0ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:10.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b7affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7c101c60 0x7f4b7c0ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40494/0 (socket says 172.21.15.142:40494) 2024-03-29T04:55:10.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b7affd700 1 -- 172.21.15.142:0/479506508 learned_addr learned my addr 172.21.15.142:0/479506508 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:10.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b7a7fc700 1 --2- 172.21.15.142:0/479506508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b7c102610 0x7f4b7c0ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:10.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b7affd700 1 -- 172.21.15.142:0/479506508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b7c102610 msgr2=0x7f4b7c0ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:10.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b7affd700 1 --2- 172.21.15.142:0/479506508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b7c102610 0x7f4b7c0ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b7affd700 1 -- 172.21.15.142:0/479506508 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b700095f0 con 0x7f4b7c101c60 2024-03-29T04:55:10.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.224+0000 7f4b7affd700 1 --2- 172.21.15.142:0/479506508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7c101c60 0x7f4b7c0ff950 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00c8a0 tx=0x7f4b6c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:10.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.225+0000 7f4b80a37700 1 -- 172.21.15.142:0/479506508 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b6c0078e0 con 0x7f4b7c101c60 2024-03-29T04:55:10.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.225+0000 7f4b80a37700 1 -- 172.21.15.142:0/479506508 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b6c00f450 con 0x7f4b7c101c60 2024-03-29T04:55:10.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.225+0000 7f4b81a39700 1 -- 172.21.15.142:0/479506508 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7c1a29b0 con 0x7f4b7c101c60 2024-03-29T04:55:10.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.225+0000 7f4b80a37700 1 -- 172.21.15.142:0/479506508 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b6c00e5e0 con 0x7f4b7c101c60 2024-03-29T04:55:10.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.225+0000 7f4b81a39700 1 -- 172.21.15.142:0/479506508 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7c1a2dc0 con 0x7f4b7c101c60 2024-03-29T04:55:10.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.225+0000 7f4b81a39700 1 -- 172.21.15.142:0/479506508 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7c108200 con 0x7f4b7c101c60 2024-03-29T04:55:10.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.226+0000 7f4b80a37700 1 -- 172.21.15.142:0/479506508 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f4b6c00f5c0 con 0x7f4b7c101c60 2024-03-29T04:55:10.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.227+0000 7f4b80a37700 1 --2- 172.21.15.142:0/479506508 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b6406dd60 0x7f4b64070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:10.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.227+0000 7f4b80a37700 1 -- 172.21.15.142:0/479506508 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f4b6c08dc60 con 0x7f4b7c101c60 2024-03-29T04:55:10.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.227+0000 7f4b7a7fc700 1 --2- 172.21.15.142:0/479506508 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b6406dd60 0x7f4b64070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:10.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.227+0000 7f4b7a7fc700 1 --2- 172.21.15.142:0/479506508 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b6406dd60 0x7f4b64070220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c101140 tx=0x7f4b7001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:10.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.231+0000 7f4b80a37700 1 -- 172.21.15.142:0/479506508 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4b6c057080 con 0x7f4b7c101c60 2024-03-29T04:55:10.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.445+0000 7f4b81a39700 1 -- 172.21.15.142:0/479506508 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b7c02cce0 con 0x7f4b6406dd60 2024-03-29T04:55:10.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.446+0000 7f4b80a37700 1 -- 172.21.15.142:0/479506508 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4b7c02cce0 con 0x7f4b6406dd60 2024-03-29T04:55:10.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.448+0000 7f4b81a39700 1 -- 172.21.15.142:0/479506508 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b6406dd60 msgr2=0x7f4b64070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:10.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.448+0000 7f4b81a39700 1 --2- 172.21.15.142:0/479506508 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b6406dd60 0x7f4b64070220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c101140 tx=0x7f4b7001a040 comp rx=0 tx=0).stop 2024-03-29T04:55:10.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.448+0000 7f4b81a39700 1 -- 172.21.15.142:0/479506508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7c101c60 msgr2=0x7f4b7c0ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:10.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.448+0000 7f4b81a39700 1 --2- 172.21.15.142:0/479506508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7c101c60 0x7f4b7c0ff950 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00c8a0 tx=0x7f4b6c00cbb0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.448+0000 7f4b81a39700 1 -- 172.21.15.142:0/479506508 shutdown_connections 2024-03-29T04:55:10.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.448+0000 7f4b81a39700 1 --2- 172.21.15.142:0/479506508 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b6406dd60 0x7f4b64070220 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.448+0000 7f4b81a39700 1 --2- 172.21.15.142:0/479506508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7c101c60 0x7f4b7c0ff950 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.448+0000 7f4b81a39700 1 --2- 172.21.15.142:0/479506508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b7c102610 0x7f4b7c0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.448+0000 7f4b81a39700 1 -- 172.21.15.142:0/479506508 >> 172.21.15.142:0/479506508 conn(0x7f4b7c0fb540 msgr2=0x7f4b7c105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:10.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.448+0000 7f4b81a39700 1 -- 172.21.15.142:0/479506508 shutdown_connections 2024-03-29T04:55:10.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.448+0000 7f4b81a39700 1 -- 172.21.15.142:0/479506508 wait complete. 2024-03-29T04:55:10.562 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.554+0000 7f6374a75700 1 -- 172.21.15.142:0/31324218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6370101750 msgr2=0x7f63701057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:10.562 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.554+0000 7f6374a75700 1 --2- 172.21.15.142:0/31324218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6370101750 0x7f63701057a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6364009910 tx=0x7f6364009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:10.562 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.554+0000 7f6374a75700 1 -- 172.21.15.142:0/31324218 shutdown_connections 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.554+0000 7f6374a75700 1 --2- 172.21.15.142:0/31324218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6370101750 0x7f63701057a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.554+0000 7f6374a75700 1 --2- 172.21.15.142:0/31324218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6370100da0 0x7f6370101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.554+0000 7f6374a75700 1 -- 172.21.15.142:0/31324218 >> 172.21.15.142:0/31324218 conn(0x7f63700fc630 msgr2=0x7f63700fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.554+0000 7f6374a75700 1 -- 172.21.15.142:0/31324218 shutdown_connections 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.554+0000 7f6374a75700 1 -- 172.21.15.142:0/31324218 wait complete. 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f6374a75700 1 Processor -- start 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f6374a75700 1 -- start start 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f6374a75700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6370100da0 0x7f6370196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f6374a75700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6370101750 0x7f6370196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f6374a75700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6370197300 con 0x7f6370101750 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f6374a75700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6370199fc0 con 0x7f6370100da0 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f636dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6370101750 0x7f6370196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:10.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f636dd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6370101750 0x7f6370196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40518/0 (socket says 172.21.15.142:40518) 2024-03-29T04:55:10.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f636dd9b700 1 -- 172.21.15.142:0/2172240426 learned_addr learned my addr 172.21.15.142:0/2172240426 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:10.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f636e59c700 1 --2- 172.21.15.142:0/2172240426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6370100da0 0x7f6370196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:10.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f636dd9b700 1 -- 172.21.15.142:0/2172240426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6370100da0 msgr2=0x7f6370196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:10.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f636dd9b700 1 --2- 172.21.15.142:0/2172240426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6370100da0 0x7f6370196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.555+0000 7f636dd9b700 1 -- 172.21.15.142:0/2172240426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63640095f0 con 0x7f6370101750 2024-03-29T04:55:10.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.556+0000 7f636dd9b700 1 --2- 172.21.15.142:0/2172240426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6370101750 0x7f6370196c70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f63640059d0 tx=0x7f636400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:10.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.556+0000 7f635f7fe700 1 -- 172.21.15.142:0/2172240426 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f636401d070 con 0x7f6370101750 2024-03-29T04:55:10.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.556+0000 7f635f7fe700 1 -- 172.21.15.142:0/2172240426 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f636400b610 con 0x7f6370101750 2024-03-29T04:55:10.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.556+0000 7f6374a75700 1 -- 172.21.15.142:0/2172240426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637019a240 con 0x7f6370101750 2024-03-29T04:55:10.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.556+0000 7f635f7fe700 1 -- 172.21.15.142:0/2172240426 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6364022470 con 0x7f6370101750 2024-03-29T04:55:10.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.556+0000 7f6374a75700 1 -- 172.21.15.142:0/2172240426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637019a730 con 0x7f6370101750 2024-03-29T04:55:10.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.557+0000 7f6374a75700 1 -- 172.21.15.142:0/2172240426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63701091a0 con 0x7f6370101750 2024-03-29T04:55:10.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.557+0000 7f635f7fe700 1 -- 172.21.15.142:0/2172240426 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6364003710 con 0x7f6370101750 2024-03-29T04:55:10.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.558+0000 7f635f7fe700 1 --2- 172.21.15.142:0/2172240426 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f635806dd60 0x7f6358070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:10.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.558+0000 7f636e59c700 1 --2- 172.21.15.142:0/2172240426 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f635806dd60 0x7f6358070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:10.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.558+0000 7f635f7fe700 1 -- 172.21.15.142:0/2172240426 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f636408f160 con 0x7f6370101750 2024-03-29T04:55:10.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.558+0000 7f636e59c700 1 --2- 172.21.15.142:0/2172240426 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f635806dd60 0x7f6358070220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f636800f540 tx=0x7f636800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:10.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.563+0000 7f635f7fe700 1 -- 172.21.15.142:0/2172240426 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6364058580 con 0x7f6370101750 2024-03-29T04:55:10.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.768+0000 7f6374a75700 1 -- 172.21.15.142:0/2172240426 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f637019a950 con 0x7f635806dd60 2024-03-29T04:55:10.775 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:10 smithi142 ceph-mon[112475]: pgmap v204: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:10.775 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:10.775 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.uhebsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:55:10.775 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:55:10.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.779+0000 7f635f7fe700 1 -- 172.21.15.142:0/2172240426 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f637019a950 con 0x7f635806dd60 2024-03-29T04:55:10.785 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:55:10.785 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (7m) 6s ago 18m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:55:10.785 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (18m) 6s ago 18m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:55:10.785 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (17m) 43s ago 17m 8870k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (4m) 6s ago 18m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (4m) 43s ago 17m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (6m) 6s ago 18m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (12s) 6s ago 12m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (28s) 6s ago 12m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (12m) 43s ago 12m 65.2M - 18.2.1 5be31c24972a de187888f020 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (12m) 43s ago 12m 122M - 18.2.1 5be31c24972a a06260b52e7d 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (8m) 6s ago 19m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (7m) 43s ago 17m 451M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (5m) 6s ago 20m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (4m) 43s ago 17m 44.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (7m) 6s ago 18m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (7m) 43s ago 17m 16.0M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:55:10.786 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (3m) 6s ago 16m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:55:10.787 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (3m) 6s ago 16m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:55:10.787 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (2m) 6s ago 15m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:55:10.787 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (115s) 43s ago 15m 99.5M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T04:55:10.787 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (81s) 43s ago 14m 87.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T04:55:10.787 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (47s) 43s ago 14m 12.7M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T04:55:10.787 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (7m) 6s ago 18m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:55:10.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.781+0000 7f6374a75700 1 -- 172.21.15.142:0/2172240426 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f635806dd60 msgr2=0x7f6358070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:10.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.781+0000 7f6374a75700 1 --2- 172.21.15.142:0/2172240426 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f635806dd60 0x7f6358070220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f636800f540 tx=0x7f636800b040 comp rx=0 tx=0).stop 2024-03-29T04:55:10.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.781+0000 7f6374a75700 1 -- 172.21.15.142:0/2172240426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6370101750 msgr2=0x7f6370196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:10.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.781+0000 7f6374a75700 1 --2- 172.21.15.142:0/2172240426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6370101750 0x7f6370196c70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f63640059d0 tx=0x7f636400f690 comp rx=0 tx=0).stop 2024-03-29T04:55:10.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.781+0000 7f6374a75700 1 -- 172.21.15.142:0/2172240426 shutdown_connections 2024-03-29T04:55:10.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.781+0000 7f6374a75700 1 --2- 172.21.15.142:0/2172240426 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f635806dd60 0x7f6358070220 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.781+0000 7f6374a75700 1 --2- 172.21.15.142:0/2172240426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6370100da0 0x7f6370196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.781+0000 7f6374a75700 1 --2- 172.21.15.142:0/2172240426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6370101750 0x7f6370196c70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.781+0000 7f6374a75700 1 -- 172.21.15.142:0/2172240426 >> 172.21.15.142:0/2172240426 conn(0x7f63700fc630 msgr2=0x7f63700fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:10.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.781+0000 7f6374a75700 1 -- 172.21.15.142:0/2172240426 shutdown_connections 2024-03-29T04:55:10.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.781+0000 7f6374a75700 1 -- 172.21.15.142:0/2172240426 wait complete. 2024-03-29T04:55:10.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.884+0000 7f0777c2f700 1 -- 172.21.15.142:0/3219761649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0770101780 msgr2=0x7f07701057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:10.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.884+0000 7f0777c2f700 1 --2- 172.21.15.142:0/3219761649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0770101780 0x7f07701057d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0760009910 tx=0x7f0760009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:10.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.885+0000 7f0777c2f700 1 -- 172.21.15.142:0/3219761649 shutdown_connections 2024-03-29T04:55:10.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.885+0000 7f0777c2f700 1 --2- 172.21.15.142:0/3219761649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0770101780 0x7f07701057d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.885+0000 7f0777c2f700 1 --2- 172.21.15.142:0/3219761649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0770100dd0 0x7f07701011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.885+0000 7f0777c2f700 1 -- 172.21.15.142:0/3219761649 >> 172.21.15.142:0/3219761649 conn(0x7f07700fc640 msgr2=0x7f07700fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:10.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.885+0000 7f0777c2f700 1 -- 172.21.15.142:0/3219761649 shutdown_connections 2024-03-29T04:55:10.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.885+0000 7f0777c2f700 1 -- 172.21.15.142:0/3219761649 wait complete. 2024-03-29T04:55:10.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.885+0000 7f0777c2f700 1 Processor -- start 2024-03-29T04:55:10.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.885+0000 7f0777c2f700 1 -- start start 2024-03-29T04:55:10.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.885+0000 7f0777c2f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0770100dd0 0x7f077019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:10.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.885+0000 7f0777c2f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0770101780 0x7f077019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:10.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.885+0000 7f0777c2f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077019d160 con 0x7f0770100dd0 2024-03-29T04:55:10.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.885+0000 7f0777c2f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0770196610 con 0x7f0770101780 2024-03-29T04:55:10.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f07759cb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0770100dd0 0x7f077019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:10.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f07759cb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0770100dd0 0x7f077019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40538/0 (socket says 172.21.15.142:40538) 2024-03-29T04:55:10.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f07759cb700 1 -- 172.21.15.142:0/1873030826 learned_addr learned my addr 172.21.15.142:0/1873030826 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:10.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f07751ca700 1 --2- 172.21.15.142:0/1873030826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0770101780 0x7f077019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:10.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f07759cb700 1 -- 172.21.15.142:0/1873030826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0770101780 msgr2=0x7f077019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:10.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f07759cb700 1 --2- 172.21.15.142:0/1873030826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0770101780 0x7f077019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:10.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f07759cb700 1 -- 172.21.15.142:0/1873030826 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07600095f0 con 0x7f0770100dd0 2024-03-29T04:55:10.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f07759cb700 1 --2- 172.21.15.142:0/1873030826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0770100dd0 0x7f077019c590 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f076c00d900 tx=0x7f076c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:10.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f0766ffd700 1 -- 172.21.15.142:0/1873030826 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f076c0041f0 con 0x7f0770100dd0 2024-03-29T04:55:10.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f0766ffd700 1 -- 172.21.15.142:0/1873030826 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f076c004cf0 con 0x7f0770100dd0 2024-03-29T04:55:10.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f0766ffd700 1 -- 172.21.15.142:0/1873030826 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f076c00b750 con 0x7f0770100dd0 2024-03-29T04:55:10.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f0777c2f700 1 -- 172.21.15.142:0/1873030826 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07701968f0 con 0x7f0770100dd0 2024-03-29T04:55:10.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.886+0000 7f0777c2f700 1 -- 172.21.15.142:0/1873030826 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0770196e40 con 0x7f0770100dd0 2024-03-29T04:55:10.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.887+0000 7f0777c2f700 1 -- 172.21.15.142:0/1873030826 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07701091d0 con 0x7f0770100dd0 2024-03-29T04:55:10.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.888+0000 7f0766ffd700 1 -- 172.21.15.142:0/1873030826 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f076c004750 con 0x7f0770100dd0 2024-03-29T04:55:10.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.888+0000 7f0766ffd700 1 --2- 172.21.15.142:0/1873030826 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f075c06dd60 0x7f075c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:10.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.888+0000 7f0766ffd700 1 -- 172.21.15.142:0/1873030826 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f076c08da80 con 0x7f0770100dd0 2024-03-29T04:55:10.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.888+0000 7f07751ca700 1 --2- 172.21.15.142:0/1873030826 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f075c06dd60 0x7f075c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:10.896 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.889+0000 7f07751ca700 1 --2- 172.21.15.142:0/1873030826 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f075c06dd60 0x7f075c070220 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f07700fcd70 tx=0x7f076001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:10.898 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:10.893+0000 7f0766ffd700 1 -- 172.21.15.142:0/1873030826 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f076c05aa50 con 0x7f0770100dd0 2024-03-29T04:55:11.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:10 smithi152 ceph-mon[88186]: pgmap v204: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:11.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:11.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.uhebsi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:55:11.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:55:11.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.166+0000 7f0777c2f700 1 -- 172.21.15.142:0/1873030826 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0770196a80 con 0x7f0770100dd0 2024-03-29T04:55:11.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.167+0000 7f0766ffd700 1 -- 172.21.15.142:0/1873030826 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f0770196a80 con 0x7f0770100dd0 2024-03-29T04:55:11.173 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:55:11.173 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:55:11.173 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-03-29T04:55:11.174 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 12 2024-03-29T04:55:11.175 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:55:11.175 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:55:11.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.169+0000 7f0777c2f700 1 -- 172.21.15.142:0/1873030826 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f075c06dd60 msgr2=0x7f075c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:11.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.169+0000 7f0777c2f700 1 --2- 172.21.15.142:0/1873030826 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f075c06dd60 0x7f075c070220 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f07700fcd70 tx=0x7f076001a040 comp rx=0 tx=0).stop 2024-03-29T04:55:11.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.169+0000 7f0777c2f700 1 -- 172.21.15.142:0/1873030826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0770100dd0 msgr2=0x7f077019c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:11.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.169+0000 7f0777c2f700 1 --2- 172.21.15.142:0/1873030826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0770100dd0 0x7f077019c590 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f076c00d900 tx=0x7f076c00dc10 comp rx=0 tx=0).stop 2024-03-29T04:55:11.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.169+0000 7f0777c2f700 1 -- 172.21.15.142:0/1873030826 shutdown_connections 2024-03-29T04:55:11.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.169+0000 7f0777c2f700 1 --2- 172.21.15.142:0/1873030826 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f075c06dd60 0x7f075c070220 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.169+0000 7f0777c2f700 1 --2- 172.21.15.142:0/1873030826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0770100dd0 0x7f077019c590 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.169+0000 7f0777c2f700 1 --2- 172.21.15.142:0/1873030826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0770101780 0x7f077019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.169+0000 7f0777c2f700 1 -- 172.21.15.142:0/1873030826 >> 172.21.15.142:0/1873030826 conn(0x7f07700fc640 msgr2=0x7f07700fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:11.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.169+0000 7f0777c2f700 1 -- 172.21.15.142:0/1873030826 shutdown_connections 2024-03-29T04:55:11.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.169+0000 7f0777c2f700 1 -- 172.21.15.142:0/1873030826 wait complete. 2024-03-29T04:55:11.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.272+0000 7fad4b46b700 1 -- 172.21.15.142:0/1331412786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad44105a40 msgr2=0x7fad44105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:11.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.272+0000 7fad4b46b700 1 --2- 172.21.15.142:0/1331412786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad44105a40 0x7fad44105e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fad40009910 tx=0x7fad40009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:11.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.272+0000 7fad4b46b700 1 -- 172.21.15.142:0/1331412786 shutdown_connections 2024-03-29T04:55:11.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.272+0000 7fad4b46b700 1 --2- 172.21.15.142:0/1331412786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad440ffbd0 0x7fad44100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.272+0000 7fad4b46b700 1 --2- 172.21.15.142:0/1331412786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad44105a40 0x7fad44105e20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.272+0000 7fad4b46b700 1 -- 172.21.15.142:0/1331412786 >> 172.21.15.142:0/1331412786 conn(0x7fad440755a0 msgr2=0x7fad440759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:11.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.272+0000 7fad4b46b700 1 -- 172.21.15.142:0/1331412786 shutdown_connections 2024-03-29T04:55:11.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.272+0000 7fad4b46b700 1 -- 172.21.15.142:0/1331412786 wait complete. 2024-03-29T04:55:11.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.273+0000 7fad4b46b700 1 Processor -- start 2024-03-29T04:55:11.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.273+0000 7fad4b46b700 1 -- start start 2024-03-29T04:55:11.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.273+0000 7fad4b46b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad440ffbd0 0x7fad4419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:11.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.273+0000 7fad4b46b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad44105a40 0x7fad4419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:11.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.273+0000 7fad4b46b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4419d060 con 0x7fad440ffbd0 2024-03-29T04:55:11.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.273+0000 7fad4b46b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad44196520 con 0x7fad44105a40 2024-03-29T04:55:11.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.273+0000 7fad49207700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad440ffbd0 0x7fad4419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:11.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.273+0000 7fad49207700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad440ffbd0 0x7fad4419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40558/0 (socket says 172.21.15.142:40558) 2024-03-29T04:55:11.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.273+0000 7fad49207700 1 -- 172.21.15.142:0/2992987335 learned_addr learned my addr 172.21.15.142:0/2992987335 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:11.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.273+0000 7fad48a06700 1 --2- 172.21.15.142:0/2992987335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad44105a40 0x7fad4419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:11.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.274+0000 7fad49207700 1 -- 172.21.15.142:0/2992987335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad44105a40 msgr2=0x7fad4419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:11.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.274+0000 7fad49207700 1 --2- 172.21.15.142:0/2992987335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad44105a40 0x7fad4419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.274+0000 7fad49207700 1 -- 172.21.15.142:0/2992987335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad400095f0 con 0x7fad440ffbd0 2024-03-29T04:55:11.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.274+0000 7fad49207700 1 --2- 172.21.15.142:0/2992987335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad440ffbd0 0x7fad4419c440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad40006010 tx=0x7fad4000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:11.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.274+0000 7fad3a7fc700 1 -- 172.21.15.142:0/2992987335 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad4001d070 con 0x7fad440ffbd0 2024-03-29T04:55:11.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.274+0000 7fad3a7fc700 1 -- 172.21.15.142:0/2992987335 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad40005dc0 con 0x7fad440ffbd0 2024-03-29T04:55:11.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.274+0000 7fad4b46b700 1 -- 172.21.15.142:0/2992987335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad441967a0 con 0x7fad440ffbd0 2024-03-29T04:55:11.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.274+0000 7fad3a7fc700 1 -- 172.21.15.142:0/2992987335 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad40017400 con 0x7fad440ffbd0 2024-03-29T04:55:11.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.274+0000 7fad4b46b700 1 -- 172.21.15.142:0/2992987335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad44196c90 con 0x7fad440ffbd0 2024-03-29T04:55:11.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.275+0000 7fad4b46b700 1 -- 172.21.15.142:0/2992987335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad44109050 con 0x7fad440ffbd0 2024-03-29T04:55:11.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.276+0000 7fad3a7fc700 1 -- 172.21.15.142:0/2992987335 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fad40003a60 con 0x7fad440ffbd0 2024-03-29T04:55:11.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.276+0000 7fad3a7fc700 1 --2- 172.21.15.142:0/2992987335 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad3006dd60 0x7fad30070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:11.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.276+0000 7fad48a06700 1 --2- 172.21.15.142:0/2992987335 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad3006dd60 0x7fad30070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:11.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.276+0000 7fad3a7fc700 1 -- 172.21.15.142:0/2992987335 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fad4008f0f0 con 0x7fad440ffbd0 2024-03-29T04:55:11.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.277+0000 7fad48a06700 1 --2- 172.21.15.142:0/2992987335 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad3006dd60 0x7fad30070220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad44197d10 tx=0x7fad3400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:11.287 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.281+0000 7fad3a7fc700 1 -- 172.21.15.142:0/2992987335 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fad4005c040 con 0x7fad440ffbd0 2024-03-29T04:55:11.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.520+0000 7fad4b46b700 1 -- 172.21.15.142:0/2992987335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fad44196f70 con 0x7fad440ffbd0 2024-03-29T04:55:11.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.521+0000 7fad3a7fc700 1 -- 172.21.15.142:0/2992987335 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1875 (secure 0 0 0) 0x7fad4005bbd0 con 0x7fad440ffbd0 2024-03-29T04:55:11.527 INFO:teuthology.orchestra.run.smithi142.stdout:e28 2024-03-29T04:55:11.527 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:55:11.527 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:55:11.527 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:55:11.528 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:55:11.529 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:55:11.530 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:55:11.530 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:55:11.530 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T04:55:11.530 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:55:11.530 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:55:11.530 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:55:11.530 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44255} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/869112786,v1:172.21.15.152:6827/869112786] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:55:11.530 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/924845457,v1:172.21.15.152:6825/924845457] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T04:55:11.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.523+0000 7fad4b46b700 1 -- 172.21.15.142:0/2992987335 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad3006dd60 msgr2=0x7fad30070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:11.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.523+0000 7fad4b46b700 1 --2- 172.21.15.142:0/2992987335 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad3006dd60 0x7fad30070220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad44197d10 tx=0x7fad3400b410 comp rx=0 tx=0).stop 2024-03-29T04:55:11.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.523+0000 7fad4b46b700 1 -- 172.21.15.142:0/2992987335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad440ffbd0 msgr2=0x7fad4419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:11.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.523+0000 7fad4b46b700 1 --2- 172.21.15.142:0/2992987335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad440ffbd0 0x7fad4419c440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad40006010 tx=0x7fad4000bd70 comp rx=0 tx=0).stop 2024-03-29T04:55:11.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.523+0000 7fad4b46b700 1 -- 172.21.15.142:0/2992987335 shutdown_connections 2024-03-29T04:55:11.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.523+0000 7fad4b46b700 1 --2- 172.21.15.142:0/2992987335 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad3006dd60 0x7fad30070220 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.523+0000 7fad4b46b700 1 --2- 172.21.15.142:0/2992987335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad440ffbd0 0x7fad4419c440 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.523+0000 7fad4b46b700 1 --2- 172.21.15.142:0/2992987335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad44105a40 0x7fad4419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.523+0000 7fad4b46b700 1 -- 172.21.15.142:0/2992987335 >> 172.21.15.142:0/2992987335 conn(0x7fad440755a0 msgr2=0x7fad440685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:11.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.523+0000 7fad4b46b700 1 -- 172.21.15.142:0/2992987335 shutdown_connections 2024-03-29T04:55:11.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.523+0000 7fad4b46b700 1 -- 172.21.15.142:0/2992987335 wait complete. 2024-03-29T04:55:11.532 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 28 2024-03-29T04:55:11.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.629+0000 7f1e04a6c700 1 -- 172.21.15.142:0/499537291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e000ffa40 msgr2=0x7f1e000ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:11.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.629+0000 7f1e04a6c700 1 --2- 172.21.15.142:0/499537291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e000ffa40 0x7f1e000ffec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009910 tx=0x7f1df4009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:11.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.629+0000 7f1e04a6c700 1 -- 172.21.15.142:0/499537291 shutdown_connections 2024-03-29T04:55:11.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.629+0000 7f1e04a6c700 1 --2- 172.21.15.142:0/499537291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e000ffa40 0x7f1e000ffec0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.629+0000 7f1e04a6c700 1 --2- 172.21.15.142:0/499537291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e00105a40 0x7f1e00105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.629+0000 7f1e04a6c700 1 -- 172.21.15.142:0/499537291 >> 172.21.15.142:0/499537291 conn(0x7f1e000fb570 msgr2=0x7f1e000fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:11.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.629+0000 7f1e04a6c700 1 -- 172.21.15.142:0/499537291 shutdown_connections 2024-03-29T04:55:11.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.629+0000 7f1e04a6c700 1 -- 172.21.15.142:0/499537291 wait complete. 2024-03-29T04:55:11.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1e04a6c700 1 Processor -- start 2024-03-29T04:55:11.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1e04a6c700 1 -- start start 2024-03-29T04:55:11.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1e04a6c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e000ffa40 0x7f1e0019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:11.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1e04a6c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00105a40 0x7f1e0019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:11.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1e04a6c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0019f2c0 con 0x7f1e00105a40 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1e04a6c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00198770 con 0x7f1e000ffa40 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1dfdd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00105a40 0x7f1e0019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1dfdd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00105a40 0x7f1e0019ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40572/0 (socket says 172.21.15.142:40572) 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1dfdd9b700 1 -- 172.21.15.142:0/4110372120 learned_addr learned my addr 172.21.15.142:0/4110372120 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1dfe59c700 1 --2- 172.21.15.142:0/4110372120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e000ffa40 0x7f1e0019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1dfdd9b700 1 -- 172.21.15.142:0/4110372120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e000ffa40 msgr2=0x7f1e0019e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1dfdd9b700 1 --2- 172.21.15.142:0/4110372120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e000ffa40 0x7f1e0019e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.630+0000 7f1dfdd9b700 1 -- 172.21.15.142:0/4110372120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df40095f0 con 0x7f1e00105a40 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.631+0000 7f1dfdd9b700 1 --2- 172.21.15.142:0/4110372120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00105a40 0x7f1e0019ec30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1df4005f50 tx=0x7f1df400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.631+0000 7f1def7fe700 1 -- 172.21.15.142:0/4110372120 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1df401d070 con 0x7f1e00105a40 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.631+0000 7f1def7fe700 1 -- 172.21.15.142:0/4110372120 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1df400b610 con 0x7f1e00105a40 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.631+0000 7f1e04a6c700 1 -- 172.21.15.142:0/4110372120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e001989f0 con 0x7f1e00105a40 2024-03-29T04:55:11.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.631+0000 7f1def7fe700 1 -- 172.21.15.142:0/4110372120 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1df4022470 con 0x7f1e00105a40 2024-03-29T04:55:11.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.631+0000 7f1e04a6c700 1 -- 172.21.15.142:0/4110372120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e00198ee0 con 0x7f1e00105a40 2024-03-29T04:55:11.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.632+0000 7f1e04a6c700 1 -- 172.21.15.142:0/4110372120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0004ea90 con 0x7f1e00105a40 2024-03-29T04:55:11.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.632+0000 7f1def7fe700 1 -- 172.21.15.142:0/4110372120 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f1df4003780 con 0x7f1e00105a40 2024-03-29T04:55:11.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.633+0000 7f1def7fe700 1 --2- 172.21.15.142:0/4110372120 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1de806dd60 0x7f1de8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:11.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.633+0000 7f1def7fe700 1 -- 172.21.15.142:0/4110372120 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1df408f040 con 0x7f1e00105a40 2024-03-29T04:55:11.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.633+0000 7f1dfe59c700 1 --2- 172.21.15.142:0/4110372120 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1de806dd60 0x7f1de8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:11.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.633+0000 7f1dfe59c700 1 --2- 172.21.15.142:0/4110372120 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1de806dd60 0x7f1de8070220 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009c60 tx=0x7f1df0009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:11.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.637+0000 7f1def7fe700 1 -- 172.21.15.142:0/4110372120 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1df405bf90 con 0x7f1e00105a40 2024-03-29T04:55:11.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.855+0000 7f1e04a6c700 1 -- 172.21.15.142:0/4110372120 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e001992b0 con 0x7f1de806dd60 2024-03-29T04:55:11.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:11 smithi142 ceph-mon[112475]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:11.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:11 smithi142 ceph-mon[112475]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:11.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:11 smithi142 ceph-mon[112475]: Upgrade: Updating mds.cephfs.smithi152.uhebsi 2024-03-29T04:55:11.861 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:11 smithi142 ceph-mon[112475]: Deploying daemon mds.cephfs.smithi152.uhebsi on smithi152 2024-03-29T04:55:11.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:11 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1873030826' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:11.862 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:11 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2992987335' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:55:11.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.856+0000 7f1def7fe700 1 -- 172.21.15.142:0/4110372120 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1e001992b0 con 0x7f1de806dd60 2024-03-29T04:55:11.862 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:55:11.862 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:55:11.862 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:55:11.863 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:55:11.863 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:55:11.863 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:55:11.863 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:55:11.863 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T04:55:11.863 INFO:teuthology.orchestra.run.smithi142.stdout: "mon" 2024-03-29T04:55:11.863 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:55:11.863 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "14/23 daemons upgraded", 2024-03-29T04:55:11.863 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T04:55:11.863 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:55:11.863 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:55:11.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.859+0000 7f1e04a6c700 1 -- 172.21.15.142:0/4110372120 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1de806dd60 msgr2=0x7f1de8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:11.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.859+0000 7f1e04a6c700 1 --2- 172.21.15.142:0/4110372120 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1de806dd60 0x7f1de8070220 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009c60 tx=0x7f1df0009260 comp rx=0 tx=0).stop 2024-03-29T04:55:11.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.859+0000 7f1e04a6c700 1 -- 172.21.15.142:0/4110372120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00105a40 msgr2=0x7f1e0019ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:11.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.859+0000 7f1e04a6c700 1 --2- 172.21.15.142:0/4110372120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00105a40 0x7f1e0019ec30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1df4005f50 tx=0x7f1df400f690 comp rx=0 tx=0).stop 2024-03-29T04:55:11.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.859+0000 7f1e04a6c700 1 -- 172.21.15.142:0/4110372120 shutdown_connections 2024-03-29T04:55:11.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.859+0000 7f1e04a6c700 1 --2- 172.21.15.142:0/4110372120 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1de806dd60 0x7f1de8070220 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.859+0000 7f1e04a6c700 1 --2- 172.21.15.142:0/4110372120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e000ffa40 0x7f1e0019e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.859+0000 7f1e04a6c700 1 --2- 172.21.15.142:0/4110372120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00105a40 0x7f1e0019ec30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.859+0000 7f1e04a6c700 1 -- 172.21.15.142:0/4110372120 >> 172.21.15.142:0/4110372120 conn(0x7f1e000fb570 msgr2=0x7f1e00068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:11.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.859+0000 7f1e04a6c700 1 -- 172.21.15.142:0/4110372120 shutdown_connections 2024-03-29T04:55:11.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.859+0000 7f1e04a6c700 1 -- 172.21.15.142:0/4110372120 wait complete. 2024-03-29T04:55:11.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.963+0000 7f136ccda700 1 -- 172.21.15.142:0/160905071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1368105a40 msgr2=0x7f1368105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:11.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.963+0000 7f136ccda700 1 --2- 172.21.15.142:0/160905071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1368105a40 0x7f1368105e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f135c009910 tx=0x7f135c009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:11.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.963+0000 7f136ccda700 1 -- 172.21.15.142:0/160905071 shutdown_connections 2024-03-29T04:55:11.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.963+0000 7f136ccda700 1 --2- 172.21.15.142:0/160905071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13680ffa40 0x7f13680ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.963+0000 7f136ccda700 1 --2- 172.21.15.142:0/160905071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1368105a40 0x7f1368105e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.963+0000 7f136ccda700 1 -- 172.21.15.142:0/160905071 >> 172.21.15.142:0/160905071 conn(0x7f13680fb570 msgr2=0x7f13680fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:11.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.963+0000 7f136ccda700 1 -- 172.21.15.142:0/160905071 shutdown_connections 2024-03-29T04:55:11.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.963+0000 7f136ccda700 1 -- 172.21.15.142:0/160905071 wait complete. 2024-03-29T04:55:11.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.964+0000 7f136ccda700 1 Processor -- start 2024-03-29T04:55:11.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.964+0000 7f136ccda700 1 -- start start 2024-03-29T04:55:11.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.964+0000 7f136ccda700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13680ffa40 0x7f136819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:11.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.964+0000 7f136ccda700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1368105a40 0x7f136819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:11.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.964+0000 7f136ccda700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819d110 con 0x7f13680ffa40 2024-03-29T04:55:11.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.964+0000 7f136ccda700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13681965c0 con 0x7f1368105a40 2024-03-29T04:55:11.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f136659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13680ffa40 0x7f136819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:11.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f136659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13680ffa40 0x7f136819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40598/0 (socket says 172.21.15.142:40598) 2024-03-29T04:55:11.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f136659c700 1 -- 172.21.15.142:0/1083532317 learned_addr learned my addr 172.21.15.142:0/1083532317 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:11.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f1365d9b700 1 --2- 172.21.15.142:0/1083532317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1368105a40 0x7f136819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:11.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f136659c700 1 -- 172.21.15.142:0/1083532317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1368105a40 msgr2=0x7f136819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:11.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f136659c700 1 --2- 172.21.15.142:0/1083532317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1368105a40 0x7f136819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:11.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f136659c700 1 -- 172.21.15.142:0/1083532317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c0095f0 con 0x7f13680ffa40 2024-03-29T04:55:11.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f136659c700 1 --2- 172.21.15.142:0/1083532317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13680ffa40 0x7f136819c540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f135c006010 tx=0x7f135c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:11.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f13577fe700 1 -- 172.21.15.142:0/1083532317 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f135c01d070 con 0x7f13680ffa40 2024-03-29T04:55:11.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f13577fe700 1 -- 172.21.15.142:0/1083532317 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f135c005dc0 con 0x7f13680ffa40 2024-03-29T04:55:11.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f136ccda700 1 -- 172.21.15.142:0/1083532317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1368196840 con 0x7f13680ffa40 2024-03-29T04:55:11.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f13577fe700 1 -- 172.21.15.142:0/1083532317 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f135c017400 con 0x7f13680ffa40 2024-03-29T04:55:11.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.965+0000 7f136ccda700 1 -- 172.21.15.142:0/1083532317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1368196d30 con 0x7f13680ffa40 2024-03-29T04:55:11.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.966+0000 7f136ccda700 1 -- 172.21.15.142:0/1083532317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136804ea90 con 0x7f13680ffa40 2024-03-29T04:55:11.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.967+0000 7f13577fe700 1 -- 172.21.15.142:0/1083532317 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f135c003a60 con 0x7f13680ffa40 2024-03-29T04:55:11.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.967+0000 7f13577fe700 1 --2- 172.21.15.142:0/1083532317 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f135006dd60 0x7f1350070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:11.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.967+0000 7f1365d9b700 1 --2- 172.21.15.142:0/1083532317 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f135006dd60 0x7f1350070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:11.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.967+0000 7f13577fe700 1 -- 172.21.15.142:0/1083532317 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f135c08f050 con 0x7f13680ffa40 2024-03-29T04:55:11.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.968+0000 7f1365d9b700 1 --2- 172.21.15.142:0/1083532317 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f135006dd60 0x7f1350070220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13680fbb50 tx=0x7f13580073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:11.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:11.972+0000 7f13577fe700 1 -- 172.21.15.142:0/1083532317 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f135c05c020 con 0x7f13680ffa40 2024-03-29T04:55:12.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:11 smithi152 ceph-mon[88186]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:12.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:11 smithi152 ceph-mon[88186]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:12.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:11 smithi152 ceph-mon[88186]: Upgrade: Updating mds.cephfs.smithi152.uhebsi 2024-03-29T04:55:12.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:11 smithi152 ceph-mon[88186]: Deploying daemon mds.cephfs.smithi152.uhebsi on smithi152 2024-03-29T04:55:12.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:11 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1873030826' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:12.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:11 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2992987335' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:55:12.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.251+0000 7f136ccda700 1 -- 172.21.15.142:0/1083532317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1368197240 con 0x7f13680ffa40 2024-03-29T04:55:12.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.251+0000 7f13577fe700 1 -- 172.21.15.142:0/1083532317 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f135c05bbb0 con 0x7f13680ffa40 2024-03-29T04:55:12.257 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T04:55:12.257 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T04:55:12.258 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T04:55:12.258 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T04:55:12.258 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T04:55:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.253+0000 7f136ccda700 1 -- 172.21.15.142:0/1083532317 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f135006dd60 msgr2=0x7f1350070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.253+0000 7f136ccda700 1 --2- 172.21.15.142:0/1083532317 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f135006dd60 0x7f1350070220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13680fbb50 tx=0x7f13580073d0 comp rx=0 tx=0).stop 2024-03-29T04:55:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.253+0000 7f136ccda700 1 -- 172.21.15.142:0/1083532317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13680ffa40 msgr2=0x7f136819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.253+0000 7f136ccda700 1 --2- 172.21.15.142:0/1083532317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13680ffa40 0x7f136819c540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f135c006010 tx=0x7f135c00bd70 comp rx=0 tx=0).stop 2024-03-29T04:55:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.254+0000 7f136ccda700 1 -- 172.21.15.142:0/1083532317 shutdown_connections 2024-03-29T04:55:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.254+0000 7f136ccda700 1 --2- 172.21.15.142:0/1083532317 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f135006dd60 0x7f1350070220 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.254+0000 7f136ccda700 1 --2- 172.21.15.142:0/1083532317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13680ffa40 0x7f136819c540 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.254+0000 7f136ccda700 1 --2- 172.21.15.142:0/1083532317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1368105a40 0x7f136819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.254+0000 7f136ccda700 1 -- 172.21.15.142:0/1083532317 >> 172.21.15.142:0/1083532317 conn(0x7f13680fb570 msgr2=0x7f1368103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.254+0000 7f136ccda700 1 -- 172.21.15.142:0/1083532317 shutdown_connections 2024-03-29T04:55:12.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:12.254+0000 7f136ccda700 1 -- 172.21.15.142:0/1083532317 wait complete. 2024-03-29T04:55:12.914 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:12 smithi152 ceph-mon[88186]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:12.915 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:12 smithi152 ceph-mon[88186]: pgmap v205: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:12.915 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:12 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1083532317' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:55:13.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:12 smithi142 ceph-mon[112475]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:13.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:12 smithi142 ceph-mon[112475]: pgmap v205: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:13.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:12 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1083532317' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:55:14.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:13 smithi152 ceph-mon[88186]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:14.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:13 smithi152 ceph-mon[88186]: osdmap e98: 6 total, 6 up, 6 in 2024-03-29T04:55:14.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:13 smithi152 ceph-mon[88186]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T04:55:14.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:13 smithi142 ceph-mon[112475]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:14.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:13 smithi142 ceph-mon[112475]: osdmap e98: 6 total, 6 up, 6 in 2024-03-29T04:55:14.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:13 smithi142 ceph-mon[112475]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T04:55:15.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:14 smithi152 ceph-mon[88186]: pgmap v207: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:15.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:55:15.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:55:15.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:14 smithi142 ceph-mon[112475]: pgmap v207: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:15.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:55:15.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:55:16.956 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:16 smithi152 ceph-mon[88186]: pgmap v208: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:17.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:16 smithi142 ceph-mon[112475]: pgmap v208: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:19.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:18 smithi152 ceph-mon[88186]: pgmap v209: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:19.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:18 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:19.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:18 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:19.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:18 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:55:19.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:18 smithi142 ceph-mon[112475]: pgmap v209: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:19.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:18 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:19.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:18 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:19.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:18 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:55:19.824 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:19 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] up:boot 2024-03-29T04:55:19.824 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:19 smithi152 ceph-mon[88186]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T04:55:19.824 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:19 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:55:20.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:19 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] up:boot 2024-03-29T04:55:20.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:19 smithi142 ceph-mon[112475]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T04:55:20.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:19 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T04:55:20.847 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:20 smithi152 ceph-mon[88186]: pgmap v210: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:21.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:20 smithi142 ceph-mon[112475]: pgmap v210: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:22.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:22 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:22.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:22 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:22.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:22 smithi152 ceph-mon[88186]: pgmap v211: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:22.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:22 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:22.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:22 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:22.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:22 smithi142 ceph-mon[112475]: pgmap v211: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:23.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:23 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:23.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:23 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:23.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:23 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:23.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:23 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:24.727 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:24 smithi152 ceph-mon[88186]: pgmap v212: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:24.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:24 smithi142 ceph-mon[112475]: pgmap v212: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:26.960 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:26 smithi152 ceph-mon[88186]: pgmap v213: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:26.960 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:26.960 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:26.960 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:55:26.960 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:55:26.960 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:26.961 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:55:26.961 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:26.961 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:26.961 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:26.961 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:26 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:27.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:26 smithi142 ceph-mon[112475]: pgmap v213: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:27.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:27.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:27.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:55:27.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T04:55:27.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:27.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:55:27.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:27.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:27.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:27.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:26 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:28.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:28 smithi152 ceph-mon[88186]: pgmap v214: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:28.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:28 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:28.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:28 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.hiilum", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:55:28.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:28 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:55:28.930 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:28 smithi142 ceph-mon[112475]: pgmap v214: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:28.931 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:28 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:28.931 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:28 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi152.hiilum", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T04:55:28.931 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:28 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T04:55:29.847 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:29 smithi152 ceph-mon[88186]: Upgrade: Updating mds.cephfs.smithi152.hiilum 2024-03-29T04:55:29.847 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:29 smithi152 ceph-mon[88186]: Deploying daemon mds.cephfs.smithi152.hiilum on smithi152 2024-03-29T04:55:29.848 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:55:29.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:29 smithi142 ceph-mon[112475]: Upgrade: Updating mds.cephfs.smithi152.hiilum 2024-03-29T04:55:29.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:29 smithi142 ceph-mon[112475]: Deploying daemon mds.cephfs.smithi152.hiilum on smithi152 2024-03-29T04:55:29.950 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:55:31.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:30 smithi152 ceph-mon[88186]: pgmap v215: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:31.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:30 smithi142 ceph-mon[112475]: pgmap v215: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:31.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:31 smithi142 ceph-mon[112475]: osdmap e99: 6 total, 6 up, 6 in 2024-03-29T04:55:31.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:31 smithi142 ceph-mon[112475]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T04:55:32.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:31 smithi152 ceph-mon[88186]: osdmap e99: 6 total, 6 up, 6 in 2024-03-29T04:55:32.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:31 smithi152 ceph-mon[88186]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T04:55:33.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:32 smithi152 ceph-mon[88186]: pgmap v217: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:33.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:32 smithi142 ceph-mon[112475]: pgmap v217: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:35.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:34 smithi152 ceph-mon[88186]: pgmap v218: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:35.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:34 smithi142 ceph-mon[112475]: pgmap v218: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:35.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:35 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] up:boot 2024-03-29T04:55:35.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:35 smithi142 ceph-mon[112475]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T04:55:35.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T04:55:35.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:35.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:35.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:55:36.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:35 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] up:boot 2024-03-29T04:55:36.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:35 smithi152 ceph-mon[88186]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T04:55:36.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T04:55:36.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:36.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:36.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T04:55:36.943 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:36 smithi152 ceph-mon[88186]: pgmap v219: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:37.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:36 smithi142 ceph-mon[112475]: pgmap v219: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:39.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:38 smithi152 ceph-mon[88186]: pgmap v220: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:39.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:38 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:39.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:38 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:39.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:38 smithi142 ceph-mon[112475]: pgmap v220: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:39.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:38 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:39.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:38 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:40.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:39 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:40.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:39 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:40.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:39 smithi142 ceph-mon[112475]: pgmap v221: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:40.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:39 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:40.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:39 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T04:55:40.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:39 smithi152 ceph-mon[88186]: pgmap v221: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:42.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.372+0000 7f6feb25e700 1 -- 172.21.15.142:0/1675635496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe4100dc0 msgr2=0x7f6fe41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:42.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.372+0000 7f6feb25e700 1 --2- 172.21.15.142:0/1675635496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe4100dc0 0x7f6fe41011a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009910 tx=0x7f6fd8009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:42.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.373+0000 7f6feb25e700 1 -- 172.21.15.142:0/1675635496 shutdown_connections 2024-03-29T04:55:42.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.373+0000 7f6feb25e700 1 --2- 172.21.15.142:0/1675635496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fe4101770 0x7f6fe41057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:42.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.373+0000 7f6feb25e700 1 --2- 172.21.15.142:0/1675635496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe4100dc0 0x7f6fe41011a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:42.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.373+0000 7f6feb25e700 1 -- 172.21.15.142:0/1675635496 >> 172.21.15.142:0/1675635496 conn(0x7f6fe40fc650 msgr2=0x7f6fe40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:42.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.373+0000 7f6feb25e700 1 -- 172.21.15.142:0/1675635496 shutdown_connections 2024-03-29T04:55:42.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.373+0000 7f6feb25e700 1 -- 172.21.15.142:0/1675635496 wait complete. 2024-03-29T04:55:42.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.373+0000 7f6feb25e700 1 Processor -- start 2024-03-29T04:55:42.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.373+0000 7f6feb25e700 1 -- start start 2024-03-29T04:55:42.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6feb25e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fe4100dc0 0x7f6fe4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:42.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6feb25e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe4101770 0x7f6fe40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:42.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6feb25e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe4073b10 con 0x7f6fe4101770 2024-03-29T04:55:42.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6feb25e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe4073c50 con 0x7f6fe4100dc0 2024-03-29T04:55:42.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6fe3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe4101770 0x7f6fe40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:42.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6fe3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe4101770 0x7f6fe40735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39742/0 (socket says 172.21.15.142:39742) 2024-03-29T04:55:42.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6fe3fff700 1 -- 172.21.15.142:0/2176786325 learned_addr learned my addr 172.21.15.142:0/2176786325 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:42.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6fe8ffa700 1 --2- 172.21.15.142:0/2176786325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fe4100dc0 0x7f6fe4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:42.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6fe3fff700 1 -- 172.21.15.142:0/2176786325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fe4100dc0 msgr2=0x7f6fe4073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:42.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6fe3fff700 1 --2- 172.21.15.142:0/2176786325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fe4100dc0 0x7f6fe4073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:42.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6fe3fff700 1 -- 172.21.15.142:0/2176786325 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd80095f0 con 0x7f6fe4101770 2024-03-29T04:55:42.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6fe3fff700 1 --2- 172.21.15.142:0/2176786325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe4101770 0x7f6fe40735d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000d900 tx=0x7f6fd000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:42.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6fe1ffb700 1 -- 172.21.15.142:0/2176786325 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fd0009900 con 0x7f6fe4101770 2024-03-29T04:55:42.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6fe1ffb700 1 -- 172.21.15.142:0/2176786325 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fd0010460 con 0x7f6fe4101770 2024-03-29T04:55:42.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6feb25e700 1 -- 172.21.15.142:0/2176786325 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe41a2900 con 0x7f6fe4101770 2024-03-29T04:55:42.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.374+0000 7f6fe1ffb700 1 -- 172.21.15.142:0/2176786325 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fd000f5f0 con 0x7f6fe4101770 2024-03-29T04:55:42.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.375+0000 7f6feb25e700 1 -- 172.21.15.142:0/2176786325 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe41a2d10 con 0x7f6fe4101770 2024-03-29T04:55:42.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.376+0000 7f6feb25e700 1 -- 172.21.15.142:0/2176786325 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe4109160 con 0x7f6fe4101770 2024-03-29T04:55:42.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.376+0000 7f6fe1ffb700 1 -- 172.21.15.142:0/2176786325 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6fd00105d0 con 0x7f6fe4101770 2024-03-29T04:55:42.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.376+0000 7f6fe1ffb700 1 --2- 172.21.15.142:0/2176786325 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6fd406dd60 0x7f6fd4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:42.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.377+0000 7f6fe1ffb700 1 -- 172.21.15.142:0/2176786325 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6fd008d870 con 0x7f6fe4101770 2024-03-29T04:55:42.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.377+0000 7f6fe8ffa700 1 --2- 172.21.15.142:0/2176786325 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6fd406dd60 0x7f6fd4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:42.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.377+0000 7f6fe8ffa700 1 --2- 172.21.15.142:0/2176786325 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6fd406dd60 0x7f6fd4070220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800b5c0 tx=0x7f6fd801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:42.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.381+0000 7f6fe1ffb700 1 -- 172.21.15.142:0/2176786325 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6fd0056c90 con 0x7f6fe4101770 2024-03-29T04:55:42.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.594+0000 7f6feb25e700 1 -- 172.21.15.142:0/2176786325 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fe402cda0 con 0x7f6fd406dd60 2024-03-29T04:55:42.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.596+0000 7f6fe1ffb700 1 -- 172.21.15.142:0/2176786325 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6fe402cda0 con 0x7f6fd406dd60 2024-03-29T04:55:42.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.598+0000 7f6feb25e700 1 -- 172.21.15.142:0/2176786325 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6fd406dd60 msgr2=0x7f6fd4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:42.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.598+0000 7f6feb25e700 1 --2- 172.21.15.142:0/2176786325 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6fd406dd60 0x7f6fd4070220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800b5c0 tx=0x7f6fd801a040 comp rx=0 tx=0).stop 2024-03-29T04:55:42.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.598+0000 7f6feb25e700 1 -- 172.21.15.142:0/2176786325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe4101770 msgr2=0x7f6fe40735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:42.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.598+0000 7f6feb25e700 1 --2- 172.21.15.142:0/2176786325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe4101770 0x7f6fe40735d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000d900 tx=0x7f6fd000dc10 comp rx=0 tx=0).stop 2024-03-29T04:55:42.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.598+0000 7f6feb25e700 1 -- 172.21.15.142:0/2176786325 shutdown_connections 2024-03-29T04:55:42.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.598+0000 7f6feb25e700 1 --2- 172.21.15.142:0/2176786325 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6fd406dd60 0x7f6fd4070220 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:42.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.598+0000 7f6feb25e700 1 --2- 172.21.15.142:0/2176786325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fe4100dc0 0x7f6fe4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:42.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.598+0000 7f6feb25e700 1 --2- 172.21.15.142:0/2176786325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fe4101770 0x7f6fe40735d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:42.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.598+0000 7f6feb25e700 1 -- 172.21.15.142:0/2176786325 >> 172.21.15.142:0/2176786325 conn(0x7f6fe40fc650 msgr2=0x7f6fe40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:42.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.598+0000 7f6feb25e700 1 -- 172.21.15.142:0/2176786325 shutdown_connections 2024-03-29T04:55:42.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.598+0000 7f6feb25e700 1 -- 172.21.15.142:0/2176786325 wait complete. 2024-03-29T04:55:42.615 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:55:42.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.705+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/3906355235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e780ffa40 msgr2=0x7f1e780ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:42.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.705+0000 7f1e7eb8e700 1 --2- 172.21.15.142:0/3906355235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e780ffa40 0x7f1e780ffec0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009910 tx=0x7f1e64009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.705+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/3906355235 shutdown_connections 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.705+0000 7f1e7eb8e700 1 --2- 172.21.15.142:0/3906355235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e780ffa40 0x7f1e780ffec0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.705+0000 7f1e7eb8e700 1 --2- 172.21.15.142:0/3906355235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e78105a40 0x7f1e78105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.705+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/3906355235 >> 172.21.15.142:0/3906355235 conn(0x7f1e780fb570 msgr2=0x7f1e780fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.705+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/3906355235 shutdown_connections 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.705+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/3906355235 wait complete. 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.706+0000 7f1e7eb8e700 1 Processor -- start 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.706+0000 7f1e7eb8e700 1 -- start start 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.706+0000 7f1e7eb8e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e780ffa40 0x7f1e78196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.706+0000 7f1e7eb8e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e78105a40 0x7f1e78196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.706+0000 7f1e7eb8e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e781972e0 con 0x7f1e78105a40 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.706+0000 7f1e7eb8e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7819a010 con 0x7f1e780ffa40 2024-03-29T04:55:42.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.706+0000 7f1e77fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e78105a40 0x7f1e78196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:42.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.706+0000 7f1e77fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e78105a40 0x7f1e78196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39764/0 (socket says 172.21.15.142:39764) 2024-03-29T04:55:42.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.706+0000 7f1e77fff700 1 -- 172.21.15.142:0/565836972 learned_addr learned my addr 172.21.15.142:0/565836972 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:42.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.706+0000 7f1e7c92a700 1 --2- 172.21.15.142:0/565836972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e780ffa40 0x7f1e78196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:42.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.707+0000 7f1e77fff700 1 -- 172.21.15.142:0/565836972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e780ffa40 msgr2=0x7f1e78196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:42.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.707+0000 7f1e77fff700 1 --2- 172.21.15.142:0/565836972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e780ffa40 0x7f1e78196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:42.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.707+0000 7f1e77fff700 1 -- 172.21.15.142:0/565836972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e640095f0 con 0x7f1e78105a40 2024-03-29T04:55:42.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.707+0000 7f1e77fff700 1 --2- 172.21.15.142:0/565836972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e78105a40 0x7f1e78196c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1e64006010 tx=0x7f1e6400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:42.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.707+0000 7f1e75ffb700 1 -- 172.21.15.142:0/565836972 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e6401d070 con 0x7f1e78105a40 2024-03-29T04:55:42.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.707+0000 7f1e75ffb700 1 -- 172.21.15.142:0/565836972 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e6400f460 con 0x7f1e78105a40 2024-03-29T04:55:42.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.707+0000 7f1e75ffb700 1 -- 172.21.15.142:0/565836972 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e64017610 con 0x7f1e78105a40 2024-03-29T04:55:42.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.707+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/565836972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7819a290 con 0x7f1e78105a40 2024-03-29T04:55:42.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.707+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/565836972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7819a780 con 0x7f1e78105a40 2024-03-29T04:55:42.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.708+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/565836972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e78108fe0 con 0x7f1e78105a40 2024-03-29T04:55:42.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.709+0000 7f1e75ffb700 1 -- 172.21.15.142:0/565836972 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f1e6400f5d0 con 0x7f1e78105a40 2024-03-29T04:55:42.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.709+0000 7f1e75ffb700 1 --2- 172.21.15.142:0/565836972 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e6806dd60 0x7f1e68070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:42.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.709+0000 7f1e75ffb700 1 -- 172.21.15.142:0/565836972 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1e6408f090 con 0x7f1e78105a40 2024-03-29T04:55:42.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.709+0000 7f1e7c92a700 1 --2- 172.21.15.142:0/565836972 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e6806dd60 0x7f1e68070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:42.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.709+0000 7f1e7c92a700 1 --2- 172.21.15.142:0/565836972 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e6806dd60 0x7f1e68070220 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c005950 tx=0x7f1e6c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:42.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.714+0000 7f1e75ffb700 1 -- 172.21.15.142:0/565836972 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1e64058530 con 0x7f1e78105a40 2024-03-29T04:55:42.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.925+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/565836972 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e7819a9f0 con 0x7f1e6806dd60 2024-03-29T04:55:42.931 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:42 smithi142 ceph-mon[112475]: pgmap v222: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:42.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.926+0000 7f1e75ffb700 1 -- 172.21.15.142:0/565836972 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1e7819a9f0 con 0x7f1e6806dd60 2024-03-29T04:55:42.934 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.928+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/565836972 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e6806dd60 msgr2=0x7f1e68070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:42.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.928+0000 7f1e7eb8e700 1 --2- 172.21.15.142:0/565836972 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e6806dd60 0x7f1e68070220 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c005950 tx=0x7f1e6c00b410 comp rx=0 tx=0).stop 2024-03-29T04:55:42.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.928+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/565836972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e78105a40 msgr2=0x7f1e78196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:42.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.928+0000 7f1e7eb8e700 1 --2- 172.21.15.142:0/565836972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e78105a40 0x7f1e78196c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1e64006010 tx=0x7f1e6400bd70 comp rx=0 tx=0).stop 2024-03-29T04:55:42.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.928+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/565836972 shutdown_connections 2024-03-29T04:55:42.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.928+0000 7f1e7eb8e700 1 --2- 172.21.15.142:0/565836972 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e6806dd60 0x7f1e68070220 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:42.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.928+0000 7f1e7eb8e700 1 --2- 172.21.15.142:0/565836972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e780ffa40 0x7f1e78196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:42.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.928+0000 7f1e7eb8e700 1 --2- 172.21.15.142:0/565836972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e78105a40 0x7f1e78196c50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:42.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.928+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/565836972 >> 172.21.15.142:0/565836972 conn(0x7f1e780fb570 msgr2=0x7f1e78069070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:42.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.928+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/565836972 shutdown_connections 2024-03-29T04:55:42.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:42.928+0000 7f1e7eb8e700 1 -- 172.21.15.142:0/565836972 wait complete. 2024-03-29T04:55:43.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.033+0000 7f45fbbd6700 1 -- 172.21.15.142:0/3990452132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f40ff530 msgr2=0x7f45f40ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:43.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.033+0000 7f45fbbd6700 1 --2- 172.21.15.142:0/3990452132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f40ff530 0x7f45f40ff910 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009910 tx=0x7f45f0009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:43.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.033+0000 7f45fbbd6700 1 -- 172.21.15.142:0/3990452132 shutdown_connections 2024-03-29T04:55:43.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.033+0000 7f45fbbd6700 1 --2- 172.21.15.142:0/3990452132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45f40ffee0 0x7f45f4105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.033+0000 7f45fbbd6700 1 --2- 172.21.15.142:0/3990452132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f40ff530 0x7f45f40ff910 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.033+0000 7f45fbbd6700 1 -- 172.21.15.142:0/3990452132 >> 172.21.15.142:0/3990452132 conn(0x7f45f4074df0 msgr2=0x7f45f4075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:43.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.033+0000 7f45fbbd6700 1 -- 172.21.15.142:0/3990452132 shutdown_connections 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.033+0000 7f45fbbd6700 1 -- 172.21.15.142:0/3990452132 wait complete. 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45fbbd6700 1 Processor -- start 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45fbbd6700 1 -- start start 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45fbbd6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f40ff530 0x7f45f419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45fbbd6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45f40ffee0 0x7f45f419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45fbbd6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f419d160 con 0x7f45f40ff530 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45fbbd6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4196610 con 0x7f45f40ffee0 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45f9972700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f40ff530 0x7f45f419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45f9972700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f40ff530 0x7f45f419c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39782/0 (socket says 172.21.15.142:39782) 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45f9972700 1 -- 172.21.15.142:0/1702609048 learned_addr learned my addr 172.21.15.142:0/1702609048 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45f9171700 1 --2- 172.21.15.142:0/1702609048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45f40ffee0 0x7f45f419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45f9972700 1 -- 172.21.15.142:0/1702609048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45f40ffee0 msgr2=0x7f45f419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:43.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45f9972700 1 --2- 172.21.15.142:0/1702609048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45f40ffee0 0x7f45f419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45f9972700 1 -- 172.21.15.142:0/1702609048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f00095f0 con 0x7f45f40ff530 2024-03-29T04:55:43.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45f9972700 1 --2- 172.21.15.142:0/1702609048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f40ff530 0x7f45f419c590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f45f0005f50 tx=0x7f45f000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:43.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.034+0000 7f45eaffd700 1 -- 172.21.15.142:0/1702609048 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45f001d070 con 0x7f45f40ff530 2024-03-29T04:55:43.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.035+0000 7f45eaffd700 1 -- 172.21.15.142:0/1702609048 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45f000b610 con 0x7f45f40ff530 2024-03-29T04:55:43.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.035+0000 7f45fbbd6700 1 -- 172.21.15.142:0/1702609048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f4196890 con 0x7f45f40ff530 2024-03-29T04:55:43.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.035+0000 7f45eaffd700 1 -- 172.21.15.142:0/1702609048 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45f0022470 con 0x7f45f40ff530 2024-03-29T04:55:43.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.035+0000 7f45fbbd6700 1 -- 172.21.15.142:0/1702609048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f4196d80 con 0x7f45f40ff530 2024-03-29T04:55:43.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.036+0000 7f45fbbd6700 1 -- 172.21.15.142:0/1702609048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f41091e0 con 0x7f45f40ff530 2024-03-29T04:55:43.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.036+0000 7f45eaffd700 1 -- 172.21.15.142:0/1702609048 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f45f000fca0 con 0x7f45f40ff530 2024-03-29T04:55:43.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.036+0000 7f45eaffd700 1 --2- 172.21.15.142:0/1702609048 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45e006dd60 0x7f45e0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:43.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.037+0000 7f45eaffd700 1 -- 172.21.15.142:0/1702609048 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f45f008f0f0 con 0x7f45f40ff530 2024-03-29T04:55:43.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.037+0000 7f45f9171700 1 --2- 172.21.15.142:0/1702609048 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45e006dd60 0x7f45e0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:43.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.037+0000 7f45f9171700 1 --2- 172.21.15.142:0/1702609048 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45e006dd60 0x7f45e0070220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45f41071f0 tx=0x7f45e4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:43.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:42 smithi152 ceph-mon[88186]: pgmap v222: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:43.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.041+0000 7f45eaffd700 1 -- 172.21.15.142:0/1702609048 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f45f005c040 con 0x7f45f40ff530 2024-03-29T04:55:43.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.243+0000 7f45fbbd6700 1 -- 172.21.15.142:0/1702609048 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f45f4196fa0 con 0x7f45e006dd60 2024-03-29T04:55:43.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.255+0000 7f45eaffd700 1 -- 172.21.15.142:0/1702609048 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f45f4196fa0 con 0x7f45e006dd60 2024-03-29T04:55:43.262 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:55:43.262 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (7m) 39s ago 19m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:55:43.262 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (19m) 39s ago 19m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:55:43.262 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (18m) 5s ago 18m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:55:43.262 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (4m) 39s ago 19m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:55:43.262 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (4m) 5s ago 18m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:55:43.262 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (7m) 39s ago 18m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:55:43.262 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (44s) 39s ago 13m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T04:55:43.262 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (60s) 39s ago 13m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T04:55:43.262 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (8s) 5s ago 13m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (25s) 5s ago 13m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (9m) 39s ago 20m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (8m) 5s ago 18m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (5m) 39s ago 20m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (5m) 5s ago 17m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (8m) 39s ago 19m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (8m) 5s ago 18m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (4m) 39s ago 17m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (3m) 39s ago 16m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (3m) 39s ago 16m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (2m) 5s ago 15m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (114s) 5s ago 15m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T04:55:43.263 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (79s) 5s ago 15m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T04:55:43.264 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (7m) 39s ago 18m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:55:43.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.258+0000 7f45fbbd6700 1 -- 172.21.15.142:0/1702609048 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45e006dd60 msgr2=0x7f45e0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:43.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.258+0000 7f45fbbd6700 1 --2- 172.21.15.142:0/1702609048 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45e006dd60 0x7f45e0070220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45f41071f0 tx=0x7f45e4009260 comp rx=0 tx=0).stop 2024-03-29T04:55:43.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.258+0000 7f45fbbd6700 1 -- 172.21.15.142:0/1702609048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f40ff530 msgr2=0x7f45f419c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:43.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.258+0000 7f45fbbd6700 1 --2- 172.21.15.142:0/1702609048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f40ff530 0x7f45f419c590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f45f0005f50 tx=0x7f45f000f740 comp rx=0 tx=0).stop 2024-03-29T04:55:43.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.258+0000 7f45fbbd6700 1 -- 172.21.15.142:0/1702609048 shutdown_connections 2024-03-29T04:55:43.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.258+0000 7f45fbbd6700 1 --2- 172.21.15.142:0/1702609048 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45e006dd60 0x7f45e0070220 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.258+0000 7f45fbbd6700 1 --2- 172.21.15.142:0/1702609048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45f40ff530 0x7f45f419c590 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.258+0000 7f45fbbd6700 1 --2- 172.21.15.142:0/1702609048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45f40ffee0 0x7f45f419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.258+0000 7f45fbbd6700 1 -- 172.21.15.142:0/1702609048 >> 172.21.15.142:0/1702609048 conn(0x7f45f4074df0 msgr2=0x7f45f40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:43.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.258+0000 7f45fbbd6700 1 -- 172.21.15.142:0/1702609048 shutdown_connections 2024-03-29T04:55:43.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.258+0000 7f45fbbd6700 1 -- 172.21.15.142:0/1702609048 wait complete. 2024-03-29T04:55:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.361+0000 7f9fc9289700 1 -- 172.21.15.142:0/2785099887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc4105a40 msgr2=0x7f9fc4105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.361+0000 7f9fc9289700 1 --2- 172.21.15.142:0/2785099887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc4105a40 0x7f9fc4105e20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009910 tx=0x7f9fb4009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.361+0000 7f9fc9289700 1 -- 172.21.15.142:0/2785099887 shutdown_connections 2024-03-29T04:55:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.361+0000 7f9fc9289700 1 --2- 172.21.15.142:0/2785099887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fc40ffbd0 0x7f9fc4100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.361+0000 7f9fc9289700 1 --2- 172.21.15.142:0/2785099887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc4105a40 0x7f9fc4105e20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.361+0000 7f9fc9289700 1 -- 172.21.15.142:0/2785099887 >> 172.21.15.142:0/2785099887 conn(0x7f9fc40755a0 msgr2=0x7f9fc40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:43.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.361+0000 7f9fc9289700 1 -- 172.21.15.142:0/2785099887 shutdown_connections 2024-03-29T04:55:43.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.361+0000 7f9fc9289700 1 -- 172.21.15.142:0/2785099887 wait complete. 2024-03-29T04:55:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.361+0000 7f9fc9289700 1 Processor -- start 2024-03-29T04:55:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.361+0000 7f9fc9289700 1 -- start start 2024-03-29T04:55:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc9289700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc40ffbd0 0x7f9fc419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc9289700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fc4105a40 0x7f9fc419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc9289700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc419d1c0 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc9289700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc4196670 con 0x7f9fc4105a40 2024-03-29T04:55:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc40ffbd0 0x7f9fc419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:43.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc40ffbd0 0x7f9fc419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39798/0 (socket says 172.21.15.142:39798) 2024-03-29T04:55:43.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc2ffd700 1 -- 172.21.15.142:0/3980350341 learned_addr learned my addr 172.21.15.142:0/3980350341 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:43.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fba7fc700 1 --2- 172.21.15.142:0/3980350341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fc4105a40 0x7f9fc419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:43.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc2ffd700 1 -- 172.21.15.142:0/3980350341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fc4105a40 msgr2=0x7f9fc419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:43.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc2ffd700 1 --2- 172.21.15.142:0/3980350341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fc4105a40 0x7f9fc419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc2ffd700 1 -- 172.21.15.142:0/3980350341 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40095f0 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fba7fc700 1 --2- 172.21.15.142:0/3980350341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fc4105a40 0x7f9fc419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:55:43.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc2ffd700 1 --2- 172.21.15.142:0/3980350341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc40ffbd0 0x7f9fc419c5f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4006010 tx=0x7f9fb400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:43.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc0ff9700 1 -- 172.21.15.142:0/3980350341 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fb401d070 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc0ff9700 1 -- 172.21.15.142:0/3980350341 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9fb4004fb0 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc9289700 1 -- 172.21.15.142:0/3980350341 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc41968f0 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.362+0000 7f9fc0ff9700 1 -- 172.21.15.142:0/3980350341 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fb4022ea0 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.363+0000 7f9fc9289700 1 -- 172.21.15.142:0/3980350341 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc4196de0 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.364+0000 7f9fc9289700 1 -- 172.21.15.142:0/3980350341 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc4109050 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.364+0000 7f9fc0ff9700 1 -- 172.21.15.142:0/3980350341 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f9fb4022470 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.364+0000 7f9fc0ff9700 1 --2- 172.21.15.142:0/3980350341 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9fb006dd60 0x7f9fb0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:43.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.365+0000 7f9fc0ff9700 1 -- 172.21.15.142:0/3980350341 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f9fb408f180 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.365+0000 7f9fba7fc700 1 --2- 172.21.15.142:0/3980350341 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9fb006dd60 0x7f9fb0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:43.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.365+0000 7f9fba7fc700 1 --2- 172.21.15.142:0/3980350341 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9fb006dd60 0x7f9fb0070220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4104df0 tx=0x7f9fac008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:43.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.369+0000 7f9fc0ff9700 1 -- 172.21.15.142:0/3980350341 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9fb405c200 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.650+0000 7f9fc9289700 1 -- 172.21.15.142:0/3980350341 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9fc402cc70 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.652+0000 7f9fc0ff9700 1 -- 172.21.15.142:0/3980350341 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9fb4022720 con 0x7f9fc40ffbd0 2024-03-29T04:55:43.658 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:55:43.658 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:55:43.658 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:55:43.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:55:43.659 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:55:43.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.654+0000 7f9fc9289700 1 -- 172.21.15.142:0/3980350341 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9fb006dd60 msgr2=0x7f9fb0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:43.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.654+0000 7f9fc9289700 1 --2- 172.21.15.142:0/3980350341 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9fb006dd60 0x7f9fb0070220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4104df0 tx=0x7f9fac008040 comp rx=0 tx=0).stop 2024-03-29T04:55:43.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.654+0000 7f9fc9289700 1 -- 172.21.15.142:0/3980350341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc40ffbd0 msgr2=0x7f9fc419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:43.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.654+0000 7f9fc9289700 1 --2- 172.21.15.142:0/3980350341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc40ffbd0 0x7f9fc419c5f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4006010 tx=0x7f9fb400be20 comp rx=0 tx=0).stop 2024-03-29T04:55:43.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.654+0000 7f9fc9289700 1 -- 172.21.15.142:0/3980350341 shutdown_connections 2024-03-29T04:55:43.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.654+0000 7f9fc9289700 1 --2- 172.21.15.142:0/3980350341 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9fb006dd60 0x7f9fb0070220 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.654+0000 7f9fc9289700 1 --2- 172.21.15.142:0/3980350341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc40ffbd0 0x7f9fc419c5f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.654+0000 7f9fc9289700 1 --2- 172.21.15.142:0/3980350341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fc4105a40 0x7f9fc419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.654+0000 7f9fc9289700 1 -- 172.21.15.142:0/3980350341 >> 172.21.15.142:0/3980350341 conn(0x7f9fc40755a0 msgr2=0x7f9fc40686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:43.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.654+0000 7f9fc9289700 1 -- 172.21.15.142:0/3980350341 shutdown_connections 2024-03-29T04:55:43.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.654+0000 7f9fc9289700 1 -- 172.21.15.142:0/3980350341 wait complete. 2024-03-29T04:55:43.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.758+0000 7efeb61d7700 1 -- 172.21.15.142:0/4074790757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb0105a40 msgr2=0x7efeb0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:43.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.758+0000 7efeb61d7700 1 --2- 172.21.15.142:0/4074790757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb0105a40 0x7efeb0105e20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efea0009910 tx=0x7efea0009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:43.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.758+0000 7efeb61d7700 1 -- 172.21.15.142:0/4074790757 shutdown_connections 2024-03-29T04:55:43.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.758+0000 7efeb61d7700 1 --2- 172.21.15.142:0/4074790757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efeb00ffbd0 0x7efeb0100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.758+0000 7efeb61d7700 1 --2- 172.21.15.142:0/4074790757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb0105a40 0x7efeb0105e20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.758+0000 7efeb61d7700 1 -- 172.21.15.142:0/4074790757 >> 172.21.15.142:0/4074790757 conn(0x7efeb00755a0 msgr2=0x7efeb00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:43.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.758+0000 7efeb61d7700 1 -- 172.21.15.142:0/4074790757 shutdown_connections 2024-03-29T04:55:43.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.758+0000 7efeb61d7700 1 -- 172.21.15.142:0/4074790757 wait complete. 2024-03-29T04:55:43.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeb61d7700 1 Processor -- start 2024-03-29T04:55:43.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeb61d7700 1 -- start start 2024-03-29T04:55:43.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeb61d7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb00ffbd0 0x7efeb0101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:43.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeb61d7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efeb0105a40 0x7efeb0101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:43.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeb61d7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0102120 con 0x7efeb00ffbd0 2024-03-29T04:55:43.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeb61d7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0102290 con 0x7efeb0105a40 2024-03-29T04:55:43.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeaf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb00ffbd0 0x7efeb0101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:43.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeaf7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb00ffbd0 0x7efeb0101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39830/0 (socket says 172.21.15.142:39830) 2024-03-29T04:55:43.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeaf7fe700 1 -- 172.21.15.142:0/3764677305 learned_addr learned my addr 172.21.15.142:0/3764677305 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:43.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeaf7fe700 1 -- 172.21.15.142:0/3764677305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efeb0105a40 msgr2=0x7efeb0101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:43.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeaeffd700 1 --2- 172.21.15.142:0/3764677305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efeb0105a40 0x7efeb0101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:43.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeaf7fe700 1 --2- 172.21.15.142:0/3764677305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efeb0105a40 0x7efeb0101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:43.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.759+0000 7efeaf7fe700 1 -- 172.21.15.142:0/3764677305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00095f0 con 0x7efeb00ffbd0 2024-03-29T04:55:43.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.760+0000 7efeaf7fe700 1 --2- 172.21.15.142:0/3764677305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb00ffbd0 0x7efeb0101610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efea00092e0 tx=0x7efea000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:43.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.760+0000 7efeacff9700 1 -- 172.21.15.142:0/3764677305 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efea001d070 con 0x7efeb00ffbd0 2024-03-29T04:55:43.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.760+0000 7efeacff9700 1 -- 172.21.15.142:0/3764677305 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efea0005e00 con 0x7efeb00ffbd0 2024-03-29T04:55:43.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.760+0000 7efeb61d7700 1 -- 172.21.15.142:0/3764677305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb01a28c0 con 0x7efeb00ffbd0 2024-03-29T04:55:43.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.760+0000 7efeacff9700 1 -- 172.21.15.142:0/3764677305 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efea0017400 con 0x7efeb00ffbd0 2024-03-29T04:55:43.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.760+0000 7efeb61d7700 1 -- 172.21.15.142:0/3764677305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb01a2c70 con 0x7efeb00ffbd0 2024-03-29T04:55:43.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.761+0000 7efeb61d7700 1 -- 172.21.15.142:0/3764677305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb0109050 con 0x7efeb00ffbd0 2024-03-29T04:55:43.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.761+0000 7efeacff9700 1 -- 172.21.15.142:0/3764677305 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7efea000f460 con 0x7efeb00ffbd0 2024-03-29T04:55:43.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.762+0000 7efeacff9700 1 --2- 172.21.15.142:0/3764677305 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7efe9806dd60 0x7efe98070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:43.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.762+0000 7efeacff9700 1 -- 172.21.15.142:0/3764677305 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7efea008f0b0 con 0x7efeb00ffbd0 2024-03-29T04:55:43.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.762+0000 7efeaeffd700 1 --2- 172.21.15.142:0/3764677305 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7efe9806dd60 0x7efe98070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:43.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.762+0000 7efeaeffd700 1 --2- 172.21.15.142:0/3764677305 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7efe9806dd60 0x7efe98070220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efea4005950 tx=0x7efea40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:43.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:43.766+0000 7efeacff9700 1 -- 172.21.15.142:0/3764677305 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efea0058550 con 0x7efeb00ffbd0 2024-03-29T04:55:44.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.005+0000 7efeb61d7700 1 -- 172.21.15.142:0/3764677305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efeb002cc70 con 0x7efeb00ffbd0 2024-03-29T04:55:44.012 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:43 smithi142 ceph-mon[112475]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:44.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.006+0000 7efeacff9700 1 -- 172.21.15.142:0/3764677305 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7efea005bc10 con 0x7efeb00ffbd0 2024-03-29T04:55:44.013 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T04:55:44.013 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:55:44.013 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:55:44.013 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:55:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:55:44.013 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:55:44.013 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:55:44.013 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T04:55:44.013 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T04:55:44.013 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:55:44.013 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T04:55:44.014 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:55:44.015 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:55:44.016 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:55:44.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.008+0000 7efeb61d7700 1 -- 172.21.15.142:0/3764677305 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7efe9806dd60 msgr2=0x7efe98070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:44.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.008+0000 7efeb61d7700 1 --2- 172.21.15.142:0/3764677305 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7efe9806dd60 0x7efe98070220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efea4005950 tx=0x7efea40058e0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.008+0000 7efeb61d7700 1 -- 172.21.15.142:0/3764677305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb00ffbd0 msgr2=0x7efeb0101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:44.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.008+0000 7efeb61d7700 1 --2- 172.21.15.142:0/3764677305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb00ffbd0 0x7efeb0101610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efea00092e0 tx=0x7efea000be20 comp rx=0 tx=0).stop 2024-03-29T04:55:44.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.008+0000 7efeb61d7700 1 -- 172.21.15.142:0/3764677305 shutdown_connections 2024-03-29T04:55:44.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.008+0000 7efeb61d7700 1 --2- 172.21.15.142:0/3764677305 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7efe9806dd60 0x7efe98070220 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.008+0000 7efeb61d7700 1 --2- 172.21.15.142:0/3764677305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efeb00ffbd0 0x7efeb0101610 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.008+0000 7efeb61d7700 1 --2- 172.21.15.142:0/3764677305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efeb0105a40 0x7efeb0101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.008+0000 7efeb61d7700 1 -- 172.21.15.142:0/3764677305 >> 172.21.15.142:0/3764677305 conn(0x7efeb00755a0 msgr2=0x7efeb00685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:44.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.008+0000 7efeb61d7700 1 -- 172.21.15.142:0/3764677305 shutdown_connections 2024-03-29T04:55:44.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.008+0000 7efeb61d7700 1 -- 172.21.15.142:0/3764677305 wait complete. 2024-03-29T04:55:44.018 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T04:55:44.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:43 smithi152 ceph-mon[88186]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:44.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.115+0000 7f33a90f6700 1 -- 172.21.15.142:0/1436994805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a4101d30 msgr2=0x7f33a4102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:44.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.115+0000 7f33a90f6700 1 --2- 172.21.15.142:0/1436994805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a4101d30 0x7f33a4102110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3394009910 tx=0x7f3394009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:44.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.115+0000 7f33a90f6700 1 -- 172.21.15.142:0/1436994805 shutdown_connections 2024-03-29T04:55:44.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.115+0000 7f33a90f6700 1 --2- 172.21.15.142:0/1436994805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33a4102650 0x7f33a410ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.115+0000 7f33a90f6700 1 --2- 172.21.15.142:0/1436994805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a4101d30 0x7f33a4102110 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.115+0000 7f33a90f6700 1 -- 172.21.15.142:0/1436994805 >> 172.21.15.142:0/1436994805 conn(0x7f33a40fb560 msgr2=0x7f33a40fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:44.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.115+0000 7f33a90f6700 1 -- 172.21.15.142:0/1436994805 shutdown_connections 2024-03-29T04:55:44.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.115+0000 7f33a90f6700 1 -- 172.21.15.142:0/1436994805 wait complete. 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.116+0000 7f33a90f6700 1 Processor -- start 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.116+0000 7f33a90f6700 1 -- start start 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a90f6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33a4101d30 0x7f33a419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a90f6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a4102650 0x7f33a419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a90f6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a419d120 con 0x7f33a4102650 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a90f6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a41965d0 con 0x7f33a4101d30 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a4102650 0x7f33a419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a4102650 0x7f33a419ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39844/0 (socket says 172.21.15.142:39844) 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a259c700 1 -- 172.21.15.142:0/2819550262 learned_addr learned my addr 172.21.15.142:0/2819550262 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a2d9d700 1 --2- 172.21.15.142:0/2819550262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33a4101d30 0x7f33a419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a259c700 1 -- 172.21.15.142:0/2819550262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33a4101d30 msgr2=0x7f33a419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a259c700 1 --2- 172.21.15.142:0/2819550262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33a4101d30 0x7f33a419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a259c700 1 -- 172.21.15.142:0/2819550262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33940095f0 con 0x7f33a4102650 2024-03-29T04:55:44.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a259c700 1 --2- 172.21.15.142:0/2819550262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a4102650 0x7f33a419ca90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f339800d8d0 tx=0x7f339800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:44.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f338bfff700 1 -- 172.21.15.142:0/2819550262 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33980041f0 con 0x7f33a4102650 2024-03-29T04:55:44.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a90f6700 1 -- 172.21.15.142:0/2819550262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a41968b0 con 0x7f33a4102650 2024-03-29T04:55:44.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f338bfff700 1 -- 172.21.15.142:0/2819550262 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3398004350 con 0x7f33a4102650 2024-03-29T04:55:44.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f338bfff700 1 -- 172.21.15.142:0/2819550262 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3398003c80 con 0x7f33a4102650 2024-03-29T04:55:44.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.117+0000 7f33a90f6700 1 -- 172.21.15.142:0/2819550262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a4196e00 con 0x7f33a4102650 2024-03-29T04:55:44.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.118+0000 7f33a90f6700 1 -- 172.21.15.142:0/2819550262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a4108240 con 0x7f33a4102650 2024-03-29T04:55:44.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.119+0000 7f338bfff700 1 -- 172.21.15.142:0/2819550262 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f339800f690 con 0x7f33a4102650 2024-03-29T04:55:44.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.119+0000 7f338bfff700 1 --2- 172.21.15.142:0/2819550262 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f338c06dd60 0x7f338c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:44.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.119+0000 7f338bfff700 1 -- 172.21.15.142:0/2819550262 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f339808d820 con 0x7f33a4102650 2024-03-29T04:55:44.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.119+0000 7f33a2d9d700 1 --2- 172.21.15.142:0/2819550262 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f338c06dd60 0x7f338c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:44.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.120+0000 7f33a2d9d700 1 --2- 172.21.15.142:0/2819550262 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f338c06dd60 0x7f338c070220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3394006010 tx=0x7f339401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:44.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.124+0000 7f338bfff700 1 -- 172.21.15.142:0/2819550262 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f339805a770 con 0x7f33a4102650 2024-03-29T04:55:44.342 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.336+0000 7f33a90f6700 1 -- 172.21.15.142:0/2819550262 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f33a41970a0 con 0x7f338c06dd60 2024-03-29T04:55:44.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.337+0000 7f338bfff700 1 -- 172.21.15.142:0/2819550262 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f33a41970a0 con 0x7f338c06dd60 2024-03-29T04:55:44.343 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:55:44.344 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:55:44.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.339+0000 7f33a90f6700 1 -- 172.21.15.142:0/2819550262 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f338c06dd60 msgr2=0x7f338c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:44.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.339+0000 7f33a90f6700 1 --2- 172.21.15.142:0/2819550262 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f338c06dd60 0x7f338c070220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3394006010 tx=0x7f339401a040 comp rx=0 tx=0).stop 2024-03-29T04:55:44.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.339+0000 7f33a90f6700 1 -- 172.21.15.142:0/2819550262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a4102650 msgr2=0x7f33a419ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:44.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.339+0000 7f33a90f6700 1 --2- 172.21.15.142:0/2819550262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a4102650 0x7f33a419ca90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f339800d8d0 tx=0x7f339800dbe0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.339+0000 7f33a90f6700 1 -- 172.21.15.142:0/2819550262 shutdown_connections 2024-03-29T04:55:44.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.339+0000 7f33a90f6700 1 --2- 172.21.15.142:0/2819550262 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f338c06dd60 0x7f338c070220 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.339+0000 7f33a90f6700 1 --2- 172.21.15.142:0/2819550262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33a4101d30 0x7f33a419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.339+0000 7f33a90f6700 1 --2- 172.21.15.142:0/2819550262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a4102650 0x7f33a419ca90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.339+0000 7f33a90f6700 1 -- 172.21.15.142:0/2819550262 >> 172.21.15.142:0/2819550262 conn(0x7f33a40fb560 msgr2=0x7f33a4105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:44.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.339+0000 7f33a90f6700 1 -- 172.21.15.142:0/2819550262 shutdown_connections 2024-03-29T04:55:44.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.339+0000 7f33a90f6700 1 -- 172.21.15.142:0/2819550262 wait complete. 2024-03-29T04:55:44.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 -- 172.21.15.142:0/22963570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a40ffa70 msgr2=0x7fd2a40ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:44.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 --2- 172.21.15.142:0/22963570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a40ffa70 0x7fd2a40ffef0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009910 tx=0x7fd2a0009c20 comp rx=0 tx=0).stop 2024-03-29T04:55:44.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 -- 172.21.15.142:0/22963570 shutdown_connections 2024-03-29T04:55:44.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 --2- 172.21.15.142:0/22963570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a40ffa70 0x7fd2a40ffef0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 --2- 172.21.15.142:0/22963570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a4105a70 0x7fd2a4105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 -- 172.21.15.142:0/22963570 >> 172.21.15.142:0/22963570 conn(0x7fd2a40fb560 msgr2=0x7fd2a40fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:44.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 -- 172.21.15.142:0/22963570 shutdown_connections 2024-03-29T04:55:44.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 -- 172.21.15.142:0/22963570 wait complete. 2024-03-29T04:55:44.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 Processor -- start 2024-03-29T04:55:44.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 -- start start 2024-03-29T04:55:44.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a40ffa70 0x7fd2a40fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:44.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a4105a70 0x7fd2a40fc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.443+0000 7fd2ac26a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a40fc9a0 con 0x7fd2a40ffa70 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd2ac26a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a40fcae0 con 0x7fd2a4105a70 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd2aa006700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a40ffa70 0x7fd2a40fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd2aa006700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a40ffa70 0x7fd2a40fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39868/0 (socket says 172.21.15.142:39868) 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd2aa006700 1 -- 172.21.15.142:0/1998387281 learned_addr learned my addr 172.21.15.142:0/1998387281 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd2a9805700 1 --2- 172.21.15.142:0/1998387281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a4105a70 0x7fd2a40fc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd2aa006700 1 -- 172.21.15.142:0/1998387281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a4105a70 msgr2=0x7fd2a40fc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd2aa006700 1 --2- 172.21.15.142:0/1998387281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a4105a70 0x7fd2a40fc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd2aa006700 1 -- 172.21.15.142:0/1998387281 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a00095f0 con 0x7fd2a40ffa70 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd2aa006700 1 --2- 172.21.15.142:0/1998387281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a40ffa70 0x7fd2a40fbf20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd29400d8d0 tx=0x7fd29400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd29b7fe700 1 -- 172.21.15.142:0/1998387281 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd294009940 con 0x7fd2a40ffa70 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd29b7fe700 1 -- 172.21.15.142:0/1998387281 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd294010460 con 0x7fd2a40ffa70 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd2ac26a700 1 -- 172.21.15.142:0/1998387281 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a41a2930 con 0x7fd2a40ffa70 2024-03-29T04:55:44.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd29b7fe700 1 -- 172.21.15.142:0/1998387281 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd29400f5f0 con 0x7fd2a40ffa70 2024-03-29T04:55:44.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.444+0000 7fd2ac26a700 1 -- 172.21.15.142:0/1998387281 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a41a2d40 con 0x7fd2a40ffa70 2024-03-29T04:55:44.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.445+0000 7fd2ac26a700 1 -- 172.21.15.142:0/1998387281 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a4109010 con 0x7fd2a40ffa70 2024-03-29T04:55:44.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.446+0000 7fd29b7fe700 1 -- 172.21.15.142:0/1998387281 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fd2940105d0 con 0x7fd2a40ffa70 2024-03-29T04:55:44.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.446+0000 7fd29b7fe700 1 --2- 172.21.15.142:0/1998387281 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd29006dd60 0x7fd290070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:55:44.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.446+0000 7fd29b7fe700 1 -- 172.21.15.142:0/1998387281 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd29408dcc0 con 0x7fd2a40ffa70 2024-03-29T04:55:44.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.446+0000 7fd2a9805700 1 --2- 172.21.15.142:0/1998387281 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd29006dd60 0x7fd290070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:55:44.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.447+0000 7fd2a9805700 1 --2- 172.21.15.142:0/1998387281 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd29006dd60 0x7fd290070220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0005b40 tx=0x7fd2a001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:55:44.457 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.451+0000 7fd29b7fe700 1 -- 172.21.15.142:0/1998387281 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd2940570e0 con 0x7fd2a40ffa70 2024-03-29T04:55:44.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.732+0000 7fd2ac26a700 1 -- 172.21.15.142:0/1998387281 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd2a404ea90 con 0x7fd2a40ffa70 2024-03-29T04:55:44.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.732+0000 7fd29b7fe700 1 -- 172.21.15.142:0/1998387281 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fd294016100 con 0x7fd2a40ffa70 2024-03-29T04:55:44.739 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T04:55:44.739 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T04:55:44.739 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T04:55:44.739 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T04:55:44.739 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T04:55:44.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.735+0000 7fd2ac26a700 1 -- 172.21.15.142:0/1998387281 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd29006dd60 msgr2=0x7fd290070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:44.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.735+0000 7fd2ac26a700 1 --2- 172.21.15.142:0/1998387281 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd29006dd60 0x7fd290070220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0005b40 tx=0x7fd2a001a040 comp rx=0 tx=0).stop 2024-03-29T04:55:44.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.735+0000 7fd2ac26a700 1 -- 172.21.15.142:0/1998387281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a40ffa70 msgr2=0x7fd2a40fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:55:44.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.735+0000 7fd2ac26a700 1 --2- 172.21.15.142:0/1998387281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a40ffa70 0x7fd2a40fbf20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd29400d8d0 tx=0x7fd29400dc90 comp rx=0 tx=0).stop 2024-03-29T04:55:44.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.735+0000 7fd2ac26a700 1 -- 172.21.15.142:0/1998387281 shutdown_connections 2024-03-29T04:55:44.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.735+0000 7fd2ac26a700 1 --2- 172.21.15.142:0/1998387281 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd29006dd60 0x7fd290070220 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.735+0000 7fd2ac26a700 1 --2- 172.21.15.142:0/1998387281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a40ffa70 0x7fd2a40fbf20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.735+0000 7fd2ac26a700 1 --2- 172.21.15.142:0/1998387281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a4105a70 0x7fd2a40fc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:55:44.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.735+0000 7fd2ac26a700 1 -- 172.21.15.142:0/1998387281 >> 172.21.15.142:0/1998387281 conn(0x7fd2a40fb560 msgr2=0x7fd2a40690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:55:44.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.735+0000 7fd2ac26a700 1 -- 172.21.15.142:0/1998387281 shutdown_connections 2024-03-29T04:55:44.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:55:44.735+0000 7fd2ac26a700 1 -- 172.21.15.142:0/1998387281 wait complete. 2024-03-29T04:55:45.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:44 smithi152 ceph-mon[88186]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:45.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:44 smithi152 ceph-mon[88186]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:45.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:44 smithi152 ceph-mon[88186]: pgmap v223: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:45.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:44 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3980350341' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:45.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:44 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3764677305' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:55:45.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:55:45.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:44 smithi142 ceph-mon[112475]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:45.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:44 smithi142 ceph-mon[112475]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:45.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:44 smithi142 ceph-mon[112475]: pgmap v223: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:45.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:44 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3980350341' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:55:45.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:44 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3764677305' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:55:45.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:55:45.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:45 smithi142 ceph-mon[112475]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:45.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:45 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1998387281' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:55:46.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:45 smithi152 ceph-mon[88186]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:55:46.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:45 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1998387281' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:55:47.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:46 smithi152 ceph-mon[88186]: pgmap v224: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:47.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:46 smithi142 ceph-mon[112475]: pgmap v224: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:49.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:48 smithi152 ceph-mon[88186]: pgmap v225: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:49.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:48 smithi142 ceph-mon[112475]: pgmap v225: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:50 smithi142 ceph-mon[112475]: pgmap v226: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:51.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:50 smithi152 ceph-mon[88186]: pgmap v226: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:53.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:52 smithi152 ceph-mon[88186]: pgmap v227: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:53.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:52 smithi142 ceph-mon[112475]: pgmap v227: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:55.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:54 smithi152 ceph-mon[88186]: pgmap v228: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:55.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:54 smithi142 ceph-mon[112475]: pgmap v228: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:57.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:56 smithi152 ceph-mon[88186]: pgmap v229: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:57.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:56 smithi142 ceph-mon[112475]: pgmap v229: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:59.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:58 smithi152 ceph-mon[88186]: pgmap v230: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:55:59.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:58 smithi142 ceph-mon[112475]: pgmap v230: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:00.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:55:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:56:00.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:55:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:56:01.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:00 smithi152 ceph-mon[88186]: pgmap v231: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:01.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:00 smithi142 ceph-mon[112475]: pgmap v231: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:03.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:02 smithi152 ceph-mon[88186]: pgmap v232: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:03.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:02 smithi142 ceph-mon[112475]: pgmap v232: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:05.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:04 smithi152 ceph-mon[88186]: pgmap v233: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:05.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:04 smithi142 ceph-mon[112475]: pgmap v233: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:07.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:06 smithi152 ceph-mon[88186]: pgmap v234: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:07.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:06 smithi142 ceph-mon[112475]: pgmap v234: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:09.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:08 smithi152 ceph-mon[88186]: pgmap v235: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:09.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:08 smithi142 ceph-mon[112475]: pgmap v235: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:11.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:10 smithi142 ceph-mon[112475]: pgmap v236: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:11.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:10 smithi152 ceph-mon[88186]: pgmap v236: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:13.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:12 smithi152 ceph-mon[88186]: pgmap v237: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:13.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:12 smithi142 ceph-mon[112475]: pgmap v237: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:14.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.851+0000 7f3f1762e700 1 -- 172.21.15.142:0/2887518496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10100dc0 msgr2=0x7f3f101011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:14.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.851+0000 7f3f1762e700 1 --2- 172.21.15.142:0/2887518496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10100dc0 0x7f3f101011a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009910 tx=0x7f3f0c009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:14.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.852+0000 7f3f1762e700 1 -- 172.21.15.142:0/2887518496 shutdown_connections 2024-03-29T04:56:14.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.852+0000 7f3f1762e700 1 --2- 172.21.15.142:0/2887518496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f10101770 0x7f3f101057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:14.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.852+0000 7f3f1762e700 1 --2- 172.21.15.142:0/2887518496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10100dc0 0x7f3f101011a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:14.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.852+0000 7f3f1762e700 1 -- 172.21.15.142:0/2887518496 >> 172.21.15.142:0/2887518496 conn(0x7f3f100fc650 msgr2=0x7f3f100fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:14.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.852+0000 7f3f1762e700 1 -- 172.21.15.142:0/2887518496 shutdown_connections 2024-03-29T04:56:14.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.852+0000 7f3f1762e700 1 -- 172.21.15.142:0/2887518496 wait complete. 2024-03-29T04:56:14.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.852+0000 7f3f1762e700 1 Processor -- start 2024-03-29T04:56:14.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.852+0000 7f3f1762e700 1 -- start start 2024-03-29T04:56:14.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.852+0000 7f3f1762e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f10100dc0 0x7f3f1019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:14.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.852+0000 7f3f1762e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10101770 0x7f3f1019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:14.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.852+0000 7f3f1762e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1019d110 con 0x7f3f10101770 2024-03-29T04:56:14.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.852+0000 7f3f1762e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f101965c0 con 0x7f3f10100dc0 2024-03-29T04:56:14.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f14bc9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10101770 0x7f3f1019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:14.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f14bc9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10101770 0x7f3f1019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54506/0 (socket says 172.21.15.142:54506) 2024-03-29T04:56:14.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f14bc9700 1 -- 172.21.15.142:0/2009450397 learned_addr learned my addr 172.21.15.142:0/2009450397 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:14.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f153ca700 1 --2- 172.21.15.142:0/2009450397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f10100dc0 0x7f3f1019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:14.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f14bc9700 1 -- 172.21.15.142:0/2009450397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f10100dc0 msgr2=0x7f3f1019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:14.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f14bc9700 1 --2- 172.21.15.142:0/2009450397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f10100dc0 0x7f3f1019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:14.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f14bc9700 1 -- 172.21.15.142:0/2009450397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0095f0 con 0x7f3f10101770 2024-03-29T04:56:14.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f14bc9700 1 --2- 172.21.15.142:0/2009450397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10101770 0x7f3f1019ca80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000d8d0 tx=0x7f3f0000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f067fc700 1 -- 172.21.15.142:0/2009450397 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f000041f0 con 0x7f3f10101770 2024-03-29T04:56:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f067fc700 1 -- 172.21.15.142:0/2009450397 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f00007620 con 0x7f3f10101770 2024-03-29T04:56:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f067fc700 1 -- 172.21.15.142:0/2009450397 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f000056b0 con 0x7f3f10101770 2024-03-29T04:56:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f1762e700 1 -- 172.21.15.142:0/2009450397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f101968a0 con 0x7f3f10101770 2024-03-29T04:56:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.853+0000 7f3f1762e700 1 -- 172.21.15.142:0/2009450397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f10196df0 con 0x7f3f10101770 2024-03-29T04:56:14.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.855+0000 7f3f1762e700 1 -- 172.21.15.142:0/2009450397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f10109160 con 0x7f3f10101770 2024-03-29T04:56:14.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.855+0000 7f3f067fc700 1 -- 172.21.15.142:0/2009450397 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f3f00004750 con 0x7f3f10101770 2024-03-29T04:56:14.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.856+0000 7f3f067fc700 1 --2- 172.21.15.142:0/2009450397 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3efc06dd60 0x7f3efc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:14.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.856+0000 7f3f067fc700 1 -- 172.21.15.142:0/2009450397 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f3f0008dad0 con 0x7f3f10101770 2024-03-29T04:56:14.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.856+0000 7f3f153ca700 1 --2- 172.21.15.142:0/2009450397 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3efc06dd60 0x7f3efc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:14.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.856+0000 7f3f153ca700 1 --2- 172.21.15.142:0/2009450397 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3efc06dd60 0x7f3efc070220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c006010 tx=0x7f3f0c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:14.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:14.860+0000 7f3f067fc700 1 -- 172.21.15.142:0/2009450397 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3f0005aa20 con 0x7f3f10101770 2024-03-29T04:56:15.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.078+0000 7f3f1762e700 1 -- 172.21.15.142:0/2009450397 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f101970a0 con 0x7f3efc06dd60 2024-03-29T04:56:15.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.079+0000 7f3f067fc700 1 -- 172.21.15.142:0/2009450397 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3f101970a0 con 0x7f3efc06dd60 2024-03-29T04:56:15.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.081+0000 7f3f1762e700 1 -- 172.21.15.142:0/2009450397 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3efc06dd60 msgr2=0x7f3efc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:15.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.081+0000 7f3f1762e700 1 --2- 172.21.15.142:0/2009450397 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3efc06dd60 0x7f3efc070220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c006010 tx=0x7f3f0c01a040 comp rx=0 tx=0).stop 2024-03-29T04:56:15.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.081+0000 7f3f1762e700 1 -- 172.21.15.142:0/2009450397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10101770 msgr2=0x7f3f1019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:15.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.081+0000 7f3f1762e700 1 --2- 172.21.15.142:0/2009450397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10101770 0x7f3f1019ca80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000d8d0 tx=0x7f3f0000dbe0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.081+0000 7f3f1762e700 1 -- 172.21.15.142:0/2009450397 shutdown_connections 2024-03-29T04:56:15.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.081+0000 7f3f1762e700 1 --2- 172.21.15.142:0/2009450397 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3efc06dd60 0x7f3efc070220 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.081+0000 7f3f1762e700 1 --2- 172.21.15.142:0/2009450397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f10100dc0 0x7f3f1019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.081+0000 7f3f1762e700 1 --2- 172.21.15.142:0/2009450397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f10101770 0x7f3f1019ca80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.081+0000 7f3f1762e700 1 -- 172.21.15.142:0/2009450397 >> 172.21.15.142:0/2009450397 conn(0x7f3f100fc650 msgr2=0x7f3f100fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:15.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.081+0000 7f3f1762e700 1 -- 172.21.15.142:0/2009450397 shutdown_connections 2024-03-29T04:56:15.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.081+0000 7f3f1762e700 1 -- 172.21.15.142:0/2009450397 wait complete. 2024-03-29T04:56:15.099 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:56:15.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:14 smithi142 ceph-mon[112475]: pgmap v238: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:15.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:56:15.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.187+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3918760833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac40ffbd0 msgr2=0x7f1ac4100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:15.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.187+0000 7f1ac8e75700 1 --2- 172.21.15.142:0/3918760833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac40ffbd0 0x7f1ac4100050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009910 tx=0x7f1ab8009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:15.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.187+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3918760833 shutdown_connections 2024-03-29T04:56:15.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.187+0000 7f1ac8e75700 1 --2- 172.21.15.142:0/3918760833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac40ffbd0 0x7f1ac4100050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.187+0000 7f1ac8e75700 1 --2- 172.21.15.142:0/3918760833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4105a40 0x7f1ac4105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.187+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3918760833 >> 172.21.15.142:0/3918760833 conn(0x7f1ac40755a0 msgr2=0x7f1ac40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:15.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.187+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3918760833 shutdown_connections 2024-03-29T04:56:15.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.187+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3918760833 wait complete. 2024-03-29T04:56:15.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.188+0000 7f1ac8e75700 1 Processor -- start 2024-03-29T04:56:15.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.188+0000 7f1ac8e75700 1 -- start start 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.188+0000 7f1ac8e75700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac40ffbd0 0x7f1ac419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.188+0000 7f1ac8e75700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4105a40 0x7f1ac419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.188+0000 7f1ac8e75700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac4196550 con 0x7f1ac4105a40 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.188+0000 7f1ac8e75700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac41966c0 con 0x7f1ac40ffbd0 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ac1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4105a40 0x7f1ac419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ac1d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4105a40 0x7f1ac419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54524/0 (socket says 172.21.15.142:54524) 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ac1d9b700 1 -- 172.21.15.142:0/3248247816 learned_addr learned my addr 172.21.15.142:0/3248247816 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ac259c700 1 --2- 172.21.15.142:0/3248247816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac40ffbd0 0x7f1ac419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ac1d9b700 1 -- 172.21.15.142:0/3248247816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac40ffbd0 msgr2=0x7f1ac419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ac1d9b700 1 --2- 172.21.15.142:0/3248247816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac40ffbd0 0x7f1ac419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ac1d9b700 1 -- 172.21.15.142:0/3248247816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab80095f0 con 0x7f1ac4105a40 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ac259c700 1 --2- 172.21.15.142:0/3248247816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac40ffbd0 0x7f1ac419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:56:15.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ac1d9b700 1 --2- 172.21.15.142:0/3248247816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4105a40 0x7f1ac419caa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006010 tx=0x7f1ab800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:15.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ab37fe700 1 -- 172.21.15.142:0/3248247816 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ab801d070 con 0x7f1ac4105a40 2024-03-29T04:56:15.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3248247816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac4196940 con 0x7f1ac4105a40 2024-03-29T04:56:15.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ab37fe700 1 -- 172.21.15.142:0/3248247816 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ab8004fb0 con 0x7f1ac4105a40 2024-03-29T04:56:15.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ab37fe700 1 -- 172.21.15.142:0/3248247816 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ab8022ea0 con 0x7f1ac4105a40 2024-03-29T04:56:15.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.189+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3248247816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac4196e30 con 0x7f1ac4105a40 2024-03-29T04:56:15.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.190+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3248247816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac404ea90 con 0x7f1ac4105a40 2024-03-29T04:56:15.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.191+0000 7f1ab37fe700 1 -- 172.21.15.142:0/3248247816 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f1ab8022470 con 0x7f1ac4105a40 2024-03-29T04:56:15.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.191+0000 7f1ab37fe700 1 --2- 172.21.15.142:0/3248247816 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1aac06dd60 0x7f1aac070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:15.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.191+0000 7f1ab37fe700 1 -- 172.21.15.142:0/3248247816 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1ab808f230 con 0x7f1ac4105a40 2024-03-29T04:56:15.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.191+0000 7f1ac259c700 1 --2- 172.21.15.142:0/3248247816 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1aac06dd60 0x7f1aac070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:15.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.192+0000 7f1ac259c700 1 --2- 172.21.15.142:0/3248247816 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1aac06dd60 0x7f1aac070220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4005950 tx=0x7f1ab4005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:15.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.196+0000 7f1ab37fe700 1 -- 172.21.15.142:0/3248247816 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1ab8058700 con 0x7f1ac4105a40 2024-03-29T04:56:15.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:14 smithi152 ceph-mon[88186]: pgmap v238: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:15.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:56:15.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.409+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3248247816 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ac4197160 con 0x7f1aac06dd60 2024-03-29T04:56:15.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.410+0000 7f1ab37fe700 1 -- 172.21.15.142:0/3248247816 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1ac4197160 con 0x7f1aac06dd60 2024-03-29T04:56:15.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.413+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3248247816 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1aac06dd60 msgr2=0x7f1aac070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:15.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.413+0000 7f1ac8e75700 1 --2- 172.21.15.142:0/3248247816 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1aac06dd60 0x7f1aac070220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4005950 tx=0x7f1ab4005f50 comp rx=0 tx=0).stop 2024-03-29T04:56:15.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.413+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3248247816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4105a40 msgr2=0x7f1ac419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:15.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.413+0000 7f1ac8e75700 1 --2- 172.21.15.142:0/3248247816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4105a40 0x7f1ac419caa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006010 tx=0x7f1ab800be20 comp rx=0 tx=0).stop 2024-03-29T04:56:15.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.413+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3248247816 shutdown_connections 2024-03-29T04:56:15.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.413+0000 7f1ac8e75700 1 --2- 172.21.15.142:0/3248247816 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1aac06dd60 0x7f1aac070220 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.413+0000 7f1ac8e75700 1 --2- 172.21.15.142:0/3248247816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac40ffbd0 0x7f1ac419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.413+0000 7f1ac8e75700 1 --2- 172.21.15.142:0/3248247816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4105a40 0x7f1ac419caa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.413+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3248247816 >> 172.21.15.142:0/3248247816 conn(0x7f1ac40755a0 msgr2=0x7f1ac4103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:15.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.413+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3248247816 shutdown_connections 2024-03-29T04:56:15.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.413+0000 7f1ac8e75700 1 -- 172.21.15.142:0/3248247816 wait complete. 2024-03-29T04:56:15.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.517+0000 7f573e7d1700 1 -- 172.21.15.142:0/3684688821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738101770 msgr2=0x7f57381057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:15.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.517+0000 7f573e7d1700 1 --2- 172.21.15.142:0/3684688821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738101770 0x7f57381057c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5728009910 tx=0x7f5728009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:15.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.517+0000 7f573e7d1700 1 -- 172.21.15.142:0/3684688821 shutdown_connections 2024-03-29T04:56:15.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.517+0000 7f573e7d1700 1 --2- 172.21.15.142:0/3684688821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738101770 0x7f57381057c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.517+0000 7f573e7d1700 1 --2- 172.21.15.142:0/3684688821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5738100dc0 0x7f57381011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.517+0000 7f573e7d1700 1 -- 172.21.15.142:0/3684688821 >> 172.21.15.142:0/3684688821 conn(0x7f57380fc650 msgr2=0x7f57380fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:15.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.517+0000 7f573e7d1700 1 -- 172.21.15.142:0/3684688821 shutdown_connections 2024-03-29T04:56:15.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.517+0000 7f573e7d1700 1 -- 172.21.15.142:0/3684688821 wait complete. 2024-03-29T04:56:15.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.518+0000 7f573e7d1700 1 Processor -- start 2024-03-29T04:56:15.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.518+0000 7f573e7d1700 1 -- start start 2024-03-29T04:56:15.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.518+0000 7f573e7d1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738100dc0 0x7f573819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:15.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.518+0000 7f573e7d1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5738101770 0x7f573819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:15.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.518+0000 7f573e7d1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573819d130 con 0x7f5738100dc0 2024-03-29T04:56:15.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.518+0000 7f573e7d1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57381965e0 con 0x7f5738101770 2024-03-29T04:56:15.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.518+0000 7f5737fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738100dc0 0x7f573819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:15.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.518+0000 7f5737fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738100dc0 0x7f573819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54542/0 (socket says 172.21.15.142:54542) 2024-03-29T04:56:15.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.518+0000 7f5737fff700 1 -- 172.21.15.142:0/2676860235 learned_addr learned my addr 172.21.15.142:0/2676860235 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:15.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.518+0000 7f57377fe700 1 --2- 172.21.15.142:0/2676860235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5738101770 0x7f573819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:15.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.519+0000 7f5737fff700 1 -- 172.21.15.142:0/2676860235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5738101770 msgr2=0x7f573819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:15.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.519+0000 7f5737fff700 1 --2- 172.21.15.142:0/2676860235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5738101770 0x7f573819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.519+0000 7f5737fff700 1 -- 172.21.15.142:0/2676860235 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57280095f0 con 0x7f5738100dc0 2024-03-29T04:56:15.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.519+0000 7f5737fff700 1 --2- 172.21.15.142:0/2676860235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738100dc0 0x7f573819c560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d8d0 tx=0x7f572c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:15.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.519+0000 7f57357fa700 1 -- 172.21.15.142:0/2676860235 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f572c0098a0 con 0x7f5738100dc0 2024-03-29T04:56:15.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.519+0000 7f573e7d1700 1 -- 172.21.15.142:0/2676860235 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57381968c0 con 0x7f5738100dc0 2024-03-29T04:56:15.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.519+0000 7f57357fa700 1 -- 172.21.15.142:0/2676860235 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f572c010460 con 0x7f5738100dc0 2024-03-29T04:56:15.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.519+0000 7f57357fa700 1 -- 172.21.15.142:0/2676860235 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f572c00f5f0 con 0x7f5738100dc0 2024-03-29T04:56:15.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.519+0000 7f573e7d1700 1 -- 172.21.15.142:0/2676860235 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5738196e10 con 0x7f5738100dc0 2024-03-29T04:56:15.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.520+0000 7f573e7d1700 1 -- 172.21.15.142:0/2676860235 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57381091c0 con 0x7f5738100dc0 2024-03-29T04:56:15.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.521+0000 7f57357fa700 1 -- 172.21.15.142:0/2676860235 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f572c009a00 con 0x7f5738100dc0 2024-03-29T04:56:15.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.521+0000 7f57357fa700 1 --2- 172.21.15.142:0/2676860235 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f572006dd60 0x7f5720070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:15.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.521+0000 7f57357fa700 1 -- 172.21.15.142:0/2676860235 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f572c08db40 con 0x7f5738100dc0 2024-03-29T04:56:15.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.521+0000 7f57377fe700 1 --2- 172.21.15.142:0/2676860235 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f572006dd60 0x7f5720070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:15.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.522+0000 7f57377fe700 1 --2- 172.21.15.142:0/2676860235 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f572006dd60 0x7f5720070220 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f57380fcea0 tx=0x7f572801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:15.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.526+0000 7f57357fa700 1 -- 172.21.15.142:0/2676860235 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f572c05aa90 con 0x7f5738100dc0 2024-03-29T04:56:15.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.732+0000 7f573e7d1700 1 -- 172.21.15.142:0/2676860235 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f57381970b0 con 0x7f572006dd60 2024-03-29T04:56:15.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.743+0000 7f57357fa700 1 -- 172.21.15.142:0/2676860235 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f57381970b0 con 0x7f572006dd60 2024-03-29T04:56:15.750 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:56:15.750 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (8m) 71s ago 19m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:56:15.750 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (19m) 71s ago 19m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:56:15.750 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (18m) 38s ago 18m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:56:15.750 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (5m) 71s ago 19m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:56:15.750 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (5m) 38s ago 18m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:56:15.750 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (7m) 71s ago 19m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:56:15.750 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (77s) 71s ago 13m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T04:56:15.750 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (93s) 71s ago 14m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T04:56:15.750 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (41s) 38s ago 13m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (57s) 38s ago 13m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (9m) 71s ago 21m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (8m) 38s ago 18m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (6m) 71s ago 21m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (5m) 38s ago 18m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (8m) 71s ago 19m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (8m) 38s ago 18m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (4m) 71s ago 17m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (4m) 71s ago 17m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (3m) 71s ago 16m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (3m) 38s ago 16m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (2m) 38s ago 16m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (112s) 38s ago 15m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T04:56:15.751 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (8m) 71s ago 19m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:56:15.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.745+0000 7f573e7d1700 1 -- 172.21.15.142:0/2676860235 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f572006dd60 msgr2=0x7f5720070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:15.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.745+0000 7f573e7d1700 1 --2- 172.21.15.142:0/2676860235 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f572006dd60 0x7f5720070220 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f57380fcea0 tx=0x7f572801a040 comp rx=0 tx=0).stop 2024-03-29T04:56:15.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.745+0000 7f573e7d1700 1 -- 172.21.15.142:0/2676860235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738100dc0 msgr2=0x7f573819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:15.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.745+0000 7f573e7d1700 1 --2- 172.21.15.142:0/2676860235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738100dc0 0x7f573819c560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d8d0 tx=0x7f572c00dbe0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.745+0000 7f573e7d1700 1 -- 172.21.15.142:0/2676860235 shutdown_connections 2024-03-29T04:56:15.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.745+0000 7f573e7d1700 1 --2- 172.21.15.142:0/2676860235 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f572006dd60 0x7f5720070220 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.745+0000 7f573e7d1700 1 --2- 172.21.15.142:0/2676860235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5738100dc0 0x7f573819c560 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.745+0000 7f573e7d1700 1 --2- 172.21.15.142:0/2676860235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5738101770 0x7f573819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.745+0000 7f573e7d1700 1 -- 172.21.15.142:0/2676860235 >> 172.21.15.142:0/2676860235 conn(0x7f57380fc650 msgr2=0x7f57380fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:15.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.745+0000 7f573e7d1700 1 -- 172.21.15.142:0/2676860235 shutdown_connections 2024-03-29T04:56:15.753 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.745+0000 7f573e7d1700 1 -- 172.21.15.142:0/2676860235 wait complete. 2024-03-29T04:56:15.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.849+0000 7fef30834700 1 -- 172.21.15.142:0/1800675911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef2c1026d0 msgr2=0x7fef2c10abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:15.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.849+0000 7fef30834700 1 --2- 172.21.15.142:0/1800675911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef2c1026d0 0x7fef2c10abc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fef20009910 tx=0x7fef20009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:15.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.849+0000 7fef30834700 1 -- 172.21.15.142:0/1800675911 shutdown_connections 2024-03-29T04:56:15.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.849+0000 7fef30834700 1 --2- 172.21.15.142:0/1800675911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef2c1026d0 0x7fef2c10abc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.849+0000 7fef30834700 1 --2- 172.21.15.142:0/1800675911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef2c101db0 0x7fef2c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.849+0000 7fef30834700 1 -- 172.21.15.142:0/1800675911 >> 172.21.15.142:0/1800675911 conn(0x7fef2c0fb540 msgr2=0x7fef2c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:15.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.849+0000 7fef30834700 1 -- 172.21.15.142:0/1800675911 shutdown_connections 2024-03-29T04:56:15.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.849+0000 7fef30834700 1 -- 172.21.15.142:0/1800675911 wait complete. 2024-03-29T04:56:15.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.850+0000 7fef30834700 1 Processor -- start 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.850+0000 7fef30834700 1 -- start start 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.850+0000 7fef30834700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef2c101db0 0x7fef2c19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.850+0000 7fef30834700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef2c1026d0 0x7fef2c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.850+0000 7fef30834700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c19d090 con 0x7fef2c1026d0 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.850+0000 7fef30834700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c1965d0 con 0x7fef2c101db0 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.850+0000 7fef29d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef2c1026d0 0x7fef2c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.850+0000 7fef29d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef2c1026d0 0x7fef2c19ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54562/0 (socket says 172.21.15.142:54562) 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.850+0000 7fef29d9b700 1 -- 172.21.15.142:0/2934231560 learned_addr learned my addr 172.21.15.142:0/2934231560 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.850+0000 7fef2a59c700 1 --2- 172.21.15.142:0/2934231560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef2c101db0 0x7fef2c19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.851+0000 7fef29d9b700 1 -- 172.21.15.142:0/2934231560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef2c101db0 msgr2=0x7fef2c19c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.851+0000 7fef29d9b700 1 --2- 172.21.15.142:0/2934231560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef2c101db0 0x7fef2c19c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.851+0000 7fef29d9b700 1 -- 172.21.15.142:0/2934231560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef200095f0 con 0x7fef2c1026d0 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.851+0000 7fef29d9b700 1 --2- 172.21.15.142:0/2934231560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef2c1026d0 0x7fef2c19ca90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fef2000bda0 tx=0x7fef2000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:15.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.851+0000 7fef1b7fe700 1 -- 172.21.15.142:0/2934231560 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef2001d070 con 0x7fef2c1026d0 2024-03-29T04:56:15.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.851+0000 7fef30834700 1 -- 172.21.15.142:0/2934231560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2c196850 con 0x7fef2c1026d0 2024-03-29T04:56:15.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.851+0000 7fef1b7fe700 1 -- 172.21.15.142:0/2934231560 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef2000f460 con 0x7fef2c1026d0 2024-03-29T04:56:15.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.851+0000 7fef1b7fe700 1 -- 172.21.15.142:0/2934231560 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef20017630 con 0x7fef2c1026d0 2024-03-29T04:56:15.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.851+0000 7fef30834700 1 -- 172.21.15.142:0/2934231560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2c196d40 con 0x7fef2c1026d0 2024-03-29T04:56:15.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.852+0000 7fef30834700 1 -- 172.21.15.142:0/2934231560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2c108250 con 0x7fef2c1026d0 2024-03-29T04:56:15.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.853+0000 7fef1b7fe700 1 -- 172.21.15.142:0/2934231560 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fef2000fab0 con 0x7fef2c1026d0 2024-03-29T04:56:15.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.853+0000 7fef1b7fe700 1 --2- 172.21.15.142:0/2934231560 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fef1406dd60 0x7fef14070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:15.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.853+0000 7fef1b7fe700 1 -- 172.21.15.142:0/2934231560 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fef2008f190 con 0x7fef2c1026d0 2024-03-29T04:56:15.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.853+0000 7fef2a59c700 1 --2- 172.21.15.142:0/2934231560 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fef1406dd60 0x7fef14070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:15.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.854+0000 7fef2a59c700 1 --2- 172.21.15.142:0/2934231560 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fef1406dd60 0x7fef14070220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00ac50 tx=0x7fef1c00a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:15.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:15.858+0000 7fef1b7fe700 1 -- 172.21.15.142:0/2934231560 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fef2005c0e0 con 0x7fef2c1026d0 2024-03-29T04:56:16.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:15 smithi142 ceph-mon[112475]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:16.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:15 smithi142 ceph-mon[112475]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:16.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.145+0000 7fef30834700 1 -- 172.21.15.142:0/2934231560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fef2c04ea90 con 0x7fef2c1026d0 2024-03-29T04:56:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.145+0000 7fef1b7fe700 1 -- 172.21.15.142:0/2934231560 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fef2005bc70 con 0x7fef2c1026d0 2024-03-29T04:56:16.152 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:56:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:56:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:56:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:56:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:56:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:56:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:56:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:56:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T04:56:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:56:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:56:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T04:56:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:56:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:56:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T04:56:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:56:16.153 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:56:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.148+0000 7fef30834700 1 -- 172.21.15.142:0/2934231560 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fef1406dd60 msgr2=0x7fef14070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.148+0000 7fef30834700 1 --2- 172.21.15.142:0/2934231560 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fef1406dd60 0x7fef14070220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00ac50 tx=0x7fef1c00a380 comp rx=0 tx=0).stop 2024-03-29T04:56:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.148+0000 7fef30834700 1 -- 172.21.15.142:0/2934231560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef2c1026d0 msgr2=0x7fef2c19ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:16.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.148+0000 7fef30834700 1 --2- 172.21.15.142:0/2934231560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef2c1026d0 0x7fef2c19ca90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fef2000bda0 tx=0x7fef2000bdd0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.148+0000 7fef30834700 1 -- 172.21.15.142:0/2934231560 shutdown_connections 2024-03-29T04:56:16.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.148+0000 7fef30834700 1 --2- 172.21.15.142:0/2934231560 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fef1406dd60 0x7fef14070220 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.148+0000 7fef30834700 1 --2- 172.21.15.142:0/2934231560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef2c101db0 0x7fef2c19c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.148+0000 7fef30834700 1 --2- 172.21.15.142:0/2934231560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef2c1026d0 0x7fef2c19ca90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.148+0000 7fef30834700 1 -- 172.21.15.142:0/2934231560 >> 172.21.15.142:0/2934231560 conn(0x7fef2c0fb540 msgr2=0x7fef2c105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:16.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.148+0000 7fef30834700 1 -- 172.21.15.142:0/2934231560 shutdown_connections 2024-03-29T04:56:16.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.148+0000 7fef30834700 1 -- 172.21.15.142:0/2934231560 wait complete. 2024-03-29T04:56:16.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.252+0000 7ffa4e573700 1 -- 172.21.15.142:0/496816467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa480ffbd0 msgr2=0x7ffa48100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:16.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.252+0000 7ffa4e573700 1 --2- 172.21.15.142:0/496816467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa480ffbd0 0x7ffa48100050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009910 tx=0x7ffa38009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:16.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.252+0000 7ffa4e573700 1 -- 172.21.15.142:0/496816467 shutdown_connections 2024-03-29T04:56:16.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.252+0000 7ffa4e573700 1 --2- 172.21.15.142:0/496816467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa480ffbd0 0x7ffa48100050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.252+0000 7ffa4e573700 1 --2- 172.21.15.142:0/496816467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa48105a40 0x7ffa48105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.252+0000 7ffa4e573700 1 -- 172.21.15.142:0/496816467 >> 172.21.15.142:0/496816467 conn(0x7ffa480755a0 msgr2=0x7ffa480759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:16.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.252+0000 7ffa4e573700 1 -- 172.21.15.142:0/496816467 shutdown_connections 2024-03-29T04:56:16.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.252+0000 7ffa4e573700 1 -- 172.21.15.142:0/496816467 wait complete. 2024-03-29T04:56:16.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.253+0000 7ffa4e573700 1 Processor -- start 2024-03-29T04:56:16.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.253+0000 7ffa4e573700 1 -- start start 2024-03-29T04:56:16.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.253+0000 7ffa4e573700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa480ffbd0 0x7ffa48101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:16.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.253+0000 7ffa4e573700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa48105a40 0x7ffa48101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:16.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.253+0000 7ffa4e573700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa48102090 con 0x7ffa480ffbd0 2024-03-29T04:56:16.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.253+0000 7ffa4e573700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa481021d0 con 0x7ffa48105a40 2024-03-29T04:56:16.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.253+0000 7ffa47fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa480ffbd0 0x7ffa48101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:16.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.253+0000 7ffa47fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa480ffbd0 0x7ffa48101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54584/0 (socket says 172.21.15.142:54584) 2024-03-29T04:56:16.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.253+0000 7ffa47fff700 1 -- 172.21.15.142:0/2331525493 learned_addr learned my addr 172.21.15.142:0/2331525493 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:16.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.254+0000 7ffa477fe700 1 --2- 172.21.15.142:0/2331525493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa48105a40 0x7ffa48101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:16.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.254+0000 7ffa47fff700 1 -- 172.21.15.142:0/2331525493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa48105a40 msgr2=0x7ffa48101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:16.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.254+0000 7ffa47fff700 1 --2- 172.21.15.142:0/2331525493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa48105a40 0x7ffa48101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.263 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.254+0000 7ffa47fff700 1 -- 172.21.15.142:0/2331525493 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa380095f0 con 0x7ffa480ffbd0 2024-03-29T04:56:16.263 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.254+0000 7ffa47fff700 1 --2- 172.21.15.142:0/2331525493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa480ffbd0 0x7ffa48101610 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00d8d0 tx=0x7ffa3c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:16.263 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.254+0000 7ffa457fa700 1 -- 172.21.15.142:0/2331525493 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3c0098a0 con 0x7ffa480ffbd0 2024-03-29T04:56:16.263 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.254+0000 7ffa457fa700 1 -- 172.21.15.142:0/2331525493 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa3c010460 con 0x7ffa480ffbd0 2024-03-29T04:56:16.263 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.254+0000 7ffa4e573700 1 -- 172.21.15.142:0/2331525493 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa481a2900 con 0x7ffa480ffbd0 2024-03-29T04:56:16.263 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.254+0000 7ffa457fa700 1 -- 172.21.15.142:0/2331525493 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3c00f5f0 con 0x7ffa480ffbd0 2024-03-29T04:56:16.263 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.254+0000 7ffa4e573700 1 -- 172.21.15.142:0/2331525493 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa481a2d10 con 0x7ffa480ffbd0 2024-03-29T04:56:16.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.255+0000 7ffa4e573700 1 -- 172.21.15.142:0/2331525493 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa48108fe0 con 0x7ffa480ffbd0 2024-03-29T04:56:16.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.256+0000 7ffa457fa700 1 -- 172.21.15.142:0/2331525493 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ffa3c009a00 con 0x7ffa480ffbd0 2024-03-29T04:56:16.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.256+0000 7ffa457fa700 1 --2- 172.21.15.142:0/2331525493 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffa3006dd60 0x7ffa30070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:16.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.256+0000 7ffa457fa700 1 -- 172.21.15.142:0/2331525493 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ffa3c08db40 con 0x7ffa480ffbd0 2024-03-29T04:56:16.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.256+0000 7ffa477fe700 1 --2- 172.21.15.142:0/2331525493 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffa3006dd60 0x7ffa30070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:16.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.257+0000 7ffa477fe700 1 --2- 172.21.15.142:0/2331525493 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffa3006dd60 0x7ffa30070220 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffa48102e00 tx=0x7ffa3801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:16.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.261+0000 7ffa457fa700 1 -- 172.21.15.142:0/2331525493 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ffa3c056f60 con 0x7ffa480ffbd0 2024-03-29T04:56:16.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:15 smithi152 ceph-mon[88186]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:16.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:15 smithi152 ceph-mon[88186]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:16.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.499+0000 7ffa4e573700 1 -- 172.21.15.142:0/2331525493 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffa4802cdd0 con 0x7ffa480ffbd0 2024-03-29T04:56:16.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.499+0000 7ffa457fa700 1 -- 172.21.15.142:0/2331525493 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7ffa3c05a620 con 0x7ffa480ffbd0 2024-03-29T04:56:16.506 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T04:56:16.506 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:56:16.506 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:56:16.506 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:56:16.506 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:56:16.507 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:56:16.508 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:56:16.509 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:56:16.509 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:56:16.509 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:56:16.509 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:56:16.509 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T04:56:16.509 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:56:16.509 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:56:16.509 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:56:16.509 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:56:16.509 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:56:16.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.502+0000 7ffa4e573700 1 -- 172.21.15.142:0/2331525493 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffa3006dd60 msgr2=0x7ffa30070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:16.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.502+0000 7ffa4e573700 1 --2- 172.21.15.142:0/2331525493 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffa3006dd60 0x7ffa30070220 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffa48102e00 tx=0x7ffa3801a040 comp rx=0 tx=0).stop 2024-03-29T04:56:16.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.502+0000 7ffa4e573700 1 -- 172.21.15.142:0/2331525493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa480ffbd0 msgr2=0x7ffa48101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:16.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.502+0000 7ffa4e573700 1 --2- 172.21.15.142:0/2331525493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa480ffbd0 0x7ffa48101610 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00d8d0 tx=0x7ffa3c00dbe0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.502+0000 7ffa4e573700 1 -- 172.21.15.142:0/2331525493 shutdown_connections 2024-03-29T04:56:16.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.502+0000 7ffa4e573700 1 --2- 172.21.15.142:0/2331525493 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffa3006dd60 0x7ffa30070220 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.502+0000 7ffa4e573700 1 --2- 172.21.15.142:0/2331525493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa480ffbd0 0x7ffa48101610 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.502+0000 7ffa4e573700 1 --2- 172.21.15.142:0/2331525493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa48105a40 0x7ffa48101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.502+0000 7ffa4e573700 1 -- 172.21.15.142:0/2331525493 >> 172.21.15.142:0/2331525493 conn(0x7ffa480755a0 msgr2=0x7ffa48068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:16.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.502+0000 7ffa4e573700 1 -- 172.21.15.142:0/2331525493 shutdown_connections 2024-03-29T04:56:16.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.502+0000 7ffa4e573700 1 -- 172.21.15.142:0/2331525493 wait complete. 2024-03-29T04:56:16.511 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T04:56:16.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.607+0000 7f6cefa3e700 1 -- 172.21.15.142:0/1202231065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ce8073130 msgr2=0x7f6ce8073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:16.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.608+0000 7f6cefa3e700 1 --2- 172.21.15.142:0/1202231065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ce8073130 0x7f6ce8073510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009910 tx=0x7f6ce4009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:16.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.608+0000 7f6cefa3e700 1 -- 172.21.15.142:0/1202231065 shutdown_connections 2024-03-29T04:56:16.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.608+0000 7f6cefa3e700 1 --2- 172.21.15.142:0/1202231065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ce8073a50 0x7f6ce8111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.608+0000 7f6cefa3e700 1 --2- 172.21.15.142:0/1202231065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ce8073130 0x7f6ce8073510 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.608+0000 7f6cefa3e700 1 -- 172.21.15.142:0/1202231065 >> 172.21.15.142:0/1202231065 conn(0x7f6ce80fc650 msgr2=0x7f6ce80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.608+0000 7f6cefa3e700 1 -- 172.21.15.142:0/1202231065 shutdown_connections 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.608+0000 7f6cefa3e700 1 -- 172.21.15.142:0/1202231065 wait complete. 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.608+0000 7f6cefa3e700 1 Processor -- start 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.608+0000 7f6cefa3e700 1 -- start start 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6cefa3e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ce8073130 0x7f6ce81a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6cefa3e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ce8073a50 0x7f6ce81a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6cefa3e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce81a36a0 con 0x7f6ce8073130 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6cefa3e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce819cba0 con 0x7f6ce8073a50 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6ced7da700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ce8073130 0x7f6ce81a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6ced7da700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ce8073130 0x7f6ce81a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54606/0 (socket says 172.21.15.142:54606) 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6ced7da700 1 -- 172.21.15.142:0/4151012194 learned_addr learned my addr 172.21.15.142:0/4151012194 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6cecfd9700 1 --2- 172.21.15.142:0/4151012194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ce8073a50 0x7f6ce81a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6ced7da700 1 -- 172.21.15.142:0/4151012194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ce8073a50 msgr2=0x7f6ce81a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:16.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6ced7da700 1 --2- 172.21.15.142:0/4151012194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ce8073a50 0x7f6ce81a3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.618 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6ced7da700 1 -- 172.21.15.142:0/4151012194 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce40095f0 con 0x7f6ce8073130 2024-03-29T04:56:16.618 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6cecfd9700 1 --2- 172.21.15.142:0/4151012194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ce8073a50 0x7f6ce81a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:56:16.618 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6ced7da700 1 --2- 172.21.15.142:0/4151012194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ce8073130 0x7f6ce81a2ad0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4006010 tx=0x7f6ce400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:16.618 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6cde7fc700 1 -- 172.21.15.142:0/4151012194 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ce401d070 con 0x7f6ce8073130 2024-03-29T04:56:16.618 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6cde7fc700 1 -- 172.21.15.142:0/4151012194 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ce400f460 con 0x7f6ce8073130 2024-03-29T04:56:16.618 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6cde7fc700 1 -- 172.21.15.142:0/4151012194 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ce4017630 con 0x7f6ce8073130 2024-03-29T04:56:16.618 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6cefa3e700 1 -- 172.21.15.142:0/4151012194 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce819ce80 con 0x7f6ce8073130 2024-03-29T04:56:16.618 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.609+0000 7f6cefa3e700 1 -- 172.21.15.142:0/4151012194 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce819d3d0 con 0x7f6ce8073130 2024-03-29T04:56:16.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.610+0000 7f6cefa3e700 1 -- 172.21.15.142:0/4151012194 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce810edb0 con 0x7f6ce8073130 2024-03-29T04:56:16.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.611+0000 7f6cde7fc700 1 -- 172.21.15.142:0/4151012194 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6ce400f5d0 con 0x7f6ce8073130 2024-03-29T04:56:16.619 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.611+0000 7f6cde7fc700 1 --2- 172.21.15.142:0/4151012194 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6cd406dd60 0x7f6cd4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:16.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.611+0000 7f6cde7fc700 1 -- 172.21.15.142:0/4151012194 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6ce408f0c0 con 0x7f6ce8073130 2024-03-29T04:56:16.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.611+0000 7f6cecfd9700 1 --2- 172.21.15.142:0/4151012194 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6cd406dd60 0x7f6cd4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:16.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.612+0000 7f6cecfd9700 1 --2- 172.21.15.142:0/4151012194 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6cd406dd60 0x7f6cd4070220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80fde30 tx=0x7f6cd800b5f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:16.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.616+0000 7f6cde7fc700 1 -- 172.21.15.142:0/4151012194 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6ce405c010 con 0x7f6ce8073130 2024-03-29T04:56:16.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.828+0000 7f6cefa3e700 1 -- 172.21.15.142:0/4151012194 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ce819d640 con 0x7f6cd406dd60 2024-03-29T04:56:16.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.830+0000 7f6cde7fc700 1 -- 172.21.15.142:0/4151012194 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6ce819d640 con 0x7f6cd406dd60 2024-03-29T04:56:16.836 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:56:16.836 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:56:16.837 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:56:16.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.832+0000 7f6cefa3e700 1 -- 172.21.15.142:0/4151012194 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6cd406dd60 msgr2=0x7f6cd4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:16.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.832+0000 7f6cefa3e700 1 --2- 172.21.15.142:0/4151012194 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6cd406dd60 0x7f6cd4070220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80fde30 tx=0x7f6cd800b5f0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.832+0000 7f6cefa3e700 1 -- 172.21.15.142:0/4151012194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ce8073130 msgr2=0x7f6ce81a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:16.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.832+0000 7f6cefa3e700 1 --2- 172.21.15.142:0/4151012194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ce8073130 0x7f6ce81a2ad0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4006010 tx=0x7f6ce400be20 comp rx=0 tx=0).stop 2024-03-29T04:56:16.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.832+0000 7f6cefa3e700 1 -- 172.21.15.142:0/4151012194 shutdown_connections 2024-03-29T04:56:16.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.832+0000 7f6cefa3e700 1 --2- 172.21.15.142:0/4151012194 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6cd406dd60 0x7f6cd4070220 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.832+0000 7f6cefa3e700 1 --2- 172.21.15.142:0/4151012194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ce8073130 0x7f6ce81a2ad0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.832+0000 7f6cefa3e700 1 --2- 172.21.15.142:0/4151012194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ce8073a50 0x7f6ce81a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.832+0000 7f6cefa3e700 1 -- 172.21.15.142:0/4151012194 >> 172.21.15.142:0/4151012194 conn(0x7f6ce80fc650 msgr2=0x7f6ce8103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:16.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.832+0000 7f6cefa3e700 1 -- 172.21.15.142:0/4151012194 shutdown_connections 2024-03-29T04:56:16.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.832+0000 7f6cefa3e700 1 -- 172.21.15.142:0/4151012194 wait complete. 2024-03-29T04:56:16.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.937+0000 7f5c6bf84700 1 -- 172.21.15.142:0/3595870433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64068670 msgr2=0x7f5c64068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:16.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.937+0000 7f5c6bf84700 1 --2- 172.21.15.142:0/3595870433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64068670 0x7f5c64068a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009910 tx=0x7f5c60009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:16.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.937+0000 7f5c6bf84700 1 -- 172.21.15.142:0/3595870433 shutdown_connections 2024-03-29T04:56:16.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.937+0000 7f5c6bf84700 1 --2- 172.21.15.142:0/3595870433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c64069020 0x7f5c64105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.937+0000 7f5c6bf84700 1 --2- 172.21.15.142:0/3595870433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64068670 0x7f5c64068a50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.937+0000 7f5c6bf84700 1 -- 172.21.15.142:0/3595870433 >> 172.21.15.142:0/3595870433 conn(0x7f5c640755a0 msgr2=0x7f5c640759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:16.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.937+0000 7f5c6bf84700 1 -- 172.21.15.142:0/3595870433 shutdown_connections 2024-03-29T04:56:16.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.937+0000 7f5c6bf84700 1 -- 172.21.15.142:0/3595870433 wait complete. 2024-03-29T04:56:16.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.937+0000 7f5c6bf84700 1 Processor -- start 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.937+0000 7f5c6bf84700 1 -- start start 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6bf84700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c64068670 0x7f5c6419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6bf84700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64069020 0x7f5c6419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6bf84700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6419d0e0 con 0x7f5c64069020 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6bf84700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c64196590 con 0x7f5c64068670 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6951f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64069020 0x7f5c6419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6951f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64069020 0x7f5c6419ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54616/0 (socket says 172.21.15.142:54616) 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6951f700 1 -- 172.21.15.142:0/1786153803 learned_addr learned my addr 172.21.15.142:0/1786153803 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c69d20700 1 --2- 172.21.15.142:0/1786153803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c64068670 0x7f5c6419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6951f700 1 -- 172.21.15.142:0/1786153803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c64068670 msgr2=0x7f5c6419c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6951f700 1 --2- 172.21.15.142:0/1786153803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c64068670 0x7f5c6419c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6951f700 1 -- 172.21.15.142:0/1786153803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c600095f0 con 0x7f5c64069020 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6951f700 1 --2- 172.21.15.142:0/1786153803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64069020 0x7f5c6419ca50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400d8d0 tx=0x7f5c5400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:16.946 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c5affd700 1 -- 172.21.15.142:0/1786153803 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c540041f0 con 0x7f5c64069020 2024-03-29T04:56:16.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6bf84700 1 -- 172.21.15.142:0/1786153803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c64196870 con 0x7f5c64069020 2024-03-29T04:56:16.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c5affd700 1 -- 172.21.15.142:0/1786153803 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c54007620 con 0x7f5c64069020 2024-03-29T04:56:16.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c5affd700 1 -- 172.21.15.142:0/1786153803 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c540056b0 con 0x7f5c64069020 2024-03-29T04:56:16.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.938+0000 7f5c6bf84700 1 -- 172.21.15.142:0/1786153803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c64196dc0 con 0x7f5c64069020 2024-03-29T04:56:16.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.940+0000 7f5c6bf84700 1 -- 172.21.15.142:0/1786153803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c64109130 con 0x7f5c64069020 2024-03-29T04:56:16.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.940+0000 7f5c5affd700 1 -- 172.21.15.142:0/1786153803 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f5c54004750 con 0x7f5c64069020 2024-03-29T04:56:16.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.940+0000 7f5c5affd700 1 --2- 172.21.15.142:0/1786153803 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5c5006dd60 0x7f5c50070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:16.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.940+0000 7f5c5affd700 1 -- 172.21.15.142:0/1786153803 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f5c5408dad0 con 0x7f5c64069020 2024-03-29T04:56:16.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.940+0000 7f5c69d20700 1 --2- 172.21.15.142:0/1786153803 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5c5006dd60 0x7f5c50070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:16.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.941+0000 7f5c69d20700 1 --2- 172.21.15.142:0/1786153803 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5c5006dd60 0x7f5c50070220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c60006010 tx=0x7f5c6001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:16.951 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:16.945+0000 7f5c5affd700 1 -- 172.21.15.142:0/1786153803 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5c5405aa20 con 0x7f5c64069020 2024-03-29T04:56:17.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:16 smithi142 ceph-mon[112475]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:17.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:16 smithi142 ceph-mon[112475]: pgmap v239: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:17.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:16 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2934231560' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:56:17.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:16 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2331525493' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:56:17.224 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.218+0000 7f5c6bf84700 1 -- 172.21.15.142:0/1786153803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5c64197060 con 0x7f5c64069020 2024-03-29T04:56:17.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.218+0000 7f5c5affd700 1 -- 172.21.15.142:0/1786153803 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f5c5405a5b0 con 0x7f5c64069020 2024-03-29T04:56:17.225 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T04:56:17.225 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T04:56:17.225 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T04:56:17.225 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T04:56:17.225 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T04:56:17.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.220+0000 7f5c6bf84700 1 -- 172.21.15.142:0/1786153803 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5c5006dd60 msgr2=0x7f5c50070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:17.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.220+0000 7f5c6bf84700 1 --2- 172.21.15.142:0/1786153803 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5c5006dd60 0x7f5c50070220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c60006010 tx=0x7f5c6001a040 comp rx=0 tx=0).stop 2024-03-29T04:56:17.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.220+0000 7f5c6bf84700 1 -- 172.21.15.142:0/1786153803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64069020 msgr2=0x7f5c6419ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:17.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.220+0000 7f5c6bf84700 1 --2- 172.21.15.142:0/1786153803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64069020 0x7f5c6419ca50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400d8d0 tx=0x7f5c5400dbe0 comp rx=0 tx=0).stop 2024-03-29T04:56:17.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.220+0000 7f5c6bf84700 1 -- 172.21.15.142:0/1786153803 shutdown_connections 2024-03-29T04:56:17.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.220+0000 7f5c6bf84700 1 --2- 172.21.15.142:0/1786153803 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5c5006dd60 0x7f5c50070220 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:17.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.220+0000 7f5c6bf84700 1 --2- 172.21.15.142:0/1786153803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c64068670 0x7f5c6419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:17.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.220+0000 7f5c6bf84700 1 --2- 172.21.15.142:0/1786153803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c64069020 0x7f5c6419ca50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:17.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.220+0000 7f5c6bf84700 1 -- 172.21.15.142:0/1786153803 >> 172.21.15.142:0/1786153803 conn(0x7f5c640755a0 msgr2=0x7f5c640fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:17.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.220+0000 7f5c6bf84700 1 -- 172.21.15.142:0/1786153803 shutdown_connections 2024-03-29T04:56:17.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:17.220+0000 7f5c6bf84700 1 -- 172.21.15.142:0/1786153803 wait complete. 2024-03-29T04:56:17.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:16 smithi152 ceph-mon[88186]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:17.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:16 smithi152 ceph-mon[88186]: pgmap v239: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:17.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:16 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2934231560' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:56:17.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:16 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2331525493' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:56:18.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:17 smithi152 ceph-mon[88186]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:18.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:17 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1786153803' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:56:18.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:17 smithi142 ceph-mon[112475]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:18.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:17 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1786153803' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:56:19.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:18 smithi152 ceph-mon[88186]: pgmap v240: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:19.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:18 smithi142 ceph-mon[112475]: pgmap v240: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:21.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:20 smithi152 ceph-mon[88186]: pgmap v241: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:21.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:20 smithi142 ceph-mon[112475]: pgmap v241: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:23.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:22 smithi152 ceph-mon[88186]: pgmap v242: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:23.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:22 smithi142 ceph-mon[112475]: pgmap v242: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:25.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:24 smithi152 ceph-mon[88186]: pgmap v243: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:25.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:24 smithi142 ceph-mon[112475]: pgmap v243: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:27.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:26 smithi152 ceph-mon[88186]: pgmap v244: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:27.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:26 smithi142 ceph-mon[112475]: pgmap v244: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:29.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:28 smithi152 ceph-mon[88186]: pgmap v245: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:29.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:28 smithi142 ceph-mon[112475]: pgmap v245: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:30.298 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:56:30.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:56:31.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:30 smithi152 ceph-mon[88186]: pgmap v246: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:31.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:30 smithi142 ceph-mon[112475]: pgmap v246: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:33.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:32 smithi152 ceph-mon[88186]: pgmap v247: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:33.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:32 smithi142 ceph-mon[112475]: pgmap v247: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:35.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:34 smithi152 ceph-mon[88186]: pgmap v248: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:35.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:34 smithi142 ceph-mon[112475]: pgmap v248: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:37.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:36 smithi152 ceph-mon[88186]: pgmap v249: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:37.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:36 smithi142 ceph-mon[112475]: pgmap v249: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:39.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:38 smithi152 ceph-mon[88186]: pgmap v250: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:39.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:38 smithi142 ceph-mon[112475]: pgmap v250: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:41.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:40 smithi152 ceph-mon[88186]: pgmap v251: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:41.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:40 smithi142 ceph-mon[112475]: pgmap v251: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:43.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:42 smithi152 ceph-mon[88186]: pgmap v252: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:43.377 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:42 smithi142 ceph-mon[112475]: pgmap v252: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:45.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:44 smithi152 ceph-mon[88186]: pgmap v253: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:45.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:56:45.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:44 smithi142 ceph-mon[112475]: pgmap v253: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:45.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:56:47.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:46 smithi152 ceph-mon[88186]: pgmap v254: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:47.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.339+0000 7ff89b62f700 1 -- 172.21.15.142:0/2825043876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff894101790 msgr2=0x7ff8941057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:47.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.339+0000 7ff89b62f700 1 --2- 172.21.15.142:0/2825043876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff894101790 0x7ff8941057e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff884009910 tx=0x7ff884009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:47.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.339+0000 7ff89b62f700 1 -- 172.21.15.142:0/2825043876 shutdown_connections 2024-03-29T04:56:47.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.339+0000 7ff89b62f700 1 --2- 172.21.15.142:0/2825043876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff894101790 0x7ff8941057e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.339+0000 7ff89b62f700 1 --2- 172.21.15.142:0/2825043876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff894100de0 0x7ff8941011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.339+0000 7ff89b62f700 1 -- 172.21.15.142:0/2825043876 >> 172.21.15.142:0/2825043876 conn(0x7ff8940fc670 msgr2=0x7ff8940fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:47.347 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:46 smithi142 ceph-mon[112475]: pgmap v254: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:47.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.339+0000 7ff89b62f700 1 -- 172.21.15.142:0/2825043876 shutdown_connections 2024-03-29T04:56:47.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.339+0000 7ff89b62f700 1 -- 172.21.15.142:0/2825043876 wait complete. 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.340+0000 7ff89b62f700 1 Processor -- start 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.340+0000 7ff89b62f700 1 -- start start 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.340+0000 7ff89b62f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff894100de0 0x7ff89419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.340+0000 7ff89b62f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff894101790 0x7ff89419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.340+0000 7ff89b62f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89419d130 con 0x7ff894101790 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.340+0000 7ff89b62f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8941965e0 con 0x7ff894100de0 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.340+0000 7ff898bca700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff894101790 0x7ff89419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.340+0000 7ff898bca700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff894101790 0x7ff89419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44484/0 (socket says 172.21.15.142:44484) 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.340+0000 7ff898bca700 1 -- 172.21.15.142:0/1827182697 learned_addr learned my addr 172.21.15.142:0/1827182697 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.340+0000 7ff8993cb700 1 --2- 172.21.15.142:0/1827182697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff894100de0 0x7ff89419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.341+0000 7ff898bca700 1 -- 172.21.15.142:0/1827182697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff894100de0 msgr2=0x7ff89419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.341+0000 7ff898bca700 1 --2- 172.21.15.142:0/1827182697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff894100de0 0x7ff89419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.341+0000 7ff898bca700 1 -- 172.21.15.142:0/1827182697 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8840095f0 con 0x7ff894101790 2024-03-29T04:56:47.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.341+0000 7ff898bca700 1 --2- 172.21.15.142:0/1827182697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff894101790 0x7ff89419caa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff8840092e0 tx=0x7ff88400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:47.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.341+0000 7ff88a7fc700 1 -- 172.21.15.142:0/1827182697 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff88401d070 con 0x7ff894101790 2024-03-29T04:56:47.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.341+0000 7ff89b62f700 1 -- 172.21.15.142:0/1827182697 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff894196860 con 0x7ff894101790 2024-03-29T04:56:47.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.341+0000 7ff88a7fc700 1 -- 172.21.15.142:0/1827182697 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff884005dc0 con 0x7ff894101790 2024-03-29T04:56:47.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.341+0000 7ff88a7fc700 1 -- 172.21.15.142:0/1827182697 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff884017400 con 0x7ff894101790 2024-03-29T04:56:47.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.341+0000 7ff89b62f700 1 -- 172.21.15.142:0/1827182697 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff894196d50 con 0x7ff894101790 2024-03-29T04:56:47.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.342+0000 7ff89b62f700 1 -- 172.21.15.142:0/1827182697 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8941091e0 con 0x7ff894101790 2024-03-29T04:56:47.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.343+0000 7ff88a7fc700 1 -- 172.21.15.142:0/1827182697 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff88400f6f0 con 0x7ff894101790 2024-03-29T04:56:47.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.343+0000 7ff88a7fc700 1 --2- 172.21.15.142:0/1827182697 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff88006dd60 0x7ff880070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:47.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.343+0000 7ff88a7fc700 1 -- 172.21.15.142:0/1827182697 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff88408f060 con 0x7ff894101790 2024-03-29T04:56:47.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.343+0000 7ff8993cb700 1 --2- 172.21.15.142:0/1827182697 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff88006dd60 0x7ff880070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:47.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.344+0000 7ff8993cb700 1 --2- 172.21.15.142:0/1827182697 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff88006dd60 0x7ff880070220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff890009550 tx=0x7ff8900091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:47.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.348+0000 7ff88a7fc700 1 -- 172.21.15.142:0/1827182697 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff88405bfb0 con 0x7ff894101790 2024-03-29T04:56:47.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.562+0000 7ff89b62f700 1 -- 172.21.15.142:0/1827182697 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff89402cd00 con 0x7ff88006dd60 2024-03-29T04:56:47.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.563+0000 7ff88a7fc700 1 -- 172.21.15.142:0/1827182697 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff89402cd00 con 0x7ff88006dd60 2024-03-29T04:56:47.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.565+0000 7ff89b62f700 1 -- 172.21.15.142:0/1827182697 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff88006dd60 msgr2=0x7ff880070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:47.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.565+0000 7ff89b62f700 1 --2- 172.21.15.142:0/1827182697 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff88006dd60 0x7ff880070220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff890009550 tx=0x7ff8900091b0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.565+0000 7ff89b62f700 1 -- 172.21.15.142:0/1827182697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff894101790 msgr2=0x7ff89419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:47.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.565+0000 7ff89b62f700 1 --2- 172.21.15.142:0/1827182697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff894101790 0x7ff89419caa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff8840092e0 tx=0x7ff88400bd70 comp rx=0 tx=0).stop 2024-03-29T04:56:47.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.565+0000 7ff89b62f700 1 -- 172.21.15.142:0/1827182697 shutdown_connections 2024-03-29T04:56:47.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.565+0000 7ff89b62f700 1 --2- 172.21.15.142:0/1827182697 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff88006dd60 0x7ff880070220 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.565+0000 7ff89b62f700 1 --2- 172.21.15.142:0/1827182697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff894100de0 0x7ff89419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.565+0000 7ff89b62f700 1 --2- 172.21.15.142:0/1827182697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff894101790 0x7ff89419caa0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.565+0000 7ff89b62f700 1 -- 172.21.15.142:0/1827182697 >> 172.21.15.142:0/1827182697 conn(0x7ff8940fc670 msgr2=0x7ff8940fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:47.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.565+0000 7ff89b62f700 1 -- 172.21.15.142:0/1827182697 shutdown_connections 2024-03-29T04:56:47.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.565+0000 7ff89b62f700 1 -- 172.21.15.142:0/1827182697 wait complete. 2024-03-29T04:56:47.583 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:56:47.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.670+0000 7f638e3ed700 1 -- 172.21.15.142:0/1456620375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63880ffa40 msgr2=0x7f63880ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:47.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.670+0000 7f638e3ed700 1 --2- 172.21.15.142:0/1456620375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63880ffa40 0x7f63880ffec0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f637c009910 tx=0x7f637c009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.670+0000 7f638e3ed700 1 -- 172.21.15.142:0/1456620375 shutdown_connections 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.670+0000 7f638e3ed700 1 --2- 172.21.15.142:0/1456620375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63880ffa40 0x7f63880ffec0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.670+0000 7f638e3ed700 1 --2- 172.21.15.142:0/1456620375 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6388105a40 0x7f6388105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.670+0000 7f638e3ed700 1 -- 172.21.15.142:0/1456620375 >> 172.21.15.142:0/1456620375 conn(0x7f63880fb550 msgr2=0x7f63880fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.671+0000 7f638e3ed700 1 -- 172.21.15.142:0/1456620375 shutdown_connections 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.671+0000 7f638e3ed700 1 -- 172.21.15.142:0/1456620375 wait complete. 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.671+0000 7f638e3ed700 1 Processor -- start 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.671+0000 7f638e3ed700 1 -- start start 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.671+0000 7f638e3ed700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880ffa40 0x7f63881975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.671+0000 7f638e3ed700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6388105a40 0x7f6388197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f638e3ed700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6388198110 con 0x7f6388105a40 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f638e3ed700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6388198250 con 0x7f63880ffa40 2024-03-29T04:56:47.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f63877fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6388105a40 0x7f6388197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f63877fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6388105a40 0x7f6388197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44506/0 (socket says 172.21.15.142:44506) 2024-03-29T04:56:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f63877fe700 1 -- 172.21.15.142:0/1731745039 learned_addr learned my addr 172.21.15.142:0/1731745039 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f6387fff700 1 --2- 172.21.15.142:0/1731745039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880ffa40 0x7f63881975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f63877fe700 1 -- 172.21.15.142:0/1731745039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880ffa40 msgr2=0x7f63881975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f63877fe700 1 --2- 172.21.15.142:0/1731745039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880ffa40 0x7f63881975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f63877fe700 1 -- 172.21.15.142:0/1731745039 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c0095f0 con 0x7f6388105a40 2024-03-29T04:56:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f63877fe700 1 --2- 172.21.15.142:0/1731745039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6388105a40 0x7f6388197af0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f637c006010 tx=0x7f637c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f63857fa700 1 -- 172.21.15.142:0/1731745039 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f637c01d070 con 0x7f6388105a40 2024-03-29T04:56:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f63857fa700 1 -- 172.21.15.142:0/1731745039 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f637c00f460 con 0x7f6388105a40 2024-03-29T04:56:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f638e3ed700 1 -- 172.21.15.142:0/1731745039 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63881a0620 con 0x7f6388105a40 2024-03-29T04:56:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f63857fa700 1 -- 172.21.15.142:0/1731745039 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f637c017630 con 0x7f6388105a40 2024-03-29T04:56:47.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.672+0000 7f638e3ed700 1 -- 172.21.15.142:0/1731745039 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63881a0a30 con 0x7f6388105a40 2024-03-29T04:56:47.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.673+0000 7f638e3ed700 1 -- 172.21.15.142:0/1731745039 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6388107fd0 con 0x7f6388105a40 2024-03-29T04:56:47.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.674+0000 7f63857fa700 1 -- 172.21.15.142:0/1731745039 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f637c00f5d0 con 0x7f6388105a40 2024-03-29T04:56:47.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.674+0000 7f63857fa700 1 --2- 172.21.15.142:0/1731745039 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f637006dd60 0x7f6370070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:47.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.674+0000 7f63857fa700 1 -- 172.21.15.142:0/1731745039 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f637c08f140 con 0x7f6388105a40 2024-03-29T04:56:47.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.674+0000 7f6387fff700 1 --2- 172.21.15.142:0/1731745039 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f637006dd60 0x7f6370070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:47.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.675+0000 7f6387fff700 1 --2- 172.21.15.142:0/1731745039 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f637006dd60 0x7f6370070220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6378005950 tx=0x7f63780058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:47.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.679+0000 7f63857fa700 1 -- 172.21.15.142:0/1731745039 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f637c058560 con 0x7f6388105a40 2024-03-29T04:56:47.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.901+0000 7f638e3ed700 1 -- 172.21.15.142:0/1731745039 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f638802cd30 con 0x7f637006dd60 2024-03-29T04:56:47.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.902+0000 7f63857fa700 1 -- 172.21.15.142:0/1731745039 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f638802cd30 con 0x7f637006dd60 2024-03-29T04:56:47.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.904+0000 7f638e3ed700 1 -- 172.21.15.142:0/1731745039 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f637006dd60 msgr2=0x7f6370070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:47.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.904+0000 7f638e3ed700 1 --2- 172.21.15.142:0/1731745039 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f637006dd60 0x7f6370070220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6378005950 tx=0x7f63780058e0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.904+0000 7f638e3ed700 1 -- 172.21.15.142:0/1731745039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6388105a40 msgr2=0x7f6388197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:47.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.904+0000 7f638e3ed700 1 --2- 172.21.15.142:0/1731745039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6388105a40 0x7f6388197af0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f637c006010 tx=0x7f637c00be20 comp rx=0 tx=0).stop 2024-03-29T04:56:47.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.904+0000 7f638e3ed700 1 -- 172.21.15.142:0/1731745039 shutdown_connections 2024-03-29T04:56:47.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.905+0000 7f638e3ed700 1 --2- 172.21.15.142:0/1731745039 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f637006dd60 0x7f6370070220 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.905+0000 7f638e3ed700 1 --2- 172.21.15.142:0/1731745039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880ffa40 0x7f63881975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.905+0000 7f638e3ed700 1 --2- 172.21.15.142:0/1731745039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6388105a40 0x7f6388197af0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:47.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.905+0000 7f638e3ed700 1 -- 172.21.15.142:0/1731745039 >> 172.21.15.142:0/1731745039 conn(0x7f63880fb550 msgr2=0x7f63880fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:47.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.905+0000 7f638e3ed700 1 -- 172.21.15.142:0/1731745039 shutdown_connections 2024-03-29T04:56:47.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:47.905+0000 7f638e3ed700 1 -- 172.21.15.142:0/1731745039 wait complete. 2024-03-29T04:56:48.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.010+0000 7f35ea53a700 1 -- 172.21.15.142:0/2859423648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4101770 msgr2=0x7f35e41057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:48.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.010+0000 7f35ea53a700 1 --2- 172.21.15.142:0/2859423648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4101770 0x7f35e41057c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009910 tx=0x7f35d8009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:48.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.010+0000 7f35ea53a700 1 -- 172.21.15.142:0/2859423648 shutdown_connections 2024-03-29T04:56:48.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.010+0000 7f35ea53a700 1 --2- 172.21.15.142:0/2859423648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4101770 0x7f35e41057c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.010+0000 7f35ea53a700 1 --2- 172.21.15.142:0/2859423648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e4100dc0 0x7f35e41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.010+0000 7f35ea53a700 1 -- 172.21.15.142:0/2859423648 >> 172.21.15.142:0/2859423648 conn(0x7f35e40fc650 msgr2=0x7f35e40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:48.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.010+0000 7f35ea53a700 1 -- 172.21.15.142:0/2859423648 shutdown_connections 2024-03-29T04:56:48.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.010+0000 7f35ea53a700 1 -- 172.21.15.142:0/2859423648 wait complete. 2024-03-29T04:56:48.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.010+0000 7f35ea53a700 1 Processor -- start 2024-03-29T04:56:48.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.010+0000 7f35ea53a700 1 -- start start 2024-03-29T04:56:48.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35ea53a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4100dc0 0x7f35e419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:48.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35ea53a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e4101770 0x7f35e419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:48.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35ea53a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e419d0a0 con 0x7f35e4100dc0 2024-03-29T04:56:48.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35ea53a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e41965e0 con 0x7f35e4101770 2024-03-29T04:56:48.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35e3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4100dc0 0x7f35e419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35e3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4100dc0 0x7f35e419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44532/0 (socket says 172.21.15.142:44532) 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35e3fff700 1 -- 172.21.15.142:0/2707252692 learned_addr learned my addr 172.21.15.142:0/2707252692 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35e37fe700 1 --2- 172.21.15.142:0/2707252692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e4101770 0x7f35e419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35e3fff700 1 -- 172.21.15.142:0/2707252692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e4101770 msgr2=0x7f35e419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35e3fff700 1 --2- 172.21.15.142:0/2707252692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e4101770 0x7f35e419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35e3fff700 1 -- 172.21.15.142:0/2707252692 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d80095f0 con 0x7f35e4100dc0 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35e3fff700 1 --2- 172.21.15.142:0/2707252692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4100dc0 0x7f35e419c560 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c930 tx=0x7f35d400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35e17fa700 1 -- 172.21.15.142:0/2707252692 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35d4007ad0 con 0x7f35e4100dc0 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35e17fa700 1 -- 172.21.15.142:0/2707252692 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35d4007c30 con 0x7f35e4100dc0 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35ea53a700 1 -- 172.21.15.142:0/2707252692 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e41968c0 con 0x7f35e4100dc0 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35e17fa700 1 -- 172.21.15.142:0/2707252692 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35d4018610 con 0x7f35e4100dc0 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.011+0000 7f35ea53a700 1 -- 172.21.15.142:0/2707252692 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e4196e10 con 0x7f35e4100dc0 2024-03-29T04:56:48.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.012+0000 7f35ea53a700 1 -- 172.21.15.142:0/2707252692 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e41091c0 con 0x7f35e4100dc0 2024-03-29T04:56:48.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.013+0000 7f35e17fa700 1 -- 172.21.15.142:0/2707252692 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f35d400f450 con 0x7f35e4100dc0 2024-03-29T04:56:48.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.013+0000 7f35e17fa700 1 --2- 172.21.15.142:0/2707252692 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f35cc06dd60 0x7f35cc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:48.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.014+0000 7f35e37fe700 1 --2- 172.21.15.142:0/2707252692 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f35cc06dd60 0x7f35cc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:48.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.014+0000 7f35e17fa700 1 -- 172.21.15.142:0/2707252692 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f35d408de60 con 0x7f35e4100dc0 2024-03-29T04:56:48.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.014+0000 7f35e37fe700 1 --2- 172.21.15.142:0/2707252692 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f35cc06dd60 0x7f35cc070220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f35d8006010 tx=0x7f35d8005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:48.025 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.018+0000 7f35e17fa700 1 -- 172.21.15.142:0/2707252692 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f35d405adb0 con 0x7f35e4100dc0 2024-03-29T04:56:48.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.223+0000 7f35ea53a700 1 -- 172.21.15.142:0/2707252692 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f35e41970f0 con 0x7f35cc06dd60 2024-03-29T04:56:48.241 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.234+0000 7f35e17fa700 1 -- 172.21.15.142:0/2707252692 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f35e41970f0 con 0x7f35cc06dd60 2024-03-29T04:56:48.241 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:56:48.241 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (8m) 104s ago 20m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:56:48.241 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (20m) 104s ago 20m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:56:48.241 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (19m) 70s ago 19m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:56:48.241 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (5m) 104s ago 20m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:56:48.241 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (5m) 70s ago 19m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (8m) 104s ago 19m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (109s) 104s ago 14m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (2m) 104s ago 14m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (73s) 70s ago 14m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (90s) 70s ago 14m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (10m) 104s ago 21m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (9m) 70s ago 19m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (6m) 104s ago 21m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (6m) 70s ago 19m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (9m) 104s ago 20m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (9m) 70s ago 19m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (5m) 104s ago 18m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:56:48.242 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (4m) 104s ago 17m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:56:48.243 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (4m) 104s ago 17m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:56:48.243 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (3m) 70s ago 17m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T04:56:48.243 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (2m) 70s ago 16m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T04:56:48.243 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (2m) 70s ago 16m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T04:56:48.243 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (8m) 104s ago 19m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:56:48.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.237+0000 7f35ea53a700 1 -- 172.21.15.142:0/2707252692 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f35cc06dd60 msgr2=0x7f35cc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:48.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.237+0000 7f35ea53a700 1 --2- 172.21.15.142:0/2707252692 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f35cc06dd60 0x7f35cc070220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f35d8006010 tx=0x7f35d8005960 comp rx=0 tx=0).stop 2024-03-29T04:56:48.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.237+0000 7f35ea53a700 1 -- 172.21.15.142:0/2707252692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4100dc0 msgr2=0x7f35e419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:48.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.237+0000 7f35ea53a700 1 --2- 172.21.15.142:0/2707252692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4100dc0 0x7f35e419c560 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c930 tx=0x7f35d400ccf0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.237+0000 7f35ea53a700 1 -- 172.21.15.142:0/2707252692 shutdown_connections 2024-03-29T04:56:48.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.237+0000 7f35ea53a700 1 --2- 172.21.15.142:0/2707252692 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f35cc06dd60 0x7f35cc070220 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.237+0000 7f35ea53a700 1 --2- 172.21.15.142:0/2707252692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4100dc0 0x7f35e419c560 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.237+0000 7f35ea53a700 1 --2- 172.21.15.142:0/2707252692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e4101770 0x7f35e419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.237+0000 7f35ea53a700 1 -- 172.21.15.142:0/2707252692 >> 172.21.15.142:0/2707252692 conn(0x7f35e40fc650 msgr2=0x7f35e40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:48.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.237+0000 7f35ea53a700 1 -- 172.21.15.142:0/2707252692 shutdown_connections 2024-03-29T04:56:48.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.237+0000 7f35ea53a700 1 -- 172.21.15.142:0/2707252692 wait complete. 2024-03-29T04:56:48.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:47 smithi152 ceph-mon[88186]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:48.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.340+0000 7ff309678700 1 -- 172.21.15.142:0/3565025858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3041017a0 msgr2=0x7ff3041057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:48.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.340+0000 7ff309678700 1 --2- 172.21.15.142:0/3565025858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3041017a0 0x7ff3041057f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009910 tx=0x7ff2f8009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:48.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.340+0000 7ff309678700 1 -- 172.21.15.142:0/3565025858 shutdown_connections 2024-03-29T04:56:48.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.340+0000 7ff309678700 1 --2- 172.21.15.142:0/3565025858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3041017a0 0x7ff3041057f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.340+0000 7ff309678700 1 --2- 172.21.15.142:0/3565025858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff304100df0 0x7ff3041011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.340+0000 7ff309678700 1 -- 172.21.15.142:0/3565025858 >> 172.21.15.142:0/3565025858 conn(0x7ff3040fc6a0 msgr2=0x7ff3040feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:48.347 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:47 smithi142 ceph-mon[112475]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:48.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.340+0000 7ff309678700 1 -- 172.21.15.142:0/3565025858 shutdown_connections 2024-03-29T04:56:48.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.340+0000 7ff309678700 1 -- 172.21.15.142:0/3565025858 wait complete. 2024-03-29T04:56:48.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.341+0000 7ff309678700 1 Processor -- start 2024-03-29T04:56:48.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.341+0000 7ff309678700 1 -- start start 2024-03-29T04:56:48.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.341+0000 7ff309678700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff304100df0 0x7ff30419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:48.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.341+0000 7ff309678700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3041017a0 0x7ff30419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.341+0000 7ff309678700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30419d130 con 0x7ff3041017a0 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.341+0000 7ff309678700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3041965e0 con 0x7ff304100df0 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.341+0000 7ff3027fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3041017a0 0x7ff30419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.341+0000 7ff3027fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3041017a0 0x7ff30419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44558/0 (socket says 172.21.15.142:44558) 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.341+0000 7ff3027fc700 1 -- 172.21.15.142:0/3867400750 learned_addr learned my addr 172.21.15.142:0/3867400750 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.341+0000 7ff302ffd700 1 --2- 172.21.15.142:0/3867400750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff304100df0 0x7ff30419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.342+0000 7ff3027fc700 1 -- 172.21.15.142:0/3867400750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff304100df0 msgr2=0x7ff30419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.342+0000 7ff3027fc700 1 --2- 172.21.15.142:0/3867400750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff304100df0 0x7ff30419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.342+0000 7ff3027fc700 1 -- 172.21.15.142:0/3867400750 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f80095f0 con 0x7ff3041017a0 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.342+0000 7ff3027fc700 1 --2- 172.21.15.142:0/3867400750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3041017a0 0x7ff30419caa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80059d0 tx=0x7ff2f800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.342+0000 7ff2ebfff700 1 -- 172.21.15.142:0/3867400750 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f801d070 con 0x7ff3041017a0 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.342+0000 7ff2ebfff700 1 -- 172.21.15.142:0/3867400750 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2f800b610 con 0x7ff3041017a0 2024-03-29T04:56:48.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.342+0000 7ff309678700 1 -- 172.21.15.142:0/3867400750 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff304196860 con 0x7ff3041017a0 2024-03-29T04:56:48.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.342+0000 7ff2ebfff700 1 -- 172.21.15.142:0/3867400750 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f8022470 con 0x7ff3041017a0 2024-03-29T04:56:48.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.342+0000 7ff309678700 1 -- 172.21.15.142:0/3867400750 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff304196d50 con 0x7ff3041017a0 2024-03-29T04:56:48.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.343+0000 7ff309678700 1 -- 172.21.15.142:0/3867400750 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff304109100 con 0x7ff3041017a0 2024-03-29T04:56:48.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.344+0000 7ff2ebfff700 1 -- 172.21.15.142:0/3867400750 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff2f8003710 con 0x7ff3041017a0 2024-03-29T04:56:48.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.344+0000 7ff2ebfff700 1 --2- 172.21.15.142:0/3867400750 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2ec06dd60 0x7ff2ec070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:48.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.344+0000 7ff2ebfff700 1 -- 172.21.15.142:0/3867400750 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff2f808f160 con 0x7ff3041017a0 2024-03-29T04:56:48.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.344+0000 7ff302ffd700 1 --2- 172.21.15.142:0/3867400750 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2ec06dd60 0x7ff2ec070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:48.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.344+0000 7ff302ffd700 1 --2- 172.21.15.142:0/3867400750 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2ec06dd60 0x7ff2ec070220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009c60 tx=0x7ff2f4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:48.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.349+0000 7ff2ebfff700 1 -- 172.21.15.142:0/3867400750 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff2f805c0b0 con 0x7ff3041017a0 2024-03-29T04:56:48.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.620+0000 7ff309678700 1 -- 172.21.15.142:0/3867400750 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff30404ea90 con 0x7ff3041017a0 2024-03-29T04:56:48.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.621+0000 7ff2ebfff700 1 -- 172.21.15.142:0/3867400750 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff2f805bc40 con 0x7ff3041017a0 2024-03-29T04:56:48.628 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:56:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:56:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:56:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:56:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:56:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:56:48.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:56:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:56:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T04:56:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:56:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:56:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T04:56:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:56:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:56:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T04:56:48.629 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:56:48.629 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:56:48.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.623+0000 7ff309678700 1 -- 172.21.15.142:0/3867400750 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2ec06dd60 msgr2=0x7ff2ec070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:48.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.623+0000 7ff309678700 1 --2- 172.21.15.142:0/3867400750 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2ec06dd60 0x7ff2ec070220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009c60 tx=0x7ff2f4009260 comp rx=0 tx=0).stop 2024-03-29T04:56:48.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.623+0000 7ff309678700 1 -- 172.21.15.142:0/3867400750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3041017a0 msgr2=0x7ff30419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:48.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.623+0000 7ff309678700 1 --2- 172.21.15.142:0/3867400750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3041017a0 0x7ff30419caa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80059d0 tx=0x7ff2f800f690 comp rx=0 tx=0).stop 2024-03-29T04:56:48.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.623+0000 7ff309678700 1 -- 172.21.15.142:0/3867400750 shutdown_connections 2024-03-29T04:56:48.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.623+0000 7ff309678700 1 --2- 172.21.15.142:0/3867400750 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2ec06dd60 0x7ff2ec070220 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.623+0000 7ff309678700 1 --2- 172.21.15.142:0/3867400750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff304100df0 0x7ff30419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.623+0000 7ff309678700 1 --2- 172.21.15.142:0/3867400750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3041017a0 0x7ff30419caa0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.623+0000 7ff309678700 1 -- 172.21.15.142:0/3867400750 >> 172.21.15.142:0/3867400750 conn(0x7ff3040fc6a0 msgr2=0x7ff304105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:48.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.623+0000 7ff309678700 1 -- 172.21.15.142:0/3867400750 shutdown_connections 2024-03-29T04:56:48.631 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.623+0000 7ff309678700 1 -- 172.21.15.142:0/3867400750 wait complete. 2024-03-29T04:56:48.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.727+0000 7f6b8e8db700 1 -- 172.21.15.142:0/2119238400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b88105a40 msgr2=0x7f6b88105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:48.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.728+0000 7f6b8e8db700 1 --2- 172.21.15.142:0/2119238400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b88105a40 0x7f6b88105e20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009910 tx=0x7f6b78009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:48.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.728+0000 7f6b8e8db700 1 -- 172.21.15.142:0/2119238400 shutdown_connections 2024-03-29T04:56:48.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.728+0000 7f6b8e8db700 1 --2- 172.21.15.142:0/2119238400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b880ffa40 0x7f6b880ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.728+0000 7f6b8e8db700 1 --2- 172.21.15.142:0/2119238400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b88105a40 0x7f6b88105e20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.728+0000 7f6b8e8db700 1 -- 172.21.15.142:0/2119238400 >> 172.21.15.142:0/2119238400 conn(0x7f6b880fb550 msgr2=0x7f6b880fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:48.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.728+0000 7f6b8e8db700 1 -- 172.21.15.142:0/2119238400 shutdown_connections 2024-03-29T04:56:48.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.728+0000 7f6b8e8db700 1 -- 172.21.15.142:0/2119238400 wait complete. 2024-03-29T04:56:48.735 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.728+0000 7f6b8e8db700 1 Processor -- start 2024-03-29T04:56:48.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.728+0000 7f6b8e8db700 1 -- start start 2024-03-29T04:56:48.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b8e8db700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b880ffa40 0x7f6b8819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:48.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b8e8db700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b88105a40 0x7f6b8819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:48.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b8e8db700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8819d230 con 0x7f6b880ffa40 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b8e8db700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b881966b0 con 0x7f6b88105a40 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b87fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b880ffa40 0x7f6b8819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b87fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b880ffa40 0x7f6b8819c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39596/0 (socket says 172.21.15.142:39596) 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b87fff700 1 -- 172.21.15.142:0/1135752704 learned_addr learned my addr 172.21.15.142:0/1135752704 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b877fe700 1 --2- 172.21.15.142:0/1135752704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b88105a40 0x7f6b8819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b87fff700 1 -- 172.21.15.142:0/1135752704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b88105a40 msgr2=0x7f6b8819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b87fff700 1 --2- 172.21.15.142:0/1135752704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b88105a40 0x7f6b8819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b87fff700 1 -- 172.21.15.142:0/1135752704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780095f0 con 0x7f6b880ffa40 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b87fff700 1 --2- 172.21.15.142:0/1135752704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b880ffa40 0x7f6b8819c610 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6b78006010 tx=0x7f6b7800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b857fa700 1 -- 172.21.15.142:0/1135752704 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b7801d070 con 0x7f6b880ffa40 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b857fa700 1 -- 172.21.15.142:0/1135752704 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b7800f460 con 0x7f6b880ffa40 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b8e8db700 1 -- 172.21.15.142:0/1135752704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b88196930 con 0x7f6b880ffa40 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b857fa700 1 -- 172.21.15.142:0/1135752704 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b78017610 con 0x7f6b880ffa40 2024-03-29T04:56:48.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.729+0000 7f6b8e8db700 1 -- 172.21.15.142:0/1135752704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b88196e20 con 0x7f6b880ffa40 2024-03-29T04:56:48.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.730+0000 7f6b8e8db700 1 -- 172.21.15.142:0/1135752704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b88108040 con 0x7f6b880ffa40 2024-03-29T04:56:48.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.731+0000 7f6b857fa700 1 -- 172.21.15.142:0/1135752704 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6b7800f5d0 con 0x7f6b880ffa40 2024-03-29T04:56:48.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.731+0000 7f6b857fa700 1 --2- 172.21.15.142:0/1135752704 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b7006dd60 0x7f6b70070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:48.739 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.731+0000 7f6b857fa700 1 -- 172.21.15.142:0/1135752704 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6b7808f1f0 con 0x7f6b880ffa40 2024-03-29T04:56:48.740 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.731+0000 7f6b877fe700 1 --2- 172.21.15.142:0/1135752704 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b7006dd60 0x7f6b70070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:48.740 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.732+0000 7f6b877fe700 1 --2- 172.21.15.142:0/1135752704 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b7006dd60 0x7f6b70070220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6b88197ea0 tx=0x7f6b7c009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:48.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.736+0000 7f6b857fa700 1 -- 172.21.15.142:0/1135752704 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6b7805c140 con 0x7f6b880ffa40 2024-03-29T04:56:48.983 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.976+0000 7f6b8e8db700 1 -- 172.21.15.142:0/1135752704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6b8802cd10 con 0x7f6b880ffa40 2024-03-29T04:56:48.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.977+0000 7f6b857fa700 1 -- 172.21.15.142:0/1135752704 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f6b7805bcd0 con 0x7f6b880ffa40 2024-03-29T04:56:48.984 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T04:56:48.984 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:56:48.984 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:56:48.984 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:56:48.984 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:56:48.984 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:56:48.984 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:56:48.984 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T04:56:48.984 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:56:48.985 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:56:48.986 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:56:48.986 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T04:56:48.986 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T04:56:48.986 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:56:48.986 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:56:48.986 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:56:48.986 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:56:48.986 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:56:48.986 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:56:48.986 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:56:48.986 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:56:48.986 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:56:48.987 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:56:48.987 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:56:48.987 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T04:56:48.987 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:56:48.987 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:56:48.987 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:56:48.987 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:56:48.987 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:56:48.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.979+0000 7f6b8e8db700 1 -- 172.21.15.142:0/1135752704 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b7006dd60 msgr2=0x7f6b70070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:48.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.979+0000 7f6b8e8db700 1 --2- 172.21.15.142:0/1135752704 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b7006dd60 0x7f6b70070220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6b88197ea0 tx=0x7f6b7c009190 comp rx=0 tx=0).stop 2024-03-29T04:56:48.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.979+0000 7f6b8e8db700 1 -- 172.21.15.142:0/1135752704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b880ffa40 msgr2=0x7f6b8819c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:48.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.979+0000 7f6b8e8db700 1 --2- 172.21.15.142:0/1135752704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b880ffa40 0x7f6b8819c610 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6b78006010 tx=0x7f6b7800bd70 comp rx=0 tx=0).stop 2024-03-29T04:56:48.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.979+0000 7f6b8e8db700 1 -- 172.21.15.142:0/1135752704 shutdown_connections 2024-03-29T04:56:48.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.979+0000 7f6b8e8db700 1 --2- 172.21.15.142:0/1135752704 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6b7006dd60 0x7f6b70070220 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.979+0000 7f6b8e8db700 1 --2- 172.21.15.142:0/1135752704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b880ffa40 0x7f6b8819c610 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.979+0000 7f6b8e8db700 1 --2- 172.21.15.142:0/1135752704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b88105a40 0x7f6b8819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:48.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.979+0000 7f6b8e8db700 1 -- 172.21.15.142:0/1135752704 >> 172.21.15.142:0/1135752704 conn(0x7f6b880fb550 msgr2=0x7f6b880fc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:48.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.979+0000 7f6b8e8db700 1 -- 172.21.15.142:0/1135752704 shutdown_connections 2024-03-29T04:56:48.988 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:48.979+0000 7f6b8e8db700 1 -- 172.21.15.142:0/1135752704 wait complete. 2024-03-29T04:56:48.989 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T04:56:49.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.085+0000 7f85424af700 1 -- 172.21.15.142:0/18225581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f853c105280 msgr2=0x7f853c105660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:49.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.085+0000 7f85424af700 1 --2- 172.21.15.142:0/18225581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f853c105280 0x7f853c105660 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f852c00aa20 tx=0x7f852c00ad30 comp rx=0 tx=0).stop 2024-03-29T04:56:49.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.085+0000 7f85424af700 1 -- 172.21.15.142:0/18225581 shutdown_connections 2024-03-29T04:56:49.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.085+0000 7f85424af700 1 --2- 172.21.15.142:0/18225581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c0ff2d0 0x7f853c0ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.085+0000 7f85424af700 1 --2- 172.21.15.142:0/18225581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f853c105280 0x7f853c105660 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.085+0000 7f85424af700 1 -- 172.21.15.142:0/18225581 >> 172.21.15.142:0/18225581 conn(0x7f853c074d20 msgr2=0x7f853c075130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:49.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.085+0000 7f85424af700 1 -- 172.21.15.142:0/18225581 shutdown_connections 2024-03-29T04:56:49.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.085+0000 7f85424af700 1 -- 172.21.15.142:0/18225581 wait complete. 2024-03-29T04:56:49.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f85424af700 1 Processor -- start 2024-03-29T04:56:49.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f85424af700 1 -- start start 2024-03-29T04:56:49.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f85424af700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f853c0ff2d0 0x7f853c194560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:49.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f85424af700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c105280 0x7f853c194aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f85424af700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c195180 con 0x7f853c0ff2d0 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f85424af700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c198f10 con 0x7f853c105280 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f853bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f853c0ff2d0 0x7f853c194560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f853bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f853c0ff2d0 0x7f853c194560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39602/0 (socket says 172.21.15.142:39602) 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f853bfff700 1 -- 172.21.15.142:0/913157656 learned_addr learned my addr 172.21.15.142:0/913157656 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f853b7fe700 1 --2- 172.21.15.142:0/913157656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c105280 0x7f853c194aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f853bfff700 1 -- 172.21.15.142:0/913157656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c105280 msgr2=0x7f853c194aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f853bfff700 1 --2- 172.21.15.142:0/913157656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c105280 0x7f853c194aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.086+0000 7f853bfff700 1 -- 172.21.15.142:0/913157656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c00a700 con 0x7f853c0ff2d0 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.087+0000 7f853bfff700 1 --2- 172.21.15.142:0/913157656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f853c0ff2d0 0x7f853c194560 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f852c00a3f0 tx=0x7f852c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.087+0000 7f85397fa700 1 -- 172.21.15.142:0/913157656 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f852c004b40 con 0x7f853c0ff2d0 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.087+0000 7f85424af700 1 -- 172.21.15.142:0/913157656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853c199190 con 0x7f853c0ff2d0 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.087+0000 7f85397fa700 1 -- 172.21.15.142:0/913157656 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f852c004ca0 con 0x7f853c0ff2d0 2024-03-29T04:56:49.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.087+0000 7f85397fa700 1 -- 172.21.15.142:0/913157656 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f852c0167a0 con 0x7f853c0ff2d0 2024-03-29T04:56:49.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.087+0000 7f85424af700 1 -- 172.21.15.142:0/913157656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c199680 con 0x7f853c0ff2d0 2024-03-29T04:56:49.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.088+0000 7f85424af700 1 -- 172.21.15.142:0/913157656 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853c107780 con 0x7f853c0ff2d0 2024-03-29T04:56:49.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.088+0000 7f85397fa700 1 -- 172.21.15.142:0/913157656 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f852c00eba0 con 0x7f853c0ff2d0 2024-03-29T04:56:49.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.089+0000 7f85397fa700 1 --2- 172.21.15.142:0/913157656 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f852406dd60 0x7f8524070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:49.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.089+0000 7f85397fa700 1 -- 172.21.15.142:0/913157656 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f852c012070 con 0x7f853c0ff2d0 2024-03-29T04:56:49.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.089+0000 7f853b7fe700 1 --2- 172.21.15.142:0/913157656 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f852406dd60 0x7f8524070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:49.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.089+0000 7f853b7fe700 1 --2- 172.21.15.142:0/913157656 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f852406dd60 0x7f8524070220 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f853c195b80 tx=0x7f853000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:49.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.093+0000 7f85397fa700 1 -- 172.21.15.142:0/913157656 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f852c05a070 con 0x7f853c0ff2d0 2024-03-29T04:56:49.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:48 smithi152 ceph-mon[88186]: pgmap v255: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:49.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:48 smithi152 ceph-mon[88186]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:49.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:48 smithi152 ceph-mon[88186]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:49.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:48 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3867400750' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:56:49.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:48 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1135752704' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:56:49.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.309+0000 7f85424af700 1 -- 172.21.15.142:0/913157656 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f853c199960 con 0x7f852406dd60 2024-03-29T04:56:49.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:48 smithi142 ceph-mon[112475]: pgmap v255: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:49.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:48 smithi142 ceph-mon[112475]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:49.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:48 smithi142 ceph-mon[112475]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:49.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:48 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3867400750' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:56:49.316 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:48 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1135752704' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:56:49.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.310+0000 7f85397fa700 1 -- 172.21.15.142:0/913157656 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f853c199960 con 0x7f852406dd60 2024-03-29T04:56:49.317 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:56:49.318 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:56:49.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.312+0000 7f85424af700 1 -- 172.21.15.142:0/913157656 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f852406dd60 msgr2=0x7f8524070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:49.319 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.312+0000 7f85424af700 1 --2- 172.21.15.142:0/913157656 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f852406dd60 0x7f8524070220 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f853c195b80 tx=0x7f853000b410 comp rx=0 tx=0).stop 2024-03-29T04:56:49.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.312+0000 7f85424af700 1 -- 172.21.15.142:0/913157656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f853c0ff2d0 msgr2=0x7f853c194560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:49.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.312+0000 7f85424af700 1 --2- 172.21.15.142:0/913157656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f853c0ff2d0 0x7f853c194560 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f852c00a3f0 tx=0x7f852c0048a0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.313+0000 7f85424af700 1 -- 172.21.15.142:0/913157656 shutdown_connections 2024-03-29T04:56:49.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.313+0000 7f85424af700 1 --2- 172.21.15.142:0/913157656 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f852406dd60 0x7f8524070220 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.313+0000 7f85424af700 1 --2- 172.21.15.142:0/913157656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f853c0ff2d0 0x7f853c194560 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.313+0000 7f85424af700 1 --2- 172.21.15.142:0/913157656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c105280 0x7f853c194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.313+0000 7f85424af700 1 -- 172.21.15.142:0/913157656 >> 172.21.15.142:0/913157656 conn(0x7f853c074d20 msgr2=0x7f853c0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:49.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.313+0000 7f85424af700 1 -- 172.21.15.142:0/913157656 shutdown_connections 2024-03-29T04:56:49.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.313+0000 7f85424af700 1 -- 172.21.15.142:0/913157656 wait complete. 2024-03-29T04:56:49.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.416+0000 7f3262769700 1 -- 172.21.15.142:0/1694599964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f325c068670 msgr2=0x7f325c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:49.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.416+0000 7f3262769700 1 --2- 172.21.15.142:0/1694599964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f325c068670 0x7f325c068a50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3250009910 tx=0x7f3250009c20 comp rx=0 tx=0).stop 2024-03-29T04:56:49.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.416+0000 7f3262769700 1 -- 172.21.15.142:0/1694599964 shutdown_connections 2024-03-29T04:56:49.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.416+0000 7f3262769700 1 --2- 172.21.15.142:0/1694599964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f325c069020 0x7f325c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.416+0000 7f3262769700 1 --2- 172.21.15.142:0/1694599964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f325c068670 0x7f325c068a50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.416+0000 7f3262769700 1 -- 172.21.15.142:0/1694599964 >> 172.21.15.142:0/1694599964 conn(0x7f325c0755a0 msgr2=0x7f325c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:49.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.416+0000 7f3262769700 1 -- 172.21.15.142:0/1694599964 shutdown_connections 2024-03-29T04:56:49.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.416+0000 7f3262769700 1 -- 172.21.15.142:0/1694599964 wait complete. 2024-03-29T04:56:49.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.417+0000 7f3262769700 1 Processor -- start 2024-03-29T04:56:49.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.417+0000 7f3262769700 1 -- start start 2024-03-29T04:56:49.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.417+0000 7f3262769700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f325c068670 0x7f325c19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:49.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.417+0000 7f3262769700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f325c069020 0x7f325c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:49.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.417+0000 7f3262769700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c19d0e0 con 0x7f325c068670 2024-03-29T04:56:49.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.417+0000 7f3262769700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c196590 con 0x7f325c069020 2024-03-29T04:56:49.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.417+0000 7f325bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f325c068670 0x7f325c19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:49.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.417+0000 7f325bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f325c068670 0x7f325c19c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39618/0 (socket says 172.21.15.142:39618) 2024-03-29T04:56:49.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.417+0000 7f325bfff700 1 -- 172.21.15.142:0/2601541580 learned_addr learned my addr 172.21.15.142:0/2601541580 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:56:49.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.418+0000 7f325b7fe700 1 --2- 172.21.15.142:0/2601541580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f325c069020 0x7f325c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:49.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.418+0000 7f325bfff700 1 -- 172.21.15.142:0/2601541580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f325c069020 msgr2=0x7f325c19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:49.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.418+0000 7f325bfff700 1 --2- 172.21.15.142:0/2601541580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f325c069020 0x7f325c19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.418+0000 7f325bfff700 1 -- 172.21.15.142:0/2601541580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32500095f0 con 0x7f325c068670 2024-03-29T04:56:49.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.418+0000 7f325b7fe700 1 --2- 172.21.15.142:0/2601541580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f325c069020 0x7f325c19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:56:49.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.418+0000 7f325bfff700 1 --2- 172.21.15.142:0/2601541580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f325c068670 0x7f325c19c510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3250005f50 tx=0x7f325000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:49.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.418+0000 7f32597fa700 1 -- 172.21.15.142:0/2601541580 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f325001d070 con 0x7f325c068670 2024-03-29T04:56:49.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.418+0000 7f32597fa700 1 -- 172.21.15.142:0/2601541580 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f325000b610 con 0x7f325c068670 2024-03-29T04:56:49.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.418+0000 7f3262769700 1 -- 172.21.15.142:0/2601541580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325c196810 con 0x7f325c068670 2024-03-29T04:56:49.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.418+0000 7f32597fa700 1 -- 172.21.15.142:0/2601541580 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3250022470 con 0x7f325c068670 2024-03-29T04:56:49.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.418+0000 7f3262769700 1 -- 172.21.15.142:0/2601541580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325c196d00 con 0x7f325c068670 2024-03-29T04:56:49.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.419+0000 7f3262769700 1 -- 172.21.15.142:0/2601541580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325c109130 con 0x7f325c068670 2024-03-29T04:56:49.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.420+0000 7f32597fa700 1 -- 172.21.15.142:0/2601541580 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f325000fca0 con 0x7f325c068670 2024-03-29T04:56:49.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.420+0000 7f32597fa700 1 --2- 172.21.15.142:0/2601541580 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f324406dd60 0x7f3244070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:56:49.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.420+0000 7f32597fa700 1 -- 172.21.15.142:0/2601541580 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f325008f040 con 0x7f325c068670 2024-03-29T04:56:49.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.420+0000 7f325b7fe700 1 --2- 172.21.15.142:0/2601541580 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f324406dd60 0x7f3244070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:56:49.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.420+0000 7f325b7fe700 1 --2- 172.21.15.142:0/2601541580 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f324406dd60 0x7f3244070220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f325c107750 tx=0x7f324c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:56:49.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.425+0000 7f32597fa700 1 -- 172.21.15.142:0/2601541580 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f325005c010 con 0x7f325c068670 2024-03-29T04:56:49.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.702+0000 7f3262769700 1 -- 172.21.15.142:0/2601541580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f325c197030 con 0x7f325c068670 2024-03-29T04:56:49.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.702+0000 7f32597fa700 1 -- 172.21.15.142:0/2601541580 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f325005bba0 con 0x7f325c068670 2024-03-29T04:56:49.709 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T04:56:49.709 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T04:56:49.709 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T04:56:49.709 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T04:56:49.709 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T04:56:49.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.704+0000 7f3262769700 1 -- 172.21.15.142:0/2601541580 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f324406dd60 msgr2=0x7f3244070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:49.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.704+0000 7f3262769700 1 --2- 172.21.15.142:0/2601541580 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f324406dd60 0x7f3244070220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f325c107750 tx=0x7f324c009260 comp rx=0 tx=0).stop 2024-03-29T04:56:49.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.704+0000 7f3262769700 1 -- 172.21.15.142:0/2601541580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f325c068670 msgr2=0x7f325c19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:56:49.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.704+0000 7f3262769700 1 --2- 172.21.15.142:0/2601541580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f325c068670 0x7f325c19c510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3250005f50 tx=0x7f325000f740 comp rx=0 tx=0).stop 2024-03-29T04:56:49.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.704+0000 7f3262769700 1 -- 172.21.15.142:0/2601541580 shutdown_connections 2024-03-29T04:56:49.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.704+0000 7f3262769700 1 --2- 172.21.15.142:0/2601541580 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f324406dd60 0x7f3244070220 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.704+0000 7f3262769700 1 --2- 172.21.15.142:0/2601541580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f325c068670 0x7f325c19c510 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.704+0000 7f3262769700 1 --2- 172.21.15.142:0/2601541580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f325c069020 0x7f325c19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:56:49.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.704+0000 7f3262769700 1 -- 172.21.15.142:0/2601541580 >> 172.21.15.142:0/2601541580 conn(0x7f325c0755a0 msgr2=0x7f325c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:56:49.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.704+0000 7f3262769700 1 -- 172.21.15.142:0/2601541580 shutdown_connections 2024-03-29T04:56:49.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:56:49.704+0000 7f3262769700 1 -- 172.21.15.142:0/2601541580 wait complete. 2024-03-29T04:56:50.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:50 smithi152 ceph-mon[88186]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:50.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:50 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2601541580' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:56:50.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:49 smithi142 ceph-mon[112475]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:56:50.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:49 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2601541580' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:56:51.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:51 smithi152 ceph-mon[88186]: pgmap v256: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:51.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:51 smithi142 ceph-mon[112475]: pgmap v256: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:53.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:53 smithi152 ceph-mon[88186]: pgmap v257: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:53.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:53 smithi142 ceph-mon[112475]: pgmap v257: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:55.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:55 smithi152 ceph-mon[88186]: pgmap v258: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:55.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:55 smithi142 ceph-mon[112475]: pgmap v258: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:57.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:57 smithi152 ceph-mon[88186]: pgmap v259: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:57.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:57 smithi142 ceph-mon[112475]: pgmap v259: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:59.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:56:59 smithi152 ceph-mon[88186]: pgmap v260: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:56:59.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:56:59 smithi142 ceph-mon[112475]: pgmap v260: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:00.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:57:00.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:57:01.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:01 smithi152 ceph-mon[88186]: pgmap v261: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:01.343 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:01 smithi142 ceph-mon[112475]: pgmap v261: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:03.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:03 smithi152 ceph-mon[88186]: pgmap v262: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:03.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:03 smithi142 ceph-mon[112475]: pgmap v262: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:05.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:05 smithi152 ceph-mon[88186]: pgmap v263: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:05.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:05 smithi142 ceph-mon[112475]: pgmap v263: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:07.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:07 smithi152 ceph-mon[88186]: pgmap v264: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:07.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:07 smithi142 ceph-mon[112475]: pgmap v264: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:09.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:09 smithi152 ceph-mon[88186]: pgmap v265: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:09.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:09 smithi142 ceph-mon[112475]: pgmap v265: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:11.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:11 smithi152 ceph-mon[88186]: pgmap v266: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:11.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:11 smithi142 ceph-mon[112475]: pgmap v266: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:13.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:13 smithi152 ceph-mon[88186]: pgmap v267: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:13.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:13 smithi142 ceph-mon[112475]: pgmap v267: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:15.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:15 smithi152 ceph-mon[88186]: pgmap v268: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:15.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:57:15.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:15 smithi142 ceph-mon[112475]: pgmap v268: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:15.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:57:17.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:17 smithi152 ceph-mon[88186]: pgmap v269: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:17.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:17 smithi142 ceph-mon[112475]: pgmap v269: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:19.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:19 smithi152 ceph-mon[88186]: pgmap v270: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:19.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:19 smithi142 ceph-mon[112475]: pgmap v270: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:19.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.822+0000 7f25546b8700 1 -- 172.21.15.142:0/2213001406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f254c0ffa60 msgr2=0x7f254c0ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:19.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.822+0000 7f25546b8700 1 --2- 172.21.15.142:0/2213001406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f254c0ffa60 0x7f254c0ffee0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2548009910 tx=0x7f2548009c20 comp rx=0 tx=0).stop 2024-03-29T04:57:19.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.822+0000 7f25546b8700 1 -- 172.21.15.142:0/2213001406 shutdown_connections 2024-03-29T04:57:19.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.822+0000 7f25546b8700 1 --2- 172.21.15.142:0/2213001406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f254c0ffa60 0x7f254c0ffee0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:19.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.822+0000 7f25546b8700 1 --2- 172.21.15.142:0/2213001406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f254c105a60 0x7f254c105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:19.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.822+0000 7f25546b8700 1 -- 172.21.15.142:0/2213001406 >> 172.21.15.142:0/2213001406 conn(0x7f254c0fb550 msgr2=0x7f254c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:19.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.822+0000 7f25546b8700 1 -- 172.21.15.142:0/2213001406 shutdown_connections 2024-03-29T04:57:19.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.822+0000 7f25546b8700 1 -- 172.21.15.142:0/2213001406 wait complete. 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.822+0000 7f25546b8700 1 Processor -- start 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.822+0000 7f25546b8700 1 -- start start 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f25546b8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f254c0ffa60 0x7f254c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f25546b8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f254c105a60 0x7f254c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f25546b8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c19d0f0 con 0x7f254c0ffa60 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f25546b8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c1965a0 con 0x7f254c105a60 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f2552454700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f254c0ffa60 0x7f254c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f2552454700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f254c0ffa60 0x7f254c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40238/0 (socket says 172.21.15.142:40238) 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f2552454700 1 -- 172.21.15.142:0/1761402668 learned_addr learned my addr 172.21.15.142:0/1761402668 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f2551c53700 1 --2- 172.21.15.142:0/1761402668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f254c105a60 0x7f254c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f2552454700 1 -- 172.21.15.142:0/1761402668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f254c105a60 msgr2=0x7f254c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f2552454700 1 --2- 172.21.15.142:0/1761402668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f254c105a60 0x7f254c19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f2552454700 1 -- 172.21.15.142:0/1761402668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25480095f0 con 0x7f254c0ffa60 2024-03-29T04:57:19.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f2552454700 1 --2- 172.21.15.142:0/1761402668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f254c0ffa60 0x7f254c19c520 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f253c00d900 tx=0x7f253c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:19.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f25437fe700 1 -- 172.21.15.142:0/1761402668 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f253c004a00 con 0x7f254c0ffa60 2024-03-29T04:57:19.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f25437fe700 1 -- 172.21.15.142:0/1761402668 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f253c0054e0 con 0x7f254c0ffa60 2024-03-29T04:57:19.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f25546b8700 1 -- 172.21.15.142:0/1761402668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c196880 con 0x7f254c0ffa60 2024-03-29T04:57:19.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f25437fe700 1 -- 172.21.15.142:0/1761402668 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f253c009d00 con 0x7f254c0ffa60 2024-03-29T04:57:19.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.823+0000 7f25546b8700 1 -- 172.21.15.142:0/1761402668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c196dd0 con 0x7f254c0ffa60 2024-03-29T04:57:19.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.824+0000 7f25546b8700 1 -- 172.21.15.142:0/1761402668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c109000 con 0x7f254c0ffa60 2024-03-29T04:57:19.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.825+0000 7f25437fe700 1 -- 172.21.15.142:0/1761402668 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f253c005020 con 0x7f254c0ffa60 2024-03-29T04:57:19.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.825+0000 7f25437fe700 1 --2- 172.21.15.142:0/1761402668 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f253806dd60 0x7f2538070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:19.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.826+0000 7f25437fe700 1 -- 172.21.15.142:0/1761402668 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f253c08dc50 con 0x7f254c0ffa60 2024-03-29T04:57:19.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.826+0000 7f2551c53700 1 --2- 172.21.15.142:0/1761402668 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f253806dd60 0x7f2538070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:19.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.826+0000 7f2551c53700 1 --2- 172.21.15.142:0/1761402668 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f253806dd60 0x7f2538070220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2548005090 tx=0x7f254801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:19.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:19.830+0000 7f25437fe700 1 -- 172.21.15.142:0/1761402668 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f253c05aba0 con 0x7f254c0ffa60 2024-03-29T04:57:20.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.043+0000 7f25546b8700 1 -- 172.21.15.142:0/1761402668 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f254c196ff0 con 0x7f253806dd60 2024-03-29T04:57:20.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.044+0000 7f25437fe700 1 -- 172.21.15.142:0/1761402668 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f254c196ff0 con 0x7f253806dd60 2024-03-29T04:57:20.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.047+0000 7f25546b8700 1 -- 172.21.15.142:0/1761402668 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f253806dd60 msgr2=0x7f2538070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:20.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.047+0000 7f25546b8700 1 --2- 172.21.15.142:0/1761402668 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f253806dd60 0x7f2538070220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2548005090 tx=0x7f254801a040 comp rx=0 tx=0).stop 2024-03-29T04:57:20.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.047+0000 7f25546b8700 1 -- 172.21.15.142:0/1761402668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f254c0ffa60 msgr2=0x7f254c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:20.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.047+0000 7f25546b8700 1 --2- 172.21.15.142:0/1761402668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f254c0ffa60 0x7f254c19c520 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f253c00d900 tx=0x7f253c00dc10 comp rx=0 tx=0).stop 2024-03-29T04:57:20.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.047+0000 7f25546b8700 1 -- 172.21.15.142:0/1761402668 shutdown_connections 2024-03-29T04:57:20.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.047+0000 7f25546b8700 1 --2- 172.21.15.142:0/1761402668 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f253806dd60 0x7f2538070220 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.047+0000 7f25546b8700 1 --2- 172.21.15.142:0/1761402668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f254c0ffa60 0x7f254c19c520 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.047+0000 7f25546b8700 1 --2- 172.21.15.142:0/1761402668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f254c105a60 0x7f254c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.047+0000 7f25546b8700 1 -- 172.21.15.142:0/1761402668 >> 172.21.15.142:0/1761402668 conn(0x7f254c0fb550 msgr2=0x7f254c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:20.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.047+0000 7f25546b8700 1 -- 172.21.15.142:0/1761402668 shutdown_connections 2024-03-29T04:57:20.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.047+0000 7f25546b8700 1 -- 172.21.15.142:0/1761402668 wait complete. 2024-03-29T04:57:20.065 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:57:20.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.153+0000 7f317bf0f700 1 -- 172.21.15.142:0/3466304964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174101770 msgr2=0x7f31741057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:20.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.153+0000 7f317bf0f700 1 --2- 172.21.15.142:0/3466304964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174101770 0x7f31741057c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3170009910 tx=0x7f3170009c20 comp rx=0 tx=0).stop 2024-03-29T04:57:20.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.153+0000 7f317bf0f700 1 -- 172.21.15.142:0/3466304964 shutdown_connections 2024-03-29T04:57:20.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.153+0000 7f317bf0f700 1 --2- 172.21.15.142:0/3466304964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174101770 0x7f31741057c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.153+0000 7f317bf0f700 1 --2- 172.21.15.142:0/3466304964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3174100dc0 0x7f31741011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.153+0000 7f317bf0f700 1 -- 172.21.15.142:0/3466304964 >> 172.21.15.142:0/3466304964 conn(0x7f31740fc650 msgr2=0x7f31740fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:20.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.154+0000 7f317bf0f700 1 -- 172.21.15.142:0/3466304964 shutdown_connections 2024-03-29T04:57:20.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.154+0000 7f317bf0f700 1 -- 172.21.15.142:0/3466304964 wait complete. 2024-03-29T04:57:20.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.154+0000 7f317bf0f700 1 Processor -- start 2024-03-29T04:57:20.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.154+0000 7f317bf0f700 1 -- start start 2024-03-29T04:57:20.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.154+0000 7f317bf0f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3174100dc0 0x7f317419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:20.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.154+0000 7f317bf0f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174101770 0x7f317419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:20.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.154+0000 7f317bf0f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317419d130 con 0x7f3174101770 2024-03-29T04:57:20.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.154+0000 7f317bf0f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31741965e0 con 0x7f3174100dc0 2024-03-29T04:57:20.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f31794aa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174101770 0x7f317419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:20.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f31794aa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174101770 0x7f317419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40250/0 (socket says 172.21.15.142:40250) 2024-03-29T04:57:20.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f31794aa700 1 -- 172.21.15.142:0/3665454215 learned_addr learned my addr 172.21.15.142:0/3665454215 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:20.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f3179cab700 1 --2- 172.21.15.142:0/3665454215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3174100dc0 0x7f317419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:20.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f31794aa700 1 -- 172.21.15.142:0/3665454215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3174100dc0 msgr2=0x7f317419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:20.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f31794aa700 1 --2- 172.21.15.142:0/3665454215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3174100dc0 0x7f317419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f31794aa700 1 -- 172.21.15.142:0/3665454215 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31700095f0 con 0x7f3174101770 2024-03-29T04:57:20.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f31794aa700 1 --2- 172.21.15.142:0/3665454215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174101770 0x7f317419caa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f31700092e0 tx=0x7f317000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:20.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f316affd700 1 -- 172.21.15.142:0/3665454215 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f317001d070 con 0x7f3174101770 2024-03-29T04:57:20.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f316affd700 1 -- 172.21.15.142:0/3665454215 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3170005dc0 con 0x7f3174101770 2024-03-29T04:57:20.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f316affd700 1 -- 172.21.15.142:0/3665454215 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3170017400 con 0x7f3174101770 2024-03-29T04:57:20.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f317bf0f700 1 -- 172.21.15.142:0/3665454215 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3174196860 con 0x7f3174101770 2024-03-29T04:57:20.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.155+0000 7f317bf0f700 1 -- 172.21.15.142:0/3665454215 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3174196d50 con 0x7f3174101770 2024-03-29T04:57:20.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.156+0000 7f317bf0f700 1 -- 172.21.15.142:0/3665454215 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31741091c0 con 0x7f3174101770 2024-03-29T04:57:20.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.157+0000 7f316affd700 1 -- 172.21.15.142:0/3665454215 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f317000f6f0 con 0x7f3174101770 2024-03-29T04:57:20.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.157+0000 7f316affd700 1 --2- 172.21.15.142:0/3665454215 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f316006dd60 0x7f3160070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:20.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.157+0000 7f3179cab700 1 --2- 172.21.15.142:0/3665454215 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f316006dd60 0x7f3160070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:20.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.157+0000 7f316affd700 1 -- 172.21.15.142:0/3665454215 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f317008efa0 con 0x7f3174101770 2024-03-29T04:57:20.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.158+0000 7f3179cab700 1 --2- 172.21.15.142:0/3665454215 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f316006dd60 0x7f3160070220 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3164005950 tx=0x7f316400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:20.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.162+0000 7f316affd700 1 -- 172.21.15.142:0/3665454215 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f317005bf70 con 0x7f3174101770 2024-03-29T04:57:20.380 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.373+0000 7f317bf0f700 1 -- 172.21.15.142:0/3665454215 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3174197080 con 0x7f316006dd60 2024-03-29T04:57:20.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.374+0000 7f316affd700 1 -- 172.21.15.142:0/3665454215 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3174197080 con 0x7f316006dd60 2024-03-29T04:57:20.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.376+0000 7f317bf0f700 1 -- 172.21.15.142:0/3665454215 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f316006dd60 msgr2=0x7f3160070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:20.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.376+0000 7f317bf0f700 1 --2- 172.21.15.142:0/3665454215 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f316006dd60 0x7f3160070220 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3164005950 tx=0x7f316400b410 comp rx=0 tx=0).stop 2024-03-29T04:57:20.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.377+0000 7f317bf0f700 1 -- 172.21.15.142:0/3665454215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174101770 msgr2=0x7f317419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:20.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.377+0000 7f317bf0f700 1 --2- 172.21.15.142:0/3665454215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174101770 0x7f317419caa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f31700092e0 tx=0x7f317000bd70 comp rx=0 tx=0).stop 2024-03-29T04:57:20.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.377+0000 7f317bf0f700 1 -- 172.21.15.142:0/3665454215 shutdown_connections 2024-03-29T04:57:20.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.377+0000 7f317bf0f700 1 --2- 172.21.15.142:0/3665454215 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f316006dd60 0x7f3160070220 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.377+0000 7f317bf0f700 1 --2- 172.21.15.142:0/3665454215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3174100dc0 0x7f317419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.377+0000 7f317bf0f700 1 --2- 172.21.15.142:0/3665454215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3174101770 0x7f317419caa0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.377+0000 7f317bf0f700 1 -- 172.21.15.142:0/3665454215 >> 172.21.15.142:0/3665454215 conn(0x7f31740fc650 msgr2=0x7f31740fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:20.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.377+0000 7f317bf0f700 1 -- 172.21.15.142:0/3665454215 shutdown_connections 2024-03-29T04:57:20.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.377+0000 7f317bf0f700 1 -- 172.21.15.142:0/3665454215 wait complete. 2024-03-29T04:57:20.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.480+0000 7fada5bb3700 1 -- 172.21.15.142:0/692478517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00ffbd0 msgr2=0x7fada0100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:20.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.480+0000 7fada5bb3700 1 --2- 172.21.15.142:0/692478517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00ffbd0 0x7fada0100050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fad94009910 tx=0x7fad94009c20 comp rx=0 tx=0).stop 2024-03-29T04:57:20.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.480+0000 7fada5bb3700 1 -- 172.21.15.142:0/692478517 shutdown_connections 2024-03-29T04:57:20.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.480+0000 7fada5bb3700 1 --2- 172.21.15.142:0/692478517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00ffbd0 0x7fada0100050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.480+0000 7fada5bb3700 1 --2- 172.21.15.142:0/692478517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fada0105a40 0x7fada0105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.480+0000 7fada5bb3700 1 -- 172.21.15.142:0/692478517 >> 172.21.15.142:0/692478517 conn(0x7fada00755a0 msgr2=0x7fada00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:20.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.481+0000 7fada5bb3700 1 -- 172.21.15.142:0/692478517 shutdown_connections 2024-03-29T04:57:20.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.481+0000 7fada5bb3700 1 -- 172.21.15.142:0/692478517 wait complete. 2024-03-29T04:57:20.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.481+0000 7fada5bb3700 1 Processor -- start 2024-03-29T04:57:20.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.481+0000 7fada5bb3700 1 -- start start 2024-03-29T04:57:20.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fada5bb3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00ffbd0 0x7fada0195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:20.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fada5bb3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fada0105a40 0x7fada0195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:20.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fada5bb3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada0196240 con 0x7fada00ffbd0 2024-03-29T04:57:20.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fada5bb3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada0199fd0 con 0x7fada0105a40 2024-03-29T04:57:20.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fad9f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00ffbd0 0x7fada0195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:20.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fad9f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00ffbd0 0x7fada0195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40258/0 (socket says 172.21.15.142:40258) 2024-03-29T04:57:20.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fad9f7fe700 1 -- 172.21.15.142:0/1729861813 learned_addr learned my addr 172.21.15.142:0/1729861813 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:20.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fad9effd700 1 --2- 172.21.15.142:0/1729861813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fada0105a40 0x7fada0195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:20.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fad9f7fe700 1 -- 172.21.15.142:0/1729861813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fada0105a40 msgr2=0x7fada0195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:20.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fad9f7fe700 1 --2- 172.21.15.142:0/1729861813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fada0105a40 0x7fada0195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fad9f7fe700 1 -- 172.21.15.142:0/1729861813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad940095f0 con 0x7fada00ffbd0 2024-03-29T04:57:20.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fad9f7fe700 1 --2- 172.21.15.142:0/1729861813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00ffbd0 0x7fada0195620 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fad9000d8d0 tx=0x7fad9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:20.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.482+0000 7fad9cff9700 1 -- 172.21.15.142:0/1729861813 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad90009940 con 0x7fada00ffbd0 2024-03-29T04:57:20.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.483+0000 7fad9cff9700 1 -- 172.21.15.142:0/1729861813 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad90010460 con 0x7fada00ffbd0 2024-03-29T04:57:20.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.483+0000 7fada5bb3700 1 -- 172.21.15.142:0/1729861813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada019a2b0 con 0x7fada00ffbd0 2024-03-29T04:57:20.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.483+0000 7fad9cff9700 1 -- 172.21.15.142:0/1729861813 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad9000f5f0 con 0x7fada00ffbd0 2024-03-29T04:57:20.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.483+0000 7fada5bb3700 1 -- 172.21.15.142:0/1729861813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada019a800 con 0x7fada00ffbd0 2024-03-29T04:57:20.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.484+0000 7fada5bb3700 1 -- 172.21.15.142:0/1729861813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada0107fd0 con 0x7fada00ffbd0 2024-03-29T04:57:20.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.484+0000 7fad9cff9700 1 -- 172.21.15.142:0/1729861813 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fad900105d0 con 0x7fada00ffbd0 2024-03-29T04:57:20.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.484+0000 7fad9cff9700 1 --2- 172.21.15.142:0/1729861813 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad8806dd60 0x7fad88070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:20.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.485+0000 7fad9cff9700 1 -- 172.21.15.142:0/1729861813 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fad9008dcc0 con 0x7fada00ffbd0 2024-03-29T04:57:20.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.485+0000 7fad9effd700 1 --2- 172.21.15.142:0/1729861813 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad8806dd60 0x7fad88070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:20.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.485+0000 7fad9effd700 1 --2- 172.21.15.142:0/1729861813 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad8806dd60 0x7fad88070220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad94005b40 tx=0x7fad9401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:20.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.490+0000 7fad9cff9700 1 -- 172.21.15.142:0/1729861813 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fad900570e0 con 0x7fada00ffbd0 2024-03-29T04:57:20.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.693+0000 7fada5bb3700 1 -- 172.21.15.142:0/1729861813 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fada019ab30 con 0x7fad8806dd60 2024-03-29T04:57:20.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.704+0000 7fad9cff9700 1 -- 172.21.15.142:0/1729861813 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fada019ab30 con 0x7fad8806dd60 2024-03-29T04:57:20.711 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:57:20.711 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (9m) 2m ago 20m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (20m) 2m ago 20m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (19m) 102s ago 19m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (6m) 2m ago 20m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (6m) 102s ago 19m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (8m) 2m ago 20m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (2m) 2m ago 15m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (2m) 2m ago 15m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (106s) 102s ago 14m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (2m) 102s ago 15m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (10m) 2m ago 22m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (10m) 102s ago 19m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (7m) 2m ago 22m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (7m) 102s ago 19m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:57:20.712 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (9m) 2m ago 20m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:57:20.713 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (9m) 102s ago 19m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:57:20.713 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (5m) 2m ago 18m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:57:20.713 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (5m) 2m ago 18m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:57:20.713 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (4m) 2m ago 17m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:57:20.713 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (4m) 102s ago 17m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T04:57:20.713 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (3m) 102s ago 17m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T04:57:20.713 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (2m) 102s ago 16m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T04:57:20.713 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (9m) 2m ago 20m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:57:20.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.706+0000 7fada5bb3700 1 -- 172.21.15.142:0/1729861813 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad8806dd60 msgr2=0x7fad88070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:20.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.706+0000 7fada5bb3700 1 --2- 172.21.15.142:0/1729861813 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad8806dd60 0x7fad88070220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad94005b40 tx=0x7fad9401a040 comp rx=0 tx=0).stop 2024-03-29T04:57:20.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.706+0000 7fada5bb3700 1 -- 172.21.15.142:0/1729861813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00ffbd0 msgr2=0x7fada0195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:20.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.706+0000 7fada5bb3700 1 --2- 172.21.15.142:0/1729861813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00ffbd0 0x7fada0195620 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fad9000d8d0 tx=0x7fad9000dc90 comp rx=0 tx=0).stop 2024-03-29T04:57:20.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.706+0000 7fada5bb3700 1 -- 172.21.15.142:0/1729861813 shutdown_connections 2024-03-29T04:57:20.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.706+0000 7fada5bb3700 1 --2- 172.21.15.142:0/1729861813 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad8806dd60 0x7fad88070220 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.706+0000 7fada5bb3700 1 --2- 172.21.15.142:0/1729861813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fada00ffbd0 0x7fada0195620 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.706+0000 7fada5bb3700 1 --2- 172.21.15.142:0/1729861813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fada0105a40 0x7fada0195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.706+0000 7fada5bb3700 1 -- 172.21.15.142:0/1729861813 >> 172.21.15.142:0/1729861813 conn(0x7fada00755a0 msgr2=0x7fada00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:20.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.707+0000 7fada5bb3700 1 -- 172.21.15.142:0/1729861813 shutdown_connections 2024-03-29T04:57:20.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.707+0000 7fada5bb3700 1 -- 172.21.15.142:0/1729861813 wait complete. 2024-03-29T04:57:20.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.812+0000 7fb997e1f700 1 -- 172.21.15.142:0/3828227490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990105a40 msgr2=0x7fb990105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:20.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.812+0000 7fb997e1f700 1 --2- 172.21.15.142:0/3828227490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990105a40 0x7fb990105e20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009910 tx=0x7fb98c009c20 comp rx=0 tx=0).stop 2024-03-29T04:57:20.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.812+0000 7fb997e1f700 1 -- 172.21.15.142:0/3828227490 shutdown_connections 2024-03-29T04:57:20.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.812+0000 7fb997e1f700 1 --2- 172.21.15.142:0/3828227490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9900ffbd0 0x7fb990100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.812+0000 7fb997e1f700 1 --2- 172.21.15.142:0/3828227490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990105a40 0x7fb990105e20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.812+0000 7fb997e1f700 1 -- 172.21.15.142:0/3828227490 >> 172.21.15.142:0/3828227490 conn(0x7fb9900755a0 msgr2=0x7fb9900759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:20.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.812+0000 7fb997e1f700 1 -- 172.21.15.142:0/3828227490 shutdown_connections 2024-03-29T04:57:20.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.812+0000 7fb997e1f700 1 -- 172.21.15.142:0/3828227490 wait complete. 2024-03-29T04:57:20.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb997e1f700 1 Processor -- start 2024-03-29T04:57:20.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb997e1f700 1 -- start start 2024-03-29T04:57:20.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb997e1f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9900ffbd0 0x7fb990101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:20.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb997e1f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb990105a40 0x7fb990101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:20.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb997e1f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb990102120 con 0x7fb9900ffbd0 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb997e1f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb990102290 con 0x7fb990105a40 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb995bbb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9900ffbd0 0x7fb990101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb995bbb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9900ffbd0 0x7fb990101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40288/0 (socket says 172.21.15.142:40288) 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb995bbb700 1 -- 172.21.15.142:0/1563574450 learned_addr learned my addr 172.21.15.142:0/1563574450 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb9953ba700 1 --2- 172.21.15.142:0/1563574450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb990105a40 0x7fb990101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb995bbb700 1 -- 172.21.15.142:0/1563574450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb990105a40 msgr2=0x7fb990101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb995bbb700 1 --2- 172.21.15.142:0/1563574450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb990105a40 0x7fb990101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb995bbb700 1 -- 172.21.15.142:0/1563574450 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c0095f0 con 0x7fb9900ffbd0 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.813+0000 7fb9953ba700 1 --2- 172.21.15.142:0/1563574450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb990105a40 0x7fb990101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.814+0000 7fb995bbb700 1 --2- 172.21.15.142:0/1563574450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9900ffbd0 0x7fb990101610 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb98c006010 tx=0x7fb98c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.814+0000 7fb986ffd700 1 -- 172.21.15.142:0/1563574450 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb98c01d070 con 0x7fb9900ffbd0 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.814+0000 7fb986ffd700 1 -- 172.21.15.142:0/1563574450 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb98c004fb0 con 0x7fb9900ffbd0 2024-03-29T04:57:20.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.814+0000 7fb986ffd700 1 -- 172.21.15.142:0/1563574450 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb98c022ea0 con 0x7fb9900ffbd0 2024-03-29T04:57:20.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.814+0000 7fb997e1f700 1 -- 172.21.15.142:0/1563574450 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9901a28c0 con 0x7fb9900ffbd0 2024-03-29T04:57:20.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.814+0000 7fb997e1f700 1 -- 172.21.15.142:0/1563574450 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9901a2c70 con 0x7fb9900ffbd0 2024-03-29T04:57:20.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.815+0000 7fb997e1f700 1 -- 172.21.15.142:0/1563574450 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb990109050 con 0x7fb9900ffbd0 2024-03-29T04:57:20.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.815+0000 7fb986ffd700 1 -- 172.21.15.142:0/1563574450 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fb98c022470 con 0x7fb9900ffbd0 2024-03-29T04:57:20.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.816+0000 7fb986ffd700 1 --2- 172.21.15.142:0/1563574450 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb97c06dd60 0x7fb97c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:20.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.816+0000 7fb986ffd700 1 -- 172.21.15.142:0/1563574450 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb98c08f180 con 0x7fb9900ffbd0 2024-03-29T04:57:20.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.816+0000 7fb9953ba700 1 --2- 172.21.15.142:0/1563574450 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb97c06dd60 0x7fb97c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:20.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.816+0000 7fb9953ba700 1 --2- 172.21.15.142:0/1563574450 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb97c06dd60 0x7fb97c070220 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb990102e00 tx=0x7fb980005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:20.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:20.820+0000 7fb986ffd700 1 -- 172.21.15.142:0/1563574450 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb98c0586d0 con 0x7fb9900ffbd0 2024-03-29T04:57:21.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.092+0000 7fb997e1f700 1 -- 172.21.15.142:0/1563574450 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb990108a30 con 0x7fb9900ffbd0 2024-03-29T04:57:21.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.093+0000 7fb986ffd700 1 -- 172.21.15.142:0/1563574450 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb98c022720 con 0x7fb9900ffbd0 2024-03-29T04:57:21.101 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:57:21.101 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:57:21.101 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:57:21.101 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:57:21.101 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:57:21.101 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:57:21.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:57:21.102 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:57:21.102 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T04:57:21.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:57:21.102 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:57:21.102 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T04:57:21.102 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:57:21.102 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:57:21.102 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T04:57:21.102 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:57:21.102 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:57:21.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.095+0000 7fb997e1f700 1 -- 172.21.15.142:0/1563574450 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb97c06dd60 msgr2=0x7fb97c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:21.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.095+0000 7fb997e1f700 1 --2- 172.21.15.142:0/1563574450 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb97c06dd60 0x7fb97c070220 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb990102e00 tx=0x7fb980005f50 comp rx=0 tx=0).stop 2024-03-29T04:57:21.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.095+0000 7fb997e1f700 1 -- 172.21.15.142:0/1563574450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9900ffbd0 msgr2=0x7fb990101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:21.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.095+0000 7fb997e1f700 1 --2- 172.21.15.142:0/1563574450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9900ffbd0 0x7fb990101610 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb98c006010 tx=0x7fb98c00be20 comp rx=0 tx=0).stop 2024-03-29T04:57:21.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.095+0000 7fb997e1f700 1 -- 172.21.15.142:0/1563574450 shutdown_connections 2024-03-29T04:57:21.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.095+0000 7fb997e1f700 1 --2- 172.21.15.142:0/1563574450 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb97c06dd60 0x7fb97c070220 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.095+0000 7fb997e1f700 1 --2- 172.21.15.142:0/1563574450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9900ffbd0 0x7fb990101610 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.095+0000 7fb997e1f700 1 --2- 172.21.15.142:0/1563574450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb990105a40 0x7fb990101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.095+0000 7fb997e1f700 1 -- 172.21.15.142:0/1563574450 >> 172.21.15.142:0/1563574450 conn(0x7fb9900755a0 msgr2=0x7fb9900685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:21.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.095+0000 7fb997e1f700 1 -- 172.21.15.142:0/1563574450 shutdown_connections 2024-03-29T04:57:21.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.095+0000 7fb997e1f700 1 -- 172.21.15.142:0/1563574450 wait complete. 2024-03-29T04:57:21.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.199+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/65712225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb8105a40 msgr2=0x7f3cb8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:21.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.199+0000 7f3cbf1d2700 1 --2- 172.21.15.142:0/65712225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb8105a40 0x7f3cb8105e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009910 tx=0x7f3cac009c20 comp rx=0 tx=0).stop 2024-03-29T04:57:21.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.199+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/65712225 shutdown_connections 2024-03-29T04:57:21.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.199+0000 7f3cbf1d2700 1 --2- 172.21.15.142:0/65712225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb80ffa40 0x7f3cb80ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.199+0000 7f3cbf1d2700 1 --2- 172.21.15.142:0/65712225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb8105a40 0x7f3cb8105e20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.200+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/65712225 >> 172.21.15.142:0/65712225 conn(0x7f3cb80fb550 msgr2=0x7f3cb80fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:21.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.200+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/65712225 shutdown_connections 2024-03-29T04:57:21.207 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.200+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/65712225 wait complete. 2024-03-29T04:57:21.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.200+0000 7f3cbf1d2700 1 Processor -- start 2024-03-29T04:57:21.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.200+0000 7f3cbf1d2700 1 -- start start 2024-03-29T04:57:21.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbf1d2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb80ffa40 0x7f3cb8197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:21.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbf1d2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb8105a40 0x7f3cb8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:21.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbf1d2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb81980d0 con 0x7f3cb80ffa40 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbf1d2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb8198240 con 0x7f3cb8105a40 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbcf6e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb80ffa40 0x7f3cb8197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbcf6e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb80ffa40 0x7f3cb8197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40302/0 (socket says 172.21.15.142:40302) 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbcf6e700 1 -- 172.21.15.142:0/258574922 learned_addr learned my addr 172.21.15.142:0/258574922 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cb7fff700 1 --2- 172.21.15.142:0/258574922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb8105a40 0x7f3cb8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbcf6e700 1 -- 172.21.15.142:0/258574922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb8105a40 msgr2=0x7f3cb8197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbcf6e700 1 --2- 172.21.15.142:0/258574922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb8105a40 0x7f3cb8197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbcf6e700 1 -- 172.21.15.142:0/258574922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac0095f0 con 0x7f3cb80ffa40 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cb7fff700 1 --2- 172.21.15.142:0/258574922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb8105a40 0x7f3cb8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbcf6e700 1 --2- 172.21.15.142:0/258574922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb80ffa40 0x7f3cb8197570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0092e0 tx=0x7f3cac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cb5ffb700 1 -- 172.21.15.142:0/258574922 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cac01d070 con 0x7f3cb80ffa40 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cb5ffb700 1 -- 172.21.15.142:0/258574922 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cac005e00 con 0x7f3cb80ffa40 2024-03-29T04:57:21.210 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/258574922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb81a05e0 con 0x7f3cb80ffa40 2024-03-29T04:57:21.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cb5ffb700 1 -- 172.21.15.142:0/258574922 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cac017400 con 0x7f3cb80ffa40 2024-03-29T04:57:21.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.201+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/258574922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb81a0a50 con 0x7f3cb80ffa40 2024-03-29T04:57:21.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.202+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/258574922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb8108040 con 0x7f3cb80ffa40 2024-03-29T04:57:21.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.203+0000 7f3cb5ffb700 1 -- 172.21.15.142:0/258574922 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f3cac00f460 con 0x7f3cb80ffa40 2024-03-29T04:57:21.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.203+0000 7f3cb5ffb700 1 --2- 172.21.15.142:0/258574922 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3ca806dd60 0x7f3ca8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:21.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.203+0000 7f3cb5ffb700 1 -- 172.21.15.142:0/258574922 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f3cac08f1d0 con 0x7f3cb80ffa40 2024-03-29T04:57:21.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.203+0000 7f3cb7fff700 1 --2- 172.21.15.142:0/258574922 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3ca806dd60 0x7f3ca8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:21.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.204+0000 7f3cb7fff700 1 --2- 172.21.15.142:0/258574922 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3ca806dd60 0x7f3ca8070220 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8069680 tx=0x7f3ca40092a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:21.215 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.208+0000 7f3cb5ffb700 1 -- 172.21.15.142:0/258574922 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3cac0585f0 con 0x7f3cb80ffa40 2024-03-29T04:57:21.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:21 smithi142 ceph-mon[112475]: pgmap v271: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:21.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:21 smithi142 ceph-mon[112475]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:21.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:21 smithi142 ceph-mon[112475]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:21.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:21 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1563574450' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:57:21.457 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.449+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/258574922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3cb802cd60 con 0x7f3cb80ffa40 2024-03-29T04:57:21.457 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.450+0000 7f3cb5ffb700 1 -- 172.21.15.142:0/258574922 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f3cac05bcb0 con 0x7f3cb80ffa40 2024-03-29T04:57:21.457 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T04:57:21.458 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:57:21.458 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:57:21.458 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:57:21.458 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:57:21.458 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:57:21.458 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:57:21.459 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T04:57:21.459 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T04:57:21.459 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:57:21.459 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T04:57:21.459 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:57:21.459 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:57:21.459 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:57:21.459 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:57:21.459 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:57:21.459 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:57:21.460 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:57:21.460 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:57:21.460 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T04:57:21.460 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:57:21.460 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:57:21.460 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:57:21.460 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T04:57:21.460 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T04:57:21.460 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:57:21.460 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:57:21.460 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:57:21.461 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:57:21.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.452+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/258574922 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3ca806dd60 msgr2=0x7f3ca8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:21.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.452+0000 7f3cbf1d2700 1 --2- 172.21.15.142:0/258574922 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3ca806dd60 0x7f3ca8070220 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8069680 tx=0x7f3ca40092a0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.452+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/258574922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb80ffa40 msgr2=0x7f3cb8197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:21.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.452+0000 7f3cbf1d2700 1 --2- 172.21.15.142:0/258574922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb80ffa40 0x7f3cb8197570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0092e0 tx=0x7f3cac00be20 comp rx=0 tx=0).stop 2024-03-29T04:57:21.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.452+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/258574922 shutdown_connections 2024-03-29T04:57:21.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.452+0000 7f3cbf1d2700 1 --2- 172.21.15.142:0/258574922 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3ca806dd60 0x7f3ca8070220 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.452+0000 7f3cbf1d2700 1 --2- 172.21.15.142:0/258574922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3cb80ffa40 0x7f3cb8197570 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.452+0000 7f3cbf1d2700 1 --2- 172.21.15.142:0/258574922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3cb8105a40 0x7f3cb8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.452+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/258574922 >> 172.21.15.142:0/258574922 conn(0x7f3cb80fb550 msgr2=0x7f3cb80fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:21.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.453+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/258574922 shutdown_connections 2024-03-29T04:57:21.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.453+0000 7f3cbf1d2700 1 -- 172.21.15.142:0/258574922 wait complete. 2024-03-29T04:57:21.463 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T04:57:21.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:21 smithi152 ceph-mon[88186]: pgmap v271: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:21.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:21 smithi152 ceph-mon[88186]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:21.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:21 smithi152 ceph-mon[88186]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:21.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:21 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1563574450' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:57:21.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.558+0000 7fa806209700 1 -- 172.21.15.142:0/3940063798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa800100de0 msgr2=0x7fa8001011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:21.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.558+0000 7fa806209700 1 --2- 172.21.15.142:0/3940063798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa800100de0 0x7fa8001011c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800b0a0 tx=0x7fa7f800b3b0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.558+0000 7fa806209700 1 -- 172.21.15.142:0/3940063798 shutdown_connections 2024-03-29T04:57:21.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.558+0000 7fa806209700 1 --2- 172.21.15.142:0/3940063798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa800101790 0x7fa8001057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.558+0000 7fa806209700 1 --2- 172.21.15.142:0/3940063798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa800100de0 0x7fa8001011c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.558+0000 7fa806209700 1 -- 172.21.15.142:0/3940063798 >> 172.21.15.142:0/3940063798 conn(0x7fa8000fc670 msgr2=0x7fa8000fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:21.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.558+0000 7fa806209700 1 -- 172.21.15.142:0/3940063798 shutdown_connections 2024-03-29T04:57:21.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.558+0000 7fa806209700 1 -- 172.21.15.142:0/3940063798 wait complete. 2024-03-29T04:57:21.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.559+0000 7fa806209700 1 Processor -- start 2024-03-29T04:57:21.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.559+0000 7fa806209700 1 -- start start 2024-03-29T04:57:21.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.559+0000 7fa806209700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa800100de0 0x7fa800073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:21.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.559+0000 7fa806209700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa800101790 0x7fa8000735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:21.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.559+0000 7fa806209700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa800073b10 con 0x7fa800101790 2024-03-29T04:57:21.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.559+0000 7fa806209700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa800073c50 con 0x7fa800100de0 2024-03-29T04:57:21.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.559+0000 7fa7feffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa800101790 0x7fa8000735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:21.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.559+0000 7fa7feffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa800101790 0x7fa8000735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40316/0 (socket says 172.21.15.142:40316) 2024-03-29T04:57:21.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.559+0000 7fa7feffd700 1 -- 172.21.15.142:0/478237053 learned_addr learned my addr 172.21.15.142:0/478237053 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:21.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.560+0000 7fa7ff7fe700 1 --2- 172.21.15.142:0/478237053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa800100de0 0x7fa800073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:21.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.560+0000 7fa7feffd700 1 -- 172.21.15.142:0/478237053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa800100de0 msgr2=0x7fa800073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:21.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.560+0000 7fa7feffd700 1 --2- 172.21.15.142:0/478237053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa800100de0 0x7fa800073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.560+0000 7fa7feffd700 1 -- 172.21.15.142:0/478237053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f8009d00 con 0x7fa800101790 2024-03-29T04:57:21.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.560+0000 7fa7feffd700 1 --2- 172.21.15.142:0/478237053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa800101790 0x7fa8000735d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800e8f0 tx=0x7fa7e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:21.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.560+0000 7fa7fcff9700 1 -- 172.21.15.142:0/478237053 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7e8009790 con 0x7fa800101790 2024-03-29T04:57:21.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.560+0000 7fa7fcff9700 1 -- 172.21.15.142:0/478237053 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7e8004cf0 con 0x7fa800101790 2024-03-29T04:57:21.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.560+0000 7fa806209700 1 -- 172.21.15.142:0/478237053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8001a2930 con 0x7fa800101790 2024-03-29T04:57:21.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.560+0000 7fa7fcff9700 1 -- 172.21.15.142:0/478237053 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7e8005690 con 0x7fa800101790 2024-03-29T04:57:21.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.560+0000 7fa806209700 1 -- 172.21.15.142:0/478237053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8001a2d40 con 0x7fa800101790 2024-03-29T04:57:21.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.561+0000 7fa806209700 1 -- 172.21.15.142:0/478237053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa800109180 con 0x7fa800101790 2024-03-29T04:57:21.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.561+0000 7fa7fcff9700 1 -- 172.21.15.142:0/478237053 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fa7e80104e0 con 0x7fa800101790 2024-03-29T04:57:21.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.562+0000 7fa7fcff9700 1 --2- 172.21.15.142:0/478237053 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa7ec06dd60 0x7fa7ec070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:21.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.562+0000 7fa7fcff9700 1 -- 172.21.15.142:0/478237053 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa7e8014070 con 0x7fa800101790 2024-03-29T04:57:21.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.562+0000 7fa7ff7fe700 1 --2- 172.21.15.142:0/478237053 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa7ec06dd60 0x7fa7ec070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:21.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.563+0000 7fa7ff7fe700 1 --2- 172.21.15.142:0/478237053 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa7ec06dd60 0x7fa7ec070220 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800b780 tx=0x7fa7f8006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:21.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.567+0000 7fa7fcff9700 1 -- 172.21.15.142:0/478237053 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa7e80574b0 con 0x7fa800101790 2024-03-29T04:57:21.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.784+0000 7fa806209700 1 -- 172.21.15.142:0/478237053 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa80002cce0 con 0x7fa7ec06dd60 2024-03-29T04:57:21.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.786+0000 7fa7fcff9700 1 -- 172.21.15.142:0/478237053 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa80002cce0 con 0x7fa7ec06dd60 2024-03-29T04:57:21.793 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:57:21.793 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:57:21.793 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:57:21.793 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:57:21.793 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:57:21.793 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:57:21.793 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:57:21.793 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T04:57:21.793 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T04:57:21.793 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T04:57:21.793 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:57:21.794 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T04:57:21.794 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T04:57:21.794 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:57:21.794 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:57:21.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.788+0000 7fa806209700 1 -- 172.21.15.142:0/478237053 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa7ec06dd60 msgr2=0x7fa7ec070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:21.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.788+0000 7fa806209700 1 --2- 172.21.15.142:0/478237053 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa7ec06dd60 0x7fa7ec070220 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800b780 tx=0x7fa7f8006000 comp rx=0 tx=0).stop 2024-03-29T04:57:21.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.788+0000 7fa806209700 1 -- 172.21.15.142:0/478237053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa800101790 msgr2=0x7fa8000735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:21.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.788+0000 7fa806209700 1 --2- 172.21.15.142:0/478237053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa800101790 0x7fa8000735d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800e8f0 tx=0x7fa7e800ecb0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.788+0000 7fa806209700 1 -- 172.21.15.142:0/478237053 shutdown_connections 2024-03-29T04:57:21.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.788+0000 7fa806209700 1 --2- 172.21.15.142:0/478237053 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa7ec06dd60 0x7fa7ec070220 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.788+0000 7fa806209700 1 --2- 172.21.15.142:0/478237053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa800100de0 0x7fa800073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.788+0000 7fa806209700 1 --2- 172.21.15.142:0/478237053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa800101790 0x7fa8000735d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.788+0000 7fa806209700 1 -- 172.21.15.142:0/478237053 >> 172.21.15.142:0/478237053 conn(0x7fa8000fc670 msgr2=0x7fa8000fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:21.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.788+0000 7fa806209700 1 -- 172.21.15.142:0/478237053 shutdown_connections 2024-03-29T04:57:21.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.788+0000 7fa806209700 1 -- 172.21.15.142:0/478237053 wait complete. 2024-03-29T04:57:21.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.892+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2874139273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfa80ffbd0 msgr2=0x7fcfa8100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:21.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.892+0000 7fcfac8c5700 1 --2- 172.21.15.142:0/2874139273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfa80ffbd0 0x7fcfa8100050 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009910 tx=0x7fcf9c009c20 comp rx=0 tx=0).stop 2024-03-29T04:57:21.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.892+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2874139273 shutdown_connections 2024-03-29T04:57:21.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.892+0000 7fcfac8c5700 1 --2- 172.21.15.142:0/2874139273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfa80ffbd0 0x7fcfa8100050 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.892+0000 7fcfac8c5700 1 --2- 172.21.15.142:0/2874139273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa8105a40 0x7fcfa8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.892+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2874139273 >> 172.21.15.142:0/2874139273 conn(0x7fcfa80755a0 msgr2=0x7fcfa80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:21.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.892+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2874139273 shutdown_connections 2024-03-29T04:57:21.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.892+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2874139273 wait complete. 2024-03-29T04:57:21.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.893+0000 7fcfac8c5700 1 Processor -- start 2024-03-29T04:57:21.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.893+0000 7fcfac8c5700 1 -- start start 2024-03-29T04:57:21.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.893+0000 7fcfac8c5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa80ffbd0 0x7fcfa819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.893+0000 7fcfac8c5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfa8105a40 0x7fcfa819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.893+0000 7fcfac8c5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa8196550 con 0x7fcfa8105a40 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.893+0000 7fcfac8c5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa81966c0 con 0x7fcfa80ffbd0 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.893+0000 7fcfa5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfa8105a40 0x7fcfa819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.893+0000 7fcfa5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfa8105a40 0x7fcfa819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40340/0 (socket says 172.21.15.142:40340) 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.893+0000 7fcfa5d9b700 1 -- 172.21.15.142:0/2880419181 learned_addr learned my addr 172.21.15.142:0/2880419181 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.893+0000 7fcfa659c700 1 --2- 172.21.15.142:0/2880419181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa80ffbd0 0x7fcfa819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.894+0000 7fcfa5d9b700 1 -- 172.21.15.142:0/2880419181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa80ffbd0 msgr2=0x7fcfa819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.894+0000 7fcfa5d9b700 1 --2- 172.21.15.142:0/2880419181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa80ffbd0 0x7fcfa819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.894+0000 7fcfa5d9b700 1 -- 172.21.15.142:0/2880419181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c0095f0 con 0x7fcfa8105a40 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.894+0000 7fcfa5d9b700 1 --2- 172.21.15.142:0/2880419181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfa8105a40 0x7fcfa819caa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c006010 tx=0x7fcf9c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.894+0000 7fcf977fe700 1 -- 172.21.15.142:0/2880419181 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf9c01d070 con 0x7fcfa8105a40 2024-03-29T04:57:21.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.894+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2880419181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa8196940 con 0x7fcfa8105a40 2024-03-29T04:57:21.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.894+0000 7fcf977fe700 1 -- 172.21.15.142:0/2880419181 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf9c005dc0 con 0x7fcfa8105a40 2024-03-29T04:57:21.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.894+0000 7fcf977fe700 1 -- 172.21.15.142:0/2880419181 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf9c017400 con 0x7fcfa8105a40 2024-03-29T04:57:21.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.894+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2880419181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa8196e30 con 0x7fcfa8105a40 2024-03-29T04:57:21.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.895+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2880419181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa804ea90 con 0x7fcfa8105a40 2024-03-29T04:57:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.895+0000 7fcf977fe700 1 -- 172.21.15.142:0/2880419181 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fcf9c003a60 con 0x7fcfa8105a40 2024-03-29T04:57:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.896+0000 7fcf977fe700 1 --2- 172.21.15.142:0/2880419181 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf9006dd60 0x7fcf90070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.896+0000 7fcf977fe700 1 -- 172.21.15.142:0/2880419181 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fcf9c08f080 con 0x7fcfa8105a40 2024-03-29T04:57:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.896+0000 7fcfa659c700 1 --2- 172.21.15.142:0/2880419181 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf9006dd60 0x7fcf90070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:21.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.896+0000 7fcfa659c700 1 --2- 172.21.15.142:0/2880419181 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf9006dd60 0x7fcf90070220 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcf98005950 tx=0x7fcf9800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:21.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:21.900+0000 7fcf977fe700 1 -- 172.21.15.142:0/2880419181 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcf9c058520 con 0x7fcfa8105a40 2024-03-29T04:57:22.181 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.174+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2880419181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcfa8197140 con 0x7fcfa8105a40 2024-03-29T04:57:22.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.174+0000 7fcf977fe700 1 -- 172.21.15.142:0/2880419181 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fcf9c05bbe0 con 0x7fcfa8105a40 2024-03-29T04:57:22.182 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T04:57:22.182 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T04:57:22.182 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T04:57:22.182 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T04:57:22.182 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T04:57:22.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.176+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2880419181 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf9006dd60 msgr2=0x7fcf90070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:22.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.176+0000 7fcfac8c5700 1 --2- 172.21.15.142:0/2880419181 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf9006dd60 0x7fcf90070220 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcf98005950 tx=0x7fcf9800b410 comp rx=0 tx=0).stop 2024-03-29T04:57:22.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.176+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2880419181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfa8105a40 msgr2=0x7fcfa819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:22.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.176+0000 7fcfac8c5700 1 --2- 172.21.15.142:0/2880419181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfa8105a40 0x7fcfa819caa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c006010 tx=0x7fcf9c00bd70 comp rx=0 tx=0).stop 2024-03-29T04:57:22.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.177+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2880419181 shutdown_connections 2024-03-29T04:57:22.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.177+0000 7fcfac8c5700 1 --2- 172.21.15.142:0/2880419181 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf9006dd60 0x7fcf90070220 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:22.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.177+0000 7fcfac8c5700 1 --2- 172.21.15.142:0/2880419181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfa80ffbd0 0x7fcfa819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:22.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.177+0000 7fcfac8c5700 1 --2- 172.21.15.142:0/2880419181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfa8105a40 0x7fcfa819caa0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:22.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.177+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2880419181 >> 172.21.15.142:0/2880419181 conn(0x7fcfa80755a0 msgr2=0x7fcfa8103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:22.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.177+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2880419181 shutdown_connections 2024-03-29T04:57:22.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:22.177+0000 7fcfac8c5700 1 -- 172.21.15.142:0/2880419181 wait complete. 2024-03-29T04:57:22.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:22 smithi152 ceph-mon[88186]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:22.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:22 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/258574922' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:57:22.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:22 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2880419181' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:57:22.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:22 smithi142 ceph-mon[112475]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:22.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:22 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/258574922' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:57:22.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:22 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2880419181' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:57:23.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:23 smithi152 ceph-mon[88186]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:23.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:23 smithi152 ceph-mon[88186]: pgmap v272: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:23.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:23 smithi142 ceph-mon[112475]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:23.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:23 smithi142 ceph-mon[112475]: pgmap v272: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:25.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:25 smithi152 ceph-mon[88186]: pgmap v273: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:25.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:25 smithi142 ceph-mon[112475]: pgmap v273: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:27.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:27 smithi152 ceph-mon[88186]: pgmap v274: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:27.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:27 smithi142 ceph-mon[112475]: pgmap v274: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:29.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:29 smithi152 ceph-mon[88186]: pgmap v275: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:29.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:29 smithi142 ceph-mon[112475]: pgmap v275: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:30.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:57:30.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:57:31.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:31 smithi152 ceph-mon[88186]: pgmap v276: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:31.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:31 smithi142 ceph-mon[112475]: pgmap v276: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:33.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:33 smithi152 ceph-mon[88186]: pgmap v277: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:33.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:33 smithi142 ceph-mon[112475]: pgmap v277: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:35.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:35 smithi152 ceph-mon[88186]: pgmap v278: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:35.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:35 smithi142 ceph-mon[112475]: pgmap v278: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:37.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:37 smithi152 ceph-mon[88186]: pgmap v279: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:37.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:37 smithi142 ceph-mon[112475]: pgmap v279: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:39.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:39 smithi152 ceph-mon[88186]: pgmap v280: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:39.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:39 smithi142 ceph-mon[112475]: pgmap v280: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:41.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:41 smithi152 ceph-mon[88186]: pgmap v281: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:41.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:41 smithi142 ceph-mon[112475]: pgmap v281: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:43.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:43 smithi152 ceph-mon[88186]: pgmap v282: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:43.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:43 smithi142 ceph-mon[112475]: pgmap v282: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:45.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:45 smithi152 ceph-mon[88186]: pgmap v283: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:45.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:57:45.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:45 smithi142 ceph-mon[112475]: pgmap v283: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:45.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:57:47.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:47 smithi152 ceph-mon[88186]: pgmap v284: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:47.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:47 smithi142 ceph-mon[112475]: pgmap v284: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:49.573 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:49 smithi152 ceph-mon[88186]: pgmap v285: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:49.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:49 smithi142 ceph-mon[112475]: pgmap v285: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:51.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:51 smithi142 ceph-mon[112475]: pgmap v286: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:51.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:51 smithi152 ceph-mon[88186]: pgmap v286: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:52.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.294+0000 7f92b74c9700 1 -- 172.21.15.142:0/3290398820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b0100dc0 msgr2=0x7f92b01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:52.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.294+0000 7f92b74c9700 1 --2- 172.21.15.142:0/3290398820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b0100dc0 0x7f92b01011a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009910 tx=0x7f92ac009c20 comp rx=0 tx=0).stop 2024-03-29T04:57:52.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.294+0000 7f92b74c9700 1 -- 172.21.15.142:0/3290398820 shutdown_connections 2024-03-29T04:57:52.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.294+0000 7f92b74c9700 1 --2- 172.21.15.142:0/3290398820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b0101770 0x7f92b01057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.294+0000 7f92b74c9700 1 --2- 172.21.15.142:0/3290398820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b0100dc0 0x7f92b01011a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.294+0000 7f92b74c9700 1 -- 172.21.15.142:0/3290398820 >> 172.21.15.142:0/3290398820 conn(0x7f92b00fc650 msgr2=0x7f92b00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:52.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.294+0000 7f92b74c9700 1 -- 172.21.15.142:0/3290398820 shutdown_connections 2024-03-29T04:57:52.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.294+0000 7f92b74c9700 1 -- 172.21.15.142:0/3290398820 wait complete. 2024-03-29T04:57:52.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.294+0000 7f92b74c9700 1 Processor -- start 2024-03-29T04:57:52.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.294+0000 7f92b74c9700 1 -- start start 2024-03-29T04:57:52.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.295+0000 7f92b74c9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b0100dc0 0x7f92b019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:52.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.295+0000 7f92b74c9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b0101770 0x7f92b019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:52.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.295+0000 7f92b74c9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b019d110 con 0x7f92b0100dc0 2024-03-29T04:57:52.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.295+0000 7f92b74c9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b01965c0 con 0x7f92b0101770 2024-03-29T04:57:52.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.295+0000 7f92b5265700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b0100dc0 0x7f92b019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:52.304 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.295+0000 7f92b5265700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b0100dc0 0x7f92b019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:36666/0 (socket says 172.21.15.142:36666) 2024-03-29T04:57:52.304 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.295+0000 7f92b5265700 1 -- 172.21.15.142:0/2629805468 learned_addr learned my addr 172.21.15.142:0/2629805468 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:52.304 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.295+0000 7f92b4a64700 1 --2- 172.21.15.142:0/2629805468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b0101770 0x7f92b019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:52.304 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.295+0000 7f92b4a64700 1 -- 172.21.15.142:0/2629805468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b0100dc0 msgr2=0x7f92b019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:52.304 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.295+0000 7f92b4a64700 1 --2- 172.21.15.142:0/2629805468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b0100dc0 0x7f92b019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.304 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.295+0000 7f92b4a64700 1 -- 172.21.15.142:0/2629805468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0095f0 con 0x7f92b0101770 2024-03-29T04:57:52.304 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.295+0000 7f92b4a64700 1 --2- 172.21.15.142:0/2629805468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b0101770 0x7f92b019ca80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f92a000e940 tx=0x7f92a000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:52.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.297+0000 7f92a67fc700 1 -- 172.21.15.142:0/2629805468 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92a000c990 con 0x7f92b0101770 2024-03-29T04:57:52.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.297+0000 7f92a67fc700 1 -- 172.21.15.142:0/2629805468 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92a0010430 con 0x7f92b0101770 2024-03-29T04:57:52.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.297+0000 7f92b74c9700 1 -- 172.21.15.142:0/2629805468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b01968a0 con 0x7f92b0101770 2024-03-29T04:57:52.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.297+0000 7f92a67fc700 1 -- 172.21.15.142:0/2629805468 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92a0018650 con 0x7f92b0101770 2024-03-29T04:57:52.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.297+0000 7f92b74c9700 1 -- 172.21.15.142:0/2629805468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b0196df0 con 0x7f92b0101770 2024-03-29T04:57:52.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.298+0000 7f92b74c9700 1 -- 172.21.15.142:0/2629805468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b0109160 con 0x7f92b0101770 2024-03-29T04:57:52.306 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.299+0000 7f92a67fc700 1 -- 172.21.15.142:0/2629805468 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f92a000caf0 con 0x7f92b0101770 2024-03-29T04:57:52.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.299+0000 7f92a67fc700 1 --2- 172.21.15.142:0/2629805468 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f929c06ddb0 0x7f929c070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:52.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.299+0000 7f92a67fc700 1 -- 172.21.15.142:0/2629805468 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f92a0014070 con 0x7f92b0101770 2024-03-29T04:57:52.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.299+0000 7f92b5265700 1 --2- 172.21.15.142:0/2629805468 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f929c06ddb0 0x7f929c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:52.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.300+0000 7f92b5265700 1 --2- 172.21.15.142:0/2629805468 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f929c06ddb0 0x7f929c070270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00b760 tx=0x7f92ac005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:52.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.304+0000 7f92a67fc700 1 -- 172.21.15.142:0/2629805468 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f92a0053280 con 0x7f92b0101770 2024-03-29T04:57:52.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.520+0000 7f92b74c9700 1 -- 172.21.15.142:0/2629805468 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92b01970d0 con 0x7f929c06ddb0 2024-03-29T04:57:52.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.522+0000 7f92a67fc700 1 -- 172.21.15.142:0/2629805468 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f92b01970d0 con 0x7f929c06ddb0 2024-03-29T04:57:52.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.524+0000 7f92b74c9700 1 -- 172.21.15.142:0/2629805468 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f929c06ddb0 msgr2=0x7f929c070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:52.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.524+0000 7f92b74c9700 1 --2- 172.21.15.142:0/2629805468 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f929c06ddb0 0x7f929c070270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00b760 tx=0x7f92ac005a90 comp rx=0 tx=0).stop 2024-03-29T04:57:52.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.524+0000 7f92b74c9700 1 -- 172.21.15.142:0/2629805468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b0101770 msgr2=0x7f92b019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:52.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.524+0000 7f92b74c9700 1 --2- 172.21.15.142:0/2629805468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b0101770 0x7f92b019ca80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f92a000e940 tx=0x7f92a000ec50 comp rx=0 tx=0).stop 2024-03-29T04:57:52.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.524+0000 7f92b74c9700 1 -- 172.21.15.142:0/2629805468 shutdown_connections 2024-03-29T04:57:52.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.524+0000 7f92b74c9700 1 --2- 172.21.15.142:0/2629805468 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f929c06ddb0 0x7f929c070270 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.524+0000 7f92b74c9700 1 --2- 172.21.15.142:0/2629805468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92b0100dc0 0x7f92b019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.524+0000 7f92b74c9700 1 --2- 172.21.15.142:0/2629805468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b0101770 0x7f92b019ca80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.524+0000 7f92b74c9700 1 -- 172.21.15.142:0/2629805468 >> 172.21.15.142:0/2629805468 conn(0x7f92b00fc650 msgr2=0x7f92b00fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:52.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.524+0000 7f92b74c9700 1 -- 172.21.15.142:0/2629805468 shutdown_connections 2024-03-29T04:57:52.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.524+0000 7f92b74c9700 1 -- 172.21.15.142:0/2629805468 wait complete. 2024-03-29T04:57:52.541 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:57:52.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.629+0000 7fba8106d700 1 -- 172.21.15.142:0/1466880730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c106be0 msgr2=0x7fba7c106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:52.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.629+0000 7fba8106d700 1 --2- 172.21.15.142:0/1466880730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c106be0 0x7fba7c106fc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fba7400b0a0 tx=0x7fba7400b3b0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.629+0000 7fba8106d700 1 -- 172.21.15.142:0/1466880730 shutdown_connections 2024-03-29T04:57:52.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.629+0000 7fba8106d700 1 --2- 172.21.15.142:0/1466880730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba7c0ff260 0x7fba7c0ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.629+0000 7fba8106d700 1 --2- 172.21.15.142:0/1466880730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c106be0 0x7fba7c106fc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.629+0000 7fba8106d700 1 -- 172.21.15.142:0/1466880730 >> 172.21.15.142:0/1466880730 conn(0x7fba7c074ce0 msgr2=0x7fba7c0750f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:52.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.629+0000 7fba8106d700 1 -- 172.21.15.142:0/1466880730 shutdown_connections 2024-03-29T04:57:52.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.629+0000 7fba8106d700 1 -- 172.21.15.142:0/1466880730 wait complete. 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.630+0000 7fba8106d700 1 Processor -- start 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.630+0000 7fba8106d700 1 -- start start 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.630+0000 7fba8106d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c0ff260 0x7fba7c100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.630+0000 7fba8106d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba7c106be0 0x7fba7c100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.630+0000 7fba8106d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c100ec0 con 0x7fba7c0ff260 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.630+0000 7fba8106d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c101000 con 0x7fba7c106be0 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.630+0000 7fba7ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c0ff260 0x7fba7c100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.630+0000 7fba7ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c0ff260 0x7fba7c100440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:36686/0 (socket says 172.21.15.142:36686) 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.630+0000 7fba7ad9d700 1 -- 172.21.15.142:0/599697076 learned_addr learned my addr 172.21.15.142:0/599697076 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.630+0000 7fba7a59c700 1 --2- 172.21.15.142:0/599697076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba7c106be0 0x7fba7c100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.631+0000 7fba7ad9d700 1 -- 172.21.15.142:0/599697076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba7c106be0 msgr2=0x7fba7c100980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.631+0000 7fba7ad9d700 1 --2- 172.21.15.142:0/599697076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba7c106be0 0x7fba7c100980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.631+0000 7fba7ad9d700 1 -- 172.21.15.142:0/599697076 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba74009d00 con 0x7fba7c0ff260 2024-03-29T04:57:52.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.631+0000 7fba7a59c700 1 --2- 172.21.15.142:0/599697076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba7c106be0 0x7fba7c100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:57:52.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.631+0000 7fba7ad9d700 1 --2- 172.21.15.142:0/599697076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c0ff260 0x7fba7c100440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fba74000f80 tx=0x7fba7400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:52.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.631+0000 7fba73fff700 1 -- 172.21.15.142:0/599697076 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba7400e040 con 0x7fba7c0ff260 2024-03-29T04:57:52.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.631+0000 7fba8106d700 1 -- 172.21.15.142:0/599697076 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7c101280 con 0x7fba7c0ff260 2024-03-29T04:57:52.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.631+0000 7fba73fff700 1 -- 172.21.15.142:0/599697076 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba74003e60 con 0x7fba7c0ff260 2024-03-29T04:57:52.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.631+0000 7fba73fff700 1 -- 172.21.15.142:0/599697076 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba7401d910 con 0x7fba7c0ff260 2024-03-29T04:57:52.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.631+0000 7fba8106d700 1 -- 172.21.15.142:0/599697076 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c1a2ca0 con 0x7fba7c0ff260 2024-03-29T04:57:52.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.632+0000 7fba8106d700 1 -- 172.21.15.142:0/599697076 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7c104050 con 0x7fba7c0ff260 2024-03-29T04:57:52.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.632+0000 7fba73fff700 1 -- 172.21.15.142:0/599697076 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fba74004820 con 0x7fba7c0ff260 2024-03-29T04:57:52.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.633+0000 7fba73fff700 1 --2- 172.21.15.142:0/599697076 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fba6806dd60 0x7fba68070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:52.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.633+0000 7fba7a59c700 1 --2- 172.21.15.142:0/599697076 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fba6806dd60 0x7fba68070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:52.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.633+0000 7fba73fff700 1 -- 172.21.15.142:0/599697076 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fba7408f590 con 0x7fba7c0ff260 2024-03-29T04:57:52.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.633+0000 7fba7a59c700 1 --2- 172.21.15.142:0/599697076 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fba6806dd60 0x7fba68070220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fba7c101c30 tx=0x7fba64008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:52.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.637+0000 7fba73fff700 1 -- 172.21.15.142:0/599697076 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fba740589b0 con 0x7fba7c0ff260 2024-03-29T04:57:52.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.853+0000 7fba8106d700 1 -- 172.21.15.142:0/599697076 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fba7c02cce0 con 0x7fba6806dd60 2024-03-29T04:57:52.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.854+0000 7fba73fff700 1 -- 172.21.15.142:0/599697076 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fba7c02cce0 con 0x7fba6806dd60 2024-03-29T04:57:52.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.857+0000 7fba8106d700 1 -- 172.21.15.142:0/599697076 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fba6806dd60 msgr2=0x7fba68070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:52.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.857+0000 7fba8106d700 1 --2- 172.21.15.142:0/599697076 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fba6806dd60 0x7fba68070220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fba7c101c30 tx=0x7fba64008040 comp rx=0 tx=0).stop 2024-03-29T04:57:52.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.857+0000 7fba8106d700 1 -- 172.21.15.142:0/599697076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c0ff260 msgr2=0x7fba7c100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:52.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.857+0000 7fba8106d700 1 --2- 172.21.15.142:0/599697076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c0ff260 0x7fba7c100440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fba74000f80 tx=0x7fba7400cdd0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.857+0000 7fba8106d700 1 -- 172.21.15.142:0/599697076 shutdown_connections 2024-03-29T04:57:52.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.857+0000 7fba8106d700 1 --2- 172.21.15.142:0/599697076 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fba6806dd60 0x7fba68070220 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.857+0000 7fba8106d700 1 --2- 172.21.15.142:0/599697076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c0ff260 0x7fba7c100440 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.857+0000 7fba8106d700 1 --2- 172.21.15.142:0/599697076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba7c106be0 0x7fba7c100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.857+0000 7fba8106d700 1 -- 172.21.15.142:0/599697076 >> 172.21.15.142:0/599697076 conn(0x7fba7c074ce0 msgr2=0x7fba7c0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:52.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.857+0000 7fba8106d700 1 -- 172.21.15.142:0/599697076 shutdown_connections 2024-03-29T04:57:52.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.857+0000 7fba8106d700 1 -- 172.21.15.142:0/599697076 wait complete. 2024-03-29T04:57:52.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.960+0000 7fd80ec65700 1 -- 172.21.15.142:0/4156817792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd808100dc0 msgr2=0x7fd8081011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:52.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.960+0000 7fd80ec65700 1 --2- 172.21.15.142:0/4156817792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd808100dc0 0x7fd8081011a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009910 tx=0x7fd7fc009c20 comp rx=0 tx=0).stop 2024-03-29T04:57:52.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.960+0000 7fd80ec65700 1 -- 172.21.15.142:0/4156817792 shutdown_connections 2024-03-29T04:57:52.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.960+0000 7fd80ec65700 1 --2- 172.21.15.142:0/4156817792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd808101770 0x7fd8081057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.960+0000 7fd80ec65700 1 --2- 172.21.15.142:0/4156817792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd808100dc0 0x7fd8081011a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.960+0000 7fd80ec65700 1 -- 172.21.15.142:0/4156817792 >> 172.21.15.142:0/4156817792 conn(0x7fd8080fc650 msgr2=0x7fd8080fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:52.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.960+0000 7fd80ec65700 1 -- 172.21.15.142:0/4156817792 shutdown_connections 2024-03-29T04:57:52.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.960+0000 7fd80ec65700 1 -- 172.21.15.142:0/4156817792 wait complete. 2024-03-29T04:57:52.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.961+0000 7fd80ec65700 1 Processor -- start 2024-03-29T04:57:52.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.961+0000 7fd80ec65700 1 -- start start 2024-03-29T04:57:52.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.961+0000 7fd80ec65700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd808100dc0 0x7fd808073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:52.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.961+0000 7fd80ec65700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd808101770 0x7fd8080735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:52.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.961+0000 7fd80ec65700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd808073b10 con 0x7fd808101770 2024-03-29T04:57:52.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.961+0000 7fd80ec65700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd808073c50 con 0x7fd808100dc0 2024-03-29T04:57:52.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.961+0000 7fd807fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd808101770 0x7fd8080735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:52.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.961+0000 7fd807fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd808101770 0x7fd8080735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:36710/0 (socket says 172.21.15.142:36710) 2024-03-29T04:57:52.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.961+0000 7fd807fff700 1 -- 172.21.15.142:0/1639368944 learned_addr learned my addr 172.21.15.142:0/1639368944 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:52.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.961+0000 7fd80ca01700 1 --2- 172.21.15.142:0/1639368944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd808100dc0 0x7fd808073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:52.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.962+0000 7fd807fff700 1 -- 172.21.15.142:0/1639368944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd808100dc0 msgr2=0x7fd808073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:52.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.962+0000 7fd807fff700 1 --2- 172.21.15.142:0/1639368944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd808100dc0 0x7fd808073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:52.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.962+0000 7fd807fff700 1 -- 172.21.15.142:0/1639368944 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc0095f0 con 0x7fd808101770 2024-03-29T04:57:52.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.962+0000 7fd807fff700 1 --2- 172.21.15.142:0/1639368944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd808101770 0x7fd8080735d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400d900 tx=0x7fd7f400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:52.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.962+0000 7fd805ffb700 1 -- 172.21.15.142:0/1639368944 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7f4009900 con 0x7fd808101770 2024-03-29T04:57:52.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.962+0000 7fd805ffb700 1 -- 172.21.15.142:0/1639368944 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7f4010460 con 0x7fd808101770 2024-03-29T04:57:52.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.962+0000 7fd80ec65700 1 -- 172.21.15.142:0/1639368944 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8081a2940 con 0x7fd808101770 2024-03-29T04:57:52.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.962+0000 7fd805ffb700 1 -- 172.21.15.142:0/1639368944 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7f400f5f0 con 0x7fd808101770 2024-03-29T04:57:52.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.962+0000 7fd80ec65700 1 -- 172.21.15.142:0/1639368944 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8081a2d50 con 0x7fd808101770 2024-03-29T04:57:52.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.963+0000 7fd80ec65700 1 -- 172.21.15.142:0/1639368944 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd808109160 con 0x7fd808101770 2024-03-29T04:57:52.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.964+0000 7fd805ffb700 1 -- 172.21.15.142:0/1639368944 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fd7f40105d0 con 0x7fd808101770 2024-03-29T04:57:52.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.964+0000 7fd805ffb700 1 --2- 172.21.15.142:0/1639368944 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd7f806dd60 0x7fd7f8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:52.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.964+0000 7fd805ffb700 1 -- 172.21.15.142:0/1639368944 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd7f408d870 con 0x7fd808101770 2024-03-29T04:57:52.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.964+0000 7fd80ca01700 1 --2- 172.21.15.142:0/1639368944 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd7f806dd60 0x7fd7f8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:52.975 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.965+0000 7fd80ca01700 1 --2- 172.21.15.142:0/1639368944 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd7f806dd60 0x7fd7f8070220 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00b5c0 tx=0x7fd7fc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:52.976 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:52.969+0000 7fd805ffb700 1 -- 172.21.15.142:0/1639368944 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd7f4056c90 con 0x7fd808101770 2024-03-29T04:57:53.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.171+0000 7fd80ec65700 1 -- 172.21.15.142:0/1639368944 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd80802cda0 con 0x7fd7f806dd60 2024-03-29T04:57:53.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.182+0000 7fd805ffb700 1 -- 172.21.15.142:0/1639368944 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd80802cda0 con 0x7fd7f806dd60 2024-03-29T04:57:53.189 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:57:53.189 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (9m) 2m ago 21m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:57:53.189 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (21m) 2m ago 21m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:57:53.189 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (20m) 2m ago 20m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:57:53.189 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (6m) 2m ago 21m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:57:53.189 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (6m) 2m ago 20m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:57:53.189 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (9m) 2m ago 20m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (2m) 2m ago 15m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (3m) 2m ago 15m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (2m) 2m ago 15m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (2m) 2m ago 15m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (11m) 2m ago 22m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (10m) 2m ago 20m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (8m) 2m ago 22m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (7m) 2m ago 20m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (10m) 2m ago 21m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (10m) 2m ago 20m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (6m) 2m ago 19m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (5m) 2m ago 18m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (5m) 2m ago 18m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:57:53.190 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (4m) 2m ago 18m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T04:57:53.191 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (4m) 2m ago 17m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T04:57:53.191 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (3m) 2m ago 17m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T04:57:53.191 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (10m) 2m ago 20m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:57:53.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.184+0000 7fd80ec65700 1 -- 172.21.15.142:0/1639368944 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd7f806dd60 msgr2=0x7fd7f8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:53.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.184+0000 7fd80ec65700 1 --2- 172.21.15.142:0/1639368944 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd7f806dd60 0x7fd7f8070220 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00b5c0 tx=0x7fd7fc01a040 comp rx=0 tx=0).stop 2024-03-29T04:57:53.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.184+0000 7fd80ec65700 1 -- 172.21.15.142:0/1639368944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd808101770 msgr2=0x7fd8080735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:53.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.184+0000 7fd80ec65700 1 --2- 172.21.15.142:0/1639368944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd808101770 0x7fd8080735d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400d900 tx=0x7fd7f400dc10 comp rx=0 tx=0).stop 2024-03-29T04:57:53.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.184+0000 7fd80ec65700 1 -- 172.21.15.142:0/1639368944 shutdown_connections 2024-03-29T04:57:53.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.184+0000 7fd80ec65700 1 --2- 172.21.15.142:0/1639368944 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd7f806dd60 0x7fd7f8070220 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.184+0000 7fd80ec65700 1 --2- 172.21.15.142:0/1639368944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd808100dc0 0x7fd808073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.184+0000 7fd80ec65700 1 --2- 172.21.15.142:0/1639368944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd808101770 0x7fd8080735d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.184+0000 7fd80ec65700 1 -- 172.21.15.142:0/1639368944 >> 172.21.15.142:0/1639368944 conn(0x7fd8080fc650 msgr2=0x7fd8080fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:53.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.185+0000 7fd80ec65700 1 -- 172.21.15.142:0/1639368944 shutdown_connections 2024-03-29T04:57:53.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.185+0000 7fd80ec65700 1 -- 172.21.15.142:0/1639368944 wait complete. 2024-03-29T04:57:53.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.287+0000 7f2ec3137700 1 -- 172.21.15.142:0/831907269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc100dc0 msgr2=0x7f2ebc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:53.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.287+0000 7f2ec3137700 1 --2- 172.21.15.142:0/831907269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc100dc0 0x7f2ebc1011a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009910 tx=0x7f2eb4009c20 comp rx=0 tx=0).stop 2024-03-29T04:57:53.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.287+0000 7f2ec3137700 1 -- 172.21.15.142:0/831907269 shutdown_connections 2024-03-29T04:57:53.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.287+0000 7f2ec3137700 1 --2- 172.21.15.142:0/831907269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ebc101770 0x7f2ebc1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.287+0000 7f2ec3137700 1 --2- 172.21.15.142:0/831907269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc100dc0 0x7f2ebc1011a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.287+0000 7f2ec3137700 1 -- 172.21.15.142:0/831907269 >> 172.21.15.142:0/831907269 conn(0x7f2ebc0fc650 msgr2=0x7f2ebc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:53.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.287+0000 7f2ec3137700 1 -- 172.21.15.142:0/831907269 shutdown_connections 2024-03-29T04:57:53.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.287+0000 7f2ec3137700 1 -- 172.21.15.142:0/831907269 wait complete. 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.288+0000 7f2ec3137700 1 Processor -- start 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.288+0000 7f2ec3137700 1 -- start start 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.288+0000 7f2ec3137700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc100dc0 0x7f2ebc19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.288+0000 7f2ec3137700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ebc101770 0x7f2ebc19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.288+0000 7f2ec3137700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc19d110 con 0x7f2ebc100dc0 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.288+0000 7f2ec3137700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc1965c0 con 0x7f2ebc101770 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.288+0000 7f2ec0ed3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc100dc0 0x7f2ebc19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.288+0000 7f2ec0ed3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc100dc0 0x7f2ebc19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:36718/0 (socket says 172.21.15.142:36718) 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.288+0000 7f2ec0ed3700 1 -- 172.21.15.142:0/3163419412 learned_addr learned my addr 172.21.15.142:0/3163419412 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.288+0000 7f2ebbfff700 1 --2- 172.21.15.142:0/3163419412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ebc101770 0x7f2ebc19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.289+0000 7f2ec0ed3700 1 -- 172.21.15.142:0/3163419412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ebc101770 msgr2=0x7f2ebc19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.289+0000 7f2ec0ed3700 1 --2- 172.21.15.142:0/3163419412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ebc101770 0x7f2ebc19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.289+0000 7f2ec0ed3700 1 -- 172.21.15.142:0/3163419412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb40095f0 con 0x7f2ebc100dc0 2024-03-29T04:57:53.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.289+0000 7f2ec0ed3700 1 --2- 172.21.15.142:0/3163419412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc100dc0 0x7f2ebc19c540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4006010 tx=0x7f2eb400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:53.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.289+0000 7f2eb9ffb700 1 -- 172.21.15.142:0/3163419412 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eb401d070 con 0x7f2ebc100dc0 2024-03-29T04:57:53.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.289+0000 7f2eb9ffb700 1 -- 172.21.15.142:0/3163419412 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2eb400f460 con 0x7f2ebc100dc0 2024-03-29T04:57:53.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.289+0000 7f2ec3137700 1 -- 172.21.15.142:0/3163419412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc196840 con 0x7f2ebc100dc0 2024-03-29T04:57:53.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.289+0000 7f2eb9ffb700 1 -- 172.21.15.142:0/3163419412 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eb4017610 con 0x7f2ebc100dc0 2024-03-29T04:57:53.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.289+0000 7f2ec3137700 1 -- 172.21.15.142:0/3163419412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ebc196d30 con 0x7f2ebc100dc0 2024-03-29T04:57:53.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.290+0000 7f2ec3137700 1 -- 172.21.15.142:0/3163419412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ebc109160 con 0x7f2ebc100dc0 2024-03-29T04:57:53.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.291+0000 7f2eb9ffb700 1 -- 172.21.15.142:0/3163419412 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f2eb400f5d0 con 0x7f2ebc100dc0 2024-03-29T04:57:53.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.291+0000 7f2eb9ffb700 1 --2- 172.21.15.142:0/3163419412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2ea806dd60 0x7f2ea8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:53.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.291+0000 7f2eb9ffb700 1 -- 172.21.15.142:0/3163419412 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f2eb408f200 con 0x7f2ebc100dc0 2024-03-29T04:57:53.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.291+0000 7f2ebbfff700 1 --2- 172.21.15.142:0/3163419412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2ea806dd60 0x7f2ea8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:53.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.292+0000 7f2ebbfff700 1 --2- 172.21.15.142:0/3163419412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2ea806dd60 0x7f2ea8070220 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc1059b0 tx=0x7f2eac00c040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:53.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.296+0000 7f2eb9ffb700 1 -- 172.21.15.142:0/3163419412 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2eb405c150 con 0x7f2ebc100dc0 2024-03-29T04:57:53.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.572+0000 7f2ec3137700 1 -- 172.21.15.142:0/3163419412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2ebc04ea90 con 0x7f2ebc100dc0 2024-03-29T04:57:53.581 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:53 smithi142 ceph-mon[112475]: pgmap v287: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:53.581 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:53 smithi142 ceph-mon[112475]: from='client.44421 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:53.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.573+0000 7f2eb9ffb700 1 -- 172.21.15.142:0/3163419412 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f2eb405bce0 con 0x7f2ebc100dc0 2024-03-29T04:57:53.582 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:57:53.582 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:57:53.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:57:53.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:57:53.582 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:57:53.583 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:57:53.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:57:53.583 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:57:53.583 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T04:57:53.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:57:53.583 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:57:53.584 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T04:57:53.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:57:53.584 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:57:53.584 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T04:57:53.584 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:57:53.584 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:57:53.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.575+0000 7f2ec3137700 1 -- 172.21.15.142:0/3163419412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2ea806dd60 msgr2=0x7f2ea8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:53.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.575+0000 7f2ec3137700 1 --2- 172.21.15.142:0/3163419412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2ea806dd60 0x7f2ea8070220 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc1059b0 tx=0x7f2eac00c040 comp rx=0 tx=0).stop 2024-03-29T04:57:53.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.575+0000 7f2ec3137700 1 -- 172.21.15.142:0/3163419412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc100dc0 msgr2=0x7f2ebc19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:53.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.575+0000 7f2ec3137700 1 --2- 172.21.15.142:0/3163419412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc100dc0 0x7f2ebc19c540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4006010 tx=0x7f2eb400bd70 comp rx=0 tx=0).stop 2024-03-29T04:57:53.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.575+0000 7f2ec3137700 1 -- 172.21.15.142:0/3163419412 shutdown_connections 2024-03-29T04:57:53.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.575+0000 7f2ec3137700 1 --2- 172.21.15.142:0/3163419412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2ea806dd60 0x7f2ea8070220 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.575+0000 7f2ec3137700 1 --2- 172.21.15.142:0/3163419412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc100dc0 0x7f2ebc19c540 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.575+0000 7f2ec3137700 1 --2- 172.21.15.142:0/3163419412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ebc101770 0x7f2ebc19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.575+0000 7f2ec3137700 1 -- 172.21.15.142:0/3163419412 >> 172.21.15.142:0/3163419412 conn(0x7f2ebc0fc650 msgr2=0x7f2ebc0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:53.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.576+0000 7f2ec3137700 1 -- 172.21.15.142:0/3163419412 shutdown_connections 2024-03-29T04:57:53.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.576+0000 7f2ec3137700 1 -- 172.21.15.142:0/3163419412 wait complete. 2024-03-29T04:57:53.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.680+0000 7ff302529700 1 -- 172.21.15.142:0/3339037673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2fc101770 msgr2=0x7ff2fc1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:53.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.680+0000 7ff302529700 1 --2- 172.21.15.142:0/3339037673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2fc101770 0x7ff2fc1057c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009910 tx=0x7ff2f0009c20 comp rx=0 tx=0).stop 2024-03-29T04:57:53.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.680+0000 7ff302529700 1 -- 172.21.15.142:0/3339037673 shutdown_connections 2024-03-29T04:57:53.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.680+0000 7ff302529700 1 --2- 172.21.15.142:0/3339037673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2fc101770 0x7ff2fc1057c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.680+0000 7ff302529700 1 --2- 172.21.15.142:0/3339037673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc100dc0 0x7ff2fc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.680+0000 7ff302529700 1 -- 172.21.15.142:0/3339037673 >> 172.21.15.142:0/3339037673 conn(0x7ff2fc0fc650 msgr2=0x7ff2fc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:53.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.681+0000 7ff302529700 1 -- 172.21.15.142:0/3339037673 shutdown_connections 2024-03-29T04:57:53.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.681+0000 7ff302529700 1 -- 172.21.15.142:0/3339037673 wait complete. 2024-03-29T04:57:53.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.681+0000 7ff302529700 1 Processor -- start 2024-03-29T04:57:53.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.681+0000 7ff302529700 1 -- start start 2024-03-29T04:57:53.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.681+0000 7ff302529700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2fc100dc0 0x7ff2fc19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:53.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.681+0000 7ff302529700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc101770 0x7ff2fc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:53.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.681+0000 7ff302529700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc19d240 con 0x7ff2fc100dc0 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.681+0000 7ff302529700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc1966f0 con 0x7ff2fc101770 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff2fbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2fc100dc0 0x7ff2fc19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff2fbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2fc100dc0 0x7ff2fc19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:36732/0 (socket says 172.21.15.142:36732) 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff2fbfff700 1 -- 172.21.15.142:0/3615667593 learned_addr learned my addr 172.21.15.142:0/3615667593 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff2fb7fe700 1 --2- 172.21.15.142:0/3615667593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc101770 0x7ff2fc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff2fbfff700 1 -- 172.21.15.142:0/3615667593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc101770 msgr2=0x7ff2fc19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff2fbfff700 1 --2- 172.21.15.142:0/3615667593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc101770 0x7ff2fc19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff2fbfff700 1 -- 172.21.15.142:0/3615667593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f00095f0 con 0x7ff2fc100dc0 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff2fb7fe700 1 --2- 172.21.15.142:0/3615667593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc101770 0x7ff2fc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff2fbfff700 1 --2- 172.21.15.142:0/3615667593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2fc100dc0 0x7ff2fc19c670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00b4b0 tx=0x7ff2ec00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff2f97fa700 1 -- 172.21.15.142:0/3615667593 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2ec011800 con 0x7ff2fc100dc0 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff2f97fa700 1 -- 172.21.15.142:0/3615667593 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2ec011e20 con 0x7ff2fc100dc0 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff2f97fa700 1 -- 172.21.15.142:0/3615667593 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2ec00f2e0 con 0x7ff2fc100dc0 2024-03-29T04:57:53.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff302529700 1 -- 172.21.15.142:0/3615667593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc1969d0 con 0x7ff2fc100dc0 2024-03-29T04:57:53.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.682+0000 7ff302529700 1 -- 172.21.15.142:0/3615667593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc196f20 con 0x7ff2fc100dc0 2024-03-29T04:57:53.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.683+0000 7ff302529700 1 -- 172.21.15.142:0/3615667593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc1090f0 con 0x7ff2fc100dc0 2024-03-29T04:57:53.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.684+0000 7ff2f97fa700 1 -- 172.21.15.142:0/3615667593 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff2ec010380 con 0x7ff2fc100dc0 2024-03-29T04:57:53.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.684+0000 7ff2f97fa700 1 --2- 172.21.15.142:0/3615667593 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2e406dd60 0x7ff2e4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:53.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.684+0000 7ff2fb7fe700 1 --2- 172.21.15.142:0/3615667593 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2e406dd60 0x7ff2e4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:53.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.684+0000 7ff2f97fa700 1 -- 172.21.15.142:0/3615667593 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff2ec08d890 con 0x7ff2fc100dc0 2024-03-29T04:57:53.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.685+0000 7ff2fb7fe700 1 --2- 172.21.15.142:0/3615667593 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2e406dd60 0x7ff2e4070220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc197b60 tx=0x7ff2f001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:53.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.689+0000 7ff2f97fa700 1 -- 172.21.15.142:0/3615667593 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff2ec05a7e0 con 0x7ff2fc100dc0 2024-03-29T04:57:53.698 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:53 smithi152 ceph-mon[88186]: pgmap v287: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:53.698 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:53 smithi152 ceph-mon[88186]: from='client.44421 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:53.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.932+0000 7ff302529700 1 -- 172.21.15.142:0/3615667593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff2fc1971a0 con 0x7ff2fc100dc0 2024-03-29T04:57:53.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.933+0000 7ff2f97fa700 1 -- 172.21.15.142:0/3615667593 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7ff2ec05a370 con 0x7ff2fc100dc0 2024-03-29T04:57:53.940 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T04:57:53.940 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:57:53.940 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:57:53.941 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:57:53.942 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:57:53.943 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:57:53.943 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:57:53.943 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:57:53.943 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:57:53.943 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:57:53.943 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T04:57:53.943 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:57:53.943 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:57:53.943 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:57:53.943 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:57:53.943 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:57:53.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.935+0000 7ff302529700 1 -- 172.21.15.142:0/3615667593 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2e406dd60 msgr2=0x7ff2e4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:53.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.935+0000 7ff302529700 1 --2- 172.21.15.142:0/3615667593 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2e406dd60 0x7ff2e4070220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc197b60 tx=0x7ff2f001a040 comp rx=0 tx=0).stop 2024-03-29T04:57:53.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.935+0000 7ff302529700 1 -- 172.21.15.142:0/3615667593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2fc100dc0 msgr2=0x7ff2fc19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:53.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.935+0000 7ff302529700 1 --2- 172.21.15.142:0/3615667593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2fc100dc0 0x7ff2fc19c670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00b4b0 tx=0x7ff2ec00b870 comp rx=0 tx=0).stop 2024-03-29T04:57:53.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.935+0000 7ff302529700 1 -- 172.21.15.142:0/3615667593 shutdown_connections 2024-03-29T04:57:53.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.935+0000 7ff302529700 1 --2- 172.21.15.142:0/3615667593 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2e406dd60 0x7ff2e4070220 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.935+0000 7ff302529700 1 --2- 172.21.15.142:0/3615667593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2fc100dc0 0x7ff2fc19c670 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.935+0000 7ff302529700 1 --2- 172.21.15.142:0/3615667593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2fc101770 0x7ff2fc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:53.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.935+0000 7ff302529700 1 -- 172.21.15.142:0/3615667593 >> 172.21.15.142:0/3615667593 conn(0x7ff2fc0fc650 msgr2=0x7ff2fc0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:53.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.935+0000 7ff302529700 1 -- 172.21.15.142:0/3615667593 shutdown_connections 2024-03-29T04:57:53.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:53.935+0000 7ff302529700 1 -- 172.21.15.142:0/3615667593 wait complete. 2024-03-29T04:57:53.945 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T04:57:54.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.040+0000 7f71c7b49700 1 -- 172.21.15.142:0/1998710228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c01017a0 msgr2=0x7f71c01057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:54.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.040+0000 7f71c7b49700 1 --2- 172.21.15.142:0/1998710228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c01017a0 0x7f71c01057f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009910 tx=0x7f71b0009c20 comp rx=0 tx=0).stop 2024-03-29T04:57:54.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.040+0000 7f71c7b49700 1 -- 172.21.15.142:0/1998710228 shutdown_connections 2024-03-29T04:57:54.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.040+0000 7f71c7b49700 1 --2- 172.21.15.142:0/1998710228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c01017a0 0x7f71c01057f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.040+0000 7f71c7b49700 1 --2- 172.21.15.142:0/1998710228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71c0100df0 0x7f71c01011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.040+0000 7f71c7b49700 1 -- 172.21.15.142:0/1998710228 >> 172.21.15.142:0/1998710228 conn(0x7f71c00fc6a0 msgr2=0x7f71c00feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.040+0000 7f71c7b49700 1 -- 172.21.15.142:0/1998710228 shutdown_connections 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.040+0000 7f71c7b49700 1 -- 172.21.15.142:0/1998710228 wait complete. 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c7b49700 1 Processor -- start 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c7b49700 1 -- start start 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c7b49700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c0100df0 0x7f71c019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c7b49700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71c01017a0 0x7f71c019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c7b49700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c019d260 con 0x7f71c0100df0 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c7b49700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c0196710 con 0x7f71c01017a0 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c58e5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c0100df0 0x7f71c019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c58e5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c0100df0 0x7f71c019c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:36748/0 (socket says 172.21.15.142:36748) 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c58e5700 1 -- 172.21.15.142:0/3153297334 learned_addr learned my addr 172.21.15.142:0/3153297334 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c50e4700 1 --2- 172.21.15.142:0/3153297334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71c01017a0 0x7f71c019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c58e5700 1 -- 172.21.15.142:0/3153297334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71c01017a0 msgr2=0x7f71c019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:54.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c58e5700 1 --2- 172.21.15.142:0/3153297334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71c01017a0 0x7f71c019cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.041+0000 7f71c58e5700 1 -- 172.21.15.142:0/3153297334 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b00095f0 con 0x7f71c0100df0 2024-03-29T04:57:54.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.042+0000 7f71c58e5700 1 --2- 172.21.15.142:0/3153297334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c0100df0 0x7f71c019c690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b4b0 tx=0x7f71bc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:54.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.042+0000 7f71b6ffd700 1 -- 172.21.15.142:0/3153297334 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71bc011800 con 0x7f71c0100df0 2024-03-29T04:57:54.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.042+0000 7f71c7b49700 1 -- 172.21.15.142:0/3153297334 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c01969f0 con 0x7f71c0100df0 2024-03-29T04:57:54.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.042+0000 7f71b6ffd700 1 -- 172.21.15.142:0/3153297334 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71bc011e20 con 0x7f71c0100df0 2024-03-29T04:57:54.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.042+0000 7f71b6ffd700 1 -- 172.21.15.142:0/3153297334 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71bc00f2e0 con 0x7f71c0100df0 2024-03-29T04:57:54.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.042+0000 7f71c7b49700 1 -- 172.21.15.142:0/3153297334 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c0196f40 con 0x7f71c0100df0 2024-03-29T04:57:54.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.043+0000 7f71c7b49700 1 -- 172.21.15.142:0/3153297334 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c0109120 con 0x7f71c0100df0 2024-03-29T04:57:54.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.043+0000 7f71b6ffd700 1 -- 172.21.15.142:0/3153297334 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f71bc010380 con 0x7f71c0100df0 2024-03-29T04:57:54.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.044+0000 7f71b6ffd700 1 --2- 172.21.15.142:0/3153297334 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f71ac06dd60 0x7f71ac070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:54.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.044+0000 7f71b6ffd700 1 -- 172.21.15.142:0/3153297334 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f71bc08d810 con 0x7f71c0100df0 2024-03-29T04:57:54.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.044+0000 7f71c50e4700 1 --2- 172.21.15.142:0/3153297334 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f71ac06dd60 0x7f71ac070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:54.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.044+0000 7f71c50e4700 1 --2- 172.21.15.142:0/3153297334 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f71ac06dd60 0x7f71ac070220 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f71c0197b80 tx=0x7f71b001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:54.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.048+0000 7f71b6ffd700 1 -- 172.21.15.142:0/3153297334 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f71bc05a760 con 0x7f71c0100df0 2024-03-29T04:57:54.270 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.263+0000 7f71c7b49700 1 -- 172.21.15.142:0/3153297334 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71c0197220 con 0x7f71ac06dd60 2024-03-29T04:57:54.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.264+0000 7f71b6ffd700 1 -- 172.21.15.142:0/3153297334 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f71c0197220 con 0x7f71ac06dd60 2024-03-29T04:57:54.271 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:57:54.271 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:57:54.271 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:57:54.271 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:57:54.271 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:57:54.272 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:57:54.272 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:57:54.272 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T04:57:54.272 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T04:57:54.272 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T04:57:54.272 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:57:54.272 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T04:57:54.272 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T04:57:54.272 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:57:54.272 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:57:54.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.266+0000 7f71c7b49700 1 -- 172.21.15.142:0/3153297334 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f71ac06dd60 msgr2=0x7f71ac070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:54.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.266+0000 7f71c7b49700 1 --2- 172.21.15.142:0/3153297334 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f71ac06dd60 0x7f71ac070220 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f71c0197b80 tx=0x7f71b001a040 comp rx=0 tx=0).stop 2024-03-29T04:57:54.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.266+0000 7f71c7b49700 1 -- 172.21.15.142:0/3153297334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c0100df0 msgr2=0x7f71c019c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:54.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.266+0000 7f71c7b49700 1 --2- 172.21.15.142:0/3153297334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c0100df0 0x7f71c019c690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b4b0 tx=0x7f71bc00b7c0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.266+0000 7f71c7b49700 1 -- 172.21.15.142:0/3153297334 shutdown_connections 2024-03-29T04:57:54.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.266+0000 7f71c7b49700 1 --2- 172.21.15.142:0/3153297334 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f71ac06dd60 0x7f71ac070220 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.266+0000 7f71c7b49700 1 --2- 172.21.15.142:0/3153297334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c0100df0 0x7f71c019c690 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.266+0000 7f71c7b49700 1 --2- 172.21.15.142:0/3153297334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71c01017a0 0x7f71c019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.266+0000 7f71c7b49700 1 -- 172.21.15.142:0/3153297334 >> 172.21.15.142:0/3153297334 conn(0x7f71c00fc6a0 msgr2=0x7f71c00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:54.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.266+0000 7f71c7b49700 1 -- 172.21.15.142:0/3153297334 shutdown_connections 2024-03-29T04:57:54.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.266+0000 7f71c7b49700 1 -- 172.21.15.142:0/3153297334 wait complete. 2024-03-29T04:57:54.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.371+0000 7ff27e677700 1 -- 172.21.15.142:0/2943680527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2780ff500 msgr2=0x7ff2780ff8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:54.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.371+0000 7ff27e677700 1 --2- 172.21.15.142:0/2943680527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2780ff500 0x7ff2780ff8e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff268009910 tx=0x7ff268009c20 comp rx=0 tx=0).stop 2024-03-29T04:57:54.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.371+0000 7ff27e677700 1 -- 172.21.15.142:0/2943680527 shutdown_connections 2024-03-29T04:57:54.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.371+0000 7ff27e677700 1 --2- 172.21.15.142:0/2943680527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2780ffeb0 0x7ff278105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.379 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.371+0000 7ff27e677700 1 --2- 172.21.15.142:0/2943680527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2780ff500 0x7ff2780ff8e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.379 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.371+0000 7ff27e677700 1 -- 172.21.15.142:0/2943680527 >> 172.21.15.142:0/2943680527 conn(0x7ff278074d80 msgr2=0x7ff278075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:54.379 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.371+0000 7ff27e677700 1 -- 172.21.15.142:0/2943680527 shutdown_connections 2024-03-29T04:57:54.379 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.371+0000 7ff27e677700 1 -- 172.21.15.142:0/2943680527 wait complete. 2024-03-29T04:57:54.379 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.371+0000 7ff27e677700 1 Processor -- start 2024-03-29T04:57:54.379 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.371+0000 7ff27e677700 1 -- start start 2024-03-29T04:57:54.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff27e677700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2780ff500 0x7ff278100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:54.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff27e677700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2780ffeb0 0x7ff2781009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:54.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff27e677700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278100f00 con 0x7ff2780ffeb0 2024-03-29T04:57:54.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff27e677700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278101040 con 0x7ff2780ff500 2024-03-29T04:57:54.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff2777fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2780ffeb0 0x7ff2781009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:54.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff2777fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2780ffeb0 0x7ff2781009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:36758/0 (socket says 172.21.15.142:36758) 2024-03-29T04:57:54.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff2777fe700 1 -- 172.21.15.142:0/3193366140 learned_addr learned my addr 172.21.15.142:0/3193366140 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:57:54.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff277fff700 1 --2- 172.21.15.142:0/3193366140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2780ff500 0x7ff278100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:54.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff2777fe700 1 -- 172.21.15.142:0/3193366140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2780ff500 msgr2=0x7ff278100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:54.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff2777fe700 1 --2- 172.21.15.142:0/3193366140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2780ff500 0x7ff278100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff2777fe700 1 -- 172.21.15.142:0/3193366140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2680095f0 con 0x7ff2780ffeb0 2024-03-29T04:57:54.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff2777fe700 1 --2- 172.21.15.142:0/3193366140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2780ffeb0 0x7ff2781009c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00d900 tx=0x7ff26c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:54.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff2757fa700 1 -- 172.21.15.142:0/3193366140 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff26c009900 con 0x7ff2780ffeb0 2024-03-29T04:57:54.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff2757fa700 1 -- 172.21.15.142:0/3193366140 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff26c010460 con 0x7ff2780ffeb0 2024-03-29T04:57:54.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff27e677700 1 -- 172.21.15.142:0/3193366140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2781a2a10 con 0x7ff2780ffeb0 2024-03-29T04:57:54.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff2757fa700 1 -- 172.21.15.142:0/3193366140 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff26c00f5f0 con 0x7ff2780ffeb0 2024-03-29T04:57:54.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.372+0000 7ff27e677700 1 -- 172.21.15.142:0/3193366140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2781a2e20 con 0x7ff2780ffeb0 2024-03-29T04:57:54.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.373+0000 7ff27e677700 1 -- 172.21.15.142:0/3193366140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff278109190 con 0x7ff2780ffeb0 2024-03-29T04:57:54.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.374+0000 7ff2757fa700 1 -- 172.21.15.142:0/3193366140 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff26c0105d0 con 0x7ff2780ffeb0 2024-03-29T04:57:54.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.374+0000 7ff2757fa700 1 --2- 172.21.15.142:0/3193366140 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff26006dd60 0x7ff260070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:57:54.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.374+0000 7ff2757fa700 1 -- 172.21.15.142:0/3193366140 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff26c08d870 con 0x7ff2780ffeb0 2024-03-29T04:57:54.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.374+0000 7ff277fff700 1 --2- 172.21.15.142:0/3193366140 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff26006dd60 0x7ff260070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:57:54.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.375+0000 7ff277fff700 1 --2- 172.21.15.142:0/3193366140 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff26006dd60 0x7ff260070220 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff268006010 tx=0x7ff26801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:57:54.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.379+0000 7ff2757fa700 1 -- 172.21.15.142:0/3193366140 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff26c056c90 con 0x7ff2780ffeb0 2024-03-29T04:57:54.626 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:54 smithi152 ceph-mon[88186]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:54.627 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:54 smithi152 ceph-mon[88186]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:54.627 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:54 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3163419412' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:57:54.627 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:54 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3615667593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:57:54.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:54 smithi142 ceph-mon[112475]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:54.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:54 smithi142 ceph-mon[112475]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:54.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:54 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3163419412' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:57:54.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:54 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3615667593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:57:54.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.656+0000 7ff27e677700 1 -- 172.21.15.142:0/3193366140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff27804ea90 con 0x7ff2780ffeb0 2024-03-29T04:57:54.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.656+0000 7ff2757fa700 1 -- 172.21.15.142:0/3193366140 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff26c05a350 con 0x7ff2780ffeb0 2024-03-29T04:57:54.663 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T04:57:54.663 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T04:57:54.664 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T04:57:54.664 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T04:57:54.664 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T04:57:54.665 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.658+0000 7ff27e677700 1 -- 172.21.15.142:0/3193366140 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff26006dd60 msgr2=0x7ff260070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:54.666 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.658+0000 7ff27e677700 1 --2- 172.21.15.142:0/3193366140 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff26006dd60 0x7ff260070220 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff268006010 tx=0x7ff26801a040 comp rx=0 tx=0).stop 2024-03-29T04:57:54.666 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.658+0000 7ff27e677700 1 -- 172.21.15.142:0/3193366140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2780ffeb0 msgr2=0x7ff2781009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:57:54.666 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.658+0000 7ff27e677700 1 --2- 172.21.15.142:0/3193366140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2780ffeb0 0x7ff2781009c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00d900 tx=0x7ff26c00dc10 comp rx=0 tx=0).stop 2024-03-29T04:57:54.666 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.658+0000 7ff27e677700 1 -- 172.21.15.142:0/3193366140 shutdown_connections 2024-03-29T04:57:54.666 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.658+0000 7ff27e677700 1 --2- 172.21.15.142:0/3193366140 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff26006dd60 0x7ff260070220 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.666 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.658+0000 7ff27e677700 1 --2- 172.21.15.142:0/3193366140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2780ff500 0x7ff278100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.666 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.658+0000 7ff27e677700 1 --2- 172.21.15.142:0/3193366140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2780ffeb0 0x7ff2781009c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:57:54.666 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.658+0000 7ff27e677700 1 -- 172.21.15.142:0/3193366140 >> 172.21.15.142:0/3193366140 conn(0x7ff278074d80 msgr2=0x7ff2780fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:57:54.666 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.658+0000 7ff27e677700 1 -- 172.21.15.142:0/3193366140 shutdown_connections 2024-03-29T04:57:54.666 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:57:54.658+0000 7ff27e677700 1 -- 172.21.15.142:0/3193366140 wait complete. 2024-03-29T04:57:55.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:55 smithi142 ceph-mon[112475]: pgmap v288: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:55.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:55 smithi142 ceph-mon[112475]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:55.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:55 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3193366140' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:57:55.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:55 smithi152 ceph-mon[88186]: pgmap v288: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:55.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:55 smithi152 ceph-mon[88186]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:57:55.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:55 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3193366140' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:57:57.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:57 smithi142 ceph-mon[112475]: pgmap v289: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:57.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:57 smithi152 ceph-mon[88186]: pgmap v289: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:59.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:57:59 smithi142 ceph-mon[112475]: pgmap v290: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:57:59.681 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:57:59 smithi152 ceph-mon[88186]: pgmap v290: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:00.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:58:00.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:58:01.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:01 smithi142 ceph-mon[112475]: pgmap v291: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:01.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:01 smithi152 ceph-mon[88186]: pgmap v291: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:03.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:03 smithi142 ceph-mon[112475]: pgmap v292: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:03.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:03 smithi152 ceph-mon[88186]: pgmap v292: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:05.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:05 smithi142 ceph-mon[112475]: pgmap v293: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:05.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:05 smithi152 ceph-mon[88186]: pgmap v293: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:07.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:07 smithi142 ceph-mon[112475]: pgmap v294: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:07.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:07 smithi152 ceph-mon[88186]: pgmap v294: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:09.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:09 smithi142 ceph-mon[112475]: pgmap v295: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:09.788 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:09 smithi152 ceph-mon[88186]: pgmap v295: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:11.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:11 smithi142 ceph-mon[112475]: pgmap v296: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:11.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:11 smithi152 ceph-mon[88186]: pgmap v296: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:13.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:13 smithi142 ceph-mon[112475]: pgmap v297: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:13.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:13 smithi152 ceph-mon[88186]: pgmap v297: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:15.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:15 smithi142 ceph-mon[112475]: pgmap v298: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:15.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:58:15.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:15 smithi152 ceph-mon[88186]: pgmap v298: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:15.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:58:17.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:17 smithi142 ceph-mon[112475]: pgmap v299: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:17.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:17 smithi152 ceph-mon[88186]: pgmap v299: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:19.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:19 smithi142 ceph-mon[112475]: pgmap v300: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:19.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:19 smithi152 ceph-mon[88186]: pgmap v300: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:21.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:21 smithi142 ceph-mon[112475]: pgmap v301: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:21.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:21 smithi152 ceph-mon[88186]: pgmap v301: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:23.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:23 smithi142 ceph-mon[112475]: pgmap v302: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:23.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:23 smithi152 ceph-mon[88186]: pgmap v302: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:24.784 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.776+0000 7f358f395700 1 -- 172.21.15.142:0/3574773037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3588105a40 msgr2=0x7f3588105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:24.784 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.776+0000 7f358f395700 1 --2- 172.21.15.142:0/3574773037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3588105a40 0x7f3588105e20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3584009910 tx=0x7f3584009c20 comp rx=0 tx=0).stop 2024-03-29T04:58:24.784 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.776+0000 7f358f395700 1 -- 172.21.15.142:0/3574773037 shutdown_connections 2024-03-29T04:58:24.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.776+0000 7f358f395700 1 --2- 172.21.15.142:0/3574773037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35880ffbd0 0x7f3588100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:24.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.776+0000 7f358f395700 1 --2- 172.21.15.142:0/3574773037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3588105a40 0x7f3588105e20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:24.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.776+0000 7f358f395700 1 -- 172.21.15.142:0/3574773037 >> 172.21.15.142:0/3574773037 conn(0x7f35880755a0 msgr2=0x7f35880759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:24.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.776+0000 7f358f395700 1 -- 172.21.15.142:0/3574773037 shutdown_connections 2024-03-29T04:58:24.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.776+0000 7f358f395700 1 -- 172.21.15.142:0/3574773037 wait complete. 2024-03-29T04:58:24.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.776+0000 7f358f395700 1 Processor -- start 2024-03-29T04:58:24.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358f395700 1 -- start start 2024-03-29T04:58:24.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358f395700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35880ffbd0 0x7f3588198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:24.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358f395700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3588105a40 0x7f3588198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:24.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358f395700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3588199530 con 0x7f3588105a40 2024-03-29T04:58:24.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358f395700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358819d340 con 0x7f35880ffbd0 2024-03-29T04:58:24.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358c930700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3588105a40 0x7f3588198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:24.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358c930700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3588105a40 0x7f3588198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40184/0 (socket says 172.21.15.142:40184) 2024-03-29T04:58:24.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358c930700 1 -- 172.21.15.142:0/658185757 learned_addr learned my addr 172.21.15.142:0/658185757 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:24.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358d131700 1 --2- 172.21.15.142:0/658185757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35880ffbd0 0x7f3588198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:24.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358c930700 1 -- 172.21.15.142:0/658185757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35880ffbd0 msgr2=0x7f3588198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:24.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358c930700 1 --2- 172.21.15.142:0/658185757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35880ffbd0 0x7f3588198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:24.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358c930700 1 -- 172.21.15.142:0/658185757 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35840095f0 con 0x7f3588105a40 2024-03-29T04:58:24.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.777+0000 7f358c930700 1 --2- 172.21.15.142:0/658185757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3588105a40 0x7f3588198ea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f357800c8a0 tx=0x7f357800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:24.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.778+0000 7f357e7fc700 1 -- 172.21.15.142:0/658185757 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f357800cea0 con 0x7f3588105a40 2024-03-29T04:58:24.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.778+0000 7f358f395700 1 -- 172.21.15.142:0/658185757 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358819d620 con 0x7f3588105a40 2024-03-29T04:58:24.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.778+0000 7f357e7fc700 1 -- 172.21.15.142:0/658185757 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3578004830 con 0x7f3588105a40 2024-03-29T04:58:24.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.778+0000 7f357e7fc700 1 -- 172.21.15.142:0/658185757 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35780055b0 con 0x7f3588105a40 2024-03-29T04:58:24.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.778+0000 7f358f395700 1 -- 172.21.15.142:0/658185757 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358819db70 con 0x7f3588105a40 2024-03-29T04:58:24.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.779+0000 7f358f395700 1 -- 172.21.15.142:0/658185757 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358804ea90 con 0x7f3588105a40 2024-03-29T04:58:24.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.779+0000 7f357e7fc700 1 -- 172.21.15.142:0/658185757 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f35780049a0 con 0x7f3588105a40 2024-03-29T04:58:24.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.780+0000 7f357e7fc700 1 --2- 172.21.15.142:0/658185757 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f357406dd60 0x7f3574070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:24.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.780+0000 7f357e7fc700 1 -- 172.21.15.142:0/658185757 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f357808d980 con 0x7f3588105a40 2024-03-29T04:58:24.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.780+0000 7f358d131700 1 --2- 172.21.15.142:0/658185757 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f357406dd60 0x7f3574070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:24.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.780+0000 7f358d131700 1 --2- 172.21.15.142:0/658185757 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f357406dd60 0x7f3574070220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3584006010 tx=0x7f358401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:24.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.784+0000 7f357e7fc700 1 -- 172.21.15.142:0/658185757 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f357805a8d0 con 0x7f3588105a40 2024-03-29T04:58:25.003 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.995+0000 7f358f395700 1 -- 172.21.15.142:0/658185757 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f358819ded0 con 0x7f357406dd60 2024-03-29T04:58:25.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.997+0000 7f357e7fc700 1 -- 172.21.15.142:0/658185757 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f358819ded0 con 0x7f357406dd60 2024-03-29T04:58:25.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.999+0000 7f358f395700 1 -- 172.21.15.142:0/658185757 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f357406dd60 msgr2=0x7f3574070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:25.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.999+0000 7f358f395700 1 --2- 172.21.15.142:0/658185757 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f357406dd60 0x7f3574070220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3584006010 tx=0x7f358401a040 comp rx=0 tx=0).stop 2024-03-29T04:58:25.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.999+0000 7f358f395700 1 -- 172.21.15.142:0/658185757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3588105a40 msgr2=0x7f3588198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:25.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.999+0000 7f358f395700 1 --2- 172.21.15.142:0/658185757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3588105a40 0x7f3588198ea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f357800c8a0 tx=0x7f357800cc60 comp rx=0 tx=0).stop 2024-03-29T04:58:25.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.999+0000 7f358f395700 1 -- 172.21.15.142:0/658185757 shutdown_connections 2024-03-29T04:58:25.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.999+0000 7f358f395700 1 --2- 172.21.15.142:0/658185757 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f357406dd60 0x7f3574070220 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.999+0000 7f358f395700 1 --2- 172.21.15.142:0/658185757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35880ffbd0 0x7f3588198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.999+0000 7f358f395700 1 --2- 172.21.15.142:0/658185757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3588105a40 0x7f3588198ea0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.999+0000 7f358f395700 1 -- 172.21.15.142:0/658185757 >> 172.21.15.142:0/658185757 conn(0x7f35880755a0 msgr2=0x7f35880fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:25.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.999+0000 7f358f395700 1 -- 172.21.15.142:0/658185757 shutdown_connections 2024-03-29T04:58:25.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:24.999+0000 7f358f395700 1 -- 172.21.15.142:0/658185757 wait complete. 2024-03-29T04:58:25.017 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:58:25.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.104+0000 7f36fd588700 1 -- 172.21.15.142:0/1046690684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f8102670 msgr2=0x7f36f810ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:25.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.104+0000 7f36fd588700 1 --2- 172.21.15.142:0/1046690684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f8102670 0x7f36f810ab60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009940 tx=0x7f36ec009c50 comp rx=0 tx=0).stop 2024-03-29T04:58:25.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.104+0000 7f36fd588700 1 -- 172.21.15.142:0/1046690684 shutdown_connections 2024-03-29T04:58:25.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.104+0000 7f36fd588700 1 --2- 172.21.15.142:0/1046690684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f8102670 0x7f36f810ab60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.104+0000 7f36fd588700 1 --2- 172.21.15.142:0/1046690684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36f8101d50 0x7f36f8102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.104+0000 7f36fd588700 1 -- 172.21.15.142:0/1046690684 >> 172.21.15.142:0/1046690684 conn(0x7f36f80fb520 msgr2=0x7f36f80fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:25.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.104+0000 7f36fd588700 1 -- 172.21.15.142:0/1046690684 shutdown_connections 2024-03-29T04:58:25.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.104+0000 7f36fd588700 1 -- 172.21.15.142:0/1046690684 wait complete. 2024-03-29T04:58:25.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.105+0000 7f36fd588700 1 Processor -- start 2024-03-29T04:58:25.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.105+0000 7f36fd588700 1 -- start start 2024-03-29T04:58:25.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.105+0000 7f36fd588700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f8101d50 0x7f36f819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.105+0000 7f36fd588700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36f8102670 0x7f36f819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.105+0000 7f36fd588700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f819d200 con 0x7f36f8101d50 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.105+0000 7f36fd588700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f81966b0 con 0x7f36f8102670 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.105+0000 7f36f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f8101d50 0x7f36f819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.106+0000 7f36f6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f8101d50 0x7f36f819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40188/0 (socket says 172.21.15.142:40188) 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.106+0000 7f36f6ffd700 1 -- 172.21.15.142:0/23942450 learned_addr learned my addr 172.21.15.142:0/23942450 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.106+0000 7f36f67fc700 1 --2- 172.21.15.142:0/23942450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36f8102670 0x7f36f819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.106+0000 7f36f6ffd700 1 -- 172.21.15.142:0/23942450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36f8102670 msgr2=0x7f36f819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.106+0000 7f36f6ffd700 1 --2- 172.21.15.142:0/23942450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36f8102670 0x7f36f819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.106+0000 7f36f6ffd700 1 -- 172.21.15.142:0/23942450 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec0095f0 con 0x7f36f8101d50 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.106+0000 7f36f6ffd700 1 --2- 172.21.15.142:0/23942450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f8101d50 0x7f36f819c630 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b4b0 tx=0x7f36e800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.106+0000 7f36dffff700 1 -- 172.21.15.142:0/23942450 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36e8011800 con 0x7f36f8101d50 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.106+0000 7f36dffff700 1 -- 172.21.15.142:0/23942450 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36e8011e20 con 0x7f36f8101d50 2024-03-29T04:58:25.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.106+0000 7f36fd588700 1 -- 172.21.15.142:0/23942450 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f8196990 con 0x7f36f8101d50 2024-03-29T04:58:25.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.106+0000 7f36dffff700 1 -- 172.21.15.142:0/23942450 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36e800f2e0 con 0x7f36f8101d50 2024-03-29T04:58:25.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.106+0000 7f36fd588700 1 -- 172.21.15.142:0/23942450 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f8196eb0 con 0x7f36f8101d50 2024-03-29T04:58:25.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.107+0000 7f36fd588700 1 -- 172.21.15.142:0/23942450 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f8108260 con 0x7f36f8101d50 2024-03-29T04:58:25.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.108+0000 7f36dffff700 1 -- 172.21.15.142:0/23942450 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f36e8010380 con 0x7f36f8101d50 2024-03-29T04:58:25.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.108+0000 7f36dffff700 1 --2- 172.21.15.142:0/23942450 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f36e006dd60 0x7f36e0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:25.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.108+0000 7f36dffff700 1 -- 172.21.15.142:0/23942450 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f36e808d720 con 0x7f36f8101d50 2024-03-29T04:58:25.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.108+0000 7f36f67fc700 1 --2- 172.21.15.142:0/23942450 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f36e006dd60 0x7f36e0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:25.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.109+0000 7f36f67fc700 1 --2- 172.21.15.142:0/23942450 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f36e006dd60 0x7f36e0070220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009910 tx=0x7f36ec00b790 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:25.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.113+0000 7f36dffff700 1 -- 172.21.15.142:0/23942450 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f36e805a6f0 con 0x7f36f8101d50 2024-03-29T04:58:25.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.324+0000 7f36fd588700 1 -- 172.21.15.142:0/23942450 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36f8197180 con 0x7f36e006dd60 2024-03-29T04:58:25.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.325+0000 7f36dffff700 1 -- 172.21.15.142:0/23942450 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f36f8197180 con 0x7f36e006dd60 2024-03-29T04:58:25.334 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.327+0000 7f36fd588700 1 -- 172.21.15.142:0/23942450 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f36e006dd60 msgr2=0x7f36e0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:25.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.327+0000 7f36fd588700 1 --2- 172.21.15.142:0/23942450 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f36e006dd60 0x7f36e0070220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009910 tx=0x7f36ec00b790 comp rx=0 tx=0).stop 2024-03-29T04:58:25.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.327+0000 7f36fd588700 1 -- 172.21.15.142:0/23942450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f8101d50 msgr2=0x7f36f819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:25.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.327+0000 7f36fd588700 1 --2- 172.21.15.142:0/23942450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f8101d50 0x7f36f819c630 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b4b0 tx=0x7f36e800b7c0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.327+0000 7f36fd588700 1 -- 172.21.15.142:0/23942450 shutdown_connections 2024-03-29T04:58:25.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.327+0000 7f36fd588700 1 --2- 172.21.15.142:0/23942450 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f36e006dd60 0x7f36e0070220 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.327+0000 7f36fd588700 1 --2- 172.21.15.142:0/23942450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f8101d50 0x7f36f819c630 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.327+0000 7f36fd588700 1 --2- 172.21.15.142:0/23942450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36f8102670 0x7f36f819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.327+0000 7f36fd588700 1 -- 172.21.15.142:0/23942450 >> 172.21.15.142:0/23942450 conn(0x7f36f80fb520 msgr2=0x7f36f81053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:25.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.327+0000 7f36fd588700 1 -- 172.21.15.142:0/23942450 shutdown_connections 2024-03-29T04:58:25.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.327+0000 7f36fd588700 1 -- 172.21.15.142:0/23942450 wait complete. 2024-03-29T04:58:25.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:25 smithi142 ceph-mon[112475]: pgmap v303: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:25.441 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.433+0000 7fd0cfe70700 1 -- 172.21.15.142:0/542155142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0c80ffa40 msgr2=0x7fd0c80ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:25.441 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.433+0000 7fd0cfe70700 1 --2- 172.21.15.142:0/542155142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0c80ffa40 0x7fd0c80ffec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8009910 tx=0x7fd0b8009c20 comp rx=0 tx=0).stop 2024-03-29T04:58:25.441 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.433+0000 7fd0cfe70700 1 -- 172.21.15.142:0/542155142 shutdown_connections 2024-03-29T04:58:25.441 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.433+0000 7fd0cfe70700 1 --2- 172.21.15.142:0/542155142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0c80ffa40 0x7fd0c80ffec0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.441 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.433+0000 7fd0cfe70700 1 --2- 172.21.15.142:0/542155142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0c8105a40 0x7fd0c8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.441 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.433+0000 7fd0cfe70700 1 -- 172.21.15.142:0/542155142 >> 172.21.15.142:0/542155142 conn(0x7fd0c80fb570 msgr2=0x7fd0c80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:25.441 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.433+0000 7fd0cfe70700 1 -- 172.21.15.142:0/542155142 shutdown_connections 2024-03-29T04:58:25.441 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.433+0000 7fd0cfe70700 1 -- 172.21.15.142:0/542155142 wait complete. 2024-03-29T04:58:25.441 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cfe70700 1 Processor -- start 2024-03-29T04:58:25.442 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cfe70700 1 -- start start 2024-03-29T04:58:25.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cfe70700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0c80ffa40 0x7fd0c8195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:25.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cfe70700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0c8105a40 0x7fd0c81958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:25.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cfe70700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c8195e00 con 0x7fd0c80ffa40 2024-03-29T04:58:25.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cfe70700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c8195f70 con 0x7fd0c8105a40 2024-03-29T04:58:25.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cdc0c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0c80ffa40 0x7fd0c8195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cdc0c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0c80ffa40 0x7fd0c8195380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40212/0 (socket says 172.21.15.142:40212) 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cdc0c700 1 -- 172.21.15.142:0/1985284094 learned_addr learned my addr 172.21.15.142:0/1985284094 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cdc0c700 1 -- 172.21.15.142:0/1985284094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0c8105a40 msgr2=0x7fd0c81958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cdc0c700 1 --2- 172.21.15.142:0/1985284094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0c8105a40 0x7fd0c81958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cdc0c700 1 -- 172.21.15.142:0/1985284094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b80095f0 con 0x7fd0c80ffa40 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cdc0c700 1 --2- 172.21.15.142:0/1985284094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0c80ffa40 0x7fd0c8195380 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00060b0 tx=0x7fd0c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0beffd700 1 -- 172.21.15.142:0/1985284094 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c000dda0 con 0x7fd0c80ffa40 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0cfe70700 1 -- 172.21.15.142:0/1985284094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c81a29f0 con 0x7fd0c80ffa40 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.434+0000 7fd0beffd700 1 -- 172.21.15.142:0/1985284094 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0c000f040 con 0x7fd0c80ffa40 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.435+0000 7fd0beffd700 1 -- 172.21.15.142:0/1985284094 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c0014790 con 0x7fd0c80ffa40 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.435+0000 7fd0cfe70700 1 -- 172.21.15.142:0/1985284094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c81a2d50 con 0x7fd0c80ffa40 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.436+0000 7fd0cfe70700 1 -- 172.21.15.142:0/1985284094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c8107fd0 con 0x7fd0c80ffa40 2024-03-29T04:58:25.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.436+0000 7fd0beffd700 1 -- 172.21.15.142:0/1985284094 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fd0c00148f0 con 0x7fd0c80ffa40 2024-03-29T04:58:25.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.436+0000 7fd0beffd700 1 --2- 172.21.15.142:0/1985284094 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd0b406dd60 0x7fd0b4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:25.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.437+0000 7fd0beffd700 1 -- 172.21.15.142:0/1985284094 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd0c008d630 con 0x7fd0c80ffa40 2024-03-29T04:58:25.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.437+0000 7fd0cd40b700 1 --2- 172.21.15.142:0/1985284094 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd0b406dd60 0x7fd0b4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:25.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.437+0000 7fd0cd40b700 1 --2- 172.21.15.142:0/1985284094 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd0b406dd60 0x7fd0b4070220 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800b5c0 tx=0x7fd0b8005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:25.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.441+0000 7fd0beffd700 1 -- 172.21.15.142:0/1985284094 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd0c005a6b0 con 0x7fd0c80ffa40 2024-03-29T04:58:25.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.645+0000 7fd0cfe70700 1 -- 172.21.15.142:0/1985284094 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0c802cd80 con 0x7fd0b406dd60 2024-03-29T04:58:25.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.656+0000 7fd0beffd700 1 -- 172.21.15.142:0/1985284094 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd0c802cd80 con 0x7fd0b406dd60 2024-03-29T04:58:25.664 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:58:25.664 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (10m) 3m ago 21m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:58:25.664 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (22m) 3m ago 22m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:58:25.664 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (20m) 2m ago 20m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:58:25.664 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (7m) 3m ago 21m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:58:25.664 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (7m) 2m ago 20m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:58:25.664 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (9m) 3m ago 21m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:58:25.664 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (3m) 3m ago 16m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T04:58:25.665 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (3m) 3m ago 16m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T04:58:25.665 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (2m) 2m ago 16m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T04:58:25.665 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (3m) 2m ago 16m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T04:58:25.665 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (11m) 3m ago 23m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:58:25.665 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (11m) 2m ago 20m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:58:25.665 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (8m) 3m ago 23m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:58:25.666 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (8m) 2m ago 20m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:58:25.666 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (10m) 3m ago 21m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:58:25.666 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (10m) 2m ago 20m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:58:25.666 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (6m) 3m ago 19m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:58:25.666 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (6m) 3m ago 19m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:58:25.666 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (5m) 3m ago 19m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:58:25.666 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (5m) 2m ago 18m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T04:58:25.666 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (4m) 2m ago 18m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T04:58:25.667 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (4m) 2m ago 17m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T04:58:25.667 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (10m) 3m ago 21m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:58:25.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.658+0000 7fd0cfe70700 1 -- 172.21.15.142:0/1985284094 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd0b406dd60 msgr2=0x7fd0b4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:25.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.658+0000 7fd0cfe70700 1 --2- 172.21.15.142:0/1985284094 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd0b406dd60 0x7fd0b4070220 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800b5c0 tx=0x7fd0b8005960 comp rx=0 tx=0).stop 2024-03-29T04:58:25.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.658+0000 7fd0cfe70700 1 -- 172.21.15.142:0/1985284094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0c80ffa40 msgr2=0x7fd0c8195380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:25.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.658+0000 7fd0cfe70700 1 --2- 172.21.15.142:0/1985284094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0c80ffa40 0x7fd0c8195380 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00060b0 tx=0x7fd0c000d3b0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.669 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.658+0000 7fd0cfe70700 1 -- 172.21.15.142:0/1985284094 shutdown_connections 2024-03-29T04:58:25.669 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.658+0000 7fd0cfe70700 1 --2- 172.21.15.142:0/1985284094 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd0b406dd60 0x7fd0b4070220 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.669 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.658+0000 7fd0cfe70700 1 --2- 172.21.15.142:0/1985284094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0c80ffa40 0x7fd0c8195380 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.669 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.658+0000 7fd0cfe70700 1 --2- 172.21.15.142:0/1985284094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0c8105a40 0x7fd0c81958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.669 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.658+0000 7fd0cfe70700 1 -- 172.21.15.142:0/1985284094 >> 172.21.15.142:0/1985284094 conn(0x7fd0c80fb570 msgr2=0x7fd0c80fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:25.669 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.658+0000 7fd0cfe70700 1 -- 172.21.15.142:0/1985284094 shutdown_connections 2024-03-29T04:58:25.669 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.658+0000 7fd0cfe70700 1 -- 172.21.15.142:0/1985284094 wait complete. 2024-03-29T04:58:25.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.761+0000 7f8cfa12f700 1 -- 172.21.15.142:0/2133038990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf4100dd0 msgr2=0x7f8cf41011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:25.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.761+0000 7f8cfa12f700 1 --2- 172.21.15.142:0/2133038990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf4100dd0 0x7f8cf41011b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009910 tx=0x7f8cec009c20 comp rx=0 tx=0).stop 2024-03-29T04:58:25.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.761+0000 7f8cfa12f700 1 -- 172.21.15.142:0/2133038990 shutdown_connections 2024-03-29T04:58:25.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.761+0000 7f8cfa12f700 1 --2- 172.21.15.142:0/2133038990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf4101780 0x7f8cf41057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.761+0000 7f8cfa12f700 1 --2- 172.21.15.142:0/2133038990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf4100dd0 0x7f8cf41011b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.761+0000 7f8cfa12f700 1 -- 172.21.15.142:0/2133038990 >> 172.21.15.142:0/2133038990 conn(0x7f8cf40fc640 msgr2=0x7f8cf40fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:25.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.761+0000 7f8cfa12f700 1 -- 172.21.15.142:0/2133038990 shutdown_connections 2024-03-29T04:58:25.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.761+0000 7f8cfa12f700 1 -- 172.21.15.142:0/2133038990 wait complete. 2024-03-29T04:58:25.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.761+0000 7f8cfa12f700 1 Processor -- start 2024-03-29T04:58:25.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cfa12f700 1 -- start start 2024-03-29T04:58:25.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cfa12f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf4100dd0 0x7f8cf4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:25.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cfa12f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf4101780 0x7f8cf40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:25.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cfa12f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf4073b10 con 0x7f8cf4101780 2024-03-29T04:58:25.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cfa12f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf4073c50 con 0x7f8cf4100dd0 2024-03-29T04:58:25.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cf2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf4101780 0x7f8cf40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:25.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cf2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf4101780 0x7f8cf40735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40236/0 (socket says 172.21.15.142:40236) 2024-03-29T04:58:25.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cf2ffd700 1 -- 172.21.15.142:0/851352588 learned_addr learned my addr 172.21.15.142:0/851352588 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:25.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cf37fe700 1 --2- 172.21.15.142:0/851352588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf4100dd0 0x7f8cf4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:25.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cf2ffd700 1 -- 172.21.15.142:0/851352588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf4100dd0 msgr2=0x7f8cf4073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:25.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cf2ffd700 1 --2- 172.21.15.142:0/851352588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf4100dd0 0x7f8cf4073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:25.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cf2ffd700 1 -- 172.21.15.142:0/851352588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec0095f0 con 0x7f8cf4101780 2024-03-29T04:58:25.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.762+0000 7f8cf37fe700 1 --2- 172.21.15.142:0/851352588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf4100dd0 0x7f8cf4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:58:25.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.763+0000 7f8cf2ffd700 1 --2- 172.21.15.142:0/851352588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf4101780 0x7f8cf40735d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800efc0 tx=0x7f8ce800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:25.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.763+0000 7f8cf0ff9700 1 -- 172.21.15.142:0/851352588 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ce800ca50 con 0x7f8cf4101780 2024-03-29T04:58:25.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.763+0000 7f8cf0ff9700 1 -- 172.21.15.142:0/851352588 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ce800f040 con 0x7f8cf4101780 2024-03-29T04:58:25.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.763+0000 7f8cfa12f700 1 -- 172.21.15.142:0/851352588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf4073ed0 con 0x7f8cf4101780 2024-03-29T04:58:25.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.763+0000 7f8cf0ff9700 1 -- 172.21.15.142:0/851352588 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ce80146b0 con 0x7f8cf4101780 2024-03-29T04:58:25.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.763+0000 7f8cfa12f700 1 -- 172.21.15.142:0/851352588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf41a2ca0 con 0x7f8cf4101780 2024-03-29T04:58:25.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.764+0000 7f8cfa12f700 1 -- 172.21.15.142:0/851352588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf4109170 con 0x7f8cf4101780 2024-03-29T04:58:25.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.765+0000 7f8cf0ff9700 1 -- 172.21.15.142:0/851352588 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f8ce800cbb0 con 0x7f8cf4101780 2024-03-29T04:58:25.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.765+0000 7f8cf0ff9700 1 --2- 172.21.15.142:0/851352588 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8cdc06dd60 0x7f8cdc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:25.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.765+0000 7f8cf0ff9700 1 -- 172.21.15.142:0/851352588 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f8ce808e0c0 con 0x7f8cf4101780 2024-03-29T04:58:25.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.765+0000 7f8cf37fe700 1 --2- 172.21.15.142:0/851352588 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8cdc06dd60 0x7f8cdc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:25.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.766+0000 7f8cf37fe700 1 --2- 172.21.15.142:0/851352588 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8cdc06dd60 0x7f8cdc070220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000c00 tx=0x7f8cec009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:25.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:25.770+0000 7f8cf0ff9700 1 -- 172.21.15.142:0/851352588 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8ce80574e0 con 0x7f8cf4101780 2024-03-29T04:58:25.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:25 smithi152 ceph-mon[88186]: pgmap v303: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:26.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.045+0000 7f8cfa12f700 1 -- 172.21.15.142:0/851352588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8cf404ea90 con 0x7f8cf4101780 2024-03-29T04:58:26.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.045+0000 7f8cf0ff9700 1 -- 172.21.15.142:0/851352588 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8ce805aba0 con 0x7f8cf4101780 2024-03-29T04:58:26.053 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:58:26.053 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:58:26.053 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:58:26.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:58:26.054 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:58:26.054 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:58:26.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:58:26.054 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:58:26.054 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T04:58:26.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:58:26.054 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:58:26.054 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T04:58:26.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:58:26.054 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:58:26.055 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T04:58:26.055 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:58:26.055 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:58:26.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.048+0000 7f8cfa12f700 1 -- 172.21.15.142:0/851352588 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8cdc06dd60 msgr2=0x7f8cdc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:26.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.048+0000 7f8cfa12f700 1 --2- 172.21.15.142:0/851352588 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8cdc06dd60 0x7f8cdc070220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000c00 tx=0x7f8cec009f70 comp rx=0 tx=0).stop 2024-03-29T04:58:26.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.048+0000 7f8cfa12f700 1 -- 172.21.15.142:0/851352588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf4101780 msgr2=0x7f8cf40735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:26.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.048+0000 7f8cfa12f700 1 --2- 172.21.15.142:0/851352588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf4101780 0x7f8cf40735d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800efc0 tx=0x7f8ce800c3a0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.048+0000 7f8cfa12f700 1 -- 172.21.15.142:0/851352588 shutdown_connections 2024-03-29T04:58:26.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.048+0000 7f8cfa12f700 1 --2- 172.21.15.142:0/851352588 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8cdc06dd60 0x7f8cdc070220 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.048+0000 7f8cfa12f700 1 --2- 172.21.15.142:0/851352588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf4100dd0 0x7f8cf4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.048+0000 7f8cfa12f700 1 --2- 172.21.15.142:0/851352588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf4101780 0x7f8cf40735d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.048+0000 7f8cfa12f700 1 -- 172.21.15.142:0/851352588 >> 172.21.15.142:0/851352588 conn(0x7f8cf40fc640 msgr2=0x7f8cf40fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:26.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.048+0000 7f8cfa12f700 1 -- 172.21.15.142:0/851352588 shutdown_connections 2024-03-29T04:58:26.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.048+0000 7f8cfa12f700 1 -- 172.21.15.142:0/851352588 wait complete. 2024-03-29T04:58:26.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.151+0000 7f5856e40700 1 -- 172.21.15.142:0/4260737175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5850105a40 msgr2=0x7f5850105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:26.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.151+0000 7f5856e40700 1 --2- 172.21.15.142:0/4260737175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5850105a40 0x7f5850105e20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5844009910 tx=0x7f5844009c20 comp rx=0 tx=0).stop 2024-03-29T04:58:26.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.151+0000 7f5856e40700 1 -- 172.21.15.142:0/4260737175 shutdown_connections 2024-03-29T04:58:26.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.151+0000 7f5856e40700 1 --2- 172.21.15.142:0/4260737175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58500ffbd0 0x7f5850100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.151+0000 7f5856e40700 1 --2- 172.21.15.142:0/4260737175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5850105a40 0x7f5850105e20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.151+0000 7f5856e40700 1 -- 172.21.15.142:0/4260737175 >> 172.21.15.142:0/4260737175 conn(0x7f58500755a0 msgr2=0x7f58500759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:26.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.151+0000 7f5856e40700 1 -- 172.21.15.142:0/4260737175 shutdown_connections 2024-03-29T04:58:26.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.151+0000 7f5856e40700 1 -- 172.21.15.142:0/4260737175 wait complete. 2024-03-29T04:58:26.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.152+0000 7f5856e40700 1 Processor -- start 2024-03-29T04:58:26.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.152+0000 7f5856e40700 1 -- start start 2024-03-29T04:58:26.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.152+0000 7f5856e40700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58500ffbd0 0x7f5850101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:26.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.152+0000 7f5856e40700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5850105a40 0x7f5850101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:26.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.152+0000 7f5856e40700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5850102090 con 0x7f5850105a40 2024-03-29T04:58:26.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.152+0000 7f5856e40700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58501021d0 con 0x7f58500ffbd0 2024-03-29T04:58:26.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.152+0000 7f584ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5850105a40 0x7f5850101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:26.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.152+0000 7f584ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5850105a40 0x7f5850101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40258/0 (socket says 172.21.15.142:40258) 2024-03-29T04:58:26.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.152+0000 7f584ffff700 1 -- 172.21.15.142:0/1272538968 learned_addr learned my addr 172.21.15.142:0/1272538968 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:26.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.152+0000 7f5854bdc700 1 --2- 172.21.15.142:0/1272538968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58500ffbd0 0x7f5850101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:26.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.152+0000 7f584ffff700 1 -- 172.21.15.142:0/1272538968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58500ffbd0 msgr2=0x7f5850101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:26.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.152+0000 7f584ffff700 1 --2- 172.21.15.142:0/1272538968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58500ffbd0 0x7f5850101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.153+0000 7f584ffff700 1 -- 172.21.15.142:0/1272538968 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58440095f0 con 0x7f5850105a40 2024-03-29T04:58:26.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.153+0000 7f584ffff700 1 --2- 172.21.15.142:0/1272538968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5850105a40 0x7f5850101b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f583c00d8d0 tx=0x7f583c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:26.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.153+0000 7f584dffb700 1 -- 172.21.15.142:0/1272538968 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f583c0098a0 con 0x7f5850105a40 2024-03-29T04:58:26.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.153+0000 7f5856e40700 1 -- 172.21.15.142:0/1272538968 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58501a28e0 con 0x7f5850105a40 2024-03-29T04:58:26.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.153+0000 7f584dffb700 1 -- 172.21.15.142:0/1272538968 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f583c010460 con 0x7f5850105a40 2024-03-29T04:58:26.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.153+0000 7f584dffb700 1 -- 172.21.15.142:0/1272538968 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f583c00f5f0 con 0x7f5850105a40 2024-03-29T04:58:26.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.153+0000 7f5856e40700 1 -- 172.21.15.142:0/1272538968 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58501a2cf0 con 0x7f5850105a40 2024-03-29T04:58:26.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.154+0000 7f5856e40700 1 -- 172.21.15.142:0/1272538968 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585004ea90 con 0x7f5850105a40 2024-03-29T04:58:26.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.154+0000 7f584dffb700 1 -- 172.21.15.142:0/1272538968 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f583c009a00 con 0x7f5850105a40 2024-03-29T04:58:26.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.155+0000 7f584dffb700 1 --2- 172.21.15.142:0/1272538968 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f584006dd60 0x7f5840070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:26.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.155+0000 7f584dffb700 1 -- 172.21.15.142:0/1272538968 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f583c08db40 con 0x7f5850105a40 2024-03-29T04:58:26.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.155+0000 7f5854bdc700 1 --2- 172.21.15.142:0/1272538968 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f584006dd60 0x7f5840070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:26.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.155+0000 7f5854bdc700 1 --2- 172.21.15.142:0/1272538968 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f584006dd60 0x7f5840070220 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5844006010 tx=0x7f584401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:26.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.159+0000 7f584dffb700 1 -- 172.21.15.142:0/1272538968 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f583c056f60 con 0x7f5850105a40 2024-03-29T04:58:26.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.403+0000 7f5856e40700 1 -- 172.21.15.142:0/1272538968 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f58501a30b0 con 0x7f5850105a40 2024-03-29T04:58:26.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.404+0000 7f584dffb700 1 -- 172.21.15.142:0/1272538968 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f583c05a620 con 0x7f5850105a40 2024-03-29T04:58:26.411 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T04:58:26.411 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:58:26.411 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:58:26.412 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:58:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:58:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:58:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:58:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T04:58:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:58:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:58:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:58:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T04:58:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T04:58:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:58:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:58:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:58:26.414 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:58:26.414 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:58:26.414 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:58:26.414 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:58:26.414 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:58:26.414 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:58:26.414 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:58:26.414 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:58:26.414 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T04:58:26.414 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:58:26.414 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:58:26.415 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:58:26.415 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:58:26.415 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:58:26.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.406+0000 7f5856e40700 1 -- 172.21.15.142:0/1272538968 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f584006dd60 msgr2=0x7f5840070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:26.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.406+0000 7f5856e40700 1 --2- 172.21.15.142:0/1272538968 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f584006dd60 0x7f5840070220 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5844006010 tx=0x7f584401a040 comp rx=0 tx=0).stop 2024-03-29T04:58:26.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.406+0000 7f5856e40700 1 -- 172.21.15.142:0/1272538968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5850105a40 msgr2=0x7f5850101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:26.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.406+0000 7f5856e40700 1 --2- 172.21.15.142:0/1272538968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5850105a40 0x7f5850101b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f583c00d8d0 tx=0x7f583c00dbe0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.406+0000 7f5856e40700 1 -- 172.21.15.142:0/1272538968 shutdown_connections 2024-03-29T04:58:26.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.406+0000 7f5856e40700 1 --2- 172.21.15.142:0/1272538968 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f584006dd60 0x7f5840070220 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.406+0000 7f5856e40700 1 --2- 172.21.15.142:0/1272538968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58500ffbd0 0x7f5850101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.406+0000 7f5856e40700 1 --2- 172.21.15.142:0/1272538968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5850105a40 0x7f5850101b50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.406+0000 7f5856e40700 1 -- 172.21.15.142:0/1272538968 >> 172.21.15.142:0/1272538968 conn(0x7f58500755a0 msgr2=0x7f58501037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:26.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.406+0000 7f5856e40700 1 -- 172.21.15.142:0/1272538968 shutdown_connections 2024-03-29T04:58:26.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.406+0000 7f5856e40700 1 -- 172.21.15.142:0/1272538968 wait complete. 2024-03-29T04:58:26.417 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T04:58:26.518 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.511+0000 7ffb15ffa700 1 -- 172.21.15.142:0/556899916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb10105a40 msgr2=0x7ffb10105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:26.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.511+0000 7ffb15ffa700 1 --2- 172.21.15.142:0/556899916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb10105a40 0x7ffb10105e20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009910 tx=0x7ffb00009c20 comp rx=0 tx=0).stop 2024-03-29T04:58:26.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.511+0000 7ffb15ffa700 1 -- 172.21.15.142:0/556899916 shutdown_connections 2024-03-29T04:58:26.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.511+0000 7ffb15ffa700 1 --2- 172.21.15.142:0/556899916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb100ffa40 0x7ffb100ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.511+0000 7ffb15ffa700 1 --2- 172.21.15.142:0/556899916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb10105a40 0x7ffb10105e20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.511+0000 7ffb15ffa700 1 -- 172.21.15.142:0/556899916 >> 172.21.15.142:0/556899916 conn(0x7ffb100fb570 msgr2=0x7ffb100fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:26.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.511+0000 7ffb15ffa700 1 -- 172.21.15.142:0/556899916 shutdown_connections 2024-03-29T04:58:26.519 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.511+0000 7ffb15ffa700 1 -- 172.21.15.142:0/556899916 wait complete. 2024-03-29T04:58:26.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb15ffa700 1 Processor -- start 2024-03-29T04:58:26.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb15ffa700 1 -- start start 2024-03-29T04:58:26.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb15ffa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb100ffa40 0x7ffb1019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:26.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb15ffa700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb10105a40 0x7ffb1019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:26.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb15ffa700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1019f2d0 con 0x7ffb100ffa40 2024-03-29T04:58:26.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb15ffa700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb10198780 con 0x7ffb10105a40 2024-03-29T04:58:26.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb0f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb100ffa40 0x7ffb1019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:26.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb0f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb100ffa40 0x7ffb1019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40282/0 (socket says 172.21.15.142:40282) 2024-03-29T04:58:26.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb0f7fe700 1 -- 172.21.15.142:0/3702284000 learned_addr learned my addr 172.21.15.142:0/3702284000 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:26.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb0f7fe700 1 -- 172.21.15.142:0/3702284000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb10105a40 msgr2=0x7ffb1019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:26.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb0f7fe700 1 --2- 172.21.15.142:0/3702284000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb10105a40 0x7ffb1019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb0f7fe700 1 -- 172.21.15.142:0/3702284000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb000095f0 con 0x7ffb100ffa40 2024-03-29T04:58:26.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.512+0000 7ffb0f7fe700 1 --2- 172.21.15.142:0/3702284000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb100ffa40 0x7ffb1019e700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffb00006010 tx=0x7ffb0000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:26.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.513+0000 7ffb0d7fa700 1 -- 172.21.15.142:0/3702284000 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb0001d070 con 0x7ffb100ffa40 2024-03-29T04:58:26.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.513+0000 7ffb0d7fa700 1 -- 172.21.15.142:0/3702284000 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb0000f460 con 0x7ffb100ffa40 2024-03-29T04:58:26.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.513+0000 7ffb15ffa700 1 -- 172.21.15.142:0/3702284000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb10198a00 con 0x7ffb100ffa40 2024-03-29T04:58:26.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.513+0000 7ffb0d7fa700 1 -- 172.21.15.142:0/3702284000 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb00017630 con 0x7ffb100ffa40 2024-03-29T04:58:26.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.513+0000 7ffb15ffa700 1 -- 172.21.15.142:0/3702284000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb10198ef0 con 0x7ffb100ffa40 2024-03-29T04:58:26.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.514+0000 7ffb15ffa700 1 -- 172.21.15.142:0/3702284000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1004ea90 con 0x7ffb100ffa40 2024-03-29T04:58:26.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.514+0000 7ffb0d7fa700 1 -- 172.21.15.142:0/3702284000 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ffb0000f5d0 con 0x7ffb100ffa40 2024-03-29T04:58:26.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.515+0000 7ffb0d7fa700 1 --2- 172.21.15.142:0/3702284000 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffafc06dd60 0x7ffafc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:26.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.515+0000 7ffb0d7fa700 1 -- 172.21.15.142:0/3702284000 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ffb0008f0c0 con 0x7ffb100ffa40 2024-03-29T04:58:26.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.515+0000 7ffb06ffd700 1 --2- 172.21.15.142:0/3702284000 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffafc06dd60 0x7ffafc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:26.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.515+0000 7ffb06ffd700 1 --2- 172.21.15.142:0/3702284000 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffafc06dd60 0x7ffafc070220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8005950 tx=0x7ffaf80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:26.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.519+0000 7ffb0d7fa700 1 -- 172.21.15.142:0/3702284000 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ffb0005c090 con 0x7ffb100ffa40 2024-03-29T04:58:26.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:26 smithi142 ceph-mon[112475]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:26.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:26 smithi142 ceph-mon[112475]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:26.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:26 smithi142 ceph-mon[112475]: from='client.34606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:26.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:26 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/851352588' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:58:26.740 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.733+0000 7ffb15ffa700 1 -- 172.21.15.142:0/3702284000 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb101992e0 con 0x7ffafc06dd60 2024-03-29T04:58:26.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.734+0000 7ffb0d7fa700 1 -- 172.21.15.142:0/3702284000 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ffb101992e0 con 0x7ffafc06dd60 2024-03-29T04:58:26.741 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T04:58:26.742 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:58:26.743 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:58:26.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.736+0000 7ffb15ffa700 1 -- 172.21.15.142:0/3702284000 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffafc06dd60 msgr2=0x7ffafc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:26.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.736+0000 7ffb15ffa700 1 --2- 172.21.15.142:0/3702284000 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffafc06dd60 0x7ffafc070220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8005950 tx=0x7ffaf80058e0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.736+0000 7ffb15ffa700 1 -- 172.21.15.142:0/3702284000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb100ffa40 msgr2=0x7ffb1019e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:26.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.736+0000 7ffb15ffa700 1 --2- 172.21.15.142:0/3702284000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb100ffa40 0x7ffb1019e700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffb00006010 tx=0x7ffb0000be20 comp rx=0 tx=0).stop 2024-03-29T04:58:26.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.737+0000 7ffb15ffa700 1 -- 172.21.15.142:0/3702284000 shutdown_connections 2024-03-29T04:58:26.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.737+0000 7ffb15ffa700 1 --2- 172.21.15.142:0/3702284000 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffafc06dd60 0x7ffafc070220 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.737+0000 7ffb15ffa700 1 --2- 172.21.15.142:0/3702284000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb100ffa40 0x7ffb1019e700 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.745 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.737+0000 7ffb15ffa700 1 --2- 172.21.15.142:0/3702284000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb10105a40 0x7ffb1019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.745 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.737+0000 7ffb15ffa700 1 -- 172.21.15.142:0/3702284000 >> 172.21.15.142:0/3702284000 conn(0x7ffb100fb570 msgr2=0x7ffb10068d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:26.745 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.737+0000 7ffb15ffa700 1 -- 172.21.15.142:0/3702284000 shutdown_connections 2024-03-29T04:58:26.745 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.737+0000 7ffb15ffa700 1 -- 172.21.15.142:0/3702284000 wait complete. 2024-03-29T04:58:26.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:26 smithi152 ceph-mon[88186]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:26.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:26 smithi152 ceph-mon[88186]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:26.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:26 smithi152 ceph-mon[88186]: from='client.34606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:26.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:26 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/851352588' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:58:26.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.841+0000 7ff2f8513700 1 -- 172.21.15.142:0/3273903252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2f0100230 msgr2=0x7ff2f010cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:26.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.841+0000 7ff2f8513700 1 --2- 172.21.15.142:0/3273903252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2f0100230 0x7ff2f010cf80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009910 tx=0x7ff2e0009c20 comp rx=0 tx=0).stop 2024-03-29T04:58:26.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.841+0000 7ff2f8513700 1 -- 172.21.15.142:0/3273903252 shutdown_connections 2024-03-29T04:58:26.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.841+0000 7ff2f8513700 1 --2- 172.21.15.142:0/3273903252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2f0100230 0x7ff2f010cf80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.841+0000 7ff2f8513700 1 --2- 172.21.15.142:0/3273903252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2f00ff910 0x7ff2f00ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.841+0000 7ff2f8513700 1 -- 172.21.15.142:0/3273903252 >> 172.21.15.142:0/3273903252 conn(0x7ff2f00fb570 msgr2=0x7ff2f00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:26.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.842+0000 7ff2f8513700 1 -- 172.21.15.142:0/3273903252 shutdown_connections 2024-03-29T04:58:26.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.842+0000 7ff2f8513700 1 -- 172.21.15.142:0/3273903252 wait complete. 2024-03-29T04:58:26.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.842+0000 7ff2f8513700 1 Processor -- start 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.842+0000 7ff2f8513700 1 -- start start 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.842+0000 7ff2f8513700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2f00ff910 0x7ff2f019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.842+0000 7ff2f8513700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2f0100230 0x7ff2f019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.842+0000 7ff2f8513700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f019f400 con 0x7ff2f0100230 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.842+0000 7ff2f8513700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f01988b0 con 0x7ff2f00ff910 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2f5aae700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2f0100230 0x7ff2f019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2f5aae700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2f0100230 0x7ff2f019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40294/0 (socket says 172.21.15.142:40294) 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2f5aae700 1 -- 172.21.15.142:0/2638714581 learned_addr learned my addr 172.21.15.142:0/2638714581 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2f62af700 1 --2- 172.21.15.142:0/2638714581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2f00ff910 0x7ff2f019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2f5aae700 1 -- 172.21.15.142:0/2638714581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2f00ff910 msgr2=0x7ff2f019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2f5aae700 1 --2- 172.21.15.142:0/2638714581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2f00ff910 0x7ff2f019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2f5aae700 1 -- 172.21.15.142:0/2638714581 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e00095f0 con 0x7ff2f0100230 2024-03-29T04:58:26.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2f62af700 1 --2- 172.21.15.142:0/2638714581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2f00ff910 0x7ff2f019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:58:26.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2f5aae700 1 --2- 172.21.15.142:0/2638714581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2f0100230 0x7ff2f019ed70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00092e0 tx=0x7ff2e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:26.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2e77fe700 1 -- 172.21.15.142:0/2638714581 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e001d070 con 0x7ff2f0100230 2024-03-29T04:58:26.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2f8513700 1 -- 172.21.15.142:0/2638714581 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f0198b30 con 0x7ff2f0100230 2024-03-29T04:58:26.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2e77fe700 1 -- 172.21.15.142:0/2638714581 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2e000f460 con 0x7ff2f0100230 2024-03-29T04:58:26.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2e77fe700 1 -- 172.21.15.142:0/2638714581 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e00175f0 con 0x7ff2f0100230 2024-03-29T04:58:26.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.843+0000 7ff2f8513700 1 -- 172.21.15.142:0/2638714581 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f0199020 con 0x7ff2f0100230 2024-03-29T04:58:26.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.844+0000 7ff2f8513700 1 -- 172.21.15.142:0/2638714581 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f010a6f0 con 0x7ff2f0100230 2024-03-29T04:58:26.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.845+0000 7ff2e77fe700 1 -- 172.21.15.142:0/2638714581 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff2e000f5d0 con 0x7ff2f0100230 2024-03-29T04:58:26.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.845+0000 7ff2e77fe700 1 --2- 172.21.15.142:0/2638714581 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2dc06dd60 0x7ff2dc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:26.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.845+0000 7ff2e77fe700 1 -- 172.21.15.142:0/2638714581 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff2e008f1f0 con 0x7ff2f0100230 2024-03-29T04:58:26.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.845+0000 7ff2f62af700 1 --2- 172.21.15.142:0/2638714581 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2dc06dd60 0x7ff2dc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:26.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.846+0000 7ff2f62af700 1 --2- 172.21.15.142:0/2638714581 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2dc06dd60 0x7ff2dc070220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0099d0 tx=0x7ff2ec008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:26.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:26.850+0000 7ff2e77fe700 1 -- 172.21.15.142:0/2638714581 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff2e005c140 con 0x7ff2f0100230 2024-03-29T04:58:27.131 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.124+0000 7ff2f8513700 1 -- 172.21.15.142:0/2638714581 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff2f004ea90 con 0x7ff2f0100230 2024-03-29T04:58:27.132 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.124+0000 7ff2e77fe700 1 -- 172.21.15.142:0/2638714581 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff2e005bcd0 con 0x7ff2f0100230 2024-03-29T04:58:27.132 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T04:58:27.132 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T04:58:27.132 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T04:58:27.132 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T04:58:27.132 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T04:58:27.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.126+0000 7ff2f8513700 1 -- 172.21.15.142:0/2638714581 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2dc06dd60 msgr2=0x7ff2dc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:27.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.126+0000 7ff2f8513700 1 --2- 172.21.15.142:0/2638714581 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2dc06dd60 0x7ff2dc070220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0099d0 tx=0x7ff2ec008040 comp rx=0 tx=0).stop 2024-03-29T04:58:27.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.126+0000 7ff2f8513700 1 -- 172.21.15.142:0/2638714581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2f0100230 msgr2=0x7ff2f019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:27.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.126+0000 7ff2f8513700 1 --2- 172.21.15.142:0/2638714581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2f0100230 0x7ff2f019ed70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00092e0 tx=0x7ff2e000be20 comp rx=0 tx=0).stop 2024-03-29T04:58:27.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.126+0000 7ff2f8513700 1 -- 172.21.15.142:0/2638714581 shutdown_connections 2024-03-29T04:58:27.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.126+0000 7ff2f8513700 1 --2- 172.21.15.142:0/2638714581 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2dc06dd60 0x7ff2dc070220 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:27.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.126+0000 7ff2f8513700 1 --2- 172.21.15.142:0/2638714581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2f00ff910 0x7ff2f019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:27.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.126+0000 7ff2f8513700 1 --2- 172.21.15.142:0/2638714581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2f0100230 0x7ff2f019ed70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:27.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.126+0000 7ff2f8513700 1 -- 172.21.15.142:0/2638714581 >> 172.21.15.142:0/2638714581 conn(0x7ff2f00fb570 msgr2=0x7ff2f00fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:27.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.126+0000 7ff2f8513700 1 -- 172.21.15.142:0/2638714581 shutdown_connections 2024-03-29T04:58:27.135 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:27.126+0000 7ff2f8513700 1 -- 172.21.15.142:0/2638714581 wait complete. 2024-03-29T04:58:27.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:27 smithi142 ceph-mon[112475]: pgmap v304: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:27.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:27 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1272538968' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:58:27.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:27 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2638714581' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:58:27.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:27 smithi152 ceph-mon[88186]: pgmap v304: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:27.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:27 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1272538968' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:58:27.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:27 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2638714581' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:58:28.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:28 smithi142 ceph-mon[112475]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:28.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:28 smithi152 ceph-mon[88186]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:29.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:29 smithi142 ceph-mon[112475]: pgmap v305: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:29.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:29 smithi152 ceph-mon[88186]: pgmap v305: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:30.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:58:30.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:58:31.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:31 smithi152 ceph-mon[88186]: pgmap v306: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:31.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:31 smithi142 ceph-mon[112475]: pgmap v306: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:33.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:33 smithi152 ceph-mon[88186]: pgmap v307: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:33.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:33 smithi142 ceph-mon[112475]: pgmap v307: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:35.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:35 smithi152 ceph-mon[88186]: pgmap v308: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:35.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:35 smithi142 ceph-mon[112475]: pgmap v308: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:37.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:37 smithi142 ceph-mon[112475]: pgmap v309: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:37.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:37 smithi152 ceph-mon[88186]: pgmap v309: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:39.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:39 smithi152 ceph-mon[88186]: pgmap v310: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:39.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:39 smithi142 ceph-mon[112475]: pgmap v310: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:41.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:41 smithi152 ceph-mon[88186]: pgmap v311: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:41.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:41 smithi142 ceph-mon[112475]: pgmap v311: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:43.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:43 smithi152 ceph-mon[88186]: pgmap v312: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:43.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:43 smithi142 ceph-mon[112475]: pgmap v312: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:45.547 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:45 smithi152 ceph-mon[88186]: pgmap v313: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:45.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:58:45.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:45 smithi142 ceph-mon[112475]: pgmap v313: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:45.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:58:47.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:47 smithi152 ceph-mon[88186]: pgmap v314: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:47.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:47 smithi142 ceph-mon[112475]: pgmap v314: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:49.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:49 smithi152 ceph-mon[88186]: pgmap v315: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:49.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:49 smithi142 ceph-mon[112475]: pgmap v315: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:51.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:51 smithi152 ceph-mon[88186]: pgmap v316: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:51.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:51 smithi142 ceph-mon[112475]: pgmap v316: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:53.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:53 smithi152 ceph-mon[88186]: pgmap v317: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:53.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:53 smithi142 ceph-mon[112475]: pgmap v317: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:55.548 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:55 smithi152 ceph-mon[88186]: pgmap v318: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:55.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:55 smithi142 ceph-mon[112475]: pgmap v318: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.244+0000 7fdcaa975700 1 -- 172.21.15.142:0/2333734964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdca4105b10 msgr2=0x7fdca4105ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.244+0000 7fdcaa975700 1 --2- 172.21.15.142:0/2333734964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdca4105b10 0x7fdca4105ef0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009910 tx=0x7fdc94009c20 comp rx=0 tx=0).stop 2024-03-29T04:58:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.244+0000 7fdcaa975700 1 -- 172.21.15.142:0/2333734964 shutdown_connections 2024-03-29T04:58:57.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.244+0000 7fdcaa975700 1 --2- 172.21.15.142:0/2333734964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca4100420 0x7fdca4100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.244+0000 7fdcaa975700 1 --2- 172.21.15.142:0/2333734964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdca4105b10 0x7fdca4105ef0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.244+0000 7fdcaa975700 1 -- 172.21.15.142:0/2333734964 >> 172.21.15.142:0/2333734964 conn(0x7fdca4074bf0 msgr2=0x7fdca4075000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:57.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.244+0000 7fdcaa975700 1 -- 172.21.15.142:0/2333734964 shutdown_connections 2024-03-29T04:58:57.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.244+0000 7fdcaa975700 1 -- 172.21.15.142:0/2333734964 wait complete. 2024-03-29T04:58:57.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.244+0000 7fdcaa975700 1 Processor -- start 2024-03-29T04:58:57.256 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.244+0000 7fdcaa975700 1 -- start start 2024-03-29T04:58:57.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdcaa975700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdca4100420 0x7fdca4101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:57.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdcaa975700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca4105b10 0x7fdca4101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:57.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdcaa975700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca4102290 con 0x7fdca4100420 2024-03-29T04:58:57.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdcaa975700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca4102400 con 0x7fdca4105b10 2024-03-29T04:58:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdca3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdca4100420 0x7fdca4101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdca3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdca4100420 0x7fdca4101780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47742/0 (socket says 172.21.15.142:47742) 2024-03-29T04:58:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdca3fff700 1 -- 172.21.15.142:0/1889406861 learned_addr learned my addr 172.21.15.142:0/1889406861 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:57.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdca37fe700 1 --2- 172.21.15.142:0/1889406861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca4105b10 0x7fdca4101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:57.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdca3fff700 1 -- 172.21.15.142:0/1889406861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca4105b10 msgr2=0x7fdca4101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:57.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdca3fff700 1 --2- 172.21.15.142:0/1889406861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca4105b10 0x7fdca4101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdca3fff700 1 -- 172.21.15.142:0/1889406861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc940095f0 con 0x7fdca4100420 2024-03-29T04:58:57.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdca37fe700 1 --2- 172.21.15.142:0/1889406861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca4105b10 0x7fdca4101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:58:57.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.245+0000 7fdca3fff700 1 --2- 172.21.15.142:0/1889406861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdca4100420 0x7fdca4101780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdc94006010 tx=0x7fdc9400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:57.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.246+0000 7fdca17fa700 1 -- 172.21.15.142:0/1889406861 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc9401d070 con 0x7fdca4100420 2024-03-29T04:58:57.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.246+0000 7fdcaa975700 1 -- 172.21.15.142:0/1889406861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca41a28e0 con 0x7fdca4100420 2024-03-29T04:58:57.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.246+0000 7fdca17fa700 1 -- 172.21.15.142:0/1889406861 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc9400f460 con 0x7fdca4100420 2024-03-29T04:58:57.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.246+0000 7fdca17fa700 1 -- 172.21.15.142:0/1889406861 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc94017630 con 0x7fdca4100420 2024-03-29T04:58:57.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.246+0000 7fdcaa975700 1 -- 172.21.15.142:0/1889406861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca41a2c90 con 0x7fdca4100420 2024-03-29T04:58:57.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.247+0000 7fdcaa975700 1 -- 172.21.15.142:0/1889406861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca4109040 con 0x7fdca4100420 2024-03-29T04:58:57.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.247+0000 7fdca17fa700 1 -- 172.21.15.142:0/1889406861 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fdc9400f5d0 con 0x7fdca4100420 2024-03-29T04:58:57.262 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.248+0000 7fdca17fa700 1 --2- 172.21.15.142:0/1889406861 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdc8c06dd60 0x7fdc8c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:57.263 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.248+0000 7fdca17fa700 1 -- 172.21.15.142:0/1889406861 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fdc9408f140 con 0x7fdca4100420 2024-03-29T04:58:57.263 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.248+0000 7fdca37fe700 1 --2- 172.21.15.142:0/1889406861 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdc8c06dd60 0x7fdc8c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:57.263 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.248+0000 7fdca37fe700 1 --2- 172.21.15.142:0/1889406861 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdc8c06dd60 0x7fdc8c070220 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdc98005fd0 tx=0x7fdc98005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:57.263 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.253+0000 7fdca17fa700 1 -- 172.21.15.142:0/1889406861 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdc94058560 con 0x7fdca4100420 2024-03-29T04:58:57.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.469+0000 7fdcaa975700 1 -- 172.21.15.142:0/1889406861 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdca402cce0 con 0x7fdc8c06dd60 2024-03-29T04:58:57.478 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.470+0000 7fdca17fa700 1 -- 172.21.15.142:0/1889406861 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdca402cce0 con 0x7fdc8c06dd60 2024-03-29T04:58:57.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.473+0000 7fdcaa975700 1 -- 172.21.15.142:0/1889406861 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdc8c06dd60 msgr2=0x7fdc8c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:57.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.473+0000 7fdcaa975700 1 --2- 172.21.15.142:0/1889406861 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdc8c06dd60 0x7fdc8c070220 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdc98005fd0 tx=0x7fdc98005ee0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.473+0000 7fdcaa975700 1 -- 172.21.15.142:0/1889406861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdca4100420 msgr2=0x7fdca4101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:57.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.473+0000 7fdcaa975700 1 --2- 172.21.15.142:0/1889406861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdca4100420 0x7fdca4101780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdc94006010 tx=0x7fdc9400be20 comp rx=0 tx=0).stop 2024-03-29T04:58:57.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.473+0000 7fdcaa975700 1 -- 172.21.15.142:0/1889406861 shutdown_connections 2024-03-29T04:58:57.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.473+0000 7fdcaa975700 1 --2- 172.21.15.142:0/1889406861 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdc8c06dd60 0x7fdc8c070220 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.473+0000 7fdcaa975700 1 --2- 172.21.15.142:0/1889406861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdca4100420 0x7fdca4101780 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.473+0000 7fdcaa975700 1 --2- 172.21.15.142:0/1889406861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca4105b10 0x7fdca4101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.473+0000 7fdcaa975700 1 -- 172.21.15.142:0/1889406861 >> 172.21.15.142:0/1889406861 conn(0x7fdca4074bf0 msgr2=0x7fdca4104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:57.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.473+0000 7fdcaa975700 1 -- 172.21.15.142:0/1889406861 shutdown_connections 2024-03-29T04:58:57.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.473+0000 7fdcaa975700 1 -- 172.21.15.142:0/1889406861 wait complete. 2024-03-29T04:58:57.493 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:58:57.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:57 smithi142 ceph-mon[112475]: pgmap v319: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:57.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.580+0000 7f3042475700 1 -- 172.21.15.142:0/508285727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303c101750 msgr2=0x7f303c1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:57.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.580+0000 7f3042475700 1 --2- 172.21.15.142:0/508285727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303c101750 0x7f303c1057a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3030009910 tx=0x7f3030009c20 comp rx=0 tx=0).stop 2024-03-29T04:58:57.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.580+0000 7f3042475700 1 -- 172.21.15.142:0/508285727 shutdown_connections 2024-03-29T04:58:57.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.580+0000 7f3042475700 1 --2- 172.21.15.142:0/508285727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303c101750 0x7f303c1057a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.580+0000 7f3042475700 1 --2- 172.21.15.142:0/508285727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f303c100da0 0x7f303c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.580+0000 7f3042475700 1 -- 172.21.15.142:0/508285727 >> 172.21.15.142:0/508285727 conn(0x7f303c0fc630 msgr2=0x7f303c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:57.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.580+0000 7f3042475700 1 -- 172.21.15.142:0/508285727 shutdown_connections 2024-03-29T04:58:57.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.580+0000 7f3042475700 1 -- 172.21.15.142:0/508285727 wait complete. 2024-03-29T04:58:57.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.581+0000 7f3042475700 1 Processor -- start 2024-03-29T04:58:57.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.581+0000 7f3042475700 1 -- start start 2024-03-29T04:58:57.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.581+0000 7f3042475700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f303c100da0 0x7f303c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:57.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.581+0000 7f3042475700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303c101750 0x7f303c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:57.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.581+0000 7f3042475700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c19d270 con 0x7f303c101750 2024-03-29T04:58:57.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.581+0000 7f3042475700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c196720 con 0x7f303c100da0 2024-03-29T04:58:57.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.581+0000 7f303b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303c101750 0x7f303c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:57.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.581+0000 7f303b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303c101750 0x7f303c19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47756/0 (socket says 172.21.15.142:47756) 2024-03-29T04:58:57.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.581+0000 7f303b7fe700 1 -- 172.21.15.142:0/844439128 learned_addr learned my addr 172.21.15.142:0/844439128 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:57.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.581+0000 7f303bfff700 1 --2- 172.21.15.142:0/844439128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f303c100da0 0x7f303c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:57.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.582+0000 7f303b7fe700 1 -- 172.21.15.142:0/844439128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f303c100da0 msgr2=0x7f303c19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:57.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.582+0000 7f303b7fe700 1 --2- 172.21.15.142:0/844439128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f303c100da0 0x7f303c19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.582+0000 7f303b7fe700 1 -- 172.21.15.142:0/844439128 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30300095f0 con 0x7f303c101750 2024-03-29T04:58:57.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.582+0000 7f303b7fe700 1 --2- 172.21.15.142:0/844439128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303c101750 0x7f303c19cbe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f30300092e0 tx=0x7f303000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:57.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.582+0000 7f30397fa700 1 -- 172.21.15.142:0/844439128 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f303001d070 con 0x7f303c101750 2024-03-29T04:58:57.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.582+0000 7f30397fa700 1 -- 172.21.15.142:0/844439128 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f303000f460 con 0x7f303c101750 2024-03-29T04:58:57.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.582+0000 7f3042475700 1 -- 172.21.15.142:0/844439128 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c1969a0 con 0x7f303c101750 2024-03-29T04:58:57.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.582+0000 7f30397fa700 1 -- 172.21.15.142:0/844439128 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30300175f0 con 0x7f303c101750 2024-03-29T04:58:57.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.582+0000 7f3042475700 1 -- 172.21.15.142:0/844439128 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303c196e90 con 0x7f303c101750 2024-03-29T04:58:57.592 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.583+0000 7f3042475700 1 -- 172.21.15.142:0/844439128 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303c1090f0 con 0x7f303c101750 2024-03-29T04:58:57.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.584+0000 7f30397fa700 1 -- 172.21.15.142:0/844439128 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f303000f5d0 con 0x7f303c101750 2024-03-29T04:58:57.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.584+0000 7f30397fa700 1 --2- 172.21.15.142:0/844439128 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f302406dd60 0x7f3024070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:57.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.584+0000 7f30397fa700 1 -- 172.21.15.142:0/844439128 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f303008f130 con 0x7f303c101750 2024-03-29T04:58:57.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.584+0000 7f303bfff700 1 --2- 172.21.15.142:0/844439128 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f302406dd60 0x7f3024070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:57.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.585+0000 7f303bfff700 1 --2- 172.21.15.142:0/844439128 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f302406dd60 0x7f3024070220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f302c006fd0 tx=0x7f302c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:57.596 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.589+0000 7f30397fa700 1 -- 172.21.15.142:0/844439128 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f303005c100 con 0x7f303c101750 2024-03-29T04:58:57.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:57 smithi152 ceph-mon[88186]: pgmap v319: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:57.815 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.808+0000 7f3042475700 1 -- 172.21.15.142:0/844439128 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f303c02cce0 con 0x7f302406dd60 2024-03-29T04:58:57.816 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.809+0000 7f30397fa700 1 -- 172.21.15.142:0/844439128 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f303c02cce0 con 0x7f302406dd60 2024-03-29T04:58:57.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.811+0000 7f3042475700 1 -- 172.21.15.142:0/844439128 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f302406dd60 msgr2=0x7f3024070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:57.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.811+0000 7f3042475700 1 --2- 172.21.15.142:0/844439128 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f302406dd60 0x7f3024070220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f302c006fd0 tx=0x7f302c008040 comp rx=0 tx=0).stop 2024-03-29T04:58:57.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.811+0000 7f3042475700 1 -- 172.21.15.142:0/844439128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303c101750 msgr2=0x7f303c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:57.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.811+0000 7f3042475700 1 --2- 172.21.15.142:0/844439128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303c101750 0x7f303c19cbe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f30300092e0 tx=0x7f303000be20 comp rx=0 tx=0).stop 2024-03-29T04:58:57.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.811+0000 7f3042475700 1 -- 172.21.15.142:0/844439128 shutdown_connections 2024-03-29T04:58:57.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.811+0000 7f3042475700 1 --2- 172.21.15.142:0/844439128 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f302406dd60 0x7f3024070220 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.811+0000 7f3042475700 1 --2- 172.21.15.142:0/844439128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f303c100da0 0x7f303c19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.811+0000 7f3042475700 1 --2- 172.21.15.142:0/844439128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303c101750 0x7f303c19cbe0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.819 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.811+0000 7f3042475700 1 -- 172.21.15.142:0/844439128 >> 172.21.15.142:0/844439128 conn(0x7f303c0fc630 msgr2=0x7f303c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:57.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.811+0000 7f3042475700 1 -- 172.21.15.142:0/844439128 shutdown_connections 2024-03-29T04:58:57.820 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.811+0000 7f3042475700 1 -- 172.21.15.142:0/844439128 wait complete. 2024-03-29T04:58:57.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.916+0000 7fa19dd4a700 1 -- 172.21.15.142:0/4292599259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa198102640 msgr2=0x7fa19810ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:57.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.916+0000 7fa19dd4a700 1 --2- 172.21.15.142:0/4292599259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa198102640 0x7fa19810ab30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009910 tx=0x7fa18c009c20 comp rx=0 tx=0).stop 2024-03-29T04:58:57.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.916+0000 7fa19dd4a700 1 -- 172.21.15.142:0/4292599259 shutdown_connections 2024-03-29T04:58:57.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.916+0000 7fa19dd4a700 1 --2- 172.21.15.142:0/4292599259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa198102640 0x7fa19810ab30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.916+0000 7fa19dd4a700 1 --2- 172.21.15.142:0/4292599259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa198101c90 0x7fa198102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.916+0000 7fa19dd4a700 1 -- 172.21.15.142:0/4292599259 >> 172.21.15.142:0/4292599259 conn(0x7fa1980fb550 msgr2=0x7fa1980fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:57.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.916+0000 7fa19dd4a700 1 -- 172.21.15.142:0/4292599259 shutdown_connections 2024-03-29T04:58:57.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.916+0000 7fa19dd4a700 1 -- 172.21.15.142:0/4292599259 wait complete. 2024-03-29T04:58:57.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.916+0000 7fa19dd4a700 1 Processor -- start 2024-03-29T04:58:57.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.916+0000 7fa19dd4a700 1 -- start start 2024-03-29T04:58:57.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa19dd4a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa198101c90 0x7fa19819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:57.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa19dd4a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa198102640 0x7fa19819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:57.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa19dd4a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19819d100 con 0x7fa198101c90 2024-03-29T04:58:57.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa19dd4a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1981965b0 con 0x7fa198102640 2024-03-29T04:58:57.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa1977fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa198101c90 0x7fa19819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa1977fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa198101c90 0x7fa19819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47778/0 (socket says 172.21.15.142:47778) 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa1977fe700 1 -- 172.21.15.142:0/1754013220 learned_addr learned my addr 172.21.15.142:0/1754013220 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa196ffd700 1 --2- 172.21.15.142:0/1754013220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa198102640 0x7fa19819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa1977fe700 1 -- 172.21.15.142:0/1754013220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa198102640 msgr2=0x7fa19819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa1977fe700 1 --2- 172.21.15.142:0/1754013220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa198102640 0x7fa19819ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa1977fe700 1 -- 172.21.15.142:0/1754013220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c0095f0 con 0x7fa198101c90 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa1977fe700 1 --2- 172.21.15.142:0/1754013220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa198101c90 0x7fa19819c530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa18800d8d0 tx=0x7fa18800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa194ff9700 1 -- 172.21.15.142:0/1754013220 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa188009940 con 0x7fa198101c90 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa194ff9700 1 -- 172.21.15.142:0/1754013220 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa188010460 con 0x7fa198101c90 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa194ff9700 1 -- 172.21.15.142:0/1754013220 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa18800f5f0 con 0x7fa198101c90 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa19dd4a700 1 -- 172.21.15.142:0/1754013220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa198196890 con 0x7fa198101c90 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.917+0000 7fa19dd4a700 1 -- 172.21.15.142:0/1754013220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa198196de0 con 0x7fa198101c90 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.918+0000 7fa19dd4a700 1 -- 172.21.15.142:0/1754013220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa198108230 con 0x7fa198101c90 2024-03-29T04:58:57.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.919+0000 7fa194ff9700 1 -- 172.21.15.142:0/1754013220 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fa1880105d0 con 0x7fa198101c90 2024-03-29T04:58:57.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.919+0000 7fa194ff9700 1 --2- 172.21.15.142:0/1754013220 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa18006dd60 0x7fa180070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:57.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.919+0000 7fa194ff9700 1 -- 172.21.15.142:0/1754013220 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa18808dcd0 con 0x7fa198101c90 2024-03-29T04:58:57.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.919+0000 7fa196ffd700 1 --2- 172.21.15.142:0/1754013220 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa18006dd60 0x7fa180070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:57.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.920+0000 7fa196ffd700 1 --2- 172.21.15.142:0/1754013220 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa18006dd60 0x7fa180070220 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0059d0 tx=0x7fa18c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:57.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:57.924+0000 7fa194ff9700 1 -- 172.21.15.142:0/1754013220 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa18805ac20 con 0x7fa198101c90 2024-03-29T04:58:58.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.125+0000 7fa19dd4a700 1 -- 172.21.15.142:0/1754013220 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa198197080 con 0x7fa18006dd60 2024-03-29T04:58:58.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.136+0000 7fa194ff9700 1 -- 172.21.15.142:0/1754013220 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa198197080 con 0x7fa18006dd60 2024-03-29T04:58:58.144 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:58:58.145 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (10m) 3m ago 22m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:58:58.145 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (22m) 3m ago 22m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:58:58.145 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (21m) 3m ago 21m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:58:58.145 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (7m) 3m ago 22m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:58:58.145 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (7m) 3m ago 21m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:58:58.145 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (10m) 3m ago 22m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:58:58.145 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (3m) 3m ago 16m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T04:58:58.145 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (4m) 3m ago 16m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T04:58:58.145 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (3m) 3m ago 16m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (3m) 3m ago 16m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (12m) 3m ago 23m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (11m) 3m ago 21m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (9m) 3m ago 23m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (8m) 3m ago 21m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (11m) 3m ago 22m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (11m) 3m ago 21m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (7m) 3m ago 20m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (6m) 3m ago 20m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (6m) 3m ago 19m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (5m) 3m ago 19m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (5m) 3m ago 18m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T04:58:58.146 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (4m) 3m ago 18m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T04:58:58.147 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (11m) 3m ago 21m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:58:58.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.138+0000 7fa19dd4a700 1 -- 172.21.15.142:0/1754013220 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa18006dd60 msgr2=0x7fa180070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:58.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.138+0000 7fa19dd4a700 1 --2- 172.21.15.142:0/1754013220 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa18006dd60 0x7fa180070220 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0059d0 tx=0x7fa18c005960 comp rx=0 tx=0).stop 2024-03-29T04:58:58.147 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.138+0000 7fa19dd4a700 1 -- 172.21.15.142:0/1754013220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa198101c90 msgr2=0x7fa19819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:58.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.138+0000 7fa19dd4a700 1 --2- 172.21.15.142:0/1754013220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa198101c90 0x7fa19819c530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa18800d8d0 tx=0x7fa18800dc90 comp rx=0 tx=0).stop 2024-03-29T04:58:58.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.138+0000 7fa19dd4a700 1 -- 172.21.15.142:0/1754013220 shutdown_connections 2024-03-29T04:58:58.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.138+0000 7fa19dd4a700 1 --2- 172.21.15.142:0/1754013220 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa18006dd60 0x7fa180070220 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.138+0000 7fa19dd4a700 1 --2- 172.21.15.142:0/1754013220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa198101c90 0x7fa19819c530 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.138+0000 7fa19dd4a700 1 --2- 172.21.15.142:0/1754013220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa198102640 0x7fa19819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.138+0000 7fa19dd4a700 1 -- 172.21.15.142:0/1754013220 >> 172.21.15.142:0/1754013220 conn(0x7fa1980fb550 msgr2=0x7fa198105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:58.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.139+0000 7fa19dd4a700 1 -- 172.21.15.142:0/1754013220 shutdown_connections 2024-03-29T04:58:58.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.139+0000 7fa19dd4a700 1 -- 172.21.15.142:0/1754013220 wait complete. 2024-03-29T04:58:58.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.241+0000 7f95d445c700 1 -- 172.21.15.142:0/3292169961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95cc100dc0 msgr2=0x7f95cc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:58.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.241+0000 7f95d445c700 1 --2- 172.21.15.142:0/3292169961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95cc100dc0 0x7f95cc1011a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f95c400b0a0 tx=0x7f95c400b3b0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.241+0000 7f95d445c700 1 -- 172.21.15.142:0/3292169961 shutdown_connections 2024-03-29T04:58:58.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.241+0000 7f95d445c700 1 --2- 172.21.15.142:0/3292169961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95cc101770 0x7f95cc1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.241+0000 7f95d445c700 1 --2- 172.21.15.142:0/3292169961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95cc100dc0 0x7f95cc1011a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.241+0000 7f95d445c700 1 -- 172.21.15.142:0/3292169961 >> 172.21.15.142:0/3292169961 conn(0x7f95cc0fc650 msgr2=0x7f95cc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:58.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.241+0000 7f95d445c700 1 -- 172.21.15.142:0/3292169961 shutdown_connections 2024-03-29T04:58:58.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.241+0000 7f95d445c700 1 -- 172.21.15.142:0/3292169961 wait complete. 2024-03-29T04:58:58.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d445c700 1 Processor -- start 2024-03-29T04:58:58.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d445c700 1 -- start start 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d445c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95cc100dc0 0x7f95cc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d445c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95cc101770 0x7f95cc198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d445c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc199560 con 0x7f95cc100dc0 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d445c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc19d370 con 0x7f95cc101770 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d21f8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95cc100dc0 0x7f95cc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d21f8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95cc100dc0 0x7f95cc198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47794/0 (socket says 172.21.15.142:47794) 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d21f8700 1 -- 172.21.15.142:0/1649557665 learned_addr learned my addr 172.21.15.142:0/1649557665 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d19f7700 1 --2- 172.21.15.142:0/1649557665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95cc101770 0x7f95cc198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d21f8700 1 -- 172.21.15.142:0/1649557665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95cc101770 msgr2=0x7f95cc198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d21f8700 1 --2- 172.21.15.142:0/1649557665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95cc101770 0x7f95cc198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d21f8700 1 -- 172.21.15.142:0/1649557665 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4009d00 con 0x7f95cc100dc0 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.242+0000 7f95d19f7700 1 --2- 172.21.15.142:0/1649557665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95cc101770 0x7f95cc198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:58:58.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.243+0000 7f95d21f8700 1 --2- 172.21.15.142:0/1649557665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95cc100dc0 0x7f95cc198990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f95c400b070 tx=0x7f95c400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:58.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.243+0000 7f95c37fe700 1 -- 172.21.15.142:0/1649557665 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95c400e040 con 0x7f95cc100dc0 2024-03-29T04:58:58.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.243+0000 7f95d445c700 1 -- 172.21.15.142:0/1649557665 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc19d5f0 con 0x7f95cc100dc0 2024-03-29T04:58:58.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.243+0000 7f95c37fe700 1 -- 172.21.15.142:0/1649557665 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95c4003de0 con 0x7f95cc100dc0 2024-03-29T04:58:58.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.243+0000 7f95d445c700 1 -- 172.21.15.142:0/1649557665 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc19dae0 con 0x7f95cc100dc0 2024-03-29T04:58:58.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.243+0000 7f95c37fe700 1 -- 172.21.15.142:0/1649557665 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95c401d8e0 con 0x7f95cc100dc0 2024-03-29T04:58:58.253 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.244+0000 7f95d445c700 1 -- 172.21.15.142:0/1649557665 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc104f40 con 0x7f95cc100dc0 2024-03-29T04:58:58.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.244+0000 7f95c37fe700 1 -- 172.21.15.142:0/1649557665 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f95c4004040 con 0x7f95cc100dc0 2024-03-29T04:58:58.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.245+0000 7f95c37fe700 1 --2- 172.21.15.142:0/1649557665 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f95b806dd60 0x7f95b8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:58.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.245+0000 7f95d19f7700 1 --2- 172.21.15.142:0/1649557665 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f95b806dd60 0x7f95b8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:58.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.245+0000 7f95c37fe700 1 -- 172.21.15.142:0/1649557665 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f95c408f500 con 0x7f95cc100dc0 2024-03-29T04:58:58.254 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.246+0000 7f95d19f7700 1 --2- 172.21.15.142:0/1649557665 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f95b806dd60 0x7f95b8070220 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f95bc005fd0 tx=0x7f95bc005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:58.257 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.250+0000 7f95c37fe700 1 -- 172.21.15.142:0/1649557665 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f95c405c450 con 0x7f95cc100dc0 2024-03-29T04:58:58.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.524+0000 7f95d445c700 1 -- 172.21.15.142:0/1649557665 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f95cc04ea90 con 0x7f95cc100dc0 2024-03-29T04:58:58.532 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:58 smithi142 ceph-mon[112475]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:58.533 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.525+0000 7f95c37fe700 1 -- 172.21.15.142:0/1649557665 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f95c405bfe0 con 0x7f95cc100dc0 2024-03-29T04:58:58.533 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:58:58.533 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:58:58.533 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:58:58.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:58:58.533 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:58:58.533 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:58:58.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:58:58.534 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:58:58.534 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T04:58:58.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:58:58.534 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:58:58.534 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T04:58:58.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:58:58.534 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:58:58.534 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T04:58:58.534 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:58:58.534 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:58:58.535 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.527+0000 7f95d445c700 1 -- 172.21.15.142:0/1649557665 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f95b806dd60 msgr2=0x7f95b8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:58.535 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.527+0000 7f95d445c700 1 --2- 172.21.15.142:0/1649557665 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f95b806dd60 0x7f95b8070220 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f95bc005fd0 tx=0x7f95bc005ee0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.535 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.527+0000 7f95d445c700 1 -- 172.21.15.142:0/1649557665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95cc100dc0 msgr2=0x7f95cc198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:58.535 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.527+0000 7f95d445c700 1 --2- 172.21.15.142:0/1649557665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95cc100dc0 0x7f95cc198990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f95c400b070 tx=0x7f95c400cdd0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.536 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.527+0000 7f95d445c700 1 -- 172.21.15.142:0/1649557665 shutdown_connections 2024-03-29T04:58:58.536 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.527+0000 7f95d445c700 1 --2- 172.21.15.142:0/1649557665 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f95b806dd60 0x7f95b8070220 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.536 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.527+0000 7f95d445c700 1 --2- 172.21.15.142:0/1649557665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95cc100dc0 0x7f95cc198990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.536 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.527+0000 7f95d445c700 1 --2- 172.21.15.142:0/1649557665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95cc101770 0x7f95cc198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.536 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.527+0000 7f95d445c700 1 -- 172.21.15.142:0/1649557665 >> 172.21.15.142:0/1649557665 conn(0x7f95cc0fc650 msgr2=0x7f95cc0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:58.536 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.528+0000 7f95d445c700 1 -- 172.21.15.142:0/1649557665 shutdown_connections 2024-03-29T04:58:58.536 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.528+0000 7f95d445c700 1 -- 172.21.15.142:0/1649557665 wait complete. 2024-03-29T04:58:58.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/3187298314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b280ffec0 msgr2=0x7f2b281057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:58.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 --2- 172.21.15.142:0/3187298314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b280ffec0 0x7f2b281057b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009910 tx=0x7f2b14009c20 comp rx=0 tx=0).stop 2024-03-29T04:58:58.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/3187298314 shutdown_connections 2024-03-29T04:58:58.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 --2- 172.21.15.142:0/3187298314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b280ffec0 0x7f2b281057b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 --2- 172.21.15.142:0/3187298314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b280ff510 0x7f2b280ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/3187298314 >> 172.21.15.142:0/3187298314 conn(0x7f2b28074e00 msgr2=0x7f2b28075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:58.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/3187298314 shutdown_connections 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/3187298314 wait complete. 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 Processor -- start 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 -- start start 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b280ff510 0x7f2b2819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b280ffec0 0x7f2b2819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2819d110 con 0x7f2b280ffec0 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.631+0000 7f2b2ecbc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b281965c0 con 0x7f2b280ff510 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b27fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b280ffec0 0x7f2b2819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b27fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b280ffec0 0x7f2b2819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59860/0 (socket says 172.21.15.142:59860) 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b27fff700 1 -- 172.21.15.142:0/2560700165 learned_addr learned my addr 172.21.15.142:0/2560700165 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b2ca58700 1 --2- 172.21.15.142:0/2560700165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b280ff510 0x7f2b2819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b27fff700 1 -- 172.21.15.142:0/2560700165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b280ff510 msgr2=0x7f2b2819c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b27fff700 1 --2- 172.21.15.142:0/2560700165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b280ff510 0x7f2b2819c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b27fff700 1 -- 172.21.15.142:0/2560700165 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b140095f0 con 0x7f2b280ffec0 2024-03-29T04:58:58.642 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b27fff700 1 --2- 172.21.15.142:0/2560700165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b280ffec0 0x7f2b2819ca80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b140092e0 tx=0x7f2b1400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:58.642 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b25ffb700 1 -- 172.21.15.142:0/2560700165 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b1401d070 con 0x7f2b280ffec0 2024-03-29T04:58:58.642 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/2560700165 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b28196840 con 0x7f2b280ffec0 2024-03-29T04:58:58.642 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b25ffb700 1 -- 172.21.15.142:0/2560700165 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b1400f460 con 0x7f2b280ffec0 2024-03-29T04:58:58.642 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b25ffb700 1 -- 172.21.15.142:0/2560700165 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b140175d0 con 0x7f2b280ffec0 2024-03-29T04:58:58.642 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.632+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/2560700165 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b28196d30 con 0x7f2b280ffec0 2024-03-29T04:58:58.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.633+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/2560700165 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b28109170 con 0x7f2b280ffec0 2024-03-29T04:58:58.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.634+0000 7f2b25ffb700 1 -- 172.21.15.142:0/2560700165 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f2b1400f5d0 con 0x7f2b280ffec0 2024-03-29T04:58:58.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.634+0000 7f2b25ffb700 1 --2- 172.21.15.142:0/2560700165 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2b1806dd60 0x7f2b18070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:58.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.634+0000 7f2b25ffb700 1 -- 172.21.15.142:0/2560700165 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f2b1408f150 con 0x7f2b280ffec0 2024-03-29T04:58:58.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.634+0000 7f2b2ca58700 1 --2- 172.21.15.142:0/2560700165 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2b1806dd60 0x7f2b18070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:58.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.635+0000 7f2b2ca58700 1 --2- 172.21.15.142:0/2560700165 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2b1806dd60 0x7f2b18070220 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c005950 tx=0x7f2b1c00a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:58.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.639+0000 7f2b25ffb700 1 -- 172.21.15.142:0/2560700165 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2b1405c0a0 con 0x7f2b280ffec0 2024-03-29T04:58:58.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:58 smithi152 ceph-mon[88186]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:58.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.876+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/2560700165 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2b2804ea90 con 0x7f2b280ffec0 2024-03-29T04:58:58.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.876+0000 7f2b25ffb700 1 -- 172.21.15.142:0/2560700165 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f2b1405bc30 con 0x7f2b280ffec0 2024-03-29T04:58:58.884 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T04:58:58.885 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:58:58.885 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:58:58.885 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:58:58.885 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:58:58.885 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:58:58.886 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:58:58.886 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T04:58:58.886 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T04:58:58.886 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:58:58.886 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T04:58:58.886 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:58:58.887 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:58:58.887 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:58:58.887 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:58:58.887 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:58:58.887 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:58:58.887 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:58:58.888 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:58:58.888 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T04:58:58.888 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:58:58.888 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:58:58.888 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:58:58.888 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T04:58:58.889 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T04:58:58.889 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:58:58.889 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:58:58.889 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:58:58.889 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:58:58.889 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:58:58.890 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:58:58.890 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:58:58.890 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:58:58.890 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:58:58.890 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:58:58.891 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:58:58.891 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T04:58:58.891 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:58:58.891 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:58:58.891 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:58:58.891 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:58:58.892 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:58:58.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.878+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/2560700165 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2b1806dd60 msgr2=0x7f2b18070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:58.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.878+0000 7f2b2ecbc700 1 --2- 172.21.15.142:0/2560700165 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2b1806dd60 0x7f2b18070220 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c005950 tx=0x7f2b1c00a300 comp rx=0 tx=0).stop 2024-03-29T04:58:58.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.878+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/2560700165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b280ffec0 msgr2=0x7f2b2819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:58.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.878+0000 7f2b2ecbc700 1 --2- 172.21.15.142:0/2560700165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b280ffec0 0x7f2b2819ca80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b140092e0 tx=0x7f2b1400bd70 comp rx=0 tx=0).stop 2024-03-29T04:58:58.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.878+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/2560700165 shutdown_connections 2024-03-29T04:58:58.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.878+0000 7f2b2ecbc700 1 --2- 172.21.15.142:0/2560700165 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2b1806dd60 0x7f2b18070220 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.878+0000 7f2b2ecbc700 1 --2- 172.21.15.142:0/2560700165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b280ff510 0x7f2b2819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.878+0000 7f2b2ecbc700 1 --2- 172.21.15.142:0/2560700165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b280ffec0 0x7f2b2819ca80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.878+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/2560700165 >> 172.21.15.142:0/2560700165 conn(0x7f2b28074e00 msgr2=0x7f2b280fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:58.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.878+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/2560700165 shutdown_connections 2024-03-29T04:58:58.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.878+0000 7f2b2ecbc700 1 -- 172.21.15.142:0/2560700165 wait complete. 2024-03-29T04:58:58.895 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T04:58:58.992 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 -- 172.21.15.142:0/2267123509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef0100dc0 msgr2=0x7f4ef01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:58.992 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 --2- 172.21.15.142:0/2267123509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef0100dc0 0x7f4ef01011a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009910 tx=0x7f4ee0009c20 comp rx=0 tx=0).stop 2024-03-29T04:58:58.992 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 -- 172.21.15.142:0/2267123509 shutdown_connections 2024-03-29T04:58:58.992 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 --2- 172.21.15.142:0/2267123509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ef0101770 0x7f4ef01057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 --2- 172.21.15.142:0/2267123509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef0100dc0 0x7f4ef01011a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 -- 172.21.15.142:0/2267123509 >> 172.21.15.142:0/2267123509 conn(0x7f4ef00fc650 msgr2=0x7f4ef00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:58.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 -- 172.21.15.142:0/2267123509 shutdown_connections 2024-03-29T04:58:58.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 -- 172.21.15.142:0/2267123509 wait complete. 2024-03-29T04:58:58.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 Processor -- start 2024-03-29T04:58:58.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 -- start start 2024-03-29T04:58:58.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef0100dc0 0x7f4ef0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:58.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ef0101770 0x7f4ef00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:58.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef0073b10 con 0x7f4ef0100dc0 2024-03-29T04:58:58.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.983+0000 7f4ef6996700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef0073c50 con 0x7f4ef0101770 2024-03-29T04:58:58.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4eeffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef0100dc0 0x7f4ef0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:58.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4eeffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef0100dc0 0x7f4ef0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59874/0 (socket says 172.21.15.142:59874) 2024-03-29T04:58:58.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4eeffff700 1 -- 172.21.15.142:0/2592148802 learned_addr learned my addr 172.21.15.142:0/2592148802 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:58.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4ee7fff700 1 --2- 172.21.15.142:0/2592148802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ef0101770 0x7f4ef00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:58.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4eeffff700 1 -- 172.21.15.142:0/2592148802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ef0101770 msgr2=0x7f4ef00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:58.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4eeffff700 1 --2- 172.21.15.142:0/2592148802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ef0101770 0x7f4ef00735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:58.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4eeffff700 1 -- 172.21.15.142:0/2592148802 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00095f0 con 0x7f4ef0100dc0 2024-03-29T04:58:58.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4ee7fff700 1 --2- 172.21.15.142:0/2592148802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ef0101770 0x7f4ef00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:58:58.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4eeffff700 1 --2- 172.21.15.142:0/2592148802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef0100dc0 0x7f4ef0073090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0005f50 tx=0x7f4ee000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:58.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4eedffb700 1 -- 172.21.15.142:0/2592148802 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ee001d070 con 0x7f4ef0100dc0 2024-03-29T04:58:58.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4eedffb700 1 -- 172.21.15.142:0/2592148802 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ee000b610 con 0x7f4ef0100dc0 2024-03-29T04:58:58.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4ef6996700 1 -- 172.21.15.142:0/2592148802 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef0073ed0 con 0x7f4ef0100dc0 2024-03-29T04:58:58.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4eedffb700 1 -- 172.21.15.142:0/2592148802 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ee0022470 con 0x7f4ef0100dc0 2024-03-29T04:58:58.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.984+0000 7f4ef6996700 1 -- 172.21.15.142:0/2592148802 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef01a2c70 con 0x7f4ef0100dc0 2024-03-29T04:58:58.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.985+0000 7f4ef6996700 1 -- 172.21.15.142:0/2592148802 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef0109160 con 0x7f4ef0100dc0 2024-03-29T04:58:58.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.986+0000 7f4eedffb700 1 -- 172.21.15.142:0/2592148802 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f4ee000fca0 con 0x7f4ef0100dc0 2024-03-29T04:58:58.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.986+0000 7f4eedffb700 1 --2- 172.21.15.142:0/2592148802 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4edc06dd60 0x7f4edc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:58.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.986+0000 7f4eedffb700 1 -- 172.21.15.142:0/2592148802 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f4ee008f040 con 0x7f4ef0100dc0 2024-03-29T04:58:58.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.986+0000 7f4ee7fff700 1 --2- 172.21.15.142:0/2592148802 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4edc06dd60 0x7f4edc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:58.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.987+0000 7f4ee7fff700 1 --2- 172.21.15.142:0/2592148802 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4edc06dd60 0x7f4edc070220 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0074880 tx=0x7f4ed8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:58.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:58.991+0000 7f4eedffb700 1 -- 172.21.15.142:0/2592148802 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4ee00584e0 con 0x7f4ef0100dc0 2024-03-29T04:58:59.213 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.206+0000 7f4ef6996700 1 -- 172.21.15.142:0/2592148802 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ef00611d0 con 0x7f4edc06dd60 2024-03-29T04:58:59.215 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.207+0000 7f4eedffb700 1 -- 172.21.15.142:0/2592148802 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4ef00611d0 con 0x7f4edc06dd60 2024-03-29T04:58:59.215 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:58:59.215 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:58:59.215 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:58:59.216 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:58:59.216 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:58:59.216 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:58:59.216 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:58:59.216 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T04:58:59.216 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T04:58:59.216 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T04:58:59.216 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:58:59.216 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T04:58:59.216 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T04:58:59.216 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:58:59.216 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:58:59.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.210+0000 7f4ef6996700 1 -- 172.21.15.142:0/2592148802 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4edc06dd60 msgr2=0x7f4edc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:59.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.210+0000 7f4ef6996700 1 --2- 172.21.15.142:0/2592148802 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4edc06dd60 0x7f4edc070220 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0074880 tx=0x7f4ed8009260 comp rx=0 tx=0).stop 2024-03-29T04:58:59.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.210+0000 7f4ef6996700 1 -- 172.21.15.142:0/2592148802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef0100dc0 msgr2=0x7f4ef0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:59.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.210+0000 7f4ef6996700 1 --2- 172.21.15.142:0/2592148802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef0100dc0 0x7f4ef0073090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0005f50 tx=0x7f4ee000f740 comp rx=0 tx=0).stop 2024-03-29T04:58:59.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.210+0000 7f4ef6996700 1 -- 172.21.15.142:0/2592148802 shutdown_connections 2024-03-29T04:58:59.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.210+0000 7f4ef6996700 1 --2- 172.21.15.142:0/2592148802 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4edc06dd60 0x7f4edc070220 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:59.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.210+0000 7f4ef6996700 1 --2- 172.21.15.142:0/2592148802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef0100dc0 0x7f4ef0073090 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:59.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.210+0000 7f4ef6996700 1 --2- 172.21.15.142:0/2592148802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ef0101770 0x7f4ef00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:59.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.210+0000 7f4ef6996700 1 -- 172.21.15.142:0/2592148802 >> 172.21.15.142:0/2592148802 conn(0x7f4ef00fc650 msgr2=0x7f4ef00fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:59.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.210+0000 7f4ef6996700 1 -- 172.21.15.142:0/2592148802 shutdown_connections 2024-03-29T04:58:59.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.210+0000 7f4ef6996700 1 -- 172.21.15.142:0/2592148802 wait complete. 2024-03-29T04:58:59.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.315+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2399290548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc102630 msgr2=0x7f84bc10ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:59.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.315+0000 7f84c3ea0700 1 --2- 172.21.15.142:0/2399290548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc102630 0x7f84bc10ab20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009910 tx=0x7f84ac009c20 comp rx=0 tx=0).stop 2024-03-29T04:58:59.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.315+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2399290548 shutdown_connections 2024-03-29T04:58:59.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.315+0000 7f84c3ea0700 1 --2- 172.21.15.142:0/2399290548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc102630 0x7f84bc10ab20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:59.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.315+0000 7f84c3ea0700 1 --2- 172.21.15.142:0/2399290548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84bc101c80 0x7f84bc102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:59.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.315+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2399290548 >> 172.21.15.142:0/2399290548 conn(0x7f84bc0fb540 msgr2=0x7f84bc0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:59.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.315+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2399290548 shutdown_connections 2024-03-29T04:58:59.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.315+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2399290548 wait complete. 2024-03-29T04:58:59.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.315+0000 7f84c3ea0700 1 Processor -- start 2024-03-29T04:58:59.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c3ea0700 1 -- start start 2024-03-29T04:58:59.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c3ea0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc101c80 0x7f84bc0ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:59.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c3ea0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84bc102630 0x7f84bc0ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:59.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c3ea0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc1003f0 con 0x7f84bc101c80 2024-03-29T04:58:59.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c3ea0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc100530 con 0x7f84bc102630 2024-03-29T04:58:59.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c1c3c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc101c80 0x7f84bc0ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:59.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c1c3c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc101c80 0x7f84bc0ff970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59906/0 (socket says 172.21.15.142:59906) 2024-03-29T04:58:59.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c1c3c700 1 -- 172.21.15.142:0/2110038673 learned_addr learned my addr 172.21.15.142:0/2110038673 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:58:59.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c143b700 1 --2- 172.21.15.142:0/2110038673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84bc102630 0x7f84bc0ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:59.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c1c3c700 1 -- 172.21.15.142:0/2110038673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84bc102630 msgr2=0x7f84bc0ffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:59.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c1c3c700 1 --2- 172.21.15.142:0/2110038673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84bc102630 0x7f84bc0ffeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:59.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c1c3c700 1 -- 172.21.15.142:0/2110038673 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac0095f0 con 0x7f84bc101c80 2024-03-29T04:58:59.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c1c3c700 1 --2- 172.21.15.142:0/2110038673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc101c80 0x7f84bc0ff970 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f84b40060b0 tx=0x7f84b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:59.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84b2ffd700 1 -- 172.21.15.142:0/2110038673 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b400dda0 con 0x7f84bc101c80 2024-03-29T04:58:59.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2110038673 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc1a2920 con 0x7f84bc101c80 2024-03-29T04:58:59.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84b2ffd700 1 -- 172.21.15.142:0/2110038673 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84b400f040 con 0x7f84bc101c80 2024-03-29T04:58:59.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.316+0000 7f84b2ffd700 1 -- 172.21.15.142:0/2110038673 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b4014790 con 0x7f84bc101c80 2024-03-29T04:58:59.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.317+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2110038673 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc1a2d30 con 0x7f84bc101c80 2024-03-29T04:58:59.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.317+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2110038673 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84bc108220 con 0x7f84bc101c80 2024-03-29T04:58:59.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.318+0000 7f84b2ffd700 1 -- 172.21.15.142:0/2110038673 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f84b4014980 con 0x7f84bc101c80 2024-03-29T04:58:59.333 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.318+0000 7f84b2ffd700 1 --2- 172.21.15.142:0/2110038673 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f84a806dd60 0x7f84a8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:58:59.333 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.318+0000 7f84b2ffd700 1 -- 172.21.15.142:0/2110038673 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f84b408d720 con 0x7f84bc101c80 2024-03-29T04:58:59.333 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.318+0000 7f84c143b700 1 --2- 172.21.15.142:0/2110038673 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f84a806dd60 0x7f84a8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:58:59.333 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.319+0000 7f84c143b700 1 --2- 172.21.15.142:0/2110038673 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f84a806dd60 0x7f84a8070220 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84bc101160 tx=0x7f84ac01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:58:59.333 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.323+0000 7f84b2ffd700 1 -- 172.21.15.142:0/2110038673 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f84b4056bf0 con 0x7f84bc101c80 2024-03-29T04:58:59.610 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.602+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2110038673 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f84bc04ea90 con 0x7f84bc101c80 2024-03-29T04:58:59.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:59 smithi142 ceph-mon[112475]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:59.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:59 smithi142 ceph-mon[112475]: pgmap v320: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:59.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:59 smithi142 ceph-mon[112475]: from='client.34634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:59.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:59 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1649557665' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:58:59.611 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:58:59 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2560700165' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:58:59.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.603+0000 7f84b2ffd700 1 -- 172.21.15.142:0/2110038673 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f84b405a2b0 con 0x7f84bc101c80 2024-03-29T04:58:59.611 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T04:58:59.612 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T04:58:59.612 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T04:58:59.612 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T04:58:59.612 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T04:58:59.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.605+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2110038673 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f84a806dd60 msgr2=0x7f84a8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:59.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.605+0000 7f84c3ea0700 1 --2- 172.21.15.142:0/2110038673 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f84a806dd60 0x7f84a8070220 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84bc101160 tx=0x7f84ac01a040 comp rx=0 tx=0).stop 2024-03-29T04:58:59.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.605+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2110038673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc101c80 msgr2=0x7f84bc0ff970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:58:59.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.605+0000 7f84c3ea0700 1 --2- 172.21.15.142:0/2110038673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc101c80 0x7f84bc0ff970 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f84b40060b0 tx=0x7f84b400d3b0 comp rx=0 tx=0).stop 2024-03-29T04:58:59.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.605+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2110038673 shutdown_connections 2024-03-29T04:58:59.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.605+0000 7f84c3ea0700 1 --2- 172.21.15.142:0/2110038673 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f84a806dd60 0x7f84a8070220 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:59.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.605+0000 7f84c3ea0700 1 --2- 172.21.15.142:0/2110038673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc101c80 0x7f84bc0ff970 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:59.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.605+0000 7f84c3ea0700 1 --2- 172.21.15.142:0/2110038673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84bc102630 0x7f84bc0ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:58:59.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.605+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2110038673 >> 172.21.15.142:0/2110038673 conn(0x7f84bc0fb540 msgr2=0x7f84bc105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:58:59.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.605+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2110038673 shutdown_connections 2024-03-29T04:58:59.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:58:59.605+0000 7f84c3ea0700 1 -- 172.21.15.142:0/2110038673 wait complete. 2024-03-29T04:58:59.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:59 smithi152 ceph-mon[88186]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:59.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:59 smithi152 ceph-mon[88186]: pgmap v320: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:58:59.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:59 smithi152 ceph-mon[88186]: from='client.34634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:58:59.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:59 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1649557665' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:58:59.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:58:59 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2560700165' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:59:00.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:00 smithi152 ceph-mon[88186]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:59:00.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:59:00.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:00 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2110038673' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:59:00.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:00 smithi142 ceph-mon[112475]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:59:00.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:59:00.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:00 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2110038673' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:59:01.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:01 smithi152 ceph-mon[88186]: pgmap v321: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:01.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:01 smithi142 ceph-mon[112475]: pgmap v321: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:03.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:03 smithi152 ceph-mon[88186]: pgmap v322: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:03.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:03 smithi142 ceph-mon[112475]: pgmap v322: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:05 smithi152 ceph-mon[88186]: pgmap v323: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:05.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:05 smithi142 ceph-mon[112475]: pgmap v323: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:07.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:07 smithi152 ceph-mon[88186]: pgmap v324: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:07.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:07 smithi142 ceph-mon[112475]: pgmap v324: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:09.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:09 smithi152 ceph-mon[88186]: pgmap v325: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:09.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:09 smithi142 ceph-mon[112475]: pgmap v325: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:11.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:11 smithi152 ceph-mon[88186]: pgmap v326: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:11.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:11 smithi142 ceph-mon[112475]: pgmap v326: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:13.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:13 smithi152 ceph-mon[88186]: pgmap v327: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:13.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:13 smithi142 ceph-mon[112475]: pgmap v327: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:15.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:15 smithi152 ceph-mon[88186]: pgmap v328: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:15.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:59:15.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:15 smithi142 ceph-mon[112475]: pgmap v328: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:15.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:59:17.788 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:17 smithi142 ceph-mon[112475]: pgmap v329: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:17.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:17 smithi152 ceph-mon[88186]: pgmap v329: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:19.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:19 smithi152 ceph-mon[88186]: pgmap v330: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:19.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:19 smithi142 ceph-mon[112475]: pgmap v330: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:21.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:21 smithi152 ceph-mon[88186]: pgmap v331: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:21.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:21 smithi142 ceph-mon[112475]: pgmap v331: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:23.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:23 smithi152 ceph-mon[88186]: pgmap v332: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:23.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:23 smithi142 ceph-mon[112475]: pgmap v332: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:25.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:25 smithi152 ceph-mon[88186]: pgmap v333: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:25.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:25 smithi142 ceph-mon[112475]: pgmap v333: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:27.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:27 smithi152 ceph-mon[88186]: pgmap v334: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:27.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:27 smithi142 ceph-mon[112475]: pgmap v334: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:29.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.722+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3374141587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c5c068670 msgr2=0x7f7c5c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:29.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.722+0000 7f7c61f6e700 1 --2- 172.21.15.142:0/3374141587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c5c068670 0x7f7c5c068a50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00b0a0 tx=0x7f7c4c00b3b0 comp rx=0 tx=0).stop 2024-03-29T04:59:29.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.722+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3374141587 shutdown_connections 2024-03-29T04:59:29.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.722+0000 7f7c61f6e700 1 --2- 172.21.15.142:0/3374141587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c5c069020 0x7f7c5c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:29.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.722+0000 7f7c61f6e700 1 --2- 172.21.15.142:0/3374141587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c5c068670 0x7f7c5c068a50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:29.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.722+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3374141587 >> 172.21.15.142:0/3374141587 conn(0x7f7c5c0755a0 msgr2=0x7f7c5c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:29.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.722+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3374141587 shutdown_connections 2024-03-29T04:59:29.731 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:29 smithi142 ceph-mon[112475]: pgmap v335: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:29.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.722+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3374141587 wait complete. 2024-03-29T04:59:29.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.722+0000 7f7c61f6e700 1 Processor -- start 2024-03-29T04:59:29.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c61f6e700 1 -- start start 2024-03-29T04:59:29.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c61f6e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c5c068670 0x7f7c5c0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:29.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c61f6e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c5c069020 0x7f7c5c1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:29.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c61f6e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c100720 con 0x7f7c5c069020 2024-03-29T04:59:29.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c61f6e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c100860 con 0x7f7c5c068670 2024-03-29T04:59:29.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c5affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c5c069020 0x7f7c5c1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:29.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c5affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c5c069020 0x7f7c5c1001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42448/0 (socket says 172.21.15.142:42448) 2024-03-29T04:59:29.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c5affd700 1 -- 172.21.15.142:0/3147320034 learned_addr learned my addr 172.21.15.142:0/3147320034 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:59:29.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c5b7fe700 1 --2- 172.21.15.142:0/3147320034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c5c068670 0x7f7c5c0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:29.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c5affd700 1 -- 172.21.15.142:0/3147320034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c5c068670 msgr2=0x7f7c5c0ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:29.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c5affd700 1 --2- 172.21.15.142:0/3147320034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c5c068670 0x7f7c5c0ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:29.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c5affd700 1 -- 172.21.15.142:0/3147320034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c009d00 con 0x7f7c5c069020 2024-03-29T04:59:29.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.723+0000 7f7c5affd700 1 --2- 172.21.15.142:0/3147320034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c5c069020 0x7f7c5c1001e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000b480 tx=0x7f7c5000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:29.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.724+0000 7f7c58ff9700 1 -- 172.21.15.142:0/3147320034 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c500118c0 con 0x7f7c5c069020 2024-03-29T04:59:29.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.724+0000 7f7c58ff9700 1 -- 172.21.15.142:0/3147320034 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c50010380 con 0x7f7c5c069020 2024-03-29T04:59:29.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.724+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3147320034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5c1a2ac0 con 0x7f7c5c069020 2024-03-29T04:59:29.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.724+0000 7f7c58ff9700 1 -- 172.21.15.142:0/3147320034 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c5000f4f0 con 0x7f7c5c069020 2024-03-29T04:59:29.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.724+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3147320034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5c1a2ed0 con 0x7f7c5c069020 2024-03-29T04:59:29.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.725+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3147320034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5c109130 con 0x7f7c5c069020 2024-03-29T04:59:29.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.725+0000 7f7c58ff9700 1 -- 172.21.15.142:0/3147320034 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f7c500104f0 con 0x7f7c5c069020 2024-03-29T04:59:29.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.726+0000 7f7c58ff9700 1 --2- 172.21.15.142:0/3147320034 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7c4406dd60 0x7f7c44070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:29.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.726+0000 7f7c58ff9700 1 -- 172.21.15.142:0/3147320034 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f7c5008d8f0 con 0x7f7c5c069020 2024-03-29T04:59:29.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.726+0000 7f7c5b7fe700 1 --2- 172.21.15.142:0/3147320034 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7c4406dd60 0x7f7c44070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:29.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.726+0000 7f7c5b7fe700 1 --2- 172.21.15.142:0/3147320034 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7c4406dd60 0x7f7c44070220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00b830 tx=0x7f7c4c0097c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:29.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.730+0000 7f7c58ff9700 1 -- 172.21.15.142:0/3147320034 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7c50056d10 con 0x7f7c5c069020 2024-03-29T04:59:29.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:29 smithi152 ceph-mon[88186]: pgmap v335: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:29.950 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.942+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3147320034 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c5c02cce0 con 0x7f7c4406dd60 2024-03-29T04:59:29.951 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.943+0000 7f7c58ff9700 1 -- 172.21.15.142:0/3147320034 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7c5c02cce0 con 0x7f7c4406dd60 2024-03-29T04:59:29.953 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.945+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3147320034 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7c4406dd60 msgr2=0x7f7c44070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:29.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.945+0000 7f7c61f6e700 1 --2- 172.21.15.142:0/3147320034 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7c4406dd60 0x7f7c44070220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00b830 tx=0x7f7c4c0097c0 comp rx=0 tx=0).stop 2024-03-29T04:59:29.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.945+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3147320034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c5c069020 msgr2=0x7f7c5c1001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:29.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.945+0000 7f7c61f6e700 1 --2- 172.21.15.142:0/3147320034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c5c069020 0x7f7c5c1001e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000b480 tx=0x7f7c5000b840 comp rx=0 tx=0).stop 2024-03-29T04:59:29.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.946+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3147320034 shutdown_connections 2024-03-29T04:59:29.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.946+0000 7f7c61f6e700 1 --2- 172.21.15.142:0/3147320034 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7c4406dd60 0x7f7c44070220 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:29.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.946+0000 7f7c61f6e700 1 --2- 172.21.15.142:0/3147320034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c5c068670 0x7f7c5c0ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:29.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.946+0000 7f7c61f6e700 1 --2- 172.21.15.142:0/3147320034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c5c069020 0x7f7c5c1001e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:29.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.946+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3147320034 >> 172.21.15.142:0/3147320034 conn(0x7f7c5c0755a0 msgr2=0x7f7c5c0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:29.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.946+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3147320034 shutdown_connections 2024-03-29T04:59:29.954 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:29.946+0000 7f7c61f6e700 1 -- 172.21.15.142:0/3147320034 wait complete. 2024-03-29T04:59:29.964 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T04:59:30.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.050+0000 7f4ba0529700 1 -- 172.21.15.142:0/1317827713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b980ffa00 msgr2=0x7f4b980ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:30.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.050+0000 7f4ba0529700 1 --2- 172.21.15.142:0/1317827713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b980ffa00 0x7f4b980ffe80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009910 tx=0x7f4b88009c20 comp rx=0 tx=0).stop 2024-03-29T04:59:30.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.050+0000 7f4ba0529700 1 -- 172.21.15.142:0/1317827713 shutdown_connections 2024-03-29T04:59:30.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.050+0000 7f4ba0529700 1 --2- 172.21.15.142:0/1317827713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b980ffa00 0x7f4b980ffe80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.050+0000 7f4ba0529700 1 --2- 172.21.15.142:0/1317827713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b98105a00 0x7f4b98105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.050+0000 7f4ba0529700 1 -- 172.21.15.142:0/1317827713 >> 172.21.15.142:0/1317827713 conn(0x7f4b980fb4f0 msgr2=0x7f4b980fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:30.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.050+0000 7f4ba0529700 1 -- 172.21.15.142:0/1317827713 shutdown_connections 2024-03-29T04:59:30.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.050+0000 7f4ba0529700 1 -- 172.21.15.142:0/1317827713 wait complete. 2024-03-29T04:59:30.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.050+0000 7f4ba0529700 1 Processor -- start 2024-03-29T04:59:30.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.050+0000 7f4ba0529700 1 -- start start 2024-03-29T04:59:30.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.051+0000 7f4ba0529700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b980ffa00 0x7f4b98197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:30.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.051+0000 7f4ba0529700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b98105a00 0x7f4b98197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:30.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.051+0000 7f4ba0529700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b98198060 con 0x7f4b980ffa00 2024-03-29T04:59:30.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.051+0000 7f4ba0529700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b981981d0 con 0x7f4b98105a00 2024-03-29T04:59:30.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.051+0000 7f4b9e2c5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b980ffa00 0x7f4b98197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:30.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.051+0000 7f4b9e2c5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b980ffa00 0x7f4b98197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42468/0 (socket says 172.21.15.142:42468) 2024-03-29T04:59:30.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.051+0000 7f4b9e2c5700 1 -- 172.21.15.142:0/311979650 learned_addr learned my addr 172.21.15.142:0/311979650 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:59:30.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.051+0000 7f4b9e2c5700 1 -- 172.21.15.142:0/311979650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b98105a00 msgr2=0x7f4b98197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:30.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.051+0000 7f4b9e2c5700 1 --2- 172.21.15.142:0/311979650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b98105a00 0x7f4b98197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.051+0000 7f4b9e2c5700 1 -- 172.21.15.142:0/311979650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b880095f0 con 0x7f4b980ffa00 2024-03-29T04:59:30.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.051+0000 7f4b9e2c5700 1 --2- 172.21.15.142:0/311979650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b980ffa00 0x7f4b98197500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400e920 tx=0x7f4b9400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:30.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.051+0000 7f4b8f7fe700 1 -- 172.21.15.142:0/311979650 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b94009790 con 0x7f4b980ffa00 2024-03-29T04:59:30.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.052+0000 7f4b8f7fe700 1 -- 172.21.15.142:0/311979650 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b9400cba0 con 0x7f4b980ffa00 2024-03-29T04:59:30.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.052+0000 7f4ba0529700 1 -- 172.21.15.142:0/311979650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b981a0570 con 0x7f4b980ffa00 2024-03-29T04:59:30.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.052+0000 7f4b8f7fe700 1 -- 172.21.15.142:0/311979650 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b94018650 con 0x7f4b980ffa00 2024-03-29T04:59:30.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.052+0000 7f4ba0529700 1 -- 172.21.15.142:0/311979650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b981a0a60 con 0x7f4b980ffa00 2024-03-29T04:59:30.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.053+0000 7f4b8d7fa700 1 -- 172.21.15.142:0/311979650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b800052f0 con 0x7f4b980ffa00 2024-03-29T04:59:30.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.053+0000 7f4b8f7fe700 1 -- 172.21.15.142:0/311979650 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f4b9400cd10 con 0x7f4b980ffa00 2024-03-29T04:59:30.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.054+0000 7f4b8f7fe700 1 --2- 172.21.15.142:0/311979650 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b8406dd60 0x7f4b84070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:30.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.054+0000 7f4b8f7fe700 1 -- 172.21.15.142:0/311979650 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f4b94014070 con 0x7f4b980ffa00 2024-03-29T04:59:30.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.054+0000 7f4b9dac4700 1 --2- 172.21.15.142:0/311979650 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b8406dd60 0x7f4b84070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:30.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.054+0000 7f4b9dac4700 1 --2- 172.21.15.142:0/311979650 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b8406dd60 0x7f4b84070220 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009fd0 tx=0x7f4b88000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:30.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.058+0000 7f4b8f7fe700 1 -- 172.21.15.142:0/311979650 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4b94053280 con 0x7f4b980ffa00 2024-03-29T04:59:30.277 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.269+0000 7f4b8d7fa700 1 -- 172.21.15.142:0/311979650 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b80000bc0 con 0x7f4b8406dd60 2024-03-29T04:59:30.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.270+0000 7f4b8f7fe700 1 -- 172.21.15.142:0/311979650 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4b80000bc0 con 0x7f4b8406dd60 2024-03-29T04:59:30.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.272+0000 7f4b8d7fa700 1 -- 172.21.15.142:0/311979650 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b8406dd60 msgr2=0x7f4b84070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:30.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.272+0000 7f4b8d7fa700 1 --2- 172.21.15.142:0/311979650 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b8406dd60 0x7f4b84070220 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009fd0 tx=0x7f4b88000bc0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.272+0000 7f4b8d7fa700 1 -- 172.21.15.142:0/311979650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b980ffa00 msgr2=0x7f4b98197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:30.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.272+0000 7f4b8d7fa700 1 --2- 172.21.15.142:0/311979650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b980ffa00 0x7f4b98197500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400e920 tx=0x7f4b9400ece0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.272+0000 7f4b8d7fa700 1 -- 172.21.15.142:0/311979650 shutdown_connections 2024-03-29T04:59:30.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.272+0000 7f4b8d7fa700 1 --2- 172.21.15.142:0/311979650 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b8406dd60 0x7f4b84070220 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.272+0000 7f4b8d7fa700 1 --2- 172.21.15.142:0/311979650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b980ffa00 0x7f4b98197500 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.272+0000 7f4b8d7fa700 1 --2- 172.21.15.142:0/311979650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b98105a00 0x7f4b98197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.272+0000 7f4b8d7fa700 1 -- 172.21.15.142:0/311979650 >> 172.21.15.142:0/311979650 conn(0x7f4b980fb4f0 msgr2=0x7f4b980fbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:30.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.272+0000 7f4b8d7fa700 1 -- 172.21.15.142:0/311979650 shutdown_connections 2024-03-29T04:59:30.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.272+0000 7f4b8d7fa700 1 -- 172.21.15.142:0/311979650 wait complete. 2024-03-29T04:59:30.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.376+0000 7fd06ce4b700 1 -- 172.21.15.142:0/1756909534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0680ffaf0 msgr2=0x7fd0680ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:30.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.376+0000 7fd06ce4b700 1 --2- 172.21.15.142:0/1756909534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0680ffaf0 0x7fd0680ffed0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd058009910 tx=0x7fd058009c20 comp rx=0 tx=0).stop 2024-03-29T04:59:30.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.376+0000 7fd06ce4b700 1 -- 172.21.15.142:0/1756909534 shutdown_connections 2024-03-29T04:59:30.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.376+0000 7fd06ce4b700 1 --2- 172.21.15.142:0/1756909534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd068100410 0x7fd0681048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.376+0000 7fd06ce4b700 1 --2- 172.21.15.142:0/1756909534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0680ffaf0 0x7fd0680ffed0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.376+0000 7fd06ce4b700 1 -- 172.21.15.142:0/1756909534 >> 172.21.15.142:0/1756909534 conn(0x7fd0680fb550 msgr2=0x7fd0680fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:30.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.376+0000 7fd06ce4b700 1 -- 172.21.15.142:0/1756909534 shutdown_connections 2024-03-29T04:59:30.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.376+0000 7fd06ce4b700 1 -- 172.21.15.142:0/1756909534 wait complete. 2024-03-29T04:59:30.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.376+0000 7fd06ce4b700 1 Processor -- start 2024-03-29T04:59:30.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd06ce4b700 1 -- start start 2024-03-29T04:59:30.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd06ce4b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0680ffaf0 0x7fd06819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd06ce4b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd068100410 0x7fd06819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd06ce4b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06819db20 con 0x7fd068100410 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd06ce4b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0681a18b0 con 0x7fd0680ffaf0 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd05ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd068100410 0x7fd06819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd05ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd068100410 0x7fd06819d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42482/0 (socket says 172.21.15.142:42482) 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd05ffff700 1 -- 172.21.15.142:0/2087276268 learned_addr learned my addr 172.21.15.142:0/2087276268 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd06659c700 1 --2- 172.21.15.142:0/2087276268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0680ffaf0 0x7fd06819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd05ffff700 1 -- 172.21.15.142:0/2087276268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0680ffaf0 msgr2=0x7fd06819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd05ffff700 1 --2- 172.21.15.142:0/2087276268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0680ffaf0 0x7fd06819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd05ffff700 1 -- 172.21.15.142:0/2087276268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0580095f0 con 0x7fd068100410 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd06659c700 1 --2- 172.21.15.142:0/2087276268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0680ffaf0 0x7fd06819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd05ffff700 1 --2- 172.21.15.142:0/2087276268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd068100410 0x7fd06819d440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd05000b480 tx=0x7fd05000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:30.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd05f7fe700 1 -- 172.21.15.142:0/2087276268 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0500118c0 con 0x7fd068100410 2024-03-29T04:59:30.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd06ce4b700 1 -- 172.21.15.142:0/2087276268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0681a1b90 con 0x7fd068100410 2024-03-29T04:59:30.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.377+0000 7fd05f7fe700 1 -- 172.21.15.142:0/2087276268 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd050010380 con 0x7fd068100410 2024-03-29T04:59:30.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.378+0000 7fd05f7fe700 1 -- 172.21.15.142:0/2087276268 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd05000f4f0 con 0x7fd068100410 2024-03-29T04:59:30.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.378+0000 7fd06ce4b700 1 -- 172.21.15.142:0/2087276268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0681a20e0 con 0x7fd068100410 2024-03-29T04:59:30.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.378+0000 7fd06ce4b700 1 -- 172.21.15.142:0/2087276268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd068102010 con 0x7fd068100410 2024-03-29T04:59:30.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.379+0000 7fd05f7fe700 1 -- 172.21.15.142:0/2087276268 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fd0500104f0 con 0x7fd068100410 2024-03-29T04:59:30.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.379+0000 7fd05f7fe700 1 --2- 172.21.15.142:0/2087276268 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd05406dd60 0x7fd054070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:30.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.380+0000 7fd05f7fe700 1 -- 172.21.15.142:0/2087276268 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd05008d8f0 con 0x7fd068100410 2024-03-29T04:59:30.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.380+0000 7fd06659c700 1 --2- 172.21.15.142:0/2087276268 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd05406dd60 0x7fd054070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:30.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.380+0000 7fd06659c700 1 --2- 172.21.15.142:0/2087276268 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd05406dd60 0x7fd054070220 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd05800b5c0 tx=0x7fd058005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:30.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.384+0000 7fd05f7fe700 1 -- 172.21.15.142:0/2087276268 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd050056d10 con 0x7fd068100410 2024-03-29T04:59:30.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.590+0000 7fd06ce4b700 1 -- 172.21.15.142:0/2087276268 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0681a2440 con 0x7fd05406dd60 2024-03-29T04:59:30.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.601+0000 7fd05f7fe700 1 -- 172.21.15.142:0/2087276268 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd0681a2440 con 0x7fd05406dd60 2024-03-29T04:59:30.609 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T04:59:30.609 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (11m) 4m ago 22m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T04:59:30.609 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (23m) 4m ago 23m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T04:59:30.609 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (22m) 3m ago 22m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T04:59:30.609 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (8m) 4m ago 23m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T04:59:30.609 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (8m) 3m ago 21m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T04:59:30.609 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (10m) 4m ago 22m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T04:59:30.609 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (4m) 4m ago 17m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T04:59:30.609 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (4m) 4m ago 17m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (3m) 3m ago 17m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (4m) 3m ago 17m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (12m) 4m ago 24m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (12m) 3m ago 21m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (9m) 4m ago 24m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (9m) 3m ago 21m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (12m) 4m ago 22m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (11m) 3m ago 21m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (8m) 4m ago 21m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (7m) 4m ago 20m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (6m) 4m ago 20m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (6m) 3m ago 19m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T04:59:30.610 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (5m) 3m ago 19m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T04:59:30.611 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (5m) 3m ago 18m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T04:59:30.611 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (11m) 4m ago 22m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T04:59:30.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.603+0000 7fd06ce4b700 1 -- 172.21.15.142:0/2087276268 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd05406dd60 msgr2=0x7fd054070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:30.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.603+0000 7fd06ce4b700 1 --2- 172.21.15.142:0/2087276268 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd05406dd60 0x7fd054070220 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd05800b5c0 tx=0x7fd058005480 comp rx=0 tx=0).stop 2024-03-29T04:59:30.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.603+0000 7fd06ce4b700 1 -- 172.21.15.142:0/2087276268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd068100410 msgr2=0x7fd06819d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:30.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.603+0000 7fd06ce4b700 1 --2- 172.21.15.142:0/2087276268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd068100410 0x7fd06819d440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd05000b480 tx=0x7fd05000b840 comp rx=0 tx=0).stop 2024-03-29T04:59:30.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.603+0000 7fd06ce4b700 1 -- 172.21.15.142:0/2087276268 shutdown_connections 2024-03-29T04:59:30.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.603+0000 7fd06ce4b700 1 --2- 172.21.15.142:0/2087276268 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd05406dd60 0x7fd054070220 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.603+0000 7fd06ce4b700 1 --2- 172.21.15.142:0/2087276268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0680ffaf0 0x7fd06819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.603+0000 7fd06ce4b700 1 --2- 172.21.15.142:0/2087276268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd068100410 0x7fd06819d440 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.603+0000 7fd06ce4b700 1 -- 172.21.15.142:0/2087276268 >> 172.21.15.142:0/2087276268 conn(0x7fd0680fb550 msgr2=0x7fd068110f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:30.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.603+0000 7fd06ce4b700 1 -- 172.21.15.142:0/2087276268 shutdown_connections 2024-03-29T04:59:30.612 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.603+0000 7fd06ce4b700 1 -- 172.21.15.142:0/2087276268 wait complete. 2024-03-29T04:59:30.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.705+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2339957160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c0c105a40 msgr2=0x7f7c0c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:30.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.705+0000 7f7c12d3a700 1 --2- 172.21.15.142:0/2339957160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c0c105a40 0x7f7c0c105e20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009910 tx=0x7f7c00009c20 comp rx=0 tx=0).stop 2024-03-29T04:59:30.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.705+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2339957160 shutdown_connections 2024-03-29T04:59:30.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.705+0000 7f7c12d3a700 1 --2- 172.21.15.142:0/2339957160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c0ffa40 0x7f7c0c0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.705+0000 7f7c12d3a700 1 --2- 172.21.15.142:0/2339957160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c0c105a40 0x7f7c0c105e20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.705+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2339957160 >> 172.21.15.142:0/2339957160 conn(0x7f7c0c0fb570 msgr2=0x7f7c0c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:30.714 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:59:30.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.705+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2339957160 shutdown_connections 2024-03-29T04:59:30.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.705+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2339957160 wait complete. 2024-03-29T04:59:30.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.706+0000 7f7c12d3a700 1 Processor -- start 2024-03-29T04:59:30.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.706+0000 7f7c12d3a700 1 -- start start 2024-03-29T04:59:30.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.706+0000 7f7c12d3a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c0ffa40 0x7f7c0c19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:30.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.706+0000 7f7c12d3a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c0c105a40 0x7f7c0c19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:30.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.706+0000 7f7c12d3a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c19d080 con 0x7f7c0c105a40 2024-03-29T04:59:30.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.706+0000 7f7c12d3a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c196540 con 0x7f7c0c0ffa40 2024-03-29T04:59:30.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.706+0000 7f7c0bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c0c105a40 0x7f7c0c19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.706+0000 7f7c0bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c0c105a40 0x7f7c0c19c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42498/0 (socket says 172.21.15.142:42498) 2024-03-29T04:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.706+0000 7f7c0bfff700 1 -- 172.21.15.142:0/2290623275 learned_addr learned my addr 172.21.15.142:0/2290623275 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.706+0000 7f7c10ad6700 1 --2- 172.21.15.142:0/2290623275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c0ffa40 0x7f7c0c19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.707+0000 7f7c0bfff700 1 -- 172.21.15.142:0/2290623275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c0ffa40 msgr2=0x7f7c0c19c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.707+0000 7f7c0bfff700 1 --2- 172.21.15.142:0/2290623275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c0ffa40 0x7f7c0c19c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.707+0000 7f7c0bfff700 1 -- 172.21.15.142:0/2290623275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c000095f0 con 0x7f7c0c105a40 2024-03-29T04:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.707+0000 7f7c0bfff700 1 --2- 172.21.15.142:0/2290623275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c0c105a40 0x7f7c0c19c9a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800d900 tx=0x7f7bf800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.707+0000 7f7c09ffb700 1 -- 172.21.15.142:0/2290623275 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf8004a00 con 0x7f7c0c105a40 2024-03-29T04:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.707+0000 7f7c09ffb700 1 -- 172.21.15.142:0/2290623275 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bf80054e0 con 0x7f7c0c105a40 2024-03-29T04:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.708+0000 7f7c09ffb700 1 -- 172.21.15.142:0/2290623275 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf8009d40 con 0x7f7c0c105a40 2024-03-29T04:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.708+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2290623275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0c196820 con 0x7f7c0c105a40 2024-03-29T04:59:30.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.708+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2290623275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0c196d20 con 0x7f7c0c105a40 2024-03-29T04:59:30.718 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.709+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2290623275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0c109050 con 0x7f7c0c105a40 2024-03-29T04:59:30.718 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.710+0000 7f7c09ffb700 1 -- 172.21.15.142:0/2290623275 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f7bf8005000 con 0x7f7c0c105a40 2024-03-29T04:59:30.718 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.710+0000 7f7c09ffb700 1 --2- 172.21.15.142:0/2290623275 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7bfc06dd60 0x7f7bfc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:30.718 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.710+0000 7f7c09ffb700 1 -- 172.21.15.142:0/2290623275 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f7bf808d5c0 con 0x7f7c0c105a40 2024-03-29T04:59:30.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.710+0000 7f7c10ad6700 1 --2- 172.21.15.142:0/2290623275 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7bfc06dd60 0x7f7bfc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:30.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.711+0000 7f7c10ad6700 1 --2- 172.21.15.142:0/2290623275 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7bfc06dd60 0x7f7bfc070220 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000b5c0 tx=0x7f7c0001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:30.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.715+0000 7f7c09ffb700 1 -- 172.21.15.142:0/2290623275 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7bf805a590 con 0x7f7c0c105a40 2024-03-29T04:59:30.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:59:31.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.991+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2290623275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7c0c04ea90 con 0x7f7c0c105a40 2024-03-29T04:59:31.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.991+0000 7f7c09ffb700 1 -- 172.21.15.142:0/2290623275 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7bf805a120 con 0x7f7c0c105a40 2024-03-29T04:59:31.000 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:59:31.001 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T04:59:31.001 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:59:31.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:59:31.001 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T04:59:31.001 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T04:59:31.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:59:31.002 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T04:59:31.004 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T04:59:31.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:59:31.004 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T04:59:31.004 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T04:59:31.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T04:59:31.004 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T04:59:31.005 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T04:59:31.005 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T04:59:31.005 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:59:31.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.994+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2290623275 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7bfc06dd60 msgr2=0x7f7bfc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:31.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.994+0000 7f7c12d3a700 1 --2- 172.21.15.142:0/2290623275 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7bfc06dd60 0x7f7bfc070220 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000b5c0 tx=0x7f7c0001a040 comp rx=0 tx=0).stop 2024-03-29T04:59:31.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.994+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2290623275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c0c105a40 msgr2=0x7f7c0c19c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:31.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.994+0000 7f7c12d3a700 1 --2- 172.21.15.142:0/2290623275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c0c105a40 0x7f7c0c19c9a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800d900 tx=0x7f7bf800dc10 comp rx=0 tx=0).stop 2024-03-29T04:59:31.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.994+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2290623275 shutdown_connections 2024-03-29T04:59:31.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.994+0000 7f7c12d3a700 1 --2- 172.21.15.142:0/2290623275 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7bfc06dd60 0x7f7bfc070220 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.994+0000 7f7c12d3a700 1 --2- 172.21.15.142:0/2290623275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c0ffa40 0x7f7c0c19c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.994+0000 7f7c12d3a700 1 --2- 172.21.15.142:0/2290623275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c0c105a40 0x7f7c0c19c9a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.994+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2290623275 >> 172.21.15.142:0/2290623275 conn(0x7f7c0c0fb570 msgr2=0x7f7c0c068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:31.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.994+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2290623275 shutdown_connections 2024-03-29T04:59:31.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:30.994+0000 7f7c12d3a700 1 -- 172.21.15.142:0/2290623275 wait complete. 2024-03-29T04:59:31.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.097+0000 7fc056082700 1 -- 172.21.15.142:0/744983109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0500ffbd0 msgr2=0x7fc050100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:31.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.097+0000 7fc056082700 1 --2- 172.21.15.142:0/744983109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0500ffbd0 0x7fc050100050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc044009910 tx=0x7fc044009c20 comp rx=0 tx=0).stop 2024-03-29T04:59:31.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.097+0000 7fc056082700 1 -- 172.21.15.142:0/744983109 shutdown_connections 2024-03-29T04:59:31.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.097+0000 7fc056082700 1 --2- 172.21.15.142:0/744983109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0500ffbd0 0x7fc050100050 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.107 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.097+0000 7fc056082700 1 --2- 172.21.15.142:0/744983109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc050105a40 0x7fc050105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.107 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.097+0000 7fc056082700 1 -- 172.21.15.142:0/744983109 >> 172.21.15.142:0/744983109 conn(0x7fc0500755a0 msgr2=0x7fc0500759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:31.107 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.097+0000 7fc056082700 1 -- 172.21.15.142:0/744983109 shutdown_connections 2024-03-29T04:59:31.107 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.097+0000 7fc056082700 1 -- 172.21.15.142:0/744983109 wait complete. 2024-03-29T04:59:31.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.098+0000 7fc056082700 1 Processor -- start 2024-03-29T04:59:31.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.098+0000 7fc056082700 1 -- start start 2024-03-29T04:59:31.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.098+0000 7fc056082700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0500ffbd0 0x7fc05019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:31.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.098+0000 7fc056082700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc050105a40 0x7fc05019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:31.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.098+0000 7fc056082700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05019d0d0 con 0x7fc050105a40 2024-03-29T04:59:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.098+0000 7fc056082700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc050196580 con 0x7fc0500ffbd0 2024-03-29T04:59:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.098+0000 7fc04effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc050105a40 0x7fc05019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.098+0000 7fc04effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc050105a40 0x7fc05019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42520/0 (socket says 172.21.15.142:42520) 2024-03-29T04:59:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.099+0000 7fc04effd700 1 -- 172.21.15.142:0/3424360065 learned_addr learned my addr 172.21.15.142:0/3424360065 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:59:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.099+0000 7fc04f7fe700 1 --2- 172.21.15.142:0/3424360065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0500ffbd0 0x7fc05019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.099+0000 7fc04effd700 1 -- 172.21.15.142:0/3424360065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0500ffbd0 msgr2=0x7fc05019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.099+0000 7fc04effd700 1 --2- 172.21.15.142:0/3424360065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0500ffbd0 0x7fc05019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.099+0000 7fc04effd700 1 -- 172.21.15.142:0/3424360065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440095f0 con 0x7fc050105a40 2024-03-29T04:59:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.099+0000 7fc04effd700 1 --2- 172.21.15.142:0/3424360065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc050105a40 0x7fc05019ca40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc044006010 tx=0x7fc04400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.099+0000 7fc04cff9700 1 -- 172.21.15.142:0/3424360065 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc04401d070 con 0x7fc050105a40 2024-03-29T04:59:31.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.099+0000 7fc056082700 1 -- 172.21.15.142:0/3424360065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc050196800 con 0x7fc050105a40 2024-03-29T04:59:31.111 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.099+0000 7fc04cff9700 1 -- 172.21.15.142:0/3424360065 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc044005dc0 con 0x7fc050105a40 2024-03-29T04:59:31.111 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.099+0000 7fc04cff9700 1 -- 172.21.15.142:0/3424360065 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc044017400 con 0x7fc050105a40 2024-03-29T04:59:31.111 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.099+0000 7fc056082700 1 -- 172.21.15.142:0/3424360065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc050196cf0 con 0x7fc050105a40 2024-03-29T04:59:31.111 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.100+0000 7fc056082700 1 -- 172.21.15.142:0/3424360065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc050108fe0 con 0x7fc050105a40 2024-03-29T04:59:31.111 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.101+0000 7fc04cff9700 1 -- 172.21.15.142:0/3424360065 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fc044003a60 con 0x7fc050105a40 2024-03-29T04:59:31.111 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.101+0000 7fc04cff9700 1 --2- 172.21.15.142:0/3424360065 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc03806dd60 0x7fc038070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:31.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.101+0000 7fc04cff9700 1 -- 172.21.15.142:0/3424360065 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fc04408f0f0 con 0x7fc050105a40 2024-03-29T04:59:31.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.101+0000 7fc04f7fe700 1 --2- 172.21.15.142:0/3424360065 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc03806dd60 0x7fc038070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:31.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.101+0000 7fc04f7fe700 1 --2- 172.21.15.142:0/3424360065 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc03806dd60 0x7fc038070220 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc040005950 tx=0x7fc04000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:31.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.106+0000 7fc04cff9700 1 -- 172.21.15.142:0/3424360065 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc04405c040 con 0x7fc050105a40 2024-03-29T04:59:31.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.345+0000 7fc056082700 1 -- 172.21.15.142:0/3424360065 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc05002cca0 con 0x7fc050105a40 2024-03-29T04:59:31.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.345+0000 7fc04cff9700 1 -- 172.21.15.142:0/3424360065 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fc04405bbd0 con 0x7fc050105a40 2024-03-29T04:59:31.354 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T04:59:31.354 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T04:59:31.354 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:59:31.354 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T04:59:31.354 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:59:31.354 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T04:59:31.354 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T04:59:31.354 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T04:59:31.354 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T04:59:31.355 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T04:59:31.356 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T04:59:31.356 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T04:59:31.356 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T04:59:31.356 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T04:59:31.356 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T04:59:31.356 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T04:59:31.356 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T04:59:31.356 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T04:59:31.356 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T04:59:31.356 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T04:59:31.356 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T04:59:31.356 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:59:31.357 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:59:31.357 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T04:59:31.357 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T04:59:31.357 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:59:31.357 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:59:31.357 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:59:31.357 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T04:59:31.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.348+0000 7fc056082700 1 -- 172.21.15.142:0/3424360065 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc03806dd60 msgr2=0x7fc038070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:31.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.348+0000 7fc056082700 1 --2- 172.21.15.142:0/3424360065 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc03806dd60 0x7fc038070220 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc040005950 tx=0x7fc04000b410 comp rx=0 tx=0).stop 2024-03-29T04:59:31.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.348+0000 7fc056082700 1 -- 172.21.15.142:0/3424360065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc050105a40 msgr2=0x7fc05019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:31.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.348+0000 7fc056082700 1 --2- 172.21.15.142:0/3424360065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc050105a40 0x7fc05019ca40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc044006010 tx=0x7fc04400bd70 comp rx=0 tx=0).stop 2024-03-29T04:59:31.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.348+0000 7fc056082700 1 -- 172.21.15.142:0/3424360065 shutdown_connections 2024-03-29T04:59:31.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.348+0000 7fc056082700 1 --2- 172.21.15.142:0/3424360065 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc03806dd60 0x7fc038070220 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.348+0000 7fc056082700 1 --2- 172.21.15.142:0/3424360065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0500ffbd0 0x7fc05019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.348+0000 7fc056082700 1 --2- 172.21.15.142:0/3424360065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc050105a40 0x7fc05019ca40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.348+0000 7fc056082700 1 -- 172.21.15.142:0/3424360065 >> 172.21.15.142:0/3424360065 conn(0x7fc0500755a0 msgr2=0x7fc050068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:31.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.348+0000 7fc056082700 1 -- 172.21.15.142:0/3424360065 shutdown_connections 2024-03-29T04:59:31.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.348+0000 7fc056082700 1 -- 172.21.15.142:0/3424360065 wait complete. 2024-03-29T04:59:31.359 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T04:59:31.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.453+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1850389858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac8102640 msgr2=0x7f6ac810ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:31.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.453+0000 7f6acfdb0700 1 --2- 172.21.15.142:0/1850389858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac8102640 0x7f6ac810ab30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009910 tx=0x7f6ac4009c20 comp rx=0 tx=0).stop 2024-03-29T04:59:31.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.453+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1850389858 shutdown_connections 2024-03-29T04:59:31.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.453+0000 7f6acfdb0700 1 --2- 172.21.15.142:0/1850389858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac8102640 0x7f6ac810ab30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.453+0000 7f6acfdb0700 1 --2- 172.21.15.142:0/1850389858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8101c90 0x7f6ac8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.453+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1850389858 >> 172.21.15.142:0/1850389858 conn(0x7f6ac80fb550 msgr2=0x7f6ac80fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:31.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.453+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1850389858 shutdown_connections 2024-03-29T04:59:31.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.453+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1850389858 wait complete. 2024-03-29T04:59:31.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acfdb0700 1 Processor -- start 2024-03-29T04:59:31.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acfdb0700 1 -- start start 2024-03-29T04:59:31.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acfdb0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8101c90 0x7f6ac819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:31.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acfdb0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac8102640 0x7f6ac819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:31.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acfdb0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac819d120 con 0x7f6ac8102640 2024-03-29T04:59:31.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acfdb0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac81965d0 con 0x7f6ac8101c90 2024-03-29T04:59:31.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acd34b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac8102640 0x7f6ac819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:31.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acd34b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac8102640 0x7f6ac819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42546/0 (socket says 172.21.15.142:42546) 2024-03-29T04:59:31.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acd34b700 1 -- 172.21.15.142:0/1135703684 learned_addr learned my addr 172.21.15.142:0/1135703684 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:59:31.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acdb4c700 1 --2- 172.21.15.142:0/1135703684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8101c90 0x7f6ac819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:31.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acd34b700 1 -- 172.21.15.142:0/1135703684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8101c90 msgr2=0x7f6ac819c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:31.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acd34b700 1 --2- 172.21.15.142:0/1135703684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8101c90 0x7f6ac819c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acd34b700 1 -- 172.21.15.142:0/1135703684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac40095f0 con 0x7f6ac8102640 2024-03-29T04:59:31.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.454+0000 7f6acdb4c700 1 --2- 172.21.15.142:0/1135703684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8101c90 0x7f6ac819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:59:31.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.455+0000 7f6acd34b700 1 --2- 172.21.15.142:0/1135703684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac8102640 0x7f6ac819ca90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40059d0 tx=0x7f6ac400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:31.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.455+0000 7f6abeffd700 1 -- 172.21.15.142:0/1135703684 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ac401d070 con 0x7f6ac8102640 2024-03-29T04:59:31.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.455+0000 7f6abeffd700 1 -- 172.21.15.142:0/1135703684 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ac400b610 con 0x7f6ac8102640 2024-03-29T04:59:31.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.455+0000 7f6abeffd700 1 -- 172.21.15.142:0/1135703684 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ac4022470 con 0x7f6ac8102640 2024-03-29T04:59:31.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.455+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1135703684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac8196850 con 0x7f6ac8102640 2024-03-29T04:59:31.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.455+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1135703684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac8196d40 con 0x7f6ac8102640 2024-03-29T04:59:31.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.456+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1135703684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac8108230 con 0x7f6ac8102640 2024-03-29T04:59:31.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.456+0000 7f6abeffd700 1 -- 172.21.15.142:0/1135703684 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6ac400fc70 con 0x7f6ac8102640 2024-03-29T04:59:31.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.457+0000 7f6abeffd700 1 --2- 172.21.15.142:0/1135703684 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ab406dd60 0x7f6ab4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:31.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.457+0000 7f6abeffd700 1 -- 172.21.15.142:0/1135703684 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6ac408eff0 con 0x7f6ac8102640 2024-03-29T04:59:31.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.457+0000 7f6acdb4c700 1 --2- 172.21.15.142:0/1135703684 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ab406dd60 0x7f6ab4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:31.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.457+0000 7f6acdb4c700 1 --2- 172.21.15.142:0/1135703684 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ab406dd60 0x7f6ab4070220 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009be0 tx=0x7f6ab8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:31.469 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.462+0000 7f6abeffd700 1 -- 172.21.15.142:0/1135703684 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6ac405bfc0 con 0x7f6ac8102640 2024-03-29T04:59:31.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:31 smithi142 ceph-mon[112475]: pgmap v336: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:31.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:31 smithi142 ceph-mon[112475]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:59:31.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:31 smithi142 ceph-mon[112475]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:59:31.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:31 smithi142 ceph-mon[112475]: from='client.34662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:59:31.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:31 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2290623275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:59:31.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:31 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3424360065' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:59:31.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.677+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1135703684 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ac802ccf0 con 0x7f6ab406dd60 2024-03-29T04:59:31.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.679+0000 7f6abeffd700 1 -- 172.21.15.142:0/1135703684 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6ac802ccf0 con 0x7f6ab406dd60 2024-03-29T04:59:31.687 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T04:59:31.687 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T04:59:31.687 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T04:59:31.687 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T04:59:31.687 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T04:59:31.687 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T04:59:31.687 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T04:59:31.687 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T04:59:31.687 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T04:59:31.687 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T04:59:31.687 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T04:59:31.687 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T04:59:31.688 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T04:59:31.688 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T04:59:31.688 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T04:59:31.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.681+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1135703684 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ab406dd60 msgr2=0x7f6ab4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:31.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.681+0000 7f6acfdb0700 1 --2- 172.21.15.142:0/1135703684 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ab406dd60 0x7f6ab4070220 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009be0 tx=0x7f6ab8009260 comp rx=0 tx=0).stop 2024-03-29T04:59:31.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.681+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1135703684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac8102640 msgr2=0x7f6ac819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:31.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.681+0000 7f6acfdb0700 1 --2- 172.21.15.142:0/1135703684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac8102640 0x7f6ac819ca90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40059d0 tx=0x7f6ac400f740 comp rx=0 tx=0).stop 2024-03-29T04:59:31.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.681+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1135703684 shutdown_connections 2024-03-29T04:59:31.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.681+0000 7f6acfdb0700 1 --2- 172.21.15.142:0/1135703684 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ab406dd60 0x7f6ab4070220 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.681+0000 7f6acfdb0700 1 --2- 172.21.15.142:0/1135703684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8101c90 0x7f6ac819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.681+0000 7f6acfdb0700 1 --2- 172.21.15.142:0/1135703684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ac8102640 0x7f6ac819ca90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.681+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1135703684 >> 172.21.15.142:0/1135703684 conn(0x7f6ac80fb550 msgr2=0x7f6ac8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:31.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.681+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1135703684 shutdown_connections 2024-03-29T04:59:31.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.681+0000 7f6acfdb0700 1 -- 172.21.15.142:0/1135703684 wait complete. 2024-03-29T04:59:31.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.784+0000 7f91a7491700 1 -- 172.21.15.142:0/247391263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a0105a40 msgr2=0x7f91a0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:31.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.784+0000 7f91a7491700 1 --2- 172.21.15.142:0/247391263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a0105a40 0x7f91a0105e20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9190009910 tx=0x7f9190009c20 comp rx=0 tx=0).stop 2024-03-29T04:59:31.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.784+0000 7f91a7491700 1 -- 172.21.15.142:0/247391263 shutdown_connections 2024-03-29T04:59:31.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.784+0000 7f91a7491700 1 --2- 172.21.15.142:0/247391263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a00ffa40 0x7f91a00ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.784+0000 7f91a7491700 1 --2- 172.21.15.142:0/247391263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a0105a40 0x7f91a0105e20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.784+0000 7f91a7491700 1 -- 172.21.15.142:0/247391263 >> 172.21.15.142:0/247391263 conn(0x7f91a00fb550 msgr2=0x7f91a00fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:31.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.784+0000 7f91a7491700 1 -- 172.21.15.142:0/247391263 shutdown_connections 2024-03-29T04:59:31.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.784+0000 7f91a7491700 1 -- 172.21.15.142:0/247391263 wait complete. 2024-03-29T04:59:31.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a7491700 1 Processor -- start 2024-03-29T04:59:31.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a7491700 1 -- start start 2024-03-29T04:59:31.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a7491700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a00ffa40 0x7f91a019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:31.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a7491700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a0105a40 0x7f91a019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:31.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a7491700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a019f2b0 con 0x7f91a00ffa40 2024-03-29T04:59:31.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a7491700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a0198760 con 0x7f91a0105a40 2024-03-29T04:59:31.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a522d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a00ffa40 0x7f91a019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:31.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a522d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a00ffa40 0x7f91a019e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42556/0 (socket says 172.21.15.142:42556) 2024-03-29T04:59:31.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a522d700 1 -- 172.21.15.142:0/926205374 learned_addr learned my addr 172.21.15.142:0/926205374 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T04:59:31.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a4a2c700 1 --2- 172.21.15.142:0/926205374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a0105a40 0x7f91a019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:31.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a522d700 1 -- 172.21.15.142:0/926205374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a0105a40 msgr2=0x7f91a019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:31.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a522d700 1 --2- 172.21.15.142:0/926205374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a0105a40 0x7f91a019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:31.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a522d700 1 -- 172.21.15.142:0/926205374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91900095f0 con 0x7f91a00ffa40 2024-03-29T04:59:31.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.785+0000 7f91a4a2c700 1 --2- 172.21.15.142:0/926205374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a0105a40 0x7f91a019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:59:31.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.786+0000 7f91a522d700 1 --2- 172.21.15.142:0/926205374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a00ffa40 0x7f91a019e6e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9190006010 tx=0x7f919000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:31.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.786+0000 7f91967fc700 1 -- 172.21.15.142:0/926205374 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f919001d070 con 0x7f91a00ffa40 2024-03-29T04:59:31.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.786+0000 7f91967fc700 1 -- 172.21.15.142:0/926205374 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9190004fb0 con 0x7f91a00ffa40 2024-03-29T04:59:31.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.786+0000 7f91a7491700 1 -- 172.21.15.142:0/926205374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a01989e0 con 0x7f91a00ffa40 2024-03-29T04:59:31.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.786+0000 7f91967fc700 1 -- 172.21.15.142:0/926205374 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9190022ea0 con 0x7f91a00ffa40 2024-03-29T04:59:31.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.786+0000 7f91a7491700 1 -- 172.21.15.142:0/926205374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a0198ed0 con 0x7f91a00ffa40 2024-03-29T04:59:31.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.787+0000 7f91a7491700 1 -- 172.21.15.142:0/926205374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a004ea90 con 0x7f91a00ffa40 2024-03-29T04:59:31.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.787+0000 7f91967fc700 1 -- 172.21.15.142:0/926205374 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f9190022470 con 0x7f91a00ffa40 2024-03-29T04:59:31.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.788+0000 7f91967fc700 1 --2- 172.21.15.142:0/926205374 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f918c06dd60 0x7f918c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:59:31.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.788+0000 7f91967fc700 1 -- 172.21.15.142:0/926205374 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f919008f230 con 0x7f91a00ffa40 2024-03-29T04:59:31.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.788+0000 7f91a4a2c700 1 --2- 172.21.15.142:0/926205374 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f918c06dd60 0x7f918c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:59:31.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.788+0000 7f91a4a2c700 1 --2- 172.21.15.142:0/926205374 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f918c06dd60 0x7f918c070220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f919c005fd0 tx=0x7f919c005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:59:31.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:31 smithi152 ceph-mon[88186]: pgmap v336: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:31.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:31 smithi152 ceph-mon[88186]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:59:31.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:31 smithi152 ceph-mon[88186]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:59:31.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:31 smithi152 ceph-mon[88186]: from='client.34662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:59:31.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:31 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2290623275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T04:59:31.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:31 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3424360065' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T04:59:31.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:31.792+0000 7f91967fc700 1 -- 172.21.15.142:0/926205374 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f919005c230 con 0x7f91a00ffa40 2024-03-29T04:59:32.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.067+0000 7f91a7491700 1 -- 172.21.15.142:0/926205374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f91a0199c70 con 0x7f91a00ffa40 2024-03-29T04:59:32.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.067+0000 7f91967fc700 1 -- 172.21.15.142:0/926205374 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f9190022720 con 0x7f91a00ffa40 2024-03-29T04:59:32.076 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T04:59:32.076 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T04:59:32.076 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T04:59:32.076 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T04:59:32.076 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T04:59:32.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.070+0000 7f91a7491700 1 -- 172.21.15.142:0/926205374 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f918c06dd60 msgr2=0x7f918c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:32.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.070+0000 7f91a7491700 1 --2- 172.21.15.142:0/926205374 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f918c06dd60 0x7f918c070220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f919c005fd0 tx=0x7f919c005dc0 comp rx=0 tx=0).stop 2024-03-29T04:59:32.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.070+0000 7f91a7491700 1 -- 172.21.15.142:0/926205374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a00ffa40 msgr2=0x7f91a019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:59:32.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.070+0000 7f91a7491700 1 --2- 172.21.15.142:0/926205374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a00ffa40 0x7f91a019e6e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9190006010 tx=0x7f919000be20 comp rx=0 tx=0).stop 2024-03-29T04:59:32.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.070+0000 7f91a7491700 1 -- 172.21.15.142:0/926205374 shutdown_connections 2024-03-29T04:59:32.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.070+0000 7f91a7491700 1 --2- 172.21.15.142:0/926205374 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f918c06dd60 0x7f918c070220 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:32.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.070+0000 7f91a7491700 1 --2- 172.21.15.142:0/926205374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a00ffa40 0x7f91a019e6e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:32.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.070+0000 7f91a7491700 1 --2- 172.21.15.142:0/926205374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a0105a40 0x7f91a019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:59:32.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.070+0000 7f91a7491700 1 -- 172.21.15.142:0/926205374 >> 172.21.15.142:0/926205374 conn(0x7f91a00fb550 msgr2=0x7f91a0068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:59:32.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.070+0000 7f91a7491700 1 -- 172.21.15.142:0/926205374 shutdown_connections 2024-03-29T04:59:32.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T04:59:32.070+0000 7f91a7491700 1 -- 172.21.15.142:0/926205374 wait complete. 2024-03-29T04:59:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:32 smithi152 ceph-mon[88186]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:59:32.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:32 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/926205374' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:59:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:32 smithi142 ceph-mon[112475]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T04:59:32.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:32 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/926205374' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T04:59:33.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:33 smithi152 ceph-mon[88186]: pgmap v337: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:33.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:33 smithi142 ceph-mon[112475]: pgmap v337: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:35.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:35 smithi152 ceph-mon[88186]: pgmap v338: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:35.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:35 smithi142 ceph-mon[112475]: pgmap v338: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:37.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:37 smithi152 ceph-mon[88186]: pgmap v339: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:37.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:37 smithi142 ceph-mon[112475]: pgmap v339: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:39.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:39 smithi152 ceph-mon[88186]: pgmap v340: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:39.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:39 smithi142 ceph-mon[112475]: pgmap v340: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:41.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:41 smithi152 ceph-mon[88186]: pgmap v341: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:41.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:41 smithi142 ceph-mon[112475]: pgmap v341: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:43.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:43 smithi152 ceph-mon[88186]: pgmap v342: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:43.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:43 smithi142 ceph-mon[112475]: pgmap v342: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:45.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:45 smithi152 ceph-mon[88186]: pgmap v343: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:45.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:59:45.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:45 smithi142 ceph-mon[112475]: pgmap v343: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:45.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T04:59:47.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:47 smithi152 ceph-mon[88186]: pgmap v344: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:47.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:47 smithi142 ceph-mon[112475]: pgmap v344: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:49.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:49 smithi152 ceph-mon[88186]: pgmap v345: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:49.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:49 smithi142 ceph-mon[112475]: pgmap v345: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:51.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:51 smithi152 ceph-mon[88186]: pgmap v346: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:51.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:51 smithi142 ceph-mon[112475]: pgmap v346: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:53.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:53 smithi152 ceph-mon[88186]: pgmap v347: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:53.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:53 smithi142 ceph-mon[112475]: pgmap v347: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:55.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:55 smithi152 ceph-mon[88186]: pgmap v348: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:55.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:55 smithi142 ceph-mon[112475]: pgmap v348: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:57.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:57 smithi152 ceph-mon[88186]: pgmap v349: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:57.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:57 smithi142 ceph-mon[112475]: pgmap v349: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:59.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 04:59:59 smithi152 ceph-mon[88186]: pgmap v350: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T04:59:59.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 04:59:59 smithi142 ceph-mon[112475]: pgmap v350: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:00.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:00 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[112471]: 2024-03-29T04:59:59.999+0000 7f6a4d4b4700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:00:00.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:00 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[112471]: 2024-03-29T04:59:59.999+0000 7f6a4d4b4700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:00:00.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:00 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[112471]: 2024-03-29T04:59:59.999+0000 7f6a4d4b4700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-03-29T05:00:00.379 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:00 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[112471]: 2024-03-29T04:59:59.999+0000 7f6a4d4b4700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:00:00.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:00 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[112471]: 2024-03-29T04:59:59.999+0000 7f6a4d4b4700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-03-29T05:00:00.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:00:00.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:00 smithi152 ceph-mon[88186]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:00:00.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:00 smithi152 ceph-mon[88186]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:00:00.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:00 smithi152 ceph-mon[88186]: fs cephfs is degraded 2024-03-29T05:00:00.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:00 smithi152 ceph-mon[88186]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:00:00.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:00 smithi152 ceph-mon[88186]: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:00:00.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:00:00.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:00 smithi142 ceph-mon[112475]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:00:00.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:00 smithi142 ceph-mon[112475]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:00:00.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:00 smithi142 ceph-mon[112475]: fs cephfs is degraded 2024-03-29T05:00:00.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:00 smithi142 ceph-mon[112475]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:00:00.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:00 smithi142 ceph-mon[112475]: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:00:01.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:01 smithi152 ceph-mon[88186]: pgmap v351: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:01.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:01 smithi142 ceph-mon[112475]: pgmap v351: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:02.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.187+0000 7f6aa2742700 1 -- 172.21.15.142:0/1040600707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c0ffa40 msgr2=0x7f6a9c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:02.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.187+0000 7f6aa2742700 1 --2- 172.21.15.142:0/1040600707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c0ffa40 0x7f6a9c0ffec0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6a90009910 tx=0x7f6a90009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:02.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.187+0000 7f6aa2742700 1 -- 172.21.15.142:0/1040600707 shutdown_connections 2024-03-29T05:00:02.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.187+0000 7f6aa2742700 1 --2- 172.21.15.142:0/1040600707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c0ffa40 0x7f6a9c0ffec0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.187+0000 7f6aa2742700 1 --2- 172.21.15.142:0/1040600707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a9c105a40 0x7f6a9c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.187+0000 7f6aa2742700 1 -- 172.21.15.142:0/1040600707 >> 172.21.15.142:0/1040600707 conn(0x7f6a9c0fb550 msgr2=0x7f6a9c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:02.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.187+0000 7f6aa2742700 1 -- 172.21.15.142:0/1040600707 shutdown_connections 2024-03-29T05:00:02.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.187+0000 7f6aa2742700 1 -- 172.21.15.142:0/1040600707 wait complete. 2024-03-29T05:00:02.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.187+0000 7f6aa2742700 1 Processor -- start 2024-03-29T05:00:02.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.188+0000 7f6aa2742700 1 -- start start 2024-03-29T05:00:02.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.188+0000 7f6aa2742700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c0ffa40 0x7f6a9c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:02.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.188+0000 7f6aa2742700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a9c105a40 0x7f6a9c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:02.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.188+0000 7f6aa2742700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c198110 con 0x7f6a9c0ffa40 2024-03-29T05:00:02.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.188+0000 7f6aa2742700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c198250 con 0x7f6a9c105a40 2024-03-29T05:00:02.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.188+0000 7f6a9bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c0ffa40 0x7f6a9c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.188+0000 7f6a9bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c0ffa40 0x7f6a9c1975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37548/0 (socket says 172.21.15.142:37548) 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.188+0000 7f6a9bfff700 1 -- 172.21.15.142:0/826293533 learned_addr learned my addr 172.21.15.142:0/826293533 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.188+0000 7f6a9b7fe700 1 --2- 172.21.15.142:0/826293533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a9c105a40 0x7f6a9c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.188+0000 7f6a9bfff700 1 -- 172.21.15.142:0/826293533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a9c105a40 msgr2=0x7f6a9c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.188+0000 7f6a9bfff700 1 --2- 172.21.15.142:0/826293533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a9c105a40 0x7f6a9c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.188+0000 7f6a9bfff700 1 -- 172.21.15.142:0/826293533 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a900095f0 con 0x7f6a9c0ffa40 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.189+0000 7f6a9bfff700 1 --2- 172.21.15.142:0/826293533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c0ffa40 0x7f6a9c1975b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00d8d0 tx=0x7f6a8c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.189+0000 7f6a997fa700 1 -- 172.21.15.142:0/826293533 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a8c0098a0 con 0x7f6a9c0ffa40 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.189+0000 7f6a997fa700 1 -- 172.21.15.142:0/826293533 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a8c010460 con 0x7f6a9c0ffa40 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.189+0000 7f6aa2742700 1 -- 172.21.15.142:0/826293533 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9c1a0620 con 0x7f6a9c0ffa40 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.189+0000 7f6a997fa700 1 -- 172.21.15.142:0/826293533 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a8c00f5f0 con 0x7f6a9c0ffa40 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.189+0000 7f6aa2742700 1 -- 172.21.15.142:0/826293533 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9c1a0ae0 con 0x7f6a9c0ffa40 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.190+0000 7f6aa2742700 1 -- 172.21.15.142:0/826293533 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9c107fd0 con 0x7f6a9c0ffa40 2024-03-29T05:00:02.199 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.190+0000 7f6a997fa700 1 -- 172.21.15.142:0/826293533 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6a8c009a00 con 0x7f6a9c0ffa40 2024-03-29T05:00:02.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.191+0000 7f6a997fa700 1 --2- 172.21.15.142:0/826293533 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6a8406dd60 0x7f6a84070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:02.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.191+0000 7f6a9b7fe700 1 --2- 172.21.15.142:0/826293533 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6a8406dd60 0x7f6a84070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:02.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.191+0000 7f6a997fa700 1 -- 172.21.15.142:0/826293533 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6a8c08db40 con 0x7f6a9c0ffa40 2024-03-29T05:00:02.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.191+0000 7f6a9b7fe700 1 --2- 172.21.15.142:0/826293533 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6a8406dd60 0x7f6a84070220 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c069660 tx=0x7f6a9001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:02.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.196+0000 7f6a997fa700 1 -- 172.21.15.142:0/826293533 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6a8c056f60 con 0x7f6a9c0ffa40 2024-03-29T05:00:02.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.406+0000 7f6aa2742700 1 -- 172.21.15.142:0/826293533 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a9c02cce0 con 0x7f6a8406dd60 2024-03-29T05:00:02.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.408+0000 7f6a997fa700 1 -- 172.21.15.142:0/826293533 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6a9c02cce0 con 0x7f6a8406dd60 2024-03-29T05:00:02.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.410+0000 7f6aa2742700 1 -- 172.21.15.142:0/826293533 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6a8406dd60 msgr2=0x7f6a84070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:02.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.410+0000 7f6aa2742700 1 --2- 172.21.15.142:0/826293533 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6a8406dd60 0x7f6a84070220 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c069660 tx=0x7f6a9001a040 comp rx=0 tx=0).stop 2024-03-29T05:00:02.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.410+0000 7f6aa2742700 1 -- 172.21.15.142:0/826293533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c0ffa40 msgr2=0x7f6a9c1975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:02.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.410+0000 7f6aa2742700 1 --2- 172.21.15.142:0/826293533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c0ffa40 0x7f6a9c1975b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00d8d0 tx=0x7f6a8c00dbe0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.410+0000 7f6aa2742700 1 -- 172.21.15.142:0/826293533 shutdown_connections 2024-03-29T05:00:02.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.410+0000 7f6aa2742700 1 --2- 172.21.15.142:0/826293533 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6a8406dd60 0x7f6a84070220 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.410+0000 7f6aa2742700 1 --2- 172.21.15.142:0/826293533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a9c0ffa40 0x7f6a9c1975b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.410+0000 7f6aa2742700 1 --2- 172.21.15.142:0/826293533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a9c105a40 0x7f6a9c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.410+0000 7f6aa2742700 1 -- 172.21.15.142:0/826293533 >> 172.21.15.142:0/826293533 conn(0x7f6a9c0fb550 msgr2=0x7f6a9c0fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:02.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.410+0000 7f6aa2742700 1 -- 172.21.15.142:0/826293533 shutdown_connections 2024-03-29T05:00:02.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.410+0000 7f6aa2742700 1 -- 172.21.15.142:0/826293533 wait complete. 2024-03-29T05:00:02.429 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:00:02.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.515+0000 7f5464db2700 1 -- 172.21.15.142:0/520448137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5460102630 msgr2=0x7f546010ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:02.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.515+0000 7f5464db2700 1 --2- 172.21.15.142:0/520448137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5460102630 0x7f546010ab20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5448009910 tx=0x7f5448009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:02.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.515+0000 7f5464db2700 1 -- 172.21.15.142:0/520448137 shutdown_connections 2024-03-29T05:00:02.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.515+0000 7f5464db2700 1 --2- 172.21.15.142:0/520448137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5460102630 0x7f546010ab20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.515+0000 7f5464db2700 1 --2- 172.21.15.142:0/520448137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5460101d10 0x7f54601020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.515+0000 7f5464db2700 1 -- 172.21.15.142:0/520448137 >> 172.21.15.142:0/520448137 conn(0x7f54600fb540 msgr2=0x7f54600fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:02.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.515+0000 7f5464db2700 1 -- 172.21.15.142:0/520448137 shutdown_connections 2024-03-29T05:00:02.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.515+0000 7f5464db2700 1 -- 172.21.15.142:0/520448137 wait complete. 2024-03-29T05:00:02.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.516+0000 7f5464db2700 1 Processor -- start 2024-03-29T05:00:02.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.516+0000 7f5464db2700 1 -- start start 2024-03-29T05:00:02.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.516+0000 7f5464db2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5460101d10 0x7f546019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.516+0000 7f5464db2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5460102630 0x7f546019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.516+0000 7f5464db2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546019d090 con 0x7f5460101d10 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.516+0000 7f5464db2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601965d0 con 0x7f5460102630 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.516+0000 7f545e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5460101d10 0x7f546019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.517+0000 7f545e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5460101d10 0x7f546019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37558/0 (socket says 172.21.15.142:37558) 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.517+0000 7f545e59c700 1 -- 172.21.15.142:0/1476688308 learned_addr learned my addr 172.21.15.142:0/1476688308 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.517+0000 7f5455d9b700 1 --2- 172.21.15.142:0/1476688308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5460102630 0x7f546019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.517+0000 7f545e59c700 1 -- 172.21.15.142:0/1476688308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5460102630 msgr2=0x7f546019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.517+0000 7f545e59c700 1 --2- 172.21.15.142:0/1476688308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5460102630 0x7f546019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.517+0000 7f545e59c700 1 -- 172.21.15.142:0/1476688308 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54480095f0 con 0x7f5460101d10 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.517+0000 7f545e59c700 1 --2- 172.21.15.142:0/1476688308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5460101d10 0x7f546019c550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f545000c930 tx=0x7f545000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.517+0000 7f5457fff700 1 -- 172.21.15.142:0/1476688308 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5450007ad0 con 0x7f5460101d10 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.517+0000 7f5457fff700 1 -- 172.21.15.142:0/1476688308 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f545000ce80 con 0x7f5460101d10 2024-03-29T05:00:02.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.517+0000 7f5457fff700 1 -- 172.21.15.142:0/1476688308 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5450018630 con 0x7f5460101d10 2024-03-29T05:00:02.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.517+0000 7f5464db2700 1 -- 172.21.15.142:0/1476688308 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54601968b0 con 0x7f5460101d10 2024-03-29T05:00:02.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.517+0000 7f5464db2700 1 -- 172.21.15.142:0/1476688308 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5460196dd0 con 0x7f5460101d10 2024-03-29T05:00:02.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.518+0000 7f5464db2700 1 -- 172.21.15.142:0/1476688308 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5460108220 con 0x7f5460101d10 2024-03-29T05:00:02.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.519+0000 7f5457fff700 1 -- 172.21.15.142:0/1476688308 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f5450007c30 con 0x7f5460101d10 2024-03-29T05:00:02.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.519+0000 7f5457fff700 1 --2- 172.21.15.142:0/1476688308 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f544c06dd60 0x7f544c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:02.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.519+0000 7f5455d9b700 1 --2- 172.21.15.142:0/1476688308 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f544c06dd60 0x7f544c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:02.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.519+0000 7f5457fff700 1 -- 172.21.15.142:0/1476688308 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f545008dc90 con 0x7f5460101d10 2024-03-29T05:00:02.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.520+0000 7f5455d9b700 1 --2- 172.21.15.142:0/1476688308 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f544c06dd60 0x7f544c070220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5448006010 tx=0x7f544801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:02.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.524+0000 7f5457fff700 1 -- 172.21.15.142:0/1476688308 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f545005abe0 con 0x7f5460101d10 2024-03-29T05:00:02.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.740+0000 7f5464db2700 1 -- 172.21.15.142:0/1476688308 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5460197050 con 0x7f544c06dd60 2024-03-29T05:00:02.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.741+0000 7f5457fff700 1 -- 172.21.15.142:0/1476688308 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5460197050 con 0x7f544c06dd60 2024-03-29T05:00:02.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.743+0000 7f5464db2700 1 -- 172.21.15.142:0/1476688308 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f544c06dd60 msgr2=0x7f544c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:02.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.743+0000 7f5464db2700 1 --2- 172.21.15.142:0/1476688308 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f544c06dd60 0x7f544c070220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5448006010 tx=0x7f544801a040 comp rx=0 tx=0).stop 2024-03-29T05:00:02.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.743+0000 7f5464db2700 1 -- 172.21.15.142:0/1476688308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5460101d10 msgr2=0x7f546019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:02.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.743+0000 7f5464db2700 1 --2- 172.21.15.142:0/1476688308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5460101d10 0x7f546019c550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f545000c930 tx=0x7f545000cc40 comp rx=0 tx=0).stop 2024-03-29T05:00:02.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.743+0000 7f5464db2700 1 -- 172.21.15.142:0/1476688308 shutdown_connections 2024-03-29T05:00:02.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.743+0000 7f5464db2700 1 --2- 172.21.15.142:0/1476688308 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f544c06dd60 0x7f544c070220 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.743+0000 7f5464db2700 1 --2- 172.21.15.142:0/1476688308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5460101d10 0x7f546019c550 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.743+0000 7f5464db2700 1 --2- 172.21.15.142:0/1476688308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5460102630 0x7f546019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.743+0000 7f5464db2700 1 -- 172.21.15.142:0/1476688308 >> 172.21.15.142:0/1476688308 conn(0x7f54600fb540 msgr2=0x7f5460105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:02.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.743+0000 7f5464db2700 1 -- 172.21.15.142:0/1476688308 shutdown_connections 2024-03-29T05:00:02.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.743+0000 7f5464db2700 1 -- 172.21.15.142:0/1476688308 wait complete. 2024-03-29T05:00:02.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.849+0000 7fb54034f700 1 -- 172.21.15.142:0/336972360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5380ff530 msgr2=0x7fb5380ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:02.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.849+0000 7fb54034f700 1 --2- 172.21.15.142:0/336972360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5380ff530 0x7fb5380ff910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb528009910 tx=0x7fb528009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:02.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.849+0000 7fb54034f700 1 -- 172.21.15.142:0/336972360 shutdown_connections 2024-03-29T05:00:02.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.849+0000 7fb54034f700 1 --2- 172.21.15.142:0/336972360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5380ffe50 0x7fb538104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.849+0000 7fb54034f700 1 --2- 172.21.15.142:0/336972360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5380ff530 0x7fb5380ff910 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.849+0000 7fb54034f700 1 -- 172.21.15.142:0/336972360 >> 172.21.15.142:0/336972360 conn(0x7fb538074df0 msgr2=0x7fb538075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:02.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.849+0000 7fb54034f700 1 -- 172.21.15.142:0/336972360 shutdown_connections 2024-03-29T05:00:02.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.849+0000 7fb54034f700 1 -- 172.21.15.142:0/336972360 wait complete. 2024-03-29T05:00:02.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.849+0000 7fb54034f700 1 Processor -- start 2024-03-29T05:00:02.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb54034f700 1 -- start start 2024-03-29T05:00:02.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb54034f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5380ff530 0x7fb53819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:02.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb54034f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5380ffe50 0x7fb53819d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:02.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb54034f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53819da80 con 0x7fb5380ffe50 2024-03-29T05:00:02.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb54034f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5381a1810 con 0x7fb5380ff530 2024-03-29T05:00:02.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb53d8ea700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5380ffe50 0x7fb53819d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:02.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb53d8ea700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5380ffe50 0x7fb53819d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37586/0 (socket says 172.21.15.142:37586) 2024-03-29T05:00:02.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb53d8ea700 1 -- 172.21.15.142:0/3920771587 learned_addr learned my addr 172.21.15.142:0/3920771587 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:02.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb53e0eb700 1 --2- 172.21.15.142:0/3920771587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5380ff530 0x7fb53819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:02.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb53d8ea700 1 -- 172.21.15.142:0/3920771587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5380ff530 msgr2=0x7fb53819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:02.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb53d8ea700 1 --2- 172.21.15.142:0/3920771587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5380ff530 0x7fb53819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:02.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb53d8ea700 1 -- 172.21.15.142:0/3920771587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5280095f0 con 0x7fb5380ffe50 2024-03-29T05:00:02.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.850+0000 7fb53d8ea700 1 --2- 172.21.15.142:0/3920771587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5380ffe50 0x7fb53819d3a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb53400c8f0 tx=0x7fb53400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:02.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.851+0000 7fb52f7fe700 1 -- 172.21.15.142:0/3920771587 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb534007a50 con 0x7fb5380ffe50 2024-03-29T05:00:02.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.851+0000 7fb54034f700 1 -- 172.21.15.142:0/3920771587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5381a1af0 con 0x7fb5380ffe50 2024-03-29T05:00:02.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.851+0000 7fb52f7fe700 1 -- 172.21.15.142:0/3920771587 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb534007bb0 con 0x7fb5380ffe50 2024-03-29T05:00:02.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.851+0000 7fb52f7fe700 1 -- 172.21.15.142:0/3920771587 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb534018630 con 0x7fb5380ffe50 2024-03-29T05:00:02.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.851+0000 7fb54034f700 1 -- 172.21.15.142:0/3920771587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5381a2040 con 0x7fb5380ffe50 2024-03-29T05:00:02.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.852+0000 7fb54034f700 1 -- 172.21.15.142:0/3920771587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb538101f70 con 0x7fb5380ffe50 2024-03-29T05:00:02.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.852+0000 7fb52f7fe700 1 -- 172.21.15.142:0/3920771587 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fb534007d20 con 0x7fb5380ffe50 2024-03-29T05:00:02.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.853+0000 7fb52f7fe700 1 --2- 172.21.15.142:0/3920771587 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb52406dd60 0x7fb524070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:02.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.853+0000 7fb52f7fe700 1 -- 172.21.15.142:0/3920771587 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb53408dc70 con 0x7fb5380ffe50 2024-03-29T05:00:02.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.853+0000 7fb53e0eb700 1 --2- 172.21.15.142:0/3920771587 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb52406dd60 0x7fb524070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:02.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.853+0000 7fb53e0eb700 1 --2- 172.21.15.142:0/3920771587 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb52406dd60 0x7fb524070220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb52800b5c0 tx=0x7fb52801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:02.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:02.857+0000 7fb52f7fe700 1 -- 172.21.15.142:0/3920771587 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb534057090 con 0x7fb5380ffe50 2024-03-29T05:00:03.067 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.058+0000 7fb54034f700 1 -- 172.21.15.142:0/3920771587 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5381a22c0 con 0x7fb52406dd60 2024-03-29T05:00:03.077 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.069+0000 7fb52f7fe700 1 -- 172.21.15.142:0/3920771587 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb5381a22c0 con 0x7fb52406dd60 2024-03-29T05:00:03.077 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:00:03.077 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (11m) 4m ago 23m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:00:03.077 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (23m) 4m ago 23m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:00:03.077 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (22m) 4m ago 22m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:00:03.077 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (9m) 4m ago 23m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:00:03.077 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (8m) 4m ago 22m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:00:03.077 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (11m) 4m ago 23m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:00:03.077 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (5m) 4m ago 17m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (5m) 4m ago 17m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (4m) 4m ago 17m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (4m) 4m ago 17m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (13m) 4m ago 24m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (12m) 4m ago 22m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (10m) 4m ago 24m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (9m) 4m ago 22m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (12m) 4m ago 23m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (12m) 4m ago 22m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (8m) 4m ago 21m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (8m) 4m ago 21m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (7m) 4m ago 20m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:00:03.078 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (6m) 4m ago 20m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:00:03.079 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (6m) 4m ago 19m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:00:03.079 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (5m) 4m ago 19m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:00:03.079 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (12m) 4m ago 22m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:00:03.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.071+0000 7fb54034f700 1 -- 172.21.15.142:0/3920771587 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb52406dd60 msgr2=0x7fb524070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:03.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.071+0000 7fb54034f700 1 --2- 172.21.15.142:0/3920771587 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb52406dd60 0x7fb524070220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb52800b5c0 tx=0x7fb52801a040 comp rx=0 tx=0).stop 2024-03-29T05:00:03.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.071+0000 7fb54034f700 1 -- 172.21.15.142:0/3920771587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5380ffe50 msgr2=0x7fb53819d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:03.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.071+0000 7fb54034f700 1 --2- 172.21.15.142:0/3920771587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5380ffe50 0x7fb53819d3a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb53400c8f0 tx=0x7fb53400ccb0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.071+0000 7fb54034f700 1 -- 172.21.15.142:0/3920771587 shutdown_connections 2024-03-29T05:00:03.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.071+0000 7fb54034f700 1 --2- 172.21.15.142:0/3920771587 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb52406dd60 0x7fb524070220 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.071+0000 7fb54034f700 1 --2- 172.21.15.142:0/3920771587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5380ff530 0x7fb53819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.071+0000 7fb54034f700 1 --2- 172.21.15.142:0/3920771587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5380ffe50 0x7fb53819d3a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.071+0000 7fb54034f700 1 -- 172.21.15.142:0/3920771587 >> 172.21.15.142:0/3920771587 conn(0x7fb538074df0 msgr2=0x7fb538111ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:03.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.071+0000 7fb54034f700 1 -- 172.21.15.142:0/3920771587 shutdown_connections 2024-03-29T05:00:03.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.071+0000 7fb54034f700 1 -- 172.21.15.142:0/3920771587 wait complete. 2024-03-29T05:00:03.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.174+0000 7f1e0dad6700 1 -- 172.21.15.142:0/2943143337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e080ffa50 msgr2=0x7f1e080ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:03.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.174+0000 7f1e0dad6700 1 --2- 172.21.15.142:0/2943143337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e080ffa50 0x7f1e080ffed0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009910 tx=0x7f1dfc009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:03.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.174+0000 7f1e0dad6700 1 -- 172.21.15.142:0/2943143337 shutdown_connections 2024-03-29T05:00:03.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.174+0000 7f1e0dad6700 1 --2- 172.21.15.142:0/2943143337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e080ffa50 0x7f1e080ffed0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.174+0000 7f1e0dad6700 1 --2- 172.21.15.142:0/2943143337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e08105a50 0x7f1e08105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.174+0000 7f1e0dad6700 1 -- 172.21.15.142:0/2943143337 >> 172.21.15.142:0/2943143337 conn(0x7f1e080fb540 msgr2=0x7f1e080fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:03.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.174+0000 7f1e0dad6700 1 -- 172.21.15.142:0/2943143337 shutdown_connections 2024-03-29T05:00:03.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.174+0000 7f1e0dad6700 1 -- 172.21.15.142:0/2943143337 wait complete. 2024-03-29T05:00:03.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.175+0000 7f1e0dad6700 1 Processor -- start 2024-03-29T05:00:03.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.175+0000 7f1e0dad6700 1 -- start start 2024-03-29T05:00:03.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.175+0000 7f1e0dad6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e080ffa50 0x7f1e08195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:03.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.175+0000 7f1e0dad6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e08105a50 0x7f1e08195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:03.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.175+0000 7f1e0dad6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e08196280 con 0x7f1e080ffa50 2024-03-29T05:00:03.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.175+0000 7f1e0dad6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0819a010 con 0x7f1e08105a50 2024-03-29T05:00:03.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.175+0000 7f1e077fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e080ffa50 0x7f1e08195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:03.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.175+0000 7f1e077fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e080ffa50 0x7f1e08195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37608/0 (socket says 172.21.15.142:37608) 2024-03-29T05:00:03.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.175+0000 7f1e077fe700 1 -- 172.21.15.142:0/1642652217 learned_addr learned my addr 172.21.15.142:0/1642652217 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.176+0000 7f1e06ffd700 1 --2- 172.21.15.142:0/1642652217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e08105a50 0x7f1e08195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.176+0000 7f1e077fe700 1 -- 172.21.15.142:0/1642652217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e08105a50 msgr2=0x7f1e08195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.176+0000 7f1e077fe700 1 --2- 172.21.15.142:0/1642652217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e08105a50 0x7f1e08195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.176+0000 7f1e077fe700 1 -- 172.21.15.142:0/1642652217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc0095f0 con 0x7f1e080ffa50 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.176+0000 7f1e06ffd700 1 --2- 172.21.15.142:0/1642652217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e08105a50 0x7f1e08195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.176+0000 7f1e077fe700 1 --2- 172.21.15.142:0/1642652217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e080ffa50 0x7f1e08195660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000ada0 tx=0x7f1e0000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.176+0000 7f1e04ff9700 1 -- 172.21.15.142:0/1642652217 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e0000f070 con 0x7f1e080ffa50 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.176+0000 7f1e04ff9700 1 -- 172.21.15.142:0/1642652217 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e00007bf0 con 0x7f1e080ffa50 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.176+0000 7f1e0dad6700 1 -- 172.21.15.142:0/1642652217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0819a2f0 con 0x7f1e080ffa50 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.176+0000 7f1e04ff9700 1 -- 172.21.15.142:0/1642652217 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e00005690 con 0x7f1e080ffa50 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.176+0000 7f1e0dad6700 1 -- 172.21.15.142:0/1642652217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0819a840 con 0x7f1e080ffa50 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.177+0000 7f1e0dad6700 1 -- 172.21.15.142:0/1642652217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e08107fe0 con 0x7f1e080ffa50 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.177+0000 7f1e04ff9700 1 -- 172.21.15.142:0/1642652217 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f1e00005850 con 0x7f1e080ffa50 2024-03-29T05:00:03.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.178+0000 7f1e04ff9700 1 --2- 172.21.15.142:0/1642652217 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1df006dd60 0x7f1df0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:03.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.178+0000 7f1e06ffd700 1 --2- 172.21.15.142:0/1642652217 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1df006dd60 0x7f1df0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:03.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.178+0000 7f1e04ff9700 1 -- 172.21.15.142:0/1642652217 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1e0008d610 con 0x7f1e080ffa50 2024-03-29T05:00:03.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.178+0000 7f1e06ffd700 1 --2- 172.21.15.142:0/1642652217 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1df006dd60 0x7f1df0070220 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00b5c0 tx=0x7f1dfc005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:03.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.182+0000 7f1e04ff9700 1 -- 172.21.15.142:0/1642652217 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1e00056ab0 con 0x7f1e080ffa50 2024-03-29T05:00:03.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.469+0000 7f1e0dad6700 1 -- 172.21.15.142:0/1642652217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1e0819ab50 con 0x7f1e080ffa50 2024-03-29T05:00:03.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.469+0000 7f1e04ff9700 1 -- 172.21.15.142:0/1642652217 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1e0005a170 con 0x7f1e080ffa50 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:00:03.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:00:03.479 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:00:03.479 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:00:03.479 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:00:03.479 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:00:03.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.472+0000 7f1e0dad6700 1 -- 172.21.15.142:0/1642652217 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1df006dd60 msgr2=0x7f1df0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:03.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.472+0000 7f1e0dad6700 1 --2- 172.21.15.142:0/1642652217 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1df006dd60 0x7f1df0070220 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00b5c0 tx=0x7f1dfc005480 comp rx=0 tx=0).stop 2024-03-29T05:00:03.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.472+0000 7f1e0dad6700 1 -- 172.21.15.142:0/1642652217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e080ffa50 msgr2=0x7f1e08195660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:03.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.472+0000 7f1e0dad6700 1 --2- 172.21.15.142:0/1642652217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e080ffa50 0x7f1e08195660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000ada0 tx=0x7f1e0000c330 comp rx=0 tx=0).stop 2024-03-29T05:00:03.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.472+0000 7f1e0dad6700 1 -- 172.21.15.142:0/1642652217 shutdown_connections 2024-03-29T05:00:03.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.472+0000 7f1e0dad6700 1 --2- 172.21.15.142:0/1642652217 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1df006dd60 0x7f1df0070220 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.472+0000 7f1e0dad6700 1 --2- 172.21.15.142:0/1642652217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e080ffa50 0x7f1e08195660 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.472+0000 7f1e0dad6700 1 --2- 172.21.15.142:0/1642652217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e08105a50 0x7f1e08195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.472+0000 7f1e0dad6700 1 -- 172.21.15.142:0/1642652217 >> 172.21.15.142:0/1642652217 conn(0x7f1e080fb540 msgr2=0x7f1e080fbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:03.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.472+0000 7f1e0dad6700 1 -- 172.21.15.142:0/1642652217 shutdown_connections 2024-03-29T05:00:03.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.472+0000 7f1e0dad6700 1 -- 172.21.15.142:0/1642652217 wait complete. 2024-03-29T05:00:03.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.577+0000 7ff8890ee700 1 -- 172.21.15.142:0/3025141372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8840ff310 msgr2=0x7ff8840ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:03.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.577+0000 7ff8890ee700 1 --2- 172.21.15.142:0/3025141372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8840ff310 0x7ff8840ff770 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff874009910 tx=0x7ff874009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:03.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.577+0000 7ff8890ee700 1 -- 172.21.15.142:0/3025141372 shutdown_connections 2024-03-29T05:00:03.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.577+0000 7ff8890ee700 1 --2- 172.21.15.142:0/3025141372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8840ff310 0x7ff8840ff770 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.577+0000 7ff8890ee700 1 --2- 172.21.15.142:0/3025141372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff884106c10 0x7ff884106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.577+0000 7ff8890ee700 1 -- 172.21.15.142:0/3025141372 >> 172.21.15.142:0/3025141372 conn(0x7ff884074bf0 msgr2=0x7ff884075000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:03.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.577+0000 7ff8890ee700 1 -- 172.21.15.142:0/3025141372 shutdown_connections 2024-03-29T05:00:03.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.577+0000 7ff8890ee700 1 -- 172.21.15.142:0/3025141372 wait complete. 2024-03-29T05:00:03.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.578+0000 7ff8890ee700 1 Processor -- start 2024-03-29T05:00:03.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.578+0000 7ff8890ee700 1 -- start start 2024-03-29T05:00:03.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff8890ee700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8840ff310 0x7ff88419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:03.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff8890ee700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff884106c10 0x7ff88419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:03.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff8890ee700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88419d1e0 con 0x7ff884106c10 2024-03-29T05:00:03.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff8890ee700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8841966e0 con 0x7ff8840ff310 2024-03-29T05:00:03.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff88259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff884106c10 0x7ff88419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:03.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff88259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff884106c10 0x7ff88419cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37624/0 (socket says 172.21.15.142:37624) 2024-03-29T05:00:03.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff88259c700 1 -- 172.21.15.142:0/4173414609 learned_addr learned my addr 172.21.15.142:0/4173414609 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:03.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff882d9d700 1 --2- 172.21.15.142:0/4173414609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8840ff310 0x7ff88419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:03.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff88259c700 1 -- 172.21.15.142:0/4173414609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8840ff310 msgr2=0x7ff88419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:03.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff88259c700 1 --2- 172.21.15.142:0/4173414609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8840ff310 0x7ff88419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff88259c700 1 -- 172.21.15.142:0/4173414609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8740095f0 con 0x7ff884106c10 2024-03-29T05:00:03.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff88259c700 1 --2- 172.21.15.142:0/4173414609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff884106c10 0x7ff88419cb50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff874006010 tx=0x7ff87400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:03.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.579+0000 7ff86bfff700 1 -- 172.21.15.142:0/4173414609 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff87401d070 con 0x7ff884106c10 2024-03-29T05:00:03.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.580+0000 7ff8890ee700 1 -- 172.21.15.142:0/4173414609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff884196960 con 0x7ff884106c10 2024-03-29T05:00:03.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.580+0000 7ff86bfff700 1 -- 172.21.15.142:0/4173414609 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff87400f460 con 0x7ff884106c10 2024-03-29T05:00:03.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.580+0000 7ff86bfff700 1 -- 172.21.15.142:0/4173414609 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff874017610 con 0x7ff884106c10 2024-03-29T05:00:03.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.580+0000 7ff8890ee700 1 -- 172.21.15.142:0/4173414609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff884196e50 con 0x7ff884106c10 2024-03-29T05:00:03.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.581+0000 7ff8890ee700 1 -- 172.21.15.142:0/4173414609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff884109130 con 0x7ff884106c10 2024-03-29T05:00:03.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.581+0000 7ff86bfff700 1 -- 172.21.15.142:0/4173414609 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff87400f5d0 con 0x7ff884106c10 2024-03-29T05:00:03.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.581+0000 7ff86bfff700 1 --2- 172.21.15.142:0/4173414609 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff86c06dd60 0x7ff86c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:03.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.581+0000 7ff86bfff700 1 -- 172.21.15.142:0/4173414609 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff87408f200 con 0x7ff884106c10 2024-03-29T05:00:03.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.581+0000 7ff882d9d700 1 --2- 172.21.15.142:0/4173414609 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff86c06dd60 0x7ff86c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:03.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.582+0000 7ff882d9d700 1 --2- 172.21.15.142:0/4173414609 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff86c06dd60 0x7ff86c070220 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff878009a10 tx=0x7ff878009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:03.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.586+0000 7ff86bfff700 1 -- 172.21.15.142:0/4173414609 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff87405c150 con 0x7ff884106c10 2024-03-29T05:00:03.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:03 smithi152 ceph-mon[88186]: pgmap v352: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:03.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:03 smithi152 ceph-mon[88186]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:03.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:03 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1642652217' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:00:03.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.824+0000 7ff8890ee700 1 -- 172.21.15.142:0/4173414609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff88404ea90 con 0x7ff884106c10 2024-03-29T05:00:03.833 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:03 smithi142 ceph-mon[112475]: pgmap v352: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:03.833 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:03 smithi142 ceph-mon[112475]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:03.833 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:03 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1642652217' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:00:03.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.824+0000 7ff86bfff700 1 -- 172.21.15.142:0/4173414609 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7ff87405bce0 con 0x7ff884106c10 2024-03-29T05:00:03.833 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:00:03.833 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:00:03.833 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:00:03.833 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:00:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:00:03.835 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:00:03.836 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:00:03.836 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:00:03.836 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:00:03.836 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:00:03.836 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:00:03.836 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:00:03.836 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:00:03.836 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:00:03.836 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:00:03.836 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:00:03.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.827+0000 7ff8890ee700 1 -- 172.21.15.142:0/4173414609 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff86c06dd60 msgr2=0x7ff86c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:03.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.827+0000 7ff8890ee700 1 --2- 172.21.15.142:0/4173414609 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff86c06dd60 0x7ff86c070220 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff878009a10 tx=0x7ff878009190 comp rx=0 tx=0).stop 2024-03-29T05:00:03.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.827+0000 7ff8890ee700 1 -- 172.21.15.142:0/4173414609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff884106c10 msgr2=0x7ff88419cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:03.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.827+0000 7ff8890ee700 1 --2- 172.21.15.142:0/4173414609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff884106c10 0x7ff88419cb50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff874006010 tx=0x7ff87400bd70 comp rx=0 tx=0).stop 2024-03-29T05:00:03.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.827+0000 7ff8890ee700 1 -- 172.21.15.142:0/4173414609 shutdown_connections 2024-03-29T05:00:03.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.827+0000 7ff8890ee700 1 --2- 172.21.15.142:0/4173414609 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff86c06dd60 0x7ff86c070220 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.827+0000 7ff8890ee700 1 --2- 172.21.15.142:0/4173414609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8840ff310 0x7ff88419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.827+0000 7ff8890ee700 1 --2- 172.21.15.142:0/4173414609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff884106c10 0x7ff88419cb50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.827+0000 7ff8890ee700 1 -- 172.21.15.142:0/4173414609 >> 172.21.15.142:0/4173414609 conn(0x7ff884074bf0 msgr2=0x7ff8840fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:03.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.827+0000 7ff8890ee700 1 -- 172.21.15.142:0/4173414609 shutdown_connections 2024-03-29T05:00:03.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.827+0000 7ff8890ee700 1 -- 172.21.15.142:0/4173414609 wait complete. 2024-03-29T05:00:03.838 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:00:03.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.932+0000 7f891692b700 1 -- 172.21.15.142:0/3544046698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89101003e0 msgr2=0x7f8910100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:03.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.932+0000 7f891692b700 1 --2- 172.21.15.142:0/3544046698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89101003e0 0x7f8910100860 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8900009910 tx=0x7f8900009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:03.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.932+0000 7f891692b700 1 -- 172.21.15.142:0/3544046698 shutdown_connections 2024-03-29T05:00:03.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.932+0000 7f891692b700 1 --2- 172.21.15.142:0/3544046698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89101003e0 0x7f8910100860 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.932+0000 7f891692b700 1 --2- 172.21.15.142:0/3544046698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8910104ae0 0x7f8910104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.932+0000 7f891692b700 1 -- 172.21.15.142:0/3544046698 >> 172.21.15.142:0/3544046698 conn(0x7f8910074df0 msgr2=0x7f8910075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:03.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.932+0000 7f891692b700 1 -- 172.21.15.142:0/3544046698 shutdown_connections 2024-03-29T05:00:03.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.932+0000 7f891692b700 1 -- 172.21.15.142:0/3544046698 wait complete. 2024-03-29T05:00:03.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.932+0000 7f891692b700 1 Processor -- start 2024-03-29T05:00:03.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f891692b700 1 -- start start 2024-03-29T05:00:03.942 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f891692b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89101003e0 0x7f891019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:03.942 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f891692b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8910104ae0 0x7f891019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f891692b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891019d170 con 0x7f8910104ae0 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f891692b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8910196620 con 0x7f89101003e0 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f890f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8910104ae0 0x7f891019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f890f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8910104ae0 0x7f891019cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37642/0 (socket says 172.21.15.142:37642) 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f890f7fe700 1 -- 172.21.15.142:0/3351106149 learned_addr learned my addr 172.21.15.142:0/3351106149 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f890ffff700 1 --2- 172.21.15.142:0/3351106149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89101003e0 0x7f891019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f890f7fe700 1 -- 172.21.15.142:0/3351106149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89101003e0 msgr2=0x7f891019c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f890f7fe700 1 --2- 172.21.15.142:0/3351106149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89101003e0 0x7f891019c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f890f7fe700 1 -- 172.21.15.142:0/3351106149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89000095f0 con 0x7f8910104ae0 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.933+0000 7f890f7fe700 1 --2- 172.21.15.142:0/3351106149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8910104ae0 0x7f891019cae0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8900006010 tx=0x7f890000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.934+0000 7f890d7fa700 1 -- 172.21.15.142:0/3351106149 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f890001d070 con 0x7f8910104ae0 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.934+0000 7f891692b700 1 -- 172.21.15.142:0/3351106149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89101968a0 con 0x7f8910104ae0 2024-03-29T05:00:03.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.934+0000 7f890d7fa700 1 -- 172.21.15.142:0/3351106149 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8900005dc0 con 0x7f8910104ae0 2024-03-29T05:00:03.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.934+0000 7f890d7fa700 1 -- 172.21.15.142:0/3351106149 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8900017400 con 0x7f8910104ae0 2024-03-29T05:00:03.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.934+0000 7f891692b700 1 -- 172.21.15.142:0/3351106149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8910196d90 con 0x7f8910104ae0 2024-03-29T05:00:03.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.935+0000 7f891692b700 1 -- 172.21.15.142:0/3351106149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891004ea90 con 0x7f8910104ae0 2024-03-29T05:00:03.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.935+0000 7f890d7fa700 1 -- 172.21.15.142:0/3351106149 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f8900003a60 con 0x7f8910104ae0 2024-03-29T05:00:03.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.935+0000 7f890d7fa700 1 --2- 172.21.15.142:0/3351106149 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f88f806dd60 0x7f88f8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:03.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.936+0000 7f890ffff700 1 --2- 172.21.15.142:0/3351106149 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f88f806dd60 0x7f88f8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:03.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.936+0000 7f890d7fa700 1 -- 172.21.15.142:0/3351106149 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f890008f130 con 0x7f8910104ae0 2024-03-29T05:00:03.945 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.936+0000 7f890ffff700 1 --2- 172.21.15.142:0/3351106149 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f88f806dd60 0x7f88f8070220 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8904005950 tx=0x7f890400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:03.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:03.940+0000 7f890d7fa700 1 -- 172.21.15.142:0/3351106149 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f890005c080 con 0x7f8910104ae0 2024-03-29T05:00:04.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.156+0000 7f891692b700 1 -- 172.21.15.142:0/3351106149 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f89101970d0 con 0x7f88f806dd60 2024-03-29T05:00:04.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.158+0000 7f890d7fa700 1 -- 172.21.15.142:0/3351106149 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f89101970d0 con 0x7f88f806dd60 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:00:04.167 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:00:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:00:04.168 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:00:04.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.160+0000 7f891692b700 1 -- 172.21.15.142:0/3351106149 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f88f806dd60 msgr2=0x7f88f8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:04.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.160+0000 7f891692b700 1 --2- 172.21.15.142:0/3351106149 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f88f806dd60 0x7f88f8070220 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8904005950 tx=0x7f890400b410 comp rx=0 tx=0).stop 2024-03-29T05:00:04.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.160+0000 7f891692b700 1 -- 172.21.15.142:0/3351106149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8910104ae0 msgr2=0x7f891019cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:04.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.160+0000 7f891692b700 1 --2- 172.21.15.142:0/3351106149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8910104ae0 0x7f891019cae0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8900006010 tx=0x7f890000bd70 comp rx=0 tx=0).stop 2024-03-29T05:00:04.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.160+0000 7f891692b700 1 -- 172.21.15.142:0/3351106149 shutdown_connections 2024-03-29T05:00:04.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.160+0000 7f891692b700 1 --2- 172.21.15.142:0/3351106149 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f88f806dd60 0x7f88f8070220 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:04.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.160+0000 7f891692b700 1 --2- 172.21.15.142:0/3351106149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89101003e0 0x7f891019c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:04.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.160+0000 7f891692b700 1 --2- 172.21.15.142:0/3351106149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8910104ae0 0x7f891019cae0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:04.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.160+0000 7f891692b700 1 -- 172.21.15.142:0/3351106149 >> 172.21.15.142:0/3351106149 conn(0x7f8910074df0 msgr2=0x7f89100ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:04.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.160+0000 7f891692b700 1 -- 172.21.15.142:0/3351106149 shutdown_connections 2024-03-29T05:00:04.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.160+0000 7f891692b700 1 -- 172.21.15.142:0/3351106149 wait complete. 2024-03-29T05:00:04.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.265+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3428171893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf0100de0 msgr2=0x7f8cf01011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:04.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.265+0000 7f8cf4dc1700 1 --2- 172.21.15.142:0/3428171893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf0100de0 0x7f8cf01011c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009910 tx=0x7f8ce0009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:04.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.265+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3428171893 shutdown_connections 2024-03-29T05:00:04.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.265+0000 7f8cf4dc1700 1 --2- 172.21.15.142:0/3428171893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf0101790 0x7f8cf01057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:04.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.265+0000 7f8cf4dc1700 1 --2- 172.21.15.142:0/3428171893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf0100de0 0x7f8cf01011c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:04.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.265+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3428171893 >> 172.21.15.142:0/3428171893 conn(0x7f8cf00fc6b0 msgr2=0x7f8cf00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:04.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.265+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3428171893 shutdown_connections 2024-03-29T05:00:04.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.265+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3428171893 wait complete. 2024-03-29T05:00:04.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.265+0000 7f8cf4dc1700 1 Processor -- start 2024-03-29T05:00:04.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8cf4dc1700 1 -- start start 2024-03-29T05:00:04.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8cf4dc1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf0100de0 0x7f8cf019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:04.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8cf4dc1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf0101790 0x7f8cf019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:04.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8cf4dc1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf019d190 con 0x7f8cf0101790 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8cf4dc1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf0196640 con 0x7f8cf0100de0 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8ce5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf0101790 0x7f8cf019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8ce5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf0101790 0x7f8cf019cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37650/0 (socket says 172.21.15.142:37650) 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8ce5d9b700 1 -- 172.21.15.142:0/3089291451 learned_addr learned my addr 172.21.15.142:0/3089291451 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8cee59c700 1 --2- 172.21.15.142:0/3089291451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf0100de0 0x7f8cf019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8ce5d9b700 1 -- 172.21.15.142:0/3089291451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf0100de0 msgr2=0x7f8cf019c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8ce5d9b700 1 --2- 172.21.15.142:0/3089291451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf0100de0 0x7f8cf019c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8ce5d9b700 1 -- 172.21.15.142:0/3089291451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce00095f0 con 0x7f8cf0101790 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.266+0000 7f8ce5d9b700 1 --2- 172.21.15.142:0/3089291451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf0101790 0x7f8cf019cb00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800c8f0 tx=0x7f8cd800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.267+0000 7f8ce7fff700 1 -- 172.21.15.142:0/3089291451 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cd8007a50 con 0x7f8cf0101790 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.267+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3089291451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf0196920 con 0x7f8cf0101790 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.267+0000 7f8ce7fff700 1 -- 172.21.15.142:0/3089291451 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cd8007bb0 con 0x7f8cf0101790 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.267+0000 7f8ce7fff700 1 -- 172.21.15.142:0/3089291451 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cd8018630 con 0x7f8cf0101790 2024-03-29T05:00:04.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.267+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3089291451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf0196e70 con 0x7f8cf0101790 2024-03-29T05:00:04.277 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.268+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3089291451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf0109180 con 0x7f8cf0101790 2024-03-29T05:00:04.277 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.268+0000 7f8ce7fff700 1 -- 172.21.15.142:0/3089291451 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f8cd8007d20 con 0x7f8cf0101790 2024-03-29T05:00:04.277 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.268+0000 7f8ce7fff700 1 --2- 172.21.15.142:0/3089291451 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8cdc06dd60 0x7f8cdc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:04.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.269+0000 7f8ce7fff700 1 -- 172.21.15.142:0/3089291451 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f8cd808dc70 con 0x7f8cf0101790 2024-03-29T05:00:04.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.269+0000 7f8cee59c700 1 --2- 172.21.15.142:0/3089291451 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8cdc06dd60 0x7f8cdc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:04.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.269+0000 7f8cee59c700 1 --2- 172.21.15.142:0/3089291451 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8cdc06dd60 0x7f8cdc070220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000b5c0 tx=0x7f8ce001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:04.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.273+0000 7f8ce7fff700 1 -- 172.21.15.142:0/3089291451 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8cd805abc0 con 0x7f8cf0101790 2024-03-29T05:00:04.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.554+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3089291451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8cf01970f0 con 0x7f8cf0101790 2024-03-29T05:00:04.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.555+0000 7f8ce7fff700 1 -- 172.21.15.142:0/3089291451 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f8cd805a750 con 0x7f8cf0101790 2024-03-29T05:00:04.563 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:00:04.563 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:00:04.563 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:00:04.563 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:00:04.564 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:00:04.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.557+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3089291451 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8cdc06dd60 msgr2=0x7f8cdc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:04.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.557+0000 7f8cf4dc1700 1 --2- 172.21.15.142:0/3089291451 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8cdc06dd60 0x7f8cdc070220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000b5c0 tx=0x7f8ce001a040 comp rx=0 tx=0).stop 2024-03-29T05:00:04.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.557+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3089291451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf0101790 msgr2=0x7f8cf019cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:04.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.557+0000 7f8cf4dc1700 1 --2- 172.21.15.142:0/3089291451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf0101790 0x7f8cf019cb00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800c8f0 tx=0x7f8cd800ccb0 comp rx=0 tx=0).stop 2024-03-29T05:00:04.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.557+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3089291451 shutdown_connections 2024-03-29T05:00:04.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.557+0000 7f8cf4dc1700 1 --2- 172.21.15.142:0/3089291451 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8cdc06dd60 0x7f8cdc070220 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:04.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.557+0000 7f8cf4dc1700 1 --2- 172.21.15.142:0/3089291451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8cf0100de0 0x7f8cf019c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:04.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.557+0000 7f8cf4dc1700 1 --2- 172.21.15.142:0/3089291451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cf0101790 0x7f8cf019cb00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:04.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.557+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3089291451 >> 172.21.15.142:0/3089291451 conn(0x7f8cf00fc6b0 msgr2=0x7f8cf00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:04.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.557+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3089291451 shutdown_connections 2024-03-29T05:00:04.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:04.558+0000 7f8cf4dc1700 1 -- 172.21.15.142:0/3089291451 wait complete. 2024-03-29T05:00:04.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:04 smithi152 ceph-mon[88186]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:04.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:04 smithi152 ceph-mon[88186]: from='client.34690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:04.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:04 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/4173414609' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:00:04.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:04 smithi142 ceph-mon[112475]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:04.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:04 smithi142 ceph-mon[112475]: from='client.34690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:04.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:04 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/4173414609' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:00:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:05 smithi152 ceph-mon[88186]: pgmap v353: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:05 smithi152 ceph-mon[88186]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:05 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3089291451' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:00:05.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:05 smithi142 ceph-mon[112475]: pgmap v353: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:05.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:05 smithi142 ceph-mon[112475]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:05.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:05 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3089291451' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:00:07.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:07 smithi142 ceph-mon[112475]: pgmap v354: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:08.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:07 smithi152 ceph-mon[88186]: pgmap v354: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:09.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:09 smithi142 ceph-mon[112475]: pgmap v355: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:10.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:09 smithi152 ceph-mon[88186]: pgmap v355: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:11.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:11 smithi142 ceph-mon[112475]: pgmap v356: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:12.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:11 smithi152 ceph-mon[88186]: pgmap v356: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:13.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:13 smithi142 ceph-mon[112475]: pgmap v357: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:14.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:13 smithi152 ceph-mon[88186]: pgmap v357: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:15.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:15 smithi142 ceph-mon[112475]: pgmap v358: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:15.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:00:16.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:15 smithi152 ceph-mon[88186]: pgmap v358: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:16.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:00:17.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:17 smithi142 ceph-mon[112475]: pgmap v359: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:18.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:17 smithi152 ceph-mon[88186]: pgmap v359: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:19.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:19 smithi142 ceph-mon[112475]: pgmap v360: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:20.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:19 smithi152 ceph-mon[88186]: pgmap v360: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:21.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:21 smithi142 ceph-mon[112475]: pgmap v361: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:22.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:21 smithi152 ceph-mon[88186]: pgmap v361: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:23.850 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:23 smithi152 ceph-mon[88186]: pgmap v362: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:23.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:23 smithi142 ceph-mon[112475]: pgmap v362: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:25.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:25 smithi142 ceph-mon[112475]: pgmap v363: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:26.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:25 smithi152 ceph-mon[88186]: pgmap v363: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:27.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:27 smithi142 ceph-mon[112475]: pgmap v364: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:28.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:27 smithi152 ceph-mon[88186]: pgmap v364: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:29.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:29 smithi142 ceph-mon[112475]: pgmap v365: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:30.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:29 smithi152 ceph-mon[88186]: pgmap v365: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:30.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:00:31.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:00:31.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:31 smithi142 ceph-mon[112475]: pgmap v366: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:32.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:31 smithi152 ceph-mon[88186]: pgmap v366: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:33.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:33 smithi142 ceph-mon[112475]: pgmap v367: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:34.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:33 smithi152 ceph-mon[88186]: pgmap v367: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:34.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.676+0000 7f7f056a7700 1 -- 172.21.15.142:0/3964709957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f000ffa40 msgr2=0x7f7f000ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:34.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.676+0000 7f7f056a7700 1 --2- 172.21.15.142:0/3964709957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f000ffa40 0x7f7f000ffec0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009910 tx=0x7f7ef4009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:34.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.676+0000 7f7f056a7700 1 -- 172.21.15.142:0/3964709957 shutdown_connections 2024-03-29T05:00:34.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.676+0000 7f7f056a7700 1 --2- 172.21.15.142:0/3964709957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f000ffa40 0x7f7f000ffec0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:34.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.676+0000 7f7f056a7700 1 --2- 172.21.15.142:0/3964709957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f00105a40 0x7f7f00105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:34.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.676+0000 7f7f056a7700 1 -- 172.21.15.142:0/3964709957 >> 172.21.15.142:0/3964709957 conn(0x7f7f000fb550 msgr2=0x7f7f000fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:34.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.676+0000 7f7f056a7700 1 -- 172.21.15.142:0/3964709957 shutdown_connections 2024-03-29T05:00:34.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.676+0000 7f7f056a7700 1 -- 172.21.15.142:0/3964709957 wait complete. 2024-03-29T05:00:34.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.676+0000 7f7f056a7700 1 Processor -- start 2024-03-29T05:00:34.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.677+0000 7f7f056a7700 1 -- start start 2024-03-29T05:00:34.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.677+0000 7f7f056a7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f000ffa40 0x7f7f001975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:34.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.677+0000 7f7f056a7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f00105a40 0x7f7f00197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:34.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.677+0000 7f7f056a7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00198110 con 0x7f7f000ffa40 2024-03-29T05:00:34.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.677+0000 7f7f056a7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00198250 con 0x7f7f00105a40 2024-03-29T05:00:34.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.677+0000 7f7efeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f000ffa40 0x7f7f001975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:34.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.677+0000 7f7efeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f000ffa40 0x7f7f001975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43880/0 (socket says 172.21.15.142:43880) 2024-03-29T05:00:34.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.677+0000 7f7efeffd700 1 -- 172.21.15.142:0/2436881777 learned_addr learned my addr 172.21.15.142:0/2436881777 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:34.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.677+0000 7f7efe7fc700 1 --2- 172.21.15.142:0/2436881777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f00105a40 0x7f7f00197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:34.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.677+0000 7f7efeffd700 1 -- 172.21.15.142:0/2436881777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f00105a40 msgr2=0x7f7f00197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:34.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.677+0000 7f7efeffd700 1 --2- 172.21.15.142:0/2436881777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f00105a40 0x7f7f00197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:34.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.677+0000 7f7efeffd700 1 -- 172.21.15.142:0/2436881777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef40095f0 con 0x7f7f000ffa40 2024-03-29T05:00:34.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.678+0000 7f7efeffd700 1 --2- 172.21.15.142:0/2436881777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f000ffa40 0x7f7f001975b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000ada0 tx=0x7f7ef000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:34.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.678+0000 7f7ee7fff700 1 -- 172.21.15.142:0/2436881777 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ef000f070 con 0x7f7f000ffa40 2024-03-29T05:00:34.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.678+0000 7f7f056a7700 1 -- 172.21.15.142:0/2436881777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f001a0620 con 0x7f7f000ffa40 2024-03-29T05:00:34.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.678+0000 7f7ee7fff700 1 -- 172.21.15.142:0/2436881777 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ef0007bf0 con 0x7f7f000ffa40 2024-03-29T05:00:34.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.678+0000 7f7ee7fff700 1 -- 172.21.15.142:0/2436881777 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ef0005690 con 0x7f7f000ffa40 2024-03-29T05:00:34.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.678+0000 7f7f056a7700 1 -- 172.21.15.142:0/2436881777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f001a0ae0 con 0x7f7f000ffa40 2024-03-29T05:00:34.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.679+0000 7f7f056a7700 1 -- 172.21.15.142:0/2436881777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f00107fd0 con 0x7f7f000ffa40 2024-03-29T05:00:34.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.679+0000 7f7ee7fff700 1 -- 172.21.15.142:0/2436881777 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f7ef0005900 con 0x7f7f000ffa40 2024-03-29T05:00:34.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.680+0000 7f7ee7fff700 1 --2- 172.21.15.142:0/2436881777 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7ee806dd60 0x7f7ee8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:34.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.680+0000 7f7ee7fff700 1 -- 172.21.15.142:0/2436881777 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f7ef008d6e0 con 0x7f7f000ffa40 2024-03-29T05:00:34.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.680+0000 7f7efe7fc700 1 --2- 172.21.15.142:0/2436881777 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7ee806dd60 0x7f7ee8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:34.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.680+0000 7f7efe7fc700 1 --2- 172.21.15.142:0/2436881777 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7ee806dd60 0x7f7ee8070220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7f00069660 tx=0x7f7ef401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:34.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.685+0000 7f7ee7fff700 1 -- 172.21.15.142:0/2436881777 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7ef0056b00 con 0x7f7f000ffa40 2024-03-29T05:00:34.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.899+0000 7f7f056a7700 1 -- 172.21.15.142:0/2436881777 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f0002cd80 con 0x7f7ee806dd60 2024-03-29T05:00:34.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.901+0000 7f7ee7fff700 1 -- 172.21.15.142:0/2436881777 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7f0002cd80 con 0x7f7ee806dd60 2024-03-29T05:00:34.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.903+0000 7f7f056a7700 1 -- 172.21.15.142:0/2436881777 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7ee806dd60 msgr2=0x7f7ee8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:34.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.903+0000 7f7f056a7700 1 --2- 172.21.15.142:0/2436881777 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7ee806dd60 0x7f7ee8070220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7f00069660 tx=0x7f7ef401a040 comp rx=0 tx=0).stop 2024-03-29T05:00:34.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.903+0000 7f7f056a7700 1 -- 172.21.15.142:0/2436881777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f000ffa40 msgr2=0x7f7f001975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:34.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.903+0000 7f7f056a7700 1 --2- 172.21.15.142:0/2436881777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f000ffa40 0x7f7f001975b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000ada0 tx=0x7f7ef000c330 comp rx=0 tx=0).stop 2024-03-29T05:00:34.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.903+0000 7f7f056a7700 1 -- 172.21.15.142:0/2436881777 shutdown_connections 2024-03-29T05:00:34.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.903+0000 7f7f056a7700 1 --2- 172.21.15.142:0/2436881777 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7ee806dd60 0x7f7ee8070220 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:34.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.903+0000 7f7f056a7700 1 --2- 172.21.15.142:0/2436881777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f000ffa40 0x7f7f001975b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:34.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.903+0000 7f7f056a7700 1 --2- 172.21.15.142:0/2436881777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f00105a40 0x7f7f00197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:34.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.903+0000 7f7f056a7700 1 -- 172.21.15.142:0/2436881777 >> 172.21.15.142:0/2436881777 conn(0x7f7f000fb550 msgr2=0x7f7f000fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:34.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.903+0000 7f7f056a7700 1 -- 172.21.15.142:0/2436881777 shutdown_connections 2024-03-29T05:00:34.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:34.903+0000 7f7f056a7700 1 -- 172.21.15.142:0/2436881777 wait complete. 2024-03-29T05:00:34.922 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:00:35.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.007+0000 7fd16b35a700 1 -- 172.21.15.142:0/3889406176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd164101770 msgr2=0x7fd1641057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:35.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.007+0000 7fd16b35a700 1 --2- 172.21.15.142:0/3889406176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd164101770 0x7fd1641057c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd154009910 tx=0x7fd154009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:35.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.008+0000 7fd16b35a700 1 -- 172.21.15.142:0/3889406176 shutdown_connections 2024-03-29T05:00:35.016 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.008+0000 7fd16b35a700 1 --2- 172.21.15.142:0/3889406176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd164101770 0x7fd1641057c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.008+0000 7fd16b35a700 1 --2- 172.21.15.142:0/3889406176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd164100dc0 0x7fd1641011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.008+0000 7fd16b35a700 1 -- 172.21.15.142:0/3889406176 >> 172.21.15.142:0/3889406176 conn(0x7fd1640fc650 msgr2=0x7fd1640fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:35.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.008+0000 7fd16b35a700 1 -- 172.21.15.142:0/3889406176 shutdown_connections 2024-03-29T05:00:35.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.008+0000 7fd16b35a700 1 -- 172.21.15.142:0/3889406176 wait complete. 2024-03-29T05:00:35.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.008+0000 7fd16b35a700 1 Processor -- start 2024-03-29T05:00:35.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.008+0000 7fd16b35a700 1 -- start start 2024-03-29T05:00:35.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd16b35a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd164100dc0 0x7fd16419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:35.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd16b35a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd164101770 0x7fd16419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:35.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd16b35a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16419d130 con 0x7fd164101770 2024-03-29T05:00:35.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd16b35a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1641965e0 con 0x7fd164100dc0 2024-03-29T05:00:35.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd1688f5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd164101770 0x7fd16419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:35.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd1688f5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd164101770 0x7fd16419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43904/0 (socket says 172.21.15.142:43904) 2024-03-29T05:00:35.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd1688f5700 1 -- 172.21.15.142:0/1495868922 learned_addr learned my addr 172.21.15.142:0/1495868922 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:35.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd1690f6700 1 --2- 172.21.15.142:0/1495868922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd164100dc0 0x7fd16419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:35.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd1688f5700 1 -- 172.21.15.142:0/1495868922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd164100dc0 msgr2=0x7fd16419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:35.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd1688f5700 1 --2- 172.21.15.142:0/1495868922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd164100dc0 0x7fd16419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd1688f5700 1 -- 172.21.15.142:0/1495868922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1540095f0 con 0x7fd164101770 2024-03-29T05:00:35.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd1688f5700 1 --2- 172.21.15.142:0/1495868922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd164101770 0x7fd16419caa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd15400bda0 tx=0x7fd15400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:35.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd15a7fc700 1 -- 172.21.15.142:0/1495868922 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15401d070 con 0x7fd164101770 2024-03-29T05:00:35.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd15a7fc700 1 -- 172.21.15.142:0/1495868922 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd15400f460 con 0x7fd164101770 2024-03-29T05:00:35.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd15a7fc700 1 -- 172.21.15.142:0/1495868922 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd154017630 con 0x7fd164101770 2024-03-29T05:00:35.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.009+0000 7fd16b35a700 1 -- 172.21.15.142:0/1495868922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd164196860 con 0x7fd164101770 2024-03-29T05:00:35.020 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.010+0000 7fd16b35a700 1 -- 172.21.15.142:0/1495868922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd164196d50 con 0x7fd164101770 2024-03-29T05:00:35.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.011+0000 7fd16b35a700 1 -- 172.21.15.142:0/1495868922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1641091c0 con 0x7fd164101770 2024-03-29T05:00:35.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.011+0000 7fd15a7fc700 1 -- 172.21.15.142:0/1495868922 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fd15400fa40 con 0x7fd164101770 2024-03-29T05:00:35.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.011+0000 7fd15a7fc700 1 --2- 172.21.15.142:0/1495868922 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd15006dd60 0x7fd150070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:35.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.012+0000 7fd15a7fc700 1 -- 172.21.15.142:0/1495868922 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd15408f150 con 0x7fd164101770 2024-03-29T05:00:35.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.012+0000 7fd1690f6700 1 --2- 172.21.15.142:0/1495868922 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd15006dd60 0x7fd150070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:35.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.012+0000 7fd1690f6700 1 --2- 172.21.15.142:0/1495868922 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd15006dd60 0x7fd150070220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd160005950 tx=0x7fd1600058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:35.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.016+0000 7fd15a7fc700 1 -- 172.21.15.142:0/1495868922 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd15405c0a0 con 0x7fd164101770 2024-03-29T05:00:35.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.232+0000 7fd16b35a700 1 -- 172.21.15.142:0/1495868922 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd164197080 con 0x7fd15006dd60 2024-03-29T05:00:35.241 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.233+0000 7fd15a7fc700 1 -- 172.21.15.142:0/1495868922 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd164197080 con 0x7fd15006dd60 2024-03-29T05:00:35.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.235+0000 7fd16b35a700 1 -- 172.21.15.142:0/1495868922 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd15006dd60 msgr2=0x7fd150070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:35.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.235+0000 7fd16b35a700 1 --2- 172.21.15.142:0/1495868922 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd15006dd60 0x7fd150070220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd160005950 tx=0x7fd1600058e0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.235+0000 7fd16b35a700 1 -- 172.21.15.142:0/1495868922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd164101770 msgr2=0x7fd16419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:35.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.235+0000 7fd16b35a700 1 --2- 172.21.15.142:0/1495868922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd164101770 0x7fd16419caa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd15400bda0 tx=0x7fd15400be80 comp rx=0 tx=0).stop 2024-03-29T05:00:35.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.235+0000 7fd16b35a700 1 -- 172.21.15.142:0/1495868922 shutdown_connections 2024-03-29T05:00:35.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.235+0000 7fd16b35a700 1 --2- 172.21.15.142:0/1495868922 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd15006dd60 0x7fd150070220 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.235+0000 7fd16b35a700 1 --2- 172.21.15.142:0/1495868922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd164100dc0 0x7fd16419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.235+0000 7fd16b35a700 1 --2- 172.21.15.142:0/1495868922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd164101770 0x7fd16419caa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.235+0000 7fd16b35a700 1 -- 172.21.15.142:0/1495868922 >> 172.21.15.142:0/1495868922 conn(0x7fd1640fc650 msgr2=0x7fd1640fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:35.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.235+0000 7fd16b35a700 1 -- 172.21.15.142:0/1495868922 shutdown_connections 2024-03-29T05:00:35.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.236+0000 7fd16b35a700 1 -- 172.21.15.142:0/1495868922 wait complete. 2024-03-29T05:00:35.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.338+0000 7f44e2571700 1 -- 172.21.15.142:0/901786998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44dc1026f0 msgr2=0x7f44dc10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:35.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.338+0000 7f44e2571700 1 --2- 172.21.15.142:0/901786998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44dc1026f0 0x7f44dc10abe0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009940 tx=0x7f44cc009c50 comp rx=0 tx=0).stop 2024-03-29T05:00:35.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.338+0000 7f44e2571700 1 -- 172.21.15.142:0/901786998 shutdown_connections 2024-03-29T05:00:35.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.338+0000 7f44e2571700 1 --2- 172.21.15.142:0/901786998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44dc1026f0 0x7f44dc10abe0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.338+0000 7f44e2571700 1 --2- 172.21.15.142:0/901786998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44dc101dd0 0x7f44dc1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.338+0000 7f44e2571700 1 -- 172.21.15.142:0/901786998 >> 172.21.15.142:0/901786998 conn(0x7f44dc0fb540 msgr2=0x7f44dc0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:35.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.338+0000 7f44e2571700 1 -- 172.21.15.142:0/901786998 shutdown_connections 2024-03-29T05:00:35.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.338+0000 7f44e2571700 1 -- 172.21.15.142:0/901786998 wait complete. 2024-03-29T05:00:35.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.338+0000 7f44e2571700 1 Processor -- start 2024-03-29T05:00:35.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.339+0000 7f44e2571700 1 -- start start 2024-03-29T05:00:35.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.339+0000 7f44e2571700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44dc101dd0 0x7f44dc0ffa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:35.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.339+0000 7f44e2571700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44dc1026f0 0x7f44dc0fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:35.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.339+0000 7f44e2571700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1004b0 con 0x7f44dc1026f0 2024-03-29T05:00:35.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.339+0000 7f44e2571700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1005f0 con 0x7f44dc101dd0 2024-03-29T05:00:35.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.339+0000 7f44db7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44dc1026f0 0x7f44dc0fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:35.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.339+0000 7f44db7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44dc1026f0 0x7f44dc0fff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43930/0 (socket says 172.21.15.142:43930) 2024-03-29T05:00:35.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.339+0000 7f44db7fe700 1 -- 172.21.15.142:0/2903854445 learned_addr learned my addr 172.21.15.142:0/2903854445 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:35.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.339+0000 7f44dbfff700 1 --2- 172.21.15.142:0/2903854445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44dc101dd0 0x7f44dc0ffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:35.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.339+0000 7f44db7fe700 1 -- 172.21.15.142:0/2903854445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44dc101dd0 msgr2=0x7f44dc0ffa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:35.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.339+0000 7f44db7fe700 1 --2- 172.21.15.142:0/2903854445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44dc101dd0 0x7f44dc0ffa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.339+0000 7f44db7fe700 1 -- 172.21.15.142:0/2903854445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc0095f0 con 0x7f44dc1026f0 2024-03-29T05:00:35.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.340+0000 7f44db7fe700 1 --2- 172.21.15.142:0/2903854445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44dc1026f0 0x7f44dc0fff70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00b580 tx=0x7f44cc0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:35.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.340+0000 7f44d97fa700 1 -- 172.21.15.142:0/2903854445 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44cc01d070 con 0x7f44dc1026f0 2024-03-29T05:00:35.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.340+0000 7f44d97fa700 1 -- 172.21.15.142:0/2903854445 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44cc005dc0 con 0x7f44dc1026f0 2024-03-29T05:00:35.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.340+0000 7f44d97fa700 1 -- 172.21.15.142:0/2903854445 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44cc021410 con 0x7f44dc1026f0 2024-03-29T05:00:35.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.340+0000 7f44e2571700 1 -- 172.21.15.142:0/2903854445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44dc100870 con 0x7f44dc1026f0 2024-03-29T05:00:35.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.340+0000 7f44e2571700 1 -- 172.21.15.142:0/2903854445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44dc1a2c20 con 0x7f44dc1026f0 2024-03-29T05:00:35.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.341+0000 7f44e2571700 1 -- 172.21.15.142:0/2903854445 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44dc1082e0 con 0x7f44dc1026f0 2024-03-29T05:00:35.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.341+0000 7f44d97fa700 1 -- 172.21.15.142:0/2903854445 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f44cc003a60 con 0x7f44dc1026f0 2024-03-29T05:00:35.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.342+0000 7f44d97fa700 1 --2- 172.21.15.142:0/2903854445 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f44c806dd60 0x7f44c8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:35.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.342+0000 7f44d97fa700 1 -- 172.21.15.142:0/2903854445 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f44cc08ed90 con 0x7f44dc1026f0 2024-03-29T05:00:35.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.342+0000 7f44dbfff700 1 --2- 172.21.15.142:0/2903854445 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f44c806dd60 0x7f44c8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:35.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.342+0000 7f44dbfff700 1 --2- 172.21.15.142:0/2903854445 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f44c806dd60 0x7f44c8070220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f44c4005950 tx=0x7f44c400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:35.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.346+0000 7f44d97fa700 1 -- 172.21.15.142:0/2903854445 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f44cc058230 con 0x7f44dc1026f0 2024-03-29T05:00:35.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.550+0000 7f44e2571700 1 -- 172.21.15.142:0/2903854445 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f44dc0611d0 con 0x7f44c806dd60 2024-03-29T05:00:35.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.561+0000 7f44d97fa700 1 -- 172.21.15.142:0/2903854445 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f44dc0611d0 con 0x7f44c806dd60 2024-03-29T05:00:35.569 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:00:35.569 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (12m) 5m ago 23m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:00:35.569 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (24m) 5m ago 24m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:00:35.569 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (23m) 4m ago 23m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:00:35.569 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (9m) 5m ago 24m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (9m) 4m ago 23m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (12m) 5m ago 23m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (5m) 5m ago 18m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (5m) 5m ago 18m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (5m) 4m ago 18m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (5m) 4m ago 18m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (13m) 5m ago 25m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (13m) 4m ago 22m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (10m) 5m ago 25m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (10m) 4m ago 22m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (13m) 5m ago 24m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (13m) 4m ago 22m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:00:35.570 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (9m) 5m ago 22m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:00:35.571 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (8m) 5m ago 21m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:00:35.571 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (7m) 5m ago 21m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:00:35.571 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (7m) 4m ago 20m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:00:35.571 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (6m) 4m ago 20m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:00:35.571 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (6m) 4m ago 19m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:00:35.571 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (12m) 5m ago 23m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:00:35.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.563+0000 7f44e2571700 1 -- 172.21.15.142:0/2903854445 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f44c806dd60 msgr2=0x7f44c8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:35.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.563+0000 7f44e2571700 1 --2- 172.21.15.142:0/2903854445 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f44c806dd60 0x7f44c8070220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f44c4005950 tx=0x7f44c400b410 comp rx=0 tx=0).stop 2024-03-29T05:00:35.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.563+0000 7f44e2571700 1 -- 172.21.15.142:0/2903854445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44dc1026f0 msgr2=0x7f44dc0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:35.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.563+0000 7f44e2571700 1 --2- 172.21.15.142:0/2903854445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44dc1026f0 0x7f44dc0fff70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00b580 tx=0x7f44cc0055f0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.563+0000 7f44e2571700 1 -- 172.21.15.142:0/2903854445 shutdown_connections 2024-03-29T05:00:35.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.563+0000 7f44e2571700 1 --2- 172.21.15.142:0/2903854445 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f44c806dd60 0x7f44c8070220 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.563+0000 7f44e2571700 1 --2- 172.21.15.142:0/2903854445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44dc101dd0 0x7f44dc0ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.563+0000 7f44e2571700 1 --2- 172.21.15.142:0/2903854445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44dc1026f0 0x7f44dc0fff70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.563+0000 7f44e2571700 1 -- 172.21.15.142:0/2903854445 >> 172.21.15.142:0/2903854445 conn(0x7f44dc0fb540 msgr2=0x7f44dc105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:35.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.563+0000 7f44e2571700 1 -- 172.21.15.142:0/2903854445 shutdown_connections 2024-03-29T05:00:35.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.563+0000 7f44e2571700 1 -- 172.21.15.142:0/2903854445 wait complete. 2024-03-29T05:00:35.629 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:35 smithi142 ceph-mon[112475]: pgmap v368: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:35.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.666+0000 7f0ee393f700 1 -- 172.21.15.142:0/2257366977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0edc105a70 msgr2=0x7f0edc105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:35.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.666+0000 7f0ee393f700 1 --2- 172.21.15.142:0/2257366977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0edc105a70 0x7f0edc105e50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009910 tx=0x7f0ed8009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:35.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.666+0000 7f0ee393f700 1 -- 172.21.15.142:0/2257366977 shutdown_connections 2024-03-29T05:00:35.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.666+0000 7f0ee393f700 1 --2- 172.21.15.142:0/2257366977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0edc0ffa70 0x7f0edc0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.666+0000 7f0ee393f700 1 --2- 172.21.15.142:0/2257366977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0edc105a70 0x7f0edc105e50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.666+0000 7f0ee393f700 1 -- 172.21.15.142:0/2257366977 >> 172.21.15.142:0/2257366977 conn(0x7f0edc0fb560 msgr2=0x7f0edc0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:35.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.667+0000 7f0ee393f700 1 -- 172.21.15.142:0/2257366977 shutdown_connections 2024-03-29T05:00:35.676 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.667+0000 7f0ee393f700 1 -- 172.21.15.142:0/2257366977 wait complete. 2024-03-29T05:00:35.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.667+0000 7f0ee393f700 1 Processor -- start 2024-03-29T05:00:35.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.667+0000 7f0ee393f700 1 -- start start 2024-03-29T05:00:35.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.667+0000 7f0ee393f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0edc0ffa70 0x7f0edc197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:35.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.667+0000 7f0ee393f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0edc105a70 0x7f0edc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:35.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.667+0000 7f0ee393f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc1980e0 con 0x7f0edc0ffa70 2024-03-29T05:00:35.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.667+0000 7f0ee393f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc198250 con 0x7f0edc105a70 2024-03-29T05:00:35.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.668+0000 7f0ee16db700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0edc0ffa70 0x7f0edc197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:35.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.668+0000 7f0ee16db700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0edc0ffa70 0x7f0edc197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43960/0 (socket says 172.21.15.142:43960) 2024-03-29T05:00:35.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.668+0000 7f0ee16db700 1 -- 172.21.15.142:0/837601631 learned_addr learned my addr 172.21.15.142:0/837601631 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:35.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.668+0000 7f0ee0eda700 1 --2- 172.21.15.142:0/837601631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0edc105a70 0x7f0edc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:35.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.668+0000 7f0ee16db700 1 -- 172.21.15.142:0/837601631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0edc105a70 msgr2=0x7f0edc197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:35.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.668+0000 7f0ee16db700 1 --2- 172.21.15.142:0/837601631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0edc105a70 0x7f0edc197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.668+0000 7f0ee16db700 1 -- 172.21.15.142:0/837601631 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80095f0 con 0x7f0edc0ffa70 2024-03-29T05:00:35.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.668+0000 7f0ee16db700 1 --2- 172.21.15.142:0/837601631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0edc0ffa70 0x7f0edc197580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8006010 tx=0x7f0ed800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:35.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.668+0000 7f0ed27fc700 1 -- 172.21.15.142:0/837601631 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ed801d070 con 0x7f0edc0ffa70 2024-03-29T05:00:35.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.668+0000 7f0ed27fc700 1 -- 172.21.15.142:0/837601631 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ed800f460 con 0x7f0edc0ffa70 2024-03-29T05:00:35.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.668+0000 7f0ee393f700 1 -- 172.21.15.142:0/837601631 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0edc1a05f0 con 0x7f0edc0ffa70 2024-03-29T05:00:35.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.668+0000 7f0ed27fc700 1 -- 172.21.15.142:0/837601631 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ed8017610 con 0x7f0edc0ffa70 2024-03-29T05:00:35.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.669+0000 7f0ee393f700 1 -- 172.21.15.142:0/837601631 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0edc1a0a60 con 0x7f0edc0ffa70 2024-03-29T05:00:35.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.669+0000 7f0ee393f700 1 -- 172.21.15.142:0/837601631 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0edc04ea90 con 0x7f0edc0ffa70 2024-03-29T05:00:35.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.670+0000 7f0ed27fc700 1 -- 172.21.15.142:0/837601631 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f0ed800f5d0 con 0x7f0edc0ffa70 2024-03-29T05:00:35.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.670+0000 7f0ed27fc700 1 --2- 172.21.15.142:0/837601631 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0ec806dd60 0x7f0ec8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:35.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.670+0000 7f0ed27fc700 1 -- 172.21.15.142:0/837601631 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f0ed808f150 con 0x7f0edc0ffa70 2024-03-29T05:00:35.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.670+0000 7f0ee0eda700 1 --2- 172.21.15.142:0/837601631 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0ec806dd60 0x7f0ec8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:35.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.671+0000 7f0ee0eda700 1 --2- 172.21.15.142:0/837601631 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0ec806dd60 0x7f0ec8070220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0696a0 tx=0x7f0ecc00a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:35.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.675+0000 7f0ed27fc700 1 -- 172.21.15.142:0/837601631 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0ed80585f0 con 0x7f0edc0ffa70 2024-03-29T05:00:35.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.955+0000 7f0ee393f700 1 -- 172.21.15.142:0/837601631 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0edc1a0e70 con 0x7f0edc0ffa70 2024-03-29T05:00:35.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.956+0000 7f0ed27fc700 1 -- 172.21.15.142:0/837601631 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f0ed805bcb0 con 0x7f0edc0ffa70 2024-03-29T05:00:35.964 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:00:35.964 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:00:35.964 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:00:35.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:00:35.965 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:00:35.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.958+0000 7f0ee393f700 1 -- 172.21.15.142:0/837601631 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0ec806dd60 msgr2=0x7f0ec8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:35.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.958+0000 7f0ee393f700 1 --2- 172.21.15.142:0/837601631 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0ec806dd60 0x7f0ec8070220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0696a0 tx=0x7f0ecc00a300 comp rx=0 tx=0).stop 2024-03-29T05:00:35.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.958+0000 7f0ee393f700 1 -- 172.21.15.142:0/837601631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0edc0ffa70 msgr2=0x7f0edc197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:35.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.958+0000 7f0ee393f700 1 --2- 172.21.15.142:0/837601631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0edc0ffa70 0x7f0edc197580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8006010 tx=0x7f0ed800bd70 comp rx=0 tx=0).stop 2024-03-29T05:00:35.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.958+0000 7f0ee393f700 1 -- 172.21.15.142:0/837601631 shutdown_connections 2024-03-29T05:00:35.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.958+0000 7f0ee393f700 1 --2- 172.21.15.142:0/837601631 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0ec806dd60 0x7f0ec8070220 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.958+0000 7f0ee393f700 1 --2- 172.21.15.142:0/837601631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0edc0ffa70 0x7f0edc197580 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.958+0000 7f0ee393f700 1 --2- 172.21.15.142:0/837601631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0edc105a70 0x7f0edc197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:35.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.958+0000 7f0ee393f700 1 -- 172.21.15.142:0/837601631 >> 172.21.15.142:0/837601631 conn(0x7f0edc0fb560 msgr2=0x7f0edc0fbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:35.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.958+0000 7f0ee393f700 1 -- 172.21.15.142:0/837601631 shutdown_connections 2024-03-29T05:00:35.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:35.958+0000 7f0ee393f700 1 -- 172.21.15.142:0/837601631 wait complete. 2024-03-29T05:00:36.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:35 smithi152 ceph-mon[88186]: pgmap v368: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 -- 172.21.15.142:0/3850324819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e60105a40 msgr2=0x7f5e60105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 --2- 172.21.15.142:0/3850324819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e60105a40 0x7f5e60105e20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009910 tx=0x7f5e5c009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 -- 172.21.15.142:0/3850324819 shutdown_connections 2024-03-29T05:00:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 --2- 172.21.15.142:0/3850324819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e600ffa40 0x7f5e600ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 --2- 172.21.15.142:0/3850324819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e60105a40 0x7f5e60105e20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 -- 172.21.15.142:0/3850324819 >> 172.21.15.142:0/3850324819 conn(0x7f5e600fb550 msgr2=0x7f5e600fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:36.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 -- 172.21.15.142:0/3850324819 shutdown_connections 2024-03-29T05:00:36.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 -- 172.21.15.142:0/3850324819 wait complete. 2024-03-29T05:00:36.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 Processor -- start 2024-03-29T05:00:36.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 -- start start 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e600ffa40 0x7f5e6019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e60105a40 0x7f5e6019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.062+0000 7f5e68718700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e60196570 con 0x7f5e60105a40 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e68718700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e601966e0 con 0x7f5e600ffa40 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e65cb3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e60105a40 0x7f5e6019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e65cb3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e60105a40 0x7f5e6019cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43972/0 (socket says 172.21.15.142:43972) 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e65cb3700 1 -- 172.21.15.142:0/272281050 learned_addr learned my addr 172.21.15.142:0/272281050 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e664b4700 1 --2- 172.21.15.142:0/272281050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e600ffa40 0x7f5e6019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e65cb3700 1 -- 172.21.15.142:0/272281050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e600ffa40 msgr2=0x7f5e6019c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e65cb3700 1 --2- 172.21.15.142:0/272281050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e600ffa40 0x7f5e6019c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e65cb3700 1 -- 172.21.15.142:0/272281050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c0095f0 con 0x7f5e60105a40 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e65cb3700 1 --2- 172.21.15.142:0/272281050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e60105a40 0x7f5e6019cac0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000d900 tx=0x7f5e5000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e577fe700 1 -- 172.21.15.142:0/272281050 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e50004a00 con 0x7f5e60105a40 2024-03-29T05:00:36.073 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e68718700 1 -- 172.21.15.142:0/272281050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e601969c0 con 0x7f5e60105a40 2024-03-29T05:00:36.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e577fe700 1 -- 172.21.15.142:0/272281050 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e500054e0 con 0x7f5e60105a40 2024-03-29T05:00:36.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e577fe700 1 -- 172.21.15.142:0/272281050 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e50009d00 con 0x7f5e60105a40 2024-03-29T05:00:36.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.063+0000 7f5e68718700 1 -- 172.21.15.142:0/272281050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e60196f10 con 0x7f5e60105a40 2024-03-29T05:00:36.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.064+0000 7f5e68718700 1 -- 172.21.15.142:0/272281050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e60109050 con 0x7f5e60105a40 2024-03-29T05:00:36.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.065+0000 7f5e577fe700 1 -- 172.21.15.142:0/272281050 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f5e50005020 con 0x7f5e60105a40 2024-03-29T05:00:36.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.065+0000 7f5e577fe700 1 --2- 172.21.15.142:0/272281050 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5e4c06dd60 0x7f5e4c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:36.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.065+0000 7f5e577fe700 1 -- 172.21.15.142:0/272281050 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f5e5008dc50 con 0x7f5e60105a40 2024-03-29T05:00:36.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.065+0000 7f5e664b4700 1 --2- 172.21.15.142:0/272281050 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5e4c06dd60 0x7f5e4c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:36.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.066+0000 7f5e664b4700 1 --2- 172.21.15.142:0/272281050 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5e4c06dd60 0x7f5e4c070220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c006010 tx=0x7f5e5c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:36.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.070+0000 7f5e577fe700 1 -- 172.21.15.142:0/272281050 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5e50057070 con 0x7f5e60105a40 2024-03-29T05:00:36.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.308+0000 7f5e68718700 1 -- 172.21.15.142:0/272281050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5e6004ea90 con 0x7f5e60105a40 2024-03-29T05:00:36.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.309+0000 7f5e577fe700 1 -- 172.21.15.142:0/272281050 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f5e50017070 con 0x7f5e60105a40 2024-03-29T05:00:36.317 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:00:36.317 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:00:36.317 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:00:36.317 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:00:36.318 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:00:36.318 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:00:36.318 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:00:36.318 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:00:36.318 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:00:36.318 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:00:36.318 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:00:36.318 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:00:36.318 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:00:36.318 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:00:36.318 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:00:36.318 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:00:36.319 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:00:36.320 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:00:36.320 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:00:36.320 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:00:36.320 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:00:36.320 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:00:36.320 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:00:36.320 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:00:36.320 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:00:36.320 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:00:36.320 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:00:36.320 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:00:36.320 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:00:36.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.312+0000 7f5e68718700 1 -- 172.21.15.142:0/272281050 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5e4c06dd60 msgr2=0x7f5e4c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:36.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.312+0000 7f5e68718700 1 --2- 172.21.15.142:0/272281050 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5e4c06dd60 0x7f5e4c070220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c006010 tx=0x7f5e5c01a040 comp rx=0 tx=0).stop 2024-03-29T05:00:36.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.312+0000 7f5e68718700 1 -- 172.21.15.142:0/272281050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e60105a40 msgr2=0x7f5e6019cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:36.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.312+0000 7f5e68718700 1 --2- 172.21.15.142:0/272281050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e60105a40 0x7f5e6019cac0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000d900 tx=0x7f5e5000dc10 comp rx=0 tx=0).stop 2024-03-29T05:00:36.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.312+0000 7f5e68718700 1 -- 172.21.15.142:0/272281050 shutdown_connections 2024-03-29T05:00:36.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.312+0000 7f5e68718700 1 --2- 172.21.15.142:0/272281050 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5e4c06dd60 0x7f5e4c070220 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.312+0000 7f5e68718700 1 --2- 172.21.15.142:0/272281050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e600ffa40 0x7f5e6019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.312+0000 7f5e68718700 1 --2- 172.21.15.142:0/272281050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e60105a40 0x7f5e6019cac0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.312+0000 7f5e68718700 1 -- 172.21.15.142:0/272281050 >> 172.21.15.142:0/272281050 conn(0x7f5e600fb550 msgr2=0x7f5e60068f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:36.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.312+0000 7f5e68718700 1 -- 172.21.15.142:0/272281050 shutdown_connections 2024-03-29T05:00:36.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.312+0000 7f5e68718700 1 -- 172.21.15.142:0/272281050 wait complete. 2024-03-29T05:00:36.322 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:00:36.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.416+0000 7f2d978bb700 1 -- 172.21.15.142:0/3573044098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d90100dc0 msgr2=0x7f2d901011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:36.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.416+0000 7f2d978bb700 1 --2- 172.21.15.142:0/3573044098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d90100dc0 0x7f2d901011a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009910 tx=0x7f2d8c009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:36.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.417+0000 7f2d978bb700 1 -- 172.21.15.142:0/3573044098 shutdown_connections 2024-03-29T05:00:36.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.417+0000 7f2d978bb700 1 --2- 172.21.15.142:0/3573044098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d90101770 0x7f2d901057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.417+0000 7f2d978bb700 1 --2- 172.21.15.142:0/3573044098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d90100dc0 0x7f2d901011a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.417+0000 7f2d978bb700 1 -- 172.21.15.142:0/3573044098 >> 172.21.15.142:0/3573044098 conn(0x7f2d900fc650 msgr2=0x7f2d900fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:36.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.417+0000 7f2d978bb700 1 -- 172.21.15.142:0/3573044098 shutdown_connections 2024-03-29T05:00:36.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.417+0000 7f2d978bb700 1 -- 172.21.15.142:0/3573044098 wait complete. 2024-03-29T05:00:36.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.417+0000 7f2d978bb700 1 Processor -- start 2024-03-29T05:00:36.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.417+0000 7f2d978bb700 1 -- start start 2024-03-29T05:00:36.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d978bb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d90100dc0 0x7f2d90073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d978bb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d90101770 0x7f2d900735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d978bb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d90073b10 con 0x7f2d90101770 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d978bb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d90073c50 con 0x7f2d90100dc0 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d94e56700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d90101770 0x7f2d900735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d94e56700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d90101770 0x7f2d900735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43990/0 (socket says 172.21.15.142:43990) 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d94e56700 1 -- 172.21.15.142:0/1282293889 learned_addr learned my addr 172.21.15.142:0/1282293889 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d95657700 1 --2- 172.21.15.142:0/1282293889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d90100dc0 0x7f2d90073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d94e56700 1 -- 172.21.15.142:0/1282293889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d90100dc0 msgr2=0x7f2d90073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d94e56700 1 --2- 172.21.15.142:0/1282293889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d90100dc0 0x7f2d90073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d94e56700 1 -- 172.21.15.142:0/1282293889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c0095f0 con 0x7f2d90101770 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d94e56700 1 --2- 172.21.15.142:0/1282293889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d90101770 0x7f2d900735d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000c960 tx=0x7f2d8000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d867fc700 1 -- 172.21.15.142:0/1282293889 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d80007a70 con 0x7f2d90101770 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d867fc700 1 -- 172.21.15.142:0/1282293889 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d80007bd0 con 0x7f2d90101770 2024-03-29T05:00:36.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d978bb700 1 -- 172.21.15.142:0/1282293889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d901a2940 con 0x7f2d90101770 2024-03-29T05:00:36.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d867fc700 1 -- 172.21.15.142:0/1282293889 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d800186c0 con 0x7f2d90101770 2024-03-29T05:00:36.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.418+0000 7f2d978bb700 1 -- 172.21.15.142:0/1282293889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d901a2ca0 con 0x7f2d90101770 2024-03-29T05:00:36.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.419+0000 7f2d978bb700 1 -- 172.21.15.142:0/1282293889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d90109160 con 0x7f2d90101770 2024-03-29T05:00:36.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.420+0000 7f2d867fc700 1 -- 172.21.15.142:0/1282293889 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f2d8000faa0 con 0x7f2d90101770 2024-03-29T05:00:36.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.420+0000 7f2d867fc700 1 --2- 172.21.15.142:0/1282293889 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2d7c06dd60 0x7f2d7c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:36.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.420+0000 7f2d867fc700 1 -- 172.21.15.142:0/1282293889 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f2d8008dce0 con 0x7f2d90101770 2024-03-29T05:00:36.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.420+0000 7f2d95657700 1 --2- 172.21.15.142:0/1282293889 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2d7c06dd60 0x7f2d7c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:36.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.421+0000 7f2d95657700 1 --2- 172.21.15.142:0/1282293889 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2d7c06dd60 0x7f2d7c070220 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00b5c0 tx=0x7f2d8c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:36.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.426+0000 7f2d867fc700 1 -- 172.21.15.142:0/1282293889 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2d8005ac30 con 0x7f2d90101770 2024-03-29T05:00:36.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.640+0000 7f2d978bb700 1 -- 172.21.15.142:0/1282293889 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d9002cd60 con 0x7f2d7c06dd60 2024-03-29T05:00:36.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.641+0000 7f2d867fc700 1 -- 172.21.15.142:0/1282293889 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2d9002cd60 con 0x7f2d7c06dd60 2024-03-29T05:00:36.649 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:00:36.650 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:00:36.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.643+0000 7f2d978bb700 1 -- 172.21.15.142:0/1282293889 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2d7c06dd60 msgr2=0x7f2d7c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:36.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.643+0000 7f2d978bb700 1 --2- 172.21.15.142:0/1282293889 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2d7c06dd60 0x7f2d7c070220 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00b5c0 tx=0x7f2d8c01a040 comp rx=0 tx=0).stop 2024-03-29T05:00:36.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.643+0000 7f2d978bb700 1 -- 172.21.15.142:0/1282293889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d90101770 msgr2=0x7f2d900735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:36.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.643+0000 7f2d978bb700 1 --2- 172.21.15.142:0/1282293889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d90101770 0x7f2d900735d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000c960 tx=0x7f2d8000cd20 comp rx=0 tx=0).stop 2024-03-29T05:00:36.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.643+0000 7f2d978bb700 1 -- 172.21.15.142:0/1282293889 shutdown_connections 2024-03-29T05:00:36.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.643+0000 7f2d978bb700 1 --2- 172.21.15.142:0/1282293889 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2d7c06dd60 0x7f2d7c070220 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.643+0000 7f2d978bb700 1 --2- 172.21.15.142:0/1282293889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d90100dc0 0x7f2d90073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.643+0000 7f2d978bb700 1 --2- 172.21.15.142:0/1282293889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d90101770 0x7f2d900735d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.643+0000 7f2d978bb700 1 -- 172.21.15.142:0/1282293889 >> 172.21.15.142:0/1282293889 conn(0x7f2d900fc650 msgr2=0x7f2d900fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:36.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.643+0000 7f2d978bb700 1 -- 172.21.15.142:0/1282293889 shutdown_connections 2024-03-29T05:00:36.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.643+0000 7f2d978bb700 1 -- 172.21.15.142:0/1282293889 wait complete. 2024-03-29T05:00:36.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.747+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/940825538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c040ffc40 msgr2=0x7f0c04105780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:36.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.747+0000 7f0c0c3cf700 1 --2- 172.21.15.142:0/940825538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c040ffc40 0x7f0c04105780 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009910 tx=0x7f0c00009c20 comp rx=0 tx=0).stop 2024-03-29T05:00:36.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.747+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/940825538 shutdown_connections 2024-03-29T05:00:36.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.747+0000 7f0c0c3cf700 1 --2- 172.21.15.142:0/940825538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c040ffc40 0x7f0c04105780 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.747+0000 7f0c0c3cf700 1 --2- 172.21.15.142:0/940825538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c040ff290 0x7f0c040ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.747+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/940825538 >> 172.21.15.142:0/940825538 conn(0x7f0c04075e30 msgr2=0x7f0c040fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:36.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.747+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/940825538 shutdown_connections 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.747+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/940825538 wait complete. 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0c3cf700 1 Processor -- start 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0c3cf700 1 -- start start 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0c3cf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c040ff290 0x7f0c04196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0c3cf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c040ffc40 0x7f0c04196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0c3cf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c041972d0 con 0x7f0c040ffc40 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0c3cf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0419a000 con 0x7f0c040ff290 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0996a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c040ffc40 0x7f0c04196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0996a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c040ffc40 0x7f0c04196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44004/0 (socket says 172.21.15.142:44004) 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0996a700 1 -- 172.21.15.142:0/983863052 learned_addr learned my addr 172.21.15.142:0/983863052 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0a16b700 1 --2- 172.21.15.142:0/983863052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c040ff290 0x7f0c04196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0996a700 1 -- 172.21.15.142:0/983863052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c040ff290 msgr2=0x7f0c04196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:36.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0996a700 1 --2- 172.21.15.142:0/983863052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c040ff290 0x7f0c04196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:36.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.748+0000 7f0c0996a700 1 -- 172.21.15.142:0/983863052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c000095f0 con 0x7f0c040ffc40 2024-03-29T05:00:36.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.749+0000 7f0c0996a700 1 --2- 172.21.15.142:0/983863052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c040ffc40 0x7f0c04196c40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0c000059d0 tx=0x7f0c0000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:36.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.749+0000 7f0bfb7fe700 1 -- 172.21.15.142:0/983863052 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c0001d070 con 0x7f0c040ffc40 2024-03-29T05:00:36.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.749+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/983863052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0419a280 con 0x7f0c040ffc40 2024-03-29T05:00:36.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.749+0000 7f0bfb7fe700 1 -- 172.21.15.142:0/983863052 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c0000b610 con 0x7f0c040ffc40 2024-03-29T05:00:36.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.749+0000 7f0bfb7fe700 1 -- 172.21.15.142:0/983863052 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c00022470 con 0x7f0c040ffc40 2024-03-29T05:00:36.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.749+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/983863052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0419a770 con 0x7f0c040ffc40 2024-03-29T05:00:36.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.750+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/983863052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c04109180 con 0x7f0c040ffc40 2024-03-29T05:00:36.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.750+0000 7f0bfb7fe700 1 -- 172.21.15.142:0/983863052 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f0c00003710 con 0x7f0c040ffc40 2024-03-29T05:00:36.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.750+0000 7f0bfb7fe700 1 --2- 172.21.15.142:0/983863052 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0bf006dd60 0x7f0bf0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:00:36.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.751+0000 7f0bfb7fe700 1 -- 172.21.15.142:0/983863052 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f0c0008f090 con 0x7f0c040ffc40 2024-03-29T05:00:36.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.751+0000 7f0c0a16b700 1 --2- 172.21.15.142:0/983863052 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0bf006dd60 0x7f0bf0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:00:36.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.751+0000 7f0c0a16b700 1 --2- 172.21.15.142:0/983863052 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0bf006dd60 0x7f0bf0070220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009c60 tx=0x7f0bf4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:00:36.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:36.755+0000 7f0bfb7fe700 1 -- 172.21.15.142:0/983863052 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0c00058530 con 0x7f0c040ffc40 2024-03-29T05:00:36.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:36 smithi152 ceph-mon[88186]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:36.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:36 smithi152 ceph-mon[88186]: from='client.34714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:36.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:36 smithi152 ceph-mon[88186]: from='client.34718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:36.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:36 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/837601631' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:00:36.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:36 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/272281050' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:00:36.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:36 smithi142 ceph-mon[112475]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:36.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:36 smithi142 ceph-mon[112475]: from='client.34714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:36.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:36 smithi142 ceph-mon[112475]: from='client.34718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:36.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:36 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/837601631' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:00:36.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:36 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/272281050' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:00:37.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.032+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/983863052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0c0402cd30 con 0x7f0c040ffc40 2024-03-29T05:00:37.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.032+0000 7f0bfb7fe700 1 -- 172.21.15.142:0/983863052 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f0c0005bbf0 con 0x7f0c040ffc40 2024-03-29T05:00:37.042 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:00:37.042 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:00:37.042 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:00:37.042 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:00:37.042 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:00:37.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.034+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/983863052 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0bf006dd60 msgr2=0x7f0bf0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:37.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.034+0000 7f0c0c3cf700 1 --2- 172.21.15.142:0/983863052 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0bf006dd60 0x7f0bf0070220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009c60 tx=0x7f0bf4009260 comp rx=0 tx=0).stop 2024-03-29T05:00:37.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.034+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/983863052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c040ffc40 msgr2=0x7f0c04196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:00:37.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.034+0000 7f0c0c3cf700 1 --2- 172.21.15.142:0/983863052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c040ffc40 0x7f0c04196c40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0c000059d0 tx=0x7f0c0000f690 comp rx=0 tx=0).stop 2024-03-29T05:00:37.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.034+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/983863052 shutdown_connections 2024-03-29T05:00:37.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.035+0000 7f0c0c3cf700 1 --2- 172.21.15.142:0/983863052 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0bf006dd60 0x7f0bf0070220 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:37.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.035+0000 7f0c0c3cf700 1 --2- 172.21.15.142:0/983863052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c040ff290 0x7f0c04196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:37.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.035+0000 7f0c0c3cf700 1 --2- 172.21.15.142:0/983863052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c040ffc40 0x7f0c04196c40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:00:37.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.035+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/983863052 >> 172.21.15.142:0/983863052 conn(0x7f0c04075e30 msgr2=0x7f0c04104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:00:37.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.035+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/983863052 shutdown_connections 2024-03-29T05:00:37.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:00:37.035+0000 7f0c0c3cf700 1 -- 172.21.15.142:0/983863052 wait complete. 2024-03-29T05:00:37.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:37 smithi142 ceph-mon[112475]: pgmap v369: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:37.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:37 smithi142 ceph-mon[112475]: from='client.34730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:37.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:37 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/983863052' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:00:38.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:37 smithi152 ceph-mon[88186]: pgmap v369: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:38.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:37 smithi152 ceph-mon[88186]: from='client.34730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:00:38.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:37 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/983863052' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:00:39.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:39 smithi142 ceph-mon[112475]: pgmap v370: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:40.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:39 smithi152 ceph-mon[88186]: pgmap v370: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:41.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:41 smithi152 ceph-mon[88186]: pgmap v371: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:41.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:41 smithi142 ceph-mon[112475]: pgmap v371: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:43.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:43 smithi142 ceph-mon[112475]: pgmap v372: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:44.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:43 smithi152 ceph-mon[88186]: pgmap v372: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:44.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:00:45.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:00:45.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:45 smithi142 ceph-mon[112475]: pgmap v373: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:46.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:45 smithi152 ceph-mon[88186]: pgmap v373: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:47.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:47 smithi142 ceph-mon[112475]: pgmap v374: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:48.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:47 smithi152 ceph-mon[88186]: pgmap v374: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:49.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:49 smithi142 ceph-mon[112475]: pgmap v375: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:50.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:49 smithi152 ceph-mon[88186]: pgmap v375: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:51.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:51 smithi152 ceph-mon[88186]: pgmap v376: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:51.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:51 smithi142 ceph-mon[112475]: pgmap v376: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:54.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:53 smithi152 ceph-mon[88186]: pgmap v377: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:54.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:53 smithi142 ceph-mon[112475]: pgmap v377: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:56.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:55 smithi152 ceph-mon[88186]: pgmap v378: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:56.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:55 smithi142 ceph-mon[112475]: pgmap v378: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:58.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:57 smithi152 ceph-mon[88186]: pgmap v379: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:00:58.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:57 smithi142 ceph-mon[112475]: pgmap v379: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:00.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:59 smithi152 ceph-mon[88186]: pgmap v380: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:00.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:00:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:01:00.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:59 smithi142 ceph-mon[112475]: pgmap v380: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:00.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:00:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:01:01.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:01 smithi142 ceph-mon[112475]: pgmap v381: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:02.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:01 smithi152 ceph-mon[88186]: pgmap v381: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:03.914 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:03 smithi142 ceph-mon[112475]: pgmap v382: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:04.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:03 smithi152 ceph-mon[88186]: pgmap v382: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:06.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:05 smithi152 ceph-mon[88186]: pgmap v383: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:05 smithi142 ceph-mon[112475]: pgmap v383: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:07.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.154+0000 7fb327d07700 1 -- 172.21.15.142:0/3588028971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb320105a40 msgr2=0x7fb320105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:07.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.154+0000 7fb327d07700 1 --2- 172.21.15.142:0/3588028971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb320105a40 0x7fb320105e20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009910 tx=0x7fb31c009c20 comp rx=0 tx=0).stop 2024-03-29T05:01:07.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.154+0000 7fb327d07700 1 -- 172.21.15.142:0/3588028971 shutdown_connections 2024-03-29T05:01:07.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.154+0000 7fb327d07700 1 --2- 172.21.15.142:0/3588028971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3200ffa40 0x7fb3200ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.154+0000 7fb327d07700 1 --2- 172.21.15.142:0/3588028971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb320105a40 0x7fb320105e20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.154+0000 7fb327d07700 1 -- 172.21.15.142:0/3588028971 >> 172.21.15.142:0/3588028971 conn(0x7fb3200fb550 msgr2=0x7fb3200fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:07.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.154+0000 7fb327d07700 1 -- 172.21.15.142:0/3588028971 shutdown_connections 2024-03-29T05:01:07.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.154+0000 7fb327d07700 1 -- 172.21.15.142:0/3588028971 wait complete. 2024-03-29T05:01:07.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.154+0000 7fb327d07700 1 Processor -- start 2024-03-29T05:01:07.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb327d07700 1 -- start start 2024-03-29T05:01:07.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb327d07700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3200ffa40 0x7fb320197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:07.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb327d07700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb320105a40 0x7fb320197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:07.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb327d07700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3201980f0 con 0x7fb3200ffa40 2024-03-29T05:01:07.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb327d07700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb320198230 con 0x7fb320105a40 2024-03-29T05:01:07.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb325aa3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3200ffa40 0x7fb320197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:07.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb325aa3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3200ffa40 0x7fb320197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47330/0 (socket says 172.21.15.142:47330) 2024-03-29T05:01:07.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb325aa3700 1 -- 172.21.15.142:0/3506252293 learned_addr learned my addr 172.21.15.142:0/3506252293 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:07.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb3252a2700 1 --2- 172.21.15.142:0/3506252293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb320105a40 0x7fb320197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:07.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb325aa3700 1 -- 172.21.15.142:0/3506252293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb320105a40 msgr2=0x7fb320197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:07.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb325aa3700 1 --2- 172.21.15.142:0/3506252293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb320105a40 0x7fb320197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb325aa3700 1 -- 172.21.15.142:0/3506252293 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c0095f0 con 0x7fb3200ffa40 2024-03-29T05:01:07.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb3252a2700 1 --2- 172.21.15.142:0/3506252293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb320105a40 0x7fb320197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:01:07.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb325aa3700 1 --2- 172.21.15.142:0/3506252293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3200ffa40 0x7fb320197590 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb31c005f50 tx=0x7fb31c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:07.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb316ffd700 1 -- 172.21.15.142:0/3506252293 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31c01d070 con 0x7fb3200ffa40 2024-03-29T05:01:07.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.155+0000 7fb316ffd700 1 -- 172.21.15.142:0/3506252293 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb31c00b610 con 0x7fb3200ffa40 2024-03-29T05:01:07.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.156+0000 7fb327d07700 1 -- 172.21.15.142:0/3506252293 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3201a0600 con 0x7fb3200ffa40 2024-03-29T05:01:07.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.156+0000 7fb316ffd700 1 -- 172.21.15.142:0/3506252293 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31c022470 con 0x7fb3200ffa40 2024-03-29T05:01:07.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.156+0000 7fb327d07700 1 -- 172.21.15.142:0/3506252293 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3201a0a10 con 0x7fb3200ffa40 2024-03-29T05:01:07.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.156+0000 7fb327d07700 1 -- 172.21.15.142:0/3506252293 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb320108040 con 0x7fb3200ffa40 2024-03-29T05:01:07.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.157+0000 7fb316ffd700 1 -- 172.21.15.142:0/3506252293 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fb31c00fca0 con 0x7fb3200ffa40 2024-03-29T05:01:07.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.157+0000 7fb316ffd700 1 --2- 172.21.15.142:0/3506252293 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb30c06dd60 0x7fb30c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:07.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.157+0000 7fb316ffd700 1 -- 172.21.15.142:0/3506252293 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb31c08f0f0 con 0x7fb3200ffa40 2024-03-29T05:01:07.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.157+0000 7fb3252a2700 1 --2- 172.21.15.142:0/3506252293 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb30c06dd60 0x7fb30c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:07.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.158+0000 7fb3252a2700 1 --2- 172.21.15.142:0/3506252293 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb30c06dd60 0x7fb30c070220 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb320069660 tx=0x7fb310009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:07.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.162+0000 7fb316ffd700 1 -- 172.21.15.142:0/3506252293 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb31c058510 con 0x7fb3200ffa40 2024-03-29T05:01:07.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.373+0000 7fb327d07700 1 -- 172.21.15.142:0/3506252293 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb32002cce0 con 0x7fb30c06dd60 2024-03-29T05:01:07.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.375+0000 7fb316ffd700 1 -- 172.21.15.142:0/3506252293 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb32002cce0 con 0x7fb30c06dd60 2024-03-29T05:01:07.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.377+0000 7fb327d07700 1 -- 172.21.15.142:0/3506252293 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb30c06dd60 msgr2=0x7fb30c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:07.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.377+0000 7fb327d07700 1 --2- 172.21.15.142:0/3506252293 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb30c06dd60 0x7fb30c070220 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb320069660 tx=0x7fb310009260 comp rx=0 tx=0).stop 2024-03-29T05:01:07.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.377+0000 7fb327d07700 1 -- 172.21.15.142:0/3506252293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3200ffa40 msgr2=0x7fb320197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:07.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.377+0000 7fb327d07700 1 --2- 172.21.15.142:0/3506252293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3200ffa40 0x7fb320197590 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb31c005f50 tx=0x7fb31c00f740 comp rx=0 tx=0).stop 2024-03-29T05:01:07.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.377+0000 7fb327d07700 1 -- 172.21.15.142:0/3506252293 shutdown_connections 2024-03-29T05:01:07.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.377+0000 7fb327d07700 1 --2- 172.21.15.142:0/3506252293 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb30c06dd60 0x7fb30c070220 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.377+0000 7fb327d07700 1 --2- 172.21.15.142:0/3506252293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3200ffa40 0x7fb320197590 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.377+0000 7fb327d07700 1 --2- 172.21.15.142:0/3506252293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb320105a40 0x7fb320197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.377+0000 7fb327d07700 1 -- 172.21.15.142:0/3506252293 >> 172.21.15.142:0/3506252293 conn(0x7fb3200fb550 msgr2=0x7fb3200fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:07.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.377+0000 7fb327d07700 1 -- 172.21.15.142:0/3506252293 shutdown_connections 2024-03-29T05:01:07.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.377+0000 7fb327d07700 1 -- 172.21.15.142:0/3506252293 wait complete. 2024-03-29T05:01:07.396 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:01:07.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.480+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/2068640519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea4105a40 msgr2=0x7f1ea4105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:07.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.480+0000 7f1eaa7c2700 1 --2- 172.21.15.142:0/2068640519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea4105a40 0x7f1ea4105e20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009910 tx=0x7f1e94009c20 comp rx=0 tx=0).stop 2024-03-29T05:01:07.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.480+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/2068640519 shutdown_connections 2024-03-29T05:01:07.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.480+0000 7f1eaa7c2700 1 --2- 172.21.15.142:0/2068640519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ea40ffa40 0x7f1ea40ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.480+0000 7f1eaa7c2700 1 --2- 172.21.15.142:0/2068640519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea4105a40 0x7f1ea4105e20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.480+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/2068640519 >> 172.21.15.142:0/2068640519 conn(0x7f1ea40fb530 msgr2=0x7f1ea40fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:07.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.481+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/2068640519 shutdown_connections 2024-03-29T05:01:07.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.481+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/2068640519 wait complete. 2024-03-29T05:01:07.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.481+0000 7f1eaa7c2700 1 Processor -- start 2024-03-29T05:01:07.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.481+0000 7f1eaa7c2700 1 -- start start 2024-03-29T05:01:07.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1eaa7c2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea40ffa40 0x7f1ea419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:07.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1eaa7c2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ea4105a40 0x7f1ea419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:07.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1eaa7c2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea419f2a0 con 0x7f1ea40ffa40 2024-03-29T05:01:07.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1eaa7c2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea4198750 con 0x7f1ea4105a40 2024-03-29T05:01:07.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1ea3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea40ffa40 0x7f1ea419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:07.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1ea3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea40ffa40 0x7f1ea419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47344/0 (socket says 172.21.15.142:47344) 2024-03-29T05:01:07.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1ea3fff700 1 -- 172.21.15.142:0/3431420729 learned_addr learned my addr 172.21.15.142:0/3431420729 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:07.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1ea37fe700 1 --2- 172.21.15.142:0/3431420729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ea4105a40 0x7f1ea419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:07.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1ea3fff700 1 -- 172.21.15.142:0/3431420729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ea4105a40 msgr2=0x7f1ea419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:07.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1ea3fff700 1 --2- 172.21.15.142:0/3431420729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ea4105a40 0x7f1ea419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1ea3fff700 1 -- 172.21.15.142:0/3431420729 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e940095f0 con 0x7f1ea40ffa40 2024-03-29T05:01:07.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1ea3fff700 1 --2- 172.21.15.142:0/3431420729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea40ffa40 0x7f1ea419e6d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1e94006010 tx=0x7f1e9400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:07.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1ea17fa700 1 -- 172.21.15.142:0/3431420729 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e9401d070 con 0x7f1ea40ffa40 2024-03-29T05:01:07.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1ea17fa700 1 -- 172.21.15.142:0/3431420729 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e94005dc0 con 0x7f1ea40ffa40 2024-03-29T05:01:07.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.482+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/3431420729 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea41989d0 con 0x7f1ea40ffa40 2024-03-29T05:01:07.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.483+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/3431420729 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea4198ec0 con 0x7f1ea40ffa40 2024-03-29T05:01:07.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.483+0000 7f1ea17fa700 1 -- 172.21.15.142:0/3431420729 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e94017400 con 0x7f1ea40ffa40 2024-03-29T05:01:07.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.484+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/3431420729 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea404ea90 con 0x7f1ea40ffa40 2024-03-29T05:01:07.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.484+0000 7f1ea17fa700 1 -- 172.21.15.142:0/3431420729 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f1e94003a60 con 0x7f1ea40ffa40 2024-03-29T05:01:07.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.485+0000 7f1ea17fa700 1 --2- 172.21.15.142:0/3431420729 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e8c06dd60 0x7f1e8c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:07.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.485+0000 7f1ea17fa700 1 -- 172.21.15.142:0/3431420729 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1e9408f100 con 0x7f1ea40ffa40 2024-03-29T05:01:07.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.485+0000 7f1ea37fe700 1 --2- 172.21.15.142:0/3431420729 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e8c06dd60 0x7f1e8c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:07.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.485+0000 7f1ea37fe700 1 --2- 172.21.15.142:0/3431420729 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e8c06dd60 0x7f1e8c070220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4199bc0 tx=0x7f1e9800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:07.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.491+0000 7f1ea17fa700 1 -- 172.21.15.142:0/3431420729 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1e9405c050 con 0x7f1ea40ffa40 2024-03-29T05:01:07.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.704+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/3431420729 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ea4199240 con 0x7f1e8c06dd60 2024-03-29T05:01:07.714 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.706+0000 7f1ea17fa700 1 -- 172.21.15.142:0/3431420729 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1ea4199240 con 0x7f1e8c06dd60 2024-03-29T05:01:07.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.708+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/3431420729 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e8c06dd60 msgr2=0x7f1e8c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:07.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.708+0000 7f1eaa7c2700 1 --2- 172.21.15.142:0/3431420729 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e8c06dd60 0x7f1e8c070220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4199bc0 tx=0x7f1e9800b410 comp rx=0 tx=0).stop 2024-03-29T05:01:07.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.708+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/3431420729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea40ffa40 msgr2=0x7f1ea419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:07.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.708+0000 7f1eaa7c2700 1 --2- 172.21.15.142:0/3431420729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea40ffa40 0x7f1ea419e6d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1e94006010 tx=0x7f1e9400bd70 comp rx=0 tx=0).stop 2024-03-29T05:01:07.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.708+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/3431420729 shutdown_connections 2024-03-29T05:01:07.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.708+0000 7f1eaa7c2700 1 --2- 172.21.15.142:0/3431420729 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e8c06dd60 0x7f1e8c070220 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.708+0000 7f1eaa7c2700 1 --2- 172.21.15.142:0/3431420729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea40ffa40 0x7f1ea419e6d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.708+0000 7f1eaa7c2700 1 --2- 172.21.15.142:0/3431420729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ea4105a40 0x7f1ea419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.708+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/3431420729 >> 172.21.15.142:0/3431420729 conn(0x7f1ea40fb530 msgr2=0x7f1ea4068e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:07.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.708+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/3431420729 shutdown_connections 2024-03-29T05:01:07.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.708+0000 7f1eaa7c2700 1 -- 172.21.15.142:0/3431420729 wait complete. 2024-03-29T05:01:07.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.812+0000 7fc1285e9700 1 -- 172.21.15.142:0/546169263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc120105a70 msgr2=0x7fc120105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:07.821 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.812+0000 7fc1285e9700 1 --2- 172.21.15.142:0/546169263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc120105a70 0x7fc120105e50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009910 tx=0x7fc11c009c20 comp rx=0 tx=0).stop 2024-03-29T05:01:07.821 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:07 smithi142 ceph-mon[112475]: pgmap v384: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:07.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.813+0000 7fc1285e9700 1 -- 172.21.15.142:0/546169263 shutdown_connections 2024-03-29T05:01:07.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.813+0000 7fc1285e9700 1 --2- 172.21.15.142:0/546169263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1200ffa70 0x7fc1200ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.813+0000 7fc1285e9700 1 --2- 172.21.15.142:0/546169263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc120105a70 0x7fc120105e50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.813+0000 7fc1285e9700 1 -- 172.21.15.142:0/546169263 >> 172.21.15.142:0/546169263 conn(0x7fc1200fb560 msgr2=0x7fc1200fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.813+0000 7fc1285e9700 1 -- 172.21.15.142:0/546169263 shutdown_connections 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.813+0000 7fc1285e9700 1 -- 172.21.15.142:0/546169263 wait complete. 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.813+0000 7fc1285e9700 1 Processor -- start 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.813+0000 7fc1285e9700 1 -- start start 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.813+0000 7fc1285e9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1200ffa70 0x7fc120101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.813+0000 7fc1285e9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc120105a70 0x7fc120101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.813+0000 7fc1285e9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1201020c0 con 0x7fc120105a70 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.813+0000 7fc1285e9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc120102200 con 0x7fc1200ffa70 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.814+0000 7fc125b84700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc120105a70 0x7fc120101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.814+0000 7fc125b84700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc120105a70 0x7fc120101b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47364/0 (socket says 172.21.15.142:47364) 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.814+0000 7fc125b84700 1 -- 172.21.15.142:0/877261916 learned_addr learned my addr 172.21.15.142:0/877261916 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.814+0000 7fc126385700 1 --2- 172.21.15.142:0/877261916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1200ffa70 0x7fc120101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.814+0000 7fc125b84700 1 -- 172.21.15.142:0/877261916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1200ffa70 msgr2=0x7fc120101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.814+0000 7fc125b84700 1 --2- 172.21.15.142:0/877261916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1200ffa70 0x7fc120101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.814+0000 7fc125b84700 1 -- 172.21.15.142:0/877261916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c0095f0 con 0x7fc120105a70 2024-03-29T05:01:07.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.814+0000 7fc125b84700 1 --2- 172.21.15.142:0/877261916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc120105a70 0x7fc120101b80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc11000d8d0 tx=0x7fc11000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:07.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.814+0000 7fc1177fe700 1 -- 172.21.15.142:0/877261916 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1100098a0 con 0x7fc120105a70 2024-03-29T05:01:07.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.814+0000 7fc1177fe700 1 -- 172.21.15.142:0/877261916 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc110010460 con 0x7fc120105a70 2024-03-29T05:01:07.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.814+0000 7fc1285e9700 1 -- 172.21.15.142:0/877261916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1201a2900 con 0x7fc120105a70 2024-03-29T05:01:07.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.814+0000 7fc1177fe700 1 -- 172.21.15.142:0/877261916 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc11000f5f0 con 0x7fc120105a70 2024-03-29T05:01:07.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.815+0000 7fc1285e9700 1 -- 172.21.15.142:0/877261916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1201a2d10 con 0x7fc120105a70 2024-03-29T05:01:07.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.816+0000 7fc1285e9700 1 -- 172.21.15.142:0/877261916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12004ea90 con 0x7fc120105a70 2024-03-29T05:01:07.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.816+0000 7fc1177fe700 1 -- 172.21.15.142:0/877261916 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fc110009a00 con 0x7fc120105a70 2024-03-29T05:01:07.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.816+0000 7fc1177fe700 1 --2- 172.21.15.142:0/877261916 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc10c06dd60 0x7fc10c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:07.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.817+0000 7fc1177fe700 1 -- 172.21.15.142:0/877261916 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fc11008da90 con 0x7fc120105a70 2024-03-29T05:01:07.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.817+0000 7fc126385700 1 --2- 172.21.15.142:0/877261916 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc10c06dd60 0x7fc10c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:07.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.817+0000 7fc126385700 1 --2- 172.21.15.142:0/877261916 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc10c06dd60 0x7fc10c070220 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc11c006010 tx=0x7fc11c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:07.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:07.821+0000 7fc1177fe700 1 -- 172.21.15.142:0/877261916 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc110056f30 con 0x7fc120105a70 2024-03-29T05:01:08.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.024+0000 7fc1285e9700 1 -- 172.21.15.142:0/877261916 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1200fbde0 con 0x7fc10c06dd60 2024-03-29T05:01:08.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.035+0000 7fc1177fe700 1 -- 172.21.15.142:0/877261916 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc1200fbde0 con 0x7fc10c06dd60 2024-03-29T05:01:08.043 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:01:08.043 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (13m) 6m ago 24m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:01:08.044 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (24m) 6m ago 24m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:01:08.044 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (23m) 5m ago 23m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:01:08.044 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (10m) 6m ago 24m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:01:08.044 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (10m) 5m ago 23m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:01:08.044 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (12m) 6m ago 24m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:01:08.044 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (6m) 6m ago 18m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:01:08.044 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (6m) 6m ago 18m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:01:08.044 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (5m) 5m ago 18m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:01:08.044 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (5m) 5m ago 18m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:01:08.044 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (14m) 6m ago 25m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:01:08.044 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (13m) 5m ago 23m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:01:08.044 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (11m) 6m ago 26m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:01:08.045 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (10m) 5m ago 23m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:01:08.045 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (13m) 6m ago 24m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:01:08.045 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (13m) 5m ago 23m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:01:08.045 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (9m) 6m ago 22m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:01:08.045 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (9m) 6m ago 22m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:01:08.045 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (8m) 6m ago 21m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:01:08.045 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (7m) 5m ago 21m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:01:08.045 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (7m) 5m ago 20m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:01:08.045 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (6m) 5m ago 20m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:01:08.045 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (13m) 6m ago 23m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:01:08.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.037+0000 7fc1285e9700 1 -- 172.21.15.142:0/877261916 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc10c06dd60 msgr2=0x7fc10c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:08.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.037+0000 7fc1285e9700 1 --2- 172.21.15.142:0/877261916 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc10c06dd60 0x7fc10c070220 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc11c006010 tx=0x7fc11c01a040 comp rx=0 tx=0).stop 2024-03-29T05:01:08.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.037+0000 7fc1285e9700 1 -- 172.21.15.142:0/877261916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc120105a70 msgr2=0x7fc120101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:08.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.037+0000 7fc1285e9700 1 --2- 172.21.15.142:0/877261916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc120105a70 0x7fc120101b80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc11000d8d0 tx=0x7fc11000dbe0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.037+0000 7fc1285e9700 1 -- 172.21.15.142:0/877261916 shutdown_connections 2024-03-29T05:01:08.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.037+0000 7fc1285e9700 1 --2- 172.21.15.142:0/877261916 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc10c06dd60 0x7fc10c070220 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.037+0000 7fc1285e9700 1 --2- 172.21.15.142:0/877261916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1200ffa70 0x7fc120101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.037+0000 7fc1285e9700 1 --2- 172.21.15.142:0/877261916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc120105a70 0x7fc120101b80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.037+0000 7fc1285e9700 1 -- 172.21.15.142:0/877261916 >> 172.21.15.142:0/877261916 conn(0x7fc1200fb560 msgr2=0x7fc120103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:08.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.037+0000 7fc1285e9700 1 -- 172.21.15.142:0/877261916 shutdown_connections 2024-03-29T05:01:08.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.037+0000 7fc1285e9700 1 -- 172.21.15.142:0/877261916 wait complete. 2024-03-29T05:01:08.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:07 smithi152 ceph-mon[88186]: pgmap v384: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:08.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.139+0000 7f63ea4e6700 1 -- 172.21.15.142:0/3648868718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e40ffa70 msgr2=0x7f63e40ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:08.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.139+0000 7f63ea4e6700 1 --2- 172.21.15.142:0/3648868718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e40ffa70 0x7f63e40ffef0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009910 tx=0x7f63d8009c20 comp rx=0 tx=0).stop 2024-03-29T05:01:08.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.139+0000 7f63ea4e6700 1 -- 172.21.15.142:0/3648868718 shutdown_connections 2024-03-29T05:01:08.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.139+0000 7f63ea4e6700 1 --2- 172.21.15.142:0/3648868718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e40ffa70 0x7f63e40ffef0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.139+0000 7f63ea4e6700 1 --2- 172.21.15.142:0/3648868718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63e4105a70 0x7f63e4105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.139+0000 7f63ea4e6700 1 -- 172.21.15.142:0/3648868718 >> 172.21.15.142:0/3648868718 conn(0x7f63e40fb5a0 msgr2=0x7f63e40fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:08.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.139+0000 7f63ea4e6700 1 -- 172.21.15.142:0/3648868718 shutdown_connections 2024-03-29T05:01:08.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.139+0000 7f63ea4e6700 1 -- 172.21.15.142:0/3648868718 wait complete. 2024-03-29T05:01:08.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63ea4e6700 1 Processor -- start 2024-03-29T05:01:08.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63ea4e6700 1 -- start start 2024-03-29T05:01:08.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63ea4e6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e40ffa70 0x7f63e419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:08.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63ea4e6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63e4105a70 0x7f63e419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:08.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63ea4e6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e419d0e0 con 0x7f63e40ffa70 2024-03-29T05:01:08.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63ea4e6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4196590 con 0x7f63e4105a70 2024-03-29T05:01:08.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63e3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e40ffa70 0x7f63e419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:08.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63e3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e40ffa70 0x7f63e419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47382/0 (socket says 172.21.15.142:47382) 2024-03-29T05:01:08.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63e3fff700 1 -- 172.21.15.142:0/573217641 learned_addr learned my addr 172.21.15.142:0/573217641 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:08.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63e37fe700 1 --2- 172.21.15.142:0/573217641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63e4105a70 0x7f63e419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:08.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63e3fff700 1 -- 172.21.15.142:0/573217641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63e4105a70 msgr2=0x7f63e419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:08.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63e3fff700 1 --2- 172.21.15.142:0/573217641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63e4105a70 0x7f63e419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63e3fff700 1 -- 172.21.15.142:0/573217641 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80095f0 con 0x7f63e40ffa70 2024-03-29T05:01:08.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.140+0000 7f63e37fe700 1 --2- 172.21.15.142:0/573217641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63e4105a70 0x7f63e419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:01:08.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.141+0000 7f63e3fff700 1 --2- 172.21.15.142:0/573217641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e40ffa70 0x7f63e419c510 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f63d400c8a0 tx=0x7f63d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:08.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.141+0000 7f63e17fa700 1 -- 172.21.15.142:0/573217641 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63d400cea0 con 0x7f63e40ffa70 2024-03-29T05:01:08.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.141+0000 7f63ea4e6700 1 -- 172.21.15.142:0/573217641 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e4196870 con 0x7f63e40ffa70 2024-03-29T05:01:08.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.141+0000 7f63e17fa700 1 -- 172.21.15.142:0/573217641 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63d4004830 con 0x7f63e40ffa70 2024-03-29T05:01:08.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.141+0000 7f63e17fa700 1 -- 172.21.15.142:0/573217641 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63d40055b0 con 0x7f63e40ffa70 2024-03-29T05:01:08.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.141+0000 7f63ea4e6700 1 -- 172.21.15.142:0/573217641 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e4196dc0 con 0x7f63e40ffa70 2024-03-29T05:01:08.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.142+0000 7f63ea4e6700 1 -- 172.21.15.142:0/573217641 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e4109010 con 0x7f63e40ffa70 2024-03-29T05:01:08.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.142+0000 7f63e17fa700 1 -- 172.21.15.142:0/573217641 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f63d40049a0 con 0x7f63e40ffa70 2024-03-29T05:01:08.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.143+0000 7f63e17fa700 1 --2- 172.21.15.142:0/573217641 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f63cc06dd60 0x7f63cc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:08.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.143+0000 7f63e17fa700 1 -- 172.21.15.142:0/573217641 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f63d408d980 con 0x7f63e40ffa70 2024-03-29T05:01:08.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.143+0000 7f63e37fe700 1 --2- 172.21.15.142:0/573217641 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f63cc06dd60 0x7f63cc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:08.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.143+0000 7f63e37fe700 1 --2- 172.21.15.142:0/573217641 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f63cc06dd60 0x7f63cc070220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f63d800b5c0 tx=0x7f63d8005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:08.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.147+0000 7f63e17fa700 1 -- 172.21.15.142:0/573217641 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f63d405a8d0 con 0x7f63e40ffa70 2024-03-29T05:01:08.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.425+0000 7f63ea4e6700 1 -- 172.21.15.142:0/573217641 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f63e4197090 con 0x7f63e40ffa70 2024-03-29T05:01:08.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.425+0000 7f63e17fa700 1 -- 172.21.15.142:0/573217641 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f63d4004c50 con 0x7f63e40ffa70 2024-03-29T05:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:01:08.434 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:01:08.435 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:01:08.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.427+0000 7f63ea4e6700 1 -- 172.21.15.142:0/573217641 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f63cc06dd60 msgr2=0x7f63cc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:08.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.427+0000 7f63ea4e6700 1 --2- 172.21.15.142:0/573217641 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f63cc06dd60 0x7f63cc070220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f63d800b5c0 tx=0x7f63d8005960 comp rx=0 tx=0).stop 2024-03-29T05:01:08.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.427+0000 7f63ea4e6700 1 -- 172.21.15.142:0/573217641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e40ffa70 msgr2=0x7f63e419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:08.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.427+0000 7f63ea4e6700 1 --2- 172.21.15.142:0/573217641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e40ffa70 0x7f63e419c510 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f63d400c8a0 tx=0x7f63d400cc60 comp rx=0 tx=0).stop 2024-03-29T05:01:08.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.428+0000 7f63ea4e6700 1 -- 172.21.15.142:0/573217641 shutdown_connections 2024-03-29T05:01:08.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.428+0000 7f63ea4e6700 1 --2- 172.21.15.142:0/573217641 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f63cc06dd60 0x7f63cc070220 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.428+0000 7f63ea4e6700 1 --2- 172.21.15.142:0/573217641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63e40ffa70 0x7f63e419c510 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.428+0000 7f63ea4e6700 1 --2- 172.21.15.142:0/573217641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63e4105a70 0x7f63e419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.428+0000 7f63ea4e6700 1 -- 172.21.15.142:0/573217641 >> 172.21.15.142:0/573217641 conn(0x7f63e40fb5a0 msgr2=0x7f63e4069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:08.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.428+0000 7f63ea4e6700 1 -- 172.21.15.142:0/573217641 shutdown_connections 2024-03-29T05:01:08.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.428+0000 7f63ea4e6700 1 -- 172.21.15.142:0/573217641 wait complete. 2024-03-29T05:01:08.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.531+0000 7f6d032d6700 1 -- 172.21.15.142:0/1279282337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cfc0ffeb0 msgr2=0x7f6cfc103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:08.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.531+0000 7f6d032d6700 1 --2- 172.21.15.142:0/1279282337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cfc0ffeb0 0x7f6cfc103e80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8009910 tx=0x7f6cf8009c20 comp rx=0 tx=0).stop 2024-03-29T05:01:08.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.531+0000 7f6d032d6700 1 -- 172.21.15.142:0/1279282337 shutdown_connections 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.531+0000 7f6d032d6700 1 --2- 172.21.15.142:0/1279282337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cfc0ffeb0 0x7f6cfc103e80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.531+0000 7f6d032d6700 1 --2- 172.21.15.142:0/1279282337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cfc0ff500 0x7f6cfc0ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.531+0000 7f6d032d6700 1 -- 172.21.15.142:0/1279282337 >> 172.21.15.142:0/1279282337 conn(0x7f6cfc074d80 msgr2=0x7f6cfc075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.531+0000 7f6d032d6700 1 -- 172.21.15.142:0/1279282337 shutdown_connections 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.531+0000 7f6d032d6700 1 -- 172.21.15.142:0/1279282337 wait complete. 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.532+0000 7f6d032d6700 1 Processor -- start 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.532+0000 7f6d032d6700 1 -- start start 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.532+0000 7f6d032d6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cfc0ff500 0x7f6cfc0fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.532+0000 7f6d032d6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cfc0ffeb0 0x7f6cfc0fd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.532+0000 7f6d032d6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc0fd790 con 0x7f6cfc0ff500 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.532+0000 7f6d032d6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc0fd8d0 con 0x7f6cfc0ffeb0 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.532+0000 7f6d01072700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cfc0ff500 0x7f6cfc0fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.532+0000 7f6d01072700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cfc0ff500 0x7f6cfc0fcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56684/0 (socket says 172.21.15.142:56684) 2024-03-29T05:01:08.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.532+0000 7f6d01072700 1 -- 172.21.15.142:0/1967287081 learned_addr learned my addr 172.21.15.142:0/1967287081 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.532+0000 7f6d00871700 1 --2- 172.21.15.142:0/1967287081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cfc0ffeb0 0x7f6cfc0fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.533+0000 7f6d01072700 1 -- 172.21.15.142:0/1967287081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cfc0ffeb0 msgr2=0x7f6cfc0fd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.533+0000 7f6d01072700 1 --2- 172.21.15.142:0/1967287081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cfc0ffeb0 0x7f6cfc0fd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.533+0000 7f6d01072700 1 -- 172.21.15.142:0/1967287081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80095f0 con 0x7f6cfc0ff500 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.533+0000 7f6d01072700 1 --2- 172.21.15.142:0/1967287081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cfc0ff500 0x7f6cfc0fcd10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00d8d0 tx=0x7f6cec00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.533+0000 7f6cf27fc700 1 -- 172.21.15.142:0/1967287081 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cec0098a0 con 0x7f6cfc0ff500 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.533+0000 7f6cf27fc700 1 -- 172.21.15.142:0/1967287081 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cec010460 con 0x7f6cfc0ff500 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.533+0000 7f6cf27fc700 1 -- 172.21.15.142:0/1967287081 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cec00f5f0 con 0x7f6cfc0ff500 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.533+0000 7f6d032d6700 1 -- 172.21.15.142:0/1967287081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cfc1a2a10 con 0x7f6cfc0ff500 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.533+0000 7f6d032d6700 1 -- 172.21.15.142:0/1967287081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cfc1a2e20 con 0x7f6cfc0ff500 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.534+0000 7f6d032d6700 1 -- 172.21.15.142:0/1967287081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cfc04ea90 con 0x7f6cfc0ff500 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.534+0000 7f6cf27fc700 1 -- 172.21.15.142:0/1967287081 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6cec009a00 con 0x7f6cfc0ff500 2024-03-29T05:01:08.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.535+0000 7f6cf27fc700 1 --2- 172.21.15.142:0/1967287081 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ce806dd60 0x7f6ce8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:08.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.535+0000 7f6cf27fc700 1 -- 172.21.15.142:0/1967287081 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6cec08daa0 con 0x7f6cfc0ff500 2024-03-29T05:01:08.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.535+0000 7f6d00871700 1 --2- 172.21.15.142:0/1967287081 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ce806dd60 0x7f6ce8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:08.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.535+0000 7f6d00871700 1 --2- 172.21.15.142:0/1967287081 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ce806dd60 0x7f6ce8070220 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0fe500 tx=0x7f6cf801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:08.548 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.540+0000 7f6cf27fc700 1 -- 172.21.15.142:0/1967287081 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6cec056f40 con 0x7f6cfc0ff500 2024-03-29T05:01:08.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.786+0000 7f6d032d6700 1 -- 172.21.15.142:0/1967287081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6cfc1a31a0 con 0x7f6cfc0ff500 2024-03-29T05:01:08.796 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:08 smithi142 ceph-mon[112475]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:08.796 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:08 smithi142 ceph-mon[112475]: from='client.34742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:08.796 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:08 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/573217641' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:01:08.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.787+0000 7f6cf27fc700 1 -- 172.21.15.142:0/1967287081 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f6cec05a600 con 0x7f6cfc0ff500 2024-03-29T05:01:08.797 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:01:08.797 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:01:08.797 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:01:08.798 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:01:08.798 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:01:08.798 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:01:08.798 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:01:08.798 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:01:08.799 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:01:08.799 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:01:08.799 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:01:08.799 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:01:08.799 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:01:08.799 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:01:08.799 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:01:08.800 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:01:08.800 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:01:08.800 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:01:08.800 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:01:08.801 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:01:08.801 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:01:08.801 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:01:08.801 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:01:08.801 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:01:08.801 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:01:08.801 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:01:08.802 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:01:08.802 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:01:08.802 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:01:08.802 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:01:08.802 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:01:08.802 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:01:08.803 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:01:08.803 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:01:08.803 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:01:08.803 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:01:08.803 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:01:08.803 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:01:08.803 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:01:08.803 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:01:08.804 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:01:08.804 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:01:08.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.790+0000 7f6d032d6700 1 -- 172.21.15.142:0/1967287081 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ce806dd60 msgr2=0x7f6ce8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:08.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.790+0000 7f6d032d6700 1 --2- 172.21.15.142:0/1967287081 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ce806dd60 0x7f6ce8070220 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0fe500 tx=0x7f6cf801a040 comp rx=0 tx=0).stop 2024-03-29T05:01:08.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.790+0000 7f6d032d6700 1 -- 172.21.15.142:0/1967287081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cfc0ff500 msgr2=0x7f6cfc0fcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:08.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.790+0000 7f6d032d6700 1 --2- 172.21.15.142:0/1967287081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cfc0ff500 0x7f6cfc0fcd10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00d8d0 tx=0x7f6cec00dbe0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.790+0000 7f6d032d6700 1 -- 172.21.15.142:0/1967287081 shutdown_connections 2024-03-29T05:01:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.790+0000 7f6d032d6700 1 --2- 172.21.15.142:0/1967287081 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ce806dd60 0x7f6ce8070220 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.790+0000 7f6d032d6700 1 --2- 172.21.15.142:0/1967287081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cfc0ff500 0x7f6cfc0fcd10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.790+0000 7f6d032d6700 1 --2- 172.21.15.142:0/1967287081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cfc0ffeb0 0x7f6cfc0fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.790+0000 7f6d032d6700 1 -- 172.21.15.142:0/1967287081 >> 172.21.15.142:0/1967287081 conn(0x7f6cfc074d80 msgr2=0x7f6cfc102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.790+0000 7f6d032d6700 1 -- 172.21.15.142:0/1967287081 shutdown_connections 2024-03-29T05:01:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.790+0000 7f6d032d6700 1 -- 172.21.15.142:0/1967287081 wait complete. 2024-03-29T05:01:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:01:08.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.894+0000 7ff249b9f700 1 -- 172.21.15.142:0/3107209403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244068670 msgr2=0x7ff244068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:08.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.894+0000 7ff249b9f700 1 --2- 172.21.15.142:0/3107209403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244068670 0x7ff244068a50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00b0a0 tx=0x7ff23c00b3b0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.894+0000 7ff249b9f700 1 -- 172.21.15.142:0/3107209403 shutdown_connections 2024-03-29T05:01:08.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.894+0000 7ff249b9f700 1 --2- 172.21.15.142:0/3107209403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff244069020 0x7ff244105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.894+0000 7ff249b9f700 1 --2- 172.21.15.142:0/3107209403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244068670 0x7ff244068a50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.894+0000 7ff249b9f700 1 -- 172.21.15.142:0/3107209403 >> 172.21.15.142:0/3107209403 conn(0x7ff2440755a0 msgr2=0x7ff2440759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:08.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.894+0000 7ff249b9f700 1 -- 172.21.15.142:0/3107209403 shutdown_connections 2024-03-29T05:01:08.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.894+0000 7ff249b9f700 1 -- 172.21.15.142:0/3107209403 wait complete. 2024-03-29T05:01:08.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.895+0000 7ff249b9f700 1 Processor -- start 2024-03-29T05:01:08.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.895+0000 7ff249b9f700 1 -- start start 2024-03-29T05:01:08.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.895+0000 7ff249b9f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff244068670 0x7ff24419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:08.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.895+0000 7ff249b9f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244069020 0x7ff24419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:08.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.895+0000 7ff249b9f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24419d0e0 con 0x7ff244069020 2024-03-29T05:01:08.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.895+0000 7ff249b9f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff244196590 con 0x7ff244068670 2024-03-29T05:01:08.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.895+0000 7ff242ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244069020 0x7ff24419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:08.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.895+0000 7ff242ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244069020 0x7ff24419ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56702/0 (socket says 172.21.15.142:56702) 2024-03-29T05:01:08.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.895+0000 7ff2437fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff244068670 0x7ff24419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:08.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.895+0000 7ff242ffd700 1 -- 172.21.15.142:0/1604539689 learned_addr learned my addr 172.21.15.142:0/1604539689 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:08.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.896+0000 7ff242ffd700 1 -- 172.21.15.142:0/1604539689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff244068670 msgr2=0x7ff24419c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:08.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.896+0000 7ff242ffd700 1 --2- 172.21.15.142:0/1604539689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff244068670 0x7ff24419c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:08.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.896+0000 7ff242ffd700 1 -- 172.21.15.142:0/1604539689 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c009d00 con 0x7ff244069020 2024-03-29T05:01:08.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.896+0000 7ff2437fe700 1 --2- 172.21.15.142:0/1604539689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff244068670 0x7ff24419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T05:01:08.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.896+0000 7ff242ffd700 1 --2- 172.21.15.142:0/1604539689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244069020 0x7ff24419ca50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00e8f0 tx=0x7ff22c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:08.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.896+0000 7ff240ff9700 1 -- 172.21.15.142:0/1604539689 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff22c009790 con 0x7ff244069020 2024-03-29T05:01:08.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.896+0000 7ff240ff9700 1 -- 172.21.15.142:0/1604539689 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff22c00c810 con 0x7ff244069020 2024-03-29T05:01:08.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.896+0000 7ff249b9f700 1 -- 172.21.15.142:0/1604539689 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff244196870 con 0x7ff244069020 2024-03-29T05:01:08.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.896+0000 7ff240ff9700 1 -- 172.21.15.142:0/1604539689 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff22c005580 con 0x7ff244069020 2024-03-29T05:01:08.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.896+0000 7ff249b9f700 1 -- 172.21.15.142:0/1604539689 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff244196dc0 con 0x7ff244069020 2024-03-29T05:01:08.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.897+0000 7ff249b9f700 1 -- 172.21.15.142:0/1604539689 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff244109130 con 0x7ff244069020 2024-03-29T05:01:08.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.897+0000 7ff240ff9700 1 -- 172.21.15.142:0/1604539689 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff22c00c980 con 0x7ff244069020 2024-03-29T05:01:08.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.898+0000 7ff240ff9700 1 --2- 172.21.15.142:0/1604539689 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff23006dd60 0x7ff230070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:08.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.898+0000 7ff240ff9700 1 -- 172.21.15.142:0/1604539689 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff22c014070 con 0x7ff244069020 2024-03-29T05:01:08.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.898+0000 7ff2437fe700 1 --2- 172.21.15.142:0/1604539689 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff23006dd60 0x7ff230070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:08.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.898+0000 7ff2437fe700 1 --2- 172.21.15.142:0/1604539689 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff23006dd60 0x7ff230070220 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff23c000f80 tx=0x7ff23c0060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:08.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:08.902+0000 7ff240ff9700 1 -- 172.21.15.142:0/1604539689 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff22c092070 con 0x7ff244069020 2024-03-29T05:01:09.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:08 smithi152 ceph-mon[88186]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:09.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:08 smithi152 ceph-mon[88186]: from='client.34742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:09.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:08 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/573217641' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:01:09.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.124+0000 7ff249b9f700 1 -- 172.21.15.142:0/1604539689 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff244197130 con 0x7ff23006dd60 2024-03-29T05:01:09.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.125+0000 7ff240ff9700 1 -- 172.21.15.142:0/1604539689 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff244197130 con 0x7ff23006dd60 2024-03-29T05:01:09.134 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:01:09.134 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:01:09.135 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:01:09.135 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:01:09.135 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:01:09.135 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:01:09.135 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:01:09.135 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:01:09.135 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:01:09.136 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:01:09.136 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:01:09.136 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:01:09.136 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:01:09.136 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:01:09.136 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:01:09.137 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.127+0000 7ff249b9f700 1 -- 172.21.15.142:0/1604539689 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff23006dd60 msgr2=0x7ff230070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:09.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.127+0000 7ff249b9f700 1 --2- 172.21.15.142:0/1604539689 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff23006dd60 0x7ff230070220 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff23c000f80 tx=0x7ff23c0060c0 comp rx=0 tx=0).stop 2024-03-29T05:01:09.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.127+0000 7ff249b9f700 1 -- 172.21.15.142:0/1604539689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244069020 msgr2=0x7ff24419ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:09.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.127+0000 7ff249b9f700 1 --2- 172.21.15.142:0/1604539689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244069020 0x7ff24419ca50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00e8f0 tx=0x7ff22c00ec00 comp rx=0 tx=0).stop 2024-03-29T05:01:09.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.127+0000 7ff249b9f700 1 -- 172.21.15.142:0/1604539689 shutdown_connections 2024-03-29T05:01:09.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.127+0000 7ff249b9f700 1 --2- 172.21.15.142:0/1604539689 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff23006dd60 0x7ff230070220 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:09.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.127+0000 7ff249b9f700 1 --2- 172.21.15.142:0/1604539689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff244068670 0x7ff24419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:09.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.127+0000 7ff249b9f700 1 --2- 172.21.15.142:0/1604539689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff244069020 0x7ff24419ca50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:09.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.128+0000 7ff249b9f700 1 -- 172.21.15.142:0/1604539689 >> 172.21.15.142:0/1604539689 conn(0x7ff2440755a0 msgr2=0x7ff2440fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:09.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.128+0000 7ff249b9f700 1 -- 172.21.15.142:0/1604539689 shutdown_connections 2024-03-29T05:01:09.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.128+0000 7ff249b9f700 1 -- 172.21.15.142:0/1604539689 wait complete. 2024-03-29T05:01:09.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.231+0000 7f2e60e65700 1 -- 172.21.15.142:0/904458214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c0ff530 msgr2=0x7f2e5c0ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:09.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.231+0000 7f2e60e65700 1 --2- 172.21.15.142:0/904458214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c0ff530 0x7f2e5c0ff910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009910 tx=0x7f2e4c009c20 comp rx=0 tx=0).stop 2024-03-29T05:01:09.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.232+0000 7f2e60e65700 1 -- 172.21.15.142:0/904458214 shutdown_connections 2024-03-29T05:01:09.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.232+0000 7f2e60e65700 1 --2- 172.21.15.142:0/904458214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e5c0ffee0 0x7f2e5c103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:09.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.232+0000 7f2e60e65700 1 --2- 172.21.15.142:0/904458214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c0ff530 0x7f2e5c0ff910 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.232+0000 7f2e60e65700 1 -- 172.21.15.142:0/904458214 >> 172.21.15.142:0/904458214 conn(0x7f2e5c074df0 msgr2=0x7f2e5c075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.232+0000 7f2e60e65700 1 -- 172.21.15.142:0/904458214 shutdown_connections 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.232+0000 7f2e60e65700 1 -- 172.21.15.142:0/904458214 wait complete. 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.232+0000 7f2e60e65700 1 Processor -- start 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e60e65700 1 -- start start 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e60e65700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c0ff530 0x7f2e5c100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e60e65700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e5c0ffee0 0x7f2e5c101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e60e65700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c1019d0 con 0x7f2e5c0ff530 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e60e65700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c101b10 con 0x7f2e5c0ffee0 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e5a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c0ff530 0x7f2e5c100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e5a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c0ff530 0x7f2e5c100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56712/0 (socket says 172.21.15.142:56712) 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e5a59c700 1 -- 172.21.15.142:0/812919469 learned_addr learned my addr 172.21.15.142:0/812919469 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e51d9b700 1 --2- 172.21.15.142:0/812919469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e5c0ffee0 0x7f2e5c101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e5a59c700 1 -- 172.21.15.142:0/812919469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e5c0ffee0 msgr2=0x7f2e5c101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:09.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e5a59c700 1 --2- 172.21.15.142:0/812919469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e5c0ffee0 0x7f2e5c101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:09.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e5a59c700 1 -- 172.21.15.142:0/812919469 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c0095f0 con 0x7f2e5c0ff530 2024-03-29T05:01:09.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.233+0000 7f2e5a59c700 1 --2- 172.21.15.142:0/812919469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c0ff530 0x7f2e5c100f50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c006010 tx=0x7f2e4c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:09.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.234+0000 7f2e53fff700 1 -- 172.21.15.142:0/812919469 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e4c01d070 con 0x7f2e5c0ff530 2024-03-29T05:01:09.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.234+0000 7f2e60e65700 1 -- 172.21.15.142:0/812919469 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5c101d90 con 0x7f2e5c0ff530 2024-03-29T05:01:09.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.234+0000 7f2e53fff700 1 -- 172.21.15.142:0/812919469 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e4c00f460 con 0x7f2e5c0ff530 2024-03-29T05:01:09.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.234+0000 7f2e53fff700 1 -- 172.21.15.142:0/812919469 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e4c017630 con 0x7f2e5c0ff530 2024-03-29T05:01:09.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.234+0000 7f2e60e65700 1 -- 172.21.15.142:0/812919469 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5c1a2c60 con 0x7f2e5c0ff530 2024-03-29T05:01:09.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.235+0000 7f2e60e65700 1 -- 172.21.15.142:0/812919469 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e5c04ea90 con 0x7f2e5c0ff530 2024-03-29T05:01:09.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.235+0000 7f2e53fff700 1 -- 172.21.15.142:0/812919469 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f2e4c00f5d0 con 0x7f2e5c0ff530 2024-03-29T05:01:09.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.235+0000 7f2e53fff700 1 --2- 172.21.15.142:0/812919469 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2e4806dd60 0x7f2e48070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:09.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.235+0000 7f2e53fff700 1 -- 172.21.15.142:0/812919469 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f2e4c08f1f0 con 0x7f2e5c0ff530 2024-03-29T05:01:09.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.235+0000 7f2e51d9b700 1 --2- 172.21.15.142:0/812919469 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2e4806dd60 0x7f2e48070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:09.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.236+0000 7f2e51d9b700 1 --2- 172.21.15.142:0/812919469 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2e4806dd60 0x7f2e48070220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c102740 tx=0x7f2e4400a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:09.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.240+0000 7f2e53fff700 1 -- 172.21.15.142:0/812919469 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2e4c058610 con 0x7f2e5c0ff530 2024-03-29T05:01:09.523 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.514+0000 7f2e60e65700 1 -- 172.21.15.142:0/812919469 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2e5c1a2fe0 con 0x7f2e5c0ff530 2024-03-29T05:01:09.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.515+0000 7f2e53fff700 1 -- 172.21.15.142:0/812919469 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f2e4c05bcd0 con 0x7f2e5c0ff530 2024-03-29T05:01:09.524 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:01:09.524 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:01:09.524 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:01:09.524 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:01:09.524 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:01:09.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.517+0000 7f2e60e65700 1 -- 172.21.15.142:0/812919469 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2e4806dd60 msgr2=0x7f2e48070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:09.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.517+0000 7f2e60e65700 1 --2- 172.21.15.142:0/812919469 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2e4806dd60 0x7f2e48070220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c102740 tx=0x7f2e4400a380 comp rx=0 tx=0).stop 2024-03-29T05:01:09.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.518+0000 7f2e60e65700 1 -- 172.21.15.142:0/812919469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c0ff530 msgr2=0x7f2e5c100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:09.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.518+0000 7f2e60e65700 1 --2- 172.21.15.142:0/812919469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c0ff530 0x7f2e5c100f50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c006010 tx=0x7f2e4c00bd70 comp rx=0 tx=0).stop 2024-03-29T05:01:09.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.518+0000 7f2e60e65700 1 -- 172.21.15.142:0/812919469 shutdown_connections 2024-03-29T05:01:09.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.518+0000 7f2e60e65700 1 --2- 172.21.15.142:0/812919469 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2e4806dd60 0x7f2e48070220 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:09.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.518+0000 7f2e60e65700 1 --2- 172.21.15.142:0/812919469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c0ff530 0x7f2e5c100f50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:09.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.518+0000 7f2e60e65700 1 --2- 172.21.15.142:0/812919469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e5c0ffee0 0x7f2e5c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:09.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.518+0000 7f2e60e65700 1 -- 172.21.15.142:0/812919469 >> 172.21.15.142:0/812919469 conn(0x7f2e5c074df0 msgr2=0x7f2e5c103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:09.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.518+0000 7f2e60e65700 1 -- 172.21.15.142:0/812919469 shutdown_connections 2024-03-29T05:01:09.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:09.518+0000 7f2e60e65700 1 -- 172.21.15.142:0/812919469 wait complete. 2024-03-29T05:01:09.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:09 smithi142 ceph-mon[112475]: pgmap v385: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:09.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:09 smithi142 ceph-mon[112475]: from='client.34746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:09.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:09 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1967287081' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:01:09.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:09 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/812919469' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:01:10.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:09 smithi152 ceph-mon[88186]: pgmap v385: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:10.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:09 smithi152 ceph-mon[88186]: from='client.34746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:10.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:09 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1967287081' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:01:10.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:09 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/812919469' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:01:11.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:10 smithi152 ceph-mon[88186]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:11.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:10 smithi142 ceph-mon[112475]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:11.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:11 smithi142 ceph-mon[112475]: pgmap v386: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:12.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:11 smithi152 ceph-mon[88186]: pgmap v386: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:14.022 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:13 smithi142 ceph-mon[112475]: pgmap v387: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:14.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:13 smithi152 ceph-mon[88186]: pgmap v387: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:15.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:01:15.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:01:16.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:15 smithi152 ceph-mon[88186]: pgmap v388: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:16.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:15 smithi142 ceph-mon[112475]: pgmap v388: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:18.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:17 smithi152 ceph-mon[88186]: pgmap v389: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:18.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:17 smithi142 ceph-mon[112475]: pgmap v389: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:20.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:19 smithi152 ceph-mon[88186]: pgmap v390: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:20.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:19 smithi142 ceph-mon[112475]: pgmap v390: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:21.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:21 smithi142 ceph-mon[112475]: pgmap v391: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:22.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:21 smithi152 ceph-mon[88186]: pgmap v391: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:24.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:23 smithi152 ceph-mon[88186]: pgmap v392: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:24.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:23 smithi142 ceph-mon[112475]: pgmap v392: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:26.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:25 smithi152 ceph-mon[88186]: pgmap v393: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:26.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:25 smithi142 ceph-mon[112475]: pgmap v393: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:28.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:27 smithi152 ceph-mon[88186]: pgmap v394: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:28.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:27 smithi142 ceph-mon[112475]: pgmap v394: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:30.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:29 smithi152 ceph-mon[88186]: pgmap v395: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:30.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:01:30.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:29 smithi142 ceph-mon[112475]: pgmap v395: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:30.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:01:31.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:31 smithi142 ceph-mon[112475]: pgmap v396: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:32.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:31 smithi152 ceph-mon[88186]: pgmap v396: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:34.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:33 smithi152 ceph-mon[88186]: pgmap v397: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:34.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:33 smithi142 ceph-mon[112475]: pgmap v397: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:36.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:35 smithi152 ceph-mon[88186]: pgmap v398: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:36.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:35 smithi142 ceph-mon[112475]: pgmap v398: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:38.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:37 smithi152 ceph-mon[88186]: pgmap v399: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:38.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:37 smithi142 ceph-mon[112475]: pgmap v399: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:39.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.632+0000 7f460cb5c700 1 -- 172.21.15.142:0/1225276616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46080ff500 msgr2=0x7f46080ff8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:39.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.632+0000 7f460cb5c700 1 --2- 172.21.15.142:0/1225276616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46080ff500 0x7f46080ff8e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f45f800aa20 tx=0x7f45f800ad30 comp rx=0 tx=0).stop 2024-03-29T05:01:39.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.632+0000 7f460cb5c700 1 -- 172.21.15.142:0/1225276616 shutdown_connections 2024-03-29T05:01:39.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.632+0000 7f460cb5c700 1 --2- 172.21.15.142:0/1225276616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46080ffeb0 0x7f4608103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:39.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.632+0000 7f460cb5c700 1 --2- 172.21.15.142:0/1225276616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46080ff500 0x7f46080ff8e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:39.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.632+0000 7f460cb5c700 1 -- 172.21.15.142:0/1225276616 >> 172.21.15.142:0/1225276616 conn(0x7f4608074d80 msgr2=0x7f4608075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:39.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.632+0000 7f460cb5c700 1 -- 172.21.15.142:0/1225276616 shutdown_connections 2024-03-29T05:01:39.642 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.632+0000 7f460cb5c700 1 -- 172.21.15.142:0/1225276616 wait complete. 2024-03-29T05:01:39.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f460cb5c700 1 Processor -- start 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f460cb5c700 1 -- start start 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f460cb5c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46080ff500 0x7f460819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f460cb5c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46080ffeb0 0x7f460819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f460cb5c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460819d140 con 0x7f46080ffeb0 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f460cb5c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46081965f0 con 0x7f46080ff500 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f4605d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46080ffeb0 0x7f460819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f4605d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46080ffeb0 0x7f460819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37746/0 (socket says 172.21.15.142:37746) 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f4605d9b700 1 -- 172.21.15.142:0/2820311700 learned_addr learned my addr 172.21.15.142:0/2820311700 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f460659c700 1 --2- 172.21.15.142:0/2820311700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46080ff500 0x7f460819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f4605d9b700 1 -- 172.21.15.142:0/2820311700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46080ff500 msgr2=0x7f460819c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f4605d9b700 1 --2- 172.21.15.142:0/2820311700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46080ff500 0x7f460819c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f4605d9b700 1 -- 172.21.15.142:0/2820311700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f800a700 con 0x7f46080ffeb0 2024-03-29T05:01:39.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.633+0000 7f4605d9b700 1 --2- 172.21.15.142:0/2820311700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46080ffeb0 0x7f460819cab0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00d900 tx=0x7f45fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:39.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.634+0000 7f45f77fe700 1 -- 172.21.15.142:0/2820311700 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45fc009900 con 0x7f46080ffeb0 2024-03-29T05:01:39.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.634+0000 7f460cb5c700 1 -- 172.21.15.142:0/2820311700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46081968d0 con 0x7f46080ffeb0 2024-03-29T05:01:39.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.634+0000 7f45f77fe700 1 -- 172.21.15.142:0/2820311700 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45fc010460 con 0x7f46080ffeb0 2024-03-29T05:01:39.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.634+0000 7f45f77fe700 1 -- 172.21.15.142:0/2820311700 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45fc00f5f0 con 0x7f46080ffeb0 2024-03-29T05:01:39.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.634+0000 7f460cb5c700 1 -- 172.21.15.142:0/2820311700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4608196e20 con 0x7f46080ffeb0 2024-03-29T05:01:39.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.635+0000 7f460cb5c700 1 -- 172.21.15.142:0/2820311700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4608107820 con 0x7f46080ffeb0 2024-03-29T05:01:39.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.635+0000 7f45f77fe700 1 -- 172.21.15.142:0/2820311700 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f45fc009a60 con 0x7f46080ffeb0 2024-03-29T05:01:39.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.635+0000 7f45f77fe700 1 --2- 172.21.15.142:0/2820311700 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45f006dd60 0x7f45f0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:39.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.636+0000 7f45f77fe700 1 -- 172.21.15.142:0/2820311700 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f45fc08d820 con 0x7f46080ffeb0 2024-03-29T05:01:39.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.636+0000 7f460659c700 1 --2- 172.21.15.142:0/2820311700 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45f006dd60 0x7f45f0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:39.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.636+0000 7f460659c700 1 --2- 172.21.15.142:0/2820311700 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45f006dd60 0x7f45f0070220 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45f8000c00 tx=0x7f45f8005de0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:39.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.640+0000 7f45f77fe700 1 -- 172.21.15.142:0/2820311700 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f45fc05a770 con 0x7f46080ffeb0 2024-03-29T05:01:39.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.855+0000 7f460cb5c700 1 -- 172.21.15.142:0/2820311700 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46081970c0 con 0x7f45f006dd60 2024-03-29T05:01:39.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.857+0000 7f45f77fe700 1 -- 172.21.15.142:0/2820311700 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f46081970c0 con 0x7f45f006dd60 2024-03-29T05:01:39.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.859+0000 7f460cb5c700 1 -- 172.21.15.142:0/2820311700 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45f006dd60 msgr2=0x7f45f0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:39.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.859+0000 7f460cb5c700 1 --2- 172.21.15.142:0/2820311700 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45f006dd60 0x7f45f0070220 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45f8000c00 tx=0x7f45f8005de0 comp rx=0 tx=0).stop 2024-03-29T05:01:39.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.859+0000 7f460cb5c700 1 -- 172.21.15.142:0/2820311700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46080ffeb0 msgr2=0x7f460819cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:39.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.859+0000 7f460cb5c700 1 --2- 172.21.15.142:0/2820311700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46080ffeb0 0x7f460819cab0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00d900 tx=0x7f45fc00dc10 comp rx=0 tx=0).stop 2024-03-29T05:01:39.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.859+0000 7f460cb5c700 1 -- 172.21.15.142:0/2820311700 shutdown_connections 2024-03-29T05:01:39.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.859+0000 7f460cb5c700 1 --2- 172.21.15.142:0/2820311700 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f45f006dd60 0x7f45f0070220 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:39.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.859+0000 7f460cb5c700 1 --2- 172.21.15.142:0/2820311700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46080ff500 0x7f460819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:39.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.859+0000 7f460cb5c700 1 --2- 172.21.15.142:0/2820311700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46080ffeb0 0x7f460819cab0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:39.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.859+0000 7f460cb5c700 1 -- 172.21.15.142:0/2820311700 >> 172.21.15.142:0/2820311700 conn(0x7f4608074d80 msgr2=0x7f46080feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:39.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.859+0000 7f460cb5c700 1 -- 172.21.15.142:0/2820311700 shutdown_connections 2024-03-29T05:01:39.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.859+0000 7f460cb5c700 1 -- 172.21.15.142:0/2820311700 wait complete. 2024-03-29T05:01:39.878 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:01:39.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.965+0000 7fa09ce6e700 1 -- 172.21.15.142:0/3085047870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0980ffbd0 msgr2=0x7fa098100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:39.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.965+0000 7fa09ce6e700 1 --2- 172.21.15.142:0/3085047870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0980ffbd0 0x7fa098100050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa088009910 tx=0x7fa088009c20 comp rx=0 tx=0).stop 2024-03-29T05:01:39.974 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:39 smithi142 ceph-mon[112475]: pgmap v400: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:39.977 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.965+0000 7fa09ce6e700 1 -- 172.21.15.142:0/3085047870 shutdown_connections 2024-03-29T05:01:39.977 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.965+0000 7fa09ce6e700 1 --2- 172.21.15.142:0/3085047870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0980ffbd0 0x7fa098100050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:39.977 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.965+0000 7fa09ce6e700 1 --2- 172.21.15.142:0/3085047870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa098105a40 0x7fa098105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:39.977 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.965+0000 7fa09ce6e700 1 -- 172.21.15.142:0/3085047870 >> 172.21.15.142:0/3085047870 conn(0x7fa0980755a0 msgr2=0x7fa0980759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:39.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.965+0000 7fa09ce6e700 1 -- 172.21.15.142:0/3085047870 shutdown_connections 2024-03-29T05:01:39.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.965+0000 7fa09ce6e700 1 -- 172.21.15.142:0/3085047870 wait complete. 2024-03-29T05:01:39.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.966+0000 7fa09ce6e700 1 Processor -- start 2024-03-29T05:01:39.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.966+0000 7fa09ce6e700 1 -- start start 2024-03-29T05:01:39.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.966+0000 7fa09ce6e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0980ffbd0 0x7fa098101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:39.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.966+0000 7fa09ce6e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa098105a40 0x7fa098101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:39.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.966+0000 7fa09ce6e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa098102090 con 0x7fa0980ffbd0 2024-03-29T05:01:39.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.966+0000 7fa09ce6e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0981021d0 con 0x7fa098105a40 2024-03-29T05:01:39.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.966+0000 7fa09659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0980ffbd0 0x7fa098101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:39.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.966+0000 7fa09659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0980ffbd0 0x7fa098101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37760/0 (socket says 172.21.15.142:37760) 2024-03-29T05:01:39.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.966+0000 7fa09659c700 1 -- 172.21.15.142:0/1045378815 learned_addr learned my addr 172.21.15.142:0/1045378815 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:39.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.967+0000 7fa095d9b700 1 --2- 172.21.15.142:0/1045378815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa098105a40 0x7fa098101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:39.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.967+0000 7fa09659c700 1 -- 172.21.15.142:0/1045378815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa098105a40 msgr2=0x7fa098101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:39.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.967+0000 7fa09659c700 1 --2- 172.21.15.142:0/1045378815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa098105a40 0x7fa098101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:39.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.967+0000 7fa09659c700 1 -- 172.21.15.142:0/1045378815 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0880095f0 con 0x7fa0980ffbd0 2024-03-29T05:01:39.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.967+0000 7fa09659c700 1 --2- 172.21.15.142:0/1045378815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0980ffbd0 0x7fa098101610 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00d900 tx=0x7fa08c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:39.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.967+0000 7fa0877fe700 1 -- 172.21.15.142:0/1045378815 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa08c0041f0 con 0x7fa0980ffbd0 2024-03-29T05:01:39.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.967+0000 7fa09ce6e700 1 -- 172.21.15.142:0/1045378815 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0981a2900 con 0x7fa0980ffbd0 2024-03-29T05:01:39.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.967+0000 7fa0877fe700 1 -- 172.21.15.142:0/1045378815 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa08c004cf0 con 0x7fa0980ffbd0 2024-03-29T05:01:39.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.967+0000 7fa0877fe700 1 -- 172.21.15.142:0/1045378815 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa08c00b750 con 0x7fa0980ffbd0 2024-03-29T05:01:39.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.967+0000 7fa09ce6e700 1 -- 172.21.15.142:0/1045378815 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0981a2d10 con 0x7fa0980ffbd0 2024-03-29T05:01:39.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.968+0000 7fa09ce6e700 1 -- 172.21.15.142:0/1045378815 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa098108fe0 con 0x7fa0980ffbd0 2024-03-29T05:01:39.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.969+0000 7fa0877fe700 1 -- 172.21.15.142:0/1045378815 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fa08c00b8b0 con 0x7fa0980ffbd0 2024-03-29T05:01:39.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.969+0000 7fa0877fe700 1 --2- 172.21.15.142:0/1045378815 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa08006dd60 0x7fa080070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:39.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.969+0000 7fa0877fe700 1 -- 172.21.15.142:0/1045378815 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa08c08db50 con 0x7fa0980ffbd0 2024-03-29T05:01:39.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.969+0000 7fa095d9b700 1 --2- 172.21.15.142:0/1045378815 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa08006dd60 0x7fa080070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:39.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.969+0000 7fa095d9b700 1 --2- 172.21.15.142:0/1045378815 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa08006dd60 0x7fa080070220 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa098102e00 tx=0x7fa08801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:39.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:39.976+0000 7fa0877fe700 1 -- 172.21.15.142:0/1045378815 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa08c056f70 con 0x7fa0980ffbd0 2024-03-29T05:01:40.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:39 smithi152 ceph-mon[88186]: pgmap v400: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:40.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.188+0000 7fa09ce6e700 1 -- 172.21.15.142:0/1045378815 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa09802cdd0 con 0x7fa08006dd60 2024-03-29T05:01:40.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.189+0000 7fa0877fe700 1 -- 172.21.15.142:0/1045378815 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa09802cdd0 con 0x7fa08006dd60 2024-03-29T05:01:40.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.191+0000 7fa09ce6e700 1 -- 172.21.15.142:0/1045378815 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa08006dd60 msgr2=0x7fa080070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:40.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.191+0000 7fa09ce6e700 1 --2- 172.21.15.142:0/1045378815 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa08006dd60 0x7fa080070220 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa098102e00 tx=0x7fa08801a040 comp rx=0 tx=0).stop 2024-03-29T05:01:40.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.191+0000 7fa09ce6e700 1 -- 172.21.15.142:0/1045378815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0980ffbd0 msgr2=0x7fa098101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:40.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.191+0000 7fa09ce6e700 1 --2- 172.21.15.142:0/1045378815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0980ffbd0 0x7fa098101610 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00d900 tx=0x7fa08c00dc10 comp rx=0 tx=0).stop 2024-03-29T05:01:40.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.191+0000 7fa09ce6e700 1 -- 172.21.15.142:0/1045378815 shutdown_connections 2024-03-29T05:01:40.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.191+0000 7fa09ce6e700 1 --2- 172.21.15.142:0/1045378815 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa08006dd60 0x7fa080070220 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.191+0000 7fa09ce6e700 1 --2- 172.21.15.142:0/1045378815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0980ffbd0 0x7fa098101610 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.191+0000 7fa09ce6e700 1 --2- 172.21.15.142:0/1045378815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa098105a40 0x7fa098101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.191+0000 7fa09ce6e700 1 -- 172.21.15.142:0/1045378815 >> 172.21.15.142:0/1045378815 conn(0x7fa0980755a0 msgr2=0x7fa098068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:40.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.191+0000 7fa09ce6e700 1 -- 172.21.15.142:0/1045378815 shutdown_connections 2024-03-29T05:01:40.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.191+0000 7fa09ce6e700 1 -- 172.21.15.142:0/1045378815 wait complete. 2024-03-29T05:01:40.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.296+0000 7fadb3a78700 1 -- 172.21.15.142:0/3459978639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac101cc0 msgr2=0x7fadac1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:40.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.296+0000 7fadb3a78700 1 --2- 172.21.15.142:0/3459978639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac101cc0 0x7fadac1020a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fada4009910 tx=0x7fada4009c20 comp rx=0 tx=0).stop 2024-03-29T05:01:40.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.296+0000 7fadb3a78700 1 -- 172.21.15.142:0/3459978639 shutdown_connections 2024-03-29T05:01:40.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.296+0000 7fadb3a78700 1 --2- 172.21.15.142:0/3459978639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadac1025e0 0x7fadac10aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.296+0000 7fadb3a78700 1 --2- 172.21.15.142:0/3459978639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac101cc0 0x7fadac1020a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.296+0000 7fadb3a78700 1 -- 172.21.15.142:0/3459978639 >> 172.21.15.142:0/3459978639 conn(0x7fadac0fb570 msgr2=0x7fadac0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:40.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.296+0000 7fadb3a78700 1 -- 172.21.15.142:0/3459978639 shutdown_connections 2024-03-29T05:01:40.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.296+0000 7fadb3a78700 1 -- 172.21.15.142:0/3459978639 wait complete. 2024-03-29T05:01:40.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.296+0000 7fadb3a78700 1 Processor -- start 2024-03-29T05:01:40.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb3a78700 1 -- start start 2024-03-29T05:01:40.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb3a78700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac101cc0 0x7fadac1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:40.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb3a78700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadac1025e0 0x7fadac196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:40.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb3a78700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac197290 con 0x7fadac101cc0 2024-03-29T05:01:40.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb3a78700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac199fc0 con 0x7fadac1025e0 2024-03-29T05:01:40.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb1814700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac101cc0 0x7fadac1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:40.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb1814700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac101cc0 0x7fadac1966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37776/0 (socket says 172.21.15.142:37776) 2024-03-29T05:01:40.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb1814700 1 -- 172.21.15.142:0/2134376649 learned_addr learned my addr 172.21.15.142:0/2134376649 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:40.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb1013700 1 --2- 172.21.15.142:0/2134376649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadac1025e0 0x7fadac196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:40.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb1814700 1 -- 172.21.15.142:0/2134376649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadac1025e0 msgr2=0x7fadac196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:40.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb1814700 1 --2- 172.21.15.142:0/2134376649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadac1025e0 0x7fadac196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb1814700 1 -- 172.21.15.142:0/2134376649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada40095f0 con 0x7fadac101cc0 2024-03-29T05:01:40.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb1814700 1 --2- 172.21.15.142:0/2134376649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac101cc0 0x7fadac1966c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fada4006010 tx=0x7fada400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:40.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadaaffd700 1 -- 172.21.15.142:0/2134376649 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fada401d070 con 0x7fadac101cc0 2024-03-29T05:01:40.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadaaffd700 1 -- 172.21.15.142:0/2134376649 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fada400f460 con 0x7fadac101cc0 2024-03-29T05:01:40.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.297+0000 7fadb3a78700 1 -- 172.21.15.142:0/2134376649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadac19a240 con 0x7fadac101cc0 2024-03-29T05:01:40.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.298+0000 7fadaaffd700 1 -- 172.21.15.142:0/2134376649 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fada4017630 con 0x7fadac101cc0 2024-03-29T05:01:40.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.298+0000 7fadb3a78700 1 -- 172.21.15.142:0/2134376649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac19a730 con 0x7fadac101cc0 2024-03-29T05:01:40.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.298+0000 7fadb3a78700 1 -- 172.21.15.142:0/2134376649 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac1081d0 con 0x7fadac101cc0 2024-03-29T05:01:40.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.299+0000 7fadaaffd700 1 -- 172.21.15.142:0/2134376649 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fada400f5d0 con 0x7fadac101cc0 2024-03-29T05:01:40.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.300+0000 7fadaaffd700 1 --2- 172.21.15.142:0/2134376649 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad9406dd60 0x7fad94070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:40.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.300+0000 7fadaaffd700 1 -- 172.21.15.142:0/2134376649 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fada408f150 con 0x7fadac101cc0 2024-03-29T05:01:40.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.300+0000 7fadb1013700 1 --2- 172.21.15.142:0/2134376649 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad9406dd60 0x7fad94070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:40.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.300+0000 7fadb1013700 1 --2- 172.21.15.142:0/2134376649 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad9406dd60 0x7fad94070220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fada0005950 tx=0x7fada00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:40.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.304+0000 7fadaaffd700 1 -- 172.21.15.142:0/2134376649 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fada4058570 con 0x7fadac101cc0 2024-03-29T05:01:40.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.507+0000 7fadb3a78700 1 -- 172.21.15.142:0/2134376649 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fadac19aa10 con 0x7fad9406dd60 2024-03-29T05:01:40.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.518+0000 7fadaaffd700 1 -- 172.21.15.142:0/2134376649 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fadac19aa10 con 0x7fad9406dd60 2024-03-29T05:01:40.527 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:01:40.527 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (13m) 6m ago 24m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:01:40.527 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (25m) 6m ago 25m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:01:40.527 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (24m) 6m ago 24m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:01:40.527 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (10m) 6m ago 25m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:01:40.527 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (10m) 6m ago 24m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (13m) 6m ago 24m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (6m) 6m ago 19m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (6m) 6m ago 19m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (6m) 6m ago 19m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (6m) 6m ago 19m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (15m) 6m ago 26m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (14m) 6m ago 24m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (11m) 6m ago 26m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (11m) 6m ago 23m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (14m) 6m ago 25m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (14m) 6m ago 24m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (10m) 6m ago 23m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (9m) 6m ago 22m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (9m) 6m ago 22m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:01:40.528 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (8m) 6m ago 21m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:01:40.529 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (7m) 6m ago 21m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:01:40.529 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (7m) 6m ago 21m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:01:40.529 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (13m) 6m ago 24m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:01:40.529 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.520+0000 7fadb3a78700 1 -- 172.21.15.142:0/2134376649 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad9406dd60 msgr2=0x7fad94070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:40.529 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.520+0000 7fadb3a78700 1 --2- 172.21.15.142:0/2134376649 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad9406dd60 0x7fad94070220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fada0005950 tx=0x7fada00058e0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.520+0000 7fadb3a78700 1 -- 172.21.15.142:0/2134376649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac101cc0 msgr2=0x7fadac1966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:40.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.520+0000 7fadb3a78700 1 --2- 172.21.15.142:0/2134376649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac101cc0 0x7fadac1966c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fada4006010 tx=0x7fada400be20 comp rx=0 tx=0).stop 2024-03-29T05:01:40.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.520+0000 7fadb3a78700 1 -- 172.21.15.142:0/2134376649 shutdown_connections 2024-03-29T05:01:40.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.520+0000 7fadb3a78700 1 --2- 172.21.15.142:0/2134376649 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fad9406dd60 0x7fad94070220 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.520+0000 7fadb3a78700 1 --2- 172.21.15.142:0/2134376649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac101cc0 0x7fadac1966c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.520+0000 7fadb3a78700 1 --2- 172.21.15.142:0/2134376649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadac1025e0 0x7fadac196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.520+0000 7fadb3a78700 1 -- 172.21.15.142:0/2134376649 >> 172.21.15.142:0/2134376649 conn(0x7fadac0fb570 msgr2=0x7fadac105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:40.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.521+0000 7fadb3a78700 1 -- 172.21.15.142:0/2134376649 shutdown_connections 2024-03-29T05:01:40.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.521+0000 7fadb3a78700 1 -- 172.21.15.142:0/2134376649 wait complete. 2024-03-29T05:01:40.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.623+0000 7f5ccf146700 1 -- 172.21.15.142:0/2704490376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8105a40 msgr2=0x7f5cc8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:40.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.623+0000 7f5ccf146700 1 --2- 172.21.15.142:0/2704490376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8105a40 0x7f5cc8105e20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009910 tx=0x7f5cbc009c20 comp rx=0 tx=0).stop 2024-03-29T05:01:40.633 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.623+0000 7f5ccf146700 1 -- 172.21.15.142:0/2704490376 shutdown_connections 2024-03-29T05:01:40.633 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.623+0000 7f5ccf146700 1 --2- 172.21.15.142:0/2704490376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc80ffbd0 0x7f5cc8100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.633 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.623+0000 7f5ccf146700 1 --2- 172.21.15.142:0/2704490376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8105a40 0x7f5cc8105e20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.633 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.623+0000 7f5ccf146700 1 -- 172.21.15.142:0/2704490376 >> 172.21.15.142:0/2704490376 conn(0x7f5cc80755a0 msgr2=0x7f5cc80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:40.633 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.623+0000 7f5ccf146700 1 -- 172.21.15.142:0/2704490376 shutdown_connections 2024-03-29T05:01:40.633 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.623+0000 7f5ccf146700 1 -- 172.21.15.142:0/2704490376 wait complete. 2024-03-29T05:01:40.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.624+0000 7f5ccf146700 1 Processor -- start 2024-03-29T05:01:40.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.624+0000 7f5ccf146700 1 -- start start 2024-03-29T05:01:40.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5ccf146700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc80ffbd0 0x7f5cc81975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:40.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5ccf146700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc8105a40 0x7f5cc8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:40.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5ccf146700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8198110 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5ccf146700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8198250 con 0x7f5cc8105a40 2024-03-29T05:01:40.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5cccee2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc80ffbd0 0x7f5cc81975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5cccee2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc80ffbd0 0x7f5cc81975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37794/0 (socket says 172.21.15.142:37794) 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5cccee2700 1 -- 172.21.15.142:0/2904196871 learned_addr learned my addr 172.21.15.142:0/2904196871 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5cc7fff700 1 --2- 172.21.15.142:0/2904196871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc8105a40 0x7f5cc8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5cccee2700 1 -- 172.21.15.142:0/2904196871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc8105a40 msgr2=0x7f5cc8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5cccee2700 1 --2- 172.21.15.142:0/2904196871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc8105a40 0x7f5cc8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5cccee2700 1 -- 172.21.15.142:0/2904196871 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc0095f0 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5cc7fff700 1 --2- 172.21.15.142:0/2904196871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc8105a40 0x7f5cc8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5cccee2700 1 --2- 172.21.15.142:0/2904196871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc80ffbd0 0x7f5cc81975b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc006010 tx=0x7f5cbc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5cc5ffb700 1 -- 172.21.15.142:0/2904196871 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cbc01d070 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5cc5ffb700 1 -- 172.21.15.142:0/2904196871 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cbc004fb0 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5cc5ffb700 1 -- 172.21.15.142:0/2904196871 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cbc022ea0 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.625+0000 7f5ccf146700 1 -- 172.21.15.142:0/2904196871 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc81a0620 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.626+0000 7f5ccf146700 1 -- 172.21.15.142:0/2904196871 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc81a0a30 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.627+0000 7f5ccf146700 1 -- 172.21.15.142:0/2904196871 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc8108040 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.627+0000 7f5cc5ffb700 1 -- 172.21.15.142:0/2904196871 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f5cbc022470 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.627+0000 7f5cc5ffb700 1 --2- 172.21.15.142:0/2904196871 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5cb806dd60 0x7f5cb8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:40.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.627+0000 7f5cc5ffb700 1 -- 172.21.15.142:0/2904196871 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f5cbc08f180 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.628+0000 7f5cc7fff700 1 --2- 172.21.15.142:0/2904196871 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5cb806dd60 0x7f5cb8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:40.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.628+0000 7f5cc7fff700 1 --2- 172.21.15.142:0/2904196871 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5cb806dd60 0x7f5cb8070220 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4005fd0 tx=0x7f5cb4005de0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:40.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.632+0000 7f5cc5ffb700 1 -- 172.21.15.142:0/2904196871 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5cbc0586d0 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.910+0000 7f5ccf146700 1 -- 172.21.15.142:0/2904196871 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5cc802cd30 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.911+0000 7f5cc5ffb700 1 -- 172.21.15.142:0/2904196871 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5cbc022720 con 0x7f5cc80ffbd0 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:01:40.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:01:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:01:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:01:40.921 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:01:40.921 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:01:40.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.913+0000 7f5ccf146700 1 -- 172.21.15.142:0/2904196871 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5cb806dd60 msgr2=0x7f5cb8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:40.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.913+0000 7f5ccf146700 1 --2- 172.21.15.142:0/2904196871 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5cb806dd60 0x7f5cb8070220 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4005fd0 tx=0x7f5cb4005de0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.913+0000 7f5ccf146700 1 -- 172.21.15.142:0/2904196871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc80ffbd0 msgr2=0x7f5cc81975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:40.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.913+0000 7f5ccf146700 1 --2- 172.21.15.142:0/2904196871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc80ffbd0 0x7f5cc81975b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc006010 tx=0x7f5cbc00be20 comp rx=0 tx=0).stop 2024-03-29T05:01:40.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.913+0000 7f5ccf146700 1 -- 172.21.15.142:0/2904196871 shutdown_connections 2024-03-29T05:01:40.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.913+0000 7f5ccf146700 1 --2- 172.21.15.142:0/2904196871 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5cb806dd60 0x7f5cb8070220 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.913+0000 7f5ccf146700 1 --2- 172.21.15.142:0/2904196871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc80ffbd0 0x7f5cc81975b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.913+0000 7f5ccf146700 1 --2- 172.21.15.142:0/2904196871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cc8105a40 0x7f5cc8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:40.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.913+0000 7f5ccf146700 1 -- 172.21.15.142:0/2904196871 >> 172.21.15.142:0/2904196871 conn(0x7f5cc80755a0 msgr2=0x7f5cc80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:40.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.913+0000 7f5ccf146700 1 -- 172.21.15.142:0/2904196871 shutdown_connections 2024-03-29T05:01:40.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:40.913+0000 7f5ccf146700 1 -- 172.21.15.142:0/2904196871 wait complete. 2024-03-29T05:01:41.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.017+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3496663653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec105a40 msgr2=0x7f3fec105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:41.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.017+0000 7f3ff47c5700 1 --2- 172.21.15.142:0/3496663653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec105a40 0x7f3fec105e20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009910 tx=0x7f3fdc009c20 comp rx=0 tx=0).stop 2024-03-29T05:01:41.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.017+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3496663653 shutdown_connections 2024-03-29T05:01:41.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.017+0000 7f3ff47c5700 1 --2- 172.21.15.142:0/3496663653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fec0ffbd0 0x7f3fec100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.017+0000 7f3ff47c5700 1 --2- 172.21.15.142:0/3496663653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec105a40 0x7f3fec105e20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.017+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3496663653 >> 172.21.15.142:0/3496663653 conn(0x7f3fec0755a0 msgr2=0x7f3fec0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.017+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3496663653 shutdown_connections 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.017+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3496663653 wait complete. 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff47c5700 1 Processor -- start 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff47c5700 1 -- start start 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff47c5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec0ffbd0 0x7f3fec1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff47c5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fec105a40 0x7f3fec197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff47c5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec198300 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff47c5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec198470 con 0x7f3fec105a40 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff2561700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec0ffbd0 0x7f3fec1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff2561700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec0ffbd0 0x7f3fec1976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37806/0 (socket says 172.21.15.142:37806) 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff2561700 1 -- 172.21.15.142:0/3482179565 learned_addr learned my addr 172.21.15.142:0/3482179565 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:41.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff2561700 1 -- 172.21.15.142:0/3482179565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fec105a40 msgr2=0x7f3fec197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:41.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff2561700 1 --2- 172.21.15.142:0/3482179565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fec105a40 0x7f3fec197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff2561700 1 -- 172.21.15.142:0/3482179565 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc0095f0 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3ff2561700 1 --2- 172.21.15.142:0/3482179565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec0ffbd0 0x7f3fec1976e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc006010 tx=0x7f3fdc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:41.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3fe37fe700 1 -- 172.21.15.142:0/3482179565 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fdc01d070 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.018+0000 7f3fe37fe700 1 -- 172.21.15.142:0/3482179565 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fdc004fb0 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.019+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3482179565 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fec1a0750 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.019+0000 7f3fe37fe700 1 -- 172.21.15.142:0/3482179565 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fdc022ea0 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.019+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3482179565 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec1a0bf0 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.019+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3482179565 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fec108040 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.020+0000 7f3fe37fe700 1 -- 172.21.15.142:0/3482179565 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f3fdc022470 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.020+0000 7f3fe37fe700 1 --2- 172.21.15.142:0/3482179565 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3fd806dd60 0x7f3fd8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:41.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.020+0000 7f3fe37fe700 1 -- 172.21.15.142:0/3482179565 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f3fdc08f3b0 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.020+0000 7f3ff1d60700 1 --2- 172.21.15.142:0/3482179565 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3fd806dd60 0x7f3fd8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:41.031 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.021+0000 7f3ff1d60700 1 --2- 172.21.15.142:0/3482179565 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3fd806dd60 0x7f3fd8070220 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3fec069060 tx=0x7f3fe8006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:41.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.025+0000 7f3fe37fe700 1 -- 172.21.15.142:0/3482179565 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3fdc058a20 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.270 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.261+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3482179565 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3fec04ea90 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.261+0000 7f3fe37fe700 1 -- 172.21.15.142:0/3482179565 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f3fdc022720 con 0x7f3fec0ffbd0 2024-03-29T05:01:41.271 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:01:41.271 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:01:41.271 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:01:41.271 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:01:41.271 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:01:41.271 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:01:41.271 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:01:41.271 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:01:41.271 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:01:41.271 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:01:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:01:41.273 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:01:41.274 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:01:41.274 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:01:41.274 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:01:41.274 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:01:41.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.265+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3482179565 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3fd806dd60 msgr2=0x7f3fd8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:41.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.265+0000 7f3ff47c5700 1 --2- 172.21.15.142:0/3482179565 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3fd806dd60 0x7f3fd8070220 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3fec069060 tx=0x7f3fe8006d20 comp rx=0 tx=0).stop 2024-03-29T05:01:41.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.265+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3482179565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec0ffbd0 msgr2=0x7f3fec1976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:41.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.265+0000 7f3ff47c5700 1 --2- 172.21.15.142:0/3482179565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec0ffbd0 0x7f3fec1976e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc006010 tx=0x7f3fdc00be20 comp rx=0 tx=0).stop 2024-03-29T05:01:41.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.265+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3482179565 shutdown_connections 2024-03-29T05:01:41.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.265+0000 7f3ff47c5700 1 --2- 172.21.15.142:0/3482179565 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3fd806dd60 0x7f3fd8070220 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.265+0000 7f3ff47c5700 1 --2- 172.21.15.142:0/3482179565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec0ffbd0 0x7f3fec1976e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.265+0000 7f3ff47c5700 1 --2- 172.21.15.142:0/3482179565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fec105a40 0x7f3fec197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.265+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3482179565 >> 172.21.15.142:0/3482179565 conn(0x7f3fec0755a0 msgr2=0x7f3fec0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:41.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.265+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3482179565 shutdown_connections 2024-03-29T05:01:41.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.265+0000 7f3ff47c5700 1 -- 172.21.15.142:0/3482179565 wait complete. 2024-03-29T05:01:41.275 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:01:41.380 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.371+0000 7f4c663d7700 1 -- 172.21.15.142:0/1442690140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c60101770 msgr2=0x7f4c601057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:41.380 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.371+0000 7f4c663d7700 1 --2- 172.21.15.142:0/1442690140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c60101770 0x7f4c601057c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009910 tx=0x7f4c50009c20 comp rx=0 tx=0).stop 2024-03-29T05:01:41.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.371+0000 7f4c663d7700 1 -- 172.21.15.142:0/1442690140 shutdown_connections 2024-03-29T05:01:41.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.371+0000 7f4c663d7700 1 --2- 172.21.15.142:0/1442690140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c60101770 0x7f4c601057c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.371+0000 7f4c663d7700 1 --2- 172.21.15.142:0/1442690140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c60100dc0 0x7f4c601011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.371+0000 7f4c663d7700 1 -- 172.21.15.142:0/1442690140 >> 172.21.15.142:0/1442690140 conn(0x7f4c600fc650 msgr2=0x7f4c600fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:41.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.371+0000 7f4c663d7700 1 -- 172.21.15.142:0/1442690140 shutdown_connections 2024-03-29T05:01:41.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.371+0000 7f4c663d7700 1 -- 172.21.15.142:0/1442690140 wait complete. 2024-03-29T05:01:41.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c663d7700 1 Processor -- start 2024-03-29T05:01:41.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c663d7700 1 -- start start 2024-03-29T05:01:41.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c663d7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c60100dc0 0x7f4c60073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:41.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c663d7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c60101770 0x7f4c600735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:41.383 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c663d7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c60073b10 con 0x7f4c60100dc0 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c663d7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c60073c50 con 0x7f4c60101770 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c5ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c60100dc0 0x7f4c60073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c5ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c60100dc0 0x7f4c60073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37820/0 (socket says 172.21.15.142:37820) 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c5ffff700 1 -- 172.21.15.142:0/3759468743 learned_addr learned my addr 172.21.15.142:0/3759468743 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c5ffff700 1 -- 172.21.15.142:0/3759468743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c60101770 msgr2=0x7f4c600735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c5f7fe700 1 --2- 172.21.15.142:0/3759468743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c60101770 0x7f4c600735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c5ffff700 1 --2- 172.21.15.142:0/3759468743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c60101770 0x7f4c600735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c5ffff700 1 -- 172.21.15.142:0/3759468743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c500095f0 con 0x7f4c60100dc0 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.372+0000 7f4c5f7fe700 1 --2- 172.21.15.142:0/3759468743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c60101770 0x7f4c600735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.373+0000 7f4c5ffff700 1 --2- 172.21.15.142:0/3759468743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c60100dc0 0x7f4c60073090 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800b4b0 tx=0x7f4c4800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.373+0000 7f4c5d7fa700 1 -- 172.21.15.142:0/3759468743 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c48011800 con 0x7f4c60100dc0 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.373+0000 7f4c663d7700 1 -- 172.21.15.142:0/3759468743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c601a2a70 con 0x7f4c60100dc0 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.373+0000 7f4c5d7fa700 1 -- 172.21.15.142:0/3759468743 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c48011e20 con 0x7f4c60100dc0 2024-03-29T05:01:41.384 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.373+0000 7f4c5d7fa700 1 -- 172.21.15.142:0/3759468743 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4800f2e0 con 0x7f4c60100dc0 2024-03-29T05:01:41.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.373+0000 7f4c663d7700 1 -- 172.21.15.142:0/3759468743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c601a2e80 con 0x7f4c60100dc0 2024-03-29T05:01:41.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.374+0000 7f4c663d7700 1 -- 172.21.15.142:0/3759468743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c601090f0 con 0x7f4c60100dc0 2024-03-29T05:01:41.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.374+0000 7f4c5d7fa700 1 -- 172.21.15.142:0/3759468743 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f4c48010380 con 0x7f4c60100dc0 2024-03-29T05:01:41.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.375+0000 7f4c5d7fa700 1 --2- 172.21.15.142:0/3759468743 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4c4c06dd60 0x7f4c4c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:41.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.375+0000 7f4c5d7fa700 1 -- 172.21.15.142:0/3759468743 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f4c4808d890 con 0x7f4c60100dc0 2024-03-29T05:01:41.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.375+0000 7f4c5f7fe700 1 --2- 172.21.15.142:0/3759468743 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4c4c06dd60 0x7f4c4c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:41.385 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.375+0000 7f4c5f7fe700 1 --2- 172.21.15.142:0/3759468743 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4c4c06dd60 0x7f4c4c070220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c50006010 tx=0x7f4c5000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:41.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.379+0000 7f4c5d7fa700 1 -- 172.21.15.142:0/3759468743 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4c48056cb0 con 0x7f4c60100dc0 2024-03-29T05:01:41.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.592+0000 7f4c663d7700 1 -- 172.21.15.142:0/3759468743 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c6002ccc0 con 0x7f4c4c06dd60 2024-03-29T05:01:41.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.593+0000 7f4c5d7fa700 1 -- 172.21.15.142:0/3759468743 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4c6002ccc0 con 0x7f4c4c06dd60 2024-03-29T05:01:41.602 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:01:41.602 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:01:41.602 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:01:41.602 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:01:41.602 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:01:41.603 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:01:41.603 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:01:41.603 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:01:41.603 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:01:41.603 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:01:41.603 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:01:41.603 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:01:41.603 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:01:41.603 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:01:41.603 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:01:41.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.595+0000 7f4c663d7700 1 -- 172.21.15.142:0/3759468743 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4c4c06dd60 msgr2=0x7f4c4c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:41.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.595+0000 7f4c663d7700 1 --2- 172.21.15.142:0/3759468743 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4c4c06dd60 0x7f4c4c070220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c50006010 tx=0x7f4c5000b540 comp rx=0 tx=0).stop 2024-03-29T05:01:41.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.595+0000 7f4c663d7700 1 -- 172.21.15.142:0/3759468743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c60100dc0 msgr2=0x7f4c60073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:41.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.595+0000 7f4c663d7700 1 --2- 172.21.15.142:0/3759468743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c60100dc0 0x7f4c60073090 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800b4b0 tx=0x7f4c4800b870 comp rx=0 tx=0).stop 2024-03-29T05:01:41.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.596+0000 7f4c663d7700 1 -- 172.21.15.142:0/3759468743 shutdown_connections 2024-03-29T05:01:41.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.596+0000 7f4c663d7700 1 --2- 172.21.15.142:0/3759468743 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4c4c06dd60 0x7f4c4c070220 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.596+0000 7f4c663d7700 1 --2- 172.21.15.142:0/3759468743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c60100dc0 0x7f4c60073090 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.596+0000 7f4c663d7700 1 --2- 172.21.15.142:0/3759468743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c60101770 0x7f4c600735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.596+0000 7f4c663d7700 1 -- 172.21.15.142:0/3759468743 >> 172.21.15.142:0/3759468743 conn(0x7f4c600fc650 msgr2=0x7f4c600fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:41.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.596+0000 7f4c663d7700 1 -- 172.21.15.142:0/3759468743 shutdown_connections 2024-03-29T05:01:41.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.596+0000 7f4c663d7700 1 -- 172.21.15.142:0/3759468743 wait complete. 2024-03-29T05:01:41.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.699+0000 7fe88691d700 1 -- 172.21.15.142:0/96090716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8800ffbd0 msgr2=0x7fe880100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:41.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.699+0000 7fe88691d700 1 --2- 172.21.15.142:0/96090716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8800ffbd0 0x7fe880100050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe874009910 tx=0x7fe874009c20 comp rx=0 tx=0).stop 2024-03-29T05:01:41.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.699+0000 7fe88691d700 1 -- 172.21.15.142:0/96090716 shutdown_connections 2024-03-29T05:01:41.708 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.699+0000 7fe88691d700 1 --2- 172.21.15.142:0/96090716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8800ffbd0 0x7fe880100050 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.699+0000 7fe88691d700 1 --2- 172.21.15.142:0/96090716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe880105a40 0x7fe880105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.699+0000 7fe88691d700 1 -- 172.21.15.142:0/96090716 >> 172.21.15.142:0/96090716 conn(0x7fe8800755a0 msgr2=0x7fe8800759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:41.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.699+0000 7fe88691d700 1 -- 172.21.15.142:0/96090716 shutdown_connections 2024-03-29T05:01:41.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.699+0000 7fe88691d700 1 -- 172.21.15.142:0/96090716 wait complete. 2024-03-29T05:01:41.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.700+0000 7fe88691d700 1 Processor -- start 2024-03-29T05:01:41.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.700+0000 7fe88691d700 1 -- start start 2024-03-29T05:01:41.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.700+0000 7fe88691d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8800ffbd0 0x7fe880197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:41.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.700+0000 7fe88691d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe880105a40 0x7fe880197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:41.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.700+0000 7fe88691d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe880198190 con 0x7fe8800ffbd0 2024-03-29T05:01:41.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.700+0000 7fe88691d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe880198300 con 0x7fe880105a40 2024-03-29T05:01:41.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.700+0000 7fe87ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8800ffbd0 0x7fe880197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:41.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.700+0000 7fe87ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8800ffbd0 0x7fe880197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37848/0 (socket says 172.21.15.142:37848) 2024-03-29T05:01:41.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.700+0000 7fe87ffff700 1 -- 172.21.15.142:0/231227951 learned_addr learned my addr 172.21.15.142:0/231227951 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:01:41.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.701+0000 7fe87f7fe700 1 --2- 172.21.15.142:0/231227951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe880105a40 0x7fe880197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:41.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.701+0000 7fe87ffff700 1 -- 172.21.15.142:0/231227951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe880105a40 msgr2=0x7fe880197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:41.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.701+0000 7fe87ffff700 1 --2- 172.21.15.142:0/231227951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe880105a40 0x7fe880197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:41.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.701+0000 7fe87ffff700 1 -- 172.21.15.142:0/231227951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8740095f0 con 0x7fe8800ffbd0 2024-03-29T05:01:41.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.701+0000 7fe87ffff700 1 --2- 172.21.15.142:0/231227951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8800ffbd0 0x7fe880197570 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe87000d8d0 tx=0x7fe87000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:41.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.701+0000 7fe87d7fa700 1 -- 172.21.15.142:0/231227951 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8700098a0 con 0x7fe8800ffbd0 2024-03-29T05:01:41.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.701+0000 7fe87d7fa700 1 -- 172.21.15.142:0/231227951 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe870010460 con 0x7fe8800ffbd0 2024-03-29T05:01:41.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.701+0000 7fe87d7fa700 1 -- 172.21.15.142:0/231227951 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe87000f5f0 con 0x7fe8800ffbd0 2024-03-29T05:01:41.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.701+0000 7fe88691d700 1 -- 172.21.15.142:0/231227951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8801a05e0 con 0x7fe8800ffbd0 2024-03-29T05:01:41.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.701+0000 7fe88691d700 1 -- 172.21.15.142:0/231227951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8801a0ae0 con 0x7fe8800ffbd0 2024-03-29T05:01:41.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.702+0000 7fe88691d700 1 -- 172.21.15.142:0/231227951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe880107fd0 con 0x7fe8800ffbd0 2024-03-29T05:01:41.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.703+0000 7fe87d7fa700 1 -- 172.21.15.142:0/231227951 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fe870009a00 con 0x7fe8800ffbd0 2024-03-29T05:01:41.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.703+0000 7fe87d7fa700 1 --2- 172.21.15.142:0/231227951 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe86806dd60 0x7fe868070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:01:41.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.703+0000 7fe87d7fa700 1 -- 172.21.15.142:0/231227951 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fe87008db40 con 0x7fe8800ffbd0 2024-03-29T05:01:41.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.703+0000 7fe87f7fe700 1 --2- 172.21.15.142:0/231227951 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe86806dd60 0x7fe868070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:01:41.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.704+0000 7fe87f7fe700 1 --2- 172.21.15.142:0/231227951 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe86806dd60 0x7fe868070220 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe880069060 tx=0x7fe87401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:01:41.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.708+0000 7fe87d7fa700 1 -- 172.21.15.142:0/231227951 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe870056f60 con 0x7fe8800ffbd0 2024-03-29T05:01:41.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:41 smithi142 ceph-mon[112475]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:41.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:41 smithi142 ceph-mon[112475]: pgmap v401: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:41.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:41 smithi142 ceph-mon[112475]: from='client.34770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:41.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:41 smithi142 ceph-mon[112475]: from='client.34774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:41.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:41 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2904196871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:01:41.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:41 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3482179565' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:01:41.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.989+0000 7fe88691d700 1 -- 172.21.15.142:0/231227951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe88004ea90 con 0x7fe8800ffbd0 2024-03-29T05:01:41.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.990+0000 7fe87d7fa700 1 -- 172.21.15.142:0/231227951 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fe87005a620 con 0x7fe8800ffbd0 2024-03-29T05:01:41.999 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:01:41.999 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:01:41.999 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:01:41.999 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:01:41.999 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:01:42.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.992+0000 7fe88691d700 1 -- 172.21.15.142:0/231227951 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe86806dd60 msgr2=0x7fe868070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:42.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.992+0000 7fe88691d700 1 --2- 172.21.15.142:0/231227951 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe86806dd60 0x7fe868070220 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe880069060 tx=0x7fe87401a040 comp rx=0 tx=0).stop 2024-03-29T05:01:42.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.992+0000 7fe88691d700 1 -- 172.21.15.142:0/231227951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8800ffbd0 msgr2=0x7fe880197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:01:42.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.992+0000 7fe88691d700 1 --2- 172.21.15.142:0/231227951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8800ffbd0 0x7fe880197570 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe87000d8d0 tx=0x7fe87000dbe0 comp rx=0 tx=0).stop 2024-03-29T05:01:42.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.992+0000 7fe88691d700 1 -- 172.21.15.142:0/231227951 shutdown_connections 2024-03-29T05:01:42.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.992+0000 7fe88691d700 1 --2- 172.21.15.142:0/231227951 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe86806dd60 0x7fe868070220 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:42.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.992+0000 7fe88691d700 1 --2- 172.21.15.142:0/231227951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8800ffbd0 0x7fe880197570 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:42.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.992+0000 7fe88691d700 1 --2- 172.21.15.142:0/231227951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe880105a40 0x7fe880197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:01:42.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.992+0000 7fe88691d700 1 -- 172.21.15.142:0/231227951 >> 172.21.15.142:0/231227951 conn(0x7fe8800755a0 msgr2=0x7fe8800fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:01:42.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.992+0000 7fe88691d700 1 -- 172.21.15.142:0/231227951 shutdown_connections 2024-03-29T05:01:42.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:01:41.992+0000 7fe88691d700 1 -- 172.21.15.142:0/231227951 wait complete. 2024-03-29T05:01:42.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:41 smithi152 ceph-mon[88186]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:42.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:41 smithi152 ceph-mon[88186]: pgmap v401: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:42.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:41 smithi152 ceph-mon[88186]: from='client.34770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:42.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:41 smithi152 ceph-mon[88186]: from='client.34774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:42.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:41 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2904196871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:01:42.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:41 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3482179565' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:01:43.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:42 smithi152 ceph-mon[88186]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:43.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:42 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/231227951' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:01:43.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:42 smithi142 ceph-mon[112475]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:01:43.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:42 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/231227951' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:01:44.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:43 smithi152 ceph-mon[88186]: pgmap v402: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:44.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:43 smithi142 ceph-mon[112475]: pgmap v402: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:45.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:01:45.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:01:46.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:45 smithi152 ceph-mon[88186]: pgmap v403: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:46.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:45 smithi142 ceph-mon[112475]: pgmap v403: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:48.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:47 smithi152 ceph-mon[88186]: pgmap v404: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:48.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:47 smithi142 ceph-mon[112475]: pgmap v404: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:50.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:49 smithi152 ceph-mon[88186]: pgmap v405: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:50.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:49 smithi142 ceph-mon[112475]: pgmap v405: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:51.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:50 smithi152 ceph-mon[88186]: pgmap v406: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:51.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:50 smithi142 ceph-mon[112475]: pgmap v406: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:53.299 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:52 smithi152 ceph-mon[88186]: pgmap v407: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:53.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:52 smithi142 ceph-mon[112475]: pgmap v407: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:55.299 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:55 smithi152 ceph-mon[88186]: pgmap v408: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:55.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:55 smithi142 ceph-mon[112475]: pgmap v408: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:57.299 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:57 smithi152 ceph-mon[88186]: pgmap v409: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:57.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:57 smithi142 ceph-mon[112475]: pgmap v409: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:59.299 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:01:59 smithi152 ceph-mon[88186]: pgmap v410: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:01:59.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:01:59 smithi142 ceph-mon[112475]: pgmap v410: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:00.299 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:02:00.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:02:01.299 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:01 smithi152 ceph-mon[88186]: pgmap v411: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:01.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:01 smithi142 ceph-mon[112475]: pgmap v411: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:03.299 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:03 smithi152 ceph-mon[88186]: pgmap v412: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:03.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:03 smithi142 ceph-mon[112475]: pgmap v412: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:05.299 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:05 smithi152 ceph-mon[88186]: pgmap v413: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:05.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:05 smithi142 ceph-mon[112475]: pgmap v413: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:07.299 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:07 smithi152 ceph-mon[88186]: pgmap v414: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:07.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:07 smithi142 ceph-mon[112475]: pgmap v414: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:09.299 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:09 smithi152 ceph-mon[88186]: pgmap v415: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:09.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:09 smithi142 ceph-mon[112475]: pgmap v415: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:11.299 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:11 smithi152 ceph-mon[88186]: pgmap v416: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:11.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:11 smithi142 ceph-mon[112475]: pgmap v416: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:12.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.109+0000 7f0a72714700 1 -- 172.21.15.142:0/4156785405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c105a30 msgr2=0x7f0a6c105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:12.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.109+0000 7f0a72714700 1 --2- 172.21.15.142:0/4156785405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c105a30 0x7f0a6c105e10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009910 tx=0x7f0a5c009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:12.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.109+0000 7f0a72714700 1 -- 172.21.15.142:0/4156785405 shutdown_connections 2024-03-29T05:02:12.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.109+0000 7f0a72714700 1 --2- 172.21.15.142:0/4156785405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a6c0ffa30 0x7f0a6c0ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.109+0000 7f0a72714700 1 --2- 172.21.15.142:0/4156785405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c105a30 0x7f0a6c105e10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.109+0000 7f0a72714700 1 -- 172.21.15.142:0/4156785405 >> 172.21.15.142:0/4156785405 conn(0x7f0a6c0fb540 msgr2=0x7f0a6c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:12.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.109+0000 7f0a72714700 1 -- 172.21.15.142:0/4156785405 shutdown_connections 2024-03-29T05:02:12.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.109+0000 7f0a72714700 1 -- 172.21.15.142:0/4156785405 wait complete. 2024-03-29T05:02:12.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.109+0000 7f0a72714700 1 Processor -- start 2024-03-29T05:02:12.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.109+0000 7f0a72714700 1 -- start start 2024-03-29T05:02:12.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.110+0000 7f0a72714700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c0ffa30 0x7f0a6c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:12.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.110+0000 7f0a72714700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a6c105a30 0x7f0a6c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:12.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.110+0000 7f0a72714700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c19d070 con 0x7f0a6c0ffa30 2024-03-29T05:02:12.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.110+0000 7f0a72714700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c196530 con 0x7f0a6c105a30 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.110+0000 7f0a6bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c0ffa30 0x7f0a6c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.110+0000 7f0a6bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c0ffa30 0x7f0a6c19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44928/0 (socket says 172.21.15.142:44928) 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.110+0000 7f0a6bfff700 1 -- 172.21.15.142:0/2043114725 learned_addr learned my addr 172.21.15.142:0/2043114725 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.110+0000 7f0a6b7fe700 1 --2- 172.21.15.142:0/2043114725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a6c105a30 0x7f0a6c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.110+0000 7f0a6bfff700 1 -- 172.21.15.142:0/2043114725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a6c105a30 msgr2=0x7f0a6c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.110+0000 7f0a6bfff700 1 --2- 172.21.15.142:0/2043114725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a6c105a30 0x7f0a6c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.110+0000 7f0a6bfff700 1 -- 172.21.15.142:0/2043114725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c0095f0 con 0x7f0a6c0ffa30 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.110+0000 7f0a6bfff700 1 --2- 172.21.15.142:0/2043114725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c0ffa30 0x7f0a6c19c450 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c006010 tx=0x7f0a5c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.111+0000 7f0a697fa700 1 -- 172.21.15.142:0/2043114725 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a5c01d070 con 0x7f0a6c0ffa30 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.111+0000 7f0a697fa700 1 -- 172.21.15.142:0/2043114725 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a5c005dc0 con 0x7f0a6c0ffa30 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.111+0000 7f0a72714700 1 -- 172.21.15.142:0/2043114725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c1967b0 con 0x7f0a6c0ffa30 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.111+0000 7f0a697fa700 1 -- 172.21.15.142:0/2043114725 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a5c017400 con 0x7f0a6c0ffa30 2024-03-29T05:02:12.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.111+0000 7f0a72714700 1 -- 172.21.15.142:0/2043114725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c196ca0 con 0x7f0a6c0ffa30 2024-03-29T05:02:12.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.112+0000 7f0a72714700 1 -- 172.21.15.142:0/2043114725 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6c109040 con 0x7f0a6c0ffa30 2024-03-29T05:02:12.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.112+0000 7f0a697fa700 1 -- 172.21.15.142:0/2043114725 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f0a5c003a60 con 0x7f0a6c0ffa30 2024-03-29T05:02:12.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.113+0000 7f0a697fa700 1 --2- 172.21.15.142:0/2043114725 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0a5406dd60 0x7f0a54070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:12.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.113+0000 7f0a697fa700 1 -- 172.21.15.142:0/2043114725 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f0a5c08f0f0 con 0x7f0a6c0ffa30 2024-03-29T05:02:12.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.113+0000 7f0a6b7fe700 1 --2- 172.21.15.142:0/2043114725 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0a5406dd60 0x7f0a54070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:12.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.113+0000 7f0a6b7fe700 1 --2- 172.21.15.142:0/2043114725 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0a5406dd60 0x7f0a54070220 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c197d20 tx=0x7f0a6000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:12.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.117+0000 7f0a697fa700 1 -- 172.21.15.142:0/2043114725 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0a5c05c040 con 0x7f0a6c0ffa30 2024-03-29T05:02:12.340 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.331+0000 7f0a72714700 1 -- 172.21.15.142:0/2043114725 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a6c196ff0 con 0x7f0a5406dd60 2024-03-29T05:02:12.341 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.332+0000 7f0a697fa700 1 -- 172.21.15.142:0/2043114725 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0a6c196ff0 con 0x7f0a5406dd60 2024-03-29T05:02:12.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.334+0000 7f0a72714700 1 -- 172.21.15.142:0/2043114725 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0a5406dd60 msgr2=0x7f0a54070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:12.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.334+0000 7f0a72714700 1 --2- 172.21.15.142:0/2043114725 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0a5406dd60 0x7f0a54070220 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c197d20 tx=0x7f0a6000b410 comp rx=0 tx=0).stop 2024-03-29T05:02:12.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.334+0000 7f0a72714700 1 -- 172.21.15.142:0/2043114725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c0ffa30 msgr2=0x7f0a6c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:12.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.334+0000 7f0a72714700 1 --2- 172.21.15.142:0/2043114725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c0ffa30 0x7f0a6c19c450 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c006010 tx=0x7f0a5c00bd70 comp rx=0 tx=0).stop 2024-03-29T05:02:12.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.334+0000 7f0a72714700 1 -- 172.21.15.142:0/2043114725 shutdown_connections 2024-03-29T05:02:12.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.334+0000 7f0a72714700 1 --2- 172.21.15.142:0/2043114725 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0a5406dd60 0x7f0a54070220 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.334+0000 7f0a72714700 1 --2- 172.21.15.142:0/2043114725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a6c0ffa30 0x7f0a6c19c450 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.334+0000 7f0a72714700 1 --2- 172.21.15.142:0/2043114725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a6c105a30 0x7f0a6c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.334+0000 7f0a72714700 1 -- 172.21.15.142:0/2043114725 >> 172.21.15.142:0/2043114725 conn(0x7f0a6c0fb540 msgr2=0x7f0a6c069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:12.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.334+0000 7f0a72714700 1 -- 172.21.15.142:0/2043114725 shutdown_connections 2024-03-29T05:02:12.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.335+0000 7f0a72714700 1 -- 172.21.15.142:0/2043114725 wait complete. 2024-03-29T05:02:12.354 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:02:12.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.440+0000 7f1b545a9700 1 -- 172.21.15.142:0/274142613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c100de0 msgr2=0x7f1b4c1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:12.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.440+0000 7f1b545a9700 1 --2- 172.21.15.142:0/274142613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c100de0 0x7f1b4c1011c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009910 tx=0x7f1b48009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:12.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.441+0000 7f1b545a9700 1 -- 172.21.15.142:0/274142613 shutdown_connections 2024-03-29T05:02:12.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.441+0000 7f1b545a9700 1 --2- 172.21.15.142:0/274142613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b4c101790 0x7f1b4c1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.441+0000 7f1b545a9700 1 --2- 172.21.15.142:0/274142613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c100de0 0x7f1b4c1011c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.441+0000 7f1b545a9700 1 -- 172.21.15.142:0/274142613 >> 172.21.15.142:0/274142613 conn(0x7f1b4c0fc670 msgr2=0x7f1b4c0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:12.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.441+0000 7f1b545a9700 1 -- 172.21.15.142:0/274142613 shutdown_connections 2024-03-29T05:02:12.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.441+0000 7f1b545a9700 1 -- 172.21.15.142:0/274142613 wait complete. 2024-03-29T05:02:12.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.441+0000 7f1b545a9700 1 Processor -- start 2024-03-29T05:02:12.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.441+0000 7f1b545a9700 1 -- start start 2024-03-29T05:02:12.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b545a9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b4c100de0 0x7f1b4c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:12.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b545a9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c101790 0x7f1b4c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:12.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b545a9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c19d070 con 0x7f1b4c101790 2024-03-29T05:02:12.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b545a9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c1965b0 con 0x7f1b4c100de0 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b51b44700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c101790 0x7f1b4c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b51b44700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c101790 0x7f1b4c19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44944/0 (socket says 172.21.15.142:44944) 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b51b44700 1 -- 172.21.15.142:0/1289315078 learned_addr learned my addr 172.21.15.142:0/1289315078 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b52345700 1 --2- 172.21.15.142:0/1289315078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b4c100de0 0x7f1b4c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b51b44700 1 -- 172.21.15.142:0/1289315078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b4c100de0 msgr2=0x7f1b4c19c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b51b44700 1 --2- 172.21.15.142:0/1289315078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b4c100de0 0x7f1b4c19c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b51b44700 1 -- 172.21.15.142:0/1289315078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b480095f0 con 0x7f1b4c101790 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b51b44700 1 --2- 172.21.15.142:0/1289315078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c101790 0x7f1b4c19ca70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00c960 tx=0x7f1b3c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b437fe700 1 -- 172.21.15.142:0/1289315078 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b3c007a70 con 0x7f1b4c101790 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b437fe700 1 -- 172.21.15.142:0/1289315078 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b3c007bd0 con 0x7f1b4c101790 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b545a9700 1 -- 172.21.15.142:0/1289315078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c196890 con 0x7f1b4c101790 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b437fe700 1 -- 172.21.15.142:0/1289315078 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b3c018630 con 0x7f1b4c101790 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.442+0000 7f1b545a9700 1 -- 172.21.15.142:0/1289315078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c196de0 con 0x7f1b4c101790 2024-03-29T05:02:12.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.443+0000 7f1b545a9700 1 -- 172.21.15.142:0/1289315078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4c109180 con 0x7f1b4c101790 2024-03-29T05:02:12.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.444+0000 7f1b437fe700 1 -- 172.21.15.142:0/1289315078 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f1b3c00f450 con 0x7f1b4c101790 2024-03-29T05:02:12.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.444+0000 7f1b437fe700 1 --2- 172.21.15.142:0/1289315078 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1b3806dd60 0x7f1b38070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:12.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.445+0000 7f1b52345700 1 --2- 172.21.15.142:0/1289315078 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1b3806dd60 0x7f1b38070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:12.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.445+0000 7f1b437fe700 1 -- 172.21.15.142:0/1289315078 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1b3c08db30 con 0x7f1b4c101790 2024-03-29T05:02:12.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.445+0000 7f1b52345700 1 --2- 172.21.15.142:0/1289315078 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1b3806dd60 0x7f1b38070220 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b48006010 tx=0x7f1b4801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:12.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.449+0000 7f1b437fe700 1 -- 172.21.15.142:0/1289315078 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1b3c05abb0 con 0x7f1b4c101790 2024-03-29T05:02:12.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.661+0000 7f1b545a9700 1 -- 172.21.15.142:0/1289315078 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b4c1970e0 con 0x7f1b3806dd60 2024-03-29T05:02:12.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.662+0000 7f1b437fe700 1 -- 172.21.15.142:0/1289315078 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1b4c1970e0 con 0x7f1b3806dd60 2024-03-29T05:02:12.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.664+0000 7f1b545a9700 1 -- 172.21.15.142:0/1289315078 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1b3806dd60 msgr2=0x7f1b38070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:12.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.664+0000 7f1b545a9700 1 --2- 172.21.15.142:0/1289315078 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1b3806dd60 0x7f1b38070220 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b48006010 tx=0x7f1b4801a040 comp rx=0 tx=0).stop 2024-03-29T05:02:12.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.664+0000 7f1b545a9700 1 -- 172.21.15.142:0/1289315078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c101790 msgr2=0x7f1b4c19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:12.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.664+0000 7f1b545a9700 1 --2- 172.21.15.142:0/1289315078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c101790 0x7f1b4c19ca70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00c960 tx=0x7f1b3c00cc70 comp rx=0 tx=0).stop 2024-03-29T05:02:12.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.664+0000 7f1b545a9700 1 -- 172.21.15.142:0/1289315078 shutdown_connections 2024-03-29T05:02:12.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.664+0000 7f1b545a9700 1 --2- 172.21.15.142:0/1289315078 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1b3806dd60 0x7f1b38070220 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.664+0000 7f1b545a9700 1 --2- 172.21.15.142:0/1289315078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b4c100de0 0x7f1b4c19c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.664+0000 7f1b545a9700 1 --2- 172.21.15.142:0/1289315078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c101790 0x7f1b4c19ca70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.664+0000 7f1b545a9700 1 -- 172.21.15.142:0/1289315078 >> 172.21.15.142:0/1289315078 conn(0x7f1b4c0fc670 msgr2=0x7f1b4c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:12.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.664+0000 7f1b545a9700 1 -- 172.21.15.142:0/1289315078 shutdown_connections 2024-03-29T05:02:12.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.665+0000 7f1b545a9700 1 -- 172.21.15.142:0/1289315078 wait complete. 2024-03-29T05:02:12.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.769+0000 7fbc478b4700 1 -- 172.21.15.142:0/2666545064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc400ff510 msgr2=0x7fbc400ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:12.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.769+0000 7fbc478b4700 1 --2- 172.21.15.142:0/2666545064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc400ff510 0x7fbc400ff8f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbc30009910 tx=0x7fbc30009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:12.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.769+0000 7fbc478b4700 1 -- 172.21.15.142:0/2666545064 shutdown_connections 2024-03-29T05:02:12.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.769+0000 7fbc478b4700 1 --2- 172.21.15.142:0/2666545064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc400ffec0 0x7fbc40105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.769+0000 7fbc478b4700 1 --2- 172.21.15.142:0/2666545064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc400ff510 0x7fbc400ff8f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.769+0000 7fbc478b4700 1 -- 172.21.15.142:0/2666545064 >> 172.21.15.142:0/2666545064 conn(0x7fbc40074d80 msgr2=0x7fbc40075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:12.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.769+0000 7fbc478b4700 1 -- 172.21.15.142:0/2666545064 shutdown_connections 2024-03-29T05:02:12.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.769+0000 7fbc478b4700 1 -- 172.21.15.142:0/2666545064 wait complete. 2024-03-29T05:02:12.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc478b4700 1 Processor -- start 2024-03-29T05:02:12.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc478b4700 1 -- start start 2024-03-29T05:02:12.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc478b4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc400ff510 0x7fbc4019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:12.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc478b4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc400ffec0 0x7fbc4019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:12.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc478b4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4019f2e0 con 0x7fbc400ff510 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc478b4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc40198790 con 0x7fbc400ffec0 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc45650700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc400ff510 0x7fbc4019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc45650700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc400ff510 0x7fbc4019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44968/0 (socket says 172.21.15.142:44968) 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc45650700 1 -- 172.21.15.142:0/1064910766 learned_addr learned my addr 172.21.15.142:0/1064910766 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc44e4f700 1 --2- 172.21.15.142:0/1064910766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc400ffec0 0x7fbc4019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc45650700 1 -- 172.21.15.142:0/1064910766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc400ffec0 msgr2=0x7fbc4019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc45650700 1 --2- 172.21.15.142:0/1064910766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc400ffec0 0x7fbc4019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.770+0000 7fbc45650700 1 -- 172.21.15.142:0/1064910766 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc300095f0 con 0x7fbc400ff510 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.771+0000 7fbc45650700 1 --2- 172.21.15.142:0/1064910766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc400ff510 0x7fbc4019e710 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbc30006010 tx=0x7fbc3000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.771+0000 7fbc367fc700 1 -- 172.21.15.142:0/1064910766 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc3001d070 con 0x7fbc400ff510 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.771+0000 7fbc367fc700 1 -- 172.21.15.142:0/1064910766 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc30005dc0 con 0x7fbc400ff510 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.771+0000 7fbc478b4700 1 -- 172.21.15.142:0/1064910766 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc40198a10 con 0x7fbc400ff510 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.771+0000 7fbc367fc700 1 -- 172.21.15.142:0/1064910766 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc30017400 con 0x7fbc400ff510 2024-03-29T05:02:12.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.771+0000 7fbc478b4700 1 -- 172.21.15.142:0/1064910766 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc40198f00 con 0x7fbc400ff510 2024-03-29T05:02:12.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.772+0000 7fbc478b4700 1 -- 172.21.15.142:0/1064910766 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc40104fd0 con 0x7fbc400ff510 2024-03-29T05:02:12.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.773+0000 7fbc367fc700 1 -- 172.21.15.142:0/1064910766 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fbc30003a60 con 0x7fbc400ff510 2024-03-29T05:02:12.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.773+0000 7fbc367fc700 1 --2- 172.21.15.142:0/1064910766 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbc2c06dd60 0x7fbc2c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:12.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.773+0000 7fbc367fc700 1 -- 172.21.15.142:0/1064910766 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fbc3008f130 con 0x7fbc400ff510 2024-03-29T05:02:12.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.773+0000 7fbc44e4f700 1 --2- 172.21.15.142:0/1064910766 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbc2c06dd60 0x7fbc2c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:12.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.774+0000 7fbc44e4f700 1 --2- 172.21.15.142:0/1064910766 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbc2c06dd60 0x7fbc2c070220 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc40199c00 tx=0x7fbc3c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:12.786 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.778+0000 7fbc367fc700 1 -- 172.21.15.142:0/1064910766 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbc3005c080 con 0x7fbc400ff510 2024-03-29T05:02:12.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.985+0000 7fbc478b4700 1 -- 172.21.15.142:0/1064910766 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc4002cce0 con 0x7fbc2c06dd60 2024-03-29T05:02:13.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.995+0000 7fbc367fc700 1 -- 172.21.15.142:0/1064910766 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbc4002cce0 con 0x7fbc2c06dd60 2024-03-29T05:02:13.004 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:02:13.004 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (14m) 7m ago 25m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (25m) 7m ago 25m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (24m) 6m ago 24m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (11m) 7m ago 25m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (11m) 6m ago 24m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (13m) 7m ago 25m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (7m) 7m ago 19m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (7m) 7m ago 19m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (6m) 6m ago 19m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (6m) 6m ago 19m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (15m) 7m ago 26m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (14m) 6m ago 24m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (12m) 7m ago 27m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (11m) 6m ago 24m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:02:13.005 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (14m) 7m ago 25m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:02:13.006 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (14m) 6m ago 24m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:02:13.006 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (10m) 7m ago 23m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:02:13.006 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (10m) 7m ago 23m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:02:13.006 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (9m) 7m ago 22m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:02:13.006 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (8m) 6m ago 22m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:02:13.006 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (8m) 6m ago 21m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:02:13.006 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (7m) 6m ago 21m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:02:13.006 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (14m) 7m ago 25m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:02:13.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.998+0000 7fbc478b4700 1 -- 172.21.15.142:0/1064910766 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbc2c06dd60 msgr2=0x7fbc2c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:13.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.998+0000 7fbc478b4700 1 --2- 172.21.15.142:0/1064910766 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbc2c06dd60 0x7fbc2c070220 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc40199c00 tx=0x7fbc3c00b410 comp rx=0 tx=0).stop 2024-03-29T05:02:13.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.998+0000 7fbc478b4700 1 -- 172.21.15.142:0/1064910766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc400ff510 msgr2=0x7fbc4019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:13.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.998+0000 7fbc478b4700 1 --2- 172.21.15.142:0/1064910766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc400ff510 0x7fbc4019e710 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbc30006010 tx=0x7fbc3000bd70 comp rx=0 tx=0).stop 2024-03-29T05:02:13.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.998+0000 7fbc478b4700 1 -- 172.21.15.142:0/1064910766 shutdown_connections 2024-03-29T05:02:13.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.998+0000 7fbc478b4700 1 --2- 172.21.15.142:0/1064910766 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbc2c06dd60 0x7fbc2c070220 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.998+0000 7fbc478b4700 1 --2- 172.21.15.142:0/1064910766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc400ff510 0x7fbc4019e710 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.998+0000 7fbc478b4700 1 --2- 172.21.15.142:0/1064910766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc400ffec0 0x7fbc4019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.998+0000 7fbc478b4700 1 -- 172.21.15.142:0/1064910766 >> 172.21.15.142:0/1064910766 conn(0x7fbc40074d80 msgr2=0x7fbc400feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:13.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.998+0000 7fbc478b4700 1 -- 172.21.15.142:0/1064910766 shutdown_connections 2024-03-29T05:02:13.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:12.998+0000 7fbc478b4700 1 -- 172.21.15.142:0/1064910766 wait complete. 2024-03-29T05:02:13.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.100+0000 7f7758ebf700 1 -- 172.21.15.142:0/924021079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7754102660 msgr2=0x7f775410ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:13.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.100+0000 7f7758ebf700 1 --2- 172.21.15.142:0/924021079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7754102660 0x7f775410ab50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7748009910 tx=0x7f7748009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:13.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.100+0000 7f7758ebf700 1 -- 172.21.15.142:0/924021079 shutdown_connections 2024-03-29T05:02:13.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.100+0000 7f7758ebf700 1 --2- 172.21.15.142:0/924021079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7754102660 0x7f775410ab50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.100+0000 7f7758ebf700 1 --2- 172.21.15.142:0/924021079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7754101cb0 0x7f7754102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.100+0000 7f7758ebf700 1 -- 172.21.15.142:0/924021079 >> 172.21.15.142:0/924021079 conn(0x7f77540fb590 msgr2=0x7f77540fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:13.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.100+0000 7f7758ebf700 1 -- 172.21.15.142:0/924021079 shutdown_connections 2024-03-29T05:02:13.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.100+0000 7f7758ebf700 1 -- 172.21.15.142:0/924021079 wait complete. 2024-03-29T05:02:13.111 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f7758ebf700 1 Processor -- start 2024-03-29T05:02:13.111 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f7758ebf700 1 -- start start 2024-03-29T05:02:13.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f7758ebf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7754101cb0 0x7f7754196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:13.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f7758ebf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7754102660 0x7f7754196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:13.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f7758ebf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77541972d0 con 0x7f7754102660 2024-03-29T05:02:13.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f7758ebf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775419a000 con 0x7f7754101cb0 2024-03-29T05:02:13.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f7751d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7754102660 0x7f7754196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f7751d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7754102660 0x7f7754196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44980/0 (socket says 172.21.15.142:44980) 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f775259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7754101cb0 0x7f7754196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f7751d9b700 1 -- 172.21.15.142:0/1082132692 learned_addr learned my addr 172.21.15.142:0/1082132692 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f7751d9b700 1 -- 172.21.15.142:0/1082132692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7754101cb0 msgr2=0x7f7754196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f7751d9b700 1 --2- 172.21.15.142:0/1082132692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7754101cb0 0x7f7754196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.101+0000 7f7751d9b700 1 -- 172.21.15.142:0/1082132692 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77480095f0 con 0x7f7754102660 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.102+0000 7f7751d9b700 1 --2- 172.21.15.142:0/1082132692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7754102660 0x7f7754196c40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f77480092e0 tx=0x7f774800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.102+0000 7f77437fe700 1 -- 172.21.15.142:0/1082132692 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f774801d070 con 0x7f7754102660 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.102+0000 7f77437fe700 1 -- 172.21.15.142:0/1082132692 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f774800f460 con 0x7f7754102660 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.102+0000 7f77437fe700 1 -- 172.21.15.142:0/1082132692 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77480175d0 con 0x7f7754102660 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.102+0000 7f7758ebf700 1 -- 172.21.15.142:0/1082132692 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775419a280 con 0x7f7754102660 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.102+0000 7f7758ebf700 1 -- 172.21.15.142:0/1082132692 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775419a770 con 0x7f7754102660 2024-03-29T05:02:13.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.103+0000 7f7758ebf700 1 -- 172.21.15.142:0/1082132692 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7754108250 con 0x7f7754102660 2024-03-29T05:02:13.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.104+0000 7f77437fe700 1 -- 172.21.15.142:0/1082132692 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f774800f5d0 con 0x7f7754102660 2024-03-29T05:02:13.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.105+0000 7f77437fe700 1 --2- 172.21.15.142:0/1082132692 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f773c06dd60 0x7f773c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:13.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.105+0000 7f77437fe700 1 -- 172.21.15.142:0/1082132692 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f774808f060 con 0x7f7754102660 2024-03-29T05:02:13.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.105+0000 7f775259c700 1 --2- 172.21.15.142:0/1082132692 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f773c06dd60 0x7f773c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:13.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.105+0000 7f775259c700 1 --2- 172.21.15.142:0/1082132692 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f773c06dd60 0x7f773c070220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7744005950 tx=0x7f774400a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:13.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.109+0000 7f77437fe700 1 -- 172.21.15.142:0/1082132692 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7748058500 con 0x7f7754102660 2024-03-29T05:02:13.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:13 smithi142 ceph-mon[112475]: pgmap v417: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:13.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:13 smithi142 ceph-mon[112475]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:13.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:13 smithi142 ceph-mon[112475]: from='client.34798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:13.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.387+0000 7f7758ebf700 1 -- 172.21.15.142:0/1082132692 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f775404ea90 con 0x7f7754102660 2024-03-29T05:02:13.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.387+0000 7f77437fe700 1 -- 172.21.15.142:0/1082132692 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f774805bbc0 con 0x7f7754102660 2024-03-29T05:02:13.396 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:02:13.396 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:02:13.397 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:02:13.398 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:02:13.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.389+0000 7f7758ebf700 1 -- 172.21.15.142:0/1082132692 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f773c06dd60 msgr2=0x7f773c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:13.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.389+0000 7f7758ebf700 1 --2- 172.21.15.142:0/1082132692 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f773c06dd60 0x7f773c070220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7744005950 tx=0x7f774400a300 comp rx=0 tx=0).stop 2024-03-29T05:02:13.398 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.389+0000 7f7758ebf700 1 -- 172.21.15.142:0/1082132692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7754102660 msgr2=0x7f7754196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:13.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.389+0000 7f7758ebf700 1 --2- 172.21.15.142:0/1082132692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7754102660 0x7f7754196c40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f77480092e0 tx=0x7f774800bd70 comp rx=0 tx=0).stop 2024-03-29T05:02:13.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.389+0000 7f7758ebf700 1 -- 172.21.15.142:0/1082132692 shutdown_connections 2024-03-29T05:02:13.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.389+0000 7f7758ebf700 1 --2- 172.21.15.142:0/1082132692 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f773c06dd60 0x7f773c070220 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.389+0000 7f7758ebf700 1 --2- 172.21.15.142:0/1082132692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7754101cb0 0x7f7754196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.389+0000 7f7758ebf700 1 --2- 172.21.15.142:0/1082132692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7754102660 0x7f7754196c40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.389+0000 7f7758ebf700 1 -- 172.21.15.142:0/1082132692 >> 172.21.15.142:0/1082132692 conn(0x7f77540fb590 msgr2=0x7f7754105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:13.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.390+0000 7f7758ebf700 1 -- 172.21.15.142:0/1082132692 shutdown_connections 2024-03-29T05:02:13.399 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.390+0000 7f7758ebf700 1 -- 172.21.15.142:0/1082132692 wait complete. 2024-03-29T05:02:13.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.493+0000 7fcb210e4700 1 -- 172.21.15.142:0/2396644784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb1c101770 msgr2=0x7fcb1c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:13.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.493+0000 7fcb210e4700 1 --2- 172.21.15.142:0/2396644784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb1c101770 0x7fcb1c1057c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009910 tx=0x7fcb0c009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:13.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.493+0000 7fcb210e4700 1 -- 172.21.15.142:0/2396644784 shutdown_connections 2024-03-29T05:02:13.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.493+0000 7fcb210e4700 1 --2- 172.21.15.142:0/2396644784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb1c101770 0x7fcb1c1057c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.493+0000 7fcb210e4700 1 --2- 172.21.15.142:0/2396644784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb1c100dc0 0x7fcb1c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.493+0000 7fcb210e4700 1 -- 172.21.15.142:0/2396644784 >> 172.21.15.142:0/2396644784 conn(0x7fcb1c0fc650 msgr2=0x7fcb1c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:13.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.493+0000 7fcb210e4700 1 -- 172.21.15.142:0/2396644784 shutdown_connections 2024-03-29T05:02:13.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.493+0000 7fcb210e4700 1 -- 172.21.15.142:0/2396644784 wait complete. 2024-03-29T05:02:13.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.494+0000 7fcb210e4700 1 Processor -- start 2024-03-29T05:02:13.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.494+0000 7fcb210e4700 1 -- start start 2024-03-29T05:02:13.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.494+0000 7fcb210e4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb1c100dc0 0x7fcb1c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:13.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.494+0000 7fcb210e4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb1c101770 0x7fcb1c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:13.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.494+0000 7fcb210e4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c073b10 con 0x7fcb1c100dc0 2024-03-29T05:02:13.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.494+0000 7fcb210e4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c073c50 con 0x7fcb1c101770 2024-03-29T05:02:13.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.494+0000 7fcb1ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb1c100dc0 0x7fcb1c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.494+0000 7fcb1ad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb1c100dc0 0x7fcb1c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44992/0 (socket says 172.21.15.142:44992) 2024-03-29T05:02:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.494+0000 7fcb1ad9d700 1 -- 172.21.15.142:0/4106264823 learned_addr learned my addr 172.21.15.142:0/4106264823 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.494+0000 7fcb1a59c700 1 --2- 172.21.15.142:0/4106264823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb1c101770 0x7fcb1c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.494+0000 7fcb1ad9d700 1 -- 172.21.15.142:0/4106264823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb1c101770 msgr2=0x7fcb1c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.495+0000 7fcb1ad9d700 1 --2- 172.21.15.142:0/4106264823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb1c101770 0x7fcb1c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.495+0000 7fcb1ad9d700 1 -- 172.21.15.142:0/4106264823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c0095f0 con 0x7fcb1c100dc0 2024-03-29T05:02:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.495+0000 7fcb1ad9d700 1 --2- 172.21.15.142:0/4106264823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb1c100dc0 0x7fcb1c073090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000d8d0 tx=0x7fcb1000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.495+0000 7fcb03fff700 1 -- 172.21.15.142:0/4106264823 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb100098a0 con 0x7fcb1c100dc0 2024-03-29T05:02:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.495+0000 7fcb03fff700 1 -- 172.21.15.142:0/4106264823 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb10010460 con 0x7fcb1c100dc0 2024-03-29T05:02:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.495+0000 7fcb210e4700 1 -- 172.21.15.142:0/4106264823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1c1a2960 con 0x7fcb1c100dc0 2024-03-29T05:02:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.495+0000 7fcb03fff700 1 -- 172.21.15.142:0/4106264823 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb1000f5f0 con 0x7fcb1c100dc0 2024-03-29T05:02:13.505 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.495+0000 7fcb210e4700 1 -- 172.21.15.142:0/4106264823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c1a2d70 con 0x7fcb1c100dc0 2024-03-29T05:02:13.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.496+0000 7fcb210e4700 1 -- 172.21.15.142:0/4106264823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1c1091c0 con 0x7fcb1c100dc0 2024-03-29T05:02:13.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.496+0000 7fcb03fff700 1 -- 172.21.15.142:0/4106264823 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fcb10009a00 con 0x7fcb1c100dc0 2024-03-29T05:02:13.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.497+0000 7fcb03fff700 1 --2- 172.21.15.142:0/4106264823 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcb0406dd60 0x7fcb04070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:13.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.497+0000 7fcb03fff700 1 -- 172.21.15.142:0/4106264823 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fcb1008db40 con 0x7fcb1c100dc0 2024-03-29T05:02:13.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.497+0000 7fcb1a59c700 1 --2- 172.21.15.142:0/4106264823 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcb0406dd60 0x7fcb04070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:13.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.497+0000 7fcb1a59c700 1 --2- 172.21.15.142:0/4106264823 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcb0406dd60 0x7fcb04070220 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c074880 tx=0x7fcb0c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:13.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.502+0000 7fcb03fff700 1 -- 172.21.15.142:0/4106264823 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcb10056f60 con 0x7fcb1c100dc0 2024-03-29T05:02:13.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:13 smithi152 ceph-mon[88186]: pgmap v417: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:13.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:13 smithi152 ceph-mon[88186]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:13.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:13 smithi152 ceph-mon[88186]: from='client.34798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:13.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.741+0000 7fcb210e4700 1 -- 172.21.15.142:0/4106264823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcb1c04ea90 con 0x7fcb1c100dc0 2024-03-29T05:02:13.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.741+0000 7fcb03fff700 1 -- 172.21.15.142:0/4106264823 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fcb1005a620 con 0x7fcb1c100dc0 2024-03-29T05:02:13.751 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:02:13.751 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:02:13.751 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:02:13.751 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:02:13.751 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:02:13.751 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:02:13.751 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:02:13.751 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:02:13.751 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:02:13.751 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:02:13.751 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:02:13.752 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:02:13.753 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:02:13.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.743+0000 7fcb210e4700 1 -- 172.21.15.142:0/4106264823 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcb0406dd60 msgr2=0x7fcb04070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:13.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.743+0000 7fcb210e4700 1 --2- 172.21.15.142:0/4106264823 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcb0406dd60 0x7fcb04070220 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c074880 tx=0x7fcb0c01a040 comp rx=0 tx=0).stop 2024-03-29T05:02:13.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.744+0000 7fcb210e4700 1 -- 172.21.15.142:0/4106264823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb1c100dc0 msgr2=0x7fcb1c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:13.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.744+0000 7fcb210e4700 1 --2- 172.21.15.142:0/4106264823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb1c100dc0 0x7fcb1c073090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000d8d0 tx=0x7fcb1000dbe0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.744+0000 7fcb210e4700 1 -- 172.21.15.142:0/4106264823 shutdown_connections 2024-03-29T05:02:13.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.744+0000 7fcb210e4700 1 --2- 172.21.15.142:0/4106264823 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcb0406dd60 0x7fcb04070220 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.744+0000 7fcb210e4700 1 --2- 172.21.15.142:0/4106264823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb1c100dc0 0x7fcb1c073090 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.744+0000 7fcb210e4700 1 --2- 172.21.15.142:0/4106264823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb1c101770 0x7fcb1c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.744+0000 7fcb210e4700 1 -- 172.21.15.142:0/4106264823 >> 172.21.15.142:0/4106264823 conn(0x7fcb1c0fc650 msgr2=0x7fcb1c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:13.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.744+0000 7fcb210e4700 1 -- 172.21.15.142:0/4106264823 shutdown_connections 2024-03-29T05:02:13.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.744+0000 7fcb210e4700 1 -- 172.21.15.142:0/4106264823 wait complete. 2024-03-29T05:02:13.755 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:02:13.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.849+0000 7fc9c233b700 1 -- 172.21.15.142:0/1859346151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9bc101780 msgr2=0x7fc9bc1057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:13.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.849+0000 7fc9c233b700 1 --2- 172.21.15.142:0/1859346151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9bc101780 0x7fc9bc1057d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009910 tx=0x7fc9b0009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:13.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.850+0000 7fc9c233b700 1 -- 172.21.15.142:0/1859346151 shutdown_connections 2024-03-29T05:02:13.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.850+0000 7fc9c233b700 1 --2- 172.21.15.142:0/1859346151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9bc101780 0x7fc9bc1057d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.850+0000 7fc9c233b700 1 --2- 172.21.15.142:0/1859346151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9bc100dd0 0x7fc9bc1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.850+0000 7fc9c233b700 1 -- 172.21.15.142:0/1859346151 >> 172.21.15.142:0/1859346151 conn(0x7fc9bc0fc640 msgr2=0x7fc9bc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:13.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.850+0000 7fc9c233b700 1 -- 172.21.15.142:0/1859346151 shutdown_connections 2024-03-29T05:02:13.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.850+0000 7fc9c233b700 1 -- 172.21.15.142:0/1859346151 wait complete. 2024-03-29T05:02:13.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.850+0000 7fc9c233b700 1 Processor -- start 2024-03-29T05:02:13.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.850+0000 7fc9c233b700 1 -- start start 2024-03-29T05:02:13.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9c233b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9bc100dd0 0x7fc9bc19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:13.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9c233b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9bc101780 0x7fc9bc19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:13.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9c233b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc19d140 con 0x7fc9bc101780 2024-03-29T05:02:13.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9c233b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc1965f0 con 0x7fc9bc100dd0 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9bb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9bc101780 0x7fc9bc19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9bb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9bc101780 0x7fc9bc19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45008/0 (socket says 172.21.15.142:45008) 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9bb7fe700 1 -- 172.21.15.142:0/401131389 learned_addr learned my addr 172.21.15.142:0/401131389 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9bbfff700 1 --2- 172.21.15.142:0/401131389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9bc100dd0 0x7fc9bc19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9bb7fe700 1 -- 172.21.15.142:0/401131389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9bc100dd0 msgr2=0x7fc9bc19c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9bb7fe700 1 --2- 172.21.15.142:0/401131389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9bc100dd0 0x7fc9bc19c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9bb7fe700 1 -- 172.21.15.142:0/401131389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b00095f0 con 0x7fc9bc101780 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9bb7fe700 1 --2- 172.21.15.142:0/401131389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9bc101780 0x7fc9bc19cab0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00092e0 tx=0x7fc9b000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.851+0000 7fc9b97fa700 1 -- 172.21.15.142:0/401131389 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9b001d070 con 0x7fc9bc101780 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.852+0000 7fc9c233b700 1 -- 172.21.15.142:0/401131389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9bc196870 con 0x7fc9bc101780 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.852+0000 7fc9b97fa700 1 -- 172.21.15.142:0/401131389 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9b0005dc0 con 0x7fc9bc101780 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.852+0000 7fc9b97fa700 1 -- 172.21.15.142:0/401131389 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9b0017400 con 0x7fc9bc101780 2024-03-29T05:02:13.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.852+0000 7fc9c233b700 1 -- 172.21.15.142:0/401131389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9bc196d60 con 0x7fc9bc101780 2024-03-29T05:02:13.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.853+0000 7fc9c233b700 1 -- 172.21.15.142:0/401131389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9bc1091d0 con 0x7fc9bc101780 2024-03-29T05:02:13.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.853+0000 7fc9b97fa700 1 -- 172.21.15.142:0/401131389 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fc9b0017560 con 0x7fc9bc101780 2024-03-29T05:02:13.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.853+0000 7fc9b97fa700 1 --2- 172.21.15.142:0/401131389 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc9a406dd60 0x7fc9a4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:13.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.853+0000 7fc9b97fa700 1 -- 172.21.15.142:0/401131389 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fc9b008f050 con 0x7fc9bc101780 2024-03-29T05:02:13.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.853+0000 7fc9bbfff700 1 --2- 172.21.15.142:0/401131389 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc9a406dd60 0x7fc9a4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:13.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.854+0000 7fc9bbfff700 1 --2- 172.21.15.142:0/401131389 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc9a406dd60 0x7fc9a4070220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac005950 tx=0x7fc9ac00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:13.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:13.858+0000 7fc9b97fa700 1 -- 172.21.15.142:0/401131389 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc9b005c010 con 0x7fc9bc101780 2024-03-29T05:02:14.079 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.070+0000 7fc9c233b700 1 -- 172.21.15.142:0/401131389 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9bc02ccf0 con 0x7fc9a406dd60 2024-03-29T05:02:14.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.072+0000 7fc9b97fa700 1 -- 172.21.15.142:0/401131389 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc9bc02ccf0 con 0x7fc9a406dd60 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:02:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:02:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:02:14.082 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:02:14.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.074+0000 7fc9c233b700 1 -- 172.21.15.142:0/401131389 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc9a406dd60 msgr2=0x7fc9a4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:14.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.074+0000 7fc9c233b700 1 --2- 172.21.15.142:0/401131389 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc9a406dd60 0x7fc9a4070220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac005950 tx=0x7fc9ac00b410 comp rx=0 tx=0).stop 2024-03-29T05:02:14.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.074+0000 7fc9c233b700 1 -- 172.21.15.142:0/401131389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9bc101780 msgr2=0x7fc9bc19cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:14.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.074+0000 7fc9c233b700 1 --2- 172.21.15.142:0/401131389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9bc101780 0x7fc9bc19cab0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00092e0 tx=0x7fc9b000bd70 comp rx=0 tx=0).stop 2024-03-29T05:02:14.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.074+0000 7fc9c233b700 1 -- 172.21.15.142:0/401131389 shutdown_connections 2024-03-29T05:02:14.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.074+0000 7fc9c233b700 1 --2- 172.21.15.142:0/401131389 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc9a406dd60 0x7fc9a4070220 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:14.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.074+0000 7fc9c233b700 1 --2- 172.21.15.142:0/401131389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9bc100dd0 0x7fc9bc19c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:14.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.074+0000 7fc9c233b700 1 --2- 172.21.15.142:0/401131389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9bc101780 0x7fc9bc19cab0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:14.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.074+0000 7fc9c233b700 1 -- 172.21.15.142:0/401131389 >> 172.21.15.142:0/401131389 conn(0x7fc9bc0fc640 msgr2=0x7fc9bc0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:14.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.074+0000 7fc9c233b700 1 -- 172.21.15.142:0/401131389 shutdown_connections 2024-03-29T05:02:14.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.074+0000 7fc9c233b700 1 -- 172.21.15.142:0/401131389 wait complete. 2024-03-29T05:02:14.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.178+0000 7ff44fc37700 1 -- 172.21.15.142:0/512933341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff448100dc0 msgr2=0x7ff4481011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:14.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.178+0000 7ff44fc37700 1 --2- 172.21.15.142:0/512933341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff448100dc0 0x7ff4481011a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b0a0 tx=0x7ff44000b3b0 comp rx=0 tx=0).stop 2024-03-29T05:02:14.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.178+0000 7ff44fc37700 1 -- 172.21.15.142:0/512933341 shutdown_connections 2024-03-29T05:02:14.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.178+0000 7ff44fc37700 1 --2- 172.21.15.142:0/512933341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff448101770 0x7ff4481057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:14.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.178+0000 7ff44fc37700 1 --2- 172.21.15.142:0/512933341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff448100dc0 0x7ff4481011a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:14.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.178+0000 7ff44fc37700 1 -- 172.21.15.142:0/512933341 >> 172.21.15.142:0/512933341 conn(0x7ff4480fc650 msgr2=0x7ff4480fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:14.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.178+0000 7ff44fc37700 1 -- 172.21.15.142:0/512933341 shutdown_connections 2024-03-29T05:02:14.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.178+0000 7ff44fc37700 1 -- 172.21.15.142:0/512933341 wait complete. 2024-03-29T05:02:14.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44fc37700 1 Processor -- start 2024-03-29T05:02:14.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44fc37700 1 -- start start 2024-03-29T05:02:14.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44fc37700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff448100dc0 0x7ff44819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44fc37700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff448101770 0x7ff44819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44fc37700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44819d100 con 0x7ff448100dc0 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44fc37700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4481965b0 con 0x7ff448101770 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44d9d3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff448100dc0 0x7ff44819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44d9d3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff448100dc0 0x7ff44819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45016/0 (socket says 172.21.15.142:45016) 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44d9d3700 1 -- 172.21.15.142:0/3722645261 learned_addr learned my addr 172.21.15.142:0/3722645261 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44d1d2700 1 --2- 172.21.15.142:0/3722645261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff448101770 0x7ff44819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44d9d3700 1 -- 172.21.15.142:0/3722645261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff448101770 msgr2=0x7ff44819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44d9d3700 1 --2- 172.21.15.142:0/3722645261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff448101770 0x7ff44819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44d9d3700 1 -- 172.21.15.142:0/3722645261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff440009d00 con 0x7ff448100dc0 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.179+0000 7ff44d9d3700 1 --2- 172.21.15.142:0/3722645261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff448100dc0 0x7ff44819c530 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff440000f80 tx=0x7ff44000bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.180+0000 7ff43effd700 1 -- 172.21.15.142:0/3722645261 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff44000e040 con 0x7ff448100dc0 2024-03-29T05:02:14.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.180+0000 7ff43effd700 1 -- 172.21.15.142:0/3722645261 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff440003c10 con 0x7ff448100dc0 2024-03-29T05:02:14.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.180+0000 7ff44fc37700 1 -- 172.21.15.142:0/3722645261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff448196830 con 0x7ff448100dc0 2024-03-29T05:02:14.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.180+0000 7ff43effd700 1 -- 172.21.15.142:0/3722645261 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff44001d730 con 0x7ff448100dc0 2024-03-29T05:02:14.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.180+0000 7ff44fc37700 1 -- 172.21.15.142:0/3722645261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff448196d20 con 0x7ff448100dc0 2024-03-29T05:02:14.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.181+0000 7ff44fc37700 1 -- 172.21.15.142:0/3722645261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff448109160 con 0x7ff448100dc0 2024-03-29T05:02:14.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.181+0000 7ff43effd700 1 -- 172.21.15.142:0/3722645261 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff440019070 con 0x7ff448100dc0 2024-03-29T05:02:14.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.182+0000 7ff43effd700 1 --2- 172.21.15.142:0/3722645261 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff43406dd60 0x7ff434070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:14.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.182+0000 7ff43effd700 1 -- 172.21.15.142:0/3722645261 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff44008f240 con 0x7ff448100dc0 2024-03-29T05:02:14.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.182+0000 7ff44d1d2700 1 --2- 172.21.15.142:0/3722645261 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff43406dd60 0x7ff434070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:14.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.182+0000 7ff44d1d2700 1 --2- 172.21.15.142:0/3722645261 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff43406dd60 0x7ff434070220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff4481059d0 tx=0x7ff438009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:14.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.186+0000 7ff43effd700 1 -- 172.21.15.142:0/3722645261 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff44005c190 con 0x7ff448100dc0 2024-03-29T05:02:14.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:14 smithi142 ceph-mon[112475]: from='client.34802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:14.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:14 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1082132692' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:02:14.380 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:14 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/4106264823' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:02:14.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.465+0000 7ff44fc37700 1 -- 172.21.15.142:0/3722645261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff448197000 con 0x7ff448100dc0 2024-03-29T05:02:14.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.465+0000 7ff43effd700 1 -- 172.21.15.142:0/3722645261 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff44005bd20 con 0x7ff448100dc0 2024-03-29T05:02:14.474 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:02:14.474 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:02:14.474 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:02:14.474 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:02:14.474 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:02:14.476 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.467+0000 7ff44fc37700 1 -- 172.21.15.142:0/3722645261 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff43406dd60 msgr2=0x7ff434070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:14.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.467+0000 7ff44fc37700 1 --2- 172.21.15.142:0/3722645261 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff43406dd60 0x7ff434070220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff4481059d0 tx=0x7ff438009260 comp rx=0 tx=0).stop 2024-03-29T05:02:14.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.467+0000 7ff44fc37700 1 -- 172.21.15.142:0/3722645261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff448100dc0 msgr2=0x7ff44819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:14.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.467+0000 7ff44fc37700 1 --2- 172.21.15.142:0/3722645261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff448100dc0 0x7ff44819c530 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff440000f80 tx=0x7ff44000bd10 comp rx=0 tx=0).stop 2024-03-29T05:02:14.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.467+0000 7ff44fc37700 1 -- 172.21.15.142:0/3722645261 shutdown_connections 2024-03-29T05:02:14.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.467+0000 7ff44fc37700 1 --2- 172.21.15.142:0/3722645261 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff43406dd60 0x7ff434070220 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:14.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.467+0000 7ff44fc37700 1 --2- 172.21.15.142:0/3722645261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff448100dc0 0x7ff44819c530 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:14.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.467+0000 7ff44fc37700 1 --2- 172.21.15.142:0/3722645261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff448101770 0x7ff44819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:14.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.467+0000 7ff44fc37700 1 -- 172.21.15.142:0/3722645261 >> 172.21.15.142:0/3722645261 conn(0x7ff4480fc650 msgr2=0x7ff4480fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:14.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.467+0000 7ff44fc37700 1 -- 172.21.15.142:0/3722645261 shutdown_connections 2024-03-29T05:02:14.477 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:14.467+0000 7ff44fc37700 1 -- 172.21.15.142:0/3722645261 wait complete. 2024-03-29T05:02:14.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:14 smithi152 ceph-mon[88186]: from='client.34802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:14.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:14 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1082132692' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:02:14.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:14 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/4106264823' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:02:15.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:15 smithi152 ceph-mon[88186]: pgmap v418: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:15.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:15 smithi152 ceph-mon[88186]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:15.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:15 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3722645261' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:02:15.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:02:15.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:15 smithi142 ceph-mon[112475]: pgmap v418: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:15.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:15 smithi142 ceph-mon[112475]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:15.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:15 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3722645261' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:02:15.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:02:17.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:17 smithi152 ceph-mon[88186]: pgmap v419: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:17.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:17 smithi142 ceph-mon[112475]: pgmap v419: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:19.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:19 smithi142 ceph-mon[112475]: pgmap v420: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:19.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:19 smithi152 ceph-mon[88186]: pgmap v420: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:21.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:21 smithi142 ceph-mon[112475]: pgmap v421: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:21.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:21 smithi152 ceph-mon[88186]: pgmap v421: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:23.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:23 smithi142 ceph-mon[112475]: pgmap v422: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:23.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:23 smithi152 ceph-mon[88186]: pgmap v422: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:25.601 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:25 smithi152 ceph-mon[88186]: pgmap v423: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:25.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:25 smithi142 ceph-mon[112475]: pgmap v423: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:27.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:27 smithi142 ceph-mon[112475]: pgmap v424: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:27.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:27 smithi152 ceph-mon[88186]: pgmap v424: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:29.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:29 smithi142 ceph-mon[112475]: pgmap v425: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:29.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:29 smithi152 ceph-mon[88186]: pgmap v425: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:30.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:02:30.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:02:31.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:31 smithi142 ceph-mon[112475]: pgmap v426: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:31.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:31 smithi152 ceph-mon[88186]: pgmap v426: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:33.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:33 smithi142 ceph-mon[112475]: pgmap v427: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:33.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:33 smithi152 ceph-mon[88186]: pgmap v427: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:35.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:35 smithi142 ceph-mon[112475]: pgmap v428: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:35.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:35 smithi152 ceph-mon[88186]: pgmap v428: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:37.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:37 smithi142 ceph-mon[112475]: pgmap v429: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:37.644 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:37 smithi152 ceph-mon[88186]: pgmap v429: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:39.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:39 smithi142 ceph-mon[112475]: pgmap v430: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:39.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:39 smithi152 ceph-mon[88186]: pgmap v430: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:41.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:41 smithi142 ceph-mon[112475]: pgmap v431: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:41.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:41 smithi152 ceph-mon[88186]: pgmap v431: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:43.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:43 smithi142 ceph-mon[112475]: pgmap v432: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:43.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:43 smithi152 ceph-mon[88186]: pgmap v432: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:44.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.584+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/2844798259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c105a60 msgr2=0x7f2a9c105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:44.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.584+0000 7f2aa0e7a700 1 --2- 172.21.15.142:0/2844798259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c105a60 0x7f2a9c105e40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009910 tx=0x7f2a8c009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:44.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.584+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/2844798259 shutdown_connections 2024-03-29T05:02:44.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.584+0000 7f2aa0e7a700 1 --2- 172.21.15.142:0/2844798259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a9c0ffa60 0x7f2a9c0ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:44.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.584+0000 7f2aa0e7a700 1 --2- 172.21.15.142:0/2844798259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c105a60 0x7f2a9c105e40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:44.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.584+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/2844798259 >> 172.21.15.142:0/2844798259 conn(0x7f2a9c0fb570 msgr2=0x7f2a9c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:44.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.584+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/2844798259 shutdown_connections 2024-03-29T05:02:44.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.584+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/2844798259 wait complete. 2024-03-29T05:02:44.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2aa0e7a700 1 Processor -- start 2024-03-29T05:02:44.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2aa0e7a700 1 -- start start 2024-03-29T05:02:44.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2aa0e7a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a9c0ffa60 0x7f2a9c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:44.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2aa0e7a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c105a60 0x7f2a9c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:44.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2aa0e7a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c1981d0 con 0x7f2a9c105a60 2024-03-29T05:02:44.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2aa0e7a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c198340 con 0x7f2a9c0ffa60 2024-03-29T05:02:44.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2a99d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c105a60 0x7f2a9c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:44.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2a99d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c105a60 0x7f2a9c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57126/0 (socket says 172.21.15.142:57126) 2024-03-29T05:02:44.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2a99d9b700 1 -- 172.21.15.142:0/1728184496 learned_addr learned my addr 172.21.15.142:0/1728184496 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:44.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2a9a59c700 1 --2- 172.21.15.142:0/1728184496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a9c0ffa60 0x7f2a9c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:44.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2a99d9b700 1 -- 172.21.15.142:0/1728184496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a9c0ffa60 msgr2=0x7f2a9c1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:44.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2a99d9b700 1 --2- 172.21.15.142:0/1728184496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a9c0ffa60 0x7f2a9c1975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:44.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2a99d9b700 1 -- 172.21.15.142:0/1728184496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c0095f0 con 0x7f2a9c105a60 2024-03-29T05:02:44.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.585+0000 7f2a99d9b700 1 --2- 172.21.15.142:0/1728184496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c105a60 0x7f2a9c197af0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000d8d0 tx=0x7f2a9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:44.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.586+0000 7f2a8b7fe700 1 -- 172.21.15.142:0/1728184496 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a90009940 con 0x7f2a9c105a60 2024-03-29T05:02:44.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.586+0000 7f2a8b7fe700 1 -- 172.21.15.142:0/1728184496 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a90010460 con 0x7f2a9c105a60 2024-03-29T05:02:44.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.586+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/1728184496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c1a0620 con 0x7f2a9c105a60 2024-03-29T05:02:44.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.586+0000 7f2a8b7fe700 1 -- 172.21.15.142:0/1728184496 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a9000f5f0 con 0x7f2a9c105a60 2024-03-29T05:02:44.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.586+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/1728184496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c1a0b70 con 0x7f2a9c105a60 2024-03-29T05:02:44.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.587+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/1728184496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c108060 con 0x7f2a9c105a60 2024-03-29T05:02:44.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.587+0000 7f2a8b7fe700 1 -- 172.21.15.142:0/1728184496 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f2a90009aa0 con 0x7f2a9c105a60 2024-03-29T05:02:44.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.588+0000 7f2a8b7fe700 1 --2- 172.21.15.142:0/1728184496 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2a8406dd60 0x7f2a84070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:44.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.588+0000 7f2a9a59c700 1 --2- 172.21.15.142:0/1728184496 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2a8406dd60 0x7f2a84070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:44.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.588+0000 7f2a8b7fe700 1 -- 172.21.15.142:0/1728184496 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f2a9008dca0 con 0x7f2a9c105a60 2024-03-29T05:02:44.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.589+0000 7f2a9a59c700 1 --2- 172.21.15.142:0/1728184496 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2a8406dd60 0x7f2a84070220 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00b5c0 tx=0x7f2a8c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:44.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.593+0000 7f2a8b7fe700 1 -- 172.21.15.142:0/1728184496 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2a90092070 con 0x7f2a9c105a60 2024-03-29T05:02:44.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.816+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/1728184496 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a9c02cce0 con 0x7f2a8406dd60 2024-03-29T05:02:44.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.817+0000 7f2a8b7fe700 1 -- 172.21.15.142:0/1728184496 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2a9c02cce0 con 0x7f2a8406dd60 2024-03-29T05:02:44.828 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.819+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/1728184496 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2a8406dd60 msgr2=0x7f2a84070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:44.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.819+0000 7f2aa0e7a700 1 --2- 172.21.15.142:0/1728184496 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2a8406dd60 0x7f2a84070220 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00b5c0 tx=0x7f2a8c01a040 comp rx=0 tx=0).stop 2024-03-29T05:02:44.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.819+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/1728184496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c105a60 msgr2=0x7f2a9c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:44.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.819+0000 7f2aa0e7a700 1 --2- 172.21.15.142:0/1728184496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c105a60 0x7f2a9c197af0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000d8d0 tx=0x7f2a9000dc90 comp rx=0 tx=0).stop 2024-03-29T05:02:44.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.819+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/1728184496 shutdown_connections 2024-03-29T05:02:44.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.819+0000 7f2aa0e7a700 1 --2- 172.21.15.142:0/1728184496 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2a8406dd60 0x7f2a84070220 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:44.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.819+0000 7f2aa0e7a700 1 --2- 172.21.15.142:0/1728184496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a9c0ffa60 0x7f2a9c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:44.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.819+0000 7f2aa0e7a700 1 --2- 172.21.15.142:0/1728184496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c105a60 0x7f2a9c197af0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:44.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.819+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/1728184496 >> 172.21.15.142:0/1728184496 conn(0x7f2a9c0fb570 msgr2=0x7f2a9c0fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:44.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.819+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/1728184496 shutdown_connections 2024-03-29T05:02:44.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.819+0000 7f2aa0e7a700 1 -- 172.21.15.142:0/1728184496 wait complete. 2024-03-29T05:02:44.839 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:02:44.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.925+0000 7f8243b48700 1 -- 172.21.15.142:0/4170074932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823c100dd0 msgr2=0x7f823c1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:44.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.925+0000 7f8243b48700 1 --2- 172.21.15.142:0/4170074932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823c100dd0 0x7f823c1011b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8238009910 tx=0x7f8238009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.925+0000 7f8243b48700 1 -- 172.21.15.142:0/4170074932 shutdown_connections 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.925+0000 7f8243b48700 1 --2- 172.21.15.142:0/4170074932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f823c101780 0x7f823c1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.925+0000 7f8243b48700 1 --2- 172.21.15.142:0/4170074932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823c100dd0 0x7f823c1011b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.925+0000 7f8243b48700 1 -- 172.21.15.142:0/4170074932 >> 172.21.15.142:0/4170074932 conn(0x7f823c0fc640 msgr2=0x7f823c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.926+0000 7f8243b48700 1 -- 172.21.15.142:0/4170074932 shutdown_connections 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.926+0000 7f8243b48700 1 -- 172.21.15.142:0/4170074932 wait complete. 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.926+0000 7f8243b48700 1 Processor -- start 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.926+0000 7f8243b48700 1 -- start start 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.926+0000 7f8243b48700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f823c100dd0 0x7f823c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.926+0000 7f8243b48700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823c101780 0x7f823c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.926+0000 7f8243b48700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c073b10 con 0x7f823c101780 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.926+0000 7f8243b48700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c073c50 con 0x7f823c100dd0 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.926+0000 7f82410e3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823c101780 0x7f823c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:44.937 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.927+0000 7f82410e3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823c101780 0x7f823c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57146/0 (socket says 172.21.15.142:57146) 2024-03-29T05:02:44.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.927+0000 7f82410e3700 1 -- 172.21.15.142:0/2753190053 learned_addr learned my addr 172.21.15.142:0/2753190053 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:44.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.927+0000 7f82418e4700 1 --2- 172.21.15.142:0/2753190053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f823c100dd0 0x7f823c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:44.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.927+0000 7f82410e3700 1 -- 172.21.15.142:0/2753190053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f823c100dd0 msgr2=0x7f823c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:44.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.927+0000 7f82410e3700 1 --2- 172.21.15.142:0/2753190053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f823c100dd0 0x7f823c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:44.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.927+0000 7f82410e3700 1 -- 172.21.15.142:0/2753190053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82380095f0 con 0x7f823c101780 2024-03-29T05:02:44.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.927+0000 7f82410e3700 1 --2- 172.21.15.142:0/2753190053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823c101780 0x7f823c0735d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f822c00b480 tx=0x7f822c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:44.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.927+0000 7f8232ffd700 1 -- 172.21.15.142:0/2753190053 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f822c0118c0 con 0x7f823c101780 2024-03-29T05:02:44.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.927+0000 7f8232ffd700 1 -- 172.21.15.142:0/2753190053 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f822c010380 con 0x7f823c101780 2024-03-29T05:02:44.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.927+0000 7f8243b48700 1 -- 172.21.15.142:0/2753190053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c1a2a50 con 0x7f823c101780 2024-03-29T05:02:44.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.927+0000 7f8232ffd700 1 -- 172.21.15.142:0/2753190053 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f822c00f4f0 con 0x7f823c101780 2024-03-29T05:02:44.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.927+0000 7f8243b48700 1 -- 172.21.15.142:0/2753190053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c1a2e60 con 0x7f823c101780 2024-03-29T05:02:44.939 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.928+0000 7f8243b48700 1 -- 172.21.15.142:0/2753190053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823c109170 con 0x7f823c101780 2024-03-29T05:02:44.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.929+0000 7f8232ffd700 1 -- 172.21.15.142:0/2753190053 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f822c0104f0 con 0x7f823c101780 2024-03-29T05:02:44.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.929+0000 7f8232ffd700 1 --2- 172.21.15.142:0/2753190053 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f822806dd60 0x7f8228070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:44.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.929+0000 7f8232ffd700 1 -- 172.21.15.142:0/2753190053 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f822c08d8f0 con 0x7f823c101780 2024-03-29T05:02:44.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.929+0000 7f82418e4700 1 --2- 172.21.15.142:0/2753190053 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f822806dd60 0x7f8228070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:44.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.930+0000 7f82418e4700 1 --2- 172.21.15.142:0/2753190053 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f822806dd60 0x7f8228070220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f823800b5c0 tx=0x7f8238005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:44.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:44.934+0000 7f8232ffd700 1 -- 172.21.15.142:0/2753190053 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f822c056d10 con 0x7f823c101780 2024-03-29T05:02:45.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.153+0000 7f8243b48700 1 -- 172.21.15.142:0/2753190053 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f823c02cd50 con 0x7f822806dd60 2024-03-29T05:02:45.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.155+0000 7f8232ffd700 1 -- 172.21.15.142:0/2753190053 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f823c02cd50 con 0x7f822806dd60 2024-03-29T05:02:45.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.157+0000 7f8243b48700 1 -- 172.21.15.142:0/2753190053 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f822806dd60 msgr2=0x7f8228070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:45.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.157+0000 7f8243b48700 1 --2- 172.21.15.142:0/2753190053 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f822806dd60 0x7f8228070220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f823800b5c0 tx=0x7f8238005960 comp rx=0 tx=0).stop 2024-03-29T05:02:45.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.157+0000 7f8243b48700 1 -- 172.21.15.142:0/2753190053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823c101780 msgr2=0x7f823c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:45.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.157+0000 7f8243b48700 1 --2- 172.21.15.142:0/2753190053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823c101780 0x7f823c0735d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f822c00b480 tx=0x7f822c00b840 comp rx=0 tx=0).stop 2024-03-29T05:02:45.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.157+0000 7f8243b48700 1 -- 172.21.15.142:0/2753190053 shutdown_connections 2024-03-29T05:02:45.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.157+0000 7f8243b48700 1 --2- 172.21.15.142:0/2753190053 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f822806dd60 0x7f8228070220 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.157+0000 7f8243b48700 1 --2- 172.21.15.142:0/2753190053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f823c100dd0 0x7f823c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.157+0000 7f8243b48700 1 --2- 172.21.15.142:0/2753190053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823c101780 0x7f823c0735d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.157+0000 7f8243b48700 1 -- 172.21.15.142:0/2753190053 >> 172.21.15.142:0/2753190053 conn(0x7f823c0fc640 msgr2=0x7f823c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:45.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.157+0000 7f8243b48700 1 -- 172.21.15.142:0/2753190053 shutdown_connections 2024-03-29T05:02:45.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.157+0000 7f8243b48700 1 -- 172.21.15.142:0/2753190053 wait complete. 2024-03-29T05:02:45.270 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.261+0000 7f3a7e131700 1 -- 172.21.15.142:0/1072681418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a780ffa40 msgr2=0x7f3a780ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:45.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.261+0000 7f3a7e131700 1 --2- 172.21.15.142:0/1072681418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a780ffa40 0x7f3a780ffec0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009910 tx=0x7f3a6c009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:45.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.261+0000 7f3a7e131700 1 -- 172.21.15.142:0/1072681418 shutdown_connections 2024-03-29T05:02:45.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.261+0000 7f3a7e131700 1 --2- 172.21.15.142:0/1072681418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a780ffa40 0x7f3a780ffec0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.261+0000 7f3a7e131700 1 --2- 172.21.15.142:0/1072681418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a78105a40 0x7f3a78105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.261+0000 7f3a7e131700 1 -- 172.21.15.142:0/1072681418 >> 172.21.15.142:0/1072681418 conn(0x7f3a780fb530 msgr2=0x7f3a780fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:45.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.261+0000 7f3a7e131700 1 -- 172.21.15.142:0/1072681418 shutdown_connections 2024-03-29T05:02:45.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.261+0000 7f3a7e131700 1 -- 172.21.15.142:0/1072681418 wait complete. 2024-03-29T05:02:45.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.261+0000 7f3a7e131700 1 Processor -- start 2024-03-29T05:02:45.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a7e131700 1 -- start start 2024-03-29T05:02:45.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a7e131700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a780ffa40 0x7f3a78195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:45.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a7e131700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a78105a40 0x7f3a78195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:45.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a7e131700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a78196250 con 0x7f3a780ffa40 2024-03-29T05:02:45.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a7e131700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a78199fe0 con 0x7f3a78105a40 2024-03-29T05:02:45.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a777fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a780ffa40 0x7f3a78195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:45.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a777fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a780ffa40 0x7f3a78195630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57162/0 (socket says 172.21.15.142:57162) 2024-03-29T05:02:45.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a777fe700 1 -- 172.21.15.142:0/3661265914 learned_addr learned my addr 172.21.15.142:0/3661265914 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:45.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a76ffd700 1 --2- 172.21.15.142:0/3661265914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a78105a40 0x7f3a78195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:45.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a777fe700 1 -- 172.21.15.142:0/3661265914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a78105a40 msgr2=0x7f3a78195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:45.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a777fe700 1 --2- 172.21.15.142:0/3661265914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a78105a40 0x7f3a78195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a777fe700 1 -- 172.21.15.142:0/3661265914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c0095f0 con 0x7f3a780ffa40 2024-03-29T05:02:45.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.262+0000 7f3a76ffd700 1 --2- 172.21.15.142:0/3661265914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a78105a40 0x7f3a78195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:02:45.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.263+0000 7f3a777fe700 1 --2- 172.21.15.142:0/3661265914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a780ffa40 0x7f3a78195630 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800cfb0 tx=0x7f3a6800e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:45.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.263+0000 7f3a74ff9700 1 -- 172.21.15.142:0/3661265914 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a68011070 con 0x7f3a780ffa40 2024-03-29T05:02:45.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.263+0000 7f3a7e131700 1 -- 172.21.15.142:0/3661265914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7819a260 con 0x7f3a780ffa40 2024-03-29T05:02:45.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.263+0000 7f3a74ff9700 1 -- 172.21.15.142:0/3661265914 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a6800c4b0 con 0x7f3a780ffa40 2024-03-29T05:02:45.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.263+0000 7f3a7e131700 1 -- 172.21.15.142:0/3661265914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7819a7b0 con 0x7f3a780ffa40 2024-03-29T05:02:45.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.263+0000 7f3a74ff9700 1 -- 172.21.15.142:0/3661265914 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a68015620 con 0x7f3a780ffa40 2024-03-29T05:02:45.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.264+0000 7f3a7e131700 1 -- 172.21.15.142:0/3661265914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7804ea90 con 0x7f3a780ffa40 2024-03-29T05:02:45.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.264+0000 7f3a74ff9700 1 -- 172.21.15.142:0/3661265914 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f3a6800e9f0 con 0x7f3a780ffa40 2024-03-29T05:02:45.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.265+0000 7f3a74ff9700 1 --2- 172.21.15.142:0/3661265914 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3a6006dd60 0x7f3a60070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:45.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.265+0000 7f3a74ff9700 1 -- 172.21.15.142:0/3661265914 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f3a6808fa70 con 0x7f3a780ffa40 2024-03-29T05:02:45.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.265+0000 7f3a76ffd700 1 --2- 172.21.15.142:0/3661265914 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3a6006dd60 0x7f3a60070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:45.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.265+0000 7f3a76ffd700 1 --2- 172.21.15.142:0/3661265914 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3a6006dd60 0x7f3a60070220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0059d0 tx=0x7f3a6c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:45.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.269+0000 7f3a74ff9700 1 -- 172.21.15.142:0/3661265914 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3a68058e90 con 0x7f3a780ffa40 2024-03-29T05:02:45.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.479+0000 7f3a7e131700 1 -- 172.21.15.142:0/3661265914 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a7819ad40 con 0x7f3a6006dd60 2024-03-29T05:02:45.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.490+0000 7f3a74ff9700 1 -- 172.21.15.142:0/3661265914 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3a7819ad40 con 0x7f3a6006dd60 2024-03-29T05:02:45.499 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:02:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (14m) 7m ago 26m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:02:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (26m) 7m ago 26m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:02:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (25m) 7m ago 25m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:02:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (11m) 7m ago 26m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:02:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (11m) 7m ago 25m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:02:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (14m) 7m ago 25m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:02:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (7m) 7m ago 20m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:02:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (8m) 7m ago 20m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:02:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (7m) 7m ago 20m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:02:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (7m) 7m ago 20m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:02:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (16m) 7m ago 27m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:02:45.500 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (15m) 7m ago 25m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:02:45.501 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (12m) 7m ago 27m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:02:45.501 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (12m) 7m ago 25m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:02:45.501 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (15m) 7m ago 26m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:02:45.501 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (15m) 7m ago 25m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:02:45.501 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (11m) 7m ago 24m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:02:45.501 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (10m) 7m ago 23m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:02:45.501 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (10m) 7m ago 23m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:02:45.501 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (9m) 7m ago 22m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:02:45.501 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (8m) 7m ago 22m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:02:45.501 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (8m) 7m ago 22m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:02:45.501 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (14m) 7m ago 25m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:02:45.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.493+0000 7f3a7e131700 1 -- 172.21.15.142:0/3661265914 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3a6006dd60 msgr2=0x7f3a60070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:45.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.493+0000 7f3a7e131700 1 --2- 172.21.15.142:0/3661265914 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3a6006dd60 0x7f3a60070220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0059d0 tx=0x7f3a6c01a040 comp rx=0 tx=0).stop 2024-03-29T05:02:45.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.493+0000 7f3a7e131700 1 -- 172.21.15.142:0/3661265914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a780ffa40 msgr2=0x7f3a78195630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:45.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.493+0000 7f3a7e131700 1 --2- 172.21.15.142:0/3661265914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a780ffa40 0x7f3a78195630 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800cfb0 tx=0x7f3a6800e330 comp rx=0 tx=0).stop 2024-03-29T05:02:45.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.493+0000 7f3a7e131700 1 -- 172.21.15.142:0/3661265914 shutdown_connections 2024-03-29T05:02:45.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.493+0000 7f3a7e131700 1 --2- 172.21.15.142:0/3661265914 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3a6006dd60 0x7f3a60070220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.493+0000 7f3a7e131700 1 --2- 172.21.15.142:0/3661265914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a780ffa40 0x7f3a78195630 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.493+0000 7f3a7e131700 1 --2- 172.21.15.142:0/3661265914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a78105a40 0x7f3a78195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.493+0000 7f3a7e131700 1 -- 172.21.15.142:0/3661265914 >> 172.21.15.142:0/3661265914 conn(0x7f3a780fb530 msgr2=0x7f3a780fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:45.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.493+0000 7f3a7e131700 1 -- 172.21.15.142:0/3661265914 shutdown_connections 2024-03-29T05:02:45.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.493+0000 7f3a7e131700 1 -- 172.21.15.142:0/3661265914 wait complete. 2024-03-29T05:02:45.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.594+0000 7f5d32d95700 1 -- 172.21.15.142:0/2492710413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2c105a40 msgr2=0x7f5d2c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:45.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.594+0000 7f5d32d95700 1 --2- 172.21.15.142:0/2492710413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2c105a40 0x7f5d2c105e20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009910 tx=0x7f5d20009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:45.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.594+0000 7f5d32d95700 1 -- 172.21.15.142:0/2492710413 shutdown_connections 2024-03-29T05:02:45.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.594+0000 7f5d32d95700 1 --2- 172.21.15.142:0/2492710413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d2c0ffa40 0x7f5d2c0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.594+0000 7f5d32d95700 1 --2- 172.21.15.142:0/2492710413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2c105a40 0x7f5d2c105e20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.594+0000 7f5d32d95700 1 -- 172.21.15.142:0/2492710413 >> 172.21.15.142:0/2492710413 conn(0x7f5d2c0fb550 msgr2=0x7f5d2c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:45.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.594+0000 7f5d32d95700 1 -- 172.21.15.142:0/2492710413 shutdown_connections 2024-03-29T05:02:45.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.594+0000 7f5d32d95700 1 -- 172.21.15.142:0/2492710413 wait complete. 2024-03-29T05:02:45.604 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:45 smithi142 ceph-mon[112475]: pgmap v433: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:45.604 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:02:45.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.594+0000 7f5d32d95700 1 Processor -- start 2024-03-29T05:02:45.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.594+0000 7f5d32d95700 1 -- start start 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d32d95700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d2c0ffa40 0x7f5d2c0fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d32d95700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2c105a40 0x7f5d2c0fc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d32d95700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0fc8c0 con 0x7f5d2c105a40 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d32d95700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0fca00 con 0x7f5d2c0ffa40 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d2bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2c105a40 0x7f5d2c0fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d2bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2c105a40 0x7f5d2c0fc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57184/0 (socket says 172.21.15.142:57184) 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d2bfff700 1 -- 172.21.15.142:0/2668395966 learned_addr learned my addr 172.21.15.142:0/2668395966 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d30b31700 1 --2- 172.21.15.142:0/2668395966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d2c0ffa40 0x7f5d2c0fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d2bfff700 1 -- 172.21.15.142:0/2668395966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d2c0ffa40 msgr2=0x7f5d2c0fbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d2bfff700 1 --2- 172.21.15.142:0/2668395966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d2c0ffa40 0x7f5d2c0fbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d2bfff700 1 -- 172.21.15.142:0/2668395966 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d200095f0 con 0x7f5d2c105a40 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d2bfff700 1 --2- 172.21.15.142:0/2668395966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2c105a40 0x7f5d2c0fc380 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800d8d0 tx=0x7f5d1800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:45.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d29ffb700 1 -- 172.21.15.142:0/2668395966 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d180098a0 con 0x7f5d2c105a40 2024-03-29T05:02:45.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d32d95700 1 -- 172.21.15.142:0/2668395966 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c1a29f0 con 0x7f5d2c105a40 2024-03-29T05:02:45.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d29ffb700 1 -- 172.21.15.142:0/2668395966 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d18010460 con 0x7f5d2c105a40 2024-03-29T05:02:45.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.595+0000 7f5d29ffb700 1 -- 172.21.15.142:0/2668395966 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d1800f5f0 con 0x7f5d2c105a40 2024-03-29T05:02:45.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.596+0000 7f5d32d95700 1 -- 172.21.15.142:0/2668395966 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c1a2e00 con 0x7f5d2c105a40 2024-03-29T05:02:45.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.597+0000 7f5d32d95700 1 -- 172.21.15.142:0/2668395966 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c109050 con 0x7f5d2c105a40 2024-03-29T05:02:45.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.597+0000 7f5d29ffb700 1 -- 172.21.15.142:0/2668395966 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f5d18009a00 con 0x7f5d2c105a40 2024-03-29T05:02:45.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.597+0000 7f5d29ffb700 1 --2- 172.21.15.142:0/2668395966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5d1c06dd60 0x7f5d1c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:45.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.598+0000 7f5d29ffb700 1 -- 172.21.15.142:0/2668395966 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f5d1808db40 con 0x7f5d2c105a40 2024-03-29T05:02:45.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.598+0000 7f5d30b31700 1 --2- 172.21.15.142:0/2668395966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5d1c06dd60 0x7f5d1c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:45.608 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.598+0000 7f5d30b31700 1 --2- 172.21.15.142:0/2668395966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5d1c06dd60 0x7f5d1c070220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5d20006010 tx=0x7f5d2001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:45.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.602+0000 7f5d29ffb700 1 -- 172.21.15.142:0/2668395966 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5d18056f60 con 0x7f5d2c105a40 2024-03-29T05:02:45.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:45 smithi152 ceph-mon[88186]: pgmap v433: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:45.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:02:45.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.881+0000 7f5d32d95700 1 -- 172.21.15.142:0/2668395966 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5d2c04ea90 con 0x7f5d2c105a40 2024-03-29T05:02:45.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.881+0000 7f5d29ffb700 1 -- 172.21.15.142:0/2668395966 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5d1805a620 con 0x7f5d2c105a40 2024-03-29T05:02:45.891 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:02:45.891 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:02:45.891 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:02:45.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:02:45.891 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:02:45.891 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:02:45.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:02:45.891 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:02:45.891 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:02:45.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:02:45.892 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:02:45.892 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:02:45.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:02:45.892 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:02:45.892 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:02:45.892 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:02:45.892 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:02:45.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.884+0000 7f5d32d95700 1 -- 172.21.15.142:0/2668395966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5d1c06dd60 msgr2=0x7f5d1c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:45.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.884+0000 7f5d32d95700 1 --2- 172.21.15.142:0/2668395966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5d1c06dd60 0x7f5d1c070220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5d20006010 tx=0x7f5d2001a040 comp rx=0 tx=0).stop 2024-03-29T05:02:45.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.884+0000 7f5d32d95700 1 -- 172.21.15.142:0/2668395966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2c105a40 msgr2=0x7f5d2c0fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:45.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.884+0000 7f5d32d95700 1 --2- 172.21.15.142:0/2668395966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2c105a40 0x7f5d2c0fc380 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800d8d0 tx=0x7f5d1800dbe0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.884+0000 7f5d32d95700 1 -- 172.21.15.142:0/2668395966 shutdown_connections 2024-03-29T05:02:45.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.884+0000 7f5d32d95700 1 --2- 172.21.15.142:0/2668395966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5d1c06dd60 0x7f5d1c070220 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.884+0000 7f5d32d95700 1 --2- 172.21.15.142:0/2668395966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d2c0ffa40 0x7f5d2c0fbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.884+0000 7f5d32d95700 1 --2- 172.21.15.142:0/2668395966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2c105a40 0x7f5d2c0fc380 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.884+0000 7f5d32d95700 1 -- 172.21.15.142:0/2668395966 >> 172.21.15.142:0/2668395966 conn(0x7f5d2c0fb550 msgr2=0x7f5d2c069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:45.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.884+0000 7f5d32d95700 1 -- 172.21.15.142:0/2668395966 shutdown_connections 2024-03-29T05:02:45.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.884+0000 7f5d32d95700 1 -- 172.21.15.142:0/2668395966 wait complete. 2024-03-29T05:02:45.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.988+0000 7f652b4ba700 1 -- 172.21.15.142:0/2236358395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65240ffbd0 msgr2=0x7f6524100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:45.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.988+0000 7f652b4ba700 1 --2- 172.21.15.142:0/2236358395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65240ffbd0 0x7f6524100050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6520009910 tx=0x7f6520009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:45.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.988+0000 7f652b4ba700 1 -- 172.21.15.142:0/2236358395 shutdown_connections 2024-03-29T05:02:45.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.988+0000 7f652b4ba700 1 --2- 172.21.15.142:0/2236358395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65240ffbd0 0x7f6524100050 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.988+0000 7f652b4ba700 1 --2- 172.21.15.142:0/2236358395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6524105a40 0x7f6524105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:45.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.988+0000 7f652b4ba700 1 -- 172.21.15.142:0/2236358395 >> 172.21.15.142:0/2236358395 conn(0x7f65240755a0 msgr2=0x7f65240759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:45.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.988+0000 7f652b4ba700 1 -- 172.21.15.142:0/2236358395 shutdown_connections 2024-03-29T05:02:45.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.988+0000 7f652b4ba700 1 -- 172.21.15.142:0/2236358395 wait complete. 2024-03-29T05:02:46.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f652b4ba700 1 Processor -- start 2024-03-29T05:02:46.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f652b4ba700 1 -- start start 2024-03-29T05:02:46.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f652b4ba700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65240ffbd0 0x7f65241976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:46.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f652b4ba700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6524105a40 0x7f6524197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:46.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f652b4ba700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6524198200 con 0x7f6524105a40 2024-03-29T05:02:46.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f652b4ba700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6524198340 con 0x7f65240ffbd0 2024-03-29T05:02:46.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f6528a55700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6524105a40 0x7f6524197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:46.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f6528a55700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6524105a40 0x7f6524197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57214/0 (socket says 172.21.15.142:57214) 2024-03-29T05:02:46.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f6528a55700 1 -- 172.21.15.142:0/89097304 learned_addr learned my addr 172.21.15.142:0/89097304 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:46.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f6529256700 1 --2- 172.21.15.142:0/89097304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65240ffbd0 0x7f65241976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:46.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f6528a55700 1 -- 172.21.15.142:0/89097304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65240ffbd0 msgr2=0x7f65241976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:46.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f6528a55700 1 --2- 172.21.15.142:0/89097304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65240ffbd0 0x7f65241976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f6528a55700 1 -- 172.21.15.142:0/89097304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200095f0 con 0x7f6524105a40 2024-03-29T05:02:46.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.989+0000 7f6528a55700 1 --2- 172.21.15.142:0/89097304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6524105a40 0x7f6524197be0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6520006010 tx=0x7f652000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:46.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.990+0000 7f651a7fc700 1 -- 172.21.15.142:0/89097304 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f652001d070 con 0x7f6524105a40 2024-03-29T05:02:46.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.990+0000 7f651a7fc700 1 -- 172.21.15.142:0/89097304 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6520004fb0 con 0x7f6524105a40 2024-03-29T05:02:46.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.990+0000 7f652b4ba700 1 -- 172.21.15.142:0/89097304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65241a0710 con 0x7f6524105a40 2024-03-29T05:02:46.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.990+0000 7f651a7fc700 1 -- 172.21.15.142:0/89097304 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6520022ea0 con 0x7f6524105a40 2024-03-29T05:02:46.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.990+0000 7f652b4ba700 1 -- 172.21.15.142:0/89097304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65241a0b20 con 0x7f6524105a40 2024-03-29T05:02:46.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.991+0000 7f652b4ba700 1 -- 172.21.15.142:0/89097304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6524108060 con 0x7f6524105a40 2024-03-29T05:02:46.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.991+0000 7f651a7fc700 1 -- 172.21.15.142:0/89097304 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6520022470 con 0x7f6524105a40 2024-03-29T05:02:46.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.991+0000 7f651a7fc700 1 --2- 172.21.15.142:0/89097304 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f651006dd60 0x7f6510070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:46.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.992+0000 7f651a7fc700 1 -- 172.21.15.142:0/89097304 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f652008f180 con 0x7f6524105a40 2024-03-29T05:02:46.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.992+0000 7f6529256700 1 --2- 172.21.15.142:0/89097304 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f651006dd60 0x7f6510070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:46.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.992+0000 7f6529256700 1 --2- 172.21.15.142:0/89097304 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f651006dd60 0x7f6510070220 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6514006fd0 tx=0x7f6514008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:46.005 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:45.996+0000 7f651a7fc700 1 -- 172.21.15.142:0/89097304 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f65200586d0 con 0x7f6524105a40 2024-03-29T05:02:46.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.235+0000 7f652b4ba700 1 -- 172.21.15.142:0/89097304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f652404ea90 con 0x7f6524105a40 2024-03-29T05:02:46.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.235+0000 7f651a7fc700 1 -- 172.21.15.142:0/89097304 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f6520027770 con 0x7f6524105a40 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:02:46.245 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:02:46.246 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:02:46.247 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:02:46.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.238+0000 7f652b4ba700 1 -- 172.21.15.142:0/89097304 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f651006dd60 msgr2=0x7f6510070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:46.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.238+0000 7f652b4ba700 1 --2- 172.21.15.142:0/89097304 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f651006dd60 0x7f6510070220 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6514006fd0 tx=0x7f6514008040 comp rx=0 tx=0).stop 2024-03-29T05:02:46.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.238+0000 7f652b4ba700 1 -- 172.21.15.142:0/89097304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6524105a40 msgr2=0x7f6524197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:46.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.238+0000 7f652b4ba700 1 --2- 172.21.15.142:0/89097304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6524105a40 0x7f6524197be0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6520006010 tx=0x7f652000be20 comp rx=0 tx=0).stop 2024-03-29T05:02:46.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.238+0000 7f652b4ba700 1 -- 172.21.15.142:0/89097304 shutdown_connections 2024-03-29T05:02:46.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.238+0000 7f652b4ba700 1 --2- 172.21.15.142:0/89097304 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f651006dd60 0x7f6510070220 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.238+0000 7f652b4ba700 1 --2- 172.21.15.142:0/89097304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65240ffbd0 0x7f65241976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.238+0000 7f652b4ba700 1 --2- 172.21.15.142:0/89097304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6524105a40 0x7f6524197be0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.238+0000 7f652b4ba700 1 -- 172.21.15.142:0/89097304 >> 172.21.15.142:0/89097304 conn(0x7f65240755a0 msgr2=0x7f65240fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:46.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.238+0000 7f652b4ba700 1 -- 172.21.15.142:0/89097304 shutdown_connections 2024-03-29T05:02:46.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.238+0000 7f652b4ba700 1 -- 172.21.15.142:0/89097304 wait complete. 2024-03-29T05:02:46.249 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:02:46.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.343+0000 7f8ef7d95700 1 -- 172.21.15.142:0/2892200321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef01026c0 msgr2=0x7f8ef010abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:46.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.343+0000 7f8ef7d95700 1 --2- 172.21.15.142:0/2892200321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef01026c0 0x7f8ef010abb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009910 tx=0x7f8ee0009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:46.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.343+0000 7f8ef7d95700 1 -- 172.21.15.142:0/2892200321 shutdown_connections 2024-03-29T05:02:46.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.343+0000 7f8ef7d95700 1 --2- 172.21.15.142:0/2892200321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef01026c0 0x7f8ef010abb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.343+0000 7f8ef7d95700 1 --2- 172.21.15.142:0/2892200321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef0101da0 0x7f8ef0102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.343+0000 7f8ef7d95700 1 -- 172.21.15.142:0/2892200321 >> 172.21.15.142:0/2892200321 conn(0x7f8ef00fb570 msgr2=0x7f8ef00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:46.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.343+0000 7f8ef7d95700 1 -- 172.21.15.142:0/2892200321 shutdown_connections 2024-03-29T05:02:46.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.343+0000 7f8ef7d95700 1 -- 172.21.15.142:0/2892200321 wait complete. 2024-03-29T05:02:46.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.343+0000 7f8ef7d95700 1 Processor -- start 2024-03-29T05:02:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.343+0000 7f8ef7d95700 1 -- start start 2024-03-29T05:02:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef7d95700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0101da0 0x7f8ef019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef7d95700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef01026c0 0x7f8ef019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef7d95700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef019d110 con 0x7f8ef0101da0 2024-03-29T05:02:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef7d95700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef01965c0 con 0x7f8ef01026c0 2024-03-29T05:02:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef5b31700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0101da0 0x7f8ef019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef5b31700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0101da0 0x7f8ef019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57236/0 (socket says 172.21.15.142:57236) 2024-03-29T05:02:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef5b31700 1 -- 172.21.15.142:0/4011824998 learned_addr learned my addr 172.21.15.142:0/4011824998 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef5330700 1 --2- 172.21.15.142:0/4011824998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef01026c0 0x7f8ef019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef5b31700 1 -- 172.21.15.142:0/4011824998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef01026c0 msgr2=0x7f8ef019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:46.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef5b31700 1 --2- 172.21.15.142:0/4011824998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef01026c0 0x7f8ef019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef5b31700 1 -- 172.21.15.142:0/4011824998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee00095f0 con 0x7f8ef0101da0 2024-03-29T05:02:46.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef5330700 1 --2- 172.21.15.142:0/4011824998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef01026c0 0x7f8ef019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:02:46.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef5b31700 1 --2- 172.21.15.142:0/4011824998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0101da0 0x7f8ef019c540 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00e920 tx=0x7f8eec00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:46.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ee6ffd700 1 -- 172.21.15.142:0/4011824998 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8eec009790 con 0x7f8ef0101da0 2024-03-29T05:02:46.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ee6ffd700 1 -- 172.21.15.142:0/4011824998 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8eec00cba0 con 0x7f8ef0101da0 2024-03-29T05:02:46.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ef7d95700 1 -- 172.21.15.142:0/4011824998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef01968a0 con 0x7f8ef0101da0 2024-03-29T05:02:46.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.344+0000 7f8ee6ffd700 1 -- 172.21.15.142:0/4011824998 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8eec018650 con 0x7f8ef0101da0 2024-03-29T05:02:46.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.345+0000 7f8ef7d95700 1 -- 172.21.15.142:0/4011824998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef0196df0 con 0x7f8ef0101da0 2024-03-29T05:02:46.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.345+0000 7f8ef7d95700 1 -- 172.21.15.142:0/4011824998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef01082b0 con 0x7f8ef0101da0 2024-03-29T05:02:46.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.346+0000 7f8ee6ffd700 1 -- 172.21.15.142:0/4011824998 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f8eec00cd10 con 0x7f8ef0101da0 2024-03-29T05:02:46.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.346+0000 7f8ee6ffd700 1 --2- 172.21.15.142:0/4011824998 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8edc06dd60 0x7f8edc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:46.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.347+0000 7f8ef5330700 1 --2- 172.21.15.142:0/4011824998 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8edc06dd60 0x7f8edc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:46.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.347+0000 7f8ee6ffd700 1 -- 172.21.15.142:0/4011824998 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f8eec014070 con 0x7f8ef0101da0 2024-03-29T05:02:46.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.347+0000 7f8ef5330700 1 --2- 172.21.15.142:0/4011824998 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8edc06dd60 0x7f8edc070220 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0000c00 tx=0x7f8ee0009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:46.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.351+0000 7f8ee6ffd700 1 -- 172.21.15.142:0/4011824998 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8eec053280 con 0x7f8ef0101da0 2024-03-29T05:02:46.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.563+0000 7f8ef7d95700 1 -- 172.21.15.142:0/4011824998 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ef0197090 con 0x7f8edc06dd60 2024-03-29T05:02:46.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:46 smithi142 ceph-mon[112475]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:46.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:46 smithi142 ceph-mon[112475]: from='client.34826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:46.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:46 smithi142 ceph-mon[112475]: from='client.34830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:46.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:46 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2668395966' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:02:46.573 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:46 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/89097304' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:02:46.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.565+0000 7f8ee6ffd700 1 -- 172.21.15.142:0/4011824998 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8ef0197090 con 0x7f8edc06dd60 2024-03-29T05:02:46.574 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:02:46.574 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:02:46.574 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:02:46.575 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:02:46.575 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:02:46.575 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:02:46.575 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:02:46.575 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:02:46.575 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:02:46.575 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:02:46.575 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:02:46.575 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:02:46.575 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:02:46.575 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:02:46.575 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:02:46.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.567+0000 7f8ef7d95700 1 -- 172.21.15.142:0/4011824998 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8edc06dd60 msgr2=0x7f8edc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:46.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.567+0000 7f8ef7d95700 1 --2- 172.21.15.142:0/4011824998 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8edc06dd60 0x7f8edc070220 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0000c00 tx=0x7f8ee0009f70 comp rx=0 tx=0).stop 2024-03-29T05:02:46.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.567+0000 7f8ef7d95700 1 -- 172.21.15.142:0/4011824998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0101da0 msgr2=0x7f8ef019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:46.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.567+0000 7f8ef7d95700 1 --2- 172.21.15.142:0/4011824998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0101da0 0x7f8ef019c540 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00e920 tx=0x7f8eec00ece0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.567+0000 7f8ef7d95700 1 -- 172.21.15.142:0/4011824998 shutdown_connections 2024-03-29T05:02:46.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.567+0000 7f8ef7d95700 1 --2- 172.21.15.142:0/4011824998 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8edc06dd60 0x7f8edc070220 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.567+0000 7f8ef7d95700 1 --2- 172.21.15.142:0/4011824998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0101da0 0x7f8ef019c540 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.567+0000 7f8ef7d95700 1 --2- 172.21.15.142:0/4011824998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef01026c0 0x7f8ef019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.567+0000 7f8ef7d95700 1 -- 172.21.15.142:0/4011824998 >> 172.21.15.142:0/4011824998 conn(0x7f8ef00fb570 msgr2=0x7f8ef01053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:46.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.567+0000 7f8ef7d95700 1 -- 172.21.15.142:0/4011824998 shutdown_connections 2024-03-29T05:02:46.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.567+0000 7f8ef7d95700 1 -- 172.21.15.142:0/4011824998 wait complete. 2024-03-29T05:02:46.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.671+0000 7f5e464dd700 1 -- 172.21.15.142:0/2144470469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e40105a40 msgr2=0x7f5e40105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:46.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.671+0000 7f5e464dd700 1 --2- 172.21.15.142:0/2144470469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e40105a40 0x7f5e40105e20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009910 tx=0x7f5e34009c20 comp rx=0 tx=0).stop 2024-03-29T05:02:46.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.671+0000 7f5e464dd700 1 -- 172.21.15.142:0/2144470469 shutdown_connections 2024-03-29T05:02:46.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.671+0000 7f5e464dd700 1 --2- 172.21.15.142:0/2144470469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e400ffbd0 0x7f5e40100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.671+0000 7f5e464dd700 1 --2- 172.21.15.142:0/2144470469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e40105a40 0x7f5e40105e20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.671+0000 7f5e464dd700 1 -- 172.21.15.142:0/2144470469 >> 172.21.15.142:0/2144470469 conn(0x7f5e400755a0 msgr2=0x7f5e400759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:46.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.671+0000 7f5e464dd700 1 -- 172.21.15.142:0/2144470469 shutdown_connections 2024-03-29T05:02:46.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.671+0000 7f5e464dd700 1 -- 172.21.15.142:0/2144470469 wait complete. 2024-03-29T05:02:46.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.672+0000 7f5e464dd700 1 Processor -- start 2024-03-29T05:02:46.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.672+0000 7f5e464dd700 1 -- start start 2024-03-29T05:02:46.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.672+0000 7f5e464dd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e400ffbd0 0x7f5e40196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:46.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.672+0000 7f5e464dd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e40105a40 0x7f5e40196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:46.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.672+0000 7f5e464dd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e401972e0 con 0x7f5e400ffbd0 2024-03-29T05:02:46.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.672+0000 7f5e464dd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4019a010 con 0x7f5e40105a40 2024-03-29T05:02:46.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.672+0000 7f5e3ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e400ffbd0 0x7f5e40196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:46.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.672+0000 7f5e3ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e400ffbd0 0x7f5e40196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57256/0 (socket says 172.21.15.142:57256) 2024-03-29T05:02:46.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.672+0000 7f5e3ffff700 1 -- 172.21.15.142:0/285286943 learned_addr learned my addr 172.21.15.142:0/285286943 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:02:46.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.672+0000 7f5e3f7fe700 1 --2- 172.21.15.142:0/285286943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e40105a40 0x7f5e40196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:46.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.673+0000 7f5e3ffff700 1 -- 172.21.15.142:0/285286943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e40105a40 msgr2=0x7f5e40196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:46.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.673+0000 7f5e3ffff700 1 --2- 172.21.15.142:0/285286943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e40105a40 0x7f5e40196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.673+0000 7f5e3ffff700 1 -- 172.21.15.142:0/285286943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e340095f0 con 0x7f5e400ffbd0 2024-03-29T05:02:46.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.673+0000 7f5e3ffff700 1 --2- 172.21.15.142:0/285286943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e400ffbd0 0x7f5e40196710 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5e34006010 tx=0x7f5e3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:46.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.673+0000 7f5e3d7fa700 1 -- 172.21.15.142:0/285286943 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e3401d070 con 0x7f5e400ffbd0 2024-03-29T05:02:46.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.673+0000 7f5e3d7fa700 1 -- 172.21.15.142:0/285286943 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e34004fb0 con 0x7f5e400ffbd0 2024-03-29T05:02:46.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.673+0000 7f5e3d7fa700 1 -- 172.21.15.142:0/285286943 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e34022ea0 con 0x7f5e400ffbd0 2024-03-29T05:02:46.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.673+0000 7f5e464dd700 1 -- 172.21.15.142:0/285286943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4019a290 con 0x7f5e400ffbd0 2024-03-29T05:02:46.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.673+0000 7f5e464dd700 1 -- 172.21.15.142:0/285286943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4019a780 con 0x7f5e400ffbd0 2024-03-29T05:02:46.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.674+0000 7f5e464dd700 1 -- 172.21.15.142:0/285286943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4004ea90 con 0x7f5e400ffbd0 2024-03-29T05:02:46.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.675+0000 7f5e3d7fa700 1 -- 172.21.15.142:0/285286943 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f5e34022470 con 0x7f5e400ffbd0 2024-03-29T05:02:46.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.675+0000 7f5e3d7fa700 1 --2- 172.21.15.142:0/285286943 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5e2806dd60 0x7f5e28070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:02:46.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.675+0000 7f5e3d7fa700 1 -- 172.21.15.142:0/285286943 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f5e3408f180 con 0x7f5e400ffbd0 2024-03-29T05:02:46.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.675+0000 7f5e3f7fe700 1 --2- 172.21.15.142:0/285286943 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5e2806dd60 0x7f5e28070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:02:46.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.675+0000 7f5e3f7fe700 1 --2- 172.21.15.142:0/285286943 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5e2806dd60 0x7f5e28070220 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e30005fd0 tx=0x7f5e3000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:02:46.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.680+0000 7f5e3d7fa700 1 -- 172.21.15.142:0/285286943 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5e340586d0 con 0x7f5e400ffbd0 2024-03-29T05:02:46.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:46 smithi152 ceph-mon[88186]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:46.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:46 smithi152 ceph-mon[88186]: from='client.34826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:46.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:46 smithi152 ceph-mon[88186]: from='client.34830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:46.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:46 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2668395966' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:02:46.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:46 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/89097304' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:02:46.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.957+0000 7f5e464dd700 1 -- 172.21.15.142:0/285286943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5e4019aaf0 con 0x7f5e400ffbd0 2024-03-29T05:02:46.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.958+0000 7f5e3d7fa700 1 -- 172.21.15.142:0/285286943 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f5e34027100 con 0x7f5e400ffbd0 2024-03-29T05:02:46.967 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:02:46.967 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:02:46.967 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:02:46.968 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:02:46.968 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:02:46.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.960+0000 7f5e464dd700 1 -- 172.21.15.142:0/285286943 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5e2806dd60 msgr2=0x7f5e28070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:46.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.960+0000 7f5e464dd700 1 --2- 172.21.15.142:0/285286943 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5e2806dd60 0x7f5e28070220 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e30005fd0 tx=0x7f5e3000a560 comp rx=0 tx=0).stop 2024-03-29T05:02:46.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.960+0000 7f5e464dd700 1 -- 172.21.15.142:0/285286943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e400ffbd0 msgr2=0x7f5e40196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:02:46.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.960+0000 7f5e464dd700 1 --2- 172.21.15.142:0/285286943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e400ffbd0 0x7f5e40196710 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5e34006010 tx=0x7f5e3400be20 comp rx=0 tx=0).stop 2024-03-29T05:02:46.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.960+0000 7f5e464dd700 1 -- 172.21.15.142:0/285286943 shutdown_connections 2024-03-29T05:02:46.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.960+0000 7f5e464dd700 1 --2- 172.21.15.142:0/285286943 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5e2806dd60 0x7f5e28070220 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.960+0000 7f5e464dd700 1 --2- 172.21.15.142:0/285286943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e400ffbd0 0x7f5e40196710 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.960+0000 7f5e464dd700 1 --2- 172.21.15.142:0/285286943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e40105a40 0x7f5e40196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:02:46.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.960+0000 7f5e464dd700 1 -- 172.21.15.142:0/285286943 >> 172.21.15.142:0/285286943 conn(0x7f5e400755a0 msgr2=0x7f5e401037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:02:46.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.960+0000 7f5e464dd700 1 -- 172.21.15.142:0/285286943 shutdown_connections 2024-03-29T05:02:46.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:02:46.960+0000 7f5e464dd700 1 -- 172.21.15.142:0/285286943 wait complete. 2024-03-29T05:02:47.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:47 smithi142 ceph-mon[112475]: pgmap v434: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:47.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:47 smithi142 ceph-mon[112475]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:47.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:47 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/285286943' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:02:47.752 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:47 smithi152 ceph-mon[88186]: pgmap v434: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:47.753 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:47 smithi152 ceph-mon[88186]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:02:47.753 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:47 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/285286943' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:02:49.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:49 smithi142 ceph-mon[112475]: pgmap v435: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:49.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:49 smithi152 ceph-mon[88186]: pgmap v435: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:51.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:51 smithi142 ceph-mon[112475]: pgmap v436: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:51.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:51 smithi152 ceph-mon[88186]: pgmap v436: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:53.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:53 smithi142 ceph-mon[112475]: pgmap v437: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:53.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:53 smithi152 ceph-mon[88186]: pgmap v437: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:55.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:55 smithi142 ceph-mon[112475]: pgmap v438: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:55.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:55 smithi152 ceph-mon[88186]: pgmap v438: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:57.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:57 smithi142 ceph-mon[112475]: pgmap v439: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:57.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:57 smithi152 ceph-mon[88186]: pgmap v439: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:59.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:02:59 smithi142 ceph-mon[112475]: pgmap v440: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:02:59.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:02:59 smithi152 ceph-mon[88186]: pgmap v440: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:00.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:03:00.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:03:01.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:01 smithi142 ceph-mon[112475]: pgmap v441: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:01.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:01 smithi152 ceph-mon[88186]: pgmap v441: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:03.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:03 smithi142 ceph-mon[112475]: pgmap v442: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:03.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:03 smithi152 ceph-mon[88186]: pgmap v442: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:05.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:05 smithi142 ceph-mon[112475]: pgmap v443: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:05 smithi152 ceph-mon[88186]: pgmap v443: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:07.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:07 smithi152 ceph-mon[88186]: pgmap v444: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:07.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:07 smithi142 ceph-mon[112475]: pgmap v444: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:09.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:09 smithi152 ceph-mon[88186]: pgmap v445: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:09.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:09 smithi142 ceph-mon[112475]: pgmap v445: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:11.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:11 smithi152 ceph-mon[88186]: pgmap v446: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:11.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:11 smithi142 ceph-mon[112475]: pgmap v446: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:13.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:13 smithi152 ceph-mon[88186]: pgmap v447: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:13.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:13 smithi142 ceph-mon[112475]: pgmap v447: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:15.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:15 smithi142 ceph-mon[112475]: pgmap v448: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:15.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:03:15.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:15 smithi152 ceph-mon[88186]: pgmap v448: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:15.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:03:17.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.077+0000 7feab557c700 1 -- 172.21.15.142:0/920717998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0101770 msgr2=0x7feab01057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:17.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.077+0000 7feab557c700 1 --2- 172.21.15.142:0/920717998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0101770 0x7feab01057c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009910 tx=0x7feaa0009c20 comp rx=0 tx=0).stop 2024-03-29T05:03:17.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.077+0000 7feab557c700 1 -- 172.21.15.142:0/920717998 shutdown_connections 2024-03-29T05:03:17.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.077+0000 7feab557c700 1 --2- 172.21.15.142:0/920717998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0101770 0x7feab01057c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.077+0000 7feab557c700 1 --2- 172.21.15.142:0/920717998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feab0100dc0 0x7feab01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.077+0000 7feab557c700 1 -- 172.21.15.142:0/920717998 >> 172.21.15.142:0/920717998 conn(0x7feab00fc650 msgr2=0x7feab00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:17.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.077+0000 7feab557c700 1 -- 172.21.15.142:0/920717998 shutdown_connections 2024-03-29T05:03:17.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.077+0000 7feab557c700 1 -- 172.21.15.142:0/920717998 wait complete. 2024-03-29T05:03:17.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.078+0000 7feab557c700 1 Processor -- start 2024-03-29T05:03:17.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.078+0000 7feab557c700 1 -- start start 2024-03-29T05:03:17.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.078+0000 7feab557c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feab0100dc0 0x7feab0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:17.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.078+0000 7feab557c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0101770 0x7feab00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:17.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.078+0000 7feab557c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab0073b10 con 0x7feab0101770 2024-03-29T05:03:17.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.078+0000 7feab557c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab0073c50 con 0x7feab0100dc0 2024-03-29T05:03:17.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.078+0000 7feaae7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0101770 0x7feab00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:17.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.078+0000 7feaae7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0101770 0x7feab00735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:51704/0 (socket says 172.21.15.142:51704) 2024-03-29T05:03:17.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.078+0000 7feaae7fc700 1 -- 172.21.15.142:0/2090581210 learned_addr learned my addr 172.21.15.142:0/2090581210 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:17.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.079+0000 7feaaeffd700 1 --2- 172.21.15.142:0/2090581210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feab0100dc0 0x7feab0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:17.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.079+0000 7feaae7fc700 1 -- 172.21.15.142:0/2090581210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feab0100dc0 msgr2=0x7feab0073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:17.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.079+0000 7feaae7fc700 1 --2- 172.21.15.142:0/2090581210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feab0100dc0 0x7feab0073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.079+0000 7feaae7fc700 1 -- 172.21.15.142:0/2090581210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa00095f0 con 0x7feab0101770 2024-03-29T05:03:17.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.079+0000 7feaae7fc700 1 --2- 172.21.15.142:0/2090581210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0101770 0x7feab00735d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7feaa00059d0 tx=0x7feaa000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:17.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.079+0000 7fea97fff700 1 -- 172.21.15.142:0/2090581210 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaa001d070 con 0x7feab0101770 2024-03-29T05:03:17.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.079+0000 7fea97fff700 1 -- 172.21.15.142:0/2090581210 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feaa000b610 con 0x7feab0101770 2024-03-29T05:03:17.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.079+0000 7feab557c700 1 -- 172.21.15.142:0/2090581210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab0073ed0 con 0x7feab0101770 2024-03-29T05:03:17.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.079+0000 7fea97fff700 1 -- 172.21.15.142:0/2090581210 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaa0022470 con 0x7feab0101770 2024-03-29T05:03:17.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.079+0000 7feab557c700 1 -- 172.21.15.142:0/2090581210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab01a2c60 con 0x7feab0101770 2024-03-29T05:03:17.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.081+0000 7feab557c700 1 -- 172.21.15.142:0/2090581210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab01091c0 con 0x7feab0101770 2024-03-29T05:03:17.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.081+0000 7fea97fff700 1 -- 172.21.15.142:0/2090581210 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7feaa00226e0 con 0x7feab0101770 2024-03-29T05:03:17.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.081+0000 7fea97fff700 1 --2- 172.21.15.142:0/2090581210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fea9806dd60 0x7fea98070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:17.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.081+0000 7fea97fff700 1 -- 172.21.15.142:0/2090581210 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7feaa008f140 con 0x7feab0101770 2024-03-29T05:03:17.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.081+0000 7feaaeffd700 1 --2- 172.21.15.142:0/2090581210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fea9806dd60 0x7fea98070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:17.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.082+0000 7feaaeffd700 1 --2- 172.21.15.142:0/2090581210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fea9806dd60 0x7fea98070220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009c60 tx=0x7feaa4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:17.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.086+0000 7fea97fff700 1 -- 172.21.15.142:0/2090581210 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7feaa00585d0 con 0x7feab0101770 2024-03-29T05:03:17.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.301+0000 7feab557c700 1 -- 172.21.15.142:0/2090581210 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feab00611d0 con 0x7fea9806dd60 2024-03-29T05:03:17.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.303+0000 7fea97fff700 1 -- 172.21.15.142:0/2090581210 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7feab00611d0 con 0x7fea9806dd60 2024-03-29T05:03:17.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.305+0000 7feab557c700 1 -- 172.21.15.142:0/2090581210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fea9806dd60 msgr2=0x7fea98070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:17.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.305+0000 7feab557c700 1 --2- 172.21.15.142:0/2090581210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fea9806dd60 0x7fea98070220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009c60 tx=0x7feaa4009260 comp rx=0 tx=0).stop 2024-03-29T05:03:17.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.305+0000 7feab557c700 1 -- 172.21.15.142:0/2090581210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0101770 msgr2=0x7feab00735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:17.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.305+0000 7feab557c700 1 --2- 172.21.15.142:0/2090581210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0101770 0x7feab00735d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7feaa00059d0 tx=0x7feaa000f690 comp rx=0 tx=0).stop 2024-03-29T05:03:17.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.305+0000 7feab557c700 1 -- 172.21.15.142:0/2090581210 shutdown_connections 2024-03-29T05:03:17.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.305+0000 7feab557c700 1 --2- 172.21.15.142:0/2090581210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fea9806dd60 0x7fea98070220 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.305+0000 7feab557c700 1 --2- 172.21.15.142:0/2090581210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feab0100dc0 0x7feab0073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.305+0000 7feab557c700 1 --2- 172.21.15.142:0/2090581210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0101770 0x7feab00735d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.305+0000 7feab557c700 1 -- 172.21.15.142:0/2090581210 >> 172.21.15.142:0/2090581210 conn(0x7feab00fc650 msgr2=0x7feab00fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:17.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.305+0000 7feab557c700 1 -- 172.21.15.142:0/2090581210 shutdown_connections 2024-03-29T05:03:17.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.305+0000 7feab557c700 1 -- 172.21.15.142:0/2090581210 wait complete. 2024-03-29T05:03:17.325 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:03:17.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.412+0000 7fb01693b700 1 -- 172.21.15.142:0/2843149038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb010105a40 msgr2=0x7fb010105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:17.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.412+0000 7fb01693b700 1 --2- 172.21.15.142:0/2843149038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb010105a40 0x7fb010105e20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb000009910 tx=0x7fb000009c20 comp rx=0 tx=0).stop 2024-03-29T05:03:17.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.412+0000 7fb01693b700 1 -- 172.21.15.142:0/2843149038 shutdown_connections 2024-03-29T05:03:17.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.412+0000 7fb01693b700 1 --2- 172.21.15.142:0/2843149038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0100ffbd0 0x7fb010100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.412+0000 7fb01693b700 1 --2- 172.21.15.142:0/2843149038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb010105a40 0x7fb010105e20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.412+0000 7fb01693b700 1 -- 172.21.15.142:0/2843149038 >> 172.21.15.142:0/2843149038 conn(0x7fb0100755a0 msgr2=0x7fb0100759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:17.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.412+0000 7fb01693b700 1 -- 172.21.15.142:0/2843149038 shutdown_connections 2024-03-29T05:03:17.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.412+0000 7fb01693b700 1 -- 172.21.15.142:0/2843149038 wait complete. 2024-03-29T05:03:17.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.412+0000 7fb01693b700 1 Processor -- start 2024-03-29T05:03:17.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.413+0000 7fb01693b700 1 -- start start 2024-03-29T05:03:17.424 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.413+0000 7fb01693b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0100ffbd0 0x7fb01019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:17.424 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.413+0000 7fb01693b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010105a40 0x7fb01019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:17.424 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.413+0000 7fb01693b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01019d060 con 0x7fb0100ffbd0 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.413+0000 7fb01693b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb010196520 con 0x7fb010105a40 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.413+0000 7fb00ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0100ffbd0 0x7fb01019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.413+0000 7fb00ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0100ffbd0 0x7fb01019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:51722/0 (socket says 172.21.15.142:51722) 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.413+0000 7fb00ffff700 1 -- 172.21.15.142:0/3424871842 learned_addr learned my addr 172.21.15.142:0/3424871842 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.413+0000 7fb00f7fe700 1 --2- 172.21.15.142:0/3424871842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010105a40 0x7fb01019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.413+0000 7fb00ffff700 1 -- 172.21.15.142:0/3424871842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010105a40 msgr2=0x7fb01019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.413+0000 7fb00ffff700 1 --2- 172.21.15.142:0/3424871842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010105a40 0x7fb01019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.413+0000 7fb00ffff700 1 -- 172.21.15.142:0/3424871842 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0000095f0 con 0x7fb0100ffbd0 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.414+0000 7fb00ffff700 1 --2- 172.21.15.142:0/3424871842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0100ffbd0 0x7fb01019c440 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb000006010 tx=0x7fb00000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.414+0000 7fb00d7fa700 1 -- 172.21.15.142:0/3424871842 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb00001d070 con 0x7fb0100ffbd0 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.414+0000 7fb00d7fa700 1 -- 172.21.15.142:0/3424871842 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb000005dc0 con 0x7fb0100ffbd0 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.414+0000 7fb01693b700 1 -- 172.21.15.142:0/3424871842 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0101967a0 con 0x7fb0100ffbd0 2024-03-29T05:03:17.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.414+0000 7fb00d7fa700 1 -- 172.21.15.142:0/3424871842 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb000017400 con 0x7fb0100ffbd0 2024-03-29T05:03:17.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.414+0000 7fb01693b700 1 -- 172.21.15.142:0/3424871842 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb010196c90 con 0x7fb0100ffbd0 2024-03-29T05:03:17.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.415+0000 7fb01693b700 1 -- 172.21.15.142:0/3424871842 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb010109050 con 0x7fb0100ffbd0 2024-03-29T05:03:17.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.415+0000 7fb00d7fa700 1 -- 172.21.15.142:0/3424871842 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fb000003a60 con 0x7fb0100ffbd0 2024-03-29T05:03:17.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.416+0000 7fb00d7fa700 1 --2- 172.21.15.142:0/3424871842 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7faff806dd60 0x7faff8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:17.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.416+0000 7fb00f7fe700 1 --2- 172.21.15.142:0/3424871842 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7faff806dd60 0x7faff8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:17.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.416+0000 7fb00d7fa700 1 -- 172.21.15.142:0/3424871842 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb00008f050 con 0x7fb0100ffbd0 2024-03-29T05:03:17.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.416+0000 7fb00f7fe700 1 --2- 172.21.15.142:0/3424871842 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7faff806dd60 0x7faff8070220 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb010197d10 tx=0x7fb00400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:17.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.420+0000 7fb00d7fa700 1 -- 172.21.15.142:0/3424871842 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb00005c020 con 0x7fb0100ffbd0 2024-03-29T05:03:17.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.640+0000 7fb01693b700 1 -- 172.21.15.142:0/3424871842 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb01002cce0 con 0x7faff806dd60 2024-03-29T05:03:17.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.641+0000 7fb00d7fa700 1 -- 172.21.15.142:0/3424871842 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb01002cce0 con 0x7faff806dd60 2024-03-29T05:03:17.650 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:17 smithi142 ceph-mon[112475]: pgmap v449: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:17.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.644+0000 7fb01693b700 1 -- 172.21.15.142:0/3424871842 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7faff806dd60 msgr2=0x7faff8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:17.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.644+0000 7fb01693b700 1 --2- 172.21.15.142:0/3424871842 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7faff806dd60 0x7faff8070220 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb010197d10 tx=0x7fb00400b410 comp rx=0 tx=0).stop 2024-03-29T05:03:17.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.644+0000 7fb01693b700 1 -- 172.21.15.142:0/3424871842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0100ffbd0 msgr2=0x7fb01019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:17.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.644+0000 7fb01693b700 1 --2- 172.21.15.142:0/3424871842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0100ffbd0 0x7fb01019c440 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb000006010 tx=0x7fb00000bd70 comp rx=0 tx=0).stop 2024-03-29T05:03:17.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.644+0000 7fb01693b700 1 -- 172.21.15.142:0/3424871842 shutdown_connections 2024-03-29T05:03:17.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.644+0000 7fb01693b700 1 --2- 172.21.15.142:0/3424871842 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7faff806dd60 0x7faff8070220 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.644+0000 7fb01693b700 1 --2- 172.21.15.142:0/3424871842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0100ffbd0 0x7fb01019c440 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.644+0000 7fb01693b700 1 --2- 172.21.15.142:0/3424871842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb010105a40 0x7fb01019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.644+0000 7fb01693b700 1 -- 172.21.15.142:0/3424871842 >> 172.21.15.142:0/3424871842 conn(0x7fb0100755a0 msgr2=0x7fb0100685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:17.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.644+0000 7fb01693b700 1 -- 172.21.15.142:0/3424871842 shutdown_connections 2024-03-29T05:03:17.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.644+0000 7fb01693b700 1 -- 172.21.15.142:0/3424871842 wait complete. 2024-03-29T05:03:17.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.748+0000 7f9d62039700 1 -- 172.21.15.142:0/860047881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d5c0ffa40 msgr2=0x7f9d5c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:17.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.748+0000 7f9d62039700 1 --2- 172.21.15.142:0/860047881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d5c0ffa40 0x7f9d5c0ffec0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009910 tx=0x7f9d50009c20 comp rx=0 tx=0).stop 2024-03-29T05:03:17.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.748+0000 7f9d62039700 1 -- 172.21.15.142:0/860047881 shutdown_connections 2024-03-29T05:03:17.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.748+0000 7f9d62039700 1 --2- 172.21.15.142:0/860047881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d5c0ffa40 0x7f9d5c0ffec0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.748+0000 7f9d62039700 1 --2- 172.21.15.142:0/860047881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c105a40 0x7f9d5c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.748+0000 7f9d62039700 1 -- 172.21.15.142:0/860047881 >> 172.21.15.142:0/860047881 conn(0x7f9d5c0fb570 msgr2=0x7f9d5c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:17.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.748+0000 7f9d62039700 1 -- 172.21.15.142:0/860047881 shutdown_connections 2024-03-29T05:03:17.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.748+0000 7f9d62039700 1 -- 172.21.15.142:0/860047881 wait complete. 2024-03-29T05:03:17.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.749+0000 7f9d62039700 1 Processor -- start 2024-03-29T05:03:17.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.749+0000 7f9d62039700 1 -- start start 2024-03-29T05:03:17.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.749+0000 7f9d62039700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c0ffa40 0x7f9d5c197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:17.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.749+0000 7f9d62039700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d5c105a40 0x7f9d5c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:17.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.749+0000 7f9d62039700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c1980d0 con 0x7f9d5c105a40 2024-03-29T05:03:17.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.749+0000 7f9d62039700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c198240 con 0x7f9d5c0ffa40 2024-03-29T05:03:17.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d5affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d5c105a40 0x7f9d5c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:17.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d5affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d5c105a40 0x7f9d5c197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:51738/0 (socket says 172.21.15.142:51738) 2024-03-29T05:03:17.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d5affd700 1 -- 172.21.15.142:0/3013545364 learned_addr learned my addr 172.21.15.142:0/3013545364 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:17.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d5b7fe700 1 --2- 172.21.15.142:0/3013545364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c0ffa40 0x7f9d5c197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:17.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d5affd700 1 -- 172.21.15.142:0/3013545364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c0ffa40 msgr2=0x7f9d5c197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:17.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d5affd700 1 --2- 172.21.15.142:0/3013545364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c0ffa40 0x7f9d5c197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d5affd700 1 -- 172.21.15.142:0/3013545364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d500095f0 con 0x7f9d5c105a40 2024-03-29T05:03:17.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d5affd700 1 --2- 172.21.15.142:0/3013545364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d5c105a40 0x7f9d5c197ab0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9d50006010 tx=0x7f9d5000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:17.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d58ff9700 1 -- 172.21.15.142:0/3013545364 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5001d070 con 0x7f9d5c105a40 2024-03-29T05:03:17.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d58ff9700 1 -- 172.21.15.142:0/3013545364 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d50005dc0 con 0x7f9d5c105a40 2024-03-29T05:03:17.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d62039700 1 -- 172.21.15.142:0/3013545364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5c1a05e0 con 0x7f9d5c105a40 2024-03-29T05:03:17.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d58ff9700 1 -- 172.21.15.142:0/3013545364 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d50017400 con 0x7f9d5c105a40 2024-03-29T05:03:17.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.750+0000 7f9d62039700 1 -- 172.21.15.142:0/3013545364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c1a0a50 con 0x7f9d5c105a40 2024-03-29T05:03:17.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.751+0000 7f9d62039700 1 -- 172.21.15.142:0/3013545364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5c107fd0 con 0x7f9d5c105a40 2024-03-29T05:03:17.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.752+0000 7f9d58ff9700 1 -- 172.21.15.142:0/3013545364 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f9d50003a60 con 0x7f9d5c105a40 2024-03-29T05:03:17.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.752+0000 7f9d58ff9700 1 --2- 172.21.15.142:0/3013545364 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d4406dd60 0x7f9d44070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:17.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.752+0000 7f9d5b7fe700 1 --2- 172.21.15.142:0/3013545364 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d4406dd60 0x7f9d44070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:17.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.752+0000 7f9d58ff9700 1 -- 172.21.15.142:0/3013545364 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f9d5008f040 con 0x7f9d5c105a40 2024-03-29T05:03:17.762 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.753+0000 7f9d5b7fe700 1 --2- 172.21.15.142:0/3013545364 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d4406dd60 0x7f9d44070220 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c007be0 tx=0x7f9d4c0073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:17.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.757+0000 7f9d58ff9700 1 -- 172.21.15.142:0/3013545364 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9d500584e0 con 0x7f9d5c105a40 2024-03-29T05:03:17.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:17 smithi152 ceph-mon[88186]: pgmap v449: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:17.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.964+0000 7f9d62039700 1 -- 172.21.15.142:0/3013545364 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d5c02cce0 con 0x7f9d4406dd60 2024-03-29T05:03:17.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.975+0000 7f9d58ff9700 1 -- 172.21.15.142:0/3013545364 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9d5c02cce0 con 0x7f9d4406dd60 2024-03-29T05:03:17.984 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (15m) 8m ago 26m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (26m) 8m ago 26m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (25m) 7m ago 25m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (12m) 8m ago 26m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (12m) 7m ago 25m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (14m) 8m ago 26m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (8m) 8m ago 20m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (8m) 8m ago 21m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (7m) 7m ago 20m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (8m) 7m ago 21m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (16m) 8m ago 28m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (16m) 7m ago 25m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (13m) 8m ago 28m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:03:17.985 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (13m) 7m ago 25m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:03:17.986 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (15m) 8m ago 26m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:03:17.986 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (15m) 7m ago 25m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:03:17.986 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (11m) 8m ago 24m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:03:17.986 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (11m) 8m ago 24m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:03:17.986 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (10m) 8m ago 23m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:03:17.986 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (10m) 7m ago 23m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:03:17.986 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (9m) 7m ago 23m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:03:17.986 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (8m) 7m ago 22m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:03:17.986 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (15m) 8m ago 26m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:03:17.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.977+0000 7f9d62039700 1 -- 172.21.15.142:0/3013545364 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d4406dd60 msgr2=0x7f9d44070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:17.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.977+0000 7f9d62039700 1 --2- 172.21.15.142:0/3013545364 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d4406dd60 0x7f9d44070220 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c007be0 tx=0x7f9d4c0073d0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.977+0000 7f9d62039700 1 -- 172.21.15.142:0/3013545364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d5c105a40 msgr2=0x7f9d5c197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:17.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.977+0000 7f9d62039700 1 --2- 172.21.15.142:0/3013545364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d5c105a40 0x7f9d5c197ab0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9d50006010 tx=0x7f9d5000bd70 comp rx=0 tx=0).stop 2024-03-29T05:03:17.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.977+0000 7f9d62039700 1 -- 172.21.15.142:0/3013545364 shutdown_connections 2024-03-29T05:03:17.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.977+0000 7f9d62039700 1 --2- 172.21.15.142:0/3013545364 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d4406dd60 0x7f9d44070220 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.977+0000 7f9d62039700 1 --2- 172.21.15.142:0/3013545364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d5c0ffa40 0x7f9d5c197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.977+0000 7f9d62039700 1 --2- 172.21.15.142:0/3013545364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d5c105a40 0x7f9d5c197ab0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:17.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.977+0000 7f9d62039700 1 -- 172.21.15.142:0/3013545364 >> 172.21.15.142:0/3013545364 conn(0x7f9d5c0fb570 msgr2=0x7f9d5c0fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:17.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.977+0000 7f9d62039700 1 -- 172.21.15.142:0/3013545364 shutdown_connections 2024-03-29T05:03:17.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:17.977+0000 7f9d62039700 1 -- 172.21.15.142:0/3013545364 wait complete. 2024-03-29T05:03:18.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.079+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3497323232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0100440 msgr2=0x7f83a0104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:18.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.079+0000 7f83a7e8b700 1 --2- 172.21.15.142:0/3497323232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0100440 0x7f83a0104910 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8390009910 tx=0x7f8390009c20 comp rx=0 tx=0).stop 2024-03-29T05:03:18.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.079+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3497323232 shutdown_connections 2024-03-29T05:03:18.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.079+0000 7f83a7e8b700 1 --2- 172.21.15.142:0/3497323232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0100440 0x7f83a0104910 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.079+0000 7f83a7e8b700 1 --2- 172.21.15.142:0/3497323232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a00ffb20 0x7f83a00fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.079+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3497323232 >> 172.21.15.142:0/3497323232 conn(0x7f83a00fb560 msgr2=0x7f83a00fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:18.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.079+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3497323232 shutdown_connections 2024-03-29T05:03:18.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.079+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3497323232 wait complete. 2024-03-29T05:03:18.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.079+0000 7f83a7e8b700 1 Processor -- start 2024-03-29T05:03:18.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a7e8b700 1 -- start start 2024-03-29T05:03:18.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a7e8b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a00ffb20 0x7f83a019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:18.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a7e8b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0100440 0x7f83a019d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:18.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a7e8b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a019d9f0 con 0x7f83a0100440 2024-03-29T05:03:18.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a7e8b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a01a1780 con 0x7f83a00ffb20 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a5426700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0100440 0x7f83a019d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a5426700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0100440 0x7f83a019d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:51756/0 (socket says 172.21.15.142:51756) 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a5426700 1 -- 172.21.15.142:0/3201237475 learned_addr learned my addr 172.21.15.142:0/3201237475 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a5c27700 1 --2- 172.21.15.142:0/3201237475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a00ffb20 0x7f83a019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a5426700 1 -- 172.21.15.142:0/3201237475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a00ffb20 msgr2=0x7f83a019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a5426700 1 --2- 172.21.15.142:0/3201237475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a00ffb20 0x7f83a019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a5426700 1 -- 172.21.15.142:0/3201237475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83900095f0 con 0x7f83a0100440 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a5c27700 1 --2- 172.21.15.142:0/3201237475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a00ffb20 0x7f83a019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a5426700 1 --2- 172.21.15.142:0/3201237475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0100440 0x7f83a019d310 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f83900092e0 tx=0x7f839000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f8396ffd700 1 -- 172.21.15.142:0/3201237475 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f839001d070 con 0x7f83a0100440 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f8396ffd700 1 -- 172.21.15.142:0/3201237475 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8390005dc0 con 0x7f83a0100440 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3201237475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a01a1a00 con 0x7f83a0100440 2024-03-29T05:03:18.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f8396ffd700 1 -- 172.21.15.142:0/3201237475 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8390017400 con 0x7f83a0100440 2024-03-29T05:03:18.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.080+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3201237475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a01a1ef0 con 0x7f83a0100440 2024-03-29T05:03:18.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.081+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3201237475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a0102000 con 0x7f83a0100440 2024-03-29T05:03:18.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.082+0000 7f8396ffd700 1 -- 172.21.15.142:0/3201237475 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f839000f460 con 0x7f83a0100440 2024-03-29T05:03:18.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.082+0000 7f8396ffd700 1 --2- 172.21.15.142:0/3201237475 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f838c06dd60 0x7f838c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:18.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.082+0000 7f8396ffd700 1 -- 172.21.15.142:0/3201237475 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f839008f0e0 con 0x7f83a0100440 2024-03-29T05:03:18.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.082+0000 7f83a5c27700 1 --2- 172.21.15.142:0/3201237475 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f838c06dd60 0x7f838c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:18.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.083+0000 7f83a5c27700 1 --2- 172.21.15.142:0/3201237475 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f838c06dd60 0x7f838c070220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f839c005950 tx=0x7f839c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:18.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.087+0000 7f8396ffd700 1 -- 172.21.15.142:0/3201237475 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8390058580 con 0x7f83a0100440 2024-03-29T05:03:18.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.363+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3201237475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f83a002cc70 con 0x7f83a0100440 2024-03-29T05:03:18.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.364+0000 7f8396ffd700 1 -- 172.21.15.142:0/3201237475 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f839005bc40 con 0x7f83a0100440 2024-03-29T05:03:18.373 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:03:18.373 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:03:18.373 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:03:18.373 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:03:18.373 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:03:18.373 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:03:18.374 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:03:18.374 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:03:18.374 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:03:18.374 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:03:18.374 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:03:18.374 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:03:18.374 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:03:18.374 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:03:18.374 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:03:18.374 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:03:18.374 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:03:18.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.366+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3201237475 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f838c06dd60 msgr2=0x7f838c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:18.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.366+0000 7f83a7e8b700 1 --2- 172.21.15.142:0/3201237475 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f838c06dd60 0x7f838c070220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f839c005950 tx=0x7f839c0058e0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.367+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3201237475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0100440 msgr2=0x7f83a019d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:18.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.367+0000 7f83a7e8b700 1 --2- 172.21.15.142:0/3201237475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0100440 0x7f83a019d310 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f83900092e0 tx=0x7f839000be20 comp rx=0 tx=0).stop 2024-03-29T05:03:18.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.367+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3201237475 shutdown_connections 2024-03-29T05:03:18.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.367+0000 7f83a7e8b700 1 --2- 172.21.15.142:0/3201237475 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f838c06dd60 0x7f838c070220 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.367+0000 7f83a7e8b700 1 --2- 172.21.15.142:0/3201237475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83a00ffb20 0x7f83a019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.367+0000 7f83a7e8b700 1 --2- 172.21.15.142:0/3201237475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0100440 0x7f83a019d310 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.367+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3201237475 >> 172.21.15.142:0/3201237475 conn(0x7f83a00fb560 msgr2=0x7f83a0110f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:18.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.367+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3201237475 shutdown_connections 2024-03-29T05:03:18.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.367+0000 7f83a7e8b700 1 -- 172.21.15.142:0/3201237475 wait complete. 2024-03-29T05:03:18.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.472+0000 7f724566f700 1 -- 172.21.15.142:0/1832110795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7240100dc0 msgr2=0x7f72401011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:18.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.472+0000 7f724566f700 1 --2- 172.21.15.142:0/1832110795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7240100dc0 0x7f72401011a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f723000b0a0 tx=0x7f723000b3b0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.472+0000 7f724566f700 1 -- 172.21.15.142:0/1832110795 shutdown_connections 2024-03-29T05:03:18.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.472+0000 7f724566f700 1 --2- 172.21.15.142:0/1832110795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7240101770 0x7f72401057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.472+0000 7f724566f700 1 --2- 172.21.15.142:0/1832110795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7240100dc0 0x7f72401011a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.472+0000 7f724566f700 1 -- 172.21.15.142:0/1832110795 >> 172.21.15.142:0/1832110795 conn(0x7f72400fc650 msgr2=0x7f72400fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:18.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.472+0000 7f724566f700 1 -- 172.21.15.142:0/1832110795 shutdown_connections 2024-03-29T05:03:18.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.472+0000 7f724566f700 1 -- 172.21.15.142:0/1832110795 wait complete. 2024-03-29T05:03:18.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.472+0000 7f724566f700 1 Processor -- start 2024-03-29T05:03:18.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.472+0000 7f724566f700 1 -- start start 2024-03-29T05:03:18.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f724566f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7240100dc0 0x7f724019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:18.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f724566f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7240101770 0x7f724019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:18.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f724566f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724019d110 con 0x7f7240101770 2024-03-29T05:03:18.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f724566f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72401965c0 con 0x7f7240100dc0 2024-03-29T05:03:18.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f723e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7240101770 0x7f724019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:18.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f723e7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7240101770 0x7f724019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59438/0 (socket says 172.21.15.142:59438) 2024-03-29T05:03:18.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f723e7fc700 1 -- 172.21.15.142:0/3811669366 learned_addr learned my addr 172.21.15.142:0/3811669366 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:18.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f723effd700 1 --2- 172.21.15.142:0/3811669366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7240100dc0 0x7f724019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:18.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f723e7fc700 1 -- 172.21.15.142:0/3811669366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7240100dc0 msgr2=0x7f724019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:18.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f723e7fc700 1 --2- 172.21.15.142:0/3811669366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7240100dc0 0x7f724019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f723e7fc700 1 -- 172.21.15.142:0/3811669366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7230009d00 con 0x7f7240101770 2024-03-29T05:03:18.485 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f723e7fc700 1 --2- 172.21.15.142:0/3811669366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7240101770 0x7f724019ca80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f722800d900 tx=0x7f722800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:18.485 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f7237fff700 1 -- 172.21.15.142:0/3811669366 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7228009900 con 0x7f7240101770 2024-03-29T05:03:18.485 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f7237fff700 1 -- 172.21.15.142:0/3811669366 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7228010460 con 0x7f7240101770 2024-03-29T05:03:18.485 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f724566f700 1 -- 172.21.15.142:0/3811669366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72401968a0 con 0x7f7240101770 2024-03-29T05:03:18.485 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.473+0000 7f7237fff700 1 -- 172.21.15.142:0/3811669366 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f722800f5f0 con 0x7f7240101770 2024-03-29T05:03:18.485 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.474+0000 7f724566f700 1 -- 172.21.15.142:0/3811669366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7240196df0 con 0x7f7240101770 2024-03-29T05:03:18.485 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.474+0000 7f724566f700 1 -- 172.21.15.142:0/3811669366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7240109160 con 0x7f7240101770 2024-03-29T05:03:18.486 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.475+0000 7f7237fff700 1 -- 172.21.15.142:0/3811669366 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f72280105d0 con 0x7f7240101770 2024-03-29T05:03:18.486 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.475+0000 7f7237fff700 1 --2- 172.21.15.142:0/3811669366 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f722c06dd60 0x7f722c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:18.486 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.476+0000 7f723effd700 1 --2- 172.21.15.142:0/3811669366 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f722c06dd60 0x7f722c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:18.486 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.476+0000 7f7237fff700 1 -- 172.21.15.142:0/3811669366 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f722808d740 con 0x7f7240101770 2024-03-29T05:03:18.486 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.476+0000 7f723effd700 1 --2- 172.21.15.142:0/3811669366 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f722c06dd60 0x7f722c070220 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7230000f80 tx=0x7f72300060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:18.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.480+0000 7f7237fff700 1 -- 172.21.15.142:0/3811669366 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f722805a710 con 0x7f7240101770 2024-03-29T05:03:18.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:18 smithi142 ceph-mon[112475]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:18.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:18 smithi142 ceph-mon[112475]: from='client.34854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:18.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:18 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3201237475' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:03:18.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.720+0000 7f724566f700 1 -- 172.21.15.142:0/3811669366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f724004ea90 con 0x7f7240101770 2024-03-29T05:03:18.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.720+0000 7f7237fff700 1 -- 172.21.15.142:0/3811669366 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f722805a2a0 con 0x7f7240101770 2024-03-29T05:03:18.730 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:03:18.730 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:03:18.730 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:03:18.730 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:03:18.730 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:03:18.730 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:03:18.730 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:03:18.730 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:03:18.730 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:03:18.730 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:03:18.730 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:03:18.730 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:03:18.731 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:03:18.731 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:03:18.731 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:03:18.731 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:03:18.731 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:03:18.731 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:03:18.731 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:03:18.732 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:03:18.732 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:03:18.732 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:03:18.732 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:03:18.732 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:03:18.732 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:03:18.732 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:03:18.733 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:03:18.733 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:03:18.733 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:03:18.733 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:03:18.733 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:03:18.733 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:03:18.733 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:03:18.734 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:03:18.734 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:03:18.734 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:03:18.734 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:03:18.734 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:03:18.734 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:03:18.735 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:03:18.735 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:03:18.735 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:03:18.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.722+0000 7f724566f700 1 -- 172.21.15.142:0/3811669366 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f722c06dd60 msgr2=0x7f722c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:18.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.722+0000 7f724566f700 1 --2- 172.21.15.142:0/3811669366 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f722c06dd60 0x7f722c070220 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7230000f80 tx=0x7f72300060c0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.722+0000 7f724566f700 1 -- 172.21.15.142:0/3811669366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7240101770 msgr2=0x7f724019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:18.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.722+0000 7f724566f700 1 --2- 172.21.15.142:0/3811669366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7240101770 0x7f724019ca80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f722800d900 tx=0x7f722800dc10 comp rx=0 tx=0).stop 2024-03-29T05:03:18.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.722+0000 7f724566f700 1 -- 172.21.15.142:0/3811669366 shutdown_connections 2024-03-29T05:03:18.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.722+0000 7f724566f700 1 --2- 172.21.15.142:0/3811669366 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f722c06dd60 0x7f722c070220 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.722+0000 7f724566f700 1 --2- 172.21.15.142:0/3811669366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7240100dc0 0x7f724019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.722+0000 7f724566f700 1 --2- 172.21.15.142:0/3811669366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7240101770 0x7f724019ca80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.723+0000 7f724566f700 1 -- 172.21.15.142:0/3811669366 >> 172.21.15.142:0/3811669366 conn(0x7f72400fc650 msgr2=0x7f72400fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:18.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.723+0000 7f724566f700 1 -- 172.21.15.142:0/3811669366 shutdown_connections 2024-03-29T05:03:18.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.723+0000 7f724566f700 1 -- 172.21.15.142:0/3811669366 wait complete. 2024-03-29T05:03:18.738 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:03:18.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:18 smithi152 ceph-mon[88186]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:18.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:18 smithi152 ceph-mon[88186]: from='client.34854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:18.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:18 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3201237475' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:03:18.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.827+0000 7ffa25075700 1 -- 172.21.15.142:0/1532877203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200ffa60 msgr2=0x7ffa200ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:18.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.827+0000 7ffa25075700 1 --2- 172.21.15.142:0/1532877203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200ffa60 0x7ffa200ffee0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009910 tx=0x7ffa14009c20 comp rx=0 tx=0).stop 2024-03-29T05:03:18.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.827+0000 7ffa25075700 1 -- 172.21.15.142:0/1532877203 shutdown_connections 2024-03-29T05:03:18.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.827+0000 7ffa25075700 1 --2- 172.21.15.142:0/1532877203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200ffa60 0x7ffa200ffee0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.827+0000 7ffa25075700 1 --2- 172.21.15.142:0/1532877203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa20105a60 0x7ffa20105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.827+0000 7ffa25075700 1 -- 172.21.15.142:0/1532877203 >> 172.21.15.142:0/1532877203 conn(0x7ffa200fb570 msgr2=0x7ffa200fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:18.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.828+0000 7ffa25075700 1 -- 172.21.15.142:0/1532877203 shutdown_connections 2024-03-29T05:03:18.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.828+0000 7ffa25075700 1 -- 172.21.15.142:0/1532877203 wait complete. 2024-03-29T05:03:18.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.828+0000 7ffa25075700 1 Processor -- start 2024-03-29T05:03:18.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.828+0000 7ffa25075700 1 -- start start 2024-03-29T05:03:18.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa25075700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200ffa60 0x7ffa2019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:18.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa25075700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa20105a60 0x7ffa2019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:18.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa25075700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2019f3f0 con 0x7ffa200ffa60 2024-03-29T05:03:18.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa25075700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa201988a0 con 0x7ffa20105a60 2024-03-29T05:03:18.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa1ed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200ffa60 0x7ffa2019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:18.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa1ed9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200ffa60 0x7ffa2019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59460/0 (socket says 172.21.15.142:59460) 2024-03-29T05:03:18.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa1ed9d700 1 -- 172.21.15.142:0/3417549268 learned_addr learned my addr 172.21.15.142:0/3417549268 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:18.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa1e59c700 1 --2- 172.21.15.142:0/3417549268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa20105a60 0x7ffa2019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:18.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa1ed9d700 1 -- 172.21.15.142:0/3417549268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa20105a60 msgr2=0x7ffa2019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:18.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa1ed9d700 1 --2- 172.21.15.142:0/3417549268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa20105a60 0x7ffa2019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:18.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa1ed9d700 1 -- 172.21.15.142:0/3417549268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa140095f0 con 0x7ffa200ffa60 2024-03-29T05:03:18.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa1ed9d700 1 --2- 172.21.15.142:0/3417549268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200ffa60 0x7ffa2019e820 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000b4b0 tx=0x7ffa1000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:18.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa07fff700 1 -- 172.21.15.142:0/3417549268 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa10011800 con 0x7ffa200ffa60 2024-03-29T05:03:18.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa07fff700 1 -- 172.21.15.142:0/3417549268 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa10011e20 con 0x7ffa200ffa60 2024-03-29T05:03:18.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa25075700 1 -- 172.21.15.142:0/3417549268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa20198b80 con 0x7ffa200ffa60 2024-03-29T05:03:18.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa07fff700 1 -- 172.21.15.142:0/3417549268 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa1000f2e0 con 0x7ffa200ffa60 2024-03-29T05:03:18.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.829+0000 7ffa25075700 1 -- 172.21.15.142:0/3417549268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa201990d0 con 0x7ffa200ffa60 2024-03-29T05:03:18.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.830+0000 7ffa25075700 1 -- 172.21.15.142:0/3417549268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2004ea90 con 0x7ffa200ffa60 2024-03-29T05:03:18.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.831+0000 7ffa07fff700 1 -- 172.21.15.142:0/3417549268 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ffa10010380 con 0x7ffa200ffa60 2024-03-29T05:03:18.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.831+0000 7ffa07fff700 1 --2- 172.21.15.142:0/3417549268 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffa0806dd60 0x7ffa08070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:18.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.831+0000 7ffa07fff700 1 -- 172.21.15.142:0/3417549268 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ffa1008d810 con 0x7ffa200ffa60 2024-03-29T05:03:18.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.831+0000 7ffa1e59c700 1 --2- 172.21.15.142:0/3417549268 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffa0806dd60 0x7ffa08070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:18.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.832+0000 7ffa1e59c700 1 --2- 172.21.15.142:0/3417549268 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffa0806dd60 0x7ffa08070220 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa20104e20 tx=0x7ffa1401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:18.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:18.836+0000 7ffa07fff700 1 -- 172.21.15.142:0/3417549268 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ffa1005a760 con 0x7ffa200ffa60 2024-03-29T05:03:19.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.048+0000 7ffa25075700 1 -- 172.21.15.142:0/3417549268 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa20199590 con 0x7ffa0806dd60 2024-03-29T05:03:19.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.050+0000 7ffa07fff700 1 -- 172.21.15.142:0/3417549268 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ffa20199590 con 0x7ffa0806dd60 2024-03-29T05:03:19.059 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:03:19.059 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:03:19.059 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:03:19.059 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:03:19.059 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:03:19.059 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:03:19.060 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:03:19.060 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:03:19.060 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:03:19.060 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:03:19.060 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:03:19.060 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:03:19.060 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:03:19.060 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:03:19.060 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:03:19.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.052+0000 7ffa25075700 1 -- 172.21.15.142:0/3417549268 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffa0806dd60 msgr2=0x7ffa08070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:19.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.052+0000 7ffa25075700 1 --2- 172.21.15.142:0/3417549268 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffa0806dd60 0x7ffa08070220 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa20104e20 tx=0x7ffa1401a040 comp rx=0 tx=0).stop 2024-03-29T05:03:19.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.052+0000 7ffa25075700 1 -- 172.21.15.142:0/3417549268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200ffa60 msgr2=0x7ffa2019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:19.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.052+0000 7ffa25075700 1 --2- 172.21.15.142:0/3417549268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200ffa60 0x7ffa2019e820 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000b4b0 tx=0x7ffa1000b7c0 comp rx=0 tx=0).stop 2024-03-29T05:03:19.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.052+0000 7ffa25075700 1 -- 172.21.15.142:0/3417549268 shutdown_connections 2024-03-29T05:03:19.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.052+0000 7ffa25075700 1 --2- 172.21.15.142:0/3417549268 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffa0806dd60 0x7ffa08070220 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:19.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.052+0000 7ffa25075700 1 --2- 172.21.15.142:0/3417549268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200ffa60 0x7ffa2019e820 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:19.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.052+0000 7ffa25075700 1 --2- 172.21.15.142:0/3417549268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa20105a60 0x7ffa2019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:19.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.052+0000 7ffa25075700 1 -- 172.21.15.142:0/3417549268 >> 172.21.15.142:0/3417549268 conn(0x7ffa200fb570 msgr2=0x7ffa20068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:19.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.052+0000 7ffa25075700 1 -- 172.21.15.142:0/3417549268 shutdown_connections 2024-03-29T05:03:19.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.052+0000 7ffa25075700 1 -- 172.21.15.142:0/3417549268 wait complete. 2024-03-29T05:03:19.164 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.155+0000 7fb6ee420700 1 -- 172.21.15.142:0/1732694493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e8101790 msgr2=0x7fb6e81057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:19.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.155+0000 7fb6ee420700 1 --2- 172.21.15.142:0/1732694493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e8101790 0x7fb6e81057e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009910 tx=0x7fb6d8009c20 comp rx=0 tx=0).stop 2024-03-29T05:03:19.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.155+0000 7fb6ee420700 1 -- 172.21.15.142:0/1732694493 shutdown_connections 2024-03-29T05:03:19.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.155+0000 7fb6ee420700 1 --2- 172.21.15.142:0/1732694493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e8101790 0x7fb6e81057e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:19.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.155+0000 7fb6ee420700 1 --2- 172.21.15.142:0/1732694493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6e8100de0 0x7fb6e81011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:19.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.155+0000 7fb6ee420700 1 -- 172.21.15.142:0/1732694493 >> 172.21.15.142:0/1732694493 conn(0x7fb6e80fc6b0 msgr2=0x7fb6e80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:19.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.155+0000 7fb6ee420700 1 -- 172.21.15.142:0/1732694493 shutdown_connections 2024-03-29T05:03:19.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.155+0000 7fb6ee420700 1 -- 172.21.15.142:0/1732694493 wait complete. 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.156+0000 7fb6ee420700 1 Processor -- start 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.156+0000 7fb6ee420700 1 -- start start 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.156+0000 7fb6ee420700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e8100de0 0x7fb6e8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.156+0000 7fb6ee420700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6e8101790 0x7fb6e8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.156+0000 7fb6ee420700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e81994a0 con 0x7fb6e8100de0 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.156+0000 7fb6ee420700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e8199610 con 0x7fb6e8101790 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.156+0000 7fb6e7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e8100de0 0x7fb6e8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.156+0000 7fb6e7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e8100de0 0x7fb6e8198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59478/0 (socket says 172.21.15.142:59478) 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.156+0000 7fb6e7fff700 1 -- 172.21.15.142:0/820357619 learned_addr learned my addr 172.21.15.142:0/820357619 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.156+0000 7fb6e77fe700 1 --2- 172.21.15.142:0/820357619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6e8101790 0x7fb6e8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.157+0000 7fb6e7fff700 1 -- 172.21.15.142:0/820357619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6e8101790 msgr2=0x7fb6e8198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.157+0000 7fb6e7fff700 1 --2- 172.21.15.142:0/820357619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6e8101790 0x7fb6e8198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:19.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.157+0000 7fb6e7fff700 1 -- 172.21.15.142:0/820357619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d80095f0 con 0x7fb6e8100de0 2024-03-29T05:03:19.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.157+0000 7fb6e7fff700 1 --2- 172.21.15.142:0/820357619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e8100de0 0x7fb6e8198960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00c930 tx=0x7fb6dc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:19.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.157+0000 7fb6e57fa700 1 -- 172.21.15.142:0/820357619 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6dc007ad0 con 0x7fb6e8100de0 2024-03-29T05:03:19.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.157+0000 7fb6e57fa700 1 -- 172.21.15.142:0/820357619 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6dc00ce80 con 0x7fb6e8100de0 2024-03-29T05:03:19.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.157+0000 7fb6ee420700 1 -- 172.21.15.142:0/820357619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e819d490 con 0x7fb6e8100de0 2024-03-29T05:03:19.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.157+0000 7fb6e57fa700 1 -- 172.21.15.142:0/820357619 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6dc018630 con 0x7fb6e8100de0 2024-03-29T05:03:19.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.157+0000 7fb6ee420700 1 -- 172.21.15.142:0/820357619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e819d9b0 con 0x7fb6e8100de0 2024-03-29T05:03:19.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.158+0000 7fb6ee420700 1 -- 172.21.15.142:0/820357619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e8104f60 con 0x7fb6e8100de0 2024-03-29T05:03:19.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.158+0000 7fb6e57fa700 1 -- 172.21.15.142:0/820357619 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fb6dc007c30 con 0x7fb6e8100de0 2024-03-29T05:03:19.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.159+0000 7fb6e57fa700 1 --2- 172.21.15.142:0/820357619 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb6d006dd60 0x7fb6d0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:19.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.159+0000 7fb6e57fa700 1 -- 172.21.15.142:0/820357619 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb6dc08dc20 con 0x7fb6e8100de0 2024-03-29T05:03:19.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.159+0000 7fb6e77fe700 1 --2- 172.21.15.142:0/820357619 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb6d006dd60 0x7fb6d0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:19.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.159+0000 7fb6e77fe700 1 --2- 172.21.15.142:0/820357619 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb6d006dd60 0x7fb6d0070220 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb6e81998d0 tx=0x7fb6d801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:19.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.163+0000 7fb6e57fa700 1 -- 172.21.15.142:0/820357619 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb6dc05ab70 con 0x7fb6e8100de0 2024-03-29T05:03:19.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.444+0000 7fb6ee420700 1 -- 172.21.15.142:0/820357619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb6e802cc70 con 0x7fb6e8100de0 2024-03-29T05:03:19.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.444+0000 7fb6e57fa700 1 -- 172.21.15.142:0/820357619 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fb6dc05a700 con 0x7fb6e8100de0 2024-03-29T05:03:19.454 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:03:19.454 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:03:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:03:19.454 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:03:19.454 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:03:19.456 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.446+0000 7fb6ee420700 1 -- 172.21.15.142:0/820357619 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb6d006dd60 msgr2=0x7fb6d0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:19.456 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.446+0000 7fb6ee420700 1 --2- 172.21.15.142:0/820357619 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb6d006dd60 0x7fb6d0070220 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb6e81998d0 tx=0x7fb6d801a040 comp rx=0 tx=0).stop 2024-03-29T05:03:19.456 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.446+0000 7fb6ee420700 1 -- 172.21.15.142:0/820357619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e8100de0 msgr2=0x7fb6e8198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:19.456 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.446+0000 7fb6ee420700 1 --2- 172.21.15.142:0/820357619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e8100de0 0x7fb6e8198960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00c930 tx=0x7fb6dc00cc40 comp rx=0 tx=0).stop 2024-03-29T05:03:19.456 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.446+0000 7fb6ee420700 1 -- 172.21.15.142:0/820357619 shutdown_connections 2024-03-29T05:03:19.456 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.446+0000 7fb6ee420700 1 --2- 172.21.15.142:0/820357619 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb6d006dd60 0x7fb6d0070220 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:19.456 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.446+0000 7fb6ee420700 1 --2- 172.21.15.142:0/820357619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6e8100de0 0x7fb6e8198960 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:19.456 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.446+0000 7fb6ee420700 1 --2- 172.21.15.142:0/820357619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6e8101790 0x7fb6e8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:19.456 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.446+0000 7fb6ee420700 1 -- 172.21.15.142:0/820357619 >> 172.21.15.142:0/820357619 conn(0x7fb6e80fc6b0 msgr2=0x7fb6e80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:19.456 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.447+0000 7fb6ee420700 1 -- 172.21.15.142:0/820357619 shutdown_connections 2024-03-29T05:03:19.456 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:19.447+0000 7fb6ee420700 1 -- 172.21.15.142:0/820357619 wait complete. 2024-03-29T05:03:19.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:19 smithi152 ceph-mon[88186]: from='client.34858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:19.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:19 smithi152 ceph-mon[88186]: pgmap v450: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:19.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:19 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3811669366' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:03:19.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:19 smithi142 ceph-mon[112475]: from='client.34858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:19.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:19 smithi142 ceph-mon[112475]: pgmap v450: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:19.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:19 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3811669366' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:03:20.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:20 smithi142 ceph-mon[112475]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:20.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:20 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/820357619' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:03:20.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:20 smithi152 ceph-mon[88186]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:20.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:20 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/820357619' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:03:21.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:21 smithi152 ceph-mon[88186]: pgmap v451: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:21.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:21 smithi142 ceph-mon[112475]: pgmap v451: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:23.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:23 smithi152 ceph-mon[88186]: pgmap v452: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:23.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:23 smithi142 ceph-mon[112475]: pgmap v452: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:25.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:25 smithi152 ceph-mon[88186]: pgmap v453: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:25.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:25 smithi142 ceph-mon[112475]: pgmap v453: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:27.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:27 smithi152 ceph-mon[88186]: pgmap v454: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:27.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:27 smithi142 ceph-mon[112475]: pgmap v454: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:29.682 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:29 smithi142 ceph-mon[112475]: pgmap v455: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:29.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:29 smithi152 ceph-mon[88186]: pgmap v455: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:30.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:03:30.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:03:31.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:31 smithi152 ceph-mon[88186]: pgmap v456: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:31.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:31 smithi142 ceph-mon[112475]: pgmap v456: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:33.549 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:33 smithi152 ceph-mon[88186]: pgmap v457: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:33.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:33 smithi142 ceph-mon[112475]: pgmap v457: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:35.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:35 smithi152 ceph-mon[88186]: pgmap v458: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:35.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:35 smithi142 ceph-mon[112475]: pgmap v458: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:37.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:37 smithi152 ceph-mon[88186]: pgmap v459: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:37.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:37 smithi142 ceph-mon[112475]: pgmap v459: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:39.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:39 smithi152 ceph-mon[88186]: pgmap v460: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:39.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:39 smithi142 ceph-mon[112475]: pgmap v460: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:41.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:41 smithi152 ceph-mon[88186]: pgmap v461: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:41.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:41 smithi142 ceph-mon[112475]: pgmap v461: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:43.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:43 smithi152 ceph-mon[88186]: pgmap v462: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:43.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:43 smithi142 ceph-mon[112475]: pgmap v462: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:45.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:45 smithi152 ceph-mon[88186]: pgmap v463: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:45.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:03:45.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:45 smithi142 ceph-mon[112475]: pgmap v463: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:45.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:03:47.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:47 smithi152 ceph-mon[88186]: pgmap v464: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:47.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:47 smithi142 ceph-mon[112475]: pgmap v464: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:49.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.563+0000 7f18f7388700 1 -- 172.21.15.142:0/3039476143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18f0100dc0 msgr2=0x7f18f01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:49.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.563+0000 7f18f7388700 1 --2- 172.21.15.142:0/3039476143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18f0100dc0 0x7f18f01011a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009910 tx=0x7f18ec009c20 comp rx=0 tx=0).stop 2024-03-29T05:03:49.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.563+0000 7f18f7388700 1 -- 172.21.15.142:0/3039476143 shutdown_connections 2024-03-29T05:03:49.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.563+0000 7f18f7388700 1 --2- 172.21.15.142:0/3039476143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18f0101770 0x7f18f01057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:49.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.563+0000 7f18f7388700 1 --2- 172.21.15.142:0/3039476143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18f0100dc0 0x7f18f01011a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:49.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.563+0000 7f18f7388700 1 -- 172.21.15.142:0/3039476143 >> 172.21.15.142:0/3039476143 conn(0x7f18f00fc650 msgr2=0x7f18f00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:49.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.563+0000 7f18f7388700 1 -- 172.21.15.142:0/3039476143 shutdown_connections 2024-03-29T05:03:49.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.563+0000 7f18f7388700 1 -- 172.21.15.142:0/3039476143 wait complete. 2024-03-29T05:03:49.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.563+0000 7f18f7388700 1 Processor -- start 2024-03-29T05:03:49.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.563+0000 7f18f7388700 1 -- start start 2024-03-29T05:03:49.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f7388700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18f0100dc0 0x7f18f0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:49.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f7388700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18f0101770 0x7f18f00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:49.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f7388700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f0073b10 con 0x7f18f0100dc0 2024-03-29T05:03:49.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f7388700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f0073c50 con 0x7f18f0101770 2024-03-29T05:03:49.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f5124700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18f0100dc0 0x7f18f0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:49.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f5124700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18f0100dc0 0x7f18f0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58710/0 (socket says 172.21.15.142:58710) 2024-03-29T05:03:49.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f5124700 1 -- 172.21.15.142:0/147781066 learned_addr learned my addr 172.21.15.142:0/147781066 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:49.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f4923700 1 --2- 172.21.15.142:0/147781066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18f0101770 0x7f18f00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:49.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f5124700 1 -- 172.21.15.142:0/147781066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18f0101770 msgr2=0x7f18f00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:49.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f5124700 1 --2- 172.21.15.142:0/147781066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18f0101770 0x7f18f00735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:49.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f5124700 1 -- 172.21.15.142:0/147781066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec0095f0 con 0x7f18f0100dc0 2024-03-29T05:03:49.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f4923700 1 --2- 172.21.15.142:0/147781066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18f0101770 0x7f18f00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:03:49.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f5124700 1 --2- 172.21.15.142:0/147781066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18f0100dc0 0x7f18f0073090 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f18ec006010 tx=0x7f18ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:49.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18e67fc700 1 -- 172.21.15.142:0/147781066 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18ec01d070 con 0x7f18f0100dc0 2024-03-29T05:03:49.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18e67fc700 1 -- 172.21.15.142:0/147781066 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18ec00f460 con 0x7f18f0100dc0 2024-03-29T05:03:49.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f7388700 1 -- 172.21.15.142:0/147781066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f0073ed0 con 0x7f18f0100dc0 2024-03-29T05:03:49.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18e67fc700 1 -- 172.21.15.142:0/147781066 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18ec017630 con 0x7f18f0100dc0 2024-03-29T05:03:49.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.564+0000 7f18f7388700 1 -- 172.21.15.142:0/147781066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f01a2c90 con 0x7f18f0100dc0 2024-03-29T05:03:49.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.565+0000 7f18f7388700 1 -- 172.21.15.142:0/147781066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f0109160 con 0x7f18f0100dc0 2024-03-29T05:03:49.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.566+0000 7f18e67fc700 1 -- 172.21.15.142:0/147781066 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f18ec00f5d0 con 0x7f18f0100dc0 2024-03-29T05:03:49.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.566+0000 7f18e67fc700 1 --2- 172.21.15.142:0/147781066 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f18dc06dd60 0x7f18dc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:49.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.567+0000 7f18e67fc700 1 -- 172.21.15.142:0/147781066 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f18ec08f0c0 con 0x7f18f0100dc0 2024-03-29T05:03:49.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.567+0000 7f18f4923700 1 --2- 172.21.15.142:0/147781066 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f18dc06dd60 0x7f18dc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:49.578 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.567+0000 7f18f4923700 1 --2- 172.21.15.142:0/147781066 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f18dc06dd60 0x7f18dc070220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f18f0074880 tx=0x7f18e000b2c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:49.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.572+0000 7f18e67fc700 1 -- 172.21.15.142:0/147781066 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f18ec058560 con 0x7f18f0100dc0 2024-03-29T05:03:49.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.785+0000 7f18f7388700 1 -- 172.21.15.142:0/147781066 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18f002cce0 con 0x7f18dc06dd60 2024-03-29T05:03:49.795 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:49 smithi142 ceph-mon[112475]: pgmap v465: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:49.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.786+0000 7f18e67fc700 1 -- 172.21.15.142:0/147781066 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f18f002cce0 con 0x7f18dc06dd60 2024-03-29T05:03:49.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.789+0000 7f18f7388700 1 -- 172.21.15.142:0/147781066 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f18dc06dd60 msgr2=0x7f18dc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:49.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.789+0000 7f18f7388700 1 --2- 172.21.15.142:0/147781066 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f18dc06dd60 0x7f18dc070220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f18f0074880 tx=0x7f18e000b2c0 comp rx=0 tx=0).stop 2024-03-29T05:03:49.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.789+0000 7f18f7388700 1 -- 172.21.15.142:0/147781066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18f0100dc0 msgr2=0x7f18f0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:49.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.789+0000 7f18f7388700 1 --2- 172.21.15.142:0/147781066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18f0100dc0 0x7f18f0073090 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f18ec006010 tx=0x7f18ec00be20 comp rx=0 tx=0).stop 2024-03-29T05:03:49.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.789+0000 7f18f7388700 1 -- 172.21.15.142:0/147781066 shutdown_connections 2024-03-29T05:03:49.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.789+0000 7f18f7388700 1 --2- 172.21.15.142:0/147781066 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f18dc06dd60 0x7f18dc070220 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:49.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.789+0000 7f18f7388700 1 --2- 172.21.15.142:0/147781066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18f0100dc0 0x7f18f0073090 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:49.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.789+0000 7f18f7388700 1 --2- 172.21.15.142:0/147781066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18f0101770 0x7f18f00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:49.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.789+0000 7f18f7388700 1 -- 172.21.15.142:0/147781066 >> 172.21.15.142:0/147781066 conn(0x7f18f00fc650 msgr2=0x7f18f00fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:49.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.789+0000 7f18f7388700 1 -- 172.21.15.142:0/147781066 shutdown_connections 2024-03-29T05:03:49.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.789+0000 7f18f7388700 1 -- 172.21.15.142:0/147781066 wait complete. 2024-03-29T05:03:49.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:49 smithi152 ceph-mon[88186]: pgmap v465: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:49.809 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:03:49.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.893+0000 7f6101010700 1 -- 172.21.15.142:0/3221176167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc101770 msgr2=0x7f60fc1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:49.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.893+0000 7f6101010700 1 --2- 172.21.15.142:0/3221176167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc101770 0x7f60fc1057c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009910 tx=0x7f60ec009c20 comp rx=0 tx=0).stop 2024-03-29T05:03:49.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.893+0000 7f6101010700 1 -- 172.21.15.142:0/3221176167 shutdown_connections 2024-03-29T05:03:49.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.893+0000 7f6101010700 1 --2- 172.21.15.142:0/3221176167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc101770 0x7f60fc1057c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:49.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.893+0000 7f6101010700 1 --2- 172.21.15.142:0/3221176167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60fc100dc0 0x7f60fc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:49.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.893+0000 7f6101010700 1 -- 172.21.15.142:0/3221176167 >> 172.21.15.142:0/3221176167 conn(0x7f60fc0fc650 msgr2=0x7f60fc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:49.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.893+0000 7f6101010700 1 -- 172.21.15.142:0/3221176167 shutdown_connections 2024-03-29T05:03:49.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.893+0000 7f6101010700 1 -- 172.21.15.142:0/3221176167 wait complete. 2024-03-29T05:03:49.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f6101010700 1 Processor -- start 2024-03-29T05:03:49.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f6101010700 1 -- start start 2024-03-29T05:03:49.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f6101010700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc100dc0 0x7f60fc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:49.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f6101010700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60fc101770 0x7f60fc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:49.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f6101010700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc073b10 con 0x7f60fc100dc0 2024-03-29T05:03:49.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f6101010700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc073c50 con 0x7f60fc101770 2024-03-29T05:03:49.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f60fad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc100dc0 0x7f60fc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:49.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f60fad9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc100dc0 0x7f60fc073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58732/0 (socket says 172.21.15.142:58732) 2024-03-29T05:03:49.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f60fad9d700 1 -- 172.21.15.142:0/2103170818 learned_addr learned my addr 172.21.15.142:0/2103170818 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:49.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f60fa59c700 1 --2- 172.21.15.142:0/2103170818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60fc101770 0x7f60fc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:49.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f60fad9d700 1 -- 172.21.15.142:0/2103170818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60fc101770 msgr2=0x7f60fc0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:49.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f60fad9d700 1 --2- 172.21.15.142:0/2103170818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60fc101770 0x7f60fc0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:49.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f60fad9d700 1 -- 172.21.15.142:0/2103170818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec0095f0 con 0x7f60fc100dc0 2024-03-29T05:03:49.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.894+0000 7f60fa59c700 1 --2- 172.21.15.142:0/2103170818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60fc101770 0x7f60fc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:03:49.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.895+0000 7f60fad9d700 1 --2- 172.21.15.142:0/2103170818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc100dc0 0x7f60fc073090 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f60f000c930 tx=0x7f60f000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:49.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.895+0000 7f60e3fff700 1 -- 172.21.15.142:0/2103170818 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60f0007ad0 con 0x7f60fc100dc0 2024-03-29T05:03:49.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.895+0000 7f6101010700 1 -- 172.21.15.142:0/2103170818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60fc1a2960 con 0x7f60fc100dc0 2024-03-29T05:03:49.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.895+0000 7f60e3fff700 1 -- 172.21.15.142:0/2103170818 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60f0007c30 con 0x7f60fc100dc0 2024-03-29T05:03:49.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.895+0000 7f60e3fff700 1 -- 172.21.15.142:0/2103170818 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60f0018610 con 0x7f60fc100dc0 2024-03-29T05:03:49.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.895+0000 7f6101010700 1 -- 172.21.15.142:0/2103170818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60fc1a2cc0 con 0x7f60fc100dc0 2024-03-29T05:03:49.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.896+0000 7f6101010700 1 -- 172.21.15.142:0/2103170818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60fc1091c0 con 0x7f60fc100dc0 2024-03-29T05:03:49.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.896+0000 7f60e3fff700 1 -- 172.21.15.142:0/2103170818 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f60f000f450 con 0x7f60fc100dc0 2024-03-29T05:03:49.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.897+0000 7f60e3fff700 1 --2- 172.21.15.142:0/2103170818 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f60e406dd60 0x7f60e4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:49.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.897+0000 7f60e3fff700 1 -- 172.21.15.142:0/2103170818 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f60f008dec0 con 0x7f60fc100dc0 2024-03-29T05:03:49.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.897+0000 7f60fa59c700 1 --2- 172.21.15.142:0/2103170818 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f60e406dd60 0x7f60e4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:49.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.897+0000 7f60fa59c700 1 --2- 172.21.15.142:0/2103170818 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f60e406dd60 0x7f60e4070220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f60ec006010 tx=0x7f60ec005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:49.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:49.901+0000 7f60e3fff700 1 -- 172.21.15.142:0/2103170818 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f60f0057280 con 0x7f60fc100dc0 2024-03-29T05:03:50.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.116+0000 7f6101010700 1 -- 172.21.15.142:0/2103170818 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60fc0611d0 con 0x7f60e406dd60 2024-03-29T05:03:50.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.117+0000 7f60e3fff700 1 -- 172.21.15.142:0/2103170818 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f60fc0611d0 con 0x7f60e406dd60 2024-03-29T05:03:50.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.119+0000 7f6101010700 1 -- 172.21.15.142:0/2103170818 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f60e406dd60 msgr2=0x7f60e4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:50.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.119+0000 7f6101010700 1 --2- 172.21.15.142:0/2103170818 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f60e406dd60 0x7f60e4070220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f60ec006010 tx=0x7f60ec005960 comp rx=0 tx=0).stop 2024-03-29T05:03:50.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.119+0000 7f6101010700 1 -- 172.21.15.142:0/2103170818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc100dc0 msgr2=0x7f60fc073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:50.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.119+0000 7f6101010700 1 --2- 172.21.15.142:0/2103170818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc100dc0 0x7f60fc073090 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f60f000c930 tx=0x7f60f000ccf0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.119+0000 7f6101010700 1 -- 172.21.15.142:0/2103170818 shutdown_connections 2024-03-29T05:03:50.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.119+0000 7f6101010700 1 --2- 172.21.15.142:0/2103170818 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f60e406dd60 0x7f60e4070220 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.119+0000 7f6101010700 1 --2- 172.21.15.142:0/2103170818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60fc100dc0 0x7f60fc073090 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.119+0000 7f6101010700 1 --2- 172.21.15.142:0/2103170818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60fc101770 0x7f60fc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.119+0000 7f6101010700 1 -- 172.21.15.142:0/2103170818 >> 172.21.15.142:0/2103170818 conn(0x7f60fc0fc650 msgr2=0x7f60fc0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:50.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.119+0000 7f6101010700 1 -- 172.21.15.142:0/2103170818 shutdown_connections 2024-03-29T05:03:50.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.119+0000 7f6101010700 1 -- 172.21.15.142:0/2103170818 wait complete. 2024-03-29T05:03:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.223+0000 7fc890997700 1 -- 172.21.15.142:0/1798384652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c105a20 msgr2=0x7fc88c105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.223+0000 7fc890997700 1 --2- 172.21.15.142:0/1798384652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c105a20 0x7fc88c105e00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc88400b0a0 tx=0x7fc88400b3b0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.223+0000 7fc890997700 1 -- 172.21.15.142:0/1798384652 shutdown_connections 2024-03-29T05:03:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.223+0000 7fc890997700 1 --2- 172.21.15.142:0/1798384652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc88c0ffa20 0x7fc88c0ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.223+0000 7fc890997700 1 --2- 172.21.15.142:0/1798384652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c105a20 0x7fc88c105e00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.223+0000 7fc890997700 1 -- 172.21.15.142:0/1798384652 >> 172.21.15.142:0/1798384652 conn(0x7fc88c0fb530 msgr2=0x7fc88c0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.223+0000 7fc890997700 1 -- 172.21.15.142:0/1798384652 shutdown_connections 2024-03-29T05:03:50.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.223+0000 7fc890997700 1 -- 172.21.15.142:0/1798384652 wait complete. 2024-03-29T05:03:50.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc890997700 1 Processor -- start 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc890997700 1 -- start start 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc890997700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c0ffa20 0x7fc88c1966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc890997700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc88c105a20 0x7fc88c196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc890997700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c197270 con 0x7fc88c0ffa20 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc890997700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c199fa0 con 0x7fc88c105a20 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc88a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c0ffa20 0x7fc88c1966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc88a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c0ffa20 0x7fc88c1966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58750/0 (socket says 172.21.15.142:58750) 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc88a59c700 1 -- 172.21.15.142:0/2126106992 learned_addr learned my addr 172.21.15.142:0/2126106992 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc889d9b700 1 --2- 172.21.15.142:0/2126106992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc88c105a20 0x7fc88c196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc88a59c700 1 -- 172.21.15.142:0/2126106992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc88c105a20 msgr2=0x7fc88c196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc88a59c700 1 --2- 172.21.15.142:0/2126106992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc88c105a20 0x7fc88c196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc88a59c700 1 -- 172.21.15.142:0/2126106992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc884009d00 con 0x7fc88c0ffa20 2024-03-29T05:03:50.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.224+0000 7fc88a59c700 1 --2- 172.21.15.142:0/2126106992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c0ffa20 0x7fc88c1966a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc884000f80 tx=0x7fc88400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:50.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.225+0000 7fc87b7fe700 1 -- 172.21.15.142:0/2126106992 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc88400e040 con 0x7fc88c0ffa20 2024-03-29T05:03:50.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.225+0000 7fc87b7fe700 1 -- 172.21.15.142:0/2126106992 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc884003de0 con 0x7fc88c0ffa20 2024-03-29T05:03:50.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.225+0000 7fc890997700 1 -- 172.21.15.142:0/2126106992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c19a220 con 0x7fc88c0ffa20 2024-03-29T05:03:50.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.225+0000 7fc87b7fe700 1 -- 172.21.15.142:0/2126106992 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc88401d790 con 0x7fc88c0ffa20 2024-03-29T05:03:50.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.225+0000 7fc890997700 1 -- 172.21.15.142:0/2126106992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c19a710 con 0x7fc88c0ffa20 2024-03-29T05:03:50.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.225+0000 7fc890997700 1 -- 172.21.15.142:0/2126106992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c04ea90 con 0x7fc88c0ffa20 2024-03-29T05:03:50.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.226+0000 7fc87b7fe700 1 -- 172.21.15.142:0/2126106992 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fc884004040 con 0x7fc88c0ffa20 2024-03-29T05:03:50.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.226+0000 7fc87b7fe700 1 --2- 172.21.15.142:0/2126106992 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc87406dd60 0x7fc874070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:50.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.227+0000 7fc87b7fe700 1 -- 172.21.15.142:0/2126106992 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fc88408f4c0 con 0x7fc88c0ffa20 2024-03-29T05:03:50.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.227+0000 7fc889d9b700 1 --2- 172.21.15.142:0/2126106992 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc87406dd60 0x7fc874070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:50.237 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.227+0000 7fc889d9b700 1 --2- 172.21.15.142:0/2126106992 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc87406dd60 0x7fc874070220 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc88c197cc0 tx=0x7fc8800094f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:50.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.231+0000 7fc87b7fe700 1 -- 172.21.15.142:0/2126106992 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc8840588e0 con 0x7fc88c0ffa20 2024-03-29T05:03:50.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.435+0000 7fc890997700 1 -- 172.21.15.142:0/2126106992 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc88c19ab00 con 0x7fc87406dd60 2024-03-29T05:03:50.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.446+0000 7fc87b7fe700 1 -- 172.21.15.142:0/2126106992 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc88c19ab00 con 0x7fc87406dd60 2024-03-29T05:03:50.455 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:03:50.455 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (15m) 8m ago 27m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:03:50.455 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (27m) 8m ago 27m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (26m) 8m ago 26m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (12m) 8m ago 27m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (12m) 8m ago 26m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (15m) 8m ago 26m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (8m) 8m ago 21m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (9m) 8m ago 21m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (8m) 8m ago 21m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (8m) 8m ago 21m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (17m) 8m ago 28m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (16m) 8m ago 26m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (13m) 8m ago 28m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (13m) 8m ago 26m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (16m) 8m ago 27m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (16m) 8m ago 26m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:03:50.456 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (12m) 8m ago 25m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:03:50.457 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (11m) 8m ago 24m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:03:50.457 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (11m) 8m ago 24m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:03:50.457 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (10m) 8m ago 24m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:03:50.457 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (10m) 8m ago 23m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:03:50.457 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (9m) 8m ago 23m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:03:50.457 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (15m) 8m ago 26m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:03:50.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.448+0000 7fc890997700 1 -- 172.21.15.142:0/2126106992 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc87406dd60 msgr2=0x7fc874070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:50.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.448+0000 7fc890997700 1 --2- 172.21.15.142:0/2126106992 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc87406dd60 0x7fc874070220 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc88c197cc0 tx=0x7fc8800094f0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.448+0000 7fc890997700 1 -- 172.21.15.142:0/2126106992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c0ffa20 msgr2=0x7fc88c1966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:50.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.448+0000 7fc890997700 1 --2- 172.21.15.142:0/2126106992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c0ffa20 0x7fc88c1966a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc884000f80 tx=0x7fc88400cdd0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.448+0000 7fc890997700 1 -- 172.21.15.142:0/2126106992 shutdown_connections 2024-03-29T05:03:50.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.448+0000 7fc890997700 1 --2- 172.21.15.142:0/2126106992 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc87406dd60 0x7fc874070220 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.448+0000 7fc890997700 1 --2- 172.21.15.142:0/2126106992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c0ffa20 0x7fc88c1966a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.448+0000 7fc890997700 1 --2- 172.21.15.142:0/2126106992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc88c105a20 0x7fc88c196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.448+0000 7fc890997700 1 -- 172.21.15.142:0/2126106992 >> 172.21.15.142:0/2126106992 conn(0x7fc88c0fb530 msgr2=0x7fc88c103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:50.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.448+0000 7fc890997700 1 -- 172.21.15.142:0/2126106992 shutdown_connections 2024-03-29T05:03:50.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.448+0000 7fc890997700 1 -- 172.21.15.142:0/2126106992 wait complete. 2024-03-29T05:03:50.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.552+0000 7fbde4bed700 1 -- 172.21.15.142:0/264254937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0101c70 msgr2=0x7fbde0102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:50.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.552+0000 7fbde4bed700 1 --2- 172.21.15.142:0/264254937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0101c70 0x7fbde0102050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009910 tx=0x7fbdd0009c20 comp rx=0 tx=0).stop 2024-03-29T05:03:50.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.552+0000 7fbde4bed700 1 -- 172.21.15.142:0/264254937 shutdown_connections 2024-03-29T05:03:50.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.552+0000 7fbde4bed700 1 --2- 172.21.15.142:0/264254937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbde0102620 0x7fbde010ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.563 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.552+0000 7fbde4bed700 1 --2- 172.21.15.142:0/264254937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0101c70 0x7fbde0102050 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.552+0000 7fbde4bed700 1 -- 172.21.15.142:0/264254937 >> 172.21.15.142:0/264254937 conn(0x7fbde00fb530 msgr2=0x7fbde00fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.552+0000 7fbde4bed700 1 -- 172.21.15.142:0/264254937 shutdown_connections 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.552+0000 7fbde4bed700 1 -- 172.21.15.142:0/264254937 wait complete. 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.552+0000 7fbde4bed700 1 Processor -- start 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbde4bed700 1 -- start start 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbde4bed700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbde0101c70 0x7fbde019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbde4bed700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0102620 0x7fbde019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbde4bed700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde019d0c0 con 0x7fbde0102620 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbde4bed700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde0196570 con 0x7fbde0101c70 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbdddd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0102620 0x7fbde019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbdddd9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0102620 0x7fbde019ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58776/0 (socket says 172.21.15.142:58776) 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbdddd9b700 1 -- 172.21.15.142:0/1077724687 learned_addr learned my addr 172.21.15.142:0/1077724687 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:50.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbdde59c700 1 --2- 172.21.15.142:0/1077724687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbde0101c70 0x7fbde019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:50.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbdddd9b700 1 -- 172.21.15.142:0/1077724687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbde0101c70 msgr2=0x7fbde019c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:50.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbdddd9b700 1 --2- 172.21.15.142:0/1077724687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbde0101c70 0x7fbde019c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbdddd9b700 1 -- 172.21.15.142:0/1077724687 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd00095f0 con 0x7fbde0102620 2024-03-29T05:03:50.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbdde59c700 1 --2- 172.21.15.142:0/1077724687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbde0101c70 0x7fbde019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:03:50.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbdddd9b700 1 --2- 172.21.15.142:0/1077724687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0102620 0x7fbde019ca30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400e8f0 tx=0x7fbdd400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:50.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.553+0000 7fbdcf7fe700 1 -- 172.21.15.142:0/1077724687 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd4009790 con 0x7fbde0102620 2024-03-29T05:03:50.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.554+0000 7fbdcf7fe700 1 -- 172.21.15.142:0/1077724687 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdd4004cf0 con 0x7fbde0102620 2024-03-29T05:03:50.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.554+0000 7fbde4bed700 1 -- 172.21.15.142:0/1077724687 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde0196850 con 0x7fbde0102620 2024-03-29T05:03:50.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.554+0000 7fbdcf7fe700 1 -- 172.21.15.142:0/1077724687 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd4005690 con 0x7fbde0102620 2024-03-29T05:03:50.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.554+0000 7fbde4bed700 1 -- 172.21.15.142:0/1077724687 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde0196da0 con 0x7fbde0102620 2024-03-29T05:03:50.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.554+0000 7fbde4bed700 1 -- 172.21.15.142:0/1077724687 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde0108210 con 0x7fbde0102620 2024-03-29T05:03:50.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.555+0000 7fbdcf7fe700 1 -- 172.21.15.142:0/1077724687 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fbdd4004750 con 0x7fbde0102620 2024-03-29T05:03:50.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.555+0000 7fbdcf7fe700 1 --2- 172.21.15.142:0/1077724687 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbdc8072170 0x7fbdc8074630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:50.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.556+0000 7fbdcf7fe700 1 -- 172.21.15.142:0/1077724687 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fbdd4014070 con 0x7fbde0102620 2024-03-29T05:03:50.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.556+0000 7fbdde59c700 1 --2- 172.21.15.142:0/1077724687 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbdc8072170 0x7fbdc8074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:50.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.556+0000 7fbdde59c700 1 --2- 172.21.15.142:0/1077724687 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbdc8072170 0x7fbdc8074630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000b760 tx=0x7fbdd0005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:50.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.560+0000 7fbdcf7fe700 1 -- 172.21.15.142:0/1077724687 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbdd405b000 con 0x7fbde0102620 2024-03-29T05:03:50.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.849+0000 7fbde4bed700 1 -- 172.21.15.142:0/1077724687 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbde004ea90 con 0x7fbde0102620 2024-03-29T05:03:50.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.849+0000 7fbdcf7fe700 1 -- 172.21.15.142:0/1077724687 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fbdd405ab90 con 0x7fbde0102620 2024-03-29T05:03:50.859 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:03:50.859 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:03:50.859 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:03:50.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:03:50.859 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:03:50.859 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:03:50.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:03:50.859 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:03:50.859 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:03:50.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:03:50.860 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:03:50.860 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:03:50.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:03:50.860 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:03:50.860 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:03:50.860 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:03:50.860 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:03:50.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.851+0000 7fbde4bed700 1 -- 172.21.15.142:0/1077724687 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbdc8072170 msgr2=0x7fbdc8074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:50.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.852+0000 7fbde4bed700 1 --2- 172.21.15.142:0/1077724687 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbdc8072170 0x7fbdc8074630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000b760 tx=0x7fbdd0005940 comp rx=0 tx=0).stop 2024-03-29T05:03:50.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.852+0000 7fbde4bed700 1 -- 172.21.15.142:0/1077724687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0102620 msgr2=0x7fbde019ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:50.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.852+0000 7fbde4bed700 1 --2- 172.21.15.142:0/1077724687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0102620 0x7fbde019ca30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400e8f0 tx=0x7fbdd400ecb0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.852+0000 7fbde4bed700 1 -- 172.21.15.142:0/1077724687 shutdown_connections 2024-03-29T05:03:50.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.852+0000 7fbde4bed700 1 --2- 172.21.15.142:0/1077724687 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbdc8072170 0x7fbdc8074630 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.852+0000 7fbde4bed700 1 --2- 172.21.15.142:0/1077724687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbde0101c70 0x7fbde019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.852+0000 7fbde4bed700 1 --2- 172.21.15.142:0/1077724687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0102620 0x7fbde019ca30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.852+0000 7fbde4bed700 1 -- 172.21.15.142:0/1077724687 >> 172.21.15.142:0/1077724687 conn(0x7fbde00fb530 msgr2=0x7fbde0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:50.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.852+0000 7fbde4bed700 1 -- 172.21.15.142:0/1077724687 shutdown_connections 2024-03-29T05:03:50.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.852+0000 7fbde4bed700 1 -- 172.21.15.142:0/1077724687 wait complete. 2024-03-29T05:03:50.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.956+0000 7fd190dee700 1 -- 172.21.15.142:0/609859079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd18c105a30 msgr2=0x7fd18c105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:50.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.956+0000 7fd190dee700 1 --2- 172.21.15.142:0/609859079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd18c105a30 0x7fd18c105e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd180009910 tx=0x7fd180009c20 comp rx=0 tx=0).stop 2024-03-29T05:03:50.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.956+0000 7fd190dee700 1 -- 172.21.15.142:0/609859079 shutdown_connections 2024-03-29T05:03:50.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.956+0000 7fd190dee700 1 --2- 172.21.15.142:0/609859079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd18c0ffa30 0x7fd18c0ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.956+0000 7fd190dee700 1 --2- 172.21.15.142:0/609859079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd18c105a30 0x7fd18c105e10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.956+0000 7fd190dee700 1 -- 172.21.15.142:0/609859079 >> 172.21.15.142:0/609859079 conn(0x7fd18c0fb540 msgr2=0x7fd18c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:50.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.956+0000 7fd190dee700 1 -- 172.21.15.142:0/609859079 shutdown_connections 2024-03-29T05:03:50.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.956+0000 7fd190dee700 1 -- 172.21.15.142:0/609859079 wait complete. 2024-03-29T05:03:50.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.957+0000 7fd190dee700 1 Processor -- start 2024-03-29T05:03:50.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.957+0000 7fd190dee700 1 -- start start 2024-03-29T05:03:50.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.957+0000 7fd190dee700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd18c0ffa30 0x7fd18c10a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:50.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.957+0000 7fd190dee700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd18c105a30 0x7fd18c10af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:50.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.957+0000 7fd190dee700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c10b440 con 0x7fd18c0ffa30 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.957+0000 7fd190dee700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c10b580 con 0x7fd18c105a30 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd18a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd18c0ffa30 0x7fd18c10a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd18a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd18c0ffa30 0x7fd18c10a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58794/0 (socket says 172.21.15.142:58794) 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd18a59c700 1 -- 172.21.15.142:0/2570092606 learned_addr learned my addr 172.21.15.142:0/2570092606 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd189d9b700 1 --2- 172.21.15.142:0/2570092606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd18c105a30 0x7fd18c10af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd18a59c700 1 -- 172.21.15.142:0/2570092606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd18c105a30 msgr2=0x7fd18c10af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd18a59c700 1 --2- 172.21.15.142:0/2570092606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd18c105a30 0x7fd18c10af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd18a59c700 1 -- 172.21.15.142:0/2570092606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1800095f0 con 0x7fd18c0ffa30 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd18a59c700 1 --2- 172.21.15.142:0/2570092606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd18c0ffa30 0x7fd18c10a9c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd180006010 tx=0x7fd18000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd17b7fe700 1 -- 172.21.15.142:0/2570092606 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd18001d070 con 0x7fd18c0ffa30 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd190dee700 1 -- 172.21.15.142:0/2570092606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c10b800 con 0x7fd18c0ffa30 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd17b7fe700 1 -- 172.21.15.142:0/2570092606 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd18000f460 con 0x7fd18c0ffa30 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd17b7fe700 1 -- 172.21.15.142:0/2570092606 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd180017630 con 0x7fd18c0ffa30 2024-03-29T05:03:50.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.958+0000 7fd190dee700 1 -- 172.21.15.142:0/2570092606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c1a2ca0 con 0x7fd18c0ffa30 2024-03-29T05:03:50.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.959+0000 7fd190dee700 1 -- 172.21.15.142:0/2570092606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18c04ea90 con 0x7fd18c0ffa30 2024-03-29T05:03:50.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.960+0000 7fd17b7fe700 1 -- 172.21.15.142:0/2570092606 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fd18000f5d0 con 0x7fd18c0ffa30 2024-03-29T05:03:50.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.960+0000 7fd17b7fe700 1 --2- 172.21.15.142:0/2570092606 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd17406dd60 0x7fd174070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:50.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.960+0000 7fd17b7fe700 1 -- 172.21.15.142:0/2570092606 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd18008f160 con 0x7fd18c0ffa30 2024-03-29T05:03:50.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.960+0000 7fd189d9b700 1 --2- 172.21.15.142:0/2570092606 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd17406dd60 0x7fd174070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:50.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.961+0000 7fd189d9b700 1 --2- 172.21.15.142:0/2570092606 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd17406dd60 0x7fd174070220 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd17c005950 tx=0x7fd17c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:50.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:50.965+0000 7fd17b7fe700 1 -- 172.21.15.142:0/2570092606 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd180058580 con 0x7fd18c0ffa30 2024-03-29T05:03:51.213 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.203+0000 7fd190dee700 1 -- 172.21.15.142:0/2570092606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd18c10bef0 con 0x7fd18c0ffa30 2024-03-29T05:03:51.213 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.204+0000 7fd17b7fe700 1 -- 172.21.15.142:0/2570092606 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fd18005bc40 con 0x7fd18c0ffa30 2024-03-29T05:03:51.214 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:03:51.214 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:03:51.214 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:03:51.214 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:03:51.214 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:03:51.214 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:03:51.214 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:03:51.214 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:03:51.214 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:03:51.214 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:03:51.214 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:03:51.214 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:03:51.215 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:03:51.216 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:03:51.217 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:03:51.217 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:03:51.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.206+0000 7fd190dee700 1 -- 172.21.15.142:0/2570092606 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd17406dd60 msgr2=0x7fd174070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:51.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.206+0000 7fd190dee700 1 --2- 172.21.15.142:0/2570092606 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd17406dd60 0x7fd174070220 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd17c005950 tx=0x7fd17c0058e0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.206+0000 7fd190dee700 1 -- 172.21.15.142:0/2570092606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd18c0ffa30 msgr2=0x7fd18c10a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:51.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.206+0000 7fd190dee700 1 --2- 172.21.15.142:0/2570092606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd18c0ffa30 0x7fd18c10a9c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd180006010 tx=0x7fd18000be20 comp rx=0 tx=0).stop 2024-03-29T05:03:51.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.206+0000 7fd190dee700 1 -- 172.21.15.142:0/2570092606 shutdown_connections 2024-03-29T05:03:51.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.206+0000 7fd190dee700 1 --2- 172.21.15.142:0/2570092606 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd17406dd60 0x7fd174070220 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.206+0000 7fd190dee700 1 --2- 172.21.15.142:0/2570092606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd18c0ffa30 0x7fd18c10a9c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.206+0000 7fd190dee700 1 --2- 172.21.15.142:0/2570092606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd18c105a30 0x7fd18c10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.206+0000 7fd190dee700 1 -- 172.21.15.142:0/2570092606 >> 172.21.15.142:0/2570092606 conn(0x7fd18c0fb540 msgr2=0x7fd18c0fbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:51.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.206+0000 7fd190dee700 1 -- 172.21.15.142:0/2570092606 shutdown_connections 2024-03-29T05:03:51.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.206+0000 7fd190dee700 1 -- 172.21.15.142:0/2570092606 wait complete. 2024-03-29T05:03:51.218 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:03:51.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.310+0000 7f55af7db700 1 -- 172.21.15.142:0/1322424026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a8105a40 msgr2=0x7f55a8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:51.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.310+0000 7f55af7db700 1 --2- 172.21.15.142:0/1322424026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a8105a40 0x7f55a8105e20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009910 tx=0x7f55a4009c20 comp rx=0 tx=0).stop 2024-03-29T05:03:51.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.310+0000 7f55af7db700 1 -- 172.21.15.142:0/1322424026 shutdown_connections 2024-03-29T05:03:51.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.310+0000 7f55af7db700 1 --2- 172.21.15.142:0/1322424026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a80ffbd0 0x7f55a8100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.320 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.310+0000 7f55af7db700 1 --2- 172.21.15.142:0/1322424026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a8105a40 0x7f55a8105e20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.310+0000 7f55af7db700 1 -- 172.21.15.142:0/1322424026 >> 172.21.15.142:0/1322424026 conn(0x7f55a80755a0 msgr2=0x7f55a80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:51.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.311+0000 7f55af7db700 1 -- 172.21.15.142:0/1322424026 shutdown_connections 2024-03-29T05:03:51.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.311+0000 7f55af7db700 1 -- 172.21.15.142:0/1322424026 wait complete. 2024-03-29T05:03:51.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.311+0000 7f55af7db700 1 Processor -- start 2024-03-29T05:03:51.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.311+0000 7f55af7db700 1 -- start start 2024-03-29T05:03:51.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.311+0000 7f55af7db700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a80ffbd0 0x7f55a8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:51.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.311+0000 7f55af7db700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a8105a40 0x7f55a8197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:51.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.311+0000 7f55af7db700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8198320 con 0x7f55a8105a40 2024-03-29T05:03:51.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.311+0000 7f55af7db700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8198490 con 0x7f55a80ffbd0 2024-03-29T05:03:51.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.311+0000 7f55acd76700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a8105a40 0x7f55a8197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:51.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.312+0000 7f55acd76700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a8105a40 0x7f55a8197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58816/0 (socket says 172.21.15.142:58816) 2024-03-29T05:03:51.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.312+0000 7f55acd76700 1 -- 172.21.15.142:0/2050730371 learned_addr learned my addr 172.21.15.142:0/2050730371 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:51.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.312+0000 7f55ad577700 1 --2- 172.21.15.142:0/2050730371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a80ffbd0 0x7f55a8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:51.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.312+0000 7f55acd76700 1 -- 172.21.15.142:0/2050730371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a80ffbd0 msgr2=0x7f55a8197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:51.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.312+0000 7f55acd76700 1 --2- 172.21.15.142:0/2050730371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a80ffbd0 0x7f55a8197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.312+0000 7f55acd76700 1 -- 172.21.15.142:0/2050730371 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a40095f0 con 0x7f55a8105a40 2024-03-29T05:03:51.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.312+0000 7f55acd76700 1 --2- 172.21.15.142:0/2050730371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a8105a40 0x7f55a8197c40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f559800b4b0 tx=0x7f559800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:51.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.312+0000 7f559e7fc700 1 -- 172.21.15.142:0/2050730371 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5598011800 con 0x7f55a8105a40 2024-03-29T05:03:51.323 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.312+0000 7f55af7db700 1 -- 172.21.15.142:0/2050730371 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a81a0770 con 0x7f55a8105a40 2024-03-29T05:03:51.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.312+0000 7f559e7fc700 1 -- 172.21.15.142:0/2050730371 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5598011e20 con 0x7f55a8105a40 2024-03-29T05:03:51.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.312+0000 7f55af7db700 1 -- 172.21.15.142:0/2050730371 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a81a0cc0 con 0x7f55a8105a40 2024-03-29T05:03:51.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.312+0000 7f559e7fc700 1 -- 172.21.15.142:0/2050730371 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f559800f2e0 con 0x7f55a8105a40 2024-03-29T05:03:51.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.313+0000 7f55af7db700 1 -- 172.21.15.142:0/2050730371 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a8108040 con 0x7f55a8105a40 2024-03-29T05:03:51.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.314+0000 7f559e7fc700 1 -- 172.21.15.142:0/2050730371 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f5598010380 con 0x7f55a8105a40 2024-03-29T05:03:51.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.314+0000 7f559e7fc700 1 --2- 172.21.15.142:0/2050730371 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f559406dd60 0x7f5594070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:51.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.314+0000 7f559e7fc700 1 -- 172.21.15.142:0/2050730371 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f559808d7b0 con 0x7f55a8105a40 2024-03-29T05:03:51.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.314+0000 7f55ad577700 1 --2- 172.21.15.142:0/2050730371 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f559406dd60 0x7f5594070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:51.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.315+0000 7f55ad577700 1 --2- 172.21.15.142:0/2050730371 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f559406dd60 0x7f5594070220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55a4006010 tx=0x7f55a401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:51.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.319+0000 7f559e7fc700 1 -- 172.21.15.142:0/2050730371 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5598056bd0 con 0x7f55a8105a40 2024-03-29T05:03:51.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.536+0000 7f55af7db700 1 -- 172.21.15.142:0/2050730371 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55a80611d0 con 0x7f559406dd60 2024-03-29T05:03:51.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.537+0000 7f559e7fc700 1 -- 172.21.15.142:0/2050730371 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f55a80611d0 con 0x7f559406dd60 2024-03-29T05:03:51.546 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:03:51.547 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:03:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.539+0000 7f55af7db700 1 -- 172.21.15.142:0/2050730371 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f559406dd60 msgr2=0x7f5594070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.539+0000 7f55af7db700 1 --2- 172.21.15.142:0/2050730371 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f559406dd60 0x7f5594070220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55a4006010 tx=0x7f55a401a040 comp rx=0 tx=0).stop 2024-03-29T05:03:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.539+0000 7f55af7db700 1 -- 172.21.15.142:0/2050730371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a8105a40 msgr2=0x7f55a8197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.539+0000 7f55af7db700 1 --2- 172.21.15.142:0/2050730371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a8105a40 0x7f55a8197c40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f559800b4b0 tx=0x7f559800b7c0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.539+0000 7f55af7db700 1 -- 172.21.15.142:0/2050730371 shutdown_connections 2024-03-29T05:03:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.539+0000 7f55af7db700 1 --2- 172.21.15.142:0/2050730371 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f559406dd60 0x7f5594070220 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.539+0000 7f55af7db700 1 --2- 172.21.15.142:0/2050730371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55a80ffbd0 0x7f55a8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.539+0000 7f55af7db700 1 --2- 172.21.15.142:0/2050730371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55a8105a40 0x7f55a8197c40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.539+0000 7f55af7db700 1 -- 172.21.15.142:0/2050730371 >> 172.21.15.142:0/2050730371 conn(0x7f55a80755a0 msgr2=0x7f55a80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.539+0000 7f55af7db700 1 -- 172.21.15.142:0/2050730371 shutdown_connections 2024-03-29T05:03:51.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.539+0000 7f55af7db700 1 -- 172.21.15.142:0/2050730371 wait complete. 2024-03-29T05:03:51.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.643+0000 7f2d2f411700 1 -- 172.21.15.142:0/2074127763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d280ff290 msgr2=0x7f2d280ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:51.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.643+0000 7f2d2f411700 1 --2- 172.21.15.142:0/2074127763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d280ff290 0x7f2d280ff670 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009910 tx=0x7f2d18009c20 comp rx=0 tx=0).stop 2024-03-29T05:03:51.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.643+0000 7f2d2f411700 1 -- 172.21.15.142:0/2074127763 shutdown_connections 2024-03-29T05:03:51.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.643+0000 7f2d2f411700 1 --2- 172.21.15.142:0/2074127763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d280ffc40 0x7f2d28105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.643+0000 7f2d2f411700 1 --2- 172.21.15.142:0/2074127763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d280ff290 0x7f2d280ff670 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.643+0000 7f2d2f411700 1 -- 172.21.15.142:0/2074127763 >> 172.21.15.142:0/2074127763 conn(0x7f2d28075e30 msgr2=0x7f2d280fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:51.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.643+0000 7f2d2f411700 1 -- 172.21.15.142:0/2074127763 shutdown_connections 2024-03-29T05:03:51.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.643+0000 7f2d2f411700 1 -- 172.21.15.142:0/2074127763 wait complete. 2024-03-29T05:03:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.643+0000 7f2d2f411700 1 Processor -- start 2024-03-29T05:03:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2f411700 1 -- start start 2024-03-29T05:03:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2f411700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d280ff290 0x7f2d2819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2f411700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d280ffc40 0x7f2d2819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2f411700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2819d270 con 0x7f2d280ffc40 2024-03-29T05:03:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2f411700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d28196720 con 0x7f2d280ff290 2024-03-29T05:03:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2c9ac700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d280ffc40 0x7f2d2819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2c9ac700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d280ffc40 0x7f2d2819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58834/0 (socket says 172.21.15.142:58834) 2024-03-29T05:03:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2c9ac700 1 -- 172.21.15.142:0/2842452251 learned_addr learned my addr 172.21.15.142:0/2842452251 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:03:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2d1ad700 1 --2- 172.21.15.142:0/2842452251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d280ff290 0x7f2d2819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2c9ac700 1 -- 172.21.15.142:0/2842452251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d280ff290 msgr2=0x7f2d2819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:51.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2c9ac700 1 --2- 172.21.15.142:0/2842452251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d280ff290 0x7f2d2819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2c9ac700 1 -- 172.21.15.142:0/2842452251 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d180095f0 con 0x7f2d280ffc40 2024-03-29T05:03:51.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d2c9ac700 1 --2- 172.21.15.142:0/2842452251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d280ffc40 0x7f2d2819cbe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400b480 tx=0x7f2d2400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:51.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.644+0000 7f2d1e7fc700 1 -- 172.21.15.142:0/2842452251 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d240118c0 con 0x7f2d280ffc40 2024-03-29T05:03:51.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.645+0000 7f2d1e7fc700 1 -- 172.21.15.142:0/2842452251 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d24010380 con 0x7f2d280ffc40 2024-03-29T05:03:51.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.645+0000 7f2d2f411700 1 -- 172.21.15.142:0/2842452251 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d28196a00 con 0x7f2d280ffc40 2024-03-29T05:03:51.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.645+0000 7f2d1e7fc700 1 -- 172.21.15.142:0/2842452251 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d2400f4f0 con 0x7f2d280ffc40 2024-03-29T05:03:51.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.645+0000 7f2d2f411700 1 -- 172.21.15.142:0/2842452251 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d28196f50 con 0x7f2d280ffc40 2024-03-29T05:03:51.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.646+0000 7f2d2f411700 1 -- 172.21.15.142:0/2842452251 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d28109120 con 0x7f2d280ffc40 2024-03-29T05:03:51.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.646+0000 7f2d1e7fc700 1 -- 172.21.15.142:0/2842452251 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f2d240104f0 con 0x7f2d280ffc40 2024-03-29T05:03:51.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.646+0000 7f2d1e7fc700 1 --2- 172.21.15.142:0/2842452251 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2d1406dd60 0x7f2d14070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:03:51.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.647+0000 7f2d1e7fc700 1 -- 172.21.15.142:0/2842452251 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f2d2408d800 con 0x7f2d280ffc40 2024-03-29T05:03:51.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.647+0000 7f2d2d1ad700 1 --2- 172.21.15.142:0/2842452251 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2d1406dd60 0x7f2d14070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:03:51.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.647+0000 7f2d2d1ad700 1 --2- 172.21.15.142:0/2842452251 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2d1406dd60 0x7f2d14070220 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800b5c0 tx=0x7f2d1801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:03:51.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.651+0000 7f2d1e7fc700 1 -- 172.21.15.142:0/2842452251 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2d2405a7d0 con 0x7f2d280ffc40 2024-03-29T05:03:51.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:51 smithi152 ceph-mon[88186]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:51.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:51 smithi152 ceph-mon[88186]: pgmap v466: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:51.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:51 smithi152 ceph-mon[88186]: from='client.34882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:51.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:51 smithi152 ceph-mon[88186]: from='client.34886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:51.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:51 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1077724687' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:03:51.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:51 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2570092606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:03:51.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:51 smithi142 ceph-mon[112475]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:51.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:51 smithi142 ceph-mon[112475]: pgmap v466: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:51.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:51 smithi142 ceph-mon[112475]: from='client.34882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:51.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:51 smithi142 ceph-mon[112475]: from='client.34886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:51.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:51 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1077724687' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:03:51.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:51 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2570092606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:03:51.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.928+0000 7f2d2f411700 1 -- 172.21.15.142:0/2842452251 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2d2804ea90 con 0x7f2d280ffc40 2024-03-29T05:03:51.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.928+0000 7f2d1e7fc700 1 -- 172.21.15.142:0/2842452251 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f2d2405a360 con 0x7f2d280ffc40 2024-03-29T05:03:51.938 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:03:51.938 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:03:51.938 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:03:51.938 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:03:51.938 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:03:51.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.931+0000 7f2d2f411700 1 -- 172.21.15.142:0/2842452251 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2d1406dd60 msgr2=0x7f2d14070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:51.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.931+0000 7f2d2f411700 1 --2- 172.21.15.142:0/2842452251 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2d1406dd60 0x7f2d14070220 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800b5c0 tx=0x7f2d1801a040 comp rx=0 tx=0).stop 2024-03-29T05:03:51.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.931+0000 7f2d2f411700 1 -- 172.21.15.142:0/2842452251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d280ffc40 msgr2=0x7f2d2819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:03:51.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.931+0000 7f2d2f411700 1 --2- 172.21.15.142:0/2842452251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d280ffc40 0x7f2d2819cbe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400b480 tx=0x7f2d2400b840 comp rx=0 tx=0).stop 2024-03-29T05:03:51.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.931+0000 7f2d2f411700 1 -- 172.21.15.142:0/2842452251 shutdown_connections 2024-03-29T05:03:51.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.931+0000 7f2d2f411700 1 --2- 172.21.15.142:0/2842452251 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2d1406dd60 0x7f2d14070220 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.931+0000 7f2d2f411700 1 --2- 172.21.15.142:0/2842452251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d280ff290 0x7f2d2819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.931+0000 7f2d2f411700 1 --2- 172.21.15.142:0/2842452251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d280ffc40 0x7f2d2819cbe0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:03:51.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.931+0000 7f2d2f411700 1 -- 172.21.15.142:0/2842452251 >> 172.21.15.142:0/2842452251 conn(0x7f2d28075e30 msgr2=0x7f2d28104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:03:51.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.931+0000 7f2d2f411700 1 -- 172.21.15.142:0/2842452251 shutdown_connections 2024-03-29T05:03:51.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:03:51.931+0000 7f2d2f411700 1 -- 172.21.15.142:0/2842452251 wait complete. 2024-03-29T05:03:52.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:52 smithi152 ceph-mon[88186]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:52.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:52 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2842452251' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:03:52.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:52 smithi142 ceph-mon[112475]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:03:52.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:52 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2842452251' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:03:53.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:53 smithi152 ceph-mon[88186]: pgmap v467: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:53.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:53 smithi142 ceph-mon[112475]: pgmap v467: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:55.737 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:55 smithi142 ceph-mon[112475]: pgmap v468: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:55.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:55 smithi152 ceph-mon[88186]: pgmap v468: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:57.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:57 smithi152 ceph-mon[88186]: pgmap v469: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:57.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:57 smithi142 ceph-mon[112475]: pgmap v469: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:59.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:03:59 smithi152 ceph-mon[88186]: pgmap v470: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:03:59.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:03:59 smithi142 ceph-mon[112475]: pgmap v470: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:00.790 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:04:00.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:04:01.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:01 smithi152 ceph-mon[88186]: pgmap v471: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:01.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:01 smithi142 ceph-mon[112475]: pgmap v471: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:03.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:03 smithi152 ceph-mon[88186]: pgmap v472: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:03.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:03 smithi142 ceph-mon[112475]: pgmap v472: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:05 smithi152 ceph-mon[88186]: pgmap v473: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:05.844 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:05 smithi142 ceph-mon[112475]: pgmap v473: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:07.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:07 smithi152 ceph-mon[88186]: pgmap v474: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:07.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:07 smithi142 ceph-mon[112475]: pgmap v474: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:09.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:09 smithi152 ceph-mon[88186]: pgmap v475: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:09.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:09 smithi142 ceph-mon[112475]: pgmap v475: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:11.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:11 smithi152 ceph-mon[88186]: pgmap v476: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:11.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:11 smithi142 ceph-mon[112475]: pgmap v476: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:13.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:13 smithi152 ceph-mon[88186]: pgmap v477: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:13.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:13 smithi142 ceph-mon[112475]: pgmap v477: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:15.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:15 smithi152 ceph-mon[88186]: pgmap v478: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:15.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:04:15.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:15 smithi142 ceph-mon[112475]: pgmap v478: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:15.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:04:17.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:17 smithi152 ceph-mon[88186]: pgmap v479: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:17.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:17 smithi142 ceph-mon[112475]: pgmap v479: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:19.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:19 smithi152 ceph-mon[88186]: pgmap v480: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:19.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:19 smithi142 ceph-mon[112475]: pgmap v480: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:21.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:21 smithi152 ceph-mon[88186]: pgmap v481: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:21.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:21 smithi142 ceph-mon[112475]: pgmap v481: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:22.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.049+0000 7f59b5c02700 1 -- 172.21.15.142:0/3045101389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59b0105a40 msgr2=0x7f59b0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:22.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.049+0000 7f59b5c02700 1 --2- 172.21.15.142:0/3045101389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59b0105a40 0x7f59b0105e20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009910 tx=0x7f59a0009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:22.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.049+0000 7f59b5c02700 1 -- 172.21.15.142:0/3045101389 shutdown_connections 2024-03-29T05:04:22.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.049+0000 7f59b5c02700 1 --2- 172.21.15.142:0/3045101389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59b00ffa40 0x7f59b00ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.049+0000 7f59b5c02700 1 --2- 172.21.15.142:0/3045101389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59b0105a40 0x7f59b0105e20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.049+0000 7f59b5c02700 1 -- 172.21.15.142:0/3045101389 >> 172.21.15.142:0/3045101389 conn(0x7f59b00fb550 msgr2=0x7f59b00fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:22.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.049+0000 7f59b5c02700 1 -- 172.21.15.142:0/3045101389 shutdown_connections 2024-03-29T05:04:22.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.049+0000 7f59b5c02700 1 -- 172.21.15.142:0/3045101389 wait complete. 2024-03-29T05:04:22.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.049+0000 7f59b5c02700 1 Processor -- start 2024-03-29T05:04:22.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59b5c02700 1 -- start start 2024-03-29T05:04:22.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59b5c02700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59b00ffa40 0x7f59b019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:22.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59b5c02700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59b0105a40 0x7f59b019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:22.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59b5c02700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b019d0f0 con 0x7f59b00ffa40 2024-03-29T05:04:22.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59b5c02700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b01965a0 con 0x7f59b0105a40 2024-03-29T05:04:22.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59af7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59b00ffa40 0x7f59b019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:22.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59af7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59b00ffa40 0x7f59b019c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55170/0 (socket says 172.21.15.142:55170) 2024-03-29T05:04:22.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59af7fe700 1 -- 172.21.15.142:0/2334385483 learned_addr learned my addr 172.21.15.142:0/2334385483 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:22.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59aeffd700 1 --2- 172.21.15.142:0/2334385483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59b0105a40 0x7f59b019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:22.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59af7fe700 1 -- 172.21.15.142:0/2334385483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59b0105a40 msgr2=0x7f59b019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:22.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59af7fe700 1 --2- 172.21.15.142:0/2334385483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59b0105a40 0x7f59b019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59af7fe700 1 -- 172.21.15.142:0/2334385483 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a00095f0 con 0x7f59b00ffa40 2024-03-29T05:04:22.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.050+0000 7f59af7fe700 1 --2- 172.21.15.142:0/2334385483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59b00ffa40 0x7f59b019c520 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f59a0006010 tx=0x7f59a000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:22.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.051+0000 7f59acff9700 1 -- 172.21.15.142:0/2334385483 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59a001d070 con 0x7f59b00ffa40 2024-03-29T05:04:22.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.051+0000 7f59b5c02700 1 -- 172.21.15.142:0/2334385483 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b0196820 con 0x7f59b00ffa40 2024-03-29T05:04:22.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.051+0000 7f59acff9700 1 -- 172.21.15.142:0/2334385483 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59a0005dc0 con 0x7f59b00ffa40 2024-03-29T05:04:22.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.051+0000 7f59acff9700 1 -- 172.21.15.142:0/2334385483 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59a0017400 con 0x7f59b00ffa40 2024-03-29T05:04:22.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.051+0000 7f59b5c02700 1 -- 172.21.15.142:0/2334385483 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b0196d10 con 0x7f59b00ffa40 2024-03-29T05:04:22.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.052+0000 7f59b5c02700 1 -- 172.21.15.142:0/2334385483 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b004ea90 con 0x7f59b00ffa40 2024-03-29T05:04:22.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.052+0000 7f59acff9700 1 -- 172.21.15.142:0/2334385483 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f59a0003a60 con 0x7f59b00ffa40 2024-03-29T05:04:22.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.052+0000 7f59acff9700 1 --2- 172.21.15.142:0/2334385483 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f599806dd60 0x7f5998070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:22.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.053+0000 7f59acff9700 1 -- 172.21.15.142:0/2334385483 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f59a008f050 con 0x7f59b00ffa40 2024-03-29T05:04:22.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.053+0000 7f59aeffd700 1 --2- 172.21.15.142:0/2334385483 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f599806dd60 0x7f5998070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:22.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.053+0000 7f59aeffd700 1 --2- 172.21.15.142:0/2334385483 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f599806dd60 0x7f5998070220 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f59b0104e00 tx=0x7f59a40073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:22.067 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.057+0000 7f59acff9700 1 -- 172.21.15.142:0/2334385483 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f59a005c020 con 0x7f59b00ffa40 2024-03-29T05:04:22.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.269+0000 7f59b5c02700 1 -- 172.21.15.142:0/2334385483 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f59b0197210 con 0x7f599806dd60 2024-03-29T05:04:22.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.270+0000 7f59acff9700 1 -- 172.21.15.142:0/2334385483 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f59b0197210 con 0x7f599806dd60 2024-03-29T05:04:22.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.272+0000 7f59b5c02700 1 -- 172.21.15.142:0/2334385483 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f599806dd60 msgr2=0x7f5998070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:22.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.272+0000 7f59b5c02700 1 --2- 172.21.15.142:0/2334385483 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f599806dd60 0x7f5998070220 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f59b0104e00 tx=0x7f59a40073d0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.272+0000 7f59b5c02700 1 -- 172.21.15.142:0/2334385483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59b00ffa40 msgr2=0x7f59b019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:22.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.272+0000 7f59b5c02700 1 --2- 172.21.15.142:0/2334385483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59b00ffa40 0x7f59b019c520 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f59a0006010 tx=0x7f59a000bd70 comp rx=0 tx=0).stop 2024-03-29T05:04:22.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.272+0000 7f59b5c02700 1 -- 172.21.15.142:0/2334385483 shutdown_connections 2024-03-29T05:04:22.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.272+0000 7f59b5c02700 1 --2- 172.21.15.142:0/2334385483 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f599806dd60 0x7f5998070220 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.272+0000 7f59b5c02700 1 --2- 172.21.15.142:0/2334385483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59b00ffa40 0x7f59b019c520 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.272+0000 7f59b5c02700 1 --2- 172.21.15.142:0/2334385483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59b0105a40 0x7f59b019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.272+0000 7f59b5c02700 1 -- 172.21.15.142:0/2334385483 >> 172.21.15.142:0/2334385483 conn(0x7f59b00fb550 msgr2=0x7f59b0103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:22.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.272+0000 7f59b5c02700 1 -- 172.21.15.142:0/2334385483 shutdown_connections 2024-03-29T05:04:22.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.272+0000 7f59b5c02700 1 -- 172.21.15.142:0/2334385483 wait complete. 2024-03-29T05:04:22.292 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:04:22.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.376+0000 7f449d13f700 1 -- 172.21.15.142:0/3188759737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498105a40 msgr2=0x7f4498105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:22.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.376+0000 7f449d13f700 1 --2- 172.21.15.142:0/3188759737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498105a40 0x7f4498105e20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4488009910 tx=0x7f4488009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:22.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.376+0000 7f449d13f700 1 -- 172.21.15.142:0/3188759737 shutdown_connections 2024-03-29T05:04:22.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.376+0000 7f449d13f700 1 --2- 172.21.15.142:0/3188759737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44980ffbd0 0x7f4498100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.376+0000 7f449d13f700 1 --2- 172.21.15.142:0/3188759737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498105a40 0x7f4498105e20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.376+0000 7f449d13f700 1 -- 172.21.15.142:0/3188759737 >> 172.21.15.142:0/3188759737 conn(0x7f44980755a0 msgr2=0x7f44980759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:22.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.377+0000 7f449d13f700 1 -- 172.21.15.142:0/3188759737 shutdown_connections 2024-03-29T05:04:22.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.377+0000 7f449d13f700 1 -- 172.21.15.142:0/3188759737 wait complete. 2024-03-29T05:04:22.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.377+0000 7f449d13f700 1 Processor -- start 2024-03-29T05:04:22.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.377+0000 7f449d13f700 1 -- start start 2024-03-29T05:04:22.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f449d13f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44980ffbd0 0x7f4498101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:22.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f449d13f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498105a40 0x7f4498101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:22.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f449d13f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4498102120 con 0x7f4498105a40 2024-03-29T05:04:22.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f449d13f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4498102290 con 0x7f44980ffbd0 2024-03-29T05:04:22.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f448e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498105a40 0x7f4498101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:22.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f448e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498105a40 0x7f4498101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55194/0 (socket says 172.21.15.142:55194) 2024-03-29T05:04:22.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f448e59c700 1 -- 172.21.15.142:0/3078615044 learned_addr learned my addr 172.21.15.142:0/3078615044 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:22.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f4496d9d700 1 --2- 172.21.15.142:0/3078615044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44980ffbd0 0x7f4498101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:22.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f448e59c700 1 -- 172.21.15.142:0/3078615044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44980ffbd0 msgr2=0x7f4498101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:22.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f448e59c700 1 --2- 172.21.15.142:0/3078615044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44980ffbd0 0x7f4498101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f448e59c700 1 -- 172.21.15.142:0/3078615044 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44880095f0 con 0x7f4498105a40 2024-03-29T05:04:22.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f448e59c700 1 --2- 172.21.15.142:0/3078615044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498105a40 0x7f4498101b50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f448000d8d0 tx=0x7f448000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:22.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f4494d99700 1 -- 172.21.15.142:0/3078615044 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4480009940 con 0x7f4498105a40 2024-03-29T05:04:22.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f4494d99700 1 -- 172.21.15.142:0/3078615044 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4480010460 con 0x7f4498105a40 2024-03-29T05:04:22.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f449d13f700 1 -- 172.21.15.142:0/3078615044 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44981a28c0 con 0x7f4498105a40 2024-03-29T05:04:22.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f4494d99700 1 -- 172.21.15.142:0/3078615044 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f448000f5f0 con 0x7f4498105a40 2024-03-29T05:04:22.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.378+0000 7f449d13f700 1 -- 172.21.15.142:0/3078615044 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44981a2d30 con 0x7f4498105a40 2024-03-29T05:04:22.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.380+0000 7f449d13f700 1 -- 172.21.15.142:0/3078615044 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4498109050 con 0x7f4498105a40 2024-03-29T05:04:22.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.380+0000 7f4494d99700 1 -- 172.21.15.142:0/3078615044 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f44800105d0 con 0x7f4498105a40 2024-03-29T05:04:22.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.380+0000 7f4494d99700 1 --2- 172.21.15.142:0/3078615044 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f448406dd60 0x7f4484070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:22.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.381+0000 7f4494d99700 1 -- 172.21.15.142:0/3078615044 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f448008dcd0 con 0x7f4498105a40 2024-03-29T05:04:22.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.381+0000 7f4496d9d700 1 --2- 172.21.15.142:0/3078615044 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f448406dd60 0x7f4484070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:22.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.381+0000 7f4496d9d700 1 --2- 172.21.15.142:0/3078615044 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f448406dd60 0x7f4484070220 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f448800b5c0 tx=0x7f448801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:22.395 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.385+0000 7f4494d99700 1 -- 172.21.15.142:0/3078615044 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f44800570f0 con 0x7f4498105a40 2024-03-29T05:04:22.610 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.601+0000 7f449d13f700 1 -- 172.21.15.142:0/3078615044 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f449802cce0 con 0x7f448406dd60 2024-03-29T05:04:22.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.602+0000 7f4494d99700 1 -- 172.21.15.142:0/3078615044 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f449802cce0 con 0x7f448406dd60 2024-03-29T05:04:22.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.604+0000 7f449d13f700 1 -- 172.21.15.142:0/3078615044 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f448406dd60 msgr2=0x7f4484070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:22.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.604+0000 7f449d13f700 1 --2- 172.21.15.142:0/3078615044 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f448406dd60 0x7f4484070220 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f448800b5c0 tx=0x7f448801a040 comp rx=0 tx=0).stop 2024-03-29T05:04:22.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.604+0000 7f449d13f700 1 -- 172.21.15.142:0/3078615044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498105a40 msgr2=0x7f4498101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:22.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.604+0000 7f449d13f700 1 --2- 172.21.15.142:0/3078615044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498105a40 0x7f4498101b50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f448000d8d0 tx=0x7f448000dc90 comp rx=0 tx=0).stop 2024-03-29T05:04:22.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.604+0000 7f449d13f700 1 -- 172.21.15.142:0/3078615044 shutdown_connections 2024-03-29T05:04:22.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.604+0000 7f449d13f700 1 --2- 172.21.15.142:0/3078615044 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f448406dd60 0x7f4484070220 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.604+0000 7f449d13f700 1 --2- 172.21.15.142:0/3078615044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44980ffbd0 0x7f4498101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.604+0000 7f449d13f700 1 --2- 172.21.15.142:0/3078615044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498105a40 0x7f4498101b50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.604+0000 7f449d13f700 1 -- 172.21.15.142:0/3078615044 >> 172.21.15.142:0/3078615044 conn(0x7f44980755a0 msgr2=0x7f44980685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:22.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.604+0000 7f449d13f700 1 -- 172.21.15.142:0/3078615044 shutdown_connections 2024-03-29T05:04:22.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.604+0000 7f449d13f700 1 -- 172.21.15.142:0/3078615044 wait complete. 2024-03-29T05:04:22.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.707+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/2366734323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e94105a40 msgr2=0x7f8e94105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:22.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.707+0000 7f8e9b6d0700 1 --2- 172.21.15.142:0/2366734323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e94105a40 0x7f8e94105e20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009910 tx=0x7f8e84009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:22.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.707+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/2366734323 shutdown_connections 2024-03-29T05:04:22.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.707+0000 7f8e9b6d0700 1 --2- 172.21.15.142:0/2366734323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e940ffbd0 0x7f8e94100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.707+0000 7f8e9b6d0700 1 --2- 172.21.15.142:0/2366734323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e94105a40 0x7f8e94105e20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.707+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/2366734323 >> 172.21.15.142:0/2366734323 conn(0x7f8e940755a0 msgr2=0x7f8e940759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:22.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.707+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/2366734323 shutdown_connections 2024-03-29T05:04:22.717 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.707+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/2366734323 wait complete. 2024-03-29T05:04:22.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e9b6d0700 1 Processor -- start 2024-03-29T05:04:22.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e9b6d0700 1 -- start start 2024-03-29T05:04:22.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e9b6d0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e940ffbd0 0x7f8e94101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:22.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e9b6d0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94105a40 0x7f8e94101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:22.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e9b6d0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94102090 con 0x7f8e940ffbd0 2024-03-29T05:04:22.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e9b6d0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e941021d0 con 0x7f8e94105a40 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e9946c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e940ffbd0 0x7f8e94101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e9946c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e940ffbd0 0x7f8e94101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55212/0 (socket says 172.21.15.142:55212) 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e9946c700 1 -- 172.21.15.142:0/3416160476 learned_addr learned my addr 172.21.15.142:0/3416160476 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e98c6b700 1 --2- 172.21.15.142:0/3416160476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94105a40 0x7f8e94101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e9946c700 1 -- 172.21.15.142:0/3416160476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94105a40 msgr2=0x7f8e94101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e9946c700 1 --2- 172.21.15.142:0/3416160476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94105a40 0x7f8e94101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e9946c700 1 -- 172.21.15.142:0/3416160476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e840095f0 con 0x7f8e940ffbd0 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.708+0000 7f8e98c6b700 1 --2- 172.21.15.142:0/3416160476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94105a40 0x7f8e94101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.709+0000 7f8e9946c700 1 --2- 172.21.15.142:0/3416160476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e940ffbd0 0x7f8e94101610 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8e84006010 tx=0x7f8e8400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.709+0000 7f8e8a7fc700 1 -- 172.21.15.142:0/3416160476 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e8401d070 con 0x7f8e940ffbd0 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.709+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/3416160476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e94102450 con 0x7f8e940ffbd0 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.709+0000 7f8e8a7fc700 1 -- 172.21.15.142:0/3416160476 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e84004fb0 con 0x7f8e940ffbd0 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.709+0000 7f8e8a7fc700 1 -- 172.21.15.142:0/3416160476 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e84022ea0 con 0x7f8e940ffbd0 2024-03-29T05:04:22.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.709+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/3416160476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e941a2d20 con 0x7f8e940ffbd0 2024-03-29T05:04:22.721 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.710+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/3416160476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e94109050 con 0x7f8e940ffbd0 2024-03-29T05:04:22.721 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.710+0000 7f8e8a7fc700 1 -- 172.21.15.142:0/3416160476 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f8e84022470 con 0x7f8e940ffbd0 2024-03-29T05:04:22.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.711+0000 7f8e8a7fc700 1 --2- 172.21.15.142:0/3416160476 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8e8006dd60 0x7f8e80070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:22.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.711+0000 7f8e8a7fc700 1 -- 172.21.15.142:0/3416160476 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f8e8408f1f0 con 0x7f8e940ffbd0 2024-03-29T05:04:22.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.711+0000 7f8e98c6b700 1 --2- 172.21.15.142:0/3416160476 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8e8006dd60 0x7f8e80070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:22.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.711+0000 7f8e98c6b700 1 --2- 172.21.15.142:0/3416160476 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8e8006dd60 0x7f8e80070220 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8e94102e00 tx=0x7f8e90008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:22.725 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.715+0000 7f8e8a7fc700 1 -- 172.21.15.142:0/3416160476 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8e840586c0 con 0x7f8e940ffbd0 2024-03-29T05:04:22.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.920+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/3416160476 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e940611d0 con 0x7f8e8006dd60 2024-03-29T05:04:22.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.931+0000 7f8e8a7fc700 1 -- 172.21.15.142:0/3416160476 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8e940611d0 con 0x7f8e8006dd60 2024-03-29T05:04:22.941 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:04:22.941 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (16m) 9m ago 27m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:04:22.941 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (28m) 9m ago 27m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:04:22.941 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (26m) 8m ago 26m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:04:22.941 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (13m) 9m ago 27m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:04:22.941 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (13m) 8m ago 26m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:04:22.941 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (15m) 9m ago 27m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:04:22.941 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (9m) 9m ago 22m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:04:22.941 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (9m) 9m ago 22m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:04:22.941 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (8m) 8m ago 21m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:04:22.941 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (9m) 8m ago 22m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (17m) 9m ago 29m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (17m) 8m ago 26m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (14m) 9m ago 29m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (14m) 8m ago 26m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (16m) 9m ago 27m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (16m) 8m ago 26m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (12m) 9m ago 25m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (12m) 9m ago 25m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (11m) 9m ago 24m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (11m) 8m ago 24m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (10m) 8m ago 24m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (9m) 8m ago 23m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:04:22.942 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (16m) 9m ago 27m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:04:22.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.933+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/3416160476 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8e8006dd60 msgr2=0x7f8e80070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:22.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.933+0000 7f8e9b6d0700 1 --2- 172.21.15.142:0/3416160476 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8e8006dd60 0x7f8e80070220 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8e94102e00 tx=0x7f8e90008040 comp rx=0 tx=0).stop 2024-03-29T05:04:22.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.933+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/3416160476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e940ffbd0 msgr2=0x7f8e94101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:22.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.933+0000 7f8e9b6d0700 1 --2- 172.21.15.142:0/3416160476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e940ffbd0 0x7f8e94101610 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8e84006010 tx=0x7f8e8400be20 comp rx=0 tx=0).stop 2024-03-29T05:04:22.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.933+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/3416160476 shutdown_connections 2024-03-29T05:04:22.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.933+0000 7f8e9b6d0700 1 --2- 172.21.15.142:0/3416160476 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8e8006dd60 0x7f8e80070220 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.933+0000 7f8e9b6d0700 1 --2- 172.21.15.142:0/3416160476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e940ffbd0 0x7f8e94101610 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.933+0000 7f8e9b6d0700 1 --2- 172.21.15.142:0/3416160476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e94105a40 0x7f8e94101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:22.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.933+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/3416160476 >> 172.21.15.142:0/3416160476 conn(0x7f8e940755a0 msgr2=0x7f8e940686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:22.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.934+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/3416160476 shutdown_connections 2024-03-29T05:04:22.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:22.934+0000 7f8e9b6d0700 1 -- 172.21.15.142:0/3416160476 wait complete. 2024-03-29T05:04:23.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.036+0000 7f2ac9d15700 1 -- 172.21.15.142:0/610901700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4073130 msgr2=0x7f2ac4073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:23.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.036+0000 7f2ac9d15700 1 --2- 172.21.15.142:0/610901700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4073130 0x7f2ac4073510 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009910 tx=0x7f2ab4009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:23.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.036+0000 7f2ac9d15700 1 -- 172.21.15.142:0/610901700 shutdown_connections 2024-03-29T05:04:23.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.036+0000 7f2ac9d15700 1 --2- 172.21.15.142:0/610901700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac4073a50 0x7f2ac4111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.036+0000 7f2ac9d15700 1 --2- 172.21.15.142:0/610901700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4073130 0x7f2ac4073510 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.036+0000 7f2ac9d15700 1 -- 172.21.15.142:0/610901700 >> 172.21.15.142:0/610901700 conn(0x7f2ac40fc650 msgr2=0x7f2ac40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:23.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.036+0000 7f2ac9d15700 1 -- 172.21.15.142:0/610901700 shutdown_connections 2024-03-29T05:04:23.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.037+0000 7f2ac9d15700 1 -- 172.21.15.142:0/610901700 wait complete. 2024-03-29T05:04:23.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.037+0000 7f2ac9d15700 1 Processor -- start 2024-03-29T05:04:23.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.037+0000 7f2ac9d15700 1 -- start start 2024-03-29T05:04:23.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2ac9d15700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac4073130 0x7f2ac41a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:23.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2ac9d15700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4073a50 0x7f2ac41a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:23.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2ac9d15700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac41a36a0 con 0x7f2ac4073a50 2024-03-29T05:04:23.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2ac9d15700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac419cba0 con 0x7f2ac4073130 2024-03-29T05:04:23.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2abbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4073a50 0x7f2ac41a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:23.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2abbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4073a50 0x7f2ac41a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55230/0 (socket says 172.21.15.142:55230) 2024-03-29T05:04:23.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2abbfff700 1 -- 172.21.15.142:0/600262528 learned_addr learned my addr 172.21.15.142:0/600262528 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:23.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2ac37fe700 1 --2- 172.21.15.142:0/600262528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac4073130 0x7f2ac41a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:23.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2abbfff700 1 -- 172.21.15.142:0/600262528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac4073130 msgr2=0x7f2ac41a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:23.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2abbfff700 1 --2- 172.21.15.142:0/600262528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac4073130 0x7f2ac41a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2abbfff700 1 -- 172.21.15.142:0/600262528 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab40095f0 con 0x7f2ac4073a50 2024-03-29T05:04:23.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2ac37fe700 1 --2- 172.21.15.142:0/600262528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac4073130 0x7f2ac41a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:04:23.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2abbfff700 1 --2- 172.21.15.142:0/600262528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4073a50 0x7f2ac41a3010 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00c8a0 tx=0x7f2aac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:23.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2ac17fa700 1 -- 172.21.15.142:0/600262528 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2aac00cea0 con 0x7f2ac4073a50 2024-03-29T05:04:23.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2ac9d15700 1 -- 172.21.15.142:0/600262528 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac419ce80 con 0x7f2ac4073a50 2024-03-29T05:04:23.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2ac17fa700 1 -- 172.21.15.142:0/600262528 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2aac004830 con 0x7f2ac4073a50 2024-03-29T05:04:23.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2ac17fa700 1 -- 172.21.15.142:0/600262528 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2aac0055b0 con 0x7f2ac4073a50 2024-03-29T05:04:23.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.038+0000 7f2ac9d15700 1 -- 172.21.15.142:0/600262528 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac419d3d0 con 0x7f2ac4073a50 2024-03-29T05:04:23.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.040+0000 7f2ac9d15700 1 -- 172.21.15.142:0/600262528 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac410edb0 con 0x7f2ac4073a50 2024-03-29T05:04:23.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.040+0000 7f2ac17fa700 1 -- 172.21.15.142:0/600262528 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f2aac0049a0 con 0x7f2ac4073a50 2024-03-29T05:04:23.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.041+0000 7f2ac17fa700 1 --2- 172.21.15.142:0/600262528 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2ab006dd60 0x7f2ab0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:23.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.041+0000 7f2ac37fe700 1 --2- 172.21.15.142:0/600262528 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2ab006dd60 0x7f2ab0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:23.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.041+0000 7f2ac17fa700 1 -- 172.21.15.142:0/600262528 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f2aac08da20 con 0x7f2ac4073a50 2024-03-29T05:04:23.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.041+0000 7f2ac37fe700 1 --2- 172.21.15.142:0/600262528 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2ab006dd60 0x7f2ab0070220 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40059d0 tx=0x7f2ab401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:23.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.045+0000 7f2ac17fa700 1 -- 172.21.15.142:0/600262528 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2aac05a8d0 con 0x7f2ac4073a50 2024-03-29T05:04:23.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.318+0000 7f2ac9d15700 1 -- 172.21.15.142:0/600262528 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2ac404ea90 con 0x7f2ac4073a50 2024-03-29T05:04:23.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.319+0000 7f2ac17fa700 1 -- 172.21.15.142:0/600262528 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f2aac004c50 con 0x7f2ac4073a50 2024-03-29T05:04:23.328 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:04:23.329 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:04:23.330 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:04:23.330 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:04:23.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.321+0000 7f2ac9d15700 1 -- 172.21.15.142:0/600262528 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2ab006dd60 msgr2=0x7f2ab0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:23.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.321+0000 7f2ac9d15700 1 --2- 172.21.15.142:0/600262528 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2ab006dd60 0x7f2ab0070220 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40059d0 tx=0x7f2ab401a040 comp rx=0 tx=0).stop 2024-03-29T05:04:23.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.321+0000 7f2ac9d15700 1 -- 172.21.15.142:0/600262528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4073a50 msgr2=0x7f2ac41a3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:23.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.321+0000 7f2ac9d15700 1 --2- 172.21.15.142:0/600262528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4073a50 0x7f2ac41a3010 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00c8a0 tx=0x7f2aac00cc60 comp rx=0 tx=0).stop 2024-03-29T05:04:23.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.321+0000 7f2ac9d15700 1 -- 172.21.15.142:0/600262528 shutdown_connections 2024-03-29T05:04:23.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.321+0000 7f2ac9d15700 1 --2- 172.21.15.142:0/600262528 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2ab006dd60 0x7f2ab0070220 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.321+0000 7f2ac9d15700 1 --2- 172.21.15.142:0/600262528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac4073130 0x7f2ac41a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.321+0000 7f2ac9d15700 1 --2- 172.21.15.142:0/600262528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4073a50 0x7f2ac41a3010 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.321+0000 7f2ac9d15700 1 -- 172.21.15.142:0/600262528 >> 172.21.15.142:0/600262528 conn(0x7f2ac40fc650 msgr2=0x7f2ac4103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:23.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.321+0000 7f2ac9d15700 1 -- 172.21.15.142:0/600262528 shutdown_connections 2024-03-29T05:04:23.331 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.321+0000 7f2ac9d15700 1 -- 172.21.15.142:0/600262528 wait complete. 2024-03-29T05:04:23.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.424+0000 7f80a6d60700 1 -- 172.21.15.142:0/1607075594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a0105a40 msgr2=0x7f80a0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:23.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.424+0000 7f80a6d60700 1 --2- 172.21.15.142:0/1607075594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a0105a40 0x7f80a0105e20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8094009910 tx=0x7f8094009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:23.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.424+0000 7f80a6d60700 1 -- 172.21.15.142:0/1607075594 shutdown_connections 2024-03-29T05:04:23.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.424+0000 7f80a6d60700 1 --2- 172.21.15.142:0/1607075594 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80a00ffbd0 0x7f80a0100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.424+0000 7f80a6d60700 1 --2- 172.21.15.142:0/1607075594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a0105a40 0x7f80a0105e20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.424+0000 7f80a6d60700 1 -- 172.21.15.142:0/1607075594 >> 172.21.15.142:0/1607075594 conn(0x7f80a00755a0 msgr2=0x7f80a00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.424+0000 7f80a6d60700 1 -- 172.21.15.142:0/1607075594 shutdown_connections 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.424+0000 7f80a6d60700 1 -- 172.21.15.142:0/1607075594 wait complete. 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f80a6d60700 1 Processor -- start 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f80a6d60700 1 -- start start 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f80a6d60700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80a00ffbd0 0x7f80a019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f80a6d60700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a0105a40 0x7f80a019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f80a6d60700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a019d060 con 0x7f80a0105a40 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f80a6d60700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a0196520 con 0x7f80a00ffbd0 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f809ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a0105a40 0x7f80a019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f809ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a0105a40 0x7f80a019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55250/0 (socket says 172.21.15.142:55250) 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f809ffff700 1 -- 172.21.15.142:0/3140721624 learned_addr learned my addr 172.21.15.142:0/3140721624 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f80a4afc700 1 --2- 172.21.15.142:0/3140721624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80a00ffbd0 0x7f80a019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:23.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f809ffff700 1 -- 172.21.15.142:0/3140721624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80a00ffbd0 msgr2=0x7f80a019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:23.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f809ffff700 1 --2- 172.21.15.142:0/3140721624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80a00ffbd0 0x7f80a019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f809ffff700 1 -- 172.21.15.142:0/3140721624 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80940095f0 con 0x7f80a0105a40 2024-03-29T05:04:23.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.425+0000 7f809ffff700 1 --2- 172.21.15.142:0/3140721624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a0105a40 0x7f80a019c980 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f808c00d8d0 tx=0x7f808c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:23.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.426+0000 7f809dffb700 1 -- 172.21.15.142:0/3140721624 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f808c009940 con 0x7f80a0105a40 2024-03-29T05:04:23.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.426+0000 7f80a6d60700 1 -- 172.21.15.142:0/3140721624 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a0196800 con 0x7f80a0105a40 2024-03-29T05:04:23.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.426+0000 7f809dffb700 1 -- 172.21.15.142:0/3140721624 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f808c010460 con 0x7f80a0105a40 2024-03-29T05:04:23.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.426+0000 7f809dffb700 1 -- 172.21.15.142:0/3140721624 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f808c00f5f0 con 0x7f80a0105a40 2024-03-29T05:04:23.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.426+0000 7f80a6d60700 1 -- 172.21.15.142:0/3140721624 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a0196d50 con 0x7f80a0105a40 2024-03-29T05:04:23.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.427+0000 7f80a6d60700 1 -- 172.21.15.142:0/3140721624 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a0109050 con 0x7f80a0105a40 2024-03-29T05:04:23.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.427+0000 7f809dffb700 1 -- 172.21.15.142:0/3140721624 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f808c0105d0 con 0x7f80a0105a40 2024-03-29T05:04:23.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.428+0000 7f809dffb700 1 --2- 172.21.15.142:0/3140721624 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f809006dd60 0x7f8090070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:23.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.428+0000 7f809dffb700 1 -- 172.21.15.142:0/3140721624 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f808c08dcd0 con 0x7f80a0105a40 2024-03-29T05:04:23.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.428+0000 7f80a4afc700 1 --2- 172.21.15.142:0/3140721624 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f809006dd60 0x7f8090070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:23.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.428+0000 7f80a4afc700 1 --2- 172.21.15.142:0/3140721624 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f809006dd60 0x7f8090070220 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f809400b5c0 tx=0x7f809401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:23.442 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.432+0000 7f809dffb700 1 -- 172.21.15.142:0/3140721624 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f808c05ac20 con 0x7f80a0105a40 2024-03-29T05:04:23.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:23 smithi142 ceph-mon[112475]: pgmap v482: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:23.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:23 smithi142 ceph-mon[112475]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:23.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:23 smithi142 ceph-mon[112475]: from='client.34910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:23.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:23 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/600262528' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:04:23.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.670+0000 7f80a6d60700 1 -- 172.21.15.142:0/3140721624 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f80a004ea90 con 0x7f80a0105a40 2024-03-29T05:04:23.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.670+0000 7f809dffb700 1 -- 172.21.15.142:0/3140721624 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f808c016770 con 0x7f80a0105a40 2024-03-29T05:04:23.680 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:04:23.680 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:04:23.680 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:04:23.680 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:04:23.681 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:04:23.681 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:04:23.681 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:04:23.681 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:04:23.681 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:04:23.681 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:04:23.681 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:04:23.681 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:04:23.681 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:04:23.681 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:04:23.681 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:04:23.681 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:04:23.682 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:04:23.683 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:04:23.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.672+0000 7f80a6d60700 1 -- 172.21.15.142:0/3140721624 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f809006dd60 msgr2=0x7f8090070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:23.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.672+0000 7f80a6d60700 1 --2- 172.21.15.142:0/3140721624 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f809006dd60 0x7f8090070220 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f809400b5c0 tx=0x7f809401a040 comp rx=0 tx=0).stop 2024-03-29T05:04:23.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.672+0000 7f80a6d60700 1 -- 172.21.15.142:0/3140721624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a0105a40 msgr2=0x7f80a019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:23.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.672+0000 7f80a6d60700 1 --2- 172.21.15.142:0/3140721624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a0105a40 0x7f80a019c980 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f808c00d8d0 tx=0x7f808c00dc90 comp rx=0 tx=0).stop 2024-03-29T05:04:23.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.672+0000 7f80a6d60700 1 -- 172.21.15.142:0/3140721624 shutdown_connections 2024-03-29T05:04:23.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.672+0000 7f80a6d60700 1 --2- 172.21.15.142:0/3140721624 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f809006dd60 0x7f8090070220 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.672+0000 7f80a6d60700 1 --2- 172.21.15.142:0/3140721624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80a00ffbd0 0x7f80a019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.672+0000 7f80a6d60700 1 --2- 172.21.15.142:0/3140721624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80a0105a40 0x7f80a019c980 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.672+0000 7f80a6d60700 1 -- 172.21.15.142:0/3140721624 >> 172.21.15.142:0/3140721624 conn(0x7f80a00755a0 msgr2=0x7f80a00685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:23.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.673+0000 7f80a6d60700 1 -- 172.21.15.142:0/3140721624 shutdown_connections 2024-03-29T05:04:23.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.673+0000 7f80a6d60700 1 -- 172.21.15.142:0/3140721624 wait complete. 2024-03-29T05:04:23.685 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:04:23.776 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:23 smithi152 ceph-mon[88186]: pgmap v482: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:23.777 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:23 smithi152 ceph-mon[88186]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:23.777 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:23 smithi152 ceph-mon[88186]: from='client.34910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:23.777 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:23 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/600262528' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:04:23.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.781+0000 7fcca12f9700 1 -- 172.21.15.142:0/3214261517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c0739d0 msgr2=0x7fcc9c10cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:23.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.781+0000 7fcca12f9700 1 --2- 172.21.15.142:0/3214261517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c0739d0 0x7fcc9c10cf10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009910 tx=0x7fcc90009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:23.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.781+0000 7fcca12f9700 1 -- 172.21.15.142:0/3214261517 shutdown_connections 2024-03-29T05:04:23.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.781+0000 7fcca12f9700 1 --2- 172.21.15.142:0/3214261517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c0739d0 0x7fcc9c10cf10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.781+0000 7fcca12f9700 1 --2- 172.21.15.142:0/3214261517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc9c0730b0 0x7fcc9c073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.781+0000 7fcca12f9700 1 -- 172.21.15.142:0/3214261517 >> 172.21.15.142:0/3214261517 conn(0x7fcc9c0fc6a0 msgr2=0x7fcc9c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:23.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.781+0000 7fcca12f9700 1 -- 172.21.15.142:0/3214261517 shutdown_connections 2024-03-29T05:04:23.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.781+0000 7fcca12f9700 1 -- 172.21.15.142:0/3214261517 wait complete. 2024-03-29T05:04:23.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcca12f9700 1 Processor -- start 2024-03-29T05:04:23.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcca12f9700 1 -- start start 2024-03-29T05:04:23.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcca12f9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c0730b0 0x7fcc9c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcca12f9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc9c0739d0 0x7fcc9c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcca12f9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c199550 con 0x7fcc9c0730b0 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcca12f9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c19d360 con 0x7fcc9c0739d0 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcc9affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c0730b0 0x7fcc9c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcc9affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c0730b0 0x7fcc9c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55274/0 (socket says 172.21.15.142:55274) 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcc9affd700 1 -- 172.21.15.142:0/3994797895 learned_addr learned my addr 172.21.15.142:0/3994797895 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcc9a7fc700 1 --2- 172.21.15.142:0/3994797895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc9c0739d0 0x7fcc9c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcc9affd700 1 -- 172.21.15.142:0/3994797895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc9c0739d0 msgr2=0x7fcc9c198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcc9affd700 1 --2- 172.21.15.142:0/3994797895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc9c0739d0 0x7fcc9c198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcc9affd700 1 -- 172.21.15.142:0/3994797895 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc900095f0 con 0x7fcc9c0730b0 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcc9a7fc700 1 --2- 172.21.15.142:0/3994797895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc9c0739d0 0x7fcc9c198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.782+0000 7fcc9affd700 1 --2- 172.21.15.142:0/3994797895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c0730b0 0x7fcc9c198980 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00e920 tx=0x7fcc8c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:23.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.783+0000 7fcc83fff700 1 -- 172.21.15.142:0/3994797895 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc8c009790 con 0x7fcc9c0730b0 2024-03-29T05:04:23.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.783+0000 7fcca12f9700 1 -- 172.21.15.142:0/3994797895 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9c19d640 con 0x7fcc9c0730b0 2024-03-29T05:04:23.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.783+0000 7fcca12f9700 1 -- 172.21.15.142:0/3994797895 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9c19db90 con 0x7fcc9c0730b0 2024-03-29T05:04:23.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.783+0000 7fcc83fff700 1 -- 172.21.15.142:0/3994797895 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc8c00cba0 con 0x7fcc9c0730b0 2024-03-29T05:04:23.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.783+0000 7fcc83fff700 1 -- 172.21.15.142:0/3994797895 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc8c018650 con 0x7fcc9c0730b0 2024-03-29T05:04:23.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.784+0000 7fcca12f9700 1 -- 172.21.15.142:0/3994797895 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc88005320 con 0x7fcc9c0730b0 2024-03-29T05:04:23.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.784+0000 7fcc83fff700 1 -- 172.21.15.142:0/3994797895 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fcc8c010430 con 0x7fcc9c0730b0 2024-03-29T05:04:23.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.785+0000 7fcc83fff700 1 --2- 172.21.15.142:0/3994797895 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcc8406dd60 0x7fcc84070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:23.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.785+0000 7fcc83fff700 1 -- 172.21.15.142:0/3994797895 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fcc8c014070 con 0x7fcc9c0730b0 2024-03-29T05:04:23.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.785+0000 7fcc9a7fc700 1 --2- 172.21.15.142:0/3994797895 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcc8406dd60 0x7fcc84070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:23.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.785+0000 7fcc9a7fc700 1 --2- 172.21.15.142:0/3994797895 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcc8406dd60 0x7fcc84070220 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcc900059d0 tx=0x7fcc90005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:23.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:23.791+0000 7fcc83fff700 1 -- 172.21.15.142:0/3994797895 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcc8c053280 con 0x7fcc9c0730b0 2024-03-29T05:04:24.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.003+0000 7fcca12f9700 1 -- 172.21.15.142:0/3994797895 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc88000bf0 con 0x7fcc8406dd60 2024-03-29T05:04:24.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.004+0000 7fcc83fff700 1 -- 172.21.15.142:0/3994797895 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcc88000bf0 con 0x7fcc8406dd60 2024-03-29T05:04:24.014 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:04:24.014 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:04:24.014 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:04:24.015 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:04:24.015 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:04:24.015 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:04:24.015 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:04:24.015 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:04:24.015 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:04:24.015 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:04:24.015 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:04:24.015 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:04:24.015 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:04:24.015 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:04:24.015 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:04:24.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.007+0000 7fcca12f9700 1 -- 172.21.15.142:0/3994797895 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcc8406dd60 msgr2=0x7fcc84070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:24.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.007+0000 7fcca12f9700 1 --2- 172.21.15.142:0/3994797895 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcc8406dd60 0x7fcc84070220 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcc900059d0 tx=0x7fcc90005940 comp rx=0 tx=0).stop 2024-03-29T05:04:24.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.007+0000 7fcca12f9700 1 -- 172.21.15.142:0/3994797895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c0730b0 msgr2=0x7fcc9c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:24.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.007+0000 7fcca12f9700 1 --2- 172.21.15.142:0/3994797895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c0730b0 0x7fcc9c198980 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00e920 tx=0x7fcc8c00ece0 comp rx=0 tx=0).stop 2024-03-29T05:04:24.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.007+0000 7fcca12f9700 1 -- 172.21.15.142:0/3994797895 shutdown_connections 2024-03-29T05:04:24.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.007+0000 7fcca12f9700 1 --2- 172.21.15.142:0/3994797895 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcc8406dd60 0x7fcc84070220 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:24.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.007+0000 7fcca12f9700 1 --2- 172.21.15.142:0/3994797895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc9c0730b0 0x7fcc9c198980 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:24.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.007+0000 7fcca12f9700 1 --2- 172.21.15.142:0/3994797895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc9c0739d0 0x7fcc9c198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:24.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.007+0000 7fcca12f9700 1 -- 172.21.15.142:0/3994797895 >> 172.21.15.142:0/3994797895 conn(0x7fcc9c0fc6a0 msgr2=0x7fcc9c107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:24.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.007+0000 7fcca12f9700 1 -- 172.21.15.142:0/3994797895 shutdown_connections 2024-03-29T05:04:24.018 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.007+0000 7fcca12f9700 1 -- 172.21.15.142:0/3994797895 wait complete. 2024-03-29T05:04:24.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.113+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1852795994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4101750 msgr2=0x7f8de41057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:24.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.113+0000 7f8dea5ca700 1 --2- 172.21.15.142:0/1852795994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4101750 0x7f8de41057a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009910 tx=0x7f8dd8009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:24.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.113+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1852795994 shutdown_connections 2024-03-29T05:04:24.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.113+0000 7f8dea5ca700 1 --2- 172.21.15.142:0/1852795994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4101750 0x7f8de41057a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:24.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.113+0000 7f8dea5ca700 1 --2- 172.21.15.142:0/1852795994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de4100da0 0x7f8de4101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:24.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.113+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1852795994 >> 172.21.15.142:0/1852795994 conn(0x7f8de40fc630 msgr2=0x7f8de40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:24.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.113+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1852795994 shutdown_connections 2024-03-29T05:04:24.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.113+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1852795994 wait complete. 2024-03-29T05:04:24.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.113+0000 7f8dea5ca700 1 Processor -- start 2024-03-29T05:04:24.124 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8dea5ca700 1 -- start start 2024-03-29T05:04:24.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8dea5ca700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de4100da0 0x7f8de419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:24.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8dea5ca700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4101750 0x7f8de419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:24.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8dea5ca700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de419d0d0 con 0x7f8de4101750 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8dea5ca700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de4196580 con 0x7f8de4100da0 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8de37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4101750 0x7f8de419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8de37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4101750 0x7f8de419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55284/0 (socket says 172.21.15.142:55284) 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8de37fe700 1 -- 172.21.15.142:0/1543610389 learned_addr learned my addr 172.21.15.142:0/1543610389 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8de3fff700 1 --2- 172.21.15.142:0/1543610389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de4100da0 0x7f8de419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8de37fe700 1 -- 172.21.15.142:0/1543610389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de4100da0 msgr2=0x7f8de419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8de37fe700 1 --2- 172.21.15.142:0/1543610389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de4100da0 0x7f8de419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8de37fe700 1 -- 172.21.15.142:0/1543610389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd80095f0 con 0x7f8de4101750 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.114+0000 7f8de3fff700 1 --2- 172.21.15.142:0/1543610389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de4100da0 0x7f8de419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.115+0000 7f8de37fe700 1 --2- 172.21.15.142:0/1543610389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4101750 0x7f8de419ca40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80092e0 tx=0x7f8dd800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.115+0000 7f8de17fa700 1 -- 172.21.15.142:0/1543610389 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dd801d070 con 0x7f8de4101750 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.115+0000 7f8de17fa700 1 -- 172.21.15.142:0/1543610389 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8dd8005dc0 con 0x7f8de4101750 2024-03-29T05:04:24.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.115+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1543610389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de4196800 con 0x7f8de4101750 2024-03-29T05:04:24.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.115+0000 7f8de17fa700 1 -- 172.21.15.142:0/1543610389 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dd8017400 con 0x7f8de4101750 2024-03-29T05:04:24.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.115+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1543610389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de4196cf0 con 0x7f8de4101750 2024-03-29T05:04:24.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.116+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1543610389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de41091a0 con 0x7f8de4101750 2024-03-29T05:04:24.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.116+0000 7f8de17fa700 1 -- 172.21.15.142:0/1543610389 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f8dd800f460 con 0x7f8de4101750 2024-03-29T05:04:24.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.117+0000 7f8de17fa700 1 --2- 172.21.15.142:0/1543610389 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8dcc06dd60 0x7f8dcc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:24.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.117+0000 7f8de17fa700 1 -- 172.21.15.142:0/1543610389 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f8dd808f1b0 con 0x7f8de4101750 2024-03-29T05:04:24.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.117+0000 7f8de3fff700 1 --2- 172.21.15.142:0/1543610389 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8dcc06dd60 0x7f8dcc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:24.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.117+0000 7f8de3fff700 1 --2- 172.21.15.142:0/1543610389 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8dcc06dd60 0x7f8dcc070220 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4005950 tx=0x7f8dd40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:24.131 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.121+0000 7f8de17fa700 1 -- 172.21.15.142:0/1543610389 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8dd805c100 con 0x7f8de4101750 2024-03-29T05:04:24.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.392+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1543610389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8de402ccf0 con 0x7f8de4101750 2024-03-29T05:04:24.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.393+0000 7f8de17fa700 1 -- 172.21.15.142:0/1543610389 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f8dd805bc90 con 0x7f8de4101750 2024-03-29T05:04:24.403 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:04:24.403 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:04:24.403 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:04:24.403 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:04:24.403 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:04:24.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.395+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1543610389 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8dcc06dd60 msgr2=0x7f8dcc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:24.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.395+0000 7f8dea5ca700 1 --2- 172.21.15.142:0/1543610389 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8dcc06dd60 0x7f8dcc070220 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4005950 tx=0x7f8dd40058e0 comp rx=0 tx=0).stop 2024-03-29T05:04:24.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.395+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1543610389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4101750 msgr2=0x7f8de419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:24.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.395+0000 7f8dea5ca700 1 --2- 172.21.15.142:0/1543610389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4101750 0x7f8de419ca40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80092e0 tx=0x7f8dd800be20 comp rx=0 tx=0).stop 2024-03-29T05:04:24.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.395+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1543610389 shutdown_connections 2024-03-29T05:04:24.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.395+0000 7f8dea5ca700 1 --2- 172.21.15.142:0/1543610389 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8dcc06dd60 0x7f8dcc070220 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:24.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.395+0000 7f8dea5ca700 1 --2- 172.21.15.142:0/1543610389 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de4100da0 0x7f8de419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:24.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.395+0000 7f8dea5ca700 1 --2- 172.21.15.142:0/1543610389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4101750 0x7f8de419ca40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:24.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.395+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1543610389 >> 172.21.15.142:0/1543610389 conn(0x7f8de40fc630 msgr2=0x7f8de40fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:24.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.395+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1543610389 shutdown_connections 2024-03-29T05:04:24.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:24.395+0000 7f8dea5ca700 1 -- 172.21.15.142:0/1543610389 wait complete. 2024-03-29T05:04:24.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:24 smithi152 ceph-mon[88186]: from='client.34914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:24.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:24 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3140721624' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:04:24.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:24 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1543610389' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:04:24.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:24 smithi142 ceph-mon[112475]: from='client.34914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:24.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:24 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3140721624' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:04:24.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:24 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1543610389' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:04:25.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:25 smithi152 ceph-mon[88186]: pgmap v483: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:25.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:25 smithi152 ceph-mon[88186]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:25.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:25 smithi142 ceph-mon[112475]: pgmap v483: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:25.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:25 smithi142 ceph-mon[112475]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:27.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:27 smithi152 ceph-mon[88186]: pgmap v484: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:27.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:27 smithi142 ceph-mon[112475]: pgmap v484: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:29.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:29 smithi152 ceph-mon[88186]: pgmap v485: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:29.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:29 smithi142 ceph-mon[112475]: pgmap v485: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:30.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:04:30.882 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:04:31.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:31 smithi152 ceph-mon[88186]: pgmap v486: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:31.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:31 smithi142 ceph-mon[112475]: pgmap v486: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:33.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:33 smithi152 ceph-mon[88186]: pgmap v487: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:33.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:33 smithi142 ceph-mon[112475]: pgmap v487: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:35.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:35 smithi152 ceph-mon[88186]: pgmap v488: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:35.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:35 smithi142 ceph-mon[112475]: pgmap v488: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:37.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:37 smithi142 ceph-mon[112475]: pgmap v489: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:38.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:37 smithi152 ceph-mon[88186]: pgmap v489: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:39.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:39 smithi142 ceph-mon[112475]: pgmap v490: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:40.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:39 smithi152 ceph-mon[88186]: pgmap v490: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:41.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:41 smithi142 ceph-mon[112475]: pgmap v491: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:42.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:41 smithi152 ceph-mon[88186]: pgmap v491: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:43.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:43 smithi142 ceph-mon[112475]: pgmap v492: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:43.992 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:43 smithi152 ceph-mon[88186]: pgmap v492: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:45.882 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:45 smithi142 ceph-mon[112475]: pgmap v493: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:45.882 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:04:46.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:45 smithi152 ceph-mon[88186]: pgmap v493: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:46.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:04:47.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:47 smithi142 ceph-mon[112475]: pgmap v494: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:48.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:47 smithi152 ceph-mon[88186]: pgmap v494: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:49.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:49 smithi142 ceph-mon[112475]: pgmap v495: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:50.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:49 smithi152 ceph-mon[88186]: pgmap v495: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:51.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:51 smithi142 ceph-mon[112475]: pgmap v496: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:52.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:51 smithi152 ceph-mon[88186]: pgmap v496: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:53.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:53 smithi142 ceph-mon[112475]: pgmap v497: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:54.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:53 smithi152 ceph-mon[88186]: pgmap v497: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:54.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.512+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/1781578183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e58100dd0 msgr2=0x7f1e581011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:54.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.512+0000 7f1e5f0e7700 1 --2- 172.21.15.142:0/1781578183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e58100dd0 0x7f1e581011b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009910 tx=0x7f1e4c009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:54.524 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.512+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/1781578183 shutdown_connections 2024-03-29T05:04:54.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.512+0000 7f1e5f0e7700 1 --2- 172.21.15.142:0/1781578183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e58101780 0x7f1e581057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:54.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.512+0000 7f1e5f0e7700 1 --2- 172.21.15.142:0/1781578183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e58100dd0 0x7f1e581011b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:54.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.512+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/1781578183 >> 172.21.15.142:0/1781578183 conn(0x7f1e580fc640 msgr2=0x7f1e580fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:54.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.512+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/1781578183 shutdown_connections 2024-03-29T05:04:54.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.512+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/1781578183 wait complete. 2024-03-29T05:04:54.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.513+0000 7f1e5f0e7700 1 Processor -- start 2024-03-29T05:04:54.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.513+0000 7f1e5f0e7700 1 -- start start 2024-03-29T05:04:54.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.513+0000 7f1e5f0e7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e58100dd0 0x7f1e5819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:54.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.513+0000 7f1e5f0e7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e58101780 0x7f1e5819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:54.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.513+0000 7f1e5f0e7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5819d050 con 0x7f1e58100dd0 2024-03-29T05:04:54.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.513+0000 7f1e5f0e7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e58196590 con 0x7f1e58101780 2024-03-29T05:04:54.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.513+0000 7f1e5ce83700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e58100dd0 0x7f1e5819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.513+0000 7f1e5ce83700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e58100dd0 0x7f1e5819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54802/0 (socket says 172.21.15.142:54802) 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.513+0000 7f1e5ce83700 1 -- 172.21.15.142:0/3633799202 learned_addr learned my addr 172.21.15.142:0/3633799202 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.514+0000 7f1e57fff700 1 --2- 172.21.15.142:0/3633799202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e58101780 0x7f1e5819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.514+0000 7f1e5ce83700 1 -- 172.21.15.142:0/3633799202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e58101780 msgr2=0x7f1e5819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.514+0000 7f1e5ce83700 1 --2- 172.21.15.142:0/3633799202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e58101780 0x7f1e5819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.514+0000 7f1e5ce83700 1 -- 172.21.15.142:0/3633799202 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c0095f0 con 0x7f1e58100dd0 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.514+0000 7f1e57fff700 1 --2- 172.21.15.142:0/3633799202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e58101780 0x7f1e5819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.514+0000 7f1e5ce83700 1 --2- 172.21.15.142:0/3633799202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e58100dd0 0x7f1e5819c510 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c006010 tx=0x7f1e4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.514+0000 7f1e55ffb700 1 -- 172.21.15.142:0/3633799202 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e4c01d070 con 0x7f1e58100dd0 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.514+0000 7f1e55ffb700 1 -- 172.21.15.142:0/3633799202 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e4c004fb0 con 0x7f1e58100dd0 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.514+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/3633799202 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e58196810 con 0x7f1e58100dd0 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.514+0000 7f1e55ffb700 1 -- 172.21.15.142:0/3633799202 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e4c022ea0 con 0x7f1e58100dd0 2024-03-29T05:04:54.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.514+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/3633799202 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e58196d00 con 0x7f1e58100dd0 2024-03-29T05:04:54.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.515+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/3633799202 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e58109170 con 0x7f1e58100dd0 2024-03-29T05:04:54.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.516+0000 7f1e55ffb700 1 -- 172.21.15.142:0/3633799202 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f1e4c004af0 con 0x7f1e58100dd0 2024-03-29T05:04:54.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.516+0000 7f1e55ffb700 1 --2- 172.21.15.142:0/3633799202 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e4806dd60 0x7f1e48070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:54.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.516+0000 7f1e55ffb700 1 -- 172.21.15.142:0/3633799202 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1e4c08f130 con 0x7f1e58100dd0 2024-03-29T05:04:54.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.516+0000 7f1e57fff700 1 --2- 172.21.15.142:0/3633799202 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e4806dd60 0x7f1e48070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:54.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.517+0000 7f1e57fff700 1 --2- 172.21.15.142:0/3633799202 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e4806dd60 0x7f1e48070220 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005fd0 tx=0x7f1e44005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:54.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.521+0000 7f1e55ffb700 1 -- 172.21.15.142:0/3633799202 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1e4c05c2d0 con 0x7f1e58100dd0 2024-03-29T05:04:54.746 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.736+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/3633799202 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e58196fd0 con 0x7f1e4806dd60 2024-03-29T05:04:54.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.737+0000 7f1e55ffb700 1 -- 172.21.15.142:0/3633799202 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1e58196fd0 con 0x7f1e4806dd60 2024-03-29T05:04:54.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.739+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/3633799202 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e4806dd60 msgr2=0x7f1e48070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.739+0000 7f1e5f0e7700 1 --2- 172.21.15.142:0/3633799202 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e4806dd60 0x7f1e48070220 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005fd0 tx=0x7f1e44005e20 comp rx=0 tx=0).stop 2024-03-29T05:04:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.739+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/3633799202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e58100dd0 msgr2=0x7f1e5819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.739+0000 7f1e5f0e7700 1 --2- 172.21.15.142:0/3633799202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e58100dd0 0x7f1e5819c510 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c006010 tx=0x7f1e4c00be20 comp rx=0 tx=0).stop 2024-03-29T05:04:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.739+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/3633799202 shutdown_connections 2024-03-29T05:04:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.739+0000 7f1e5f0e7700 1 --2- 172.21.15.142:0/3633799202 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e4806dd60 0x7f1e48070220 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.739+0000 7f1e5f0e7700 1 --2- 172.21.15.142:0/3633799202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e58100dd0 0x7f1e5819c510 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.739+0000 7f1e5f0e7700 1 --2- 172.21.15.142:0/3633799202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e58101780 0x7f1e5819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.739+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/3633799202 >> 172.21.15.142:0/3633799202 conn(0x7f1e580fc640 msgr2=0x7f1e580fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.739+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/3633799202 shutdown_connections 2024-03-29T05:04:54.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.739+0000 7f1e5f0e7700 1 -- 172.21.15.142:0/3633799202 wait complete. 2024-03-29T05:04:54.760 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:04:54.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.845+0000 7fb5d3627700 1 -- 172.21.15.142:0/1762997625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5cc100420 msgr2=0x7fb5cc100880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:54.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.845+0000 7fb5d3627700 1 --2- 172.21.15.142:0/1762997625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5cc100420 0x7fb5cc100880 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009910 tx=0x7fb5c0009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:54.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.845+0000 7fb5d3627700 1 -- 172.21.15.142:0/1762997625 shutdown_connections 2024-03-29T05:04:54.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.845+0000 7fb5d3627700 1 --2- 172.21.15.142:0/1762997625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5cc100420 0x7fb5cc100880 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:54.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.845+0000 7fb5d3627700 1 --2- 172.21.15.142:0/1762997625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5cc105b10 0x7fb5cc105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:54.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.845+0000 7fb5d3627700 1 -- 172.21.15.142:0/1762997625 >> 172.21.15.142:0/1762997625 conn(0x7fb5cc074bf0 msgr2=0x7fb5cc075000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:54.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.845+0000 7fb5d3627700 1 -- 172.21.15.142:0/1762997625 shutdown_connections 2024-03-29T05:04:54.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.845+0000 7fb5d3627700 1 -- 172.21.15.142:0/1762997625 wait complete. 2024-03-29T05:04:54.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.846+0000 7fb5d3627700 1 Processor -- start 2024-03-29T05:04:54.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.846+0000 7fb5d3627700 1 -- start start 2024-03-29T05:04:54.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.846+0000 7fb5d3627700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5cc100420 0x7fb5cc101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:54.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.846+0000 7fb5d3627700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5cc105b10 0x7fb5cc101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:54.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.846+0000 7fb5d3627700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc102200 con 0x7fb5cc100420 2024-03-29T05:04:54.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.846+0000 7fb5d3627700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc102340 con 0x7fb5cc105b10 2024-03-29T05:04:54.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.846+0000 7fb5d13c3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5cc100420 0x7fb5cc101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.846+0000 7fb5d13c3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5cc100420 0x7fb5cc101780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54816/0 (socket says 172.21.15.142:54816) 2024-03-29T05:04:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.846+0000 7fb5d13c3700 1 -- 172.21.15.142:0/2577812317 learned_addr learned my addr 172.21.15.142:0/2577812317 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.846+0000 7fb5d0bc2700 1 --2- 172.21.15.142:0/2577812317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5cc105b10 0x7fb5cc101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.846+0000 7fb5d13c3700 1 -- 172.21.15.142:0/2577812317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5cc105b10 msgr2=0x7fb5cc101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.846+0000 7fb5d13c3700 1 --2- 172.21.15.142:0/2577812317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5cc105b10 0x7fb5cc101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.847+0000 7fb5d13c3700 1 -- 172.21.15.142:0/2577812317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c00095f0 con 0x7fb5cc100420 2024-03-29T05:04:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.847+0000 7fb5d13c3700 1 --2- 172.21.15.142:0/2577812317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5cc100420 0x7fb5cc101780 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00c8a0 tx=0x7fb5bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.847+0000 7fb5ca7fc700 1 -- 172.21.15.142:0/2577812317 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5bc00cea0 con 0x7fb5cc100420 2024-03-29T05:04:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.847+0000 7fb5ca7fc700 1 -- 172.21.15.142:0/2577812317 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5bc004830 con 0x7fb5cc100420 2024-03-29T05:04:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.847+0000 7fb5d3627700 1 -- 172.21.15.142:0/2577812317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5cc1a28e0 con 0x7fb5cc100420 2024-03-29T05:04:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.847+0000 7fb5ca7fc700 1 -- 172.21.15.142:0/2577812317 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5bc0055b0 con 0x7fb5cc100420 2024-03-29T05:04:54.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.847+0000 7fb5d3627700 1 -- 172.21.15.142:0/2577812317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5cc1a2cf0 con 0x7fb5cc100420 2024-03-29T05:04:54.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.848+0000 7fb5d3627700 1 -- 172.21.15.142:0/2577812317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5cc109020 con 0x7fb5cc100420 2024-03-29T05:04:54.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.848+0000 7fb5ca7fc700 1 -- 172.21.15.142:0/2577812317 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fb5bc0049a0 con 0x7fb5cc100420 2024-03-29T05:04:54.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.849+0000 7fb5ca7fc700 1 --2- 172.21.15.142:0/2577812317 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb5b406dd60 0x7fb5b4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:54.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.849+0000 7fb5ca7fc700 1 -- 172.21.15.142:0/2577812317 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb5bc08d880 con 0x7fb5cc100420 2024-03-29T05:04:54.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.849+0000 7fb5d0bc2700 1 --2- 172.21.15.142:0/2577812317 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb5b406dd60 0x7fb5b4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:54.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.849+0000 7fb5d0bc2700 1 --2- 172.21.15.142:0/2577812317 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb5b406dd60 0x7fb5b4070220 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00059d0 tx=0x7fb5c001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:54.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:54.853+0000 7fb5ca7fc700 1 -- 172.21.15.142:0/2577812317 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb5bc056d20 con 0x7fb5cc100420 2024-03-29T05:04:55.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.068+0000 7fb5d3627700 1 -- 172.21.15.142:0/2577812317 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5cc02cd80 con 0x7fb5b406dd60 2024-03-29T05:04:55.078 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.069+0000 7fb5ca7fc700 1 -- 172.21.15.142:0/2577812317 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb5cc02cd80 con 0x7fb5b406dd60 2024-03-29T05:04:55.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.071+0000 7fb5d3627700 1 -- 172.21.15.142:0/2577812317 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb5b406dd60 msgr2=0x7fb5b4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:55.081 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.071+0000 7fb5d3627700 1 --2- 172.21.15.142:0/2577812317 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb5b406dd60 0x7fb5b4070220 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00059d0 tx=0x7fb5c001a040 comp rx=0 tx=0).stop 2024-03-29T05:04:55.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.071+0000 7fb5d3627700 1 -- 172.21.15.142:0/2577812317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5cc100420 msgr2=0x7fb5cc101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:55.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.071+0000 7fb5d3627700 1 --2- 172.21.15.142:0/2577812317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5cc100420 0x7fb5cc101780 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00c8a0 tx=0x7fb5bc00cc60 comp rx=0 tx=0).stop 2024-03-29T05:04:55.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.071+0000 7fb5d3627700 1 -- 172.21.15.142:0/2577812317 shutdown_connections 2024-03-29T05:04:55.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.071+0000 7fb5d3627700 1 --2- 172.21.15.142:0/2577812317 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb5b406dd60 0x7fb5b4070220 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.071+0000 7fb5d3627700 1 --2- 172.21.15.142:0/2577812317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5cc100420 0x7fb5cc101780 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.071+0000 7fb5d3627700 1 --2- 172.21.15.142:0/2577812317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5cc105b10 0x7fb5cc101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.071+0000 7fb5d3627700 1 -- 172.21.15.142:0/2577812317 >> 172.21.15.142:0/2577812317 conn(0x7fb5cc074bf0 msgr2=0x7fb5cc104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:55.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.071+0000 7fb5d3627700 1 -- 172.21.15.142:0/2577812317 shutdown_connections 2024-03-29T05:04:55.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.071+0000 7fb5d3627700 1 -- 172.21.15.142:0/2577812317 wait complete. 2024-03-29T05:04:55.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.176+0000 7fbace317700 1 -- 172.21.15.142:0/2525066358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbac80ffe90 msgr2=0x7fbac8111260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:55.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.176+0000 7fbace317700 1 --2- 172.21.15.142:0/2525066358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbac80ffe90 0x7fbac8111260 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009910 tx=0x7fbabc009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:55.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.176+0000 7fbace317700 1 -- 172.21.15.142:0/2525066358 shutdown_connections 2024-03-29T05:04:55.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.176+0000 7fbace317700 1 --2- 172.21.15.142:0/2525066358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbac80ffe90 0x7fbac8111260 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.176+0000 7fbace317700 1 --2- 172.21.15.142:0/2525066358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac80ff570 0x7fbac80ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.176+0000 7fbace317700 1 -- 172.21.15.142:0/2525066358 >> 172.21.15.142:0/2525066358 conn(0x7fbac8074bf0 msgr2=0x7fbac8075000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:55.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.176+0000 7fbace317700 1 -- 172.21.15.142:0/2525066358 shutdown_connections 2024-03-29T05:04:55.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.176+0000 7fbace317700 1 -- 172.21.15.142:0/2525066358 wait complete. 2024-03-29T05:04:55.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.176+0000 7fbace317700 1 Processor -- start 2024-03-29T05:04:55.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.176+0000 7fbace317700 1 -- start start 2024-03-29T05:04:55.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbace317700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbac80ff570 0x7fbac819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:55.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbace317700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac80ffe90 0x7fbac819d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:55.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbace317700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac819d9d0 con 0x7fbac80ff570 2024-03-29T05:04:55.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbace317700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac81a1760 con 0x7fbac80ffe90 2024-03-29T05:04:55.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbac7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbac80ff570 0x7fbac819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:55.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbac7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbac80ff570 0x7fbac819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54834/0 (socket says 172.21.15.142:54834) 2024-03-29T05:04:55.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbac7fff700 1 -- 172.21.15.142:0/3722905901 learned_addr learned my addr 172.21.15.142:0/3722905901 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:55.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbac77fe700 1 --2- 172.21.15.142:0/3722905901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac80ffe90 0x7fbac819d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:55.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbac7fff700 1 -- 172.21.15.142:0/3722905901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac80ffe90 msgr2=0x7fbac819d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:55.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbac7fff700 1 --2- 172.21.15.142:0/3722905901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac80ffe90 0x7fbac819d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbac7fff700 1 -- 172.21.15.142:0/3722905901 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc0095f0 con 0x7fbac80ff570 2024-03-29T05:04:55.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbac77fe700 1 --2- 172.21.15.142:0/3722905901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac80ffe90 0x7fbac819d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:04:55.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.177+0000 7fbac7fff700 1 --2- 172.21.15.142:0/3722905901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbac80ff570 0x7fbac819cdb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbab800d8d0 tx=0x7fbab800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:55.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.178+0000 7fbac57fa700 1 -- 172.21.15.142:0/3722905901 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbab8009940 con 0x7fbac80ff570 2024-03-29T05:04:55.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.178+0000 7fbac57fa700 1 -- 172.21.15.142:0/3722905901 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbab8010460 con 0x7fbac80ff570 2024-03-29T05:04:55.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.178+0000 7fbace317700 1 -- 172.21.15.142:0/3722905901 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac81a1a40 con 0x7fbac80ff570 2024-03-29T05:04:55.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.178+0000 7fbac57fa700 1 -- 172.21.15.142:0/3722905901 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbab800f5f0 con 0x7fbac80ff570 2024-03-29T05:04:55.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.178+0000 7fbace317700 1 -- 172.21.15.142:0/3722905901 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac81a1f90 con 0x7fbac80ff570 2024-03-29T05:04:55.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.179+0000 7fbace317700 1 -- 172.21.15.142:0/3722905901 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac804ea90 con 0x7fbac80ff570 2024-03-29T05:04:55.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.179+0000 7fbac57fa700 1 -- 172.21.15.142:0/3722905901 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fbab80105d0 con 0x7fbac80ff570 2024-03-29T05:04:55.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.179+0000 7fbac57fa700 1 --2- 172.21.15.142:0/3722905901 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbab006dd60 0x7fbab0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:55.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.180+0000 7fbac57fa700 1 -- 172.21.15.142:0/3722905901 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fbab808db80 con 0x7fbac80ff570 2024-03-29T05:04:55.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.180+0000 7fbac77fe700 1 --2- 172.21.15.142:0/3722905901 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbab006dd60 0x7fbab0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:55.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.180+0000 7fbac77fe700 1 --2- 172.21.15.142:0/3722905901 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbab006dd60 0x7fbab0070220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbac819e3d0 tx=0x7fbabc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:55.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.184+0000 7fbac57fa700 1 -- 172.21.15.142:0/3722905901 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbab8057020 con 0x7fbac80ff570 2024-03-29T05:04:55.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.387+0000 7fbace317700 1 -- 172.21.15.142:0/3722905901 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbac80008d0 con 0x7fbab006dd60 2024-03-29T05:04:55.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.397+0000 7fbac57fa700 1 -- 172.21.15.142:0/3722905901 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbac80008d0 con 0x7fbab006dd60 2024-03-29T05:04:55.408 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:04:55.408 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (16m) 9m ago 28m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:04:55.408 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (28m) 9m ago 28m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:04:55.408 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (27m) 9m ago 27m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:04:55.408 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (13m) 9m ago 28m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:04:55.408 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (13m) 9m ago 27m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:04:55.408 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (16m) 9m ago 27m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:04:55.408 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (9m) 9m ago 22m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:04:55.408 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (10m) 9m ago 22m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:04:55.408 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (9m) 9m ago 22m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:04:55.408 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (9m) 9m ago 22m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:04:55.408 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (18m) 9m ago 29m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:04:55.409 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (17m) 9m ago 27m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:04:55.409 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (15m) 9m ago 29m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:04:55.409 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (14m) 9m ago 27m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:04:55.409 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (17m) 9m ago 28m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:04:55.409 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (17m) 9m ago 27m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:04:55.409 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (13m) 9m ago 26m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:04:55.409 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (12m) 9m ago 25m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:04:55.409 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (12m) 9m ago 25m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:04:55.409 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (11m) 9m ago 25m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:04:55.409 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (11m) 9m ago 24m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:04:55.409 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (10m) 9m ago 24m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:04:55.409 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (17m) 9m ago 27m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:04:55.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.399+0000 7fbace317700 1 -- 172.21.15.142:0/3722905901 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbab006dd60 msgr2=0x7fbab0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:55.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.399+0000 7fbace317700 1 --2- 172.21.15.142:0/3722905901 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbab006dd60 0x7fbab0070220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbac819e3d0 tx=0x7fbabc01a040 comp rx=0 tx=0).stop 2024-03-29T05:04:55.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.399+0000 7fbace317700 1 -- 172.21.15.142:0/3722905901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbac80ff570 msgr2=0x7fbac819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:55.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.399+0000 7fbace317700 1 --2- 172.21.15.142:0/3722905901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbac80ff570 0x7fbac819cdb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbab800d8d0 tx=0x7fbab800dc90 comp rx=0 tx=0).stop 2024-03-29T05:04:55.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.400+0000 7fbace317700 1 -- 172.21.15.142:0/3722905901 shutdown_connections 2024-03-29T05:04:55.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.400+0000 7fbace317700 1 --2- 172.21.15.142:0/3722905901 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbab006dd60 0x7fbab0070220 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.400+0000 7fbace317700 1 --2- 172.21.15.142:0/3722905901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbac80ff570 0x7fbac819cdb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.400+0000 7fbace317700 1 --2- 172.21.15.142:0/3722905901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbac80ffe90 0x7fbac819d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.400+0000 7fbace317700 1 -- 172.21.15.142:0/3722905901 >> 172.21.15.142:0/3722905901 conn(0x7fbac8074bf0 msgr2=0x7fbac810f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:55.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.400+0000 7fbace317700 1 -- 172.21.15.142:0/3722905901 shutdown_connections 2024-03-29T05:04:55.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.400+0000 7fbace317700 1 -- 172.21.15.142:0/3722905901 wait complete. 2024-03-29T05:04:55.512 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.502+0000 7f312e163700 1 -- 172.21.15.142:0/2396046625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3128105a40 msgr2=0x7f3128105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:55.512 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.502+0000 7f312e163700 1 --2- 172.21.15.142:0/2396046625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3128105a40 0x7f3128105e20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3118009910 tx=0x7f3118009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:55.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.502+0000 7f312e163700 1 -- 172.21.15.142:0/2396046625 shutdown_connections 2024-03-29T05:04:55.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.502+0000 7f312e163700 1 --2- 172.21.15.142:0/2396046625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31280ffbd0 0x7f3128100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.502+0000 7f312e163700 1 --2- 172.21.15.142:0/2396046625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3128105a40 0x7f3128105e20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.502+0000 7f312e163700 1 -- 172.21.15.142:0/2396046625 >> 172.21.15.142:0/2396046625 conn(0x7f31280755a0 msgr2=0x7f31280759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:55.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.502+0000 7f312e163700 1 -- 172.21.15.142:0/2396046625 shutdown_connections 2024-03-29T05:04:55.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.502+0000 7f312e163700 1 -- 172.21.15.142:0/2396046625 wait complete. 2024-03-29T05:04:55.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.503+0000 7f312e163700 1 Processor -- start 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.503+0000 7f312e163700 1 -- start start 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.503+0000 7f312e163700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31280ffbd0 0x7f31281975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.503+0000 7f312e163700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3128105a40 0x7f3128197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.503+0000 7f312e163700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3128198110 con 0x7f31280ffbd0 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.503+0000 7f312e163700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3128198250 con 0x7f3128105a40 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.503+0000 7f31277fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31280ffbd0 0x7f31281975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.503+0000 7f31277fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31280ffbd0 0x7f31281975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54852/0 (socket says 172.21.15.142:54852) 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.503+0000 7f31277fe700 1 -- 172.21.15.142:0/1644518097 learned_addr learned my addr 172.21.15.142:0/1644518097 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.504+0000 7f3126ffd700 1 --2- 172.21.15.142:0/1644518097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3128105a40 0x7f3128197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.504+0000 7f31277fe700 1 -- 172.21.15.142:0/1644518097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3128105a40 msgr2=0x7f3128197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.504+0000 7f31277fe700 1 --2- 172.21.15.142:0/1644518097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3128105a40 0x7f3128197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.504+0000 7f31277fe700 1 -- 172.21.15.142:0/1644518097 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31180095f0 con 0x7f31280ffbd0 2024-03-29T05:04:55.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.504+0000 7f31277fe700 1 --2- 172.21.15.142:0/1644518097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31280ffbd0 0x7f31281975b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3118006010 tx=0x7f311800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:55.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.504+0000 7f3124ff9700 1 -- 172.21.15.142:0/1644518097 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f311801d070 con 0x7f31280ffbd0 2024-03-29T05:04:55.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.504+0000 7f3124ff9700 1 -- 172.21.15.142:0/1644518097 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3118005dc0 con 0x7f31280ffbd0 2024-03-29T05:04:55.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.504+0000 7f3124ff9700 1 -- 172.21.15.142:0/1644518097 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3118017400 con 0x7f31280ffbd0 2024-03-29T05:04:55.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.504+0000 7f312e163700 1 -- 172.21.15.142:0/1644518097 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31281a0620 con 0x7f31280ffbd0 2024-03-29T05:04:55.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.504+0000 7f312e163700 1 -- 172.21.15.142:0/1644518097 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31281a0a30 con 0x7f31280ffbd0 2024-03-29T05:04:55.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.505+0000 7f312e163700 1 -- 172.21.15.142:0/1644518097 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3128108040 con 0x7f31280ffbd0 2024-03-29T05:04:55.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.506+0000 7f3124ff9700 1 -- 172.21.15.142:0/1644518097 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f3118003a60 con 0x7f31280ffbd0 2024-03-29T05:04:55.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.506+0000 7f3124ff9700 1 --2- 172.21.15.142:0/1644518097 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f311006dd60 0x7f3110070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:55.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.506+0000 7f3124ff9700 1 -- 172.21.15.142:0/1644518097 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f311808f0b0 con 0x7f31280ffbd0 2024-03-29T05:04:55.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.506+0000 7f3126ffd700 1 --2- 172.21.15.142:0/1644518097 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f311006dd60 0x7f3110070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:55.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.507+0000 7f3126ffd700 1 --2- 172.21.15.142:0/1644518097 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f311006dd60 0x7f3110070220 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3128068cc0 tx=0x7f311c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:55.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.511+0000 7f3124ff9700 1 -- 172.21.15.142:0/1644518097 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f31180584f0 con 0x7f31280ffbd0 2024-03-29T05:04:55.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.794+0000 7f312e163700 1 -- 172.21.15.142:0/1644518097 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f312804ea90 con 0x7f31280ffbd0 2024-03-29T05:04:55.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:55 smithi142 ceph-mon[112475]: pgmap v498: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:55.805 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:55 smithi142 ceph-mon[112475]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:55.805 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.795+0000 7f3124ff9700 1 -- 172.21.15.142:0/1644518097 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f311805bbb0 con 0x7f31280ffbd0 2024-03-29T05:04:55.805 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:04:55.805 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:04:55.805 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:04:55.806 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:04:55.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.797+0000 7f312e163700 1 -- 172.21.15.142:0/1644518097 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f311006dd60 msgr2=0x7f3110070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:55.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.797+0000 7f312e163700 1 --2- 172.21.15.142:0/1644518097 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f311006dd60 0x7f3110070220 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3128068cc0 tx=0x7f311c00b410 comp rx=0 tx=0).stop 2024-03-29T05:04:55.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.797+0000 7f312e163700 1 -- 172.21.15.142:0/1644518097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31280ffbd0 msgr2=0x7f31281975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:55.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.797+0000 7f312e163700 1 --2- 172.21.15.142:0/1644518097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31280ffbd0 0x7f31281975b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3118006010 tx=0x7f311800bd70 comp rx=0 tx=0).stop 2024-03-29T05:04:55.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.797+0000 7f312e163700 1 -- 172.21.15.142:0/1644518097 shutdown_connections 2024-03-29T05:04:55.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.797+0000 7f312e163700 1 --2- 172.21.15.142:0/1644518097 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f311006dd60 0x7f3110070220 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.797+0000 7f312e163700 1 --2- 172.21.15.142:0/1644518097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31280ffbd0 0x7f31281975b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.797+0000 7f312e163700 1 --2- 172.21.15.142:0/1644518097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3128105a40 0x7f3128197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.797+0000 7f312e163700 1 -- 172.21.15.142:0/1644518097 >> 172.21.15.142:0/1644518097 conn(0x7f31280755a0 msgr2=0x7f31280fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:55.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.797+0000 7f312e163700 1 -- 172.21.15.142:0/1644518097 shutdown_connections 2024-03-29T05:04:55.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.797+0000 7f312e163700 1 -- 172.21.15.142:0/1644518097 wait complete. 2024-03-29T05:04:55.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.900+0000 7f41c82a0700 1 -- 172.21.15.142:0/4062041318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0101750 msgr2=0x7f41c01057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:55.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.900+0000 7f41c82a0700 1 --2- 172.21.15.142:0/4062041318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0101750 0x7f41c01057a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009910 tx=0x7f41b0009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:55.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.900+0000 7f41c82a0700 1 -- 172.21.15.142:0/4062041318 shutdown_connections 2024-03-29T05:04:55.912 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.900+0000 7f41c82a0700 1 --2- 172.21.15.142:0/4062041318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0101750 0x7f41c01057a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.900+0000 7f41c82a0700 1 --2- 172.21.15.142:0/4062041318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41c0100da0 0x7f41c0101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.900+0000 7f41c82a0700 1 -- 172.21.15.142:0/4062041318 >> 172.21.15.142:0/4062041318 conn(0x7f41c00fc650 msgr2=0x7f41c00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:55.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.900+0000 7f41c82a0700 1 -- 172.21.15.142:0/4062041318 shutdown_connections 2024-03-29T05:04:55.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.900+0000 7f41c82a0700 1 -- 172.21.15.142:0/4062041318 wait complete. 2024-03-29T05:04:55.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.900+0000 7f41c82a0700 1 Processor -- start 2024-03-29T05:04:55.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.900+0000 7f41c82a0700 1 -- start start 2024-03-29T05:04:55.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c82a0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41c0100da0 0x7f41c019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:55.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c82a0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0101750 0x7f41c019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:55.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c82a0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c019d0e0 con 0x7f41c0101750 2024-03-29T05:04:55.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c82a0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c0196590 con 0x7f41c0100da0 2024-03-29T05:04:55.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c583b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0101750 0x7f41c019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:55.913 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c583b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0101750 0x7f41c019ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54866/0 (socket says 172.21.15.142:54866) 2024-03-29T05:04:55.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c583b700 1 -- 172.21.15.142:0/1792088279 learned_addr learned my addr 172.21.15.142:0/1792088279 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:55.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c603c700 1 --2- 172.21.15.142:0/1792088279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41c0100da0 0x7f41c019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:55.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c583b700 1 -- 172.21.15.142:0/1792088279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41c0100da0 msgr2=0x7f41c019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:55.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c583b700 1 --2- 172.21.15.142:0/1792088279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41c0100da0 0x7f41c019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:55.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c583b700 1 -- 172.21.15.142:0/1792088279 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b00095f0 con 0x7f41c0101750 2024-03-29T05:04:55.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c583b700 1 --2- 172.21.15.142:0/1792088279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0101750 0x7f41c019ca50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f41b00059d0 tx=0x7f41b000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:55.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41b77fe700 1 -- 172.21.15.142:0/1792088279 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b001d070 con 0x7f41c0101750 2024-03-29T05:04:55.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41b77fe700 1 -- 172.21.15.142:0/1792088279 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41b000b610 con 0x7f41c0101750 2024-03-29T05:04:55.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c82a0700 1 -- 172.21.15.142:0/1792088279 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c0196810 con 0x7f41c0101750 2024-03-29T05:04:55.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41b77fe700 1 -- 172.21.15.142:0/1792088279 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b0022470 con 0x7f41c0101750 2024-03-29T05:04:55.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.901+0000 7f41c82a0700 1 -- 172.21.15.142:0/1792088279 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c0196d00 con 0x7f41c0101750 2024-03-29T05:04:55.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.902+0000 7f41c82a0700 1 -- 172.21.15.142:0/1792088279 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c01091a0 con 0x7f41c0101750 2024-03-29T05:04:55.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.903+0000 7f41b77fe700 1 -- 172.21.15.142:0/1792088279 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f41b0003710 con 0x7f41c0101750 2024-03-29T05:04:55.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.903+0000 7f41b77fe700 1 --2- 172.21.15.142:0/1792088279 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f41ac06dd60 0x7f41ac070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:55.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.903+0000 7f41b77fe700 1 -- 172.21.15.142:0/1792088279 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f41b008f090 con 0x7f41c0101750 2024-03-29T05:04:55.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.903+0000 7f41c603c700 1 --2- 172.21.15.142:0/1792088279 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f41ac06dd60 0x7f41ac070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:55.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.904+0000 7f41c603c700 1 --2- 172.21.15.142:0/1792088279 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f41ac06dd60 0x7f41ac070220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009c60 tx=0x7f41bc009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:55.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:55.908+0000 7f41b77fe700 1 -- 172.21.15.142:0/1792088279 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f41b005c060 con 0x7f41c0101750 2024-03-29T05:04:56.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:55 smithi152 ceph-mon[88186]: pgmap v498: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:56.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:55 smithi152 ceph-mon[88186]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:56.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.150+0000 7f41c82a0700 1 -- 172.21.15.142:0/1792088279 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f41c004ea90 con 0x7f41c0101750 2024-03-29T05:04:56.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.150+0000 7f41b77fe700 1 -- 172.21.15.142:0/1792088279 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f41b005bbf0 con 0x7f41c0101750 2024-03-29T05:04:56.161 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:04:56.161 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:04:56.161 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:04:56.161 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:04:56.161 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:04:56.161 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:04:56.161 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:04:56.161 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:04:56.161 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:04:56.161 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:04:56.161 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:04:56.162 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:04:56.162 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:04:56.162 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:04:56.162 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:04:56.162 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:04:56.162 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:04:56.162 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:04:56.162 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:04:56.162 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:04:56.162 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:04:56.162 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:04:56.162 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:04:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:04:56.164 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:04:56.164 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:04:56.164 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:04:56.164 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:04:56.164 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:04:56.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.152+0000 7f41c82a0700 1 -- 172.21.15.142:0/1792088279 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f41ac06dd60 msgr2=0x7f41ac070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:56.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.152+0000 7f41c82a0700 1 --2- 172.21.15.142:0/1792088279 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f41ac06dd60 0x7f41ac070220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009c60 tx=0x7f41bc009260 comp rx=0 tx=0).stop 2024-03-29T05:04:56.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.152+0000 7f41c82a0700 1 -- 172.21.15.142:0/1792088279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0101750 msgr2=0x7f41c019ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:56.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.152+0000 7f41c82a0700 1 --2- 172.21.15.142:0/1792088279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0101750 0x7f41c019ca50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f41b00059d0 tx=0x7f41b000f690 comp rx=0 tx=0).stop 2024-03-29T05:04:56.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.152+0000 7f41c82a0700 1 -- 172.21.15.142:0/1792088279 shutdown_connections 2024-03-29T05:04:56.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.152+0000 7f41c82a0700 1 --2- 172.21.15.142:0/1792088279 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f41ac06dd60 0x7f41ac070220 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.152+0000 7f41c82a0700 1 --2- 172.21.15.142:0/1792088279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41c0100da0 0x7f41c019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.152+0000 7f41c82a0700 1 --2- 172.21.15.142:0/1792088279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0101750 0x7f41c019ca50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.152+0000 7f41c82a0700 1 -- 172.21.15.142:0/1792088279 >> 172.21.15.142:0/1792088279 conn(0x7f41c00fc650 msgr2=0x7f41c00fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:56.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.152+0000 7f41c82a0700 1 -- 172.21.15.142:0/1792088279 shutdown_connections 2024-03-29T05:04:56.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.152+0000 7f41c82a0700 1 -- 172.21.15.142:0/1792088279 wait complete. 2024-03-29T05:04:56.165 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:04:56.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.256+0000 7f82b7af9700 1 -- 172.21.15.142:0/2619021250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b00ffe50 msgr2=0x7f82b0104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:56.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.256+0000 7f82b7af9700 1 --2- 172.21.15.142:0/2619021250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b00ffe50 0x7f82b0104810 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009910 tx=0x7f82a0009c20 comp rx=0 tx=0).stop 2024-03-29T05:04:56.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.256+0000 7f82b7af9700 1 -- 172.21.15.142:0/2619021250 shutdown_connections 2024-03-29T05:04:56.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.256+0000 7f82b7af9700 1 --2- 172.21.15.142:0/2619021250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b00ffe50 0x7f82b0104810 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.256+0000 7f82b7af9700 1 --2- 172.21.15.142:0/2619021250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b00ff530 0x7f82b00ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.256+0000 7f82b7af9700 1 -- 172.21.15.142:0/2619021250 >> 172.21.15.142:0/2619021250 conn(0x7f82b0074df0 msgr2=0x7f82b0075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:56.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.256+0000 7f82b7af9700 1 -- 172.21.15.142:0/2619021250 shutdown_connections 2024-03-29T05:04:56.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.256+0000 7f82b7af9700 1 -- 172.21.15.142:0/2619021250 wait complete. 2024-03-29T05:04:56.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.256+0000 7f82b7af9700 1 Processor -- start 2024-03-29T05:04:56.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.256+0000 7f82b7af9700 1 -- start start 2024-03-29T05:04:56.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b7af9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b00ff530 0x7f82b019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:56.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b7af9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b00ffe50 0x7f82b019d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:56.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b7af9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b019d990 con 0x7f82b00ffe50 2024-03-29T05:04:56.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b7af9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b019dad0 con 0x7f82b00ff530 2024-03-29T05:04:56.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b5094700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b00ffe50 0x7f82b019d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:56.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b5094700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b00ffe50 0x7f82b019d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54878/0 (socket says 172.21.15.142:54878) 2024-03-29T05:04:56.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b5094700 1 -- 172.21.15.142:0/2523449421 learned_addr learned my addr 172.21.15.142:0/2523449421 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:56.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b5895700 1 --2- 172.21.15.142:0/2523449421 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b00ff530 0x7f82b019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:56.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b5094700 1 -- 172.21.15.142:0/2523449421 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b00ff530 msgr2=0x7f82b019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:56.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b5094700 1 --2- 172.21.15.142:0/2523449421 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b00ff530 0x7f82b019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b5094700 1 -- 172.21.15.142:0/2523449421 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a00095f0 con 0x7f82b00ffe50 2024-03-29T05:04:56.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b5094700 1 --2- 172.21.15.142:0/2523449421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b00ffe50 0x7f82b019d340 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f82a000bda0 tx=0x7f82a000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:56.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82a6ffd700 1 -- 172.21.15.142:0/2523449421 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82a001d070 con 0x7f82b00ffe50 2024-03-29T05:04:56.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82a6ffd700 1 -- 172.21.15.142:0/2523449421 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82a000f460 con 0x7f82b00ffe50 2024-03-29T05:04:56.270 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b7af9700 1 -- 172.21.15.142:0/2523449421 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b01a18c0 con 0x7f82b00ffe50 2024-03-29T05:04:56.270 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82a6ffd700 1 -- 172.21.15.142:0/2523449421 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82a0017630 con 0x7f82b00ffe50 2024-03-29T05:04:56.270 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.257+0000 7f82b7af9700 1 -- 172.21.15.142:0/2523449421 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b01a1db0 con 0x7f82b00ffe50 2024-03-29T05:04:56.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.258+0000 7f82b7af9700 1 -- 172.21.15.142:0/2523449421 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b004ea90 con 0x7f82b00ffe50 2024-03-29T05:04:56.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.259+0000 7f82a6ffd700 1 -- 172.21.15.142:0/2523449421 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f82a000fab0 con 0x7f82b00ffe50 2024-03-29T05:04:56.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.259+0000 7f82a6ffd700 1 --2- 172.21.15.142:0/2523449421 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f829c06dd60 0x7f829c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:56.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.259+0000 7f82a6ffd700 1 -- 172.21.15.142:0/2523449421 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f82a008f160 con 0x7f82b00ffe50 2024-03-29T05:04:56.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.259+0000 7f82b5895700 1 --2- 172.21.15.142:0/2523449421 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f829c06dd60 0x7f829c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:56.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.260+0000 7f82b5895700 1 --2- 172.21.15.142:0/2523449421 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f829c06dd60 0x7f829c070220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00ac50 tx=0x7f82ac00a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:56.274 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.264+0000 7f82a6ffd700 1 -- 172.21.15.142:0/2523449421 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f82a0058580 con 0x7f82b00ffe50 2024-03-29T05:04:56.489 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.479+0000 7f82b7af9700 1 -- 172.21.15.142:0/2523449421 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f82b0101c10 con 0x7f829c06dd60 2024-03-29T05:04:56.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.481+0000 7f82a6ffd700 1 -- 172.21.15.142:0/2523449421 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f82b0101c10 con 0x7f829c06dd60 2024-03-29T05:04:56.491 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:04:56.491 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:04:56.491 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:04:56.491 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:04:56.491 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:04:56.491 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:04:56.492 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:04:56.492 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:04:56.492 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:04:56.492 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:04:56.492 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:04:56.492 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:04:56.492 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:04:56.492 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:04:56.492 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:04:56.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.483+0000 7f82b7af9700 1 -- 172.21.15.142:0/2523449421 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f829c06dd60 msgr2=0x7f829c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:56.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.483+0000 7f82b7af9700 1 --2- 172.21.15.142:0/2523449421 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f829c06dd60 0x7f829c070220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00ac50 tx=0x7f82ac00a380 comp rx=0 tx=0).stop 2024-03-29T05:04:56.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.483+0000 7f82b7af9700 1 -- 172.21.15.142:0/2523449421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b00ffe50 msgr2=0x7f82b019d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:56.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.483+0000 7f82b7af9700 1 --2- 172.21.15.142:0/2523449421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b00ffe50 0x7f82b019d340 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f82a000bda0 tx=0x7f82a000bdd0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.483+0000 7f82b7af9700 1 -- 172.21.15.142:0/2523449421 shutdown_connections 2024-03-29T05:04:56.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.483+0000 7f82b7af9700 1 --2- 172.21.15.142:0/2523449421 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f829c06dd60 0x7f829c070220 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.483+0000 7f82b7af9700 1 --2- 172.21.15.142:0/2523449421 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b00ff530 0x7f82b019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.483+0000 7f82b7af9700 1 --2- 172.21.15.142:0/2523449421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b00ffe50 0x7f82b019d340 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.483+0000 7f82b7af9700 1 -- 172.21.15.142:0/2523449421 >> 172.21.15.142:0/2523449421 conn(0x7f82b0074df0 msgr2=0x7f82b0111ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:56.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.483+0000 7f82b7af9700 1 -- 172.21.15.142:0/2523449421 shutdown_connections 2024-03-29T05:04:56.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.483+0000 7f82b7af9700 1 -- 172.21.15.142:0/2523449421 wait complete. 2024-03-29T05:04:56.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.586+0000 7f16b5b45700 1 -- 172.21.15.142:0/4059526715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b0100dd0 msgr2=0x7f16b01011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:56.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.586+0000 7f16b5b45700 1 --2- 172.21.15.142:0/4059526715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b0100dd0 0x7f16b01011b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f16a800b0a0 tx=0x7f16a800b3b0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.586+0000 7f16b5b45700 1 -- 172.21.15.142:0/4059526715 shutdown_connections 2024-03-29T05:04:56.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.586+0000 7f16b5b45700 1 --2- 172.21.15.142:0/4059526715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16b0101780 0x7f16b01057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.586+0000 7f16b5b45700 1 --2- 172.21.15.142:0/4059526715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b0100dd0 0x7f16b01011b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.586+0000 7f16b5b45700 1 -- 172.21.15.142:0/4059526715 >> 172.21.15.142:0/4059526715 conn(0x7f16b00fc640 msgr2=0x7f16b00fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:56.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.586+0000 7f16b5b45700 1 -- 172.21.15.142:0/4059526715 shutdown_connections 2024-03-29T05:04:56.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.586+0000 7f16b5b45700 1 -- 172.21.15.142:0/4059526715 wait complete. 2024-03-29T05:04:56.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.587+0000 7f16b5b45700 1 Processor -- start 2024-03-29T05:04:56.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.587+0000 7f16b5b45700 1 -- start start 2024-03-29T05:04:56.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.587+0000 7f16b5b45700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16b0100dd0 0x7f16b019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:56.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.587+0000 7f16b5b45700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b0101780 0x7f16b019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:56.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.587+0000 7f16b5b45700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b019d0e0 con 0x7f16b0101780 2024-03-29T05:04:56.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.587+0000 7f16b5b45700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b0196590 con 0x7f16b0100dd0 2024-03-29T05:04:56.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.587+0000 7f16aeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b0101780 0x7f16b019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:56.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.587+0000 7f16aeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b0101780 0x7f16b019ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54892/0 (socket says 172.21.15.142:54892) 2024-03-29T05:04:56.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.587+0000 7f16aeffd700 1 -- 172.21.15.142:0/863454327 learned_addr learned my addr 172.21.15.142:0/863454327 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:04:56.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.587+0000 7f16af7fe700 1 --2- 172.21.15.142:0/863454327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16b0100dd0 0x7f16b019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:56.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.587+0000 7f16aeffd700 1 -- 172.21.15.142:0/863454327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16b0100dd0 msgr2=0x7f16b019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:56.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.587+0000 7f16aeffd700 1 --2- 172.21.15.142:0/863454327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16b0100dd0 0x7f16b019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.588+0000 7f16aeffd700 1 -- 172.21.15.142:0/863454327 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a8009d00 con 0x7f16b0101780 2024-03-29T05:04:56.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.588+0000 7f16af7fe700 1 --2- 172.21.15.142:0/863454327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16b0100dd0 0x7f16b019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:04:56.600 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.588+0000 7f16aeffd700 1 --2- 172.21.15.142:0/863454327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b0101780 0x7f16b019ca50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f169800d900 tx=0x7f169800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:56.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.588+0000 7f16acff9700 1 -- 172.21.15.142:0/863454327 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1698009900 con 0x7f16b0101780 2024-03-29T05:04:56.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.588+0000 7f16acff9700 1 -- 172.21.15.142:0/863454327 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1698010460 con 0x7f16b0101780 2024-03-29T05:04:56.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.588+0000 7f16b5b45700 1 -- 172.21.15.142:0/863454327 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b0196870 con 0x7f16b0101780 2024-03-29T05:04:56.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.588+0000 7f16acff9700 1 -- 172.21.15.142:0/863454327 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f169800f5f0 con 0x7f16b0101780 2024-03-29T05:04:56.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.588+0000 7f16b5b45700 1 -- 172.21.15.142:0/863454327 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b0196dc0 con 0x7f16b0101780 2024-03-29T05:04:56.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.589+0000 7f16b5b45700 1 -- 172.21.15.142:0/863454327 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b0109170 con 0x7f16b0101780 2024-03-29T05:04:56.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.589+0000 7f16acff9700 1 -- 172.21.15.142:0/863454327 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f16980105d0 con 0x7f16b0101780 2024-03-29T05:04:56.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.590+0000 7f16acff9700 1 --2- 172.21.15.142:0/863454327 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f169c06dd60 0x7f169c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:04:56.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.590+0000 7f16acff9700 1 -- 172.21.15.142:0/863454327 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f169808db40 con 0x7f16b0101780 2024-03-29T05:04:56.602 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.590+0000 7f16af7fe700 1 --2- 172.21.15.142:0/863454327 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f169c06dd60 0x7f169c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:04:56.603 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.590+0000 7f16af7fe700 1 --2- 172.21.15.142:0/863454327 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f169c06dd60 0x7f169c070220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f16a80062a0 tx=0x7f16a8006210 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:04:56.604 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.595+0000 7f16acff9700 1 -- 172.21.15.142:0/863454327 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f169805aa90 con 0x7f16b0101780 2024-03-29T05:04:56.877 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.867+0000 7f16b5b45700 1 -- 172.21.15.142:0/863454327 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f16b0197060 con 0x7f16b0101780 2024-03-29T05:04:56.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:56 smithi142 ceph-mon[112475]: from='client.34938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:56.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:56 smithi142 ceph-mon[112475]: from='client.34942 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:56.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:56 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1644518097' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:04:56.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:56 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1792088279' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:04:56.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.867+0000 7f16acff9700 1 -- 172.21.15.142:0/863454327 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f169805a620 con 0x7f16b0101780 2024-03-29T05:04:56.878 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:04:56.879 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:04:56.879 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:04:56.879 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:04:56.879 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:04:56.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.870+0000 7f16b5b45700 1 -- 172.21.15.142:0/863454327 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f169c06dd60 msgr2=0x7f169c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:56.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.870+0000 7f16b5b45700 1 --2- 172.21.15.142:0/863454327 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f169c06dd60 0x7f169c070220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f16a80062a0 tx=0x7f16a8006210 comp rx=0 tx=0).stop 2024-03-29T05:04:56.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.870+0000 7f16b5b45700 1 -- 172.21.15.142:0/863454327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b0101780 msgr2=0x7f16b019ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:04:56.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.870+0000 7f16b5b45700 1 --2- 172.21.15.142:0/863454327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b0101780 0x7f16b019ca50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f169800d900 tx=0x7f169800dcc0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.870+0000 7f16b5b45700 1 -- 172.21.15.142:0/863454327 shutdown_connections 2024-03-29T05:04:56.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.870+0000 7f16b5b45700 1 --2- 172.21.15.142:0/863454327 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f169c06dd60 0x7f169c070220 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.870+0000 7f16b5b45700 1 --2- 172.21.15.142:0/863454327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16b0100dd0 0x7f16b019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.870+0000 7f16b5b45700 1 --2- 172.21.15.142:0/863454327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b0101780 0x7f16b019ca50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:04:56.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.870+0000 7f16b5b45700 1 -- 172.21.15.142:0/863454327 >> 172.21.15.142:0/863454327 conn(0x7f16b00fc640 msgr2=0x7f16b00fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:04:56.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.870+0000 7f16b5b45700 1 -- 172.21.15.142:0/863454327 shutdown_connections 2024-03-29T05:04:56.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:04:56.870+0000 7f16b5b45700 1 -- 172.21.15.142:0/863454327 wait complete. 2024-03-29T05:04:57.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:56 smithi152 ceph-mon[88186]: from='client.34938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:57.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:56 smithi152 ceph-mon[88186]: from='client.34942 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:57.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:56 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1644518097' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:04:57.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:56 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1792088279' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:04:57.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:57 smithi142 ceph-mon[112475]: pgmap v499: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:57.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:57 smithi142 ceph-mon[112475]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:57.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:57 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/863454327' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:04:58.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:57 smithi152 ceph-mon[88186]: pgmap v499: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:04:58.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:57 smithi152 ceph-mon[88186]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:04:58.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:57 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/863454327' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:04:59.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:04:59 smithi142 ceph-mon[112475]: pgmap v500: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:00.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:04:59 smithi152 ceph-mon[88186]: pgmap v500: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:00.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:05:01.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:05:01.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:01 smithi142 ceph-mon[112475]: pgmap v501: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:02.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:01 smithi152 ceph-mon[88186]: pgmap v501: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:03.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:03 smithi142 ceph-mon[112475]: pgmap v502: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:04.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:03 smithi152 ceph-mon[88186]: pgmap v502: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:05.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:05 smithi142 ceph-mon[112475]: pgmap v503: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:06.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:05 smithi152 ceph-mon[88186]: pgmap v503: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:07.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:07 smithi142 ceph-mon[112475]: pgmap v504: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:08.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:07 smithi152 ceph-mon[88186]: pgmap v504: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:09.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:09 smithi142 ceph-mon[112475]: pgmap v505: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:10.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:09 smithi152 ceph-mon[88186]: pgmap v505: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:11.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:11 smithi142 ceph-mon[112475]: pgmap v506: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:12.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:11 smithi152 ceph-mon[88186]: pgmap v506: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:13.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:13 smithi142 ceph-mon[112475]: pgmap v507: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:14.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:13 smithi152 ceph-mon[88186]: pgmap v507: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:14.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:05:15.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:05:15.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:15 smithi142 ceph-mon[112475]: pgmap v508: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:16.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:15 smithi152 ceph-mon[88186]: pgmap v508: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:17.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:17 smithi142 ceph-mon[112475]: pgmap v509: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:18.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:17 smithi152 ceph-mon[88186]: pgmap v509: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:19.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:19 smithi142 ceph-mon[112475]: pgmap v510: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:20.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:19 smithi152 ceph-mon[88186]: pgmap v510: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:21.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:21 smithi142 ceph-mon[112475]: pgmap v511: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:22.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:21 smithi152 ceph-mon[88186]: pgmap v511: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:24.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:23 smithi152 ceph-mon[88186]: pgmap v512: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:24.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:23 smithi142 ceph-mon[112475]: pgmap v512: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:26.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:25 smithi152 ceph-mon[88186]: pgmap v513: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:26.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:25 smithi142 ceph-mon[112475]: pgmap v513: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:26.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.985+0000 7f568d62a700 1 -- 172.21.15.142:0/1496225001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56880ffa50 msgr2=0x7f56880ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:26.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.985+0000 7f568d62a700 1 --2- 172.21.15.142:0/1496225001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56880ffa50 0x7f56880ffed0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5670009910 tx=0x7f5670009c20 comp rx=0 tx=0).stop 2024-03-29T05:05:26.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.985+0000 7f568d62a700 1 -- 172.21.15.142:0/1496225001 shutdown_connections 2024-03-29T05:05:26.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.985+0000 7f568d62a700 1 --2- 172.21.15.142:0/1496225001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56880ffa50 0x7f56880ffed0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:26.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.985+0000 7f568d62a700 1 --2- 172.21.15.142:0/1496225001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5688105a50 0x7f5688105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:26.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.985+0000 7f568d62a700 1 -- 172.21.15.142:0/1496225001 >> 172.21.15.142:0/1496225001 conn(0x7f56880fb540 msgr2=0x7f56880fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:26.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.985+0000 7f568d62a700 1 -- 172.21.15.142:0/1496225001 shutdown_connections 2024-03-29T05:05:26.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.985+0000 7f568d62a700 1 -- 172.21.15.142:0/1496225001 wait complete. 2024-03-29T05:05:26.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.986+0000 7f568d62a700 1 Processor -- start 2024-03-29T05:05:26.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.986+0000 7f568d62a700 1 -- start start 2024-03-29T05:05:26.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.986+0000 7f568d62a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56880ffa50 0x7f56881966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:26.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.986+0000 7f568d62a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5688105a50 0x7f5688196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:26.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.986+0000 7f568d62a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56881972b0 con 0x7f56880ffa50 2024-03-29T05:05:26.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.986+0000 7f568d62a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5688199fe0 con 0x7f5688105a50 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.986+0000 7f5686ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56880ffa50 0x7f56881966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f5686ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56880ffa50 0x7f56881966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56792/0 (socket says 172.21.15.142:56792) 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f5686ffd700 1 -- 172.21.15.142:0/3629361071 learned_addr learned my addr 172.21.15.142:0/3629361071 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f567e7fc700 1 --2- 172.21.15.142:0/3629361071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5688105a50 0x7f5688196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f5686ffd700 1 -- 172.21.15.142:0/3629361071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5688105a50 msgr2=0x7f5688196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f5686ffd700 1 --2- 172.21.15.142:0/3629361071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5688105a50 0x7f5688196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f5686ffd700 1 -- 172.21.15.142:0/3629361071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56700095f0 con 0x7f56880ffa50 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f567e7fc700 1 --2- 172.21.15.142:0/3629361071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5688105a50 0x7f5688196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f5686ffd700 1 --2- 172.21.15.142:0/3629361071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56880ffa50 0x7f56881966e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f567800e920 tx=0x7f567800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f5684ff9700 1 -- 172.21.15.142:0/3629361071 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5678009790 con 0x7f56880ffa50 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f5684ff9700 1 -- 172.21.15.142:0/3629361071 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f567800cba0 con 0x7f56880ffa50 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f5684ff9700 1 -- 172.21.15.142:0/3629361071 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5678018650 con 0x7f56880ffa50 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f568d62a700 1 -- 172.21.15.142:0/3629361071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568819a2c0 con 0x7f56880ffa50 2024-03-29T05:05:26.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.987+0000 7f568d62a700 1 -- 172.21.15.142:0/3629361071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568819a810 con 0x7f56880ffa50 2024-03-29T05:05:27.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.988+0000 7f568d62a700 1 -- 172.21.15.142:0/3629361071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5688108ff0 con 0x7f56880ffa50 2024-03-29T05:05:27.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.989+0000 7f5684ff9700 1 -- 172.21.15.142:0/3629361071 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f567800cd10 con 0x7f56880ffa50 2024-03-29T05:05:27.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.989+0000 7f5684ff9700 1 --2- 172.21.15.142:0/3629361071 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f567406dd60 0x7f5674070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:27.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.989+0000 7f5684ff9700 1 -- 172.21.15.142:0/3629361071 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f5678014070 con 0x7f56880ffa50 2024-03-29T05:05:27.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.989+0000 7f567e7fc700 1 --2- 172.21.15.142:0/3629361071 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f567406dd60 0x7f5674070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:27.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.990+0000 7f567e7fc700 1 --2- 172.21.15.142:0/3629361071 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f567406dd60 0x7f5674070220 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5670000c00 tx=0x7f5670009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:27.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:26.994+0000 7f5684ff9700 1 -- 172.21.15.142:0/3629361071 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5678053280 con 0x7f56880ffa50 2024-03-29T05:05:27.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.202+0000 7f568d62a700 1 -- 172.21.15.142:0/3629361071 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f568819aa30 con 0x7f567406dd60 2024-03-29T05:05:27.213 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.204+0000 7f5684ff9700 1 -- 172.21.15.142:0/3629361071 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f568819aa30 con 0x7f567406dd60 2024-03-29T05:05:27.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.206+0000 7f568d62a700 1 -- 172.21.15.142:0/3629361071 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f567406dd60 msgr2=0x7f5674070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:27.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.206+0000 7f568d62a700 1 --2- 172.21.15.142:0/3629361071 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f567406dd60 0x7f5674070220 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5670000c00 tx=0x7f5670009f70 comp rx=0 tx=0).stop 2024-03-29T05:05:27.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.206+0000 7f568d62a700 1 -- 172.21.15.142:0/3629361071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56880ffa50 msgr2=0x7f56881966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:27.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.206+0000 7f568d62a700 1 --2- 172.21.15.142:0/3629361071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56880ffa50 0x7f56881966e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f567800e920 tx=0x7f567800ece0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.206+0000 7f568d62a700 1 -- 172.21.15.142:0/3629361071 shutdown_connections 2024-03-29T05:05:27.217 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.206+0000 7f568d62a700 1 --2- 172.21.15.142:0/3629361071 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f567406dd60 0x7f5674070220 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.217 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.206+0000 7f568d62a700 1 --2- 172.21.15.142:0/3629361071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56880ffa50 0x7f56881966e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.217 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.206+0000 7f568d62a700 1 --2- 172.21.15.142:0/3629361071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5688105a50 0x7f5688196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.217 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.206+0000 7f568d62a700 1 -- 172.21.15.142:0/3629361071 >> 172.21.15.142:0/3629361071 conn(0x7f56880fb540 msgr2=0x7f5688068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:27.217 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.206+0000 7f568d62a700 1 -- 172.21.15.142:0/3629361071 shutdown_connections 2024-03-29T05:05:27.217 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.206+0000 7f568d62a700 1 -- 172.21.15.142:0/3629361071 wait complete. 2024-03-29T05:05:27.226 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:05:27.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.310+0000 7f18020d4700 1 -- 172.21.15.142:0/2083921489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc068670 msgr2=0x7f17fc068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:27.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.310+0000 7f18020d4700 1 --2- 172.21.15.142:0/2083921489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc068670 0x7f17fc068a50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009910 tx=0x7f17f0009c20 comp rx=0 tx=0).stop 2024-03-29T05:05:27.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.310+0000 7f18020d4700 1 -- 172.21.15.142:0/2083921489 shutdown_connections 2024-03-29T05:05:27.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.310+0000 7f18020d4700 1 --2- 172.21.15.142:0/2083921489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17fc069020 0x7f17fc105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.310+0000 7f18020d4700 1 --2- 172.21.15.142:0/2083921489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc068670 0x7f17fc068a50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.310+0000 7f18020d4700 1 -- 172.21.15.142:0/2083921489 >> 172.21.15.142:0/2083921489 conn(0x7f17fc0755a0 msgr2=0x7f17fc0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:27.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.310+0000 7f18020d4700 1 -- 172.21.15.142:0/2083921489 shutdown_connections 2024-03-29T05:05:27.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.311+0000 7f18020d4700 1 -- 172.21.15.142:0/2083921489 wait complete. 2024-03-29T05:05:27.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.311+0000 7f18020d4700 1 Processor -- start 2024-03-29T05:05:27.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.311+0000 7f18020d4700 1 -- start start 2024-03-29T05:05:27.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.312+0000 7f18020d4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17fc068670 0x7f17fc0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:27.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.312+0000 7f18020d4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc069020 0x7f17fc1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:27.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.312+0000 7f18020d4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc100720 con 0x7f17fc069020 2024-03-29T05:05:27.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.312+0000 7f18020d4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc100860 con 0x7f17fc068670 2024-03-29T05:05:27.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.312+0000 7f17faffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc069020 0x7f17fc1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:27.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.312+0000 7f17faffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc069020 0x7f17fc1001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56798/0 (socket says 172.21.15.142:56798) 2024-03-29T05:05:27.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.312+0000 7f17faffd700 1 -- 172.21.15.142:0/1842932734 learned_addr learned my addr 172.21.15.142:0/1842932734 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:05:27.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.312+0000 7f17fb7fe700 1 --2- 172.21.15.142:0/1842932734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17fc068670 0x7f17fc0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:27.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.312+0000 7f17faffd700 1 -- 172.21.15.142:0/1842932734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17fc068670 msgr2=0x7f17fc0ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:27.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.312+0000 7f17faffd700 1 --2- 172.21.15.142:0/1842932734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17fc068670 0x7f17fc0ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.312+0000 7f17faffd700 1 -- 172.21.15.142:0/1842932734 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f00095f0 con 0x7f17fc069020 2024-03-29T05:05:27.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.312+0000 7f17faffd700 1 --2- 172.21.15.142:0/1842932734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc069020 0x7f17fc1001e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00c960 tx=0x7f17ec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:27.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.313+0000 7f17f8ff9700 1 -- 172.21.15.142:0/1842932734 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17ec007a70 con 0x7f17fc069020 2024-03-29T05:05:27.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.313+0000 7f17f8ff9700 1 -- 172.21.15.142:0/1842932734 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17ec007bd0 con 0x7f17fc069020 2024-03-29T05:05:27.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.313+0000 7f17f8ff9700 1 -- 172.21.15.142:0/1842932734 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17ec018630 con 0x7f17fc069020 2024-03-29T05:05:27.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.313+0000 7f18020d4700 1 -- 172.21.15.142:0/1842932734 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17fc1a2910 con 0x7f17fc069020 2024-03-29T05:05:27.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.313+0000 7f18020d4700 1 -- 172.21.15.142:0/1842932734 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17fc1a2c70 con 0x7f17fc069020 2024-03-29T05:05:27.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.314+0000 7f18020d4700 1 -- 172.21.15.142:0/1842932734 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17fc109130 con 0x7f17fc069020 2024-03-29T05:05:27.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.314+0000 7f17f8ff9700 1 -- 172.21.15.142:0/1842932734 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f17ec00f450 con 0x7f17fc069020 2024-03-29T05:05:27.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.315+0000 7f17f8ff9700 1 --2- 172.21.15.142:0/1842932734 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f17e406dd60 0x7f17e4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:27.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.315+0000 7f17f8ff9700 1 -- 172.21.15.142:0/1842932734 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f17ec08dbf0 con 0x7f17fc069020 2024-03-29T05:05:27.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.315+0000 7f17fb7fe700 1 --2- 172.21.15.142:0/1842932734 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f17e406dd60 0x7f17e4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:27.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.315+0000 7f17fb7fe700 1 --2- 172.21.15.142:0/1842932734 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f17e406dd60 0x7f17e4070220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f17f0006010 tx=0x7f17f001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:27.330 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.320+0000 7f17f8ff9700 1 -- 172.21.15.142:0/1842932734 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f17ec057010 con 0x7f17fc069020 2024-03-29T05:05:27.543 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.533+0000 7f18020d4700 1 -- 172.21.15.142:0/1842932734 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f17fc02cce0 con 0x7f17e406dd60 2024-03-29T05:05:27.544 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.534+0000 7f17f8ff9700 1 -- 172.21.15.142:0/1842932734 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f17fc02cce0 con 0x7f17e406dd60 2024-03-29T05:05:27.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.536+0000 7f18020d4700 1 -- 172.21.15.142:0/1842932734 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f17e406dd60 msgr2=0x7f17e4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:27.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.536+0000 7f18020d4700 1 --2- 172.21.15.142:0/1842932734 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f17e406dd60 0x7f17e4070220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f17f0006010 tx=0x7f17f001a040 comp rx=0 tx=0).stop 2024-03-29T05:05:27.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.536+0000 7f18020d4700 1 -- 172.21.15.142:0/1842932734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc069020 msgr2=0x7f17fc1001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:27.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.536+0000 7f18020d4700 1 --2- 172.21.15.142:0/1842932734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc069020 0x7f17fc1001e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00c960 tx=0x7f17ec00cc70 comp rx=0 tx=0).stop 2024-03-29T05:05:27.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.536+0000 7f18020d4700 1 -- 172.21.15.142:0/1842932734 shutdown_connections 2024-03-29T05:05:27.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.536+0000 7f18020d4700 1 --2- 172.21.15.142:0/1842932734 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f17e406dd60 0x7f17e4070220 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.536+0000 7f18020d4700 1 --2- 172.21.15.142:0/1842932734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17fc068670 0x7f17fc0ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.536+0000 7f18020d4700 1 --2- 172.21.15.142:0/1842932734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17fc069020 0x7f17fc1001e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.536+0000 7f18020d4700 1 -- 172.21.15.142:0/1842932734 >> 172.21.15.142:0/1842932734 conn(0x7f17fc0755a0 msgr2=0x7f17fc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:27.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.537+0000 7f18020d4700 1 -- 172.21.15.142:0/1842932734 shutdown_connections 2024-03-29T05:05:27.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.537+0000 7f18020d4700 1 -- 172.21.15.142:0/1842932734 wait complete. 2024-03-29T05:05:27.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.639+0000 7f7a19259700 1 -- 172.21.15.142:0/4229900701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a140ff530 msgr2=0x7f7a140ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:27.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.639+0000 7f7a19259700 1 --2- 172.21.15.142:0/4229900701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a140ff530 0x7f7a140ff910 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7a04009910 tx=0x7f7a04009c20 comp rx=0 tx=0).stop 2024-03-29T05:05:27.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.639+0000 7f7a19259700 1 -- 172.21.15.142:0/4229900701 shutdown_connections 2024-03-29T05:05:27.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.639+0000 7f7a19259700 1 --2- 172.21.15.142:0/4229900701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a140ffee0 0x7f7a14105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.639+0000 7f7a19259700 1 --2- 172.21.15.142:0/4229900701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a140ff530 0x7f7a140ff910 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.639+0000 7f7a19259700 1 -- 172.21.15.142:0/4229900701 >> 172.21.15.142:0/4229900701 conn(0x7f7a14074df0 msgr2=0x7f7a14075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:27.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.639+0000 7f7a19259700 1 -- 172.21.15.142:0/4229900701 shutdown_connections 2024-03-29T05:05:27.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.639+0000 7f7a19259700 1 -- 172.21.15.142:0/4229900701 wait complete. 2024-03-29T05:05:27.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.640+0000 7f7a19259700 1 Processor -- start 2024-03-29T05:05:27.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.640+0000 7f7a19259700 1 -- start start 2024-03-29T05:05:27.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a19259700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a140ff530 0x7f7a1419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:27.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a19259700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a140ffee0 0x7f7a1419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:27.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a19259700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1419d0d0 con 0x7f7a140ffee0 2024-03-29T05:05:27.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a19259700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a14196610 con 0x7f7a140ff530 2024-03-29T05:05:27.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a1259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a140ffee0 0x7f7a1419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:27.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a1259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a140ffee0 0x7f7a1419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56820/0 (socket says 172.21.15.142:56820) 2024-03-29T05:05:27.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a1259c700 1 -- 172.21.15.142:0/3646474643 learned_addr learned my addr 172.21.15.142:0/3646474643 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:05:27.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a12d9d700 1 --2- 172.21.15.142:0/3646474643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a140ff530 0x7f7a1419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:27.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a1259c700 1 -- 172.21.15.142:0/3646474643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a140ff530 msgr2=0x7f7a1419c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:27.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a1259c700 1 --2- 172.21.15.142:0/3646474643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a140ff530 0x7f7a1419c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a1259c700 1 -- 172.21.15.142:0/3646474643 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a040095f0 con 0x7f7a140ffee0 2024-03-29T05:05:27.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a1259c700 1 --2- 172.21.15.142:0/3646474643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a140ffee0 0x7f7a1419cad0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800c960 tx=0x7f7a0800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:27.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f79fbfff700 1 -- 172.21.15.142:0/3646474643 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a08007a70 con 0x7f7a140ffee0 2024-03-29T05:05:27.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f79fbfff700 1 -- 172.21.15.142:0/3646474643 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a08007bd0 con 0x7f7a140ffee0 2024-03-29T05:05:27.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a19259700 1 -- 172.21.15.142:0/3646474643 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a141968f0 con 0x7f7a140ffee0 2024-03-29T05:05:27.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f79fbfff700 1 -- 172.21.15.142:0/3646474643 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a08018630 con 0x7f7a140ffee0 2024-03-29T05:05:27.653 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.641+0000 7f7a19259700 1 -- 172.21.15.142:0/3646474643 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a14196e40 con 0x7f7a140ffee0 2024-03-29T05:05:27.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.642+0000 7f7a19259700 1 -- 172.21.15.142:0/3646474643 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a141091e0 con 0x7f7a140ffee0 2024-03-29T05:05:27.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.643+0000 7f79fbfff700 1 -- 172.21.15.142:0/3646474643 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f7a0800f450 con 0x7f7a140ffee0 2024-03-29T05:05:27.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.643+0000 7f79fbfff700 1 --2- 172.21.15.142:0/3646474643 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f79fc06dd60 0x7f79fc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:27.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.643+0000 7f79fbfff700 1 -- 172.21.15.142:0/3646474643 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f7a0808dbf0 con 0x7f7a140ffee0 2024-03-29T05:05:27.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.643+0000 7f7a12d9d700 1 --2- 172.21.15.142:0/3646474643 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f79fc06dd60 0x7f79fc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:27.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.644+0000 7f7a12d9d700 1 --2- 172.21.15.142:0/3646474643 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f79fc06dd60 0x7f79fc070220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a04006010 tx=0x7f7a0401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:27.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.648+0000 7f79fbfff700 1 -- 172.21.15.142:0/3646474643 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7a0805ab40 con 0x7f7a140ffee0 2024-03-29T05:05:27.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.851+0000 7f7a19259700 1 -- 172.21.15.142:0/3646474643 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a141970c0 con 0x7f79fc06dd60 2024-03-29T05:05:27.872 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.862+0000 7f79fbfff700 1 -- 172.21.15.142:0/3646474643 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7a141970c0 con 0x7f79fc06dd60 2024-03-29T05:05:27.872 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:05:27.872 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (17m) 10m ago 28m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (29m) 10m ago 29m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (28m) 9m ago 28m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (14m) 10m ago 29m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (14m) 9m ago 27m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (16m) 10m ago 28m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (10m) 10m ago 23m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (10m) 10m ago 23m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (9m) 9m ago 23m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (10m) 9m ago 23m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (18m) 10m ago 30m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (18m) 9m ago 27m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (15m) 10m ago 30m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (15m) 9m ago 27m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (17m) 10m ago 28m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:05:27.873 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (17m) 9m ago 27m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:05:27.874 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (14m) 10m ago 26m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:05:27.874 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (13m) 10m ago 26m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:05:27.874 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (12m) 10m ago 26m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:05:27.874 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (12m) 9m ago 25m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:05:27.874 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (11m) 9m ago 25m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:05:27.874 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (11m) 9m ago 24m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:05:27.874 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (17m) 10m ago 28m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:05:27.875 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.864+0000 7f7a19259700 1 -- 172.21.15.142:0/3646474643 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f79fc06dd60 msgr2=0x7f79fc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:27.875 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.864+0000 7f7a19259700 1 --2- 172.21.15.142:0/3646474643 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f79fc06dd60 0x7f79fc070220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a04006010 tx=0x7f7a0401a040 comp rx=0 tx=0).stop 2024-03-29T05:05:27.875 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.864+0000 7f7a19259700 1 -- 172.21.15.142:0/3646474643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a140ffee0 msgr2=0x7f7a1419cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:27.875 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.864+0000 7f7a19259700 1 --2- 172.21.15.142:0/3646474643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a140ffee0 0x7f7a1419cad0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800c960 tx=0x7f7a0800cc70 comp rx=0 tx=0).stop 2024-03-29T05:05:27.875 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.864+0000 7f7a19259700 1 -- 172.21.15.142:0/3646474643 shutdown_connections 2024-03-29T05:05:27.875 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.864+0000 7f7a19259700 1 --2- 172.21.15.142:0/3646474643 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f79fc06dd60 0x7f79fc070220 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.875 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.864+0000 7f7a19259700 1 --2- 172.21.15.142:0/3646474643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a140ff530 0x7f7a1419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.875 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.864+0000 7f7a19259700 1 --2- 172.21.15.142:0/3646474643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a140ffee0 0x7f7a1419cad0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.875 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.864+0000 7f7a19259700 1 -- 172.21.15.142:0/3646474643 >> 172.21.15.142:0/3646474643 conn(0x7f7a14074df0 msgr2=0x7f7a140feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:27.875 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.865+0000 7f7a19259700 1 -- 172.21.15.142:0/3646474643 shutdown_connections 2024-03-29T05:05:27.876 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.865+0000 7f7a19259700 1 -- 172.21.15.142:0/3646474643 wait complete. 2024-03-29T05:05:27.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.968+0000 7f9691e9b700 1 -- 172.21.15.142:0/3727991464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f968c101750 msgr2=0x7f968c1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:27.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.968+0000 7f9691e9b700 1 --2- 172.21.15.142:0/3727991464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f968c101750 0x7f968c1057a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9680009910 tx=0x7f9680009c20 comp rx=0 tx=0).stop 2024-03-29T05:05:27.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.968+0000 7f9691e9b700 1 -- 172.21.15.142:0/3727991464 shutdown_connections 2024-03-29T05:05:27.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.968+0000 7f9691e9b700 1 --2- 172.21.15.142:0/3727991464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f968c101750 0x7f968c1057a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.968+0000 7f9691e9b700 1 --2- 172.21.15.142:0/3727991464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f968c100da0 0x7f968c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.968+0000 7f9691e9b700 1 -- 172.21.15.142:0/3727991464 >> 172.21.15.142:0/3727991464 conn(0x7f968c0fc650 msgr2=0x7f968c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:27.979 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:27 smithi142 ceph-mon[112475]: pgmap v514: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:27.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.968+0000 7f9691e9b700 1 -- 172.21.15.142:0/3727991464 shutdown_connections 2024-03-29T05:05:27.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.968+0000 7f9691e9b700 1 -- 172.21.15.142:0/3727991464 wait complete. 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.968+0000 7f9691e9b700 1 Processor -- start 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.969+0000 7f9691e9b700 1 -- start start 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.969+0000 7f9691e9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f968c100da0 0x7f968c19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.969+0000 7f9691e9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f968c101750 0x7f968c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.969+0000 7f9691e9b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c19d0e0 con 0x7f968c101750 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.969+0000 7f9691e9b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c196590 con 0x7f968c100da0 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.969+0000 7f968affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f968c101750 0x7f968c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.969+0000 7f968affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f968c101750 0x7f968c19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56844/0 (socket says 172.21.15.142:56844) 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.969+0000 7f968affd700 1 -- 172.21.15.142:0/2382183011 learned_addr learned my addr 172.21.15.142:0/2382183011 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.969+0000 7f968b7fe700 1 --2- 172.21.15.142:0/2382183011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f968c100da0 0x7f968c19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.969+0000 7f968affd700 1 -- 172.21.15.142:0/2382183011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f968c100da0 msgr2=0x7f968c19c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.969+0000 7f968affd700 1 --2- 172.21.15.142:0/2382183011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f968c100da0 0x7f968c19c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:27.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.969+0000 7f968affd700 1 -- 172.21.15.142:0/2382183011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96800095f0 con 0x7f968c101750 2024-03-29T05:05:27.983 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.970+0000 7f968affd700 1 --2- 172.21.15.142:0/2382183011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f968c101750 0x7f968c19ca50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f96800092e0 tx=0x7f968000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:27.983 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.970+0000 7f9688ff9700 1 -- 172.21.15.142:0/2382183011 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f968001d070 con 0x7f968c101750 2024-03-29T05:05:27.983 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.970+0000 7f9688ff9700 1 -- 172.21.15.142:0/2382183011 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9680005dc0 con 0x7f968c101750 2024-03-29T05:05:27.983 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.970+0000 7f9691e9b700 1 -- 172.21.15.142:0/2382183011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c196810 con 0x7f968c101750 2024-03-29T05:05:27.983 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.970+0000 7f9688ff9700 1 -- 172.21.15.142:0/2382183011 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9680017400 con 0x7f968c101750 2024-03-29T05:05:27.983 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.970+0000 7f9691e9b700 1 -- 172.21.15.142:0/2382183011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c196d00 con 0x7f968c101750 2024-03-29T05:05:27.983 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.971+0000 7f9691e9b700 1 -- 172.21.15.142:0/2382183011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c1091a0 con 0x7f968c101750 2024-03-29T05:05:27.983 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.972+0000 7f9688ff9700 1 -- 172.21.15.142:0/2382183011 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f968000f6f0 con 0x7f968c101750 2024-03-29T05:05:27.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.972+0000 7f9688ff9700 1 --2- 172.21.15.142:0/2382183011 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f967406dd60 0x7f9674070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:27.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.972+0000 7f9688ff9700 1 -- 172.21.15.142:0/2382183011 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f968008efa0 con 0x7f968c101750 2024-03-29T05:05:27.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.972+0000 7f968b7fe700 1 --2- 172.21.15.142:0/2382183011 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f967406dd60 0x7f9674070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:27.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.973+0000 7f968b7fe700 1 --2- 172.21.15.142:0/2382183011 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f967406dd60 0x7f9674070220 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f967c009550 tx=0x7f967c0091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:27.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:27.977+0000 7f9688ff9700 1 -- 172.21.15.142:0/2382183011 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f968005bf70 con 0x7f968c101750 2024-03-29T05:05:28.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:27 smithi152 ceph-mon[88186]: pgmap v514: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:28.265 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.255+0000 7f9691e9b700 1 -- 172.21.15.142:0/2382183011 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f968c04ea90 con 0x7f968c101750 2024-03-29T05:05:28.266 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.256+0000 7f9688ff9700 1 -- 172.21.15.142:0/2382183011 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f968005bb00 con 0x7f968c101750 2024-03-29T05:05:28.266 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:05:28.266 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:05:28.266 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:05:28.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:05:28.266 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:05:28.266 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:05:28.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:05:28.266 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:05:28.267 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:05:28.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:05:28.267 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:05:28.267 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:05:28.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:05:28.267 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:05:28.267 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:05:28.267 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:05:28.267 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:05:28.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.258+0000 7f9691e9b700 1 -- 172.21.15.142:0/2382183011 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f967406dd60 msgr2=0x7f9674070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:28.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.258+0000 7f9691e9b700 1 --2- 172.21.15.142:0/2382183011 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f967406dd60 0x7f9674070220 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f967c009550 tx=0x7f967c0091b0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.258+0000 7f9691e9b700 1 -- 172.21.15.142:0/2382183011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f968c101750 msgr2=0x7f968c19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:28.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.258+0000 7f9691e9b700 1 --2- 172.21.15.142:0/2382183011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f968c101750 0x7f968c19ca50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f96800092e0 tx=0x7f968000bd70 comp rx=0 tx=0).stop 2024-03-29T05:05:28.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.258+0000 7f9691e9b700 1 -- 172.21.15.142:0/2382183011 shutdown_connections 2024-03-29T05:05:28.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.258+0000 7f9691e9b700 1 --2- 172.21.15.142:0/2382183011 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f967406dd60 0x7f9674070220 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.258+0000 7f9691e9b700 1 --2- 172.21.15.142:0/2382183011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f968c100da0 0x7f968c19c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.258+0000 7f9691e9b700 1 --2- 172.21.15.142:0/2382183011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f968c101750 0x7f968c19ca50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.258+0000 7f9691e9b700 1 -- 172.21.15.142:0/2382183011 >> 172.21.15.142:0/2382183011 conn(0x7f968c0fc650 msgr2=0x7f968c0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:28.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.258+0000 7f9691e9b700 1 -- 172.21.15.142:0/2382183011 shutdown_connections 2024-03-29T05:05:28.269 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.258+0000 7f9691e9b700 1 -- 172.21.15.142:0/2382183011 wait complete. 2024-03-29T05:05:28.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.361+0000 7f284ded5700 1 -- 172.21.15.142:0/1188374921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28480ffa40 msgr2=0x7f28480ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:28.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.361+0000 7f284ded5700 1 --2- 172.21.15.142:0/1188374921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28480ffa40 0x7f28480ffec0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f283c009910 tx=0x7f283c009c20 comp rx=0 tx=0).stop 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.362+0000 7f284ded5700 1 -- 172.21.15.142:0/1188374921 shutdown_connections 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.362+0000 7f284ded5700 1 --2- 172.21.15.142:0/1188374921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28480ffa40 0x7f28480ffec0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.362+0000 7f284ded5700 1 --2- 172.21.15.142:0/1188374921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2848105a40 0x7f2848105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.362+0000 7f284ded5700 1 -- 172.21.15.142:0/1188374921 >> 172.21.15.142:0/1188374921 conn(0x7f28480fb550 msgr2=0x7f28480fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.362+0000 7f284ded5700 1 -- 172.21.15.142:0/1188374921 shutdown_connections 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.362+0000 7f284ded5700 1 -- 172.21.15.142:0/1188374921 wait complete. 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.362+0000 7f284ded5700 1 Processor -- start 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.362+0000 7f284ded5700 1 -- start start 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.362+0000 7f284ded5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28480ffa40 0x7f28481953a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.362+0000 7f284ded5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2848105a40 0x7f28481958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f284ded5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2848195e20 con 0x7f28480ffa40 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f284ded5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2848195f90 con 0x7f2848105a40 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f28477fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28480ffa40 0x7f28481953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:28.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f28477fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28480ffa40 0x7f28481953a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56868/0 (socket says 172.21.15.142:56868) 2024-03-29T05:05:28.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f28477fe700 1 -- 172.21.15.142:0/3348982529 learned_addr learned my addr 172.21.15.142:0/3348982529 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:05:28.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f2846ffd700 1 --2- 172.21.15.142:0/3348982529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2848105a40 0x7f28481958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:28.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f28477fe700 1 -- 172.21.15.142:0/3348982529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2848105a40 msgr2=0x7f28481958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:28.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f28477fe700 1 --2- 172.21.15.142:0/3348982529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2848105a40 0x7f28481958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f28477fe700 1 -- 172.21.15.142:0/3348982529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c0095f0 con 0x7f28480ffa40 2024-03-29T05:05:28.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f28477fe700 1 --2- 172.21.15.142:0/3348982529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28480ffa40 0x7f28481953a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f283800d8d0 tx=0x7f283800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:28.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f2844ff9700 1 -- 172.21.15.142:0/3348982529 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2838009940 con 0x7f28480ffa40 2024-03-29T05:05:28.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f2844ff9700 1 -- 172.21.15.142:0/3348982529 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2838010460 con 0x7f28480ffa40 2024-03-29T05:05:28.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f284ded5700 1 -- 172.21.15.142:0/3348982529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28481a2a10 con 0x7f28480ffa40 2024-03-29T05:05:28.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f2844ff9700 1 -- 172.21.15.142:0/3348982529 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f283800f5f0 con 0x7f28480ffa40 2024-03-29T05:05:28.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.363+0000 7f284ded5700 1 -- 172.21.15.142:0/3348982529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28481a2d70 con 0x7f28480ffa40 2024-03-29T05:05:28.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.364+0000 7f284ded5700 1 -- 172.21.15.142:0/3348982529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2848107fd0 con 0x7f28480ffa40 2024-03-29T05:05:28.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.365+0000 7f2844ff9700 1 -- 172.21.15.142:0/3348982529 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f28380105d0 con 0x7f28480ffa40 2024-03-29T05:05:28.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.365+0000 7f2844ff9700 1 --2- 172.21.15.142:0/3348982529 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f283006dd60 0x7f2830070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:28.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.365+0000 7f2844ff9700 1 -- 172.21.15.142:0/3348982529 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f283808db40 con 0x7f28480ffa40 2024-03-29T05:05:28.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.365+0000 7f2846ffd700 1 --2- 172.21.15.142:0/3348982529 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f283006dd60 0x7f2830070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:28.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.366+0000 7f2846ffd700 1 --2- 172.21.15.142:0/3348982529 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f283006dd60 0x7f2830070220 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f283c0059d0 tx=0x7f283c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:28.380 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.370+0000 7f2844ff9700 1 -- 172.21.15.142:0/3348982529 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f283805ab10 con 0x7f28480ffa40 2024-03-29T05:05:28.618 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.607+0000 7f284ded5700 1 -- 172.21.15.142:0/3348982529 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f284804ea90 con 0x7f28480ffa40 2024-03-29T05:05:28.618 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.608+0000 7f2844ff9700 1 -- 172.21.15.142:0/3348982529 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f2838020770 con 0x7f28480ffa40 2024-03-29T05:05:28.618 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:05:28.618 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:05:28.619 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:05:28.620 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:05:28.621 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:05:28.621 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:05:28.621 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:05:28.621 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:05:28.621 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:05:28.621 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:05:28.621 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:05:28.621 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:05:28.621 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:05:28.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.611+0000 7f284ded5700 1 -- 172.21.15.142:0/3348982529 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f283006dd60 msgr2=0x7f2830070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:28.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.611+0000 7f284ded5700 1 --2- 172.21.15.142:0/3348982529 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f283006dd60 0x7f2830070220 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f283c0059d0 tx=0x7f283c005960 comp rx=0 tx=0).stop 2024-03-29T05:05:28.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.611+0000 7f284ded5700 1 -- 172.21.15.142:0/3348982529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28480ffa40 msgr2=0x7f28481953a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:28.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.611+0000 7f284ded5700 1 --2- 172.21.15.142:0/3348982529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28480ffa40 0x7f28481953a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f283800d8d0 tx=0x7f283800dc90 comp rx=0 tx=0).stop 2024-03-29T05:05:28.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.611+0000 7f284ded5700 1 -- 172.21.15.142:0/3348982529 shutdown_connections 2024-03-29T05:05:28.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.611+0000 7f284ded5700 1 --2- 172.21.15.142:0/3348982529 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f283006dd60 0x7f2830070220 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.611+0000 7f284ded5700 1 --2- 172.21.15.142:0/3348982529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28480ffa40 0x7f28481953a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.611+0000 7f284ded5700 1 --2- 172.21.15.142:0/3348982529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2848105a40 0x7f28481958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.611+0000 7f284ded5700 1 -- 172.21.15.142:0/3348982529 >> 172.21.15.142:0/3348982529 conn(0x7f28480fb550 msgr2=0x7f28480fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:28.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.611+0000 7f284ded5700 1 -- 172.21.15.142:0/3348982529 shutdown_connections 2024-03-29T05:05:28.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.611+0000 7f284ded5700 1 -- 172.21.15.142:0/3348982529 wait complete. 2024-03-29T05:05:28.623 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:05:28.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 -- 172.21.15.142:0/2128177196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4100410 msgr2=0x7f7bb41048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:28.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 --2- 172.21.15.142:0/2128177196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4100410 0x7f7bb41048e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009910 tx=0x7f7ba0009c20 comp rx=0 tx=0).stop 2024-03-29T05:05:28.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 -- 172.21.15.142:0/2128177196 shutdown_connections 2024-03-29T05:05:28.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 --2- 172.21.15.142:0/2128177196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4100410 0x7f7bb41048e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 --2- 172.21.15.142:0/2128177196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bb40ffa60 0x7f7bb40ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 -- 172.21.15.142:0/2128177196 >> 172.21.15.142:0/2128177196 conn(0x7f7bb40fb550 msgr2=0x7f7bb40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:28.728 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 -- 172.21.15.142:0/2128177196 shutdown_connections 2024-03-29T05:05:28.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 -- 172.21.15.142:0/2128177196 wait complete. 2024-03-29T05:05:28.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 Processor -- start 2024-03-29T05:05:28.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 -- start start 2024-03-29T05:05:28.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bb40ffa60 0x7f7bb419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:28.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4100410 0x7f7bb419d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:28.732 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb419d8e0 con 0x7f7bb4100410 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.718+0000 7f7bbad01700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb419da20 con 0x7f7bb40ffa60 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bb3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4100410 0x7f7bb419d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bb3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4100410 0x7f7bb419d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:41946/0 (socket says 172.21.15.142:41946) 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bb3fff700 1 -- 172.21.15.142:0/3350168630 learned_addr learned my addr 172.21.15.142:0/3350168630 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bb8a9d700 1 --2- 172.21.15.142:0/3350168630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bb40ffa60 0x7f7bb419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bb3fff700 1 -- 172.21.15.142:0/3350168630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bb40ffa60 msgr2=0x7f7bb419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bb3fff700 1 --2- 172.21.15.142:0/3350168630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bb40ffa60 0x7f7bb419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bb3fff700 1 -- 172.21.15.142:0/3350168630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba00095f0 con 0x7f7bb4100410 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bb3fff700 1 --2- 172.21.15.142:0/3350168630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4100410 0x7f7bb419d290 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000bda0 tx=0x7f7ba000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bb1ffb700 1 -- 172.21.15.142:0/3350168630 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ba001d070 con 0x7f7bb4100410 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bb1ffb700 1 -- 172.21.15.142:0/3350168630 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ba000f460 con 0x7f7bb4100410 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bbad01700 1 -- 172.21.15.142:0/3350168630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb41a1810 con 0x7f7bb4100410 2024-03-29T05:05:28.733 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bb1ffb700 1 -- 172.21.15.142:0/3350168630 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ba0017630 con 0x7f7bb4100410 2024-03-29T05:05:28.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.719+0000 7f7bbad01700 1 -- 172.21.15.142:0/3350168630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb41a1d00 con 0x7f7bb4100410 2024-03-29T05:05:28.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.720+0000 7f7bbad01700 1 -- 172.21.15.142:0/3350168630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb404ea90 con 0x7f7bb4100410 2024-03-29T05:05:28.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.721+0000 7f7bb1ffb700 1 -- 172.21.15.142:0/3350168630 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f7ba000fab0 con 0x7f7bb4100410 2024-03-29T05:05:28.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.721+0000 7f7bb1ffb700 1 --2- 172.21.15.142:0/3350168630 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7ba406dd60 0x7f7ba4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:28.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.721+0000 7f7bb1ffb700 1 -- 172.21.15.142:0/3350168630 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f7ba008f160 con 0x7f7bb4100410 2024-03-29T05:05:28.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.721+0000 7f7bb8a9d700 1 --2- 172.21.15.142:0/3350168630 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7ba406dd60 0x7f7ba4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:28.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.722+0000 7f7bb8a9d700 1 --2- 172.21.15.142:0/3350168630 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7ba406dd60 0x7f7ba4070220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800ac50 tx=0x7f7ba800a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:28.736 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.726+0000 7f7bb1ffb700 1 -- 172.21.15.142:0/3350168630 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7ba0058580 con 0x7f7bb4100410 2024-03-29T05:05:28.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:28 smithi142 ceph-mon[112475]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:05:28.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:28 smithi142 ceph-mon[112475]: from='client.34966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:05:28.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:28 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2382183011' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:05:28.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:28 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3348982529' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:05:28.951 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.941+0000 7f7bbad01700 1 -- 172.21.15.142:0/3350168630 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bb4101d10 con 0x7f7ba406dd60 2024-03-29T05:05:28.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.942+0000 7f7bb1ffb700 1 -- 172.21.15.142:0/3350168630 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7bb4101d10 con 0x7f7ba406dd60 2024-03-29T05:05:28.952 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:05:28.952 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:05:28.952 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:05:28.953 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:05:28.953 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:05:28.953 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:05:28.953 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:05:28.953 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:05:28.953 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:05:28.953 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:05:28.953 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:05:28.953 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:05:28.953 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:05:28.953 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:05:28.953 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:05:28.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.944+0000 7f7bbad01700 1 -- 172.21.15.142:0/3350168630 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7ba406dd60 msgr2=0x7f7ba4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:28.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.945+0000 7f7bbad01700 1 --2- 172.21.15.142:0/3350168630 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7ba406dd60 0x7f7ba4070220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800ac50 tx=0x7f7ba800a380 comp rx=0 tx=0).stop 2024-03-29T05:05:28.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.945+0000 7f7bbad01700 1 -- 172.21.15.142:0/3350168630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4100410 msgr2=0x7f7bb419d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:28.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.945+0000 7f7bbad01700 1 --2- 172.21.15.142:0/3350168630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4100410 0x7f7bb419d290 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000bda0 tx=0x7f7ba000bdd0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.945+0000 7f7bbad01700 1 -- 172.21.15.142:0/3350168630 shutdown_connections 2024-03-29T05:05:28.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.945+0000 7f7bbad01700 1 --2- 172.21.15.142:0/3350168630 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7ba406dd60 0x7f7ba4070220 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.945+0000 7f7bbad01700 1 --2- 172.21.15.142:0/3350168630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bb40ffa60 0x7f7bb419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.945+0000 7f7bbad01700 1 --2- 172.21.15.142:0/3350168630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4100410 0x7f7bb419d290 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:28.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.945+0000 7f7bbad01700 1 -- 172.21.15.142:0/3350168630 >> 172.21.15.142:0/3350168630 conn(0x7f7bb40fb550 msgr2=0x7f7bb4110f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:28.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.945+0000 7f7bbad01700 1 -- 172.21.15.142:0/3350168630 shutdown_connections 2024-03-29T05:05:28.956 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:28.945+0000 7f7bbad01700 1 -- 172.21.15.142:0/3350168630 wait complete. 2024-03-29T05:05:29.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:28 smithi152 ceph-mon[88186]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:05:29.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:28 smithi152 ceph-mon[88186]: from='client.34966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:05:29.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:28 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2382183011' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:05:29.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:28 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3348982529' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:05:29.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.048+0000 7f6adc70e700 1 -- 172.21.15.142:0/2134581311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4105a40 msgr2=0x7f6ad4105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:29.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.048+0000 7f6adc70e700 1 --2- 172.21.15.142:0/2134581311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4105a40 0x7f6ad4105e20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009910 tx=0x7f6ac4009c20 comp rx=0 tx=0).stop 2024-03-29T05:05:29.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.048+0000 7f6adc70e700 1 -- 172.21.15.142:0/2134581311 shutdown_connections 2024-03-29T05:05:29.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.048+0000 7f6adc70e700 1 --2- 172.21.15.142:0/2134581311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ad40ffbd0 0x7f6ad4100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:29.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.048+0000 7f6adc70e700 1 --2- 172.21.15.142:0/2134581311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4105a40 0x7f6ad4105e20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:29.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.048+0000 7f6adc70e700 1 -- 172.21.15.142:0/2134581311 >> 172.21.15.142:0/2134581311 conn(0x7f6ad40755a0 msgr2=0x7f6ad40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:29.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.048+0000 7f6adc70e700 1 -- 172.21.15.142:0/2134581311 shutdown_connections 2024-03-29T05:05:29.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.048+0000 7f6adc70e700 1 -- 172.21.15.142:0/2134581311 wait complete. 2024-03-29T05:05:29.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.049+0000 7f6adc70e700 1 Processor -- start 2024-03-29T05:05:29.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.049+0000 7f6adc70e700 1 -- start start 2024-03-29T05:05:29.060 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.049+0000 7f6adc70e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ad40ffbd0 0x7f6ad4195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.049+0000 7f6adc70e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4105a40 0x7f6ad4195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.049+0000 7f6adc70e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad4196260 con 0x7f6ad4105a40 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.049+0000 7f6adc70e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad4199ff0 con 0x7f6ad40ffbd0 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.049+0000 7f6ad9ca9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4105a40 0x7f6ad4195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.049+0000 7f6ad9ca9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4105a40 0x7f6ad4195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:41964/0 (socket says 172.21.15.142:41964) 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.049+0000 7f6ad9ca9700 1 -- 172.21.15.142:0/1441341459 learned_addr learned my addr 172.21.15.142:0/1441341459 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.049+0000 7f6ad9ca9700 1 -- 172.21.15.142:0/1441341459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ad40ffbd0 msgr2=0x7f6ad4195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.049+0000 7f6ad9ca9700 1 --2- 172.21.15.142:0/1441341459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ad40ffbd0 0x7f6ad4195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.049+0000 7f6ad9ca9700 1 -- 172.21.15.142:0/1441341459 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac40095f0 con 0x7f6ad4105a40 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.050+0000 7f6ad9ca9700 1 --2- 172.21.15.142:0/1441341459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4105a40 0x7f6ad4195b80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000d8d0 tx=0x7f6ad000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.050+0000 7f6acb7fe700 1 -- 172.21.15.142:0/1441341459 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ad0009940 con 0x7f6ad4105a40 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.050+0000 7f6adc70e700 1 -- 172.21.15.142:0/1441341459 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad419a1f0 con 0x7f6ad4105a40 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.050+0000 7f6acb7fe700 1 -- 172.21.15.142:0/1441341459 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ad0010460 con 0x7f6ad4105a40 2024-03-29T05:05:29.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.050+0000 7f6acb7fe700 1 -- 172.21.15.142:0/1441341459 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ad000f5f0 con 0x7f6ad4105a40 2024-03-29T05:05:29.062 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.050+0000 7f6adc70e700 1 -- 172.21.15.142:0/1441341459 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad419a740 con 0x7f6ad4105a40 2024-03-29T05:05:29.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.051+0000 7f6adc70e700 1 -- 172.21.15.142:0/1441341459 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad404ea90 con 0x7f6ad4105a40 2024-03-29T05:05:29.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.051+0000 7f6acb7fe700 1 -- 172.21.15.142:0/1441341459 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6ad00105d0 con 0x7f6ad4105a40 2024-03-29T05:05:29.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.052+0000 7f6acb7fe700 1 --2- 172.21.15.142:0/1441341459 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ac006dd60 0x7f6ac0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:29.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.052+0000 7f6acb7fe700 1 -- 172.21.15.142:0/1441341459 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6ad008dcd0 con 0x7f6ad4105a40 2024-03-29T05:05:29.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.052+0000 7f6ada4aa700 1 --2- 172.21.15.142:0/1441341459 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ac006dd60 0x7f6ac0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:29.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.052+0000 7f6ada4aa700 1 --2- 172.21.15.142:0/1441341459 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ac006dd60 0x7f6ac0070220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400b5c0 tx=0x7f6ac4005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:29.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.056+0000 7f6acb7fe700 1 -- 172.21.15.142:0/1441341459 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6ad00570f0 con 0x7f6ad4105a40 2024-03-29T05:05:29.341 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.331+0000 7f6adc70e700 1 -- 172.21.15.142:0/1441341459 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6ad41969a0 con 0x7f6ad4105a40 2024-03-29T05:05:29.342 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.331+0000 7f6acb7fe700 1 -- 172.21.15.142:0/1441341459 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f6ad0016100 con 0x7f6ad4105a40 2024-03-29T05:05:29.342 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:05:29.342 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:05:29.342 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:05:29.342 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:05:29.342 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:05:29.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.334+0000 7f6adc70e700 1 -- 172.21.15.142:0/1441341459 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ac006dd60 msgr2=0x7f6ac0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:29.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.334+0000 7f6adc70e700 1 --2- 172.21.15.142:0/1441341459 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ac006dd60 0x7f6ac0070220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400b5c0 tx=0x7f6ac4005960 comp rx=0 tx=0).stop 2024-03-29T05:05:29.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.334+0000 7f6adc70e700 1 -- 172.21.15.142:0/1441341459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4105a40 msgr2=0x7f6ad4195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:29.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.334+0000 7f6adc70e700 1 --2- 172.21.15.142:0/1441341459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4105a40 0x7f6ad4195b80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000d8d0 tx=0x7f6ad000dc90 comp rx=0 tx=0).stop 2024-03-29T05:05:29.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.334+0000 7f6adc70e700 1 -- 172.21.15.142:0/1441341459 shutdown_connections 2024-03-29T05:05:29.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.334+0000 7f6adc70e700 1 --2- 172.21.15.142:0/1441341459 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6ac006dd60 0x7f6ac0070220 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:29.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.334+0000 7f6adc70e700 1 --2- 172.21.15.142:0/1441341459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ad40ffbd0 0x7f6ad4195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:29.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.334+0000 7f6adc70e700 1 --2- 172.21.15.142:0/1441341459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ad4105a40 0x7f6ad4195b80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:29.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.334+0000 7f6adc70e700 1 -- 172.21.15.142:0/1441341459 >> 172.21.15.142:0/1441341459 conn(0x7f6ad40755a0 msgr2=0x7f6ad40fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:29.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.334+0000 7f6adc70e700 1 -- 172.21.15.142:0/1441341459 shutdown_connections 2024-03-29T05:05:29.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:29.334+0000 7f6adc70e700 1 -- 172.21.15.142:0/1441341459 wait complete. 2024-03-29T05:05:29.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:29 smithi152 ceph-mon[88186]: from='client.34970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:05:29.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:29 smithi152 ceph-mon[88186]: pgmap v515: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:29.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:29 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1441341459' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:05:29.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:05:30.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:29 smithi142 ceph-mon[112475]: from='client.34970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:05:30.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:29 smithi142 ceph-mon[112475]: pgmap v515: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:30.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:29 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1441341459' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:05:30.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:05:31.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:30 smithi152 ceph-mon[88186]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:05:31.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:30 smithi142 ceph-mon[112475]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:05:31.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:31 smithi142 ceph-mon[112475]: pgmap v516: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:32.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:31 smithi152 ceph-mon[88186]: pgmap v516: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:34.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:33 smithi152 ceph-mon[88186]: pgmap v517: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:34.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:33 smithi142 ceph-mon[112475]: pgmap v517: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:36.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:35 smithi152 ceph-mon[88186]: pgmap v518: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:36.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:35 smithi142 ceph-mon[112475]: pgmap v518: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:38.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:37 smithi152 ceph-mon[88186]: pgmap v519: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:38.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:37 smithi142 ceph-mon[112475]: pgmap v519: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:40.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:39 smithi152 ceph-mon[88186]: pgmap v520: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:40.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:39 smithi142 ceph-mon[112475]: pgmap v520: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:41.860 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:41 smithi142 ceph-mon[112475]: pgmap v521: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:42.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:41 smithi152 ceph-mon[88186]: pgmap v521: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:44.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:43 smithi152 ceph-mon[88186]: pgmap v522: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:44.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:43 smithi142 ceph-mon[112475]: pgmap v522: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:45.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:05:45.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:05:46.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:45 smithi152 ceph-mon[88186]: pgmap v523: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:46.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:45 smithi142 ceph-mon[112475]: pgmap v523: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:48.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:47 smithi152 ceph-mon[88186]: pgmap v524: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:48.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:47 smithi142 ceph-mon[112475]: pgmap v524: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:50.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:49 smithi152 ceph-mon[88186]: pgmap v525: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:50.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:49 smithi142 ceph-mon[112475]: pgmap v525: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:51.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:51 smithi142 ceph-mon[112475]: pgmap v526: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:52.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:51 smithi152 ceph-mon[88186]: pgmap v526: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:54.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:53 smithi152 ceph-mon[88186]: pgmap v527: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:54.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:53 smithi142 ceph-mon[112475]: pgmap v527: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:56.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:55 smithi152 ceph-mon[88186]: pgmap v528: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:56.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:55 smithi142 ceph-mon[112475]: pgmap v528: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:58.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:57 smithi152 ceph-mon[88186]: pgmap v529: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:58.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:57 smithi142 ceph-mon[112475]: pgmap v529: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:59.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.448+0000 7f0c9043d700 1 -- 172.21.15.142:0/3868376583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c88105a40 msgr2=0x7f0c88105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:59.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.448+0000 7f0c9043d700 1 --2- 172.21.15.142:0/3868376583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c88105a40 0x7f0c88105e20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0c84009910 tx=0x7f0c84009c20 comp rx=0 tx=0).stop 2024-03-29T05:05:59.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.448+0000 7f0c9043d700 1 -- 172.21.15.142:0/3868376583 shutdown_connections 2024-03-29T05:05:59.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.448+0000 7f0c9043d700 1 --2- 172.21.15.142:0/3868376583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c880ffa40 0x7f0c880ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:59.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.448+0000 7f0c9043d700 1 --2- 172.21.15.142:0/3868376583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c88105a40 0x7f0c88105e20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:59.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.448+0000 7f0c9043d700 1 -- 172.21.15.142:0/3868376583 >> 172.21.15.142:0/3868376583 conn(0x7f0c880fb550 msgr2=0x7f0c880fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:59.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.448+0000 7f0c9043d700 1 -- 172.21.15.142:0/3868376583 shutdown_connections 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.448+0000 7f0c9043d700 1 -- 172.21.15.142:0/3868376583 wait complete. 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.448+0000 7f0c9043d700 1 Processor -- start 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.448+0000 7f0c9043d700 1 -- start start 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c9043d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c880ffa40 0x7f0c8819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c9043d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c88105a40 0x7f0c8819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c9043d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8819d150 con 0x7f0c88105a40 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c9043d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c88196670 con 0x7f0c880ffa40 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c8d9d8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c88105a40 0x7f0c8819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c8d9d8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c88105a40 0x7f0c8819cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42310/0 (socket says 172.21.15.142:42310) 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c8d9d8700 1 -- 172.21.15.142:0/1710546370 learned_addr learned my addr 172.21.15.142:0/1710546370 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c8e1d9700 1 --2- 172.21.15.142:0/1710546370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c880ffa40 0x7f0c8819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c8d9d8700 1 -- 172.21.15.142:0/1710546370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c880ffa40 msgr2=0x7f0c8819c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c8d9d8700 1 --2- 172.21.15.142:0/1710546370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c880ffa40 0x7f0c8819c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:59.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c8d9d8700 1 -- 172.21.15.142:0/1710546370 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c840095f0 con 0x7f0c88105a40 2024-03-29T05:05:59.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c8d9d8700 1 --2- 172.21.15.142:0/1710546370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c88105a40 0x7f0c8819cb30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800c8a0 tx=0x7f0c7800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:59.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c7f7fe700 1 -- 172.21.15.142:0/1710546370 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c780078e0 con 0x7f0c88105a40 2024-03-29T05:05:59.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c7f7fe700 1 -- 172.21.15.142:0/1710546370 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c7800f450 con 0x7f0c88105a40 2024-03-29T05:05:59.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c7f7fe700 1 -- 172.21.15.142:0/1710546370 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c7800e5e0 con 0x7f0c88105a40 2024-03-29T05:05:59.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.449+0000 7f0c9043d700 1 -- 172.21.15.142:0/1710546370 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c88196950 con 0x7f0c88105a40 2024-03-29T05:05:59.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.450+0000 7f0c9043d700 1 -- 172.21.15.142:0/1710546370 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c88196ea0 con 0x7f0c88105a40 2024-03-29T05:05:59.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.451+0000 7f0c9043d700 1 -- 172.21.15.142:0/1710546370 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c88109050 con 0x7f0c88105a40 2024-03-29T05:05:59.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.451+0000 7f0c7f7fe700 1 -- 172.21.15.142:0/1710546370 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f0c7800f5c0 con 0x7f0c88105a40 2024-03-29T05:05:59.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.451+0000 7f0c7f7fe700 1 --2- 172.21.15.142:0/1710546370 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0c7406dd60 0x7f0c74070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:59.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.452+0000 7f0c7f7fe700 1 -- 172.21.15.142:0/1710546370 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f0c7808db90 con 0x7f0c88105a40 2024-03-29T05:05:59.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.452+0000 7f0c8e1d9700 1 --2- 172.21.15.142:0/1710546370 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0c7406dd60 0x7f0c74070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:59.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.452+0000 7f0c8e1d9700 1 --2- 172.21.15.142:0/1710546370 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0c7406dd60 0x7f0c74070220 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0c84005f50 tx=0x7f0c8401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:59.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.456+0000 7f0c7f7fe700 1 -- 172.21.15.142:0/1710546370 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0c7805ab60 con 0x7f0c88105a40 2024-03-29T05:05:59.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.676+0000 7f0c9043d700 1 -- 172.21.15.142:0/1710546370 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c88197170 con 0x7f0c7406dd60 2024-03-29T05:05:59.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.677+0000 7f0c7f7fe700 1 -- 172.21.15.142:0/1710546370 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0c88197170 con 0x7f0c7406dd60 2024-03-29T05:05:59.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.679+0000 7f0c9043d700 1 -- 172.21.15.142:0/1710546370 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0c7406dd60 msgr2=0x7f0c74070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:59.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.679+0000 7f0c9043d700 1 --2- 172.21.15.142:0/1710546370 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0c7406dd60 0x7f0c74070220 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0c84005f50 tx=0x7f0c8401a040 comp rx=0 tx=0).stop 2024-03-29T05:05:59.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.679+0000 7f0c9043d700 1 -- 172.21.15.142:0/1710546370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c88105a40 msgr2=0x7f0c8819cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:59.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.679+0000 7f0c9043d700 1 --2- 172.21.15.142:0/1710546370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c88105a40 0x7f0c8819cb30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800c8a0 tx=0x7f0c7800cbb0 comp rx=0 tx=0).stop 2024-03-29T05:05:59.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.679+0000 7f0c9043d700 1 -- 172.21.15.142:0/1710546370 shutdown_connections 2024-03-29T05:05:59.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.679+0000 7f0c9043d700 1 --2- 172.21.15.142:0/1710546370 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0c7406dd60 0x7f0c74070220 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:59.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.679+0000 7f0c9043d700 1 --2- 172.21.15.142:0/1710546370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c880ffa40 0x7f0c8819c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:59.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.679+0000 7f0c9043d700 1 --2- 172.21.15.142:0/1710546370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c88105a40 0x7f0c8819cb30 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:59.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.679+0000 7f0c9043d700 1 -- 172.21.15.142:0/1710546370 >> 172.21.15.142:0/1710546370 conn(0x7f0c880fb550 msgr2=0x7f0c88069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:59.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.679+0000 7f0c9043d700 1 -- 172.21.15.142:0/1710546370 shutdown_connections 2024-03-29T05:05:59.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.679+0000 7f0c9043d700 1 -- 172.21.15.142:0/1710546370 wait complete. 2024-03-29T05:05:59.700 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:05:59.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.784+0000 7ff7c2a49700 1 -- 172.21.15.142:0/493829180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7bc105a40 msgr2=0x7ff7bc105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:59.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.784+0000 7ff7c2a49700 1 --2- 172.21.15.142:0/493829180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7bc105a40 0x7ff7bc105e20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009910 tx=0x7ff7ac009c20 comp rx=0 tx=0).stop 2024-03-29T05:05:59.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.785+0000 7ff7c2a49700 1 -- 172.21.15.142:0/493829180 shutdown_connections 2024-03-29T05:05:59.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.785+0000 7ff7c2a49700 1 --2- 172.21.15.142:0/493829180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7bc0ffbd0 0x7ff7bc100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:59.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.785+0000 7ff7c2a49700 1 --2- 172.21.15.142:0/493829180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7bc105a40 0x7ff7bc105e20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:59.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.785+0000 7ff7c2a49700 1 -- 172.21.15.142:0/493829180 >> 172.21.15.142:0/493829180 conn(0x7ff7bc0755a0 msgr2=0x7ff7bc0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:05:59.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.785+0000 7ff7c2a49700 1 -- 172.21.15.142:0/493829180 shutdown_connections 2024-03-29T05:05:59.796 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:59 smithi142 ceph-mon[112475]: pgmap v530: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:05:59.796 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:05:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:05:59.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.785+0000 7ff7c2a49700 1 -- 172.21.15.142:0/493829180 wait complete. 2024-03-29T05:05:59.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.786+0000 7ff7c2a49700 1 Processor -- start 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.786+0000 7ff7c2a49700 1 -- start start 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.786+0000 7ff7c2a49700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7bc0ffbd0 0x7ff7bc19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.786+0000 7ff7c2a49700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7bc105a40 0x7ff7bc19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.786+0000 7ff7c2a49700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc19f2f0 con 0x7ff7bc0ffbd0 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.786+0000 7ff7c2a49700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc198770 con 0x7ff7bc105a40 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.786+0000 7ff7bbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7bc0ffbd0 0x7ff7bc19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.786+0000 7ff7bbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7bc0ffbd0 0x7ff7bc19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42324/0 (socket says 172.21.15.142:42324) 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.786+0000 7ff7bbfff700 1 -- 172.21.15.142:0/2473897759 learned_addr learned my addr 172.21.15.142:0/2473897759 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.786+0000 7ff7bb7fe700 1 --2- 172.21.15.142:0/2473897759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7bc105a40 0x7ff7bc19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.787+0000 7ff7bbfff700 1 -- 172.21.15.142:0/2473897759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7bc105a40 msgr2=0x7ff7bc19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.787+0000 7ff7bbfff700 1 --2- 172.21.15.142:0/2473897759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7bc105a40 0x7ff7bc19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.787+0000 7ff7bbfff700 1 -- 172.21.15.142:0/2473897759 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac0095f0 con 0x7ff7bc0ffbd0 2024-03-29T05:05:59.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.787+0000 7ff7bbfff700 1 --2- 172.21.15.142:0/2473897759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7bc0ffbd0 0x7ff7bc19e6d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac006010 tx=0x7ff7ac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:59.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.787+0000 7ff7b97fa700 1 -- 172.21.15.142:0/2473897759 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7ac01d070 con 0x7ff7bc0ffbd0 2024-03-29T05:05:59.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.787+0000 7ff7b97fa700 1 -- 172.21.15.142:0/2473897759 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7ac005dc0 con 0x7ff7bc0ffbd0 2024-03-29T05:05:59.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.787+0000 7ff7c2a49700 1 -- 172.21.15.142:0/2473897759 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc1989f0 con 0x7ff7bc0ffbd0 2024-03-29T05:05:59.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.787+0000 7ff7b97fa700 1 -- 172.21.15.142:0/2473897759 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7ac017400 con 0x7ff7bc0ffbd0 2024-03-29T05:05:59.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.787+0000 7ff7c2a49700 1 -- 172.21.15.142:0/2473897759 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7bc198ee0 con 0x7ff7bc0ffbd0 2024-03-29T05:05:59.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.788+0000 7ff7c2a49700 1 -- 172.21.15.142:0/2473897759 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7bc108040 con 0x7ff7bc0ffbd0 2024-03-29T05:05:59.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.789+0000 7ff7b97fa700 1 -- 172.21.15.142:0/2473897759 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff7ac003a60 con 0x7ff7bc0ffbd0 2024-03-29T05:05:59.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.789+0000 7ff7b97fa700 1 --2- 172.21.15.142:0/2473897759 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff7a406dd60 0x7ff7a4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:05:59.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.789+0000 7ff7b97fa700 1 -- 172.21.15.142:0/2473897759 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff7ac08f120 con 0x7ff7bc0ffbd0 2024-03-29T05:05:59.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.789+0000 7ff7bb7fe700 1 --2- 172.21.15.142:0/2473897759 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff7a406dd60 0x7ff7a4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:05:59.800 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.790+0000 7ff7bb7fe700 1 --2- 172.21.15.142:0/2473897759 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff7a406dd60 0x7ff7a4070220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc199f60 tx=0x7ff7b000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:05:59.804 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:05:59.794+0000 7ff7b97fa700 1 -- 172.21.15.142:0/2473897759 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff7ac05c070 con 0x7ff7bc0ffbd0 2024-03-29T05:06:00.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.019+0000 7ff7c2a49700 1 -- 172.21.15.142:0/2473897759 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7bc02cdf0 con 0x7ff7a406dd60 2024-03-29T05:06:00.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.020+0000 7ff7b97fa700 1 -- 172.21.15.142:0/2473897759 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff7bc02cdf0 con 0x7ff7a406dd60 2024-03-29T05:06:00.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.022+0000 7ff7c2a49700 1 -- 172.21.15.142:0/2473897759 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff7a406dd60 msgr2=0x7ff7a4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:00.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.022+0000 7ff7c2a49700 1 --2- 172.21.15.142:0/2473897759 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff7a406dd60 0x7ff7a4070220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc199f60 tx=0x7ff7b000b410 comp rx=0 tx=0).stop 2024-03-29T05:06:00.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.022+0000 7ff7c2a49700 1 -- 172.21.15.142:0/2473897759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7bc0ffbd0 msgr2=0x7ff7bc19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:00.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.022+0000 7ff7c2a49700 1 --2- 172.21.15.142:0/2473897759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7bc0ffbd0 0x7ff7bc19e6d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac006010 tx=0x7ff7ac00bd70 comp rx=0 tx=0).stop 2024-03-29T05:06:00.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.022+0000 7ff7c2a49700 1 -- 172.21.15.142:0/2473897759 shutdown_connections 2024-03-29T05:06:00.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.022+0000 7ff7c2a49700 1 --2- 172.21.15.142:0/2473897759 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff7a406dd60 0x7ff7a4070220 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.022+0000 7ff7c2a49700 1 --2- 172.21.15.142:0/2473897759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7bc0ffbd0 0x7ff7bc19e6d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.022+0000 7ff7c2a49700 1 --2- 172.21.15.142:0/2473897759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7bc105a40 0x7ff7bc19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.022+0000 7ff7c2a49700 1 -- 172.21.15.142:0/2473897759 >> 172.21.15.142:0/2473897759 conn(0x7ff7bc0755a0 msgr2=0x7ff7bc0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:00.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.022+0000 7ff7c2a49700 1 -- 172.21.15.142:0/2473897759 shutdown_connections 2024-03-29T05:06:00.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.023+0000 7ff7c2a49700 1 -- 172.21.15.142:0/2473897759 wait complete. 2024-03-29T05:06:00.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:59 smithi152 ceph-mon[88186]: pgmap v530: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:00.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:05:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:06:00.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.127+0000 7f89d77ee700 1 -- 172.21.15.142:0/1163650676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d00ff510 msgr2=0x7f89d00ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.127+0000 7f89d77ee700 1 --2- 172.21.15.142:0/1163650676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d00ff510 0x7f89d00ff8f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009910 tx=0x7f89cc009c20 comp rx=0 tx=0).stop 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.128+0000 7f89d77ee700 1 -- 172.21.15.142:0/1163650676 shutdown_connections 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.128+0000 7f89d77ee700 1 --2- 172.21.15.142:0/1163650676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89d00ffec0 0x7f89d0105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.128+0000 7f89d77ee700 1 --2- 172.21.15.142:0/1163650676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d00ff510 0x7f89d00ff8f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.128+0000 7f89d77ee700 1 -- 172.21.15.142:0/1163650676 >> 172.21.15.142:0/1163650676 conn(0x7f89d0074d80 msgr2=0x7f89d0075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.128+0000 7f89d77ee700 1 -- 172.21.15.142:0/1163650676 shutdown_connections 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.128+0000 7f89d77ee700 1 -- 172.21.15.142:0/1163650676 wait complete. 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.128+0000 7f89d77ee700 1 Processor -- start 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.128+0000 7f89d77ee700 1 -- start start 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.128+0000 7f89d77ee700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d00ff510 0x7f89d0100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.128+0000 7f89d77ee700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89d00ffec0 0x7f89d01009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.128+0000 7f89d77ee700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d0100f10 con 0x7f89d00ff510 2024-03-29T05:06:00.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89d77ee700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d0101050 con 0x7f89d00ffec0 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89d558a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d00ff510 0x7f89d0100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89d558a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d00ff510 0x7f89d0100490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42350/0 (socket says 172.21.15.142:42350) 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89d558a700 1 -- 172.21.15.142:0/2445079864 learned_addr learned my addr 172.21.15.142:0/2445079864 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89d4d89700 1 --2- 172.21.15.142:0/2445079864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89d00ffec0 0x7f89d01009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89d558a700 1 -- 172.21.15.142:0/2445079864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89d00ffec0 msgr2=0x7f89d01009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89d558a700 1 --2- 172.21.15.142:0/2445079864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89d00ffec0 0x7f89d01009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89d558a700 1 -- 172.21.15.142:0/2445079864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc0095f0 con 0x7f89d00ff510 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89d558a700 1 --2- 172.21.15.142:0/2445079864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d00ff510 0x7f89d0100490 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f89cc006010 tx=0x7f89cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89c67fc700 1 -- 172.21.15.142:0/2445079864 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89cc01d070 con 0x7f89d00ff510 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89c67fc700 1 -- 172.21.15.142:0/2445079864 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89cc004fb0 con 0x7f89d00ff510 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89c67fc700 1 -- 172.21.15.142:0/2445079864 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89cc022ea0 con 0x7f89d00ff510 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89d77ee700 1 -- 172.21.15.142:0/2445079864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d01012d0 con 0x7f89d00ff510 2024-03-29T05:06:00.141 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.129+0000 7f89d77ee700 1 -- 172.21.15.142:0/2445079864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d01a2c30 con 0x7f89d00ff510 2024-03-29T05:06:00.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.130+0000 7f89d77ee700 1 -- 172.21.15.142:0/2445079864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d0109180 con 0x7f89d00ff510 2024-03-29T05:06:00.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.131+0000 7f89c67fc700 1 -- 172.21.15.142:0/2445079864 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f89cc022470 con 0x7f89d00ff510 2024-03-29T05:06:00.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.131+0000 7f89c67fc700 1 --2- 172.21.15.142:0/2445079864 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f89bc06dd60 0x7f89bc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:00.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.131+0000 7f89d4d89700 1 --2- 172.21.15.142:0/2445079864 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f89bc06dd60 0x7f89bc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:00.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.131+0000 7f89c67fc700 1 -- 172.21.15.142:0/2445079864 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f89cc08f3b0 con 0x7f89d00ff510 2024-03-29T05:06:00.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.132+0000 7f89d4d89700 1 --2- 172.21.15.142:0/2445079864 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f89bc06dd60 0x7f89bc070220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f89c0005fd0 tx=0x7f89c000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:00.146 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.136+0000 7f89c67fc700 1 -- 172.21.15.142:0/2445079864 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f89cc058a20 con 0x7f89d00ff510 2024-03-29T05:06:00.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.344+0000 7f89d77ee700 1 -- 172.21.15.142:0/2445079864 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f89d002ccc0 con 0x7f89bc06dd60 2024-03-29T05:06:00.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.355+0000 7f89c67fc700 1 -- 172.21.15.142:0/2445079864 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f89d002ccc0 con 0x7f89bc06dd60 2024-03-29T05:06:00.365 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:06:00.365 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (17m) 10m ago 29m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:06:00.365 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (29m) 10m ago 29m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:06:00.365 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (28m) 10m ago 28m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:06:00.365 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (14m) 10m ago 29m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:06:00.366 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (14m) 10m ago 28m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:06:00.366 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (17m) 10m ago 29m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:06:00.366 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (11m) 10m ago 23m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:06:00.366 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (11m) 10m ago 23m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:06:00.366 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (10m) 10m ago 23m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:06:00.366 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (10m) 10m ago 23m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:06:00.366 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (19m) 10m ago 30m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:06:00.366 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (18m) 10m ago 28m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:06:00.366 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (16m) 10m ago 30m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:06:00.366 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (15m) 10m ago 28m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:06:00.366 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (18m) 10m ago 29m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:06:00.366 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (18m) 10m ago 28m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:06:00.367 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (14m) 10m ago 27m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:06:00.367 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (13m) 10m ago 27m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:06:00.367 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (13m) 10m ago 26m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:06:00.367 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (12m) 10m ago 26m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:06:00.367 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (12m) 10m ago 25m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:06:00.367 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (11m) 10m ago 25m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:06:00.367 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (18m) 10m ago 28m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:06:00.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.357+0000 7f89d77ee700 1 -- 172.21.15.142:0/2445079864 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f89bc06dd60 msgr2=0x7f89bc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:00.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.357+0000 7f89d77ee700 1 --2- 172.21.15.142:0/2445079864 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f89bc06dd60 0x7f89bc070220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f89c0005fd0 tx=0x7f89c000a560 comp rx=0 tx=0).stop 2024-03-29T05:06:00.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.357+0000 7f89d77ee700 1 -- 172.21.15.142:0/2445079864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d00ff510 msgr2=0x7f89d0100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:00.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.357+0000 7f89d77ee700 1 --2- 172.21.15.142:0/2445079864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d00ff510 0x7f89d0100490 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f89cc006010 tx=0x7f89cc00be20 comp rx=0 tx=0).stop 2024-03-29T05:06:00.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.357+0000 7f89d77ee700 1 -- 172.21.15.142:0/2445079864 shutdown_connections 2024-03-29T05:06:00.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.357+0000 7f89d77ee700 1 --2- 172.21.15.142:0/2445079864 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f89bc06dd60 0x7f89bc070220 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.357+0000 7f89d77ee700 1 --2- 172.21.15.142:0/2445079864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d00ff510 0x7f89d0100490 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.357+0000 7f89d77ee700 1 --2- 172.21.15.142:0/2445079864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89d00ffec0 0x7f89d01009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.357+0000 7f89d77ee700 1 -- 172.21.15.142:0/2445079864 >> 172.21.15.142:0/2445079864 conn(0x7f89d0074d80 msgr2=0x7f89d00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:00.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.357+0000 7f89d77ee700 1 -- 172.21.15.142:0/2445079864 shutdown_connections 2024-03-29T05:06:00.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.357+0000 7f89d77ee700 1 -- 172.21.15.142:0/2445079864 wait complete. 2024-03-29T05:06:00.469 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.459+0000 7f41420f2700 1 -- 172.21.15.142:0/2395326439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413c100dc0 msgr2=0x7f413c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:00.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.459+0000 7f41420f2700 1 --2- 172.21.15.142:0/2395326439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413c100dc0 0x7f413c1011a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f413400b0a0 tx=0x7f413400b3b0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.459+0000 7f41420f2700 1 -- 172.21.15.142:0/2395326439 shutdown_connections 2024-03-29T05:06:00.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.459+0000 7f41420f2700 1 --2- 172.21.15.142:0/2395326439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f413c101770 0x7f413c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.459+0000 7f41420f2700 1 --2- 172.21.15.142:0/2395326439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413c100dc0 0x7f413c1011a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.459+0000 7f41420f2700 1 -- 172.21.15.142:0/2395326439 >> 172.21.15.142:0/2395326439 conn(0x7f413c0fc650 msgr2=0x7f413c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:00.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.459+0000 7f41420f2700 1 -- 172.21.15.142:0/2395326439 shutdown_connections 2024-03-29T05:06:00.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.459+0000 7f41420f2700 1 -- 172.21.15.142:0/2395326439 wait complete. 2024-03-29T05:06:00.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.460+0000 7f41420f2700 1 Processor -- start 2024-03-29T05:06:00.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.460+0000 7f41420f2700 1 -- start start 2024-03-29T05:06:00.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.460+0000 7f41420f2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f413c100dc0 0x7f413c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:00.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.460+0000 7f41420f2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413c101770 0x7f413c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:00.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.460+0000 7f41420f2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c073b10 con 0x7f413c101770 2024-03-29T05:06:00.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.460+0000 7f41420f2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c073c50 con 0x7f413c100dc0 2024-03-29T05:06:00.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.460+0000 7f413affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413c101770 0x7f413c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:00.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.460+0000 7f413affd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413c101770 0x7f413c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42372/0 (socket says 172.21.15.142:42372) 2024-03-29T05:06:00.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.460+0000 7f413affd700 1 -- 172.21.15.142:0/2510933811 learned_addr learned my addr 172.21.15.142:0/2510933811 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:06:00.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.461+0000 7f413b7fe700 1 --2- 172.21.15.142:0/2510933811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f413c100dc0 0x7f413c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:00.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.461+0000 7f413affd700 1 -- 172.21.15.142:0/2510933811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f413c100dc0 msgr2=0x7f413c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:00.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.461+0000 7f413affd700 1 --2- 172.21.15.142:0/2510933811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f413c100dc0 0x7f413c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.461+0000 7f413affd700 1 -- 172.21.15.142:0/2510933811 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4134009d00 con 0x7f413c101770 2024-03-29T05:06:00.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.461+0000 7f413affd700 1 --2- 172.21.15.142:0/2510933811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413c101770 0x7f413c0735d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f413000d8d0 tx=0x7f413000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:00.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.461+0000 7f4138ff9700 1 -- 172.21.15.142:0/2510933811 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41300041f0 con 0x7f413c101770 2024-03-29T05:06:00.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.461+0000 7f4138ff9700 1 -- 172.21.15.142:0/2510933811 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4130007620 con 0x7f413c101770 2024-03-29T05:06:00.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.461+0000 7f41420f2700 1 -- 172.21.15.142:0/2510933811 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f413c1a2940 con 0x7f413c101770 2024-03-29T05:06:00.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.461+0000 7f4138ff9700 1 -- 172.21.15.142:0/2510933811 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41300056b0 con 0x7f413c101770 2024-03-29T05:06:00.473 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.461+0000 7f41420f2700 1 -- 172.21.15.142:0/2510933811 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413c1a2d50 con 0x7f413c101770 2024-03-29T05:06:00.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.462+0000 7f41420f2700 1 -- 172.21.15.142:0/2510933811 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413c109160 con 0x7f413c101770 2024-03-29T05:06:00.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.463+0000 7f4138ff9700 1 -- 172.21.15.142:0/2510933811 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f413000f770 con 0x7f413c101770 2024-03-29T05:06:00.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.463+0000 7f4138ff9700 1 --2- 172.21.15.142:0/2510933811 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f412406dd60 0x7f4124070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:00.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.463+0000 7f4138ff9700 1 -- 172.21.15.142:0/2510933811 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f413008dab0 con 0x7f413c101770 2024-03-29T05:06:00.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.463+0000 7f413b7fe700 1 --2- 172.21.15.142:0/2510933811 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f412406dd60 0x7f4124070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:00.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.464+0000 7f413b7fe700 1 --2- 172.21.15.142:0/2510933811 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f412406dd60 0x7f4124070220 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4134000f80 tx=0x7f41340060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:00.478 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.468+0000 7f4138ff9700 1 -- 172.21.15.142:0/2510933811 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4130056ed0 con 0x7f413c101770 2024-03-29T05:06:00.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.747+0000 7f41420f2700 1 -- 172.21.15.142:0/2510933811 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f413c02cc70 con 0x7f413c101770 2024-03-29T05:06:00.757 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:00 smithi142 ceph-mon[112475]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:00.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.747+0000 7f4138ff9700 1 -- 172.21.15.142:0/2510933811 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f413005a590 con 0x7f413c101770 2024-03-29T05:06:00.758 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:06:00.758 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:06:00.758 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:06:00.758 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:06:00.758 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:06:00.758 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:06:00.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:06:00.759 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:06:00.759 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:06:00.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:06:00.759 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:06:00.759 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:06:00.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:06:00.759 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:06:00.759 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:06:00.759 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:06:00.759 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:06:00.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.749+0000 7f41420f2700 1 -- 172.21.15.142:0/2510933811 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f412406dd60 msgr2=0x7f4124070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:00.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.749+0000 7f41420f2700 1 --2- 172.21.15.142:0/2510933811 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f412406dd60 0x7f4124070220 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4134000f80 tx=0x7f41340060c0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.749+0000 7f41420f2700 1 -- 172.21.15.142:0/2510933811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413c101770 msgr2=0x7f413c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:00.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.749+0000 7f41420f2700 1 --2- 172.21.15.142:0/2510933811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413c101770 0x7f413c0735d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f413000d8d0 tx=0x7f413000dbe0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.749+0000 7f41420f2700 1 -- 172.21.15.142:0/2510933811 shutdown_connections 2024-03-29T05:06:00.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.749+0000 7f41420f2700 1 --2- 172.21.15.142:0/2510933811 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f412406dd60 0x7f4124070220 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.749+0000 7f41420f2700 1 --2- 172.21.15.142:0/2510933811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f413c100dc0 0x7f413c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.749+0000 7f41420f2700 1 --2- 172.21.15.142:0/2510933811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413c101770 0x7f413c0735d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.749+0000 7f41420f2700 1 -- 172.21.15.142:0/2510933811 >> 172.21.15.142:0/2510933811 conn(0x7f413c0fc650 msgr2=0x7f413c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:00.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.750+0000 7f41420f2700 1 -- 172.21.15.142:0/2510933811 shutdown_connections 2024-03-29T05:06:00.760 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.750+0000 7f41420f2700 1 -- 172.21.15.142:0/2510933811 wait complete. 2024-03-29T05:06:00.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.853+0000 7ff1dafa2700 1 -- 172.21.15.142:0/194302452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1d4106a40 msgr2=0x7ff1d4106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:00.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.853+0000 7ff1dafa2700 1 --2- 172.21.15.142:0/194302452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1d4106a40 0x7ff1d4106e20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00b0a0 tx=0x7ff1cc00b3b0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.853+0000 7ff1dafa2700 1 -- 172.21.15.142:0/194302452 shutdown_connections 2024-03-29T05:06:00.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.853+0000 7ff1dafa2700 1 --2- 172.21.15.142:0/194302452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1d4068410 0x7ff1d4068890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.853+0000 7ff1dafa2700 1 --2- 172.21.15.142:0/194302452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1d4106a40 0x7ff1d4106e20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.853+0000 7ff1dafa2700 1 -- 172.21.15.142:0/194302452 >> 172.21.15.142:0/194302452 conn(0x7ff1d4075330 msgr2=0x7ff1d4075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:00.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.853+0000 7ff1dafa2700 1 -- 172.21.15.142:0/194302452 shutdown_connections 2024-03-29T05:06:00.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.853+0000 7ff1dafa2700 1 -- 172.21.15.142:0/194302452 wait complete. 2024-03-29T05:06:00.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1dafa2700 1 Processor -- start 2024-03-29T05:06:00.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1dafa2700 1 -- start start 2024-03-29T05:06:00.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1dafa2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1d4068410 0x7ff1d40ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:00.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1dafa2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1d4106a40 0x7ff1d4100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:00.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1dafa2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4100560 con 0x7ff1d4068410 2024-03-29T05:06:00.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1dafa2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d41006a0 con 0x7ff1d4106a40 2024-03-29T05:06:00.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1d9fa0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1d4068410 0x7ff1d40ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:00.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1d9fa0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1d4068410 0x7ff1d40ffae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42388/0 (socket says 172.21.15.142:42388) 2024-03-29T05:06:00.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1d9fa0700 1 -- 172.21.15.142:0/2723590096 learned_addr learned my addr 172.21.15.142:0/2723590096 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:06:00.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1d979f700 1 --2- 172.21.15.142:0/2723590096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1d4106a40 0x7ff1d4100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:00.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1d9fa0700 1 -- 172.21.15.142:0/2723590096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1d4106a40 msgr2=0x7ff1d4100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:00.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1d9fa0700 1 --2- 172.21.15.142:0/2723590096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1d4106a40 0x7ff1d4100020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:00.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1d9fa0700 1 -- 172.21.15.142:0/2723590096 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc009d00 con 0x7ff1d4068410 2024-03-29T05:06:00.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.854+0000 7ff1d9fa0700 1 --2- 172.21.15.142:0/2723590096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1d4068410 0x7ff1d40ffae0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc000f80 tx=0x7ff1cc00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:00.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.855+0000 7ff1caffd700 1 -- 172.21.15.142:0/2723590096 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1cc00e040 con 0x7ff1d4068410 2024-03-29T05:06:00.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.855+0000 7ff1dafa2700 1 -- 172.21.15.142:0/2723590096 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d4100920 con 0x7ff1d4068410 2024-03-29T05:06:00.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.855+0000 7ff1caffd700 1 -- 172.21.15.142:0/2723590096 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1cc003c10 con 0x7ff1d4068410 2024-03-29T05:06:00.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.855+0000 7ff1caffd700 1 -- 172.21.15.142:0/2723590096 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1cc01d6b0 con 0x7ff1d4068410 2024-03-29T05:06:00.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.855+0000 7ff1dafa2700 1 -- 172.21.15.142:0/2723590096 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d41a2a90 con 0x7ff1d4068410 2024-03-29T05:06:00.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.855+0000 7ff1dafa2700 1 -- 172.21.15.142:0/2723590096 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d4103ce0 con 0x7ff1d4068410 2024-03-29T05:06:00.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.856+0000 7ff1caffd700 1 -- 172.21.15.142:0/2723590096 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff1cc004040 con 0x7ff1d4068410 2024-03-29T05:06:00.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.856+0000 7ff1caffd700 1 --2- 172.21.15.142:0/2723590096 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff1c006dd60 0x7ff1c0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:00.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.857+0000 7ff1caffd700 1 -- 172.21.15.142:0/2723590096 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff1cc08f400 con 0x7ff1d4068410 2024-03-29T05:06:00.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.857+0000 7ff1d979f700 1 --2- 172.21.15.142:0/2723590096 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff1c006dd60 0x7ff1c0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:00.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.857+0000 7ff1d979f700 1 --2- 172.21.15.142:0/2723590096 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff1c006dd60 0x7ff1c0070220 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff1d41012d0 tx=0x7ff1c40073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:00.871 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:00.861+0000 7ff1caffd700 1 -- 172.21.15.142:0/2723590096 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff1cc058820 con 0x7ff1d4068410 2024-03-29T05:06:01.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:00 smithi152 ceph-mon[88186]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:01.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.103+0000 7ff1dafa2700 1 -- 172.21.15.142:0/2723590096 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff1d404f2e0 con 0x7ff1d4068410 2024-03-29T05:06:01.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.103+0000 7ff1caffd700 1 -- 172.21.15.142:0/2723590096 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7ff1cc05bee0 con 0x7ff1d4068410 2024-03-29T05:06:01.114 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:06:01.114 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:06:01.114 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:06:01.114 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:06:01.114 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:06:01.114 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:06:01.114 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:06:01.114 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:06:01.114 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:06:01.114 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:06:01.114 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:06:01.114 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:06:01.115 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:06:01.116 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:06:01.117 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:06:01.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.106+0000 7ff1dafa2700 1 -- 172.21.15.142:0/2723590096 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff1c006dd60 msgr2=0x7ff1c0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:01.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.106+0000 7ff1dafa2700 1 --2- 172.21.15.142:0/2723590096 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff1c006dd60 0x7ff1c0070220 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff1d41012d0 tx=0x7ff1c40073d0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.106+0000 7ff1dafa2700 1 -- 172.21.15.142:0/2723590096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1d4068410 msgr2=0x7ff1d40ffae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:01.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.106+0000 7ff1dafa2700 1 --2- 172.21.15.142:0/2723590096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1d4068410 0x7ff1d40ffae0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc000f80 tx=0x7ff1cc00cd20 comp rx=0 tx=0).stop 2024-03-29T05:06:01.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.106+0000 7ff1dafa2700 1 -- 172.21.15.142:0/2723590096 shutdown_connections 2024-03-29T05:06:01.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.106+0000 7ff1dafa2700 1 --2- 172.21.15.142:0/2723590096 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff1c006dd60 0x7ff1c0070220 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.106+0000 7ff1dafa2700 1 --2- 172.21.15.142:0/2723590096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1d4068410 0x7ff1d40ffae0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.106+0000 7ff1dafa2700 1 --2- 172.21.15.142:0/2723590096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1d4106a40 0x7ff1d4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.106+0000 7ff1dafa2700 1 -- 172.21.15.142:0/2723590096 >> 172.21.15.142:0/2723590096 conn(0x7ff1d4075330 msgr2=0x7ff1d40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:01.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.106+0000 7ff1dafa2700 1 -- 172.21.15.142:0/2723590096 shutdown_connections 2024-03-29T05:06:01.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.106+0000 7ff1dafa2700 1 -- 172.21.15.142:0/2723590096 wait complete. 2024-03-29T05:06:01.118 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:06:01.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.212+0000 7f3c81f77700 1 -- 172.21.15.142:0/284426303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c101d90 msgr2=0x7f3c7c102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:01.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.212+0000 7f3c81f77700 1 --2- 172.21.15.142:0/284426303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c101d90 0x7f3c7c102170 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009910 tx=0x7f3c6c009c20 comp rx=0 tx=0).stop 2024-03-29T05:06:01.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.212+0000 7f3c81f77700 1 -- 172.21.15.142:0/284426303 shutdown_connections 2024-03-29T05:06:01.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.212+0000 7f3c81f77700 1 --2- 172.21.15.142:0/284426303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c7c1026b0 0x7f3c7c10aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.212+0000 7f3c81f77700 1 --2- 172.21.15.142:0/284426303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c101d90 0x7f3c7c102170 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.212+0000 7f3c81f77700 1 -- 172.21.15.142:0/284426303 >> 172.21.15.142:0/284426303 conn(0x7f3c7c0fb540 msgr2=0x7f3c7c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:01.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.212+0000 7f3c81f77700 1 -- 172.21.15.142:0/284426303 shutdown_connections 2024-03-29T05:06:01.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.212+0000 7f3c81f77700 1 -- 172.21.15.142:0/284426303 wait complete. 2024-03-29T05:06:01.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.212+0000 7f3c81f77700 1 Processor -- start 2024-03-29T05:06:01.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.213+0000 7f3c81f77700 1 -- start start 2024-03-29T05:06:01.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.213+0000 7f3c81f77700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c7c101d90 0x7f3c7c0ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:01.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.213+0000 7f3c81f77700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c1026b0 0x7f3c7c0fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:01.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.213+0000 7f3c81f77700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c100470 con 0x7f3c7c1026b0 2024-03-29T05:06:01.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.213+0000 7f3c81f77700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c1005b0 con 0x7f3c7c101d90 2024-03-29T05:06:01.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.213+0000 7f3c72ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c1026b0 0x7f3c7c0fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:01.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.213+0000 7f3c72ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c1026b0 0x7f3c7c0fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42406/0 (socket says 172.21.15.142:42406) 2024-03-29T05:06:01.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.213+0000 7f3c72ffd700 1 -- 172.21.15.142:0/1638538451 learned_addr learned my addr 172.21.15.142:0/1638538451 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:06:01.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.213+0000 7f3c72ffd700 1 -- 172.21.15.142:0/1638538451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c7c101d90 msgr2=0x7f3c7c0ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:01.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.213+0000 7f3c72ffd700 1 --2- 172.21.15.142:0/1638538451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c7c101d90 0x7f3c7c0ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.213+0000 7f3c72ffd700 1 -- 172.21.15.142:0/1638538451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c0095f0 con 0x7f3c7c1026b0 2024-03-29T05:06:01.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.213+0000 7f3c72ffd700 1 --2- 172.21.15.142:0/1638538451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c1026b0 0x7f3c7c0fff30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400c960 tx=0x7f3c6400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:01.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.214+0000 7f3c797fa700 1 -- 172.21.15.142:0/1638538451 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c64007a70 con 0x7f3c7c1026b0 2024-03-29T05:06:01.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.214+0000 7f3c81f77700 1 -- 172.21.15.142:0/1638538451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7c1a28d0 con 0x7f3c7c1026b0 2024-03-29T05:06:01.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.214+0000 7f3c797fa700 1 -- 172.21.15.142:0/1638538451 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c64007bd0 con 0x7f3c7c1026b0 2024-03-29T05:06:01.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.214+0000 7f3c797fa700 1 -- 172.21.15.142:0/1638538451 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c640186c0 con 0x7f3c7c1026b0 2024-03-29T05:06:01.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.214+0000 7f3c81f77700 1 -- 172.21.15.142:0/1638538451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7c1a2c30 con 0x7f3c7c1026b0 2024-03-29T05:06:01.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.214+0000 7f3c81f77700 1 -- 172.21.15.142:0/1638538451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7c1082a0 con 0x7f3c7c1026b0 2024-03-29T05:06:01.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.215+0000 7f3c797fa700 1 -- 172.21.15.142:0/1638538451 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f3c6400faa0 con 0x7f3c7c1026b0 2024-03-29T05:06:01.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.215+0000 7f3c797fa700 1 --2- 172.21.15.142:0/1638538451 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3c6806dd60 0x7f3c68070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:01.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.216+0000 7f3c797fa700 1 -- 172.21.15.142:0/1638538451 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f3c6408dce0 con 0x7f3c7c1026b0 2024-03-29T05:06:01.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.216+0000 7f3c7b7fe700 1 --2- 172.21.15.142:0/1638538451 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3c6806dd60 0x7f3c68070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:01.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.216+0000 7f3c7b7fe700 1 --2- 172.21.15.142:0/1638538451 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3c6806dd60 0x7f3c68070220 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00b5c0 tx=0x7f3c6c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:01.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.220+0000 7f3c797fa700 1 -- 172.21.15.142:0/1638538451 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3c6405ac30 con 0x7f3c7c1026b0 2024-03-29T05:06:01.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.433+0000 7f3c81f77700 1 -- 172.21.15.142:0/1638538451 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c7c02cd80 con 0x7f3c6806dd60 2024-03-29T05:06:01.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.434+0000 7f3c797fa700 1 -- 172.21.15.142:0/1638538451 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3c7c02cd80 con 0x7f3c6806dd60 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:06:01.445 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:06:01.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.436+0000 7f3c81f77700 1 -- 172.21.15.142:0/1638538451 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3c6806dd60 msgr2=0x7f3c68070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:01.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.436+0000 7f3c81f77700 1 --2- 172.21.15.142:0/1638538451 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3c6806dd60 0x7f3c68070220 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00b5c0 tx=0x7f3c6c005960 comp rx=0 tx=0).stop 2024-03-29T05:06:01.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.436+0000 7f3c81f77700 1 -- 172.21.15.142:0/1638538451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c1026b0 msgr2=0x7f3c7c0fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:01.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.436+0000 7f3c81f77700 1 --2- 172.21.15.142:0/1638538451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c1026b0 0x7f3c7c0fff30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400c960 tx=0x7f3c6400cd20 comp rx=0 tx=0).stop 2024-03-29T05:06:01.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.436+0000 7f3c81f77700 1 -- 172.21.15.142:0/1638538451 shutdown_connections 2024-03-29T05:06:01.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.436+0000 7f3c81f77700 1 --2- 172.21.15.142:0/1638538451 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3c6806dd60 0x7f3c68070220 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.436+0000 7f3c81f77700 1 --2- 172.21.15.142:0/1638538451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c7c101d90 0x7f3c7c0ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.436+0000 7f3c81f77700 1 --2- 172.21.15.142:0/1638538451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c7c1026b0 0x7f3c7c0fff30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.436+0000 7f3c81f77700 1 -- 172.21.15.142:0/1638538451 >> 172.21.15.142:0/1638538451 conn(0x7f3c7c0fb540 msgr2=0x7f3c7c1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:01.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.437+0000 7f3c81f77700 1 -- 172.21.15.142:0/1638538451 shutdown_connections 2024-03-29T05:06:01.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.437+0000 7f3c81f77700 1 -- 172.21.15.142:0/1638538451 wait complete. 2024-03-29T05:06:01.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.540+0000 7f4a06ea3700 1 -- 172.21.15.142:0/1255786698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a00100dc0 msgr2=0x7f4a001011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:01.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.540+0000 7f4a06ea3700 1 --2- 172.21.15.142:0/1255786698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a00100dc0 0x7f4a001011a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009910 tx=0x7f49f4009c20 comp rx=0 tx=0).stop 2024-03-29T05:06:01.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.541+0000 7f4a06ea3700 1 -- 172.21.15.142:0/1255786698 shutdown_connections 2024-03-29T05:06:01.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.541+0000 7f4a06ea3700 1 --2- 172.21.15.142:0/1255786698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a00101770 0x7f4a001057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.541+0000 7f4a06ea3700 1 --2- 172.21.15.142:0/1255786698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a00100dc0 0x7f4a001011a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.541+0000 7f4a06ea3700 1 -- 172.21.15.142:0/1255786698 >> 172.21.15.142:0/1255786698 conn(0x7f4a000fc650 msgr2=0x7f4a000fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:01.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.541+0000 7f4a06ea3700 1 -- 172.21.15.142:0/1255786698 shutdown_connections 2024-03-29T05:06:01.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.541+0000 7f4a06ea3700 1 -- 172.21.15.142:0/1255786698 wait complete. 2024-03-29T05:06:01.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.541+0000 7f4a06ea3700 1 Processor -- start 2024-03-29T05:06:01.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.541+0000 7f4a06ea3700 1 -- start start 2024-03-29T05:06:01.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f4a06ea3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a00100dc0 0x7f4a00073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:01.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f4a06ea3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a00101770 0x7f4a000735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:01.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f4a06ea3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00073b10 con 0x7f4a00100dc0 2024-03-29T05:06:01.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f4a06ea3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00073c50 con 0x7f4a00101770 2024-03-29T05:06:01.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f4a04c3f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a00100dc0 0x7f4a00073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:01.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f4a04c3f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a00100dc0 0x7f4a00073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:42416/0 (socket says 172.21.15.142:42416) 2024-03-29T05:06:01.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f4a04c3f700 1 -- 172.21.15.142:0/2183765290 learned_addr learned my addr 172.21.15.142:0/2183765290 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:06:01.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f49fffff700 1 --2- 172.21.15.142:0/2183765290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a00101770 0x7f4a000735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:01.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f4a04c3f700 1 -- 172.21.15.142:0/2183765290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a00101770 msgr2=0x7f4a000735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:01.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f4a04c3f700 1 --2- 172.21.15.142:0/2183765290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a00101770 0x7f4a000735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f4a04c3f700 1 -- 172.21.15.142:0/2183765290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f40095f0 con 0x7f4a00100dc0 2024-03-29T05:06:01.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f4a04c3f700 1 --2- 172.21.15.142:0/2183765290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a00100dc0 0x7f4a00073090 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f49f4006010 tx=0x7f49f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:01.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f49fdffb700 1 -- 172.21.15.142:0/2183765290 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49f401d070 con 0x7f4a00100dc0 2024-03-29T05:06:01.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f49fdffb700 1 -- 172.21.15.142:0/2183765290 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49f4004fb0 con 0x7f4a00100dc0 2024-03-29T05:06:01.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f49fdffb700 1 -- 172.21.15.142:0/2183765290 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49f4022ea0 con 0x7f4a00100dc0 2024-03-29T05:06:01.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.542+0000 7f4a06ea3700 1 -- 172.21.15.142:0/2183765290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a00073ed0 con 0x7f4a00100dc0 2024-03-29T05:06:01.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.543+0000 7f4a06ea3700 1 -- 172.21.15.142:0/2183765290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a001a2b70 con 0x7f4a00100dc0 2024-03-29T05:06:01.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.544+0000 7f4a06ea3700 1 -- 172.21.15.142:0/2183765290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a00109160 con 0x7f4a00100dc0 2024-03-29T05:06:01.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.544+0000 7f49fdffb700 1 -- 172.21.15.142:0/2183765290 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f49f4022470 con 0x7f4a00100dc0 2024-03-29T05:06:01.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.544+0000 7f49fdffb700 1 --2- 172.21.15.142:0/2183765290 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f49f006dd60 0x7f49f0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:01.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.544+0000 7f49fdffb700 1 -- 172.21.15.142:0/2183765290 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f49f408f1f0 con 0x7f4a00100dc0 2024-03-29T05:06:01.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.544+0000 7f49fffff700 1 --2- 172.21.15.142:0/2183765290 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f49f006dd60 0x7f49f0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:01.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.545+0000 7f49fffff700 1 --2- 172.21.15.142:0/2183765290 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f49f006dd60 0x7f49f0070220 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f49ec005ea0 tx=0x7f49ec005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:01.559 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.549+0000 7f49fdffb700 1 -- 172.21.15.142:0/2183765290 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f49f40586c0 con 0x7f4a00100dc0 2024-03-29T05:06:01.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.828+0000 7f4a06ea3700 1 -- 172.21.15.142:0/2183765290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4a0004ea90 con 0x7f4a00100dc0 2024-03-29T05:06:01.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.829+0000 7f49fdffb700 1 -- 172.21.15.142:0/2183765290 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f49f4022720 con 0x7f4a00100dc0 2024-03-29T05:06:01.839 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:06:01.840 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:06:01.840 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:06:01.840 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:06:01.840 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:06:01.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.831+0000 7f4a06ea3700 1 -- 172.21.15.142:0/2183765290 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f49f006dd60 msgr2=0x7f49f0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:01.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.831+0000 7f4a06ea3700 1 --2- 172.21.15.142:0/2183765290 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f49f006dd60 0x7f49f0070220 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f49ec005ea0 tx=0x7f49ec005e30 comp rx=0 tx=0).stop 2024-03-29T05:06:01.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.831+0000 7f4a06ea3700 1 -- 172.21.15.142:0/2183765290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a00100dc0 msgr2=0x7f4a00073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:01.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.831+0000 7f4a06ea3700 1 --2- 172.21.15.142:0/2183765290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a00100dc0 0x7f4a00073090 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f49f4006010 tx=0x7f49f400be20 comp rx=0 tx=0).stop 2024-03-29T05:06:01.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.831+0000 7f4a06ea3700 1 -- 172.21.15.142:0/2183765290 shutdown_connections 2024-03-29T05:06:01.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.831+0000 7f4a06ea3700 1 --2- 172.21.15.142:0/2183765290 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f49f006dd60 0x7f49f0070220 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.831+0000 7f4a06ea3700 1 --2- 172.21.15.142:0/2183765290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a00100dc0 0x7f4a00073090 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.831+0000 7f4a06ea3700 1 --2- 172.21.15.142:0/2183765290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a00101770 0x7f4a000735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:01.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.831+0000 7f4a06ea3700 1 -- 172.21.15.142:0/2183765290 >> 172.21.15.142:0/2183765290 conn(0x7f4a000fc650 msgr2=0x7f4a000fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:01.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.831+0000 7f4a06ea3700 1 -- 172.21.15.142:0/2183765290 shutdown_connections 2024-03-29T05:06:01.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:01.831+0000 7f4a06ea3700 1 -- 172.21.15.142:0/2183765290 wait complete. 2024-03-29T05:06:02.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:01 smithi152 ceph-mon[88186]: pgmap v531: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:02.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:01 smithi152 ceph-mon[88186]: from='client.34994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:02.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:01 smithi152 ceph-mon[88186]: from='client.34998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:02.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:01 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2510933811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:06:02.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:01 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2723590096' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:06:02.074 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:01 smithi142 ceph-mon[112475]: pgmap v531: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:02.074 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:01 smithi142 ceph-mon[112475]: from='client.34994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:02.074 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:01 smithi142 ceph-mon[112475]: from='client.34998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:02.074 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:01 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2510933811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:06:02.074 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:01 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2723590096' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:06:03.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:02 smithi152 ceph-mon[88186]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:03.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:02 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2183765290' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:06:03.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:02 smithi142 ceph-mon[112475]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:03.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:02 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2183765290' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:06:04.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:03 smithi152 ceph-mon[88186]: pgmap v532: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:04.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:03 smithi142 ceph-mon[112475]: pgmap v532: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:06.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:05 smithi152 ceph-mon[88186]: pgmap v533: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:05 smithi142 ceph-mon[112475]: pgmap v533: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:08.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:07 smithi152 ceph-mon[88186]: pgmap v534: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:08.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:07 smithi142 ceph-mon[112475]: pgmap v534: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:10.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:09 smithi152 ceph-mon[88186]: pgmap v535: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:10.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:09 smithi142 ceph-mon[112475]: pgmap v535: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:11.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:11 smithi142 ceph-mon[112475]: pgmap v536: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:12.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:11 smithi152 ceph-mon[88186]: pgmap v536: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:14.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:13 smithi152 ceph-mon[88186]: pgmap v537: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:14.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:13 smithi142 ceph-mon[112475]: pgmap v537: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:15.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:06:15.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:06:16.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:15 smithi152 ceph-mon[88186]: pgmap v538: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:16.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:15 smithi142 ceph-mon[112475]: pgmap v538: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:18.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:17 smithi152 ceph-mon[88186]: pgmap v539: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:18.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:17 smithi142 ceph-mon[112475]: pgmap v539: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:20.017 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:19 smithi152 ceph-mon[88186]: pgmap v540: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:20.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:19 smithi142 ceph-mon[112475]: pgmap v540: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:21.882 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:21 smithi142 ceph-mon[112475]: pgmap v541: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:22.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:21 smithi152 ceph-mon[88186]: pgmap v541: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:24.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:23 smithi152 ceph-mon[88186]: pgmap v542: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:24.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:23 smithi142 ceph-mon[112475]: pgmap v542: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:26.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:25 smithi152 ceph-mon[88186]: pgmap v543: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:26.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:25 smithi142 ceph-mon[112475]: pgmap v543: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:28.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:27 smithi152 ceph-mon[88186]: pgmap v544: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:28.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:27 smithi142 ceph-mon[112475]: pgmap v544: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:30.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:29 smithi152 ceph-mon[88186]: pgmap v545: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:30.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:06:30.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:29 smithi142 ceph-mon[112475]: pgmap v545: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:30.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:06:31.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:31 smithi142 ceph-mon[112475]: pgmap v546: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:31.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.948+0000 7f619ac73700 1 -- 172.21.15.142:0/4171198730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6194106be0 msgr2=0x7f6194106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:31.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.948+0000 7f619ac73700 1 --2- 172.21.15.142:0/4171198730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6194106be0 0x7f6194106fc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6188009910 tx=0x7f6188009c20 comp rx=0 tx=0).stop 2024-03-29T05:06:31.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.948+0000 7f619ac73700 1 -- 172.21.15.142:0/4171198730 shutdown_connections 2024-03-29T05:06:31.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.948+0000 7f619ac73700 1 --2- 172.21.15.142:0/4171198730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61940ff260 0x7f61940ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:31.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.948+0000 7f619ac73700 1 --2- 172.21.15.142:0/4171198730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6194106be0 0x7f6194106fc0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:31.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.948+0000 7f619ac73700 1 -- 172.21.15.142:0/4171198730 >> 172.21.15.142:0/4171198730 conn(0x7f6194074ce0 msgr2=0x7f61940750f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:31.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.948+0000 7f619ac73700 1 -- 172.21.15.142:0/4171198730 shutdown_connections 2024-03-29T05:06:31.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.948+0000 7f619ac73700 1 -- 172.21.15.142:0/4171198730 wait complete. 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.949+0000 7f619ac73700 1 Processor -- start 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.949+0000 7f619ac73700 1 -- start start 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.949+0000 7f619ac73700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61940ff260 0x7f6194196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.949+0000 7f619ac73700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6194106be0 0x7f6194196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.949+0000 7f619ac73700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6194197250 con 0x7f6194106be0 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.949+0000 7f619ac73700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619419afe0 con 0x7f61940ff260 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.949+0000 7f6193fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6194106be0 0x7f6194196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.949+0000 7f6193fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6194106be0 0x7f6194196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37698/0 (socket says 172.21.15.142:37698) 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.949+0000 7f6193fff700 1 -- 172.21.15.142:0/68560895 learned_addr learned my addr 172.21.15.142:0/68560895 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.950+0000 7f6198a0f700 1 --2- 172.21.15.142:0/68560895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61940ff260 0x7f6194196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.950+0000 7f6193fff700 1 -- 172.21.15.142:0/68560895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61940ff260 msgr2=0x7f6194196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.950+0000 7f6193fff700 1 --2- 172.21.15.142:0/68560895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61940ff260 0x7f6194196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:31.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.950+0000 7f6193fff700 1 -- 172.21.15.142:0/68560895 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61880095f0 con 0x7f6194106be0 2024-03-29T05:06:31.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.950+0000 7f6193fff700 1 --2- 172.21.15.142:0/68560895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6194106be0 0x7f6194196b70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f618000d8d0 tx=0x7f618000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:31.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.950+0000 7f6191ffb700 1 -- 172.21.15.142:0/68560895 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6180009940 con 0x7f6194106be0 2024-03-29T05:06:31.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.950+0000 7f6191ffb700 1 -- 172.21.15.142:0/68560895 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6180010460 con 0x7f6194106be0 2024-03-29T05:06:31.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.950+0000 7f619ac73700 1 -- 172.21.15.142:0/68560895 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619419b2c0 con 0x7f6194106be0 2024-03-29T05:06:31.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.951+0000 7f6191ffb700 1 -- 172.21.15.142:0/68560895 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f618000f5f0 con 0x7f6194106be0 2024-03-29T05:06:31.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.951+0000 7f619ac73700 1 -- 172.21.15.142:0/68560895 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619419b890 con 0x7f6194106be0 2024-03-29T05:06:31.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.952+0000 7f619ac73700 1 -- 172.21.15.142:0/68560895 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619404ea90 con 0x7f6194106be0 2024-03-29T05:06:31.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.952+0000 7f6191ffb700 1 -- 172.21.15.142:0/68560895 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f61800105d0 con 0x7f6194106be0 2024-03-29T05:06:31.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.953+0000 7f6191ffb700 1 --2- 172.21.15.142:0/68560895 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f618406dd60 0x7f6184070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:31.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.953+0000 7f6191ffb700 1 -- 172.21.15.142:0/68560895 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f618008db80 con 0x7f6194106be0 2024-03-29T05:06:31.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.953+0000 7f6198a0f700 1 --2- 172.21.15.142:0/68560895 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f618406dd60 0x7f6184070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:31.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.953+0000 7f6198a0f700 1 --2- 172.21.15.142:0/68560895 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f618406dd60 0x7f6184070220 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f618800b5c0 tx=0x7f618801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:31.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:31.958+0000 7f6191ffb700 1 -- 172.21.15.142:0/68560895 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6180057020 con 0x7f6194106be0 2024-03-29T05:06:32.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:31 smithi152 ceph-mon[88186]: pgmap v546: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:32.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.173+0000 7f619ac73700 1 -- 172.21.15.142:0/68560895 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f619419bcf0 con 0x7f618406dd60 2024-03-29T05:06:32.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.174+0000 7f6191ffb700 1 -- 172.21.15.142:0/68560895 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f619419bcf0 con 0x7f618406dd60 2024-03-29T05:06:32.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.176+0000 7f619ac73700 1 -- 172.21.15.142:0/68560895 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f618406dd60 msgr2=0x7f6184070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:32.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.176+0000 7f619ac73700 1 --2- 172.21.15.142:0/68560895 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f618406dd60 0x7f6184070220 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f618800b5c0 tx=0x7f618801a040 comp rx=0 tx=0).stop 2024-03-29T05:06:32.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.176+0000 7f619ac73700 1 -- 172.21.15.142:0/68560895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6194106be0 msgr2=0x7f6194196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:32.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.176+0000 7f619ac73700 1 --2- 172.21.15.142:0/68560895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6194106be0 0x7f6194196b70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f618000d8d0 tx=0x7f618000dc90 comp rx=0 tx=0).stop 2024-03-29T05:06:32.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.176+0000 7f619ac73700 1 -- 172.21.15.142:0/68560895 shutdown_connections 2024-03-29T05:06:32.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.176+0000 7f619ac73700 1 --2- 172.21.15.142:0/68560895 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f618406dd60 0x7f6184070220 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.176+0000 7f619ac73700 1 --2- 172.21.15.142:0/68560895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61940ff260 0x7f6194196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.176+0000 7f619ac73700 1 --2- 172.21.15.142:0/68560895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6194106be0 0x7f6194196b70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.176+0000 7f619ac73700 1 -- 172.21.15.142:0/68560895 >> 172.21.15.142:0/68560895 conn(0x7f6194074ce0 msgr2=0x7f61940fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:32.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.176+0000 7f619ac73700 1 -- 172.21.15.142:0/68560895 shutdown_connections 2024-03-29T05:06:32.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.176+0000 7f619ac73700 1 -- 172.21.15.142:0/68560895 wait complete. 2024-03-29T05:06:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:06:32.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.280+0000 7fa13b3b5700 1 -- 172.21.15.142:0/2225997591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa134100dc0 msgr2=0x7fa1341011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:32.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.280+0000 7fa13b3b5700 1 --2- 172.21.15.142:0/2225997591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa134100dc0 0x7fa1341011a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa130009910 tx=0x7fa130009c20 comp rx=0 tx=0).stop 2024-03-29T05:06:32.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.281+0000 7fa13b3b5700 1 -- 172.21.15.142:0/2225997591 shutdown_connections 2024-03-29T05:06:32.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.281+0000 7fa13b3b5700 1 --2- 172.21.15.142:0/2225997591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa134101770 0x7fa1341057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.281+0000 7fa13b3b5700 1 --2- 172.21.15.142:0/2225997591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa134100dc0 0x7fa1341011a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.281+0000 7fa13b3b5700 1 -- 172.21.15.142:0/2225997591 >> 172.21.15.142:0/2225997591 conn(0x7fa1340fc650 msgr2=0x7fa1340fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:32.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.281+0000 7fa13b3b5700 1 -- 172.21.15.142:0/2225997591 shutdown_connections 2024-03-29T05:06:32.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.281+0000 7fa13b3b5700 1 -- 172.21.15.142:0/2225997591 wait complete. 2024-03-29T05:06:32.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.281+0000 7fa13b3b5700 1 Processor -- start 2024-03-29T05:06:32.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.281+0000 7fa13b3b5700 1 -- start start 2024-03-29T05:06:32.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa13b3b5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa134100dc0 0x7fa13419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:32.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa13b3b5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa134101770 0x7fa13419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:32.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa13b3b5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13419d110 con 0x7fa134100dc0 2024-03-29T05:06:32.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa13b3b5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1341965c0 con 0x7fa134101770 2024-03-29T05:06:32.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa139151700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa134100dc0 0x7fa13419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:32.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa139151700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa134100dc0 0x7fa13419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37710/0 (socket says 172.21.15.142:37710) 2024-03-29T05:06:32.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa139151700 1 -- 172.21.15.142:0/392504496 learned_addr learned my addr 172.21.15.142:0/392504496 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:06:32.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa138950700 1 --2- 172.21.15.142:0/392504496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa134101770 0x7fa13419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:32.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa139151700 1 -- 172.21.15.142:0/392504496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa134101770 msgr2=0x7fa13419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:32.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa139151700 1 --2- 172.21.15.142:0/392504496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa134101770 0x7fa13419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa139151700 1 -- 172.21.15.142:0/392504496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1300095f0 con 0x7fa134100dc0 2024-03-29T05:06:32.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa139151700 1 --2- 172.21.15.142:0/392504496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa134100dc0 0x7fa13419c540 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa130006010 tx=0x7fa13000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:32.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa12a7fc700 1 -- 172.21.15.142:0/392504496 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa13001d070 con 0x7fa134100dc0 2024-03-29T05:06:32.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa12a7fc700 1 -- 172.21.15.142:0/392504496 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa130005dc0 con 0x7fa134100dc0 2024-03-29T05:06:32.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa13b3b5700 1 -- 172.21.15.142:0/392504496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa134196840 con 0x7fa134100dc0 2024-03-29T05:06:32.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa12a7fc700 1 -- 172.21.15.142:0/392504496 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa130017400 con 0x7fa134100dc0 2024-03-29T05:06:32.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.282+0000 7fa13b3b5700 1 -- 172.21.15.142:0/392504496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa134196d30 con 0x7fa134100dc0 2024-03-29T05:06:32.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.283+0000 7fa13b3b5700 1 -- 172.21.15.142:0/392504496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa134109160 con 0x7fa134100dc0 2024-03-29T05:06:32.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.284+0000 7fa12a7fc700 1 -- 172.21.15.142:0/392504496 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fa130003a60 con 0x7fa134100dc0 2024-03-29T05:06:32.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.284+0000 7fa12a7fc700 1 --2- 172.21.15.142:0/392504496 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa12006dd60 0x7fa120070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:32.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.284+0000 7fa12a7fc700 1 -- 172.21.15.142:0/392504496 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa13008f0f0 con 0x7fa134100dc0 2024-03-29T05:06:32.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.284+0000 7fa138950700 1 --2- 172.21.15.142:0/392504496 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa12006dd60 0x7fa120070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:32.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.285+0000 7fa138950700 1 --2- 172.21.15.142:0/392504496 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa12006dd60 0x7fa120070220 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa1341059b0 tx=0x7fa12400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:32.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.289+0000 7fa12a7fc700 1 -- 172.21.15.142:0/392504496 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa13005c040 con 0x7fa134100dc0 2024-03-29T05:06:32.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.506+0000 7fa13b3b5700 1 -- 172.21.15.142:0/392504496 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa13402cca0 con 0x7fa12006dd60 2024-03-29T05:06:32.517 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.507+0000 7fa12a7fc700 1 -- 172.21.15.142:0/392504496 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa13402cca0 con 0x7fa12006dd60 2024-03-29T05:06:32.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.509+0000 7fa13b3b5700 1 -- 172.21.15.142:0/392504496 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa12006dd60 msgr2=0x7fa120070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:32.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.509+0000 7fa13b3b5700 1 --2- 172.21.15.142:0/392504496 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa12006dd60 0x7fa120070220 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa1341059b0 tx=0x7fa12400b410 comp rx=0 tx=0).stop 2024-03-29T05:06:32.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.509+0000 7fa13b3b5700 1 -- 172.21.15.142:0/392504496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa134100dc0 msgr2=0x7fa13419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:32.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.509+0000 7fa13b3b5700 1 --2- 172.21.15.142:0/392504496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa134100dc0 0x7fa13419c540 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa130006010 tx=0x7fa13000bd70 comp rx=0 tx=0).stop 2024-03-29T05:06:32.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.509+0000 7fa13b3b5700 1 -- 172.21.15.142:0/392504496 shutdown_connections 2024-03-29T05:06:32.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.509+0000 7fa13b3b5700 1 --2- 172.21.15.142:0/392504496 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa12006dd60 0x7fa120070220 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.520 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.509+0000 7fa13b3b5700 1 --2- 172.21.15.142:0/392504496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa134100dc0 0x7fa13419c540 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.509+0000 7fa13b3b5700 1 --2- 172.21.15.142:0/392504496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa134101770 0x7fa13419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.509+0000 7fa13b3b5700 1 -- 172.21.15.142:0/392504496 >> 172.21.15.142:0/392504496 conn(0x7fa1340fc650 msgr2=0x7fa1340fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:32.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.510+0000 7fa13b3b5700 1 -- 172.21.15.142:0/392504496 shutdown_connections 2024-03-29T05:06:32.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.510+0000 7fa13b3b5700 1 -- 172.21.15.142:0/392504496 wait complete. 2024-03-29T05:06:32.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.614+0000 7f99a6558700 1 -- 172.21.15.142:0/4058625966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0102620 msgr2=0x7f99a010ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:32.625 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.614+0000 7f99a6558700 1 --2- 172.21.15.142:0/4058625966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0102620 0x7f99a010ab10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9994009910 tx=0x7f9994009c20 comp rx=0 tx=0).stop 2024-03-29T05:06:32.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.614+0000 7f99a6558700 1 -- 172.21.15.142:0/4058625966 shutdown_connections 2024-03-29T05:06:32.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.614+0000 7f99a6558700 1 --2- 172.21.15.142:0/4058625966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0102620 0x7f99a010ab10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.614+0000 7f99a6558700 1 --2- 172.21.15.142:0/4058625966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a0101d00 0x7f99a01020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.614+0000 7f99a6558700 1 -- 172.21.15.142:0/4058625966 >> 172.21.15.142:0/4058625966 conn(0x7f99a00fb570 msgr2=0x7f99a00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:32.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.615+0000 7f99a6558700 1 -- 172.21.15.142:0/4058625966 shutdown_connections 2024-03-29T05:06:32.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.615+0000 7f99a6558700 1 -- 172.21.15.142:0/4058625966 wait complete. 2024-03-29T05:06:32.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.615+0000 7f99a6558700 1 Processor -- start 2024-03-29T05:06:32.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.615+0000 7f99a6558700 1 -- start start 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f99a6558700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a0101d00 0x7f99a00ff960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f99a6558700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0102620 0x7f99a00ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f99a6558700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a01003e0 con 0x7f99a0102620 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f99a6558700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0100520 con 0x7f99a0101d00 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f999f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0102620 0x7f99a00ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f999f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0102620 0x7f99a00ffea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37726/0 (socket says 172.21.15.142:37726) 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f999f7fe700 1 -- 172.21.15.142:0/1377168616 learned_addr learned my addr 172.21.15.142:0/1377168616 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f999ffff700 1 --2- 172.21.15.142:0/1377168616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a0101d00 0x7f99a00ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f999f7fe700 1 -- 172.21.15.142:0/1377168616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a0101d00 msgr2=0x7f99a00ff960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f999f7fe700 1 --2- 172.21.15.142:0/1377168616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a0101d00 0x7f99a00ff960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f999f7fe700 1 -- 172.21.15.142:0/1377168616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99940095f0 con 0x7f99a0102620 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f999f7fe700 1 --2- 172.21.15.142:0/1377168616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0102620 0x7f99a00ffea0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f99940092e0 tx=0x7f999400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f999d7fa700 1 -- 172.21.15.142:0/1377168616 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f999401d070 con 0x7f99a0102620 2024-03-29T05:06:32.628 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f999d7fa700 1 -- 172.21.15.142:0/1377168616 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f999400f460 con 0x7f99a0102620 2024-03-29T05:06:32.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.616+0000 7f99a6558700 1 -- 172.21.15.142:0/1377168616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a01007a0 con 0x7f99a0102620 2024-03-29T05:06:32.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.617+0000 7f999d7fa700 1 -- 172.21.15.142:0/1377168616 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99940175f0 con 0x7f99a0102620 2024-03-29T05:06:32.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.617+0000 7f99a6558700 1 -- 172.21.15.142:0/1377168616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a01a2db0 con 0x7f99a0102620 2024-03-29T05:06:32.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.617+0000 7f99a6558700 1 -- 172.21.15.142:0/1377168616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a0108210 con 0x7f99a0102620 2024-03-29T05:06:32.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.618+0000 7f999d7fa700 1 -- 172.21.15.142:0/1377168616 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f999400f5d0 con 0x7f99a0102620 2024-03-29T05:06:32.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.619+0000 7f999d7fa700 1 --2- 172.21.15.142:0/1377168616 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f998806dd60 0x7f9988070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:32.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.619+0000 7f999d7fa700 1 -- 172.21.15.142:0/1377168616 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f999408f160 con 0x7f99a0102620 2024-03-29T05:06:32.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.619+0000 7f999ffff700 1 --2- 172.21.15.142:0/1377168616 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f998806dd60 0x7f9988070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:32.630 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.619+0000 7f999ffff700 1 --2- 172.21.15.142:0/1377168616 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f998806dd60 0x7f9988070220 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9990009a10 tx=0x7f9990009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:32.633 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.623+0000 7f999d7fa700 1 -- 172.21.15.142:0/1377168616 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f99940586a0 con 0x7f99a0102620 2024-03-29T05:06:32.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.831+0000 7f99a6558700 1 -- 172.21.15.142:0/1377168616 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f99a002cce0 con 0x7f998806dd60 2024-03-29T05:06:32.852 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.842+0000 7f999d7fa700 1 -- 172.21.15.142:0/1377168616 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f99a002cce0 con 0x7f998806dd60 2024-03-29T05:06:32.852 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:06:32.852 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (18m) 11m ago 29m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:06:32.852 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (30m) 11m ago 30m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:06:32.852 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (29m) 10m ago 29m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:06:32.852 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (15m) 11m ago 30m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:06:32.852 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (15m) 10m ago 29m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:06:32.852 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (18m) 11m ago 29m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (11m) 11m ago 24m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (11m) 11m ago 24m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (10m) 10m ago 24m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (11m) 10m ago 24m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (19m) 11m ago 31m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (19m) 10m ago 28m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (16m) 11m ago 31m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (16m) 10m ago 28m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (19m) 11m ago 30m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (18m) 10m ago 28m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (15m) 11m ago 28m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (14m) 11m ago 27m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:06:32.853 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (13m) 11m ago 27m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:06:32.854 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (13m) 10m ago 26m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:06:32.854 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (12m) 10m ago 26m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:06:32.854 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (12m) 10m ago 25m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:06:32.854 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (18m) 11m ago 29m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:06:32.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.844+0000 7f99a6558700 1 -- 172.21.15.142:0/1377168616 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f998806dd60 msgr2=0x7f9988070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:32.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.844+0000 7f99a6558700 1 --2- 172.21.15.142:0/1377168616 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f998806dd60 0x7f9988070220 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9990009a10 tx=0x7f9990009190 comp rx=0 tx=0).stop 2024-03-29T05:06:32.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.844+0000 7f99a6558700 1 -- 172.21.15.142:0/1377168616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0102620 msgr2=0x7f99a00ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:32.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.844+0000 7f99a6558700 1 --2- 172.21.15.142:0/1377168616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0102620 0x7f99a00ffea0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f99940092e0 tx=0x7f999400bd70 comp rx=0 tx=0).stop 2024-03-29T05:06:32.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.844+0000 7f99a6558700 1 -- 172.21.15.142:0/1377168616 shutdown_connections 2024-03-29T05:06:32.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.844+0000 7f99a6558700 1 --2- 172.21.15.142:0/1377168616 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f998806dd60 0x7f9988070220 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.844+0000 7f99a6558700 1 --2- 172.21.15.142:0/1377168616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a0101d00 0x7f99a00ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.844+0000 7f99a6558700 1 --2- 172.21.15.142:0/1377168616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0102620 0x7f99a00ffea0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.844+0000 7f99a6558700 1 -- 172.21.15.142:0/1377168616 >> 172.21.15.142:0/1377168616 conn(0x7f99a00fb570 msgr2=0x7f99a0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:32.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.844+0000 7f99a6558700 1 -- 172.21.15.142:0/1377168616 shutdown_connections 2024-03-29T05:06:32.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.844+0000 7f99a6558700 1 -- 172.21.15.142:0/1377168616 wait complete. 2024-03-29T05:06:32.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.947+0000 7f75a7838700 1 -- 172.21.15.142:0/66561734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a0105a70 msgr2=0x7f75a0105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:32.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.947+0000 7f75a7838700 1 --2- 172.21.15.142:0/66561734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a0105a70 0x7f75a0105e50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7590009910 tx=0x7f7590009c20 comp rx=0 tx=0).stop 2024-03-29T05:06:32.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.947+0000 7f75a7838700 1 -- 172.21.15.142:0/66561734 shutdown_connections 2024-03-29T05:06:32.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.947+0000 7f75a7838700 1 --2- 172.21.15.142:0/66561734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a00ffa70 0x7f75a00ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.947+0000 7f75a7838700 1 --2- 172.21.15.142:0/66561734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a0105a70 0x7f75a0105e50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.958 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.947+0000 7f75a7838700 1 -- 172.21.15.142:0/66561734 >> 172.21.15.142:0/66561734 conn(0x7f75a00fb560 msgr2=0x7f75a00fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:32.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.947+0000 7f75a7838700 1 -- 172.21.15.142:0/66561734 shutdown_connections 2024-03-29T05:06:32.959 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.948+0000 7f75a7838700 1 -- 172.21.15.142:0/66561734 wait complete. 2024-03-29T05:06:32.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.948+0000 7f75a7838700 1 Processor -- start 2024-03-29T05:06:32.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.948+0000 7f75a7838700 1 -- start start 2024-03-29T05:06:32.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.948+0000 7f75a7838700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a00ffa70 0x7f75a00fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:32.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.948+0000 7f75a7838700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a0105a70 0x7f75a00fc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:32.960 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.948+0000 7f75a7838700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a00fc950 con 0x7f75a0105a70 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.948+0000 7f75a7838700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a00fcac0 con 0x7f75a00ffa70 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.948+0000 7f75a4dd3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a0105a70 0x7f75a00fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.948+0000 7f75a4dd3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a0105a70 0x7f75a00fc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37742/0 (socket says 172.21.15.142:37742) 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.948+0000 7f75a4dd3700 1 -- 172.21.15.142:0/1378815318 learned_addr learned my addr 172.21.15.142:0/1378815318 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.949+0000 7f75a55d4700 1 --2- 172.21.15.142:0/1378815318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a00ffa70 0x7f75a00fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.949+0000 7f75a4dd3700 1 -- 172.21.15.142:0/1378815318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a00ffa70 msgr2=0x7f75a00fbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.949+0000 7f75a4dd3700 1 --2- 172.21.15.142:0/1378815318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a00ffa70 0x7f75a00fbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.949+0000 7f75a4dd3700 1 -- 172.21.15.142:0/1378815318 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75900095f0 con 0x7f75a0105a70 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.949+0000 7f75a4dd3700 1 --2- 172.21.15.142:0/1378815318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a0105a70 0x7f75a00fc380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f759c00d900 tx=0x7f759c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.949+0000 7f75967fc700 1 -- 172.21.15.142:0/1378815318 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f759c004a00 con 0x7f75a0105a70 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.949+0000 7f75967fc700 1 -- 172.21.15.142:0/1378815318 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f759c0054e0 con 0x7f75a0105a70 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.949+0000 7f75a7838700 1 -- 172.21.15.142:0/1378815318 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a01a2900 con 0x7f75a0105a70 2024-03-29T05:06:32.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.949+0000 7f75a7838700 1 -- 172.21.15.142:0/1378815318 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a01a2d70 con 0x7f75a0105a70 2024-03-29T05:06:32.962 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.949+0000 7f75967fc700 1 -- 172.21.15.142:0/1378815318 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f759c009d00 con 0x7f75a0105a70 2024-03-29T05:06:32.962 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.950+0000 7f75a7838700 1 -- 172.21.15.142:0/1378815318 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a0109080 con 0x7f75a0105a70 2024-03-29T05:06:32.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.951+0000 7f75967fc700 1 -- 172.21.15.142:0/1378815318 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f759c005020 con 0x7f75a0105a70 2024-03-29T05:06:32.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.951+0000 7f75967fc700 1 --2- 172.21.15.142:0/1378815318 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f758c06dd60 0x7f758c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:32.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.951+0000 7f75967fc700 1 -- 172.21.15.142:0/1378815318 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f759c08dc50 con 0x7f75a0105a70 2024-03-29T05:06:32.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.951+0000 7f75a55d4700 1 --2- 172.21.15.142:0/1378815318 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f758c06dd60 0x7f758c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:32.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.952+0000 7f75a55d4700 1 --2- 172.21.15.142:0/1378815318 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f758c06dd60 0x7f758c070220 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7590006010 tx=0x7f759001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:32.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:32.956+0000 7f75967fc700 1 -- 172.21.15.142:0/1378815318 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f759c057070 con 0x7f75a0105a70 2024-03-29T05:06:33.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.229+0000 7f75a7838700 1 -- 172.21.15.142:0/1378815318 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f75a004ea90 con 0x7f75a0105a70 2024-03-29T05:06:33.241 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.230+0000 7f75967fc700 1 -- 172.21.15.142:0/1378815318 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f759c017280 con 0x7f75a0105a70 2024-03-29T05:06:33.241 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:06:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:06:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:06:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:06:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:06:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:06:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:06:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:06:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:06:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:06:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:06:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:06:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:06:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:06:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:06:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:06:33.242 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:06:33.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.233+0000 7f75a7838700 1 -- 172.21.15.142:0/1378815318 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f758c06dd60 msgr2=0x7f758c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:33.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.233+0000 7f75a7838700 1 --2- 172.21.15.142:0/1378815318 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f758c06dd60 0x7f758c070220 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7590006010 tx=0x7f759001a040 comp rx=0 tx=0).stop 2024-03-29T05:06:33.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.233+0000 7f75a7838700 1 -- 172.21.15.142:0/1378815318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a0105a70 msgr2=0x7f75a00fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:33.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.233+0000 7f75a7838700 1 --2- 172.21.15.142:0/1378815318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a0105a70 0x7f75a00fc380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f759c00d900 tx=0x7f759c00dc10 comp rx=0 tx=0).stop 2024-03-29T05:06:33.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.233+0000 7f75a7838700 1 -- 172.21.15.142:0/1378815318 shutdown_connections 2024-03-29T05:06:33.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.233+0000 7f75a7838700 1 --2- 172.21.15.142:0/1378815318 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f758c06dd60 0x7f758c070220 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.233+0000 7f75a7838700 1 --2- 172.21.15.142:0/1378815318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a00ffa70 0x7f75a00fbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.233+0000 7f75a7838700 1 --2- 172.21.15.142:0/1378815318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a0105a70 0x7f75a00fc380 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.233+0000 7f75a7838700 1 -- 172.21.15.142:0/1378815318 >> 172.21.15.142:0/1378815318 conn(0x7f75a00fb560 msgr2=0x7f75a0068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:33.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.233+0000 7f75a7838700 1 -- 172.21.15.142:0/1378815318 shutdown_connections 2024-03-29T05:06:33.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.233+0000 7f75a7838700 1 -- 172.21.15.142:0/1378815318 wait complete. 2024-03-29T05:06:33.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.337+0000 7fe0fe63c700 1 -- 172.21.15.142:0/1690853567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0f8102680 msgr2=0x7fe0f810ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:33.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.337+0000 7fe0fe63c700 1 --2- 172.21.15.142:0/1690853567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0f8102680 0x7fe0f810ab70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009940 tx=0x7fe0ec009c50 comp rx=0 tx=0).stop 2024-03-29T05:06:33.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.337+0000 7fe0fe63c700 1 -- 172.21.15.142:0/1690853567 shutdown_connections 2024-03-29T05:06:33.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.337+0000 7fe0fe63c700 1 --2- 172.21.15.142:0/1690853567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0f8102680 0x7fe0f810ab70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.337+0000 7fe0fe63c700 1 --2- 172.21.15.142:0/1690853567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f8101d60 0x7fe0f8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.337+0000 7fe0fe63c700 1 -- 172.21.15.142:0/1690853567 >> 172.21.15.142:0/1690853567 conn(0x7fe0f80fb550 msgr2=0x7fe0f80fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:33.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.337+0000 7fe0fe63c700 1 -- 172.21.15.142:0/1690853567 shutdown_connections 2024-03-29T05:06:33.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.337+0000 7fe0fe63c700 1 -- 172.21.15.142:0/1690853567 wait complete. 2024-03-29T05:06:33.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.337+0000 7fe0fe63c700 1 Processor -- start 2024-03-29T05:06:33.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.337+0000 7fe0fe63c700 1 -- start start 2024-03-29T05:06:33.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0fe63c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f8101d60 0x7fe0f819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:33.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0fe63c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0f8102680 0x7fe0f819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:33.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0fe63c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f819d110 con 0x7fe0f8102680 2024-03-29T05:06:33.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0fe63c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f81965c0 con 0x7fe0f8101d60 2024-03-29T05:06:33.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0f77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0f8102680 0x7fe0f819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0f77fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0f8102680 0x7fe0f819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37766/0 (socket says 172.21.15.142:37766) 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0f77fe700 1 -- 172.21.15.142:0/976647676 learned_addr learned my addr 172.21.15.142:0/976647676 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0f7fff700 1 --2- 172.21.15.142:0/976647676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f8101d60 0x7fe0f819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0f77fe700 1 -- 172.21.15.142:0/976647676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f8101d60 msgr2=0x7fe0f819c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0f77fe700 1 --2- 172.21.15.142:0/976647676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f8101d60 0x7fe0f819c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0f77fe700 1 -- 172.21.15.142:0/976647676 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec0095f0 con 0x7fe0f8102680 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0f7fff700 1 --2- 172.21.15.142:0/976647676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f8101d60 0x7fe0f819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0f77fe700 1 --2- 172.21.15.142:0/976647676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0f8102680 0x7fe0f819ca80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec000c00 tx=0x7fe0ec0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0f57fa700 1 -- 172.21.15.142:0/976647676 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0ec01d070 con 0x7fe0f8102680 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0f57fa700 1 -- 172.21.15.142:0/976647676 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0ec00f460 con 0x7fe0f8102680 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0fe63c700 1 -- 172.21.15.142:0/976647676 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f8196840 con 0x7fe0f8102680 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0f57fa700 1 -- 172.21.15.142:0/976647676 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0ec021640 con 0x7fe0f8102680 2024-03-29T05:06:33.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.338+0000 7fe0fe63c700 1 -- 172.21.15.142:0/976647676 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f8196d30 con 0x7fe0f8102680 2024-03-29T05:06:33.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.339+0000 7fe0fe63c700 1 -- 172.21.15.142:0/976647676 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f8108270 con 0x7fe0f8102680 2024-03-29T05:06:33.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.340+0000 7fe0f57fa700 1 -- 172.21.15.142:0/976647676 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fe0ec00fa60 con 0x7fe0f8102680 2024-03-29T05:06:33.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.340+0000 7fe0f57fa700 1 --2- 172.21.15.142:0/976647676 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe0e006dd60 0x7fe0e0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:33.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.340+0000 7fe0f57fa700 1 -- 172.21.15.142:0/976647676 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fe0ec08ee00 con 0x7fe0f8102680 2024-03-29T05:06:33.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.340+0000 7fe0f7fff700 1 --2- 172.21.15.142:0/976647676 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe0e006dd60 0x7fe0e0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:33.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.341+0000 7fe0f7fff700 1 --2- 172.21.15.142:0/976647676 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe0e006dd60 0x7fe0e0070220 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8005950 tx=0x7fe0e80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:33.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.345+0000 7fe0f57fa700 1 -- 172.21.15.142:0/976647676 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe0ec05bdd0 con 0x7fe0f8102680 2024-03-29T05:06:33.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.583+0000 7fe0fe63c700 1 -- 172.21.15.142:0/976647676 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe0f804ea90 con 0x7fe0f8102680 2024-03-29T05:06:33.594 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.584+0000 7fe0f57fa700 1 -- 172.21.15.142:0/976647676 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fe0ec05b960 con 0x7fe0f8102680 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:06:33.595 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:06:33.596 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:06:33.597 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:06:33.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.586+0000 7fe0fe63c700 1 -- 172.21.15.142:0/976647676 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe0e006dd60 msgr2=0x7fe0e0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:33.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.586+0000 7fe0fe63c700 1 --2- 172.21.15.142:0/976647676 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe0e006dd60 0x7fe0e0070220 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8005950 tx=0x7fe0e80058e0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.586+0000 7fe0fe63c700 1 -- 172.21.15.142:0/976647676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0f8102680 msgr2=0x7fe0f819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:33.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.586+0000 7fe0fe63c700 1 --2- 172.21.15.142:0/976647676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0f8102680 0x7fe0f819ca80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec000c00 tx=0x7fe0ec0056a0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.586+0000 7fe0fe63c700 1 -- 172.21.15.142:0/976647676 shutdown_connections 2024-03-29T05:06:33.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.586+0000 7fe0fe63c700 1 --2- 172.21.15.142:0/976647676 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe0e006dd60 0x7fe0e0070220 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.586+0000 7fe0fe63c700 1 --2- 172.21.15.142:0/976647676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f8101d60 0x7fe0f819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.586+0000 7fe0fe63c700 1 --2- 172.21.15.142:0/976647676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0f8102680 0x7fe0f819ca80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.586+0000 7fe0fe63c700 1 -- 172.21.15.142:0/976647676 >> 172.21.15.142:0/976647676 conn(0x7fe0f80fb550 msgr2=0x7fe0f81053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:33.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.586+0000 7fe0fe63c700 1 -- 172.21.15.142:0/976647676 shutdown_connections 2024-03-29T05:06:33.599 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.586+0000 7fe0fe63c700 1 -- 172.21.15.142:0/976647676 wait complete. 2024-03-29T05:06:33.599 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:06:33.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.691+0000 7f5f39357700 1 -- 172.21.15.142:0/2796439442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f34105a40 msgr2=0x7f5f34105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:33.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.691+0000 7f5f39357700 1 --2- 172.21.15.142:0/2796439442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f34105a40 0x7f5f34105e20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009910 tx=0x7f5f24009c20 comp rx=0 tx=0).stop 2024-03-29T05:06:33.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.691+0000 7f5f39357700 1 -- 172.21.15.142:0/2796439442 shutdown_connections 2024-03-29T05:06:33.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.691+0000 7f5f39357700 1 --2- 172.21.15.142:0/2796439442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f340ffa40 0x7f5f340ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.691+0000 7f5f39357700 1 --2- 172.21.15.142:0/2796439442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f34105a40 0x7f5f34105e20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.691+0000 7f5f39357700 1 -- 172.21.15.142:0/2796439442 >> 172.21.15.142:0/2796439442 conn(0x7f5f340fb570 msgr2=0x7f5f340fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:33.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.691+0000 7f5f39357700 1 -- 172.21.15.142:0/2796439442 shutdown_connections 2024-03-29T05:06:33.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.691+0000 7f5f39357700 1 -- 172.21.15.142:0/2796439442 wait complete. 2024-03-29T05:06:33.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.691+0000 7f5f39357700 1 Processor -- start 2024-03-29T05:06:33.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.691+0000 7f5f39357700 1 -- start start 2024-03-29T05:06:33.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f39357700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f340ffa40 0x7f5f341975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:33.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f39357700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f34105a40 0x7f5f34197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:33.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f39357700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f34198120 con 0x7f5f340ffa40 2024-03-29T05:06:33.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f39357700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f34198260 con 0x7f5f34105a40 2024-03-29T05:06:33.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f32ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f340ffa40 0x7f5f341975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:33.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f32ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f340ffa40 0x7f5f341975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37798/0 (socket says 172.21.15.142:37798) 2024-03-29T05:06:33.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f32ffd700 1 -- 172.21.15.142:0/557594966 learned_addr learned my addr 172.21.15.142:0/557594966 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:06:33.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f327fc700 1 --2- 172.21.15.142:0/557594966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f34105a40 0x7f5f34197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:33.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f32ffd700 1 -- 172.21.15.142:0/557594966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f34105a40 msgr2=0x7f5f34197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:33.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f32ffd700 1 --2- 172.21.15.142:0/557594966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f34105a40 0x7f5f34197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f32ffd700 1 -- 172.21.15.142:0/557594966 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f240095f0 con 0x7f5f340ffa40 2024-03-29T05:06:33.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f32ffd700 1 --2- 172.21.15.142:0/557594966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f340ffa40 0x7f5f341975c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006010 tx=0x7f5f2400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:33.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f1bfff700 1 -- 172.21.15.142:0/557594966 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f2401d070 con 0x7f5f340ffa40 2024-03-29T05:06:33.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f1bfff700 1 -- 172.21.15.142:0/557594966 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f24004fb0 con 0x7f5f340ffa40 2024-03-29T05:06:33.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f39357700 1 -- 172.21.15.142:0/557594966 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f341a0630 con 0x7f5f340ffa40 2024-03-29T05:06:33.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f1bfff700 1 -- 172.21.15.142:0/557594966 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f2400f690 con 0x7f5f340ffa40 2024-03-29T05:06:33.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.692+0000 7f5f39357700 1 -- 172.21.15.142:0/557594966 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f341a0a40 con 0x7f5f340ffa40 2024-03-29T05:06:33.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.693+0000 7f5f39357700 1 -- 172.21.15.142:0/557594966 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f34108040 con 0x7f5f340ffa40 2024-03-29T05:06:33.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.694+0000 7f5f1bfff700 1 -- 172.21.15.142:0/557594966 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f5f240228a0 con 0x7f5f340ffa40 2024-03-29T05:06:33.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.694+0000 7f5f1bfff700 1 --2- 172.21.15.142:0/557594966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5f1c06dd60 0x7f5f1c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:33.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.694+0000 7f5f1bfff700 1 -- 172.21.15.142:0/557594966 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f5f2408f520 con 0x7f5f340ffa40 2024-03-29T05:06:33.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.694+0000 7f5f327fc700 1 --2- 172.21.15.142:0/557594966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5f1c06dd60 0x7f5f1c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:33.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.695+0000 7f5f327fc700 1 --2- 172.21.15.142:0/557594966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5f1c06dd60 0x7f5f1c070220 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5f34069550 tx=0x7f5f28005ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:33.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.699+0000 7f5f1bfff700 1 -- 172.21.15.142:0/557594966 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5f24058940 con 0x7f5f340ffa40 2024-03-29T05:06:33.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:33 smithi142 ceph-mon[112475]: pgmap v547: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:33.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:33 smithi142 ceph-mon[112475]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:33.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:33 smithi142 ceph-mon[112475]: from='client.35022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:33.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:33 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1378815318' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:06:33.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:33 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/976647676' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:06:33.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.912+0000 7f5f39357700 1 -- 172.21.15.142:0/557594966 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f3402cd30 con 0x7f5f1c06dd60 2024-03-29T05:06:33.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.913+0000 7f5f1bfff700 1 -- 172.21.15.142:0/557594966 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5f3402cd30 con 0x7f5f1c06dd60 2024-03-29T05:06:33.924 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:06:33.924 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:06:33.924 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:06:33.924 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:06:33.924 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:06:33.924 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:06:33.924 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:06:33.924 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:06:33.924 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:06:33.924 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:06:33.925 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:06:33.925 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:06:33.925 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:06:33.925 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:06:33.925 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:06:33.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.915+0000 7f5f39357700 1 -- 172.21.15.142:0/557594966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5f1c06dd60 msgr2=0x7f5f1c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:33.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.915+0000 7f5f39357700 1 --2- 172.21.15.142:0/557594966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5f1c06dd60 0x7f5f1c070220 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5f34069550 tx=0x7f5f28005ed0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.915+0000 7f5f39357700 1 -- 172.21.15.142:0/557594966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f340ffa40 msgr2=0x7f5f341975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:33.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.915+0000 7f5f39357700 1 --2- 172.21.15.142:0/557594966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f340ffa40 0x7f5f341975c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006010 tx=0x7f5f2400be20 comp rx=0 tx=0).stop 2024-03-29T05:06:33.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.916+0000 7f5f39357700 1 -- 172.21.15.142:0/557594966 shutdown_connections 2024-03-29T05:06:33.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.916+0000 7f5f39357700 1 --2- 172.21.15.142:0/557594966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5f1c06dd60 0x7f5f1c070220 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.926 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.916+0000 7f5f39357700 1 --2- 172.21.15.142:0/557594966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f340ffa40 0x7f5f341975c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.916+0000 7f5f39357700 1 --2- 172.21.15.142:0/557594966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f34105a40 0x7f5f34197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:33.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.916+0000 7f5f39357700 1 -- 172.21.15.142:0/557594966 >> 172.21.15.142:0/557594966 conn(0x7f5f340fb570 msgr2=0x7f5f340fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:33.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.916+0000 7f5f39357700 1 -- 172.21.15.142:0/557594966 shutdown_connections 2024-03-29T05:06:33.927 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:33.916+0000 7f5f39357700 1 -- 172.21.15.142:0/557594966 wait complete. 2024-03-29T05:06:34.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.019+0000 7f3b9273f700 1 -- 172.21.15.142:0/2968144365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b8c105a40 msgr2=0x7f3b8c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:34.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.019+0000 7f3b9273f700 1 --2- 172.21.15.142:0/2968144365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b8c105a40 0x7f3b8c105e20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009910 tx=0x7f3b7c009c20 comp rx=0 tx=0).stop 2024-03-29T05:06:34.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.019+0000 7f3b9273f700 1 -- 172.21.15.142:0/2968144365 shutdown_connections 2024-03-29T05:06:34.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.019+0000 7f3b9273f700 1 --2- 172.21.15.142:0/2968144365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b8c0ffbd0 0x7f3b8c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:34.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.019+0000 7f3b9273f700 1 --2- 172.21.15.142:0/2968144365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b8c105a40 0x7f3b8c105e20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:34.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.019+0000 7f3b9273f700 1 -- 172.21.15.142:0/2968144365 >> 172.21.15.142:0/2968144365 conn(0x7f3b8c0755a0 msgr2=0x7f3b8c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:34.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.019+0000 7f3b9273f700 1 -- 172.21.15.142:0/2968144365 shutdown_connections 2024-03-29T05:06:34.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.019+0000 7f3b9273f700 1 -- 172.21.15.142:0/2968144365 wait complete. 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.020+0000 7f3b9273f700 1 Processor -- start 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.020+0000 7f3b9273f700 1 -- start start 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.020+0000 7f3b9273f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b8c0ffbd0 0x7f3b8c1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.020+0000 7f3b9273f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b8c105a40 0x7f3b8c197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.020+0000 7f3b9273f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c198300 con 0x7f3b8c105a40 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.020+0000 7f3b9273f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c198470 con 0x7f3b8c0ffbd0 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.020+0000 7f3b8b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b8c105a40 0x7f3b8c197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.020+0000 7f3b8b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b8c105a40 0x7f3b8c197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37818/0 (socket says 172.21.15.142:37818) 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.020+0000 7f3b8b7fe700 1 -- 172.21.15.142:0/4086913652 learned_addr learned my addr 172.21.15.142:0/4086913652 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.021+0000 7f3b8bfff700 1 --2- 172.21.15.142:0/4086913652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b8c0ffbd0 0x7f3b8c1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.021+0000 7f3b8b7fe700 1 -- 172.21.15.142:0/4086913652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b8c0ffbd0 msgr2=0x7f3b8c1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.021+0000 7f3b8b7fe700 1 --2- 172.21.15.142:0/4086913652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b8c0ffbd0 0x7f3b8c1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:34.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.021+0000 7f3b8b7fe700 1 -- 172.21.15.142:0/4086913652 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c0095f0 con 0x7f3b8c105a40 2024-03-29T05:06:34.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.021+0000 7f3b8b7fe700 1 --2- 172.21.15.142:0/4086913652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b8c105a40 0x7f3b8c197c20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000b4b0 tx=0x7f3b8000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:34.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.021+0000 7f3b897fa700 1 -- 172.21.15.142:0/4086913652 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b80011800 con 0x7f3b8c105a40 2024-03-29T05:06:34.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.021+0000 7f3b9273f700 1 -- 172.21.15.142:0/4086913652 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8c1a0750 con 0x7f3b8c105a40 2024-03-29T05:06:34.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.021+0000 7f3b897fa700 1 -- 172.21.15.142:0/4086913652 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b80011e20 con 0x7f3b8c105a40 2024-03-29T05:06:34.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.021+0000 7f3b897fa700 1 -- 172.21.15.142:0/4086913652 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b8000f2e0 con 0x7f3b8c105a40 2024-03-29T05:06:34.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.021+0000 7f3b9273f700 1 -- 172.21.15.142:0/4086913652 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8c1a0ca0 con 0x7f3b8c105a40 2024-03-29T05:06:34.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.022+0000 7f3b9273f700 1 -- 172.21.15.142:0/4086913652 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8c108040 con 0x7f3b8c105a40 2024-03-29T05:06:34.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.022+0000 7f3b897fa700 1 -- 172.21.15.142:0/4086913652 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f3b80010380 con 0x7f3b8c105a40 2024-03-29T05:06:34.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.023+0000 7f3b897fa700 1 --2- 172.21.15.142:0/4086913652 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3b7406dd60 0x7f3b74070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:06:34.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.023+0000 7f3b897fa700 1 -- 172.21.15.142:0/4086913652 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f3b8008d890 con 0x7f3b8c105a40 2024-03-29T05:06:34.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.023+0000 7f3b8bfff700 1 --2- 172.21.15.142:0/4086913652 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3b7406dd60 0x7f3b74070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:06:34.034 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.023+0000 7f3b8bfff700 1 --2- 172.21.15.142:0/4086913652 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3b7406dd60 0x7f3b74070220 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00b5c0 tx=0x7f3b7c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:06:34.038 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.028+0000 7f3b897fa700 1 -- 172.21.15.142:0/4086913652 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3b80056cb0 con 0x7f3b8c105a40 2024-03-29T05:06:34.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:33 smithi152 ceph-mon[88186]: pgmap v547: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:34.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:33 smithi152 ceph-mon[88186]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:34.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:33 smithi152 ceph-mon[88186]: from='client.35022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:34.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:33 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1378815318' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:06:34.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:33 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/976647676' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:06:34.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.304+0000 7f3b9273f700 1 -- 172.21.15.142:0/4086913652 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3b8c107a20 con 0x7f3b8c105a40 2024-03-29T05:06:34.314 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.304+0000 7f3b897fa700 1 -- 172.21.15.142:0/4086913652 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f3b8005a370 con 0x7f3b8c105a40 2024-03-29T05:06:34.315 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:06:34.315 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:06:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:06:34.315 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:06:34.315 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:06:34.317 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.307+0000 7f3b9273f700 1 -- 172.21.15.142:0/4086913652 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3b7406dd60 msgr2=0x7f3b74070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:34.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.307+0000 7f3b9273f700 1 --2- 172.21.15.142:0/4086913652 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3b7406dd60 0x7f3b74070220 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00b5c0 tx=0x7f3b7c01a040 comp rx=0 tx=0).stop 2024-03-29T05:06:34.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.307+0000 7f3b9273f700 1 -- 172.21.15.142:0/4086913652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b8c105a40 msgr2=0x7f3b8c197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:06:34.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.307+0000 7f3b9273f700 1 --2- 172.21.15.142:0/4086913652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b8c105a40 0x7f3b8c197c20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000b4b0 tx=0x7f3b8000b870 comp rx=0 tx=0).stop 2024-03-29T05:06:34.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.307+0000 7f3b9273f700 1 -- 172.21.15.142:0/4086913652 shutdown_connections 2024-03-29T05:06:34.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.307+0000 7f3b9273f700 1 --2- 172.21.15.142:0/4086913652 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f3b7406dd60 0x7f3b74070220 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:34.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.307+0000 7f3b9273f700 1 --2- 172.21.15.142:0/4086913652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b8c0ffbd0 0x7f3b8c1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:34.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.307+0000 7f3b9273f700 1 --2- 172.21.15.142:0/4086913652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b8c105a40 0x7f3b8c197c20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:06:34.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.307+0000 7f3b9273f700 1 -- 172.21.15.142:0/4086913652 >> 172.21.15.142:0/4086913652 conn(0x7f3b8c0755a0 msgr2=0x7f3b8c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:06:34.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.307+0000 7f3b9273f700 1 -- 172.21.15.142:0/4086913652 shutdown_connections 2024-03-29T05:06:34.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:06:34.307+0000 7f3b9273f700 1 -- 172.21.15.142:0/4086913652 wait complete. 2024-03-29T05:06:35.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:34 smithi152 ceph-mon[88186]: from='client.35026 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:35.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:34 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/4086913652' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:06:35.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:34 smithi142 ceph-mon[112475]: from='client.35026 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:35.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:34 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/4086913652' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:06:36.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:35 smithi152 ceph-mon[88186]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:36.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:35 smithi152 ceph-mon[88186]: pgmap v548: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:36.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:35 smithi142 ceph-mon[112475]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:06:36.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:35 smithi142 ceph-mon[112475]: pgmap v548: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:38.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:37 smithi152 ceph-mon[88186]: pgmap v549: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:38.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:37 smithi142 ceph-mon[112475]: pgmap v549: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:40.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:39 smithi152 ceph-mon[88186]: pgmap v550: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:40.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:39 smithi142 ceph-mon[112475]: pgmap v550: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:42.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:41 smithi152 ceph-mon[88186]: pgmap v551: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:42.134 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:41 smithi142 ceph-mon[112475]: pgmap v551: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:44.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:43 smithi152 ceph-mon[88186]: pgmap v552: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:44.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:43 smithi142 ceph-mon[112475]: pgmap v552: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:45.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:06:45.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:06:46.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:45 smithi152 ceph-mon[88186]: pgmap v553: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:46.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:45 smithi142 ceph-mon[112475]: pgmap v553: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:47.881 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:47 smithi142 ceph-mon[112475]: pgmap v554: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:48.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:47 smithi152 ceph-mon[88186]: pgmap v554: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:49.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:48 smithi142 ceph-mon[112475]: pgmap v555: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:49.300 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:48 smithi152 ceph-mon[88186]: pgmap v555: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:51.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:51 smithi142 ceph-mon[112475]: pgmap v556: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:51.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:51 smithi152 ceph-mon[88186]: pgmap v556: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:53.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:53 smithi142 ceph-mon[112475]: pgmap v557: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:53.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:53 smithi152 ceph-mon[88186]: pgmap v557: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:55.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:55 smithi142 ceph-mon[112475]: pgmap v558: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:55.390 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:55 smithi152 ceph-mon[88186]: pgmap v558: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:57.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:57 smithi142 ceph-mon[112475]: pgmap v559: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:57.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:57 smithi152 ceph-mon[88186]: pgmap v559: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:59.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:06:59 smithi142 ceph-mon[112475]: pgmap v560: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:06:59.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:06:59 smithi152 ceph-mon[88186]: pgmap v560: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:00.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:07:00.443 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:07:01.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:01 smithi142 ceph-mon[112475]: pgmap v561: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:01.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:01 smithi152 ceph-mon[88186]: pgmap v561: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:03.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:03 smithi142 ceph-mon[112475]: pgmap v562: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:03.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:03 smithi152 ceph-mon[88186]: pgmap v562: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:04.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.425+0000 7f6583a8b700 1 -- 172.21.15.142:0/1758275330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f657c0ffa40 msgr2=0x7f657c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:04.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.425+0000 7f6583a8b700 1 --2- 172.21.15.142:0/1758275330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f657c0ffa40 0x7f657c0ffec0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f656c009910 tx=0x7f656c009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:04.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.425+0000 7f6583a8b700 1 -- 172.21.15.142:0/1758275330 shutdown_connections 2024-03-29T05:07:04.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.425+0000 7f6583a8b700 1 --2- 172.21.15.142:0/1758275330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f657c0ffa40 0x7f657c0ffec0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.425+0000 7f6583a8b700 1 --2- 172.21.15.142:0/1758275330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f657c105a40 0x7f657c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.425+0000 7f6583a8b700 1 -- 172.21.15.142:0/1758275330 >> 172.21.15.142:0/1758275330 conn(0x7f657c0fb550 msgr2=0x7f657c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:04.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.425+0000 7f6583a8b700 1 -- 172.21.15.142:0/1758275330 shutdown_connections 2024-03-29T05:07:04.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.425+0000 7f6583a8b700 1 -- 172.21.15.142:0/1758275330 wait complete. 2024-03-29T05:07:04.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6583a8b700 1 Processor -- start 2024-03-29T05:07:04.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6583a8b700 1 -- start start 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6583a8b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f657c0ffa40 0x7f657c0fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6583a8b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f657c105a40 0x7f657c0fc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6583a8b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0fc960 con 0x7f657c0ffa40 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6583a8b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0fcaa0 con 0x7f657c105a40 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6581827700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f657c0ffa40 0x7f657c0fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6581827700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f657c0ffa40 0x7f657c0fbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54434/0 (socket says 172.21.15.142:54434) 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6581827700 1 -- 172.21.15.142:0/814250741 learned_addr learned my addr 172.21.15.142:0/814250741 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6581026700 1 --2- 172.21.15.142:0/814250741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f657c105a40 0x7f657c0fc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6581827700 1 -- 172.21.15.142:0/814250741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f657c105a40 msgr2=0x7f657c0fc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6581827700 1 --2- 172.21.15.142:0/814250741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f657c105a40 0x7f657c0fc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.426+0000 7f6581827700 1 -- 172.21.15.142:0/814250741 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c0095f0 con 0x7f657c0ffa40 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.427+0000 7f6581827700 1 --2- 172.21.15.142:0/814250741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f657c0ffa40 0x7f657c0fbee0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f657800c930 tx=0x7f657800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.427+0000 7f6572ffd700 1 -- 172.21.15.142:0/814250741 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6578007ad0 con 0x7f657c0ffa40 2024-03-29T05:07:04.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.427+0000 7f6572ffd700 1 -- 172.21.15.142:0/814250741 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6578007c30 con 0x7f657c0ffa40 2024-03-29T05:07:04.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.427+0000 7f6583a8b700 1 -- 172.21.15.142:0/814250741 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c1a2920 con 0x7f657c0ffa40 2024-03-29T05:07:04.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.427+0000 7f6572ffd700 1 -- 172.21.15.142:0/814250741 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6578018610 con 0x7f657c0ffa40 2024-03-29T05:07:04.439 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.427+0000 7f6583a8b700 1 -- 172.21.15.142:0/814250741 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c1a2c80 con 0x7f657c0ffa40 2024-03-29T05:07:04.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.428+0000 7f6583a8b700 1 -- 172.21.15.142:0/814250741 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c108fe0 con 0x7f657c0ffa40 2024-03-29T05:07:04.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.428+0000 7f6572ffd700 1 -- 172.21.15.142:0/814250741 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f657800f450 con 0x7f657c0ffa40 2024-03-29T05:07:04.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.429+0000 7f6572ffd700 1 --2- 172.21.15.142:0/814250741 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f656806dd60 0x7f6568070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:04.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.429+0000 7f6572ffd700 1 -- 172.21.15.142:0/814250741 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f657808dda0 con 0x7f657c0ffa40 2024-03-29T05:07:04.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.429+0000 7f6581026700 1 --2- 172.21.15.142:0/814250741 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f656806dd60 0x7f6568070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:04.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.429+0000 7f6581026700 1 --2- 172.21.15.142:0/814250741 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f656806dd60 0x7f6568070220 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f657c0fd6d0 tx=0x7f656c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:04.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.434+0000 7f6572ffd700 1 -- 172.21.15.142:0/814250741 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6578057240 con 0x7f657c0ffa40 2024-03-29T05:07:04.654 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.644+0000 7f6583a8b700 1 -- 172.21.15.142:0/814250741 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f657c02cd80 con 0x7f656806dd60 2024-03-29T05:07:04.655 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.646+0000 7f6572ffd700 1 -- 172.21.15.142:0/814250741 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f657c02cd80 con 0x7f656806dd60 2024-03-29T05:07:04.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.648+0000 7f6583a8b700 1 -- 172.21.15.142:0/814250741 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f656806dd60 msgr2=0x7f6568070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:04.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.648+0000 7f6583a8b700 1 --2- 172.21.15.142:0/814250741 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f656806dd60 0x7f6568070220 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f657c0fd6d0 tx=0x7f656c01a040 comp rx=0 tx=0).stop 2024-03-29T05:07:04.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.648+0000 7f6583a8b700 1 -- 172.21.15.142:0/814250741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f657c0ffa40 msgr2=0x7f657c0fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:04.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.648+0000 7f6583a8b700 1 --2- 172.21.15.142:0/814250741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f657c0ffa40 0x7f657c0fbee0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f657800c930 tx=0x7f657800ccf0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.648+0000 7f6583a8b700 1 -- 172.21.15.142:0/814250741 shutdown_connections 2024-03-29T05:07:04.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.648+0000 7f6583a8b700 1 --2- 172.21.15.142:0/814250741 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f656806dd60 0x7f6568070220 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.648+0000 7f6583a8b700 1 --2- 172.21.15.142:0/814250741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f657c0ffa40 0x7f657c0fbee0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.648+0000 7f6583a8b700 1 --2- 172.21.15.142:0/814250741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f657c105a40 0x7f657c0fc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.648+0000 7f6583a8b700 1 -- 172.21.15.142:0/814250741 >> 172.21.15.142:0/814250741 conn(0x7f657c0fb550 msgr2=0x7f657c069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:04.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.648+0000 7f6583a8b700 1 -- 172.21.15.142:0/814250741 shutdown_connections 2024-03-29T05:07:04.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.648+0000 7f6583a8b700 1 -- 172.21.15.142:0/814250741 wait complete. 2024-03-29T05:07:04.668 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:07:04.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.752+0000 7fcf44fd3700 1 -- 172.21.15.142:0/1413145652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf400ffa40 msgr2=0x7fcf400ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:04.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.752+0000 7fcf44fd3700 1 --2- 172.21.15.142:0/1413145652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf400ffa40 0x7fcf400ffec0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009910 tx=0x7fcf34009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:04.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.752+0000 7fcf44fd3700 1 -- 172.21.15.142:0/1413145652 shutdown_connections 2024-03-29T05:07:04.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.752+0000 7fcf44fd3700 1 --2- 172.21.15.142:0/1413145652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf400ffa40 0x7fcf400ffec0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.752+0000 7fcf44fd3700 1 --2- 172.21.15.142:0/1413145652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf40105a40 0x7fcf40105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.752+0000 7fcf44fd3700 1 -- 172.21.15.142:0/1413145652 >> 172.21.15.142:0/1413145652 conn(0x7fcf400fb550 msgr2=0x7fcf400fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:04.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.752+0000 7fcf44fd3700 1 -- 172.21.15.142:0/1413145652 shutdown_connections 2024-03-29T05:07:04.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.753+0000 7fcf44fd3700 1 -- 172.21.15.142:0/1413145652 wait complete. 2024-03-29T05:07:04.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.753+0000 7fcf44fd3700 1 Processor -- start 2024-03-29T05:07:04.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.753+0000 7fcf44fd3700 1 -- start start 2024-03-29T05:07:04.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf44fd3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf400ffa40 0x7fcf401975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:04.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf44fd3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf40105a40 0x7fcf40197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:04.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf44fd3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf40198110 con 0x7fcf400ffa40 2024-03-29T05:07:04.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf44fd3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf40198250 con 0x7fcf40105a40 2024-03-29T05:07:04.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf3e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf400ffa40 0x7fcf401975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:04.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf3e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf400ffa40 0x7fcf401975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54456/0 (socket says 172.21.15.142:54456) 2024-03-29T05:07:04.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf3e59c700 1 -- 172.21.15.142:0/2430773419 learned_addr learned my addr 172.21.15.142:0/2430773419 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:04.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf3dd9b700 1 --2- 172.21.15.142:0/2430773419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf40105a40 0x7fcf40197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:04.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf3e59c700 1 -- 172.21.15.142:0/2430773419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf40105a40 msgr2=0x7fcf40197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:04.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf3e59c700 1 --2- 172.21.15.142:0/2430773419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf40105a40 0x7fcf40197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf3e59c700 1 -- 172.21.15.142:0/2430773419 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf340095f0 con 0x7fcf400ffa40 2024-03-29T05:07:04.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf3e59c700 1 --2- 172.21.15.142:0/2430773419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf400ffa40 0x7fcf401975b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800ef10 tx=0x7fcf3800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:04.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf2f7fe700 1 -- 172.21.15.142:0/2430773419 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf3800c9f0 con 0x7fcf400ffa40 2024-03-29T05:07:04.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.754+0000 7fcf44fd3700 1 -- 172.21.15.142:0/2430773419 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf401a0620 con 0x7fcf400ffa40 2024-03-29T05:07:04.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.755+0000 7fcf44fd3700 1 -- 172.21.15.142:0/2430773419 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf401a0a90 con 0x7fcf400ffa40 2024-03-29T05:07:04.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.755+0000 7fcf2f7fe700 1 -- 172.21.15.142:0/2430773419 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf3800f040 con 0x7fcf400ffa40 2024-03-29T05:07:04.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.755+0000 7fcf2f7fe700 1 -- 172.21.15.142:0/2430773419 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf380146b0 con 0x7fcf400ffa40 2024-03-29T05:07:04.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.755+0000 7fcf44fd3700 1 -- 172.21.15.142:0/2430773419 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf20005320 con 0x7fcf400ffa40 2024-03-29T05:07:04.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.756+0000 7fcf2f7fe700 1 -- 172.21.15.142:0/2430773419 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fcf3800cb50 con 0x7fcf400ffa40 2024-03-29T05:07:04.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.756+0000 7fcf2f7fe700 1 --2- 172.21.15.142:0/2430773419 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf2806dd60 0x7fcf28070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:04.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.757+0000 7fcf2f7fe700 1 -- 172.21.15.142:0/2430773419 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fcf3808dfc0 con 0x7fcf400ffa40 2024-03-29T05:07:04.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.757+0000 7fcf3dd9b700 1 --2- 172.21.15.142:0/2430773419 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf2806dd60 0x7fcf28070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:04.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.757+0000 7fcf3dd9b700 1 --2- 172.21.15.142:0/2430773419 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf2806dd60 0x7fcf28070220 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcf40105330 tx=0x7fcf34005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:04.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.761+0000 7fcf2f7fe700 1 -- 172.21.15.142:0/2430773419 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcf380573e0 con 0x7fcf400ffa40 2024-03-29T05:07:04.983 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.973+0000 7fcf44fd3700 1 -- 172.21.15.142:0/2430773419 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf20000bf0 con 0x7fcf2806dd60 2024-03-29T05:07:04.984 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.974+0000 7fcf2f7fe700 1 -- 172.21.15.142:0/2430773419 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcf20000bf0 con 0x7fcf2806dd60 2024-03-29T05:07:04.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.976+0000 7fcf44fd3700 1 -- 172.21.15.142:0/2430773419 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf2806dd60 msgr2=0x7fcf28070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:04.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.976+0000 7fcf44fd3700 1 --2- 172.21.15.142:0/2430773419 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf2806dd60 0x7fcf28070220 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcf40105330 tx=0x7fcf34005940 comp rx=0 tx=0).stop 2024-03-29T05:07:04.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.976+0000 7fcf44fd3700 1 -- 172.21.15.142:0/2430773419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf400ffa40 msgr2=0x7fcf401975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:04.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.976+0000 7fcf44fd3700 1 --2- 172.21.15.142:0/2430773419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf400ffa40 0x7fcf401975b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800ef10 tx=0x7fcf3800c3a0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.976+0000 7fcf44fd3700 1 -- 172.21.15.142:0/2430773419 shutdown_connections 2024-03-29T05:07:04.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.976+0000 7fcf44fd3700 1 --2- 172.21.15.142:0/2430773419 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf2806dd60 0x7fcf28070220 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.976+0000 7fcf44fd3700 1 --2- 172.21.15.142:0/2430773419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf400ffa40 0x7fcf401975b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.976+0000 7fcf44fd3700 1 --2- 172.21.15.142:0/2430773419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf40105a40 0x7fcf40197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:04.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.976+0000 7fcf44fd3700 1 -- 172.21.15.142:0/2430773419 >> 172.21.15.142:0/2430773419 conn(0x7fcf400fb550 msgr2=0x7fcf400fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:04.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.976+0000 7fcf44fd3700 1 -- 172.21.15.142:0/2430773419 shutdown_connections 2024-03-29T05:07:04.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:04.976+0000 7fcf44fd3700 1 -- 172.21.15.142:0/2430773419 wait complete. 2024-03-29T05:07:05.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.080+0000 7fa5a246c700 1 -- 172.21.15.142:0/1969919076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa59c0ffa40 msgr2=0x7fa59c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:05.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.080+0000 7fa5a246c700 1 --2- 172.21.15.142:0/1969919076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa59c0ffa40 0x7fa59c0ffec0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa590009910 tx=0x7fa590009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:05.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.080+0000 7fa5a246c700 1 -- 172.21.15.142:0/1969919076 shutdown_connections 2024-03-29T05:07:05.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.080+0000 7fa5a246c700 1 --2- 172.21.15.142:0/1969919076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa59c0ffa40 0x7fa59c0ffec0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.080+0000 7fa5a246c700 1 --2- 172.21.15.142:0/1969919076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa59c105a40 0x7fa59c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.080+0000 7fa5a246c700 1 -- 172.21.15.142:0/1969919076 >> 172.21.15.142:0/1969919076 conn(0x7fa59c0fb550 msgr2=0x7fa59c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:05.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.080+0000 7fa5a246c700 1 -- 172.21.15.142:0/1969919076 shutdown_connections 2024-03-29T05:07:05.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.080+0000 7fa5a246c700 1 -- 172.21.15.142:0/1969919076 wait complete. 2024-03-29T05:07:05.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.081+0000 7fa5a246c700 1 Processor -- start 2024-03-29T05:07:05.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.081+0000 7fa5a246c700 1 -- start start 2024-03-29T05:07:05.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.081+0000 7fa5a246c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa59c0ffa40 0x7fa59c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:05.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.081+0000 7fa5a246c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa59c105a40 0x7fa59c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:05.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.081+0000 7fa5a246c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c198110 con 0x7fa59c0ffa40 2024-03-29T05:07:05.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.081+0000 7fa5a246c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c198250 con 0x7fa59c105a40 2024-03-29T05:07:05.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa59bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa59c0ffa40 0x7fa59c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:05.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa59bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa59c0ffa40 0x7fa59c1975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54468/0 (socket says 172.21.15.142:54468) 2024-03-29T05:07:05.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa59bfff700 1 -- 172.21.15.142:0/3308515615 learned_addr learned my addr 172.21.15.142:0/3308515615 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:05.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa59b7fe700 1 --2- 172.21.15.142:0/3308515615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa59c105a40 0x7fa59c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:05.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa59bfff700 1 -- 172.21.15.142:0/3308515615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa59c105a40 msgr2=0x7fa59c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:05.093 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa59bfff700 1 --2- 172.21.15.142:0/3308515615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa59c105a40 0x7fa59c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa59bfff700 1 -- 172.21.15.142:0/3308515615 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5900095f0 con 0x7fa59c0ffa40 2024-03-29T05:07:05.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa59bfff700 1 --2- 172.21.15.142:0/3308515615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa59c0ffa40 0x7fa59c1975b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00d8d0 tx=0x7fa58c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:05.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa5997fa700 1 -- 172.21.15.142:0/3308515615 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa58c0098a0 con 0x7fa59c0ffa40 2024-03-29T05:07:05.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa5997fa700 1 -- 172.21.15.142:0/3308515615 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa58c010460 con 0x7fa59c0ffa40 2024-03-29T05:07:05.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa5a246c700 1 -- 172.21.15.142:0/3308515615 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c1a0620 con 0x7fa59c0ffa40 2024-03-29T05:07:05.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa5997fa700 1 -- 172.21.15.142:0/3308515615 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa58c00f5f0 con 0x7fa59c0ffa40 2024-03-29T05:07:05.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.082+0000 7fa5a246c700 1 -- 172.21.15.142:0/3308515615 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c1a0ae0 con 0x7fa59c0ffa40 2024-03-29T05:07:05.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.083+0000 7fa5a246c700 1 -- 172.21.15.142:0/3308515615 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59c107fd0 con 0x7fa59c0ffa40 2024-03-29T05:07:05.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.084+0000 7fa5997fa700 1 -- 172.21.15.142:0/3308515615 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fa58c009a00 con 0x7fa59c0ffa40 2024-03-29T05:07:05.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.084+0000 7fa5997fa700 1 --2- 172.21.15.142:0/3308515615 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa58406dd60 0x7fa584070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:05.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.084+0000 7fa5997fa700 1 -- 172.21.15.142:0/3308515615 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa58c08da90 con 0x7fa59c0ffa40 2024-03-29T05:07:05.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.084+0000 7fa59b7fe700 1 --2- 172.21.15.142:0/3308515615 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa58406dd60 0x7fa584070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:05.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.085+0000 7fa59b7fe700 1 --2- 172.21.15.142:0/3308515615 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa58406dd60 0x7fa584070220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa59c069660 tx=0x7fa59001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:05.098 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.089+0000 7fa5997fa700 1 -- 172.21.15.142:0/3308515615 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa58c056f30 con 0x7fa59c0ffa40 2024-03-29T05:07:05.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.291+0000 7fa5a246c700 1 -- 172.21.15.142:0/3308515615 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa59c02cd80 con 0x7fa58406dd60 2024-03-29T05:07:05.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.302+0000 7fa5997fa700 1 -- 172.21.15.142:0/3308515615 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa59c02cd80 con 0x7fa58406dd60 2024-03-29T05:07:05.312 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:07:05.312 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (18m) 12m ago 30m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:07:05.312 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (30m) 12m ago 30m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:07:05.312 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (29m) 11m ago 29m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:07:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (16m) 12m ago 30m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:07:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (15m) 11m ago 29m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:07:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (18m) 12m ago 30m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:07:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (12m) 12m ago 24m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:07:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (12m) 12m ago 24m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:07:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (11m) 11m ago 24m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:07:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (11m) 11m ago 24m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:07:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (20m) 12m ago 31m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:07:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (19m) 11m ago 29m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:07:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (17m) 12m ago 31m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:07:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (16m) 11m ago 29m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:07:05.313 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (19m) 12m ago 30m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:07:05.314 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (19m) 11m ago 29m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:07:05.314 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (15m) 12m ago 28m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:07:05.314 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (15m) 12m ago 28m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:07:05.314 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (14m) 12m ago 27m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:07:05.314 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (13m) 11m ago 27m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:07:05.314 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (13m) 11m ago 26m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:07:05.314 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (12m) 11m ago 26m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:07:05.314 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (19m) 12m ago 29m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:07:05.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.304+0000 7fa5a246c700 1 -- 172.21.15.142:0/3308515615 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa58406dd60 msgr2=0x7fa584070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:05.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.304+0000 7fa5a246c700 1 --2- 172.21.15.142:0/3308515615 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa58406dd60 0x7fa584070220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa59c069660 tx=0x7fa59001a040 comp rx=0 tx=0).stop 2024-03-29T05:07:05.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.304+0000 7fa5a246c700 1 -- 172.21.15.142:0/3308515615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa59c0ffa40 msgr2=0x7fa59c1975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:05.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.304+0000 7fa5a246c700 1 --2- 172.21.15.142:0/3308515615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa59c0ffa40 0x7fa59c1975b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00d8d0 tx=0x7fa58c00dbe0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.304+0000 7fa5a246c700 1 -- 172.21.15.142:0/3308515615 shutdown_connections 2024-03-29T05:07:05.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.304+0000 7fa5a246c700 1 --2- 172.21.15.142:0/3308515615 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fa58406dd60 0x7fa584070220 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.304+0000 7fa5a246c700 1 --2- 172.21.15.142:0/3308515615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa59c0ffa40 0x7fa59c1975b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.304+0000 7fa5a246c700 1 --2- 172.21.15.142:0/3308515615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa59c105a40 0x7fa59c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.304+0000 7fa5a246c700 1 -- 172.21.15.142:0/3308515615 >> 172.21.15.142:0/3308515615 conn(0x7fa59c0fb550 msgr2=0x7fa59c0fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:05.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.304+0000 7fa5a246c700 1 -- 172.21.15.142:0/3308515615 shutdown_connections 2024-03-29T05:07:05.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.305+0000 7fa5a246c700 1 -- 172.21.15.142:0/3308515615 wait complete. 2024-03-29T05:07:05.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:05 smithi142 ceph-mon[112475]: pgmap v563: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:05.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:05 smithi142 ceph-mon[112475]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:05.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.407+0000 7f6f61790700 1 -- 172.21.15.142:0/2539701629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c100dc0 msgr2=0x7f6f5c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:05.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.407+0000 7f6f61790700 1 --2- 172.21.15.142:0/2539701629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c100dc0 0x7f6f5c1011a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009910 tx=0x7f6f4c009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:05.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.407+0000 7f6f61790700 1 -- 172.21.15.142:0/2539701629 shutdown_connections 2024-03-29T05:07:05.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.407+0000 7f6f61790700 1 --2- 172.21.15.142:0/2539701629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f5c101770 0x7f6f5c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.407+0000 7f6f61790700 1 --2- 172.21.15.142:0/2539701629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c100dc0 0x7f6f5c1011a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.407+0000 7f6f61790700 1 -- 172.21.15.142:0/2539701629 >> 172.21.15.142:0/2539701629 conn(0x7f6f5c0fc650 msgr2=0x7f6f5c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:05.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.407+0000 7f6f61790700 1 -- 172.21.15.142:0/2539701629 shutdown_connections 2024-03-29T05:07:05.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.407+0000 7f6f61790700 1 -- 172.21.15.142:0/2539701629 wait complete. 2024-03-29T05:07:05.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.408+0000 7f6f61790700 1 Processor -- start 2024-03-29T05:07:05.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.408+0000 7f6f61790700 1 -- start start 2024-03-29T05:07:05.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.408+0000 7f6f61790700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f5c100dc0 0x7f6f5c19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.408+0000 7f6f61790700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c101770 0x7f6f5c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.408+0000 7f6f61790700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c19d210 con 0x7f6f5c101770 2024-03-29T05:07:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.408+0000 7f6f61790700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c1966c0 con 0x7f6f5c100dc0 2024-03-29T05:07:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f5a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c101770 0x7f6f5c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f5a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c101770 0x7f6f5c19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54488/0 (socket says 172.21.15.142:54488) 2024-03-29T05:07:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f5a7fc700 1 -- 172.21.15.142:0/1153010806 learned_addr learned my addr 172.21.15.142:0/1153010806 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f5affd700 1 --2- 172.21.15.142:0/1153010806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f5c100dc0 0x7f6f5c19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f5a7fc700 1 -- 172.21.15.142:0/1153010806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f5c100dc0 msgr2=0x7f6f5c19c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f5a7fc700 1 --2- 172.21.15.142:0/1153010806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f5c100dc0 0x7f6f5c19c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f5a7fc700 1 -- 172.21.15.142:0/1153010806 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c0095f0 con 0x7f6f5c101770 2024-03-29T05:07:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f5a7fc700 1 --2- 172.21.15.142:0/1153010806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c101770 0x7f6f5c19cb80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000b480 tx=0x7f6f5000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f43fff700 1 -- 172.21.15.142:0/1153010806 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f50011800 con 0x7f6f5c101770 2024-03-29T05:07:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f43fff700 1 -- 172.21.15.142:0/1153010806 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f50011e20 con 0x7f6f5c101770 2024-03-29T05:07:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f61790700 1 -- 172.21.15.142:0/1153010806 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5c1969a0 con 0x7f6f5c101770 2024-03-29T05:07:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f43fff700 1 -- 172.21.15.142:0/1153010806 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f50010ea0 con 0x7f6f5c101770 2024-03-29T05:07:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.409+0000 7f6f61790700 1 -- 172.21.15.142:0/1153010806 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5c196ef0 con 0x7f6f5c101770 2024-03-29T05:07:05.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.410+0000 7f6f61790700 1 -- 172.21.15.142:0/1153010806 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5c109160 con 0x7f6f5c101770 2024-03-29T05:07:05.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.411+0000 7f6f43fff700 1 -- 172.21.15.142:0/1153010806 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6f50011960 con 0x7f6f5c101770 2024-03-29T05:07:05.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.411+0000 7f6f43fff700 1 --2- 172.21.15.142:0/1153010806 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6f44072170 0x7f6f44074630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:05.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.411+0000 7f6f43fff700 1 -- 172.21.15.142:0/1153010806 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6f5008d6b0 con 0x7f6f5c101770 2024-03-29T05:07:05.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.411+0000 7f6f5affd700 1 --2- 172.21.15.142:0/1153010806 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6f44072170 0x7f6f44074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:05.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.412+0000 7f6f5affd700 1 --2- 172.21.15.142:0/1153010806 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6f44072170 0x7f6f44074630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c006010 tx=0x7f6f4c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:05.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.417+0000 7f6f43fff700 1 -- 172.21.15.142:0/1153010806 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6f5005a680 con 0x7f6f5c101770 2024-03-29T05:07:05.496 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:05 smithi152 ceph-mon[88186]: pgmap v563: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:05.497 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:05 smithi152 ceph-mon[88186]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:05.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.690+0000 7f6f61790700 1 -- 172.21.15.142:0/1153010806 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6f5c04ea90 con 0x7f6f5c101770 2024-03-29T05:07:05.700 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.690+0000 7f6f43fff700 1 -- 172.21.15.142:0/1153010806 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6f5005a210 con 0x7f6f5c101770 2024-03-29T05:07:05.701 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:07:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:07:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:07:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:07:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:07:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:07:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:07:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:07:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:07:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:07:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:07:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:07:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:07:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:07:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:07:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:07:05.702 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:07:05.703 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.693+0000 7f6f61790700 1 -- 172.21.15.142:0/1153010806 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6f44072170 msgr2=0x7f6f44074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:05.703 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.693+0000 7f6f61790700 1 --2- 172.21.15.142:0/1153010806 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6f44072170 0x7f6f44074630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c006010 tx=0x7f6f4c01a040 comp rx=0 tx=0).stop 2024-03-29T05:07:05.703 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.693+0000 7f6f61790700 1 -- 172.21.15.142:0/1153010806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c101770 msgr2=0x7f6f5c19cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:05.703 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.693+0000 7f6f61790700 1 --2- 172.21.15.142:0/1153010806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c101770 0x7f6f5c19cb80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000b480 tx=0x7f6f5000b790 comp rx=0 tx=0).stop 2024-03-29T05:07:05.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.693+0000 7f6f61790700 1 -- 172.21.15.142:0/1153010806 shutdown_connections 2024-03-29T05:07:05.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.693+0000 7f6f61790700 1 --2- 172.21.15.142:0/1153010806 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6f44072170 0x7f6f44074630 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.693+0000 7f6f61790700 1 --2- 172.21.15.142:0/1153010806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f5c100dc0 0x7f6f5c19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.693+0000 7f6f61790700 1 --2- 172.21.15.142:0/1153010806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f5c101770 0x7f6f5c19cb80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.693+0000 7f6f61790700 1 -- 172.21.15.142:0/1153010806 >> 172.21.15.142:0/1153010806 conn(0x7f6f5c0fc650 msgr2=0x7f6f5c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:05.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.693+0000 7f6f61790700 1 -- 172.21.15.142:0/1153010806 shutdown_connections 2024-03-29T05:07:05.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.693+0000 7f6f61790700 1 -- 172.21.15.142:0/1153010806 wait complete. 2024-03-29T05:07:05.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.798+0000 7f6a0041e700 1 -- 172.21.15.142:0/1683870586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f81026c0 msgr2=0x7f69f810abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:05.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.798+0000 7f6a0041e700 1 --2- 172.21.15.142:0/1683870586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f81026c0 0x7f69f810abb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009910 tx=0x7f69e8009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:05.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.798+0000 7f6a0041e700 1 -- 172.21.15.142:0/1683870586 shutdown_connections 2024-03-29T05:07:05.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.798+0000 7f6a0041e700 1 --2- 172.21.15.142:0/1683870586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f81026c0 0x7f69f810abb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.798+0000 7f6a0041e700 1 --2- 172.21.15.142:0/1683870586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69f8101da0 0x7f69f8102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.798+0000 7f6a0041e700 1 -- 172.21.15.142:0/1683870586 >> 172.21.15.142:0/1683870586 conn(0x7f69f80fb550 msgr2=0x7f69f80fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:05.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.798+0000 7f6a0041e700 1 -- 172.21.15.142:0/1683870586 shutdown_connections 2024-03-29T05:07:05.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.798+0000 7f6a0041e700 1 -- 172.21.15.142:0/1683870586 wait complete. 2024-03-29T05:07:05.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.798+0000 7f6a0041e700 1 Processor -- start 2024-03-29T05:07:05.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.798+0000 7f6a0041e700 1 -- start start 2024-03-29T05:07:05.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f6a0041e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69f8101da0 0x7f69f80ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:05.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f6a0041e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f81026c0 0x7f69f80fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:05.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f6a0041e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8100480 con 0x7f69f81026c0 2024-03-29T05:07:05.810 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f6a0041e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f81005c0 con 0x7f69f8101da0 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f69fd9b9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f81026c0 0x7f69f80fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f69fd9b9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f81026c0 0x7f69f80fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54500/0 (socket says 172.21.15.142:54500) 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f69fd9b9700 1 -- 172.21.15.142:0/864262182 learned_addr learned my addr 172.21.15.142:0/864262182 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f69fe1ba700 1 --2- 172.21.15.142:0/864262182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69f8101da0 0x7f69f80ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f69fd9b9700 1 -- 172.21.15.142:0/864262182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69f8101da0 msgr2=0x7f69f80ffa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f69fd9b9700 1 --2- 172.21.15.142:0/864262182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69f8101da0 0x7f69f80ffa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f69fd9b9700 1 -- 172.21.15.142:0/864262182 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e80095f0 con 0x7f69f81026c0 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f69fe1ba700 1 --2- 172.21.15.142:0/864262182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69f8101da0 0x7f69f80ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f69fd9b9700 1 --2- 172.21.15.142:0/864262182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f81026c0 0x7f69f80fff40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f69e800bda0 tx=0x7f69e800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f69ef7fe700 1 -- 172.21.15.142:0/864262182 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69e801d070 con 0x7f69f81026c0 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f69ef7fe700 1 -- 172.21.15.142:0/864262182 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69e8004fb0 con 0x7f69f81026c0 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f6a0041e700 1 -- 172.21.15.142:0/864262182 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f8100840 con 0x7f69f81026c0 2024-03-29T05:07:05.811 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f69ef7fe700 1 -- 172.21.15.142:0/864262182 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69e800f480 con 0x7f69f81026c0 2024-03-29T05:07:05.812 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.799+0000 7f6a0041e700 1 -- 172.21.15.142:0/864262182 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f81a2c10 con 0x7f69f81026c0 2024-03-29T05:07:05.812 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.800+0000 7f6a0041e700 1 -- 172.21.15.142:0/864262182 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f81082b0 con 0x7f69f81026c0 2024-03-29T05:07:05.812 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.801+0000 7f69ef7fe700 1 -- 172.21.15.142:0/864262182 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f69e8022870 con 0x7f69f81026c0 2024-03-29T05:07:05.812 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.801+0000 7f69ef7fe700 1 --2- 172.21.15.142:0/864262182 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f69e406dd60 0x7f69e4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:05.813 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.801+0000 7f69ef7fe700 1 -- 172.21.15.142:0/864262182 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f69e808f400 con 0x7f69f81026c0 2024-03-29T05:07:05.813 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.801+0000 7f69fe1ba700 1 --2- 172.21.15.142:0/864262182 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f69e406dd60 0x7f69e4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:05.813 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.802+0000 7f69fe1ba700 1 --2- 172.21.15.142:0/864262182 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f69e406dd60 0x7f69e4070220 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f69f4005fd0 tx=0x7f69f4005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:05.816 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:05.807+0000 7f69ef7fe700 1 -- 172.21.15.142:0/864262182 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f69e80589f0 con 0x7f69f81026c0 2024-03-29T05:07:06.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.043+0000 7f6a0041e700 1 -- 172.21.15.142:0/864262182 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f69f804ea90 con 0x7f69f81026c0 2024-03-29T05:07:06.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.044+0000 7f69ef7fe700 1 -- 172.21.15.142:0/864262182 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f69e8027770 con 0x7f69f81026c0 2024-03-29T05:07:06.055 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:07:06.055 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:07:06.055 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:07:06.055 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:07:06.055 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:07:06.055 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:07:06.055 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:07:06.055 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:07:06.055 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:07:06.055 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:07:06.056 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:07:06.057 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:07:06.058 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:07:06.058 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:07:06.058 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:07:06.058 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:07:06.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.046+0000 7f6a0041e700 1 -- 172.21.15.142:0/864262182 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f69e406dd60 msgr2=0x7f69e4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:06.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.046+0000 7f6a0041e700 1 --2- 172.21.15.142:0/864262182 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f69e406dd60 0x7f69e4070220 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f69f4005fd0 tx=0x7f69f4005e30 comp rx=0 tx=0).stop 2024-03-29T05:07:06.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.046+0000 7f6a0041e700 1 -- 172.21.15.142:0/864262182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f81026c0 msgr2=0x7f69f80fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:06.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.046+0000 7f6a0041e700 1 --2- 172.21.15.142:0/864262182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f81026c0 0x7f69f80fff40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f69e800bda0 tx=0x7f69e800be80 comp rx=0 tx=0).stop 2024-03-29T05:07:06.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.046+0000 7f6a0041e700 1 -- 172.21.15.142:0/864262182 shutdown_connections 2024-03-29T05:07:06.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.046+0000 7f6a0041e700 1 --2- 172.21.15.142:0/864262182 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f69e406dd60 0x7f69e4070220 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.046+0000 7f6a0041e700 1 --2- 172.21.15.142:0/864262182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69f8101da0 0x7f69f80ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.046+0000 7f6a0041e700 1 --2- 172.21.15.142:0/864262182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f81026c0 0x7f69f80fff40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.046+0000 7f6a0041e700 1 -- 172.21.15.142:0/864262182 >> 172.21.15.142:0/864262182 conn(0x7f69f80fb550 msgr2=0x7f69f81053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:06.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.046+0000 7f6a0041e700 1 -- 172.21.15.142:0/864262182 shutdown_connections 2024-03-29T05:07:06.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.046+0000 7f6a0041e700 1 -- 172.21.15.142:0/864262182 wait complete. 2024-03-29T05:07:06.059 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:07:06.162 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.152+0000 7faf7d17f700 1 -- 172.21.15.142:0/855572010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf78100de0 msgr2=0x7faf781011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:06.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.152+0000 7faf7d17f700 1 --2- 172.21.15.142:0/855572010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf78100de0 0x7faf781011c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7faf68009910 tx=0x7faf68009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:06.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.152+0000 7faf7d17f700 1 -- 172.21.15.142:0/855572010 shutdown_connections 2024-03-29T05:07:06.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.152+0000 7faf7d17f700 1 --2- 172.21.15.142:0/855572010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78101790 0x7faf781057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.152+0000 7faf7d17f700 1 --2- 172.21.15.142:0/855572010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf78100de0 0x7faf781011c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.152+0000 7faf7d17f700 1 -- 172.21.15.142:0/855572010 >> 172.21.15.142:0/855572010 conn(0x7faf780fc670 msgr2=0x7faf780fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:06.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.152+0000 7faf7d17f700 1 -- 172.21.15.142:0/855572010 shutdown_connections 2024-03-29T05:07:06.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.152+0000 7faf7d17f700 1 -- 172.21.15.142:0/855572010 wait complete. 2024-03-29T05:07:06.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.152+0000 7faf7d17f700 1 Processor -- start 2024-03-29T05:07:06.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf7d17f700 1 -- start start 2024-03-29T05:07:06.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf7d17f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf78100de0 0x7faf7819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:06.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf7d17f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78101790 0x7faf7819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:06.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf7d17f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7819d070 con 0x7faf78100de0 2024-03-29T05:07:06.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf7d17f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf781965b0 con 0x7faf78101790 2024-03-29T05:07:06.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf76d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf78100de0 0x7faf7819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:06.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf76d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf78100de0 0x7faf7819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54518/0 (socket says 172.21.15.142:54518) 2024-03-29T05:07:06.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf76d9d700 1 -- 172.21.15.142:0/1596590226 learned_addr learned my addr 172.21.15.142:0/1596590226 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:06.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf6e59c700 1 --2- 172.21.15.142:0/1596590226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78101790 0x7faf7819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:06.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf76d9d700 1 -- 172.21.15.142:0/1596590226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78101790 msgr2=0x7faf7819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:06.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf76d9d700 1 --2- 172.21.15.142:0/1596590226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78101790 0x7faf7819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf76d9d700 1 -- 172.21.15.142:0/1596590226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf680095f0 con 0x7faf78100de0 2024-03-29T05:07:06.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.153+0000 7faf6e59c700 1 --2- 172.21.15.142:0/1596590226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78101790 0x7faf7819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:07:06.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.154+0000 7faf76d9d700 1 --2- 172.21.15.142:0/1596590226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf78100de0 0x7faf7819c530 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7faf68006010 tx=0x7faf6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:06.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.154+0000 7faf74d99700 1 -- 172.21.15.142:0/1596590226 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf6801d070 con 0x7faf78100de0 2024-03-29T05:07:06.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.154+0000 7faf74d99700 1 -- 172.21.15.142:0/1596590226 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf68004fb0 con 0x7faf78100de0 2024-03-29T05:07:06.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.154+0000 7faf7d17f700 1 -- 172.21.15.142:0/1596590226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf78196830 con 0x7faf78100de0 2024-03-29T05:07:06.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.154+0000 7faf74d99700 1 -- 172.21.15.142:0/1596590226 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf68022ea0 con 0x7faf78100de0 2024-03-29T05:07:06.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.154+0000 7faf7d17f700 1 -- 172.21.15.142:0/1596590226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf78196d20 con 0x7faf78100de0 2024-03-29T05:07:06.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.155+0000 7faf7d17f700 1 -- 172.21.15.142:0/1596590226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf78109180 con 0x7faf78100de0 2024-03-29T05:07:06.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.155+0000 7faf74d99700 1 -- 172.21.15.142:0/1596590226 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7faf68022470 con 0x7faf78100de0 2024-03-29T05:07:06.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.156+0000 7faf74d99700 1 --2- 172.21.15.142:0/1596590226 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7faf6406dd60 0x7faf64070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:06.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.156+0000 7faf6e59c700 1 --2- 172.21.15.142:0/1596590226 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7faf6406dd60 0x7faf64070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:06.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.156+0000 7faf74d99700 1 -- 172.21.15.142:0/1596590226 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7faf6808f390 con 0x7faf78100de0 2024-03-29T05:07:06.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.156+0000 7faf6e59c700 1 --2- 172.21.15.142:0/1596590226 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7faf6406dd60 0x7faf64070220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faf60005fd0 tx=0x7faf60005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:06.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.161+0000 7faf74d99700 1 -- 172.21.15.142:0/1596590226 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7faf6805c530 con 0x7faf78100de0 2024-03-29T05:07:06.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:06 smithi142 ceph-mon[112475]: from='client.35050 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:06.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:06 smithi142 ceph-mon[112475]: from='client.35054 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:06.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:06 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1153010806' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:07:06.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:06 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/864262182' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:07:06.386 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.376+0000 7faf7d17f700 1 -- 172.21.15.142:0/1596590226 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faf78197050 con 0x7faf6406dd60 2024-03-29T05:07:06.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.377+0000 7faf74d99700 1 -- 172.21.15.142:0/1596590226 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faf78197050 con 0x7faf6406dd60 2024-03-29T05:07:06.387 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:07:06.388 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:07:06.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.380+0000 7faf7d17f700 1 -- 172.21.15.142:0/1596590226 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7faf6406dd60 msgr2=0x7faf64070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:06.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.380+0000 7faf7d17f700 1 --2- 172.21.15.142:0/1596590226 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7faf6406dd60 0x7faf64070220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faf60005fd0 tx=0x7faf60005e30 comp rx=0 tx=0).stop 2024-03-29T05:07:06.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.380+0000 7faf7d17f700 1 -- 172.21.15.142:0/1596590226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf78100de0 msgr2=0x7faf7819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:06.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.380+0000 7faf7d17f700 1 --2- 172.21.15.142:0/1596590226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf78100de0 0x7faf7819c530 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7faf68006010 tx=0x7faf6800be20 comp rx=0 tx=0).stop 2024-03-29T05:07:06.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.380+0000 7faf7d17f700 1 -- 172.21.15.142:0/1596590226 shutdown_connections 2024-03-29T05:07:06.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.380+0000 7faf7d17f700 1 --2- 172.21.15.142:0/1596590226 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7faf6406dd60 0x7faf64070220 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.380+0000 7faf7d17f700 1 --2- 172.21.15.142:0/1596590226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf78100de0 0x7faf7819c530 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.380+0000 7faf7d17f700 1 --2- 172.21.15.142:0/1596590226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78101790 0x7faf7819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.380+0000 7faf7d17f700 1 -- 172.21.15.142:0/1596590226 >> 172.21.15.142:0/1596590226 conn(0x7faf780fc670 msgr2=0x7faf780fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:06.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.380+0000 7faf7d17f700 1 -- 172.21.15.142:0/1596590226 shutdown_connections 2024-03-29T05:07:06.390 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.380+0000 7faf7d17f700 1 -- 172.21.15.142:0/1596590226 wait complete. 2024-03-29T05:07:06.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.482+0000 7f79cc0a4700 1 -- 172.21.15.142:0/2777236762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79c4100dc0 msgr2=0x7f79c41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:06.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.482+0000 7f79cc0a4700 1 --2- 172.21.15.142:0/2777236762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79c4100dc0 0x7f79c41011a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009910 tx=0x7f79c0009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:06.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.482+0000 7f79cc0a4700 1 -- 172.21.15.142:0/2777236762 shutdown_connections 2024-03-29T05:07:06.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.482+0000 7f79cc0a4700 1 --2- 172.21.15.142:0/2777236762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79c4101770 0x7f79c41057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.482+0000 7f79cc0a4700 1 --2- 172.21.15.142:0/2777236762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79c4100dc0 0x7f79c41011a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.482+0000 7f79cc0a4700 1 -- 172.21.15.142:0/2777236762 >> 172.21.15.142:0/2777236762 conn(0x7f79c40fc650 msgr2=0x7f79c40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:06.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.483+0000 7f79cc0a4700 1 -- 172.21.15.142:0/2777236762 shutdown_connections 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.483+0000 7f79cc0a4700 1 -- 172.21.15.142:0/2777236762 wait complete. 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.483+0000 7f79cc0a4700 1 Processor -- start 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.483+0000 7f79cc0a4700 1 -- start start 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.483+0000 7f79cc0a4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79c4100dc0 0x7f79c419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.483+0000 7f79cc0a4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79c4101770 0x7f79c419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.483+0000 7f79cc0a4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c419d110 con 0x7f79c4101770 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.483+0000 7f79cc0a4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c41965c0 con 0x7f79c4100dc0 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79c963f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79c4101770 0x7f79c419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79c963f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79c4101770 0x7f79c419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54530/0 (socket says 172.21.15.142:54530) 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79c963f700 1 -- 172.21.15.142:0/110041935 learned_addr learned my addr 172.21.15.142:0/110041935 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79c9e40700 1 --2- 172.21.15.142:0/110041935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79c4100dc0 0x7f79c419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79c963f700 1 -- 172.21.15.142:0/110041935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79c4100dc0 msgr2=0x7f79c419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79c963f700 1 --2- 172.21.15.142:0/110041935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79c4100dc0 0x7f79c419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79c963f700 1 -- 172.21.15.142:0/110041935 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c00095f0 con 0x7f79c4101770 2024-03-29T05:07:06.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79c963f700 1 --2- 172.21.15.142:0/110041935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79c4101770 0x7f79c419ca80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f79b400c8f0 tx=0x7f79b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:06.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79baffd700 1 -- 172.21.15.142:0/110041935 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79b4007a50 con 0x7f79c4101770 2024-03-29T05:07:06.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79baffd700 1 -- 172.21.15.142:0/110041935 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79b4007bb0 con 0x7f79c4101770 2024-03-29T05:07:06.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79cc0a4700 1 -- 172.21.15.142:0/110041935 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c41968a0 con 0x7f79c4101770 2024-03-29T05:07:06.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79baffd700 1 -- 172.21.15.142:0/110041935 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79b4018630 con 0x7f79c4101770 2024-03-29T05:07:06.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.484+0000 7f79cc0a4700 1 -- 172.21.15.142:0/110041935 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c4196df0 con 0x7f79c4101770 2024-03-29T05:07:06.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.485+0000 7f79cc0a4700 1 -- 172.21.15.142:0/110041935 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c4109160 con 0x7f79c4101770 2024-03-29T05:07:06.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.486+0000 7f79baffd700 1 -- 172.21.15.142:0/110041935 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f79b4007d20 con 0x7f79c4101770 2024-03-29T05:07:06.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.486+0000 7f79baffd700 1 --2- 172.21.15.142:0/110041935 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f79b006dd60 0x7f79b0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:06.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.486+0000 7f79baffd700 1 -- 172.21.15.142:0/110041935 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f79b408dc70 con 0x7f79c4101770 2024-03-29T05:07:06.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.486+0000 7f79c9e40700 1 --2- 172.21.15.142:0/110041935 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f79b006dd60 0x7f79b0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:06.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.487+0000 7f79c9e40700 1 --2- 172.21.15.142:0/110041935 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f79b006dd60 0x7f79b0070220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f79c000b5c0 tx=0x7f79c001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:06.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.491+0000 7f79baffd700 1 -- 172.21.15.142:0/110041935 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f79b405abc0 con 0x7f79c4101770 2024-03-29T05:07:06.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:06 smithi152 ceph-mon[88186]: from='client.35050 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:06.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:06 smithi152 ceph-mon[88186]: from='client.35054 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:06.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:06 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1153010806' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:07:06.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:06 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/864262182' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:07:06.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.765+0000 7f79cc0a4700 1 -- 172.21.15.142:0/110041935 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f79c41970c0 con 0x7f79c4101770 2024-03-29T05:07:06.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.766+0000 7f79baffd700 1 -- 172.21.15.142:0/110041935 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f79b405a750 con 0x7f79c4101770 2024-03-29T05:07:06.776 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:07:06.776 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:07:06.776 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:07:06.776 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:07:06.776 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:07:06.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.768+0000 7f79cc0a4700 1 -- 172.21.15.142:0/110041935 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f79b006dd60 msgr2=0x7f79b0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:06.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.768+0000 7f79cc0a4700 1 --2- 172.21.15.142:0/110041935 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f79b006dd60 0x7f79b0070220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f79c000b5c0 tx=0x7f79c001a040 comp rx=0 tx=0).stop 2024-03-29T05:07:06.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.768+0000 7f79cc0a4700 1 -- 172.21.15.142:0/110041935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79c4101770 msgr2=0x7f79c419ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:06.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.768+0000 7f79cc0a4700 1 --2- 172.21.15.142:0/110041935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79c4101770 0x7f79c419ca80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f79b400c8f0 tx=0x7f79b400ccb0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.768+0000 7f79cc0a4700 1 -- 172.21.15.142:0/110041935 shutdown_connections 2024-03-29T05:07:06.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.768+0000 7f79cc0a4700 1 --2- 172.21.15.142:0/110041935 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f79b006dd60 0x7f79b0070220 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.768+0000 7f79cc0a4700 1 --2- 172.21.15.142:0/110041935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79c4100dc0 0x7f79c419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.768+0000 7f79cc0a4700 1 --2- 172.21.15.142:0/110041935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79c4101770 0x7f79c419ca80 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:06.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.768+0000 7f79cc0a4700 1 -- 172.21.15.142:0/110041935 >> 172.21.15.142:0/110041935 conn(0x7f79c40fc650 msgr2=0x7f79c40fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:06.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.768+0000 7f79cc0a4700 1 -- 172.21.15.142:0/110041935 shutdown_connections 2024-03-29T05:07:06.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:06.768+0000 7f79cc0a4700 1 -- 172.21.15.142:0/110041935 wait complete. 2024-03-29T05:07:07.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:07 smithi142 ceph-mon[112475]: pgmap v564: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:07.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:07 smithi142 ceph-mon[112475]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:07.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:07 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/110041935' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:07:07.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:07 smithi152 ceph-mon[88186]: pgmap v564: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:07.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:07 smithi152 ceph-mon[88186]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:07.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:07 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/110041935' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:07:09.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:09 smithi142 ceph-mon[112475]: pgmap v565: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:09.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:09 smithi152 ceph-mon[88186]: pgmap v565: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:11.381 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:11 smithi142 ceph-mon[112475]: pgmap v566: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:11.550 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:11 smithi152 ceph-mon[88186]: pgmap v566: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:13.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:13 smithi142 ceph-mon[112475]: pgmap v567: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:13.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:13 smithi152 ceph-mon[88186]: pgmap v567: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:15.603 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:15 smithi152 ceph-mon[88186]: pgmap v568: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:15.604 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:07:15.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:15 smithi142 ceph-mon[112475]: pgmap v568: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:15.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:07:17.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:17 smithi142 ceph-mon[112475]: pgmap v569: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:17.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:17 smithi152 ceph-mon[88186]: pgmap v569: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:19.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:19 smithi142 ceph-mon[112475]: pgmap v570: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:19.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:19 smithi152 ceph-mon[88186]: pgmap v570: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:21.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:21 smithi142 ceph-mon[112475]: pgmap v571: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:21.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:21 smithi152 ceph-mon[88186]: pgmap v571: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:23.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:23 smithi142 ceph-mon[112475]: pgmap v572: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:23.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:23 smithi152 ceph-mon[88186]: pgmap v572: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:25.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:25 smithi142 ceph-mon[112475]: pgmap v573: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:25.710 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:25 smithi152 ceph-mon[88186]: pgmap v573: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:27.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:27 smithi142 ceph-mon[112475]: pgmap v574: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:27.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:27 smithi152 ceph-mon[88186]: pgmap v574: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:29.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:29 smithi142 ceph-mon[112475]: pgmap v575: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:29.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:29 smithi152 ceph-mon[88186]: pgmap v575: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:30.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:07:30.763 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:07:31.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:31 smithi142 ceph-mon[112475]: pgmap v576: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:31.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:31 smithi152 ceph-mon[88186]: pgmap v576: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:33.631 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:33 smithi142 ceph-mon[112475]: pgmap v577: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:33.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:33 smithi152 ceph-mon[88186]: pgmap v577: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:35.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:35 smithi142 ceph-mon[112475]: pgmap v578: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:35.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:35 smithi152 ceph-mon[88186]: pgmap v578: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:36.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.881+0000 7ff30bf2d700 1 -- 172.21.15.142:0/2679483854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304100da0 msgr2=0x7ff304101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:36.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.881+0000 7ff30bf2d700 1 --2- 172.21.15.142:0/2679483854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304100da0 0x7ff304101180 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009910 tx=0x7ff2f4009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:36.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.881+0000 7ff30bf2d700 1 -- 172.21.15.142:0/2679483854 shutdown_connections 2024-03-29T05:07:36.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.881+0000 7ff30bf2d700 1 --2- 172.21.15.142:0/2679483854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff304101750 0x7ff3041057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.881+0000 7ff30bf2d700 1 --2- 172.21.15.142:0/2679483854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304100da0 0x7ff304101180 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.881+0000 7ff30bf2d700 1 -- 172.21.15.142:0/2679483854 >> 172.21.15.142:0/2679483854 conn(0x7ff3040fc630 msgr2=0x7ff3040fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.881+0000 7ff30bf2d700 1 -- 172.21.15.142:0/2679483854 shutdown_connections 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.881+0000 7ff30bf2d700 1 -- 172.21.15.142:0/2679483854 wait complete. 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff30bf2d700 1 Processor -- start 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff30bf2d700 1 -- start start 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff30bf2d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304100da0 0x7ff30419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff30bf2d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff304101750 0x7ff30419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff30bf2d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30419d170 con 0x7ff304100da0 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff30bf2d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff304196620 con 0x7ff304101750 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff309cc9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304100da0 0x7ff30419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff309cc9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304100da0 0x7ff30419c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56782/0 (socket says 172.21.15.142:56782) 2024-03-29T05:07:36.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff309cc9700 1 -- 172.21.15.142:0/1135636852 learned_addr learned my addr 172.21.15.142:0/1135636852 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:36.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff3094c8700 1 --2- 172.21.15.142:0/1135636852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff304101750 0x7ff30419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:36.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff309cc9700 1 -- 172.21.15.142:0/1135636852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff304101750 msgr2=0x7ff30419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:36.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff309cc9700 1 --2- 172.21.15.142:0/1135636852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff304101750 0x7ff30419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:36.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.882+0000 7ff309cc9700 1 -- 172.21.15.142:0/1135636852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f40095f0 con 0x7ff304100da0 2024-03-29T05:07:36.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.883+0000 7ff309cc9700 1 --2- 172.21.15.142:0/1135636852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304100da0 0x7ff30419c5a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40092e0 tx=0x7ff2f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:36.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.883+0000 7ff2faffd700 1 -- 172.21.15.142:0/1135636852 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f401d070 con 0x7ff304100da0 2024-03-29T05:07:36.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.883+0000 7ff2faffd700 1 -- 172.21.15.142:0/1135636852 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2f4005e00 con 0x7ff304100da0 2024-03-29T05:07:36.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.883+0000 7ff30bf2d700 1 -- 172.21.15.142:0/1135636852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3041968a0 con 0x7ff304100da0 2024-03-29T05:07:36.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.883+0000 7ff2faffd700 1 -- 172.21.15.142:0/1135636852 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f4017400 con 0x7ff304100da0 2024-03-29T05:07:36.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.883+0000 7ff30bf2d700 1 -- 172.21.15.142:0/1135636852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff304196d90 con 0x7ff304100da0 2024-03-29T05:07:36.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.884+0000 7ff30bf2d700 1 -- 172.21.15.142:0/1135636852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff304109140 con 0x7ff304100da0 2024-03-29T05:07:36.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.884+0000 7ff2faffd700 1 -- 172.21.15.142:0/1135636852 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff2f400f460 con 0x7ff304100da0 2024-03-29T05:07:36.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.885+0000 7ff2faffd700 1 --2- 172.21.15.142:0/1135636852 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2f006dd60 0x7ff2f0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:36.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.885+0000 7ff2faffd700 1 -- 172.21.15.142:0/1135636852 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff2f408f1b0 con 0x7ff304100da0 2024-03-29T05:07:36.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.885+0000 7ff3094c8700 1 --2- 172.21.15.142:0/1135636852 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2f006dd60 0x7ff2f0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:36.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.885+0000 7ff3094c8700 1 --2- 172.21.15.142:0/1135636852 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2f006dd60 0x7ff2f0070220 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff300005950 tx=0x7ff3000058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:36.899 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:36.889+0000 7ff2faffd700 1 -- 172.21.15.142:0/1135636852 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff2f405c100 con 0x7ff304100da0 2024-03-29T05:07:37.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.103+0000 7ff30bf2d700 1 -- 172.21.15.142:0/1135636852 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff304197070 con 0x7ff2f006dd60 2024-03-29T05:07:37.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.104+0000 7ff2faffd700 1 -- 172.21.15.142:0/1135636852 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff304197070 con 0x7ff2f006dd60 2024-03-29T05:07:37.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.106+0000 7ff30bf2d700 1 -- 172.21.15.142:0/1135636852 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2f006dd60 msgr2=0x7ff2f0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:37.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.106+0000 7ff30bf2d700 1 --2- 172.21.15.142:0/1135636852 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2f006dd60 0x7ff2f0070220 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff300005950 tx=0x7ff3000058e0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.106+0000 7ff30bf2d700 1 -- 172.21.15.142:0/1135636852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304100da0 msgr2=0x7ff30419c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:37.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.106+0000 7ff30bf2d700 1 --2- 172.21.15.142:0/1135636852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304100da0 0x7ff30419c5a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40092e0 tx=0x7ff2f400be20 comp rx=0 tx=0).stop 2024-03-29T05:07:37.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.106+0000 7ff30bf2d700 1 -- 172.21.15.142:0/1135636852 shutdown_connections 2024-03-29T05:07:37.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.106+0000 7ff30bf2d700 1 --2- 172.21.15.142:0/1135636852 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff2f006dd60 0x7ff2f0070220 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.106+0000 7ff30bf2d700 1 --2- 172.21.15.142:0/1135636852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff304100da0 0x7ff30419c5a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.106+0000 7ff30bf2d700 1 --2- 172.21.15.142:0/1135636852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff304101750 0x7ff30419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.106+0000 7ff30bf2d700 1 -- 172.21.15.142:0/1135636852 >> 172.21.15.142:0/1135636852 conn(0x7ff3040fc630 msgr2=0x7ff3040fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:37.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.106+0000 7ff30bf2d700 1 -- 172.21.15.142:0/1135636852 shutdown_connections 2024-03-29T05:07:37.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.106+0000 7ff30bf2d700 1 -- 172.21.15.142:0/1135636852 wait complete. 2024-03-29T05:07:37.126 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:07:37.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.211+0000 7f805e297700 1 -- 172.21.15.142:0/1829683634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8058101790 msgr2=0x7f80581057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:37.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.211+0000 7f805e297700 1 --2- 172.21.15.142:0/1829683634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8058101790 0x7f80581057e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f804c009910 tx=0x7f804c009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:37.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.211+0000 7f805e297700 1 -- 172.21.15.142:0/1829683634 shutdown_connections 2024-03-29T05:07:37.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.211+0000 7f805e297700 1 --2- 172.21.15.142:0/1829683634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8058101790 0x7f80581057e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.211+0000 7f805e297700 1 --2- 172.21.15.142:0/1829683634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8058100de0 0x7f80581011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.211+0000 7f805e297700 1 -- 172.21.15.142:0/1829683634 >> 172.21.15.142:0/1829683634 conn(0x7f80580fc670 msgr2=0x7f80580fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:37.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.211+0000 7f805e297700 1 -- 172.21.15.142:0/1829683634 shutdown_connections 2024-03-29T05:07:37.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.211+0000 7f805e297700 1 -- 172.21.15.142:0/1829683634 wait complete. 2024-03-29T05:07:37.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.212+0000 7f805e297700 1 Processor -- start 2024-03-29T05:07:37.222 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.212+0000 7f805e297700 1 -- start start 2024-03-29T05:07:37.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.212+0000 7f805e297700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8058100de0 0x7f8058073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:37.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.212+0000 7f805e297700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8058101790 0x7f80580735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:37.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.212+0000 7f805e297700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8058073b10 con 0x7f8058101790 2024-03-29T05:07:37.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.212+0000 7f805e297700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8058073c50 con 0x7f8058100de0 2024-03-29T05:07:37.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.212+0000 7f80577fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8058101790 0x7f80580735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:37.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.212+0000 7f80577fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8058101790 0x7f80580735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56802/0 (socket says 172.21.15.142:56802) 2024-03-29T05:07:37.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.212+0000 7f80577fe700 1 -- 172.21.15.142:0/536693365 learned_addr learned my addr 172.21.15.142:0/536693365 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:37.224 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.213+0000 7f8057fff700 1 --2- 172.21.15.142:0/536693365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8058100de0 0x7f8058073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:37.224 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.213+0000 7f80577fe700 1 -- 172.21.15.142:0/536693365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8058100de0 msgr2=0x7f8058073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:37.224 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.213+0000 7f80577fe700 1 --2- 172.21.15.142:0/536693365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8058100de0 0x7f8058073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.224 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.213+0000 7f80577fe700 1 -- 172.21.15.142:0/536693365 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c0095f0 con 0x7f8058101790 2024-03-29T05:07:37.224 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.213+0000 7f80577fe700 1 --2- 172.21.15.142:0/536693365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8058101790 0x7f80580735d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f804c0092e0 tx=0x7f804c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:37.224 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.213+0000 7f80557fa700 1 -- 172.21.15.142:0/536693365 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f804c01d070 con 0x7f8058101790 2024-03-29T05:07:37.224 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.213+0000 7f805e297700 1 -- 172.21.15.142:0/536693365 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8058073ed0 con 0x7f8058101790 2024-03-29T05:07:37.224 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.213+0000 7f80557fa700 1 -- 172.21.15.142:0/536693365 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f804c005dc0 con 0x7f8058101790 2024-03-29T05:07:37.224 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.213+0000 7f805e297700 1 -- 172.21.15.142:0/536693365 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80581a2c90 con 0x7f8058101790 2024-03-29T05:07:37.224 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.213+0000 7f80557fa700 1 -- 172.21.15.142:0/536693365 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f804c017400 con 0x7f8058101790 2024-03-29T05:07:37.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.214+0000 7f805e297700 1 -- 172.21.15.142:0/536693365 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80581091e0 con 0x7f8058101790 2024-03-29T05:07:37.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.215+0000 7f80557fa700 1 -- 172.21.15.142:0/536693365 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f804c00f460 con 0x7f8058101790 2024-03-29T05:07:37.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.215+0000 7f80557fa700 1 --2- 172.21.15.142:0/536693365 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f804006dd60 0x7f8040070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:37.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.215+0000 7f80557fa700 1 -- 172.21.15.142:0/536693365 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f804c08f0b0 con 0x7f8058101790 2024-03-29T05:07:37.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.215+0000 7f8057fff700 1 --2- 172.21.15.142:0/536693365 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f804006dd60 0x7f8040070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:37.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.216+0000 7f8057fff700 1 --2- 172.21.15.142:0/536693365 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f804006dd60 0x7f8040070220 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8048005950 tx=0x7f80480058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:37.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.220+0000 7f80557fa700 1 -- 172.21.15.142:0/536693365 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f804c058550 con 0x7f8058101790 2024-03-29T05:07:37.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.434+0000 7f805e297700 1 -- 172.21.15.142:0/536693365 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f80580611d0 con 0x7f804006dd60 2024-03-29T05:07:37.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.435+0000 7f80557fa700 1 -- 172.21.15.142:0/536693365 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f80580611d0 con 0x7f804006dd60 2024-03-29T05:07:37.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.437+0000 7f805e297700 1 -- 172.21.15.142:0/536693365 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f804006dd60 msgr2=0x7f8040070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:37.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.437+0000 7f805e297700 1 --2- 172.21.15.142:0/536693365 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f804006dd60 0x7f8040070220 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8048005950 tx=0x7f80480058e0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.437+0000 7f805e297700 1 -- 172.21.15.142:0/536693365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8058101790 msgr2=0x7f80580735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:37.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.437+0000 7f805e297700 1 --2- 172.21.15.142:0/536693365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8058101790 0x7f80580735d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f804c0092e0 tx=0x7f804c00be20 comp rx=0 tx=0).stop 2024-03-29T05:07:37.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.437+0000 7f805e297700 1 -- 172.21.15.142:0/536693365 shutdown_connections 2024-03-29T05:07:37.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.437+0000 7f805e297700 1 --2- 172.21.15.142:0/536693365 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f804006dd60 0x7f8040070220 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.437+0000 7f805e297700 1 --2- 172.21.15.142:0/536693365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8058100de0 0x7f8058073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.437+0000 7f805e297700 1 --2- 172.21.15.142:0/536693365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8058101790 0x7f80580735d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.437+0000 7f805e297700 1 -- 172.21.15.142:0/536693365 >> 172.21.15.142:0/536693365 conn(0x7f80580fc670 msgr2=0x7f80580fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:37.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.438+0000 7f805e297700 1 -- 172.21.15.142:0/536693365 shutdown_connections 2024-03-29T05:07:37.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.438+0000 7f805e297700 1 -- 172.21.15.142:0/536693365 wait complete. 2024-03-29T05:07:37.550 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.540+0000 7f07fe7a3700 1 -- 172.21.15.142:0/2632265822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07f8101770 msgr2=0x7f07f81057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:37.550 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.540+0000 7f07fe7a3700 1 --2- 172.21.15.142:0/2632265822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07f8101770 0x7f07f81057c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009910 tx=0x7f07ec009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:37.550 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.540+0000 7f07fe7a3700 1 -- 172.21.15.142:0/2632265822 shutdown_connections 2024-03-29T05:07:37.550 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.540+0000 7f07fe7a3700 1 --2- 172.21.15.142:0/2632265822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07f8101770 0x7f07f81057c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.550 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.540+0000 7f07fe7a3700 1 --2- 172.21.15.142:0/2632265822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8100dc0 0x7f07f81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.540+0000 7f07fe7a3700 1 -- 172.21.15.142:0/2632265822 >> 172.21.15.142:0/2632265822 conn(0x7f07f80fc650 msgr2=0x7f07f80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:37.551 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:37 smithi142 ceph-mon[112475]: pgmap v579: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:37.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.541+0000 7f07fe7a3700 1 -- 172.21.15.142:0/2632265822 shutdown_connections 2024-03-29T05:07:37.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.541+0000 7f07fe7a3700 1 -- 172.21.15.142:0/2632265822 wait complete. 2024-03-29T05:07:37.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.541+0000 7f07fe7a3700 1 Processor -- start 2024-03-29T05:07:37.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.541+0000 7f07fe7a3700 1 -- start start 2024-03-29T05:07:37.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07fe7a3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07f8100dc0 0x7f07f819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:37.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07fe7a3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8101770 0x7f07f819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:37.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07fe7a3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f819d170 con 0x7f07f8100dc0 2024-03-29T05:07:37.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07fe7a3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f8196620 con 0x7f07f8101770 2024-03-29T05:07:37.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07f7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07f8100dc0 0x7f07f819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:37.553 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07f7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07f8100dc0 0x7f07f819c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56818/0 (socket says 172.21.15.142:56818) 2024-03-29T05:07:37.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07f7fff700 1 -- 172.21.15.142:0/3200698550 learned_addr learned my addr 172.21.15.142:0/3200698550 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:37.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07f77fe700 1 --2- 172.21.15.142:0/3200698550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8101770 0x7f07f819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:37.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07f7fff700 1 -- 172.21.15.142:0/3200698550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8101770 msgr2=0x7f07f819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:37.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07f7fff700 1 --2- 172.21.15.142:0/3200698550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8101770 0x7f07f819cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07f7fff700 1 -- 172.21.15.142:0/3200698550 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec0095f0 con 0x7f07f8100dc0 2024-03-29T05:07:37.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07f7fff700 1 --2- 172.21.15.142:0/3200698550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07f8100dc0 0x7f07f819c5a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f07e800c8f0 tx=0x7f07e800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:37.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07f57fa700 1 -- 172.21.15.142:0/3200698550 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07e8004a00 con 0x7f07f8100dc0 2024-03-29T05:07:37.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07f57fa700 1 -- 172.21.15.142:0/3200698550 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07e80054e0 con 0x7f07f8100dc0 2024-03-29T05:07:37.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07fe7a3700 1 -- 172.21.15.142:0/3200698550 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f8196900 con 0x7f07f8100dc0 2024-03-29T05:07:37.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.542+0000 7f07f57fa700 1 -- 172.21.15.142:0/3200698550 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07e8007cf0 con 0x7f07f8100dc0 2024-03-29T05:07:37.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.543+0000 7f07fe7a3700 1 -- 172.21.15.142:0/3200698550 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f8196e20 con 0x7f07f8100dc0 2024-03-29T05:07:37.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.544+0000 7f07fe7a3700 1 -- 172.21.15.142:0/3200698550 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f81090d0 con 0x7f07f8100dc0 2024-03-29T05:07:37.555 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.544+0000 7f07f57fa700 1 -- 172.21.15.142:0/3200698550 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f07e8005020 con 0x7f07f8100dc0 2024-03-29T05:07:37.555 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.544+0000 7f07f57fa700 1 --2- 172.21.15.142:0/3200698550 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f07e006dd60 0x7f07e0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:37.555 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.545+0000 7f07f57fa700 1 -- 172.21.15.142:0/3200698550 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f07e8013070 con 0x7f07f8100dc0 2024-03-29T05:07:37.555 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.545+0000 7f07f77fe700 1 --2- 172.21.15.142:0/3200698550 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f07e006dd60 0x7f07e0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:37.555 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.545+0000 7f07f77fe700 1 --2- 172.21.15.142:0/3200698550 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f07e006dd60 0x7f07e0070220 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f07ec006010 tx=0x7f07ec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:37.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.549+0000 7f07f57fa700 1 -- 172.21.15.142:0/3200698550 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f07e805aa20 con 0x7f07f8100dc0 2024-03-29T05:07:37.761 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.752+0000 7f07fe7a3700 1 -- 172.21.15.142:0/3200698550 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f07f8197070 con 0x7f07e006dd60 2024-03-29T05:07:37.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.763+0000 7f07f57fa700 1 -- 172.21.15.142:0/3200698550 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f07f8197070 con 0x7f07e006dd60 2024-03-29T05:07:37.772 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:07:37.773 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (19m) 12m ago 30m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:07:37.773 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (31m) 12m ago 31m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:07:37.773 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (30m) 12m ago 30m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:07:37.773 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (16m) 12m ago 31m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:07:37.773 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (16m) 12m ago 30m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:07:37.773 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (19m) 12m ago 30m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:07:37.773 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (12m) 12m ago 25m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:07:37.773 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (12m) 12m ago 25m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:07:37.773 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (12m) 12m ago 25m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:07:37.773 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (12m) 12m ago 25m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:07:37.773 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (20m) 12m ago 32m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:07:37.773 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (20m) 12m ago 29m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:07:37.774 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (17m) 12m ago 32m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:07:37.774 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (17m) 12m ago 29m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:07:37.774 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (20m) 12m ago 31m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:07:37.774 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (20m) 12m ago 30m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:07:37.774 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (16m) 12m ago 29m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:07:37.774 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (15m) 12m ago 28m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:07:37.774 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (14m) 12m ago 28m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:07:37.774 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (14m) 12m ago 27m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:07:37.774 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (13m) 12m ago 27m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:07:37.774 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (13m) 12m ago 26m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:07:37.775 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (19m) 12m ago 30m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:07:37.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.766+0000 7f07fe7a3700 1 -- 172.21.15.142:0/3200698550 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f07e006dd60 msgr2=0x7f07e0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:37.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.766+0000 7f07fe7a3700 1 --2- 172.21.15.142:0/3200698550 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f07e006dd60 0x7f07e0070220 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f07ec006010 tx=0x7f07ec01a040 comp rx=0 tx=0).stop 2024-03-29T05:07:37.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.766+0000 7f07fe7a3700 1 -- 172.21.15.142:0/3200698550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07f8100dc0 msgr2=0x7f07f819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:37.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.766+0000 7f07fe7a3700 1 --2- 172.21.15.142:0/3200698550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07f8100dc0 0x7f07f819c5a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f07e800c8f0 tx=0x7f07e800cc00 comp rx=0 tx=0).stop 2024-03-29T05:07:37.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.766+0000 7f07fe7a3700 1 -- 172.21.15.142:0/3200698550 shutdown_connections 2024-03-29T05:07:37.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.766+0000 7f07fe7a3700 1 --2- 172.21.15.142:0/3200698550 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f07e006dd60 0x7f07e0070220 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.766+0000 7f07fe7a3700 1 --2- 172.21.15.142:0/3200698550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07f8100dc0 0x7f07f819c5a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.766+0000 7f07fe7a3700 1 --2- 172.21.15.142:0/3200698550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8101770 0x7f07f819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.766+0000 7f07fe7a3700 1 -- 172.21.15.142:0/3200698550 >> 172.21.15.142:0/3200698550 conn(0x7f07f80fc650 msgr2=0x7f07f8105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:37.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.766+0000 7f07fe7a3700 1 -- 172.21.15.142:0/3200698550 shutdown_connections 2024-03-29T05:07:37.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.766+0000 7f07fe7a3700 1 -- 172.21.15.142:0/3200698550 wait complete. 2024-03-29T05:07:37.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:37 smithi152 ceph-mon[88186]: pgmap v579: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:37.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.868+0000 7f750bdd4700 1 -- 172.21.15.142:0/3512571661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7504068670 msgr2=0x7f7504068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:37.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.868+0000 7f750bdd4700 1 --2- 172.21.15.142:0/3512571661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7504068670 0x7f7504068a50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009910 tx=0x7f74fc009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:37.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.868+0000 7f750bdd4700 1 -- 172.21.15.142:0/3512571661 shutdown_connections 2024-03-29T05:07:37.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.868+0000 7f750bdd4700 1 --2- 172.21.15.142:0/3512571661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7504069020 0x7f7504105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.868+0000 7f750bdd4700 1 --2- 172.21.15.142:0/3512571661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7504068670 0x7f7504068a50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.868+0000 7f750bdd4700 1 -- 172.21.15.142:0/3512571661 >> 172.21.15.142:0/3512571661 conn(0x7f75040755a0 msgr2=0x7f75040759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:37.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.868+0000 7f750bdd4700 1 -- 172.21.15.142:0/3512571661 shutdown_connections 2024-03-29T05:07:37.878 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.868+0000 7f750bdd4700 1 -- 172.21.15.142:0/3512571661 wait complete. 2024-03-29T05:07:37.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.869+0000 7f750bdd4700 1 Processor -- start 2024-03-29T05:07:37.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.869+0000 7f750bdd4700 1 -- start start 2024-03-29T05:07:37.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.869+0000 7f750bdd4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7504069020 0x7f750419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:37.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.869+0000 7f750bdd4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750419cd60 0x7f75041968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:37.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.869+0000 7f750bdd4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750419d270 con 0x7f750419cd60 2024-03-29T05:07:37.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.869+0000 7f750bdd4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750419d3e0 con 0x7f7504069020 2024-03-29T05:07:37.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f750936f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750419cd60 0x7f75041968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:37.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f750936f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750419cd60 0x7f75041968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56836/0 (socket says 172.21.15.142:56836) 2024-03-29T05:07:37.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f750936f700 1 -- 172.21.15.142:0/1332785782 learned_addr learned my addr 172.21.15.142:0/1332785782 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:37.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f7509b70700 1 --2- 172.21.15.142:0/1332785782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7504069020 0x7f750419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:37.880 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f750936f700 1 -- 172.21.15.142:0/1332785782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7504069020 msgr2=0x7f750419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:37.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f750936f700 1 --2- 172.21.15.142:0/1332785782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7504069020 0x7f750419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:37.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f750936f700 1 -- 172.21.15.142:0/1332785782 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74fc0095f0 con 0x7f750419cd60 2024-03-29T05:07:37.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f750936f700 1 --2- 172.21.15.142:0/1332785782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750419cd60 0x7f75041968a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f74f800c8f0 tx=0x7f74f800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:37.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f7502ffd700 1 -- 172.21.15.142:0/1332785782 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74f80078a0 con 0x7f750419cd60 2024-03-29T05:07:37.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f750bdd4700 1 -- 172.21.15.142:0/1332785782 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7504196f80 con 0x7f750419cd60 2024-03-29T05:07:37.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f7502ffd700 1 -- 172.21.15.142:0/1332785782 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74f800f450 con 0x7f750419cd60 2024-03-29T05:07:37.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f7502ffd700 1 -- 172.21.15.142:0/1332785782 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74f8018610 con 0x7f750419cd60 2024-03-29T05:07:37.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.870+0000 7f750bdd4700 1 -- 172.21.15.142:0/1332785782 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75041974d0 con 0x7f750419cd60 2024-03-29T05:07:37.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.872+0000 7f750bdd4700 1 -- 172.21.15.142:0/1332785782 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7504109130 con 0x7f750419cd60 2024-03-29T05:07:37.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.872+0000 7f7502ffd700 1 -- 172.21.15.142:0/1332785782 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f74f800f5c0 con 0x7f750419cd60 2024-03-29T05:07:37.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.872+0000 7f7502ffd700 1 --2- 172.21.15.142:0/1332785782 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f74ec06dd60 0x7f74ec070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:37.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.873+0000 7f7502ffd700 1 -- 172.21.15.142:0/1332785782 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f74f808dc00 con 0x7f750419cd60 2024-03-29T05:07:37.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.873+0000 7f7509b70700 1 --2- 172.21.15.142:0/1332785782 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f74ec06dd60 0x7f74ec070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:37.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.873+0000 7f7509b70700 1 --2- 172.21.15.142:0/1332785782 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f74ec06dd60 0x7f74ec070220 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009fd0 tx=0x7f74fc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:37.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:37.877+0000 7f7502ffd700 1 -- 172.21.15.142:0/1332785782 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f74f8093020 con 0x7f750419cd60 2024-03-29T05:07:38.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.161+0000 7f750bdd4700 1 -- 172.21.15.142:0/1332785782 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7504197e00 con 0x7f750419cd60 2024-03-29T05:07:38.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.161+0000 7f7502ffd700 1 -- 172.21.15.142:0/1332785782 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f74f805ab50 con 0x7f750419cd60 2024-03-29T05:07:38.171 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:07:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:07:38.171 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:07:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:07:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:07:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:07:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:07:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:07:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:07:38.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:07:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:07:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:07:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:07:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:07:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:07:38.173 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:07:38.173 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:07:38.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.163+0000 7f750bdd4700 1 -- 172.21.15.142:0/1332785782 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f74ec06dd60 msgr2=0x7f74ec070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:38.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.163+0000 7f750bdd4700 1 --2- 172.21.15.142:0/1332785782 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f74ec06dd60 0x7f74ec070220 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009fd0 tx=0x7f74fc01a040 comp rx=0 tx=0).stop 2024-03-29T05:07:38.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.163+0000 7f750bdd4700 1 -- 172.21.15.142:0/1332785782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750419cd60 msgr2=0x7f75041968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:38.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.163+0000 7f750bdd4700 1 --2- 172.21.15.142:0/1332785782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750419cd60 0x7f75041968a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f74f800c8f0 tx=0x7f74f800cc00 comp rx=0 tx=0).stop 2024-03-29T05:07:38.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.163+0000 7f750bdd4700 1 -- 172.21.15.142:0/1332785782 shutdown_connections 2024-03-29T05:07:38.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.163+0000 7f750bdd4700 1 --2- 172.21.15.142:0/1332785782 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f74ec06dd60 0x7f74ec070220 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.163+0000 7f750bdd4700 1 --2- 172.21.15.142:0/1332785782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7504069020 0x7f750419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.163+0000 7f750bdd4700 1 --2- 172.21.15.142:0/1332785782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750419cd60 0x7f75041968a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.163+0000 7f750bdd4700 1 -- 172.21.15.142:0/1332785782 >> 172.21.15.142:0/1332785782 conn(0x7f75040755a0 msgr2=0x7f75040fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:38.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.164+0000 7f750bdd4700 1 -- 172.21.15.142:0/1332785782 shutdown_connections 2024-03-29T05:07:38.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.164+0000 7f750bdd4700 1 -- 172.21.15.142:0/1332785782 wait complete. 2024-03-29T05:07:38.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.267+0000 7fe966eb9700 1 -- 172.21.15.142:0/3122336145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe960101770 msgr2=0x7fe9601057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:38.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.267+0000 7fe966eb9700 1 --2- 172.21.15.142:0/3122336145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe960101770 0x7fe9601057c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009910 tx=0x7fe94c009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:38.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.267+0000 7fe966eb9700 1 -- 172.21.15.142:0/3122336145 shutdown_connections 2024-03-29T05:07:38.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.267+0000 7fe966eb9700 1 --2- 172.21.15.142:0/3122336145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe960101770 0x7fe9601057c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.267+0000 7fe966eb9700 1 --2- 172.21.15.142:0/3122336145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe960100dc0 0x7fe9601011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.267+0000 7fe966eb9700 1 -- 172.21.15.142:0/3122336145 >> 172.21.15.142:0/3122336145 conn(0x7fe9600fc650 msgr2=0x7fe9600fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:38.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.267+0000 7fe966eb9700 1 -- 172.21.15.142:0/3122336145 shutdown_connections 2024-03-29T05:07:38.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.267+0000 7fe966eb9700 1 -- 172.21.15.142:0/3122336145 wait complete. 2024-03-29T05:07:38.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.267+0000 7fe966eb9700 1 Processor -- start 2024-03-29T05:07:38.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.267+0000 7fe966eb9700 1 -- start start 2024-03-29T05:07:38.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe966eb9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe960100dc0 0x7fe960073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe966eb9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe960101770 0x7fe9600735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe966eb9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe960073b10 con 0x7fe960100dc0 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe966eb9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe960073c50 con 0x7fe960101770 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe964c55700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe960100dc0 0x7fe960073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe964c55700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe960100dc0 0x7fe960073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:56860/0 (socket says 172.21.15.142:56860) 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe964c55700 1 -- 172.21.15.142:0/1667862769 learned_addr learned my addr 172.21.15.142:0/1667862769 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe95ffff700 1 --2- 172.21.15.142:0/1667862769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe960101770 0x7fe9600735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe964c55700 1 -- 172.21.15.142:0/1667862769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe960101770 msgr2=0x7fe9600735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe964c55700 1 --2- 172.21.15.142:0/1667862769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe960101770 0x7fe9600735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe964c55700 1 -- 172.21.15.142:0/1667862769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c0095f0 con 0x7fe960100dc0 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe964c55700 1 --2- 172.21.15.142:0/1667862769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe960100dc0 0x7fe960073090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe95400d8d0 tx=0x7fe95400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe95dffb700 1 -- 172.21.15.142:0/1667862769 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9540098a0 con 0x7fe960100dc0 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe95dffb700 1 -- 172.21.15.142:0/1667862769 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe954010460 con 0x7fe960100dc0 2024-03-29T05:07:38.279 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe95dffb700 1 -- 172.21.15.142:0/1667862769 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe95400f5f0 con 0x7fe960100dc0 2024-03-29T05:07:38.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.268+0000 7fe966eb9700 1 -- 172.21.15.142:0/1667862769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9601a2960 con 0x7fe960100dc0 2024-03-29T05:07:38.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.269+0000 7fe966eb9700 1 -- 172.21.15.142:0/1667862769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9601a2d70 con 0x7fe960100dc0 2024-03-29T05:07:38.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.269+0000 7fe966eb9700 1 -- 172.21.15.142:0/1667862769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9601091c0 con 0x7fe960100dc0 2024-03-29T05:07:38.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.270+0000 7fe95dffb700 1 -- 172.21.15.142:0/1667862769 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fe954009a00 con 0x7fe960100dc0 2024-03-29T05:07:38.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.270+0000 7fe95dffb700 1 --2- 172.21.15.142:0/1667862769 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe95006dd60 0x7fe950070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:38.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.270+0000 7fe95dffb700 1 -- 172.21.15.142:0/1667862769 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fe95408da90 con 0x7fe960100dc0 2024-03-29T05:07:38.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.270+0000 7fe95ffff700 1 --2- 172.21.15.142:0/1667862769 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe95006dd60 0x7fe950070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:38.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.271+0000 7fe95ffff700 1 --2- 172.21.15.142:0/1667862769 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe95006dd60 0x7fe950070220 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe960074880 tx=0x7fe94c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:38.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.275+0000 7fe95dffb700 1 -- 172.21.15.142:0/1667862769 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe954056f30 con 0x7fe960100dc0 2024-03-29T05:07:38.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.515+0000 7fe966eb9700 1 -- 172.21.15.142:0/1667862769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe96004ea90 con 0x7fe960100dc0 2024-03-29T05:07:38.525 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:38 smithi142 ceph-mon[112475]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:38.525 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:38 smithi142 ceph-mon[112475]: from='client.35078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:38.525 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:38 smithi142 ceph-mon[112475]: from='client.35082 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:38.526 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:38 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1332785782' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:07:38.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.516+0000 7fe95dffb700 1 -- 172.21.15.142:0/1667862769 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fe95405a5f0 con 0x7fe960100dc0 2024-03-29T05:07:38.526 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:07:38.526 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:07:38.526 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:07:38.526 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:07:38.526 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:07:38.527 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:07:38.528 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:07:38.529 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:07:38.529 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:07:38.529 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:07:38.529 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:07:38.529 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:07:38.529 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:07:38.529 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:07:38.529 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:07:38.529 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:07:38.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.518+0000 7fe966eb9700 1 -- 172.21.15.142:0/1667862769 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe95006dd60 msgr2=0x7fe950070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:38.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.518+0000 7fe966eb9700 1 --2- 172.21.15.142:0/1667862769 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe95006dd60 0x7fe950070220 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe960074880 tx=0x7fe94c01a040 comp rx=0 tx=0).stop 2024-03-29T05:07:38.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.518+0000 7fe966eb9700 1 -- 172.21.15.142:0/1667862769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe960100dc0 msgr2=0x7fe960073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:38.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.518+0000 7fe966eb9700 1 --2- 172.21.15.142:0/1667862769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe960100dc0 0x7fe960073090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe95400d8d0 tx=0x7fe95400dbe0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.518+0000 7fe966eb9700 1 -- 172.21.15.142:0/1667862769 shutdown_connections 2024-03-29T05:07:38.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.518+0000 7fe966eb9700 1 --2- 172.21.15.142:0/1667862769 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe95006dd60 0x7fe950070220 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.530 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.518+0000 7fe966eb9700 1 --2- 172.21.15.142:0/1667862769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe960100dc0 0x7fe960073090 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.518+0000 7fe966eb9700 1 --2- 172.21.15.142:0/1667862769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe960101770 0x7fe9600735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.518+0000 7fe966eb9700 1 -- 172.21.15.142:0/1667862769 >> 172.21.15.142:0/1667862769 conn(0x7fe9600fc650 msgr2=0x7fe9600fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:38.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.518+0000 7fe966eb9700 1 -- 172.21.15.142:0/1667862769 shutdown_connections 2024-03-29T05:07:38.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.518+0000 7fe966eb9700 1 -- 172.21.15.142:0/1667862769 wait complete. 2024-03-29T05:07:38.531 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:07:38.634 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.624+0000 7f30de04f700 1 -- 172.21.15.142:0/4168491084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8100dc0 msgr2=0x7f30d81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:38.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.624+0000 7f30de04f700 1 --2- 172.21.15.142:0/4168491084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8100dc0 0x7f30d81011a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009910 tx=0x7f30c8009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:38.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.624+0000 7f30de04f700 1 -- 172.21.15.142:0/4168491084 shutdown_connections 2024-03-29T05:07:38.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.624+0000 7f30de04f700 1 --2- 172.21.15.142:0/4168491084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30d8101770 0x7f30d81057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.624+0000 7f30de04f700 1 --2- 172.21.15.142:0/4168491084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8100dc0 0x7f30d81011a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.624+0000 7f30de04f700 1 -- 172.21.15.142:0/4168491084 >> 172.21.15.142:0/4168491084 conn(0x7f30d80fc650 msgr2=0x7f30d80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:38.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.624+0000 7f30de04f700 1 -- 172.21.15.142:0/4168491084 shutdown_connections 2024-03-29T05:07:38.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.624+0000 7f30de04f700 1 -- 172.21.15.142:0/4168491084 wait complete. 2024-03-29T05:07:38.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.624+0000 7f30de04f700 1 Processor -- start 2024-03-29T05:07:38.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.624+0000 7f30de04f700 1 -- start start 2024-03-29T05:07:38.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30de04f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30d8100dc0 0x7f30d819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:38.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30de04f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8101770 0x7f30d819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:38.635 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30de04f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d819d110 con 0x7f30d8101770 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30de04f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d81965c0 con 0x7f30d8100dc0 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30d6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8101770 0x7f30d819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30d6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8101770 0x7f30d819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52962/0 (socket says 172.21.15.142:52962) 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30d6ffd700 1 -- 172.21.15.142:0/3324521023 learned_addr learned my addr 172.21.15.142:0/3324521023 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30d77fe700 1 --2- 172.21.15.142:0/3324521023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30d8100dc0 0x7f30d819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30d6ffd700 1 -- 172.21.15.142:0/3324521023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30d8100dc0 msgr2=0x7f30d819c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30d6ffd700 1 --2- 172.21.15.142:0/3324521023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30d8100dc0 0x7f30d819c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30d6ffd700 1 -- 172.21.15.142:0/3324521023 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c80095f0 con 0x7f30d8101770 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30d6ffd700 1 --2- 172.21.15.142:0/3324521023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8101770 0x7f30d819ca80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00c8f0 tx=0x7f30cc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30d4ff9700 1 -- 172.21.15.142:0/3324521023 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30cc0078a0 con 0x7f30d8101770 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30de04f700 1 -- 172.21.15.142:0/3324521023 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d81968a0 con 0x7f30d8101770 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30d4ff9700 1 -- 172.21.15.142:0/3324521023 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30cc00f450 con 0x7f30d8101770 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30d4ff9700 1 -- 172.21.15.142:0/3324521023 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30cc018610 con 0x7f30d8101770 2024-03-29T05:07:38.636 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.625+0000 7f30de04f700 1 -- 172.21.15.142:0/3324521023 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d8196df0 con 0x7f30d8101770 2024-03-29T05:07:38.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.626+0000 7f30de04f700 1 -- 172.21.15.142:0/3324521023 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d8109160 con 0x7f30d8101770 2024-03-29T05:07:38.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.627+0000 7f30d4ff9700 1 -- 172.21.15.142:0/3324521023 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f30cc00f5c0 con 0x7f30d8101770 2024-03-29T05:07:38.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.627+0000 7f30d4ff9700 1 --2- 172.21.15.142:0/3324521023 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f30c006dd60 0x7f30c0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:38.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.627+0000 7f30d4ff9700 1 -- 172.21.15.142:0/3324521023 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f30cc08dbd0 con 0x7f30d8101770 2024-03-29T05:07:38.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.627+0000 7f30d77fe700 1 --2- 172.21.15.142:0/3324521023 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f30c006dd60 0x7f30c0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:38.638 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.628+0000 7f30d77fe700 1 --2- 172.21.15.142:0/3324521023 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f30c006dd60 0x7f30c0070220 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f30c8006010 tx=0x7f30c801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:38.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.632+0000 7f30d4ff9700 1 -- 172.21.15.142:0/3324521023 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f30cc05ab20 con 0x7f30d8101770 2024-03-29T05:07:38.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:38 smithi152 ceph-mon[88186]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:38.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:38 smithi152 ceph-mon[88186]: from='client.35078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:38.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:38 smithi152 ceph-mon[88186]: from='client.35082 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:38.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:38 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1332785782' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:07:38.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.845+0000 7f30de04f700 1 -- 172.21.15.142:0/3324521023 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30d81970c0 con 0x7f30c006dd60 2024-03-29T05:07:38.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.846+0000 7f30d4ff9700 1 -- 172.21.15.142:0/3324521023 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f30d81970c0 con 0x7f30c006dd60 2024-03-29T05:07:38.855 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:07:38.856 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:07:38.856 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:07:38.856 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:07:38.856 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:07:38.856 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:07:38.856 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:07:38.856 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:07:38.856 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:07:38.856 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:07:38.856 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:07:38.856 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:07:38.856 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:07:38.857 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:07:38.857 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:07:38.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.848+0000 7f30de04f700 1 -- 172.21.15.142:0/3324521023 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f30c006dd60 msgr2=0x7f30c0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:38.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.848+0000 7f30de04f700 1 --2- 172.21.15.142:0/3324521023 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f30c006dd60 0x7f30c0070220 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f30c8006010 tx=0x7f30c801a040 comp rx=0 tx=0).stop 2024-03-29T05:07:38.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.848+0000 7f30de04f700 1 -- 172.21.15.142:0/3324521023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8101770 msgr2=0x7f30d819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:38.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.848+0000 7f30de04f700 1 --2- 172.21.15.142:0/3324521023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8101770 0x7f30d819ca80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00c8f0 tx=0x7f30cc00cc00 comp rx=0 tx=0).stop 2024-03-29T05:07:38.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.848+0000 7f30de04f700 1 -- 172.21.15.142:0/3324521023 shutdown_connections 2024-03-29T05:07:38.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.848+0000 7f30de04f700 1 --2- 172.21.15.142:0/3324521023 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f30c006dd60 0x7f30c0070220 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.848+0000 7f30de04f700 1 --2- 172.21.15.142:0/3324521023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30d8100dc0 0x7f30d819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.848+0000 7f30de04f700 1 --2- 172.21.15.142:0/3324521023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d8101770 0x7f30d819ca80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.848+0000 7f30de04f700 1 -- 172.21.15.142:0/3324521023 >> 172.21.15.142:0/3324521023 conn(0x7f30d80fc650 msgr2=0x7f30d80fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:38.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.849+0000 7f30de04f700 1 -- 172.21.15.142:0/3324521023 shutdown_connections 2024-03-29T05:07:38.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.849+0000 7f30de04f700 1 -- 172.21.15.142:0/3324521023 wait complete. 2024-03-29T05:07:38.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.952+0000 7f1025f95700 1 -- 172.21.15.142:0/1861845644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1020101750 msgr2=0x7f10201057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:38.961 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.952+0000 7f1025f95700 1 --2- 172.21.15.142:0/1861845644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1020101750 0x7f10201057a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1014009910 tx=0x7f1014009c20 comp rx=0 tx=0).stop 2024-03-29T05:07:38.962 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.952+0000 7f1025f95700 1 -- 172.21.15.142:0/1861845644 shutdown_connections 2024-03-29T05:07:38.962 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.952+0000 7f1025f95700 1 --2- 172.21.15.142:0/1861845644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1020101750 0x7f10201057a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.962 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.952+0000 7f1025f95700 1 --2- 172.21.15.142:0/1861845644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1020100da0 0x7f1020101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.962 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.952+0000 7f1025f95700 1 -- 172.21.15.142:0/1861845644 >> 172.21.15.142:0/1861845644 conn(0x7f10200fc650 msgr2=0x7f10200fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:38.962 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.952+0000 7f1025f95700 1 -- 172.21.15.142:0/1861845644 shutdown_connections 2024-03-29T05:07:38.962 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.952+0000 7f1025f95700 1 -- 172.21.15.142:0/1861845644 wait complete. 2024-03-29T05:07:38.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.953+0000 7f1025f95700 1 Processor -- start 2024-03-29T05:07:38.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.953+0000 7f1025f95700 1 -- start start 2024-03-29T05:07:38.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.953+0000 7f1025f95700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1020100da0 0x7f102019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:38.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.953+0000 7f1025f95700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1020101750 0x7f102019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:38.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.953+0000 7f1025f95700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102019d0e0 con 0x7f1020101750 2024-03-29T05:07:38.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.953+0000 7f1025f95700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1020196590 con 0x7f1020100da0 2024-03-29T05:07:38.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.953+0000 7f101effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1020101750 0x7f102019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:38.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.953+0000 7f101effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1020101750 0x7f102019ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52980/0 (socket says 172.21.15.142:52980) 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.953+0000 7f101effd700 1 -- 172.21.15.142:0/2053925978 learned_addr learned my addr 172.21.15.142:0/2053925978 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.953+0000 7f101f7fe700 1 --2- 172.21.15.142:0/2053925978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1020100da0 0x7f102019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.954+0000 7f101effd700 1 -- 172.21.15.142:0/2053925978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1020100da0 msgr2=0x7f102019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.954+0000 7f101effd700 1 --2- 172.21.15.142:0/2053925978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1020100da0 0x7f102019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.954+0000 7f101effd700 1 -- 172.21.15.142:0/2053925978 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10140095f0 con 0x7f1020101750 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.954+0000 7f101f7fe700 1 --2- 172.21.15.142:0/2053925978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1020100da0 0x7f102019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.954+0000 7f101effd700 1 --2- 172.21.15.142:0/2053925978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1020101750 0x7f102019ca50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f10140059d0 tx=0x7f101400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.954+0000 7f101cff9700 1 -- 172.21.15.142:0/2053925978 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f101401d070 con 0x7f1020101750 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.954+0000 7f101cff9700 1 -- 172.21.15.142:0/2053925978 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f101400b610 con 0x7f1020101750 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.954+0000 7f1025f95700 1 -- 172.21.15.142:0/2053925978 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1020196810 con 0x7f1020101750 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.954+0000 7f101cff9700 1 -- 172.21.15.142:0/2053925978 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1014022470 con 0x7f1020101750 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.954+0000 7f1025f95700 1 -- 172.21.15.142:0/2053925978 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1020196d00 con 0x7f1020101750 2024-03-29T05:07:38.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.955+0000 7f1025f95700 1 -- 172.21.15.142:0/2053925978 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10201091a0 con 0x7f1020101750 2024-03-29T05:07:38.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.955+0000 7f101cff9700 1 -- 172.21.15.142:0/2053925978 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f101400fc70 con 0x7f1020101750 2024-03-29T05:07:38.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.956+0000 7f101cff9700 1 --2- 172.21.15.142:0/2053925978 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f100806dd60 0x7f1008070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:07:38.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.956+0000 7f101cff9700 1 -- 172.21.15.142:0/2053925978 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f101408eff0 con 0x7f1020101750 2024-03-29T05:07:38.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.956+0000 7f101f7fe700 1 --2- 172.21.15.142:0/2053925978 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f100806dd60 0x7f1008070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:07:38.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.956+0000 7f101f7fe700 1 --2- 172.21.15.142:0/2053925978 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f100806dd60 0x7f1008070220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1010009be0 tx=0x7f1010009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:07:38.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:38.960+0000 7f101cff9700 1 -- 172.21.15.142:0/2053925978 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f101405bfc0 con 0x7f1020101750 2024-03-29T05:07:39.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.234+0000 7f1025f95700 1 -- 172.21.15.142:0/2053925978 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f102002ccf0 con 0x7f1020101750 2024-03-29T05:07:39.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.234+0000 7f101cff9700 1 -- 172.21.15.142:0/2053925978 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f101405bb50 con 0x7f1020101750 2024-03-29T05:07:39.244 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:07:39.244 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:07:39.244 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:07:39.244 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:07:39.244 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:07:39.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.236+0000 7f1025f95700 1 -- 172.21.15.142:0/2053925978 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f100806dd60 msgr2=0x7f1008070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:39.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.236+0000 7f1025f95700 1 --2- 172.21.15.142:0/2053925978 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f100806dd60 0x7f1008070220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1010009be0 tx=0x7f1010009260 comp rx=0 tx=0).stop 2024-03-29T05:07:39.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.236+0000 7f1025f95700 1 -- 172.21.15.142:0/2053925978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1020101750 msgr2=0x7f102019ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:07:39.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.236+0000 7f1025f95700 1 --2- 172.21.15.142:0/2053925978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1020101750 0x7f102019ca50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f10140059d0 tx=0x7f101400f740 comp rx=0 tx=0).stop 2024-03-29T05:07:39.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.236+0000 7f1025f95700 1 -- 172.21.15.142:0/2053925978 shutdown_connections 2024-03-29T05:07:39.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.236+0000 7f1025f95700 1 --2- 172.21.15.142:0/2053925978 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f100806dd60 0x7f1008070220 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:39.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.236+0000 7f1025f95700 1 --2- 172.21.15.142:0/2053925978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1020100da0 0x7f102019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:39.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.236+0000 7f1025f95700 1 --2- 172.21.15.142:0/2053925978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1020101750 0x7f102019ca50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:07:39.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.236+0000 7f1025f95700 1 -- 172.21.15.142:0/2053925978 >> 172.21.15.142:0/2053925978 conn(0x7f10200fc650 msgr2=0x7f10200fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:07:39.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.237+0000 7f1025f95700 1 -- 172.21.15.142:0/2053925978 shutdown_connections 2024-03-29T05:07:39.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:07:39.237+0000 7f1025f95700 1 -- 172.21.15.142:0/2053925978 wait complete. 2024-03-29T05:07:39.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:39 smithi142 ceph-mon[112475]: pgmap v580: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:39.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:39 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1667862769' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:07:39.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:39 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2053925978' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:07:39.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:39 smithi152 ceph-mon[88186]: pgmap v580: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:39.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:39 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1667862769' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:07:39.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:39 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2053925978' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:07:40.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:40 smithi152 ceph-mon[88186]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:40.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:40 smithi142 ceph-mon[112475]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:07:41.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:41 smithi152 ceph-mon[88186]: pgmap v581: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:41.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:41 smithi142 ceph-mon[112475]: pgmap v581: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:43.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:43 smithi152 ceph-mon[88186]: pgmap v582: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:43.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:43 smithi142 ceph-mon[112475]: pgmap v582: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:45.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:45 smithi152 ceph-mon[88186]: pgmap v583: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:45.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:07:45.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:45 smithi142 ceph-mon[112475]: pgmap v583: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:45.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:07:47.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:47 smithi152 ceph-mon[88186]: pgmap v584: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:47.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:47 smithi142 ceph-mon[112475]: pgmap v584: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:49.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:49 smithi152 ceph-mon[88186]: pgmap v585: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:49.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:49 smithi142 ceph-mon[112475]: pgmap v585: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:51.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:51 smithi152 ceph-mon[88186]: pgmap v586: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:51.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:51 smithi142 ceph-mon[112475]: pgmap v586: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:53.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:53 smithi142 ceph-mon[112475]: pgmap v587: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:53.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:53 smithi152 ceph-mon[88186]: pgmap v587: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:55.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:55 smithi152 ceph-mon[88186]: pgmap v588: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:55.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:55 smithi142 ceph-mon[112475]: pgmap v588: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:57.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:57 smithi152 ceph-mon[88186]: pgmap v589: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:57.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:57 smithi142 ceph-mon[112475]: pgmap v589: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:59.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:07:59 smithi152 ceph-mon[88186]: pgmap v590: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:07:59.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:07:59 smithi142 ceph-mon[112475]: pgmap v590: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:00.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:08:00.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:08:01.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:01 smithi152 ceph-mon[88186]: pgmap v591: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:01.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:01 smithi142 ceph-mon[112475]: pgmap v591: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:03.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:03 smithi142 ceph-mon[112475]: pgmap v592: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:03.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:03 smithi152 ceph-mon[88186]: pgmap v592: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:05 smithi152 ceph-mon[88186]: pgmap v593: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:05.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:05 smithi142 ceph-mon[112475]: pgmap v593: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:07.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:07 smithi152 ceph-mon[88186]: pgmap v594: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:07.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:07 smithi142 ceph-mon[112475]: pgmap v594: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:09.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.353+0000 7eff74485700 1 -- 172.21.15.142:0/52665566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6c0739d0 msgr2=0x7eff6c10cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:09.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.353+0000 7eff74485700 1 --2- 172.21.15.142:0/52665566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6c0739d0 0x7eff6c10cee0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009910 tx=0x7eff5c009c20 comp rx=0 tx=0).stop 2024-03-29T05:08:09.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.354+0000 7eff74485700 1 -- 172.21.15.142:0/52665566 shutdown_connections 2024-03-29T05:08:09.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.354+0000 7eff74485700 1 --2- 172.21.15.142:0/52665566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6c0739d0 0x7eff6c10cee0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.354+0000 7eff74485700 1 --2- 172.21.15.142:0/52665566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff6c0730b0 0x7eff6c073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.354+0000 7eff74485700 1 -- 172.21.15.142:0/52665566 >> 172.21.15.142:0/52665566 conn(0x7eff6c0fc650 msgr2=0x7eff6c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:09.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.354+0000 7eff74485700 1 -- 172.21.15.142:0/52665566 shutdown_connections 2024-03-29T05:08:09.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.354+0000 7eff74485700 1 -- 172.21.15.142:0/52665566 wait complete. 2024-03-29T05:08:09.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.354+0000 7eff74485700 1 Processor -- start 2024-03-29T05:08:09.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.354+0000 7eff74485700 1 -- start start 2024-03-29T05:08:09.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.354+0000 7eff74485700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6c0730b0 0x7eff6c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:09.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.354+0000 7eff74485700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff6c0739d0 0x7eff6c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:09.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.354+0000 7eff74485700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c19f270 con 0x7eff6c0730b0 2024-03-29T05:08:09.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.354+0000 7eff74485700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c198790 con 0x7eff6c0739d0 2024-03-29T05:08:09.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff72221700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6c0730b0 0x7eff6c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:09.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff72221700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6c0730b0 0x7eff6c19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44058/0 (socket says 172.21.15.142:44058) 2024-03-29T05:08:09.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff72221700 1 -- 172.21.15.142:0/1668371228 learned_addr learned my addr 172.21.15.142:0/1668371228 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:09.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff71a20700 1 --2- 172.21.15.142:0/1668371228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff6c0739d0 0x7eff6c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:09.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff72221700 1 -- 172.21.15.142:0/1668371228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff6c0739d0 msgr2=0x7eff6c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:09.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff72221700 1 --2- 172.21.15.142:0/1668371228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff6c0739d0 0x7eff6c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff72221700 1 -- 172.21.15.142:0/1668371228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c0095f0 con 0x7eff6c0730b0 2024-03-29T05:08:09.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff72221700 1 --2- 172.21.15.142:0/1668371228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6c0730b0 0x7eff6c19e6a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7eff6800d8d0 tx=0x7eff6800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:09.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff637fe700 1 -- 172.21.15.142:0/1668371228 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff680098a0 con 0x7eff6c0730b0 2024-03-29T05:08:09.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff637fe700 1 -- 172.21.15.142:0/1668371228 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff68010460 con 0x7eff6c0730b0 2024-03-29T05:08:09.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff74485700 1 -- 172.21.15.142:0/1668371228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6c198a70 con 0x7eff6c0730b0 2024-03-29T05:08:09.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff637fe700 1 -- 172.21.15.142:0/1668371228 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff6800f5f0 con 0x7eff6c0730b0 2024-03-29T05:08:09.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.355+0000 7eff74485700 1 -- 172.21.15.142:0/1668371228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c198fc0 con 0x7eff6c0730b0 2024-03-29T05:08:09.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.356+0000 7eff74485700 1 -- 172.21.15.142:0/1668371228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6c10a5e0 con 0x7eff6c0730b0 2024-03-29T05:08:09.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.357+0000 7eff637fe700 1 -- 172.21.15.142:0/1668371228 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7eff68009a00 con 0x7eff6c0730b0 2024-03-29T05:08:09.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.357+0000 7eff637fe700 1 --2- 172.21.15.142:0/1668371228 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7eff5806dd60 0x7eff58070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:09.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.357+0000 7eff71a20700 1 --2- 172.21.15.142:0/1668371228 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7eff5806dd60 0x7eff58070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:09.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.357+0000 7eff637fe700 1 -- 172.21.15.142:0/1668371228 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7eff6808da90 con 0x7eff6c0730b0 2024-03-29T05:08:09.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.358+0000 7eff71a20700 1 --2- 172.21.15.142:0/1668371228 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7eff5806dd60 0x7eff58070220 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7eff6c199c00 tx=0x7eff5c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:09.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.362+0000 7eff637fe700 1 -- 172.21.15.142:0/1668371228 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7eff6805aa60 con 0x7eff6c0730b0 2024-03-29T05:08:09.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.576+0000 7eff74485700 1 -- 172.21.15.142:0/1668371228 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff6c1992a0 con 0x7eff5806dd60 2024-03-29T05:08:09.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.577+0000 7eff637fe700 1 -- 172.21.15.142:0/1668371228 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7eff6c1992a0 con 0x7eff5806dd60 2024-03-29T05:08:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.579+0000 7eff74485700 1 -- 172.21.15.142:0/1668371228 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7eff5806dd60 msgr2=0x7eff58070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.579+0000 7eff74485700 1 --2- 172.21.15.142:0/1668371228 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7eff5806dd60 0x7eff58070220 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7eff6c199c00 tx=0x7eff5c01a040 comp rx=0 tx=0).stop 2024-03-29T05:08:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.579+0000 7eff74485700 1 -- 172.21.15.142:0/1668371228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6c0730b0 msgr2=0x7eff6c19e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.579+0000 7eff74485700 1 --2- 172.21.15.142:0/1668371228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6c0730b0 0x7eff6c19e6a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7eff6800d8d0 tx=0x7eff6800dbe0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.580+0000 7eff74485700 1 -- 172.21.15.142:0/1668371228 shutdown_connections 2024-03-29T05:08:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.580+0000 7eff74485700 1 --2- 172.21.15.142:0/1668371228 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7eff5806dd60 0x7eff58070220 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.580+0000 7eff74485700 1 --2- 172.21.15.142:0/1668371228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff6c0730b0 0x7eff6c19e6a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.580+0000 7eff74485700 1 --2- 172.21.15.142:0/1668371228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff6c0739d0 0x7eff6c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.580+0000 7eff74485700 1 -- 172.21.15.142:0/1668371228 >> 172.21.15.142:0/1668371228 conn(0x7eff6c0fc650 msgr2=0x7eff6c107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.580+0000 7eff74485700 1 -- 172.21.15.142:0/1668371228 shutdown_connections 2024-03-29T05:08:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.580+0000 7eff74485700 1 -- 172.21.15.142:0/1668371228 wait complete. 2024-03-29T05:08:09.599 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:08:09.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:09 smithi142 ceph-mon[112475]: pgmap v595: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:09.694 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.685+0000 7f707e0e1700 1 -- 172.21.15.142:0/2353344811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70781026a0 msgr2=0x7f707810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:09.694 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.685+0000 7f707e0e1700 1 --2- 172.21.15.142:0/2353344811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70781026a0 0x7f707810ab90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f706c009910 tx=0x7f706c009c20 comp rx=0 tx=0).stop 2024-03-29T05:08:09.694 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.685+0000 7f707e0e1700 1 -- 172.21.15.142:0/2353344811 shutdown_connections 2024-03-29T05:08:09.694 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.685+0000 7f707e0e1700 1 --2- 172.21.15.142:0/2353344811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70781026a0 0x7f707810ab90 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.694 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.685+0000 7f707e0e1700 1 --2- 172.21.15.142:0/2353344811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7078101d80 0x7f7078102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.685+0000 7f707e0e1700 1 -- 172.21.15.142:0/2353344811 >> 172.21.15.142:0/2353344811 conn(0x7f70780fb550 msgr2=0x7f70780fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:09.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.685+0000 7f707e0e1700 1 -- 172.21.15.142:0/2353344811 shutdown_connections 2024-03-29T05:08:09.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.685+0000 7f707e0e1700 1 -- 172.21.15.142:0/2353344811 wait complete. 2024-03-29T05:08:09.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.685+0000 7f707e0e1700 1 Processor -- start 2024-03-29T05:08:09.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f707e0e1700 1 -- start start 2024-03-29T05:08:09.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f707e0e1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7078101d80 0x7f707819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:09.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f707e0e1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70781026a0 0x7f707819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:09.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f707e0e1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707819d290 con 0x7f70781026a0 2024-03-29T05:08:09.696 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f707e0e1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7078196740 con 0x7f7078101d80 2024-03-29T05:08:09.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f7076ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70781026a0 0x7f707819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:09.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f7076ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70781026a0 0x7f707819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44084/0 (socket says 172.21.15.142:44084) 2024-03-29T05:08:09.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f7076ffd700 1 -- 172.21.15.142:0/2806892404 learned_addr learned my addr 172.21.15.142:0/2806892404 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:09.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f70777fe700 1 --2- 172.21.15.142:0/2806892404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7078101d80 0x7f707819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:09.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f7076ffd700 1 -- 172.21.15.142:0/2806892404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7078101d80 msgr2=0x7f707819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:09.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f7076ffd700 1 --2- 172.21.15.142:0/2806892404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7078101d80 0x7f707819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f7076ffd700 1 -- 172.21.15.142:0/2806892404 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c0095f0 con 0x7f70781026a0 2024-03-29T05:08:09.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f7076ffd700 1 --2- 172.21.15.142:0/2806892404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70781026a0 0x7f707819cc00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f706c0092e0 tx=0x7f706c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:09.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f7074ff9700 1 -- 172.21.15.142:0/2806892404 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f706c01d070 con 0x7f70781026a0 2024-03-29T05:08:09.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f7074ff9700 1 -- 172.21.15.142:0/2806892404 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f706c00f460 con 0x7f70781026a0 2024-03-29T05:08:09.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.686+0000 7f707e0e1700 1 -- 172.21.15.142:0/2806892404 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70781969c0 con 0x7f70781026a0 2024-03-29T05:08:09.697 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.687+0000 7f7074ff9700 1 -- 172.21.15.142:0/2806892404 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f706c0175f0 con 0x7f70781026a0 2024-03-29T05:08:09.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.687+0000 7f707e0e1700 1 -- 172.21.15.142:0/2806892404 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7078196eb0 con 0x7f70781026a0 2024-03-29T05:08:09.698 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.687+0000 7f707e0e1700 1 -- 172.21.15.142:0/2806892404 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7078108290 con 0x7f70781026a0 2024-03-29T05:08:09.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.688+0000 7f7074ff9700 1 -- 172.21.15.142:0/2806892404 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f706c00f5d0 con 0x7f70781026a0 2024-03-29T05:08:09.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.688+0000 7f7074ff9700 1 --2- 172.21.15.142:0/2806892404 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f706006dd60 0x7f7060070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:09.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.689+0000 7f7074ff9700 1 -- 172.21.15.142:0/2806892404 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f706c08f1f0 con 0x7f70781026a0 2024-03-29T05:08:09.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.689+0000 7f70777fe700 1 --2- 172.21.15.142:0/2806892404 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f706006dd60 0x7f7060070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:09.699 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.689+0000 7f70777fe700 1 --2- 172.21.15.142:0/2806892404 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f706006dd60 0x7f7060070220 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7068006fd0 tx=0x7f7068008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:09.702 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.693+0000 7f7074ff9700 1 -- 172.21.15.142:0/2806892404 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f706c05c140 con 0x7f70781026a0 2024-03-29T05:08:09.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:09 smithi152 ceph-mon[88186]: pgmap v595: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:09.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.906+0000 7f707e0e1700 1 -- 172.21.15.142:0/2806892404 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f707802cce0 con 0x7f706006dd60 2024-03-29T05:08:09.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.907+0000 7f7074ff9700 1 -- 172.21.15.142:0/2806892404 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f707802cce0 con 0x7f706006dd60 2024-03-29T05:08:09.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.909+0000 7f707e0e1700 1 -- 172.21.15.142:0/2806892404 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f706006dd60 msgr2=0x7f7060070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:09.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.909+0000 7f707e0e1700 1 --2- 172.21.15.142:0/2806892404 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f706006dd60 0x7f7060070220 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7068006fd0 tx=0x7f7068008040 comp rx=0 tx=0).stop 2024-03-29T05:08:09.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.909+0000 7f707e0e1700 1 -- 172.21.15.142:0/2806892404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70781026a0 msgr2=0x7f707819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:09.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.909+0000 7f707e0e1700 1 --2- 172.21.15.142:0/2806892404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70781026a0 0x7f707819cc00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f706c0092e0 tx=0x7f706c00be20 comp rx=0 tx=0).stop 2024-03-29T05:08:09.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.909+0000 7f707e0e1700 1 -- 172.21.15.142:0/2806892404 shutdown_connections 2024-03-29T05:08:09.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.910+0000 7f707e0e1700 1 --2- 172.21.15.142:0/2806892404 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f706006dd60 0x7f7060070220 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.920 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.910+0000 7f707e0e1700 1 --2- 172.21.15.142:0/2806892404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7078101d80 0x7f707819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.920 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.910+0000 7f707e0e1700 1 --2- 172.21.15.142:0/2806892404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70781026a0 0x7f707819cc00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:09.920 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.910+0000 7f707e0e1700 1 -- 172.21.15.142:0/2806892404 >> 172.21.15.142:0/2806892404 conn(0x7f70780fb550 msgr2=0x7f70781053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:09.920 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.910+0000 7f707e0e1700 1 -- 172.21.15.142:0/2806892404 shutdown_connections 2024-03-29T05:08:09.920 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:09.910+0000 7f707e0e1700 1 -- 172.21.15.142:0/2806892404 wait complete. 2024-03-29T05:08:10.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.013+0000 7f4ed39fe700 1 -- 172.21.15.142:0/2194261676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc100da0 msgr2=0x7f4ecc101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:10.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.013+0000 7f4ed39fe700 1 --2- 172.21.15.142:0/2194261676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc100da0 0x7f4ecc101180 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800b0a0 tx=0x7f4ec800b3b0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.013+0000 7f4ed39fe700 1 -- 172.21.15.142:0/2194261676 shutdown_connections 2024-03-29T05:08:10.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.013+0000 7f4ed39fe700 1 --2- 172.21.15.142:0/2194261676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ecc101750 0x7f4ecc1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.022 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.013+0000 7f4ed39fe700 1 --2- 172.21.15.142:0/2194261676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc100da0 0x7f4ecc101180 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.013+0000 7f4ed39fe700 1 -- 172.21.15.142:0/2194261676 >> 172.21.15.142:0/2194261676 conn(0x7f4ecc0fc630 msgr2=0x7f4ecc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:10.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.013+0000 7f4ed39fe700 1 -- 172.21.15.142:0/2194261676 shutdown_connections 2024-03-29T05:08:10.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.013+0000 7f4ed39fe700 1 -- 172.21.15.142:0/2194261676 wait complete. 2024-03-29T05:08:10.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.014+0000 7f4ed39fe700 1 Processor -- start 2024-03-29T05:08:10.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.014+0000 7f4ed39fe700 1 -- start start 2024-03-29T05:08:10.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.014+0000 7f4ed39fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc100da0 0x7f4ecc19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:10.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.014+0000 7f4ed39fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ecc101750 0x7f4ecc19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:10.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.014+0000 7f4ed39fe700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc19d210 con 0x7f4ecc100da0 2024-03-29T05:08:10.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.014+0000 7f4ed39fe700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc1966c0 con 0x7f4ecc101750 2024-03-29T05:08:10.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.014+0000 7f4ed179a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc100da0 0x7f4ecc19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.014+0000 7f4ed179a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc100da0 0x7f4ecc19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44100/0 (socket says 172.21.15.142:44100) 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.014+0000 7f4ed179a700 1 -- 172.21.15.142:0/205555206 learned_addr learned my addr 172.21.15.142:0/205555206 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.015+0000 7f4ed0f99700 1 --2- 172.21.15.142:0/205555206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ecc101750 0x7f4ecc19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.015+0000 7f4ed179a700 1 -- 172.21.15.142:0/205555206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ecc101750 msgr2=0x7f4ecc19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.015+0000 7f4ed179a700 1 --2- 172.21.15.142:0/205555206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ecc101750 0x7f4ecc19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.015+0000 7f4ed179a700 1 -- 172.21.15.142:0/205555206 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec8009d00 con 0x7f4ecc100da0 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.015+0000 7f4ed179a700 1 --2- 172.21.15.142:0/205555206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc100da0 0x7f4ecc19c640 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8000f80 tx=0x7f4ec800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.015+0000 7f4ec27fc700 1 -- 172.21.15.142:0/205555206 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec800e040 con 0x7f4ecc100da0 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.015+0000 7f4ec27fc700 1 -- 172.21.15.142:0/205555206 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ec8003e60 con 0x7f4ecc100da0 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.015+0000 7f4ed39fe700 1 -- 172.21.15.142:0/205555206 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc196940 con 0x7f4ecc100da0 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.015+0000 7f4ec27fc700 1 -- 172.21.15.142:0/205555206 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec801d910 con 0x7f4ecc100da0 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.015+0000 7f4ed39fe700 1 -- 172.21.15.142:0/205555206 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc196e30 con 0x7f4ecc100da0 2024-03-29T05:08:10.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.016+0000 7f4ed39fe700 1 -- 172.21.15.142:0/205555206 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc109140 con 0x7f4ecc100da0 2024-03-29T05:08:10.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.017+0000 7f4ec27fc700 1 -- 172.21.15.142:0/205555206 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f4ec8004820 con 0x7f4ecc100da0 2024-03-29T05:08:10.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.017+0000 7f4ec27fc700 1 --2- 172.21.15.142:0/205555206 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4eb806dd60 0x7f4eb8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:10.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.017+0000 7f4ec27fc700 1 -- 172.21.15.142:0/205555206 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f4ec808f5e0 con 0x7f4ecc100da0 2024-03-29T05:08:10.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.017+0000 7f4ed0f99700 1 --2- 172.21.15.142:0/205555206 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4eb806dd60 0x7f4eb8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:10.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.018+0000 7f4ed0f99700 1 --2- 172.21.15.142:0/205555206 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4eb806dd60 0x7f4eb8070220 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc006fd0 tx=0x7f4ebc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:10.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.022+0000 7f4ec27fc700 1 -- 172.21.15.142:0/205555206 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4ec805c530 con 0x7f4ecc100da0 2024-03-29T05:08:10.234 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.225+0000 7f4ed39fe700 1 -- 172.21.15.142:0/205555206 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ecc197180 con 0x7f4eb806dd60 2024-03-29T05:08:10.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.236+0000 7f4ec27fc700 1 -- 172.21.15.142:0/205555206 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4ecc197180 con 0x7f4eb806dd60 2024-03-29T05:08:10.244 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (20m) 13m ago 31m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (31m) 13m ago 31m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (30m) 12m ago 30m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (17m) 13m ago 31m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (17m) 12m ago 30m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (19m) 13m ago 31m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (13m) 13m ago 25m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (13m) 13m ago 25m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (12m) 12m ago 25m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (12m) 12m ago 25m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (21m) 13m ago 32m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (20m) 12m ago 30m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:08:10.245 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (18m) 13m ago 33m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:08:10.246 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (17m) 12m ago 30m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:08:10.246 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (20m) 13m ago 31m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:08:10.246 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (20m) 12m ago 30m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:08:10.246 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (16m) 13m ago 29m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:08:10.246 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (16m) 13m ago 29m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:08:10.246 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (15m) 13m ago 28m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:08:10.246 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (14m) 12m ago 28m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:08:10.246 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (14m) 12m ago 27m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:08:10.246 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (13m) 12m ago 27m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:08:10.246 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (20m) 13m ago 31m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:08:10.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.238+0000 7f4ed39fe700 1 -- 172.21.15.142:0/205555206 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4eb806dd60 msgr2=0x7f4eb8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:10.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.238+0000 7f4ed39fe700 1 --2- 172.21.15.142:0/205555206 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4eb806dd60 0x7f4eb8070220 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc006fd0 tx=0x7f4ebc008040 comp rx=0 tx=0).stop 2024-03-29T05:08:10.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.238+0000 7f4ed39fe700 1 -- 172.21.15.142:0/205555206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc100da0 msgr2=0x7f4ecc19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:10.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.238+0000 7f4ed39fe700 1 --2- 172.21.15.142:0/205555206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc100da0 0x7f4ecc19c640 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8000f80 tx=0x7f4ec800cdd0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.238+0000 7f4ed39fe700 1 -- 172.21.15.142:0/205555206 shutdown_connections 2024-03-29T05:08:10.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.238+0000 7f4ed39fe700 1 --2- 172.21.15.142:0/205555206 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4eb806dd60 0x7f4eb8070220 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.238+0000 7f4ed39fe700 1 --2- 172.21.15.142:0/205555206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc100da0 0x7f4ecc19c640 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.238+0000 7f4ed39fe700 1 --2- 172.21.15.142:0/205555206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ecc101750 0x7f4ecc19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.238+0000 7f4ed39fe700 1 -- 172.21.15.142:0/205555206 >> 172.21.15.142:0/205555206 conn(0x7f4ecc0fc630 msgr2=0x7f4ecc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:10.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.238+0000 7f4ed39fe700 1 -- 172.21.15.142:0/205555206 shutdown_connections 2024-03-29T05:08:10.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.238+0000 7f4ed39fe700 1 -- 172.21.15.142:0/205555206 wait complete. 2024-03-29T05:08:10.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.341+0000 7fda76cdd700 1 -- 172.21.15.142:0/1872421328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda700ffbd0 msgr2=0x7fda70100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:10.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.341+0000 7fda76cdd700 1 --2- 172.21.15.142:0/1872421328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda700ffbd0 0x7fda70100050 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009910 tx=0x7fda5c009c20 comp rx=0 tx=0).stop 2024-03-29T05:08:10.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.341+0000 7fda76cdd700 1 -- 172.21.15.142:0/1872421328 shutdown_connections 2024-03-29T05:08:10.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.341+0000 7fda76cdd700 1 --2- 172.21.15.142:0/1872421328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda700ffbd0 0x7fda70100050 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.341+0000 7fda76cdd700 1 --2- 172.21.15.142:0/1872421328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda70105a40 0x7fda70105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.341+0000 7fda76cdd700 1 -- 172.21.15.142:0/1872421328 >> 172.21.15.142:0/1872421328 conn(0x7fda700755a0 msgr2=0x7fda700759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:10.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.341+0000 7fda76cdd700 1 -- 172.21.15.142:0/1872421328 shutdown_connections 2024-03-29T05:08:10.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.341+0000 7fda76cdd700 1 -- 172.21.15.142:0/1872421328 wait complete. 2024-03-29T05:08:10.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda76cdd700 1 Processor -- start 2024-03-29T05:08:10.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda76cdd700 1 -- start start 2024-03-29T05:08:10.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda76cdd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda700ffbd0 0x7fda70195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:10.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda76cdd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda70105a40 0x7fda70195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:10.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda76cdd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda70196240 con 0x7fda700ffbd0 2024-03-29T05:08:10.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda76cdd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda70199fd0 con 0x7fda70105a40 2024-03-29T05:08:10.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda74a79700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda700ffbd0 0x7fda70195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:10.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda74a79700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda700ffbd0 0x7fda70195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44114/0 (socket says 172.21.15.142:44114) 2024-03-29T05:08:10.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda74a79700 1 -- 172.21.15.142:0/156093436 learned_addr learned my addr 172.21.15.142:0/156093436 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:10.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda6ffff700 1 --2- 172.21.15.142:0/156093436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda70105a40 0x7fda70195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:10.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda74a79700 1 -- 172.21.15.142:0/156093436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda70105a40 msgr2=0x7fda70195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:10.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda74a79700 1 --2- 172.21.15.142:0/156093436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda70105a40 0x7fda70195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.342+0000 7fda74a79700 1 -- 172.21.15.142:0/156093436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c0095f0 con 0x7fda700ffbd0 2024-03-29T05:08:10.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.343+0000 7fda74a79700 1 --2- 172.21.15.142:0/156093436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda700ffbd0 0x7fda70195620 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fda64007b50 tx=0x7fda64007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:10.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.343+0000 7fda6dffb700 1 -- 172.21.15.142:0/156093436 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda64011070 con 0x7fda700ffbd0 2024-03-29T05:08:10.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.343+0000 7fda6dffb700 1 -- 172.21.15.142:0/156093436 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda6400c2e0 con 0x7fda700ffbd0 2024-03-29T05:08:10.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.343+0000 7fda76cdd700 1 -- 172.21.15.142:0/156093436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7019a250 con 0x7fda700ffbd0 2024-03-29T05:08:10.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.343+0000 7fda6dffb700 1 -- 172.21.15.142:0/156093436 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda6400e740 con 0x7fda700ffbd0 2024-03-29T05:08:10.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.343+0000 7fda76cdd700 1 -- 172.21.15.142:0/156093436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7019a7a0 con 0x7fda700ffbd0 2024-03-29T05:08:10.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.344+0000 7fda76cdd700 1 -- 172.21.15.142:0/156093436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7004ea90 con 0x7fda700ffbd0 2024-03-29T05:08:10.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.344+0000 7fda6dffb700 1 -- 172.21.15.142:0/156093436 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fda6400e8a0 con 0x7fda700ffbd0 2024-03-29T05:08:10.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.345+0000 7fda6dffb700 1 --2- 172.21.15.142:0/156093436 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fda6006dd60 0x7fda60070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:10.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.345+0000 7fda6dffb700 1 -- 172.21.15.142:0/156093436 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fda6408f1b0 con 0x7fda700ffbd0 2024-03-29T05:08:10.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.345+0000 7fda6ffff700 1 --2- 172.21.15.142:0/156093436 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fda6006dd60 0x7fda60070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:10.355 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.346+0000 7fda6ffff700 1 --2- 172.21.15.142:0/156093436 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fda6006dd60 0x7fda60070220 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fda70196c40 tx=0x7fda5c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:10.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.350+0000 7fda6dffb700 1 -- 172.21.15.142:0/156093436 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fda64058650 con 0x7fda700ffbd0 2024-03-29T05:08:10.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:10 smithi142 ceph-mon[112475]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:10.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.631+0000 7fda76cdd700 1 -- 172.21.15.142:0/156093436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fda7019ac80 con 0x7fda700ffbd0 2024-03-29T05:08:10.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.631+0000 7fda6dffb700 1 -- 172.21.15.142:0/156093436 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fda6405bd10 con 0x7fda700ffbd0 2024-03-29T05:08:10.640 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:08:10.640 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:08:10.640 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:08:10.641 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:08:10.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.633+0000 7fda76cdd700 1 -- 172.21.15.142:0/156093436 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fda6006dd60 msgr2=0x7fda60070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:10.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.634+0000 7fda76cdd700 1 --2- 172.21.15.142:0/156093436 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fda6006dd60 0x7fda60070220 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fda70196c40 tx=0x7fda5c01a040 comp rx=0 tx=0).stop 2024-03-29T05:08:10.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.634+0000 7fda76cdd700 1 -- 172.21.15.142:0/156093436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda700ffbd0 msgr2=0x7fda70195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:10.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.634+0000 7fda76cdd700 1 --2- 172.21.15.142:0/156093436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda700ffbd0 0x7fda70195620 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fda64007b50 tx=0x7fda64007e60 comp rx=0 tx=0).stop 2024-03-29T05:08:10.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.634+0000 7fda76cdd700 1 -- 172.21.15.142:0/156093436 shutdown_connections 2024-03-29T05:08:10.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.634+0000 7fda76cdd700 1 --2- 172.21.15.142:0/156093436 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fda6006dd60 0x7fda60070220 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.634+0000 7fda76cdd700 1 --2- 172.21.15.142:0/156093436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda700ffbd0 0x7fda70195620 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.634+0000 7fda76cdd700 1 --2- 172.21.15.142:0/156093436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda70105a40 0x7fda70195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.634+0000 7fda76cdd700 1 -- 172.21.15.142:0/156093436 >> 172.21.15.142:0/156093436 conn(0x7fda700755a0 msgr2=0x7fda700fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:10.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.634+0000 7fda76cdd700 1 -- 172.21.15.142:0/156093436 shutdown_connections 2024-03-29T05:08:10.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.634+0000 7fda76cdd700 1 -- 172.21.15.142:0/156093436 wait complete. 2024-03-29T05:08:10.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.738+0000 7f6d7d969700 1 -- 172.21.15.142:0/3582237274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d780fef70 msgr2=0x7f6d780ff3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:10.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.738+0000 7f6d7d969700 1 --2- 172.21.15.142:0/3582237274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d780fef70 0x7f6d780ff3f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009910 tx=0x7f6d6c009c20 comp rx=0 tx=0).stop 2024-03-29T05:08:10.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.738+0000 7f6d7d969700 1 -- 172.21.15.142:0/3582237274 shutdown_connections 2024-03-29T05:08:10.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.738+0000 7f6d7d969700 1 --2- 172.21.15.142:0/3582237274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d780fef70 0x7f6d780ff3f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.738+0000 7f6d7d969700 1 --2- 172.21.15.142:0/3582237274 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d78105b50 0x7f6d78105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.738+0000 7f6d7d969700 1 -- 172.21.15.142:0/3582237274 >> 172.21.15.142:0/3582237274 conn(0x7f6d78075e30 msgr2=0x7f6d780fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:10.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.738+0000 7f6d7d969700 1 -- 172.21.15.142:0/3582237274 shutdown_connections 2024-03-29T05:08:10.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.738+0000 7f6d7d969700 1 -- 172.21.15.142:0/3582237274 wait complete. 2024-03-29T05:08:10.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.738+0000 7f6d7d969700 1 Processor -- start 2024-03-29T05:08:10.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.739+0000 7f6d7d969700 1 -- start start 2024-03-29T05:08:10.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.739+0000 7f6d7d969700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d780fef70 0x7f6d7819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:10.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.739+0000 7f6d7d969700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d78105b50 0x7f6d7819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:10.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.739+0000 7f6d7d969700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7819d1c0 con 0x7f6d780fef70 2024-03-29T05:08:10.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.739+0000 7f6d7d969700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d78196670 con 0x7f6d78105b50 2024-03-29T05:08:10.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.739+0000 7f6d76ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d780fef70 0x7f6d7819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:10.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.739+0000 7f6d76ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d780fef70 0x7f6d7819c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44134/0 (socket says 172.21.15.142:44134) 2024-03-29T05:08:10.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.739+0000 7f6d76ffd700 1 -- 172.21.15.142:0/1502388342 learned_addr learned my addr 172.21.15.142:0/1502388342 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:10.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.739+0000 7f6d767fc700 1 --2- 172.21.15.142:0/1502388342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d78105b50 0x7f6d7819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:10.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.739+0000 7f6d76ffd700 1 -- 172.21.15.142:0/1502388342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d78105b50 msgr2=0x7f6d7819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:10.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.739+0000 7f6d76ffd700 1 --2- 172.21.15.142:0/1502388342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d78105b50 0x7f6d7819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.739+0000 7f6d76ffd700 1 -- 172.21.15.142:0/1502388342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c0095f0 con 0x7f6d780fef70 2024-03-29T05:08:10.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.740+0000 7f6d76ffd700 1 --2- 172.21.15.142:0/1502388342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d780fef70 0x7f6d7819c5f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000ba50 tx=0x7f6d7000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:10.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.740+0000 7f6d7c967700 1 -- 172.21.15.142:0/1502388342 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d7000c900 con 0x7f6d780fef70 2024-03-29T05:08:10.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.740+0000 7f6d7c967700 1 -- 172.21.15.142:0/1502388342 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d70014440 con 0x7f6d780fef70 2024-03-29T05:08:10.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.740+0000 7f6d7d969700 1 -- 172.21.15.142:0/1502388342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d78196950 con 0x7f6d780fef70 2024-03-29T05:08:10.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.740+0000 7f6d7c967700 1 -- 172.21.15.142:0/1502388342 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d700075a0 con 0x7f6d780fef70 2024-03-29T05:08:10.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.740+0000 7f6d7d969700 1 -- 172.21.15.142:0/1502388342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d78196ea0 con 0x7f6d780fef70 2024-03-29T05:08:10.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.741+0000 7f6d7d969700 1 -- 172.21.15.142:0/1502388342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d78109120 con 0x7f6d780fef70 2024-03-29T05:08:10.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.741+0000 7f6d7c967700 1 -- 172.21.15.142:0/1502388342 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6d700145b0 con 0x7f6d780fef70 2024-03-29T05:08:10.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.742+0000 7f6d7c967700 1 --2- 172.21.15.142:0/1502388342 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6d6006dd60 0x7f6d60070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:10.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.742+0000 7f6d7c967700 1 -- 172.21.15.142:0/1502388342 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6d70006e40 con 0x7f6d780fef70 2024-03-29T05:08:10.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.742+0000 7f6d767fc700 1 --2- 172.21.15.142:0/1502388342 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6d6006dd60 0x7f6d60070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:10.752 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.742+0000 7f6d767fc700 1 --2- 172.21.15.142:0/1502388342 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6d6006dd60 0x7f6d60070220 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6d78107250 tx=0x7f6d6c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:10.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.746+0000 7f6d7c967700 1 -- 172.21.15.142:0/1502388342 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6d7005a390 con 0x7f6d780fef70 2024-03-29T05:08:10.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:10 smithi152 ceph-mon[88186]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:10.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.984+0000 7f6d7d969700 1 -- 172.21.15.142:0/1502388342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6d78197180 con 0x7f6d780fef70 2024-03-29T05:08:10.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.984+0000 7f6d7c967700 1 -- 172.21.15.142:0/1502388342 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f6d70078e60 con 0x7f6d780fef70 2024-03-29T05:08:10.993 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:08:10.994 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:08:10.995 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:08:10.995 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:08:10.995 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:08:10.995 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:08:10.995 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:08:10.995 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:08:10.995 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:08:10.995 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:08:10.995 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:08:10.995 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:08:10.995 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:08:10.995 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:08:10.996 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:08:10.997 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:08:10.997 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:08:10.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.987+0000 7f6d7d969700 1 -- 172.21.15.142:0/1502388342 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6d6006dd60 msgr2=0x7f6d60070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:10.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.987+0000 7f6d7d969700 1 --2- 172.21.15.142:0/1502388342 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6d6006dd60 0x7f6d60070220 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6d78107250 tx=0x7f6d6c01a040 comp rx=0 tx=0).stop 2024-03-29T05:08:10.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.987+0000 7f6d7d969700 1 -- 172.21.15.142:0/1502388342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d780fef70 msgr2=0x7f6d7819c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:10.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.987+0000 7f6d7d969700 1 --2- 172.21.15.142:0/1502388342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d780fef70 0x7f6d7819c5f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000ba50 tx=0x7f6d7000bd60 comp rx=0 tx=0).stop 2024-03-29T05:08:10.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.987+0000 7f6d7d969700 1 -- 172.21.15.142:0/1502388342 shutdown_connections 2024-03-29T05:08:10.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.987+0000 7f6d7d969700 1 --2- 172.21.15.142:0/1502388342 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6d6006dd60 0x7f6d60070220 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.987+0000 7f6d7d969700 1 --2- 172.21.15.142:0/1502388342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d780fef70 0x7f6d7819c5f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.987+0000 7f6d7d969700 1 --2- 172.21.15.142:0/1502388342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d78105b50 0x7f6d7819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:10.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.988+0000 7f6d7d969700 1 -- 172.21.15.142:0/1502388342 >> 172.21.15.142:0/1502388342 conn(0x7f6d78075e30 msgr2=0x7f6d781044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:10.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.988+0000 7f6d7d969700 1 -- 172.21.15.142:0/1502388342 shutdown_connections 2024-03-29T05:08:10.998 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:10.988+0000 7f6d7d969700 1 -- 172.21.15.142:0/1502388342 wait complete. 2024-03-29T05:08:10.998 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:08:11.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.091+0000 7f6821e1b700 1 -- 172.21.15.142:0/1867130240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681c101770 msgr2=0x7f681c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:11.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.091+0000 7f6821e1b700 1 --2- 172.21.15.142:0/1867130240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681c101770 0x7f681c1057c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6810009910 tx=0x7f6810009c20 comp rx=0 tx=0).stop 2024-03-29T05:08:11.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.091+0000 7f6821e1b700 1 -- 172.21.15.142:0/1867130240 shutdown_connections 2024-03-29T05:08:11.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.091+0000 7f6821e1b700 1 --2- 172.21.15.142:0/1867130240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681c101770 0x7f681c1057c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.091+0000 7f6821e1b700 1 --2- 172.21.15.142:0/1867130240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f681c100dc0 0x7f681c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.091+0000 7f6821e1b700 1 -- 172.21.15.142:0/1867130240 >> 172.21.15.142:0/1867130240 conn(0x7f681c0fc650 msgr2=0x7f681c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:11.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.091+0000 7f6821e1b700 1 -- 172.21.15.142:0/1867130240 shutdown_connections 2024-03-29T05:08:11.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.091+0000 7f6821e1b700 1 -- 172.21.15.142:0/1867130240 wait complete. 2024-03-29T05:08:11.102 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.092+0000 7f6821e1b700 1 Processor -- start 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.092+0000 7f6821e1b700 1 -- start start 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.092+0000 7f6821e1b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681c100dc0 0x7f681c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.092+0000 7f6821e1b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f681c101770 0x7f681c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.092+0000 7f6821e1b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c19d130 con 0x7f681c100dc0 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.092+0000 7f6821e1b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c1965e0 con 0x7f681c101770 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.092+0000 7f681b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681c100dc0 0x7f681c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.092+0000 7f681b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681c100dc0 0x7f681c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44150/0 (socket says 172.21.15.142:44150) 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.092+0000 7f681b7fe700 1 -- 172.21.15.142:0/2761776644 learned_addr learned my addr 172.21.15.142:0/2761776644 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.092+0000 7f681affd700 1 --2- 172.21.15.142:0/2761776644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f681c101770 0x7f681c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.093+0000 7f681b7fe700 1 -- 172.21.15.142:0/2761776644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f681c101770 msgr2=0x7f681c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.093+0000 7f681b7fe700 1 --2- 172.21.15.142:0/2761776644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f681c101770 0x7f681c19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.093+0000 7f681b7fe700 1 -- 172.21.15.142:0/2761776644 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100095f0 con 0x7f681c100dc0 2024-03-29T05:08:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.093+0000 7f681b7fe700 1 --2- 172.21.15.142:0/2761776644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681c100dc0 0x7f681c19c560 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f680c00c8a0 tx=0x7f680c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:11.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.093+0000 7f6818ff9700 1 -- 172.21.15.142:0/2761776644 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f680c0078e0 con 0x7f681c100dc0 2024-03-29T05:08:11.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.093+0000 7f6818ff9700 1 -- 172.21.15.142:0/2761776644 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f680c00f450 con 0x7f681c100dc0 2024-03-29T05:08:11.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.093+0000 7f6821e1b700 1 -- 172.21.15.142:0/2761776644 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681c1968c0 con 0x7f681c100dc0 2024-03-29T05:08:11.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.093+0000 7f6818ff9700 1 -- 172.21.15.142:0/2761776644 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f680c00e5e0 con 0x7f681c100dc0 2024-03-29T05:08:11.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.093+0000 7f6821e1b700 1 -- 172.21.15.142:0/2761776644 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681c196de0 con 0x7f681c100dc0 2024-03-29T05:08:11.104 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.094+0000 7f6821e1b700 1 -- 172.21.15.142:0/2761776644 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681c1091c0 con 0x7f681c100dc0 2024-03-29T05:08:11.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.095+0000 7f6818ff9700 1 -- 172.21.15.142:0/2761776644 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f680c00f5c0 con 0x7f681c100dc0 2024-03-29T05:08:11.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.095+0000 7f6818ff9700 1 --2- 172.21.15.142:0/2761776644 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f680406dd60 0x7f6804070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:11.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.095+0000 7f6818ff9700 1 -- 172.21.15.142:0/2761776644 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f680c08dc60 con 0x7f681c100dc0 2024-03-29T05:08:11.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.095+0000 7f681affd700 1 --2- 172.21.15.142:0/2761776644 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f680406dd60 0x7f6804070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:11.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.095+0000 7f681affd700 1 --2- 172.21.15.142:0/2761776644 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f680406dd60 0x7f6804070220 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6810006010 tx=0x7f681001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:11.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.100+0000 7f6818ff9700 1 -- 172.21.15.142:0/2761776644 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f680c05abb0 con 0x7f681c100dc0 2024-03-29T05:08:11.321 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.312+0000 7f6821e1b700 1 -- 172.21.15.142:0/2761776644 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f681c197090 con 0x7f680406dd60 2024-03-29T05:08:11.322 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.313+0000 7f6818ff9700 1 -- 172.21.15.142:0/2761776644 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f681c197090 con 0x7f680406dd60 2024-03-29T05:08:11.322 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:08:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:08:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:08:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:08:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:08:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:08:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:08:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:08:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:08:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:08:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:08:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:08:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:08:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:08:11.323 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:08:11.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.316+0000 7f6821e1b700 1 -- 172.21.15.142:0/2761776644 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f680406dd60 msgr2=0x7f6804070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:11.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.316+0000 7f6821e1b700 1 --2- 172.21.15.142:0/2761776644 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f680406dd60 0x7f6804070220 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6810006010 tx=0x7f681001a040 comp rx=0 tx=0).stop 2024-03-29T05:08:11.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.316+0000 7f6821e1b700 1 -- 172.21.15.142:0/2761776644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681c100dc0 msgr2=0x7f681c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:11.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.316+0000 7f6821e1b700 1 --2- 172.21.15.142:0/2761776644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681c100dc0 0x7f681c19c560 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f680c00c8a0 tx=0x7f680c00cbb0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.316+0000 7f6821e1b700 1 -- 172.21.15.142:0/2761776644 shutdown_connections 2024-03-29T05:08:11.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.316+0000 7f6821e1b700 1 --2- 172.21.15.142:0/2761776644 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f680406dd60 0x7f6804070220 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.316+0000 7f6821e1b700 1 --2- 172.21.15.142:0/2761776644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681c100dc0 0x7f681c19c560 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.316+0000 7f6821e1b700 1 --2- 172.21.15.142:0/2761776644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f681c101770 0x7f681c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.316+0000 7f6821e1b700 1 -- 172.21.15.142:0/2761776644 >> 172.21.15.142:0/2761776644 conn(0x7f681c0fc650 msgr2=0x7f681c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:11.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.316+0000 7f6821e1b700 1 -- 172.21.15.142:0/2761776644 shutdown_connections 2024-03-29T05:08:11.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.316+0000 7f6821e1b700 1 -- 172.21.15.142:0/2761776644 wait complete. 2024-03-29T05:08:11.430 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.420+0000 7f044c750700 1 -- 172.21.15.142:0/3783318067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0444068410 msgr2=0x7f04440687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.420+0000 7f044c750700 1 --2- 172.21.15.142:0/3783318067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0444068410 0x7f04440687f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0434009910 tx=0x7f0434009c20 comp rx=0 tx=0).stop 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.420+0000 7f044c750700 1 -- 172.21.15.142:0/3783318067 shutdown_connections 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.420+0000 7f044c750700 1 --2- 172.21.15.142:0/3783318067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0444068d30 0x7f044410d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.420+0000 7f044c750700 1 --2- 172.21.15.142:0/3783318067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0444068410 0x7f04440687f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.420+0000 7f044c750700 1 -- 172.21.15.142:0/3783318067 >> 172.21.15.142:0/3783318067 conn(0x7f04440755a0 msgr2=0x7f04440759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.420+0000 7f044c750700 1 -- 172.21.15.142:0/3783318067 shutdown_connections 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.420+0000 7f044c750700 1 -- 172.21.15.142:0/3783318067 wait complete. 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f044c750700 1 Processor -- start 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f044c750700 1 -- start start 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f044c750700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0444068410 0x7f044419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f044c750700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0444068d30 0x7f044419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f044c750700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044419f390 con 0x7f0444068d30 2024-03-29T05:08:11.431 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f044c750700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0444198840 con 0x7f0444068410 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f0449ceb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0444068d30 0x7f044419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f0449ceb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0444068d30 0x7f044419ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44180/0 (socket says 172.21.15.142:44180) 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f0449ceb700 1 -- 172.21.15.142:0/1391022813 learned_addr learned my addr 172.21.15.142:0/1391022813 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f044a4ec700 1 --2- 172.21.15.142:0/1391022813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0444068410 0x7f044419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f0449ceb700 1 -- 172.21.15.142:0/1391022813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0444068410 msgr2=0x7f044419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f0449ceb700 1 --2- 172.21.15.142:0/1391022813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0444068410 0x7f044419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f0449ceb700 1 -- 172.21.15.142:0/1391022813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04340095f0 con 0x7f0444068d30 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.421+0000 7f0449ceb700 1 --2- 172.21.15.142:0/1391022813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0444068d30 0x7f044419ed00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f044000b4b0 tx=0x7f044000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.422+0000 7f043b7fe700 1 -- 172.21.15.142:0/1391022813 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0440011800 con 0x7f0444068d30 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.422+0000 7f043b7fe700 1 -- 172.21.15.142:0/1391022813 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0440011e20 con 0x7f0444068d30 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.422+0000 7f043b7fe700 1 -- 172.21.15.142:0/1391022813 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f044000f2e0 con 0x7f0444068d30 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.422+0000 7f044c750700 1 -- 172.21.15.142:0/1391022813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0444198b20 con 0x7f0444068d30 2024-03-29T05:08:11.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.422+0000 7f044c750700 1 -- 172.21.15.142:0/1391022813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0444199070 con 0x7f0444068d30 2024-03-29T05:08:11.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.423+0000 7f044c750700 1 -- 172.21.15.142:0/1391022813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044410a9b0 con 0x7f0444068d30 2024-03-29T05:08:11.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.423+0000 7f043b7fe700 1 -- 172.21.15.142:0/1391022813 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f0440010380 con 0x7f0444068d30 2024-03-29T05:08:11.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.424+0000 7f043b7fe700 1 --2- 172.21.15.142:0/1391022813 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f043006dd60 0x7f0430070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:11.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.424+0000 7f043b7fe700 1 -- 172.21.15.142:0/1391022813 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f044008d890 con 0x7f0444068d30 2024-03-29T05:08:11.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.424+0000 7f044a4ec700 1 --2- 172.21.15.142:0/1391022813 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f043006dd60 0x7f0430070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:11.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.424+0000 7f044a4ec700 1 --2- 172.21.15.142:0/1391022813 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f043006dd60 0x7f0430070220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f04340059d0 tx=0x7f043401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:11.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.428+0000 7f043b7fe700 1 -- 172.21.15.142:0/1391022813 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f044005a7e0 con 0x7f0444068d30 2024-03-29T05:08:11.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:11 smithi142 ceph-mon[112475]: from='client.35106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:11.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:11 smithi142 ceph-mon[112475]: pgmap v596: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:11.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:11 smithi142 ceph-mon[112475]: from='client.35110 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:11.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:11 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/156093436' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:08:11.630 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:11 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1502388342' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:08:11.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.704+0000 7f044c750700 1 -- 172.21.15.142:0/1391022813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f044404ea90 con 0x7f0444068d30 2024-03-29T05:08:11.713 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.704+0000 7f043b7fe700 1 -- 172.21.15.142:0/1391022813 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f044005a370 con 0x7f0444068d30 2024-03-29T05:08:11.713 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:08:11.713 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:08:11.714 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:08:11.714 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:08:11.714 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:08:11.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.707+0000 7f044c750700 1 -- 172.21.15.142:0/1391022813 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f043006dd60 msgr2=0x7f0430070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:11.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.707+0000 7f044c750700 1 --2- 172.21.15.142:0/1391022813 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f043006dd60 0x7f0430070220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f04340059d0 tx=0x7f043401a040 comp rx=0 tx=0).stop 2024-03-29T05:08:11.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.707+0000 7f044c750700 1 -- 172.21.15.142:0/1391022813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0444068d30 msgr2=0x7f044419ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:11.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.707+0000 7f044c750700 1 --2- 172.21.15.142:0/1391022813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0444068d30 0x7f044419ed00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f044000b4b0 tx=0x7f044000b870 comp rx=0 tx=0).stop 2024-03-29T05:08:11.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.707+0000 7f044c750700 1 -- 172.21.15.142:0/1391022813 shutdown_connections 2024-03-29T05:08:11.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.707+0000 7f044c750700 1 --2- 172.21.15.142:0/1391022813 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f043006dd60 0x7f0430070220 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.707+0000 7f044c750700 1 --2- 172.21.15.142:0/1391022813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0444068410 0x7f044419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.707+0000 7f044c750700 1 --2- 172.21.15.142:0/1391022813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0444068d30 0x7f044419ed00 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:11.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.707+0000 7f044c750700 1 -- 172.21.15.142:0/1391022813 >> 172.21.15.142:0/1391022813 conn(0x7f04440755a0 msgr2=0x7f04440fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:11.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.707+0000 7f044c750700 1 -- 172.21.15.142:0/1391022813 shutdown_connections 2024-03-29T05:08:11.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:11.707+0000 7f044c750700 1 -- 172.21.15.142:0/1391022813 wait complete. 2024-03-29T05:08:11.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:11 smithi152 ceph-mon[88186]: from='client.35106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:11.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:11 smithi152 ceph-mon[88186]: pgmap v596: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:11.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:11 smithi152 ceph-mon[88186]: from='client.35110 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:11.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:11 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/156093436' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:08:11.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:11 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1502388342' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:08:12.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:12 smithi152 ceph-mon[88186]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:12.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:12 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1391022813' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:08:12.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:12 smithi142 ceph-mon[112475]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:12.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:12 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1391022813' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:08:13.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:13 smithi152 ceph-mon[88186]: pgmap v597: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:13.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:13 smithi142 ceph-mon[112475]: pgmap v597: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:15.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:15 smithi152 ceph-mon[88186]: pgmap v598: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:15.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:08:15.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:15 smithi142 ceph-mon[112475]: pgmap v598: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:15.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:08:17.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:17 smithi152 ceph-mon[88186]: pgmap v599: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:17.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:17 smithi142 ceph-mon[112475]: pgmap v599: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:19.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:19 smithi152 ceph-mon[88186]: pgmap v600: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:19.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:19 smithi142 ceph-mon[112475]: pgmap v600: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:21.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:21 smithi152 ceph-mon[88186]: pgmap v601: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:21.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:21 smithi142 ceph-mon[112475]: pgmap v601: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:23.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:23 smithi152 ceph-mon[88186]: pgmap v602: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:23.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:23 smithi142 ceph-mon[112475]: pgmap v602: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:25.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:25 smithi152 ceph-mon[88186]: pgmap v603: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:25.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:25 smithi142 ceph-mon[112475]: pgmap v603: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:27.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:27 smithi152 ceph-mon[88186]: pgmap v604: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:27.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:27 smithi142 ceph-mon[112475]: pgmap v604: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:29.781 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:29 smithi142 ceph-mon[112475]: pgmap v605: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:29.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:29 smithi152 ceph-mon[88186]: pgmap v605: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:30.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:08:30.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:08:31.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:31 smithi152 ceph-mon[88186]: pgmap v606: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:31.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:31 smithi142 ceph-mon[112475]: pgmap v606: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:33.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:33 smithi152 ceph-mon[88186]: pgmap v607: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:33.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:33 smithi142 ceph-mon[112475]: pgmap v607: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:35.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:35 smithi152 ceph-mon[88186]: pgmap v608: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:35.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:35 smithi142 ceph-mon[112475]: pgmap v608: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:37.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:37 smithi152 ceph-mon[88186]: pgmap v609: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:37.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:37 smithi142 ceph-mon[112475]: pgmap v609: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:39.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:39 smithi152 ceph-mon[88186]: pgmap v610: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:39.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:39 smithi142 ceph-mon[112475]: pgmap v610: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:41.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:41 smithi152 ceph-mon[88186]: pgmap v611: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:41.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.825+0000 7f6552fc3700 1 -- 172.21.15.142:0/2736984176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654c0ffbd0 msgr2=0x7f654c100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:41.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.825+0000 7f6552fc3700 1 --2- 172.21.15.142:0/2736984176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654c0ffbd0 0x7f654c100050 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6540009910 tx=0x7f6540009c20 comp rx=0 tx=0).stop 2024-03-29T05:08:41.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.825+0000 7f6552fc3700 1 -- 172.21.15.142:0/2736984176 shutdown_connections 2024-03-29T05:08:41.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.825+0000 7f6552fc3700 1 --2- 172.21.15.142:0/2736984176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654c0ffbd0 0x7f654c100050 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:41.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.825+0000 7f6552fc3700 1 --2- 172.21.15.142:0/2736984176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f654c105a40 0x7f654c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:41.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.825+0000 7f6552fc3700 1 -- 172.21.15.142:0/2736984176 >> 172.21.15.142:0/2736984176 conn(0x7f654c0755a0 msgr2=0x7f654c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:41.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.825+0000 7f6552fc3700 1 -- 172.21.15.142:0/2736984176 shutdown_connections 2024-03-29T05:08:41.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.825+0000 7f6552fc3700 1 -- 172.21.15.142:0/2736984176 wait complete. 2024-03-29T05:08:41.835 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:41 smithi142 ceph-mon[112475]: pgmap v611: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:41.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.826+0000 7f6552fc3700 1 Processor -- start 2024-03-29T05:08:41.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.826+0000 7f6552fc3700 1 -- start start 2024-03-29T05:08:41.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.826+0000 7f6552fc3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654c0ffbd0 0x7f654c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:41.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.826+0000 7f6552fc3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f654c105a40 0x7f654c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:41.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.826+0000 7f6552fc3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c19f1c0 con 0x7f654c0ffbd0 2024-03-29T05:08:41.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.826+0000 7f6552fc3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c198770 con 0x7f654c105a40 2024-03-29T05:08:41.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.826+0000 7f6550d5f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654c0ffbd0 0x7f654c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:41.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.826+0000 7f6550d5f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654c0ffbd0 0x7f654c19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43010/0 (socket says 172.21.15.142:43010) 2024-03-29T05:08:41.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.826+0000 7f6550d5f700 1 -- 172.21.15.142:0/3929038280 learned_addr learned my addr 172.21.15.142:0/3929038280 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:41.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.827+0000 7f654bfff700 1 --2- 172.21.15.142:0/3929038280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f654c105a40 0x7f654c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:41.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.827+0000 7f6550d5f700 1 -- 172.21.15.142:0/3929038280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f654c105a40 msgr2=0x7f654c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:41.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.827+0000 7f6550d5f700 1 --2- 172.21.15.142:0/3929038280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f654c105a40 0x7f654c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:41.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.827+0000 7f6550d5f700 1 -- 172.21.15.142:0/3929038280 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65400095f0 con 0x7f654c0ffbd0 2024-03-29T05:08:41.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.827+0000 7f654bfff700 1 --2- 172.21.15.142:0/3929038280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f654c105a40 0x7f654c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:08:41.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.827+0000 7f6550d5f700 1 --2- 172.21.15.142:0/3929038280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654c0ffbd0 0x7f654c19e680 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f654400cfd0 tx=0x7f654400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:41.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.827+0000 7f6549ffb700 1 -- 172.21.15.142:0/3929038280 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f654400ad70 con 0x7f654c0ffbd0 2024-03-29T05:08:41.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.827+0000 7f6549ffb700 1 -- 172.21.15.142:0/3929038280 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65440128e0 con 0x7f654c0ffbd0 2024-03-29T05:08:41.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.827+0000 7f6552fc3700 1 -- 172.21.15.142:0/3929038280 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654c198a50 con 0x7f654c0ffbd0 2024-03-29T05:08:41.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.827+0000 7f6549ffb700 1 -- 172.21.15.142:0/3929038280 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6544011880 con 0x7f654c0ffbd0 2024-03-29T05:08:41.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.827+0000 7f6552fc3700 1 -- 172.21.15.142:0/3929038280 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654c198fa0 con 0x7f654c0ffbd0 2024-03-29T05:08:41.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.828+0000 7f6552fc3700 1 -- 172.21.15.142:0/3929038280 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654c04ea90 con 0x7f654c0ffbd0 2024-03-29T05:08:41.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.829+0000 7f6549ffb700 1 -- 172.21.15.142:0/3929038280 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6544012420 con 0x7f654c0ffbd0 2024-03-29T05:08:41.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.829+0000 7f6549ffb700 1 --2- 172.21.15.142:0/3929038280 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f653406dd60 0x7f6534070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:41.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.829+0000 7f6549ffb700 1 -- 172.21.15.142:0/3929038280 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f654408dd10 con 0x7f654c0ffbd0 2024-03-29T05:08:41.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.829+0000 7f654bfff700 1 --2- 172.21.15.142:0/3929038280 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f653406dd60 0x7f6534070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:41.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.830+0000 7f654bfff700 1 --2- 172.21.15.142:0/3929038280 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f653406dd60 0x7f6534070220 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f654000b5c0 tx=0x7f6540005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:41.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:41.834+0000 7f6549ffb700 1 -- 172.21.15.142:0/3929038280 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f654405ace0 con 0x7f654c0ffbd0 2024-03-29T05:08:42.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.046+0000 7f6552fc3700 1 -- 172.21.15.142:0/3929038280 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f654c1993b0 con 0x7f653406dd60 2024-03-29T05:08:42.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.047+0000 7f6549ffb700 1 -- 172.21.15.142:0/3929038280 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f654c1993b0 con 0x7f653406dd60 2024-03-29T05:08:42.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.049+0000 7f6552fc3700 1 -- 172.21.15.142:0/3929038280 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f653406dd60 msgr2=0x7f6534070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:42.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.049+0000 7f6552fc3700 1 --2- 172.21.15.142:0/3929038280 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f653406dd60 0x7f6534070220 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f654000b5c0 tx=0x7f6540005960 comp rx=0 tx=0).stop 2024-03-29T05:08:42.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.049+0000 7f6552fc3700 1 -- 172.21.15.142:0/3929038280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654c0ffbd0 msgr2=0x7f654c19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:42.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.049+0000 7f6552fc3700 1 --2- 172.21.15.142:0/3929038280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654c0ffbd0 0x7f654c19e680 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f654400cfd0 tx=0x7f654400a340 comp rx=0 tx=0).stop 2024-03-29T05:08:42.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.049+0000 7f6552fc3700 1 -- 172.21.15.142:0/3929038280 shutdown_connections 2024-03-29T05:08:42.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.049+0000 7f6552fc3700 1 --2- 172.21.15.142:0/3929038280 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f653406dd60 0x7f6534070220 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.049+0000 7f6552fc3700 1 --2- 172.21.15.142:0/3929038280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f654c0ffbd0 0x7f654c19e680 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.049+0000 7f6552fc3700 1 --2- 172.21.15.142:0/3929038280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f654c105a40 0x7f654c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.049+0000 7f6552fc3700 1 -- 172.21.15.142:0/3929038280 >> 172.21.15.142:0/3929038280 conn(0x7f654c0755a0 msgr2=0x7f654c068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:42.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.049+0000 7f6552fc3700 1 -- 172.21.15.142:0/3929038280 shutdown_connections 2024-03-29T05:08:42.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.049+0000 7f6552fc3700 1 -- 172.21.15.142:0/3929038280 wait complete. 2024-03-29T05:08:42.068 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:08:42.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.154+0000 7fb080b63700 1 -- 172.21.15.142:0/1953264359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07c0ffbd0 msgr2=0x7fb07c100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:42.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.154+0000 7fb080b63700 1 --2- 172.21.15.142:0/1953264359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07c0ffbd0 0x7fb07c100050 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb06400b0a0 tx=0x7fb06400b3b0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.154+0000 7fb080b63700 1 -- 172.21.15.142:0/1953264359 shutdown_connections 2024-03-29T05:08:42.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.154+0000 7fb080b63700 1 --2- 172.21.15.142:0/1953264359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07c0ffbd0 0x7fb07c100050 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.154+0000 7fb080b63700 1 --2- 172.21.15.142:0/1953264359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb07c105a40 0x7fb07c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.154+0000 7fb080b63700 1 -- 172.21.15.142:0/1953264359 >> 172.21.15.142:0/1953264359 conn(0x7fb07c0755a0 msgr2=0x7fb07c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:42.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.154+0000 7fb080b63700 1 -- 172.21.15.142:0/1953264359 shutdown_connections 2024-03-29T05:08:42.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.154+0000 7fb080b63700 1 -- 172.21.15.142:0/1953264359 wait complete. 2024-03-29T05:08:42.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.155+0000 7fb080b63700 1 Processor -- start 2024-03-29T05:08:42.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.155+0000 7fb080b63700 1 -- start start 2024-03-29T05:08:42.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.155+0000 7fb080b63700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07c0ffbd0 0x7fb07c195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:42.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.155+0000 7fb080b63700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb07c105a40 0x7fb07c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:42.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.155+0000 7fb080b63700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c196240 con 0x7fb07c0ffbd0 2024-03-29T05:08:42.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.155+0000 7fb080b63700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c199fd0 con 0x7fb07c105a40 2024-03-29T05:08:42.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.155+0000 7fb07a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07c0ffbd0 0x7fb07c195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:42.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.155+0000 7fb07a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07c0ffbd0 0x7fb07c195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43022/0 (socket says 172.21.15.142:43022) 2024-03-29T05:08:42.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.155+0000 7fb07a59c700 1 -- 172.21.15.142:0/2249390521 learned_addr learned my addr 172.21.15.142:0/2249390521 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:42.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.156+0000 7fb079d9b700 1 --2- 172.21.15.142:0/2249390521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb07c105a40 0x7fb07c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:42.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.156+0000 7fb07a59c700 1 -- 172.21.15.142:0/2249390521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb07c105a40 msgr2=0x7fb07c195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:42.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.156+0000 7fb07a59c700 1 --2- 172.21.15.142:0/2249390521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb07c105a40 0x7fb07c195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.156+0000 7fb07a59c700 1 -- 172.21.15.142:0/2249390521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb064009d00 con 0x7fb07c0ffbd0 2024-03-29T05:08:42.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.156+0000 7fb079d9b700 1 --2- 172.21.15.142:0/2249390521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb07c105a40 0x7fb07c195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:08:42.166 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.156+0000 7fb07a59c700 1 --2- 172.21.15.142:0/2249390521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07c0ffbd0 0x7fb07c195620 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb07400d900 tx=0x7fb07400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:42.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.156+0000 7fb0737fe700 1 -- 172.21.15.142:0/2249390521 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0740041f0 con 0x7fb07c0ffbd0 2024-03-29T05:08:42.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.156+0000 7fb0737fe700 1 -- 172.21.15.142:0/2249390521 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb074004cf0 con 0x7fb07c0ffbd0 2024-03-29T05:08:42.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.156+0000 7fb080b63700 1 -- 172.21.15.142:0/2249390521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c19a2b0 con 0x7fb07c0ffbd0 2024-03-29T05:08:42.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.156+0000 7fb0737fe700 1 -- 172.21.15.142:0/2249390521 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb07400b750 con 0x7fb07c0ffbd0 2024-03-29T05:08:42.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.156+0000 7fb080b63700 1 -- 172.21.15.142:0/2249390521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c19a800 con 0x7fb07c0ffbd0 2024-03-29T05:08:42.167 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.157+0000 7fb080b63700 1 -- 172.21.15.142:0/2249390521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c107fd0 con 0x7fb07c0ffbd0 2024-03-29T05:08:42.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.158+0000 7fb0737fe700 1 -- 172.21.15.142:0/2249390521 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fb074004350 con 0x7fb07c0ffbd0 2024-03-29T05:08:42.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.158+0000 7fb0737fe700 1 --2- 172.21.15.142:0/2249390521 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb06806dd60 0x7fb068070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:42.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.158+0000 7fb079d9b700 1 --2- 172.21.15.142:0/2249390521 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb06806dd60 0x7fb068070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:42.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.158+0000 7fb0737fe700 1 -- 172.21.15.142:0/2249390521 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb07408dd30 con 0x7fb07c0ffbd0 2024-03-29T05:08:42.168 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.159+0000 7fb079d9b700 1 --2- 172.21.15.142:0/2249390521 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb06806dd60 0x7fb068070220 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb06400b830 tx=0x7fb064006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:42.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.163+0000 7fb0737fe700 1 -- 172.21.15.142:0/2249390521 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb0740571d0 con 0x7fb07c0ffbd0 2024-03-29T05:08:42.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.379+0000 7fb080b63700 1 -- 172.21.15.142:0/2249390521 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb07c19ab50 con 0x7fb06806dd60 2024-03-29T05:08:42.389 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.380+0000 7fb0737fe700 1 -- 172.21.15.142:0/2249390521 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb07c19ab50 con 0x7fb06806dd60 2024-03-29T05:08:42.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.383+0000 7fb080b63700 1 -- 172.21.15.142:0/2249390521 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb06806dd60 msgr2=0x7fb068070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:42.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.383+0000 7fb080b63700 1 --2- 172.21.15.142:0/2249390521 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb06806dd60 0x7fb068070220 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb06400b830 tx=0x7fb064006000 comp rx=0 tx=0).stop 2024-03-29T05:08:42.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.383+0000 7fb080b63700 1 -- 172.21.15.142:0/2249390521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07c0ffbd0 msgr2=0x7fb07c195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:42.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.383+0000 7fb080b63700 1 --2- 172.21.15.142:0/2249390521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07c0ffbd0 0x7fb07c195620 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb07400d900 tx=0x7fb07400dcc0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.383+0000 7fb080b63700 1 -- 172.21.15.142:0/2249390521 shutdown_connections 2024-03-29T05:08:42.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.383+0000 7fb080b63700 1 --2- 172.21.15.142:0/2249390521 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb06806dd60 0x7fb068070220 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.383+0000 7fb080b63700 1 --2- 172.21.15.142:0/2249390521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07c0ffbd0 0x7fb07c195620 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.383+0000 7fb080b63700 1 --2- 172.21.15.142:0/2249390521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb07c105a40 0x7fb07c195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.383+0000 7fb080b63700 1 -- 172.21.15.142:0/2249390521 >> 172.21.15.142:0/2249390521 conn(0x7fb07c0755a0 msgr2=0x7fb07c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:42.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.383+0000 7fb080b63700 1 -- 172.21.15.142:0/2249390521 shutdown_connections 2024-03-29T05:08:42.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.383+0000 7fb080b63700 1 -- 172.21.15.142:0/2249390521 wait complete. 2024-03-29T05:08:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.486+0000 7f14bc754700 1 -- 172.21.15.142:0/2215274135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b40ffbd0 msgr2=0x7f14b4100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.486+0000 7f14bc754700 1 --2- 172.21.15.142:0/2215274135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b40ffbd0 0x7f14b4100050 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009910 tx=0x7f14b0009c20 comp rx=0 tx=0).stop 2024-03-29T05:08:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.486+0000 7f14bc754700 1 -- 172.21.15.142:0/2215274135 shutdown_connections 2024-03-29T05:08:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.486+0000 7f14bc754700 1 --2- 172.21.15.142:0/2215274135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b40ffbd0 0x7f14b4100050 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.486+0000 7f14bc754700 1 --2- 172.21.15.142:0/2215274135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f14b4105a40 0x7f14b4105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.486+0000 7f14bc754700 1 -- 172.21.15.142:0/2215274135 >> 172.21.15.142:0/2215274135 conn(0x7f14b40755a0 msgr2=0x7f14b40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.486+0000 7f14bc754700 1 -- 172.21.15.142:0/2215274135 shutdown_connections 2024-03-29T05:08:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.486+0000 7f14bc754700 1 -- 172.21.15.142:0/2215274135 wait complete. 2024-03-29T05:08:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.487+0000 7f14bc754700 1 Processor -- start 2024-03-29T05:08:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.487+0000 7f14bc754700 1 -- start start 2024-03-29T05:08:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.487+0000 7f14bc754700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b40ffbd0 0x7f14b4195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.487+0000 7f14bc754700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f14b4105a40 0x7f14b4195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.487+0000 7f14bc754700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b4196240 con 0x7f14b40ffbd0 2024-03-29T05:08:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.487+0000 7f14bc754700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b4199fd0 con 0x7f14b4105a40 2024-03-29T05:08:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.487+0000 7f14ba4f0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b40ffbd0 0x7f14b4195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.487+0000 7f14ba4f0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b40ffbd0 0x7f14b4195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43034/0 (socket says 172.21.15.142:43034) 2024-03-29T05:08:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.487+0000 7f14ba4f0700 1 -- 172.21.15.142:0/3231055040 learned_addr learned my addr 172.21.15.142:0/3231055040 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.487+0000 7f14b9cef700 1 --2- 172.21.15.142:0/3231055040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f14b4105a40 0x7f14b4195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:42.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.488+0000 7f14ba4f0700 1 -- 172.21.15.142:0/3231055040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f14b4105a40 msgr2=0x7f14b4195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:42.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.488+0000 7f14ba4f0700 1 --2- 172.21.15.142:0/3231055040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f14b4105a40 0x7f14b4195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.488+0000 7f14ba4f0700 1 -- 172.21.15.142:0/3231055040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b00095f0 con 0x7f14b40ffbd0 2024-03-29T05:08:42.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.488+0000 7f14ba4f0700 1 --2- 172.21.15.142:0/3231055040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b40ffbd0 0x7f14b4195620 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f14a4007910 tx=0x7f14a4007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:42.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.488+0000 7f14ab7fe700 1 -- 172.21.15.142:0/3231055040 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14a4012070 con 0x7f14b40ffbd0 2024-03-29T05:08:42.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.488+0000 7f14bc754700 1 -- 172.21.15.142:0/3231055040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b419a250 con 0x7f14b40ffbd0 2024-03-29T05:08:42.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.488+0000 7f14ab7fe700 1 -- 172.21.15.142:0/3231055040 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14a4007e60 con 0x7f14b40ffbd0 2024-03-29T05:08:42.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.488+0000 7f14ab7fe700 1 -- 172.21.15.142:0/3231055040 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14a40054d0 con 0x7f14b40ffbd0 2024-03-29T05:08:42.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.488+0000 7f14bc754700 1 -- 172.21.15.142:0/3231055040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b419a7a0 con 0x7f14b40ffbd0 2024-03-29T05:08:42.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.489+0000 7f14bc754700 1 -- 172.21.15.142:0/3231055040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b404ea90 con 0x7f14b40ffbd0 2024-03-29T05:08:42.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.490+0000 7f14ab7fe700 1 -- 172.21.15.142:0/3231055040 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f14a4016070 con 0x7f14b40ffbd0 2024-03-29T05:08:42.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.490+0000 7f14ab7fe700 1 --2- 172.21.15.142:0/3231055040 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f14a006dd60 0x7f14a0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:42.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.490+0000 7f14ab7fe700 1 -- 172.21.15.142:0/3231055040 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f14a408f060 con 0x7f14b40ffbd0 2024-03-29T05:08:42.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.490+0000 7f14b9cef700 1 --2- 172.21.15.142:0/3231055040 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f14a006dd60 0x7f14a0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:42.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.491+0000 7f14b9cef700 1 --2- 172.21.15.142:0/3231055040 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f14a006dd60 0x7f14a0070220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f14b4196c40 tx=0x7f14b001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:42.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.494+0000 7f14ab7fe700 1 -- 172.21.15.142:0/3231055040 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f14a40584e0 con 0x7f14b40ffbd0 2024-03-29T05:08:42.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.697+0000 7f14bc754700 1 -- 172.21.15.142:0/3231055040 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f14b419ad70 con 0x7f14a006dd60 2024-03-29T05:08:42.716 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.708+0000 7f14ab7fe700 1 -- 172.21.15.142:0/3231055040 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f14b419ad70 con 0x7f14a006dd60 2024-03-29T05:08:42.716 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:08:42.716 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (20m) 13m ago 32m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:08:42.716 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (32m) 13m ago 32m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:08:42.716 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (31m) 13m ago 31m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:08:42.716 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (17m) 13m ago 32m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (17m) 13m ago 31m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (20m) 13m ago 31m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (13m) 13m ago 26m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (14m) 13m ago 26m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (13m) 13m ago 26m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (13m) 13m ago 26m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (22m) 13m ago 33m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (21m) 13m ago 31m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (18m) 13m ago 33m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (18m) 13m ago 30m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (21m) 13m ago 32m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (21m) 13m ago 31m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (17m) 13m ago 30m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:08:42.717 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (16m) 13m ago 29m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:08:42.718 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (16m) 13m ago 29m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:08:42.718 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (15m) 13m ago 28m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:08:42.718 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (14m) 13m ago 28m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:08:42.718 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (14m) 13m ago 28m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:08:42.718 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (20m) 13m ago 31m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:08:42.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.710+0000 7f14bc754700 1 -- 172.21.15.142:0/3231055040 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f14a006dd60 msgr2=0x7f14a0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:42.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.710+0000 7f14bc754700 1 --2- 172.21.15.142:0/3231055040 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f14a006dd60 0x7f14a0070220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f14b4196c40 tx=0x7f14b001a040 comp rx=0 tx=0).stop 2024-03-29T05:08:42.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.710+0000 7f14bc754700 1 -- 172.21.15.142:0/3231055040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b40ffbd0 msgr2=0x7f14b4195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:42.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.710+0000 7f14bc754700 1 --2- 172.21.15.142:0/3231055040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b40ffbd0 0x7f14b4195620 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f14a4007910 tx=0x7f14a4007c20 comp rx=0 tx=0).stop 2024-03-29T05:08:42.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.710+0000 7f14bc754700 1 -- 172.21.15.142:0/3231055040 shutdown_connections 2024-03-29T05:08:42.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.710+0000 7f14bc754700 1 --2- 172.21.15.142:0/3231055040 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f14a006dd60 0x7f14a0070220 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.710+0000 7f14bc754700 1 --2- 172.21.15.142:0/3231055040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b40ffbd0 0x7f14b4195620 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.710+0000 7f14bc754700 1 --2- 172.21.15.142:0/3231055040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f14b4105a40 0x7f14b4195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.710+0000 7f14bc754700 1 -- 172.21.15.142:0/3231055040 >> 172.21.15.142:0/3231055040 conn(0x7f14b40755a0 msgr2=0x7f14b40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:42.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.710+0000 7f14bc754700 1 -- 172.21.15.142:0/3231055040 shutdown_connections 2024-03-29T05:08:42.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.710+0000 7f14bc754700 1 -- 172.21.15.142:0/3231055040 wait complete. 2024-03-29T05:08:42.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.813+0000 7f8c089e1700 1 -- 172.21.15.142:0/369603373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c04105a80 msgr2=0x7f8c04105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:42.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.813+0000 7f8c089e1700 1 --2- 172.21.15.142:0/369603373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c04105a80 0x7f8c04105e60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009910 tx=0x7f8bf8009c20 comp rx=0 tx=0).stop 2024-03-29T05:08:42.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.813+0000 7f8c089e1700 1 -- 172.21.15.142:0/369603373 shutdown_connections 2024-03-29T05:08:42.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.813+0000 7f8c089e1700 1 --2- 172.21.15.142:0/369603373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c040ffa80 0x7f8c040fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.813+0000 7f8c089e1700 1 --2- 172.21.15.142:0/369603373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c04105a80 0x7f8c04105e60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.813+0000 7f8c089e1700 1 -- 172.21.15.142:0/369603373 >> 172.21.15.142:0/369603373 conn(0x7f8c040fb590 msgr2=0x7f8c040fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:42.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.813+0000 7f8c089e1700 1 -- 172.21.15.142:0/369603373 shutdown_connections 2024-03-29T05:08:42.822 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.813+0000 7f8c089e1700 1 -- 172.21.15.142:0/369603373 wait complete. 2024-03-29T05:08:42.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.814+0000 7f8c089e1700 1 Processor -- start 2024-03-29T05:08:42.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.814+0000 7f8c089e1700 1 -- start start 2024-03-29T05:08:42.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.814+0000 7f8c089e1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c040ffa80 0x7f8c04196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:42.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.814+0000 7f8c089e1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c04105a80 0x7f8c04196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:42.823 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.814+0000 7f8c089e1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c041972d0 con 0x7f8c040ffa80 2024-03-29T05:08:42.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.814+0000 7f8c089e1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0419a000 con 0x7f8c04105a80 2024-03-29T05:08:42.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8c0259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c040ffa80 0x7f8c04196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:42.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8c0259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c040ffa80 0x7f8c04196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43048/0 (socket says 172.21.15.142:43048) 2024-03-29T05:08:42.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8c0259c700 1 -- 172.21.15.142:0/1663926412 learned_addr learned my addr 172.21.15.142:0/1663926412 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:42.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8c01d9b700 1 --2- 172.21.15.142:0/1663926412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c04105a80 0x7f8c04196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:42.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8c0259c700 1 -- 172.21.15.142:0/1663926412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c04105a80 msgr2=0x7f8c04196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:42.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8c0259c700 1 --2- 172.21.15.142:0/1663926412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c04105a80 0x7f8c04196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:42.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8c0259c700 1 -- 172.21.15.142:0/1663926412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf80095f0 con 0x7f8c040ffa80 2024-03-29T05:08:42.824 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8c0259c700 1 --2- 172.21.15.142:0/1663926412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c040ffa80 0x7f8c04196700 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005f50 tx=0x7f8bf800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:42.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8bf37fe700 1 -- 172.21.15.142:0/1663926412 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf801d070 con 0x7f8c040ffa80 2024-03-29T05:08:42.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8bf37fe700 1 -- 172.21.15.142:0/1663926412 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bf800b610 con 0x7f8c040ffa80 2024-03-29T05:08:42.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8bf37fe700 1 -- 172.21.15.142:0/1663926412 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf8022470 con 0x7f8c040ffa80 2024-03-29T05:08:42.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8c089e1700 1 -- 172.21.15.142:0/1663926412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0419a280 con 0x7f8c040ffa80 2024-03-29T05:08:42.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.815+0000 7f8c089e1700 1 -- 172.21.15.142:0/1663926412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0419a770 con 0x7f8c040ffa80 2024-03-29T05:08:42.825 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.816+0000 7f8c089e1700 1 -- 172.21.15.142:0/1663926412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c0404ea90 con 0x7f8c040ffa80 2024-03-29T05:08:42.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.817+0000 7f8bf37fe700 1 -- 172.21.15.142:0/1663926412 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f8bf8003780 con 0x7f8c040ffa80 2024-03-29T05:08:42.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.818+0000 7f8bf37fe700 1 --2- 172.21.15.142:0/1663926412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8bec06dd60 0x7f8bec070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:42.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.818+0000 7f8bf37fe700 1 -- 172.21.15.142:0/1663926412 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f8bf808ef50 con 0x7f8c040ffa80 2024-03-29T05:08:42.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.818+0000 7f8c01d9b700 1 --2- 172.21.15.142:0/1663926412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8bec06dd60 0x7f8bec070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:42.826 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.818+0000 7f8c01d9b700 1 --2- 172.21.15.142:0/1663926412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8bec06dd60 0x7f8bec070220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8c04197d20 tx=0x7f8bf4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:42.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:42.822+0000 7f8bf37fe700 1 -- 172.21.15.142:0/1663926412 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8bf80c3280 con 0x7f8c040ffa80 2024-03-29T05:08:43.105 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.097+0000 7f8c089e1700 1 -- 172.21.15.142:0/1663926412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8c0419aaa0 con 0x7f8c040ffa80 2024-03-29T05:08:43.106 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.097+0000 7f8bf37fe700 1 -- 172.21.15.142:0/1663926412 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8bf80583f0 con 0x7f8c040ffa80 2024-03-29T05:08:43.106 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:08:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:08:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:08:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:08:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:08:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:08:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:08:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:08:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:08:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:08:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:08:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:08:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:08:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:08:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:08:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:08:43.107 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:08:43.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.099+0000 7f8c089e1700 1 -- 172.21.15.142:0/1663926412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8bec06dd60 msgr2=0x7f8bec070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:43.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.099+0000 7f8c089e1700 1 --2- 172.21.15.142:0/1663926412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8bec06dd60 0x7f8bec070220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8c04197d20 tx=0x7f8bf4009260 comp rx=0 tx=0).stop 2024-03-29T05:08:43.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.099+0000 7f8c089e1700 1 -- 172.21.15.142:0/1663926412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c040ffa80 msgr2=0x7f8c04196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:43.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.099+0000 7f8c089e1700 1 --2- 172.21.15.142:0/1663926412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c040ffa80 0x7f8c04196700 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005f50 tx=0x7f8bf800f690 comp rx=0 tx=0).stop 2024-03-29T05:08:43.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.099+0000 7f8c089e1700 1 -- 172.21.15.142:0/1663926412 shutdown_connections 2024-03-29T05:08:43.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.099+0000 7f8c089e1700 1 --2- 172.21.15.142:0/1663926412 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8bec06dd60 0x7f8bec070220 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.099+0000 7f8c089e1700 1 --2- 172.21.15.142:0/1663926412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c040ffa80 0x7f8c04196700 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.099+0000 7f8c089e1700 1 --2- 172.21.15.142:0/1663926412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c04105a80 0x7f8c04196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.100+0000 7f8c089e1700 1 -- 172.21.15.142:0/1663926412 >> 172.21.15.142:0/1663926412 conn(0x7f8c040fb590 msgr2=0x7f8c04103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:43.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.100+0000 7f8c089e1700 1 -- 172.21.15.142:0/1663926412 shutdown_connections 2024-03-29T05:08:43.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.100+0000 7f8c089e1700 1 -- 172.21.15.142:0/1663926412 wait complete. 2024-03-29T05:08:43.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.202+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3788517701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08bc100dc0 msgr2=0x7f08bc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:43.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.202+0000 7f08c3eb1700 1 --2- 172.21.15.142:0/3788517701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08bc100dc0 0x7f08bc1011a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009910 tx=0x7f08b8009c20 comp rx=0 tx=0).stop 2024-03-29T05:08:43.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.202+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3788517701 shutdown_connections 2024-03-29T05:08:43.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.202+0000 7f08c3eb1700 1 --2- 172.21.15.142:0/3788517701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc101770 0x7f08bc1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.202+0000 7f08c3eb1700 1 --2- 172.21.15.142:0/3788517701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08bc100dc0 0x7f08bc1011a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.211 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.202+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3788517701 >> 172.21.15.142:0/3788517701 conn(0x7f08bc0fc650 msgr2=0x7f08bc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:43.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.203+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3788517701 shutdown_connections 2024-03-29T05:08:43.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.203+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3788517701 wait complete. 2024-03-29T05:08:43.213 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.203+0000 7f08c3eb1700 1 Processor -- start 2024-03-29T05:08:43.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.203+0000 7f08c3eb1700 1 -- start start 2024-03-29T05:08:43.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.203+0000 7f08c3eb1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc100dc0 0x7f08bc19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:43.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.203+0000 7f08c3eb1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08bc101770 0x7f08bc19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:43.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.203+0000 7f08c3eb1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc19d080 con 0x7f08bc101770 2024-03-29T05:08:43.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.203+0000 7f08c3eb1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1965c0 con 0x7f08bc100dc0 2024-03-29T05:08:43.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08c144c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08bc101770 0x7f08bc19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:43.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08c144c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08bc101770 0x7f08bc19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43064/0 (socket says 172.21.15.142:43064) 2024-03-29T05:08:43.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08c144c700 1 -- 172.21.15.142:0/3948955267 learned_addr learned my addr 172.21.15.142:0/3948955267 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:43.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08c1c4d700 1 --2- 172.21.15.142:0/3948955267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc100dc0 0x7f08bc19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:43.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08c144c700 1 -- 172.21.15.142:0/3948955267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc100dc0 msgr2=0x7f08bc19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:43.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08c144c700 1 --2- 172.21.15.142:0/3948955267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc100dc0 0x7f08bc19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08c144c700 1 -- 172.21.15.142:0/3948955267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b80095f0 con 0x7f08bc101770 2024-03-29T05:08:43.215 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08c1c4d700 1 --2- 172.21.15.142:0/3948955267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc100dc0 0x7f08bc19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:08:43.215 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08c144c700 1 --2- 172.21.15.142:0/3948955267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08bc101770 0x7f08bc19ca80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00c960 tx=0x7f08ac00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:43.215 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08b2ffd700 1 -- 172.21.15.142:0/3948955267 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08ac007a70 con 0x7f08bc101770 2024-03-29T05:08:43.215 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3948955267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc1968a0 con 0x7f08bc101770 2024-03-29T05:08:43.215 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08b2ffd700 1 -- 172.21.15.142:0/3948955267 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08ac007bd0 con 0x7f08bc101770 2024-03-29T05:08:43.215 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08b2ffd700 1 -- 172.21.15.142:0/3948955267 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08ac0186c0 con 0x7f08bc101770 2024-03-29T05:08:43.215 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.204+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3948955267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc196df0 con 0x7f08bc101770 2024-03-29T05:08:43.215 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.205+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3948955267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc109160 con 0x7f08bc101770 2024-03-29T05:08:43.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.206+0000 7f08b2ffd700 1 -- 172.21.15.142:0/3948955267 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f08ac00faa0 con 0x7f08bc101770 2024-03-29T05:08:43.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.206+0000 7f08b2ffd700 1 --2- 172.21.15.142:0/3948955267 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f08a806dd60 0x7f08a8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:43.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.206+0000 7f08b2ffd700 1 -- 172.21.15.142:0/3948955267 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f08ac08dce0 con 0x7f08bc101770 2024-03-29T05:08:43.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.206+0000 7f08c1c4d700 1 --2- 172.21.15.142:0/3948955267 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f08a806dd60 0x7f08a8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:43.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.207+0000 7f08c1c4d700 1 --2- 172.21.15.142:0/3948955267 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f08a806dd60 0x7f08a8070220 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f08b8005b40 tx=0x7f08b801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:43.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.211+0000 7f08b2ffd700 1 -- 172.21.15.142:0/3948955267 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f08ac05a9a0 con 0x7f08bc101770 2024-03-29T05:08:43.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.452+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3948955267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f08bc04ea90 con 0x7f08bc101770 2024-03-29T05:08:43.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.452+0000 7f08b2ffd700 1 -- 172.21.15.142:0/3948955267 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f08ac05a7c0 con 0x7f08bc101770 2024-03-29T05:08:43.461 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:08:43.461 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:08:43.461 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:08:43.461 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:08:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:08:43.462 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:08:43.463 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:08:43.464 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:08:43.464 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:08:43.464 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:08:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:08:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:08:43.464 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:08:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:08:43.464 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:08:43.464 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:08:43.464 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:08:43.464 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:08:43.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.454+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3948955267 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f08a806dd60 msgr2=0x7f08a8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:43.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.454+0000 7f08c3eb1700 1 --2- 172.21.15.142:0/3948955267 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f08a806dd60 0x7f08a8070220 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f08b8005b40 tx=0x7f08b801a040 comp rx=0 tx=0).stop 2024-03-29T05:08:43.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.454+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3948955267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08bc101770 msgr2=0x7f08bc19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:43.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.454+0000 7f08c3eb1700 1 --2- 172.21.15.142:0/3948955267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08bc101770 0x7f08bc19ca80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00c960 tx=0x7f08ac00cd20 comp rx=0 tx=0).stop 2024-03-29T05:08:43.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.454+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3948955267 shutdown_connections 2024-03-29T05:08:43.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.454+0000 7f08c3eb1700 1 --2- 172.21.15.142:0/3948955267 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f08a806dd60 0x7f08a8070220 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.454+0000 7f08c3eb1700 1 --2- 172.21.15.142:0/3948955267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc100dc0 0x7f08bc19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.465 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.454+0000 7f08c3eb1700 1 --2- 172.21.15.142:0/3948955267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08bc101770 0x7f08bc19ca80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.454+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3948955267 >> 172.21.15.142:0/3948955267 conn(0x7f08bc0fc650 msgr2=0x7f08bc0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:43.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.454+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3948955267 shutdown_connections 2024-03-29T05:08:43.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.454+0000 7f08c3eb1700 1 -- 172.21.15.142:0/3948955267 wait complete. 2024-03-29T05:08:43.466 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:08:43.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.560+0000 7f9490c73700 1 -- 172.21.15.142:0/594243617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948c0ffa40 msgr2=0x7f948c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:43.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.560+0000 7f9490c73700 1 --2- 172.21.15.142:0/594243617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948c0ffa40 0x7f948c0ffec0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9480009910 tx=0x7f9480009c20 comp rx=0 tx=0).stop 2024-03-29T05:08:43.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.560+0000 7f9490c73700 1 -- 172.21.15.142:0/594243617 shutdown_connections 2024-03-29T05:08:43.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.560+0000 7f9490c73700 1 --2- 172.21.15.142:0/594243617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948c0ffa40 0x7f948c0ffec0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.560+0000 7f9490c73700 1 --2- 172.21.15.142:0/594243617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f948c105a40 0x7f948c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.560+0000 7f9490c73700 1 -- 172.21.15.142:0/594243617 >> 172.21.15.142:0/594243617 conn(0x7f948c0fb550 msgr2=0x7f948c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:43.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.560+0000 7f9490c73700 1 -- 172.21.15.142:0/594243617 shutdown_connections 2024-03-29T05:08:43.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.560+0000 7f9490c73700 1 -- 172.21.15.142:0/594243617 wait complete. 2024-03-29T05:08:43.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.560+0000 7f9490c73700 1 Processor -- start 2024-03-29T05:08:43.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.560+0000 7f9490c73700 1 -- start start 2024-03-29T05:08:43.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9490c73700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f948c0ffa40 0x7f948c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:43.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9490c73700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948c105a40 0x7f948c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:43.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9490c73700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c19d0f0 con 0x7f948c105a40 2024-03-29T05:08:43.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9490c73700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c1965a0 con 0x7f948c0ffa40 2024-03-29T05:08:43.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9489d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948c105a40 0x7f948c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:43.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9489d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948c105a40 0x7f948c19ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43088/0 (socket says 172.21.15.142:43088) 2024-03-29T05:08:43.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9489d9b700 1 -- 172.21.15.142:0/899389688 learned_addr learned my addr 172.21.15.142:0/899389688 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:43.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f948a59c700 1 --2- 172.21.15.142:0/899389688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f948c0ffa40 0x7f948c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:43.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9489d9b700 1 -- 172.21.15.142:0/899389688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f948c0ffa40 msgr2=0x7f948c19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:43.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9489d9b700 1 --2- 172.21.15.142:0/899389688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f948c0ffa40 0x7f948c19c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9489d9b700 1 -- 172.21.15.142:0/899389688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94800095f0 con 0x7f948c105a40 2024-03-29T05:08:43.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9489d9b700 1 --2- 172.21.15.142:0/899389688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948c105a40 0x7f948c19ca60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9480006010 tx=0x7f948000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:43.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f947b7fe700 1 -- 172.21.15.142:0/899389688 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f948001d070 con 0x7f948c105a40 2024-03-29T05:08:43.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f947b7fe700 1 -- 172.21.15.142:0/899389688 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9480005dc0 con 0x7f948c105a40 2024-03-29T05:08:43.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9490c73700 1 -- 172.21.15.142:0/899389688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c196820 con 0x7f948c105a40 2024-03-29T05:08:43.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f947b7fe700 1 -- 172.21.15.142:0/899389688 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9480017400 con 0x7f948c105a40 2024-03-29T05:08:43.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.561+0000 7f9490c73700 1 -- 172.21.15.142:0/899389688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c196d10 con 0x7f948c105a40 2024-03-29T05:08:43.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.563+0000 7f9490c73700 1 -- 172.21.15.142:0/899389688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c108fe0 con 0x7f948c105a40 2024-03-29T05:08:43.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.563+0000 7f947b7fe700 1 -- 172.21.15.142:0/899389688 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f9480003a60 con 0x7f948c105a40 2024-03-29T05:08:43.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.563+0000 7f947b7fe700 1 --2- 172.21.15.142:0/899389688 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f947406dd60 0x7f9474070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:43.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.563+0000 7f947b7fe700 1 -- 172.21.15.142:0/899389688 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f948008f050 con 0x7f948c105a40 2024-03-29T05:08:43.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.563+0000 7f948a59c700 1 --2- 172.21.15.142:0/899389688 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f947406dd60 0x7f9474070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:43.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.564+0000 7f948a59c700 1 --2- 172.21.15.142:0/899389688 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f947406dd60 0x7f9474070220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f947c005950 tx=0x7f947c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:43.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.568+0000 7f947b7fe700 1 -- 172.21.15.142:0/899389688 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f948005c020 con 0x7f948c105a40 2024-03-29T05:08:43.782 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:43 smithi142 ceph-mon[112475]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:43.782 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:43 smithi142 ceph-mon[112475]: pgmap v612: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:43.782 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:43 smithi142 ceph-mon[112475]: from='client.35134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:43.782 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:43 smithi142 ceph-mon[112475]: from='client.35138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:43.782 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:43 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1663926412' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:08:43.782 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:43 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3948955267' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:08:43.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.783+0000 7f9490c73700 1 -- 172.21.15.142:0/899389688 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f948c197040 con 0x7f947406dd60 2024-03-29T05:08:43.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.784+0000 7f947b7fe700 1 -- 172.21.15.142:0/899389688 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f948c197040 con 0x7f947406dd60 2024-03-29T05:08:43.792 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:08:43.792 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:08:43.792 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:08:43.793 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:08:43.793 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:08:43.793 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:08:43.793 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:08:43.793 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:08:43.793 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:08:43.793 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:08:43.793 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:08:43.793 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:08:43.793 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:08:43.793 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:08:43.793 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:08:43.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.787+0000 7f9490c73700 1 -- 172.21.15.142:0/899389688 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f947406dd60 msgr2=0x7f9474070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:43.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.787+0000 7f9490c73700 1 --2- 172.21.15.142:0/899389688 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f947406dd60 0x7f9474070220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f947c005950 tx=0x7f947c00b410 comp rx=0 tx=0).stop 2024-03-29T05:08:43.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.787+0000 7f9490c73700 1 -- 172.21.15.142:0/899389688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948c105a40 msgr2=0x7f948c19ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:43.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.787+0000 7f9490c73700 1 --2- 172.21.15.142:0/899389688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948c105a40 0x7f948c19ca60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9480006010 tx=0x7f948000bd70 comp rx=0 tx=0).stop 2024-03-29T05:08:43.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.787+0000 7f9490c73700 1 -- 172.21.15.142:0/899389688 shutdown_connections 2024-03-29T05:08:43.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.787+0000 7f9490c73700 1 --2- 172.21.15.142:0/899389688 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f947406dd60 0x7f9474070220 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.787+0000 7f9490c73700 1 --2- 172.21.15.142:0/899389688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f948c0ffa40 0x7f948c19c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.787+0000 7f9490c73700 1 --2- 172.21.15.142:0/899389688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948c105a40 0x7f948c19ca60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.787+0000 7f9490c73700 1 -- 172.21.15.142:0/899389688 >> 172.21.15.142:0/899389688 conn(0x7f948c0fb550 msgr2=0x7f948c069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:43.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.787+0000 7f9490c73700 1 -- 172.21.15.142:0/899389688 shutdown_connections 2024-03-29T05:08:43.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.787+0000 7f9490c73700 1 -- 172.21.15.142:0/899389688 wait complete. 2024-03-29T05:08:43.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:43 smithi152 ceph-mon[88186]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:43.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:43 smithi152 ceph-mon[88186]: pgmap v612: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:43.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:43 smithi152 ceph-mon[88186]: from='client.35134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:43.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:43 smithi152 ceph-mon[88186]: from='client.35138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:43.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:43 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1663926412' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:08:43.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:43 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3948955267' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:08:43.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.895+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/1761479253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b58100dc0 msgr2=0x7f4b581011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:43.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.895+0000 7f4b5fa0f700 1 --2- 172.21.15.142:0/1761479253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b58100dc0 0x7f4b581011a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009910 tx=0x7f4b50009c20 comp rx=0 tx=0).stop 2024-03-29T05:08:43.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.895+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/1761479253 shutdown_connections 2024-03-29T05:08:43.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.895+0000 7f4b5fa0f700 1 --2- 172.21.15.142:0/1761479253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b58101770 0x7f4b581057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.895+0000 7f4b5fa0f700 1 --2- 172.21.15.142:0/1761479253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b58100dc0 0x7f4b581011a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.895+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/1761479253 >> 172.21.15.142:0/1761479253 conn(0x7f4b580fc650 msgr2=0x7f4b580fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:43.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.895+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/1761479253 shutdown_connections 2024-03-29T05:08:43.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.895+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/1761479253 wait complete. 2024-03-29T05:08:43.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5fa0f700 1 Processor -- start 2024-03-29T05:08:43.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5fa0f700 1 -- start start 2024-03-29T05:08:43.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5fa0f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b58100dc0 0x7f4b5819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:43.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5fa0f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b58101770 0x7f4b5819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:43.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5fa0f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5819d110 con 0x7f4b58101770 2024-03-29T05:08:43.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5fa0f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b581965c0 con 0x7f4b58100dc0 2024-03-29T05:08:43.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5cfaa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b58101770 0x7f4b5819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:43.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5cfaa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b58101770 0x7f4b5819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43114/0 (socket says 172.21.15.142:43114) 2024-03-29T05:08:43.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5cfaa700 1 -- 172.21.15.142:0/2606388440 learned_addr learned my addr 172.21.15.142:0/2606388440 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:08:43.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5d7ab700 1 --2- 172.21.15.142:0/2606388440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b58100dc0 0x7f4b5819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:43.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5cfaa700 1 -- 172.21.15.142:0/2606388440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b58100dc0 msgr2=0x7f4b5819c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:43.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5cfaa700 1 --2- 172.21.15.142:0/2606388440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b58100dc0 0x7f4b5819c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:43.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.896+0000 7f4b5cfaa700 1 -- 172.21.15.142:0/2606388440 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b500095f0 con 0x7f4b58101770 2024-03-29T05:08:43.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.897+0000 7f4b5cfaa700 1 --2- 172.21.15.142:0/2606388440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b58101770 0x7f4b5819ca80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800d900 tx=0x7f4b4800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:43.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.897+0000 7f4b4e7fc700 1 -- 172.21.15.142:0/2606388440 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b48009900 con 0x7f4b58101770 2024-03-29T05:08:43.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.897+0000 7f4b4e7fc700 1 -- 172.21.15.142:0/2606388440 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b48010460 con 0x7f4b58101770 2024-03-29T05:08:43.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.897+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/2606388440 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b581968a0 con 0x7f4b58101770 2024-03-29T05:08:43.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.897+0000 7f4b4e7fc700 1 -- 172.21.15.142:0/2606388440 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b4800f5f0 con 0x7f4b58101770 2024-03-29T05:08:43.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.897+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/2606388440 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b58196df0 con 0x7f4b58101770 2024-03-29T05:08:43.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.898+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/2606388440 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b58109160 con 0x7f4b58101770 2024-03-29T05:08:43.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.898+0000 7f4b4e7fc700 1 -- 172.21.15.142:0/2606388440 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f4b480105d0 con 0x7f4b58101770 2024-03-29T05:08:43.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.899+0000 7f4b4e7fc700 1 --2- 172.21.15.142:0/2606388440 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b4406dd60 0x7f4b44070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:08:43.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.899+0000 7f4b4e7fc700 1 -- 172.21.15.142:0/2606388440 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f4b4808d870 con 0x7f4b58101770 2024-03-29T05:08:43.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.899+0000 7f4b5d7ab700 1 --2- 172.21.15.142:0/2606388440 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b4406dd60 0x7f4b44070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:08:43.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.899+0000 7f4b5d7ab700 1 --2- 172.21.15.142:0/2606388440 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b4406dd60 0x7f4b44070220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4b50006010 tx=0x7f4b5001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:08:43.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:43.903+0000 7f4b4e7fc700 1 -- 172.21.15.142:0/2606388440 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4b4805a7c0 con 0x7f4b58101770 2024-03-29T05:08:44.193 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.185+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/2606388440 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4b581970d0 con 0x7f4b58101770 2024-03-29T05:08:44.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.185+0000 7f4b4e7fc700 1 -- 172.21.15.142:0/2606388440 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f4b4805a350 con 0x7f4b58101770 2024-03-29T05:08:44.194 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:08:44.194 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:08:44.194 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:08:44.194 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:08:44.194 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:08:44.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.187+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/2606388440 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b4406dd60 msgr2=0x7f4b44070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:44.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.187+0000 7f4b5fa0f700 1 --2- 172.21.15.142:0/2606388440 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b4406dd60 0x7f4b44070220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4b50006010 tx=0x7f4b5001a040 comp rx=0 tx=0).stop 2024-03-29T05:08:44.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.187+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/2606388440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b58101770 msgr2=0x7f4b5819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:08:44.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.187+0000 7f4b5fa0f700 1 --2- 172.21.15.142:0/2606388440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b58101770 0x7f4b5819ca80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800d900 tx=0x7f4b4800dc10 comp rx=0 tx=0).stop 2024-03-29T05:08:44.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.188+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/2606388440 shutdown_connections 2024-03-29T05:08:44.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.188+0000 7f4b5fa0f700 1 --2- 172.21.15.142:0/2606388440 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4b4406dd60 0x7f4b44070220 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:44.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.188+0000 7f4b5fa0f700 1 --2- 172.21.15.142:0/2606388440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b58100dc0 0x7f4b5819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:44.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.188+0000 7f4b5fa0f700 1 --2- 172.21.15.142:0/2606388440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b58101770 0x7f4b5819ca80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:08:44.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.188+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/2606388440 >> 172.21.15.142:0/2606388440 conn(0x7f4b580fc650 msgr2=0x7f4b580fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:08:44.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.188+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/2606388440 shutdown_connections 2024-03-29T05:08:44.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:08:44.188+0000 7f4b5fa0f700 1 -- 172.21.15.142:0/2606388440 wait complete. 2024-03-29T05:08:44.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:44 smithi152 ceph-mon[88186]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:44.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:44 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2606388440' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:08:44.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:44 smithi142 ceph-mon[112475]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:08:44.880 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:44 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2606388440' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:08:45.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:45 smithi152 ceph-mon[88186]: pgmap v613: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:45.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:08:45.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:45 smithi142 ceph-mon[112475]: pgmap v613: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:45.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:08:47.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:47 smithi152 ceph-mon[88186]: pgmap v614: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:47.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:47 smithi142 ceph-mon[112475]: pgmap v614: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:49.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:49 smithi152 ceph-mon[88186]: pgmap v615: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:49.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:49 smithi142 ceph-mon[112475]: pgmap v615: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:51.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:51 smithi152 ceph-mon[88186]: pgmap v616: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:51.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:51 smithi142 ceph-mon[112475]: pgmap v616: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:53.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:53 smithi152 ceph-mon[88186]: pgmap v617: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:53.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:53 smithi142 ceph-mon[112475]: pgmap v617: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:55.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:55 smithi152 ceph-mon[88186]: pgmap v618: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:55.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:55 smithi142 ceph-mon[112475]: pgmap v618: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:57.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:57 smithi152 ceph-mon[88186]: pgmap v619: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:57.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:57 smithi142 ceph-mon[112475]: pgmap v619: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:59.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:08:59 smithi152 ceph-mon[88186]: pgmap v620: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:08:59.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:08:59 smithi142 ceph-mon[112475]: pgmap v620: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:00.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:09:00.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:09:01.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:01 smithi152 ceph-mon[88186]: pgmap v621: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:01.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:01 smithi142 ceph-mon[112475]: pgmap v621: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:03.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:03 smithi142 ceph-mon[112475]: pgmap v622: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:04.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:03 smithi152 ceph-mon[88186]: pgmap v622: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:05.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:05 smithi142 ceph-mon[112475]: pgmap v623: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:06.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:05 smithi152 ceph-mon[88186]: pgmap v623: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:07.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:07 smithi142 ceph-mon[112475]: pgmap v624: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:08.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:07 smithi152 ceph-mon[88186]: pgmap v624: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:09.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:09 smithi142 ceph-mon[112475]: pgmap v625: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:10.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:09 smithi152 ceph-mon[88186]: pgmap v625: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:11.839 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:11 smithi152 ceph-mon[88186]: pgmap v626: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:11.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:11 smithi142 ceph-mon[112475]: pgmap v626: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:13.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:13 smithi142 ceph-mon[112475]: pgmap v627: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:14.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:13 smithi152 ceph-mon[88186]: pgmap v627: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:14.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.299+0000 7fd90f560700 1 -- 172.21.15.142:0/2833755009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9080ffa50 msgr2=0x7fd9080ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:14.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.299+0000 7fd90f560700 1 --2- 172.21.15.142:0/2833755009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9080ffa50 0x7fd9080ffed0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8009910 tx=0x7fd8f8009c20 comp rx=0 tx=0).stop 2024-03-29T05:09:14.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.299+0000 7fd90f560700 1 -- 172.21.15.142:0/2833755009 shutdown_connections 2024-03-29T05:09:14.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.299+0000 7fd90f560700 1 --2- 172.21.15.142:0/2833755009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9080ffa50 0x7fd9080ffed0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.299+0000 7fd90f560700 1 --2- 172.21.15.142:0/2833755009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd908105a50 0x7fd908105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.299+0000 7fd90f560700 1 -- 172.21.15.142:0/2833755009 >> 172.21.15.142:0/2833755009 conn(0x7fd9080fb540 msgr2=0x7fd9080fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:14.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.299+0000 7fd90f560700 1 -- 172.21.15.142:0/2833755009 shutdown_connections 2024-03-29T05:09:14.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.299+0000 7fd90f560700 1 -- 172.21.15.142:0/2833755009 wait complete. 2024-03-29T05:09:14.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90f560700 1 Processor -- start 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90f560700 1 -- start start 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90f560700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9080ffa50 0x7fd908195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90f560700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd908105a50 0x7fd908195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90f560700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd908195e80 con 0x7fd908105a50 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90f560700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd908195ff0 con 0x7fd9080ffa50 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90cafb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd908105a50 0x7fd908195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90cafb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd908105a50 0x7fd908195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59094/0 (socket says 172.21.15.142:59094) 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90cafb700 1 -- 172.21.15.142:0/136585007 learned_addr learned my addr 172.21.15.142:0/136585007 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90d2fc700 1 --2- 172.21.15.142:0/136585007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9080ffa50 0x7fd908195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90cafb700 1 -- 172.21.15.142:0/136585007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9080ffa50 msgr2=0x7fd908195400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90cafb700 1 --2- 172.21.15.142:0/136585007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9080ffa50 0x7fd908195400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90cafb700 1 -- 172.21.15.142:0/136585007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f80095f0 con 0x7fd908105a50 2024-03-29T05:09:14.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.300+0000 7fd90cafb700 1 --2- 172.21.15.142:0/136585007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd908105a50 0x7fd908195940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8005f50 tx=0x7fd8f800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:14.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.301+0000 7fd8fe7fc700 1 -- 172.21.15.142:0/136585007 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8f801d070 con 0x7fd908105a50 2024-03-29T05:09:14.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.301+0000 7fd8fe7fc700 1 -- 172.21.15.142:0/136585007 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8f800b610 con 0x7fd908105a50 2024-03-29T05:09:14.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.301+0000 7fd8fe7fc700 1 -- 172.21.15.142:0/136585007 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8f8022470 con 0x7fd908105a50 2024-03-29T05:09:14.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.301+0000 7fd90f560700 1 -- 172.21.15.142:0/136585007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd908196270 con 0x7fd908105a50 2024-03-29T05:09:14.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.301+0000 7fd90f560700 1 -- 172.21.15.142:0/136585007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9081a2cf0 con 0x7fd908105a50 2024-03-29T05:09:14.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.302+0000 7fd90f560700 1 -- 172.21.15.142:0/136585007 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd908107fe0 con 0x7fd908105a50 2024-03-29T05:09:14.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.302+0000 7fd8fe7fc700 1 -- 172.21.15.142:0/136585007 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fd8f800fca0 con 0x7fd908105a50 2024-03-29T05:09:14.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.303+0000 7fd8fe7fc700 1 --2- 172.21.15.142:0/136585007 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd8f406dd60 0x7fd8f4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:14.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.303+0000 7fd8fe7fc700 1 -- 172.21.15.142:0/136585007 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd8f808eef0 con 0x7fd908105a50 2024-03-29T05:09:14.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.303+0000 7fd90d2fc700 1 --2- 172.21.15.142:0/136585007 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd8f406dd60 0x7fd8f4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:14.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.303+0000 7fd90d2fc700 1 --2- 172.21.15.142:0/136585007 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd8f406dd60 0x7fd8f4070220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd900009380 tx=0x7fd90000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:14.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.307+0000 7fd8fe7fc700 1 -- 172.21.15.142:0/136585007 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd8f805bec0 con 0x7fd908105a50 2024-03-29T05:09:14.527 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.519+0000 7fd90f560700 1 -- 172.21.15.142:0/136585007 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd908196270 con 0x7fd8f406dd60 2024-03-29T05:09:14.528 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.521+0000 7fd8fe7fc700 1 -- 172.21.15.142:0/136585007 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd908196270 con 0x7fd8f406dd60 2024-03-29T05:09:14.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.523+0000 7fd90f560700 1 -- 172.21.15.142:0/136585007 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd8f406dd60 msgr2=0x7fd8f4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:14.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.523+0000 7fd90f560700 1 --2- 172.21.15.142:0/136585007 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd8f406dd60 0x7fd8f4070220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd900009380 tx=0x7fd90000b040 comp rx=0 tx=0).stop 2024-03-29T05:09:14.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.523+0000 7fd90f560700 1 -- 172.21.15.142:0/136585007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd908105a50 msgr2=0x7fd908195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:14.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.523+0000 7fd90f560700 1 --2- 172.21.15.142:0/136585007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd908105a50 0x7fd908195940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8005f50 tx=0x7fd8f800f740 comp rx=0 tx=0).stop 2024-03-29T05:09:14.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.523+0000 7fd90f560700 1 -- 172.21.15.142:0/136585007 shutdown_connections 2024-03-29T05:09:14.531 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.523+0000 7fd90f560700 1 --2- 172.21.15.142:0/136585007 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd8f406dd60 0x7fd8f4070220 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.523+0000 7fd90f560700 1 --2- 172.21.15.142:0/136585007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9080ffa50 0x7fd908195400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.523+0000 7fd90f560700 1 --2- 172.21.15.142:0/136585007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd908105a50 0x7fd908195940 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.523+0000 7fd90f560700 1 -- 172.21.15.142:0/136585007 >> 172.21.15.142:0/136585007 conn(0x7fd9080fb540 msgr2=0x7fd9080fc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:14.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.523+0000 7fd90f560700 1 -- 172.21.15.142:0/136585007 shutdown_connections 2024-03-29T05:09:14.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.523+0000 7fd90f560700 1 -- 172.21.15.142:0/136585007 wait complete. 2024-03-29T05:09:14.541 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:09:14.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.629+0000 7fd9dec57700 1 -- 172.21.15.142:0/3778393715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9d8105a40 msgr2=0x7fd9d8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:14.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.629+0000 7fd9dec57700 1 --2- 172.21.15.142:0/3778393715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9d8105a40 0x7fd9d8105e20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400b0a0 tx=0x7fd9d400b3b0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.629+0000 7fd9dec57700 1 -- 172.21.15.142:0/3778393715 shutdown_connections 2024-03-29T05:09:14.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.629+0000 7fd9dec57700 1 --2- 172.21.15.142:0/3778393715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9d80ffa40 0x7fd9d80ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.629+0000 7fd9dec57700 1 --2- 172.21.15.142:0/3778393715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9d8105a40 0x7fd9d8105e20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.629+0000 7fd9dec57700 1 -- 172.21.15.142:0/3778393715 >> 172.21.15.142:0/3778393715 conn(0x7fd9d80fb570 msgr2=0x7fd9d80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:14.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.629+0000 7fd9dec57700 1 -- 172.21.15.142:0/3778393715 shutdown_connections 2024-03-29T05:09:14.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.629+0000 7fd9dec57700 1 -- 172.21.15.142:0/3778393715 wait complete. 2024-03-29T05:09:14.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.629+0000 7fd9dec57700 1 Processor -- start 2024-03-29T05:09:14.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dec57700 1 -- start start 2024-03-29T05:09:14.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dec57700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9d80ffa40 0x7fd9d80fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:14.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dec57700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9d8105a40 0x7fd9d80fc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:14.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dec57700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80fc900 con 0x7fd9d80ffa40 2024-03-29T05:09:14.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dec57700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80fca70 con 0x7fd9d8105a40 2024-03-29T05:09:14.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dc9f3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9d80ffa40 0x7fd9d80fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:14.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dc9f3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9d80ffa40 0x7fd9d80fbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59110/0 (socket says 172.21.15.142:59110) 2024-03-29T05:09:14.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dc9f3700 1 -- 172.21.15.142:0/2915039352 learned_addr learned my addr 172.21.15.142:0/2915039352 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:14.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9cffff700 1 --2- 172.21.15.142:0/2915039352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9d8105a40 0x7fd9d80fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:14.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dc9f3700 1 -- 172.21.15.142:0/2915039352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9d8105a40 msgr2=0x7fd9d80fc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:14.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dc9f3700 1 --2- 172.21.15.142:0/2915039352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9d8105a40 0x7fd9d80fc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dc9f3700 1 -- 172.21.15.142:0/2915039352 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d4009d00 con 0x7fd9d80ffa40 2024-03-29T05:09:14.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9cffff700 1 --2- 172.21.15.142:0/2915039352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9d8105a40 0x7fd9d80fc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:09:14.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dc9f3700 1 --2- 172.21.15.142:0/2915039352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9d80ffa40 0x7fd9d80fbdf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4000f80 tx=0x7fd9d400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:14.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9cdffb700 1 -- 172.21.15.142:0/2915039352 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9d400e040 con 0x7fd9d80ffa40 2024-03-29T05:09:14.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9cdffb700 1 -- 172.21.15.142:0/2915039352 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9d4003de0 con 0x7fd9d80ffa40 2024-03-29T05:09:14.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9dec57700 1 -- 172.21.15.142:0/2915039352 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d81a28e0 con 0x7fd9d80ffa40 2024-03-29T05:09:14.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.630+0000 7fd9cdffb700 1 -- 172.21.15.142:0/2915039352 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9d401d790 con 0x7fd9d80ffa40 2024-03-29T05:09:14.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.631+0000 7fd9dec57700 1 -- 172.21.15.142:0/2915039352 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d81a2c90 con 0x7fd9d80ffa40 2024-03-29T05:09:14.642 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.632+0000 7fd9dec57700 1 -- 172.21.15.142:0/2915039352 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d8109050 con 0x7fd9d80ffa40 2024-03-29T05:09:14.642 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.632+0000 7fd9cdffb700 1 -- 172.21.15.142:0/2915039352 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fd9d4004040 con 0x7fd9d80ffa40 2024-03-29T05:09:14.642 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.633+0000 7fd9cdffb700 1 --2- 172.21.15.142:0/2915039352 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd9c006dd60 0x7fd9c0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:14.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.633+0000 7fd9cdffb700 1 -- 172.21.15.142:0/2915039352 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd9d408f4c0 con 0x7fd9d80ffa40 2024-03-29T05:09:14.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.633+0000 7fd9cffff700 1 --2- 172.21.15.142:0/2915039352 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd9c006dd60 0x7fd9c0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:14.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.633+0000 7fd9cffff700 1 --2- 172.21.15.142:0/2915039352 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd9c006dd60 0x7fd9c0070220 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8005950 tx=0x7fd9c80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:14.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.637+0000 7fd9cdffb700 1 -- 172.21.15.142:0/2915039352 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd9d40588e0 con 0x7fd9d80ffa40 2024-03-29T05:09:14.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.850+0000 7fd9dec57700 1 -- 172.21.15.142:0/2915039352 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9d80611d0 con 0x7fd9c006dd60 2024-03-29T05:09:14.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.851+0000 7fd9cdffb700 1 -- 172.21.15.142:0/2915039352 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd9d80611d0 con 0x7fd9c006dd60 2024-03-29T05:09:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.853+0000 7fd9dec57700 1 -- 172.21.15.142:0/2915039352 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd9c006dd60 msgr2=0x7fd9c0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.853+0000 7fd9dec57700 1 --2- 172.21.15.142:0/2915039352 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd9c006dd60 0x7fd9c0070220 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8005950 tx=0x7fd9c80058e0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.853+0000 7fd9dec57700 1 -- 172.21.15.142:0/2915039352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9d80ffa40 msgr2=0x7fd9d80fbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.853+0000 7fd9dec57700 1 --2- 172.21.15.142:0/2915039352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9d80ffa40 0x7fd9d80fbdf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4000f80 tx=0x7fd9d400cdd0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.853+0000 7fd9dec57700 1 -- 172.21.15.142:0/2915039352 shutdown_connections 2024-03-29T05:09:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.854+0000 7fd9dec57700 1 --2- 172.21.15.142:0/2915039352 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd9c006dd60 0x7fd9c0070220 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.854+0000 7fd9dec57700 1 --2- 172.21.15.142:0/2915039352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9d80ffa40 0x7fd9d80fbdf0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.854+0000 7fd9dec57700 1 --2- 172.21.15.142:0/2915039352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9d8105a40 0x7fd9d80fc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.854+0000 7fd9dec57700 1 -- 172.21.15.142:0/2915039352 >> 172.21.15.142:0/2915039352 conn(0x7fd9d80fb570 msgr2=0x7fd9d8068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:14.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.854+0000 7fd9dec57700 1 -- 172.21.15.142:0/2915039352 shutdown_connections 2024-03-29T05:09:14.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.854+0000 7fd9dec57700 1 -- 172.21.15.142:0/2915039352 wait complete. 2024-03-29T05:09:14.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.957+0000 7f768888b700 1 -- 172.21.15.142:0/2949757174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7684105a50 msgr2=0x7f7684105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:14.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.957+0000 7f768888b700 1 --2- 172.21.15.142:0/2949757174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7684105a50 0x7f7684105e30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7678009910 tx=0x7f7678009c20 comp rx=0 tx=0).stop 2024-03-29T05:09:14.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.957+0000 7f768888b700 1 -- 172.21.15.142:0/2949757174 shutdown_connections 2024-03-29T05:09:14.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.957+0000 7f768888b700 1 --2- 172.21.15.142:0/2949757174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76840ffa50 0x7f76840ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.957+0000 7f768888b700 1 --2- 172.21.15.142:0/2949757174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7684105a50 0x7f7684105e30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.957+0000 7f768888b700 1 -- 172.21.15.142:0/2949757174 >> 172.21.15.142:0/2949757174 conn(0x7f76840fb540 msgr2=0x7f76840fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:14.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.957+0000 7f768888b700 1 -- 172.21.15.142:0/2949757174 shutdown_connections 2024-03-29T05:09:14.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.957+0000 7f768888b700 1 -- 172.21.15.142:0/2949757174 wait complete. 2024-03-29T05:09:14.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.958+0000 7f768888b700 1 Processor -- start 2024-03-29T05:09:14.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.958+0000 7f768888b700 1 -- start start 2024-03-29T05:09:14.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.958+0000 7f768888b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76840ffa50 0x7f768419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:14.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.958+0000 7f768888b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7684105a50 0x7f768419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:14.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.958+0000 7f768888b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7684196670 con 0x7f76840ffa50 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.958+0000 7f768888b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76841967e0 con 0x7f7684105a50 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.958+0000 7f768259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76840ffa50 0x7f768419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.958+0000 7f768259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76840ffa50 0x7f768419c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59132/0 (socket says 172.21.15.142:59132) 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.958+0000 7f768259c700 1 -- 172.21.15.142:0/2794579465 learned_addr learned my addr 172.21.15.142:0/2794579465 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.959+0000 7f7681d9b700 1 --2- 172.21.15.142:0/2794579465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7684105a50 0x7f768419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.959+0000 7f768259c700 1 -- 172.21.15.142:0/2794579465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7684105a50 msgr2=0x7f768419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.959+0000 7f768259c700 1 --2- 172.21.15.142:0/2794579465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7684105a50 0x7f768419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.959+0000 7f768259c700 1 -- 172.21.15.142:0/2794579465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76780095f0 con 0x7f76840ffa50 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.959+0000 7f7681d9b700 1 --2- 172.21.15.142:0/2794579465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7684105a50 0x7f768419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.959+0000 7f768259c700 1 --2- 172.21.15.142:0/2794579465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76840ffa50 0x7f768419c680 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7678006010 tx=0x7f767800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.959+0000 7f76737fe700 1 -- 172.21.15.142:0/2794579465 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f767801d070 con 0x7f76840ffa50 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.959+0000 7f76737fe700 1 -- 172.21.15.142:0/2794579465 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7678004fb0 con 0x7f76840ffa50 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.959+0000 7f768888b700 1 -- 172.21.15.142:0/2794579465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7684196a60 con 0x7f76840ffa50 2024-03-29T05:09:14.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.959+0000 7f76737fe700 1 -- 172.21.15.142:0/2794579465 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7678022ea0 con 0x7f76840ffa50 2024-03-29T05:09:14.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.959+0000 7f768888b700 1 -- 172.21.15.142:0/2794579465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7684196f50 con 0x7f76840ffa50 2024-03-29T05:09:14.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.960+0000 7f768888b700 1 -- 172.21.15.142:0/2794579465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7684109060 con 0x7f76840ffa50 2024-03-29T05:09:14.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.961+0000 7f76737fe700 1 -- 172.21.15.142:0/2794579465 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f7678022470 con 0x7f76840ffa50 2024-03-29T05:09:14.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.961+0000 7f76737fe700 1 --2- 172.21.15.142:0/2794579465 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f766c06dd60 0x7f766c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:14.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.961+0000 7f76737fe700 1 -- 172.21.15.142:0/2794579465 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f767808f240 con 0x7f76840ffa50 2024-03-29T05:09:14.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.961+0000 7f7681d9b700 1 --2- 172.21.15.142:0/2794579465 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f766c06dd60 0x7f766c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:14.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.961+0000 7f7681d9b700 1 --2- 172.21.15.142:0/2794579465 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f766c06dd60 0x7f766c070220 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7684197f10 tx=0x7f7674008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:14.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:14.966+0000 7f76737fe700 1 -- 172.21.15.142:0/2794579465 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7678058710 con 0x7f76840ffa50 2024-03-29T05:09:15.179 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.171+0000 7f768888b700 1 -- 172.21.15.142:0/2794579465 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7684197d10 con 0x7f766c06dd60 2024-03-29T05:09:15.189 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.181+0000 7f76737fe700 1 -- 172.21.15.142:0/2794579465 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7674007d40 con 0x7f766c06dd60 2024-03-29T05:09:15.190 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:09:15.190 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (21m) 14m ago 32m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:09:15.190 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (32m) 14m ago 32m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:09:15.190 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (31m) 13m ago 31m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:09:15.190 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (18m) 14m ago 32m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:09:15.190 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (18m) 13m ago 31m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:09:15.190 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (20m) 14m ago 32m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:09:15.190 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (14m) 14m ago 26m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:09:15.190 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (14m) 14m ago 27m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:09:15.190 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (13m) 13m ago 26m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:09:15.190 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (13m) 13m ago 26m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:09:15.190 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (22m) 14m ago 33m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:09:15.191 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (21m) 13m ago 31m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:09:15.191 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (19m) 14m ago 34m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:09:15.191 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (18m) 13m ago 31m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:09:15.191 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (21m) 14m ago 32m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:09:15.191 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (21m) 13m ago 31m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:09:15.191 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (17m) 14m ago 30m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:09:15.191 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (17m) 14m ago 30m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:09:15.191 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (16m) 14m ago 29m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:09:15.191 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (15m) 13m ago 29m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:09:15.191 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (15m) 13m ago 29m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:09:15.191 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (14m) 13m ago 28m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:09:15.191 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (21m) 14m ago 32m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:09:15.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.183+0000 7f768888b700 1 -- 172.21.15.142:0/2794579465 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f766c06dd60 msgr2=0x7f766c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:15.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.183+0000 7f768888b700 1 --2- 172.21.15.142:0/2794579465 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f766c06dd60 0x7f766c070220 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7684197f10 tx=0x7f7674008040 comp rx=0 tx=0).stop 2024-03-29T05:09:15.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.183+0000 7f768888b700 1 -- 172.21.15.142:0/2794579465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76840ffa50 msgr2=0x7f768419c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:15.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.183+0000 7f768888b700 1 --2- 172.21.15.142:0/2794579465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76840ffa50 0x7f768419c680 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7678006010 tx=0x7f767800be20 comp rx=0 tx=0).stop 2024-03-29T05:09:15.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.184+0000 7f768888b700 1 -- 172.21.15.142:0/2794579465 shutdown_connections 2024-03-29T05:09:15.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.184+0000 7f768888b700 1 --2- 172.21.15.142:0/2794579465 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f766c06dd60 0x7f766c070220 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.193 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.184+0000 7f768888b700 1 --2- 172.21.15.142:0/2794579465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76840ffa50 0x7f768419c680 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.193 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.184+0000 7f768888b700 1 --2- 172.21.15.142:0/2794579465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7684105a50 0x7f768419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.193 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.184+0000 7f768888b700 1 -- 172.21.15.142:0/2794579465 >> 172.21.15.142:0/2794579465 conn(0x7f76840fb540 msgr2=0x7f7684069070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:15.193 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.184+0000 7f768888b700 1 -- 172.21.15.142:0/2794579465 shutdown_connections 2024-03-29T05:09:15.193 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.184+0000 7f768888b700 1 -- 172.21.15.142:0/2794579465 wait complete. 2024-03-29T05:09:15.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.287+0000 7f0eb63b3700 1 -- 172.21.15.142:0/2943942309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb0068670 msgr2=0x7f0eb0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:15.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.287+0000 7f0eb63b3700 1 --2- 172.21.15.142:0/2943942309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb0068670 0x7f0eb0068a50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009910 tx=0x7f0ea4009c20 comp rx=0 tx=0).stop 2024-03-29T05:09:15.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.287+0000 7f0eb63b3700 1 -- 172.21.15.142:0/2943942309 shutdown_connections 2024-03-29T05:09:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.287+0000 7f0eb63b3700 1 --2- 172.21.15.142:0/2943942309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0eb0069020 0x7f0eb0105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.287+0000 7f0eb63b3700 1 --2- 172.21.15.142:0/2943942309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb0068670 0x7f0eb0068a50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.287+0000 7f0eb63b3700 1 -- 172.21.15.142:0/2943942309 >> 172.21.15.142:0/2943942309 conn(0x7f0eb00755a0 msgr2=0x7f0eb00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.287+0000 7f0eb63b3700 1 -- 172.21.15.142:0/2943942309 shutdown_connections 2024-03-29T05:09:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.287+0000 7f0eb63b3700 1 -- 172.21.15.142:0/2943942309 wait complete. 2024-03-29T05:09:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.288+0000 7f0eb63b3700 1 Processor -- start 2024-03-29T05:09:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.288+0000 7f0eb63b3700 1 -- start start 2024-03-29T05:09:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.288+0000 7f0eb63b3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb0068670 0x7f0eb019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.288+0000 7f0eb63b3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0eb0069020 0x7f0eb019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.288+0000 7f0eb63b3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb019d290 con 0x7f0eb0068670 2024-03-29T05:09:15.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.288+0000 7f0eb63b3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb0196740 con 0x7f0eb0069020 2024-03-29T05:09:15.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.288+0000 7f0eaffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb0068670 0x7f0eb019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:15.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.288+0000 7f0eaffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb0068670 0x7f0eb019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59140/0 (socket says 172.21.15.142:59140) 2024-03-29T05:09:15.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.288+0000 7f0eaffff700 1 -- 172.21.15.142:0/1164917210 learned_addr learned my addr 172.21.15.142:0/1164917210 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:15.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.289+0000 7f0eaffff700 1 -- 172.21.15.142:0/1164917210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0eb0069020 msgr2=0x7f0eb019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:15.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.289+0000 7f0eaffff700 1 --2- 172.21.15.142:0/1164917210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0eb0069020 0x7f0eb019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.289+0000 7f0eaffff700 1 -- 172.21.15.142:0/1164917210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea40095f0 con 0x7f0eb0068670 2024-03-29T05:09:15.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.289+0000 7f0eaffff700 1 --2- 172.21.15.142:0/1164917210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb0068670 0x7f0eb019c6c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4006010 tx=0x7f0ea400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:15.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.289+0000 7f0ead7fa700 1 -- 172.21.15.142:0/1164917210 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ea401d070 con 0x7f0eb0068670 2024-03-29T05:09:15.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.289+0000 7f0eb63b3700 1 -- 172.21.15.142:0/1164917210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb01969c0 con 0x7f0eb0068670 2024-03-29T05:09:15.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.289+0000 7f0ead7fa700 1 -- 172.21.15.142:0/1164917210 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ea4004fb0 con 0x7f0eb0068670 2024-03-29T05:09:15.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.289+0000 7f0ead7fa700 1 -- 172.21.15.142:0/1164917210 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ea4022ea0 con 0x7f0eb0068670 2024-03-29T05:09:15.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.289+0000 7f0eb63b3700 1 -- 172.21.15.142:0/1164917210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb0196eb0 con 0x7f0eb0068670 2024-03-29T05:09:15.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.290+0000 7f0eb63b3700 1 -- 172.21.15.142:0/1164917210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb0109130 con 0x7f0eb0068670 2024-03-29T05:09:15.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.290+0000 7f0ead7fa700 1 -- 172.21.15.142:0/1164917210 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f0ea4022470 con 0x7f0eb0068670 2024-03-29T05:09:15.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.291+0000 7f0ead7fa700 1 --2- 172.21.15.142:0/1164917210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0e9806dd60 0x7f0e98070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:15.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.291+0000 7f0ead7fa700 1 -- 172.21.15.142:0/1164917210 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f0ea408f230 con 0x7f0eb0068670 2024-03-29T05:09:15.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.291+0000 7f0eaf7fe700 1 --2- 172.21.15.142:0/1164917210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0e9806dd60 0x7f0e98070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:15.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.291+0000 7f0eaf7fe700 1 --2- 172.21.15.142:0/1164917210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0e9806dd60 0x7f0e98070220 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0197bb0 tx=0x7f0ea0006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:15.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.295+0000 7f0ead7fa700 1 -- 172.21.15.142:0/1164917210 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0ea405c230 con 0x7f0eb0068670 2024-03-29T05:09:15.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.573+0000 7f0eb63b3700 1 -- 172.21.15.142:0/1164917210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0eb002cc70 con 0x7f0eb0068670 2024-03-29T05:09:15.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.573+0000 7f0ead7fa700 1 -- 172.21.15.142:0/1164917210 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f0ea4022720 con 0x7f0eb0068670 2024-03-29T05:09:15.581 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:09:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:09:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:09:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:09:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:09:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:09:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:09:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:09:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:09:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:09:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:09:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:09:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:09:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:09:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:09:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:09:15.583 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:09:15.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.575+0000 7f0eb63b3700 1 -- 172.21.15.142:0/1164917210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0e9806dd60 msgr2=0x7f0e98070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:15.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.575+0000 7f0eb63b3700 1 --2- 172.21.15.142:0/1164917210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0e9806dd60 0x7f0e98070220 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0197bb0 tx=0x7f0ea0006d20 comp rx=0 tx=0).stop 2024-03-29T05:09:15.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.575+0000 7f0eb63b3700 1 -- 172.21.15.142:0/1164917210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb0068670 msgr2=0x7f0eb019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:15.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.575+0000 7f0eb63b3700 1 --2- 172.21.15.142:0/1164917210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb0068670 0x7f0eb019c6c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4006010 tx=0x7f0ea400be20 comp rx=0 tx=0).stop 2024-03-29T05:09:15.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.576+0000 7f0eb63b3700 1 -- 172.21.15.142:0/1164917210 shutdown_connections 2024-03-29T05:09:15.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.576+0000 7f0eb63b3700 1 --2- 172.21.15.142:0/1164917210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0e9806dd60 0x7f0e98070220 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.576+0000 7f0eb63b3700 1 --2- 172.21.15.142:0/1164917210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0eb0068670 0x7f0eb019c6c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.576+0000 7f0eb63b3700 1 --2- 172.21.15.142:0/1164917210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0eb0069020 0x7f0eb019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.576+0000 7f0eb63b3700 1 -- 172.21.15.142:0/1164917210 >> 172.21.15.142:0/1164917210 conn(0x7f0eb00755a0 msgr2=0x7f0eb00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:15.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.576+0000 7f0eb63b3700 1 -- 172.21.15.142:0/1164917210 shutdown_connections 2024-03-29T05:09:15.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.576+0000 7f0eb63b3700 1 -- 172.21.15.142:0/1164917210 wait complete. 2024-03-29T05:09:15.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.679+0000 7f6755172700 1 -- 172.21.15.142:0/4068350254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750105a40 msgr2=0x7f6750105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:15.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.679+0000 7f6755172700 1 --2- 172.21.15.142:0/4068350254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750105a40 0x7f6750105e20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6740009910 tx=0x7f6740009c20 comp rx=0 tx=0).stop 2024-03-29T05:09:15.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.679+0000 7f6755172700 1 -- 172.21.15.142:0/4068350254 shutdown_connections 2024-03-29T05:09:15.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.679+0000 7f6755172700 1 --2- 172.21.15.142:0/4068350254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67500ffa40 0x7f67500ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.679+0000 7f6755172700 1 --2- 172.21.15.142:0/4068350254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750105a40 0x7f6750105e20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.679+0000 7f6755172700 1 -- 172.21.15.142:0/4068350254 >> 172.21.15.142:0/4068350254 conn(0x7f67500fb550 msgr2=0x7f67500fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:15.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.679+0000 7f6755172700 1 -- 172.21.15.142:0/4068350254 shutdown_connections 2024-03-29T05:09:15.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.679+0000 7f6755172700 1 -- 172.21.15.142:0/4068350254 wait complete. 2024-03-29T05:09:15.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.679+0000 7f6755172700 1 Processor -- start 2024-03-29T05:09:15.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f6755172700 1 -- start start 2024-03-29T05:09:15.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f6755172700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67500ffa40 0x7f6750195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:15.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f6755172700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750105a40 0x7f6750195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:15.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f6755172700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6750196240 con 0x7f6750105a40 2024-03-29T05:09:15.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f6755172700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6750199fd0 con 0x7f67500ffa40 2024-03-29T05:09:15.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f674e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750105a40 0x7f6750195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:15.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f674e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750105a40 0x7f6750195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59156/0 (socket says 172.21.15.142:59156) 2024-03-29T05:09:15.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f674e59c700 1 -- 172.21.15.142:0/3073205210 learned_addr learned my addr 172.21.15.142:0/3073205210 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:15.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f674ed9d700 1 --2- 172.21.15.142:0/3073205210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67500ffa40 0x7f6750195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:15.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f674e59c700 1 -- 172.21.15.142:0/3073205210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67500ffa40 msgr2=0x7f6750195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:15.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f674e59c700 1 --2- 172.21.15.142:0/3073205210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67500ffa40 0x7f6750195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f674e59c700 1 -- 172.21.15.142:0/3073205210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67400095f0 con 0x7f6750105a40 2024-03-29T05:09:15.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f674e59c700 1 --2- 172.21.15.142:0/3073205210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750105a40 0x7f6750195b60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f674400c8a0 tx=0x7f674400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:15.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.680+0000 7f6737fff700 1 -- 172.21.15.142:0/3073205210 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f674400cea0 con 0x7f6750105a40 2024-03-29T05:09:15.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.681+0000 7f6755172700 1 -- 172.21.15.142:0/3073205210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675019a2b0 con 0x7f6750105a40 2024-03-29T05:09:15.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.681+0000 7f6737fff700 1 -- 172.21.15.142:0/3073205210 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6744004830 con 0x7f6750105a40 2024-03-29T05:09:15.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.681+0000 7f6737fff700 1 -- 172.21.15.142:0/3073205210 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67440055b0 con 0x7f6750105a40 2024-03-29T05:09:15.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.681+0000 7f6755172700 1 -- 172.21.15.142:0/3073205210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675019a800 con 0x7f6750105a40 2024-03-29T05:09:15.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.682+0000 7f6755172700 1 -- 172.21.15.142:0/3073205210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6750108040 con 0x7f6750105a40 2024-03-29T05:09:15.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.682+0000 7f6737fff700 1 -- 172.21.15.142:0/3073205210 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f67440049a0 con 0x7f6750105a40 2024-03-29T05:09:15.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.682+0000 7f6737fff700 1 --2- 172.21.15.142:0/3073205210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f673806dd60 0x7f6738070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:15.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.683+0000 7f6737fff700 1 -- 172.21.15.142:0/3073205210 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f674408d980 con 0x7f6750105a40 2024-03-29T05:09:15.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.683+0000 7f674ed9d700 1 --2- 172.21.15.142:0/3073205210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f673806dd60 0x7f6738070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:15.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.683+0000 7f674ed9d700 1 --2- 172.21.15.142:0/3073205210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f673806dd60 0x7f6738070220 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6740006010 tx=0x7f674001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:15.695 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.687+0000 7f6737fff700 1 -- 172.21.15.142:0/3073205210 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6744056da0 con 0x7f6750105a40 2024-03-29T05:09:15.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:15 smithi142 ceph-mon[112475]: pgmap v628: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:15.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:15 smithi142 ceph-mon[112475]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:15.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:09:15.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.927+0000 7f6755172700 1 -- 172.21.15.142:0/3073205210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f675004ea90 con 0x7f6750105a40 2024-03-29T05:09:15.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.928+0000 7f6737fff700 1 -- 172.21.15.142:0/3073205210 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f6744004c50 con 0x7f6750105a40 2024-03-29T05:09:15.936 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:09:15.936 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:09:15.936 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:09:15.936 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:09:15.936 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:09:15.936 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:09:15.937 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:09:15.938 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:09:15.939 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:09:15.939 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:09:15.939 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:09:15.939 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:09:15.939 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:09:15.939 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:09:15.939 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:09:15.939 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:09:15.939 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:09:15.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.931+0000 7f6755172700 1 -- 172.21.15.142:0/3073205210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f673806dd60 msgr2=0x7f6738070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:15.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.931+0000 7f6755172700 1 --2- 172.21.15.142:0/3073205210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f673806dd60 0x7f6738070220 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6740006010 tx=0x7f674001a040 comp rx=0 tx=0).stop 2024-03-29T05:09:15.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.931+0000 7f6755172700 1 -- 172.21.15.142:0/3073205210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750105a40 msgr2=0x7f6750195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:15.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.931+0000 7f6755172700 1 --2- 172.21.15.142:0/3073205210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750105a40 0x7f6750195b60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f674400c8a0 tx=0x7f674400cc60 comp rx=0 tx=0).stop 2024-03-29T05:09:15.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.931+0000 7f6755172700 1 -- 172.21.15.142:0/3073205210 shutdown_connections 2024-03-29T05:09:15.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.931+0000 7f6755172700 1 --2- 172.21.15.142:0/3073205210 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f673806dd60 0x7f6738070220 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.931+0000 7f6755172700 1 --2- 172.21.15.142:0/3073205210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67500ffa40 0x7f6750195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.931+0000 7f6755172700 1 --2- 172.21.15.142:0/3073205210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6750105a40 0x7f6750195b60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:15.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.931+0000 7f6755172700 1 -- 172.21.15.142:0/3073205210 >> 172.21.15.142:0/3073205210 conn(0x7f67500fb550 msgr2=0x7f67500fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:15.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.931+0000 7f6755172700 1 -- 172.21.15.142:0/3073205210 shutdown_connections 2024-03-29T05:09:15.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:15.931+0000 7f6755172700 1 -- 172.21.15.142:0/3073205210 wait complete. 2024-03-29T05:09:15.941 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:09:16.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.035+0000 7f0f830ff700 1 -- 172.21.15.142:0/1627727280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f7c0ffbd0 msgr2=0x7f0f7c100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:16.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.035+0000 7f0f830ff700 1 --2- 172.21.15.142:0/1627727280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f7c0ffbd0 0x7f0f7c100050 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009910 tx=0x7f0f68009c20 comp rx=0 tx=0).stop 2024-03-29T05:09:16.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.035+0000 7f0f830ff700 1 -- 172.21.15.142:0/1627727280 shutdown_connections 2024-03-29T05:09:16.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.035+0000 7f0f830ff700 1 --2- 172.21.15.142:0/1627727280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f7c0ffbd0 0x7f0f7c100050 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.035+0000 7f0f830ff700 1 --2- 172.21.15.142:0/1627727280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7c105a40 0x7f0f7c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.035+0000 7f0f830ff700 1 -- 172.21.15.142:0/1627727280 >> 172.21.15.142:0/1627727280 conn(0x7f0f7c0755a0 msgr2=0x7f0f7c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:16.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.035+0000 7f0f830ff700 1 -- 172.21.15.142:0/1627727280 shutdown_connections 2024-03-29T05:09:16.043 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.035+0000 7f0f830ff700 1 -- 172.21.15.142:0/1627727280 wait complete. 2024-03-29T05:09:16.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.035+0000 7f0f830ff700 1 Processor -- start 2024-03-29T05:09:16.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.035+0000 7f0f830ff700 1 -- start start 2024-03-29T05:09:16.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f830ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f7c0ffbd0 0x7f0f7c197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:16.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f830ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7c105a40 0x7f0f7c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f830ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c198190 con 0x7f0f7c0ffbd0 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f830ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c198300 con 0x7f0f7c105a40 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f80e9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f7c0ffbd0 0x7f0f7c197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f80e9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f7c0ffbd0 0x7f0f7c197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59186/0 (socket says 172.21.15.142:59186) 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f80e9b700 1 -- 172.21.15.142:0/631342675 learned_addr learned my addr 172.21.15.142:0/631342675 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f7bfff700 1 --2- 172.21.15.142:0/631342675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7c105a40 0x7f0f7c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f80e9b700 1 -- 172.21.15.142:0/631342675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7c105a40 msgr2=0x7f0f7c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f80e9b700 1 --2- 172.21.15.142:0/631342675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7c105a40 0x7f0f7c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f80e9b700 1 -- 172.21.15.142:0/631342675 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f680095f0 con 0x7f0f7c0ffbd0 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f80e9b700 1 --2- 172.21.15.142:0/631342675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f7c0ffbd0 0x7f0f7c197570 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000c8a0 tx=0x7f0f7000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f79ffb700 1 -- 172.21.15.142:0/631342675 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f700078e0 con 0x7f0f7c0ffbd0 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f830ff700 1 -- 172.21.15.142:0/631342675 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7c1a05e0 con 0x7f0f7c0ffbd0 2024-03-29T05:09:16.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f79ffb700 1 -- 172.21.15.142:0/631342675 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f7000f450 con 0x7f0f7c0ffbd0 2024-03-29T05:09:16.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.036+0000 7f0f79ffb700 1 -- 172.21.15.142:0/631342675 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f7000e5e0 con 0x7f0f7c0ffbd0 2024-03-29T05:09:16.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.037+0000 7f0f830ff700 1 -- 172.21.15.142:0/631342675 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7c1a0b30 con 0x7f0f7c0ffbd0 2024-03-29T05:09:16.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.038+0000 7f0f830ff700 1 -- 172.21.15.142:0/631342675 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7c107fd0 con 0x7f0f7c0ffbd0 2024-03-29T05:09:16.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.038+0000 7f0f79ffb700 1 -- 172.21.15.142:0/631342675 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f0f7000f5c0 con 0x7f0f7c0ffbd0 2024-03-29T05:09:16.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.038+0000 7f0f79ffb700 1 --2- 172.21.15.142:0/631342675 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0f6c06dd60 0x7f0f6c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:16.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.038+0000 7f0f79ffb700 1 -- 172.21.15.142:0/631342675 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f0f7008dc50 con 0x7f0f7c0ffbd0 2024-03-29T05:09:16.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.038+0000 7f0f7bfff700 1 --2- 172.21.15.142:0/631342675 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0f6c06dd60 0x7f0f6c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:16.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.039+0000 7f0f7bfff700 1 --2- 172.21.15.142:0/631342675 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0f6c06dd60 0x7f0f6c070220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c069060 tx=0x7f0f6801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:16.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:15 smithi152 ceph-mon[88186]: pgmap v628: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:16.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:15 smithi152 ceph-mon[88186]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:16.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:09:16.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.043+0000 7f0f79ffb700 1 -- 172.21.15.142:0/631342675 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0f70057070 con 0x7f0f7c0ffbd0 2024-03-29T05:09:16.267 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.259+0000 7f0f830ff700 1 -- 172.21.15.142:0/631342675 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f7c0611d0 con 0x7f0f6c06dd60 2024-03-29T05:09:16.268 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.260+0000 7f0f79ffb700 1 -- 172.21.15.142:0/631342675 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0f7c0611d0 con 0x7f0f6c06dd60 2024-03-29T05:09:16.268 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:09:16.268 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:09:16.268 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:09:16.268 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:09:16.268 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:09:16.268 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:09:16.268 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:09:16.268 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:09:16.269 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:09:16.269 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:09:16.269 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:09:16.269 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:09:16.269 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:09:16.269 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:09:16.269 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:09:16.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.262+0000 7f0f830ff700 1 -- 172.21.15.142:0/631342675 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0f6c06dd60 msgr2=0x7f0f6c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:16.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.262+0000 7f0f830ff700 1 --2- 172.21.15.142:0/631342675 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0f6c06dd60 0x7f0f6c070220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c069060 tx=0x7f0f6801a040 comp rx=0 tx=0).stop 2024-03-29T05:09:16.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.263+0000 7f0f830ff700 1 -- 172.21.15.142:0/631342675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f7c0ffbd0 msgr2=0x7f0f7c197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:16.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.263+0000 7f0f830ff700 1 --2- 172.21.15.142:0/631342675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f7c0ffbd0 0x7f0f7c197570 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000c8a0 tx=0x7f0f7000cbb0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.263+0000 7f0f830ff700 1 -- 172.21.15.142:0/631342675 shutdown_connections 2024-03-29T05:09:16.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.263+0000 7f0f830ff700 1 --2- 172.21.15.142:0/631342675 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0f6c06dd60 0x7f0f6c070220 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.263+0000 7f0f830ff700 1 --2- 172.21.15.142:0/631342675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f7c0ffbd0 0x7f0f7c197570 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.263+0000 7f0f830ff700 1 --2- 172.21.15.142:0/631342675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f7c105a40 0x7f0f7c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.263+0000 7f0f830ff700 1 -- 172.21.15.142:0/631342675 >> 172.21.15.142:0/631342675 conn(0x7f0f7c0755a0 msgr2=0x7f0f7c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:16.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.263+0000 7f0f830ff700 1 -- 172.21.15.142:0/631342675 shutdown_connections 2024-03-29T05:09:16.271 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.263+0000 7f0f830ff700 1 -- 172.21.15.142:0/631342675 wait complete. 2024-03-29T05:09:16.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.366+0000 7f88c4263700 1 -- 172.21.15.142:0/1621285803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88bc069020 msgr2=0x7f88bc105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:16.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.366+0000 7f88c4263700 1 --2- 172.21.15.142:0/1621285803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88bc069020 0x7f88bc105790 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009910 tx=0x7f88ac009c20 comp rx=0 tx=0).stop 2024-03-29T05:09:16.374 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.366+0000 7f88c4263700 1 -- 172.21.15.142:0/1621285803 shutdown_connections 2024-03-29T05:09:16.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.366+0000 7f88c4263700 1 --2- 172.21.15.142:0/1621285803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88bc069020 0x7f88bc105790 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.366+0000 7f88c4263700 1 --2- 172.21.15.142:0/1621285803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88bc068670 0x7f88bc068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.366+0000 7f88c4263700 1 -- 172.21.15.142:0/1621285803 >> 172.21.15.142:0/1621285803 conn(0x7f88bc0755a0 msgr2=0x7f88bc0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:16.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.366+0000 7f88c4263700 1 -- 172.21.15.142:0/1621285803 shutdown_connections 2024-03-29T05:09:16.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.366+0000 7f88c4263700 1 -- 172.21.15.142:0/1621285803 wait complete. 2024-03-29T05:09:16.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.367+0000 7f88c4263700 1 Processor -- start 2024-03-29T05:09:16.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.367+0000 7f88c4263700 1 -- start start 2024-03-29T05:09:16.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.367+0000 7f88c4263700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88bc068670 0x7f88bc0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:16.376 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.367+0000 7f88c4263700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88bc069020 0x7f88bc1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.367+0000 7f88c4263700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc100720 con 0x7f88bc068670 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.367+0000 7f88c4263700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc100860 con 0x7f88bc069020 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.367+0000 7f88c1fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88bc068670 0x7f88bc0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.367+0000 7f88c1fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88bc068670 0x7f88bc0ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59192/0 (socket says 172.21.15.142:59192) 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.367+0000 7f88c1fff700 1 -- 172.21.15.142:0/973310133 learned_addr learned my addr 172.21.15.142:0/973310133 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.367+0000 7f88c17fe700 1 --2- 172.21.15.142:0/973310133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88bc069020 0x7f88bc1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.368+0000 7f88c1fff700 1 -- 172.21.15.142:0/973310133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88bc069020 msgr2=0x7f88bc1001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.368+0000 7f88c1fff700 1 --2- 172.21.15.142:0/973310133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88bc069020 0x7f88bc1001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.368+0000 7f88c1fff700 1 -- 172.21.15.142:0/973310133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac0095f0 con 0x7f88bc068670 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.368+0000 7f88c1fff700 1 --2- 172.21.15.142:0/973310133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88bc068670 0x7f88bc0ffca0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f88b800d8d0 tx=0x7f88b800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.368+0000 7f88b2ffd700 1 -- 172.21.15.142:0/973310133 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88b80098a0 con 0x7f88bc068670 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.368+0000 7f88c4263700 1 -- 172.21.15.142:0/973310133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc1a28d0 con 0x7f88bc068670 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.368+0000 7f88b2ffd700 1 -- 172.21.15.142:0/973310133 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88b8010460 con 0x7f88bc068670 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.368+0000 7f88b2ffd700 1 -- 172.21.15.142:0/973310133 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88b800f5f0 con 0x7f88bc068670 2024-03-29T05:09:16.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.368+0000 7f88c4263700 1 -- 172.21.15.142:0/973310133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc1a2ce0 con 0x7f88bc068670 2024-03-29T05:09:16.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.369+0000 7f88c4263700 1 -- 172.21.15.142:0/973310133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88bc1090a0 con 0x7f88bc068670 2024-03-29T05:09:16.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.369+0000 7f88b2ffd700 1 -- 172.21.15.142:0/973310133 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f88b800f820 con 0x7f88bc068670 2024-03-29T05:09:16.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.370+0000 7f88b2ffd700 1 --2- 172.21.15.142:0/973310133 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f88a806dd60 0x7f88a8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:16.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.370+0000 7f88c17fe700 1 --2- 172.21.15.142:0/973310133 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f88a806dd60 0x7f88a8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:16.379 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.370+0000 7f88b2ffd700 1 -- 172.21.15.142:0/973310133 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f88b808dc00 con 0x7f88bc068670 2024-03-29T05:09:16.379 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.370+0000 7f88c17fe700 1 --2- 172.21.15.142:0/973310133 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f88a806dd60 0x7f88a8070220 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f88bc101490 tx=0x7f88ac01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:16.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.375+0000 7f88b2ffd700 1 -- 172.21.15.142:0/973310133 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f88b8093020 con 0x7f88bc068670 2024-03-29T05:09:16.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.649+0000 7f88c4263700 1 -- 172.21.15.142:0/973310133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f88bc04ea90 con 0x7f88bc068670 2024-03-29T05:09:16.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.649+0000 7f88b2ffd700 1 -- 172.21.15.142:0/973310133 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f88b8016100 con 0x7f88bc068670 2024-03-29T05:09:16.657 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:09:16.657 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:09:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:09:16.658 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:09:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:09:16.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.651+0000 7f88c4263700 1 -- 172.21.15.142:0/973310133 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f88a806dd60 msgr2=0x7f88a8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:16.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.651+0000 7f88c4263700 1 --2- 172.21.15.142:0/973310133 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f88a806dd60 0x7f88a8070220 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f88bc101490 tx=0x7f88ac01a040 comp rx=0 tx=0).stop 2024-03-29T05:09:16.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.651+0000 7f88c4263700 1 -- 172.21.15.142:0/973310133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88bc068670 msgr2=0x7f88bc0ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:16.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.651+0000 7f88c4263700 1 --2- 172.21.15.142:0/973310133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88bc068670 0x7f88bc0ffca0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f88b800d8d0 tx=0x7f88b800dbe0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.651+0000 7f88c4263700 1 -- 172.21.15.142:0/973310133 shutdown_connections 2024-03-29T05:09:16.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.651+0000 7f88c4263700 1 --2- 172.21.15.142:0/973310133 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f88a806dd60 0x7f88a8070220 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.651+0000 7f88c4263700 1 --2- 172.21.15.142:0/973310133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88bc068670 0x7f88bc0ffca0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.651+0000 7f88c4263700 1 --2- 172.21.15.142:0/973310133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88bc069020 0x7f88bc1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:16.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.651+0000 7f88c4263700 1 -- 172.21.15.142:0/973310133 >> 172.21.15.142:0/973310133 conn(0x7f88bc0755a0 msgr2=0x7f88bc105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:16.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.651+0000 7f88c4263700 1 -- 172.21.15.142:0/973310133 shutdown_connections 2024-03-29T05:09:16.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:16.651+0000 7f88c4263700 1 -- 172.21.15.142:0/973310133 wait complete. 2024-03-29T05:09:16.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:16 smithi142 ceph-mon[112475]: from='client.35162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:16.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:16 smithi142 ceph-mon[112475]: from='client.35166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:16.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:16 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1164917210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:09:16.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:16 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3073205210' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:09:16.892 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:16 smithi152 ceph-mon[88186]: from='client.35162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:16.892 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:16 smithi152 ceph-mon[88186]: from='client.35166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:16.893 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:16 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1164917210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:09:16.893 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:16 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3073205210' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:09:17.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:17 smithi142 ceph-mon[112475]: pgmap v629: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:17.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:17 smithi142 ceph-mon[112475]: from='client.35178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:17.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:17 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/973310133' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:09:18.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:17 smithi152 ceph-mon[88186]: pgmap v629: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:18.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:17 smithi152 ceph-mon[88186]: from='client.35178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:18.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:17 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/973310133' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:09:19.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:19 smithi142 ceph-mon[112475]: pgmap v630: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:20.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:19 smithi152 ceph-mon[88186]: pgmap v630: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:21.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:21 smithi142 ceph-mon[112475]: pgmap v631: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:21.946 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:21 smithi152 ceph-mon[88186]: pgmap v631: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:23.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:23 smithi142 ceph-mon[112475]: pgmap v632: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:24.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:23 smithi152 ceph-mon[88186]: pgmap v632: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:25.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:25 smithi142 ceph-mon[112475]: pgmap v633: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:26.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:25 smithi152 ceph-mon[88186]: pgmap v633: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:27.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:27 smithi142 ceph-mon[112475]: pgmap v634: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:28.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:27 smithi152 ceph-mon[88186]: pgmap v634: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:29.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:29 smithi142 ceph-mon[112475]: pgmap v635: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:30.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:29 smithi152 ceph-mon[88186]: pgmap v635: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:30.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:09:31.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:09:31.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:31 smithi142 ceph-mon[112475]: pgmap v636: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:32.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:31 smithi152 ceph-mon[88186]: pgmap v636: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:33.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:33 smithi142 ceph-mon[112475]: pgmap v637: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:34.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:33 smithi152 ceph-mon[88186]: pgmap v637: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:35.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:35 smithi142 ceph-mon[112475]: pgmap v638: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:36.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:35 smithi152 ceph-mon[88186]: pgmap v638: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:37.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:37 smithi142 ceph-mon[112475]: pgmap v639: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:38.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:37 smithi152 ceph-mon[88186]: pgmap v639: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:39.628 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:39 smithi142 ceph-mon[112475]: pgmap v640: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:40.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:39 smithi152 ceph-mon[88186]: pgmap v640: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:41.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:41 smithi142 ceph-mon[112475]: pgmap v641: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:42.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:41 smithi152 ceph-mon[88186]: pgmap v641: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:43.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:43 smithi142 ceph-mon[112475]: pgmap v642: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:44.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:43 smithi152 ceph-mon[88186]: pgmap v642: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:44.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:09:45.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:09:45.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:45 smithi142 ceph-mon[112475]: pgmap v643: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:46.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:45 smithi152 ceph-mon[88186]: pgmap v643: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:46.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.768+0000 7f9d2d062700 1 -- 172.21.15.142:0/4199986452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d28073a50 msgr2=0x7f9d28111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:46.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.768+0000 7f9d2d062700 1 --2- 172.21.15.142:0/4199986452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d28073a50 0x7f9d28111630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009910 tx=0x7f9d10009c20 comp rx=0 tx=0).stop 2024-03-29T05:09:46.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.768+0000 7f9d2d062700 1 -- 172.21.15.142:0/4199986452 shutdown_connections 2024-03-29T05:09:46.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.768+0000 7f9d2d062700 1 --2- 172.21.15.142:0/4199986452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d28073a50 0x7f9d28111630 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:46.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.768+0000 7f9d2d062700 1 --2- 172.21.15.142:0/4199986452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d28073130 0x7f9d28073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:46.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.768+0000 7f9d2d062700 1 -- 172.21.15.142:0/4199986452 >> 172.21.15.142:0/4199986452 conn(0x7f9d280fc670 msgr2=0x7f9d280fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:46.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.768+0000 7f9d2d062700 1 -- 172.21.15.142:0/4199986452 shutdown_connections 2024-03-29T05:09:46.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.768+0000 7f9d2d062700 1 -- 172.21.15.142:0/4199986452 wait complete. 2024-03-29T05:09:46.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.769+0000 7f9d2d062700 1 Processor -- start 2024-03-29T05:09:46.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.769+0000 7f9d2d062700 1 -- start start 2024-03-29T05:09:46.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.769+0000 7f9d2d062700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d28073130 0x7f9d2819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:46.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.769+0000 7f9d2d062700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d28073a50 0x7f9d2819d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:46.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.769+0000 7f9d2d062700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2819d800 con 0x7f9d28073130 2024-03-29T05:09:46.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.769+0000 7f9d2d062700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2819d970 con 0x7f9d28073a50 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.769+0000 7f9d26d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d28073130 0x7f9d2819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.769+0000 7f9d26d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d28073130 0x7f9d2819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44690/0 (socket says 172.21.15.142:44690) 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.769+0000 7f9d26d9d700 1 -- 172.21.15.142:0/24521021 learned_addr learned my addr 172.21.15.142:0/24521021 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.769+0000 7f9d1ffff700 1 --2- 172.21.15.142:0/24521021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d28073a50 0x7f9d2819d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.770+0000 7f9d26d9d700 1 -- 172.21.15.142:0/24521021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d28073a50 msgr2=0x7f9d2819d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.770+0000 7f9d26d9d700 1 --2- 172.21.15.142:0/24521021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d28073a50 0x7f9d2819d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.770+0000 7f9d26d9d700 1 -- 172.21.15.142:0/24521021 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d100095f0 con 0x7f9d28073130 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.770+0000 7f9d26d9d700 1 --2- 172.21.15.142:0/24521021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d28073130 0x7f9d2819ccc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800c930 tx=0x7f9d1800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.770+0000 7f9d24d99700 1 -- 172.21.15.142:0/24521021 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d18007ad0 con 0x7f9d28073130 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.770+0000 7f9d24d99700 1 -- 172.21.15.142:0/24521021 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d1800ce80 con 0x7f9d28073130 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.770+0000 7f9d2d062700 1 -- 172.21.15.142:0/24521021 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d281a0710 con 0x7f9d28073130 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.770+0000 7f9d24d99700 1 -- 172.21.15.142:0/24521021 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d18018630 con 0x7f9d28073130 2024-03-29T05:09:46.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.770+0000 7f9d2d062700 1 -- 172.21.15.142:0/24521021 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d281a0c60 con 0x7f9d28073130 2024-03-29T05:09:46.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.771+0000 7f9d2d062700 1 -- 172.21.15.142:0/24521021 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2810edb0 con 0x7f9d28073130 2024-03-29T05:09:46.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.772+0000 7f9d24d99700 1 -- 172.21.15.142:0/24521021 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f9d18007c30 con 0x7f9d28073130 2024-03-29T05:09:46.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.772+0000 7f9d24d99700 1 --2- 172.21.15.142:0/24521021 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d1406dd60 0x7f9d14070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:46.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.772+0000 7f9d24d99700 1 -- 172.21.15.142:0/24521021 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f9d1808dc90 con 0x7f9d28073130 2024-03-29T05:09:46.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.772+0000 7f9d1ffff700 1 --2- 172.21.15.142:0/24521021 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d1406dd60 0x7f9d14070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:46.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.773+0000 7f9d1ffff700 1 --2- 172.21.15.142:0/24521021 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d1406dd60 0x7f9d14070220 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9d2819e2a0 tx=0x7f9d1001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:46.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.778+0000 7f9d24d99700 1 -- 172.21.15.142:0/24521021 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9d180570b0 con 0x7f9d28073130 2024-03-29T05:09:46.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.989+0000 7f9d2d062700 1 -- 172.21.15.142:0/24521021 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d2802cd80 con 0x7f9d1406dd60 2024-03-29T05:09:46.997 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.990+0000 7f9d24d99700 1 -- 172.21.15.142:0/24521021 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9d2802cd80 con 0x7f9d1406dd60 2024-03-29T05:09:47.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.992+0000 7f9d2d062700 1 -- 172.21.15.142:0/24521021 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d1406dd60 msgr2=0x7f9d14070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:47.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.992+0000 7f9d2d062700 1 --2- 172.21.15.142:0/24521021 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d1406dd60 0x7f9d14070220 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9d2819e2a0 tx=0x7f9d1001a040 comp rx=0 tx=0).stop 2024-03-29T05:09:47.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.992+0000 7f9d2d062700 1 -- 172.21.15.142:0/24521021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d28073130 msgr2=0x7f9d2819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:47.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.992+0000 7f9d2d062700 1 --2- 172.21.15.142:0/24521021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d28073130 0x7f9d2819ccc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800c930 tx=0x7f9d1800cc40 comp rx=0 tx=0).stop 2024-03-29T05:09:47.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.992+0000 7f9d2d062700 1 -- 172.21.15.142:0/24521021 shutdown_connections 2024-03-29T05:09:47.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.992+0000 7f9d2d062700 1 --2- 172.21.15.142:0/24521021 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9d1406dd60 0x7f9d14070220 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.992+0000 7f9d2d062700 1 --2- 172.21.15.142:0/24521021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d28073130 0x7f9d2819ccc0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.992+0000 7f9d2d062700 1 --2- 172.21.15.142:0/24521021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d28073a50 0x7f9d2819d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.992+0000 7f9d2d062700 1 -- 172.21.15.142:0/24521021 >> 172.21.15.142:0/24521021 conn(0x7f9d280fc670 msgr2=0x7f9d28103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:47.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.992+0000 7f9d2d062700 1 -- 172.21.15.142:0/24521021 shutdown_connections 2024-03-29T05:09:47.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:46.992+0000 7f9d2d062700 1 -- 172.21.15.142:0/24521021 wait complete. 2024-03-29T05:09:47.010 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:09:47.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.095+0000 7f07018c9700 1 -- 172.21.15.142:0/769276896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc105a50 msgr2=0x7f06fc105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:47.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.095+0000 7f07018c9700 1 --2- 172.21.15.142:0/769276896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc105a50 0x7f06fc105e30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009910 tx=0x7f06ec009c20 comp rx=0 tx=0).stop 2024-03-29T05:09:47.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.095+0000 7f07018c9700 1 -- 172.21.15.142:0/769276896 shutdown_connections 2024-03-29T05:09:47.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.095+0000 7f07018c9700 1 --2- 172.21.15.142:0/769276896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06fc0ffa50 0x7f06fc0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.095+0000 7f07018c9700 1 --2- 172.21.15.142:0/769276896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc105a50 0x7f06fc105e30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.095+0000 7f07018c9700 1 -- 172.21.15.142:0/769276896 >> 172.21.15.142:0/769276896 conn(0x7f06fc0fb540 msgr2=0x7f06fc0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:47.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.095+0000 7f07018c9700 1 -- 172.21.15.142:0/769276896 shutdown_connections 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.095+0000 7f07018c9700 1 -- 172.21.15.142:0/769276896 wait complete. 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.096+0000 7f07018c9700 1 Processor -- start 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.096+0000 7f07018c9700 1 -- start start 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.096+0000 7f07018c9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06fc0ffa50 0x7f06fc1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f07018c9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc105a50 0x7f06fc197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f07018c9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc198070 con 0x7f06fc105a50 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f07018c9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc1981e0 con 0x7f06fc0ffa50 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f06fa7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc105a50 0x7f06fc197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f06fa7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc105a50 0x7f06fc197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44704/0 (socket says 172.21.15.142:44704) 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f06fa7fc700 1 -- 172.21.15.142:0/1052965330 learned_addr learned my addr 172.21.15.142:0/1052965330 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f06faffd700 1 --2- 172.21.15.142:0/1052965330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06fc0ffa50 0x7f06fc1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f06fa7fc700 1 -- 172.21.15.142:0/1052965330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06fc0ffa50 msgr2=0x7f06fc1975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f06fa7fc700 1 --2- 172.21.15.142:0/1052965330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06fc0ffa50 0x7f06fc1975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f06fa7fc700 1 -- 172.21.15.142:0/1052965330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ec0095f0 con 0x7f06fc105a50 2024-03-29T05:09:47.109 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f06fa7fc700 1 --2- 172.21.15.142:0/1052965330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc105a50 0x7f06fc197ae0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c930 tx=0x7f06f000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:47.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f07008c7700 1 -- 172.21.15.142:0/1052965330 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06f0007ad0 con 0x7f06fc105a50 2024-03-29T05:09:47.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.097+0000 7f07008c7700 1 -- 172.21.15.142:0/1052965330 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06f0007c30 con 0x7f06fc105a50 2024-03-29T05:09:47.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.098+0000 7f07008c7700 1 -- 172.21.15.142:0/1052965330 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06f0018610 con 0x7f06fc105a50 2024-03-29T05:09:47.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.098+0000 7f07018c9700 1 -- 172.21.15.142:0/1052965330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc1a0610 con 0x7f06fc105a50 2024-03-29T05:09:47.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.098+0000 7f07018c9700 1 -- 172.21.15.142:0/1052965330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc1a0a20 con 0x7f06fc105a50 2024-03-29T05:09:47.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.099+0000 7f07018c9700 1 -- 172.21.15.142:0/1052965330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc108050 con 0x7f06fc105a50 2024-03-29T05:09:47.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.099+0000 7f07008c7700 1 -- 172.21.15.142:0/1052965330 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f06f000f450 con 0x7f06fc105a50 2024-03-29T05:09:47.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.100+0000 7f07008c7700 1 --2- 172.21.15.142:0/1052965330 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f06e406db50 0x7f06e4070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:47.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.100+0000 7f07008c7700 1 -- 172.21.15.142:0/1052965330 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f06f008dbc0 con 0x7f06fc105a50 2024-03-29T05:09:47.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.100+0000 7f06faffd700 1 --2- 172.21.15.142:0/1052965330 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f06e406db50 0x7f06e4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:47.110 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.100+0000 7f06faffd700 1 --2- 172.21.15.142:0/1052965330 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f06e406db50 0x7f06e4070010 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f06ec005f50 tx=0x7f06ec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:47.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.105+0000 7f07008c7700 1 -- 172.21.15.142:0/1052965330 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f06f0057060 con 0x7f06fc105a50 2024-03-29T05:09:47.324 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.317+0000 7f07018c9700 1 -- 172.21.15.142:0/1052965330 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06fc0611d0 con 0x7f06e406db50 2024-03-29T05:09:47.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.318+0000 7f07008c7700 1 -- 172.21.15.142:0/1052965330 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f06fc0611d0 con 0x7f06e406db50 2024-03-29T05:09:47.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.320+0000 7f07018c9700 1 -- 172.21.15.142:0/1052965330 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f06e406db50 msgr2=0x7f06e4070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:47.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.320+0000 7f07018c9700 1 --2- 172.21.15.142:0/1052965330 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f06e406db50 0x7f06e4070010 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f06ec005f50 tx=0x7f06ec01a040 comp rx=0 tx=0).stop 2024-03-29T05:09:47.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.320+0000 7f07018c9700 1 -- 172.21.15.142:0/1052965330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc105a50 msgr2=0x7f06fc197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:47.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.320+0000 7f07018c9700 1 --2- 172.21.15.142:0/1052965330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc105a50 0x7f06fc197ae0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c930 tx=0x7f06f000ccf0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.320+0000 7f07018c9700 1 -- 172.21.15.142:0/1052965330 shutdown_connections 2024-03-29T05:09:47.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.320+0000 7f07018c9700 1 --2- 172.21.15.142:0/1052965330 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f06e406db50 0x7f06e4070010 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.320+0000 7f07018c9700 1 --2- 172.21.15.142:0/1052965330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06fc0ffa50 0x7f06fc1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.320+0000 7f07018c9700 1 --2- 172.21.15.142:0/1052965330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc105a50 0x7f06fc197ae0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.320+0000 7f07018c9700 1 -- 172.21.15.142:0/1052965330 >> 172.21.15.142:0/1052965330 conn(0x7f06fc0fb540 msgr2=0x7f06fc0fbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:47.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.320+0000 7f07018c9700 1 -- 172.21.15.142:0/1052965330 shutdown_connections 2024-03-29T05:09:47.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.320+0000 7f07018c9700 1 -- 172.21.15.142:0/1052965330 wait complete. 2024-03-29T05:09:47.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.424+0000 7fe4f7229700 1 -- 172.21.15.142:0/3787309703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f00ffa40 msgr2=0x7fe4f00ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:47.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.424+0000 7fe4f7229700 1 --2- 172.21.15.142:0/3787309703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f00ffa40 0x7fe4f00ffec0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009910 tx=0x7fe4dc009c20 comp rx=0 tx=0).stop 2024-03-29T05:09:47.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.424+0000 7fe4f7229700 1 -- 172.21.15.142:0/3787309703 shutdown_connections 2024-03-29T05:09:47.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.424+0000 7fe4f7229700 1 --2- 172.21.15.142:0/3787309703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f00ffa40 0x7fe4f00ffec0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.424+0000 7fe4f7229700 1 --2- 172.21.15.142:0/3787309703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4f0105a40 0x7fe4f0105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.424+0000 7fe4f7229700 1 -- 172.21.15.142:0/3787309703 >> 172.21.15.142:0/3787309703 conn(0x7fe4f00fb570 msgr2=0x7fe4f00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:47.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.424+0000 7fe4f7229700 1 -- 172.21.15.142:0/3787309703 shutdown_connections 2024-03-29T05:09:47.432 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.424+0000 7fe4f7229700 1 -- 172.21.15.142:0/3787309703 wait complete. 2024-03-29T05:09:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f7229700 1 Processor -- start 2024-03-29T05:09:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f7229700 1 -- start start 2024-03-29T05:09:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f7229700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f00ffa40 0x7fe4f00fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f7229700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4f0105a40 0x7fe4f00fc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f7229700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00fc960 con 0x7fe4f00ffa40 2024-03-29T05:09:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f7229700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00fcaa0 con 0x7fe4f0105a40 2024-03-29T05:09:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f4fc5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f00ffa40 0x7fe4f00fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f4fc5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f00ffa40 0x7fe4f00fbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44714/0 (socket says 172.21.15.142:44714) 2024-03-29T05:09:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f4fc5700 1 -- 172.21.15.142:0/163277609 learned_addr learned my addr 172.21.15.142:0/163277609 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4effff700 1 --2- 172.21.15.142:0/163277609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4f0105a40 0x7fe4f00fc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f4fc5700 1 -- 172.21.15.142:0/163277609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4f0105a40 msgr2=0x7fe4f00fc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f4fc5700 1 --2- 172.21.15.142:0/163277609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4f0105a40 0x7fe4f00fc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f4fc5700 1 -- 172.21.15.142:0/163277609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc0095f0 con 0x7fe4f00ffa40 2024-03-29T05:09:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4f4fc5700 1 --2- 172.21.15.142:0/163277609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f00ffa40 0x7fe4f00fbee0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400b4b0 tx=0x7fe4e400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.425+0000 7fe4edffb700 1 -- 172.21.15.142:0/163277609 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4e4011800 con 0x7fe4f00ffa40 2024-03-29T05:09:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.426+0000 7fe4f7229700 1 -- 172.21.15.142:0/163277609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f01a29e0 con 0x7fe4f00ffa40 2024-03-29T05:09:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.426+0000 7fe4edffb700 1 -- 172.21.15.142:0/163277609 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4e4011e20 con 0x7fe4f00ffa40 2024-03-29T05:09:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.426+0000 7fe4edffb700 1 -- 172.21.15.142:0/163277609 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4e400f2e0 con 0x7fe4f00ffa40 2024-03-29T05:09:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.426+0000 7fe4f7229700 1 -- 172.21.15.142:0/163277609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01a2df0 con 0x7fe4f00ffa40 2024-03-29T05:09:47.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.427+0000 7fe4f7229700 1 -- 172.21.15.142:0/163277609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f0109070 con 0x7fe4f00ffa40 2024-03-29T05:09:47.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.427+0000 7fe4edffb700 1 -- 172.21.15.142:0/163277609 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fe4e4010380 con 0x7fe4f00ffa40 2024-03-29T05:09:47.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.428+0000 7fe4edffb700 1 --2- 172.21.15.142:0/163277609 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe4e006dd60 0x7fe4e0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:47.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.428+0000 7fe4edffb700 1 -- 172.21.15.142:0/163277609 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fe4e408d810 con 0x7fe4f00ffa40 2024-03-29T05:09:47.436 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.428+0000 7fe4effff700 1 --2- 172.21.15.142:0/163277609 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe4e006dd60 0x7fe4e0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:47.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.428+0000 7fe4effff700 1 --2- 172.21.15.142:0/163277609 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe4e006dd60 0x7fe4e0070220 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00fd6d0 tx=0x7fe4dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:47.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.432+0000 7fe4edffb700 1 -- 172.21.15.142:0/163277609 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe4e4056c30 con 0x7fe4f00ffa40 2024-03-29T05:09:47.647 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.639+0000 7fe4f7229700 1 -- 172.21.15.142:0/163277609 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4f002cd80 con 0x7fe4e006dd60 2024-03-29T05:09:47.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.650+0000 7fe4edffb700 1 -- 172.21.15.142:0/163277609 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe4f002cd80 con 0x7fe4e006dd60 2024-03-29T05:09:47.658 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:09:47.658 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (21m) 14m ago 33m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:09:47.658 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (33m) 14m ago 33m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:09:47.659 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (32m) 14m ago 32m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:09:47.659 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (18m) 14m ago 33m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:09:47.659 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (18m) 14m ago 32m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:09:47.659 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (21m) 14m ago 32m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:09:47.659 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (14m) 14m ago 27m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:09:47.659 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (15m) 14m ago 27m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:09:47.659 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (14m) 14m ago 27m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:09:47.659 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (14m) 14m ago 27m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:09:47.659 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (23m) 14m ago 34m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:09:47.659 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (22m) 14m ago 32m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:09:47.660 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (19m) 14m ago 34m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:09:47.660 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (19m) 14m ago 32m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:09:47.660 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (22m) 14m ago 33m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:09:47.660 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (22m) 14m ago 32m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:09:47.660 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (18m) 14m ago 31m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:09:47.660 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (17m) 14m ago 30m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:09:47.660 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (17m) 14m ago 30m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:09:47.660 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (16m) 14m ago 30m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:09:47.660 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (15m) 14m ago 29m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:09:47.660 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (15m) 14m ago 29m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:09:47.661 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (21m) 14m ago 32m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:09:47.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.652+0000 7fe4f7229700 1 -- 172.21.15.142:0/163277609 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe4e006dd60 msgr2=0x7fe4e0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:47.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.652+0000 7fe4f7229700 1 --2- 172.21.15.142:0/163277609 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe4e006dd60 0x7fe4e0070220 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00fd6d0 tx=0x7fe4dc01a040 comp rx=0 tx=0).stop 2024-03-29T05:09:47.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.652+0000 7fe4f7229700 1 -- 172.21.15.142:0/163277609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f00ffa40 msgr2=0x7fe4f00fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:47.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.652+0000 7fe4f7229700 1 --2- 172.21.15.142:0/163277609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f00ffa40 0x7fe4f00fbee0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400b4b0 tx=0x7fe4e400b7c0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.653+0000 7fe4f7229700 1 -- 172.21.15.142:0/163277609 shutdown_connections 2024-03-29T05:09:47.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.653+0000 7fe4f7229700 1 --2- 172.21.15.142:0/163277609 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe4e006dd60 0x7fe4e0070220 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.653+0000 7fe4f7229700 1 --2- 172.21.15.142:0/163277609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4f00ffa40 0x7fe4f00fbee0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.653+0000 7fe4f7229700 1 --2- 172.21.15.142:0/163277609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4f0105a40 0x7fe4f00fc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.653+0000 7fe4f7229700 1 -- 172.21.15.142:0/163277609 >> 172.21.15.142:0/163277609 conn(0x7fe4f00fb570 msgr2=0x7fe4f0068f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:47.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.653+0000 7fe4f7229700 1 -- 172.21.15.142:0/163277609 shutdown_connections 2024-03-29T05:09:47.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.653+0000 7fe4f7229700 1 -- 172.21.15.142:0/163277609 wait complete. 2024-03-29T05:09:47.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.757+0000 7f897751d700 1 -- 172.21.15.142:0/2148693189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8970100da0 msgr2=0x7f8970101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:47.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.757+0000 7f897751d700 1 --2- 172.21.15.142:0/2148693189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8970100da0 0x7f8970101180 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f896800b0a0 tx=0x7f896800b3b0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.757+0000 7f897751d700 1 -- 172.21.15.142:0/2148693189 shutdown_connections 2024-03-29T05:09:47.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.757+0000 7f897751d700 1 --2- 172.21.15.142:0/2148693189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8970101750 0x7f89701057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.757+0000 7f897751d700 1 --2- 172.21.15.142:0/2148693189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8970100da0 0x7f8970101180 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.757+0000 7f897751d700 1 -- 172.21.15.142:0/2148693189 >> 172.21.15.142:0/2148693189 conn(0x7f89700fc650 msgr2=0x7f89700fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:47.766 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:47 smithi142 ceph-mon[112475]: pgmap v644: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:47.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.757+0000 7f897751d700 1 -- 172.21.15.142:0/2148693189 shutdown_connections 2024-03-29T05:09:47.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.757+0000 7f897751d700 1 -- 172.21.15.142:0/2148693189 wait complete. 2024-03-29T05:09:47.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.758+0000 7f897751d700 1 Processor -- start 2024-03-29T05:09:47.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.758+0000 7f897751d700 1 -- start start 2024-03-29T05:09:47.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.758+0000 7f897751d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8970100da0 0x7f897019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:47.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.758+0000 7f897751d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8970101750 0x7f897019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:47.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.758+0000 7f897751d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897019d0f0 con 0x7f8970101750 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.758+0000 7f897751d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89701965a0 con 0x7f8970100da0 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.758+0000 7f8974ab8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8970101750 0x7f897019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.758+0000 7f8974ab8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8970101750 0x7f897019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44740/0 (socket says 172.21.15.142:44740) 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.758+0000 7f8974ab8700 1 -- 172.21.15.142:0/1613989186 learned_addr learned my addr 172.21.15.142:0/1613989186 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.759+0000 7f89752b9700 1 --2- 172.21.15.142:0/1613989186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8970100da0 0x7f897019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.759+0000 7f8974ab8700 1 -- 172.21.15.142:0/1613989186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8970100da0 msgr2=0x7f897019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.759+0000 7f8974ab8700 1 --2- 172.21.15.142:0/1613989186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8970100da0 0x7f897019c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.759+0000 7f8974ab8700 1 -- 172.21.15.142:0/1613989186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8968009d00 con 0x7f8970101750 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.759+0000 7f8974ab8700 1 --2- 172.21.15.142:0/1613989186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8970101750 0x7f897019ca60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f896000e8f0 tx=0x7f896000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.759+0000 7f89667fc700 1 -- 172.21.15.142:0/1613989186 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8960009790 con 0x7f8970101750 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.759+0000 7f89667fc700 1 -- 172.21.15.142:0/1613989186 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f896000c810 con 0x7f8970101750 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.759+0000 7f897751d700 1 -- 172.21.15.142:0/1613989186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8970196880 con 0x7f8970101750 2024-03-29T05:09:47.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.759+0000 7f89667fc700 1 -- 172.21.15.142:0/1613989186 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8960005580 con 0x7f8970101750 2024-03-29T05:09:47.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.759+0000 7f897751d700 1 -- 172.21.15.142:0/1613989186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8970196dd0 con 0x7f8970101750 2024-03-29T05:09:47.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.760+0000 7f897751d700 1 -- 172.21.15.142:0/1613989186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8970109140 con 0x7f8970101750 2024-03-29T05:09:47.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.761+0000 7f89667fc700 1 -- 172.21.15.142:0/1613989186 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f896000c980 con 0x7f8970101750 2024-03-29T05:09:47.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.761+0000 7f89667fc700 1 --2- 172.21.15.142:0/1613989186 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f895c06dd60 0x7f895c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:47.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.761+0000 7f89667fc700 1 -- 172.21.15.142:0/1613989186 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f8960014070 con 0x7f8970101750 2024-03-29T05:09:47.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.761+0000 7f89752b9700 1 --2- 172.21.15.142:0/1613989186 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f895c06dd60 0x7f895c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:47.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.762+0000 7f89752b9700 1 --2- 172.21.15.142:0/1613989186 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f895c06dd60 0x7f895c070220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8968000f80 tx=0x7f89680060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:47.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:47.766+0000 7f89667fc700 1 -- 172.21.15.142:0/1613989186 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f896005aeb0 con 0x7f8970101750 2024-03-29T05:09:48.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:47 smithi152 ceph-mon[88186]: pgmap v644: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:48.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.042+0000 7f897751d700 1 -- 172.21.15.142:0/1613989186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f897004ea90 con 0x7f8970101750 2024-03-29T05:09:48.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.043+0000 7f89667fc700 1 -- 172.21.15.142:0/1613989186 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f896005aa40 con 0x7f8970101750 2024-03-29T05:09:48.050 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:09:48.050 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:09:48.050 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:09:48.051 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:09:48.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.045+0000 7f897751d700 1 -- 172.21.15.142:0/1613989186 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f895c06dd60 msgr2=0x7f895c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:48.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.045+0000 7f897751d700 1 --2- 172.21.15.142:0/1613989186 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f895c06dd60 0x7f895c070220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8968000f80 tx=0x7f89680060c0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.045+0000 7f897751d700 1 -- 172.21.15.142:0/1613989186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8970101750 msgr2=0x7f897019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:48.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.045+0000 7f897751d700 1 --2- 172.21.15.142:0/1613989186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8970101750 0x7f897019ca60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f896000e8f0 tx=0x7f896000ec00 comp rx=0 tx=0).stop 2024-03-29T05:09:48.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.045+0000 7f897751d700 1 -- 172.21.15.142:0/1613989186 shutdown_connections 2024-03-29T05:09:48.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.045+0000 7f897751d700 1 --2- 172.21.15.142:0/1613989186 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f895c06dd60 0x7f895c070220 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.045+0000 7f897751d700 1 --2- 172.21.15.142:0/1613989186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8970100da0 0x7f897019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.045+0000 7f897751d700 1 --2- 172.21.15.142:0/1613989186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8970101750 0x7f897019ca60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.045+0000 7f897751d700 1 -- 172.21.15.142:0/1613989186 >> 172.21.15.142:0/1613989186 conn(0x7f89700fc650 msgr2=0x7f89700fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:48.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.045+0000 7f897751d700 1 -- 172.21.15.142:0/1613989186 shutdown_connections 2024-03-29T05:09:48.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.045+0000 7f897751d700 1 -- 172.21.15.142:0/1613989186 wait complete. 2024-03-29T05:09:48.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.147+0000 7feb82e2f700 1 -- 172.21.15.142:0/3729686276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7c100de0 msgr2=0x7feb7c1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:48.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.147+0000 7feb82e2f700 1 --2- 172.21.15.142:0/3729686276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7c100de0 0x7feb7c1011c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7feb70009910 tx=0x7feb70009c20 comp rx=0 tx=0).stop 2024-03-29T05:09:48.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.147+0000 7feb82e2f700 1 -- 172.21.15.142:0/3729686276 shutdown_connections 2024-03-29T05:09:48.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.147+0000 7feb82e2f700 1 --2- 172.21.15.142:0/3729686276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb7c101790 0x7feb7c1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.147+0000 7feb82e2f700 1 --2- 172.21.15.142:0/3729686276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7c100de0 0x7feb7c1011c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.147+0000 7feb82e2f700 1 -- 172.21.15.142:0/3729686276 >> 172.21.15.142:0/3729686276 conn(0x7feb7c0fc6b0 msgr2=0x7feb7c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:48.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.147+0000 7feb82e2f700 1 -- 172.21.15.142:0/3729686276 shutdown_connections 2024-03-29T05:09:48.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.147+0000 7feb82e2f700 1 -- 172.21.15.142:0/3729686276 wait complete. 2024-03-29T05:09:48.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb82e2f700 1 Processor -- start 2024-03-29T05:09:48.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb82e2f700 1 -- start start 2024-03-29T05:09:48.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb82e2f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb7c100de0 0x7feb7c19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:48.157 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb82e2f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7c101790 0x7feb7c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb82e2f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c19d100 con 0x7feb7c101790 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb82e2f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c196640 con 0x7feb7c100de0 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb7bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7c101790 0x7feb7c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb7bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7c101790 0x7feb7c19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44750/0 (socket says 172.21.15.142:44750) 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb7bfff700 1 -- 172.21.15.142:0/2568626148 learned_addr learned my addr 172.21.15.142:0/2568626148 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb80bcb700 1 --2- 172.21.15.142:0/2568626148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb7c100de0 0x7feb7c19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb7bfff700 1 -- 172.21.15.142:0/2568626148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb7c100de0 msgr2=0x7feb7c19c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb7bfff700 1 --2- 172.21.15.142:0/2568626148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb7c100de0 0x7feb7c19c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.148+0000 7feb7bfff700 1 -- 172.21.15.142:0/2568626148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb700095f0 con 0x7feb7c101790 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.149+0000 7feb7bfff700 1 --2- 172.21.15.142:0/2568626148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7c101790 0x7feb7c19cb00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7feb6800c960 tx=0x7feb6800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.149+0000 7feb79ffb700 1 -- 172.21.15.142:0/2568626148 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb68007a70 con 0x7feb7c101790 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.149+0000 7feb79ffb700 1 -- 172.21.15.142:0/2568626148 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb68007bd0 con 0x7feb7c101790 2024-03-29T05:09:48.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.149+0000 7feb82e2f700 1 -- 172.21.15.142:0/2568626148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7c196920 con 0x7feb7c101790 2024-03-29T05:09:48.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.149+0000 7feb79ffb700 1 -- 172.21.15.142:0/2568626148 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb68018630 con 0x7feb7c101790 2024-03-29T05:09:48.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.149+0000 7feb82e2f700 1 -- 172.21.15.142:0/2568626148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c196e70 con 0x7feb7c101790 2024-03-29T05:09:48.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.150+0000 7feb82e2f700 1 -- 172.21.15.142:0/2568626148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7c109180 con 0x7feb7c101790 2024-03-29T05:09:48.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.150+0000 7feb79ffb700 1 -- 172.21.15.142:0/2568626148 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7feb6800f450 con 0x7feb7c101790 2024-03-29T05:09:48.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.151+0000 7feb79ffb700 1 --2- 172.21.15.142:0/2568626148 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7feb6c06dd60 0x7feb6c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:48.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.151+0000 7feb79ffb700 1 -- 172.21.15.142:0/2568626148 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7feb6808dbf0 con 0x7feb7c101790 2024-03-29T05:09:48.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.151+0000 7feb80bcb700 1 --2- 172.21.15.142:0/2568626148 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7feb6c06dd60 0x7feb6c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:48.160 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.151+0000 7feb80bcb700 1 --2- 172.21.15.142:0/2568626148 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7feb6c06dd60 0x7feb6c070220 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feb70006010 tx=0x7feb7001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:48.163 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.155+0000 7feb79ffb700 1 -- 172.21.15.142:0/2568626148 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7feb6805ab40 con 0x7feb7c101790 2024-03-29T05:09:48.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.394+0000 7feb82e2f700 1 -- 172.21.15.142:0/2568626148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7feb7c1970f0 con 0x7feb7c101790 2024-03-29T05:09:48.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.394+0000 7feb79ffb700 1 -- 172.21.15.142:0/2568626148 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7feb6805a6d0 con 0x7feb7c101790 2024-03-29T05:09:48.402 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:09:48.402 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:09:48.402 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:09:48.403 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:09:48.404 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:09:48.405 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:09:48.405 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:09:48.405 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:09:48.405 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:09:48.405 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:09:48.405 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:09:48.405 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:09:48.405 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:09:48.405 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:09:48.405 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:09:48.405 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:09:48.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.397+0000 7feb82e2f700 1 -- 172.21.15.142:0/2568626148 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7feb6c06dd60 msgr2=0x7feb6c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:48.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.397+0000 7feb82e2f700 1 --2- 172.21.15.142:0/2568626148 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7feb6c06dd60 0x7feb6c070220 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7feb70006010 tx=0x7feb7001a040 comp rx=0 tx=0).stop 2024-03-29T05:09:48.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.397+0000 7feb82e2f700 1 -- 172.21.15.142:0/2568626148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7c101790 msgr2=0x7feb7c19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:48.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.397+0000 7feb82e2f700 1 --2- 172.21.15.142:0/2568626148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7c101790 0x7feb7c19cb00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7feb6800c960 tx=0x7feb6800cc70 comp rx=0 tx=0).stop 2024-03-29T05:09:48.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.397+0000 7feb82e2f700 1 -- 172.21.15.142:0/2568626148 shutdown_connections 2024-03-29T05:09:48.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.397+0000 7feb82e2f700 1 --2- 172.21.15.142:0/2568626148 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7feb6c06dd60 0x7feb6c070220 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.397+0000 7feb82e2f700 1 --2- 172.21.15.142:0/2568626148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb7c100de0 0x7feb7c19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.397+0000 7feb82e2f700 1 --2- 172.21.15.142:0/2568626148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7c101790 0x7feb7c19cb00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.397+0000 7feb82e2f700 1 -- 172.21.15.142:0/2568626148 >> 172.21.15.142:0/2568626148 conn(0x7feb7c0fc6b0 msgr2=0x7feb7c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:48.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.397+0000 7feb82e2f700 1 -- 172.21.15.142:0/2568626148 shutdown_connections 2024-03-29T05:09:48.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.397+0000 7feb82e2f700 1 -- 172.21.15.142:0/2568626148 wait complete. 2024-03-29T05:09:48.407 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:09:48.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.502+0000 7f456e461700 1 -- 172.21.15.142:0/278481393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4568069020 msgr2=0x7f4568105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:48.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.502+0000 7f456e461700 1 --2- 172.21.15.142:0/278481393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4568069020 0x7f4568105790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f455c009910 tx=0x7f455c009c20 comp rx=0 tx=0).stop 2024-03-29T05:09:48.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.502+0000 7f456e461700 1 -- 172.21.15.142:0/278481393 shutdown_connections 2024-03-29T05:09:48.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.502+0000 7f456e461700 1 --2- 172.21.15.142:0/278481393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4568069020 0x7f4568105790 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.502+0000 7f456e461700 1 --2- 172.21.15.142:0/278481393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4568068670 0x7f4568068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.502+0000 7f456e461700 1 -- 172.21.15.142:0/278481393 >> 172.21.15.142:0/278481393 conn(0x7f45680755a0 msgr2=0x7f45680759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:48.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.502+0000 7f456e461700 1 -- 172.21.15.142:0/278481393 shutdown_connections 2024-03-29T05:09:48.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.502+0000 7f456e461700 1 -- 172.21.15.142:0/278481393 wait complete. 2024-03-29T05:09:48.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.503+0000 7f456e461700 1 Processor -- start 2024-03-29T05:09:48.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.503+0000 7f456e461700 1 -- start start 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.503+0000 7f456e461700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4568068670 0x7f4568196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.503+0000 7f456e461700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4568069020 0x7f4568196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.503+0000 7f456e461700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45681972d0 con 0x7f4568068670 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.503+0000 7f456e461700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456819a000 con 0x7f4568069020 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.503+0000 7f4567fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4568068670 0x7f4568196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.503+0000 7f4567fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4568068670 0x7f4568196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58670/0 (socket says 172.21.15.142:58670) 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.503+0000 7f4567fff700 1 -- 172.21.15.142:0/2516584613 learned_addr learned my addr 172.21.15.142:0/2516584613 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.503+0000 7f45677fe700 1 --2- 172.21.15.142:0/2516584613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4568069020 0x7f4568196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.504+0000 7f4567fff700 1 -- 172.21.15.142:0/2516584613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4568069020 msgr2=0x7f4568196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.504+0000 7f4567fff700 1 --2- 172.21.15.142:0/2516584613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4568069020 0x7f4568196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.504+0000 7f4567fff700 1 -- 172.21.15.142:0/2516584613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c0095f0 con 0x7f4568068670 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.504+0000 7f45677fe700 1 --2- 172.21.15.142:0/2516584613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4568069020 0x7f4568196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:09:48.513 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.504+0000 7f4567fff700 1 --2- 172.21.15.142:0/2516584613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4568068670 0x7f4568196700 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f455800d8d0 tx=0x7f455800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:48.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.504+0000 7f45657fa700 1 -- 172.21.15.142:0/2516584613 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4558009940 con 0x7f4568068670 2024-03-29T05:09:48.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.504+0000 7f45657fa700 1 -- 172.21.15.142:0/2516584613 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4558010460 con 0x7f4568068670 2024-03-29T05:09:48.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.504+0000 7f45657fa700 1 -- 172.21.15.142:0/2516584613 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f455800f5f0 con 0x7f4568068670 2024-03-29T05:09:48.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.504+0000 7f456e461700 1 -- 172.21.15.142:0/2516584613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456819a2e0 con 0x7f4568068670 2024-03-29T05:09:48.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.504+0000 7f456e461700 1 -- 172.21.15.142:0/2516584613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456819a800 con 0x7f4568068670 2024-03-29T05:09:48.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.505+0000 7f456e461700 1 -- 172.21.15.142:0/2516584613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45681090a0 con 0x7f4568068670 2024-03-29T05:09:48.514 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.505+0000 7f45657fa700 1 -- 172.21.15.142:0/2516584613 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f45580105d0 con 0x7f4568068670 2024-03-29T05:09:48.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.506+0000 7f45657fa700 1 --2- 172.21.15.142:0/2516584613 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f455006dd60 0x7f4550070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:48.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.506+0000 7f45657fa700 1 -- 172.21.15.142:0/2516584613 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f455808db80 con 0x7f4568068670 2024-03-29T05:09:48.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.506+0000 7f45677fe700 1 --2- 172.21.15.142:0/2516584613 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f455006dd60 0x7f4550070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:48.515 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.506+0000 7f45677fe700 1 --2- 172.21.15.142:0/2516584613 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f455006dd60 0x7f4550070220 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4568197e20 tx=0x7f455c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:48.518 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.510+0000 7f45657fa700 1 -- 172.21.15.142:0/2516584613 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4558057020 con 0x7f4568068670 2024-03-29T05:09:48.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.722+0000 7f456e461700 1 -- 172.21.15.142:0/2516584613 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f456819aa70 con 0x7f455006dd60 2024-03-29T05:09:48.730 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:48 smithi142 ceph-mon[112475]: from='client.35186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:48.730 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:48 smithi142 ceph-mon[112475]: from='client.35190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:48.731 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:48 smithi142 ceph-mon[112475]: from='client.35194 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:48.731 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:48 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1613989186' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:09:48.731 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:48 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2568626148' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:09:48.731 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.724+0000 7f45657fa700 1 -- 172.21.15.142:0/2516584613 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f456819aa70 con 0x7f455006dd60 2024-03-29T05:09:48.732 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:09:48.732 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:09:48.732 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:09:48.732 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:09:48.732 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:09:48.732 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:09:48.732 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:09:48.732 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:09:48.732 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:09:48.732 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:09:48.732 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:09:48.732 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:09:48.733 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:09:48.733 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:09:48.733 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:09:48.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.726+0000 7f456e461700 1 -- 172.21.15.142:0/2516584613 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f455006dd60 msgr2=0x7f4550070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:48.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.726+0000 7f456e461700 1 --2- 172.21.15.142:0/2516584613 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f455006dd60 0x7f4550070220 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4568197e20 tx=0x7f455c01a040 comp rx=0 tx=0).stop 2024-03-29T05:09:48.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.726+0000 7f456e461700 1 -- 172.21.15.142:0/2516584613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4568068670 msgr2=0x7f4568196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:48.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.726+0000 7f456e461700 1 --2- 172.21.15.142:0/2516584613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4568068670 0x7f4568196700 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f455800d8d0 tx=0x7f455800dc90 comp rx=0 tx=0).stop 2024-03-29T05:09:48.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.726+0000 7f456e461700 1 -- 172.21.15.142:0/2516584613 shutdown_connections 2024-03-29T05:09:48.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.726+0000 7f456e461700 1 --2- 172.21.15.142:0/2516584613 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f455006dd60 0x7f4550070220 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.726+0000 7f456e461700 1 --2- 172.21.15.142:0/2516584613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4568068670 0x7f4568196700 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.726+0000 7f456e461700 1 --2- 172.21.15.142:0/2516584613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4568069020 0x7f4568196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.726+0000 7f456e461700 1 -- 172.21.15.142:0/2516584613 >> 172.21.15.142:0/2516584613 conn(0x7f45680755a0 msgr2=0x7f4568105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:48.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.726+0000 7f456e461700 1 -- 172.21.15.142:0/2516584613 shutdown_connections 2024-03-29T05:09:48.734 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.726+0000 7f456e461700 1 -- 172.21.15.142:0/2516584613 wait complete. 2024-03-29T05:09:48.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.829+0000 7f9362390700 1 -- 172.21.15.142:0/4159512896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f935c100dc0 msgr2=0x7f935c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:48.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.829+0000 7f9362390700 1 --2- 172.21.15.142:0/4159512896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f935c100dc0 0x7f935c1011a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f934c009910 tx=0x7f934c009c20 comp rx=0 tx=0).stop 2024-03-29T05:09:48.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.829+0000 7f9362390700 1 -- 172.21.15.142:0/4159512896 shutdown_connections 2024-03-29T05:09:48.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.829+0000 7f9362390700 1 --2- 172.21.15.142:0/4159512896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f935c101770 0x7f935c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.829+0000 7f9362390700 1 --2- 172.21.15.142:0/4159512896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f935c100dc0 0x7f935c1011a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.829+0000 7f9362390700 1 -- 172.21.15.142:0/4159512896 >> 172.21.15.142:0/4159512896 conn(0x7f935c0fc650 msgr2=0x7f935c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:48.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.830+0000 7f9362390700 1 -- 172.21.15.142:0/4159512896 shutdown_connections 2024-03-29T05:09:48.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.830+0000 7f9362390700 1 -- 172.21.15.142:0/4159512896 wait complete. 2024-03-29T05:09:48.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.830+0000 7f9362390700 1 Processor -- start 2024-03-29T05:09:48.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.830+0000 7f9362390700 1 -- start start 2024-03-29T05:09:48.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.830+0000 7f9362390700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f935c100dc0 0x7f935c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:48.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.830+0000 7f9362390700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f935c101770 0x7f935c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:48.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.830+0000 7f9362390700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c19d080 con 0x7f935c100dc0 2024-03-29T05:09:48.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.830+0000 7f9362390700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c1965c0 con 0x7f935c101770 2024-03-29T05:09:48.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f935bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f935c100dc0 0x7f935c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:48.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f935bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f935c100dc0 0x7f935c19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:58692/0 (socket says 172.21.15.142:58692) 2024-03-29T05:09:48.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f935bfff700 1 -- 172.21.15.142:0/4153350312 learned_addr learned my addr 172.21.15.142:0/4153350312 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:09:48.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f935b7fe700 1 --2- 172.21.15.142:0/4153350312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f935c101770 0x7f935c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:48.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f935bfff700 1 -- 172.21.15.142:0/4153350312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f935c101770 msgr2=0x7f935c19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:48.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f935bfff700 1 --2- 172.21.15.142:0/4153350312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f935c101770 0x7f935c19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:48.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f935bfff700 1 -- 172.21.15.142:0/4153350312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c0095f0 con 0x7f935c100dc0 2024-03-29T05:09:48.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f935bfff700 1 --2- 172.21.15.142:0/4153350312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f935c100dc0 0x7f935c19c540 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f934c006010 tx=0x7f934c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:48.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f93597fa700 1 -- 172.21.15.142:0/4153350312 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f934c01d070 con 0x7f935c100dc0 2024-03-29T05:09:48.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f9362390700 1 -- 172.21.15.142:0/4153350312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c196840 con 0x7f935c100dc0 2024-03-29T05:09:48.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f93597fa700 1 -- 172.21.15.142:0/4153350312 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f934c00f460 con 0x7f935c100dc0 2024-03-29T05:09:48.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f93597fa700 1 -- 172.21.15.142:0/4153350312 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f934c017630 con 0x7f935c100dc0 2024-03-29T05:09:48.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.831+0000 7f9362390700 1 -- 172.21.15.142:0/4153350312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c196d30 con 0x7f935c100dc0 2024-03-29T05:09:48.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.832+0000 7f9362390700 1 -- 172.21.15.142:0/4153350312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c109160 con 0x7f935c100dc0 2024-03-29T05:09:48.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.833+0000 7f93597fa700 1 -- 172.21.15.142:0/4153350312 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f934c00f5d0 con 0x7f935c100dc0 2024-03-29T05:09:48.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.833+0000 7f93597fa700 1 --2- 172.21.15.142:0/4153350312 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f934406dd60 0x7f9344070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:09:48.843 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.833+0000 7f93597fa700 1 -- 172.21.15.142:0/4153350312 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f934c08f1f0 con 0x7f935c100dc0 2024-03-29T05:09:48.843 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.833+0000 7f935b7fe700 1 --2- 172.21.15.142:0/4153350312 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f934406dd60 0x7f9344070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:09:48.843 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.834+0000 7f935b7fe700 1 --2- 172.21.15.142:0/4153350312 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f934406dd60 0x7f9344070220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f935c1059b0 tx=0x7f935000a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:09:48.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:48.838+0000 7f93597fa700 1 -- 172.21.15.142:0/4153350312 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f934c05c140 con 0x7f935c100dc0 2024-03-29T05:09:49.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:48 smithi152 ceph-mon[88186]: from='client.35186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:49.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:48 smithi152 ceph-mon[88186]: from='client.35190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:49.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:48 smithi152 ceph-mon[88186]: from='client.35194 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:49.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:48 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1613989186' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:09:49.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:48 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2568626148' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:09:49.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.112+0000 7f9362390700 1 -- 172.21.15.142:0/4153350312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f935c108b40 con 0x7f935c100dc0 2024-03-29T05:09:49.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.112+0000 7f93597fa700 1 -- 172.21.15.142:0/4153350312 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f934c05bcd0 con 0x7f935c100dc0 2024-03-29T05:09:49.120 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:09:49.120 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:09:49.120 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:09:49.120 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:09:49.120 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:09:49.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.114+0000 7f9362390700 1 -- 172.21.15.142:0/4153350312 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f934406dd60 msgr2=0x7f9344070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:49.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.114+0000 7f9362390700 1 --2- 172.21.15.142:0/4153350312 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f934406dd60 0x7f9344070220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f935c1059b0 tx=0x7f935000a380 comp rx=0 tx=0).stop 2024-03-29T05:09:49.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.114+0000 7f9362390700 1 -- 172.21.15.142:0/4153350312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f935c100dc0 msgr2=0x7f935c19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:09:49.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.114+0000 7f9362390700 1 --2- 172.21.15.142:0/4153350312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f935c100dc0 0x7f935c19c540 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f934c006010 tx=0x7f934c00bd70 comp rx=0 tx=0).stop 2024-03-29T05:09:49.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.114+0000 7f9362390700 1 -- 172.21.15.142:0/4153350312 shutdown_connections 2024-03-29T05:09:49.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.114+0000 7f9362390700 1 --2- 172.21.15.142:0/4153350312 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f934406dd60 0x7f9344070220 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:49.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.114+0000 7f9362390700 1 --2- 172.21.15.142:0/4153350312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f935c100dc0 0x7f935c19c540 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:49.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.114+0000 7f9362390700 1 --2- 172.21.15.142:0/4153350312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f935c101770 0x7f935c19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:09:49.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.114+0000 7f9362390700 1 -- 172.21.15.142:0/4153350312 >> 172.21.15.142:0/4153350312 conn(0x7f935c0fc650 msgr2=0x7f935c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:09:49.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.114+0000 7f9362390700 1 -- 172.21.15.142:0/4153350312 shutdown_connections 2024-03-29T05:09:49.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:09:49.114+0000 7f9362390700 1 -- 172.21.15.142:0/4153350312 wait complete. 2024-03-29T05:09:49.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:49 smithi142 ceph-mon[112475]: pgmap v645: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:49.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:49 smithi142 ceph-mon[112475]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:49.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:49 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/4153350312' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:09:50.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:49 smithi152 ceph-mon[88186]: pgmap v645: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:50.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:49 smithi152 ceph-mon[88186]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:09:50.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:49 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/4153350312' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:09:51.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:51 smithi142 ceph-mon[112475]: pgmap v646: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:52.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:51 smithi152 ceph-mon[88186]: pgmap v646: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:54.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:53 smithi152 ceph-mon[88186]: pgmap v647: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:54.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:53 smithi142 ceph-mon[112475]: pgmap v647: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:56.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:55 smithi152 ceph-mon[88186]: pgmap v648: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:56.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:55 smithi142 ceph-mon[112475]: pgmap v648: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:58.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:57 smithi152 ceph-mon[88186]: pgmap v649: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:58.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:57 smithi142 ceph-mon[112475]: pgmap v649: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:59.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:59 smithi142 ceph-mon[112475]: pgmap v650: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:09:59.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:09:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:10:00.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:59 smithi152 ceph-mon[88186]: pgmap v650: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:00.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:09:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:10:00.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:00 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[112471]: 2024-03-29T05:09:59.999+0000 7f6a4d4b4700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:10:01.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:00 smithi152 ceph-mon[88186]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:10:01.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:00 smithi142 ceph-mon[112475]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:10:01.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:01 smithi142 ceph-mon[112475]: pgmap v651: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:02.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:01 smithi152 ceph-mon[88186]: pgmap v651: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:04.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:03 smithi152 ceph-mon[88186]: pgmap v652: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:04.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:03 smithi142 ceph-mon[112475]: pgmap v652: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:06.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:05 smithi152 ceph-mon[88186]: pgmap v653: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:06.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:05 smithi142 ceph-mon[112475]: pgmap v653: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:07.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:07 smithi152 ceph-mon[88186]: pgmap v654: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:08.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:07 smithi142 ceph-mon[112475]: pgmap v654: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:10.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:09 smithi152 ceph-mon[88186]: pgmap v655: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:10.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:09 smithi142 ceph-mon[112475]: pgmap v655: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:11.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:11 smithi142 ceph-mon[112475]: pgmap v656: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:12.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:11 smithi152 ceph-mon[88186]: pgmap v656: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:14.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:13 smithi152 ceph-mon[88186]: pgmap v657: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:14.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:13 smithi142 ceph-mon[112475]: pgmap v657: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:15.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:10:15.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:10:16.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:15 smithi152 ceph-mon[88186]: pgmap v658: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:16.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:15 smithi142 ceph-mon[112475]: pgmap v658: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:17.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:17 smithi152 ceph-mon[88186]: pgmap v659: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:18.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:17 smithi142 ceph-mon[112475]: pgmap v659: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:19.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.231+0000 7f7fee8d7700 1 -- 172.21.15.142:0/380527026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe81026b0 msgr2=0x7f7fe810aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:19.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.231+0000 7f7fee8d7700 1 --2- 172.21.15.142:0/380527026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe81026b0 0x7f7fe810aba0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009940 tx=0x7f7fdc009c50 comp rx=0 tx=0).stop 2024-03-29T05:10:19.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.231+0000 7f7fee8d7700 1 -- 172.21.15.142:0/380527026 shutdown_connections 2024-03-29T05:10:19.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.231+0000 7f7fee8d7700 1 --2- 172.21.15.142:0/380527026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe81026b0 0x7f7fe810aba0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.231+0000 7f7fee8d7700 1 --2- 172.21.15.142:0/380527026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fe8101d90 0x7f7fe8102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.231+0000 7f7fee8d7700 1 -- 172.21.15.142:0/380527026 >> 172.21.15.142:0/380527026 conn(0x7f7fe80fb540 msgr2=0x7f7fe80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:19.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.231+0000 7f7fee8d7700 1 -- 172.21.15.142:0/380527026 shutdown_connections 2024-03-29T05:10:19.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.231+0000 7f7fee8d7700 1 -- 172.21.15.142:0/380527026 wait complete. 2024-03-29T05:10:19.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fee8d7700 1 Processor -- start 2024-03-29T05:10:19.240 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fee8d7700 1 -- start start 2024-03-29T05:10:19.241 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fee8d7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe8101d90 0x7f7fe819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:19.241 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fee8d7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fe81026b0 0x7f7fe819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:19.241 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fee8d7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe819d2a0 con 0x7f7fe8101d90 2024-03-29T05:10:19.241 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fee8d7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8196750 con 0x7f7fe81026b0 2024-03-29T05:10:19.241 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fe7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe8101d90 0x7f7fe819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:19.241 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fe7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe8101d90 0x7f7fe819c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45356/0 (socket says 172.21.15.142:45356) 2024-03-29T05:10:19.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fe7fff700 1 -- 172.21.15.142:0/1586673672 learned_addr learned my addr 172.21.15.142:0/1586673672 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:19.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fe77fe700 1 --2- 172.21.15.142:0/1586673672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fe81026b0 0x7f7fe819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:19.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fe7fff700 1 -- 172.21.15.142:0/1586673672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fe81026b0 msgr2=0x7f7fe819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:19.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fe7fff700 1 --2- 172.21.15.142:0/1586673672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fe81026b0 0x7f7fe819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.232+0000 7f7fe7fff700 1 -- 172.21.15.142:0/1586673672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc0095f0 con 0x7f7fe8101d90 2024-03-29T05:10:19.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.233+0000 7f7fe7fff700 1 --2- 172.21.15.142:0/1586673672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe8101d90 0x7f7fe819c6d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800b4b0 tx=0x7f7fd800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:19.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.233+0000 7f7fe57fa700 1 -- 172.21.15.142:0/1586673672 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd8011800 con 0x7f7fe8101d90 2024-03-29T05:10:19.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.233+0000 7f7fee8d7700 1 -- 172.21.15.142:0/1586673672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe8196a30 con 0x7f7fe8101d90 2024-03-29T05:10:19.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.233+0000 7f7fe57fa700 1 -- 172.21.15.142:0/1586673672 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fd8011e20 con 0x7f7fe8101d90 2024-03-29T05:10:19.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.233+0000 7f7fe57fa700 1 -- 172.21.15.142:0/1586673672 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd800f2e0 con 0x7f7fe8101d90 2024-03-29T05:10:19.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.233+0000 7f7fee8d7700 1 -- 172.21.15.142:0/1586673672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe8196f50 con 0x7f7fe8101d90 2024-03-29T05:10:19.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.234+0000 7f7fee8d7700 1 -- 172.21.15.142:0/1586673672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe81082a0 con 0x7f7fe8101d90 2024-03-29T05:10:19.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.234+0000 7f7fe57fa700 1 -- 172.21.15.142:0/1586673672 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f7fd8010380 con 0x7f7fe8101d90 2024-03-29T05:10:19.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.235+0000 7f7fe57fa700 1 --2- 172.21.15.142:0/1586673672 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7fd006dd60 0x7f7fd0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:19.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.235+0000 7f7fe57fa700 1 -- 172.21.15.142:0/1586673672 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f7fd808d8b0 con 0x7f7fe8101d90 2024-03-29T05:10:19.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.235+0000 7f7fe77fe700 1 --2- 172.21.15.142:0/1586673672 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7fd006dd60 0x7f7fd0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:19.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.235+0000 7f7fe77fe700 1 --2- 172.21.15.142:0/1586673672 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7fd006dd60 0x7f7fd0070220 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc006010 tx=0x7f7fdc005c20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:19.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.239+0000 7f7fe57fa700 1 -- 172.21.15.142:0/1586673672 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7fd805a800 con 0x7f7fe8101d90 2024-03-29T05:10:19.457 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.450+0000 7f7fee8d7700 1 -- 172.21.15.142:0/1586673672 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fe81971d0 con 0x7f7fd006dd60 2024-03-29T05:10:19.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.451+0000 7f7fe57fa700 1 -- 172.21.15.142:0/1586673672 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7fe81971d0 con 0x7f7fd006dd60 2024-03-29T05:10:19.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.453+0000 7f7fee8d7700 1 -- 172.21.15.142:0/1586673672 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7fd006dd60 msgr2=0x7f7fd0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:19.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.453+0000 7f7fee8d7700 1 --2- 172.21.15.142:0/1586673672 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7fd006dd60 0x7f7fd0070220 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc006010 tx=0x7f7fdc005c20 comp rx=0 tx=0).stop 2024-03-29T05:10:19.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.453+0000 7f7fee8d7700 1 -- 172.21.15.142:0/1586673672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe8101d90 msgr2=0x7f7fe819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:19.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.453+0000 7f7fee8d7700 1 --2- 172.21.15.142:0/1586673672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe8101d90 0x7f7fe819c6d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800b4b0 tx=0x7f7fd800b870 comp rx=0 tx=0).stop 2024-03-29T05:10:19.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.453+0000 7f7fee8d7700 1 -- 172.21.15.142:0/1586673672 shutdown_connections 2024-03-29T05:10:19.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.453+0000 7f7fee8d7700 1 --2- 172.21.15.142:0/1586673672 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7fd006dd60 0x7f7fd0070220 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.453+0000 7f7fee8d7700 1 --2- 172.21.15.142:0/1586673672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe8101d90 0x7f7fe819c6d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.453+0000 7f7fee8d7700 1 --2- 172.21.15.142:0/1586673672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fe81026b0 0x7f7fe819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.453+0000 7f7fee8d7700 1 -- 172.21.15.142:0/1586673672 >> 172.21.15.142:0/1586673672 conn(0x7f7fe80fb540 msgr2=0x7f7fe81053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:19.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.454+0000 7f7fee8d7700 1 -- 172.21.15.142:0/1586673672 shutdown_connections 2024-03-29T05:10:19.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.454+0000 7f7fee8d7700 1 -- 172.21.15.142:0/1586673672 wait complete. 2024-03-29T05:10:19.471 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:10:19.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.557+0000 7f78eeef1700 1 -- 172.21.15.142:0/1724603094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e80ffa70 msgr2=0x7f78e80ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:19.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.557+0000 7f78eeef1700 1 --2- 172.21.15.142:0/1724603094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e80ffa70 0x7f78e80ffef0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009910 tx=0x7f78d8009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:19.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.557+0000 7f78eeef1700 1 -- 172.21.15.142:0/1724603094 shutdown_connections 2024-03-29T05:10:19.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.557+0000 7f78eeef1700 1 --2- 172.21.15.142:0/1724603094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e80ffa70 0x7f78e80ffef0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.557+0000 7f78eeef1700 1 --2- 172.21.15.142:0/1724603094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e8105a70 0x7f78e8105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.557+0000 7f78eeef1700 1 -- 172.21.15.142:0/1724603094 >> 172.21.15.142:0/1724603094 conn(0x7f78e80fb560 msgr2=0x7f78e80fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:19.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.557+0000 7f78eeef1700 1 -- 172.21.15.142:0/1724603094 shutdown_connections 2024-03-29T05:10:19.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.557+0000 7f78eeef1700 1 -- 172.21.15.142:0/1724603094 wait complete. 2024-03-29T05:10:19.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.558+0000 7f78eeef1700 1 Processor -- start 2024-03-29T05:10:19.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.558+0000 7f78eeef1700 1 -- start start 2024-03-29T05:10:19.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.558+0000 7f78eeef1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e80ffa70 0x7f78e8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:19.566 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.558+0000 7f78eeef1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e8105a70 0x7f78e8198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:19.567 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.558+0000 7f78eeef1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e81994b0 con 0x7f78e80ffa70 2024-03-29T05:10:19.567 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.558+0000 7f78eeef1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8199620 con 0x7f78e8105a70 2024-03-29T05:10:19.567 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.558+0000 7f78ecc8d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e80ffa70 0x7f78e8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:19.567 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.558+0000 7f78ecc8d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e80ffa70 0x7f78e8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45368/0 (socket says 172.21.15.142:45368) 2024-03-29T05:10:19.567 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.558+0000 7f78ecc8d700 1 -- 172.21.15.142:0/2552848312 learned_addr learned my addr 172.21.15.142:0/2552848312 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:19.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.559+0000 7f78e7fff700 1 --2- 172.21.15.142:0/2552848312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e8105a70 0x7f78e8198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:19.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.559+0000 7f78ecc8d700 1 -- 172.21.15.142:0/2552848312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e8105a70 msgr2=0x7f78e8198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.559+0000 7f78ecc8d700 1 --2- 172.21.15.142:0/2552848312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e8105a70 0x7f78e8198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.559+0000 7f78ecc8d700 1 -- 172.21.15.142:0/2552848312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d80095f0 con 0x7f78e80ffa70 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.559+0000 7f78ecc8d700 1 --2- 172.21.15.142:0/2552848312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e80ffa70 0x7f78e8198970 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f78e000cfd0 tx=0x7f78e000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.559+0000 7f78e5ffb700 1 -- 172.21.15.142:0/2552848312 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78e000ad70 con 0x7f78e80ffa70 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.559+0000 7f78e5ffb700 1 -- 172.21.15.142:0/2552848312 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78e00128e0 con 0x7f78e80ffa70 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.559+0000 7f78e5ffb700 1 -- 172.21.15.142:0/2552848312 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78e0011880 con 0x7f78e80ffa70 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.559+0000 7f78eeef1700 1 -- 172.21.15.142:0/2552848312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e819d4a0 con 0x7f78e80ffa70 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.559+0000 7f78eeef1700 1 -- 172.21.15.142:0/2552848312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e819d9f0 con 0x7f78e80ffa70 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.560+0000 7f78eeef1700 1 -- 172.21.15.142:0/2552848312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e804ea90 con 0x7f78e80ffa70 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.561+0000 7f78e5ffb700 1 -- 172.21.15.142:0/2552848312 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f78e0012420 con 0x7f78e80ffa70 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.561+0000 7f78e5ffb700 1 --2- 172.21.15.142:0/2552848312 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f78d406dd60 0x7f78d4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.561+0000 7f78e5ffb700 1 -- 172.21.15.142:0/2552848312 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f78e008dd10 con 0x7f78e80ffa70 2024-03-29T05:10:19.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.561+0000 7f78e7fff700 1 --2- 172.21.15.142:0/2552848312 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f78d406dd60 0x7f78d4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:19.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.562+0000 7f78e7fff700 1 --2- 172.21.15.142:0/2552848312 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f78d406dd60 0x7f78d4070220 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f78e8199f90 tx=0x7f78d801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:19.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.566+0000 7f78e5ffb700 1 -- 172.21.15.142:0/2552848312 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f78e005ace0 con 0x7f78e80ffa70 2024-03-29T05:10:19.784 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.777+0000 7f78eeef1700 1 -- 172.21.15.142:0/2552848312 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f78e819df70 con 0x7f78d406dd60 2024-03-29T05:10:19.785 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.778+0000 7f78e5ffb700 1 -- 172.21.15.142:0/2552848312 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f78e819df70 con 0x7f78d406dd60 2024-03-29T05:10:19.787 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.780+0000 7f78eeef1700 1 -- 172.21.15.142:0/2552848312 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f78d406dd60 msgr2=0x7f78d4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:19.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.780+0000 7f78eeef1700 1 --2- 172.21.15.142:0/2552848312 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f78d406dd60 0x7f78d4070220 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f78e8199f90 tx=0x7f78d801a040 comp rx=0 tx=0).stop 2024-03-29T05:10:19.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.780+0000 7f78eeef1700 1 -- 172.21.15.142:0/2552848312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e80ffa70 msgr2=0x7f78e8198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:19.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.780+0000 7f78eeef1700 1 --2- 172.21.15.142:0/2552848312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e80ffa70 0x7f78e8198970 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f78e000cfd0 tx=0x7f78e000a340 comp rx=0 tx=0).stop 2024-03-29T05:10:19.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.780+0000 7f78eeef1700 1 -- 172.21.15.142:0/2552848312 shutdown_connections 2024-03-29T05:10:19.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.780+0000 7f78eeef1700 1 --2- 172.21.15.142:0/2552848312 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f78d406dd60 0x7f78d4070220 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.780+0000 7f78eeef1700 1 --2- 172.21.15.142:0/2552848312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e80ffa70 0x7f78e8198970 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.780+0000 7f78eeef1700 1 --2- 172.21.15.142:0/2552848312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e8105a70 0x7f78e8198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.780+0000 7f78eeef1700 1 -- 172.21.15.142:0/2552848312 >> 172.21.15.142:0/2552848312 conn(0x7f78e80fb560 msgr2=0x7f78e80fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:19.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.780+0000 7f78eeef1700 1 -- 172.21.15.142:0/2552848312 shutdown_connections 2024-03-29T05:10:19.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.780+0000 7f78eeef1700 1 -- 172.21.15.142:0/2552848312 wait complete. 2024-03-29T05:10:19.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:19 smithi142 ceph-mon[112475]: pgmap v660: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:19.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.890+0000 7f43094e0700 1 -- 172.21.15.142:0/1780010346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4304069020 msgr2=0x7f4304105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:19.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.890+0000 7f43094e0700 1 --2- 172.21.15.142:0/1780010346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4304069020 0x7f4304105790 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009910 tx=0x7f42f8009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:19.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.890+0000 7f43094e0700 1 -- 172.21.15.142:0/1780010346 shutdown_connections 2024-03-29T05:10:19.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.890+0000 7f43094e0700 1 --2- 172.21.15.142:0/1780010346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4304069020 0x7f4304105790 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.890+0000 7f43094e0700 1 --2- 172.21.15.142:0/1780010346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4304068670 0x7f4304068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.890+0000 7f43094e0700 1 -- 172.21.15.142:0/1780010346 >> 172.21.15.142:0/1780010346 conn(0x7f43040755a0 msgr2=0x7f43040759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:19.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.890+0000 7f43094e0700 1 -- 172.21.15.142:0/1780010346 shutdown_connections 2024-03-29T05:10:19.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.890+0000 7f43094e0700 1 -- 172.21.15.142:0/1780010346 wait complete. 2024-03-29T05:10:19.900 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.891+0000 7f43094e0700 1 Processor -- start 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.891+0000 7f43094e0700 1 -- start start 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.891+0000 7f43094e0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4304068670 0x7f43040ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.891+0000 7f43094e0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4304069020 0x7f43041001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.891+0000 7f43094e0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4304100720 con 0x7f4304069020 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.891+0000 7f43094e0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4304100860 con 0x7f4304068670 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f43027fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4304069020 0x7f43041001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f43027fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4304069020 0x7f43041001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45376/0 (socket says 172.21.15.142:45376) 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f43027fc700 1 -- 172.21.15.142:0/3721603361 learned_addr learned my addr 172.21.15.142:0/3721603361 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f4302ffd700 1 --2- 172.21.15.142:0/3721603361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4304068670 0x7f43040ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f43027fc700 1 -- 172.21.15.142:0/3721603361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4304068670 msgr2=0x7f43040ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f43027fc700 1 --2- 172.21.15.142:0/3721603361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4304068670 0x7f43040ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f43027fc700 1 -- 172.21.15.142:0/3721603361 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80095f0 con 0x7f4304069020 2024-03-29T05:10:19.901 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f43027fc700 1 --2- 172.21.15.142:0/3721603361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4304069020 0x7f43041001e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f42f80092e0 tx=0x7f42f800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:19.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f42ebfff700 1 -- 172.21.15.142:0/3721603361 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f801d070 con 0x7f4304069020 2024-03-29T05:10:19.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f43094e0700 1 -- 172.21.15.142:0/3721603361 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4304100ae0 con 0x7f4304069020 2024-03-29T05:10:19.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f42ebfff700 1 -- 172.21.15.142:0/3721603361 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42f800f460 con 0x7f4304069020 2024-03-29T05:10:19.902 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f42ebfff700 1 -- 172.21.15.142:0/3721603361 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f80175d0 con 0x7f4304069020 2024-03-29T05:10:19.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.892+0000 7f43094e0700 1 -- 172.21.15.142:0/3721603361 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43041a2c00 con 0x7f4304069020 2024-03-29T05:10:19.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.893+0000 7f43094e0700 1 -- 172.21.15.142:0/3721603361 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43041090a0 con 0x7f4304069020 2024-03-29T05:10:19.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.894+0000 7f42ebfff700 1 -- 172.21.15.142:0/3721603361 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f42f800f5d0 con 0x7f4304069020 2024-03-29T05:10:19.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.894+0000 7f42ebfff700 1 --2- 172.21.15.142:0/3721603361 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f42ec06dd60 0x7f42ec070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:19.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.894+0000 7f42ebfff700 1 -- 172.21.15.142:0/3721603361 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f42f808f150 con 0x7f4304069020 2024-03-29T05:10:19.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.894+0000 7f4302ffd700 1 --2- 172.21.15.142:0/3721603361 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f42ec06dd60 0x7f42ec070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:19.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.895+0000 7f4302ffd700 1 --2- 172.21.15.142:0/3721603361 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f42ec06dd60 0x7f42ec070220 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f42f4005950 tx=0x7f42f400a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:19.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:19.899+0000 7f42ebfff700 1 -- 172.21.15.142:0/3721603361 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f42f8058570 con 0x7f4304069020 2024-03-29T05:10:20.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:19 smithi152 ceph-mon[88186]: pgmap v660: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:20.108 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.101+0000 7f43094e0700 1 -- 172.21.15.142:0/3721603361 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f430402cce0 con 0x7f42ec06dd60 2024-03-29T05:10:20.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.111+0000 7f42ebfff700 1 -- 172.21.15.142:0/3721603361 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f430402cce0 con 0x7f42ec06dd60 2024-03-29T05:10:20.119 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:10:20.119 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (22m) 15m ago 33m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:10:20.119 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (33m) 15m ago 33m 9143k - 18.2.1 5be31c24972a 3d33465f5167 2024-03-29T05:10:20.119 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (32m) 14m ago 32m 8988k - 18.2.1 5be31c24972a a3de1acd0721 2024-03-29T05:10:20.119 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (19m) 15m ago 33m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:10:20.119 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (19m) 14m ago 32m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:10:20.119 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (21m) 15m ago 33m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:10:20.119 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (15m) 15m ago 27m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:10:20.119 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (15m) 15m ago 28m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:10:20.119 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (14m) 14m ago 27m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:10:20.119 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (15m) 14m ago 28m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (23m) 15m ago 35m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (23m) 14m ago 32m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (20m) 15m ago 35m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (20m) 14m ago 32m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (22m) 15m ago 33m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (22m) 14m ago 32m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (18m) 15m ago 31m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (18m) 15m ago 31m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (17m) 15m ago 30m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (17m) 14m ago 30m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (16m) 14m ago 30m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (15m) 14m ago 29m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:10:20.120 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (22m) 15m ago 33m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:10:20.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.114+0000 7f43094e0700 1 -- 172.21.15.142:0/3721603361 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f42ec06dd60 msgr2=0x7f42ec070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:20.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.114+0000 7f43094e0700 1 --2- 172.21.15.142:0/3721603361 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f42ec06dd60 0x7f42ec070220 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f42f4005950 tx=0x7f42f400a300 comp rx=0 tx=0).stop 2024-03-29T05:10:20.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.114+0000 7f43094e0700 1 -- 172.21.15.142:0/3721603361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4304069020 msgr2=0x7f43041001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:20.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.114+0000 7f43094e0700 1 --2- 172.21.15.142:0/3721603361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4304069020 0x7f43041001e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f42f80092e0 tx=0x7f42f800bd70 comp rx=0 tx=0).stop 2024-03-29T05:10:20.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.114+0000 7f43094e0700 1 -- 172.21.15.142:0/3721603361 shutdown_connections 2024-03-29T05:10:20.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.114+0000 7f43094e0700 1 --2- 172.21.15.142:0/3721603361 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f42ec06dd60 0x7f42ec070220 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.114+0000 7f43094e0700 1 --2- 172.21.15.142:0/3721603361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4304068670 0x7f43040ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.114+0000 7f43094e0700 1 --2- 172.21.15.142:0/3721603361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4304069020 0x7f43041001e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.114+0000 7f43094e0700 1 -- 172.21.15.142:0/3721603361 >> 172.21.15.142:0/3721603361 conn(0x7f43040755a0 msgr2=0x7f4304105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:20.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.114+0000 7f43094e0700 1 -- 172.21.15.142:0/3721603361 shutdown_connections 2024-03-29T05:10:20.122 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.114+0000 7f43094e0700 1 -- 172.21.15.142:0/3721603361 wait complete. 2024-03-29T05:10:20.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.216+0000 7f8a584ac700 1 -- 172.21.15.142:0/1179779378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a50105a40 msgr2=0x7f8a50105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:20.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.216+0000 7f8a584ac700 1 --2- 172.21.15.142:0/1179779378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a50105a40 0x7f8a50105e20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009910 tx=0x7f8a4c009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:20.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.216+0000 7f8a584ac700 1 -- 172.21.15.142:0/1179779378 shutdown_connections 2024-03-29T05:10:20.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.216+0000 7f8a584ac700 1 --2- 172.21.15.142:0/1179779378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a500ffa40 0x7f8a500ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.216+0000 7f8a584ac700 1 --2- 172.21.15.142:0/1179779378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a50105a40 0x7f8a50105e20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.216+0000 7f8a584ac700 1 -- 172.21.15.142:0/1179779378 >> 172.21.15.142:0/1179779378 conn(0x7f8a500fb570 msgr2=0x7f8a500fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:20.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.216+0000 7f8a584ac700 1 -- 172.21.15.142:0/1179779378 shutdown_connections 2024-03-29T05:10:20.225 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.216+0000 7f8a584ac700 1 -- 172.21.15.142:0/1179779378 wait complete. 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a584ac700 1 Processor -- start 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a584ac700 1 -- start start 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a584ac700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a500ffa40 0x7f8a5019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a584ac700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a50105a40 0x7f8a5019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a584ac700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5019d130 con 0x7f8a500ffa40 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a584ac700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a501965e0 con 0x7f8a50105a40 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a56248700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a500ffa40 0x7f8a5019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a56248700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a500ffa40 0x7f8a5019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45388/0 (socket says 172.21.15.142:45388) 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a56248700 1 -- 172.21.15.142:0/3792305648 learned_addr learned my addr 172.21.15.142:0/3792305648 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a56248700 1 -- 172.21.15.142:0/3792305648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a50105a40 msgr2=0x7f8a5019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a56248700 1 --2- 172.21.15.142:0/3792305648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a50105a40 0x7f8a5019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a56248700 1 -- 172.21.15.142:0/3792305648 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c0095f0 con 0x7f8a500ffa40 2024-03-29T05:10:20.226 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.217+0000 7f8a56248700 1 --2- 172.21.15.142:0/3792305648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a500ffa40 0x7f8a5019c560 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c006010 tx=0x7f8a4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:20.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.218+0000 7f8a477fe700 1 -- 172.21.15.142:0/3792305648 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a4c01d070 con 0x7f8a500ffa40 2024-03-29T05:10:20.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.218+0000 7f8a584ac700 1 -- 172.21.15.142:0/3792305648 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a50196860 con 0x7f8a500ffa40 2024-03-29T05:10:20.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.218+0000 7f8a477fe700 1 -- 172.21.15.142:0/3792305648 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a4c004fb0 con 0x7f8a500ffa40 2024-03-29T05:10:20.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.218+0000 7f8a477fe700 1 -- 172.21.15.142:0/3792305648 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a4c022ea0 con 0x7f8a500ffa40 2024-03-29T05:10:20.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.218+0000 7f8a584ac700 1 -- 172.21.15.142:0/3792305648 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a50196d50 con 0x7f8a500ffa40 2024-03-29T05:10:20.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.219+0000 7f8a584ac700 1 -- 172.21.15.142:0/3792305648 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a50109050 con 0x7f8a500ffa40 2024-03-29T05:10:20.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.219+0000 7f8a477fe700 1 -- 172.21.15.142:0/3792305648 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f8a4c022470 con 0x7f8a500ffa40 2024-03-29T05:10:20.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.220+0000 7f8a477fe700 1 --2- 172.21.15.142:0/3792305648 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8a3c06dd60 0x7f8a3c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:20.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.220+0000 7f8a477fe700 1 -- 172.21.15.142:0/3792305648 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f8a4c08f230 con 0x7f8a500ffa40 2024-03-29T05:10:20.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.220+0000 7f8a55a47700 1 --2- 172.21.15.142:0/3792305648 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8a3c06dd60 0x7f8a3c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:20.229 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.220+0000 7f8a55a47700 1 --2- 172.21.15.142:0/3792305648 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8a3c06dd60 0x7f8a3c070220 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8a50105250 tx=0x7f8a40005c80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:20.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.224+0000 7f8a477fe700 1 -- 172.21.15.142:0/3792305648 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8a4c05c230 con 0x7f8a500ffa40 2024-03-29T05:10:20.506 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.499+0000 7f8a584ac700 1 -- 172.21.15.142:0/3792305648 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8a50197030 con 0x7f8a500ffa40 2024-03-29T05:10:20.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.499+0000 7f8a477fe700 1 -- 172.21.15.142:0/3792305648 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8a4c022720 con 0x7f8a500ffa40 2024-03-29T05:10:20.507 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:10:20.507 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:10:20.507 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:10:20.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:10:20.507 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:10:20.507 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:10:20.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:10:20.507 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:10:20.507 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:10:20.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:10:20.508 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:10:20.508 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:10:20.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:10:20.508 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:10:20.508 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:10:20.508 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:10:20.508 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:10:20.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.501+0000 7f8a584ac700 1 -- 172.21.15.142:0/3792305648 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8a3c06dd60 msgr2=0x7f8a3c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:20.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.501+0000 7f8a584ac700 1 --2- 172.21.15.142:0/3792305648 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8a3c06dd60 0x7f8a3c070220 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8a50105250 tx=0x7f8a40005c80 comp rx=0 tx=0).stop 2024-03-29T05:10:20.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.501+0000 7f8a584ac700 1 -- 172.21.15.142:0/3792305648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a500ffa40 msgr2=0x7f8a5019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:20.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.501+0000 7f8a584ac700 1 --2- 172.21.15.142:0/3792305648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a500ffa40 0x7f8a5019c560 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c006010 tx=0x7f8a4c00be20 comp rx=0 tx=0).stop 2024-03-29T05:10:20.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.501+0000 7f8a584ac700 1 -- 172.21.15.142:0/3792305648 shutdown_connections 2024-03-29T05:10:20.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.501+0000 7f8a584ac700 1 --2- 172.21.15.142:0/3792305648 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8a3c06dd60 0x7f8a3c070220 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.501+0000 7f8a584ac700 1 --2- 172.21.15.142:0/3792305648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a500ffa40 0x7f8a5019c560 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.501+0000 7f8a584ac700 1 --2- 172.21.15.142:0/3792305648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a50105a40 0x7f8a5019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.502+0000 7f8a584ac700 1 -- 172.21.15.142:0/3792305648 >> 172.21.15.142:0/3792305648 conn(0x7f8a500fb570 msgr2=0x7f8a50068f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:20.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.502+0000 7f8a584ac700 1 -- 172.21.15.142:0/3792305648 shutdown_connections 2024-03-29T05:10:20.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.502+0000 7f8a584ac700 1 -- 172.21.15.142:0/3792305648 wait complete. 2024-03-29T05:10:20.613 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.604+0000 7f2372da9700 1 -- 172.21.15.142:0/389143006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c100400 msgr2=0x7f236c1048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:20.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.604+0000 7f2372da9700 1 --2- 172.21.15.142:0/389143006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c100400 0x7f236c1048d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2358009910 tx=0x7f2358009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:20.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.605+0000 7f2372da9700 1 -- 172.21.15.142:0/389143006 shutdown_connections 2024-03-29T05:10:20.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.605+0000 7f2372da9700 1 --2- 172.21.15.142:0/389143006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c100400 0x7f236c1048d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.605+0000 7f2372da9700 1 --2- 172.21.15.142:0/389143006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f236c0ffa50 0x7f236c0ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.605+0000 7f2372da9700 1 -- 172.21.15.142:0/389143006 >> 172.21.15.142:0/389143006 conn(0x7f236c0fb540 msgr2=0x7f236c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:20.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.605+0000 7f2372da9700 1 -- 172.21.15.142:0/389143006 shutdown_connections 2024-03-29T05:10:20.614 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.605+0000 7f2372da9700 1 -- 172.21.15.142:0/389143006 wait complete. 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.605+0000 7f2372da9700 1 Processor -- start 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.605+0000 7f2372da9700 1 -- start start 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.605+0000 7f2372da9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f236c0ffa50 0x7f236c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.605+0000 7f2372da9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c100400 0x7f236c19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.605+0000 7f2372da9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c19d8b0 con 0x7f236c100400 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.605+0000 7f2372da9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c19d9f0 con 0x7f236c0ffa50 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f236bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c100400 0x7f236c19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f236bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c100400 0x7f236c19d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45406/0 (socket says 172.21.15.142:45406) 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f236bfff700 1 -- 172.21.15.142:0/3973057869 learned_addr learned my addr 172.21.15.142:0/3973057869 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f2370b45700 1 --2- 172.21.15.142:0/3973057869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f236c0ffa50 0x7f236c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f236bfff700 1 -- 172.21.15.142:0/3973057869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f236c0ffa50 msgr2=0x7f236c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f236bfff700 1 --2- 172.21.15.142:0/3973057869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f236c0ffa50 0x7f236c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.615 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f236bfff700 1 -- 172.21.15.142:0/3973057869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23580095f0 con 0x7f236c100400 2024-03-29T05:10:20.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f2370b45700 1 --2- 172.21.15.142:0/3973057869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f236c0ffa50 0x7f236c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:10:20.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f236bfff700 1 --2- 172.21.15.142:0/3973057869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c100400 0x7f236c19d260 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f235800bda0 tx=0x7f235800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:20.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f2369ffb700 1 -- 172.21.15.142:0/3973057869 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f235801d070 con 0x7f236c100400 2024-03-29T05:10:20.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f2369ffb700 1 -- 172.21.15.142:0/3973057869 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2358004fb0 con 0x7f236c100400 2024-03-29T05:10:20.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f2372da9700 1 -- 172.21.15.142:0/3973057869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c1a17e0 con 0x7f236c100400 2024-03-29T05:10:20.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f2369ffb700 1 -- 172.21.15.142:0/3973057869 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f235800f480 con 0x7f236c100400 2024-03-29T05:10:20.616 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.606+0000 7f2372da9700 1 -- 172.21.15.142:0/3973057869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c1a1cd0 con 0x7f236c100400 2024-03-29T05:10:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.607+0000 7f2372da9700 1 -- 172.21.15.142:0/3973057869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c102030 con 0x7f236c100400 2024-03-29T05:10:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.608+0000 7f2369ffb700 1 -- 172.21.15.142:0/3973057869 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f2358022870 con 0x7f236c100400 2024-03-29T05:10:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.608+0000 7f2369ffb700 1 --2- 172.21.15.142:0/3973057869 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f235c06dd60 0x7f235c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.608+0000 7f2369ffb700 1 -- 172.21.15.142:0/3973057869 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f235808f130 con 0x7f236c100400 2024-03-29T05:10:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.608+0000 7f2370b45700 1 --2- 172.21.15.142:0/3973057869 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f235c06dd60 0x7f235c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.609+0000 7f2370b45700 1 --2- 172.21.15.142:0/3973057869 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f235c06dd60 0x7f235c070220 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2360005fd0 tx=0x7f2360005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:20.620 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.613+0000 7f2369ffb700 1 -- 172.21.15.142:0/3973057869 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2358058680 con 0x7f236c100400 2024-03-29T05:10:20.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.855+0000 7f2372da9700 1 -- 172.21.15.142:0/3973057869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f236c04ea90 con 0x7f236c100400 2024-03-29T05:10:20.863 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:20 smithi142 ceph-mon[112475]: from='client.35214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:20.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:20 smithi142 ceph-mon[112475]: from='client.35218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:20.864 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:20 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3792305648' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:20.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.856+0000 7f2369ffb700 1 -- 172.21.15.142:0/3973057869 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f235805bd40 con 0x7f236c100400 2024-03-29T05:10:20.864 INFO:teuthology.orchestra.run.smithi142.stdout:e32 2024-03-29T05:10:20.865 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:10:20.865 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:10:20.865 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:10:20.865 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:10:20.865 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:10:20.865 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:10:20.865 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 23 2024-03-29T05:10:20.865 INFO:teuthology.orchestra.run.smithi142.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T05:10:20.865 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:10:20.866 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T04:54:33.202220+0000 2024-03-29T05:10:20.866 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:10:20.866 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:10:20.866 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:10:20.866 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:10:20.866 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:10:20.866 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:10:20.866 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:10:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:10:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:10:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:10:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:10:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:10:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:up {} 2024-03-29T05:10:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:failed 0,1 2024-03-29T05:10:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:10:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:10:20.868 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:10:20.868 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:10:20.868 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:10:20.868 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:10:20.868 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:10:20.868 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:10:20.868 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 0 members: 2024-03-29T05:10:20.868 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:10:20.869 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:10:20.869 INFO:teuthology.orchestra.run.smithi142.stdout:Standby daemons: 2024-03-29T05:10:20.869 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:10:20.869 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:10:20.869 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:10:20.869 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:10:20.869 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:10:20.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.858+0000 7f2372da9700 1 -- 172.21.15.142:0/3973057869 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f235c06dd60 msgr2=0x7f235c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:20.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.858+0000 7f2372da9700 1 --2- 172.21.15.142:0/3973057869 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f235c06dd60 0x7f235c070220 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2360005fd0 tx=0x7f2360005e30 comp rx=0 tx=0).stop 2024-03-29T05:10:20.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.858+0000 7f2372da9700 1 -- 172.21.15.142:0/3973057869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c100400 msgr2=0x7f236c19d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:20.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.858+0000 7f2372da9700 1 --2- 172.21.15.142:0/3973057869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c100400 0x7f236c19d260 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f235800bda0 tx=0x7f235800be80 comp rx=0 tx=0).stop 2024-03-29T05:10:20.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.858+0000 7f2372da9700 1 -- 172.21.15.142:0/3973057869 shutdown_connections 2024-03-29T05:10:20.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.858+0000 7f2372da9700 1 --2- 172.21.15.142:0/3973057869 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f235c06dd60 0x7f235c070220 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.858+0000 7f2372da9700 1 --2- 172.21.15.142:0/3973057869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f236c0ffa50 0x7f236c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.859+0000 7f2372da9700 1 --2- 172.21.15.142:0/3973057869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c100400 0x7f236c19d260 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.870 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.859+0000 7f2372da9700 1 -- 172.21.15.142:0/3973057869 >> 172.21.15.142:0/3973057869 conn(0x7f236c0fb540 msgr2=0x7f236c110f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:20.871 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.859+0000 7f2372da9700 1 -- 172.21.15.142:0/3973057869 shutdown_connections 2024-03-29T05:10:20.871 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.859+0000 7f2372da9700 1 -- 172.21.15.142:0/3973057869 wait complete. 2024-03-29T05:10:20.871 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:10:20.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.962+0000 7fbc6d601700 1 -- 172.21.15.142:0/272453831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc680ffa40 msgr2=0x7fbc680ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:20.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.962+0000 7fbc6d601700 1 --2- 172.21.15.142:0/272453831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc680ffa40 0x7fbc680ffec0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009910 tx=0x7fbc58009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:20.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.962+0000 7fbc6d601700 1 -- 172.21.15.142:0/272453831 shutdown_connections 2024-03-29T05:10:20.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.962+0000 7fbc6d601700 1 --2- 172.21.15.142:0/272453831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc680ffa40 0x7fbc680ffec0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.962+0000 7fbc6d601700 1 --2- 172.21.15.142:0/272453831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc68105a40 0x7fbc68105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.962+0000 7fbc6d601700 1 -- 172.21.15.142:0/272453831 >> 172.21.15.142:0/272453831 conn(0x7fbc680fb570 msgr2=0x7fbc680fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:20.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.962+0000 7fbc6d601700 1 -- 172.21.15.142:0/272453831 shutdown_connections 2024-03-29T05:10:20.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.962+0000 7fbc6d601700 1 -- 172.21.15.142:0/272453831 wait complete. 2024-03-29T05:10:20.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.963+0000 7fbc6d601700 1 Processor -- start 2024-03-29T05:10:20.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.963+0000 7fbc6d601700 1 -- start start 2024-03-29T05:10:20.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.963+0000 7fbc6d601700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc680ffa40 0x7fbc68197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:20.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.963+0000 7fbc6d601700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc68105a40 0x7fbc68197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:20.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.963+0000 7fbc6d601700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc681980d0 con 0x7fbc680ffa40 2024-03-29T05:10:20.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.963+0000 7fbc6d601700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc68198240 con 0x7fbc68105a40 2024-03-29T05:10:20.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc66ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc680ffa40 0x7fbc68197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:20.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc66ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc680ffa40 0x7fbc68197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45424/0 (socket says 172.21.15.142:45424) 2024-03-29T05:10:20.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc66ffd700 1 -- 172.21.15.142:0/429438560 learned_addr learned my addr 172.21.15.142:0/429438560 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:20.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc667fc700 1 --2- 172.21.15.142:0/429438560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc68105a40 0x7fbc68197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:20.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc66ffd700 1 -- 172.21.15.142:0/429438560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc68105a40 msgr2=0x7fbc68197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:20.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc66ffd700 1 --2- 172.21.15.142:0/429438560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc68105a40 0x7fbc68197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:20.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc66ffd700 1 -- 172.21.15.142:0/429438560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc580095f0 con 0x7fbc680ffa40 2024-03-29T05:10:20.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc66ffd700 1 --2- 172.21.15.142:0/429438560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc680ffa40 0x7fbc68197570 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000d8d0 tx=0x7fbc5000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:20.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc5ffff700 1 -- 172.21.15.142:0/429438560 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc500098a0 con 0x7fbc680ffa40 2024-03-29T05:10:20.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc5ffff700 1 -- 172.21.15.142:0/429438560 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc50010460 con 0x7fbc680ffa40 2024-03-29T05:10:20.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc6d601700 1 -- 172.21.15.142:0/429438560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc681a05e0 con 0x7fbc680ffa40 2024-03-29T05:10:20.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc5ffff700 1 -- 172.21.15.142:0/429438560 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5000f5f0 con 0x7fbc680ffa40 2024-03-29T05:10:20.973 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.964+0000 7fbc6d601700 1 -- 172.21.15.142:0/429438560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc681a0ad0 con 0x7fbc680ffa40 2024-03-29T05:10:20.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.965+0000 7fbc6d601700 1 -- 172.21.15.142:0/429438560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc68107fd0 con 0x7fbc680ffa40 2024-03-29T05:10:20.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.966+0000 7fbc5ffff700 1 -- 172.21.15.142:0/429438560 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fbc5000f820 con 0x7fbc680ffa40 2024-03-29T05:10:20.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.966+0000 7fbc5ffff700 1 --2- 172.21.15.142:0/429438560 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbc5406dd60 0x7fbc54070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:20.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.966+0000 7fbc5ffff700 1 -- 172.21.15.142:0/429438560 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fbc5008dbf0 con 0x7fbc680ffa40 2024-03-29T05:10:20.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.966+0000 7fbc667fc700 1 --2- 172.21.15.142:0/429438560 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbc5406dd60 0x7fbc54070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:20.974 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.967+0000 7fbc667fc700 1 --2- 172.21.15.142:0/429438560 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbc5406dd60 0x7fbc54070220 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbc68069570 tx=0x7fbc5801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:20.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:20.971+0000 7fbc5ffff700 1 -- 172.21.15.142:0/429438560 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbc50057080 con 0x7fbc680ffa40 2024-03-29T05:10:21.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:20 smithi152 ceph-mon[88186]: from='client.35214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:21.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:20 smithi152 ceph-mon[88186]: from='client.35218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:21.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:20 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3792305648' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:21.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.183+0000 7fbc6d601700 1 -- 172.21.15.142:0/429438560 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc6802cd80 con 0x7fbc5406dd60 2024-03-29T05:10:21.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.184+0000 7fbc5ffff700 1 -- 172.21.15.142:0/429438560 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbc6802cd80 con 0x7fbc5406dd60 2024-03-29T05:10:21.192 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:10:21.192 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:10:21.192 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:10:21.192 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:10:21.192 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:10:21.192 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:10:21.192 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:10:21.193 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:10:21.193 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:10:21.193 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:10:21.193 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:10:21.193 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "16/23 daemons upgraded", 2024-03-29T05:10:21.193 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T05:10:21.193 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:10:21.194 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:10:21.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.186+0000 7fbc6d601700 1 -- 172.21.15.142:0/429438560 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbc5406dd60 msgr2=0x7fbc54070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:21.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.186+0000 7fbc6d601700 1 --2- 172.21.15.142:0/429438560 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbc5406dd60 0x7fbc54070220 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbc68069570 tx=0x7fbc5801a040 comp rx=0 tx=0).stop 2024-03-29T05:10:21.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.186+0000 7fbc6d601700 1 -- 172.21.15.142:0/429438560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc680ffa40 msgr2=0x7fbc68197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:21.195 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.186+0000 7fbc6d601700 1 --2- 172.21.15.142:0/429438560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc680ffa40 0x7fbc68197570 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000d8d0 tx=0x7fbc5000dbe0 comp rx=0 tx=0).stop 2024-03-29T05:10:21.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.186+0000 7fbc6d601700 1 -- 172.21.15.142:0/429438560 shutdown_connections 2024-03-29T05:10:21.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.186+0000 7fbc6d601700 1 --2- 172.21.15.142:0/429438560 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbc5406dd60 0x7fbc54070220 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:21.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.186+0000 7fbc6d601700 1 --2- 172.21.15.142:0/429438560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc680ffa40 0x7fbc68197570 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:21.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.186+0000 7fbc6d601700 1 --2- 172.21.15.142:0/429438560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc68105a40 0x7fbc68197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:21.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.186+0000 7fbc6d601700 1 -- 172.21.15.142:0/429438560 >> 172.21.15.142:0/429438560 conn(0x7fbc680fb570 msgr2=0x7fbc680fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:21.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.187+0000 7fbc6d601700 1 -- 172.21.15.142:0/429438560 shutdown_connections 2024-03-29T05:10:21.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.187+0000 7fbc6d601700 1 -- 172.21.15.142:0/429438560 wait complete. 2024-03-29T05:10:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.290+0000 7f5cb44ed700 1 -- 172.21.15.142:0/1619613183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cac0ffa40 msgr2=0x7f5cac0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.290+0000 7f5cb44ed700 1 --2- 172.21.15.142:0/1619613183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cac0ffa40 0x7f5cac0ffec0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009910 tx=0x7f5ca8009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.290+0000 7f5cb44ed700 1 -- 172.21.15.142:0/1619613183 shutdown_connections 2024-03-29T05:10:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.290+0000 7f5cb44ed700 1 --2- 172.21.15.142:0/1619613183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cac0ffa40 0x7f5cac0ffec0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.290+0000 7f5cb44ed700 1 --2- 172.21.15.142:0/1619613183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cac105a40 0x7f5cac105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.290+0000 7f5cb44ed700 1 -- 172.21.15.142:0/1619613183 >> 172.21.15.142:0/1619613183 conn(0x7f5cac0fb550 msgr2=0x7f5cac0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.290+0000 7f5cb44ed700 1 -- 172.21.15.142:0/1619613183 shutdown_connections 2024-03-29T05:10:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.290+0000 7f5cb44ed700 1 -- 172.21.15.142:0/1619613183 wait complete. 2024-03-29T05:10:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.291+0000 7f5cb44ed700 1 Processor -- start 2024-03-29T05:10:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.291+0000 7f5cb44ed700 1 -- start start 2024-03-29T05:10:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.291+0000 7f5cb44ed700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cac0ffa40 0x7f5cac19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:21.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.291+0000 7f5cb44ed700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cac105a40 0x7f5cac19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.291+0000 7f5cb44ed700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac19d110 con 0x7f5cac105a40 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.291+0000 7f5cb44ed700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac1965c0 con 0x7f5cac0ffa40 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.291+0000 7f5cb1a88700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cac105a40 0x7f5cac19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.291+0000 7f5cb1a88700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cac105a40 0x7f5cac19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45456/0 (socket says 172.21.15.142:45456) 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.291+0000 7f5cb1a88700 1 -- 172.21.15.142:0/464786316 learned_addr learned my addr 172.21.15.142:0/464786316 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.292+0000 7f5cb1a88700 1 -- 172.21.15.142:0/464786316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cac0ffa40 msgr2=0x7f5cac19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.292+0000 7f5cb2289700 1 --2- 172.21.15.142:0/464786316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cac0ffa40 0x7f5cac19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.292+0000 7f5cb1a88700 1 --2- 172.21.15.142:0/464786316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cac0ffa40 0x7f5cac19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.292+0000 7f5cb1a88700 1 -- 172.21.15.142:0/464786316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca80095f0 con 0x7f5cac105a40 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.292+0000 7f5cb2289700 1 --2- 172.21.15.142:0/464786316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cac0ffa40 0x7f5cac19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.292+0000 7f5cb1a88700 1 --2- 172.21.15.142:0/464786316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cac105a40 0x7f5cac19ca80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80092e0 tx=0x7f5ca800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.292+0000 7f5ca37fe700 1 -- 172.21.15.142:0/464786316 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ca801d070 con 0x7f5cac105a40 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.292+0000 7f5ca37fe700 1 -- 172.21.15.142:0/464786316 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ca8005e00 con 0x7f5cac105a40 2024-03-29T05:10:21.301 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.292+0000 7f5cb44ed700 1 -- 172.21.15.142:0/464786316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac196840 con 0x7f5cac105a40 2024-03-29T05:10:21.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.292+0000 7f5ca37fe700 1 -- 172.21.15.142:0/464786316 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ca8017400 con 0x7f5cac105a40 2024-03-29T05:10:21.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.292+0000 7f5cb44ed700 1 -- 172.21.15.142:0/464786316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac196d30 con 0x7f5cac105a40 2024-03-29T05:10:21.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.293+0000 7f5cb44ed700 1 -- 172.21.15.142:0/464786316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac04ea90 con 0x7f5cac105a40 2024-03-29T05:10:21.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.293+0000 7f5ca37fe700 1 -- 172.21.15.142:0/464786316 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f5ca800f460 con 0x7f5cac105a40 2024-03-29T05:10:21.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.294+0000 7f5ca37fe700 1 --2- 172.21.15.142:0/464786316 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5c9806dd60 0x7f5c98070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:21.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.294+0000 7f5ca37fe700 1 -- 172.21.15.142:0/464786316 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f5ca808f1f0 con 0x7f5cac105a40 2024-03-29T05:10:21.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.294+0000 7f5cb2289700 1 --2- 172.21.15.142:0/464786316 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5c9806dd60 0x7f5c98070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:21.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.294+0000 7f5cb2289700 1 --2- 172.21.15.142:0/464786316 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5c9806dd60 0x7f5c98070220 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c005fd0 tx=0x7f5c9c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:21.305 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.298+0000 7f5ca37fe700 1 -- 172.21.15.142:0/464786316 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5ca805c140 con 0x7f5cac105a40 2024-03-29T05:10:21.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.574+0000 7f5cb44ed700 1 -- 172.21.15.142:0/464786316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5cac197130 con 0x7f5cac105a40 2024-03-29T05:10:21.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.574+0000 7f5ca37fe700 1 -- 172.21.15.142:0/464786316 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f5ca805bcd0 con 0x7f5cac105a40 2024-03-29T05:10:21.582 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T05:10:21.582 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T05:10:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is degraded 2024-03-29T05:10:21.582 INFO:teuthology.orchestra.run.smithi142.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T05:10:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T05:10:21.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.576+0000 7f5cb44ed700 1 -- 172.21.15.142:0/464786316 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5c9806dd60 msgr2=0x7f5c98070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:21.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.576+0000 7f5cb44ed700 1 --2- 172.21.15.142:0/464786316 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5c9806dd60 0x7f5c98070220 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c005fd0 tx=0x7f5c9c005ee0 comp rx=0 tx=0).stop 2024-03-29T05:10:21.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.576+0000 7f5cb44ed700 1 -- 172.21.15.142:0/464786316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cac105a40 msgr2=0x7f5cac19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:21.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.576+0000 7f5cb44ed700 1 --2- 172.21.15.142:0/464786316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cac105a40 0x7f5cac19ca80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80092e0 tx=0x7f5ca800be20 comp rx=0 tx=0).stop 2024-03-29T05:10:21.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.576+0000 7f5cb44ed700 1 -- 172.21.15.142:0/464786316 shutdown_connections 2024-03-29T05:10:21.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.576+0000 7f5cb44ed700 1 --2- 172.21.15.142:0/464786316 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f5c9806dd60 0x7f5c98070220 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:21.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.576+0000 7f5cb44ed700 1 --2- 172.21.15.142:0/464786316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cac0ffa40 0x7f5cac19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:21.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.576+0000 7f5cb44ed700 1 --2- 172.21.15.142:0/464786316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cac105a40 0x7f5cac19ca80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:21.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.576+0000 7f5cb44ed700 1 -- 172.21.15.142:0/464786316 >> 172.21.15.142:0/464786316 conn(0x7f5cac0fb550 msgr2=0x7f5cac103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:21.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.576+0000 7f5cb44ed700 1 -- 172.21.15.142:0/464786316 shutdown_connections 2024-03-29T05:10:21.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:21.576+0000 7f5cb44ed700 1 -- 172.21.15.142:0/464786316 wait complete. 2024-03-29T05:10:21.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:21 smithi142 ceph-mon[112475]: pgmap v661: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:21.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:21 smithi142 ceph-mon[112475]: from='client.35222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:21.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:21 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3973057869' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:10:21.879 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:21 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/464786316' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:10:22.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:21 smithi152 ceph-mon[88186]: pgmap v661: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:22.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:21 smithi152 ceph-mon[88186]: from='client.35222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:22.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:21 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3973057869' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:10:22.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:21 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/464786316' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:10:23.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:22 smithi152 ceph-mon[88186]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:23.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:22 smithi142 ceph-mon[112475]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:24.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:23 smithi152 ceph-mon[88186]: pgmap v662: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:24.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:23 smithi142 ceph-mon[112475]: pgmap v662: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:26.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:25 smithi152 ceph-mon[88186]: pgmap v663: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:26.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:25 smithi142 ceph-mon[112475]: pgmap v663: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:28.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:27 smithi152 ceph-mon[88186]: pgmap v664: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:28.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:27 smithi142 ceph-mon[112475]: pgmap v664: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:30.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:29 smithi152 ceph-mon[88186]: pgmap v665: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:30.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:10:30.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:29 smithi142 ceph-mon[112475]: pgmap v665: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:30.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:10:31.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:31 smithi142 ceph-mon[112475]: pgmap v666: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:32.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:31 smithi152 ceph-mon[88186]: pgmap v666: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:33 smithi152 ceph-mon[88186]: pgmap v667: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:34.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:33 smithi142 ceph-mon[112475]: pgmap v667: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:36.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:35 smithi152 ceph-mon[88186]: pgmap v668: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:36.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:10:36.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:10:36.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:35 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:36.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:35 smithi142 ceph-mon[112475]: pgmap v668: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:36.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:10:36.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:10:36.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:35 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:37.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: pgmap v669: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:37.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: Upgrade: Setting container_image for all mds 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.bmkjzf"}]': finished 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.wgmndr"}]': finished 2024-03-29T05:10:37.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T05:10:37.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi152.hiilum"}]': finished 2024-03-29T05:10:37.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T05:10:37.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi152.uhebsi"}]': finished 2024-03-29T05:10:37.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: Upgrade: Setting filesystem cephfs Joinable 2024-03-29T05:10:37.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-29T05:10:37.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: Standby daemon mds.cephfs.smithi142.wgmndr assigned to filesystem cephfs as rank 0 2024-03-29T05:10:37.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: Standby daemon mds.cephfs.smithi142.bmkjzf assigned to filesystem cephfs as rank 1 2024-03-29T05:10:37.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-29T05:10:37.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-29T05:10:37.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: fsmap cephfs:2/2 {0=cephfs.smithi142.wgmndr=up:replay,1=cephfs.smithi142.bmkjzf=up:replay} 2 up:standby 2024-03-29T05:10:37.049 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:36 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-29T05:10:37.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: pgmap v669: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-29T05:10:37.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-03-29T05:10:37.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:10:37.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:37.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:37.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: Upgrade: Setting container_image for all mds 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.bmkjzf"}]: dispatch 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.bmkjzf"}]': finished 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.wgmndr"}]: dispatch 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi142.wgmndr"}]': finished 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi152.hiilum"}]: dispatch 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi152.hiilum"}]': finished 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi152.uhebsi"}]: dispatch 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi152.uhebsi"}]': finished 2024-03-29T05:10:37.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: Upgrade: Setting filesystem cephfs Joinable 2024-03-29T05:10:37.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-29T05:10:37.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: Standby daemon mds.cephfs.smithi142.wgmndr assigned to filesystem cephfs as rank 0 2024-03-29T05:10:37.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: Standby daemon mds.cephfs.smithi142.bmkjzf assigned to filesystem cephfs as rank 1 2024-03-29T05:10:37.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-29T05:10:37.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-29T05:10:37.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: fsmap cephfs:2/2 {0=cephfs.smithi142.wgmndr=up:replay,1=cephfs.smithi142.bmkjzf=up:replay} 2 up:standby 2024-03-29T05:10:37.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:36 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-29T05:10:38.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:37 smithi152 ceph-mon[88186]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-29T05:10:38.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-29T05:10:38.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:37 smithi152 ceph-mon[88186]: fsmap cephfs:2/2 {0=cephfs.smithi142.wgmndr=up:replay,1=cephfs.smithi142.bmkjzf=up:replay} 2 up:standby 2024-03-29T05:10:38.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:38.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:38.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:38.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:38.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:38.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:37 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:38.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:37 smithi142 ceph-mon[112475]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-29T05:10:38.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-29T05:10:38.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:37 smithi142 ceph-mon[112475]: fsmap cephfs:2/2 {0=cephfs.smithi142.wgmndr=up:replay,1=cephfs.smithi142.bmkjzf=up:replay} 2 up:standby 2024-03-29T05:10:38.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:38.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:38.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:38.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:38.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:38.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:37 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:39.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:38 smithi152 ceph-mon[88186]: Upgrade: Setting container_image for all rgw 2024-03-29T05:10:39.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:38 smithi152 ceph-mon[88186]: Upgrade: Setting container_image for all rbd-mirror 2024-03-29T05:10:39.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:38 smithi152 ceph-mon[88186]: pgmap v670: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.8 MiB/s rd, 0 op/s 2024-03-29T05:10:39.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:38 smithi142 ceph-mon[112475]: Upgrade: Setting container_image for all rgw 2024-03-29T05:10:39.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:38 smithi142 ceph-mon[112475]: Upgrade: Setting container_image for all rbd-mirror 2024-03-29T05:10:39.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:38 smithi142 ceph-mon[112475]: pgmap v670: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.8 MiB/s rd, 0 op/s 2024-03-29T05:10:41.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:40 smithi152 ceph-mon[88186]: pgmap v671: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-03-29T05:10:41.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:40 smithi152 ceph-mon[88186]: Upgrade: Updating ceph-exporter.smithi142 (1/2) 2024-03-29T05:10:41.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:40 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:41.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:40 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T05:10:41.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:40 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:10:41.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:40 smithi152 ceph-mon[88186]: Deploying daemon ceph-exporter.smithi142 on smithi142 2024-03-29T05:10:41.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:40 smithi142 ceph-mon[112475]: pgmap v671: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-03-29T05:10:41.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:40 smithi142 ceph-mon[112475]: Upgrade: Updating ceph-exporter.smithi142 (1/2) 2024-03-29T05:10:41.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:40 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:41.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:40 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi142", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T05:10:41.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:40 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:10:41.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:40 smithi142 ceph-mon[112475]: Deploying daemon ceph-exporter.smithi142 on smithi142 2024-03-29T05:10:42.873 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:42 smithi142 ceph-mon[112475]: pgmap v672: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-03-29T05:10:43.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:42 smithi152 ceph-mon[88186]: pgmap v672: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-03-29T05:10:44.777 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:44 smithi152 ceph-mon[88186]: pgmap v673: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-03-29T05:10:44.777 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:44 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] up:resolve 2024-03-29T05:10:44.777 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:44 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] up:resolve 2024-03-29T05:10:44.777 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:44 smithi152 ceph-mon[88186]: fsmap cephfs:2/2 {0=cephfs.smithi142.wgmndr=up:resolve,1=cephfs.smithi142.bmkjzf=up:resolve} 2 up:standby 2024-03-29T05:10:44.777 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:44.777 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:44.777 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:44.777 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:10:45.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:44 smithi142 ceph-mon[112475]: pgmap v673: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-03-29T05:10:45.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:44 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] up:resolve 2024-03-29T05:10:45.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:44 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] up:resolve 2024-03-29T05:10:45.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:44 smithi142 ceph-mon[112475]: fsmap cephfs:2/2 {0=cephfs.smithi142.wgmndr=up:resolve,1=cephfs.smithi142.bmkjzf=up:resolve} 2 up:standby 2024-03-29T05:10:45.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:45.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:45.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:45.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:10:46.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:45 smithi152 ceph-mon[88186]: reconnect by client.24323 192.168.0.1:0/1101610675 after 0.000999982 2024-03-29T05:10:46.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:45 smithi152 ceph-mon[88186]: reconnect by client.24323 192.168.0.1:0/1101610675 after 0.000999982 2024-03-29T05:10:46.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:45 smithi152 ceph-mon[88186]: reconnect by client.14542 192.168.0.1:0/2208957537 after 0.000999982 2024-03-29T05:10:46.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:45 smithi152 ceph-mon[88186]: reconnect by client.14542 192.168.0.1:0/2208957537 after 0.000999982 2024-03-29T05:10:46.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:45 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] up:reconnect 2024-03-29T05:10:46.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:45 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] up:reconnect 2024-03-29T05:10:46.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:45 smithi152 ceph-mon[88186]: fsmap cephfs:2/2 {0=cephfs.smithi142.wgmndr=up:reconnect,1=cephfs.smithi142.bmkjzf=up:reconnect} 2 up:standby 2024-03-29T05:10:46.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:46.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi152", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T05:10:46.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:10:46.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:45 smithi142 ceph-mon[112475]: reconnect by client.24323 192.168.0.1:0/1101610675 after 0.000999982 2024-03-29T05:10:46.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:45 smithi142 ceph-mon[112475]: reconnect by client.24323 192.168.0.1:0/1101610675 after 0.000999982 2024-03-29T05:10:46.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:45 smithi142 ceph-mon[112475]: reconnect by client.14542 192.168.0.1:0/2208957537 after 0.000999982 2024-03-29T05:10:46.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:45 smithi142 ceph-mon[112475]: reconnect by client.14542 192.168.0.1:0/2208957537 after 0.000999982 2024-03-29T05:10:46.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:45 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] up:reconnect 2024-03-29T05:10:46.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:45 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] up:reconnect 2024-03-29T05:10:46.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:45 smithi142 ceph-mon[112475]: fsmap cephfs:2/2 {0=cephfs.smithi142.wgmndr=up:reconnect,1=cephfs.smithi142.bmkjzf=up:reconnect} 2 up:standby 2024-03-29T05:10:46.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:46.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi152", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-29T05:10:46.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:10:47.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:46 smithi152 ceph-mon[88186]: Upgrade: Updating ceph-exporter.smithi152 (2/2) 2024-03-29T05:10:47.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:46 smithi152 ceph-mon[88186]: Deploying daemon ceph-exporter.smithi152 on smithi152 2024-03-29T05:10:47.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:46 smithi152 ceph-mon[88186]: pgmap v674: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-03-29T05:10:47.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:46 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] up:rejoin 2024-03-29T05:10:47.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:46 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] up:rejoin 2024-03-29T05:10:47.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:46 smithi152 ceph-mon[88186]: fsmap cephfs:2/2 {0=cephfs.smithi142.wgmndr=up:rejoin,1=cephfs.smithi142.bmkjzf=up:rejoin} 2 up:standby 2024-03-29T05:10:47.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:46 smithi152 ceph-mon[88186]: daemon mds.cephfs.smithi142.wgmndr is now active in filesystem cephfs as rank 0 2024-03-29T05:10:47.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:46 smithi152 ceph-mon[88186]: daemon mds.cephfs.smithi142.bmkjzf is now active in filesystem cephfs as rank 1 2024-03-29T05:10:47.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:46 smithi152 ceph-mon[88186]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-29T05:10:47.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:46 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] up:active 2024-03-29T05:10:47.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:46 smithi152 ceph-mon[88186]: mds.? [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] up:active 2024-03-29T05:10:47.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:46 smithi152 ceph-mon[88186]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi142.bmkjzf=up:active} 2 up:standby-replay 2024-03-29T05:10:47.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:46 smithi142 ceph-mon[112475]: Upgrade: Updating ceph-exporter.smithi152 (2/2) 2024-03-29T05:10:47.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:46 smithi142 ceph-mon[112475]: Deploying daemon ceph-exporter.smithi152 on smithi152 2024-03-29T05:10:47.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:46 smithi142 ceph-mon[112475]: pgmap v674: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-03-29T05:10:47.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:46 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] up:rejoin 2024-03-29T05:10:47.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:46 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] up:rejoin 2024-03-29T05:10:47.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:46 smithi142 ceph-mon[112475]: fsmap cephfs:2/2 {0=cephfs.smithi142.wgmndr=up:rejoin,1=cephfs.smithi142.bmkjzf=up:rejoin} 2 up:standby 2024-03-29T05:10:47.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:46 smithi142 ceph-mon[112475]: daemon mds.cephfs.smithi142.wgmndr is now active in filesystem cephfs as rank 0 2024-03-29T05:10:47.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:46 smithi142 ceph-mon[112475]: daemon mds.cephfs.smithi142.bmkjzf is now active in filesystem cephfs as rank 1 2024-03-29T05:10:47.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:46 smithi142 ceph-mon[112475]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-29T05:10:47.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:46 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] up:active 2024-03-29T05:10:47.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:46 smithi142 ceph-mon[112475]: mds.? [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] up:active 2024-03-29T05:10:47.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:46 smithi142 ceph-mon[112475]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi142.bmkjzf=up:active} 2 up:standby-replay 2024-03-29T05:10:48.937 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:48 smithi152 ceph-mon[88186]: pgmap v675: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 85 B/s wr, 9 op/s 2024-03-29T05:10:49.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:48 smithi142 ceph-mon[112475]: pgmap v675: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 85 B/s wr, 9 op/s 2024-03-29T05:10:51.026 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:50 smithi152 ceph-mon[88186]: pgmap v676: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 31 MiB/s rd, 1.1 KiB/s wr, 16 op/s 2024-03-29T05:10:51.027 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:51.027 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:51.027 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:50 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:10:51.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:50 smithi142 ceph-mon[112475]: pgmap v676: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 31 MiB/s rd, 1.1 KiB/s wr, 16 op/s 2024-03-29T05:10:51.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:51.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:51.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:50 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:10:51.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.695+0000 7fec670d4700 1 -- 172.21.15.142:0/3088666570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec600ffbd0 msgr2=0x7fec60100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:51.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.695+0000 7fec670d4700 1 --2- 172.21.15.142:0/3088666570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec600ffbd0 0x7fec60100050 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009910 tx=0x7fec4c009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:51.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.695+0000 7fec670d4700 1 -- 172.21.15.142:0/3088666570 shutdown_connections 2024-03-29T05:10:51.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.695+0000 7fec670d4700 1 --2- 172.21.15.142:0/3088666570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec600ffbd0 0x7fec60100050 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:51.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.695+0000 7fec670d4700 1 --2- 172.21.15.142:0/3088666570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec60105a40 0x7fec60105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:51.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.695+0000 7fec670d4700 1 -- 172.21.15.142:0/3088666570 >> 172.21.15.142:0/3088666570 conn(0x7fec600755a0 msgr2=0x7fec600759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:51.704 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.695+0000 7fec670d4700 1 -- 172.21.15.142:0/3088666570 shutdown_connections 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.695+0000 7fec670d4700 1 -- 172.21.15.142:0/3088666570 wait complete. 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.695+0000 7fec670d4700 1 Processor -- start 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.695+0000 7fec670d4700 1 -- start start 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec670d4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec600ffbd0 0x7fec60195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec670d4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec60105a40 0x7fec60195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec670d4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec60196240 con 0x7fec600ffbd0 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec670d4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec60199fd0 con 0x7fec60105a40 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec64e70700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec600ffbd0 0x7fec60195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec64e70700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec600ffbd0 0x7fec60195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46350/0 (socket says 172.21.15.142:46350) 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec64e70700 1 -- 172.21.15.142:0/953049639 learned_addr learned my addr 172.21.15.142:0/953049639 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec5ffff700 1 --2- 172.21.15.142:0/953049639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec60105a40 0x7fec60195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec64e70700 1 -- 172.21.15.142:0/953049639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec60105a40 msgr2=0x7fec60195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec64e70700 1 --2- 172.21.15.142:0/953049639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec60105a40 0x7fec60195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:51.705 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec64e70700 1 -- 172.21.15.142:0/953049639 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c0095f0 con 0x7fec600ffbd0 2024-03-29T05:10:51.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec64e70700 1 --2- 172.21.15.142:0/953049639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec600ffbd0 0x7fec60195620 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fec5400e920 tx=0x7fec5400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:51.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec5dffb700 1 -- 172.21.15.142:0/953049639 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec5400c9d0 con 0x7fec600ffbd0 2024-03-29T05:10:51.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec5dffb700 1 -- 172.21.15.142:0/953049639 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec54010430 con 0x7fec600ffbd0 2024-03-29T05:10:51.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.696+0000 7fec670d4700 1 -- 172.21.15.142:0/953049639 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6019a2b0 con 0x7fec600ffbd0 2024-03-29T05:10:51.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.697+0000 7fec5dffb700 1 -- 172.21.15.142:0/953049639 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec54018630 con 0x7fec600ffbd0 2024-03-29T05:10:51.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.697+0000 7fec670d4700 1 -- 172.21.15.142:0/953049639 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6019a7b0 con 0x7fec600ffbd0 2024-03-29T05:10:51.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.698+0000 7fec670d4700 1 -- 172.21.15.142:0/953049639 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec60107fd0 con 0x7fec600ffbd0 2024-03-29T05:10:51.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.698+0000 7fec5dffb700 1 -- 172.21.15.142:0/953049639 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fec5400cb30 con 0x7fec600ffbd0 2024-03-29T05:10:51.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.699+0000 7fec5dffb700 1 --2- 172.21.15.142:0/953049639 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fec5006dd60 0x7fec50070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:51.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.699+0000 7fec5dffb700 1 -- 172.21.15.142:0/953049639 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fec54014070 con 0x7fec600ffbd0 2024-03-29T05:10:51.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.699+0000 7fec5ffff700 1 --2- 172.21.15.142:0/953049639 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fec5006dd60 0x7fec50070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:51.707 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.699+0000 7fec5ffff700 1 --2- 172.21.15.142:0/953049639 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fec5006dd60 0x7fec50070220 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fec60196c40 tx=0x7fec4c005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:51.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.704+0000 7fec5dffb700 1 -- 172.21.15.142:0/953049639 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fec54053280 con 0x7fec600ffbd0 2024-03-29T05:10:51.943 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.936+0000 7fec670d4700 1 -- 172.21.15.142:0/953049639 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec6019ab30 con 0x7fec5006dd60 2024-03-29T05:10:51.944 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.937+0000 7fec5dffb700 1 -- 172.21.15.142:0/953049639 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fec6019ab30 con 0x7fec5006dd60 2024-03-29T05:10:51.947 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.940+0000 7fec670d4700 1 -- 172.21.15.142:0/953049639 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fec5006dd60 msgr2=0x7fec50070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:51.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.940+0000 7fec670d4700 1 --2- 172.21.15.142:0/953049639 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fec5006dd60 0x7fec50070220 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fec60196c40 tx=0x7fec4c005940 comp rx=0 tx=0).stop 2024-03-29T05:10:51.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.940+0000 7fec670d4700 1 -- 172.21.15.142:0/953049639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec600ffbd0 msgr2=0x7fec60195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:51.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.940+0000 7fec670d4700 1 --2- 172.21.15.142:0/953049639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec600ffbd0 0x7fec60195620 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fec5400e920 tx=0x7fec5400ec30 comp rx=0 tx=0).stop 2024-03-29T05:10:51.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.941+0000 7fec670d4700 1 -- 172.21.15.142:0/953049639 shutdown_connections 2024-03-29T05:10:51.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.941+0000 7fec670d4700 1 --2- 172.21.15.142:0/953049639 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fec5006dd60 0x7fec50070220 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:51.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.941+0000 7fec670d4700 1 --2- 172.21.15.142:0/953049639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec600ffbd0 0x7fec60195620 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:51.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.941+0000 7fec670d4700 1 --2- 172.21.15.142:0/953049639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec60105a40 0x7fec60195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:51.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.941+0000 7fec670d4700 1 -- 172.21.15.142:0/953049639 >> 172.21.15.142:0/953049639 conn(0x7fec600755a0 msgr2=0x7fec600fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:51.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.941+0000 7fec670d4700 1 -- 172.21.15.142:0/953049639 shutdown_connections 2024-03-29T05:10:51.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:51.941+0000 7fec670d4700 1 -- 172.21.15.142:0/953049639 wait complete. 2024-03-29T05:10:51.961 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:10:52.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.053+0000 7f55cc740700 1 -- 172.21.15.142:0/1478390014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c4100620 msgr2=0x7f55c4100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:52.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.053+0000 7f55cc740700 1 --2- 172.21.15.142:0/1478390014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c4100620 0x7f55c4100a00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009910 tx=0x7f55c0009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:52.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.053+0000 7f55cc740700 1 -- 172.21.15.142:0/1478390014 shutdown_connections 2024-03-29T05:10:52.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.053+0000 7f55cc740700 1 --2- 172.21.15.142:0/1478390014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55c4100fd0 0x7f55c4104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.053+0000 7f55cc740700 1 --2- 172.21.15.142:0/1478390014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c4100620 0x7f55c4100a00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.053+0000 7f55cc740700 1 -- 172.21.15.142:0/1478390014 >> 172.21.15.142:0/1478390014 conn(0x7f55c4074e00 msgr2=0x7f55c4075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:52.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.053+0000 7f55cc740700 1 -- 172.21.15.142:0/1478390014 shutdown_connections 2024-03-29T05:10:52.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.053+0000 7f55cc740700 1 -- 172.21.15.142:0/1478390014 wait complete. 2024-03-29T05:10:52.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.054+0000 7f55cc740700 1 Processor -- start 2024-03-29T05:10:52.061 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.054+0000 7f55cc740700 1 -- start start 2024-03-29T05:10:52.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.054+0000 7f55cc740700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c4100620 0x7f55c41017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:52.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.054+0000 7f55cc740700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55c4100fd0 0x7f55c4101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:52.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.054+0000 7f55cc740700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c4102240 con 0x7f55c4100620 2024-03-29T05:10:52.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.054+0000 7f55cc740700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c4102380 con 0x7f55c4100fd0 2024-03-29T05:10:52.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.054+0000 7f55ca4dc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c4100620 0x7f55c41017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:52.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.054+0000 7f55ca4dc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c4100620 0x7f55c41017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46366/0 (socket says 172.21.15.142:46366) 2024-03-29T05:10:52.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.054+0000 7f55ca4dc700 1 -- 172.21.15.142:0/860190092 learned_addr learned my addr 172.21.15.142:0/860190092 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:52.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.054+0000 7f55c9cdb700 1 --2- 172.21.15.142:0/860190092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55c4100fd0 0x7f55c4101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:52.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.055+0000 7f55ca4dc700 1 -- 172.21.15.142:0/860190092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55c4100fd0 msgr2=0x7f55c4101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:52.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.055+0000 7f55ca4dc700 1 --2- 172.21.15.142:0/860190092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55c4100fd0 0x7f55c4101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.063 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.055+0000 7f55ca4dc700 1 -- 172.21.15.142:0/860190092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c00095f0 con 0x7f55c4100620 2024-03-29T05:10:52.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.055+0000 7f55ca4dc700 1 --2- 172.21.15.142:0/860190092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c4100620 0x7f55c41017c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f55c0006010 tx=0x7f55c000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:52.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.055+0000 7f55bb7fe700 1 -- 172.21.15.142:0/860190092 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55c001d070 con 0x7f55c4100620 2024-03-29T05:10:52.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.055+0000 7f55bb7fe700 1 -- 172.21.15.142:0/860190092 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55c000f460 con 0x7f55c4100620 2024-03-29T05:10:52.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.055+0000 7f55cc740700 1 -- 172.21.15.142:0/860190092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c4102600 con 0x7f55c4100620 2024-03-29T05:10:52.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.055+0000 7f55bb7fe700 1 -- 172.21.15.142:0/860190092 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55c0017610 con 0x7f55c4100620 2024-03-29T05:10:52.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.055+0000 7f55cc740700 1 -- 172.21.15.142:0/860190092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c41a2d50 con 0x7f55c4100620 2024-03-29T05:10:52.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.056+0000 7f55cc740700 1 -- 172.21.15.142:0/860190092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c41090a0 con 0x7f55c4100620 2024-03-29T05:10:52.065 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.057+0000 7f55bb7fe700 1 -- 172.21.15.142:0/860190092 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f55c000f5d0 con 0x7f55c4100620 2024-03-29T05:10:52.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.057+0000 7f55bb7fe700 1 --2- 172.21.15.142:0/860190092 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f55b006dd60 0x7f55b0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:52.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.058+0000 7f55bb7fe700 1 -- 172.21.15.142:0/860190092 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f55c008f210 con 0x7f55c4100620 2024-03-29T05:10:52.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.058+0000 7f55c9cdb700 1 --2- 172.21.15.142:0/860190092 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f55b006dd60 0x7f55b0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:52.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.058+0000 7f55c9cdb700 1 --2- 172.21.15.142:0/860190092 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f55b006dd60 0x7f55b0070220 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f55c4102fb0 tx=0x7f55b4009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:52.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.062+0000 7f55bb7fe700 1 -- 172.21.15.142:0/860190092 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f55c0058630 con 0x7f55c4100620 2024-03-29T05:10:52.080 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:51 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:52.080 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:51 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:52.080 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:51 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:52.080 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:51 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:52.080 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:51 smithi142 ceph-mon[112475]: pgmap v677: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-03-29T05:10:52.170 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:51 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:52.170 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:51 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:52.170 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:51 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:52.170 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:51 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:52.170 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:51 smithi152 ceph-mon[88186]: pgmap v677: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-03-29T05:10:52.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.285+0000 7f55cc740700 1 -- 172.21.15.142:0/860190092 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55c402cd80 con 0x7f55b006dd60 2024-03-29T05:10:52.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.287+0000 7f55bb7fe700 1 -- 172.21.15.142:0/860190092 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f55c402cd80 con 0x7f55b006dd60 2024-03-29T05:10:52.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.289+0000 7f55cc740700 1 -- 172.21.15.142:0/860190092 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f55b006dd60 msgr2=0x7f55b0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:52.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.289+0000 7f55cc740700 1 --2- 172.21.15.142:0/860190092 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f55b006dd60 0x7f55b0070220 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f55c4102fb0 tx=0x7f55b4009190 comp rx=0 tx=0).stop 2024-03-29T05:10:52.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.289+0000 7f55cc740700 1 -- 172.21.15.142:0/860190092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c4100620 msgr2=0x7f55c41017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:52.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.289+0000 7f55cc740700 1 --2- 172.21.15.142:0/860190092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c4100620 0x7f55c41017c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f55c0006010 tx=0x7f55c000bd70 comp rx=0 tx=0).stop 2024-03-29T05:10:52.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.289+0000 7f55cc740700 1 -- 172.21.15.142:0/860190092 shutdown_connections 2024-03-29T05:10:52.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.289+0000 7f55cc740700 1 --2- 172.21.15.142:0/860190092 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f55b006dd60 0x7f55b0070220 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.289+0000 7f55cc740700 1 --2- 172.21.15.142:0/860190092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c4100620 0x7f55c41017c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.289+0000 7f55cc740700 1 --2- 172.21.15.142:0/860190092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55c4100fd0 0x7f55c4101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.289+0000 7f55cc740700 1 -- 172.21.15.142:0/860190092 >> 172.21.15.142:0/860190092 conn(0x7f55c4074e00 msgr2=0x7f55c4103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:52.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.289+0000 7f55cc740700 1 -- 172.21.15.142:0/860190092 shutdown_connections 2024-03-29T05:10:52.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.289+0000 7f55cc740700 1 -- 172.21.15.142:0/860190092 wait complete. 2024-03-29T05:10:52.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.407+0000 7fde9e04e700 1 -- 172.21.15.142:0/244184379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde98104a50 msgr2=0x7fde98104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:52.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.407+0000 7fde9e04e700 1 --2- 172.21.15.142:0/244184379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde98104a50 0x7fde98104e30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fde88009910 tx=0x7fde88009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:52.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.407+0000 7fde9e04e700 1 -- 172.21.15.142:0/244184379 shutdown_connections 2024-03-29T05:10:52.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.407+0000 7fde9e04e700 1 --2- 172.21.15.142:0/244184379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde98100140 0x7fde981005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.407+0000 7fde9e04e700 1 --2- 172.21.15.142:0/244184379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde98104a50 0x7fde98104e30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.407+0000 7fde9e04e700 1 -- 172.21.15.142:0/244184379 >> 172.21.15.142:0/244184379 conn(0x7fde98075e30 msgr2=0x7fde980fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:52.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.407+0000 7fde9e04e700 1 -- 172.21.15.142:0/244184379 shutdown_connections 2024-03-29T05:10:52.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.407+0000 7fde9e04e700 1 -- 172.21.15.142:0/244184379 wait complete. 2024-03-29T05:10:52.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.408+0000 7fde9e04e700 1 Processor -- start 2024-03-29T05:10:52.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.408+0000 7fde9e04e700 1 -- start start 2024-03-29T05:10:52.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.409+0000 7fde9e04e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde98100140 0x7fde980fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:52.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.409+0000 7fde9e04e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde98104a50 0x7fde980fb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:52.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.409+0000 7fde9e04e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde980fbd20 con 0x7fde98100140 2024-03-29T05:10:52.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.409+0000 7fde9e04e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde980fbe60 con 0x7fde98104a50 2024-03-29T05:10:52.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.409+0000 7fde96ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde98104a50 0x7fde980fb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:52.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.409+0000 7fde96ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde98104a50 0x7fde980fb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.142:33052/0 (socket says 172.21.15.142:33052) 2024-03-29T05:10:52.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.409+0000 7fde96ffd700 1 -- 172.21.15.142:0/2896059966 learned_addr learned my addr 172.21.15.142:0/2896059966 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:52.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.409+0000 7fde977fe700 1 --2- 172.21.15.142:0/2896059966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde98100140 0x7fde980fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:52.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.409+0000 7fde96ffd700 1 -- 172.21.15.142:0/2896059966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde98100140 msgr2=0x7fde980fb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:52.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.409+0000 7fde96ffd700 1 --2- 172.21.15.142:0/2896059966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde98100140 0x7fde980fb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.409+0000 7fde96ffd700 1 -- 172.21.15.142:0/2896059966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde880095f0 con 0x7fde98104a50 2024-03-29T05:10:52.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.409+0000 7fde96ffd700 1 --2- 172.21.15.142:0/2896059966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde98104a50 0x7fde980fb7e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00e920 tx=0x7fde8c00ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:52.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.410+0000 7fde94ff9700 1 -- 172.21.15.142:0/2896059966 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde8c00c9d0 con 0x7fde98104a50 2024-03-29T05:10:52.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.410+0000 7fde94ff9700 1 -- 172.21.15.142:0/2896059966 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde8c010430 con 0x7fde98104a50 2024-03-29T05:10:52.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.410+0000 7fde94ff9700 1 -- 172.21.15.142:0/2896059966 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde8c018630 con 0x7fde98104a50 2024-03-29T05:10:52.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.410+0000 7fde9e04e700 1 -- 172.21.15.142:0/2896059966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde981a27e0 con 0x7fde98104a50 2024-03-29T05:10:52.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.410+0000 7fde9e04e700 1 -- 172.21.15.142:0/2896059966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde981a2bc0 con 0x7fde98104a50 2024-03-29T05:10:52.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.411+0000 7fde9e04e700 1 -- 172.21.15.142:0/2896059966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9804ea90 con 0x7fde98104a50 2024-03-29T05:10:52.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.413+0000 7fde94ff9700 1 -- 172.21.15.142:0/2896059966 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fde8c0105a0 con 0x7fde98104a50 2024-03-29T05:10:52.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.414+0000 7fde94ff9700 1 --2- 172.21.15.142:0/2896059966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fde8006dd60 0x7fde80070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:52.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.414+0000 7fde94ff9700 1 -- 172.21.15.142:0/2896059966 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fde8c014070 con 0x7fde98104a50 2024-03-29T05:10:52.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.414+0000 7fde977fe700 1 --2- 172.21.15.142:0/2896059966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fde8006dd60 0x7fde80070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:52.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.415+0000 7fde977fe700 1 --2- 172.21.15.142:0/2896059966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fde8006dd60 0x7fde80070220 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fde880059d0 tx=0x7fde88005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:52.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.421+0000 7fde94ff9700 1 -- 172.21.15.142:0/2896059966 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fde8c057370 con 0x7fde98104a50 2024-03-29T05:10:52.694 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.687+0000 7fde9e04e700 1 -- 172.21.15.142:0/2896059966 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fde981027f0 con 0x7fde8006dd60 2024-03-29T05:10:52.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.699+0000 7fde94ff9700 1 -- 172.21.15.142:0/2896059966 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fde981027f0 con 0x7fde8006dd60 2024-03-29T05:10:52.707 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:10:52.707 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (22m) 15m ago 34m 24.6M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:10:52.707 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 starting - - - - 2024-03-29T05:10:52.707 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 starting - - - - 2024-03-29T05:10:52.707 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (19m) 15m ago 34m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:10:52.707 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (19m) 15m ago 33m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:10:52.708 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (22m) 15m ago 33m 81.6M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:10:52.708 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (15m) 15m ago 28m 13.5M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:10:52.708 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (16m) 15m ago 28m 12.4M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:10:52.708 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (15m) 15m ago 28m 11.7M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:10:52.708 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (15m) 15m ago 28m 14.8M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:10:52.708 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (24m) 15m ago 35m 510M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:10:52.708 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (23m) 15m ago 33m 452M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:10:52.708 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (21m) 15m ago 35m 56.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:10:52.709 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (20m) 15m ago 33m 49.9M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:10:52.709 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (23m) 15m ago 34m 15.2M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:10:52.709 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (23m) 15m ago 33m 15.6M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:10:52.709 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (19m) 15m ago 32m 109M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:10:52.709 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (18m) 15m ago 31m 131M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:10:52.709 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (18m) 15m ago 31m 87.4M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:10:52.709 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (17m) 15m ago 31m 100M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:10:52.709 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (17m) 15m ago 30m 88.0M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:10:52.710 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (16m) 15m ago 30m 92.9M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:10:52.710 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (23m) 15m ago 33m 64.6M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:10:52.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.702+0000 7fde9e04e700 1 -- 172.21.15.142:0/2896059966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fde8006dd60 msgr2=0x7fde80070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:52.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.702+0000 7fde9e04e700 1 --2- 172.21.15.142:0/2896059966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fde8006dd60 0x7fde80070220 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fde880059d0 tx=0x7fde88005940 comp rx=0 tx=0).stop 2024-03-29T05:10:52.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.702+0000 7fde9e04e700 1 -- 172.21.15.142:0/2896059966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde98104a50 msgr2=0x7fde980fb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:52.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.702+0000 7fde9e04e700 1 --2- 172.21.15.142:0/2896059966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde98104a50 0x7fde980fb7e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00e920 tx=0x7fde8c00ec30 comp rx=0 tx=0).stop 2024-03-29T05:10:52.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.702+0000 7fde9e04e700 1 -- 172.21.15.142:0/2896059966 shutdown_connections 2024-03-29T05:10:52.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.702+0000 7fde9e04e700 1 --2- 172.21.15.142:0/2896059966 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fde8006dd60 0x7fde80070220 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.702+0000 7fde9e04e700 1 --2- 172.21.15.142:0/2896059966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde98100140 0x7fde980fb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.711 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.702+0000 7fde9e04e700 1 --2- 172.21.15.142:0/2896059966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde98104a50 0x7fde980fb7e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.702+0000 7fde9e04e700 1 -- 172.21.15.142:0/2896059966 >> 172.21.15.142:0/2896059966 conn(0x7fde98075e30 msgr2=0x7fde980fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:52.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.702+0000 7fde9e04e700 1 -- 172.21.15.142:0/2896059966 shutdown_connections 2024-03-29T05:10:52.712 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.702+0000 7fde9e04e700 1 -- 172.21.15.142:0/2896059966 wait complete. 2024-03-29T05:10:52.827 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.820+0000 7f9bb9812700 1 -- 172.21.15.142:0/2958475862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bb40ffbd0 msgr2=0x7f9bb4100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:52.827 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.820+0000 7f9bb9812700 1 --2- 172.21.15.142:0/2958475862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bb40ffbd0 0x7f9bb4100050 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009910 tx=0x7f9ba8009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:52.828 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.820+0000 7f9bb9812700 1 -- 172.21.15.142:0/2958475862 shutdown_connections 2024-03-29T05:10:52.828 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.820+0000 7f9bb9812700 1 --2- 172.21.15.142:0/2958475862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bb40ffbd0 0x7f9bb4100050 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.828 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.820+0000 7f9bb9812700 1 --2- 172.21.15.142:0/2958475862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb4105a40 0x7f9bb4105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.828 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.820+0000 7f9bb9812700 1 -- 172.21.15.142:0/2958475862 >> 172.21.15.142:0/2958475862 conn(0x7f9bb40755a0 msgr2=0x7f9bb40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:52.828 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.820+0000 7f9bb9812700 1 -- 172.21.15.142:0/2958475862 shutdown_connections 2024-03-29T05:10:52.828 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.820+0000 7f9bb9812700 1 -- 172.21.15.142:0/2958475862 wait complete. 2024-03-29T05:10:52.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.821+0000 7f9bb9812700 1 Processor -- start 2024-03-29T05:10:52.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.821+0000 7f9bb9812700 1 -- start start 2024-03-29T05:10:52.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.821+0000 7f9bb9812700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb40ffbd0 0x7f9bb419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:52.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.821+0000 7f9bb9812700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bb4105a40 0x7f9bb419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:52.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.821+0000 7f9bb9812700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb419d0d0 con 0x7f9bb4105a40 2024-03-29T05:10:52.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.821+0000 7f9bb9812700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb4196580 con 0x7f9bb40ffbd0 2024-03-29T05:10:52.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb27fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bb4105a40 0x7f9bb419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:52.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb27fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bb4105a40 0x7f9bb419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46408/0 (socket says 172.21.15.142:46408) 2024-03-29T05:10:52.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb40ffbd0 0x7f9bb419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:52.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb27fc700 1 -- 172.21.15.142:0/1767493954 learned_addr learned my addr 172.21.15.142:0/1767493954 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:52.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb27fc700 1 -- 172.21.15.142:0/1767493954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb40ffbd0 msgr2=0x7f9bb419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:52.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb27fc700 1 --2- 172.21.15.142:0/1767493954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb40ffbd0 0x7f9bb419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:52.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb27fc700 1 -- 172.21.15.142:0/1767493954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba80095f0 con 0x7f9bb4105a40 2024-03-29T05:10:52.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb27fc700 1 --2- 172.21.15.142:0/1767493954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bb4105a40 0x7f9bb419ca40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8005f50 tx=0x7f9ba800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:52.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb8810700 1 -- 172.21.15.142:0/1767493954 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ba801d070 con 0x7f9bb4105a40 2024-03-29T05:10:52.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb9812700 1 -- 172.21.15.142:0/1767493954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb4196800 con 0x7f9bb4105a40 2024-03-29T05:10:52.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb9812700 1 -- 172.21.15.142:0/1767493954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb4196cf0 con 0x7f9bb4105a40 2024-03-29T05:10:52.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb8810700 1 -- 172.21.15.142:0/1767493954 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ba800b610 con 0x7f9bb4105a40 2024-03-29T05:10:52.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.822+0000 7f9bb8810700 1 -- 172.21.15.142:0/1767493954 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ba8022470 con 0x7f9bb4105a40 2024-03-29T05:10:52.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.824+0000 7f9bb9812700 1 -- 172.21.15.142:0/1767493954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb4108fe0 con 0x7f9bb4105a40 2024-03-29T05:10:52.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.824+0000 7f9bb8810700 1 -- 172.21.15.142:0/1767493954 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f9ba8003780 con 0x7f9bb4105a40 2024-03-29T05:10:52.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.825+0000 7f9bb8810700 1 --2- 172.21.15.142:0/1767493954 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9b9c06dd60 0x7f9b9c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:52.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.825+0000 7f9bb8810700 1 -- 172.21.15.142:0/1767493954 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f9ba808ef50 con 0x7f9bb4105a40 2024-03-29T05:10:52.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.825+0000 7f9bb2ffd700 1 --2- 172.21.15.142:0/1767493954 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9b9c06dd60 0x7f9b9c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:52.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.825+0000 7f9bb2ffd700 1 --2- 172.21.15.142:0/1767493954 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9b9c06dd60 0x7f9b9c070220 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009c60 tx=0x7f9ba4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:52.839 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:52.833+0000 7f9bb8810700 1 -- 172.21.15.142:0/1767493954 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9ba80c3280 con 0x7f9bb4105a40 2024-03-29T05:10:53.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:52 smithi152 ceph-mon[88186]: from='client.35246 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:53.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:52 smithi152 ceph-mon[88186]: from='client.35250 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:53.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:52 smithi152 ceph-mon[88186]: from='client.44869 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:53.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.305+0000 7f9bb9812700 1 -- 172.21.15.142:0/1767493954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9bb404ea90 con 0x7f9bb4105a40 2024-03-29T05:10:53.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:52 smithi142 ceph-mon[112475]: from='client.35246 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:53.312 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:52 smithi142 ceph-mon[112475]: from='client.35250 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:53.313 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:52 smithi142 ceph-mon[112475]: from='client.44869 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:53.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.306+0000 7f9bb8810700 1 -- 172.21.15.142:0/1767493954 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9ba805bf20 con 0x7f9bb4105a40 2024-03-29T05:10:53.313 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:10:53.313 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:10:53.313 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:10:53.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:10:53.313 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:10:53.313 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:10:53.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:10:53.314 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:10:53.314 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:10:53.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:10:53.314 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:10:53.314 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:10:53.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:10:53.314 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:10:53.314 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:10:53.314 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:10:53.314 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:10:53.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.308+0000 7f9bb9812700 1 -- 172.21.15.142:0/1767493954 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9b9c06dd60 msgr2=0x7f9b9c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:53.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.308+0000 7f9bb9812700 1 --2- 172.21.15.142:0/1767493954 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9b9c06dd60 0x7f9b9c070220 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009c60 tx=0x7f9ba4009260 comp rx=0 tx=0).stop 2024-03-29T05:10:53.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.308+0000 7f9bb9812700 1 -- 172.21.15.142:0/1767493954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bb4105a40 msgr2=0x7f9bb419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:53.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.308+0000 7f9bb9812700 1 --2- 172.21.15.142:0/1767493954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bb4105a40 0x7f9bb419ca40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8005f50 tx=0x7f9ba800f690 comp rx=0 tx=0).stop 2024-03-29T05:10:53.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.308+0000 7f9bb9812700 1 -- 172.21.15.142:0/1767493954 shutdown_connections 2024-03-29T05:10:53.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.308+0000 7f9bb9812700 1 --2- 172.21.15.142:0/1767493954 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9b9c06dd60 0x7f9b9c070220 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:53.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.308+0000 7f9bb9812700 1 --2- 172.21.15.142:0/1767493954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bb40ffbd0 0x7f9bb419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:53.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.308+0000 7f9bb9812700 1 --2- 172.21.15.142:0/1767493954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9bb4105a40 0x7f9bb419ca40 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:53.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.308+0000 7f9bb9812700 1 -- 172.21.15.142:0/1767493954 >> 172.21.15.142:0/1767493954 conn(0x7f9bb40755a0 msgr2=0x7f9bb4068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:53.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.308+0000 7f9bb9812700 1 -- 172.21.15.142:0/1767493954 shutdown_connections 2024-03-29T05:10:53.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.308+0000 7f9bb9812700 1 -- 172.21.15.142:0/1767493954 wait complete. 2024-03-29T05:10:53.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.430+0000 7f8a034d3700 1 -- 172.21.15.142:0/2936385496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89fc068410 msgr2=0x7f89fc0687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:53.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.430+0000 7f8a034d3700 1 --2- 172.21.15.142:0/2936385496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89fc068410 0x7f89fc0687f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009910 tx=0x7f89f8009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:53.437 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.430+0000 7f8a034d3700 1 -- 172.21.15.142:0/2936385496 shutdown_connections 2024-03-29T05:10:53.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.430+0000 7f8a034d3700 1 --2- 172.21.15.142:0/2936385496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89fc068d30 0x7f89fc10d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:53.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.430+0000 7f8a034d3700 1 --2- 172.21.15.142:0/2936385496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89fc068410 0x7f89fc0687f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:53.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.430+0000 7f8a034d3700 1 -- 172.21.15.142:0/2936385496 >> 172.21.15.142:0/2936385496 conn(0x7f89fc0755a0 msgr2=0x7f89fc0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:53.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.430+0000 7f8a034d3700 1 -- 172.21.15.142:0/2936385496 shutdown_connections 2024-03-29T05:10:53.438 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.430+0000 7f8a034d3700 1 -- 172.21.15.142:0/2936385496 wait complete. 2024-03-29T05:10:53.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.431+0000 7f8a034d3700 1 Processor -- start 2024-03-29T05:10:53.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.431+0000 7f8a034d3700 1 -- start start 2024-03-29T05:10:53.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.431+0000 7f8a034d3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89fc068410 0x7f89fc19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:53.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.431+0000 7f8a034d3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89fc068d30 0x7f89fc19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:53.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.431+0000 7f8a034d3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc19f280 con 0x7f89fc068d30 2024-03-29T05:10:53.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.431+0000 7f8a034d3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc198730 con 0x7f89fc068410 2024-03-29T05:10:53.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.431+0000 7f8a00a6e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89fc068d30 0x7f89fc19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:53.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.431+0000 7f8a0126f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89fc068410 0x7f89fc19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:53.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.431+0000 7f8a00a6e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89fc068d30 0x7f89fc19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46418/0 (socket says 172.21.15.142:46418) 2024-03-29T05:10:53.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.431+0000 7f8a00a6e700 1 -- 172.21.15.142:0/3909800355 learned_addr learned my addr 172.21.15.142:0/3909800355 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:53.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.432+0000 7f8a0126f700 1 -- 172.21.15.142:0/3909800355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89fc068d30 msgr2=0x7f89fc19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:53.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.432+0000 7f8a0126f700 1 --2- 172.21.15.142:0/3909800355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89fc068d30 0x7f89fc19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:53.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.432+0000 7f8a0126f700 1 -- 172.21.15.142:0/3909800355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f80095f0 con 0x7f89fc068410 2024-03-29T05:10:53.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.432+0000 7f8a00a6e700 1 --2- 172.21.15.142:0/3909800355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89fc068d30 0x7f89fc19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T05:10:53.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.432+0000 7f8a0126f700 1 --2- 172.21.15.142:0/3909800355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89fc068410 0x7f89fc19e6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f89f8006010 tx=0x7f89f800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:53.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.432+0000 7f89f27fc700 1 -- 172.21.15.142:0/3909800355 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89f801d070 con 0x7f89fc068410 2024-03-29T05:10:53.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.432+0000 7f89f27fc700 1 -- 172.21.15.142:0/3909800355 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89f800b610 con 0x7f89fc068410 2024-03-29T05:10:53.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.432+0000 7f8a034d3700 1 -- 172.21.15.142:0/3909800355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc1989b0 con 0x7f89fc068410 2024-03-29T05:10:53.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.432+0000 7f89f27fc700 1 -- 172.21.15.142:0/3909800355 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89f8022470 con 0x7f89fc068410 2024-03-29T05:10:53.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.432+0000 7f8a034d3700 1 -- 172.21.15.142:0/3909800355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc198ed0 con 0x7f89fc068410 2024-03-29T05:10:53.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.433+0000 7f8a034d3700 1 -- 172.21.15.142:0/3909800355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc10a9b0 con 0x7f89fc068410 2024-03-29T05:10:53.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.434+0000 7f89f27fc700 1 -- 172.21.15.142:0/3909800355 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f89f80050d0 con 0x7f89fc068410 2024-03-29T05:10:53.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.434+0000 7f89f27fc700 1 --2- 172.21.15.142:0/3909800355 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f89e806dd60 0x7f89e8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:53.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.434+0000 7f89f27fc700 1 -- 172.21.15.142:0/3909800355 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f89f808f260 con 0x7f89fc068410 2024-03-29T05:10:53.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.434+0000 7f8a00a6e700 1 --2- 172.21.15.142:0/3909800355 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f89e806dd60 0x7f89e8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:53.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.434+0000 7f8a00a6e700 1 --2- 172.21.15.142:0/3909800355 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f89e806dd60 0x7f89e8070220 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f89fc199ba0 tx=0x7f89ec009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:53.445 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.439+0000 7f89f27fc700 1 -- 172.21.15.142:0/3909800355 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f89f805c1c0 con 0x7f89fc068410 2024-03-29T05:10:53.720 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.713+0000 7f8a034d3700 1 -- 172.21.15.142:0/3909800355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f89fc04ea90 con 0x7f89fc068410 2024-03-29T05:10:53.721 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.713+0000 7f89f27fc700 1 -- 172.21.15.142:0/3909800355 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1973 (secure 0 0 0) 0x7f89f805bd50 con 0x7f89fc068410 2024-03-29T05:10:53.721 INFO:teuthology.orchestra.run.smithi142.stdout:e38 2024-03-29T05:10:53.721 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:10:53.721 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:10:53.722 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:10:53.722 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:10:53.722 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:10:53.722 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:10:53.722 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 38 2024-03-29T05:10:53.722 INFO:teuthology.orchestra.run.smithi142.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T05:10:53.723 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:10:53.723 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T05:10:46.360840+0000 2024-03-29T05:10:53.723 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:10:53.723 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:10:53.723 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:10:53.723 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:10:53.724 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:10:53.724 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:10:53.724 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:10:53.724 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:10:53.724 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:10:53.724 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-29T05:10:53.725 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:10:53.725 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:10:53.725 INFO:teuthology.orchestra.run.smithi142.stdout:up {0=34420,1=34424} 2024-03-29T05:10:53.725 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T05:10:53.725 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:10:53.725 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:10:53.726 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:10:53.726 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:10:53.726 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:10:53.726 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:10:53.726 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:10:53.726 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:10:53.727 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 34420 members: 34420,34424 2024-03-29T05:10:53.727 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{0:34420} state up:active seq 245 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:10:53.727 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{0:44323} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:10:53.727 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{1:34424} state up:active seq 241 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:10:53.727 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{1:44327} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:10:53.727 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:10:53.728 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:10:53.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.716+0000 7f8a034d3700 1 -- 172.21.15.142:0/3909800355 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f89e806dd60 msgr2=0x7f89e8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:53.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.716+0000 7f8a034d3700 1 --2- 172.21.15.142:0/3909800355 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f89e806dd60 0x7f89e8070220 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f89fc199ba0 tx=0x7f89ec009260 comp rx=0 tx=0).stop 2024-03-29T05:10:53.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.716+0000 7f8a034d3700 1 -- 172.21.15.142:0/3909800355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89fc068410 msgr2=0x7f89fc19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:53.729 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.716+0000 7f8a034d3700 1 --2- 172.21.15.142:0/3909800355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89fc068410 0x7f89fc19e6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f89f8006010 tx=0x7f89f800f690 comp rx=0 tx=0).stop 2024-03-29T05:10:53.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.716+0000 7f8a034d3700 1 -- 172.21.15.142:0/3909800355 shutdown_connections 2024-03-29T05:10:53.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.716+0000 7f8a034d3700 1 --2- 172.21.15.142:0/3909800355 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f89e806dd60 0x7f89e8070220 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:53.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.716+0000 7f8a034d3700 1 --2- 172.21.15.142:0/3909800355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89fc068410 0x7f89fc19e6b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:53.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.716+0000 7f8a034d3700 1 --2- 172.21.15.142:0/3909800355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89fc068d30 0x7f89fc19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:53.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.716+0000 7f8a034d3700 1 -- 172.21.15.142:0/3909800355 >> 172.21.15.142:0/3909800355 conn(0x7f89fc0755a0 msgr2=0x7f89fc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:53.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.716+0000 7f8a034d3700 1 -- 172.21.15.142:0/3909800355 shutdown_connections 2024-03-29T05:10:53.730 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.716+0000 7f8a034d3700 1 -- 172.21.15.142:0/3909800355 wait complete. 2024-03-29T05:10:53.731 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 38 2024-03-29T05:10:53.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.838+0000 7f0309626700 1 -- 172.21.15.142:0/2923195166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0304100c90 msgr2=0x7f03041046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:53.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.838+0000 7f0309626700 1 --2- 172.21.15.142:0/2923195166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0304100c90 0x7f03041046d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009910 tx=0x7f02f8009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:53.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.838+0000 7f0309626700 1 -- 172.21.15.142:0/2923195166 shutdown_connections 2024-03-29T05:10:53.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.838+0000 7f0309626700 1 --2- 172.21.15.142:0/2923195166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0304100c90 0x7f03041046d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:53.845 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.838+0000 7f0309626700 1 --2- 172.21.15.142:0/2923195166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03041002e0 0x7f03041006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:53.846 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.838+0000 7f0309626700 1 -- 172.21.15.142:0/2923195166 >> 172.21.15.142:0/2923195166 conn(0x7f0304075e30 msgr2=0x7f03040fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:53.846 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.838+0000 7f0309626700 1 -- 172.21.15.142:0/2923195166 shutdown_connections 2024-03-29T05:10:53.846 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.838+0000 7f0309626700 1 -- 172.21.15.142:0/2923195166 wait complete. 2024-03-29T05:10:53.847 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.839+0000 7f0309626700 1 Processor -- start 2024-03-29T05:10:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.839+0000 7f0309626700 1 -- start start 2024-03-29T05:10:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.839+0000 7f0309626700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03041002e0 0x7f0304197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.839+0000 7f0309626700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0304100c90 0x7f0304197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.839+0000 7f0309626700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041981c0 con 0x7f03041002e0 2024-03-29T05:10:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.839+0000 7f0309626700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0304198300 con 0x7f0304100c90 2024-03-29T05:10:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.839+0000 7f0302ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03041002e0 0x7f0304197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.839+0000 7f03027fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0304100c90 0x7f0304197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.839+0000 7f0302ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03041002e0 0x7f0304197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46446/0 (socket says 172.21.15.142:46446) 2024-03-29T05:10:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.839+0000 7f0302ffd700 1 -- 172.21.15.142:0/3673832802 learned_addr learned my addr 172.21.15.142:0/3673832802 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.840+0000 7f03027fc700 1 -- 172.21.15.142:0/3673832802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03041002e0 msgr2=0x7f0304197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.840+0000 7f03027fc700 1 --2- 172.21.15.142:0/3673832802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03041002e0 0x7f0304197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:53.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.840+0000 7f03027fc700 1 -- 172.21.15.142:0/3673832802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f80095f0 con 0x7f0304100c90 2024-03-29T05:10:53.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.840+0000 7f0302ffd700 1 --2- 172.21.15.142:0/3673832802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03041002e0 0x7f0304197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T05:10:53.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.840+0000 7f03027fc700 1 --2- 172.21.15.142:0/3673832802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0304100c90 0x7f0304197ba0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0304073150 tx=0x7f02f801cdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:53.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.840+0000 7f02ebfff700 1 -- 172.21.15.142:0/3673832802 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02f8021070 con 0x7f0304100c90 2024-03-29T05:10:53.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.840+0000 7f02ebfff700 1 -- 172.21.15.142:0/3673832802 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02f8005dc0 con 0x7f0304100c90 2024-03-29T05:10:53.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.840+0000 7f0309626700 1 -- 172.21.15.142:0/3673832802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0304195450 con 0x7f0304100c90 2024-03-29T05:10:53.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.840+0000 7f02ebfff700 1 -- 172.21.15.142:0/3673832802 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02f800f690 con 0x7f0304100c90 2024-03-29T05:10:53.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.840+0000 7f0309626700 1 -- 172.21.15.142:0/3673832802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0304195940 con 0x7f0304100c90 2024-03-29T05:10:53.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.842+0000 7f0309626700 1 -- 172.21.15.142:0/3673832802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0304108110 con 0x7f0304100c90 2024-03-29T05:10:53.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.842+0000 7f02ebfff700 1 -- 172.21.15.142:0/3673832802 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f02f800f8d0 con 0x7f0304100c90 2024-03-29T05:10:53.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.842+0000 7f02ebfff700 1 --2- 172.21.15.142:0/3673832802 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f02ec06dd60 0x7f02ec070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:53.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.843+0000 7f02ebfff700 1 -- 172.21.15.142:0/3673832802 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f02f8093240 con 0x7f0304100c90 2024-03-29T05:10:53.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.843+0000 7f0302ffd700 1 --2- 172.21.15.142:0/3673832802 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f02ec06dd60 0x7f02ec070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:53.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.843+0000 7f0302ffd700 1 --2- 172.21.15.142:0/3673832802 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f02ec06dd60 0x7f02ec070220 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02f40104e0 tx=0x7f02f4007480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:53.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:53.848+0000 7f02ebfff700 1 -- 172.21.15.142:0/3673832802 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f02f805c600 con 0x7f0304100c90 2024-03-29T05:10:54.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.090+0000 7f0309626700 1 -- 172.21.15.142:0/3673832802 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03040611d0 con 0x7f02ec06dd60 2024-03-29T05:10:54.098 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.091+0000 7f02ebfff700 1 -- 172.21.15.142:0/3673832802 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f03040611d0 con 0x7f02ec06dd60 2024-03-29T05:10:54.098 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28", 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": true, 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [ 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr", 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: "crash", 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: "mon", 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: "osd", 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: "mds" 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": "17/23 daemons upgraded", 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:10:54.099 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:10:54.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.094+0000 7f0309626700 1 -- 172.21.15.142:0/3673832802 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f02ec06dd60 msgr2=0x7f02ec070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:54.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.094+0000 7f0309626700 1 --2- 172.21.15.142:0/3673832802 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f02ec06dd60 0x7f02ec070220 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02f40104e0 tx=0x7f02f4007480 comp rx=0 tx=0).stop 2024-03-29T05:10:54.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.094+0000 7f0309626700 1 -- 172.21.15.142:0/3673832802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0304100c90 msgr2=0x7f0304197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:54.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.094+0000 7f0309626700 1 --2- 172.21.15.142:0/3673832802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0304100c90 0x7f0304197ba0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0304073150 tx=0x7f02f801cdc0 comp rx=0 tx=0).stop 2024-03-29T05:10:54.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.094+0000 7f0309626700 1 -- 172.21.15.142:0/3673832802 shutdown_connections 2024-03-29T05:10:54.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.094+0000 7f0309626700 1 --2- 172.21.15.142:0/3673832802 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f02ec06dd60 0x7f02ec070220 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:54.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.094+0000 7f0309626700 1 --2- 172.21.15.142:0/3673832802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03041002e0 0x7f0304197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:54.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.094+0000 7f0309626700 1 --2- 172.21.15.142:0/3673832802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0304100c90 0x7f0304197ba0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:54.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.094+0000 7f0309626700 1 -- 172.21.15.142:0/3673832802 >> 172.21.15.142:0/3673832802 conn(0x7f0304075e30 msgr2=0x7f0304103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:54.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.094+0000 7f0309626700 1 -- 172.21.15.142:0/3673832802 shutdown_connections 2024-03-29T05:10:54.101 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.094+0000 7f0309626700 1 -- 172.21.15.142:0/3673832802 wait complete. 2024-03-29T05:10:54.227 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.220+0000 7fd91e25b700 1 -- 172.21.15.142:0/3112316058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd918101750 msgr2=0x7fd9181057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:54.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.220+0000 7fd91e25b700 1 --2- 172.21.15.142:0/3112316058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd918101750 0x7fd9181057a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009910 tx=0x7fd90c009c20 comp rx=0 tx=0).stop 2024-03-29T05:10:54.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.220+0000 7fd91e25b700 1 -- 172.21.15.142:0/3112316058 shutdown_connections 2024-03-29T05:10:54.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.220+0000 7fd91e25b700 1 --2- 172.21.15.142:0/3112316058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd918101750 0x7fd9181057a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:54.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.220+0000 7fd91e25b700 1 --2- 172.21.15.142:0/3112316058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd918100da0 0x7fd918101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:54.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.220+0000 7fd91e25b700 1 -- 172.21.15.142:0/3112316058 >> 172.21.15.142:0/3112316058 conn(0x7fd9180fc630 msgr2=0x7fd9180fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:54.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.220+0000 7fd91e25b700 1 -- 172.21.15.142:0/3112316058 shutdown_connections 2024-03-29T05:10:54.228 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.220+0000 7fd91e25b700 1 -- 172.21.15.142:0/3112316058 wait complete. 2024-03-29T05:10:54.228 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:53 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1767493954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:54.228 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:53 smithi142 ceph-mon[112475]: pgmap v678: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-29T05:10:54.229 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:53 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3909800355' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:10:54.229 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:53 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:54.229 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:53 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:54.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.221+0000 7fd91e25b700 1 Processor -- start 2024-03-29T05:10:54.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.221+0000 7fd91e25b700 1 -- start start 2024-03-29T05:10:54.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.221+0000 7fd91e25b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd918100da0 0x7fd91819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:54.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.221+0000 7fd91e25b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd918101750 0x7fd91819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:54.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.221+0000 7fd91e25b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91819d0d0 con 0x7fd918101750 2024-03-29T05:10:54.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.221+0000 7fd91e25b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd918196580 con 0x7fd918100da0 2024-03-29T05:10:54.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd9177fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd918100da0 0x7fd91819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:54.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd916ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd918101750 0x7fd91819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:54.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd9177fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd918100da0 0x7fd91819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.142:33156/0 (socket says 172.21.15.142:33156) 2024-03-29T05:10:54.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd916ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd918101750 0x7fd91819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46462/0 (socket says 172.21.15.142:46462) 2024-03-29T05:10:54.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd9177fe700 1 -- 172.21.15.142:0/1958622633 learned_addr learned my addr 172.21.15.142:0/1958622633 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:10:54.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd916ffd700 1 -- 172.21.15.142:0/1958622633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd918100da0 msgr2=0x7fd91819c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:54.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd916ffd700 1 --2- 172.21.15.142:0/1958622633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd918100da0 0x7fd91819c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:54.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd916ffd700 1 -- 172.21.15.142:0/1958622633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c0095f0 con 0x7fd918101750 2024-03-29T05:10:54.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd9177fe700 1 --2- 172.21.15.142:0/1958622633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd918100da0 0x7fd91819c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T05:10:54.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd916ffd700 1 --2- 172.21.15.142:0/1958622633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd918101750 0x7fd91819ca40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd91806a090 tx=0x7fd90c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:54.231 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd914ff9700 1 -- 172.21.15.142:0/1958622633 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd90c01d070 con 0x7fd918101750 2024-03-29T05:10:54.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd91e25b700 1 -- 172.21.15.142:0/1958622633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd918196800 con 0x7fd918101750 2024-03-29T05:10:54.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd914ff9700 1 -- 172.21.15.142:0/1958622633 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd90c005dc0 con 0x7fd918101750 2024-03-29T05:10:54.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd914ff9700 1 -- 172.21.15.142:0/1958622633 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd90c00fe80 con 0x7fd918101750 2024-03-29T05:10:54.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.222+0000 7fd91e25b700 1 -- 172.21.15.142:0/1958622633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd918196cf0 con 0x7fd918101750 2024-03-29T05:10:54.232 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.224+0000 7fd91e25b700 1 -- 172.21.15.142:0/1958622633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9181091a0 con 0x7fd918101750 2024-03-29T05:10:54.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.224+0000 7fd914ff9700 1 -- 172.21.15.142:0/1958622633 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fd90c00f6f0 con 0x7fd918101750 2024-03-29T05:10:54.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.225+0000 7fd914ff9700 1 --2- 172.21.15.142:0/1958622633 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd90006dd60 0x7fd900070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:10:54.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.225+0000 7fd914ff9700 1 -- 172.21.15.142:0/1958622633 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd90c08ef20 con 0x7fd918101750 2024-03-29T05:10:54.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.225+0000 7fd9177fe700 1 --2- 172.21.15.142:0/1958622633 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd90006dd60 0x7fd900070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:10:54.233 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.225+0000 7fd9177fe700 1 --2- 172.21.15.142:0/1958622633 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd90006dd60 0x7fd900070220 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd9080104e0 tx=0x7fd908007480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:10:54.236 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.229+0000 7fd914ff9700 1 -- 172.21.15.142:0/1958622633 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd90c0c3280 con 0x7fd918101750 2024-03-29T05:10:54.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:53 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1767493954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:10:54.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:53 smithi152 ceph-mon[88186]: pgmap v678: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-29T05:10:54.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:53 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3909800355' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:10:54.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:53 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:54.298 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:53 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:54.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.529+0000 7fd91e25b700 1 -- 172.21.15.142:0/1958622633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd918196fd0 con 0x7fd918101750 2024-03-29T05:10:54.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.530+0000 7fd914ff9700 1 -- 172.21.15.142:0/1958622633 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4081 (secure 0 0 0) 0x7fd90c05bef0 con 0x7fd918101750 2024-03-29T05:10:54.537 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_WARN failed to probe daemons or devices 2024-03-29T05:10:54.537 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T05:10:54.537 INFO:teuthology.orchestra.run.smithi142.stdout: host smithi152 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi152/config 2024-03-29T05:10:54.537 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 -e NODE_NAME=smithi152 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/9f96302c-ed85-11ee-95d0-87774f69a715:/var/run/ceph:z -v /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpw7p6g95h:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 inventory --format=json-pretty --filter-for-batch 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:Traceback (most recent call last): 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout: return _run_code(code, main_globals, None, 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout: exec(code, run_globals) 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout: File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 5528, in 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout: File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 5516, in main 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout: File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 406, in _infer_config 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout: File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 321, in _infer_fsid 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout: File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 434, in _infer_image 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout: File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 308, in _validate_fsid 2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout: File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 3303, in command_ceph_volume 2024-03-29T05:10:54.539 INFO:teuthology.orchestra.run.smithi142.stdout: File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-29T05:10:54.539 INFO:teuthology.orchestra.run.smithi142.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 -e NODE_NAME=smithi152 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/9f96302c-ed85-11ee-95d0-87774f69a715:/var/run/ceph:z -v /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpw7p6g95h:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 inventory --format=json-pretty --filter-for-batch 2024-03-29T05:10:54.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.532+0000 7fd91e25b700 1 -- 172.21.15.142:0/1958622633 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd90006dd60 msgr2=0x7fd900070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:54.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.532+0000 7fd91e25b700 1 --2- 172.21.15.142:0/1958622633 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd90006dd60 0x7fd900070220 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd9080104e0 tx=0x7fd908007480 comp rx=0 tx=0).stop 2024-03-29T05:10:54.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.532+0000 7fd91e25b700 1 -- 172.21.15.142:0/1958622633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd918101750 msgr2=0x7fd91819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:10:54.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.532+0000 7fd91e25b700 1 --2- 172.21.15.142:0/1958622633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd918101750 0x7fd91819ca40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd91806a090 tx=0x7fd90c00bd70 comp rx=0 tx=0).stop 2024-03-29T05:10:54.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.532+0000 7fd91e25b700 1 -- 172.21.15.142:0/1958622633 shutdown_connections 2024-03-29T05:10:54.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.532+0000 7fd91e25b700 1 --2- 172.21.15.142:0/1958622633 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd90006dd60 0x7fd900070220 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:54.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.532+0000 7fd91e25b700 1 --2- 172.21.15.142:0/1958622633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd918100da0 0x7fd91819c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:54.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.532+0000 7fd91e25b700 1 --2- 172.21.15.142:0/1958622633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd918101750 0x7fd91819ca40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:10:54.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.532+0000 7fd91e25b700 1 -- 172.21.15.142:0/1958622633 >> 172.21.15.142:0/1958622633 conn(0x7fd9180fc630 msgr2=0x7fd9180fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:10:54.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.532+0000 7fd91e25b700 1 -- 172.21.15.142:0/1958622633 shutdown_connections 2024-03-29T05:10:54.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:10:54.532+0000 7fd91e25b700 1 -- 172.21.15.142:0/1958622633 wait complete. 2024-03-29T05:10:55.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:54 smithi152 ceph-mon[88186]: from='client.44881 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:55.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:54 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1958622633' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:10:55.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:55.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:54 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:55.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:54 smithi142 ceph-mon[112475]: from='client.44881 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:10:55.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:54 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1958622633' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:10:55.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:55.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:54 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:56.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:55 smithi152 ceph-mon[88186]: pgmap v679: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-29T05:10:56.364 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:55 smithi142 ceph-mon[112475]: pgmap v679: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-29T05:10:58.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:58 smithi142 ceph-mon[112475]: pgmap v680: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-29T05:10:58.976 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:58.977 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:58 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:59.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:58 smithi152 ceph-mon[88186]: pgmap v680: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-29T05:10:59.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:10:59.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:58 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:00.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:10:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:00.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:00.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:00.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:11:00.298 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:00 smithi152 ceph-mon[88186]: pgmap v681: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 11 KiB/s wr, 17 op/s 2024-03-29T05:11:00.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:00.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:00.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:00.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:11:00.378 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:10:59 smithi142 ceph-mon[112475]: pgmap v681: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 11 KiB/s wr, 17 op/s 2024-03-29T05:11:02.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:02 smithi142 ceph-mon[112475]: pgmap v682: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 10 KiB/s wr, 10 op/s 2024-03-29T05:11:02.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:02 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:02.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:02 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:03.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:02 smithi152 ceph-mon[88186]: pgmap v682: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 10 KiB/s wr, 10 op/s 2024-03-29T05:11:03.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:02 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:03.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:02 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:05.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: pgmap v683: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 6.9 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-03-29T05:11:05.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:05.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:05.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:11:05.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:11:05.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:05.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:11:05.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.048 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:04 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-29T05:11:05.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: pgmap v683: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 6.9 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-03-29T05:11:05.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:05.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:05.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:11:05.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:11:05.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:05.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:11:05.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:04 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-29T05:11:05.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: pgmap v684: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.7 MiB/s rd, 4 op/s 2024-03-29T05:11:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: Upgrade: Setting filesystem cephfs Joinable 2024-03-29T05:11:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-03-29T05:11:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: Cluster is now healthy 2024-03-29T05:11:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-29T05:11:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi142.bmkjzf=up:active} 2 up:standby-replay 2024-03-29T05:11:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:05.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi142"}]: dispatch 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi142"}]': finished 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi152"}]: dispatch 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi152"}]': finished 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.799 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-29T05:11:05.800 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:05.801 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:11:05.802 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:06.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: pgmap v684: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.7 MiB/s rd, 4 op/s 2024-03-29T05:11:06.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: Upgrade: Setting filesystem cephfs Joinable 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: Cluster is now healthy 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: fsmap cephfs:2 {0=cephfs.smithi142.wgmndr=up:active,1=cephfs.smithi142.bmkjzf=up:active} 2 up:standby-replay 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi142"}]: dispatch 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi142"}]': finished 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi152"}]: dispatch 2024-03-29T05:11:06.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi152"}]': finished 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:06.130 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-29T05:11:06.131 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-29T05:11:06.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:06.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-29T05:11:06.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-29T05:11:06.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-29T05:11:06.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-29T05:11:06.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-29T05:11:06.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-29T05:11:06.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-29T05:11:06.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-29T05:11:06.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:06.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:06.132 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:11:06.133 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:07.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:06 smithi152 ceph-mon[88186]: Upgrade: Setting container_image for all ceph-exporter 2024-03-29T05:11:07.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:06 smithi152 ceph-mon[88186]: Upgrade: Setting container_image for all iscsi 2024-03-29T05:11:07.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:06 smithi152 ceph-mon[88186]: Upgrade: Setting container_image for all nfs 2024-03-29T05:11:07.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:06 smithi152 ceph-mon[88186]: Upgrade: Setting container_image for all nvmeof 2024-03-29T05:11:07.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:06 smithi152 ceph-mon[88186]: Upgrade: Finalizing container_image settings 2024-03-29T05:11:07.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:06 smithi152 ceph-mon[88186]: Upgrade: Complete! 2024-03-29T05:11:07.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:06 smithi142 ceph-mon[112475]: Upgrade: Setting container_image for all ceph-exporter 2024-03-29T05:11:07.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:06 smithi142 ceph-mon[112475]: Upgrade: Setting container_image for all iscsi 2024-03-29T05:11:07.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:06 smithi142 ceph-mon[112475]: Upgrade: Setting container_image for all nfs 2024-03-29T05:11:07.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:06 smithi142 ceph-mon[112475]: Upgrade: Setting container_image for all nvmeof 2024-03-29T05:11:07.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:06 smithi142 ceph-mon[112475]: Upgrade: Finalizing container_image settings 2024-03-29T05:11:07.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:06 smithi142 ceph-mon[112475]: Upgrade: Complete! 2024-03-29T05:11:08.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:07 smithi152 ceph-mon[88186]: pgmap v685: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.3 KiB/s rd, 3 op/s 2024-03-29T05:11:08.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:07 smithi142 ceph-mon[112475]: pgmap v685: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.3 KiB/s rd, 3 op/s 2024-03-29T05:11:10.050 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:09 smithi152 ceph-mon[88186]: pgmap v686: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-29T05:11:10.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:09 smithi142 ceph-mon[112475]: pgmap v686: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-29T05:11:11.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:11.128 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:10 smithi142 ceph-mon[112475]: pgmap v687: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-29T05:11:11.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:11.297 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:10 smithi152 ceph-mon[88186]: pgmap v687: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-29T05:11:13.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:13 smithi142 ceph-mon[112475]: pgmap v688: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-03-29T05:11:13.797 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:13 smithi152 ceph-mon[88186]: pgmap v688: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-03-29T05:11:15.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:15 smithi142 ceph-mon[112475]: pgmap v689: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-29T05:11:15.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:15.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:11:16.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:15 smithi152 ceph-mon[88186]: pgmap v689: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-03-29T05:11:16.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:11:16.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:11:17.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:17 smithi142 ceph-mon[112475]: pgmap v690: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:11:18.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:17 smithi152 ceph-mon[88186]: pgmap v690: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:11:19.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:19 smithi142 ceph-mon[112475]: pgmap v691: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:11:20.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:19 smithi152 ceph-mon[88186]: pgmap v691: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:11:21.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:21 smithi142 ceph-mon[112475]: pgmap v692: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:11:22.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:21 smithi152 ceph-mon[88186]: pgmap v692: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:11:23.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:23 smithi142 ceph-mon[112475]: pgmap v693: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:11:24.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:23 smithi152 ceph-mon[88186]: pgmap v693: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:11:24.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.645+0000 7fcf89659700 1 -- 172.21.15.142:0/3411628139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf84100df0 msgr2=0x7fcf841011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:24.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.645+0000 7fcf89659700 1 --2- 172.21.15.142:0/3411628139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf84100df0 0x7fcf841011d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009910 tx=0x7fcf74009c20 comp rx=0 tx=0).stop 2024-03-29T05:11:24.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.645+0000 7fcf89659700 1 -- 172.21.15.142:0/3411628139 shutdown_connections 2024-03-29T05:11:24.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.645+0000 7fcf89659700 1 --2- 172.21.15.142:0/3411628139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf841017a0 0x7fcf841057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:24.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.645+0000 7fcf89659700 1 --2- 172.21.15.142:0/3411628139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf84100df0 0x7fcf841011d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:24.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.645+0000 7fcf89659700 1 -- 172.21.15.142:0/3411628139 >> 172.21.15.142:0/3411628139 conn(0x7fcf840fc6a0 msgr2=0x7fcf840feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:11:24.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.646+0000 7fcf89659700 1 -- 172.21.15.142:0/3411628139 shutdown_connections 2024-03-29T05:11:24.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.646+0000 7fcf89659700 1 -- 172.21.15.142:0/3411628139 wait complete. 2024-03-29T05:11:24.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.646+0000 7fcf89659700 1 Processor -- start 2024-03-29T05:11:24.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.646+0000 7fcf89659700 1 -- start start 2024-03-29T05:11:24.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.646+0000 7fcf89659700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf84100df0 0x7fcf8419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:24.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.646+0000 7fcf89659700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf841017a0 0x7fcf8419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:24.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.646+0000 7fcf89659700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8419d100 con 0x7fcf84100df0 2024-03-29T05:11:24.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.646+0000 7fcf89659700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf841965b0 con 0x7fcf841017a0 2024-03-29T05:11:24.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.646+0000 7fcf82ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf84100df0 0x7fcf8419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:24.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.647+0000 7fcf82ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf84100df0 0x7fcf8419c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40280/0 (socket says 172.21.15.142:40280) 2024-03-29T05:11:24.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.647+0000 7fcf82ffd700 1 -- 172.21.15.142:0/4057668959 learned_addr learned my addr 172.21.15.142:0/4057668959 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:11:24.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.647+0000 7fcf7bfff700 1 --2- 172.21.15.142:0/4057668959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf841017a0 0x7fcf8419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:24.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.647+0000 7fcf82ffd700 1 -- 172.21.15.142:0/4057668959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf841017a0 msgr2=0x7fcf8419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:24.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.647+0000 7fcf82ffd700 1 --2- 172.21.15.142:0/4057668959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf841017a0 0x7fcf8419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:24.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.647+0000 7fcf82ffd700 1 -- 172.21.15.142:0/4057668959 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf740095f0 con 0x7fcf84100df0 2024-03-29T05:11:24.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.647+0000 7fcf82ffd700 1 --2- 172.21.15.142:0/4057668959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf84100df0 0x7fcf8419c530 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcf74006010 tx=0x7fcf7400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:11:24.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.647+0000 7fcf80ff9700 1 -- 172.21.15.142:0/4057668959 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf7401d070 con 0x7fcf84100df0 2024-03-29T05:11:24.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.647+0000 7fcf80ff9700 1 -- 172.21.15.142:0/4057668959 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf74005dc0 con 0x7fcf84100df0 2024-03-29T05:11:24.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.647+0000 7fcf89659700 1 -- 172.21.15.142:0/4057668959 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf84196830 con 0x7fcf84100df0 2024-03-29T05:11:24.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.647+0000 7fcf80ff9700 1 -- 172.21.15.142:0/4057668959 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf74017400 con 0x7fcf84100df0 2024-03-29T05:11:24.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.647+0000 7fcf89659700 1 -- 172.21.15.142:0/4057668959 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf84196d20 con 0x7fcf84100df0 2024-03-29T05:11:24.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.648+0000 7fcf89659700 1 -- 172.21.15.142:0/4057668959 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf84109190 con 0x7fcf84100df0 2024-03-29T05:11:24.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.649+0000 7fcf80ff9700 1 -- 172.21.15.142:0/4057668959 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fcf74003a60 con 0x7fcf84100df0 2024-03-29T05:11:24.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.649+0000 7fcf80ff9700 1 --2- 172.21.15.142:0/4057668959 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf7006dd60 0x7fcf70070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:24.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.650+0000 7fcf80ff9700 1 -- 172.21.15.142:0/4057668959 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fcf7408f0f0 con 0x7fcf84100df0 2024-03-29T05:11:24.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.650+0000 7fcf7bfff700 1 --2- 172.21.15.142:0/4057668959 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf7006dd60 0x7fcf70070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:24.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.650+0000 7fcf7bfff700 1 --2- 172.21.15.142:0/4057668959 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf7006dd60 0x7fcf70070220 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcf840fce50 tx=0x7fcf6c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:11:24.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.655+0000 7fcf80ff9700 1 -- 172.21.15.142:0/4057668959 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcf7405c040 con 0x7fcf84100df0 2024-03-29T05:11:24.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.874+0000 7fcf89659700 1 -- 172.21.15.142:0/4057668959 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf84197000 con 0x7fcf7006dd60 2024-03-29T05:11:24.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.876+0000 7fcf80ff9700 1 -- 172.21.15.142:0/4057668959 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fcf84197000 con 0x7fcf7006dd60 2024-03-29T05:11:24.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.878+0000 7fcf89659700 1 -- 172.21.15.142:0/4057668959 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf7006dd60 msgr2=0x7fcf70070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:24.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.878+0000 7fcf89659700 1 --2- 172.21.15.142:0/4057668959 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf7006dd60 0x7fcf70070220 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcf840fce50 tx=0x7fcf6c00b410 comp rx=0 tx=0).stop 2024-03-29T05:11:24.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.878+0000 7fcf89659700 1 -- 172.21.15.142:0/4057668959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf84100df0 msgr2=0x7fcf8419c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:24.884 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.878+0000 7fcf89659700 1 --2- 172.21.15.142:0/4057668959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf84100df0 0x7fcf8419c530 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcf74006010 tx=0x7fcf7400bd70 comp rx=0 tx=0).stop 2024-03-29T05:11:24.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.878+0000 7fcf89659700 1 -- 172.21.15.142:0/4057668959 shutdown_connections 2024-03-29T05:11:24.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.878+0000 7fcf89659700 1 --2- 172.21.15.142:0/4057668959 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcf7006dd60 0x7fcf70070220 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:24.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.878+0000 7fcf89659700 1 --2- 172.21.15.142:0/4057668959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf84100df0 0x7fcf8419c530 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:24.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.878+0000 7fcf89659700 1 --2- 172.21.15.142:0/4057668959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf841017a0 0x7fcf8419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:24.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.878+0000 7fcf89659700 1 -- 172.21.15.142:0/4057668959 >> 172.21.15.142:0/4057668959 conn(0x7fcf840fc6a0 msgr2=0x7fcf840fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:11:24.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.878+0000 7fcf89659700 1 -- 172.21.15.142:0/4057668959 shutdown_connections 2024-03-29T05:11:24.885 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:24.878+0000 7fcf89659700 1 -- 172.21.15.142:0/4057668959 wait complete. 2024-03-29T05:11:25.529 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph orch ps' 2024-03-29T05:11:25.759 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T05:11:25.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:25 smithi142 ceph-mon[112475]: pgmap v694: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:11:26.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:25 smithi152 ceph-mon[88186]: pgmap v694: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:11:26.849 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:26 smithi142 ceph-mon[112475]: from='client.35274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:11:27.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:26 smithi152 ceph-mon[88186]: from='client.35274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:11:27.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.360+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2548091304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a4101770 msgr2=0x7fb7a41057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:27.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.360+0000 7fb7abbbf700 1 --2- 172.21.15.142:0/2548091304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a4101770 0x7fb7a41057c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb794009910 tx=0x7fb794009c20 comp rx=0 tx=0).stop 2024-03-29T05:11:27.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.360+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2548091304 shutdown_connections 2024-03-29T05:11:27.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.360+0000 7fb7abbbf700 1 --2- 172.21.15.142:0/2548091304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a4101770 0x7fb7a41057c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:27.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.360+0000 7fb7abbbf700 1 --2- 172.21.15.142:0/2548091304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a4100dc0 0x7fb7a41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:27.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.360+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2548091304 >> 172.21.15.142:0/2548091304 conn(0x7fb7a40fc650 msgr2=0x7fb7a40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:11:27.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.360+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2548091304 shutdown_connections 2024-03-29T05:11:27.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.360+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2548091304 wait complete. 2024-03-29T05:11:27.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.360+0000 7fb7abbbf700 1 Processor -- start 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.360+0000 7fb7abbbf700 1 -- start start 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.361+0000 7fb7abbbf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a4100dc0 0x7fb7a4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.361+0000 7fb7abbbf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a4101770 0x7fb7a4198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.361+0000 7fb7abbbf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4199540 con 0x7fb7a4100dc0 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.361+0000 7fb7a995b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a4100dc0 0x7fb7a4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.361+0000 7fb7a995b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a4100dc0 0x7fb7a4198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40296/0 (socket says 172.21.15.142:40296) 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.361+0000 7fb7a995b700 1 -- 172.21.15.142:0/2641558892 learned_addr learned my addr 172.21.15.142:0/2641558892 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.361+0000 7fb7abbbf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a419d350 con 0x7fb7a4101770 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.361+0000 7fb7a915a700 1 --2- 172.21.15.142:0/2641558892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a4101770 0x7fb7a4198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.361+0000 7fb7a995b700 1 -- 172.21.15.142:0/2641558892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a4101770 msgr2=0x7fb7a4198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.361+0000 7fb7a995b700 1 --2- 172.21.15.142:0/2641558892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a4101770 0x7fb7a4198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.361+0000 7fb7a995b700 1 -- 172.21.15.142:0/2641558892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7940095f0 con 0x7fb7a4100dc0 2024-03-29T05:11:27.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.361+0000 7fb7a995b700 1 --2- 172.21.15.142:0/2641558892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a4100dc0 0x7fb7a4198970 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000d900 tx=0x7fb7a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:11:27.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.362+0000 7fb79affd700 1 -- 172.21.15.142:0/2641558892 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7a00041f0 con 0x7fb7a4100dc0 2024-03-29T05:11:27.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.362+0000 7fb79affd700 1 -- 172.21.15.142:0/2641558892 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7a0004cf0 con 0x7fb7a4100dc0 2024-03-29T05:11:27.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.362+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2641558892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a419d630 con 0x7fb7a4100dc0 2024-03-29T05:11:27.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.362+0000 7fb79affd700 1 -- 172.21.15.142:0/2641558892 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7a000b750 con 0x7fb7a4100dc0 2024-03-29T05:11:27.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.362+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2641558892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a419db50 con 0x7fb7a4100dc0 2024-03-29T05:11:27.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.363+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2641558892 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a4104f40 con 0x7fb7a4100dc0 2024-03-29T05:11:27.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.363+0000 7fb79affd700 1 -- 172.21.15.142:0/2641558892 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fb7a000bb80 con 0x7fb7a4100dc0 2024-03-29T05:11:27.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.363+0000 7fb79affd700 1 --2- 172.21.15.142:0/2641558892 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb79006dd60 0x7fb790070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:27.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.364+0000 7fb79affd700 1 -- 172.21.15.142:0/2641558892 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb7a008dd30 con 0x7fb7a4100dc0 2024-03-29T05:11:27.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.364+0000 7fb7a915a700 1 --2- 172.21.15.142:0/2641558892 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb79006dd60 0x7fb790070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:27.371 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.364+0000 7fb7a915a700 1 --2- 172.21.15.142:0/2641558892 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb79006dd60 0x7fb790070220 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb7a41998e0 tx=0x7fb79401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:11:27.375 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.368+0000 7fb79affd700 1 -- 172.21.15.142:0/2641558892 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb7a005ad00 con 0x7fb7a4100dc0 2024-03-29T05:11:27.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.642+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2641558892 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7a419ddc0 con 0x7fb79006dd60 2024-03-29T05:11:27.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.653+0000 7fb79affd700 1 -- 172.21.15.142:0/2641558892 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb7a419ddc0 con 0x7fb79006dd60 2024-03-29T05:11:27.660 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:11:27.660 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (23m) 30s ago 34m 24.2M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:11:27.660 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (44s) 30s ago 35m 7373k - 19.0.0-1959-g8da23624 54112f57da44 a9260541b7e8 2024-03-29T05:11:27.660 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (37s) 33s ago 34m 7109k - 19.0.0-1959-g8da23624 54112f57da44 47e6e41b3305 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (20m) 30s ago 35m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (20m) 33s ago 33m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (22m) 30s ago 34m 82.0M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (16m) 30s ago 29m 69.0M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (16m) 30s ago 29m 67.8M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (15m) 33s ago 29m 135M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (16m) 33s ago 29m 172M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (24m) 30s ago 36m 515M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (24m) 33s ago 33m 458M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (21m) 30s ago 36m 74.0M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (21m) 33s ago 33m 56.4M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (23m) 30s ago 34m 16.4M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (23m) 33s ago 33m 15.5M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:11:27.661 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (20m) 30s ago 32m 116M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:11:27.662 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (19m) 30s ago 32m 136M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:11:27.662 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (18m) 30s ago 32m 99.6M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:11:27.662 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (18m) 33s ago 31m 113M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:11:27.662 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (17m) 33s ago 31m 99.6M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:11:27.662 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (17m) 33s ago 30m 104M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:11:27.662 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (23m) 30s ago 34m 71.4M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:11:27.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.656+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2641558892 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb79006dd60 msgr2=0x7fb790070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:27.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.656+0000 7fb7abbbf700 1 --2- 172.21.15.142:0/2641558892 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb79006dd60 0x7fb790070220 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb7a41998e0 tx=0x7fb79401a040 comp rx=0 tx=0).stop 2024-03-29T05:11:27.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.656+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2641558892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a4100dc0 msgr2=0x7fb7a4198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:27.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.656+0000 7fb7abbbf700 1 --2- 172.21.15.142:0/2641558892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a4100dc0 0x7fb7a4198970 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000d900 tx=0x7fb7a000dcc0 comp rx=0 tx=0).stop 2024-03-29T05:11:27.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.656+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2641558892 shutdown_connections 2024-03-29T05:11:27.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.656+0000 7fb7abbbf700 1 --2- 172.21.15.142:0/2641558892 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb79006dd60 0x7fb790070220 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:27.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.656+0000 7fb7abbbf700 1 --2- 172.21.15.142:0/2641558892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a4100dc0 0x7fb7a4198970 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:27.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.656+0000 7fb7abbbf700 1 --2- 172.21.15.142:0/2641558892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a4101770 0x7fb7a4198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:27.664 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.656+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2641558892 >> 172.21.15.142:0/2641558892 conn(0x7fb7a40fc650 msgr2=0x7fb7a40fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:11:27.664 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.656+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2641558892 shutdown_connections 2024-03-29T05:11:27.664 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:27.656+0000 7fb7abbbf700 1 -- 172.21.15.142:0/2641558892 wait complete. 2024-03-29T05:11:28.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:27 smithi152 ceph-mon[88186]: pgmap v695: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:11:28.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:27 smithi142 ceph-mon[112475]: pgmap v695: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:11:28.419 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph orch upgrade status' 2024-03-29T05:11:28.667 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T05:11:28.723 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:28 smithi142 ceph-mon[112475]: from='client.35278 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:11:28.948 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:28 smithi152 ceph-mon[88186]: from='client.35278 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:11:29.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:29 smithi142 ceph-mon[112475]: pgmap v696: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:11:29.959 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:11:30.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:29 smithi152 ceph-mon[88186]: pgmap v696: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:11:30.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:11:30.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.274+0000 7f47b9546700 1 -- 172.21.15.142:0/2244031860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4101750 msgr2=0x7f47b41057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.274+0000 7f47b9546700 1 --2- 172.21.15.142:0/2244031860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4101750 0x7f47b41057a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009910 tx=0x7f47a8009c20 comp rx=0 tx=0).stop 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.274+0000 7f47b9546700 1 -- 172.21.15.142:0/2244031860 shutdown_connections 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.274+0000 7f47b9546700 1 --2- 172.21.15.142:0/2244031860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4101750 0x7f47b41057a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.274+0000 7f47b9546700 1 --2- 172.21.15.142:0/2244031860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47b4100da0 0x7f47b4101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.274+0000 7f47b9546700 1 -- 172.21.15.142:0/2244031860 >> 172.21.15.142:0/2244031860 conn(0x7f47b40fc630 msgr2=0x7f47b40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.274+0000 7f47b9546700 1 -- 172.21.15.142:0/2244031860 shutdown_connections 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.274+0000 7f47b9546700 1 -- 172.21.15.142:0/2244031860 wait complete. 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.275+0000 7f47b9546700 1 Processor -- start 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.275+0000 7f47b9546700 1 -- start start 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.275+0000 7f47b9546700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4100da0 0x7f47b419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.275+0000 7f47b9546700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47b4101750 0x7f47b419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.275+0000 7f47b9546700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b419d270 con 0x7f47b4100da0 2024-03-29T05:11:30.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.275+0000 7f47b9546700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b4196720 con 0x7f47b4101750 2024-03-29T05:11:30.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.275+0000 7f47b2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4100da0 0x7f47b419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:30.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.276+0000 7f47b2ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4100da0 0x7f47b419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44598/0 (socket says 172.21.15.142:44598) 2024-03-29T05:11:30.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.276+0000 7f47b2ffd700 1 -- 172.21.15.142:0/3265223682 learned_addr learned my addr 172.21.15.142:0/3265223682 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:11:30.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.276+0000 7f47b27fc700 1 --2- 172.21.15.142:0/3265223682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47b4101750 0x7f47b419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:30.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.276+0000 7f47b2ffd700 1 -- 172.21.15.142:0/3265223682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47b4101750 msgr2=0x7f47b419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:30.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.276+0000 7f47b2ffd700 1 --2- 172.21.15.142:0/3265223682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47b4101750 0x7f47b419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:30.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.276+0000 7f47b2ffd700 1 -- 172.21.15.142:0/3265223682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80095f0 con 0x7f47b4100da0 2024-03-29T05:11:30.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.276+0000 7f47b2ffd700 1 --2- 172.21.15.142:0/3265223682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4100da0 0x7f47b419c6a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f47a400b4b0 tx=0x7f47a400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:11:30.285 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.277+0000 7f479bfff700 1 -- 172.21.15.142:0/3265223682 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a4011800 con 0x7f47b4100da0 2024-03-29T05:11:30.285 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.277+0000 7f47b9546700 1 -- 172.21.15.142:0/3265223682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b4196a00 con 0x7f47b4100da0 2024-03-29T05:11:30.285 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.277+0000 7f479bfff700 1 -- 172.21.15.142:0/3265223682 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47a4011e20 con 0x7f47b4100da0 2024-03-29T05:11:30.285 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.277+0000 7f479bfff700 1 -- 172.21.15.142:0/3265223682 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a400f2e0 con 0x7f47b4100da0 2024-03-29T05:11:30.285 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.277+0000 7f47b9546700 1 -- 172.21.15.142:0/3265223682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b4196f50 con 0x7f47b4100da0 2024-03-29T05:11:30.285 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.278+0000 7f47b9546700 1 -- 172.21.15.142:0/3265223682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b41090d0 con 0x7f47b4100da0 2024-03-29T05:11:30.285 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.278+0000 7f479bfff700 1 -- 172.21.15.142:0/3265223682 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f47a400f440 con 0x7f47b4100da0 2024-03-29T05:11:30.286 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.279+0000 7f479bfff700 1 --2- 172.21.15.142:0/3265223682 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f479c06dd60 0x7f479c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:30.286 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.279+0000 7f479bfff700 1 -- 172.21.15.142:0/3265223682 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f47a408d6e0 con 0x7f47b4100da0 2024-03-29T05:11:30.286 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.279+0000 7f47b27fc700 1 --2- 172.21.15.142:0/3265223682 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f479c06dd60 0x7f479c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:30.286 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.279+0000 7f47b27fc700 1 --2- 172.21.15.142:0/3265223682 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f479c06dd60 0x7f479c070220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f47b4197b90 tx=0x7f47a801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:11:30.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.283+0000 7f479bfff700 1 -- 172.21.15.142:0/3265223682 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f47a405a790 con 0x7f47b4100da0 2024-03-29T05:11:30.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.652+0000 7f47b9546700 1 -- 172.21.15.142:0/3265223682 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47b4197270 con 0x7f479c06dd60 2024-03-29T05:11:30.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.654+0000 7f479bfff700 1 -- 172.21.15.142:0/3265223682 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f47b4197270 con 0x7f479c06dd60 2024-03-29T05:11:30.660 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:11:30.660 INFO:teuthology.orchestra.run.smithi142.stdout: "target_image": null, 2024-03-29T05:11:30.661 INFO:teuthology.orchestra.run.smithi142.stdout: "in_progress": false, 2024-03-29T05:11:30.661 INFO:teuthology.orchestra.run.smithi142.stdout: "which": "", 2024-03-29T05:11:30.661 INFO:teuthology.orchestra.run.smithi142.stdout: "services_complete": [], 2024-03-29T05:11:30.661 INFO:teuthology.orchestra.run.smithi142.stdout: "progress": null, 2024-03-29T05:11:30.661 INFO:teuthology.orchestra.run.smithi142.stdout: "message": "", 2024-03-29T05:11:30.661 INFO:teuthology.orchestra.run.smithi142.stdout: "is_paused": false 2024-03-29T05:11:30.661 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:11:30.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.656+0000 7f47b9546700 1 -- 172.21.15.142:0/3265223682 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f479c06dd60 msgr2=0x7f479c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:30.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.656+0000 7f47b9546700 1 --2- 172.21.15.142:0/3265223682 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f479c06dd60 0x7f479c070220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f47b4197b90 tx=0x7f47a801a040 comp rx=0 tx=0).stop 2024-03-29T05:11:30.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.657+0000 7f47b9546700 1 -- 172.21.15.142:0/3265223682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4100da0 msgr2=0x7f47b419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:30.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.657+0000 7f47b9546700 1 --2- 172.21.15.142:0/3265223682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4100da0 0x7f47b419c6a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f47a400b4b0 tx=0x7f47a400b7c0 comp rx=0 tx=0).stop 2024-03-29T05:11:30.664 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.657+0000 7f47b9546700 1 -- 172.21.15.142:0/3265223682 shutdown_connections 2024-03-29T05:11:30.664 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.657+0000 7f47b9546700 1 --2- 172.21.15.142:0/3265223682 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f479c06dd60 0x7f479c070220 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:30.664 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.657+0000 7f47b9546700 1 --2- 172.21.15.142:0/3265223682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b4100da0 0x7f47b419c6a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:30.664 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.657+0000 7f47b9546700 1 --2- 172.21.15.142:0/3265223682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47b4101750 0x7f47b419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:30.664 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.657+0000 7f47b9546700 1 -- 172.21.15.142:0/3265223682 >> 172.21.15.142:0/3265223682 conn(0x7f47b40fc630 msgr2=0x7f47b40fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:11:30.664 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.657+0000 7f47b9546700 1 -- 172.21.15.142:0/3265223682 shutdown_connections 2024-03-29T05:11:30.664 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:30.657+0000 7f47b9546700 1 -- 172.21.15.142:0/3265223682 wait complete. 2024-03-29T05:11:31.286 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph health detail' 2024-03-29T05:11:31.531 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T05:11:31.890 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:31 smithi142 ceph-mon[112475]: pgmap v697: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:11:31.891 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:31 smithi142 ceph-mon[112475]: from='client.35282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:11:32.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:31 smithi152 ceph-mon[88186]: pgmap v697: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:11:32.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:31 smithi152 ceph-mon[88186]: from='client.35282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:11:33.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.180+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/3805526208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80ffbd0 msgr2=0x7fd6b8100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:33.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.180+0000 7fd6bcdf4700 1 --2- 172.21.15.142:0/3805526208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80ffbd0 0x7fd6b8100050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac009910 tx=0x7fd6ac009c20 comp rx=0 tx=0).stop 2024-03-29T05:11:33.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.180+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/3805526208 shutdown_connections 2024-03-29T05:11:33.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.180+0000 7fd6bcdf4700 1 --2- 172.21.15.142:0/3805526208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80ffbd0 0x7fd6b8100050 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:33.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.180+0000 7fd6bcdf4700 1 --2- 172.21.15.142:0/3805526208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6b8105a40 0x7fd6b8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:33.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.180+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/3805526208 >> 172.21.15.142:0/3805526208 conn(0x7fd6b80755a0 msgr2=0x7fd6b80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:11:33.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.180+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/3805526208 shutdown_connections 2024-03-29T05:11:33.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.180+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/3805526208 wait complete. 2024-03-29T05:11:33.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.181+0000 7fd6bcdf4700 1 Processor -- start 2024-03-29T05:11:33.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.181+0000 7fd6bcdf4700 1 -- start start 2024-03-29T05:11:33.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6bcdf4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80ffbd0 0x7fd6b8195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:33.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6bcdf4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6b8105a40 0x7fd6b81958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:33.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6bcdf4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b8195e00 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6bcdf4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b8195f70 con 0x7fd6b8105a40 2024-03-29T05:11:33.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6b659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80ffbd0 0x7fd6b8195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:33.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6b659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80ffbd0 0x7fd6b8195380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44624/0 (socket says 172.21.15.142:44624) 2024-03-29T05:11:33.190 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6b659c700 1 -- 172.21.15.142:0/2685787078 learned_addr learned my addr 172.21.15.142:0/2685787078 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:11:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6b5d9b700 1 --2- 172.21.15.142:0/2685787078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6b8105a40 0x7fd6b81958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6b659c700 1 -- 172.21.15.142:0/2685787078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6b8105a40 msgr2=0x7fd6b81958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6b659c700 1 --2- 172.21.15.142:0/2685787078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6b8105a40 0x7fd6b81958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6b659c700 1 -- 172.21.15.142:0/2685787078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac0095f0 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6b5d9b700 1 --2- 172.21.15.142:0/2685787078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6b8105a40 0x7fd6b81958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:11:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6b659c700 1 --2- 172.21.15.142:0/2685787078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80ffbd0 0x7fd6b8195380 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800d8d0 tx=0x7fd6a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:11:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6a77fe700 1 -- 172.21.15.142:0/2685787078 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a8009940 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.182+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/2685787078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b81a29f0 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.183+0000 7fd6a77fe700 1 -- 172.21.15.142:0/2685787078 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6a8010460 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.183+0000 7fd6a77fe700 1 -- 172.21.15.142:0/2685787078 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a800f5f0 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.183+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/2685787078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b81a2d50 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.183+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/2685787078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b8107fd0 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.184+0000 7fd6a77fe700 1 -- 172.21.15.142:0/2685787078 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fd6a80105d0 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.185+0000 7fd6a77fe700 1 --2- 172.21.15.142:0/2685787078 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd6a006dd60 0x7fd6a0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:33.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.185+0000 7fd6a77fe700 1 -- 172.21.15.142:0/2685787078 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd6a808dc10 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.192 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.185+0000 7fd6b5d9b700 1 --2- 172.21.15.142:0/2685787078 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd6a006dd60 0x7fd6a0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:33.193 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.185+0000 7fd6b5d9b700 1 --2- 172.21.15.142:0/2685787078 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd6a006dd60 0x7fd6a0070220 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00b5c0 tx=0x7fd6ac0053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:11:33.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.190+0000 7fd6a77fe700 1 -- 172.21.15.142:0/2685787078 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd6a8057030 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.486+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/2685787078 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd6b804ea90 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.487+0000 7fd6a77fe700 1 -- 172.21.15.142:0/2685787078 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fd6a805a6f0 con 0x7fd6b80ffbd0 2024-03-29T05:11:33.494 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_OK 2024-03-29T05:11:33.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.489+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/2685787078 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd6a006dd60 msgr2=0x7fd6a0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:33.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.489+0000 7fd6bcdf4700 1 --2- 172.21.15.142:0/2685787078 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd6a006dd60 0x7fd6a0070220 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00b5c0 tx=0x7fd6ac0053a0 comp rx=0 tx=0).stop 2024-03-29T05:11:33.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.489+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/2685787078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80ffbd0 msgr2=0x7fd6b8195380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:33.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.489+0000 7fd6bcdf4700 1 --2- 172.21.15.142:0/2685787078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80ffbd0 0x7fd6b8195380 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800d8d0 tx=0x7fd6a800dc90 comp rx=0 tx=0).stop 2024-03-29T05:11:33.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.489+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/2685787078 shutdown_connections 2024-03-29T05:11:33.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.489+0000 7fd6bcdf4700 1 --2- 172.21.15.142:0/2685787078 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fd6a006dd60 0x7fd6a0070220 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:33.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.489+0000 7fd6bcdf4700 1 --2- 172.21.15.142:0/2685787078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6b80ffbd0 0x7fd6b8195380 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:33.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.489+0000 7fd6bcdf4700 1 --2- 172.21.15.142:0/2685787078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6b8105a40 0x7fd6b81958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:33.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.489+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/2685787078 >> 172.21.15.142:0/2685787078 conn(0x7fd6b80755a0 msgr2=0x7fd6b80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:11:33.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.489+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/2685787078 shutdown_connections 2024-03-29T05:11:33.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:33.489+0000 7fd6bcdf4700 1 -- 172.21.15.142:0/2685787078 wait complete. 2024-03-29T05:11:33.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:33 smithi142 ceph-mon[112475]: pgmap v698: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:11:33.813 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:33 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2685787078' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:11:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:33 smithi152 ceph-mon[88186]: pgmap v698: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:11:34.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:33 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2685787078' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T05:11:34.134 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph versions' 2024-03-29T05:11:34.352 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T05:11:35.842 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:35 smithi142 ceph-mon[112475]: pgmap v699: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:11:36.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:35 smithi152 ceph-mon[88186]: pgmap v699: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:11:36.067 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.061+0000 7f9c63529700 1 -- 172.21.15.142:0/145811415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c0ffbd0 msgr2=0x7f9c5c100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:36.067 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.061+0000 7f9c63529700 1 --2- 172.21.15.142:0/145811415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c0ffbd0 0x7f9c5c100050 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009910 tx=0x7f9c4c009c20 comp rx=0 tx=0).stop 2024-03-29T05:11:36.068 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.061+0000 7f9c63529700 1 -- 172.21.15.142:0/145811415 shutdown_connections 2024-03-29T05:11:36.068 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.061+0000 7f9c63529700 1 --2- 172.21.15.142:0/145811415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c0ffbd0 0x7f9c5c100050 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:36.068 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.061+0000 7f9c63529700 1 --2- 172.21.15.142:0/145811415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c105a40 0x7f9c5c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:36.068 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.061+0000 7f9c63529700 1 -- 172.21.15.142:0/145811415 >> 172.21.15.142:0/145811415 conn(0x7f9c5c0755a0 msgr2=0x7f9c5c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:11:36.068 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.061+0000 7f9c63529700 1 -- 172.21.15.142:0/145811415 shutdown_connections 2024-03-29T05:11:36.068 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.061+0000 7f9c63529700 1 -- 172.21.15.142:0/145811415 wait complete. 2024-03-29T05:11:36.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c63529700 1 Processor -- start 2024-03-29T05:11:36.069 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c63529700 1 -- start start 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c63529700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c0ffbd0 0x7f9c5c195320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c63529700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c105a40 0x7f9c5c195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c63529700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c195f40 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c63529700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c196080 con 0x7f9c5c105a40 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c612c5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c0ffbd0 0x7f9c5c195320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c612c5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c0ffbd0 0x7f9c5c195320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44638/0 (socket says 172.21.15.142:44638) 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c612c5700 1 -- 172.21.15.142:0/1775400432 learned_addr learned my addr 172.21.15.142:0/1775400432 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c60ac4700 1 --2- 172.21.15.142:0/1775400432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c105a40 0x7f9c5c195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c612c5700 1 -- 172.21.15.142:0/1775400432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c105a40 msgr2=0x7f9c5c195860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c612c5700 1 --2- 172.21.15.142:0/1775400432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c105a40 0x7f9c5c195860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c612c5700 1 -- 172.21.15.142:0/1775400432 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0095f0 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c60ac4700 1 --2- 172.21.15.142:0/1775400432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c105a40 0x7f9c5c195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:11:36.070 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.062+0000 7f9c612c5700 1 --2- 172.21.15.142:0/1775400432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c0ffbd0 0x7f9c5c195320 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800d8d0 tx=0x7f9c5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:11:36.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.063+0000 7f9c527fc700 1 -- 172.21.15.142:0/1775400432 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c58009940 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.063+0000 7f9c527fc700 1 -- 172.21.15.142:0/1775400432 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c58010460 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.063+0000 7f9c527fc700 1 -- 172.21.15.142:0/1775400432 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c5800f5f0 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.063+0000 7f9c63529700 1 -- 172.21.15.142:0/1775400432 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c19a3d0 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.063+0000 7f9c63529700 1 -- 172.21.15.142:0/1775400432 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c19a8a0 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.064+0000 7f9c63529700 1 -- 172.21.15.142:0/1775400432 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c04ea90 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.071 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.064+0000 7f9c527fc700 1 -- 172.21.15.142:0/1775400432 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f9c580105d0 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.072 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.065+0000 7f9c527fc700 1 --2- 172.21.15.142:0/1775400432 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9c4806dd60 0x7f9c48070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:11:36.074 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.065+0000 7f9c60ac4700 1 --2- 172.21.15.142:0/1775400432 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9c4806dd60 0x7f9c48070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:11:36.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.065+0000 7f9c527fc700 1 -- 172.21.15.142:0/1775400432 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f9c5808dd30 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.075 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.065+0000 7f9c60ac4700 1 --2- 172.21.15.142:0/1775400432 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9c4806dd60 0x7f9c48070220 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00b5c0 tx=0x7f9c4c0053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:11:36.076 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.070+0000 7f9c527fc700 1 -- 172.21.15.142:0/1775400432 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9c580570f0 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.362+0000 7f9c63529700 1 -- 172.21.15.142:0/1775400432 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9c5c068a60 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.363+0000 7f9c527fc700 1 -- 172.21.15.142:0/1775400432 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9c5805a7b0 con 0x7f9c5c0ffbd0 2024-03-29T05:11:36.370 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:11:36.370 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:11:36.370 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:11:36.370 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:11:36.370 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:11:36.370 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:11:36.370 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:11:36.370 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:11:36.370 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:11:36.371 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:11:36.371 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:11:36.371 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:11:36.371 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:11:36.371 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:11:36.371 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:11:36.371 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:11:36.371 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:11:36.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.366+0000 7f9c63529700 1 -- 172.21.15.142:0/1775400432 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9c4806dd60 msgr2=0x7f9c48070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:36.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.366+0000 7f9c63529700 1 --2- 172.21.15.142:0/1775400432 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9c4806dd60 0x7f9c48070220 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00b5c0 tx=0x7f9c4c0053a0 comp rx=0 tx=0).stop 2024-03-29T05:11:36.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.366+0000 7f9c63529700 1 -- 172.21.15.142:0/1775400432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c0ffbd0 msgr2=0x7f9c5c195320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:11:36.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.366+0000 7f9c63529700 1 --2- 172.21.15.142:0/1775400432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c0ffbd0 0x7f9c5c195320 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800d8d0 tx=0x7f9c5800dc90 comp rx=0 tx=0).stop 2024-03-29T05:11:36.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.366+0000 7f9c63529700 1 -- 172.21.15.142:0/1775400432 shutdown_connections 2024-03-29T05:11:36.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.366+0000 7f9c63529700 1 --2- 172.21.15.142:0/1775400432 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9c4806dd60 0x7f9c48070220 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:36.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.366+0000 7f9c63529700 1 --2- 172.21.15.142:0/1775400432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c0ffbd0 0x7f9c5c195320 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:36.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.366+0000 7f9c63529700 1 --2- 172.21.15.142:0/1775400432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c105a40 0x7f9c5c195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:11:36.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.366+0000 7f9c63529700 1 -- 172.21.15.142:0/1775400432 >> 172.21.15.142:0/1775400432 conn(0x7f9c5c0755a0 msgr2=0x7f9c5c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:11:36.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.366+0000 7f9c63529700 1 -- 172.21.15.142:0/1775400432 shutdown_connections 2024-03-29T05:11:36.373 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:11:36.366+0000 7f9c63529700 1 -- 172.21.15.142:0/1775400432 wait complete. 2024-03-29T05:11:37.003 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:36 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1775400432' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:37.024 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-29T05:11:37.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:36 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1775400432' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:11:37.248 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T05:11:38.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:37 smithi152 ceph-mon[88186]: pgmap v700: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:11:38.069 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:37 smithi142 ceph-mon[112475]: pgmap v700: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:11:38.907 INFO:teuthology.orchestra.run.smithi142.stdout:wait for servicemap items w/ changing names to refresh 2024-03-29T05:11:39.645 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'sleep 60' 2024-03-29T05:11:39.844 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:39 smithi142 ceph-mon[112475]: pgmap v701: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:11:39.867 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T05:11:40.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:39 smithi152 ceph-mon[88186]: pgmap v701: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:11:41.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:41 smithi142 ceph-mon[112475]: pgmap v702: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:11:42.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:41 smithi152 ceph-mon[88186]: pgmap v702: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:11:44.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:43 smithi152 ceph-mon[88186]: pgmap v703: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:11:44.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:43 smithi142 ceph-mon[112475]: pgmap v703: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:11:44.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:11:45.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:11:46.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:45 smithi152 ceph-mon[88186]: pgmap v704: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:11:46.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:45 smithi142 ceph-mon[112475]: pgmap v704: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:11:48.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:47 smithi152 ceph-mon[88186]: pgmap v705: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:11:48.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:47 smithi142 ceph-mon[112475]: pgmap v705: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:11:50.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:49 smithi152 ceph-mon[88186]: pgmap v706: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:11:50.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:49 smithi142 ceph-mon[112475]: pgmap v706: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:11:51.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:51 smithi142 ceph-mon[112475]: pgmap v707: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:11:52.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:51 smithi152 ceph-mon[88186]: pgmap v707: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:11:54.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:53 smithi152 ceph-mon[88186]: pgmap v708: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:11:54.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:53 smithi142 ceph-mon[112475]: pgmap v708: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:11:56.009 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:55 smithi152 ceph-mon[88186]: pgmap v709: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:11:56.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:55 smithi142 ceph-mon[112475]: pgmap v709: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:11:58.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:57 smithi152 ceph-mon[88186]: pgmap v710: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:11:58.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:57 smithi142 ceph-mon[112475]: pgmap v710: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:11:59.947 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:59 smithi142 ceph-mon[112475]: pgmap v711: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:11:59.947 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:11:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:12:00.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:59 smithi152 ceph-mon[88186]: pgmap v711: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:12:00.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:11:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:12:02.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:01 smithi152 ceph-mon[88186]: pgmap v712: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:12:02.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:01 smithi142 ceph-mon[112475]: pgmap v712: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:12:04.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:03 smithi152 ceph-mon[88186]: pgmap v713: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:12:04.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:03 smithi142 ceph-mon[112475]: pgmap v713: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:12:06.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:05 smithi152 ceph-mon[88186]: pgmap v714: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:12:06.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:05 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:12:06.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:05 smithi142 ceph-mon[112475]: pgmap v714: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:12:06.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:05 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:12:07.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:06 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:12:07.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:06 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:12:07.047 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:06 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:12:07.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:06 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:12:07.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:06 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:12:07.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:06 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:12:08.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:07 smithi152 ceph-mon[88186]: pgmap v715: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:12:08.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:07 smithi142 ceph-mon[112475]: pgmap v715: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:12:10.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:09 smithi152 ceph-mon[88186]: pgmap v716: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:12:10.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:09 smithi142 ceph-mon[112475]: pgmap v716: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:12:11.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:11 smithi142 ceph-mon[112475]: pgmap v717: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:12:12.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:11 smithi152 ceph-mon[88186]: pgmap v717: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:12:14.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:13 smithi152 ceph-mon[88186]: pgmap v718: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:12:14.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:13 smithi142 ceph-mon[112475]: pgmap v718: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:12:15.003 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:12:15.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:12:16.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:15 smithi152 ceph-mon[88186]: pgmap v719: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:12:16.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:15 smithi142 ceph-mon[112475]: pgmap v719: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:12:18.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:17 smithi152 ceph-mon[88186]: pgmap v720: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:12:18.129 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:17 smithi142 ceph-mon[112475]: pgmap v720: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:12:20.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:19 smithi152 ceph-mon[88186]: pgmap v721: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:12:20.065 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:19 smithi142 ceph-mon[112475]: pgmap v721: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:12:21.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:21 smithi142 ceph-mon[112475]: pgmap v722: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:12:22.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:21 smithi152 ceph-mon[88186]: pgmap v722: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:12:24.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:23 smithi152 ceph-mon[88186]: pgmap v723: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:12:24.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:23 smithi142 ceph-mon[112475]: pgmap v723: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:12:26.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:25 smithi152 ceph-mon[88186]: pgmap v724: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:12:26.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:25 smithi142 ceph-mon[112475]: pgmap v724: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:12:28.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:27 smithi152 ceph-mon[88186]: pgmap v725: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:12:28.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:27 smithi142 ceph-mon[112475]: pgmap v725: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:12:30.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:29 smithi152 ceph-mon[88186]: pgmap v726: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:12:30.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:12:30.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:29 smithi142 ceph-mon[112475]: pgmap v726: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:12:30.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:12:31.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:31 smithi142 ceph-mon[112475]: pgmap v727: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:12:32.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:31 smithi152 ceph-mon[88186]: pgmap v727: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:12:34.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:33 smithi152 ceph-mon[88186]: pgmap v728: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:12:34.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:33 smithi142 ceph-mon[112475]: pgmap v728: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:12:36.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:35 smithi152 ceph-mon[88186]: pgmap v729: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:12:36.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:35 smithi142 ceph-mon[112475]: pgmap v729: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:12:38.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:37 smithi152 ceph-mon[88186]: pgmap v730: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:12:38.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:37 smithi142 ceph-mon[112475]: pgmap v730: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:12:40.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:39 smithi152 ceph-mon[88186]: pgmap v731: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:12:40.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:39 smithi142 ceph-mon[112475]: pgmap v731: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:12:41.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:41 smithi152 ceph-mon[88186]: pgmap v732: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:12:42.039 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:41 smithi142 ceph-mon[112475]: pgmap v732: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:12:42.060 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph orch ps' 2024-03-29T05:12:42.332 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T05:12:43.958 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:43 smithi142 ceph-mon[112475]: pgmap v733: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:12:43.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.956+0000 7fe3eed43700 1 -- 172.21.15.142:0/633977673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3e80ffbd0 msgr2=0x7fe3e8100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:43.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.956+0000 7fe3eed43700 1 --2- 172.21.15.142:0/633977673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3e80ffbd0 0x7fe3e8100050 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009910 tx=0x7fe3d4009c20 comp rx=0 tx=0).stop 2024-03-29T05:12:43.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.956+0000 7fe3eed43700 1 -- 172.21.15.142:0/633977673 shutdown_connections 2024-03-29T05:12:43.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.956+0000 7fe3eed43700 1 --2- 172.21.15.142:0/633977673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3e80ffbd0 0x7fe3e8100050 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:43.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.956+0000 7fe3eed43700 1 --2- 172.21.15.142:0/633977673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e8105a40 0x7fe3e8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:43.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.956+0000 7fe3eed43700 1 -- 172.21.15.142:0/633977673 >> 172.21.15.142:0/633977673 conn(0x7fe3e80755a0 msgr2=0x7fe3e80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:43.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.956+0000 7fe3eed43700 1 -- 172.21.15.142:0/633977673 shutdown_connections 2024-03-29T05:12:43.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.956+0000 7fe3eed43700 1 -- 172.21.15.142:0/633977673 wait complete. 2024-03-29T05:12:43.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.957+0000 7fe3eed43700 1 Processor -- start 2024-03-29T05:12:43.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.957+0000 7fe3eed43700 1 -- start start 2024-03-29T05:12:43.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.957+0000 7fe3eed43700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3e80ffbd0 0x7fe3e8197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:43.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.957+0000 7fe3eed43700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e8105a40 0x7fe3e8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:43.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.957+0000 7fe3eed43700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8198150 con 0x7fe3e80ffbd0 2024-03-29T05:12:43.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.957+0000 7fe3eed43700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8198290 con 0x7fe3e8105a40 2024-03-29T05:12:43.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.957+0000 7fe3ecadf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3e80ffbd0 0x7fe3e8197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:43.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.957+0000 7fe3ecadf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3e80ffbd0 0x7fe3e8197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43878/0 (socket says 172.21.15.142:43878) 2024-03-29T05:12:43.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.957+0000 7fe3ecadf700 1 -- 172.21.15.142:0/629530342 learned_addr learned my addr 172.21.15.142:0/629530342 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:43.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.958+0000 7fe3e7fff700 1 --2- 172.21.15.142:0/629530342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e8105a40 0x7fe3e8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:43.965 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.958+0000 7fe3ecadf700 1 -- 172.21.15.142:0/629530342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e8105a40 msgr2=0x7fe3e8197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:43.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.958+0000 7fe3ecadf700 1 --2- 172.21.15.142:0/629530342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e8105a40 0x7fe3e8197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:43.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.958+0000 7fe3ecadf700 1 -- 172.21.15.142:0/629530342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d40095f0 con 0x7fe3e80ffbd0 2024-03-29T05:12:43.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.958+0000 7fe3e7fff700 1 --2- 172.21.15.142:0/629530342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e8105a40 0x7fe3e8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:12:43.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.958+0000 7fe3ecadf700 1 --2- 172.21.15.142:0/629530342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3e80ffbd0 0x7fe3e8197530 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00e920 tx=0x7fe3dc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:43.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.958+0000 7fe3e5ffb700 1 -- 172.21.15.142:0/629530342 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3dc009790 con 0x7fe3e80ffbd0 2024-03-29T05:12:43.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.958+0000 7fe3e5ffb700 1 -- 172.21.15.142:0/629530342 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3dc00cba0 con 0x7fe3e80ffbd0 2024-03-29T05:12:43.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.958+0000 7fe3eed43700 1 -- 172.21.15.142:0/629530342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e81a0600 con 0x7fe3e80ffbd0 2024-03-29T05:12:43.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.958+0000 7fe3e5ffb700 1 -- 172.21.15.142:0/629530342 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3dc018650 con 0x7fe3e80ffbd0 2024-03-29T05:12:43.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.958+0000 7fe3eed43700 1 -- 172.21.15.142:0/629530342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e81a0b20 con 0x7fe3e80ffbd0 2024-03-29T05:12:43.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.959+0000 7fe3eed43700 1 -- 172.21.15.142:0/629530342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e8107fd0 con 0x7fe3e80ffbd0 2024-03-29T05:12:43.966 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.960+0000 7fe3e5ffb700 1 -- 172.21.15.142:0/629530342 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fe3dc00cd10 con 0x7fe3e80ffbd0 2024-03-29T05:12:43.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.960+0000 7fe3e5ffb700 1 --2- 172.21.15.142:0/629530342 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe3d806dd60 0x7fe3d8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:43.967 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.961+0000 7fe3e5ffb700 1 -- 172.21.15.142:0/629530342 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fe3dc014070 con 0x7fe3e80ffbd0 2024-03-29T05:12:43.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.961+0000 7fe3e7fff700 1 --2- 172.21.15.142:0/629530342 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe3d806dd60 0x7fe3d8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:43.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.961+0000 7fe3e7fff700 1 --2- 172.21.15.142:0/629530342 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe3d806dd60 0x7fe3d8070220 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4000c00 tx=0x7fe3d4009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:43.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:43.965+0000 7fe3e5ffb700 1 -- 172.21.15.142:0/629530342 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe3dc053280 con 0x7fe3e80ffbd0 2024-03-29T05:12:44.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:43 smithi152 ceph-mon[88186]: pgmap v733: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:12:44.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.180+0000 7fe3eed43700 1 -- 172.21.15.142:0/629530342 --> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3e802ccc0 con 0x7fe3d806dd60 2024-03-29T05:12:44.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.191+0000 7fe3e5ffb700 1 -- 172.21.15.142:0/629530342 <== mgr.34104 v2:172.21.15.142:6800/3465978191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fe3e802ccc0 con 0x7fe3d806dd60 2024-03-29T05:12:44.197 INFO:teuthology.orchestra.run.smithi142.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T05:12:44.197 INFO:teuthology.orchestra.run.smithi142.stdout:alertmanager.smithi142 smithi142 *:9093,9094 running (24m) 106s ago 36m 24.2M - 0.25.0 c8568f914cd2 59bb553ca519 2024-03-29T05:12:44.197 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi142 smithi142 running (2m) 106s ago 36m 7373k - 19.0.0-1959-g8da23624 54112f57da44 a9260541b7e8 2024-03-29T05:12:44.197 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-exporter.smithi152 smithi152 running (114s) 110s ago 35m 7109k - 19.0.0-1959-g8da23624 54112f57da44 47e6e41b3305 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi142 smithi142 running (21m) 106s ago 36m 6665k - 19.0.0-1959-g8da23624 54112f57da44 b5a5140a543a 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:crash.smithi152 smithi152 running (21m) 110s ago 35m 6660k - 19.0.0-1959-g8da23624 54112f57da44 c0c073ee315b 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:grafana.smithi142 smithi142 *:3000 running (24m) 106s ago 35m 82.0M - 9.4.12 2bacad6d85d8 8bece2a27622 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.bmkjzf smithi142 running (17m) 106s ago 30m 69.0M - 19.0.0-1959-g8da23624 54112f57da44 e622c941a63a 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi142.wgmndr smithi142 running (18m) 106s ago 30m 67.8M - 19.0.0-1959-g8da23624 54112f57da44 fdb2f8edb341 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.hiilum smithi152 running (17m) 110s ago 30m 135M - 19.0.0-1959-g8da23624 54112f57da44 2096352490b4 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:mds.cephfs.smithi152.uhebsi smithi152 running (17m) 110s ago 30m 172M - 19.0.0-1959-g8da23624 54112f57da44 e56a24022fe8 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi142.swesrg smithi142 *:8443,9283,8765 running (26m) 106s ago 37m 515M - 19.0.0-1959-g8da23624 54112f57da44 97c9ada2f8bf 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:mgr.smithi152.usoolh smithi152 *:8443,9283,8765 running (25m) 110s ago 35m 458M - 19.0.0-1959-g8da23624 54112f57da44 fa1f633638c7 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi142 smithi142 running (22m) 106s ago 37m 74.0M 2048M 19.0.0-1959-g8da23624 54112f57da44 434f713a3fd2 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:mon.smithi152 smithi152 running (22m) 110s ago 34m 56.4M 2048M 19.0.0-1959-g8da23624 54112f57da44 4ad4708e8405 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi142 smithi142 *:9100 running (25m) 106s ago 36m 16.4M - 1.5.0 0da6a335fe13 eadef2494032 2024-03-29T05:12:44.198 INFO:teuthology.orchestra.run.smithi142.stdout:node-exporter.smithi152 smithi152 *:9100 running (25m) 110s ago 35m 15.5M - 1.5.0 0da6a335fe13 024d9e6f5b18 2024-03-29T05:12:44.199 INFO:teuthology.orchestra.run.smithi142.stdout:osd.0 smithi142 running (21m) 106s ago 34m 116M 1216M 19.0.0-1959-g8da23624 54112f57da44 5c47cbf77b86 2024-03-29T05:12:44.199 INFO:teuthology.orchestra.run.smithi142.stdout:osd.1 smithi142 running (20m) 106s ago 33m 136M 1216M 19.0.0-1959-g8da23624 54112f57da44 fadc1526c831 2024-03-29T05:12:44.199 INFO:teuthology.orchestra.run.smithi142.stdout:osd.2 smithi142 running (20m) 106s ago 33m 99.6M 1216M 19.0.0-1959-g8da23624 54112f57da44 ce7651319325 2024-03-29T05:12:44.199 INFO:teuthology.orchestra.run.smithi142.stdout:osd.3 smithi152 running (19m) 110s ago 32m 113M 2240M 19.0.0-1959-g8da23624 54112f57da44 c41086b4f6bc 2024-03-29T05:12:44.199 INFO:teuthology.orchestra.run.smithi142.stdout:osd.4 smithi152 running (18m) 110s ago 32m 99.6M 2240M 19.0.0-1959-g8da23624 54112f57da44 eca640a33744 2024-03-29T05:12:44.199 INFO:teuthology.orchestra.run.smithi142.stdout:osd.5 smithi152 running (18m) 110s ago 32m 104M 2240M 19.0.0-1959-g8da23624 54112f57da44 dfaff5665341 2024-03-29T05:12:44.199 INFO:teuthology.orchestra.run.smithi142.stdout:prometheus.smithi142 smithi142 *:9095 running (24m) 106s ago 35m 71.4M - 2.43.0 a07b618ecd1d 57d226a5e133 2024-03-29T05:12:44.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.193+0000 7fe3eed43700 1 -- 172.21.15.142:0/629530342 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe3d806dd60 msgr2=0x7fe3d8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:44.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.193+0000 7fe3eed43700 1 --2- 172.21.15.142:0/629530342 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe3d806dd60 0x7fe3d8070220 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4000c00 tx=0x7fe3d4009f70 comp rx=0 tx=0).stop 2024-03-29T05:12:44.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.193+0000 7fe3eed43700 1 -- 172.21.15.142:0/629530342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3e80ffbd0 msgr2=0x7fe3e8197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:44.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.193+0000 7fe3eed43700 1 --2- 172.21.15.142:0/629530342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3e80ffbd0 0x7fe3e8197530 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00e920 tx=0x7fe3dc00ece0 comp rx=0 tx=0).stop 2024-03-29T05:12:44.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.193+0000 7fe3eed43700 1 -- 172.21.15.142:0/629530342 shutdown_connections 2024-03-29T05:12:44.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.193+0000 7fe3eed43700 1 --2- 172.21.15.142:0/629530342 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe3d806dd60 0x7fe3d8070220 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:44.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.193+0000 7fe3eed43700 1 --2- 172.21.15.142:0/629530342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3e80ffbd0 0x7fe3e8197530 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:44.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.193+0000 7fe3eed43700 1 --2- 172.21.15.142:0/629530342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e8105a40 0x7fe3e8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:44.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.193+0000 7fe3eed43700 1 -- 172.21.15.142:0/629530342 >> 172.21.15.142:0/629530342 conn(0x7fe3e80755a0 msgr2=0x7fe3e80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:44.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.193+0000 7fe3eed43700 1 -- 172.21.15.142:0/629530342 shutdown_connections 2024-03-29T05:12:44.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:44.193+0000 7fe3eed43700 1 -- 172.21.15.142:0/629530342 wait complete. 2024-03-29T05:12:44.858 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph versions' 2024-03-29T05:12:45.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:12:45.074 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T05:12:45.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:12:46.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:45 smithi152 ceph-mon[88186]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:12:46.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:45 smithi152 ceph-mon[88186]: pgmap v734: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:12:46.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:45 smithi142 ceph-mon[112475]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T05:12:46.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:45 smithi142 ceph-mon[112475]: pgmap v734: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:12:46.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.734+0000 7f3a1a967700 1 -- 172.21.15.142:0/878992144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14100de0 msgr2=0x7f3a141011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:46.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.734+0000 7f3a1a967700 1 --2- 172.21.15.142:0/878992144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14100de0 0x7f3a141011c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009910 tx=0x7f3a04009c20 comp rx=0 tx=0).stop 2024-03-29T05:12:46.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.735+0000 7f3a1a967700 1 -- 172.21.15.142:0/878992144 shutdown_connections 2024-03-29T05:12:46.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.735+0000 7f3a1a967700 1 --2- 172.21.15.142:0/878992144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a14101790 0x7f3a141057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:46.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.735+0000 7f3a1a967700 1 --2- 172.21.15.142:0/878992144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14100de0 0x7f3a141011c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:46.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.735+0000 7f3a1a967700 1 -- 172.21.15.142:0/878992144 >> 172.21.15.142:0/878992144 conn(0x7f3a140fc670 msgr2=0x7f3a140fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:46.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.735+0000 7f3a1a967700 1 -- 172.21.15.142:0/878992144 shutdown_connections 2024-03-29T05:12:46.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.735+0000 7f3a1a967700 1 -- 172.21.15.142:0/878992144 wait complete. 2024-03-29T05:12:46.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.735+0000 7f3a1a967700 1 Processor -- start 2024-03-29T05:12:46.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.735+0000 7f3a1a967700 1 -- start start 2024-03-29T05:12:46.743 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a1a967700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14100de0 0x7f3a1419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:46.743 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a1a967700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a14101790 0x7f3a1419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:46.743 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a1a967700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1419d1f0 con 0x7f3a14100de0 2024-03-29T05:12:46.743 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a1a967700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a141966a0 con 0x7f3a14101790 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a13fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14100de0 0x7f3a1419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a13fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14100de0 0x7f3a1419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:43908/0 (socket says 172.21.15.142:43908) 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a13fff700 1 -- 172.21.15.142:0/1792730613 learned_addr learned my addr 172.21.15.142:0/1792730613 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a137fe700 1 --2- 172.21.15.142:0/1792730613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a14101790 0x7f3a1419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a13fff700 1 -- 172.21.15.142:0/1792730613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a14101790 msgr2=0x7f3a1419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a13fff700 1 --2- 172.21.15.142:0/1792730613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a14101790 0x7f3a1419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a13fff700 1 -- 172.21.15.142:0/1792730613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a040095f0 con 0x7f3a14100de0 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a13fff700 1 --2- 172.21.15.142:0/1792730613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14100de0 0x7f3a1419c620 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3a04006010 tx=0x7f3a0400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a117fa700 1 -- 172.21.15.142:0/1792730613 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a0401d070 con 0x7f3a14100de0 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a117fa700 1 -- 172.21.15.142:0/1792730613 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a0400f460 con 0x7f3a14100de0 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a1a967700 1 -- 172.21.15.142:0/1792730613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a14196920 con 0x7f3a14100de0 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.736+0000 7f3a117fa700 1 -- 172.21.15.142:0/1792730613 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a04017610 con 0x7f3a14100de0 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.737+0000 7f3a1a967700 1 -- 172.21.15.142:0/1792730613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a14196e10 con 0x7f3a14100de0 2024-03-29T05:12:46.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.737+0000 7f3a1a967700 1 -- 172.21.15.142:0/1792730613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a14109180 con 0x7f3a14100de0 2024-03-29T05:12:46.745 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.738+0000 7f3a117fa700 1 -- 172.21.15.142:0/1792730613 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f3a04003a60 con 0x7f3a14100de0 2024-03-29T05:12:46.745 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.738+0000 7f3a117fa700 1 --2- 172.21.15.142:0/1792730613 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f39fc06dd60 0x7f39fc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:46.745 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.738+0000 7f3a117fa700 1 -- 172.21.15.142:0/1792730613 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f3a0408f130 con 0x7f3a14100de0 2024-03-29T05:12:46.745 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.739+0000 7f3a137fe700 1 --2- 172.21.15.142:0/1792730613 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f39fc06dd60 0x7f39fc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:46.745 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.739+0000 7f3a137fe700 1 --2- 172.21.15.142:0/1792730613 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f39fc06dd60 0x7f39fc070220 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3a141059f0 tx=0x7f3a08009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:46.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:46.743+0000 7f3a117fa700 1 -- 172.21.15.142:0/1792730613 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3a0405c100 con 0x7f3a14100de0 2024-03-29T05:12:47.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.035+0000 7f3a1a967700 1 -- 172.21.15.142:0/1792730613 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3a1404ea90 con 0x7f3a14100de0 2024-03-29T05:12:47.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.036+0000 7f3a117fa700 1 -- 172.21.15.142:0/1792730613 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f3a0405bc90 con 0x7f3a14100de0 2024-03-29T05:12:47.042 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-29T05:12:47.042 INFO:teuthology.orchestra.run.smithi142.stdout: "mon": { 2024-03-29T05:12:47.042 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: "mgr": { 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 2 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: "osd": { 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 6 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 4 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: "overall": { 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)": 14 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-29T05:12:47.043 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-29T05:12:47.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.039+0000 7f3a1a967700 1 -- 172.21.15.142:0/1792730613 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f39fc06dd60 msgr2=0x7f39fc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:47.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.039+0000 7f3a1a967700 1 --2- 172.21.15.142:0/1792730613 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f39fc06dd60 0x7f39fc070220 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3a141059f0 tx=0x7f3a08009190 comp rx=0 tx=0).stop 2024-03-29T05:12:47.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.039+0000 7f3a1a967700 1 -- 172.21.15.142:0/1792730613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14100de0 msgr2=0x7f3a1419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:47.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.039+0000 7f3a1a967700 1 --2- 172.21.15.142:0/1792730613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14100de0 0x7f3a1419c620 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3a04006010 tx=0x7f3a0400bd70 comp rx=0 tx=0).stop 2024-03-29T05:12:47.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.039+0000 7f3a1a967700 1 -- 172.21.15.142:0/1792730613 shutdown_connections 2024-03-29T05:12:47.045 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.039+0000 7f3a1a967700 1 --2- 172.21.15.142:0/1792730613 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f39fc06dd60 0x7f39fc070220 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:47.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.039+0000 7f3a1a967700 1 --2- 172.21.15.142:0/1792730613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14100de0 0x7f3a1419c620 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:47.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.039+0000 7f3a1a967700 1 --2- 172.21.15.142:0/1792730613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a14101790 0x7f3a1419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:47.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.039+0000 7f3a1a967700 1 -- 172.21.15.142:0/1792730613 >> 172.21.15.142:0/1792730613 conn(0x7f3a140fc670 msgr2=0x7f3a140fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:47.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.039+0000 7f3a1a967700 1 -- 172.21.15.142:0/1792730613 shutdown_connections 2024-03-29T05:12:47.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:47.039+0000 7f3a1a967700 1 -- 172.21.15.142:0/1792730613 wait complete. 2024-03-29T05:12:47.748 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-29T05:12:47.985 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T05:12:48.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:47 smithi152 ceph-mon[88186]: pgmap v735: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:12:48.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:47 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1792730613' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:12:48.108 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:47 smithi142 ceph-mon[112475]: pgmap v735: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:12:48.108 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:47 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1792730613' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:12:49.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.673+0000 7f048e7b7700 1 -- 172.21.15.142:0/3346104305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488101790 msgr2=0x7f04881057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:49.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.673+0000 7f048e7b7700 1 --2- 172.21.15.142:0/3346104305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488101790 0x7f04881057e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f047c009910 tx=0x7f047c009c20 comp rx=0 tx=0).stop 2024-03-29T05:12:49.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.673+0000 7f048e7b7700 1 -- 172.21.15.142:0/3346104305 shutdown_connections 2024-03-29T05:12:49.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.673+0000 7f048e7b7700 1 --2- 172.21.15.142:0/3346104305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488101790 0x7f04881057e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:49.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.673+0000 7f048e7b7700 1 --2- 172.21.15.142:0/3346104305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0488100de0 0x7f04881011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:49.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.673+0000 7f048e7b7700 1 -- 172.21.15.142:0/3346104305 >> 172.21.15.142:0/3346104305 conn(0x7f04880fc670 msgr2=0x7f04880fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:49.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.673+0000 7f048e7b7700 1 -- 172.21.15.142:0/3346104305 shutdown_connections 2024-03-29T05:12:49.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.673+0000 7f048e7b7700 1 -- 172.21.15.142:0/3346104305 wait complete. 2024-03-29T05:12:49.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.674+0000 7f048e7b7700 1 Processor -- start 2024-03-29T05:12:49.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.674+0000 7f048e7b7700 1 -- start start 2024-03-29T05:12:49.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.674+0000 7f048e7b7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488100de0 0x7f048819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:49.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.674+0000 7f048e7b7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0488101790 0x7f048819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:49.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.674+0000 7f048e7b7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048819d0a0 con 0x7f0488100de0 2024-03-29T05:12:49.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.674+0000 7f048e7b7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04881965e0 con 0x7f0488101790 2024-03-29T05:12:49.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f0487fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488100de0 0x7f048819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:49.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f0487fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488100de0 0x7f048819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46176/0 (socket says 172.21.15.142:46176) 2024-03-29T05:12:49.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f0487fff700 1 -- 172.21.15.142:0/2600381014 learned_addr learned my addr 172.21.15.142:0/2600381014 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:49.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f04877fe700 1 --2- 172.21.15.142:0/2600381014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0488101790 0x7f048819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:49.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f0487fff700 1 -- 172.21.15.142:0/2600381014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0488101790 msgr2=0x7f048819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:49.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f0487fff700 1 --2- 172.21.15.142:0/2600381014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0488101790 0x7f048819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:49.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f0487fff700 1 -- 172.21.15.142:0/2600381014 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c0095f0 con 0x7f0488100de0 2024-03-29T05:12:49.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f0487fff700 1 --2- 172.21.15.142:0/2600381014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488100de0 0x7f048819c560 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f047800c930 tx=0x7f047800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:49.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f04857fa700 1 -- 172.21.15.142:0/2600381014 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0478007ad0 con 0x7f0488100de0 2024-03-29T05:12:49.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f048e7b7700 1 -- 172.21.15.142:0/2600381014 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04881968c0 con 0x7f0488100de0 2024-03-29T05:12:49.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f04857fa700 1 -- 172.21.15.142:0/2600381014 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f047800ce80 con 0x7f0488100de0 2024-03-29T05:12:49.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f04857fa700 1 -- 172.21.15.142:0/2600381014 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0478018630 con 0x7f0488100de0 2024-03-29T05:12:49.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.675+0000 7f048e7b7700 1 -- 172.21.15.142:0/2600381014 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0488196e10 con 0x7f0488100de0 2024-03-29T05:12:49.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.677+0000 7f048e7b7700 1 -- 172.21.15.142:0/2600381014 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04881091e0 con 0x7f0488100de0 2024-03-29T05:12:49.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.677+0000 7f04857fa700 1 -- 172.21.15.142:0/2600381014 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f0478007c30 con 0x7f0488100de0 2024-03-29T05:12:49.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.677+0000 7f04857fa700 1 --2- 172.21.15.142:0/2600381014 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0470072170 0x7f0470074630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:49.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.677+0000 7f04857fa700 1 -- 172.21.15.142:0/2600381014 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f047808dc90 con 0x7f0488100de0 2024-03-29T05:12:49.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.677+0000 7f04877fe700 1 --2- 172.21.15.142:0/2600381014 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0470072170 0x7f0470074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:49.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.678+0000 7f04877fe700 1 --2- 172.21.15.142:0/2600381014 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0470072170 0x7f0470074630 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f04880fce20 tx=0x7f047c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:49.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.682+0000 7f04857fa700 1 -- 172.21.15.142:0/2600381014 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f047805abe0 con 0x7f0488100de0 2024-03-29T05:12:49.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.972+0000 7f048e7b7700 1 -- 172.21.15.142:0/2600381014 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f048804ea90 con 0x7f0488100de0 2024-03-29T05:12:49.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.972+0000 7f04857fa700 1 -- 172.21.15.142:0/2600381014 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f047805a770 con 0x7f0488100de0 2024-03-29T05:12:49.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.975+0000 7f048e7b7700 1 -- 172.21.15.142:0/2600381014 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0470072170 msgr2=0x7f0470074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:49.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.975+0000 7f048e7b7700 1 --2- 172.21.15.142:0/2600381014 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0470072170 0x7f0470074630 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f04880fce20 tx=0x7f047c01a040 comp rx=0 tx=0).stop 2024-03-29T05:12:49.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.975+0000 7f048e7b7700 1 -- 172.21.15.142:0/2600381014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488100de0 msgr2=0x7f048819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:49.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.975+0000 7f048e7b7700 1 --2- 172.21.15.142:0/2600381014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488100de0 0x7f048819c560 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f047800c930 tx=0x7f047800cc40 comp rx=0 tx=0).stop 2024-03-29T05:12:49.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.975+0000 7f048e7b7700 1 -- 172.21.15.142:0/2600381014 shutdown_connections 2024-03-29T05:12:49.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.975+0000 7f048e7b7700 1 --2- 172.21.15.142:0/2600381014 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0470072170 0x7f0470074630 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:49.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.975+0000 7f048e7b7700 1 --2- 172.21.15.142:0/2600381014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488100de0 0x7f048819c560 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:49.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.975+0000 7f048e7b7700 1 --2- 172.21.15.142:0/2600381014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0488101790 0x7f048819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:49.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.975+0000 7f048e7b7700 1 -- 172.21.15.142:0/2600381014 >> 172.21.15.142:0/2600381014 conn(0x7f04880fc670 msgr2=0x7f04880fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:49.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.975+0000 7f048e7b7700 1 -- 172.21.15.142:0/2600381014 shutdown_connections 2024-03-29T05:12:49.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:49.975+0000 7f048e7b7700 1 -- 172.21.15.142:0/2600381014 wait complete. 2024-03-29T05:12:49.995 INFO:teuthology.orchestra.run.smithi142.stdout:true 2024-03-29T05:12:50.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:49 smithi152 ceph-mon[88186]: pgmap v736: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:12:50.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:49 smithi142 ceph-mon[112475]: pgmap v736: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:12:50.609 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -e sha1=8da23624bc17fe0b1f797227e72a0db0ce7821ba -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-29T05:12:50.827 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T05:12:50.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:50 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2600381014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:12:51.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:50 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2600381014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:12:52.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:51 smithi152 ceph-mon[88186]: pgmap v737: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:12:52.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:51 smithi142 ceph-mon[112475]: pgmap v737: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:12:52.536 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.530+0000 7f269412d700 1 -- 172.21.15.142:0/1140800078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f268c069020 msgr2=0x7f268c105780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:52.536 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.530+0000 7f269412d700 1 --2- 172.21.15.142:0/1140800078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f268c069020 0x7f268c105780 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f267c009910 tx=0x7f267c009c20 comp rx=0 tx=0).stop 2024-03-29T05:12:52.536 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.530+0000 7f269412d700 1 -- 172.21.15.142:0/1140800078 shutdown_connections 2024-03-29T05:12:52.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.530+0000 7f269412d700 1 --2- 172.21.15.142:0/1140800078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f268c069020 0x7f268c105780 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:52.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.530+0000 7f269412d700 1 --2- 172.21.15.142:0/1140800078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f268c068670 0x7f268c068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:52.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.530+0000 7f269412d700 1 -- 172.21.15.142:0/1140800078 >> 172.21.15.142:0/1140800078 conn(0x7f268c0754f0 msgr2=0x7f268c075900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:52.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.530+0000 7f269412d700 1 -- 172.21.15.142:0/1140800078 shutdown_connections 2024-03-29T05:12:52.537 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.530+0000 7f269412d700 1 -- 172.21.15.142:0/1140800078 wait complete. 2024-03-29T05:12:52.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.531+0000 7f269412d700 1 Processor -- start 2024-03-29T05:12:52.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.531+0000 7f269412d700 1 -- start start 2024-03-29T05:12:52.538 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.531+0000 7f269412d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f268c068670 0x7f268c0ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.531+0000 7f269412d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f268c069020 0x7f268c100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.531+0000 7f269412d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c1006c0 con 0x7f268c068670 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.531+0000 7f269412d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c100800 con 0x7f268c069020 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.531+0000 7f2691ec9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f268c068670 0x7f268c0ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.531+0000 7f2691ec9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f268c068670 0x7f268c0ffc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46192/0 (socket says 172.21.15.142:46192) 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.531+0000 7f2691ec9700 1 -- 172.21.15.142:0/2642463749 learned_addr learned my addr 172.21.15.142:0/2642463749 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.532+0000 7f26916c8700 1 --2- 172.21.15.142:0/2642463749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f268c069020 0x7f268c100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.532+0000 7f2691ec9700 1 -- 172.21.15.142:0/2642463749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f268c069020 msgr2=0x7f268c100180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.532+0000 7f2691ec9700 1 --2- 172.21.15.142:0/2642463749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f268c069020 0x7f268c100180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.532+0000 7f2691ec9700 1 -- 172.21.15.142:0/2642463749 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c0095f0 con 0x7f268c068670 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.532+0000 7f2691ec9700 1 --2- 172.21.15.142:0/2642463749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f268c068670 0x7f268c0ffc40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f268800d8d0 tx=0x7f268800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.532+0000 7f2682ffd700 1 -- 172.21.15.142:0/2642463749 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2688009940 con 0x7f268c068670 2024-03-29T05:12:52.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.532+0000 7f2682ffd700 1 -- 172.21.15.142:0/2642463749 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2688010460 con 0x7f268c068670 2024-03-29T05:12:52.540 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.532+0000 7f2682ffd700 1 -- 172.21.15.142:0/2642463749 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f268800f5f0 con 0x7f268c068670 2024-03-29T05:12:52.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.532+0000 7f269412d700 1 -- 172.21.15.142:0/2642463749 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c1a2900 con 0x7f268c068670 2024-03-29T05:12:52.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.533+0000 7f269412d700 1 -- 172.21.15.142:0/2642463749 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c1a2cb0 con 0x7f268c068670 2024-03-29T05:12:52.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.533+0000 7f269412d700 1 -- 172.21.15.142:0/2642463749 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c04ea90 con 0x7f268c068670 2024-03-29T05:12:52.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.534+0000 7f2682ffd700 1 -- 172.21.15.142:0/2642463749 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f2688009aa0 con 0x7f268c068670 2024-03-29T05:12:52.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.535+0000 7f2682ffd700 1 --2- 172.21.15.142:0/2642463749 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f267806db50 0x7f2678070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:52.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.535+0000 7f2682ffd700 1 -- 172.21.15.142:0/2642463749 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f268808de10 con 0x7f268c068670 2024-03-29T05:12:52.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.535+0000 7f26916c8700 1 --2- 172.21.15.142:0/2642463749 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f267806db50 0x7f2678070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:52.542 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.535+0000 7f26916c8700 1 --2- 172.21.15.142:0/2642463749 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f267806db50 0x7f2678070010 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f267c0051a0 tx=0x7f267c005fb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:52.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.539+0000 7f2682ffd700 1 -- 172.21.15.142:0/2642463749 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2688057230 con 0x7f268c068670 2024-03-29T05:12:52.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.825+0000 7f269412d700 1 -- 172.21.15.142:0/2642463749 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f268c101170 con 0x7f268c068670 2024-03-29T05:12:52.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.825+0000 7f2682ffd700 1 -- 172.21.15.142:0/2642463749 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f268805a8f0 con 0x7f268c068670 2024-03-29T05:12:52.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.828+0000 7f269412d700 1 -- 172.21.15.142:0/2642463749 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f267806db50 msgr2=0x7f2678070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:52.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.828+0000 7f269412d700 1 --2- 172.21.15.142:0/2642463749 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f267806db50 0x7f2678070010 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f267c0051a0 tx=0x7f267c005fb0 comp rx=0 tx=0).stop 2024-03-29T05:12:52.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.828+0000 7f269412d700 1 -- 172.21.15.142:0/2642463749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f268c068670 msgr2=0x7f268c0ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:52.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.828+0000 7f269412d700 1 --2- 172.21.15.142:0/2642463749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f268c068670 0x7f268c0ffc40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f268800d8d0 tx=0x7f268800dc90 comp rx=0 tx=0).stop 2024-03-29T05:12:52.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.828+0000 7f269412d700 1 -- 172.21.15.142:0/2642463749 shutdown_connections 2024-03-29T05:12:52.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.828+0000 7f269412d700 1 --2- 172.21.15.142:0/2642463749 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f267806db50 0x7f2678070010 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:52.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.828+0000 7f269412d700 1 --2- 172.21.15.142:0/2642463749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f268c068670 0x7f268c0ffc40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:52.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.828+0000 7f269412d700 1 --2- 172.21.15.142:0/2642463749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f268c069020 0x7f268c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:52.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.828+0000 7f269412d700 1 -- 172.21.15.142:0/2642463749 >> 172.21.15.142:0/2642463749 conn(0x7f268c0754f0 msgr2=0x7f268c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:52.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.828+0000 7f269412d700 1 -- 172.21.15.142:0/2642463749 shutdown_connections 2024-03-29T05:12:52.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:52.828+0000 7f269412d700 1 -- 172.21.15.142:0/2642463749 wait complete. 2024-03-29T05:12:52.848 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph version 19.0.0-1959-g8da23624 (8da23624bc17fe0b1f797227e72a0db0ce7821ba) squid (dev)" 2024-03-29T05:12:53.429 DEBUG:teuthology.parallel:result is None 2024-03-29T05:12:53.430 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T05:12:53.443 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T05:12:53.443 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-29T05:12:53.695 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi142/config 2024-03-29T05:12:54.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:53 smithi152 ceph-mon[88186]: pgmap v738: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:12:54.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:53 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2642463749' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:12:54.050 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:53 smithi142 ceph-mon[112475]: pgmap v738: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:12:54.050 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:53 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2642463749' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T05:12:55.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.287+0000 7f640e486700 1 -- 172.21.15.142:0/220236301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6408100da0 msgr2=0x7f6408101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:55.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.287+0000 7f640e486700 1 --2- 172.21.15.142:0/220236301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6408100da0 0x7f6408101180 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009910 tx=0x7f63f8009c20 comp rx=0 tx=0).stop 2024-03-29T05:12:55.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.287+0000 7f640e486700 1 -- 172.21.15.142:0/220236301 shutdown_connections 2024-03-29T05:12:55.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.287+0000 7f640e486700 1 --2- 172.21.15.142:0/220236301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6408101750 0x7f64081057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:55.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.287+0000 7f640e486700 1 --2- 172.21.15.142:0/220236301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6408100da0 0x7f6408101180 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:55.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.287+0000 7f640e486700 1 -- 172.21.15.142:0/220236301 >> 172.21.15.142:0/220236301 conn(0x7f64080fc630 msgr2=0x7f64080fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:55.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.288+0000 7f640e486700 1 -- 172.21.15.142:0/220236301 shutdown_connections 2024-03-29T05:12:55.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.288+0000 7f640e486700 1 -- 172.21.15.142:0/220236301 wait complete. 2024-03-29T05:12:55.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.288+0000 7f640e486700 1 Processor -- start 2024-03-29T05:12:55.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.288+0000 7f640e486700 1 -- start start 2024-03-29T05:12:55.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f640e486700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6408100da0 0x7f640819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:55.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f640e486700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6408101750 0x7f640819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:55.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f640e486700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640819d210 con 0x7f6408100da0 2024-03-29T05:12:55.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f640e486700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64081966c0 con 0x7f6408101750 2024-03-29T05:12:55.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f6407fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6408100da0 0x7f640819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:55.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f6407fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6408100da0 0x7f640819c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46212/0 (socket says 172.21.15.142:46212) 2024-03-29T05:12:55.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f6407fff700 1 -- 172.21.15.142:0/854957710 learned_addr learned my addr 172.21.15.142:0/854957710 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:55.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f64077fe700 1 --2- 172.21.15.142:0/854957710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6408101750 0x7f640819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:55.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f6407fff700 1 -- 172.21.15.142:0/854957710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6408101750 msgr2=0x7f640819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:55.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f6407fff700 1 --2- 172.21.15.142:0/854957710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6408101750 0x7f640819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:55.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f6407fff700 1 -- 172.21.15.142:0/854957710 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80095f0 con 0x7f6408100da0 2024-03-29T05:12:55.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f64077fe700 1 --2- 172.21.15.142:0/854957710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6408101750 0x7f640819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:12:55.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.289+0000 7f6407fff700 1 --2- 172.21.15.142:0/854957710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6408100da0 0x7f640819c640 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f63f8006010 tx=0x7f63f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:55.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.290+0000 7f64057fa700 1 -- 172.21.15.142:0/854957710 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63f801d070 con 0x7f6408100da0 2024-03-29T05:12:55.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.290+0000 7f640e486700 1 -- 172.21.15.142:0/854957710 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6408196940 con 0x7f6408100da0 2024-03-29T05:12:55.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.290+0000 7f64057fa700 1 -- 172.21.15.142:0/854957710 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63f8004fb0 con 0x7f6408100da0 2024-03-29T05:12:55.298 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.290+0000 7f64057fa700 1 -- 172.21.15.142:0/854957710 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63f8022ea0 con 0x7f6408100da0 2024-03-29T05:12:55.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.290+0000 7f640e486700 1 -- 172.21.15.142:0/854957710 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6408196e30 con 0x7f6408100da0 2024-03-29T05:12:55.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.291+0000 7f640e486700 1 -- 172.21.15.142:0/854957710 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6408109140 con 0x7f6408100da0 2024-03-29T05:12:55.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.292+0000 7f64057fa700 1 -- 172.21.15.142:0/854957710 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f63f8022470 con 0x7f6408100da0 2024-03-29T05:12:55.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.292+0000 7f64057fa700 1 --2- 172.21.15.142:0/854957710 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f63f006dd60 0x7f63f0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:55.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.292+0000 7f64057fa700 1 -- 172.21.15.142:0/854957710 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f63f808f230 con 0x7f6408100da0 2024-03-29T05:12:55.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.292+0000 7f64077fe700 1 --2- 172.21.15.142:0/854957710 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f63f006dd60 0x7f63f0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:55.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.293+0000 7f64077fe700 1 --2- 172.21.15.142:0/854957710 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f63f006dd60 0x7f63f0070220 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6408197b30 tx=0x7f63fc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:55.307 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.301+0000 7f64057fa700 1 -- 172.21.15.142:0/854957710 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f63f805c230 con 0x7f6408100da0 2024-03-29T05:12:55.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.588+0000 7f640e486700 1 -- 172.21.15.142:0/854957710 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f640804ea90 con 0x7f6408100da0 2024-03-29T05:12:55.595 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.589+0000 7f64057fa700 1 -- 172.21.15.142:0/854957710 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 39 v39) v1 ==== 76+0+1973 (secure 0 0 0) 0x7f63f8022720 con 0x7f6408100da0 2024-03-29T05:12:55.596 INFO:teuthology.orchestra.run.smithi142.stdout:e39 2024-03-29T05:12:55.596 INFO:teuthology.orchestra.run.smithi142.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T05:12:55.596 INFO:teuthology.orchestra.run.smithi142.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T05:12:55.596 INFO:teuthology.orchestra.run.smithi142.stdout:legacy client fscid: 1 2024-03-29T05:12:55.596 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:12:55.596 INFO:teuthology.orchestra.run.smithi142.stdout:Filesystem 'cephfs' (1) 2024-03-29T05:12:55.597 INFO:teuthology.orchestra.run.smithi142.stdout:fs_name cephfs 2024-03-29T05:12:55.597 INFO:teuthology.orchestra.run.smithi142.stdout:epoch 39 2024-03-29T05:12:55.597 INFO:teuthology.orchestra.run.smithi142.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-29T05:12:55.597 INFO:teuthology.orchestra.run.smithi142.stdout:created 2024-03-29T04:42:06.709178+0000 2024-03-29T05:12:55.597 INFO:teuthology.orchestra.run.smithi142.stdout:modified 2024-03-29T05:11:04.422493+0000 2024-03-29T05:12:55.597 INFO:teuthology.orchestra.run.smithi142.stdout:tableserver 0 2024-03-29T05:12:55.598 INFO:teuthology.orchestra.run.smithi142.stdout:root 0 2024-03-29T05:12:55.598 INFO:teuthology.orchestra.run.smithi142.stdout:session_timeout 60 2024-03-29T05:12:55.598 INFO:teuthology.orchestra.run.smithi142.stdout:session_autoclose 300 2024-03-29T05:12:55.598 INFO:teuthology.orchestra.run.smithi142.stdout:max_file_size 1099511627776 2024-03-29T05:12:55.598 INFO:teuthology.orchestra.run.smithi142.stdout:max_xattr_size 65536 2024-03-29T05:12:55.599 INFO:teuthology.orchestra.run.smithi142.stdout:required_client_features {} 2024-03-29T05:12:55.599 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure 0 2024-03-29T05:12:55.599 INFO:teuthology.orchestra.run.smithi142.stdout:last_failure_osd_epoch 95 2024-03-29T05:12:55.599 INFO:teuthology.orchestra.run.smithi142.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-29T05:12:55.600 INFO:teuthology.orchestra.run.smithi142.stdout:max_mds 2 2024-03-29T05:12:55.601 INFO:teuthology.orchestra.run.smithi142.stdout:in 0,1 2024-03-29T05:12:55.601 INFO:teuthology.orchestra.run.smithi142.stdout:up {0=34420,1=34424} 2024-03-29T05:12:55.601 INFO:teuthology.orchestra.run.smithi142.stdout:failed 2024-03-29T05:12:55.601 INFO:teuthology.orchestra.run.smithi142.stdout:damaged 2024-03-29T05:12:55.602 INFO:teuthology.orchestra.run.smithi142.stdout:stopped 2024-03-29T05:12:55.602 INFO:teuthology.orchestra.run.smithi142.stdout:data_pools [3] 2024-03-29T05:12:55.602 INFO:teuthology.orchestra.run.smithi142.stdout:metadata_pool 2 2024-03-29T05:12:55.602 INFO:teuthology.orchestra.run.smithi142.stdout:inline_data disabled 2024-03-29T05:12:55.602 INFO:teuthology.orchestra.run.smithi142.stdout:balancer 2024-03-29T05:12:55.602 INFO:teuthology.orchestra.run.smithi142.stdout:bal_rank_mask -1 2024-03-29T05:12:55.603 INFO:teuthology.orchestra.run.smithi142.stdout:standby_count_wanted 1 2024-03-29T05:12:55.603 INFO:teuthology.orchestra.run.smithi142.stdout:qdb_cluster leader: 34420 members: 34424,34420 2024-03-29T05:12:55.603 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.wgmndr{0:34420} state up:active seq 245 join_fscid=1 addr [v2:172.21.15.142:6826/2841622897,v1:172.21.15.142:6827/2841622897] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:12:55.603 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.uhebsi{0:44323} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.152:6824/778282077,v1:172.21.15.152:6825/778282077] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:12:55.603 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi142.bmkjzf{1:34424} state up:active seq 241 join_fscid=1 addr [v2:172.21.15.142:6828/1735570051,v1:172.21.15.142:6829/1735570051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:12:55.604 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.cephfs.smithi152.hiilum{1:44327} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.152:6826/2852002359,v1:172.21.15.152:6827/2852002359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-29T05:12:55.604 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:12:55.604 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:12:55.605 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.591+0000 7f640e486700 1 -- 172.21.15.142:0/854957710 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f63f006dd60 msgr2=0x7f63f0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:55.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.592+0000 7f640e486700 1 --2- 172.21.15.142:0/854957710 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f63f006dd60 0x7f63f0070220 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6408197b30 tx=0x7f63fc008040 comp rx=0 tx=0).stop 2024-03-29T05:12:55.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.592+0000 7f640e486700 1 -- 172.21.15.142:0/854957710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6408100da0 msgr2=0x7f640819c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:55.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.592+0000 7f640e486700 1 --2- 172.21.15.142:0/854957710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6408100da0 0x7f640819c640 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f63f8006010 tx=0x7f63f800be20 comp rx=0 tx=0).stop 2024-03-29T05:12:55.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.592+0000 7f640e486700 1 -- 172.21.15.142:0/854957710 shutdown_connections 2024-03-29T05:12:55.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.592+0000 7f640e486700 1 --2- 172.21.15.142:0/854957710 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f63f006dd60 0x7f63f0070220 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:55.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.592+0000 7f640e486700 1 --2- 172.21.15.142:0/854957710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6408100da0 0x7f640819c640 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:55.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.592+0000 7f640e486700 1 --2- 172.21.15.142:0/854957710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6408101750 0x7f640819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:55.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.592+0000 7f640e486700 1 -- 172.21.15.142:0/854957710 >> 172.21.15.142:0/854957710 conn(0x7f64080fc630 msgr2=0x7f64080fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:55.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.592+0000 7f640e486700 1 -- 172.21.15.142:0/854957710 shutdown_connections 2024-03-29T05:12:55.607 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:55.592+0000 7f640e486700 1 -- 172.21.15.142:0/854957710 wait complete. 2024-03-29T05:12:55.607 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 39 2024-03-29T05:12:56.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:55 smithi152 ceph-mon[88186]: pgmap v739: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:12:56.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:55 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/854957710' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:12:56.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:55 smithi142 ceph-mon[112475]: pgmap v739: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:12:56.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:55 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/854957710' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T05:12:56.241 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-29T05:12:56.253 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T05:12:56.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.356+0000 7f54861dd640 1 -- 172.21.15.142:0/3680603785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5480100c30 msgr2=0x7f5480104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:56.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.356+0000 7f54861dd640 1 --2- 172.21.15.142:0/3680603785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5480100c30 0x7f5480104c70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5474009880 tx=0x7f547402bce0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.356+0000 7f54861dd640 1 -- 172.21.15.142:0/3680603785 shutdown_connections 2024-03-29T05:12:56.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.356+0000 7f54861dd640 1 --2- 172.21.15.142:0/3680603785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5480100c30 0x7f5480104c70 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.356+0000 7f54861dd640 1 --2- 172.21.15.142:0/3680603785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5480100280 0x7f5480100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.356+0000 7f54861dd640 1 -- 172.21.15.142:0/3680603785 >> 172.21.15.142:0/3680603785 conn(0x7f54800fbb20 msgr2=0x7f54800fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:56.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.356+0000 7f54861dd640 1 -- 172.21.15.142:0/3680603785 shutdown_connections 2024-03-29T05:12:56.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.356+0000 7f54861dd640 1 -- 172.21.15.142:0/3680603785 wait complete. 2024-03-29T05:12:56.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.356+0000 7f54861dd640 1 Processor -- start 2024-03-29T05:12:56.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f54861dd640 1 -- start start 2024-03-29T05:12:56.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f54861dd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5480100280 0x7f548019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:56.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f54861dd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f548019bee0 0x7f5480195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:56.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f54861dd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548019c3f0 con 0x7f5480100280 2024-03-29T05:12:56.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f54861dd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548019c530 con 0x7f548019bee0 2024-03-29T05:12:56.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f547f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5480100280 0x7f548019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:56.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f547f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5480100280 0x7f548019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46232/0 (socket says 172.21.15.142:46232) 2024-03-29T05:12:56.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f547f7fe640 1 -- 172.21.15.142:0/3372339262 learned_addr learned my addr 172.21.15.142:0/3372339262 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:56.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f547effd640 1 --2- 172.21.15.142:0/3372339262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f548019bee0 0x7f5480195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:56.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f547f7fe640 1 -- 172.21.15.142:0/3372339262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f548019bee0 msgr2=0x7f5480195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:56.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f547f7fe640 1 --2- 172.21.15.142:0/3372339262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f548019bee0 0x7f5480195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f547f7fe640 1 -- 172.21.15.142:0/3372339262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5474009560 con 0x7f5480100280 2024-03-29T05:12:56.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.357+0000 7f547effd640 1 --2- 172.21.15.142:0/3372339262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f548019bee0 0x7f5480195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:12:56.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.358+0000 7f547f7fe640 1 --2- 172.21.15.142:0/3372339262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5480100280 0x7f548019b9a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f547000c840 tx=0x7f547000cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:56.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.358+0000 7f547cff9640 1 -- 172.21.15.142:0/3372339262 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54700041f0 con 0x7f5480100280 2024-03-29T05:12:56.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.358+0000 7f547cff9640 1 -- 172.21.15.142:0/3372339262 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5470007e60 con 0x7f5480100280 2024-03-29T05:12:56.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.358+0000 7f54861dd640 1 -- 172.21.15.142:0/3372339262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54801960e0 con 0x7f5480100280 2024-03-29T05:12:56.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.358+0000 7f547cff9640 1 -- 172.21.15.142:0/3372339262 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54700049d0 con 0x7f5480100280 2024-03-29T05:12:56.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.358+0000 7f54861dd640 1 -- 172.21.15.142:0/3372339262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5480196640 con 0x7f5480100280 2024-03-29T05:12:56.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.359+0000 7f54861dd640 1 -- 172.21.15.142:0/3372339262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54801968c0 con 0x7f5480100280 2024-03-29T05:12:56.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.360+0000 7f547cff9640 1 -- 172.21.15.142:0/3372339262 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f5470005120 con 0x7f5480100280 2024-03-29T05:12:56.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.360+0000 7f547cff9640 1 --2- 172.21.15.142:0/3372339262 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f546806dd90 0x7f5468070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:56.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.360+0000 7f547effd640 1 --2- 172.21.15.142:0/3372339262 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f546806dd90 0x7f5468070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:56.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.360+0000 7f547cff9640 1 -- 172.21.15.142:0/3372339262 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f547008dab0 con 0x7f5480100280 2024-03-29T05:12:56.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.361+0000 7f547effd640 1 --2- 172.21.15.142:0/3372339262 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f546806dd90 0x7f5468070250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5474005ec0 tx=0x7f5474005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:56.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.367+0000 7f547cff9640 1 -- 172.21.15.142:0/3372339262 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5470056ed0 con 0x7f5480100280 2024-03-29T05:12:56.554 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.548+0000 7f54861dd640 1 -- 172.21.15.142:0/3372339262 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f548004e040 con 0x7f5480100280 2024-03-29T05:12:56.556 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.550+0000 7f547cff9640 1 -- 172.21.15.142:0/3372339262 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17309 (secure 0 0 0) 0x7f547005a590 con 0x7f5480100280 2024-03-29T05:12:56.556 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:12:56.556 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":99,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","created":"2024-03-29T04:35:07.701328+0000","modified":"2024-03-29T04:55:30.115802+0000","last_up_change":"2024-03-29T04:54:31.016765+0000","last_in_change":"2024-03-29T04:40:27.009053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:39:31.891786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-29T04:42:05.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-29T04:42:05.937588+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d908de1a-bec3-4238-959a-f3f550431ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":51,"up_from":55,"up_thru":92,"down_at":52,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1587216149},{"type":"v1","addr":"172.21.15.142:6803","nonce":1587216149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1587216149},{"type":"v1","addr":"172.21.15.142:6805","nonce":1587216149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1587216149},{"type":"v1","addr":"172.21.15.142:6809","nonce":1587216149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1587216149},{"type":"v1","addr":"172.21.15.142:6807","nonce":1587216149}]},"public_addr":"172.21.15.142:6803/1587216149","cluster_addr":"172.21.15.142:6805/1587216149","heartbeat_back_addr":"172.21.15.142:6809/1587216149","heartbeat_front_addr":"172.21.15.142:6807/1587216149","state":["exists","up"]},{"osd":1,"uuid":"717e7135-21ab-4d33-a81e-65da6d5186c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":56,"up_from":67,"up_thru":92,"down_at":57,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":296003948},{"type":"v1","addr":"172.21.15.142:6811","nonce":296003948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":296003948},{"type":"v1","addr":"172.21.15.142:6813","nonce":296003948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":296003948},{"type":"v1","addr":"172.21.15.142:6817","nonce":296003948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":296003948},{"type":"v1","addr":"172.21.15.142:6815","nonce":296003948}]},"public_addr":"172.21.15.142:6811/296003948","cluster_addr":"172.21.15.142:6813/296003948","heartbeat_back_addr":"172.21.15.142:6817/296003948","heartbeat_front_addr":"172.21.15.142:6815/296003948","state":["exists","up"]},{"osd":2,"uuid":"977032d2-3c29-4c68-ae66-73d024ab94ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":73,"up_from":77,"up_thru":92,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":658903196},{"type":"v1","addr":"172.21.15.142:6819","nonce":658903196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":658903196},{"type":"v1","addr":"172.21.15.142:6821","nonce":658903196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":658903196},{"type":"v1","addr":"172.21.15.142:6825","nonce":658903196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":658903196},{"type":"v1","addr":"172.21.15.142:6823","nonce":658903196}]},"public_addr":"172.21.15.142:6819/658903196","cluster_addr":"172.21.15.142:6821/658903196","heartbeat_back_addr":"172.21.15.142:6825/658903196","heartbeat_front_addr":"172.21.15.142:6823/658903196","state":["exists","up"]},{"osd":3,"uuid":"39ae7173-3a83-4ae1-9866-5a0da908171c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":78,"up_from":82,"up_thru":92,"down_at":79,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1538535999},{"type":"v1","addr":"172.21.15.152:6801","nonce":1538535999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1538535999},{"type":"v1","addr":"172.21.15.152:6803","nonce":1538535999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1538535999},{"type":"v1","addr":"172.21.15.152:6807","nonce":1538535999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1538535999},{"type":"v1","addr":"172.21.15.152:6805","nonce":1538535999}]},"public_addr":"172.21.15.152:6801/1538535999","cluster_addr":"172.21.15.152:6803/1538535999","heartbeat_back_addr":"172.21.15.152:6807/1538535999","heartbeat_front_addr":"172.21.15.152:6805/1538535999","state":["exists","up"]},{"osd":4,"uuid":"f7b5fab7-0f30-4756-a481-15ad64b9e691","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":83,"up_from":87,"up_thru":92,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":3103638517},{"type":"v1","addr":"172.21.15.152:6809","nonce":3103638517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3103638517},{"type":"v1","addr":"172.21.15.152:6811","nonce":3103638517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3103638517},{"type":"v1","addr":"172.21.15.152:6815","nonce":3103638517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3103638517},{"type":"v1","addr":"172.21.15.152:6813","nonce":3103638517}]},"public_addr":"172.21.15.152:6809/3103638517","cluster_addr":"172.21.15.152:6811/3103638517","heartbeat_back_addr":"172.21.15.152:6815/3103638517","heartbeat_front_addr":"172.21.15.152:6813/3103638517","state":["exists","up"]},{"osd":5,"uuid":"8ed3caba-b201-4ad7-b055-3dc40610ab1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":88,"up_from":92,"up_thru":92,"down_at":89,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3741970037},{"type":"v1","addr":"172.21.15.152:6817","nonce":3741970037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3741970037},{"type":"v1","addr":"172.21.15.152:6819","nonce":3741970037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3741970037},{"type":"v1","addr":"172.21.15.152:6823","nonce":3741970037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3741970037},{"type":"v1","addr":"172.21.15.152:6821","nonce":3741970037}]},"public_addr":"172.21.15.152:6817/3741970037","cluster_addr":"172.21.15.152:6819/3741970037","heartbeat_back_addr":"172.21.15.152:6823/3741970037","heartbeat_front_addr":"172.21.15.152:6821/3741970037","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:38:33.605811+0000","dead_epoch":51},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:00.911691+0000","dead_epoch":56},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:28.052973+0000","dead_epoch":73},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:51.974988+0000","dead_epoch":78},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:18.104282+0000","dead_epoch":83},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:44.747877+0000","dead_epoch":88}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":0,"to":2}]},{"pgid":"2.4","mappings":[{"from":0,"to":2}]},{"pgid":"2.b","mappings":[{"from":3,"to":1}]},{"pgid":"2.f","mappings":[{"from":0,"to":1}]},{"pgid":"2.14","mappings":[{"from":3,"to":1}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":3,"to":1}]},{"pgid":"3.14","mappings":[{"from":3,"to":1}]},{"pgid":"3.1a","mappings":[{"from":4,"to":0}]},{"pgid":"3.1d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:6824/832699894":"2024-03-30T04:54:33.202001+0000","172.21.15.142:6829/1941073852":"2024-03-30T04:54:33.201978+0000","172.21.15.142:6828/1941073852":"2024-03-30T04:54:33.201978+0000","172.21.15.142:6826/2071436089":"2024-03-30T04:42:25.447288+0000","172.21.15.142:6801/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6800/2395947042":"2024-03-30T04:45:42.287114+0000","172.21.15.142:6801/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/586295950":"2024-03-30T04:37:17.579177+0000","172.21.15.152:0/2558519425":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/2666008640":"2024-03-30T04:35:33.215645+0000","172.21.15.152:6825/832699894":"2024-03-30T04:54:33.202001+0000","172.21.15.142:6827/2071436089":"2024-03-30T04:42:25.447288+0000","172.21.15.152:0/2307372541":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/484532431":"2024-03-30T04:45:42.287114+0000","172.21.15.152:0/3618978330":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/351083517":"2024-03-30T04:49:59.502486+0000","172.21.15.142:0/390665991":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/2545215192":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6800/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/672130862":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/1851570418":"2024-03-30T04:45:42.287114+0000","172.21.15.142:0/3750656759":"2024-03-30T04:45:42.287114+0000","172.21.15.152:0/2082842835":"2024-03-30T04:46:54.194028+0000","172.21.15.142:6801/2395947042":"2024-03-30T04:45:42.287114+0000","172.21.15.152:6828/3130987667":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/2370069599":"2024-03-30T04:45:42.287114+0000","172.21.15.142:6800/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/1199684691":"2024-03-30T04:49:59.502486+0000","172.21.15.152:0/2115311268":"2024-03-30T04:46:54.194028+0000","172.21.15.152:0/1140094447":"2024-03-30T04:46:54.194028+0000","172.21.15.142:6800/1771239042":"2024-03-30T04:49:59.502486+0000","172.21.15.152:6829/3130987667":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/3751399855":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/1935271872":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/3338740178":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/966342870":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6801/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:6800/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/2191110203":"2024-03-30T04:49:59.502486+0000","172.21.15.142:6801/1771239042":"2024-03-30T04:49:59.502486+0000","172.21.15.142:0/143963075":"2024-03-30T04:49:59.502486+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T05:12:56.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.552+0000 7f54861dd640 1 -- 172.21.15.142:0/3372339262 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f546806dd90 msgr2=0x7f5468070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:56.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.552+0000 7f54861dd640 1 --2- 172.21.15.142:0/3372339262 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f546806dd90 0x7f5468070250 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5474005ec0 tx=0x7f5474005fd0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.552+0000 7f54861dd640 1 -- 172.21.15.142:0/3372339262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5480100280 msgr2=0x7f548019b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:56.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.552+0000 7f54861dd640 1 --2- 172.21.15.142:0/3372339262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5480100280 0x7f548019b9a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f547000c840 tx=0x7f547000cd00 comp rx=0 tx=0).stop 2024-03-29T05:12:56.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.552+0000 7f54861dd640 1 -- 172.21.15.142:0/3372339262 shutdown_connections 2024-03-29T05:12:56.558 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.552+0000 7f54861dd640 1 --2- 172.21.15.142:0/3372339262 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f546806dd90 0x7f5468070250 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.559 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.552+0000 7f54861dd640 1 --2- 172.21.15.142:0/3372339262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f548019bee0 0x7f5480195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.559 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.552+0000 7f54861dd640 1 --2- 172.21.15.142:0/3372339262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5480100280 0x7f548019b9a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.559 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.552+0000 7f54861dd640 1 -- 172.21.15.142:0/3372339262 >> 172.21.15.142:0/3372339262 conn(0x7f54800fbb20 msgr2=0x7f54800fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:56.559 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.552+0000 7f54861dd640 1 -- 172.21.15.142:0/3372339262 shutdown_connections 2024-03-29T05:12:56.559 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.552+0000 7f54861dd640 1 -- 172.21.15.142:0/3372339262 wait complete. 2024-03-29T05:12:56.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:39:31.891786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-29T04:42:05.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-29T04:42:05.937588+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T05:12:56.570 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T05:12:56.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.673+0000 7f28a1237640 1 -- 172.21.15.142:0/2346905999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f289c0675c0 msgr2=0x7f289c0679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:56.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.673+0000 7f28a1237640 1 --2- 172.21.15.142:0/2346905999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f289c0675c0 0x7f289c0679a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2894009c20 tx=0x7f289402c1b0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.673+0000 7f28a1237640 1 -- 172.21.15.142:0/2346905999 shutdown_connections 2024-03-29T05:12:56.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.673+0000 7f28a1237640 1 --2- 172.21.15.142:0/2346905999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f289c067f70 0x7f289c104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.673+0000 7f28a1237640 1 --2- 172.21.15.142:0/2346905999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f289c0675c0 0x7f289c0679a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.673+0000 7f28a1237640 1 -- 172.21.15.142:0/2346905999 >> 172.21.15.142:0/2346905999 conn(0x7f289c074a50 msgr2=0x7f289c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:56.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.673+0000 7f28a1237640 1 -- 172.21.15.142:0/2346905999 shutdown_connections 2024-03-29T05:12:56.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.673+0000 7f28a1237640 1 -- 172.21.15.142:0/2346905999 wait complete. 2024-03-29T05:12:56.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.674+0000 7f28a1237640 1 Processor -- start 2024-03-29T05:12:56.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.674+0000 7f28a1237640 1 -- start start 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.674+0000 7f28a1237640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f289c0675c0 0x7f289c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.674+0000 7f28a1237640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f289c1962f0 0x7f289c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.674+0000 7f28a1237640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c1968f0 con 0x7f289c1962f0 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.674+0000 7f28a1237640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c196a60 con 0x7f289c0675c0 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.674+0000 7f289a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f289c1962f0 0x7f289c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.674+0000 7f289a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f289c1962f0 0x7f289c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46244/0 (socket says 172.21.15.142:46244) 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.674+0000 7f289a59c640 1 -- 172.21.15.142:0/2628933993 learned_addr learned my addr 172.21.15.142:0/2628933993 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.674+0000 7f289ad9d640 1 --2- 172.21.15.142:0/2628933993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f289c0675c0 0x7f289c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.674+0000 7f289a59c640 1 -- 172.21.15.142:0/2628933993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f289c0675c0 msgr2=0x7f289c195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.674+0000 7f289a59c640 1 --2- 172.21.15.142:0/2628933993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f289c0675c0 0x7f289c195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.675+0000 7f289a59c640 1 -- 172.21.15.142:0/2628933993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2894009900 con 0x7f289c1962f0 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.675+0000 7f289a59c640 1 --2- 172.21.15.142:0/2628933993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f289c1962f0 0x7f289c19a750 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f289000c920 tx=0x7f289000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:56.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.675+0000 7f2883fff640 1 -- 172.21.15.142:0/2628933993 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2890007b80 con 0x7f289c1962f0 2024-03-29T05:12:56.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.675+0000 7f2883fff640 1 -- 172.21.15.142:0/2628933993 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2890007ce0 con 0x7f289c1962f0 2024-03-29T05:12:56.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.675+0000 7f28a1237640 1 -- 172.21.15.142:0/2628933993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289c19ad50 con 0x7f289c1962f0 2024-03-29T05:12:56.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.675+0000 7f2883fff640 1 -- 172.21.15.142:0/2628933993 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28900187d0 con 0x7f289c1962f0 2024-03-29T05:12:56.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.675+0000 7f28a1237640 1 -- 172.21.15.142:0/2628933993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289c19b2b0 con 0x7f289c1962f0 2024-03-29T05:12:56.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.676+0000 7f28a1237640 1 -- 172.21.15.142:0/2628933993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289c19b590 con 0x7f289c1962f0 2024-03-29T05:12:56.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.677+0000 7f2883fff640 1 -- 172.21.15.142:0/2628933993 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f289000f450 con 0x7f289c1962f0 2024-03-29T05:12:56.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.677+0000 7f2883fff640 1 --2- 172.21.15.142:0/2628933993 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f288406dcd0 0x7f2884070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:56.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.677+0000 7f2883fff640 1 -- 172.21.15.142:0/2628933993 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f289008deb0 con 0x7f289c1962f0 2024-03-29T05:12:56.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.677+0000 7f289ad9d640 1 --2- 172.21.15.142:0/2628933993 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f288406dcd0 0x7f2884070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:56.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.678+0000 7f289ad9d640 1 --2- 172.21.15.142:0/2628933993 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f288406dcd0 0x7f2884070190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2894033000 tx=0x7f2894005f10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:56.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.683+0000 7f2883fff640 1 -- 172.21.15.142:0/2628933993 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f28900572d0 con 0x7f289c1962f0 2024-03-29T05:12:56.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:56 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3372339262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T05:12:56.886 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.880+0000 7f28a1237640 1 -- 172.21.15.142:0/2628933993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f289c197100 con 0x7f289c1962f0 2024-03-29T05:12:56.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.881+0000 7f2883fff640 1 -- 172.21.15.142:0/2628933993 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f289005a990 con 0x7f289c1962f0 2024-03-29T05:12:56.887 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 1 2024-03-29T05:12:56.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.883+0000 7f28a1237640 1 -- 172.21.15.142:0/2628933993 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f288406dcd0 msgr2=0x7f2884070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:56.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.883+0000 7f28a1237640 1 --2- 172.21.15.142:0/2628933993 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f288406dcd0 0x7f2884070190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2894033000 tx=0x7f2894005f10 comp rx=0 tx=0).stop 2024-03-29T05:12:56.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.883+0000 7f28a1237640 1 -- 172.21.15.142:0/2628933993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f289c1962f0 msgr2=0x7f289c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:56.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.883+0000 7f28a1237640 1 --2- 172.21.15.142:0/2628933993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f289c1962f0 0x7f289c19a750 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f289000c920 tx=0x7f289000cde0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.883+0000 7f28a1237640 1 -- 172.21.15.142:0/2628933993 shutdown_connections 2024-03-29T05:12:56.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.883+0000 7f28a1237640 1 --2- 172.21.15.142:0/2628933993 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f288406dcd0 0x7f2884070190 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.883+0000 7f28a1237640 1 --2- 172.21.15.142:0/2628933993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f289c1962f0 0x7f289c19a750 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.883+0000 7f28a1237640 1 --2- 172.21.15.142:0/2628933993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f289c0675c0 0x7f289c195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:56.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.883+0000 7f28a1237640 1 -- 172.21.15.142:0/2628933993 >> 172.21.15.142:0/2628933993 conn(0x7f289c074a50 msgr2=0x7f289c0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:56.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.883+0000 7f28a1237640 1 -- 172.21.15.142:0/2628933993 shutdown_connections 2024-03-29T05:12:56.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:56.883+0000 7f28a1237640 1 -- 172.21.15.142:0/2628933993 wait complete. 2024-03-29T05:12:56.900 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-29T05:12:57.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.000+0000 7f555a85c640 1 -- 172.21.15.142:0/4175237310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100240 msgr2=0x7f5554100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:57.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.000+0000 7f555a85c640 1 --2- 172.21.15.142:0/4175237310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100240 0x7f5554100620 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5544009880 tx=0x7f554402bce0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.000+0000 7f555a85c640 1 -- 172.21.15.142:0/4175237310 shutdown_connections 2024-03-29T05:12:57.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.000+0000 7f555a85c640 1 --2- 172.21.15.142:0/4175237310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5554100bf0 0x7f5554104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.000+0000 7f555a85c640 1 --2- 172.21.15.142:0/4175237310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100240 0x7f5554100620 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.000+0000 7f555a85c640 1 -- 172.21.15.142:0/4175237310 >> 172.21.15.142:0/4175237310 conn(0x7f55540fbae0 msgr2=0x7f55540fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:57.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.000+0000 7f555a85c640 1 -- 172.21.15.142:0/4175237310 shutdown_connections 2024-03-29T05:12:57.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.000+0000 7f555a85c640 1 -- 172.21.15.142:0/4175237310 wait complete. 2024-03-29T05:12:57.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.000+0000 7f555a85c640 1 Processor -- start 2024-03-29T05:12:57.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.000+0000 7f555a85c640 1 -- start start 2024-03-29T05:12:57.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f555a85c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100240 0x7f5554195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:57.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f555a85c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5554196290 0x7f555419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f555a85c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5554196890 con 0x7f5554100240 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f555a85c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5554196a00 con 0x7f5554196290 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f5553fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100240 0x7f5554195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f5553fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100240 0x7f5554195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46262/0 (socket says 172.21.15.142:46262) 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f5553fff640 1 -- 172.21.15.142:0/1709418383 learned_addr learned my addr 172.21.15.142:0/1709418383 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f55537fe640 1 --2- 172.21.15.142:0/1709418383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5554196290 0x7f555419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f5553fff640 1 -- 172.21.15.142:0/1709418383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5554196290 msgr2=0x7f555419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f5553fff640 1 --2- 172.21.15.142:0/1709418383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5554196290 0x7f555419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f5553fff640 1 -- 172.21.15.142:0/1709418383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5544009560 con 0x7f5554100240 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f55537fe640 1 --2- 172.21.15.142:0/1709418383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5554196290 0x7f555419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f5553fff640 1 --2- 172.21.15.142:0/1709418383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100240 0x7f5554195d50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5544009430 tx=0x7f5544005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f55517fa640 1 -- 172.21.15.142:0/1709418383 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55440044a0 con 0x7f5554100240 2024-03-29T05:12:57.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f55517fa640 1 -- 172.21.15.142:0/1709418383 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5544003ab0 con 0x7f5554100240 2024-03-29T05:12:57.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f55517fa640 1 -- 172.21.15.142:0/1709418383 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55440055f0 con 0x7f5554100240 2024-03-29T05:12:57.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f555a85c640 1 -- 172.21.15.142:0/1709418383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555419ac90 con 0x7f5554100240 2024-03-29T05:12:57.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.001+0000 7f555a85c640 1 -- 172.21.15.142:0/1709418383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555419b1f0 con 0x7f5554100240 2024-03-29T05:12:57.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.003+0000 7f555a85c640 1 -- 172.21.15.142:0/1709418383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555404e040 con 0x7f5554100240 2024-03-29T05:12:57.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.003+0000 7f55517fa640 1 -- 172.21.15.142:0/1709418383 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f5544004e60 con 0x7f5554100240 2024-03-29T05:12:57.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.003+0000 7f55517fa640 1 --2- 172.21.15.142:0/1709418383 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f553c06dcd0 0x7f553c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:57.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.004+0000 7f55537fe640 1 --2- 172.21.15.142:0/1709418383 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f553c06dcd0 0x7f553c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:57.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.004+0000 7f55517fa640 1 -- 172.21.15.142:0/1709418383 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f55440b0070 con 0x7f5554100240 2024-03-29T05:12:57.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.004+0000 7f55537fe640 1 --2- 172.21.15.142:0/1709418383 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f553c06dcd0 0x7f553c070190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5554067bd0 tx=0x7f5548005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:57.017 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.011+0000 7f55517fa640 1 -- 172.21.15.142:0/1709418383 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5544079490 con 0x7f5554100240 2024-03-29T05:12:57.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:56 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3372339262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T05:12:57.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.203+0000 7f555a85c640 1 -- 172.21.15.142:0/1709418383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f5554108540 con 0x7f5554100240 2024-03-29T05:12:57.209 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.203+0000 7f55517fa640 1 -- 172.21.15.142:0/1709418383 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v99) v1 ==== 107+0+11 (secure 0 0 0) 0x7f554407cb50 con 0x7f5554100240 2024-03-29T05:12:57.209 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 32 2024-03-29T05:12:57.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.206+0000 7f555a85c640 1 -- 172.21.15.142:0/1709418383 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f553c06dcd0 msgr2=0x7f553c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:57.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.206+0000 7f555a85c640 1 --2- 172.21.15.142:0/1709418383 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f553c06dcd0 0x7f553c070190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5554067bd0 tx=0x7f5548005f90 comp rx=0 tx=0).stop 2024-03-29T05:12:57.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.206+0000 7f555a85c640 1 -- 172.21.15.142:0/1709418383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100240 msgr2=0x7f5554195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:57.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.206+0000 7f555a85c640 1 --2- 172.21.15.142:0/1709418383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100240 0x7f5554195d50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5544009430 tx=0x7f5544005e50 comp rx=0 tx=0).stop 2024-03-29T05:12:57.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.206+0000 7f555a85c640 1 -- 172.21.15.142:0/1709418383 shutdown_connections 2024-03-29T05:12:57.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.206+0000 7f555a85c640 1 --2- 172.21.15.142:0/1709418383 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f553c06dcd0 0x7f553c070190 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.206+0000 7f555a85c640 1 --2- 172.21.15.142:0/1709418383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5554196290 0x7f555419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.206+0000 7f555a85c640 1 --2- 172.21.15.142:0/1709418383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100240 0x7f5554195d50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.206+0000 7f555a85c640 1 -- 172.21.15.142:0/1709418383 >> 172.21.15.142:0/1709418383 conn(0x7f55540fbae0 msgr2=0x7f55540fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:57.213 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.206+0000 7f555a85c640 1 -- 172.21.15.142:0/1709418383 shutdown_connections 2024-03-29T05:12:57.213 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.206+0000 7f555a85c640 1 -- 172.21.15.142:0/1709418383 wait complete. 2024-03-29T05:12:57.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-29T05:12:57.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.328+0000 7f8f48dee640 1 -- 172.21.15.142:0/381508868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f440ff260 msgr2=0x7f8f440ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:57.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.328+0000 7f8f48dee640 1 --2- 172.21.15.142:0/381508868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f440ff260 0x7f8f440ff640 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009880 tx=0x7f8f3402bce0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.328+0000 7f8f48dee640 1 -- 172.21.15.142:0/381508868 shutdown_connections 2024-03-29T05:12:57.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.328+0000 7f8f48dee640 1 --2- 172.21.15.142:0/381508868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f440ffc10 0x7f8f44103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.328+0000 7f8f48dee640 1 --2- 172.21.15.142:0/381508868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f440ff260 0x7f8f440ff640 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.328+0000 7f8f48dee640 1 -- 172.21.15.142:0/381508868 >> 172.21.15.142:0/381508868 conn(0x7f8f44074a50 msgr2=0x7f8f44074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:57.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.328+0000 7f8f48dee640 1 -- 172.21.15.142:0/381508868 shutdown_connections 2024-03-29T05:12:57.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.328+0000 7f8f48dee640 1 -- 172.21.15.142:0/381508868 wait complete. 2024-03-29T05:12:57.335 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.329+0000 7f8f48dee640 1 Processor -- start 2024-03-29T05:12:57.336 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.329+0000 7f8f48dee640 1 -- start start 2024-03-29T05:12:57.336 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.329+0000 7f8f48dee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f440ff260 0x7f8f4419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:57.336 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.329+0000 7f8f48dee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f4419afa0 0x7f8f44194ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:57.336 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.329+0000 7f8f48dee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4419b570 con 0x7f8f440ff260 2024-03-29T05:12:57.336 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.329+0000 7f8f48dee640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44195010 con 0x7f8f4419afa0 2024-03-29T05:12:57.336 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.329+0000 7f8f4259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f440ff260 0x7f8f4419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:57.336 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.329+0000 7f8f4259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f440ff260 0x7f8f4419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46276/0 (socket says 172.21.15.142:46276) 2024-03-29T05:12:57.336 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.329+0000 7f8f4259c640 1 -- 172.21.15.142:0/3418301063 learned_addr learned my addr 172.21.15.142:0/3418301063 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:57.337 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.329+0000 7f8f4259c640 1 -- 172.21.15.142:0/3418301063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f4419afa0 msgr2=0x7f8f44194ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.329+0000 7f8f4259c640 1 --2- 172.21.15.142:0/3418301063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f4419afa0 0x7f8f44194ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.329+0000 7f8f4259c640 1 -- 172.21.15.142:0/3418301063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34009560 con 0x7f8f440ff260 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.330+0000 7f8f4259c640 1 --2- 172.21.15.142:0/3418301063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f440ff260 0x7f8f4419aa60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009230 tx=0x7f8f34004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.330+0000 7f8f337fe640 1 -- 172.21.15.142:0/3418301063 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f340035f0 con 0x7f8f440ff260 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.330+0000 7f8f337fe640 1 -- 172.21.15.142:0/3418301063 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f34003c10 con 0x7f8f440ff260 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.330+0000 7f8f48dee640 1 -- 172.21.15.142:0/3418301063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f44195290 con 0x7f8f440ff260 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.330+0000 7f8f337fe640 1 -- 172.21.15.142:0/3418301063 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f34005670 con 0x7f8f440ff260 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.330+0000 7f8f48dee640 1 -- 172.21.15.142:0/3418301063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f44195790 con 0x7f8f440ff260 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.331+0000 7f8f48dee640 1 -- 172.21.15.142:0/3418301063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4404e040 con 0x7f8f440ff260 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.331+0000 7f8f337fe640 1 -- 172.21.15.142:0/3418301063 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f8f34004dd0 con 0x7f8f440ff260 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.332+0000 7f8f337fe640 1 --2- 172.21.15.142:0/3418301063 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8f2c06dcd0 0x7f8f2c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.332+0000 7f8f337fe640 1 -- 172.21.15.142:0/3418301063 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f8f340affe0 con 0x7f8f440ff260 2024-03-29T05:12:57.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.332+0000 7f8f41d9b640 1 --2- 172.21.15.142:0/3418301063 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8f2c06dcd0 0x7f8f2c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:57.339 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.332+0000 7f8f41d9b640 1 --2- 172.21.15.142:0/3418301063 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8f2c06dcd0 0x7f8f2c070190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8f38005fd0 tx=0x7f8f38005c70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:57.342 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.337+0000 7f8f337fe640 1 -- 172.21.15.142:0/3418301063 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8f34079480 con 0x7f8f440ff260 2024-03-29T05:12:57.521 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.515+0000 7f8f48dee640 1 -- 172.21.15.142:0/3418301063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f8f441960c0 con 0x7f8f440ff260 2024-03-29T05:12:57.522 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.516+0000 7f8f337fe640 1 -- 172.21.15.142:0/3418301063 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v99) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8f34041020 con 0x7f8f440ff260 2024-03-29T05:12:57.522 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 32 2024-03-29T05:12:57.525 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.518+0000 7f8f48dee640 1 -- 172.21.15.142:0/3418301063 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8f2c06dcd0 msgr2=0x7f8f2c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:57.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.518+0000 7f8f48dee640 1 --2- 172.21.15.142:0/3418301063 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8f2c06dcd0 0x7f8f2c070190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8f38005fd0 tx=0x7f8f38005c70 comp rx=0 tx=0).stop 2024-03-29T05:12:57.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.518+0000 7f8f48dee640 1 -- 172.21.15.142:0/3418301063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f440ff260 msgr2=0x7f8f4419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:57.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.518+0000 7f8f48dee640 1 --2- 172.21.15.142:0/3418301063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f440ff260 0x7f8f4419aa60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009230 tx=0x7f8f34004040 comp rx=0 tx=0).stop 2024-03-29T05:12:57.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.518+0000 7f8f48dee640 1 -- 172.21.15.142:0/3418301063 shutdown_connections 2024-03-29T05:12:57.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.518+0000 7f8f48dee640 1 --2- 172.21.15.142:0/3418301063 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f8f2c06dcd0 0x7f8f2c070190 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.518+0000 7f8f48dee640 1 --2- 172.21.15.142:0/3418301063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f4419afa0 0x7f8f44194ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.518+0000 7f8f48dee640 1 --2- 172.21.15.142:0/3418301063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f440ff260 0x7f8f4419aa60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.518+0000 7f8f48dee640 1 -- 172.21.15.142:0/3418301063 >> 172.21.15.142:0/3418301063 conn(0x7f8f44074a50 msgr2=0x7f8f440fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:57.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.518+0000 7f8f48dee640 1 -- 172.21.15.142:0/3418301063 shutdown_connections 2024-03-29T05:12:57.526 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.518+0000 7f8f48dee640 1 -- 172.21.15.142:0/3418301063 wait complete. 2024-03-29T05:12:57.538 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T05:12:57.647 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.636+0000 7f2baf084640 1 -- 172.21.15.142:0/2801276697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ba80ffb30 msgr2=0x7f2ba8103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:57.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.636+0000 7f2baf084640 1 --2- 172.21.15.142:0/2801276697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ba80ffb30 0x7f2ba8103b70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009880 tx=0x7f2b9402bce0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.636+0000 7f2baf084640 1 -- 172.21.15.142:0/2801276697 shutdown_connections 2024-03-29T05:12:57.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.636+0000 7f2baf084640 1 --2- 172.21.15.142:0/2801276697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ba80ffb30 0x7f2ba8103b70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.636+0000 7f2baf084640 1 --2- 172.21.15.142:0/2801276697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba80ff180 0x7f2ba80ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.636+0000 7f2baf084640 1 -- 172.21.15.142:0/2801276697 >> 172.21.15.142:0/2801276697 conn(0x7f2ba80faa00 msgr2=0x7f2ba80fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:57.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.636+0000 7f2baf084640 1 -- 172.21.15.142:0/2801276697 shutdown_connections 2024-03-29T05:12:57.648 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.636+0000 7f2baf084640 1 -- 172.21.15.142:0/2801276697 wait complete. 2024-03-29T05:12:57.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.636+0000 7f2baf084640 1 Processor -- start 2024-03-29T05:12:57.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2baf084640 1 -- start start 2024-03-29T05:12:57.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2baf084640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ba80ff180 0x7f2ba8195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:57.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2baf084640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba8196220 0x7f2ba81995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:57.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2baf084640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba8196790 con 0x7f2ba80ff180 2024-03-29T05:12:57.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2baf084640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba81968d0 con 0x7f2ba8196220 2024-03-29T05:12:57.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2bace20640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ba80ff180 0x7f2ba8195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:57.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2bace20640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ba80ff180 0x7f2ba8195ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46286/0 (socket says 172.21.15.142:46286) 2024-03-29T05:12:57.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2bace20640 1 -- 172.21.15.142:0/3544426448 learned_addr learned my addr 172.21.15.142:0/3544426448 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:57.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2ba7fff640 1 --2- 172.21.15.142:0/3544426448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba8196220 0x7f2ba81995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:57.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2bace20640 1 -- 172.21.15.142:0/3544426448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba8196220 msgr2=0x7f2ba81995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:57.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2bace20640 1 --2- 172.21.15.142:0/3544426448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba8196220 0x7f2ba81995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2bace20640 1 -- 172.21.15.142:0/3544426448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b94009560 con 0x7f2ba80ff180 2024-03-29T05:12:57.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2bace20640 1 --2- 172.21.15.142:0/3544426448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ba80ff180 0x7f2ba8195ce0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00c8a0 tx=0x7f2b9c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:57.651 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2ba5ffb640 1 -- 172.21.15.142:0/3544426448 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b9c007b80 con 0x7f2ba80ff180 2024-03-29T05:12:57.651 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.637+0000 7f2ba5ffb640 1 -- 172.21.15.142:0/3544426448 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b9c007ce0 con 0x7f2ba80ff180 2024-03-29T05:12:57.651 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.638+0000 7f2baf084640 1 -- 172.21.15.142:0/3544426448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba8199b10 con 0x7f2ba80ff180 2024-03-29T05:12:57.651 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.638+0000 7f2ba5ffb640 1 -- 172.21.15.142:0/3544426448 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b9c018810 con 0x7f2ba80ff180 2024-03-29T05:12:57.651 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.638+0000 7f2baf084640 1 -- 172.21.15.142:0/3544426448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba819a070 con 0x7f2ba80ff180 2024-03-29T05:12:57.651 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.639+0000 7f2baf084640 1 -- 172.21.15.142:0/3544426448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba804e040 con 0x7f2ba80ff180 2024-03-29T05:12:57.651 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.640+0000 7f2ba5ffb640 1 -- 172.21.15.142:0/3544426448 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f2b9c018a80 con 0x7f2ba80ff180 2024-03-29T05:12:57.651 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.640+0000 7f2ba5ffb640 1 --2- 172.21.15.142:0/3544426448 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2b9806dd90 0x7f2b98070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:57.651 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.640+0000 7f2ba5ffb640 1 -- 172.21.15.142:0/3544426448 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f2b9c08df40 con 0x7f2ba80ff180 2024-03-29T05:12:57.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.640+0000 7f2ba7fff640 1 --2- 172.21.15.142:0/3544426448 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2b9806dd90 0x7f2b98070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:57.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.641+0000 7f2ba7fff640 1 --2- 172.21.15.142:0/3544426448 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2b9806dd90 0x7f2b98070250 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2b94036040 tx=0x7f2b94005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:57.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.645+0000 7f2ba5ffb640 1 -- 172.21.15.142:0/3544426448 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2b9c0c2280 con 0x7f2ba80ff180 2024-03-29T05:12:57.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.919+0000 7f2baf084640 1 -- 172.21.15.142:0/3544426448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ba80008d0 con 0x7f2ba80ff180 2024-03-29T05:12:57.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.919+0000 7f2ba5ffb640 1 -- 172.21.15.142:0/3544426448 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5257 (secure 0 0 0) 0x7f2b9c057360 con 0x7f2ba80ff180 2024-03-29T05:12:57.926 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:12:57.926 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T05:11:04.422493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":0,"incarnation":33,"state":"up:active","state_seq":245,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi142.bmkjzf","rank":1,"incarnation":33,"state":"up:active","state_seq":241,"addr":"172.21.15.142:6829/1735570051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1735570051},{"type":"v1","addr":"172.21.15.142:6829","nonce":1735570051}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44323":{"gid":44323,"name":"cephfs.smithi152.uhebsi","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6825/778282077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":778282077},{"type":"v1","addr":"172.21.15.152:6825","nonce":778282077}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44327":{"gid":44327,"name":"cephfs.smithi152.hiilum","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6827/2852002359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2852002359},{"type":"v1","addr":"172.21.15.152:6827","nonce":2852002359}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34420,"qdb_cluster":[34424,34420]},"id":1}]} 2024-03-29T05:12:57.926 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 39 2024-03-29T05:12:57.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.922+0000 7f2baf084640 1 -- 172.21.15.142:0/3544426448 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2b9806dd90 msgr2=0x7f2b98070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:57.928 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.922+0000 7f2baf084640 1 --2- 172.21.15.142:0/3544426448 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2b9806dd90 0x7f2b98070250 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2b94036040 tx=0x7f2b94005fd0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.922+0000 7f2baf084640 1 -- 172.21.15.142:0/3544426448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ba80ff180 msgr2=0x7f2ba8195ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:57.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.922+0000 7f2baf084640 1 --2- 172.21.15.142:0/3544426448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ba80ff180 0x7f2ba8195ce0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00c8a0 tx=0x7f2b9c00cd60 comp rx=0 tx=0).stop 2024-03-29T05:12:57.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.922+0000 7f2baf084640 1 -- 172.21.15.142:0/3544426448 shutdown_connections 2024-03-29T05:12:57.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.922+0000 7f2baf084640 1 --2- 172.21.15.142:0/3544426448 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f2b9806dd90 0x7f2b98070250 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.922+0000 7f2baf084640 1 --2- 172.21.15.142:0/3544426448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba8196220 0x7f2ba81995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.922+0000 7f2baf084640 1 --2- 172.21.15.142:0/3544426448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ba80ff180 0x7f2ba8195ce0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:57.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.922+0000 7f2baf084640 1 -- 172.21.15.142:0/3544426448 >> 172.21.15.142:0/3544426448 conn(0x7f2ba80faa00 msgr2=0x7f2ba80683f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:57.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.923+0000 7f2baf084640 1 -- 172.21.15.142:0/3544426448 shutdown_connections 2024-03-29T05:12:57.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:57.923+0000 7f2baf084640 1 -- 172.21.15.142:0/3544426448 wait complete. 2024-03-29T05:12:57.940 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 50} 2024-03-29T05:12:57.940 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-03-29T05:12:58.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.039+0000 7f2180c15640 1 -- 172.21.15.142:0/1211065519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c0ffc10 msgr2=0x7f217c103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:58.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.039+0000 7f2180c15640 1 --2- 172.21.15.142:0/1211065519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c0ffc10 0x7f217c103b70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2170009880 tx=0x7f217002bce0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.040+0000 7f2180c15640 1 -- 172.21.15.142:0/1211065519 shutdown_connections 2024-03-29T05:12:58.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.040+0000 7f2180c15640 1 --2- 172.21.15.142:0/1211065519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c0ffc10 0x7f217c103b70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.040+0000 7f2180c15640 1 --2- 172.21.15.142:0/1211065519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c0ff260 0x7f217c0ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.040+0000 7f2180c15640 1 -- 172.21.15.142:0/1211065519 >> 172.21.15.142:0/1211065519 conn(0x7f217c074a50 msgr2=0x7f217c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:58.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.040+0000 7f2180c15640 1 -- 172.21.15.142:0/1211065519 shutdown_connections 2024-03-29T05:12:58.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.040+0000 7f2180c15640 1 -- 172.21.15.142:0/1211065519 wait complete. 2024-03-29T05:12:58.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.040+0000 7f2180c15640 1 Processor -- start 2024-03-29T05:12:58.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.040+0000 7f2180c15640 1 -- start start 2024-03-29T05:12:58.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f2180c15640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c0ff260 0x7f217c196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:58.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f2180c15640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c197090 0x7f217c1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:58.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f2180c15640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c19fc50 con 0x7f217c0ff260 2024-03-29T05:12:58.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f2180c15640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c19fd90 con 0x7f217c197090 2024-03-29T05:12:58.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f217a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c0ff260 0x7f217c196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:58.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f217a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c0ff260 0x7f217c196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46302/0 (socket says 172.21.15.142:46302) 2024-03-29T05:12:58.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f217a59c640 1 -- 172.21.15.142:0/749923415 learned_addr learned my addr 172.21.15.142:0/749923415 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:58.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f2179d9b640 1 --2- 172.21.15.142:0/749923415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c197090 0x7f217c1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:58.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f217a59c640 1 -- 172.21.15.142:0/749923415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c197090 msgr2=0x7f217c1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:58.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f217a59c640 1 --2- 172.21.15.142:0/749923415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c197090 0x7f217c1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f217a59c640 1 -- 172.21.15.142:0/749923415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2170009560 con 0x7f217c0ff260 2024-03-29T05:12:58.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f2179d9b640 1 --2- 172.21.15.142:0/749923415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c197090 0x7f217c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:12:58.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f217a59c640 1 --2- 172.21.15.142:0/749923415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c0ff260 0x7f217c196b50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f216c00c920 tx=0x7f216c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:58.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f216b7fe640 1 -- 172.21.15.142:0/749923415 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f216c007b80 con 0x7f217c0ff260 2024-03-29T05:12:58.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.041+0000 7f2180c15640 1 -- 172.21.15.142:0/749923415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c19ff90 con 0x7f217c0ff260 2024-03-29T05:12:58.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.042+0000 7f216b7fe640 1 -- 172.21.15.142:0/749923415 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f216c0054e0 con 0x7f217c0ff260 2024-03-29T05:12:58.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.042+0000 7f216b7fe640 1 -- 172.21.15.142:0/749923415 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f216c00f450 con 0x7f217c0ff260 2024-03-29T05:12:58.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.042+0000 7f2180c15640 1 -- 172.21.15.142:0/749923415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c1a0410 con 0x7f217c0ff260 2024-03-29T05:12:58.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:57 smithi152 ceph-mon[88186]: pgmap v740: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:12:58.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:57 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2628933993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T05:12:58.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:57 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1709418383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T05:12:58.051 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:57 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3418301063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T05:12:58.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.043+0000 7f2180c15640 1 -- 172.21.15.142:0/749923415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c04e040 con 0x7f217c0ff260 2024-03-29T05:12:58.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.044+0000 7f216b7fe640 1 -- 172.21.15.142:0/749923415 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f216c004f80 con 0x7f217c0ff260 2024-03-29T05:12:58.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.044+0000 7f216b7fe640 1 --2- 172.21.15.142:0/749923415 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f216406dcd0 0x7f2164070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:58.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.044+0000 7f216b7fe640 1 -- 172.21.15.142:0/749923415 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f216c08dab0 con 0x7f217c0ff260 2024-03-29T05:12:58.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.044+0000 7f2179d9b640 1 --2- 172.21.15.142:0/749923415 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f216406dcd0 0x7f2164070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:58.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.045+0000 7f2179d9b640 1 --2- 172.21.15.142:0/749923415 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f216406dcd0 0x7f2164070190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2170006010 tx=0x7f217002cc00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:58.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.049+0000 7f216b7fe640 1 -- 172.21.15.142:0/749923415 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f216c056ed0 con 0x7f217c0ff260 2024-03-29T05:12:58.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:57 smithi142 ceph-mon[112475]: pgmap v740: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:12:58.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:57 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2628933993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T05:12:58.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:57 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1709418383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T05:12:58.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:57 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3418301063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T05:12:58.275 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.269+0000 7f2180c15640 1 -- 172.21.15.142:0/749923415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f217c107480 con 0x7f217c0ff260 2024-03-29T05:12:58.276 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.270+0000 7f216b7fe640 1 -- 172.21.15.142:0/749923415 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v39) v1 ==== 107+0+4898 (secure 0 0 0) 0x7f216c05a590 con 0x7f217c0ff260 2024-03-29T05:12:58.276 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:12:58.277 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:44:10.715853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14494,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14494":{"gid":14494,"name":"cephfs.smithi142.bmkjzf","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.142:6829/1941073852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1941073852},{"type":"v1","addr":"172.21.15.142:6829","nonce":1941073852}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14506":{"gid":14506,"name":"cephfs.smithi142.wgmndr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6827/1415100534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1415100534},{"type":"v1","addr":"172.21.15.142:6827","nonce":1415100534}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi152.uhebsi","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.152:6825/832699894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":832699894},{"type":"v1","addr":"172.21.15.152:6825","nonce":832699894}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24287":{"gid":24287,"name":"cephfs.smithi152.hiilum","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.152:6827/3411697908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3411697908},{"type":"v1","addr":"172.21.15.152:6827","nonce":3411697908}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:12:58.277 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 18 2024-03-29T05:12:58.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.273+0000 7f2180c15640 1 -- 172.21.15.142:0/749923415 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f216406dcd0 msgr2=0x7f2164070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:58.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.273+0000 7f2180c15640 1 --2- 172.21.15.142:0/749923415 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f216406dcd0 0x7f2164070190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2170006010 tx=0x7f217002cc00 comp rx=0 tx=0).stop 2024-03-29T05:12:58.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.273+0000 7f2180c15640 1 -- 172.21.15.142:0/749923415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c0ff260 msgr2=0x7f217c196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:58.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.273+0000 7f2180c15640 1 --2- 172.21.15.142:0/749923415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c0ff260 0x7f217c196b50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f216c00c920 tx=0x7f216c00cde0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.273+0000 7f2180c15640 1 -- 172.21.15.142:0/749923415 shutdown_connections 2024-03-29T05:12:58.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.273+0000 7f2180c15640 1 --2- 172.21.15.142:0/749923415 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f216406dcd0 0x7f2164070190 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.273+0000 7f2180c15640 1 --2- 172.21.15.142:0/749923415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c197090 0x7f217c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.280 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.273+0000 7f2180c15640 1 --2- 172.21.15.142:0/749923415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c0ff260 0x7f217c196b50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.273+0000 7f2180c15640 1 -- 172.21.15.142:0/749923415 >> 172.21.15.142:0/749923415 conn(0x7f217c074a50 msgr2=0x7f217c0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:58.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.273+0000 7f2180c15640 1 -- 172.21.15.142:0/749923415 shutdown_connections 2024-03-29T05:12:58.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.273+0000 7f2180c15640 1 -- 172.21.15.142:0/749923415 wait complete. 2024-03-29T05:12:58.295 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-29T05:12:58.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.398+0000 7f5143727640 1 -- 172.21.15.142:0/1343069448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513c106070 msgr2=0x7f513c106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:58.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.398+0000 7f5143727640 1 --2- 172.21.15.142:0/1343069448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513c106070 0x7f513c106450 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5138009880 tx=0x7f513802bce0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.398+0000 7f5143727640 1 -- 172.21.15.142:0/1343069448 shutdown_connections 2024-03-29T05:12:58.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.398+0000 7f5143727640 1 --2- 172.21.15.142:0/1343069448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f513c0fe730 0x7f513c0feb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.398+0000 7f5143727640 1 --2- 172.21.15.142:0/1343069448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513c106070 0x7f513c106450 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.398+0000 7f5143727640 1 -- 172.21.15.142:0/1343069448 >> 172.21.15.142:0/1343069448 conn(0x7f513c074250 msgr2=0x7f513c074660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:58.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.398+0000 7f5143727640 1 -- 172.21.15.142:0/1343069448 shutdown_connections 2024-03-29T05:12:58.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.398+0000 7f5143727640 1 -- 172.21.15.142:0/1343069448 wait complete. 2024-03-29T05:12:58.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5143727640 1 Processor -- start 2024-03-29T05:12:58.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5143727640 1 -- start start 2024-03-29T05:12:58.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5143727640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f513c0fe730 0x7f513c195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:58.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5143727640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513c106070 0x7f513c196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:58.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5143727640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c196910 con 0x7f513c106070 2024-03-29T05:12:58.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5143727640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c19a6e0 con 0x7f513c0fe730 2024-03-29T05:12:58.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5140cc2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513c106070 0x7f513c196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:58.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5140cc2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513c106070 0x7f513c196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46322/0 (socket says 172.21.15.142:46322) 2024-03-29T05:12:58.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5140cc2640 1 -- 172.21.15.142:0/3023379769 learned_addr learned my addr 172.21.15.142:0/3023379769 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:58.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f51414c3640 1 --2- 172.21.15.142:0/3023379769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f513c0fe730 0x7f513c195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:58.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5140cc2640 1 -- 172.21.15.142:0/3023379769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f513c0fe730 msgr2=0x7f513c195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:58.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5140cc2640 1 --2- 172.21.15.142:0/3023379769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f513c0fe730 0x7f513c195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5140cc2640 1 -- 172.21.15.142:0/3023379769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5138009560 con 0x7f513c106070 2024-03-29T05:12:58.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.399+0000 7f5140cc2640 1 --2- 172.21.15.142:0/3023379769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513c106070 0x7f513c196280 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f512c00d8d0 tx=0x7f512c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:58.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.400+0000 7f51327fc640 1 -- 172.21.15.142:0/3023379769 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f512c00bea0 con 0x7f513c106070 2024-03-29T05:12:58.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.400+0000 7f51327fc640 1 -- 172.21.15.142:0/3023379769 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f512c009c00 con 0x7f513c106070 2024-03-29T05:12:58.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.400+0000 7f51327fc640 1 -- 172.21.15.142:0/3023379769 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f512c00f710 con 0x7f513c106070 2024-03-29T05:12:58.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.400+0000 7f5143727640 1 -- 172.21.15.142:0/3023379769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513c19a9c0 con 0x7f513c106070 2024-03-29T05:12:58.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.400+0000 7f5143727640 1 -- 172.21.15.142:0/3023379769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c19af20 con 0x7f513c106070 2024-03-29T05:12:58.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.401+0000 7f5143727640 1 -- 172.21.15.142:0/3023379769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513c04e040 con 0x7f513c106070 2024-03-29T05:12:58.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.401+0000 7f51327fc640 1 -- 172.21.15.142:0/3023379769 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f512c010460 con 0x7f513c106070 2024-03-29T05:12:58.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.402+0000 7f51327fc640 1 --2- 172.21.15.142:0/3023379769 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f512806dcd0 0x7f5128070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:58.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.402+0000 7f51327fc640 1 -- 172.21.15.142:0/3023379769 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f512c08d940 con 0x7f513c106070 2024-03-29T05:12:58.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.402+0000 7f51414c3640 1 --2- 172.21.15.142:0/3023379769 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f512806dcd0 0x7f5128070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:58.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.402+0000 7f51414c3640 1 --2- 172.21.15.142:0/3023379769 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f512806dcd0 0x7f5128070190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5138036040 tx=0x7f5138005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:58.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.410+0000 7f51327fc640 1 -- 172.21.15.142:0/3023379769 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f512c056de0 con 0x7f513c106070 2024-03-29T05:12:58.639 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.633+0000 7f5143727640 1 -- 172.21.15.142:0/3023379769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f513c108560 con 0x7f513c106070 2024-03-29T05:12:58.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.634+0000 7f51327fc640 1 -- 172.21.15.142:0/3023379769 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v39) v1 ==== 107+0+4899 (secure 0 0 0) 0x7f512c05a4a0 con 0x7f513c106070 2024-03-29T05:12:58.640 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:12:58.641 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:45:01.724489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14494,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14494":{"gid":14494,"name":"cephfs.smithi142.bmkjzf","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.142:6829/1941073852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1941073852},{"type":"v1","addr":"172.21.15.142:6829","nonce":1941073852}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14506":{"gid":14506,"name":"cephfs.smithi142.wgmndr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6827/1415100534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1415100534},{"type":"v1","addr":"172.21.15.142:6827","nonce":1415100534}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi152.uhebsi","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.152:6825/832699894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":832699894},{"type":"v1","addr":"172.21.15.152:6825","nonce":832699894}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24287":{"gid":24287,"name":"cephfs.smithi152.hiilum","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.152:6827/3411697908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3411697908},{"type":"v1","addr":"172.21.15.152:6827","nonce":3411697908}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:12:58.642 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 19 2024-03-29T05:12:58.642 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.636+0000 7f5143727640 1 -- 172.21.15.142:0/3023379769 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f512806dcd0 msgr2=0x7f5128070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:58.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.636+0000 7f5143727640 1 --2- 172.21.15.142:0/3023379769 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f512806dcd0 0x7f5128070190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5138036040 tx=0x7f5138005fd0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.636+0000 7f5143727640 1 -- 172.21.15.142:0/3023379769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513c106070 msgr2=0x7f513c196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:58.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.636+0000 7f5143727640 1 --2- 172.21.15.142:0/3023379769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513c106070 0x7f513c196280 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f512c00d8d0 tx=0x7f512c00dd90 comp rx=0 tx=0).stop 2024-03-29T05:12:58.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.636+0000 7f5143727640 1 -- 172.21.15.142:0/3023379769 shutdown_connections 2024-03-29T05:12:58.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.636+0000 7f5143727640 1 --2- 172.21.15.142:0/3023379769 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f512806dcd0 0x7f5128070190 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.636+0000 7f5143727640 1 --2- 172.21.15.142:0/3023379769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513c106070 0x7f513c196280 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.636+0000 7f5143727640 1 --2- 172.21.15.142:0/3023379769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f513c0fe730 0x7f513c195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.636+0000 7f5143727640 1 -- 172.21.15.142:0/3023379769 >> 172.21.15.142:0/3023379769 conn(0x7f513c074250 msgr2=0x7f513c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:58.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.636+0000 7f5143727640 1 -- 172.21.15.142:0/3023379769 shutdown_connections 2024-03-29T05:12:58.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.636+0000 7f5143727640 1 -- 172.21.15.142:0/3023379769 wait complete. 2024-03-29T05:12:58.657 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-29T05:12:58.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.759+0000 7f68903d2640 1 -- 172.21.15.142:0/2721048543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68881002b0 msgr2=0x7f6888100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:58.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.759+0000 7f68903d2640 1 --2- 172.21.15.142:0/2721048543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68881002b0 0x7f6888100690 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6884009880 tx=0x7f688402bce0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.759+0000 7f68903d2640 1 -- 172.21.15.142:0/2721048543 shutdown_connections 2024-03-29T05:12:58.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.759+0000 7f68903d2640 1 --2- 172.21.15.142:0/2721048543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6888100c60 0x7f6888104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.759+0000 7f68903d2640 1 --2- 172.21.15.142:0/2721048543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68881002b0 0x7f6888100690 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.759+0000 7f68903d2640 1 -- 172.21.15.142:0/2721048543 >> 172.21.15.142:0/2721048543 conn(0x7f68880fbb30 msgr2=0x7f68880fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.759+0000 7f68903d2640 1 -- 172.21.15.142:0/2721048543 shutdown_connections 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.759+0000 7f68903d2640 1 -- 172.21.15.142:0/2721048543 wait complete. 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.760+0000 7f68903d2640 1 Processor -- start 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.760+0000 7f68903d2640 1 -- start start 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.760+0000 7f68903d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68881002b0 0x7f68880725a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.760+0000 7f68903d2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6888072ae0 0x7f6888072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.760+0000 7f68903d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881a1f60 con 0x7f68881002b0 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.760+0000 7f68903d2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881a20a0 con 0x7f6888072ae0 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.760+0000 7f688e16e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68881002b0 0x7f68880725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.760+0000 7f688e16e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68881002b0 0x7f68880725a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46872/0 (socket says 172.21.15.142:46872) 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.760+0000 7f688e16e640 1 -- 172.21.15.142:0/1568054299 learned_addr learned my addr 172.21.15.142:0/1568054299 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.760+0000 7f688d96d640 1 --2- 172.21.15.142:0/1568054299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6888072ae0 0x7f6888072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:58.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.761+0000 7f688e16e640 1 -- 172.21.15.142:0/1568054299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6888072ae0 msgr2=0x7f6888072f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:58.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.761+0000 7f688e16e640 1 --2- 172.21.15.142:0/1568054299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6888072ae0 0x7f6888072f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.761+0000 7f688e16e640 1 -- 172.21.15.142:0/1568054299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6884009560 con 0x7f68881002b0 2024-03-29T05:12:58.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.761+0000 7f688e16e640 1 --2- 172.21.15.142:0/1568054299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68881002b0 0x7f68880725a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6884009430 tx=0x7f6884004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:58.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.761+0000 7f687f7fe640 1 -- 172.21.15.142:0/1568054299 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68840035f0 con 0x7f68881002b0 2024-03-29T05:12:58.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.761+0000 7f68903d2640 1 -- 172.21.15.142:0/1568054299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68881a21e0 con 0x7f68881002b0 2024-03-29T05:12:58.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.761+0000 7f687f7fe640 1 -- 172.21.15.142:0/1568054299 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6884003c10 con 0x7f68881002b0 2024-03-29T05:12:58.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.761+0000 7f687f7fe640 1 -- 172.21.15.142:0/1568054299 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6884005670 con 0x7f68881002b0 2024-03-29T05:12:58.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.761+0000 7f68903d2640 1 -- 172.21.15.142:0/1568054299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68881a2690 con 0x7f68881002b0 2024-03-29T05:12:58.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.762+0000 7f68903d2640 1 -- 172.21.15.142:0/1568054299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f688804e040 con 0x7f68881002b0 2024-03-29T05:12:58.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.763+0000 7f687f7fe640 1 -- 172.21.15.142:0/1568054299 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6884004dd0 con 0x7f68881002b0 2024-03-29T05:12:58.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.763+0000 7f687f7fe640 1 --2- 172.21.15.142:0/1568054299 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f687406dc80 0x7f6874070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:58.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.763+0000 7f687f7fe640 1 -- 172.21.15.142:0/1568054299 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f68840affd0 con 0x7f68881002b0 2024-03-29T05:12:58.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.763+0000 7f688d96d640 1 --2- 172.21.15.142:0/1568054299 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f687406dc80 0x7f6874070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:58.771 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.764+0000 7f688d96d640 1 --2- 172.21.15.142:0/1568054299 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f687406dc80 0x7f6874070140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6878005fd0 tx=0x7f687800b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:58.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.772+0000 7f687f7fe640 1 -- 172.21.15.142:0/1568054299 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f68840794a0 con 0x7f68881002b0 2024-03-29T05:12:58.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.990+0000 7f68903d2640 1 -- 172.21.15.142:0/1568054299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f68881085b0 con 0x7f68881002b0 2024-03-29T05:12:58.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.990+0000 7f687f7fe640 1 -- 172.21.15.142:0/1568054299 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v39) v1 ==== 107+0+4898 (secure 0 0 0) 0x7f688407cb60 con 0x7f68881002b0 2024-03-29T05:12:58.996 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:12:58.996 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:48:42.222447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14494,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14494":{"gid":14494,"name":"cephfs.smithi142.bmkjzf","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.142:6829/1941073852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1941073852},{"type":"v1","addr":"172.21.15.142:6829","nonce":1941073852}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14506":{"gid":14506,"name":"cephfs.smithi142.wgmndr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6827/1415100534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1415100534},{"type":"v1","addr":"172.21.15.142:6827","nonce":1415100534}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi152.uhebsi","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.152:6825/832699894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":832699894},{"type":"v1","addr":"172.21.15.152:6825","nonce":832699894}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24287":{"gid":24287,"name":"cephfs.smithi152.hiilum","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.152:6827/3411697908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3411697908},{"type":"v1","addr":"172.21.15.152:6827","nonce":3411697908}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:12:58.996 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 20 2024-03-29T05:12:58.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.992+0000 7f68903d2640 1 -- 172.21.15.142:0/1568054299 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f687406dc80 msgr2=0x7f6874070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:58.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.992+0000 7f68903d2640 1 --2- 172.21.15.142:0/1568054299 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f687406dc80 0x7f6874070140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6878005fd0 tx=0x7f687800b390 comp rx=0 tx=0).stop 2024-03-29T05:12:58.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.992+0000 7f68903d2640 1 -- 172.21.15.142:0/1568054299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68881002b0 msgr2=0x7f68880725a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:58.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.992+0000 7f68903d2640 1 --2- 172.21.15.142:0/1568054299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68881002b0 0x7f68880725a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6884009430 tx=0x7f6884004040 comp rx=0 tx=0).stop 2024-03-29T05:12:58.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.992+0000 7f68903d2640 1 -- 172.21.15.142:0/1568054299 shutdown_connections 2024-03-29T05:12:58.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.992+0000 7f68903d2640 1 --2- 172.21.15.142:0/1568054299 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f687406dc80 0x7f6874070140 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.992+0000 7f68903d2640 1 --2- 172.21.15.142:0/1568054299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6888072ae0 0x7f6888072f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.992+0000 7f68903d2640 1 --2- 172.21.15.142:0/1568054299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68881002b0 0x7f68880725a0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:58.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.992+0000 7f68903d2640 1 -- 172.21.15.142:0/1568054299 >> 172.21.15.142:0/1568054299 conn(0x7f68880fbb30 msgr2=0x7f68880fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:58.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.993+0000 7f68903d2640 1 -- 172.21.15.142:0/1568054299 shutdown_connections 2024-03-29T05:12:58.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:58.993+0000 7f68903d2640 1 -- 172.21.15.142:0/1568054299 wait complete. 2024-03-29T05:12:59.008 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:58 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3544426448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-29T05:12:59.008 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:58 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/749923415' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-29T05:12:59.008 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:58 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3023379769' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-29T05:12:59.010 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-29T05:12:59.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.108+0000 7f1e60a51640 1 -- 172.21.15.142:0/2989679838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e5c0ff180 msgr2=0x7f1e5c0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:59.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.108+0000 7f1e60a51640 1 --2- 172.21.15.142:0/2989679838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e5c0ff180 0x7f1e5c0ff560 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009880 tx=0x7f1e4c02bce0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.109+0000 7f1e60a51640 1 -- 172.21.15.142:0/2989679838 shutdown_connections 2024-03-29T05:12:59.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.109+0000 7f1e60a51640 1 --2- 172.21.15.142:0/2989679838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e5c0ffb30 0x7f1e5c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.109+0000 7f1e60a51640 1 --2- 172.21.15.142:0/2989679838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e5c0ff180 0x7f1e5c0ff560 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.109+0000 7f1e60a51640 1 -- 172.21.15.142:0/2989679838 >> 172.21.15.142:0/2989679838 conn(0x7f1e5c0faa00 msgr2=0x7f1e5c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:59.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.109+0000 7f1e60a51640 1 -- 172.21.15.142:0/2989679838 shutdown_connections 2024-03-29T05:12:59.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.109+0000 7f1e60a51640 1 -- 172.21.15.142:0/2989679838 wait complete. 2024-03-29T05:12:59.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.109+0000 7f1e60a51640 1 Processor -- start 2024-03-29T05:12:59.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.109+0000 7f1e60a51640 1 -- start start 2024-03-29T05:12:59.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e60a51640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e5c0ff180 0x7f1e5c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:59.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e60a51640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e5c1962f0 0x7f1e5c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:59.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e60a51640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1968f0 con 0x7f1e5c0ff180 2024-03-29T05:12:59.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e60a51640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c196a60 con 0x7f1e5c1962f0 2024-03-29T05:12:59.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e5a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e5c0ff180 0x7f1e5c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:59.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e5a59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e5c0ff180 0x7f1e5c195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46888/0 (socket says 172.21.15.142:46888) 2024-03-29T05:12:59.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e5a59c640 1 -- 172.21.15.142:0/2640467422 learned_addr learned my addr 172.21.15.142:0/2640467422 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:59.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e59d9b640 1 --2- 172.21.15.142:0/2640467422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e5c1962f0 0x7f1e5c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:59.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e5a59c640 1 -- 172.21.15.142:0/2640467422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e5c1962f0 msgr2=0x7f1e5c19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:59.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e5a59c640 1 --2- 172.21.15.142:0/2640467422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e5c1962f0 0x7f1e5c19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e5a59c640 1 -- 172.21.15.142:0/2640467422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c009560 con 0x7f1e5c0ff180 2024-03-29T05:12:59.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e5a59c640 1 --2- 172.21.15.142:0/2640467422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e5c0ff180 0x7f1e5c195db0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009430 tx=0x7f1e4c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:59.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.110+0000 7f1e4b7fe640 1 -- 172.21.15.142:0/2640467422 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e4c0035f0 con 0x7f1e5c0ff180 2024-03-29T05:12:59.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.111+0000 7f1e4b7fe640 1 -- 172.21.15.142:0/2640467422 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e4c003c10 con 0x7f1e5c0ff180 2024-03-29T05:12:59.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.111+0000 7f1e60a51640 1 -- 172.21.15.142:0/2640467422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c19acf0 con 0x7f1e5c0ff180 2024-03-29T05:12:59.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.111+0000 7f1e4b7fe640 1 -- 172.21.15.142:0/2640467422 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e4c005670 con 0x7f1e5c0ff180 2024-03-29T05:12:59.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.111+0000 7f1e60a51640 1 -- 172.21.15.142:0/2640467422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c19b250 con 0x7f1e5c0ff180 2024-03-29T05:12:59.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.112+0000 7f1e60a51640 1 -- 172.21.15.142:0/2640467422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c04e040 con 0x7f1e5c0ff180 2024-03-29T05:12:59.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.112+0000 7f1e4b7fe640 1 -- 172.21.15.142:0/2640467422 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f1e4c003750 con 0x7f1e5c0ff180 2024-03-29T05:12:59.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.113+0000 7f1e4b7fe640 1 --2- 172.21.15.142:0/2640467422 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e4406dc80 0x7f1e44070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:59.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.113+0000 7f1e4b7fe640 1 -- 172.21.15.142:0/2640467422 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1e4c0affc0 con 0x7f1e5c0ff180 2024-03-29T05:12:59.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.113+0000 7f1e59d9b640 1 --2- 172.21.15.142:0/2640467422 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e4406dc80 0x7f1e44070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:59.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.113+0000 7f1e59d9b640 1 --2- 172.21.15.142:0/2640467422 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e4406dc80 0x7f1e44070140 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1e50005fd0 tx=0x7f1e5000b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:59.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.121+0000 7f1e4b7fe640 1 -- 172.21.15.142:0/2640467422 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1e4c079490 con 0x7f1e5c0ff180 2024-03-29T05:12:59.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:58 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3544426448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-29T05:12:59.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:58 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/749923415' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-29T05:12:59.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:58 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3023379769' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-29T05:12:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.403+0000 7f1e60a51640 1 -- 172.21.15.142:0/2640467422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f1e5c197100 con 0x7f1e5c0ff180 2024-03-29T05:12:59.409 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.403+0000 7f1e4b7fe640 1 -- 172.21.15.142:0/2640467422 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v39) v1 ==== 107+0+4897 (secure 0 0 0) 0x7f1e4c07cb50 con 0x7f1e5c0ff180 2024-03-29T05:12:59.409 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:12:59.410 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:49:01.288637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14494,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14494":{"gid":14494,"name":"cephfs.smithi142.bmkjzf","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.142:6829/1941073852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1941073852},{"type":"v1","addr":"172.21.15.142:6829","nonce":1941073852}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14506":{"gid":14506,"name":"cephfs.smithi142.wgmndr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6827/1415100534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1415100534},{"type":"v1","addr":"172.21.15.142:6827","nonce":1415100534}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi152.uhebsi","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.152:6825/832699894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":832699894},{"type":"v1","addr":"172.21.15.152:6825","nonce":832699894}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24287":{"gid":24287,"name":"cephfs.smithi152.hiilum","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.152:6827/3411697908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3411697908},{"type":"v1","addr":"172.21.15.152:6827","nonce":3411697908}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:12:59.410 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 21 2024-03-29T05:12:59.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.406+0000 7f1e60a51640 1 -- 172.21.15.142:0/2640467422 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e4406dc80 msgr2=0x7f1e44070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:59.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.406+0000 7f1e60a51640 1 --2- 172.21.15.142:0/2640467422 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e4406dc80 0x7f1e44070140 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1e50005fd0 tx=0x7f1e5000b390 comp rx=0 tx=0).stop 2024-03-29T05:12:59.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.406+0000 7f1e60a51640 1 -- 172.21.15.142:0/2640467422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e5c0ff180 msgr2=0x7f1e5c195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:59.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.406+0000 7f1e60a51640 1 --2- 172.21.15.142:0/2640467422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e5c0ff180 0x7f1e5c195db0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009430 tx=0x7f1e4c004040 comp rx=0 tx=0).stop 2024-03-29T05:12:59.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.406+0000 7f1e60a51640 1 -- 172.21.15.142:0/2640467422 shutdown_connections 2024-03-29T05:12:59.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.406+0000 7f1e60a51640 1 --2- 172.21.15.142:0/2640467422 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f1e4406dc80 0x7f1e44070140 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.406+0000 7f1e60a51640 1 --2- 172.21.15.142:0/2640467422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e5c1962f0 0x7f1e5c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.406+0000 7f1e60a51640 1 --2- 172.21.15.142:0/2640467422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e5c0ff180 0x7f1e5c195db0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.406+0000 7f1e60a51640 1 -- 172.21.15.142:0/2640467422 >> 172.21.15.142:0/2640467422 conn(0x7f1e5c0faa00 msgr2=0x7f1e5c1032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:59.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.406+0000 7f1e60a51640 1 -- 172.21.15.142:0/2640467422 shutdown_connections 2024-03-29T05:12:59.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.406+0000 7f1e60a51640 1 -- 172.21.15.142:0/2640467422 wait complete. 2024-03-29T05:12:59.424 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-29T05:12:59.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.539+0000 7f545790a640 1 -- 172.21.15.142:0/2555744185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5450067f70 msgr2=0x7f5450104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:59.548 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.539+0000 7f545790a640 1 --2- 172.21.15.142:0/2555744185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5450067f70 0x7f5450104c70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5440009880 tx=0x7f544002bce0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.548 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.540+0000 7f545790a640 1 -- 172.21.15.142:0/2555744185 shutdown_connections 2024-03-29T05:12:59.548 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.540+0000 7f545790a640 1 --2- 172.21.15.142:0/2555744185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5450067f70 0x7f5450104c70 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.548 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.540+0000 7f545790a640 1 --2- 172.21.15.142:0/2555744185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54500675c0 0x7f54500679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.548 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.540+0000 7f545790a640 1 -- 172.21.15.142:0/2555744185 >> 172.21.15.142:0/2555744185 conn(0x7f5450074a50 msgr2=0x7f5450074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:59.548 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.540+0000 7f545790a640 1 -- 172.21.15.142:0/2555744185 shutdown_connections 2024-03-29T05:12:59.548 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.540+0000 7f545790a640 1 -- 172.21.15.142:0/2555744185 wait complete. 2024-03-29T05:12:59.548 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.540+0000 7f545790a640 1 Processor -- start 2024-03-29T05:12:59.548 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.540+0000 7f545790a640 1 -- start start 2024-03-29T05:12:59.548 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f545790a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54500675c0 0x7f54500ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f545790a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54500ff600 0x7f54500ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f545790a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54501a1f20 con 0x7f54500ff600 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f545790a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54501a2060 con 0x7f54500675c0 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f5454ea5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54500ff600 0x7f54500ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f5454ea5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54500ff600 0x7f54500ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46904/0 (socket says 172.21.15.142:46904) 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f5454ea5640 1 -- 172.21.15.142:0/569775634 learned_addr learned my addr 172.21.15.142:0/569775634 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f54556a6640 1 --2- 172.21.15.142:0/569775634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54500675c0 0x7f54500ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f5454ea5640 1 -- 172.21.15.142:0/569775634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54500675c0 msgr2=0x7f54500ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f5454ea5640 1 --2- 172.21.15.142:0/569775634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54500675c0 0x7f54500ff0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f5454ea5640 1 -- 172.21.15.142:0/569775634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5440009560 con 0x7f54500ff600 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f54556a6640 1 --2- 172.21.15.142:0/569775634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54500675c0 0x7f54500ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.541+0000 7f5454ea5640 1 --2- 172.21.15.142:0/569775634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54500ff600 0x7f54500ffa60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5440036040 tx=0x7f5440004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.542+0000 7f54467fc640 1 -- 172.21.15.142:0/569775634 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54400037c0 con 0x7f54500ff600 2024-03-29T05:12:59.549 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.542+0000 7f54467fc640 1 -- 172.21.15.142:0/569775634 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5440003de0 con 0x7f54500ff600 2024-03-29T05:12:59.550 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.542+0000 7f54467fc640 1 -- 172.21.15.142:0/569775634 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5440034440 con 0x7f54500ff600 2024-03-29T05:12:59.550 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.542+0000 7f545790a640 1 -- 172.21.15.142:0/569775634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54501a21a0 con 0x7f54500ff600 2024-03-29T05:12:59.550 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.542+0000 7f545790a640 1 -- 172.21.15.142:0/569775634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54501a2620 con 0x7f54500ff600 2024-03-29T05:12:59.550 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.543+0000 7f545790a640 1 -- 172.21.15.142:0/569775634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545004e040 con 0x7f54500ff600 2024-03-29T05:12:59.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.544+0000 7f54467fc640 1 -- 172.21.15.142:0/569775634 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f5440004fb0 con 0x7f54500ff600 2024-03-29T05:12:59.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.544+0000 7f54467fc640 1 --2- 172.21.15.142:0/569775634 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f543c06dcd0 0x7f543c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:59.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.544+0000 7f54467fc640 1 -- 172.21.15.142:0/569775634 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f54400b01a0 con 0x7f54500ff600 2024-03-29T05:12:59.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.544+0000 7f54556a6640 1 --2- 172.21.15.142:0/569775634 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f543c06dcd0 0x7f543c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:59.551 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.545+0000 7f54556a6640 1 --2- 172.21.15.142:0/569775634 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f543c06dcd0 0x7f543c070190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f544c0058c0 tx=0x7f544c005ec0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:59.555 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.549+0000 7f54467fc640 1 -- 172.21.15.142:0/569775634 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f54400795c0 con 0x7f54500ff600 2024-03-29T05:12:59.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.787+0000 7f545790a640 1 -- 172.21.15.142:0/569775634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f5450108580 con 0x7f54500ff600 2024-03-29T05:12:59.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.788+0000 7f54467fc640 1 -- 172.21.15.142:0/569775634 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v39) v1 ==== 107+0+3334 (secure 0 0 0) 0x7f5440041d20 con 0x7f54500ff600 2024-03-29T05:12:59.794 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:12:59.794 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:54:33.039768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14494,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14494":{"gid":14494,"name":"cephfs.smithi142.bmkjzf","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.142:6829/1941073852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1941073852},{"type":"v1","addr":"172.21.15.142:6829","nonce":1941073852}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi152.uhebsi","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.152:6825/832699894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":832699894},{"type":"v1","addr":"172.21.15.152:6825","nonce":832699894}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14494,"qdb_cluster":[14494,24275]},"id":1}]} 2024-03-29T05:12:59.794 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 22 2024-03-29T05:12:59.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.790+0000 7f545790a640 1 -- 172.21.15.142:0/569775634 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f543c06dcd0 msgr2=0x7f543c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:59.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.790+0000 7f545790a640 1 --2- 172.21.15.142:0/569775634 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f543c06dcd0 0x7f543c070190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f544c0058c0 tx=0x7f544c005ec0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.790+0000 7f545790a640 1 -- 172.21.15.142:0/569775634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54500ff600 msgr2=0x7f54500ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:59.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.790+0000 7f545790a640 1 --2- 172.21.15.142:0/569775634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54500ff600 0x7f54500ffa60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5440036040 tx=0x7f5440004270 comp rx=0 tx=0).stop 2024-03-29T05:12:59.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.790+0000 7f545790a640 1 -- 172.21.15.142:0/569775634 shutdown_connections 2024-03-29T05:12:59.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.790+0000 7f545790a640 1 --2- 172.21.15.142:0/569775634 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f543c06dcd0 0x7f543c070190 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.790+0000 7f545790a640 1 --2- 172.21.15.142:0/569775634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54500ff600 0x7f54500ffa60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.790+0000 7f545790a640 1 --2- 172.21.15.142:0/569775634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54500675c0 0x7f54500ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.790+0000 7f545790a640 1 -- 172.21.15.142:0/569775634 >> 172.21.15.142:0/569775634 conn(0x7f5450074a50 msgr2=0x7f54500fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:59.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.790+0000 7f545790a640 1 -- 172.21.15.142:0/569775634 shutdown_connections 2024-03-29T05:12:59.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.790+0000 7f545790a640 1 -- 172.21.15.142:0/569775634 wait complete. 2024-03-29T05:12:59.809 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-03-29T05:12:59.809 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-29T05:12:59.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.921+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/895273741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e080fe720 msgr2=0x7f9e080feb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:59.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.921+0000 7f9e0eb6e640 1 --2- 172.21.15.142:0/895273741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e080fe720 0x7f9e080feb80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009880 tx=0x7f9df402bd70 comp rx=0 tx=0).stop 2024-03-29T05:12:59.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.921+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/895273741 shutdown_connections 2024-03-29T05:12:59.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.921+0000 7f9e0eb6e640 1 --2- 172.21.15.142:0/895273741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e080fe720 0x7f9e080feb80 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.921+0000 7f9e0eb6e640 1 --2- 172.21.15.142:0/895273741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e08100fa0 0x7f9e08101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.921+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/895273741 >> 172.21.15.142:0/895273741 conn(0x7f9e08075250 msgr2=0x7f9e08075660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:12:59.929 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.922+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/895273741 shutdown_connections 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.922+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/895273741 wait complete. 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.922+0000 7f9e0eb6e640 1 Processor -- start 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.922+0000 7f9e0eb6e640 1 -- start start 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.922+0000 7f9e0eb6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e080fe720 0x7f9e08197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.922+0000 7f9e0eb6e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e08100fa0 0x7f9e081984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.922+0000 7f9e0eb6e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e08198b40 con 0x7f9e08100fa0 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.922+0000 7f9e0eb6e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0819c910 con 0x7f9e080fe720 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.922+0000 7f9e07fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e08100fa0 0x7f9e081984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.922+0000 7f9e07fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e08100fa0 0x7f9e081984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46910/0 (socket says 172.21.15.142:46910) 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.922+0000 7f9e07fff640 1 -- 172.21.15.142:0/2120480755 learned_addr learned my addr 172.21.15.142:0/2120480755 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.923+0000 7f9e0c90a640 1 --2- 172.21.15.142:0/2120480755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e080fe720 0x7f9e08197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.923+0000 7f9e07fff640 1 -- 172.21.15.142:0/2120480755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e080fe720 msgr2=0x7f9e08197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:12:59.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.923+0000 7f9e07fff640 1 --2- 172.21.15.142:0/2120480755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e080fe720 0x7f9e08197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:12:59.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.923+0000 7f9e07fff640 1 -- 172.21.15.142:0/2120480755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df4009560 con 0x7f9e08100fa0 2024-03-29T05:12:59.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.923+0000 7f9e07fff640 1 --2- 172.21.15.142:0/2120480755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e08100fa0 0x7f9e081984b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009230 tx=0x7f9df4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:59.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.923+0000 7f9e05ffb640 1 -- 172.21.15.142:0/2120480755 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9df40035f0 con 0x7f9e08100fa0 2024-03-29T05:12:59.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.923+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/2120480755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0819cb90 con 0x7f9e08100fa0 2024-03-29T05:12:59.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.923+0000 7f9e05ffb640 1 -- 172.21.15.142:0/2120480755 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9df4003c10 con 0x7f9e08100fa0 2024-03-29T05:12:59.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.923+0000 7f9e05ffb640 1 -- 172.21.15.142:0/2120480755 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9df403e450 con 0x7f9e08100fa0 2024-03-29T05:12:59.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.923+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/2120480755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0819d0f0 con 0x7f9e08100fa0 2024-03-29T05:12:59.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.925+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/2120480755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0804e040 con 0x7f9e08100fa0 2024-03-29T05:12:59.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.925+0000 7f9e05ffb640 1 -- 172.21.15.142:0/2120480755 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f9df4004f60 con 0x7f9e08100fa0 2024-03-29T05:12:59.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.926+0000 7f9e05ffb640 1 --2- 172.21.15.142:0/2120480755 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9df806dcd0 0x7f9df8070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:12:59.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.926+0000 7f9e05ffb640 1 -- 172.21.15.142:0/2120480755 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f9df40b07e0 con 0x7f9e08100fa0 2024-03-29T05:12:59.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.926+0000 7f9e0c90a640 1 --2- 172.21.15.142:0/2120480755 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9df806dcd0 0x7f9df8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:12:59.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.926+0000 7f9e0c90a640 1 --2- 172.21.15.142:0/2120480755 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9df806dcd0 0x7f9df8070190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc005fd0 tx=0x7f9dfc005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:12:59.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:12:59.930+0000 7f9e05ffb640 1 -- 172.21.15.142:0/2120480755 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9df4079da0 con 0x7f9e08100fa0 2024-03-29T05:13:00.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:59 smithi142 ceph-mon[112475]: pgmap v741: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:13:00.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:59 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1568054299' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-29T05:13:00.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:59 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2640467422' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-29T05:13:00.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:13:00.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:12:59 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/569775634' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-29T05:13:00.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.285+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/2120480755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f9e08199390 con 0x7f9e08100fa0 2024-03-29T05:13:00.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.286+0000 7f9e05ffb640 1 -- 172.21.15.142:0/2120480755 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v39) v1 ==== 107+0+3298 (secure 0 0 0) 0x7f9df4043d00 con 0x7f9e08100fa0 2024-03-29T05:13:00.293 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:00.293 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi142.wgmndr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/3774187452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3774187452},{"type":"v1","addr":"172.21.15.142:6827","nonce":3774187452}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44255,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/869112786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":869112786},{"type":"v1","addr":"172.21.15.152:6827","nonce":869112786}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:54:33.202220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:00.293 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 23 2024-03-29T05:13:00.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.288+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/2120480755 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9df806dcd0 msgr2=0x7f9df8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:00.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.288+0000 7f9e0eb6e640 1 --2- 172.21.15.142:0/2120480755 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9df806dcd0 0x7f9df8070190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc005fd0 tx=0x7f9dfc005850 comp rx=0 tx=0).stop 2024-03-29T05:13:00.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.288+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/2120480755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e08100fa0 msgr2=0x7f9e081984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:00.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.288+0000 7f9e0eb6e640 1 --2- 172.21.15.142:0/2120480755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e08100fa0 0x7f9e081984b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009230 tx=0x7f9df4004040 comp rx=0 tx=0).stop 2024-03-29T05:13:00.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.288+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/2120480755 shutdown_connections 2024-03-29T05:13:00.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.288+0000 7f9e0eb6e640 1 --2- 172.21.15.142:0/2120480755 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f9df806dcd0 0x7f9df8070190 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.288+0000 7f9e0eb6e640 1 --2- 172.21.15.142:0/2120480755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e08100fa0 0x7f9e081984b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.296 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.288+0000 7f9e0eb6e640 1 --2- 172.21.15.142:0/2120480755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e080fe720 0x7f9e08197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.288+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/2120480755 >> 172.21.15.142:0/2120480755 conn(0x7f9e08075250 msgr2=0x7f9e080fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:00.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.288+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/2120480755 shutdown_connections 2024-03-29T05:13:00.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.288+0000 7f9e0eb6e640 1 -- 172.21.15.142:0/2120480755 wait complete. 2024-03-29T05:13:00.298 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:59 smithi152 ceph-mon[88186]: pgmap v741: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:13:00.298 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:59 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1568054299' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-29T05:13:00.298 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:59 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2640467422' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-29T05:13:00.298 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:13:00.298 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:12:59 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/569775634' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-29T05:13:00.306 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T05:13:00.306 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-03-29T05:13:00.306 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-29T05:13:00.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.407+0000 7f4d58927640 1 -- 172.21.15.142:0/1604607009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d540ffc10 msgr2=0x7f4d54103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:00.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.407+0000 7f4d58927640 1 --2- 172.21.15.142:0/1604607009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d540ffc10 0x7f4d54103b70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009880 tx=0x7f4d3c02bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.407+0000 7f4d58927640 1 -- 172.21.15.142:0/1604607009 shutdown_connections 2024-03-29T05:13:00.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.407+0000 7f4d58927640 1 --2- 172.21.15.142:0/1604607009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d540ffc10 0x7f4d54103b70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.407+0000 7f4d58927640 1 --2- 172.21.15.142:0/1604607009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d540ff260 0x7f4d540ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.407+0000 7f4d58927640 1 -- 172.21.15.142:0/1604607009 >> 172.21.15.142:0/1604607009 conn(0x7f4d54074a50 msgr2=0x7f4d54074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:00.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.407+0000 7f4d58927640 1 -- 172.21.15.142:0/1604607009 shutdown_connections 2024-03-29T05:13:00.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.407+0000 7f4d58927640 1 -- 172.21.15.142:0/1604607009 wait complete. 2024-03-29T05:13:00.415 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d58927640 1 Processor -- start 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d58927640 1 -- start start 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d58927640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d540ff260 0x7f4d54194980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d58927640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d54194ec0 0x7f4d54195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d58927640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d540682f0 con 0x7f4d54194ec0 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d58927640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d541a1ff0 con 0x7f4d540ff260 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d4bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d54194ec0 0x7f4d54195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d4bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d54194ec0 0x7f4d54195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46928/0 (socket says 172.21.15.142:46928) 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d4bfff640 1 -- 172.21.15.142:0/3645941368 learned_addr learned my addr 172.21.15.142:0/3645941368 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d5259c640 1 --2- 172.21.15.142:0/3645941368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d540ff260 0x7f4d54194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d4bfff640 1 -- 172.21.15.142:0/3645941368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d540ff260 msgr2=0x7f4d54194980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d4bfff640 1 --2- 172.21.15.142:0/3645941368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d540ff260 0x7f4d54194980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d4bfff640 1 -- 172.21.15.142:0/3645941368 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c009560 con 0x7f4d54194ec0 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.408+0000 7f4d4bfff640 1 --2- 172.21.15.142:0/3645941368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d54194ec0 0x7f4d54195340 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009230 tx=0x7f4d3c02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.409+0000 7f4d4b7fe640 1 -- 172.21.15.142:0/3645941368 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d3c0035f0 con 0x7f4d54194ec0 2024-03-29T05:13:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.409+0000 7f4d58927640 1 -- 172.21.15.142:0/3645941368 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d541a2190 con 0x7f4d54194ec0 2024-03-29T05:13:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.409+0000 7f4d4b7fe640 1 -- 172.21.15.142:0/3645941368 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d3c003c10 con 0x7f4d54194ec0 2024-03-29T05:13:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.409+0000 7f4d4b7fe640 1 -- 172.21.15.142:0/3645941368 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d3c005690 con 0x7f4d54194ec0 2024-03-29T05:13:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.409+0000 7f4d58927640 1 -- 172.21.15.142:0/3645941368 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d541a26a0 con 0x7f4d54194ec0 2024-03-29T05:13:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.410+0000 7f4d58927640 1 -- 172.21.15.142:0/3645941368 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5404e040 con 0x7f4d54194ec0 2024-03-29T05:13:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.410+0000 7f4d4b7fe640 1 -- 172.21.15.142:0/3645941368 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f4d3c004ed0 con 0x7f4d54194ec0 2024-03-29T05:13:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.411+0000 7f4d4b7fe640 1 --2- 172.21.15.142:0/3645941368 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4d4006dcd0 0x7f4d40070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.411+0000 7f4d4b7fe640 1 -- 172.21.15.142:0/3645941368 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f4d3c0b0030 con 0x7f4d54194ec0 2024-03-29T05:13:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.411+0000 7f4d5259c640 1 --2- 172.21.15.142:0/3645941368 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4d4006dcd0 0x7f4d40070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.411+0000 7f4d5259c640 1 --2- 172.21.15.142:0/3645941368 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4d4006dcd0 0x7f4d40070190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400bbe0 tx=0x7f4d4400b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:00.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.416+0000 7f4d4b7fe640 1 -- 172.21.15.142:0/3645941368 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4d3c079500 con 0x7f4d54194ec0 2024-03-29T05:13:00.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.769+0000 7f4d58927640 1 -- 172.21.15.142:0/3645941368 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f4d54107480 con 0x7f4d54194ec0 2024-03-29T05:13:00.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.770+0000 7f4d4b7fe640 1 -- 172.21.15.142:0/3645941368 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v39) v1 ==== 107+0+4856 (secure 0 0 0) 0x7f4d3c07cbc0 con 0x7f4d54194ec0 2024-03-29T05:13:00.776 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:00.776 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi142.wgmndr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/3774187452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3774187452},{"type":"v1","addr":"172.21.15.142:6827","nonce":3774187452}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":34388,"name":"cephfs.smithi142.bmkjzf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6829/168150659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":168150659},{"type":"v1","addr":"172.21.15.142:6829","nonce":168150659}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44255,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/869112786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":869112786},{"type":"v1","addr":"172.21.15.152:6827","nonce":869112786}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44263,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/924845457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":924845457},{"type":"v1","addr":"172.21.15.152:6825","nonce":924845457}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:54:33.202220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:00.777 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 24 2024-03-29T05:13:00.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.772+0000 7f4d58927640 1 -- 172.21.15.142:0/3645941368 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4d4006dcd0 msgr2=0x7f4d40070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:00.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.772+0000 7f4d58927640 1 --2- 172.21.15.142:0/3645941368 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4d4006dcd0 0x7f4d40070190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400bbe0 tx=0x7f4d4400b390 comp rx=0 tx=0).stop 2024-03-29T05:13:00.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.772+0000 7f4d58927640 1 -- 172.21.15.142:0/3645941368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d54194ec0 msgr2=0x7f4d54195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:00.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.772+0000 7f4d58927640 1 --2- 172.21.15.142:0/3645941368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d54194ec0 0x7f4d54195340 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009230 tx=0x7f4d3c02c850 comp rx=0 tx=0).stop 2024-03-29T05:13:00.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.772+0000 7f4d58927640 1 -- 172.21.15.142:0/3645941368 shutdown_connections 2024-03-29T05:13:00.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.772+0000 7f4d58927640 1 --2- 172.21.15.142:0/3645941368 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f4d4006dcd0 0x7f4d40070190 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.772+0000 7f4d58927640 1 --2- 172.21.15.142:0/3645941368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d54194ec0 0x7f4d54195340 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.772+0000 7f4d58927640 1 --2- 172.21.15.142:0/3645941368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d540ff260 0x7f4d54194980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.772+0000 7f4d58927640 1 -- 172.21.15.142:0/3645941368 >> 172.21.15.142:0/3645941368 conn(0x7f4d54074a50 msgr2=0x7f4d540fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:00.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.772+0000 7f4d58927640 1 -- 172.21.15.142:0/3645941368 shutdown_connections 2024-03-29T05:13:00.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.772+0000 7f4d58927640 1 -- 172.21.15.142:0/3645941368 wait complete. 2024-03-29T05:13:00.789 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T05:13:00.790 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-03-29T05:13:00.790 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-29T05:13:00.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:00 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2120480755' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-29T05:13:00.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:00 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3645941368' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-29T05:13:00.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.897+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2097981679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc0fee00 msgr2=0x7f78cc0ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:00.904 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.897+0000 7f78d2ba2640 1 --2- 172.21.15.142:0/2097981679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc0fee00 0x7f78cc0ff240 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009880 tx=0x7f78bc02bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.897+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2097981679 shutdown_connections 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.897+0000 7f78d2ba2640 1 --2- 172.21.15.142:0/2097981679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc0fee00 0x7f78cc0ff240 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.897+0000 7f78d2ba2640 1 --2- 172.21.15.142:0/2097981679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc104e20 0x7f78cc105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.897+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2097981679 >> 172.21.15.142:0/2097981679 conn(0x7f78cc074930 msgr2=0x7f78cc074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.897+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2097981679 shutdown_connections 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.897+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2097981679 wait complete. 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.897+0000 7f78d2ba2640 1 Processor -- start 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.897+0000 7f78d2ba2640 1 -- start start 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d2ba2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc0fee00 0x7f78cc1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d2ba2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc104e20 0x7f78cc196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d2ba2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1974f0 con 0x7f78cc104e20 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d2ba2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc197660 con 0x7f78cc0fee00 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d139f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc104e20 0x7f78cc196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:00.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d139f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc104e20 0x7f78cc196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46944/0 (socket says 172.21.15.142:46944) 2024-03-29T05:13:00.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d139f640 1 -- 172.21.15.142:0/2546898308 learned_addr learned my addr 172.21.15.142:0/2546898308 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:00.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d1ba0640 1 --2- 172.21.15.142:0/2546898308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc0fee00 0x7f78cc1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:00.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d139f640 1 -- 172.21.15.142:0/2546898308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc0fee00 msgr2=0x7f78cc1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:00.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d139f640 1 --2- 172.21.15.142:0/2546898308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc0fee00 0x7f78cc1969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:00.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d139f640 1 -- 172.21.15.142:0/2546898308 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc009560 con 0x7f78cc104e20 2024-03-29T05:13:00.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d1ba0640 1 --2- 172.21.15.142:0/2546898308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc0fee00 0x7f78cc1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:13:00.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d139f640 1 --2- 172.21.15.142:0/2546898308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc104e20 0x7f78cc196ef0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009230 tx=0x7f78bc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:00.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78c2ffd640 1 -- 172.21.15.142:0/2546898308 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78bc0035f0 con 0x7f78cc104e20 2024-03-29T05:13:00.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2546898308 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78cc19fa20 con 0x7f78cc104e20 2024-03-29T05:13:00.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78c2ffd640 1 -- 172.21.15.142:0/2546898308 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78bc003c10 con 0x7f78cc104e20 2024-03-29T05:13:00.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.898+0000 7f78c2ffd640 1 -- 172.21.15.142:0/2546898308 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78bc034440 con 0x7f78cc104e20 2024-03-29T05:13:00.906 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.899+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2546898308 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc19fdf0 con 0x7f78cc104e20 2024-03-29T05:13:00.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.900+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2546898308 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78cc04e040 con 0x7f78cc104e20 2024-03-29T05:13:00.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.901+0000 7f78c2ffd640 1 -- 172.21.15.142:0/2546898308 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f78bc004fd0 con 0x7f78cc104e20 2024-03-29T05:13:00.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.901+0000 7f78c2ffd640 1 --2- 172.21.15.142:0/2546898308 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f78b806dcd0 0x7f78b8070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:00.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.901+0000 7f78c2ffd640 1 -- 172.21.15.142:0/2546898308 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f78bc0b0120 con 0x7f78cc104e20 2024-03-29T05:13:00.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.901+0000 7f78d1ba0640 1 --2- 172.21.15.142:0/2546898308 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f78b806dcd0 0x7f78b8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:00.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.902+0000 7f78d1ba0640 1 --2- 172.21.15.142:0/2546898308 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f78b806dcd0 0x7f78b8070190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f78c800abe0 tx=0x7f78c800a5c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:00.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:00.906+0000 7f78c2ffd640 1 -- 172.21.15.142:0/2546898308 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f78bc079540 con 0x7f78cc104e20 2024-03-29T05:13:01.131 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.125+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2546898308 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f78cc1072a0 con 0x7f78cc104e20 2024-03-29T05:13:01.131 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.125+0000 7f78c2ffd640 1 -- 172.21.15.142:0/2546898308 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v39) v1 ==== 107+0+4074 (secure 0 0 0) 0x7f78bc07cc00 con 0x7f78cc104e20 2024-03-29T05:13:01.132 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:01.132 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi142.bmkjzf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6829/168150659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":168150659},{"type":"v1","addr":"172.21.15.142:6829","nonce":168150659}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44255,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/869112786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":869112786},{"type":"v1","addr":"172.21.15.152:6827","nonce":869112786}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44263,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/924845457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":924845457},{"type":"v1","addr":"172.21.15.152:6825","nonce":924845457}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:54:33.202220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:01.132 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 25 2024-03-29T05:13:01.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.127+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2546898308 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f78b806dcd0 msgr2=0x7f78b8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:01.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.127+0000 7f78d2ba2640 1 --2- 172.21.15.142:0/2546898308 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f78b806dcd0 0x7f78b8070190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f78c800abe0 tx=0x7f78c800a5c0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.127+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2546898308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc104e20 msgr2=0x7f78cc196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:01.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.127+0000 7f78d2ba2640 1 --2- 172.21.15.142:0/2546898308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc104e20 0x7f78cc196ef0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009230 tx=0x7f78bc004040 comp rx=0 tx=0).stop 2024-03-29T05:13:01.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.127+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2546898308 shutdown_connections 2024-03-29T05:13:01.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.127+0000 7f78d2ba2640 1 --2- 172.21.15.142:0/2546898308 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f78b806dcd0 0x7f78b8070190 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.127+0000 7f78d2ba2640 1 --2- 172.21.15.142:0/2546898308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc104e20 0x7f78cc196ef0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.127+0000 7f78d2ba2640 1 --2- 172.21.15.142:0/2546898308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc0fee00 0x7f78cc1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.128+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2546898308 >> 172.21.15.142:0/2546898308 conn(0x7f78cc074930 msgr2=0x7f78cc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:01.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.128+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2546898308 shutdown_connections 2024-03-29T05:13:01.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.128+0000 7f78d2ba2640 1 -- 172.21.15.142:0/2546898308 wait complete. 2024-03-29T05:13:01.146 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T05:13:01.146 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-03-29T05:13:01.146 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-29T05:13:01.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.250+0000 7f48a8731640 1 -- 172.21.15.142:0/4086414120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a00675c0 msgr2=0x7f48a00679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:01.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.250+0000 7f48a8731640 1 --2- 172.21.15.142:0/4086414120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a00675c0 0x7f48a00679a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f489c009880 tx=0x7f489c02bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.250+0000 7f48a8731640 1 -- 172.21.15.142:0/4086414120 shutdown_connections 2024-03-29T05:13:01.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.250+0000 7f48a8731640 1 --2- 172.21.15.142:0/4086414120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48a0067f70 0x7f48a0104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.250+0000 7f48a8731640 1 --2- 172.21.15.142:0/4086414120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a00675c0 0x7f48a00679a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.250+0000 7f48a8731640 1 -- 172.21.15.142:0/4086414120 >> 172.21.15.142:0/4086414120 conn(0x7f48a0074a50 msgr2=0x7f48a0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:01.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.250+0000 7f48a8731640 1 -- 172.21.15.142:0/4086414120 shutdown_connections 2024-03-29T05:13:01.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.250+0000 7f48a8731640 1 -- 172.21.15.142:0/4086414120 wait complete. 2024-03-29T05:13:01.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a8731640 1 Processor -- start 2024-03-29T05:13:01.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a8731640 1 -- start start 2024-03-29T05:13:01.258 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a8731640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a00675c0 0x7f48a00ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a8731640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48a00ff600 0x7f48a00ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a8731640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a01a1f50 con 0x7f48a00675c0 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a8731640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a01a2090 con 0x7f48a00ff600 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a64cd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a00675c0 0x7f48a00ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a64cd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a00675c0 0x7f48a00ff0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46956/0 (socket says 172.21.15.142:46956) 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a64cd640 1 -- 172.21.15.142:0/973399856 learned_addr learned my addr 172.21.15.142:0/973399856 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a5ccc640 1 --2- 172.21.15.142:0/973399856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48a00ff600 0x7f48a00ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a64cd640 1 -- 172.21.15.142:0/973399856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48a00ff600 msgr2=0x7f48a00ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a64cd640 1 --2- 172.21.15.142:0/973399856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48a00ff600 0x7f48a00ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.251+0000 7f48a64cd640 1 -- 172.21.15.142:0/973399856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c009560 con 0x7f48a00675c0 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.252+0000 7f48a64cd640 1 --2- 172.21.15.142:0/973399856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a00675c0 0x7f48a00ff0c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f489c009430 tx=0x7f489c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.252+0000 7f48977fe640 1 -- 172.21.15.142:0/973399856 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f489c0035f0 con 0x7f48a00675c0 2024-03-29T05:13:01.259 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.252+0000 7f48977fe640 1 -- 172.21.15.142:0/973399856 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f489c003c10 con 0x7f48a00675c0 2024-03-29T05:13:01.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.252+0000 7f48977fe640 1 -- 172.21.15.142:0/973399856 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f489c005670 con 0x7f48a00675c0 2024-03-29T05:13:01.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.252+0000 7f48a8731640 1 -- 172.21.15.142:0/973399856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a01a21d0 con 0x7f48a00675c0 2024-03-29T05:13:01.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.252+0000 7f48a8731640 1 -- 172.21.15.142:0/973399856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a01a2680 con 0x7f48a00675c0 2024-03-29T05:13:01.260 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.253+0000 7f48a8731640 1 -- 172.21.15.142:0/973399856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a004e040 con 0x7f48a00675c0 2024-03-29T05:13:01.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.253+0000 7f48977fe640 1 -- 172.21.15.142:0/973399856 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f489c004dd0 con 0x7f48a00675c0 2024-03-29T05:13:01.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.254+0000 7f48977fe640 1 --2- 172.21.15.142:0/973399856 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f488c06dcd0 0x7f488c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:01.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.254+0000 7f48a5ccc640 1 --2- 172.21.15.142:0/973399856 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f488c06dcd0 0x7f488c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:01.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.254+0000 7f48977fe640 1 -- 172.21.15.142:0/973399856 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f489c0affe0 con 0x7f48a00675c0 2024-03-29T05:13:01.261 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.255+0000 7f48a5ccc640 1 --2- 172.21.15.142:0/973399856 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f488c06dcd0 0x7f488c070190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4890005fd0 tx=0x7f4890005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:01.264 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.259+0000 7f48977fe640 1 -- 172.21.15.142:0/973399856 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f489c079480 con 0x7f48a00675c0 2024-03-29T05:13:01.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:00 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2120480755' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-29T05:13:01.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:00 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3645941368' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-29T05:13:01.561 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.556+0000 7f48a8731640 1 -- 172.21.15.142:0/973399856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f48a0108580 con 0x7f48a00675c0 2024-03-29T05:13:01.562 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.556+0000 7f48977fe640 1 -- 172.21.15.142:0/973399856 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v39) v1 ==== 107+0+4924 (secure 0 0 0) 0x7f489c07cb40 con 0x7f48a00675c0 2024-03-29T05:13:01.563 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:01.563 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi142.bmkjzf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6829/168150659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":168150659},{"type":"v1","addr":"172.21.15.142:6829","nonce":168150659}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44255,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/869112786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":869112786},{"type":"v1","addr":"172.21.15.152:6827","nonce":869112786}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44263,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/924845457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":924845457},{"type":"v1","addr":"172.21.15.152:6825","nonce":924845457}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:54:33.202220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:01.563 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 26 2024-03-29T05:13:01.564 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.558+0000 7f48a8731640 1 -- 172.21.15.142:0/973399856 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f488c06dcd0 msgr2=0x7f488c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:01.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.558+0000 7f48a8731640 1 --2- 172.21.15.142:0/973399856 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f488c06dcd0 0x7f488c070190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4890005fd0 tx=0x7f4890005850 comp rx=0 tx=0).stop 2024-03-29T05:13:01.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.558+0000 7f48a8731640 1 -- 172.21.15.142:0/973399856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a00675c0 msgr2=0x7f48a00ff0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:01.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.558+0000 7f48a8731640 1 --2- 172.21.15.142:0/973399856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a00675c0 0x7f48a00ff0c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f489c009430 tx=0x7f489c004040 comp rx=0 tx=0).stop 2024-03-29T05:13:01.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.558+0000 7f48a8731640 1 -- 172.21.15.142:0/973399856 shutdown_connections 2024-03-29T05:13:01.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.558+0000 7f48a8731640 1 --2- 172.21.15.142:0/973399856 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f488c06dcd0 0x7f488c070190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.558+0000 7f48a8731640 1 --2- 172.21.15.142:0/973399856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48a00ff600 0x7f48a00ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.558+0000 7f48a8731640 1 --2- 172.21.15.142:0/973399856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48a00675c0 0x7f48a00ff0c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.558+0000 7f48a8731640 1 -- 172.21.15.142:0/973399856 >> 172.21.15.142:0/973399856 conn(0x7f48a0074a50 msgr2=0x7f48a00fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:01.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.558+0000 7f48a8731640 1 -- 172.21.15.142:0/973399856 shutdown_connections 2024-03-29T05:13:01.565 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.559+0000 7f48a8731640 1 -- 172.21.15.142:0/973399856 wait complete. 2024-03-29T05:13:01.575 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T05:13:01.575 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-03-29T05:13:01.575 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-29T05:13:01.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.674+0000 7f7f2381c640 1 -- 172.21.15.142:0/2427903528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f1c0fe9e0 msgr2=0x7f7f1c0fedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:01.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.674+0000 7f7f2381c640 1 --2- 172.21.15.142:0/2427903528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f1c0fe9e0 0x7f7f1c0fedc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009880 tx=0x7f7f1802bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.674+0000 7f7f2381c640 1 -- 172.21.15.142:0/2427903528 shutdown_connections 2024-03-29T05:13:01.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.674+0000 7f7f2381c640 1 --2- 172.21.15.142:0/2427903528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f1c0ff390 0x7f7f1c104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.674+0000 7f7f2381c640 1 --2- 172.21.15.142:0/2427903528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f1c0fe9e0 0x7f7f1c0fedc0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.674+0000 7f7f2381c640 1 -- 172.21.15.142:0/2427903528 >> 172.21.15.142:0/2427903528 conn(0x7f7f1c0741f0 msgr2=0x7f7f1c074600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:01.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.674+0000 7f7f2381c640 1 -- 172.21.15.142:0/2427903528 shutdown_connections 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.674+0000 7f7f2381c640 1 -- 172.21.15.142:0/2427903528 wait complete. 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f2381c640 1 Processor -- start 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f2381c640 1 -- start start 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f2381c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f1c0fe9e0 0x7f7f1c195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f2381c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f1c196310 0x7f7f1c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f2381c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c196910 con 0x7f7f1c196310 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f2381c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c196a80 con 0x7f7f1c0fe9e0 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f20db7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f1c196310 0x7f7f1c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f20db7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f1c196310 0x7f7f1c19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:46976/0 (socket says 172.21.15.142:46976) 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f20db7640 1 -- 172.21.15.142:0/2145907286 learned_addr learned my addr 172.21.15.142:0/2145907286 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f215b8640 1 --2- 172.21.15.142:0/2145907286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f1c0fe9e0 0x7f7f1c195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f20db7640 1 -- 172.21.15.142:0/2145907286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f1c0fe9e0 msgr2=0x7f7f1c195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f20db7640 1 --2- 172.21.15.142:0/2145907286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f1c0fe9e0 0x7f7f1c195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.675+0000 7f7f20db7640 1 -- 172.21.15.142:0/2145907286 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f18009560 con 0x7f7f1c196310 2024-03-29T05:13:01.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.676+0000 7f7f20db7640 1 --2- 172.21.15.142:0/2145907286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f1c196310 0x7f7f1c19a770 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00d8d0 tx=0x7f7f0c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:01.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.676+0000 7f7f127fc640 1 -- 172.21.15.142:0/2145907286 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f0c00bea0 con 0x7f7f1c196310 2024-03-29T05:13:01.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.676+0000 7f7f127fc640 1 -- 172.21.15.142:0/2145907286 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f0c009c00 con 0x7f7f1c196310 2024-03-29T05:13:01.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.676+0000 7f7f2381c640 1 -- 172.21.15.142:0/2145907286 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1c19ad70 con 0x7f7f1c196310 2024-03-29T05:13:01.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.676+0000 7f7f127fc640 1 -- 172.21.15.142:0/2145907286 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f0c010460 con 0x7f7f1c196310 2024-03-29T05:13:01.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.676+0000 7f7f2381c640 1 -- 172.21.15.142:0/2145907286 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1c19b2d0 con 0x7f7f1c196310 2024-03-29T05:13:01.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.677+0000 7f7f2381c640 1 -- 172.21.15.142:0/2145907286 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1c19b5a0 con 0x7f7f1c196310 2024-03-29T05:13:01.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.677+0000 7f7f127fc640 1 -- 172.21.15.142:0/2145907286 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f7f0c003f10 con 0x7f7f1c196310 2024-03-29T05:13:01.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.678+0000 7f7f127fc640 1 --2- 172.21.15.142:0/2145907286 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7f0806dcd0 0x7f7f08070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:01.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.678+0000 7f7f127fc640 1 -- 172.21.15.142:0/2145907286 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f7f0c08d9b0 con 0x7f7f1c196310 2024-03-29T05:13:01.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.678+0000 7f7f215b8640 1 --2- 172.21.15.142:0/2145907286 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7f0806dcd0 0x7f7f08070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:01.685 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.678+0000 7f7f215b8640 1 --2- 172.21.15.142:0/2145907286 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7f0806dcd0 0x7f7f08070190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7f18006010 tx=0x7f7f18000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:01.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.682+0000 7f7f127fc640 1 -- 172.21.15.142:0/2145907286 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7f0c056e50 con 0x7f7f1c196310 2024-03-29T05:13:01.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.925+0000 7f7f2381c640 1 -- 172.21.15.142:0/2145907286 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f7f1c197120 con 0x7f7f1c196310 2024-03-29T05:13:01.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.926+0000 7f7f127fc640 1 -- 172.21.15.142:0/2145907286 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v39) v1 ==== 107+0+4145 (secure 0 0 0) 0x7f7f0c05a510 con 0x7f7f1c196310 2024-03-29T05:13:01.932 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:01.932 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44255,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/869112786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":869112786},{"type":"v1","addr":"172.21.15.152:6827","nonce":869112786}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44263,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/924845457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":924845457},{"type":"v1","addr":"172.21.15.152:6825","nonce":924845457}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:54:33.202220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:01.932 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 27 2024-03-29T05:13:01.934 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.928+0000 7f7f2381c640 1 -- 172.21.15.142:0/2145907286 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7f0806dcd0 msgr2=0x7f7f08070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:01.934 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.928+0000 7f7f2381c640 1 --2- 172.21.15.142:0/2145907286 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7f0806dcd0 0x7f7f08070190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7f18006010 tx=0x7f7f18000bc0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.928+0000 7f7f2381c640 1 -- 172.21.15.142:0/2145907286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f1c196310 msgr2=0x7f7f1c19a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:01.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.928+0000 7f7f2381c640 1 --2- 172.21.15.142:0/2145907286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f1c196310 0x7f7f1c19a770 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00d8d0 tx=0x7f7f0c00dd90 comp rx=0 tx=0).stop 2024-03-29T05:13:01.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.928+0000 7f7f2381c640 1 -- 172.21.15.142:0/2145907286 shutdown_connections 2024-03-29T05:13:01.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.928+0000 7f7f2381c640 1 --2- 172.21.15.142:0/2145907286 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f7f0806dcd0 0x7f7f08070190 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.928+0000 7f7f2381c640 1 --2- 172.21.15.142:0/2145907286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f1c196310 0x7f7f1c19a770 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.928+0000 7f7f2381c640 1 --2- 172.21.15.142:0/2145907286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f1c0fe9e0 0x7f7f1c195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:01.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.928+0000 7f7f2381c640 1 -- 172.21.15.142:0/2145907286 >> 172.21.15.142:0/2145907286 conn(0x7f7f1c0741f0 msgr2=0x7f7f1c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:01.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.928+0000 7f7f2381c640 1 -- 172.21.15.142:0/2145907286 shutdown_connections 2024-03-29T05:13:01.935 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:01.928+0000 7f7f2381c640 1 -- 172.21.15.142:0/2145907286 wait complete. 2024-03-29T05:13:01.945 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T05:13:01.945 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-03-29T05:13:01.945 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-29T05:13:02.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:01 smithi152 ceph-mon[88186]: pgmap v742: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:13:02.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:01 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2546898308' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-29T05:13:02.046 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:01 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/973399856' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-29T05:13:02.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.044+0000 7f15ba88f640 1 -- 172.21.15.142:0/3553061169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15b4067f70 msgr2=0x7f15b4104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:02.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.044+0000 7f15ba88f640 1 --2- 172.21.15.142:0/3553061169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15b4067f70 0x7f15b4104c70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009880 tx=0x7f15a802bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.044+0000 7f15ba88f640 1 -- 172.21.15.142:0/3553061169 shutdown_connections 2024-03-29T05:13:02.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.044+0000 7f15ba88f640 1 --2- 172.21.15.142:0/3553061169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15b4067f70 0x7f15b4104c70 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.044+0000 7f15ba88f640 1 --2- 172.21.15.142:0/3553061169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15b40675c0 0x7f15b40679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.044+0000 7f15ba88f640 1 -- 172.21.15.142:0/3553061169 >> 172.21.15.142:0/3553061169 conn(0x7f15b4074a50 msgr2=0x7f15b4074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:02.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.044+0000 7f15ba88f640 1 -- 172.21.15.142:0/3553061169 shutdown_connections 2024-03-29T05:13:02.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.044+0000 7f15ba88f640 1 -- 172.21.15.142:0/3553061169 wait complete. 2024-03-29T05:13:02.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.045+0000 7f15ba88f640 1 Processor -- start 2024-03-29T05:13:02.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.045+0000 7f15ba88f640 1 -- start start 2024-03-29T05:13:02.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.045+0000 7f15ba88f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15b40675c0 0x7f15b4195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:02.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.045+0000 7f15ba88f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15b41962c0 0x7f15b419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:02.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.045+0000 7f15ba88f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b41968c0 con 0x7f15b40675c0 2024-03-29T05:13:02.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.045+0000 7f15ba88f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b4196a30 con 0x7f15b41962c0 2024-03-29T05:13:02.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.045+0000 7f15b3fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15b40675c0 0x7f15b4195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:02.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.045+0000 7f15b3fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15b40675c0 0x7f15b4195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47002/0 (socket says 172.21.15.142:47002) 2024-03-29T05:13:02.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.045+0000 7f15b3fff640 1 -- 172.21.15.142:0/2508215987 learned_addr learned my addr 172.21.15.142:0/2508215987 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:02.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.046+0000 7f15b3fff640 1 -- 172.21.15.142:0/2508215987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15b41962c0 msgr2=0x7f15b419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:02.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.046+0000 7f15b3fff640 1 --2- 172.21.15.142:0/2508215987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15b41962c0 0x7f15b419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.046+0000 7f15b3fff640 1 -- 172.21.15.142:0/2508215987 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a8009560 con 0x7f15b40675c0 2024-03-29T05:13:02.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.046+0000 7f15b3fff640 1 --2- 172.21.15.142:0/2508215987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15b40675c0 0x7f15b4195d80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f15a400d720 tx=0x7f15a400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:02.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.046+0000 7f15b17fa640 1 -- 172.21.15.142:0/2508215987 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15a40051f0 con 0x7f15b40675c0 2024-03-29T05:13:02.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.046+0000 7f15b17fa640 1 -- 172.21.15.142:0/2508215987 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15a4009ce0 con 0x7f15b40675c0 2024-03-29T05:13:02.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.046+0000 7f15ba88f640 1 -- 172.21.15.142:0/2508215987 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b419ad20 con 0x7f15b40675c0 2024-03-29T05:13:02.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.046+0000 7f15b17fa640 1 -- 172.21.15.142:0/2508215987 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15a4010460 con 0x7f15b40675c0 2024-03-29T05:13:02.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.046+0000 7f15ba88f640 1 -- 172.21.15.142:0/2508215987 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b419b280 con 0x7f15b40675c0 2024-03-29T05:13:02.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.047+0000 7f15ba88f640 1 -- 172.21.15.142:0/2508215987 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b404e040 con 0x7f15b40675c0 2024-03-29T05:13:02.055 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.048+0000 7f15b17fa640 1 -- 172.21.15.142:0/2508215987 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f15a4005350 con 0x7f15b40675c0 2024-03-29T05:13:02.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.048+0000 7f15b17fa640 1 --2- 172.21.15.142:0/2508215987 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f159c06dcd0 0x7f159c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:02.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.048+0000 7f15b17fa640 1 -- 172.21.15.142:0/2508215987 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f15a408d820 con 0x7f15b40675c0 2024-03-29T05:13:02.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.048+0000 7f15b37fe640 1 --2- 172.21.15.142:0/2508215987 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f159c06dcd0 0x7f159c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:02.056 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.049+0000 7f15b37fe640 1 --2- 172.21.15.142:0/2508215987 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f159c06dcd0 0x7f159c070190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f15a8036040 tx=0x7f15a8000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:02.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.053+0000 7f15b17fa640 1 -- 172.21.15.142:0/2508215987 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f15a4056c40 con 0x7f15b40675c0 2024-03-29T05:13:02.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:01 smithi142 ceph-mon[112475]: pgmap v742: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:13:02.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:01 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2546898308' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-29T05:13:02.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:01 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/973399856' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-29T05:13:02.277 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.271+0000 7f15ba88f640 1 -- 172.21.15.142:0/2508215987 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f15b419b500 con 0x7f15b40675c0 2024-03-29T05:13:02.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.272+0000 7f15b17fa640 1 -- 172.21.15.142:0/2508215987 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v39) v1 ==== 107+0+4995 (secure 0 0 0) 0x7f15a405a300 con 0x7f15b40675c0 2024-03-29T05:13:02.278 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:02.278 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":34424,"name":"cephfs.smithi142.bmkjzf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6829/1735570051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1735570051},{"type":"v1","addr":"172.21.15.142:6829","nonce":1735570051}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44255,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/869112786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":869112786},{"type":"v1","addr":"172.21.15.152:6827","nonce":869112786}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44263,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/924845457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":924845457},{"type":"v1","addr":"172.21.15.152:6825","nonce":924845457}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:54:33.202220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:02.278 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 28 2024-03-29T05:13:02.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.275+0000 7f15ba88f640 1 -- 172.21.15.142:0/2508215987 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f159c06dcd0 msgr2=0x7f159c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:02.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.275+0000 7f15ba88f640 1 --2- 172.21.15.142:0/2508215987 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f159c06dcd0 0x7f159c070190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f15a8036040 tx=0x7f15a8000bc0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.281 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.275+0000 7f15ba88f640 1 -- 172.21.15.142:0/2508215987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15b40675c0 msgr2=0x7f15b4195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:02.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.275+0000 7f15ba88f640 1 --2- 172.21.15.142:0/2508215987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15b40675c0 0x7f15b4195d80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f15a400d720 tx=0x7f15a400dbe0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.275+0000 7f15ba88f640 1 -- 172.21.15.142:0/2508215987 shutdown_connections 2024-03-29T05:13:02.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.275+0000 7f15ba88f640 1 --2- 172.21.15.142:0/2508215987 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f159c06dcd0 0x7f159c070190 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.275+0000 7f15ba88f640 1 --2- 172.21.15.142:0/2508215987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15b41962c0 0x7f15b419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.275+0000 7f15ba88f640 1 --2- 172.21.15.142:0/2508215987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15b40675c0 0x7f15b4195d80 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.275+0000 7f15ba88f640 1 -- 172.21.15.142:0/2508215987 >> 172.21.15.142:0/2508215987 conn(0x7f15b4074a50 msgr2=0x7f15b40fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:02.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.276+0000 7f15ba88f640 1 -- 172.21.15.142:0/2508215987 shutdown_connections 2024-03-29T05:13:02.282 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.276+0000 7f15ba88f640 1 -- 172.21.15.142:0/2508215987 wait complete. 2024-03-29T05:13:02.292 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T05:13:02.292 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-03-29T05:13:02.293 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-29T05:13:02.400 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.392+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/72150994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb180ffc10 msgr2=0x7ffb18103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:02.400 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.392+0000 7ffb1ffa4640 1 --2- 172.21.15.142:0/72150994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb180ffc10 0x7ffb18103b70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009880 tx=0x7ffb0802bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.400 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.392+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/72150994 shutdown_connections 2024-03-29T05:13:02.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.392+0000 7ffb1ffa4640 1 --2- 172.21.15.142:0/72150994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb180ffc10 0x7ffb18103b70 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.392+0000 7ffb1ffa4640 1 --2- 172.21.15.142:0/72150994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb180ff260 0x7ffb180ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.392+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/72150994 >> 172.21.15.142:0/72150994 conn(0x7ffb18074a50 msgr2=0x7ffb18074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:02.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.392+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/72150994 shutdown_connections 2024-03-29T05:13:02.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.392+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/72150994 wait complete. 2024-03-29T05:13:02.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1ffa4640 1 Processor -- start 2024-03-29T05:13:02.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1ffa4640 1 -- start start 2024-03-29T05:13:02.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1ffa4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb180ff260 0x7ffb1819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:02.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1ffa4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb1819af60 0x7ffb18194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:02.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1ffa4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1819b530 con 0x7ffb180ff260 2024-03-29T05:13:02.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1ffa4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb18194fd0 con 0x7ffb1819af60 2024-03-29T05:13:02.401 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1dd40640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb180ff260 0x7ffb1819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:02.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1dd40640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb180ff260 0x7ffb1819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47010/0 (socket says 172.21.15.142:47010) 2024-03-29T05:13:02.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1dd40640 1 -- 172.21.15.142:0/574696712 learned_addr learned my addr 172.21.15.142:0/574696712 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:02.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1d53f640 1 --2- 172.21.15.142:0/574696712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb1819af60 0x7ffb18194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:02.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1dd40640 1 -- 172.21.15.142:0/574696712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb1819af60 msgr2=0x7ffb18194a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:02.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1dd40640 1 --2- 172.21.15.142:0/574696712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb1819af60 0x7ffb18194a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1dd40640 1 -- 172.21.15.142:0/574696712 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb08009560 con 0x7ffb180ff260 2024-03-29T05:13:02.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb1dd40640 1 --2- 172.21.15.142:0/574696712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb180ff260 0x7ffb1819aa20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400c8f0 tx=0x7ffb1400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:02.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.393+0000 7ffb0effd640 1 -- 172.21.15.142:0/574696712 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb14007b80 con 0x7ffb180ff260 2024-03-29T05:13:02.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.394+0000 7ffb0effd640 1 -- 172.21.15.142:0/574696712 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb140054e0 con 0x7ffb180ff260 2024-03-29T05:13:02.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.394+0000 7ffb0effd640 1 -- 172.21.15.142:0/574696712 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb1400f450 con 0x7ffb180ff260 2024-03-29T05:13:02.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.394+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/574696712 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb181952b0 con 0x7ffb180ff260 2024-03-29T05:13:02.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.394+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/574696712 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb18195810 con 0x7ffb180ff260 2024-03-29T05:13:02.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.395+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/574696712 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1804e040 con 0x7ffb180ff260 2024-03-29T05:13:02.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.395+0000 7ffb0effd640 1 -- 172.21.15.142:0/574696712 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ffb14004f60 con 0x7ffb180ff260 2024-03-29T05:13:02.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.396+0000 7ffb0effd640 1 --2- 172.21.15.142:0/574696712 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffb0406dcd0 0x7ffb04070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:02.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.396+0000 7ffb0effd640 1 -- 172.21.15.142:0/574696712 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ffb1408da30 con 0x7ffb180ff260 2024-03-29T05:13:02.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.396+0000 7ffb1d53f640 1 --2- 172.21.15.142:0/574696712 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffb0406dcd0 0x7ffb04070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:02.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.396+0000 7ffb1d53f640 1 --2- 172.21.15.142:0/574696712 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffb0406dcd0 0x7ffb04070190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffb08006010 tx=0x7ffb08000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:02.410 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.405+0000 7ffb0effd640 1 -- 172.21.15.142:0/574696712 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ffb14056ed0 con 0x7ffb180ff260 2024-03-29T05:13:02.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.654+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/574696712 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7ffb18195fa0 con 0x7ffb180ff260 2024-03-29T05:13:02.660 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.654+0000 7ffb0effd640 1 -- 172.21.15.142:0/574696712 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v39) v1 ==== 107+0+4216 (secure 0 0 0) 0x7ffb1405a590 con 0x7ffb180ff260 2024-03-29T05:13:02.660 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:02.660 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":34424,"name":"cephfs.smithi142.bmkjzf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6829/1735570051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1735570051},{"type":"v1","addr":"172.21.15.142:6829","nonce":1735570051}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44255,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/869112786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":869112786},{"type":"v1","addr":"172.21.15.152:6827","nonce":869112786}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:54:33.202220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:02.661 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 29 2024-03-29T05:13:02.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.656+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/574696712 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffb0406dcd0 msgr2=0x7ffb04070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:02.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.656+0000 7ffb1ffa4640 1 --2- 172.21.15.142:0/574696712 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffb0406dcd0 0x7ffb04070190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffb08006010 tx=0x7ffb08000bc0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.656+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/574696712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb180ff260 msgr2=0x7ffb1819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:02.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.656+0000 7ffb1ffa4640 1 --2- 172.21.15.142:0/574696712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb180ff260 0x7ffb1819aa20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400c8f0 tx=0x7ffb1400cdb0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.656+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/574696712 shutdown_connections 2024-03-29T05:13:02.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.656+0000 7ffb1ffa4640 1 --2- 172.21.15.142:0/574696712 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ffb0406dcd0 0x7ffb04070190 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.656+0000 7ffb1ffa4640 1 --2- 172.21.15.142:0/574696712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb1819af60 0x7ffb18194a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.656+0000 7ffb1ffa4640 1 --2- 172.21.15.142:0/574696712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb180ff260 0x7ffb1819aa20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.656+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/574696712 >> 172.21.15.142:0/574696712 conn(0x7ffb18074a50 msgr2=0x7ffb180fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:02.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.657+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/574696712 shutdown_connections 2024-03-29T05:13:02.663 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.657+0000 7ffb1ffa4640 1 -- 172.21.15.142:0/574696712 wait complete. 2024-03-29T05:13:02.677 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T05:13:02.677 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-03-29T05:13:02.678 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-29T05:13:02.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.782+0000 7f0a3c96e640 1 -- 172.21.15.142:0/3666539360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a380ff260 msgr2=0x7f0a380ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:02.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.782+0000 7f0a3c96e640 1 --2- 172.21.15.142:0/3666539360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a380ff260 0x7f0a380ff640 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009880 tx=0x7f0a2802bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.783+0000 7f0a3c96e640 1 -- 172.21.15.142:0/3666539360 shutdown_connections 2024-03-29T05:13:02.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.783+0000 7f0a3c96e640 1 --2- 172.21.15.142:0/3666539360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a380ffc10 0x7f0a38103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.783+0000 7f0a3c96e640 1 --2- 172.21.15.142:0/3666539360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a380ff260 0x7f0a380ff640 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.783+0000 7f0a3c96e640 1 -- 172.21.15.142:0/3666539360 >> 172.21.15.142:0/3666539360 conn(0x7f0a38074a50 msgr2=0x7f0a38074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:02.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.783+0000 7f0a3c96e640 1 -- 172.21.15.142:0/3666539360 shutdown_connections 2024-03-29T05:13:02.789 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.783+0000 7f0a3c96e640 1 -- 172.21.15.142:0/3666539360 wait complete. 2024-03-29T05:13:02.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.783+0000 7f0a3c96e640 1 Processor -- start 2024-03-29T05:13:02.791 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.783+0000 7f0a3c96e640 1 -- start start 2024-03-29T05:13:02.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a3c96e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a380ff260 0x7f0a38100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:02.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a3c96e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a38101150 0x7f0a381015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:02.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a3c96e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a381a1f60 con 0x7f0a38101150 2024-03-29T05:13:02.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a3c96e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a381a20a0 con 0x7f0a380ff260 2024-03-29T05:13:02.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a35d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a38101150 0x7f0a381015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:02.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a35d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a38101150 0x7f0a381015b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47026/0 (socket says 172.21.15.142:47026) 2024-03-29T05:13:02.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a35d9b640 1 -- 172.21.15.142:0/2548536866 learned_addr learned my addr 172.21.15.142:0/2548536866 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:02.792 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a3659c640 1 --2- 172.21.15.142:0/2548536866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a380ff260 0x7f0a38100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:02.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a35d9b640 1 -- 172.21.15.142:0/2548536866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a380ff260 msgr2=0x7f0a38100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:02.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a35d9b640 1 --2- 172.21.15.142:0/2548536866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a380ff260 0x7f0a38100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:02.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a35d9b640 1 -- 172.21.15.142:0/2548536866 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a28009560 con 0x7f0a38101150 2024-03-29T05:13:02.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a35d9b640 1 --2- 172.21.15.142:0/2548536866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a38101150 0x7f0a381015b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00e9a0 tx=0x7f0a2c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:02.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a277fe640 1 -- 172.21.15.142:0/2548536866 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a2c00cc90 con 0x7f0a38101150 2024-03-29T05:13:02.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a3c96e640 1 -- 172.21.15.142:0/2548536866 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a381a21e0 con 0x7f0a38101150 2024-03-29T05:13:02.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.784+0000 7f0a277fe640 1 -- 172.21.15.142:0/2548536866 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a2c00cdf0 con 0x7f0a38101150 2024-03-29T05:13:02.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.785+0000 7f0a277fe640 1 -- 172.21.15.142:0/2548536866 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a2c010430 con 0x7f0a38101150 2024-03-29T05:13:02.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.785+0000 7f0a3c96e640 1 -- 172.21.15.142:0/2548536866 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a381a2740 con 0x7f0a38101150 2024-03-29T05:13:02.793 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.786+0000 7f0a3c96e640 1 -- 172.21.15.142:0/2548536866 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3804e040 con 0x7f0a38101150 2024-03-29T05:13:02.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.786+0000 7f0a277fe640 1 -- 172.21.15.142:0/2548536866 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f0a2c005020 con 0x7f0a38101150 2024-03-29T05:13:02.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.787+0000 7f0a277fe640 1 --2- 172.21.15.142:0/2548536866 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0a2006dcd0 0x7f0a20070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:02.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.787+0000 7f0a277fe640 1 -- 172.21.15.142:0/2548536866 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f0a2c014070 con 0x7f0a38101150 2024-03-29T05:13:02.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.787+0000 7f0a3659c640 1 --2- 172.21.15.142:0/2548536866 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0a2006dcd0 0x7f0a20070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:02.794 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.787+0000 7f0a3659c640 1 --2- 172.21.15.142:0/2548536866 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0a2006dcd0 0x7f0a20070190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0a28006010 tx=0x7f0a28000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:02.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:02.793+0000 7f0a277fe640 1 -- 172.21.15.142:0/2548536866 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0a2c057610 con 0x7f0a38101150 2024-03-29T05:13:03.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.024+0000 7f0a3c96e640 1 -- 172.21.15.142:0/2548536866 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f0a380ffe60 con 0x7f0a38101150 2024-03-29T05:13:03.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.025+0000 7f0a277fe640 1 -- 172.21.15.142:0/2548536866 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v39) v1 ==== 107+0+5063 (secure 0 0 0) 0x7f0a380ffe60 con 0x7f0a38101150 2024-03-29T05:13:03.031 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:03.031 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":34424,"name":"cephfs.smithi142.bmkjzf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6829/1735570051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1735570051},{"type":"v1","addr":"172.21.15.142:6829","nonce":1735570051}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44255,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/869112786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":869112786},{"type":"v1","addr":"172.21.15.152:6827","nonce":869112786}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44323,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/778282077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":778282077},{"type":"v1","addr":"172.21.15.152:6825","nonce":778282077}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:54:33.202220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:03.031 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 30 2024-03-29T05:13:03.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.027+0000 7f0a3c96e640 1 -- 172.21.15.142:0/2548536866 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0a2006dcd0 msgr2=0x7f0a20070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:03.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.027+0000 7f0a3c96e640 1 --2- 172.21.15.142:0/2548536866 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0a2006dcd0 0x7f0a20070190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0a28006010 tx=0x7f0a28000bc0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.027+0000 7f0a3c96e640 1 -- 172.21.15.142:0/2548536866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a38101150 msgr2=0x7f0a381015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:03.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.027+0000 7f0a3c96e640 1 --2- 172.21.15.142:0/2548536866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a38101150 0x7f0a381015b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00e9a0 tx=0x7f0a2c00ee60 comp rx=0 tx=0).stop 2024-03-29T05:13:03.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.027+0000 7f0a3c96e640 1 -- 172.21.15.142:0/2548536866 shutdown_connections 2024-03-29T05:13:03.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.027+0000 7f0a3c96e640 1 --2- 172.21.15.142:0/2548536866 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f0a2006dcd0 0x7f0a20070190 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.027+0000 7f0a3c96e640 1 --2- 172.21.15.142:0/2548536866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a38101150 0x7f0a381015b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.027+0000 7f0a3c96e640 1 --2- 172.21.15.142:0/2548536866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a380ff260 0x7f0a38100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.027+0000 7f0a3c96e640 1 -- 172.21.15.142:0/2548536866 >> 172.21.15.142:0/2548536866 conn(0x7f0a38074a50 msgr2=0x7f0a38103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:03.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.027+0000 7f0a3c96e640 1 -- 172.21.15.142:0/2548536866 shutdown_connections 2024-03-29T05:13:03.033 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.027+0000 7f0a3c96e640 1 -- 172.21.15.142:0/2548536866 wait complete. 2024-03-29T05:13:03.042 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:02 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2145907286' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-29T05:13:03.042 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:02 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2508215987' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-29T05:13:03.042 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:02 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/574696712' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-29T05:13:03.044 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T05:13:03.044 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-03-29T05:13:03.044 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-29T05:13:03.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.141+0000 7ff431eb6640 1 -- 172.21.15.142:0/1842668453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42c0ff1a0 msgr2=0x7ff42c0ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:03.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.141+0000 7ff431eb6640 1 --2- 172.21.15.142:0/1842668453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42c0ff1a0 0x7ff42c0ff580 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009880 tx=0x7ff41c02bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.141+0000 7ff431eb6640 1 -- 172.21.15.142:0/1842668453 shutdown_connections 2024-03-29T05:13:03.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.141+0000 7ff431eb6640 1 --2- 172.21.15.142:0/1842668453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff42c0ffb50 0x7ff42c103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.141+0000 7ff431eb6640 1 --2- 172.21.15.142:0/1842668453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42c0ff1a0 0x7ff42c0ff580 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.141+0000 7ff431eb6640 1 -- 172.21.15.142:0/1842668453 >> 172.21.15.142:0/1842668453 conn(0x7ff42c0faa60 msgr2=0x7ff42c0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.141+0000 7ff431eb6640 1 -- 172.21.15.142:0/1842668453 shutdown_connections 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.141+0000 7ff431eb6640 1 -- 172.21.15.142:0/1842668453 wait complete. 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.142+0000 7ff431eb6640 1 Processor -- start 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.142+0000 7ff431eb6640 1 -- start start 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.142+0000 7ff431eb6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff42c0ff1a0 0x7ff42c194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.142+0000 7ff431eb6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42c1952c0 0x7ff42c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.142+0000 7ff431eb6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c1958c0 con 0x7ff42c1952c0 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.142+0000 7ff431eb6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c195a30 con 0x7ff42c0ff1a0 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.142+0000 7ff42affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42c1952c0 0x7ff42c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.142+0000 7ff42affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42c1952c0 0x7ff42c199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47044/0 (socket says 172.21.15.142:47044) 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.142+0000 7ff42affd640 1 -- 172.21.15.142:0/1227577287 learned_addr learned my addr 172.21.15.142:0/1227577287 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:03.150 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.143+0000 7ff42b7fe640 1 --2- 172.21.15.142:0/1227577287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff42c0ff1a0 0x7ff42c194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:03.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.143+0000 7ff42affd640 1 -- 172.21.15.142:0/1227577287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff42c0ff1a0 msgr2=0x7ff42c194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:03.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.143+0000 7ff42affd640 1 --2- 172.21.15.142:0/1227577287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff42c0ff1a0 0x7ff42c194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.143+0000 7ff42affd640 1 -- 172.21.15.142:0/1227577287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c009560 con 0x7ff42c1952c0 2024-03-29T05:13:03.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.143+0000 7ff42b7fe640 1 --2- 172.21.15.142:0/1227577287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff42c0ff1a0 0x7ff42c194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T05:13:03.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.143+0000 7ff42affd640 1 --2- 172.21.15.142:0/1227577287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42c1952c0 0x7ff42c199720 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff42000b9f0 tx=0x7ff42000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:03.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.143+0000 7ff428ff9640 1 -- 172.21.15.142:0/1227577287 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff42000cb20 con 0x7ff42c1952c0 2024-03-29T05:13:03.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.143+0000 7ff428ff9640 1 -- 172.21.15.142:0/1227577287 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff42000cc80 con 0x7ff42c1952c0 2024-03-29T05:13:03.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.143+0000 7ff431eb6640 1 -- 172.21.15.142:0/1227577287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42c199d20 con 0x7ff42c1952c0 2024-03-29T05:13:03.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.143+0000 7ff428ff9640 1 -- 172.21.15.142:0/1227577287 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4200126f0 con 0x7ff42c1952c0 2024-03-29T05:13:03.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.143+0000 7ff431eb6640 1 -- 172.21.15.142:0/1227577287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42c19a280 con 0x7ff42c1952c0 2024-03-29T05:13:03.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.144+0000 7ff431eb6640 1 -- 172.21.15.142:0/1227577287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42c19a560 con 0x7ff42c1952c0 2024-03-29T05:13:03.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.145+0000 7ff428ff9640 1 -- 172.21.15.142:0/1227577287 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7ff420012960 con 0x7ff42c1952c0 2024-03-29T05:13:03.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.145+0000 7ff428ff9640 1 --2- 172.21.15.142:0/1227577287 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff41406dcd0 0x7ff414070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:03.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.146+0000 7ff428ff9640 1 -- 172.21.15.142:0/1227577287 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff42008d670 con 0x7ff42c1952c0 2024-03-29T05:13:03.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.146+0000 7ff42b7fe640 1 --2- 172.21.15.142:0/1227577287 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff41406dcd0 0x7ff414070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:03.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.146+0000 7ff42b7fe640 1 --2- 172.21.15.142:0/1227577287 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff41406dcd0 0x7ff414070190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff41c005ec0 tx=0x7ff41c005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:03.159 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.153+0000 7ff428ff9640 1 -- 172.21.15.142:0/1227577287 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff420056b40 con 0x7ff42c1952c0 2024-03-29T05:13:03.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:02 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2145907286' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-29T05:13:03.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:02 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2508215987' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-29T05:13:03.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:02 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/574696712' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-29T05:13:03.377 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.372+0000 7ff431eb6640 1 -- 172.21.15.142:0/1227577287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7ff42c196140 con 0x7ff42c1952c0 2024-03-29T05:13:03.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.372+0000 7ff428ff9640 1 -- 172.21.15.142:0/1227577287 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v39) v1 ==== 107+0+4284 (secure 0 0 0) 0x7ff42005a200 con 0x7ff42c1952c0 2024-03-29T05:13:03.378 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:03.379 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":34424,"name":"cephfs.smithi142.bmkjzf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6829/1735570051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1735570051},{"type":"v1","addr":"172.21.15.142:6829","nonce":1735570051}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44323,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/778282077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":778282077},{"type":"v1","addr":"172.21.15.152:6825","nonce":778282077}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:54:33.202220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:03.379 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 31 2024-03-29T05:13:03.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.375+0000 7ff431eb6640 1 -- 172.21.15.142:0/1227577287 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff41406dcd0 msgr2=0x7ff414070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:03.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.375+0000 7ff431eb6640 1 --2- 172.21.15.142:0/1227577287 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff41406dcd0 0x7ff414070190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff41c005ec0 tx=0x7ff41c005fd0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.375+0000 7ff431eb6640 1 -- 172.21.15.142:0/1227577287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42c1952c0 msgr2=0x7ff42c199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:03.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.375+0000 7ff431eb6640 1 --2- 172.21.15.142:0/1227577287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42c1952c0 0x7ff42c199720 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff42000b9f0 tx=0x7ff42000beb0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.375+0000 7ff431eb6640 1 -- 172.21.15.142:0/1227577287 shutdown_connections 2024-03-29T05:13:03.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.375+0000 7ff431eb6640 1 --2- 172.21.15.142:0/1227577287 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7ff41406dcd0 0x7ff414070190 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.375+0000 7ff431eb6640 1 --2- 172.21.15.142:0/1227577287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff42c1952c0 0x7ff42c199720 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.375+0000 7ff431eb6640 1 --2- 172.21.15.142:0/1227577287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff42c0ff1a0 0x7ff42c194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.375+0000 7ff431eb6640 1 -- 172.21.15.142:0/1227577287 >> 172.21.15.142:0/1227577287 conn(0x7ff42c0faa60 msgr2=0x7ff42c0fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:03.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.375+0000 7ff431eb6640 1 -- 172.21.15.142:0/1227577287 shutdown_connections 2024-03-29T05:13:03.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.375+0000 7ff431eb6640 1 -- 172.21.15.142:0/1227577287 wait complete. 2024-03-29T05:13:03.392 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T05:13:03.392 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-03-29T05:13:03.392 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-29T05:13:03.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.492+0000 7f46eace0640 1 -- 172.21.15.142:0/3734863671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e4067f70 msgr2=0x7f46e4104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:03.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.492+0000 7f46eace0640 1 --2- 172.21.15.142:0/3734863671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e4067f70 0x7f46e4104c70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009880 tx=0x7f46d002bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.492+0000 7f46eace0640 1 -- 172.21.15.142:0/3734863671 shutdown_connections 2024-03-29T05:13:03.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.492+0000 7f46eace0640 1 --2- 172.21.15.142:0/3734863671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e4067f70 0x7f46e4104c70 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.492+0000 7f46eace0640 1 --2- 172.21.15.142:0/3734863671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46e40675c0 0x7f46e40679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.492+0000 7f46eace0640 1 -- 172.21.15.142:0/3734863671 >> 172.21.15.142:0/3734863671 conn(0x7f46e4074a50 msgr2=0x7f46e4074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:03.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.493+0000 7f46eace0640 1 -- 172.21.15.142:0/3734863671 shutdown_connections 2024-03-29T05:13:03.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.493+0000 7f46eace0640 1 -- 172.21.15.142:0/3734863671 wait complete. 2024-03-29T05:13:03.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.493+0000 7f46eace0640 1 Processor -- start 2024-03-29T05:13:03.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.493+0000 7f46eace0640 1 -- start start 2024-03-29T05:13:03.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46eace0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e40675c0 0x7f46e4195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:03.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46eace0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46e41962a0 0x7f46e419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:03.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46eace0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e4196810 con 0x7f46e40675c0 2024-03-29T05:13:03.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46eace0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e4196980 con 0x7f46e41962a0 2024-03-29T05:13:03.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46e8a7c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e40675c0 0x7f46e4195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:03.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46e8a7c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e40675c0 0x7f46e4195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47058/0 (socket says 172.21.15.142:47058) 2024-03-29T05:13:03.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46e8a7c640 1 -- 172.21.15.142:0/253906258 learned_addr learned my addr 172.21.15.142:0/253906258 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:03.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46e3fff640 1 --2- 172.21.15.142:0/253906258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46e41962a0 0x7f46e419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:03.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46e8a7c640 1 -- 172.21.15.142:0/253906258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46e41962a0 msgr2=0x7f46e419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:03.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46e8a7c640 1 --2- 172.21.15.142:0/253906258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46e41962a0 0x7f46e419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46e8a7c640 1 -- 172.21.15.142:0/253906258 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d0009560 con 0x7f46e40675c0 2024-03-29T05:13:03.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46e8a7c640 1 --2- 172.21.15.142:0/253906258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e40675c0 0x7f46e4195d60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f46d800c8a0 tx=0x7f46d800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:03.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46e1ffb640 1 -- 172.21.15.142:0/253906258 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46d8007b80 con 0x7f46e40675c0 2024-03-29T05:13:03.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46e1ffb640 1 -- 172.21.15.142:0/253906258 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46d8007ce0 con 0x7f46e40675c0 2024-03-29T05:13:03.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46eace0640 1 -- 172.21.15.142:0/253906258 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e419aca0 con 0x7f46e40675c0 2024-03-29T05:13:03.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.494+0000 7f46e1ffb640 1 -- 172.21.15.142:0/253906258 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46d8018810 con 0x7f46e40675c0 2024-03-29T05:13:03.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.495+0000 7f46eace0640 1 -- 172.21.15.142:0/253906258 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e419b1b0 con 0x7f46e40675c0 2024-03-29T05:13:03.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.496+0000 7f46eace0640 1 -- 172.21.15.142:0/253906258 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e404e040 con 0x7f46e40675c0 2024-03-29T05:13:03.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.496+0000 7f46e1ffb640 1 -- 172.21.15.142:0/253906258 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f46d8018a80 con 0x7f46e40675c0 2024-03-29T05:13:03.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.497+0000 7f46e1ffb640 1 --2- 172.21.15.142:0/253906258 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f46d406dcd0 0x7f46d4070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:03.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.497+0000 7f46e1ffb640 1 -- 172.21.15.142:0/253906258 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f46d808df40 con 0x7f46e40675c0 2024-03-29T05:13:03.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.497+0000 7f46e3fff640 1 --2- 172.21.15.142:0/253906258 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f46d406dcd0 0x7f46d4070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:03.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.497+0000 7f46e3fff640 1 --2- 172.21.15.142:0/253906258 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f46d406dcd0 0x7f46d4070190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f46d0006010 tx=0x7f46d0000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:03.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.502+0000 7f46e1ffb640 1 -- 172.21.15.142:0/253906258 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f46d80c2280 con 0x7f46e40675c0 2024-03-29T05:13:03.737 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.731+0000 7f46eace0640 1 -- 172.21.15.142:0/253906258 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f46e4108580 con 0x7f46e40675c0 2024-03-29T05:13:03.738 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.732+0000 7f46e1ffb640 1 -- 172.21.15.142:0/253906258 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v39) v1 ==== 107+0+5134 (secure 0 0 0) 0x7f46d8057360 con 0x7f46e40675c0 2024-03-29T05:13:03.738 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:03.738 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":34424,"name":"cephfs.smithi142.bmkjzf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6829/1735570051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1735570051},{"type":"v1","addr":"172.21.15.142:6829","nonce":1735570051}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44323,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/778282077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":778282077},{"type":"v1","addr":"172.21.15.152:6825","nonce":778282077}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44327,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/2852002359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2852002359},{"type":"v1","addr":"172.21.15.152:6827","nonce":2852002359}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T04:54:33.202220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:03.738 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 32 2024-03-29T05:13:03.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.735+0000 7f46eace0640 1 -- 172.21.15.142:0/253906258 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f46d406dcd0 msgr2=0x7f46d4070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:03.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.735+0000 7f46eace0640 1 --2- 172.21.15.142:0/253906258 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f46d406dcd0 0x7f46d4070190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f46d0006010 tx=0x7f46d0000bc0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.735+0000 7f46eace0640 1 -- 172.21.15.142:0/253906258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e40675c0 msgr2=0x7f46e4195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:03.741 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.735+0000 7f46eace0640 1 --2- 172.21.15.142:0/253906258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e40675c0 0x7f46e4195d60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f46d800c8a0 tx=0x7f46d800cd60 comp rx=0 tx=0).stop 2024-03-29T05:13:03.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.735+0000 7f46eace0640 1 -- 172.21.15.142:0/253906258 shutdown_connections 2024-03-29T05:13:03.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.735+0000 7f46eace0640 1 --2- 172.21.15.142:0/253906258 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f46d406dcd0 0x7f46d4070190 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.735+0000 7f46eace0640 1 --2- 172.21.15.142:0/253906258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46e41962a0 0x7f46e419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.735+0000 7f46eace0640 1 --2- 172.21.15.142:0/253906258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46e40675c0 0x7f46e4195d60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.735+0000 7f46eace0640 1 -- 172.21.15.142:0/253906258 >> 172.21.15.142:0/253906258 conn(0x7f46e4074a50 msgr2=0x7f46e40fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:03.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.735+0000 7f46eace0640 1 -- 172.21.15.142:0/253906258 shutdown_connections 2024-03-29T05:13:03.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.735+0000 7f46eace0640 1 -- 172.21.15.142:0/253906258 wait complete. 2024-03-29T05:13:03.752 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T05:13:03.752 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-03-29T05:13:03.752 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-29T05:13:03.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 -- 172.21.15.142:0/27565741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4000675c0 msgr2=0x7fc4000679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:03.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 --2- 172.21.15.142:0/27565741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4000675c0 0x7fc4000679a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009c20 tx=0x7fc3f802c1b0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 -- 172.21.15.142:0/27565741 shutdown_connections 2024-03-29T05:13:03.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 --2- 172.21.15.142:0/27565741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc400067f70 0x7fc400104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 --2- 172.21.15.142:0/27565741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4000675c0 0x7fc4000679a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 -- 172.21.15.142:0/27565741 >> 172.21.15.142:0/27565741 conn(0x7fc400074a50 msgr2=0x7fc400074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:03.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 -- 172.21.15.142:0/27565741 shutdown_connections 2024-03-29T05:13:03.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 -- 172.21.15.142:0/27565741 wait complete. 2024-03-29T05:13:03.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 Processor -- start 2024-03-29T05:13:03.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 -- start start 2024-03-29T05:13:03.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4000675c0 0x7fc400195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:03.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4001962f0 0x7fc40019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:03.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4001968f0 con 0x7fc4001962f0 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.852+0000 7fc407959640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc400196a60 con 0x7fc4000675c0 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc404ef4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4001962f0 0x7fc40019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc404ef4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4001962f0 0x7fc40019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47084/0 (socket says 172.21.15.142:47084) 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc404ef4640 1 -- 172.21.15.142:0/3885661088 learned_addr learned my addr 172.21.15.142:0/3885661088 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc4056f5640 1 --2- 172.21.15.142:0/3885661088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4000675c0 0x7fc400195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc404ef4640 1 -- 172.21.15.142:0/3885661088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4000675c0 msgr2=0x7fc400195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc404ef4640 1 --2- 172.21.15.142:0/3885661088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4000675c0 0x7fc400195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc404ef4640 1 -- 172.21.15.142:0/3885661088 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f8009900 con 0x7fc4001962f0 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc404ef4640 1 --2- 172.21.15.142:0/3885661088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4001962f0 0x7fc40019a750 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00c920 tx=0x7fc3fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc3f27fc640 1 -- 172.21.15.142:0/3885661088 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3fc007b80 con 0x7fc4001962f0 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc3f27fc640 1 -- 172.21.15.142:0/3885661088 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3fc007ce0 con 0x7fc4001962f0 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc3f27fc640 1 -- 172.21.15.142:0/3885661088 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3fc0187d0 con 0x7fc4001962f0 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc407959640 1 -- 172.21.15.142:0/3885661088 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40019ad50 con 0x7fc4001962f0 2024-03-29T05:13:03.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.853+0000 7fc407959640 1 -- 172.21.15.142:0/3885661088 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40019b2b0 con 0x7fc4001962f0 2024-03-29T05:13:03.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.854+0000 7fc407959640 1 -- 172.21.15.142:0/3885661088 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40019b590 con 0x7fc4001962f0 2024-03-29T05:13:03.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.855+0000 7fc3f27fc640 1 -- 172.21.15.142:0/3885661088 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fc3fc00f450 con 0x7fc4001962f0 2024-03-29T05:13:03.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.855+0000 7fc3f27fc640 1 --2- 172.21.15.142:0/3885661088 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc3ec06dcd0 0x7fc3ec070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:03.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.855+0000 7fc3f27fc640 1 -- 172.21.15.142:0/3885661088 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fc3fc08deb0 con 0x7fc4001962f0 2024-03-29T05:13:03.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.855+0000 7fc4056f5640 1 --2- 172.21.15.142:0/3885661088 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc3ec06dcd0 0x7fc3ec070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:03.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.856+0000 7fc4056f5640 1 --2- 172.21.15.142:0/3885661088 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc3ec06dcd0 0x7fc3ec070190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8033000 tx=0x7fc3f8005f10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:03.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:03.864+0000 7fc3f27fc640 1 -- 172.21.15.142:0/3885661088 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc3fc0572d0 con 0x7fc4001962f0 2024-03-29T05:13:04.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.090+0000 7fc407959640 1 -- 172.21.15.142:0/3885661088 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fc400197100 con 0x7fc4001962f0 2024-03-29T05:13:04.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.091+0000 7fc3f27fc640 1 -- 172.21.15.142:0/3885661088 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v39) v1 ==== 107+0+5224 (secure 0 0 0) 0x7fc3fc05a990 con 0x7fc4001962f0 2024-03-29T05:13:04.097 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:04.097 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/778282077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":778282077},{"type":"v1","addr":"172.21.15.152:6825","nonce":778282077}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44327,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/2852002359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2852002359},{"type":"v1","addr":"172.21.15.152:6827","nonce":2852002359}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T05:10:36.358889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi142.bmkjzf","rank":1,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.142:6829/1735570051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1735570051},{"type":"v1","addr":"172.21.15.142:6829","nonce":1735570051}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:04.098 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 33 2024-03-29T05:13:04.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.093+0000 7fc407959640 1 -- 172.21.15.142:0/3885661088 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc3ec06dcd0 msgr2=0x7fc3ec070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:04.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.093+0000 7fc407959640 1 --2- 172.21.15.142:0/3885661088 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc3ec06dcd0 0x7fc3ec070190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8033000 tx=0x7fc3f8005f10 comp rx=0 tx=0).stop 2024-03-29T05:13:04.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.093+0000 7fc407959640 1 -- 172.21.15.142:0/3885661088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4001962f0 msgr2=0x7fc40019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:04.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.093+0000 7fc407959640 1 --2- 172.21.15.142:0/3885661088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4001962f0 0x7fc40019a750 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00c920 tx=0x7fc3fc00cde0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.099 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.093+0000 7fc407959640 1 -- 172.21.15.142:0/3885661088 shutdown_connections 2024-03-29T05:13:04.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.093+0000 7fc407959640 1 --2- 172.21.15.142:0/3885661088 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc3ec06dcd0 0x7fc3ec070190 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.093+0000 7fc407959640 1 --2- 172.21.15.142:0/3885661088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4001962f0 0x7fc40019a750 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.093+0000 7fc407959640 1 --2- 172.21.15.142:0/3885661088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4000675c0 0x7fc400195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.093+0000 7fc407959640 1 -- 172.21.15.142:0/3885661088 >> 172.21.15.142:0/3885661088 conn(0x7fc400074a50 msgr2=0x7fc4000fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:04.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.093+0000 7fc407959640 1 -- 172.21.15.142:0/3885661088 shutdown_connections 2024-03-29T05:13:04.100 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.093+0000 7fc407959640 1 -- 172.21.15.142:0/3885661088 wait complete. 2024-03-29T05:13:04.109 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:03 smithi142 ceph-mon[112475]: pgmap v743: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:13:04.109 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:03 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2548536866' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-29T05:13:04.109 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:03 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1227577287' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-29T05:13:04.109 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:03 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/253906258' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-29T05:13:04.110 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-03-29T05:13:04.110 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-29T05:13:04.215 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.209+0000 7f77e647a640 1 -- 172.21.15.142:0/3003027605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e00ffc10 msgr2=0x7f77e0103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:04.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.209+0000 7f77e647a640 1 --2- 172.21.15.142:0/3003027605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e00ffc10 0x7f77e0103b70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009880 tx=0x7f77d402bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.209+0000 7f77e647a640 1 -- 172.21.15.142:0/3003027605 shutdown_connections 2024-03-29T05:13:04.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.209+0000 7f77e647a640 1 --2- 172.21.15.142:0/3003027605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e00ffc10 0x7f77e0103b70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.209+0000 7f77e647a640 1 --2- 172.21.15.142:0/3003027605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77e00ff260 0x7f77e00ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.209+0000 7f77e647a640 1 -- 172.21.15.142:0/3003027605 >> 172.21.15.142:0/3003027605 conn(0x7f77e0074a50 msgr2=0x7f77e0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:04.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.209+0000 7f77e647a640 1 -- 172.21.15.142:0/3003027605 shutdown_connections 2024-03-29T05:13:04.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.209+0000 7f77e647a640 1 -- 172.21.15.142:0/3003027605 wait complete. 2024-03-29T05:13:04.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77e647a640 1 Processor -- start 2024-03-29T05:13:04.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77e647a640 1 -- start start 2024-03-29T05:13:04.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77e647a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77e00ff260 0x7f77e0195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:04.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77e647a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e01962f0 0x7f77e019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:04.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77e647a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e01968f0 con 0x7f77e01962f0 2024-03-29T05:13:04.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77e647a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e0196a60 con 0x7f77e00ff260 2024-03-29T05:13:04.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77df7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e01962f0 0x7f77e019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:04.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77df7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e01962f0 0x7f77e019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47102/0 (socket says 172.21.15.142:47102) 2024-03-29T05:13:04.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77df7fe640 1 -- 172.21.15.142:0/2033492684 learned_addr learned my addr 172.21.15.142:0/2033492684 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:04.219 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77dffff640 1 --2- 172.21.15.142:0/2033492684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77e00ff260 0x7f77e0195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77df7fe640 1 -- 172.21.15.142:0/2033492684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77e00ff260 msgr2=0x7f77e0195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77df7fe640 1 --2- 172.21.15.142:0/2033492684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77e00ff260 0x7f77e0195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.210+0000 7f77df7fe640 1 -- 172.21.15.142:0/2033492684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d4009560 con 0x7f77e01962f0 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.211+0000 7f77df7fe640 1 --2- 172.21.15.142:0/2033492684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e01962f0 0x7f77e019a750 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f77d4036040 tx=0x7f77d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.211+0000 7f77dd7fa640 1 -- 172.21.15.142:0/2033492684 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77d40037c0 con 0x7f77e01962f0 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.211+0000 7f77dd7fa640 1 -- 172.21.15.142:0/2033492684 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77d4003de0 con 0x7f77e01962f0 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.211+0000 7f77e647a640 1 -- 172.21.15.142:0/2033492684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e019acf0 con 0x7f77e01962f0 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.211+0000 7f77dd7fa640 1 -- 172.21.15.142:0/2033492684 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77d4034440 con 0x7f77e01962f0 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.211+0000 7f77e647a640 1 -- 172.21.15.142:0/2033492684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e019b250 con 0x7f77e01962f0 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.212+0000 7f77e647a640 1 -- 172.21.15.142:0/2033492684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e004e040 con 0x7f77e01962f0 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.212+0000 7f77dd7fa640 1 -- 172.21.15.142:0/2033492684 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f77d4004fb0 con 0x7f77e01962f0 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.213+0000 7f77dd7fa640 1 --2- 172.21.15.142:0/2033492684 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f77c806dcd0 0x7f77c8070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.213+0000 7f77dd7fa640 1 -- 172.21.15.142:0/2033492684 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f77d40b0160 con 0x7f77e01962f0 2024-03-29T05:13:04.220 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.213+0000 7f77dffff640 1 --2- 172.21.15.142:0/2033492684 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f77c806dcd0 0x7f77c8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:04.221 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.213+0000 7f77dffff640 1 --2- 172.21.15.142:0/2033492684 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f77c806dcd0 0x7f77c8070190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f77d00058c0 tx=0x7f77d000a4c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:04.223 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.218+0000 7f77dd7fa640 1 -- 172.21.15.142:0/2033492684 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f77d4079580 con 0x7f77e01962f0 2024-03-29T05:13:04.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:03 smithi152 ceph-mon[88186]: pgmap v743: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:13:04.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:03 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2548536866' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-29T05:13:04.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:03 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1227577287' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-29T05:13:04.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:03 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/253906258' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-29T05:13:04.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.438+0000 7f77e647a640 1 -- 172.21.15.142:0/2033492684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f77e01971e0 con 0x7f77e01962f0 2024-03-29T05:13:04.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.439+0000 7f77dd7fa640 1 -- 172.21.15.142:0/2033492684 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v39) v1 ==== 107+0+5223 (secure 0 0 0) 0x7f77e01971e0 con 0x7f77e01962f0 2024-03-29T05:13:04.445 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:04.445 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/778282077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":778282077},{"type":"v1","addr":"172.21.15.152:6825","nonce":778282077}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44327,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/2852002359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2852002359},{"type":"v1","addr":"172.21.15.152:6827","nonce":2852002359}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T05:10:36.365826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi142.bmkjzf","rank":1,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.142:6829/1735570051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1735570051},{"type":"v1","addr":"172.21.15.142:6829","nonce":1735570051}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:04.445 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 34 2024-03-29T05:13:04.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.441+0000 7f77e647a640 1 -- 172.21.15.142:0/2033492684 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f77c806dcd0 msgr2=0x7f77c8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:04.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.441+0000 7f77e647a640 1 --2- 172.21.15.142:0/2033492684 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f77c806dcd0 0x7f77c8070190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f77d00058c0 tx=0x7f77d000a4c0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.441+0000 7f77e647a640 1 -- 172.21.15.142:0/2033492684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e01962f0 msgr2=0x7f77e019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:04.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.441+0000 7f77e647a640 1 --2- 172.21.15.142:0/2033492684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e01962f0 0x7f77e019a750 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f77d4036040 tx=0x7f77d4004270 comp rx=0 tx=0).stop 2024-03-29T05:13:04.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.441+0000 7f77e647a640 1 -- 172.21.15.142:0/2033492684 shutdown_connections 2024-03-29T05:13:04.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.441+0000 7f77e647a640 1 --2- 172.21.15.142:0/2033492684 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f77c806dcd0 0x7f77c8070190 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.441+0000 7f77e647a640 1 --2- 172.21.15.142:0/2033492684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e01962f0 0x7f77e019a750 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.441+0000 7f77e647a640 1 --2- 172.21.15.142:0/2033492684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77e00ff260 0x7f77e0195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.447 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.441+0000 7f77e647a640 1 -- 172.21.15.142:0/2033492684 >> 172.21.15.142:0/2033492684 conn(0x7f77e0074a50 msgr2=0x7f77e0103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:04.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.441+0000 7f77e647a640 1 -- 172.21.15.142:0/2033492684 shutdown_connections 2024-03-29T05:13:04.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.441+0000 7f77e647a640 1 -- 172.21.15.142:0/2033492684 wait complete. 2024-03-29T05:13:04.458 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-29T05:13:04.569 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.562+0000 7f207f5ee640 1 -- 172.21.15.142:0/148928110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2078104f70 msgr2=0x7f2078105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:04.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.562+0000 7f207f5ee640 1 --2- 172.21.15.142:0/148928110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2078104f70 0x7f2078105350 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2074009880 tx=0x7f207402bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.562+0000 7f207f5ee640 1 -- 172.21.15.142:0/148928110 shutdown_connections 2024-03-29T05:13:04.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.562+0000 7f207f5ee640 1 --2- 172.21.15.142:0/148928110 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20780fefb0 0x7f20780ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.562+0000 7f207f5ee640 1 --2- 172.21.15.142:0/148928110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2078104f70 0x7f2078105350 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.562+0000 7f207f5ee640 1 -- 172.21.15.142:0/148928110 >> 172.21.15.142:0/148928110 conn(0x7f2078074a80 msgr2=0x7f2078074e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:04.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.562+0000 7f207f5ee640 1 -- 172.21.15.142:0/148928110 shutdown_connections 2024-03-29T05:13:04.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.562+0000 7f207f5ee640 1 -- 172.21.15.142:0/148928110 wait complete. 2024-03-29T05:13:04.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.562+0000 7f207f5ee640 1 Processor -- start 2024-03-29T05:13:04.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.562+0000 7f207f5ee640 1 -- start start 2024-03-29T05:13:04.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207f5ee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20780fefb0 0x7f2078100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207f5ee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2078104f70 0x7f2078101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207f5ee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078101650 con 0x7f20780fefb0 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207f5ee640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20781017c0 con 0x7f2078104f70 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207d38a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20780fefb0 0x7f2078100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207d38a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20780fefb0 0x7f2078100bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47114/0 (socket says 172.21.15.142:47114) 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207d38a640 1 -- 172.21.15.142:0/2306138606 learned_addr learned my addr 172.21.15.142:0/2306138606 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207cb89640 1 --2- 172.21.15.142:0/2306138606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2078104f70 0x7f2078101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207d38a640 1 -- 172.21.15.142:0/2306138606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2078104f70 msgr2=0x7f2078101110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207d38a640 1 --2- 172.21.15.142:0/2306138606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2078104f70 0x7f2078101110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207d38a640 1 -- 172.21.15.142:0/2306138606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2074009560 con 0x7f20780fefb0 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207cb89640 1 --2- 172.21.15.142:0/2306138606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2078104f70 0x7f2078101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207d38a640 1 --2- 172.21.15.142:0/2306138606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20780fefb0 0x7f2078100bd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2074009230 tx=0x7f2074004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:04.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f206e7fc640 1 -- 172.21.15.142:0/2306138606 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20740035f0 con 0x7f20780fefb0 2024-03-29T05:13:04.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f207f5ee640 1 -- 172.21.15.142:0/2306138606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20781019e0 con 0x7f20780fefb0 2024-03-29T05:13:04.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f206e7fc640 1 -- 172.21.15.142:0/2306138606 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2074003c10 con 0x7f20780fefb0 2024-03-29T05:13:04.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.563+0000 7f206e7fc640 1 -- 172.21.15.142:0/2306138606 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2074034440 con 0x7f20780fefb0 2024-03-29T05:13:04.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.564+0000 7f207f5ee640 1 -- 172.21.15.142:0/2306138606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20781a22e0 con 0x7f20780fefb0 2024-03-29T05:13:04.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.564+0000 7f207f5ee640 1 -- 172.21.15.142:0/2306138606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207804e040 con 0x7f20780fefb0 2024-03-29T05:13:04.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.565+0000 7f206e7fc640 1 -- 172.21.15.142:0/2306138606 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f2074004fd0 con 0x7f20780fefb0 2024-03-29T05:13:04.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.566+0000 7f206e7fc640 1 --2- 172.21.15.142:0/2306138606 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f206406dcd0 0x7f2064070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:04.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.566+0000 7f206e7fc640 1 -- 172.21.15.142:0/2306138606 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f20740b0120 con 0x7f20780fefb0 2024-03-29T05:13:04.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.566+0000 7f207cb89640 1 --2- 172.21.15.142:0/2306138606 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f206406dcd0 0x7f2064070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:04.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.566+0000 7f207cb89640 1 --2- 172.21.15.142:0/2306138606 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f206406dcd0 0x7f2064070190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f20781023c0 tx=0x7f2068008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:04.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.570+0000 7f206e7fc640 1 -- 172.21.15.142:0/2306138606 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2074079540 con 0x7f20780fefb0 2024-03-29T05:13:04.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.789+0000 7f207f5ee640 1 -- 172.21.15.142:0/2306138606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f207810c3f0 con 0x7f20780fefb0 2024-03-29T05:13:04.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.790+0000 7f206e7fc640 1 -- 172.21.15.142:0/2306138606 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v39) v1 ==== 107+0+5229 (secure 0 0 0) 0x7f207407cc00 con 0x7f20780fefb0 2024-03-29T05:13:04.796 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:04.796 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/778282077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":778282077},{"type":"v1","addr":"172.21.15.152:6825","nonce":778282077}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44327,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/2852002359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2852002359},{"type":"v1","addr":"172.21.15.152:6827","nonce":2852002359}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T05:10:43.442422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":0,"incarnation":33,"state":"up:resolve","state_seq":242,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi142.bmkjzf","rank":1,"incarnation":33,"state":"up:resolve","state_seq":238,"addr":"172.21.15.142:6829/1735570051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1735570051},{"type":"v1","addr":"172.21.15.142:6829","nonce":1735570051}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:04.796 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 35 2024-03-29T05:13:04.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.792+0000 7f207f5ee640 1 -- 172.21.15.142:0/2306138606 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f206406dcd0 msgr2=0x7f2064070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:04.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.792+0000 7f207f5ee640 1 --2- 172.21.15.142:0/2306138606 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f206406dcd0 0x7f2064070190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f20781023c0 tx=0x7f2068008040 comp rx=0 tx=0).stop 2024-03-29T05:13:04.798 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.792+0000 7f207f5ee640 1 -- 172.21.15.142:0/2306138606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20780fefb0 msgr2=0x7f2078100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:04.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.792+0000 7f207f5ee640 1 --2- 172.21.15.142:0/2306138606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20780fefb0 0x7f2078100bd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2074009230 tx=0x7f2074004040 comp rx=0 tx=0).stop 2024-03-29T05:13:04.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.792+0000 7f207f5ee640 1 -- 172.21.15.142:0/2306138606 shutdown_connections 2024-03-29T05:13:04.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.792+0000 7f207f5ee640 1 --2- 172.21.15.142:0/2306138606 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f206406dcd0 0x7f2064070190 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.792+0000 7f207f5ee640 1 --2- 172.21.15.142:0/2306138606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2078104f70 0x7f2078101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.792+0000 7f207f5ee640 1 --2- 172.21.15.142:0/2306138606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20780fefb0 0x7f2078100bd0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.792+0000 7f207f5ee640 1 -- 172.21.15.142:0/2306138606 >> 172.21.15.142:0/2306138606 conn(0x7f2078074a80 msgr2=0x7f20780fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:04.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.792+0000 7f207f5ee640 1 -- 172.21.15.142:0/2306138606 shutdown_connections 2024-03-29T05:13:04.799 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.792+0000 7f207f5ee640 1 -- 172.21.15.142:0/2306138606 wait complete. 2024-03-29T05:13:04.809 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-03-29T05:13:04.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.922+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/406575645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe640ff1b0 msgr2=0x7fbe640ff590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:04.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.922+0000 7fbe6ac9d640 1 --2- 172.21.15.142:0/406575645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe640ff1b0 0x7fbe640ff590 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009880 tx=0x7fbe5802bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.930 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.922+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/406575645 shutdown_connections 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.922+0000 7fbe6ac9d640 1 --2- 172.21.15.142:0/406575645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe640ffb60 0x7fbe64103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.922+0000 7fbe6ac9d640 1 --2- 172.21.15.142:0/406575645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe640ff1b0 0x7fbe640ff590 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.922+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/406575645 >> 172.21.15.142:0/406575645 conn(0x7fbe640faa30 msgr2=0x7fbe640fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.922+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/406575645 shutdown_connections 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.922+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/406575645 wait complete. 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.923+0000 7fbe6ac9d640 1 Processor -- start 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.923+0000 7fbe6ac9d640 1 -- start start 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.923+0000 7fbe6ac9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe640ff1b0 0x7fbe64196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.923+0000 7fbe6ac9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe64197200 0x7fbe64197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.923+0000 7fbe6ac9d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6419fdc0 con 0x7fbe64197200 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.923+0000 7fbe6ac9d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6419ff00 con 0x7fbe640ff1b0 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.923+0000 7fbe63fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe64197200 0x7fbe64197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:04.931 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.923+0000 7fbe63fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe64197200 0x7fbe64197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47146/0 (socket says 172.21.15.142:47146) 2024-03-29T05:13:04.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.923+0000 7fbe63fff640 1 -- 172.21.15.142:0/1033526186 learned_addr learned my addr 172.21.15.142:0/1033526186 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:04.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.923+0000 7fbe68a39640 1 --2- 172.21.15.142:0/1033526186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe640ff1b0 0x7fbe64196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:04.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.924+0000 7fbe63fff640 1 -- 172.21.15.142:0/1033526186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe640ff1b0 msgr2=0x7fbe64196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:04.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.924+0000 7fbe63fff640 1 --2- 172.21.15.142:0/1033526186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe640ff1b0 0x7fbe64196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:04.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.924+0000 7fbe63fff640 1 -- 172.21.15.142:0/1033526186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58009560 con 0x7fbe64197200 2024-03-29T05:13:04.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.924+0000 7fbe63fff640 1 --2- 172.21.15.142:0/1033526186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe64197200 0x7fbe64197660 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000d6f0 tx=0x7fbe5000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:04.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.924+0000 7fbe61ffb640 1 -- 172.21.15.142:0/1033526186 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe500051f0 con 0x7fbe64197200 2024-03-29T05:13:04.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.924+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/1033526186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe641a0100 con 0x7fbe64197200 2024-03-29T05:13:04.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.924+0000 7fbe61ffb640 1 -- 172.21.15.142:0/1033526186 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe50009ce0 con 0x7fbe64197200 2024-03-29T05:13:04.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.924+0000 7fbe61ffb640 1 -- 172.21.15.142:0/1033526186 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe50004e80 con 0x7fbe64197200 2024-03-29T05:13:04.932 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.924+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/1033526186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe641a0660 con 0x7fbe64197200 2024-03-29T05:13:04.933 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.925+0000 7fbe61ffb640 1 -- 172.21.15.142:0/1033526186 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fbe500046c0 con 0x7fbe64197200 2024-03-29T05:13:04.933 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.925+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/1033526186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6404e040 con 0x7fbe64197200 2024-03-29T05:13:04.933 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.926+0000 7fbe61ffb640 1 --2- 172.21.15.142:0/1033526186 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbe5406dc80 0x7fbe54070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:04.933 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.926+0000 7fbe68a39640 1 --2- 172.21.15.142:0/1033526186 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbe5406dc80 0x7fbe54070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:04.933 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.926+0000 7fbe61ffb640 1 -- 172.21.15.142:0/1033526186 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fbe5008d760 con 0x7fbe64197200 2024-03-29T05:13:04.933 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.927+0000 7fbe68a39640 1 --2- 172.21.15.142:0/1033526186 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbe5406dc80 0x7fbe54070140 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbe58006010 tx=0x7fbe58000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:04.936 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:04.931+0000 7fbe61ffb640 1 -- 172.21.15.142:0/1033526186 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbe50056b80 con 0x7fbe64197200 2024-03-29T05:13:05.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:04 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3885661088' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-29T05:13:05.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:04 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2033492684' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-29T05:13:05.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:04 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2306138606' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-29T05:13:05.289 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.284+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/1033526186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7fbe640ffb60 con 0x7fbe64197200 2024-03-29T05:13:05.290 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.284+0000 7fbe61ffb640 1 -- 172.21.15.142:0/1033526186 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v39) v1 ==== 107+0+5233 (secure 0 0 0) 0x7fbe5005a240 con 0x7fbe64197200 2024-03-29T05:13:05.290 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:05.290 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/778282077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":778282077},{"type":"v1","addr":"172.21.15.152:6825","nonce":778282077}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44327,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/2852002359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2852002359},{"type":"v1","addr":"172.21.15.152:6827","nonce":2852002359}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T05:10:43.745402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":243,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi142.bmkjzf","rank":1,"incarnation":33,"state":"up:reconnect","state_seq":239,"addr":"172.21.15.142:6829/1735570051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1735570051},{"type":"v1","addr":"172.21.15.142:6829","nonce":1735570051}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:05.291 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 36 2024-03-29T05:13:05.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.287+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/1033526186 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbe5406dc80 msgr2=0x7fbe54070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:05.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.287+0000 7fbe6ac9d640 1 --2- 172.21.15.142:0/1033526186 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbe5406dc80 0x7fbe54070140 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbe58006010 tx=0x7fbe58000bc0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.287+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/1033526186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe64197200 msgr2=0x7fbe64197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:05.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.287+0000 7fbe6ac9d640 1 --2- 172.21.15.142:0/1033526186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe64197200 0x7fbe64197660 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000d6f0 tx=0x7fbe5000dbb0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.287+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/1033526186 shutdown_connections 2024-03-29T05:13:05.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.287+0000 7fbe6ac9d640 1 --2- 172.21.15.142:0/1033526186 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fbe5406dc80 0x7fbe54070140 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.287+0000 7fbe6ac9d640 1 --2- 172.21.15.142:0/1033526186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe64197200 0x7fbe64197660 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.287+0000 7fbe6ac9d640 1 --2- 172.21.15.142:0/1033526186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe640ff1b0 0x7fbe64196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.287+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/1033526186 >> 172.21.15.142:0/1033526186 conn(0x7fbe640faa30 msgr2=0x7fbe640fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:05.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.287+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/1033526186 shutdown_connections 2024-03-29T05:13:05.293 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.287+0000 7fbe6ac9d640 1 -- 172.21.15.142:0/1033526186 wait complete. 2024-03-29T05:13:05.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:04 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3885661088' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-29T05:13:05.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:04 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2033492684' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-29T05:13:05.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:04 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2306138606' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-29T05:13:05.303 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-03-29T05:13:05.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.412+0000 7f32acbce640 1 -- 172.21.15.142:0/640297485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32a8101b90 msgr2=0x7f32a810a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.412+0000 7f32acbce640 1 --2- 172.21.15.142:0/640297485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32a8101b90 0x7f32a810a070 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f329c0098b0 tx=0x7f329c02bd60 comp rx=0 tx=0).stop 2024-03-29T05:13:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.412+0000 7f32acbce640 1 -- 172.21.15.142:0/640297485 shutdown_connections 2024-03-29T05:13:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.412+0000 7f32acbce640 1 --2- 172.21.15.142:0/640297485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32a8101b90 0x7f32a810a070 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.412+0000 7f32acbce640 1 --2- 172.21.15.142:0/640297485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32a8101270 0x7f32a8101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.412+0000 7f32acbce640 1 -- 172.21.15.142:0/640297485 >> 172.21.15.142:0/640297485 conn(0x7f32a80faa30 msgr2=0x7f32a80fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.412+0000 7f32acbce640 1 -- 172.21.15.142:0/640297485 shutdown_connections 2024-03-29T05:13:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.412+0000 7f32acbce640 1 -- 172.21.15.142:0/640297485 wait complete. 2024-03-29T05:13:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.413+0000 7f32acbce640 1 Processor -- start 2024-03-29T05:13:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.413+0000 7f32acbce640 1 -- start start 2024-03-29T05:13:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.413+0000 7f32acbce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32a8101270 0x7f32a8195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:05.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.413+0000 7f32acbce640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32a8101b90 0x7f32a81962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.413+0000 7f32acbce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a8196980 con 0x7f32a8101270 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.413+0000 7f32acbce640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a819a750 con 0x7f32a8101b90 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.413+0000 7f32a659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32a8101270 0x7f32a8195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.413+0000 7f32a659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32a8101270 0x7f32a8195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47150/0 (socket says 172.21.15.142:47150) 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.413+0000 7f32a659c640 1 -- 172.21.15.142:0/2549661149 learned_addr learned my addr 172.21.15.142:0/2549661149 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.413+0000 7f32a659c640 1 -- 172.21.15.142:0/2549661149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32a8101b90 msgr2=0x7f32a81962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.413+0000 7f32a659c640 1 --2- 172.21.15.142:0/2549661149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32a8101b90 0x7f32a81962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.413+0000 7f32a659c640 1 -- 172.21.15.142:0/2549661149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c009560 con 0x7f32a8101270 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.414+0000 7f32a659c640 1 --2- 172.21.15.142:0/2549661149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32a8101270 0x7f32a8195db0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f329800d6f0 tx=0x7f329800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.414+0000 7f32977fe640 1 -- 172.21.15.142:0/2549661149 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3298009a30 con 0x7f32a8101270 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.414+0000 7f32977fe640 1 -- 172.21.15.142:0/2549661149 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32980054e0 con 0x7f32a8101270 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.414+0000 7f32acbce640 1 -- 172.21.15.142:0/2549661149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a819aa30 con 0x7f32a8101270 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.414+0000 7f32977fe640 1 -- 172.21.15.142:0/2549661149 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3298010460 con 0x7f32a8101270 2024-03-29T05:13:05.421 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.414+0000 7f32acbce640 1 -- 172.21.15.142:0/2549661149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a819af90 con 0x7f32a8101270 2024-03-29T05:13:05.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.415+0000 7f32acbce640 1 -- 172.21.15.142:0/2549661149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a804e040 con 0x7f32a8101270 2024-03-29T05:13:05.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.415+0000 7f32977fe640 1 -- 172.21.15.142:0/2549661149 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f3298009b90 con 0x7f32a8101270 2024-03-29T05:13:05.422 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.416+0000 7f32977fe640 1 --2- 172.21.15.142:0/2549661149 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f329006dcd0 0x7f3290070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:05.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.416+0000 7f32a5d9b640 1 --2- 172.21.15.142:0/2549661149 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f329006dcd0 0x7f3290070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:05.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.416+0000 7f32977fe640 1 -- 172.21.15.142:0/2549661149 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f329808d840 con 0x7f32a8101270 2024-03-29T05:13:05.423 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.416+0000 7f32a5d9b640 1 --2- 172.21.15.142:0/2549661149 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f329006dcd0 0x7f3290070190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f329c036040 tx=0x7f329c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:05.426 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.421+0000 7f32977fe640 1 -- 172.21.15.142:0/2549661149 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3298056c60 con 0x7f32a8101270 2024-03-29T05:13:05.763 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.757+0000 7f32acbce640 1 -- 172.21.15.142:0/2549661149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f32a81076e0 con 0x7f32a8101270 2024-03-29T05:13:05.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.758+0000 7f32977fe640 1 -- 172.21.15.142:0/2549661149 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v39) v1 ==== 107+0+5227 (secure 0 0 0) 0x7f329805a320 con 0x7f32a8101270 2024-03-29T05:13:05.764 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:05.764 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi152.uhebsi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6825/778282077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":778282077},{"type":"v1","addr":"172.21.15.152:6825","nonce":778282077}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44327,"name":"cephfs.smithi152.hiilum","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6827/2852002359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2852002359},{"type":"v1","addr":"172.21.15.152:6827","nonce":2852002359}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T05:10:44.730274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":244,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi142.bmkjzf","rank":1,"incarnation":33,"state":"up:rejoin","state_seq":240,"addr":"172.21.15.142:6829/1735570051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1735570051},{"type":"v1","addr":"172.21.15.142:6829","nonce":1735570051}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-29T05:13:05.764 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 37 2024-03-29T05:13:05.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.761+0000 7f32acbce640 1 -- 172.21.15.142:0/2549661149 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f329006dcd0 msgr2=0x7f3290070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:05.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.761+0000 7f32acbce640 1 --2- 172.21.15.142:0/2549661149 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f329006dcd0 0x7f3290070190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f329c036040 tx=0x7f329c000bc0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.761+0000 7f32acbce640 1 -- 172.21.15.142:0/2549661149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32a8101270 msgr2=0x7f32a8195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:05.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.761+0000 7f32acbce640 1 --2- 172.21.15.142:0/2549661149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32a8101270 0x7f32a8195db0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f329800d6f0 tx=0x7f329800dbb0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.761+0000 7f32acbce640 1 -- 172.21.15.142:0/2549661149 shutdown_connections 2024-03-29T05:13:05.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.761+0000 7f32acbce640 1 --2- 172.21.15.142:0/2549661149 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f329006dcd0 0x7f3290070190 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.761+0000 7f32acbce640 1 --2- 172.21.15.142:0/2549661149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32a8101b90 0x7f32a81962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.761+0000 7f32acbce640 1 --2- 172.21.15.142:0/2549661149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32a8101270 0x7f32a8195db0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.761+0000 7f32acbce640 1 -- 172.21.15.142:0/2549661149 >> 172.21.15.142:0/2549661149 conn(0x7f32a80faa30 msgr2=0x7f32a81048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:05.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.761+0000 7f32acbce640 1 -- 172.21.15.142:0/2549661149 shutdown_connections 2024-03-29T05:13:05.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.761+0000 7f32acbce640 1 -- 172.21.15.142:0/2549661149 wait complete. 2024-03-29T05:13:05.778 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-03-29T05:13:05.887 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.880+0000 7f341b9ab640 1 -- 172.21.15.142:0/46170076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34140ff260 msgr2=0x7f34140ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.880+0000 7f341b9ab640 1 --2- 172.21.15.142:0/46170076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34140ff260 0x7f34140ff640 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3410009880 tx=0x7f341002bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.880+0000 7f341b9ab640 1 -- 172.21.15.142:0/46170076 shutdown_connections 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.880+0000 7f341b9ab640 1 --2- 172.21.15.142:0/46170076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34140ffc10 0x7f3414103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.880+0000 7f341b9ab640 1 --2- 172.21.15.142:0/46170076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34140ff260 0x7f34140ff640 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.880+0000 7f341b9ab640 1 -- 172.21.15.142:0/46170076 >> 172.21.15.142:0/46170076 conn(0x7f3414074a50 msgr2=0x7f3414074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.880+0000 7f341b9ab640 1 -- 172.21.15.142:0/46170076 shutdown_connections 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.880+0000 7f341b9ab640 1 -- 172.21.15.142:0/46170076 wait complete. 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.880+0000 7f341b9ab640 1 Processor -- start 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.880+0000 7f341b9ab640 1 -- start start 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.880+0000 7f341b9ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34140ff260 0x7f3414196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f341b9ab640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3414197130 0x7f3414197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f341b9ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341419fc60 con 0x7f34140ff260 2024-03-29T05:13:05.888 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f341b9ab640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341419fda0 con 0x7f3414197130 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f3419747640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34140ff260 0x7f3414196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f3419747640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34140ff260 0x7f3414196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47168/0 (socket says 172.21.15.142:47168) 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f3419747640 1 -- 172.21.15.142:0/822246959 learned_addr learned my addr 172.21.15.142:0/822246959 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f3418f46640 1 --2- 172.21.15.142:0/822246959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3414197130 0x7f3414197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f3419747640 1 -- 172.21.15.142:0/822246959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3414197130 msgr2=0x7f3414197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f3419747640 1 --2- 172.21.15.142:0/822246959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3414197130 0x7f3414197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f3419747640 1 -- 172.21.15.142:0/822246959 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3410009560 con 0x7f34140ff260 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f3419747640 1 --2- 172.21.15.142:0/822246959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34140ff260 0x7f3414196bf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3410009430 tx=0x7f3410004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f340a7fc640 1 -- 172.21.15.142:0/822246959 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34100035f0 con 0x7f34140ff260 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f340a7fc640 1 -- 172.21.15.142:0/822246959 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3410003c10 con 0x7f34140ff260 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f341b9ab640 1 -- 172.21.15.142:0/822246959 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341419ffa0 con 0x7f34140ff260 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f340a7fc640 1 -- 172.21.15.142:0/822246959 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3410005670 con 0x7f34140ff260 2024-03-29T05:13:05.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.881+0000 7f341b9ab640 1 -- 172.21.15.142:0/822246959 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34141a03d0 con 0x7f34140ff260 2024-03-29T05:13:05.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.883+0000 7f341b9ab640 1 -- 172.21.15.142:0/822246959 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341404e040 con 0x7f34140ff260 2024-03-29T05:13:05.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.883+0000 7f340a7fc640 1 -- 172.21.15.142:0/822246959 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f3410004dd0 con 0x7f34140ff260 2024-03-29T05:13:05.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.883+0000 7f340a7fc640 1 --2- 172.21.15.142:0/822246959 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f340006dcd0 0x7f3400070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:05.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.883+0000 7f340a7fc640 1 -- 172.21.15.142:0/822246959 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f34100aff30 con 0x7f34140ff260 2024-03-29T05:13:05.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.883+0000 7f3418f46640 1 --2- 172.21.15.142:0/822246959 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f340006dcd0 0x7f3400070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:05.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.884+0000 7f3418f46640 1 --2- 172.21.15.142:0/822246959 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f340006dcd0 0x7f3400070190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3404005d00 tx=0x7f340400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:05.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:05.888+0000 7f340a7fc640 1 -- 172.21.15.142:0/822246959 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f34100e4280 con 0x7f34140ff260 2024-03-29T05:13:06.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:05 smithi142 ceph-mon[112475]: pgmap v744: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:13:06.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:05 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1033526186' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-29T05:13:06.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:05 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2549661149' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-29T05:13:06.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.229+0000 7f341b9ab640 1 -- 172.21.15.142:0/822246959 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f34140009c0 con 0x7f34140ff260 2024-03-29T05:13:06.235 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.230+0000 7f340a7fc640 1 -- 172.21.15.142:0/822246959 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v39) v1 ==== 107+0+5257 (secure 0 0 0) 0x7f34140009c0 con 0x7f34140ff260 2024-03-29T05:13:06.235 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:06.236 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-29T04:42:06.709178+0000","modified":"2024-03-29T05:10:46.360840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34420,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34420":{"gid":34420,"name":"cephfs.smithi142.wgmndr","rank":0,"incarnation":33,"state":"up:active","state_seq":245,"addr":"172.21.15.142:6827/2841622897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2841622897},{"type":"v1","addr":"172.21.15.142:6827","nonce":2841622897}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi142.bmkjzf","rank":1,"incarnation":33,"state":"up:active","state_seq":241,"addr":"172.21.15.142:6829/1735570051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1735570051},{"type":"v1","addr":"172.21.15.142:6829","nonce":1735570051}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44323":{"gid":44323,"name":"cephfs.smithi152.uhebsi","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6825/778282077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":778282077},{"type":"v1","addr":"172.21.15.152:6825","nonce":778282077}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44327":{"gid":44327,"name":"cephfs.smithi152.hiilum","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6827/2852002359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2852002359},{"type":"v1","addr":"172.21.15.152:6827","nonce":2852002359}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34420,"qdb_cluster":[34420,34424]},"id":1}]} 2024-03-29T05:13:06.236 INFO:teuthology.orchestra.run.smithi142.stderr:dumped fsmap epoch 38 2024-03-29T05:13:06.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.232+0000 7f341b9ab640 1 -- 172.21.15.142:0/822246959 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f340006dcd0 msgr2=0x7f3400070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:06.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.232+0000 7f341b9ab640 1 --2- 172.21.15.142:0/822246959 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f340006dcd0 0x7f3400070190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3404005d00 tx=0x7f340400b410 comp rx=0 tx=0).stop 2024-03-29T05:13:06.238 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.232+0000 7f341b9ab640 1 -- 172.21.15.142:0/822246959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34140ff260 msgr2=0x7f3414196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:06.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.232+0000 7f341b9ab640 1 --2- 172.21.15.142:0/822246959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34140ff260 0x7f3414196bf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3410009430 tx=0x7f3410004040 comp rx=0 tx=0).stop 2024-03-29T05:13:06.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.232+0000 7f341b9ab640 1 -- 172.21.15.142:0/822246959 shutdown_connections 2024-03-29T05:13:06.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.232+0000 7f341b9ab640 1 --2- 172.21.15.142:0/822246959 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f340006dcd0 0x7f3400070190 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.232+0000 7f341b9ab640 1 --2- 172.21.15.142:0/822246959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3414197130 0x7f3414197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.232+0000 7f341b9ab640 1 --2- 172.21.15.142:0/822246959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34140ff260 0x7f3414196bf0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.232+0000 7f341b9ab640 1 -- 172.21.15.142:0/822246959 >> 172.21.15.142:0/822246959 conn(0x7f3414074a50 msgr2=0x7f34140fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:06.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.232+0000 7f341b9ab640 1 -- 172.21.15.142:0/822246959 shutdown_connections 2024-03-29T05:13:06.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.232+0000 7f341b9ab640 1 -- 172.21.15.142:0/822246959 wait complete. 2024-03-29T05:13:06.252 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-29T05:13:06.264 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-29T05:13:06.264 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T05:13:06.264 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T05:13:06.283 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T05:13:06.283 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T05:13:06.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:05 smithi152 ceph-mon[88186]: pgmap v744: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:13:06.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:05 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1033526186' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-29T05:13:06.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:05 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2549661149' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-29T05:13:06.338 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T05:13:06.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.500+0000 7f364d8bd640 1 -- 172.21.15.142:0/3400055507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648067350 msgr2=0x7f36480677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:06.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.500+0000 7f364d8bd640 1 --2- 172.21.15.142:0/3400055507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648067350 0x7f36480677b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f363c009880 tx=0x7f363c02bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.500+0000 7f364d8bd640 1 -- 172.21.15.142:0/3400055507 shutdown_connections 2024-03-29T05:13:06.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.500+0000 7f364d8bd640 1 --2- 172.21.15.142:0/3400055507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648067350 0x7f36480677b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.500+0000 7f364d8bd640 1 --2- 172.21.15.142:0/3400055507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36481061b0 0x7f3648106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.500+0000 7f364d8bd640 1 -- 172.21.15.142:0/3400055507 >> 172.21.15.142:0/3400055507 conn(0x7f3648075b40 msgr2=0x7f3648075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:06.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.501+0000 7f364d8bd640 1 -- 172.21.15.142:0/3400055507 shutdown_connections 2024-03-29T05:13:06.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.501+0000 7f364d8bd640 1 -- 172.21.15.142:0/3400055507 wait complete. 2024-03-29T05:13:06.507 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.501+0000 7f364d8bd640 1 Processor -- start 2024-03-29T05:13:06.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.501+0000 7f364d8bd640 1 -- start start 2024-03-29T05:13:06.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.502+0000 7f364d8bd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648067350 0x7f3648195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:06.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.502+0000 7f364d8bd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36481061b0 0x7f3648196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:06.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.502+0000 7f364d8bd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36481968f0 con 0x7f3648067350 2024-03-29T05:13:06.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.502+0000 7f364d8bd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36481995c0 con 0x7f36481061b0 2024-03-29T05:13:06.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.502+0000 7f3646ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648067350 0x7f3648195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:06.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.502+0000 7f3646ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648067350 0x7f3648195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47184/0 (socket says 172.21.15.142:47184) 2024-03-29T05:13:06.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.502+0000 7f3646ffd640 1 -- 172.21.15.142:0/3346842092 learned_addr learned my addr 172.21.15.142:0/3346842092 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:06.509 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.502+0000 7f36467fc640 1 --2- 172.21.15.142:0/3346842092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36481061b0 0x7f3648196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:06.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.502+0000 7f3646ffd640 1 -- 172.21.15.142:0/3346842092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36481061b0 msgr2=0x7f3648196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:06.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.502+0000 7f3646ffd640 1 --2- 172.21.15.142:0/3346842092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36481061b0 0x7f3648196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.502+0000 7f3646ffd640 1 -- 172.21.15.142:0/3346842092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c009560 con 0x7f3648067350 2024-03-29T05:13:06.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.502+0000 7f3646ffd640 1 --2- 172.21.15.142:0/3346842092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648067350 0x7f3648195cd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f363800d8d0 tx=0x7f363800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:06.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.503+0000 7f364c8bb640 1 -- 172.21.15.142:0/3346842092 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3638009c00 con 0x7f3648067350 2024-03-29T05:13:06.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.503+0000 7f364d8bd640 1 -- 172.21.15.142:0/3346842092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36481998a0 con 0x7f3648067350 2024-03-29T05:13:06.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.503+0000 7f364d8bd640 1 -- 172.21.15.142:0/3346842092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3648199dd0 con 0x7f3648067350 2024-03-29T05:13:06.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.503+0000 7f364c8bb640 1 -- 172.21.15.142:0/3346842092 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3638009d60 con 0x7f3648067350 2024-03-29T05:13:06.510 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.503+0000 7f364c8bb640 1 -- 172.21.15.142:0/3346842092 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36380049d0 con 0x7f3648067350 2024-03-29T05:13:06.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.504+0000 7f364d8bd640 1 -- 172.21.15.142:0/3346842092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364804e040 con 0x7f3648067350 2024-03-29T05:13:06.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.504+0000 7f364c8bb640 1 -- 172.21.15.142:0/3346842092 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f3638004c90 con 0x7f3648067350 2024-03-29T05:13:06.511 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.505+0000 7f364c8bb640 1 --2- 172.21.15.142:0/3346842092 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f363006dd90 0x7f3630070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:06.512 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.505+0000 7f364c8bb640 1 -- 172.21.15.142:0/3346842092 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f363808d990 con 0x7f3648067350 2024-03-29T05:13:06.512 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.505+0000 7f36467fc640 1 --2- 172.21.15.142:0/3346842092 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f363006dd90 0x7f3630070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:06.512 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.505+0000 7f36467fc640 1 --2- 172.21.15.142:0/3346842092 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f363006dd90 0x7f3630070250 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f363c006010 tx=0x7f363c005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:06.516 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.510+0000 7f364c8bb640 1 -- 172.21.15.142:0/3346842092 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3638057d70 con 0x7f3648067350 2024-03-29T05:13:06.718 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.712+0000 7f364d8bd640 1 -- 172.21.15.142:0/3346842092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3648197130 con 0x7f3648067350 2024-03-29T05:13:06.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.714+0000 7f364c8bb640 1 -- 172.21.15.142:0/3346842092 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17309 (secure 0 0 0) 0x7f3648197130 con 0x7f3648067350 2024-03-29T05:13:06.720 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:06.720 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":99,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","created":"2024-03-29T04:35:07.701328+0000","modified":"2024-03-29T04:55:30.115802+0000","last_up_change":"2024-03-29T04:54:31.016765+0000","last_in_change":"2024-03-29T04:40:27.009053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:39:31.891786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-29T04:42:05.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-29T04:42:05.937588+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d908de1a-bec3-4238-959a-f3f550431ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":51,"up_from":55,"up_thru":92,"down_at":52,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1587216149},{"type":"v1","addr":"172.21.15.142:6803","nonce":1587216149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1587216149},{"type":"v1","addr":"172.21.15.142:6805","nonce":1587216149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1587216149},{"type":"v1","addr":"172.21.15.142:6809","nonce":1587216149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1587216149},{"type":"v1","addr":"172.21.15.142:6807","nonce":1587216149}]},"public_addr":"172.21.15.142:6803/1587216149","cluster_addr":"172.21.15.142:6805/1587216149","heartbeat_back_addr":"172.21.15.142:6809/1587216149","heartbeat_front_addr":"172.21.15.142:6807/1587216149","state":["exists","up"]},{"osd":1,"uuid":"717e7135-21ab-4d33-a81e-65da6d5186c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":56,"up_from":67,"up_thru":92,"down_at":57,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":296003948},{"type":"v1","addr":"172.21.15.142:6811","nonce":296003948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":296003948},{"type":"v1","addr":"172.21.15.142:6813","nonce":296003948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":296003948},{"type":"v1","addr":"172.21.15.142:6817","nonce":296003948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":296003948},{"type":"v1","addr":"172.21.15.142:6815","nonce":296003948}]},"public_addr":"172.21.15.142:6811/296003948","cluster_addr":"172.21.15.142:6813/296003948","heartbeat_back_addr":"172.21.15.142:6817/296003948","heartbeat_front_addr":"172.21.15.142:6815/296003948","state":["exists","up"]},{"osd":2,"uuid":"977032d2-3c29-4c68-ae66-73d024ab94ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":73,"up_from":77,"up_thru":92,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":658903196},{"type":"v1","addr":"172.21.15.142:6819","nonce":658903196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":658903196},{"type":"v1","addr":"172.21.15.142:6821","nonce":658903196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":658903196},{"type":"v1","addr":"172.21.15.142:6825","nonce":658903196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":658903196},{"type":"v1","addr":"172.21.15.142:6823","nonce":658903196}]},"public_addr":"172.21.15.142:6819/658903196","cluster_addr":"172.21.15.142:6821/658903196","heartbeat_back_addr":"172.21.15.142:6825/658903196","heartbeat_front_addr":"172.21.15.142:6823/658903196","state":["exists","up"]},{"osd":3,"uuid":"39ae7173-3a83-4ae1-9866-5a0da908171c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":78,"up_from":82,"up_thru":92,"down_at":79,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1538535999},{"type":"v1","addr":"172.21.15.152:6801","nonce":1538535999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1538535999},{"type":"v1","addr":"172.21.15.152:6803","nonce":1538535999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1538535999},{"type":"v1","addr":"172.21.15.152:6807","nonce":1538535999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1538535999},{"type":"v1","addr":"172.21.15.152:6805","nonce":1538535999}]},"public_addr":"172.21.15.152:6801/1538535999","cluster_addr":"172.21.15.152:6803/1538535999","heartbeat_back_addr":"172.21.15.152:6807/1538535999","heartbeat_front_addr":"172.21.15.152:6805/1538535999","state":["exists","up"]},{"osd":4,"uuid":"f7b5fab7-0f30-4756-a481-15ad64b9e691","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":83,"up_from":87,"up_thru":92,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":3103638517},{"type":"v1","addr":"172.21.15.152:6809","nonce":3103638517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3103638517},{"type":"v1","addr":"172.21.15.152:6811","nonce":3103638517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3103638517},{"type":"v1","addr":"172.21.15.152:6815","nonce":3103638517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3103638517},{"type":"v1","addr":"172.21.15.152:6813","nonce":3103638517}]},"public_addr":"172.21.15.152:6809/3103638517","cluster_addr":"172.21.15.152:6811/3103638517","heartbeat_back_addr":"172.21.15.152:6815/3103638517","heartbeat_front_addr":"172.21.15.152:6813/3103638517","state":["exists","up"]},{"osd":5,"uuid":"8ed3caba-b201-4ad7-b055-3dc40610ab1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":88,"up_from":92,"up_thru":92,"down_at":89,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3741970037},{"type":"v1","addr":"172.21.15.152:6817","nonce":3741970037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3741970037},{"type":"v1","addr":"172.21.15.152:6819","nonce":3741970037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3741970037},{"type":"v1","addr":"172.21.15.152:6823","nonce":3741970037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3741970037},{"type":"v1","addr":"172.21.15.152:6821","nonce":3741970037}]},"public_addr":"172.21.15.152:6817/3741970037","cluster_addr":"172.21.15.152:6819/3741970037","heartbeat_back_addr":"172.21.15.152:6823/3741970037","heartbeat_front_addr":"172.21.15.152:6821/3741970037","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:38:33.605811+0000","dead_epoch":51},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:00.911691+0000","dead_epoch":56},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:28.052973+0000","dead_epoch":73},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:51.974988+0000","dead_epoch":78},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:18.104282+0000","dead_epoch":83},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:44.747877+0000","dead_epoch":88}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":0,"to":2}]},{"pgid":"2.4","mappings":[{"from":0,"to":2}]},{"pgid":"2.b","mappings":[{"from":3,"to":1}]},{"pgid":"2.f","mappings":[{"from":0,"to":1}]},{"pgid":"2.14","mappings":[{"from":3,"to":1}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":3,"to":1}]},{"pgid":"3.14","mappings":[{"from":3,"to":1}]},{"pgid":"3.1a","mappings":[{"from":4,"to":0}]},{"pgid":"3.1d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:6824/832699894":"2024-03-30T04:54:33.202001+0000","172.21.15.142:6829/1941073852":"2024-03-30T04:54:33.201978+0000","172.21.15.142:6828/1941073852":"2024-03-30T04:54:33.201978+0000","172.21.15.142:6826/2071436089":"2024-03-30T04:42:25.447288+0000","172.21.15.142:6801/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6800/2395947042":"2024-03-30T04:45:42.287114+0000","172.21.15.142:6801/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/586295950":"2024-03-30T04:37:17.579177+0000","172.21.15.152:0/2558519425":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/2666008640":"2024-03-30T04:35:33.215645+0000","172.21.15.152:6825/832699894":"2024-03-30T04:54:33.202001+0000","172.21.15.142:6827/2071436089":"2024-03-30T04:42:25.447288+0000","172.21.15.152:0/2307372541":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/484532431":"2024-03-30T04:45:42.287114+0000","172.21.15.152:0/3618978330":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/351083517":"2024-03-30T04:49:59.502486+0000","172.21.15.142:0/390665991":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/2545215192":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6800/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/672130862":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/1851570418":"2024-03-30T04:45:42.287114+0000","172.21.15.142:0/3750656759":"2024-03-30T04:45:42.287114+0000","172.21.15.152:0/2082842835":"2024-03-30T04:46:54.194028+0000","172.21.15.142:6801/2395947042":"2024-03-30T04:45:42.287114+0000","172.21.15.152:6828/3130987667":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/2370069599":"2024-03-30T04:45:42.287114+0000","172.21.15.142:6800/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/1199684691":"2024-03-30T04:49:59.502486+0000","172.21.15.152:0/2115311268":"2024-03-30T04:46:54.194028+0000","172.21.15.152:0/1140094447":"2024-03-30T04:46:54.194028+0000","172.21.15.142:6800/1771239042":"2024-03-30T04:49:59.502486+0000","172.21.15.152:6829/3130987667":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/3751399855":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/1935271872":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/3338740178":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/966342870":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6801/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:6800/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/2191110203":"2024-03-30T04:49:59.502486+0000","172.21.15.142:6801/1771239042":"2024-03-30T04:49:59.502486+0000","172.21.15.142:0/143963075":"2024-03-30T04:49:59.502486+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T05:13:06.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.716+0000 7f364d8bd640 1 -- 172.21.15.142:0/3346842092 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f363006dd90 msgr2=0x7f3630070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:06.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.716+0000 7f364d8bd640 1 --2- 172.21.15.142:0/3346842092 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f363006dd90 0x7f3630070250 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f363c006010 tx=0x7f363c005dd0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.716+0000 7f364d8bd640 1 -- 172.21.15.142:0/3346842092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648067350 msgr2=0x7f3648195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:06.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.716+0000 7f364d8bd640 1 --2- 172.21.15.142:0/3346842092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648067350 0x7f3648195cd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f363800d8d0 tx=0x7f363800dd90 comp rx=0 tx=0).stop 2024-03-29T05:13:06.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.716+0000 7f364d8bd640 1 -- 172.21.15.142:0/3346842092 shutdown_connections 2024-03-29T05:13:06.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.716+0000 7f364d8bd640 1 --2- 172.21.15.142:0/3346842092 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f363006dd90 0x7f3630070250 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.716+0000 7f364d8bd640 1 --2- 172.21.15.142:0/3346842092 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36481061b0 0x7f3648196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.716+0000 7f364d8bd640 1 --2- 172.21.15.142:0/3346842092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3648067350 0x7f3648195cd0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.716+0000 7f364d8bd640 1 -- 172.21.15.142:0/3346842092 >> 172.21.15.142:0/3346842092 conn(0x7f3648075b40 msgr2=0x7f36480fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:06.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.716+0000 7f364d8bd640 1 -- 172.21.15.142:0/3346842092 shutdown_connections 2024-03-29T05:13:06.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.716+0000 7f364d8bd640 1 -- 172.21.15.142:0/3346842092 wait complete. 2024-03-29T05:13:06.734 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:39:31.891786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-29T04:42:05.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-29T04:42:05.937588+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T05:13:06.734 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T05:13:06.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.840+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/698537419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaf01002b0 msgr2=0x7fcaf0100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:06.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.840+0000 7fcaf5bfa640 1 --2- 172.21.15.142:0/698537419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaf01002b0 0x7fcaf0100690 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009880 tx=0x7fcae002bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.840+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/698537419 shutdown_connections 2024-03-29T05:13:06.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.840+0000 7fcaf5bfa640 1 --2- 172.21.15.142:0/698537419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf0100c60 0x7fcaf0104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.840+0000 7fcaf5bfa640 1 --2- 172.21.15.142:0/698537419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaf01002b0 0x7fcaf0100690 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.840+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/698537419 >> 172.21.15.142:0/698537419 conn(0x7fcaf00fbb30 msgr2=0x7fcaf00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:06.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.840+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/698537419 shutdown_connections 2024-03-29T05:13:06.848 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.840+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/698537419 wait complete. 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.841+0000 7fcaf5bfa640 1 Processor -- start 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.841+0000 7fcaf5bfa640 1 -- start start 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.841+0000 7fcaf5bfa640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf01002b0 0x7fcaf0195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.841+0000 7fcaf5bfa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaf01963a0 0x7fcaf019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.841+0000 7fcaf5bfa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf01969a0 con 0x7fcaf01963a0 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.841+0000 7fcaf5bfa640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0196b10 con 0x7fcaf01002b0 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.841+0000 7fcaeeffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaf01963a0 0x7fcaf019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.841+0000 7fcaeeffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaf01963a0 0x7fcaf019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47202/0 (socket says 172.21.15.142:47202) 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.841+0000 7fcaeeffd640 1 -- 172.21.15.142:0/557367943 learned_addr learned my addr 172.21.15.142:0/557367943 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.841+0000 7fcaef7fe640 1 --2- 172.21.15.142:0/557367943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf01002b0 0x7fcaf0195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.842+0000 7fcaeeffd640 1 -- 172.21.15.142:0/557367943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf01002b0 msgr2=0x7fcaf0195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.842+0000 7fcaeeffd640 1 --2- 172.21.15.142:0/557367943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf01002b0 0x7fcaf0195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:06.849 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.842+0000 7fcaeeffd640 1 -- 172.21.15.142:0/557367943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae0009560 con 0x7fcaf01963a0 2024-03-29T05:13:06.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.842+0000 7fcaef7fe640 1 --2- 172.21.15.142:0/557367943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf01002b0 0x7fcaf0195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T05:13:06.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.842+0000 7fcaeeffd640 1 --2- 172.21.15.142:0/557367943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaf01963a0 0x7fcaf019a800 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcae400e9a0 tx=0x7fcae400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:06.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.842+0000 7fcaecff9640 1 -- 172.21.15.142:0/557367943 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcae400cc90 con 0x7fcaf01963a0 2024-03-29T05:13:06.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.842+0000 7fcaecff9640 1 -- 172.21.15.142:0/557367943 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcae400cdf0 con 0x7fcaf01963a0 2024-03-29T05:13:06.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.842+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/557367943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf019ae00 con 0x7fcaf01963a0 2024-03-29T05:13:06.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.842+0000 7fcaecff9640 1 -- 172.21.15.142:0/557367943 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcae4010430 con 0x7fcaf01963a0 2024-03-29T05:13:06.850 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.842+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/557367943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf019b360 con 0x7fcaf01963a0 2024-03-29T05:13:06.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.843+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/557367943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf004e040 con 0x7fcaf01963a0 2024-03-29T05:13:06.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.844+0000 7fcaecff9640 1 -- 172.21.15.142:0/557367943 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fcae4005020 con 0x7fcaf01963a0 2024-03-29T05:13:06.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.845+0000 7fcaecff9640 1 --2- 172.21.15.142:0/557367943 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcad806dcd0 0x7fcad8070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:06.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.845+0000 7fcaecff9640 1 -- 172.21.15.142:0/557367943 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fcae4014070 con 0x7fcaf01963a0 2024-03-29T05:13:06.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.845+0000 7fcaef7fe640 1 --2- 172.21.15.142:0/557367943 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcad806dcd0 0x7fcad8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:06.851 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.845+0000 7fcaef7fe640 1 --2- 172.21.15.142:0/557367943 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcad806dcd0 0x7fcad8070190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcae0006010 tx=0x7fcae0005b50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:06.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:06.854+0000 7fcaecff9640 1 -- 172.21.15.142:0/557367943 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcae4057570 con 0x7fcaf01963a0 2024-03-29T05:13:07.107 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.101+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/557367943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcaf01971b0 con 0x7fcaf01963a0 2024-03-29T05:13:07.107 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.102+0000 7fcaecff9640 1 -- 172.21.15.142:0/557367943 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcae405ac30 con 0x7fcaf01963a0 2024-03-29T05:13:07.108 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 1 2024-03-29T05:13:07.111 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.106+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/557367943 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcad806dcd0 msgr2=0x7fcad8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:07.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.106+0000 7fcaf5bfa640 1 --2- 172.21.15.142:0/557367943 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcad806dcd0 0x7fcad8070190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcae0006010 tx=0x7fcae0005b50 comp rx=0 tx=0).stop 2024-03-29T05:13:07.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.106+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/557367943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaf01963a0 msgr2=0x7fcaf019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:07.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.106+0000 7fcaf5bfa640 1 --2- 172.21.15.142:0/557367943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaf01963a0 0x7fcaf019a800 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcae400e9a0 tx=0x7fcae400ee60 comp rx=0 tx=0).stop 2024-03-29T05:13:07.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.106+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/557367943 shutdown_connections 2024-03-29T05:13:07.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.106+0000 7fcaf5bfa640 1 --2- 172.21.15.142:0/557367943 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fcad806dcd0 0x7fcad8070190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.106+0000 7fcaf5bfa640 1 --2- 172.21.15.142:0/557367943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaf01963a0 0x7fcaf019a800 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.106+0000 7fcaf5bfa640 1 --2- 172.21.15.142:0/557367943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf01002b0 0x7fcaf0195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.106+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/557367943 >> 172.21.15.142:0/557367943 conn(0x7fcaf00fbb30 msgr2=0x7fcaf00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:07.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.106+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/557367943 shutdown_connections 2024-03-29T05:13:07.112 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.106+0000 7fcaf5bfa640 1 -- 172.21.15.142:0/557367943 wait complete. 2024-03-29T05:13:07.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:06 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/822246959' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-29T05:13:07.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:06 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:13:07.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:06 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3346842092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T05:13:07.128 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-29T05:13:07.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.235+0000 7fc459dfa640 1 -- 172.21.15.142:0/2247304388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc454101210 msgr2=0x7fc4541015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:07.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.235+0000 7fc459dfa640 1 --2- 172.21.15.142:0/2247304388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc454101210 0x7fc4541015f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc444009880 tx=0x7fc44402bd70 comp rx=0 tx=0).stop 2024-03-29T05:13:07.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.235+0000 7fc459dfa640 1 -- 172.21.15.142:0/2247304388 shutdown_connections 2024-03-29T05:13:07.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.235+0000 7fc459dfa640 1 --2- 172.21.15.142:0/2247304388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc454101b30 0x7fc45410a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.235+0000 7fc459dfa640 1 --2- 172.21.15.142:0/2247304388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc454101210 0x7fc4541015f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.235+0000 7fc459dfa640 1 -- 172.21.15.142:0/2247304388 >> 172.21.15.142:0/2247304388 conn(0x7fc4540faa30 msgr2=0x7fc4540fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:07.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.235+0000 7fc459dfa640 1 -- 172.21.15.142:0/2247304388 shutdown_connections 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.235+0000 7fc459dfa640 1 -- 172.21.15.142:0/2247304388 wait complete. 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc459dfa640 1 Processor -- start 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc459dfa640 1 -- start start 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc459dfa640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc454101210 0x7fc454195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc459dfa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc454101b30 0x7fc454196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc459dfa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc454196900 con 0x7fc454101b30 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc459dfa640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45419a6d0 con 0x7fc454101210 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc452ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc454101b30 0x7fc454196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc452ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc454101b30 0x7fc454196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47220/0 (socket says 172.21.15.142:47220) 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc452ffd640 1 -- 172.21.15.142:0/1832648427 learned_addr learned my addr 172.21.15.142:0/1832648427 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc4537fe640 1 --2- 172.21.15.142:0/1832648427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc454101210 0x7fc454195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc452ffd640 1 -- 172.21.15.142:0/1832648427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc454101210 msgr2=0x7fc454195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:07.244 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc452ffd640 1 --2- 172.21.15.142:0/1832648427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc454101210 0x7fc454195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.236+0000 7fc452ffd640 1 -- 172.21.15.142:0/1832648427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444009560 con 0x7fc454101b30 2024-03-29T05:13:07.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.237+0000 7fc452ffd640 1 --2- 172.21.15.142:0/1832648427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc454101b30 0x7fc454196270 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc44800d6c0 tx=0x7fc44800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:07.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.237+0000 7fc450ff9640 1 -- 172.21.15.142:0/1832648427 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4480099f0 con 0x7fc454101b30 2024-03-29T05:13:07.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.237+0000 7fc459dfa640 1 -- 172.21.15.142:0/1832648427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45419a9b0 con 0x7fc454101b30 2024-03-29T05:13:07.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.237+0000 7fc450ff9640 1 -- 172.21.15.142:0/1832648427 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc448009b50 con 0x7fc454101b30 2024-03-29T05:13:07.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.237+0000 7fc450ff9640 1 -- 172.21.15.142:0/1832648427 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc44800f6f0 con 0x7fc454101b30 2024-03-29T05:13:07.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.237+0000 7fc459dfa640 1 -- 172.21.15.142:0/1832648427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45419af10 con 0x7fc454101b30 2024-03-29T05:13:07.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.238+0000 7fc459dfa640 1 -- 172.21.15.142:0/1832648427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45404e040 con 0x7fc454101b30 2024-03-29T05:13:07.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.239+0000 7fc450ff9640 1 -- 172.21.15.142:0/1832648427 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fc448009cc0 con 0x7fc454101b30 2024-03-29T05:13:07.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.239+0000 7fc450ff9640 1 --2- 172.21.15.142:0/1832648427 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc43c06dcd0 0x7fc43c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:07.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.239+0000 7fc450ff9640 1 -- 172.21.15.142:0/1832648427 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fc44808d8f0 con 0x7fc454101b30 2024-03-29T05:13:07.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.239+0000 7fc4537fe640 1 --2- 172.21.15.142:0/1832648427 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc43c06dcd0 0x7fc43c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:07.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.240+0000 7fc4537fe640 1 --2- 172.21.15.142:0/1832648427 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc43c06dcd0 0x7fc43c070190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc444006010 tx=0x7fc444005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:07.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.245+0000 7fc450ff9640 1 -- 172.21.15.142:0/1832648427 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc448056dc0 con 0x7fc454101b30 2024-03-29T05:13:07.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:06 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/822246959' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-29T05:13:07.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:06 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:13:07.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:06 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3346842092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T05:13:07.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.442+0000 7fc459dfa640 1 -- 172.21.15.142:0/1832648427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fc454107680 con 0x7fc454101b30 2024-03-29T05:13:07.448 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.443+0000 7fc450ff9640 1 -- 172.21.15.142:0/1832648427 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v99) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc44805a480 con 0x7fc454101b30 2024-03-29T05:13:07.449 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 32 2024-03-29T05:13:07.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.445+0000 7fc459dfa640 1 -- 172.21.15.142:0/1832648427 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc43c06dcd0 msgr2=0x7fc43c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:07.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.445+0000 7fc459dfa640 1 --2- 172.21.15.142:0/1832648427 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc43c06dcd0 0x7fc43c070190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc444006010 tx=0x7fc444005dd0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.445+0000 7fc459dfa640 1 -- 172.21.15.142:0/1832648427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc454101b30 msgr2=0x7fc454196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:07.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.445+0000 7fc459dfa640 1 --2- 172.21.15.142:0/1832648427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc454101b30 0x7fc454196270 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc44800d6c0 tx=0x7fc44800db80 comp rx=0 tx=0).stop 2024-03-29T05:13:07.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.445+0000 7fc459dfa640 1 -- 172.21.15.142:0/1832648427 shutdown_connections 2024-03-29T05:13:07.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.445+0000 7fc459dfa640 1 --2- 172.21.15.142:0/1832648427 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fc43c06dcd0 0x7fc43c070190 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.445+0000 7fc459dfa640 1 --2- 172.21.15.142:0/1832648427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc454101b30 0x7fc454196270 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.445+0000 7fc459dfa640 1 --2- 172.21.15.142:0/1832648427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc454101210 0x7fc454195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.445+0000 7fc459dfa640 1 -- 172.21.15.142:0/1832648427 >> 172.21.15.142:0/1832648427 conn(0x7fc4540faa30 msgr2=0x7fc454104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:07.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.445+0000 7fc459dfa640 1 -- 172.21.15.142:0/1832648427 shutdown_connections 2024-03-29T05:13:07.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.445+0000 7fc459dfa640 1 -- 172.21.15.142:0/1832648427 wait complete. 2024-03-29T05:13:07.463 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-29T05:13:07.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.565+0000 7fae1d927640 1 -- 172.21.15.142:0/2330173488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae180ffc10 msgr2=0x7fae18103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:07.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.565+0000 7fae1d927640 1 --2- 172.21.15.142:0/2330173488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae180ffc10 0x7fae18103b70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fae00009880 tx=0x7fae0002bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.565+0000 7fae1d927640 1 -- 172.21.15.142:0/2330173488 shutdown_connections 2024-03-29T05:13:07.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.565+0000 7fae1d927640 1 --2- 172.21.15.142:0/2330173488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae180ffc10 0x7fae18103b70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.565+0000 7fae1d927640 1 --2- 172.21.15.142:0/2330173488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae180ff260 0x7fae180ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.565+0000 7fae1d927640 1 -- 172.21.15.142:0/2330173488 >> 172.21.15.142:0/2330173488 conn(0x7fae18074a50 msgr2=0x7fae18074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:07.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.565+0000 7fae1d927640 1 -- 172.21.15.142:0/2330173488 shutdown_connections 2024-03-29T05:13:07.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.565+0000 7fae1d927640 1 -- 172.21.15.142:0/2330173488 wait complete. 2024-03-29T05:13:07.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae1d927640 1 Processor -- start 2024-03-29T05:13:07.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae1d927640 1 -- start start 2024-03-29T05:13:07.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae1d927640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae180ff260 0x7fae18195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:07.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae1d927640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae18196350 0x7fae1819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:07.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae1d927640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae18196950 con 0x7fae180ff260 2024-03-29T05:13:07.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae1d927640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae18196ac0 con 0x7fae18196350 2024-03-29T05:13:07.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae16ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae180ff260 0x7fae18195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:07.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae16ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae180ff260 0x7fae18195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47238/0 (socket says 172.21.15.142:47238) 2024-03-29T05:13:07.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae16ffd640 1 -- 172.21.15.142:0/4040119266 learned_addr learned my addr 172.21.15.142:0/4040119266 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:07.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae167fc640 1 --2- 172.21.15.142:0/4040119266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae18196350 0x7fae1819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:07.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae16ffd640 1 -- 172.21.15.142:0/4040119266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae18196350 msgr2=0x7fae1819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:07.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae16ffd640 1 --2- 172.21.15.142:0/4040119266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae18196350 0x7fae1819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.566+0000 7fae16ffd640 1 -- 172.21.15.142:0/4040119266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae00009560 con 0x7fae180ff260 2024-03-29T05:13:07.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.567+0000 7fae16ffd640 1 --2- 172.21.15.142:0/4040119266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae180ff260 0x7fae18195e10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fae1000d900 tx=0x7fae1000ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:07.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.567+0000 7fae1c925640 1 -- 172.21.15.142:0/4040119266 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae10009c00 con 0x7fae180ff260 2024-03-29T05:13:07.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.567+0000 7fae1d927640 1 -- 172.21.15.142:0/4040119266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1819adb0 con 0x7fae180ff260 2024-03-29T05:13:07.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.567+0000 7fae1c925640 1 -- 172.21.15.142:0/4040119266 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae10009d60 con 0x7fae180ff260 2024-03-29T05:13:07.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.567+0000 7fae1c925640 1 -- 172.21.15.142:0/4040119266 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae1000f710 con 0x7fae180ff260 2024-03-29T05:13:07.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.567+0000 7fae1d927640 1 -- 172.21.15.142:0/4040119266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1819b310 con 0x7fae180ff260 2024-03-29T05:13:07.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.568+0000 7fae1d927640 1 -- 172.21.15.142:0/4040119266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1804e040 con 0x7fae180ff260 2024-03-29T05:13:07.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.568+0000 7fae1c925640 1 -- 172.21.15.142:0/4040119266 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fae10010460 con 0x7fae180ff260 2024-03-29T05:13:07.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.569+0000 7fae1c925640 1 --2- 172.21.15.142:0/4040119266 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fae0406dcd0 0x7fae04070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:07.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.569+0000 7fae1c925640 1 -- 172.21.15.142:0/4040119266 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fae1008d9c0 con 0x7fae180ff260 2024-03-29T05:13:07.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.569+0000 7fae167fc640 1 --2- 172.21.15.142:0/4040119266 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fae0406dcd0 0x7fae04070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:07.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.569+0000 7fae167fc640 1 --2- 172.21.15.142:0/4040119266 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fae0406dcd0 0x7fae04070190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fae00000c00 tx=0x7fae00005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:07.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.578+0000 7fae1c925640 1 -- 172.21.15.142:0/4040119266 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fae10056e90 con 0x7fae180ff260 2024-03-29T05:13:07.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.910+0000 7fae1d927640 1 -- 172.21.15.142:0/4040119266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fae1819b590 con 0x7fae180ff260 2024-03-29T05:13:07.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.910+0000 7fae1c925640 1 -- 172.21.15.142:0/4040119266 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v99) v1 ==== 107+0+11 (secure 0 0 0) 0x7fae1005a550 con 0x7fae180ff260 2024-03-29T05:13:07.916 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 32 2024-03-29T05:13:07.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.912+0000 7fae1d927640 1 -- 172.21.15.142:0/4040119266 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fae0406dcd0 msgr2=0x7fae04070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:07.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.912+0000 7fae1d927640 1 --2- 172.21.15.142:0/4040119266 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fae0406dcd0 0x7fae04070190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fae00000c00 tx=0x7fae00005fd0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.912+0000 7fae1d927640 1 -- 172.21.15.142:0/4040119266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae180ff260 msgr2=0x7fae18195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:07.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.912+0000 7fae1d927640 1 --2- 172.21.15.142:0/4040119266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae180ff260 0x7fae18195e10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fae1000d900 tx=0x7fae1000ddc0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.912+0000 7fae1d927640 1 -- 172.21.15.142:0/4040119266 shutdown_connections 2024-03-29T05:13:07.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.912+0000 7fae1d927640 1 --2- 172.21.15.142:0/4040119266 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fae0406dcd0 0x7fae04070190 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.912+0000 7fae1d927640 1 --2- 172.21.15.142:0/4040119266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae18196350 0x7fae1819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.912+0000 7fae1d927640 1 --2- 172.21.15.142:0/4040119266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae180ff260 0x7fae18195e10 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:07.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.912+0000 7fae1d927640 1 -- 172.21.15.142:0/4040119266 >> 172.21.15.142:0/4040119266 conn(0x7fae18074a50 msgr2=0x7fae18103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:07.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.912+0000 7fae1d927640 1 -- 172.21.15.142:0/4040119266 shutdown_connections 2024-03-29T05:13:07.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:07.912+0000 7fae1d927640 1 -- 172.21.15.142:0/4040119266 wait complete. 2024-03-29T05:13:07.933 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T05:13:08.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.031+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/1162097881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f38067350 msgr2=0x7f6f380677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:08.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.031+0000 7f6f3ffb6640 1 --2- 172.21.15.142:0/1162097881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f38067350 0x7f6f380677b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009880 tx=0x7f6f2802bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.031+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/1162097881 shutdown_connections 2024-03-29T05:13:08.039 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.031+0000 7f6f3ffb6640 1 --2- 172.21.15.142:0/1162097881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f38067350 0x7f6f380677b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.031+0000 7f6f3ffb6640 1 --2- 172.21.15.142:0/1162097881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f381061b0 0x7f6f38106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.031+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/1162097881 >> 172.21.15.142:0/1162097881 conn(0x7f6f38075b40 msgr2=0x7f6f38075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.032+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/1162097881 shutdown_connections 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.032+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/1162097881 wait complete. 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.032+0000 7f6f3ffb6640 1 Processor -- start 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.032+0000 7f6f3ffb6640 1 -- start start 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.032+0000 7f6f3ffb6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f38067350 0x7f6f3819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.032+0000 7f6f3ffb6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f381061b0 0x7f6f3819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.032+0000 7f6f3ffb6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38195aa0 con 0x7f6f38067350 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.032+0000 7f6f3ffb6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38195c10 con 0x7f6f381061b0 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.032+0000 7f6f3dd52640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f38067350 0x7f6f3819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f3dd52640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f38067350 0x7f6f3819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47262/0 (socket says 172.21.15.142:47262) 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f3dd52640 1 -- 172.21.15.142:0/2612834489 learned_addr learned my addr 172.21.15.142:0/2612834489 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:08.040 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f3d551640 1 --2- 172.21.15.142:0/2612834489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f381061b0 0x7f6f3819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:08.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f3dd52640 1 -- 172.21.15.142:0/2612834489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f381061b0 msgr2=0x7f6f3819bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:08.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f3dd52640 1 --2- 172.21.15.142:0/2612834489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f381061b0 0x7f6f3819bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f3dd52640 1 -- 172.21.15.142:0/2612834489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f28009560 con 0x7f6f38067350 2024-03-29T05:13:08.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f3d551640 1 --2- 172.21.15.142:0/2612834489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f381061b0 0x7f6f3819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T05:13:08.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f3dd52640 1 --2- 172.21.15.142:0/2612834489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f38067350 0x7f6f3819ba30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400d8d0 tx=0x7f6f3400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:08.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f2effd640 1 -- 172.21.15.142:0/2612834489 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f34009c00 con 0x7f6f38067350 2024-03-29T05:13:08.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f2effd640 1 -- 172.21.15.142:0/2612834489 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f34009d60 con 0x7f6f38067350 2024-03-29T05:13:08.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/2612834489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f38195ef0 con 0x7f6f38067350 2024-03-29T05:13:08.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f2effd640 1 -- 172.21.15.142:0/2612834489 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f340049d0 con 0x7f6f38067350 2024-03-29T05:13:08.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.033+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/2612834489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f38196420 con 0x7f6f38067350 2024-03-29T05:13:08.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.035+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/2612834489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3804e040 con 0x7f6f38067350 2024-03-29T05:13:08.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.035+0000 7f6f2effd640 1 -- 172.21.15.142:0/2612834489 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f6f34004c40 con 0x7f6f38067350 2024-03-29T05:13:08.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.036+0000 7f6f2effd640 1 --2- 172.21.15.142:0/2612834489 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6f2406dd90 0x7f6f24070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:08.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.036+0000 7f6f3d551640 1 --2- 172.21.15.142:0/2612834489 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6f2406dd90 0x7f6f24070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:08.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.036+0000 7f6f2effd640 1 -- 172.21.15.142:0/2612834489 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6f3408e040 con 0x7f6f38067350 2024-03-29T05:13:08.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.037+0000 7f6f3d551640 1 --2- 172.21.15.142:0/2612834489 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6f2406dd90 0x7f6f24070250 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6f38196f10 tx=0x7f6f2802cb40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:08.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.040+0000 7f6f2effd640 1 -- 172.21.15.142:0/2612834489 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6f34057580 con 0x7f6f38067350 2024-03-29T05:13:08.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:07 smithi142 ceph-mon[112475]: pgmap v745: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:13:08.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:07 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/557367943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T05:13:08.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:07 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:13:08.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:07 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:13:08.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:07 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:13:08.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:07 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1832648427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T05:13:08.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.267+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/2612834489 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6f38103390 con 0x7f6f38067350 2024-03-29T05:13:08.273 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.267+0000 7f6f2effd640 1 -- 172.21.15.142:0/2612834489 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v99) v1 ==== 81+0+2462 (secure 0 0 0) 0x7f6f34017080 con 0x7f6f38067350 2024-03-29T05:13:08.274 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:6824/832699894 2024-03-30T04:54:33.202001+0000 2024-03-29T05:13:08.274 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6829/1941073852 2024-03-30T04:54:33.201978+0000 2024-03-29T05:13:08.274 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6828/1941073852 2024-03-30T04:54:33.201978+0000 2024-03-29T05:13:08.274 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6826/2071436089 2024-03-30T04:42:25.447288+0000 2024-03-29T05:13:08.274 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6801/2742619935 2024-03-30T04:36:06.145588+0000 2024-03-29T05:13:08.274 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6800/2395947042 2024-03-30T04:45:42.287114+0000 2024-03-29T05:13:08.274 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6801/2745119007 2024-03-30T04:35:33.215645+0000 2024-03-29T05:13:08.274 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/586295950 2024-03-30T04:37:17.579177+0000 2024-03-29T05:13:08.274 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:0/2558519425 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:08.274 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/2666008640 2024-03-30T04:35:33.215645+0000 2024-03-29T05:13:08.274 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:6825/832699894 2024-03-30T04:54:33.202001+0000 2024-03-29T05:13:08.274 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6827/2071436089 2024-03-30T04:42:25.447288+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:0/2307372541 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/484532431 2024-03-30T04:45:42.287114+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:0/3618978330 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/351083517 2024-03-30T04:49:59.502486+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/390665991 2024-03-30T04:36:06.145588+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/2545215192 2024-03-30T04:36:06.145588+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6800/2742619935 2024-03-30T04:36:06.145588+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/672130862 2024-03-30T04:37:17.579177+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/1851570418 2024-03-30T04:45:42.287114+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/3750656759 2024-03-30T04:45:42.287114+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:0/2082842835 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6801/2395947042 2024-03-30T04:45:42.287114+0000 2024-03-29T05:13:08.275 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:6828/3130987667 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/2370069599 2024-03-30T04:45:42.287114+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6800/2745119007 2024-03-30T04:35:33.215645+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/1199684691 2024-03-30T04:49:59.502486+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:0/2115311268 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:0/1140094447 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6800/1771239042 2024-03-30T04:49:59.502486+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:6829/3130987667 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/3751399855 2024-03-30T04:35:33.215645+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/1935271872 2024-03-30T04:35:33.215645+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/3338740178 2024-03-30T04:37:17.579177+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/966342870 2024-03-30T04:36:06.145588+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6801/779416791 2024-03-30T04:37:17.579177+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6800/779416791 2024-03-30T04:37:17.579177+0000 2024-03-29T05:13:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/2191110203 2024-03-30T04:49:59.502486+0000 2024-03-29T05:13:08.277 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6801/1771239042 2024-03-30T04:49:59.502486+0000 2024-03-29T05:13:08.277 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/143963075 2024-03-30T04:49:59.502486+0000 2024-03-29T05:13:08.278 INFO:teuthology.orchestra.run.smithi142.stderr:listed 41 entries 2024-03-29T05:13:08.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.269+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/2612834489 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6f2406dd90 msgr2=0x7f6f24070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:08.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.269+0000 7f6f3ffb6640 1 --2- 172.21.15.142:0/2612834489 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6f2406dd90 0x7f6f24070250 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6f38196f10 tx=0x7f6f2802cb40 comp rx=0 tx=0).stop 2024-03-29T05:13:08.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.269+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/2612834489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f38067350 msgr2=0x7f6f3819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:08.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.269+0000 7f6f3ffb6640 1 --2- 172.21.15.142:0/2612834489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f38067350 0x7f6f3819ba30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400d8d0 tx=0x7f6f3400dd90 comp rx=0 tx=0).stop 2024-03-29T05:13:08.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.269+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/2612834489 shutdown_connections 2024-03-29T05:13:08.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.269+0000 7f6f3ffb6640 1 --2- 172.21.15.142:0/2612834489 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f6f2406dd90 0x7f6f24070250 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.269+0000 7f6f3ffb6640 1 --2- 172.21.15.142:0/2612834489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f381061b0 0x7f6f3819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.269+0000 7f6f3ffb6640 1 --2- 172.21.15.142:0/2612834489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f38067350 0x7f6f3819ba30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.270+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/2612834489 >> 172.21.15.142:0/2612834489 conn(0x7f6f38075b40 msgr2=0x7f6f380fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:08.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.270+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/2612834489 shutdown_connections 2024-03-29T05:13:08.278 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.270+0000 7f6f3ffb6640 1 -- 172.21.15.142:0/2612834489 wait complete. 2024-03-29T05:13:08.287 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-29T05:13:08.287 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T05:13:08.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:07 smithi152 ceph-mon[88186]: pgmap v745: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:13:08.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:07 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/557367943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T05:13:08.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:07 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:13:08.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:07 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:13:08.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:07 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:13:08.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:07 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1832648427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T05:13:08.304 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T05:13:08.450 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.443+0000 7fe29af8b640 1 -- 172.21.15.142:0/1475926108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe294103f20 msgr2=0x7fe294104300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.443+0000 7fe29af8b640 1 --2- 172.21.15.142:0/1475926108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe294103f20 0x7fe294104300 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe288009880 tx=0x7fe28802bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.443+0000 7fe29af8b640 1 -- 172.21.15.142:0/1475926108 shutdown_connections 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.443+0000 7fe29af8b640 1 --2- 172.21.15.142:0/1475926108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2940fefe0 0x7fe2940ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.443+0000 7fe29af8b640 1 --2- 172.21.15.142:0/1475926108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe294103f20 0x7fe294104300 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.443+0000 7fe29af8b640 1 -- 172.21.15.142:0/1475926108 >> 172.21.15.142:0/1475926108 conn(0x7fe294075b40 msgr2=0x7fe294075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.443+0000 7fe29af8b640 1 -- 172.21.15.142:0/1475926108 shutdown_connections 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.443+0000 7fe29af8b640 1 -- 172.21.15.142:0/1475926108 wait complete. 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.443+0000 7fe29af8b640 1 Processor -- start 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe29af8b640 1 -- start start 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe29af8b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2940fefe0 0x7fe294196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe29af8b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe294103f20 0x7fe294197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe29af8b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe294197690 con 0x7fe2940fefe0 2024-03-29T05:13:08.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe29af8b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2941977d0 con 0x7fe294103f20 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe298d27640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2940fefe0 0x7fe294196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe298d27640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2940fefe0 0x7fe294196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47280/0 (socket says 172.21.15.142:47280) 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe298d27640 1 -- 172.21.15.142:0/2372079187 learned_addr learned my addr 172.21.15.142:0/2372079187 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe293fff640 1 --2- 172.21.15.142:0/2372079187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe294103f20 0x7fe294197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe298d27640 1 -- 172.21.15.142:0/2372079187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe294103f20 msgr2=0x7fe294197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe298d27640 1 --2- 172.21.15.142:0/2372079187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe294103f20 0x7fe294197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe298d27640 1 -- 172.21.15.142:0/2372079187 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe288009560 con 0x7fe2940fefe0 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe298d27640 1 --2- 172.21.15.142:0/2372079187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2940fefe0 0x7fe294196b30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe288009230 tx=0x7fe288004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe291ffb640 1 -- 172.21.15.142:0/2372079187 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2880035f0 con 0x7fe2940fefe0 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe29af8b640 1 -- 172.21.15.142:0/2372079187 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29419fba0 con 0x7fe2940fefe0 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe291ffb640 1 -- 172.21.15.142:0/2372079187 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe288003c10 con 0x7fe2940fefe0 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.444+0000 7fe291ffb640 1 -- 172.21.15.142:0/2372079187 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe288005670 con 0x7fe2940fefe0 2024-03-29T05:13:08.452 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.445+0000 7fe29af8b640 1 -- 172.21.15.142:0/2372079187 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29419ffc0 con 0x7fe2940fefe0 2024-03-29T05:13:08.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.446+0000 7fe29af8b640 1 -- 172.21.15.142:0/2372079187 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29404e040 con 0x7fe2940fefe0 2024-03-29T05:13:08.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.446+0000 7fe291ffb640 1 -- 172.21.15.142:0/2372079187 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fe288004dd0 con 0x7fe2940fefe0 2024-03-29T05:13:08.453 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.446+0000 7fe291ffb640 1 --2- 172.21.15.142:0/2372079187 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe28406dd30 0x7fe2840701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:08.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.447+0000 7fe291ffb640 1 -- 172.21.15.142:0/2372079187 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fe2880b00e0 con 0x7fe2940fefe0 2024-03-29T05:13:08.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.447+0000 7fe293fff640 1 --2- 172.21.15.142:0/2372079187 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe28406dd30 0x7fe2840701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:08.454 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.447+0000 7fe293fff640 1 --2- 172.21.15.142:0/2372079187 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe28406dd30 0x7fe2840701f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe280005fd0 tx=0x7fe280005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:08.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.455+0000 7fe291ffb640 1 -- 172.21.15.142:0/2372079187 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe288079500 con 0x7fe2940fefe0 2024-03-29T05:13:08.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.673+0000 7fe29af8b640 1 -- 172.21.15.142:0/2372079187 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe294067960 con 0x7fe2940fefe0 2024-03-29T05:13:08.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.674+0000 7fe291ffb640 1 -- 172.21.15.142:0/2372079187 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17309 (secure 0 0 0) 0x7fe28809b3b0 con 0x7fe2940fefe0 2024-03-29T05:13:08.680 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-29T05:13:08.680 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":99,"fsid":"9f96302c-ed85-11ee-95d0-87774f69a715","created":"2024-03-29T04:35:07.701328+0000","modified":"2024-03-29T04:55:30.115802+0000","last_up_change":"2024-03-29T04:54:31.016765+0000","last_in_change":"2024-03-29T04:40:27.009053+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:39:31.891786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-29T04:42:05.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-29T04:42:05.937588+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d908de1a-bec3-4238-959a-f3f550431ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":51,"up_from":55,"up_thru":92,"down_at":52,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1587216149},{"type":"v1","addr":"172.21.15.142:6803","nonce":1587216149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1587216149},{"type":"v1","addr":"172.21.15.142:6805","nonce":1587216149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1587216149},{"type":"v1","addr":"172.21.15.142:6809","nonce":1587216149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1587216149},{"type":"v1","addr":"172.21.15.142:6807","nonce":1587216149}]},"public_addr":"172.21.15.142:6803/1587216149","cluster_addr":"172.21.15.142:6805/1587216149","heartbeat_back_addr":"172.21.15.142:6809/1587216149","heartbeat_front_addr":"172.21.15.142:6807/1587216149","state":["exists","up"]},{"osd":1,"uuid":"717e7135-21ab-4d33-a81e-65da6d5186c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":56,"up_from":67,"up_thru":92,"down_at":57,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":296003948},{"type":"v1","addr":"172.21.15.142:6811","nonce":296003948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":296003948},{"type":"v1","addr":"172.21.15.142:6813","nonce":296003948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":296003948},{"type":"v1","addr":"172.21.15.142:6817","nonce":296003948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":296003948},{"type":"v1","addr":"172.21.15.142:6815","nonce":296003948}]},"public_addr":"172.21.15.142:6811/296003948","cluster_addr":"172.21.15.142:6813/296003948","heartbeat_back_addr":"172.21.15.142:6817/296003948","heartbeat_front_addr":"172.21.15.142:6815/296003948","state":["exists","up"]},{"osd":2,"uuid":"977032d2-3c29-4c68-ae66-73d024ab94ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":73,"up_from":77,"up_thru":92,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":658903196},{"type":"v1","addr":"172.21.15.142:6819","nonce":658903196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":658903196},{"type":"v1","addr":"172.21.15.142:6821","nonce":658903196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":658903196},{"type":"v1","addr":"172.21.15.142:6825","nonce":658903196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":658903196},{"type":"v1","addr":"172.21.15.142:6823","nonce":658903196}]},"public_addr":"172.21.15.142:6819/658903196","cluster_addr":"172.21.15.142:6821/658903196","heartbeat_back_addr":"172.21.15.142:6825/658903196","heartbeat_front_addr":"172.21.15.142:6823/658903196","state":["exists","up"]},{"osd":3,"uuid":"39ae7173-3a83-4ae1-9866-5a0da908171c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":78,"up_from":82,"up_thru":92,"down_at":79,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1538535999},{"type":"v1","addr":"172.21.15.152:6801","nonce":1538535999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1538535999},{"type":"v1","addr":"172.21.15.152:6803","nonce":1538535999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1538535999},{"type":"v1","addr":"172.21.15.152:6807","nonce":1538535999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1538535999},{"type":"v1","addr":"172.21.15.152:6805","nonce":1538535999}]},"public_addr":"172.21.15.152:6801/1538535999","cluster_addr":"172.21.15.152:6803/1538535999","heartbeat_back_addr":"172.21.15.152:6807/1538535999","heartbeat_front_addr":"172.21.15.152:6805/1538535999","state":["exists","up"]},{"osd":4,"uuid":"f7b5fab7-0f30-4756-a481-15ad64b9e691","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":83,"up_from":87,"up_thru":92,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":3103638517},{"type":"v1","addr":"172.21.15.152:6809","nonce":3103638517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3103638517},{"type":"v1","addr":"172.21.15.152:6811","nonce":3103638517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3103638517},{"type":"v1","addr":"172.21.15.152:6815","nonce":3103638517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3103638517},{"type":"v1","addr":"172.21.15.152:6813","nonce":3103638517}]},"public_addr":"172.21.15.152:6809/3103638517","cluster_addr":"172.21.15.152:6811/3103638517","heartbeat_back_addr":"172.21.15.152:6815/3103638517","heartbeat_front_addr":"172.21.15.152:6813/3103638517","state":["exists","up"]},{"osd":5,"uuid":"8ed3caba-b201-4ad7-b055-3dc40610ab1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":88,"up_from":92,"up_thru":92,"down_at":89,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3741970037},{"type":"v1","addr":"172.21.15.152:6817","nonce":3741970037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3741970037},{"type":"v1","addr":"172.21.15.152:6819","nonce":3741970037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3741970037},{"type":"v1","addr":"172.21.15.152:6823","nonce":3741970037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3741970037},{"type":"v1","addr":"172.21.15.152:6821","nonce":3741970037}]},"public_addr":"172.21.15.152:6817/3741970037","cluster_addr":"172.21.15.152:6819/3741970037","heartbeat_back_addr":"172.21.15.152:6823/3741970037","heartbeat_front_addr":"172.21.15.152:6821/3741970037","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:38:33.605811+0000","dead_epoch":51},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:00.911691+0000","dead_epoch":56},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:28.052973+0000","dead_epoch":73},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:39:51.974988+0000","dead_epoch":78},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:18.104282+0000","dead_epoch":83},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:40:44.747877+0000","dead_epoch":88}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":0,"to":2}]},{"pgid":"2.4","mappings":[{"from":0,"to":2}]},{"pgid":"2.b","mappings":[{"from":3,"to":1}]},{"pgid":"2.f","mappings":[{"from":0,"to":1}]},{"pgid":"2.14","mappings":[{"from":3,"to":1}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":3,"to":1}]},{"pgid":"3.14","mappings":[{"from":3,"to":1}]},{"pgid":"3.1a","mappings":[{"from":4,"to":0}]},{"pgid":"3.1d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:6824/832699894":"2024-03-30T04:54:33.202001+0000","172.21.15.142:6829/1941073852":"2024-03-30T04:54:33.201978+0000","172.21.15.142:6828/1941073852":"2024-03-30T04:54:33.201978+0000","172.21.15.142:6826/2071436089":"2024-03-30T04:42:25.447288+0000","172.21.15.142:6801/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6800/2395947042":"2024-03-30T04:45:42.287114+0000","172.21.15.142:6801/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/586295950":"2024-03-30T04:37:17.579177+0000","172.21.15.152:0/2558519425":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/2666008640":"2024-03-30T04:35:33.215645+0000","172.21.15.152:6825/832699894":"2024-03-30T04:54:33.202001+0000","172.21.15.142:6827/2071436089":"2024-03-30T04:42:25.447288+0000","172.21.15.152:0/2307372541":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/484532431":"2024-03-30T04:45:42.287114+0000","172.21.15.152:0/3618978330":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/351083517":"2024-03-30T04:49:59.502486+0000","172.21.15.142:0/390665991":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/2545215192":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6800/2742619935":"2024-03-30T04:36:06.145588+0000","172.21.15.142:0/672130862":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/1851570418":"2024-03-30T04:45:42.287114+0000","172.21.15.142:0/3750656759":"2024-03-30T04:45:42.287114+0000","172.21.15.152:0/2082842835":"2024-03-30T04:46:54.194028+0000","172.21.15.142:6801/2395947042":"2024-03-30T04:45:42.287114+0000","172.21.15.152:6828/3130987667":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/2370069599":"2024-03-30T04:45:42.287114+0000","172.21.15.142:6800/2745119007":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/1199684691":"2024-03-30T04:49:59.502486+0000","172.21.15.152:0/2115311268":"2024-03-30T04:46:54.194028+0000","172.21.15.152:0/1140094447":"2024-03-30T04:46:54.194028+0000","172.21.15.142:6800/1771239042":"2024-03-30T04:49:59.502486+0000","172.21.15.152:6829/3130987667":"2024-03-30T04:46:54.194028+0000","172.21.15.142:0/3751399855":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/1935271872":"2024-03-30T04:35:33.215645+0000","172.21.15.142:0/3338740178":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/966342870":"2024-03-30T04:36:06.145588+0000","172.21.15.142:6801/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:6800/779416791":"2024-03-30T04:37:17.579177+0000","172.21.15.142:0/2191110203":"2024-03-30T04:49:59.502486+0000","172.21.15.142:6801/1771239042":"2024-03-30T04:49:59.502486+0000","172.21.15.142:0/143963075":"2024-03-30T04:49:59.502486+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T05:13:08.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.677+0000 7fe29af8b640 1 -- 172.21.15.142:0/2372079187 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe28406dd30 msgr2=0x7fe2840701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:08.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.677+0000 7fe29af8b640 1 --2- 172.21.15.142:0/2372079187 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe28406dd30 0x7fe2840701f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe280005fd0 tx=0x7fe280005850 comp rx=0 tx=0).stop 2024-03-29T05:13:08.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.677+0000 7fe29af8b640 1 -- 172.21.15.142:0/2372079187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2940fefe0 msgr2=0x7fe294196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:08.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.677+0000 7fe29af8b640 1 --2- 172.21.15.142:0/2372079187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2940fefe0 0x7fe294196b30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe288009230 tx=0x7fe288004040 comp rx=0 tx=0).stop 2024-03-29T05:13:08.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.677+0000 7fe29af8b640 1 -- 172.21.15.142:0/2372079187 shutdown_connections 2024-03-29T05:13:08.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.677+0000 7fe29af8b640 1 --2- 172.21.15.142:0/2372079187 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fe28406dd30 0x7fe2840701f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.677+0000 7fe29af8b640 1 --2- 172.21.15.142:0/2372079187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe294103f20 0x7fe294197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.677+0000 7fe29af8b640 1 --2- 172.21.15.142:0/2372079187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2940fefe0 0x7fe294196b30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.677+0000 7fe29af8b640 1 -- 172.21.15.142:0/2372079187 >> 172.21.15.142:0/2372079187 conn(0x7fe294075b40 msgr2=0x7fe2940fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:08.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.677+0000 7fe29af8b640 1 -- 172.21.15.142:0/2372079187 shutdown_connections 2024-03-29T05:13:08.684 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.677+0000 7fe29af8b640 1 -- 172.21.15.142:0/2372079187 wait complete. 2024-03-29T05:13:08.694 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:39:31.891786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-29T04:42:05.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-29T04:42:05.937588+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T05:13:08.695 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T05:13:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.798+0000 7fdd2b41c640 1 -- 172.21.15.142:0/2611622238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd24067f70 msgr2=0x7fdd24104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.798+0000 7fdd2b41c640 1 --2- 172.21.15.142:0/2611622238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd24067f70 0x7fdd24104c70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009880 tx=0x7fdd2002bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.798+0000 7fdd2b41c640 1 -- 172.21.15.142:0/2611622238 shutdown_connections 2024-03-29T05:13:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.798+0000 7fdd2b41c640 1 --2- 172.21.15.142:0/2611622238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd24067f70 0x7fdd24104c70 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.806 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.798+0000 7fdd2b41c640 1 --2- 172.21.15.142:0/2611622238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd240675c0 0x7fdd240679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.798+0000 7fdd2b41c640 1 -- 172.21.15.142:0/2611622238 >> 172.21.15.142:0/2611622238 conn(0x7fdd24074a50 msgr2=0x7fdd24074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.799+0000 7fdd2b41c640 1 -- 172.21.15.142:0/2611622238 shutdown_connections 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.799+0000 7fdd2b41c640 1 -- 172.21.15.142:0/2611622238 wait complete. 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.799+0000 7fdd2b41c640 1 Processor -- start 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.799+0000 7fdd2b41c640 1 -- start start 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.799+0000 7fdd2b41c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd240675c0 0x7fdd24195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.799+0000 7fdd2b41c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd241962c0 0x7fdd2419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.799+0000 7fdd2b41c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd241968c0 con 0x7fdd240675c0 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.799+0000 7fdd2b41c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd24196a30 con 0x7fdd241962c0 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.799+0000 7fdd291b8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd240675c0 0x7fdd24195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.799+0000 7fdd291b8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd240675c0 0x7fdd24195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37790/0 (socket says 172.21.15.142:37790) 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.799+0000 7fdd291b8640 1 -- 172.21.15.142:0/609982439 learned_addr learned my addr 172.21.15.142:0/609982439 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.800+0000 7fdd289b7640 1 --2- 172.21.15.142:0/609982439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd241962c0 0x7fdd2419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:08.807 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.800+0000 7fdd291b8640 1 -- 172.21.15.142:0/609982439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd241962c0 msgr2=0x7fdd2419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:08.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.800+0000 7fdd291b8640 1 --2- 172.21.15.142:0/609982439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd241962c0 0x7fdd2419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:08.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.800+0000 7fdd291b8640 1 -- 172.21.15.142:0/609982439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd20009560 con 0x7fdd240675c0 2024-03-29T05:13:08.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.800+0000 7fdd291b8640 1 --2- 172.21.15.142:0/609982439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd240675c0 0x7fdd24195d80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400d720 tx=0x7fdd1400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:08.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.800+0000 7fdd1a7fc640 1 -- 172.21.15.142:0/609982439 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd140051f0 con 0x7fdd240675c0 2024-03-29T05:13:08.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.800+0000 7fdd2b41c640 1 -- 172.21.15.142:0/609982439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2419ad20 con 0x7fdd240675c0 2024-03-29T05:13:08.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.800+0000 7fdd2b41c640 1 -- 172.21.15.142:0/609982439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2419b280 con 0x7fdd240675c0 2024-03-29T05:13:08.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.800+0000 7fdd1a7fc640 1 -- 172.21.15.142:0/609982439 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd14009ce0 con 0x7fdd240675c0 2024-03-29T05:13:08.808 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.800+0000 7fdd1a7fc640 1 -- 172.21.15.142:0/609982439 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd14004e20 con 0x7fdd240675c0 2024-03-29T05:13:08.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.801+0000 7fdd2b41c640 1 -- 172.21.15.142:0/609982439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd08005280 con 0x7fdd240675c0 2024-03-29T05:13:08.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.801+0000 7fdd1a7fc640 1 -- 172.21.15.142:0/609982439 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fdd140046c0 con 0x7fdd240675c0 2024-03-29T05:13:08.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.802+0000 7fdd1a7fc640 1 --2- 172.21.15.142:0/609982439 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdd1006dcd0 0x7fdd10070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:08.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.802+0000 7fdd1a7fc640 1 -- 172.21.15.142:0/609982439 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fdd1408d680 con 0x7fdd240675c0 2024-03-29T05:13:08.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.802+0000 7fdd289b7640 1 --2- 172.21.15.142:0/609982439 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdd1006dcd0 0x7fdd10070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:08.809 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.803+0000 7fdd289b7640 1 --2- 172.21.15.142:0/609982439 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdd1006dcd0 0x7fdd10070190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdd20006010 tx=0x7fdd20000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:08.812 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:08.807+0000 7fdd1a7fc640 1 -- 172.21.15.142:0/609982439 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdd14056aa0 con 0x7fdd240675c0 2024-03-29T05:13:09.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.003+0000 7fdd2b41c640 1 -- 172.21.15.142:0/609982439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd08005ab0 con 0x7fdd240675c0 2024-03-29T05:13:09.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.003+0000 7fdd1a7fc640 1 -- 172.21.15.142:0/609982439 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd1405a160 con 0x7fdd240675c0 2024-03-29T05:13:09.009 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 1 2024-03-29T05:13:09.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.005+0000 7fdd2b41c640 1 -- 172.21.15.142:0/609982439 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdd1006dcd0 msgr2=0x7fdd10070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:09.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.005+0000 7fdd2b41c640 1 --2- 172.21.15.142:0/609982439 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdd1006dcd0 0x7fdd10070190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdd20006010 tx=0x7fdd20000bc0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.005+0000 7fdd2b41c640 1 -- 172.21.15.142:0/609982439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd240675c0 msgr2=0x7fdd24195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:09.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.005+0000 7fdd2b41c640 1 --2- 172.21.15.142:0/609982439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd240675c0 0x7fdd24195d80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400d720 tx=0x7fdd1400dbe0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.005+0000 7fdd2b41c640 1 -- 172.21.15.142:0/609982439 shutdown_connections 2024-03-29T05:13:09.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.005+0000 7fdd2b41c640 1 --2- 172.21.15.142:0/609982439 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fdd1006dcd0 0x7fdd10070190 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.005+0000 7fdd2b41c640 1 --2- 172.21.15.142:0/609982439 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd241962c0 0x7fdd2419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.005+0000 7fdd2b41c640 1 --2- 172.21.15.142:0/609982439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd240675c0 0x7fdd24195d80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.005+0000 7fdd2b41c640 1 -- 172.21.15.142:0/609982439 >> 172.21.15.142:0/609982439 conn(0x7fdd24074a50 msgr2=0x7fdd240fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:09.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.005+0000 7fdd2b41c640 1 -- 172.21.15.142:0/609982439 shutdown_connections 2024-03-29T05:13:09.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.005+0000 7fdd2b41c640 1 -- 172.21.15.142:0/609982439 wait complete. 2024-03-29T05:13:09.020 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:08 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/4040119266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T05:13:09.020 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:08 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2612834489' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-29T05:13:09.021 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:08 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2372079187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T05:13:09.021 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-29T05:13:09.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.120+0000 7fb798a8f640 1 -- 172.21.15.142:0/445377091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7940ffb60 msgr2=0x7fb794103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:09.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.120+0000 7fb798a8f640 1 --2- 172.21.15.142:0/445377091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7940ffb60 0x7fb794103ba0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb788009880 tx=0x7fb78802bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.120+0000 7fb798a8f640 1 -- 172.21.15.142:0/445377091 shutdown_connections 2024-03-29T05:13:09.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.120+0000 7fb798a8f640 1 --2- 172.21.15.142:0/445377091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7940ffb60 0x7fb794103ba0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.120+0000 7fb798a8f640 1 --2- 172.21.15.142:0/445377091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7940ff1b0 0x7fb7940ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.120+0000 7fb798a8f640 1 -- 172.21.15.142:0/445377091 >> 172.21.15.142:0/445377091 conn(0x7fb7940faa30 msgr2=0x7fb7940fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:09.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.120+0000 7fb798a8f640 1 -- 172.21.15.142:0/445377091 shutdown_connections 2024-03-29T05:13:09.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.120+0000 7fb798a8f640 1 -- 172.21.15.142:0/445377091 wait complete. 2024-03-29T05:13:09.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.121+0000 7fb798a8f640 1 Processor -- start 2024-03-29T05:13:09.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.121+0000 7fb798a8f640 1 -- start start 2024-03-29T05:13:09.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb798a8f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7940ff1b0 0x7fb794196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:09.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb798a8f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7941970d0 0x7fb794197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:09.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb798a8f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79419fc90 con 0x7fb7940ff1b0 2024-03-29T05:13:09.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb798a8f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79419fdd0 con 0x7fb7941970d0 2024-03-29T05:13:09.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb79259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7940ff1b0 0x7fb794196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:09.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb79259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7940ff1b0 0x7fb794196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37802/0 (socket says 172.21.15.142:37802) 2024-03-29T05:13:09.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb79259c640 1 -- 172.21.15.142:0/3934471346 learned_addr learned my addr 172.21.15.142:0/3934471346 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:09.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb791d9b640 1 --2- 172.21.15.142:0/3934471346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7941970d0 0x7fb794197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:09.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb79259c640 1 -- 172.21.15.142:0/3934471346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7941970d0 msgr2=0x7fb794197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:09.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb79259c640 1 --2- 172.21.15.142:0/3934471346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7941970d0 0x7fb794197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb79259c640 1 -- 172.21.15.142:0/3934471346 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb788009560 con 0x7fb7940ff1b0 2024-03-29T05:13:09.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb79259c640 1 --2- 172.21.15.142:0/3934471346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7940ff1b0 0x7fb794196b90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00eb70 tx=0x7fb78c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:09.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb7837fe640 1 -- 172.21.15.142:0/3934471346 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb78c00ced0 con 0x7fb7940ff1b0 2024-03-29T05:13:09.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb7837fe640 1 -- 172.21.15.142:0/3934471346 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb78c0108f0 con 0x7fb7940ff1b0 2024-03-29T05:13:09.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb798a8f640 1 -- 172.21.15.142:0/3934471346 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79419ffd0 con 0x7fb7940ff1b0 2024-03-29T05:13:09.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.122+0000 7fb7837fe640 1 -- 172.21.15.142:0/3934471346 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb78c018aa0 con 0x7fb7940ff1b0 2024-03-29T05:13:09.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.123+0000 7fb798a8f640 1 -- 172.21.15.142:0/3934471346 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7941a0450 con 0x7fb7940ff1b0 2024-03-29T05:13:09.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.123+0000 7fb798a8f640 1 -- 172.21.15.142:0/3934471346 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79404e040 con 0x7fb7940ff1b0 2024-03-29T05:13:09.131 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.124+0000 7fb7837fe640 1 -- 172.21.15.142:0/3934471346 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fb78c010430 con 0x7fb7940ff1b0 2024-03-29T05:13:09.131 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.124+0000 7fb7837fe640 1 --2- 172.21.15.142:0/3934471346 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb77c06dcd0 0x7fb77c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:09.131 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.125+0000 7fb7837fe640 1 -- 172.21.15.142:0/3934471346 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb78c014070 con 0x7fb7940ff1b0 2024-03-29T05:13:09.131 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.125+0000 7fb791d9b640 1 --2- 172.21.15.142:0/3934471346 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb77c06dcd0 0x7fb77c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:09.131 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.125+0000 7fb791d9b640 1 --2- 172.21.15.142:0/3934471346 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb77c06dcd0 0x7fb77c070190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb788000c00 tx=0x7fb788005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:09.139 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.133+0000 7fb7837fe640 1 -- 172.21.15.142:0/3934471346 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb78c053280 con 0x7fb7940ff1b0 2024-03-29T05:13:09.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:08 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/4040119266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T05:13:09.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:08 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2612834489' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-29T05:13:09.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:08 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2372079187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T05:13:09.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.352+0000 7fb798a8f640 1 -- 172.21.15.142:0/3934471346 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fb7941074b0 con 0x7fb7940ff1b0 2024-03-29T05:13:09.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.353+0000 7fb7837fe640 1 -- 172.21.15.142:0/3934471346 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v99) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb78c0577c0 con 0x7fb7940ff1b0 2024-03-29T05:13:09.359 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 32 2024-03-29T05:13:09.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.354+0000 7fb798a8f640 1 -- 172.21.15.142:0/3934471346 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb77c06dcd0 msgr2=0x7fb77c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:09.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.354+0000 7fb798a8f640 1 --2- 172.21.15.142:0/3934471346 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb77c06dcd0 0x7fb77c070190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb788000c00 tx=0x7fb788005fd0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.354+0000 7fb798a8f640 1 -- 172.21.15.142:0/3934471346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7940ff1b0 msgr2=0x7fb794196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:09.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.354+0000 7fb798a8f640 1 --2- 172.21.15.142:0/3934471346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7940ff1b0 0x7fb794196b90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00eb70 tx=0x7fb78c00c6a0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.355+0000 7fb798a8f640 1 -- 172.21.15.142:0/3934471346 shutdown_connections 2024-03-29T05:13:09.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.355+0000 7fb798a8f640 1 --2- 172.21.15.142:0/3934471346 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb77c06dcd0 0x7fb77c070190 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.355+0000 7fb798a8f640 1 --2- 172.21.15.142:0/3934471346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7941970d0 0x7fb794197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.355+0000 7fb798a8f640 1 --2- 172.21.15.142:0/3934471346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7940ff1b0 0x7fb794196b90 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.355+0000 7fb798a8f640 1 -- 172.21.15.142:0/3934471346 >> 172.21.15.142:0/3934471346 conn(0x7fb7940faa30 msgr2=0x7fb7940fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:09.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.355+0000 7fb798a8f640 1 -- 172.21.15.142:0/3934471346 shutdown_connections 2024-03-29T05:13:09.361 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.355+0000 7fb798a8f640 1 -- 172.21.15.142:0/3934471346 wait complete. 2024-03-29T05:13:09.371 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-29T05:13:09.480 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.474+0000 7fb9be2a4640 1 -- 172.21.15.142:0/1860505594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b8100c90 msgr2=0x7fb9b8104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:09.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.474+0000 7fb9be2a4640 1 --2- 172.21.15.142:0/1860505594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b8100c90 0x7fb9b8104cd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009880 tx=0x7fb9ac02bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.474+0000 7fb9be2a4640 1 -- 172.21.15.142:0/1860505594 shutdown_connections 2024-03-29T05:13:09.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.474+0000 7fb9be2a4640 1 --2- 172.21.15.142:0/1860505594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b8100c90 0x7fb9b8104cd0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.474+0000 7fb9be2a4640 1 --2- 172.21.15.142:0/1860505594 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b81002e0 0x7fb9b81006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.474+0000 7fb9be2a4640 1 -- 172.21.15.142:0/1860505594 >> 172.21.15.142:0/1860505594 conn(0x7fb9b80fbb80 msgr2=0x7fb9b80fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:09.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.474+0000 7fb9be2a4640 1 -- 172.21.15.142:0/1860505594 shutdown_connections 2024-03-29T05:13:09.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.474+0000 7fb9be2a4640 1 -- 172.21.15.142:0/1860505594 wait complete. 2024-03-29T05:13:09.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.475+0000 7fb9be2a4640 1 Processor -- start 2024-03-29T05:13:09.481 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.475+0000 7fb9be2a4640 1 -- start start 2024-03-29T05:13:09.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.475+0000 7fb9be2a4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b81002e0 0x7fb9b8195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:09.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.475+0000 7fb9be2a4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b8196300 0x7fb9b819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:09.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.475+0000 7fb9be2a4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b8196900 con 0x7fb9b8196300 2024-03-29T05:13:09.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.475+0000 7fb9be2a4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b8196a70 con 0x7fb9b81002e0 2024-03-29T05:13:09.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.475+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b8196300 0x7fb9b819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:09.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.475+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b8196300 0x7fb9b819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37822/0 (socket says 172.21.15.142:37822) 2024-03-29T05:13:09.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.475+0000 7fb9b77fe640 1 -- 172.21.15.142:0/2719572755 learned_addr learned my addr 172.21.15.142:0/2719572755 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:09.483 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.476+0000 7fb9b7fff640 1 --2- 172.21.15.142:0/2719572755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b81002e0 0x7fb9b8195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:09.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.476+0000 7fb9b77fe640 1 -- 172.21.15.142:0/2719572755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b81002e0 msgr2=0x7fb9b8195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:09.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.476+0000 7fb9b77fe640 1 --2- 172.21.15.142:0/2719572755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b81002e0 0x7fb9b8195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.476+0000 7fb9b77fe640 1 -- 172.21.15.142:0/2719572755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac009560 con 0x7fb9b8196300 2024-03-29T05:13:09.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.476+0000 7fb9b77fe640 1 --2- 172.21.15.142:0/2719572755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b8196300 0x7fb9b819a760 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac036040 tx=0x7fb9ac02c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:09.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.476+0000 7fb9b57fa640 1 -- 172.21.15.142:0/2719572755 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9ac0035f0 con 0x7fb9b8196300 2024-03-29T05:13:09.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.476+0000 7fb9be2a4640 1 -- 172.21.15.142:0/2719572755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b819ad00 con 0x7fb9b8196300 2024-03-29T05:13:09.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.476+0000 7fb9b57fa640 1 -- 172.21.15.142:0/2719572755 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9ac003c10 con 0x7fb9b8196300 2024-03-29T05:13:09.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.476+0000 7fb9b57fa640 1 -- 172.21.15.142:0/2719572755 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9ac005630 con 0x7fb9b8196300 2024-03-29T05:13:09.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.476+0000 7fb9be2a4640 1 -- 172.21.15.142:0/2719572755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b819b260 con 0x7fb9b8196300 2024-03-29T05:13:09.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.477+0000 7fb9be2a4640 1 -- 172.21.15.142:0/2719572755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b804e040 con 0x7fb9b8196300 2024-03-29T05:13:09.484 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.478+0000 7fb9b57fa640 1 -- 172.21.15.142:0/2719572755 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7fb9ac003750 con 0x7fb9b8196300 2024-03-29T05:13:09.485 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.478+0000 7fb9b57fa640 1 --2- 172.21.15.142:0/2719572755 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb9a006dcd0 0x7fb9a0070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:09.485 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.478+0000 7fb9b7fff640 1 --2- 172.21.15.142:0/2719572755 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb9a006dcd0 0x7fb9a0070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:09.485 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.478+0000 7fb9b57fa640 1 -- 172.21.15.142:0/2719572755 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb9ac0b0060 con 0x7fb9b8196300 2024-03-29T05:13:09.485 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.479+0000 7fb9b7fff640 1 --2- 172.21.15.142:0/2719572755 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb9a006dcd0 0x7fb9a0070190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009aa0 tx=0x7fb9a8005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:09.488 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.483+0000 7fb9b57fa640 1 -- 172.21.15.142:0/2719572755 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb9ac079480 con 0x7fb9b8196300 2024-03-29T05:13:09.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.700+0000 7fb9be2a4640 1 -- 172.21.15.142:0/2719572755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fb9b8197110 con 0x7fb9b8196300 2024-03-29T05:13:09.706 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.700+0000 7fb9b57fa640 1 -- 172.21.15.142:0/2719572755 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v99) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb9ac07cb40 con 0x7fb9b8196300 2024-03-29T05:13:09.707 INFO:teuthology.orchestra.run.smithi142.stdout:pg_num: 32 2024-03-29T05:13:09.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.703+0000 7fb9be2a4640 1 -- 172.21.15.142:0/2719572755 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb9a006dcd0 msgr2=0x7fb9a0070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:09.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.703+0000 7fb9be2a4640 1 --2- 172.21.15.142:0/2719572755 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb9a006dcd0 0x7fb9a0070190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009aa0 tx=0x7fb9a8005f90 comp rx=0 tx=0).stop 2024-03-29T05:13:09.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.703+0000 7fb9be2a4640 1 -- 172.21.15.142:0/2719572755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b8196300 msgr2=0x7fb9b819a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:09.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.703+0000 7fb9be2a4640 1 --2- 172.21.15.142:0/2719572755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b8196300 0x7fb9b819a760 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac036040 tx=0x7fb9ac02c750 comp rx=0 tx=0).stop 2024-03-29T05:13:09.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.703+0000 7fb9be2a4640 1 -- 172.21.15.142:0/2719572755 shutdown_connections 2024-03-29T05:13:09.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.703+0000 7fb9be2a4640 1 --2- 172.21.15.142:0/2719572755 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7fb9a006dcd0 0x7fb9a0070190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.703+0000 7fb9be2a4640 1 --2- 172.21.15.142:0/2719572755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b8196300 0x7fb9b819a760 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.703+0000 7fb9be2a4640 1 --2- 172.21.15.142:0/2719572755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b81002e0 0x7fb9b8195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.703+0000 7fb9be2a4640 1 -- 172.21.15.142:0/2719572755 >> 172.21.15.142:0/2719572755 conn(0x7fb9b80fbb80 msgr2=0x7fb9b80fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:09.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.703+0000 7fb9be2a4640 1 -- 172.21.15.142:0/2719572755 shutdown_connections 2024-03-29T05:13:09.710 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.703+0000 7fb9be2a4640 1 -- 172.21.15.142:0/2719572755 wait complete. 2024-03-29T05:13:09.720 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T05:13:09.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.825+0000 7f34e2421640 1 -- 172.21.15.142:0/2319970105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc0ff1a0 msgr2=0x7f34dc0ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:09.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.825+0000 7f34e2421640 1 --2- 172.21.15.142:0/2319970105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc0ff1a0 0x7f34dc0ff580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009880 tx=0x7f34cc02bce0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.825+0000 7f34e2421640 1 -- 172.21.15.142:0/2319970105 shutdown_connections 2024-03-29T05:13:09.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.825+0000 7f34e2421640 1 --2- 172.21.15.142:0/2319970105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc0ffb50 0x7f34dc103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.825+0000 7f34e2421640 1 --2- 172.21.15.142:0/2319970105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc0ff1a0 0x7f34dc0ff580 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.825+0000 7f34e2421640 1 -- 172.21.15.142:0/2319970105 >> 172.21.15.142:0/2319970105 conn(0x7f34dc0faa60 msgr2=0x7f34dc0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:09.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.825+0000 7f34e2421640 1 -- 172.21.15.142:0/2319970105 shutdown_connections 2024-03-29T05:13:09.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.825+0000 7f34e2421640 1 -- 172.21.15.142:0/2319970105 wait complete. 2024-03-29T05:13:09.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.826+0000 7f34e2421640 1 Processor -- start 2024-03-29T05:13:09.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.826+0000 7f34e2421640 1 -- start start 2024-03-29T05:13:09.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.826+0000 7f34e2421640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc0ff1a0 0x7f34dc19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:09.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.826+0000 7f34e2421640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc19c010 0x7f34dc195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:09.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.826+0000 7f34e2421640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc19c5e0 con 0x7f34dc0ff1a0 2024-03-29T05:13:09.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.826+0000 7f34e2421640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc196080 con 0x7f34dc19c010 2024-03-29T05:13:09.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.826+0000 7f34dbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc0ff1a0 0x7f34dc19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:09.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.826+0000 7f34dbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc0ff1a0 0x7f34dc19bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:37854/0 (socket says 172.21.15.142:37854) 2024-03-29T05:13:09.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.826+0000 7f34dbfff640 1 -- 172.21.15.142:0/1393606890 learned_addr learned my addr 172.21.15.142:0/1393606890 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-29T05:13:09.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.826+0000 7f34db7fe640 1 --2- 172.21.15.142:0/1393606890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc19c010 0x7f34dc195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:09.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.827+0000 7f34dbfff640 1 -- 172.21.15.142:0/1393606890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc19c010 msgr2=0x7f34dc195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:09.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.827+0000 7f34dbfff640 1 --2- 172.21.15.142:0/1393606890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc19c010 0x7f34dc195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:09.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.827+0000 7f34dbfff640 1 -- 172.21.15.142:0/1393606890 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc009560 con 0x7f34dc0ff1a0 2024-03-29T05:13:09.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.827+0000 7f34dbfff640 1 --2- 172.21.15.142:0/1393606890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc0ff1a0 0x7f34dc19bad0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009230 tx=0x7f34cc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:09.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.827+0000 7f34d97fa640 1 -- 172.21.15.142:0/1393606890 <== mon.0 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34cc0035f0 con 0x7f34dc0ff1a0 2024-03-29T05:13:09.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.827+0000 7f34e2421640 1 -- 172.21.15.142:0/1393606890 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc196300 con 0x7f34dc0ff1a0 2024-03-29T05:13:09.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.827+0000 7f34d97fa640 1 -- 172.21.15.142:0/1393606890 <== mon.0 v2:172.21.15.142:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34cc003c10 con 0x7f34dc0ff1a0 2024-03-29T05:13:09.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.827+0000 7f34d97fa640 1 -- 172.21.15.142:0/1393606890 <== mon.0 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34cc005670 con 0x7f34dc0ff1a0 2024-03-29T05:13:09.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.827+0000 7f34e2421640 1 -- 172.21.15.142:0/1393606890 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc196800 con 0x7f34dc0ff1a0 2024-03-29T05:13:09.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.828+0000 7f34e2421640 1 -- 172.21.15.142:0/1393606890 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc04e040 con 0x7f34dc0ff1a0 2024-03-29T05:13:09.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.829+0000 7f34d97fa640 1 -- 172.21.15.142:0/1393606890 <== mon.0 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 40) v1 ==== 91863+0+0 (secure 0 0 0) 0x7f34cc004dd0 con 0x7f34dc0ff1a0 2024-03-29T05:13:09.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.829+0000 7f34d97fa640 1 --2- 172.21.15.142:0/1393606890 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f34c406dd30 0x7f34c40701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T05:13:09.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.829+0000 7f34d97fa640 1 -- 172.21.15.142:0/1393606890 <== mon.0 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f34cc0b00e0 con 0x7f34dc0ff1a0 2024-03-29T05:13:09.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.829+0000 7f34db7fe640 1 --2- 172.21.15.142:0/1393606890 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f34c406dd30 0x7f34c40701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T05:13:09.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.830+0000 7f34db7fe640 1 --2- 172.21.15.142:0/1393606890 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f34c406dd30 0x7f34c40701f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f34d0005fd0 tx=0x7f34d000b590 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T05:13:09.843 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:09.838+0000 7f34d97fa640 1 -- 172.21.15.142:0/1393606890 <== mon.0 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f34cc079500 con 0x7f34dc0ff1a0 2024-03-29T05:13:10.037 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.031+0000 7f34e2421640 1 -- 172.21.15.142:0/1393606890 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f34dc196fe0 con 0x7f34dc0ff1a0 2024-03-29T05:13:10.037 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.032+0000 7f34d97fa640 1 -- 172.21.15.142:0/1393606890 <== mon.0 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v99) v1 ==== 81+0+2462 (secure 0 0 0) 0x7f34cc07cbc0 con 0x7f34dc0ff1a0 2024-03-29T05:13:10.037 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:6824/832699894 2024-03-30T04:54:33.202001+0000 2024-03-29T05:13:10.038 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6829/1941073852 2024-03-30T04:54:33.201978+0000 2024-03-29T05:13:10.038 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6828/1941073852 2024-03-30T04:54:33.201978+0000 2024-03-29T05:13:10.038 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6826/2071436089 2024-03-30T04:42:25.447288+0000 2024-03-29T05:13:10.038 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6801/2742619935 2024-03-30T04:36:06.145588+0000 2024-03-29T05:13:10.038 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6800/2395947042 2024-03-30T04:45:42.287114+0000 2024-03-29T05:13:10.038 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6801/2745119007 2024-03-30T04:35:33.215645+0000 2024-03-29T05:13:10.038 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/586295950 2024-03-30T04:37:17.579177+0000 2024-03-29T05:13:10.038 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:0/2558519425 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:10.038 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/2666008640 2024-03-30T04:35:33.215645+0000 2024-03-29T05:13:10.038 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:6825/832699894 2024-03-30T04:54:33.202001+0000 2024-03-29T05:13:10.038 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6827/2071436089 2024-03-30T04:42:25.447288+0000 2024-03-29T05:13:10.038 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:0/2307372541 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/484532431 2024-03-30T04:45:42.287114+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:0/3618978330 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/351083517 2024-03-30T04:49:59.502486+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/390665991 2024-03-30T04:36:06.145588+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/2545215192 2024-03-30T04:36:06.145588+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6800/2742619935 2024-03-30T04:36:06.145588+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/672130862 2024-03-30T04:37:17.579177+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/1851570418 2024-03-30T04:45:42.287114+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/3750656759 2024-03-30T04:45:42.287114+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:0/2082842835 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6801/2395947042 2024-03-30T04:45:42.287114+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:6828/3130987667 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:10.039 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/2370069599 2024-03-30T04:45:42.287114+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6800/2745119007 2024-03-30T04:35:33.215645+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/1199684691 2024-03-30T04:49:59.502486+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:0/2115311268 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:0/1140094447 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6800/1771239042 2024-03-30T04:49:59.502486+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.152:6829/3130987667 2024-03-30T04:46:54.194028+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/3751399855 2024-03-30T04:35:33.215645+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/1935271872 2024-03-30T04:35:33.215645+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/3338740178 2024-03-30T04:37:17.579177+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/966342870 2024-03-30T04:36:06.145588+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6801/779416791 2024-03-30T04:37:17.579177+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6800/779416791 2024-03-30T04:37:17.579177+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/2191110203 2024-03-30T04:49:59.502486+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:6801/1771239042 2024-03-30T04:49:59.502486+0000 2024-03-29T05:13:10.040 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142:0/143963075 2024-03-30T04:49:59.502486+0000 2024-03-29T05:13:10.041 INFO:teuthology.orchestra.run.smithi142.stderr:listed 41 entries 2024-03-29T05:13:10.041 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.034+0000 7f34e2421640 1 -- 172.21.15.142:0/1393606890 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f34c406dd30 msgr2=0x7f34c40701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:10.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.034+0000 7f34e2421640 1 --2- 172.21.15.142:0/1393606890 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f34c406dd30 0x7f34c40701f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f34d0005fd0 tx=0x7f34d000b590 comp rx=0 tx=0).stop 2024-03-29T05:13:10.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.034+0000 7f34e2421640 1 -- 172.21.15.142:0/1393606890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc0ff1a0 msgr2=0x7f34dc19bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T05:13:10.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.034+0000 7f34e2421640 1 --2- 172.21.15.142:0/1393606890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc0ff1a0 0x7f34dc19bad0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009230 tx=0x7f34cc004040 comp rx=0 tx=0).stop 2024-03-29T05:13:10.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.034+0000 7f34e2421640 1 -- 172.21.15.142:0/1393606890 shutdown_connections 2024-03-29T05:13:10.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.034+0000 7f34e2421640 1 --2- 172.21.15.142:0/1393606890 >> [v2:172.21.15.142:6800/3465978191,v1:172.21.15.142:6801/3465978191] conn(0x7f34c406dd30 0x7f34c40701f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:10.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.034+0000 7f34e2421640 1 --2- 172.21.15.142:0/1393606890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc19c010 0x7f34dc195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:10.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.034+0000 7f34e2421640 1 --2- 172.21.15.142:0/1393606890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc0ff1a0 0x7f34dc19bad0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T05:13:10.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.034+0000 7f34e2421640 1 -- 172.21.15.142:0/1393606890 >> 172.21.15.142:0/1393606890 conn(0x7f34dc0faa60 msgr2=0x7f34dc0fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T05:13:10.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.034+0000 7f34e2421640 1 -- 172.21.15.142:0/1393606890 shutdown_connections 2024-03-29T05:13:10.042 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-29T05:13:10.034+0000 7f34e2421640 1 -- 172.21.15.142:0/1393606890 wait complete. 2024-03-29T05:13:10.049 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:09 smithi142 ceph-mon[112475]: pgmap v746: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:13:10.049 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:09 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/609982439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T05:13:10.050 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:09 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/3934471346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T05:13:10.050 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:09 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/2719572755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T05:13:10.051 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi142.front.sepia.ceph.com... 2024-03-29T05:13:10.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T05:13:10.051 DEBUG:teuthology.orchestra.run.smithi142:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T05:13:10.079 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T05:13:10.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:09 smithi152 ceph-mon[88186]: pgmap v746: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:13:10.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:09 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/609982439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T05:13:10.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:09 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/3934471346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T05:13:10.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:09 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/2719572755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T05:13:11.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:10 smithi142 ceph-mon[112475]: from='client.? 172.21.15.142:0/1393606890' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-29T05:13:11.127 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:10 smithi142 ceph-mon[112475]: pgmap v747: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:13:11.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:10 smithi152 ceph-mon[88186]: from='client.? 172.21.15.142:0/1393606890' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-29T05:13:11.296 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:10 smithi152 ceph-mon[88186]: pgmap v747: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:13:13.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:13 smithi142 ceph-mon[112475]: pgmap v748: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:13:13.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:13 smithi152 ceph-mon[88186]: pgmap v748: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:13:15.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:15 smithi142 ceph-mon[112475]: pgmap v749: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:13:15.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:13:15.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:15 smithi152 ceph-mon[88186]: pgmap v749: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:13:15.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:13:17.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:17 smithi142 ceph-mon[112475]: pgmap v750: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:13:17.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:17 smithi152 ceph-mon[88186]: pgmap v750: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:13:19.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:19 smithi142 ceph-mon[112475]: pgmap v751: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:13:19.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:19 smithi152 ceph-mon[88186]: pgmap v751: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:13:21.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:21 smithi142 ceph-mon[112475]: pgmap v752: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:13:21.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:21 smithi152 ceph-mon[88186]: pgmap v752: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:13:23.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:23 smithi142 ceph-mon[112475]: pgmap v753: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:13:23.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:23 smithi152 ceph-mon[88186]: pgmap v753: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:13:25.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:25 smithi142 ceph-mon[112475]: pgmap v754: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:13:25.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:25 smithi152 ceph-mon[88186]: pgmap v754: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:13:27.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:27 smithi142 ceph-mon[112475]: pgmap v755: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:13:27.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:27 smithi152 ceph-mon[88186]: pgmap v755: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:13:29.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:29 smithi142 ceph-mon[112475]: pgmap v756: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:13:29.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:29 smithi152 ceph-mon[88186]: pgmap v756: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:13:30.627 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:13:30.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:13:31.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:31 smithi152 ceph-mon[88186]: pgmap v757: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:13:31.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:31 smithi142 ceph-mon[112475]: pgmap v757: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:13:33.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:33 smithi152 ceph-mon[88186]: pgmap v758: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:13:33.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:33 smithi142 ceph-mon[112475]: pgmap v758: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:13:35.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:35 smithi152 ceph-mon[88186]: pgmap v759: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:13:35.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:35 smithi142 ceph-mon[112475]: pgmap v759: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:13:37.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:37 smithi152 ceph-mon[88186]: pgmap v760: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-29T05:13:37.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:37 smithi142 ceph-mon[112475]: pgmap v760: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-29T05:13:39.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:39 smithi152 ceph-mon[88186]: pgmap v761: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-29T05:13:39.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:39 smithi142 ceph-mon[112475]: pgmap v761: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-29T05:13:41.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:41 smithi152 ceph-mon[88186]: pgmap v762: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-29T05:13:41.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:41 smithi142 ceph-mon[112475]: pgmap v762: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-29T05:13:43.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:43 smithi152 ceph-mon[88186]: pgmap v763: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4 op/s 2024-03-29T05:13:43.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:43 smithi142 ceph-mon[112475]: pgmap v763: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 4 op/s 2024-03-29T05:13:45.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:45 smithi152 ceph-mon[88186]: pgmap v764: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-29T05:13:45.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:13:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:45 smithi142 ceph-mon[112475]: pgmap v764: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-29T05:13:45.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:13:47.546 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:47 smithi152 ceph-mon[88186]: pgmap v765: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-29T05:13:47.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:47 smithi142 ceph-mon[112475]: pgmap v765: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-29T05:13:49.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:49 smithi152 ceph-mon[88186]: pgmap v766: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:13:49.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:49 smithi142 ceph-mon[112475]: pgmap v766: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:13:51.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:51 smithi142 ceph-mon[112475]: pgmap v767: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:13:51.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:51 smithi152 ceph-mon[88186]: pgmap v767: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:13:53.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:53 smithi152 ceph-mon[88186]: pgmap v768: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:13:53.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:53 smithi142 ceph-mon[112475]: pgmap v768: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:13:55.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:55 smithi152 ceph-mon[88186]: pgmap v769: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:13:55.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:55 smithi142 ceph-mon[112475]: pgmap v769: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:13:57.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:57 smithi152 ceph-mon[88186]: pgmap v770: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:13:57.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:57 smithi142 ceph-mon[112475]: pgmap v770: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:13:59.678 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:13:59 smithi142 ceph-mon[112475]: pgmap v771: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:13:59.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:13:59 smithi152 ceph-mon[88186]: pgmap v771: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:14:00.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:14:00.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:14:01.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:01 smithi142 ceph-mon[112475]: pgmap v772: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:14:01.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:01 smithi152 ceph-mon[88186]: pgmap v772: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:14:03.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:03 smithi152 ceph-mon[88186]: pgmap v773: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:14:03.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:03 smithi142 ceph-mon[112475]: pgmap v773: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:14:05.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:05 smithi152 ceph-mon[88186]: pgmap v774: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:14:05.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:05 smithi142 ceph-mon[112475]: pgmap v774: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:14:07.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:07 smithi152 ceph-mon[88186]: pgmap v775: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:14:07.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:07 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:14:07.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:07 smithi142 ceph-mon[112475]: pgmap v775: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:14:07.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:07 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:14:08.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:08 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:14:08.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:08 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:14:08.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:08 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:14:08.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:08 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:14:08.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:08 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:14:08.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:08 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:14:09.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:09 smithi152 ceph-mon[88186]: pgmap v776: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:14:09.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:09 smithi142 ceph-mon[112475]: pgmap v776: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:14:11.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:11 smithi152 ceph-mon[88186]: pgmap v777: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:14:11.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:11 smithi142 ceph-mon[112475]: pgmap v777: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:14:13.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:13 smithi152 ceph-mon[88186]: pgmap v778: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:14:13.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:13 smithi142 ceph-mon[112475]: pgmap v778: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:14:15.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:15 smithi152 ceph-mon[88186]: pgmap v779: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:14:15.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:14:15.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:15 smithi142 ceph-mon[112475]: pgmap v779: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:14:15.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:14:17.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:17 smithi152 ceph-mon[88186]: pgmap v780: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:14:17.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:17 smithi142 ceph-mon[112475]: pgmap v780: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:14:19.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:19 smithi152 ceph-mon[88186]: pgmap v781: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:14:19.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:19 smithi142 ceph-mon[112475]: pgmap v781: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:14:21.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:21 smithi152 ceph-mon[88186]: pgmap v782: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:14:21.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:21 smithi142 ceph-mon[112475]: pgmap v782: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:14:23.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:23 smithi152 ceph-mon[88186]: pgmap v783: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:14:23.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:23 smithi142 ceph-mon[112475]: pgmap v783: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:14:25.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:25 smithi152 ceph-mon[88186]: pgmap v784: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:14:25.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:25 smithi142 ceph-mon[112475]: pgmap v784: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:14:27.742 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:27 smithi152 ceph-mon[88186]: pgmap v785: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:14:27.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:27 smithi142 ceph-mon[112475]: pgmap v785: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:14:29.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:29 smithi152 ceph-mon[88186]: pgmap v786: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:14:29.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:29 smithi142 ceph-mon[112475]: pgmap v786: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:14:30.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:14:30.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:14:31.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:31 smithi152 ceph-mon[88186]: pgmap v787: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:14:31.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:31 smithi142 ceph-mon[112475]: pgmap v787: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:14:33.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:33 smithi152 ceph-mon[88186]: pgmap v788: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:14:33.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:33 smithi142 ceph-mon[112475]: pgmap v788: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:14:35.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:35 smithi152 ceph-mon[88186]: pgmap v789: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:14:35.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:35 smithi142 ceph-mon[112475]: pgmap v789: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:14:37.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:37 smithi152 ceph-mon[88186]: pgmap v790: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:14:37.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:37 smithi142 ceph-mon[112475]: pgmap v790: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:14:39.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:39 smithi152 ceph-mon[88186]: pgmap v791: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:14:39.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:39 smithi142 ceph-mon[112475]: pgmap v791: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:14:41.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:41 smithi152 ceph-mon[88186]: pgmap v792: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:14:41.823 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:41 smithi142 ceph-mon[112475]: pgmap v792: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:14:43.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:43 smithi152 ceph-mon[88186]: pgmap v793: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:14:43.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:43 smithi142 ceph-mon[112475]: pgmap v793: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:14:45.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:45 smithi152 ceph-mon[88186]: pgmap v794: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:14:45.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:14:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:45 smithi142 ceph-mon[112475]: pgmap v794: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:14:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:14:47.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:47 smithi152 ceph-mon[88186]: pgmap v795: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:14:47.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:47 smithi142 ceph-mon[112475]: pgmap v795: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:14:49.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:49 smithi152 ceph-mon[88186]: pgmap v796: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:14:49.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:49 smithi142 ceph-mon[112475]: pgmap v796: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:14:51.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:51 smithi152 ceph-mon[88186]: pgmap v797: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:14:51.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:51 smithi142 ceph-mon[112475]: pgmap v797: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:14:53.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:53 smithi152 ceph-mon[88186]: pgmap v798: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:14:53.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:53 smithi142 ceph-mon[112475]: pgmap v798: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:14:55.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:55 smithi152 ceph-mon[88186]: pgmap v799: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:14:55.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:55 smithi142 ceph-mon[112475]: pgmap v799: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:14:57.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:57 smithi152 ceph-mon[88186]: pgmap v800: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:14:57.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:57 smithi142 ceph-mon[112475]: pgmap v800: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:14:59.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:14:59 smithi152 ceph-mon[88186]: pgmap v801: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:14:59.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:14:59 smithi142 ceph-mon[112475]: pgmap v801: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:15:00.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:15:00.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:15:01.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:01 smithi152 ceph-mon[88186]: pgmap v802: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:15:01.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:01 smithi142 ceph-mon[112475]: pgmap v802: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:15:03.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:03 smithi152 ceph-mon[88186]: pgmap v803: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:15:03.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:03 smithi142 ceph-mon[112475]: pgmap v803: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:15:05.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:05 smithi152 ceph-mon[88186]: pgmap v804: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:15:05.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:05 smithi142 ceph-mon[112475]: pgmap v804: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:15:07.798 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:07 smithi152 ceph-mon[88186]: pgmap v805: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:15:07.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:07 smithi142 ceph-mon[112475]: pgmap v805: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:15:08.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:08 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:15:08.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:08 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:15:09.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:09 smithi152 ceph-mon[88186]: pgmap v806: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:15:09.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:09 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:15:09.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:09 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:15:09.796 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:09 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:15:09.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:09 smithi142 ceph-mon[112475]: pgmap v806: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:15:09.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:09 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:15:09.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:09 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:15:09.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:09 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:15:11.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:11 smithi152 ceph-mon[88186]: pgmap v807: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:15:11.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:11 smithi142 ceph-mon[112475]: pgmap v807: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:15:13.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:13 smithi152 ceph-mon[88186]: pgmap v808: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:15:13.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:13 smithi142 ceph-mon[112475]: pgmap v808: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:15:15.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:15 smithi152 ceph-mon[88186]: pgmap v809: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:15:15.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:15:15.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:15 smithi142 ceph-mon[112475]: pgmap v809: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:15:15.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:15:17.626 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:17 smithi142 ceph-mon[112475]: pgmap v810: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:15:17.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:17 smithi152 ceph-mon[88186]: pgmap v810: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:15:19.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:19 smithi152 ceph-mon[88186]: pgmap v811: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:15:19.877 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:19 smithi142 ceph-mon[112475]: pgmap v811: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:15:21.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:21 smithi152 ceph-mon[88186]: pgmap v812: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:15:21.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:21 smithi142 ceph-mon[112475]: pgmap v812: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:15:23.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:23 smithi152 ceph-mon[88186]: pgmap v813: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:15:23.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:23 smithi142 ceph-mon[112475]: pgmap v813: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:15:25.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:25 smithi152 ceph-mon[88186]: pgmap v814: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:15:25.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:25 smithi142 ceph-mon[112475]: pgmap v814: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:15:27.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:27 smithi152 ceph-mon[88186]: pgmap v815: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:15:27.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:27 smithi142 ceph-mon[112475]: pgmap v815: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:15:29.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:29 smithi152 ceph-mon[88186]: pgmap v816: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:15:29.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:29 smithi142 ceph-mon[112475]: pgmap v816: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:15:30.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:30 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:15:30.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:30 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:15:31.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:31 smithi152 ceph-mon[88186]: pgmap v817: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:15:31.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:31 smithi142 ceph-mon[112475]: pgmap v817: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:15:33.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:33 smithi152 ceph-mon[88186]: pgmap v818: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:15:33.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:33 smithi142 ceph-mon[112475]: pgmap v818: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:15:35.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:35 smithi142 ceph-mon[112475]: pgmap v819: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:15:36.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:35 smithi152 ceph-mon[88186]: pgmap v819: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:15:37.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:37 smithi142 ceph-mon[112475]: pgmap v820: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:15:38.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:37 smithi152 ceph-mon[88186]: pgmap v820: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:15:39.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:39 smithi142 ceph-mon[112475]: pgmap v821: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:15:40.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:39 smithi152 ceph-mon[88186]: pgmap v821: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:15:41.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:41 smithi142 ceph-mon[112475]: pgmap v822: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:15:42.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:41 smithi152 ceph-mon[88186]: pgmap v822: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:15:43.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:43 smithi142 ceph-mon[112475]: pgmap v823: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:15:44.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:43 smithi152 ceph-mon[88186]: pgmap v823: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:15:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:45 smithi142 ceph-mon[112475]: pgmap v824: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:15:45.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:45 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:15:46.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:45 smithi152 ceph-mon[88186]: pgmap v824: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:15:46.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:45 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:15:47.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:47 smithi142 ceph-mon[112475]: pgmap v825: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:15:48.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:47 smithi152 ceph-mon[88186]: pgmap v825: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:15:49.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:49 smithi142 ceph-mon[112475]: pgmap v826: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:15:50.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:49 smithi152 ceph-mon[88186]: pgmap v826: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:15:51.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:51 smithi142 ceph-mon[112475]: pgmap v827: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:15:52.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:51 smithi152 ceph-mon[88186]: pgmap v827: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:15:53.878 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:53 smithi142 ceph-mon[112475]: pgmap v828: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:15:54.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:53 smithi152 ceph-mon[88186]: pgmap v828: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:15:55.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:55 smithi142 ceph-mon[112475]: pgmap v829: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:15:56.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:55 smithi152 ceph-mon[88186]: pgmap v829: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:15:57.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:57 smithi142 ceph-mon[112475]: pgmap v830: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:15:58.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:57 smithi152 ceph-mon[88186]: pgmap v830: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:15:59.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:15:59 smithi142 ceph-mon[112475]: pgmap v831: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:15:59.946 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:15:59 smithi152 ceph-mon[88186]: pgmap v831: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:16:00.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:00 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:16:01.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:00 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:16:01.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:01 smithi142 ceph-mon[112475]: pgmap v832: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:16:02.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:01 smithi152 ceph-mon[88186]: pgmap v832: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:16:03.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:03 smithi142 ceph-mon[112475]: pgmap v833: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:16:04.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:03 smithi152 ceph-mon[88186]: pgmap v833: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:16:05.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:05 smithi142 ceph-mon[112475]: pgmap v834: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:16:06.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:05 smithi152 ceph-mon[88186]: pgmap v834: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:16:07.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:07 smithi142 ceph-mon[112475]: pgmap v835: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:16:08.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:07 smithi152 ceph-mon[88186]: pgmap v835: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:16:09.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:09 smithi142 ceph-mon[112475]: pgmap v836: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:16:09.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:09 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:16:10.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:09 smithi152 ceph-mon[88186]: pgmap v836: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:16:10.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:09 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:16:10.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:16:10.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:16:10.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:16:11.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:16:11.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:16:11.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:16:11.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:11 smithi142 ceph-mon[112475]: pgmap v837: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:16:12.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:11 smithi152 ceph-mon[88186]: pgmap v837: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:16:13.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:13 smithi142 ceph-mon[112475]: pgmap v838: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:16:13.942 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:13 smithi152 ceph-mon[88186]: pgmap v838: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:16:15.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:15 smithi142 ceph-mon[112475]: pgmap v839: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:16:15.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:15 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:16:16.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:15 smithi152 ceph-mon[88186]: pgmap v839: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:16:16.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:15 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:16:17.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:17 smithi142 ceph-mon[112475]: pgmap v840: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:16:18.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:17 smithi152 ceph-mon[88186]: pgmap v840: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:16:19.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:19 smithi142 ceph-mon[112475]: pgmap v841: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:16:20.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:19 smithi152 ceph-mon[88186]: pgmap v841: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:16:21.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:21 smithi142 ceph-mon[112475]: pgmap v842: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:16:22.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:21 smithi152 ceph-mon[88186]: pgmap v842: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:16:23.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:23 smithi142 ceph-mon[112475]: pgmap v843: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:16:24.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:23 smithi152 ceph-mon[88186]: pgmap v843: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:16:25.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:25 smithi142 ceph-mon[112475]: pgmap v844: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:16:26.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:25 smithi152 ceph-mon[88186]: pgmap v844: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:16:27.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:27 smithi142 ceph-mon[112475]: pgmap v845: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:16:28.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:27 smithi152 ceph-mon[88186]: pgmap v845: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:16:29.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:29 smithi142 ceph-mon[112475]: pgmap v846: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:16:29.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:16:30.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:29 smithi152 ceph-mon[88186]: pgmap v846: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:16:30.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:16:31.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:31 smithi142 ceph-mon[112475]: pgmap v847: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:16:32.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:31 smithi152 ceph-mon[88186]: pgmap v847: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:16:34.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:33 smithi152 ceph-mon[88186]: pgmap v848: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:16:34.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:33 smithi142 ceph-mon[112475]: pgmap v848: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:16:36.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:35 smithi152 ceph-mon[88186]: pgmap v849: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:16:36.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:35 smithi142 ceph-mon[112475]: pgmap v849: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:16:38.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:37 smithi152 ceph-mon[88186]: pgmap v850: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:16:38.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:37 smithi142 ceph-mon[112475]: pgmap v850: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:16:40.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:39 smithi152 ceph-mon[88186]: pgmap v851: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:16:40.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:39 smithi142 ceph-mon[112475]: pgmap v851: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:16:41.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:41 smithi142 ceph-mon[112475]: pgmap v852: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:16:42.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:41 smithi152 ceph-mon[88186]: pgmap v852: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:16:44.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:43 smithi152 ceph-mon[88186]: pgmap v853: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:16:44.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:43 smithi142 ceph-mon[112475]: pgmap v853: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:16:45.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:16:45.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:16:46.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:45 smithi152 ceph-mon[88186]: pgmap v854: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:16:46.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:45 smithi142 ceph-mon[112475]: pgmap v854: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:16:48.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:47 smithi152 ceph-mon[88186]: pgmap v855: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:16:48.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:47 smithi142 ceph-mon[112475]: pgmap v855: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:16:50.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:49 smithi152 ceph-mon[88186]: pgmap v856: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:16:50.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:49 smithi142 ceph-mon[112475]: pgmap v856: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:16:51.876 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:51 smithi142 ceph-mon[112475]: pgmap v857: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:16:52.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:51 smithi152 ceph-mon[88186]: pgmap v857: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:16:53.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:53 smithi142 ceph-mon[112475]: pgmap v858: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:16:54.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:53 smithi152 ceph-mon[88186]: pgmap v858: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:16:56.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:55 smithi152 ceph-mon[88186]: pgmap v859: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:16:56.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:55 smithi142 ceph-mon[112475]: pgmap v859: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:16:58.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:57 smithi152 ceph-mon[88186]: pgmap v860: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:16:58.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:57 smithi142 ceph-mon[112475]: pgmap v860: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:16:59.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:59 smithi152 ceph-mon[88186]: pgmap v861: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:16:59.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:16:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:17:00.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:59 smithi142 ceph-mon[112475]: pgmap v861: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:17:00.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:16:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:17:01.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:01 smithi142 ceph-mon[112475]: pgmap v862: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:17:02.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:01 smithi152 ceph-mon[88186]: pgmap v862: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:17:04.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:03 smithi152 ceph-mon[88186]: pgmap v863: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:17:04.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:03 smithi142 ceph-mon[112475]: pgmap v863: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:17:06.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:05 smithi152 ceph-mon[88186]: pgmap v864: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:17:06.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:05 smithi142 ceph-mon[112475]: pgmap v864: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:17:08.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:07 smithi152 ceph-mon[88186]: pgmap v865: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:17:08.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:07 smithi142 ceph-mon[112475]: pgmap v865: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:17:10.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:09 smithi152 ceph-mon[88186]: pgmap v866: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:17:10.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:09 smithi142 ceph-mon[112475]: pgmap v866: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:17:11.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:17:11.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:17:11.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:17:11.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:10 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:17:11.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T05:17:11.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T05:17:11.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T05:17:11.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:10 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' 2024-03-29T05:17:11.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:11 smithi142 ceph-mon[112475]: pgmap v867: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:17:12.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:11 smithi152 ceph-mon[88186]: pgmap v867: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:17:14.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:13 smithi152 ceph-mon[88186]: pgmap v868: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:17:14.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:13 smithi142 ceph-mon[112475]: pgmap v868: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:17:15.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:14 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:17:15.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:14 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:17:16.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:15 smithi152 ceph-mon[88186]: pgmap v869: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:17:16.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:15 smithi142 ceph-mon[112475]: pgmap v869: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:17:18.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:17 smithi152 ceph-mon[88186]: pgmap v870: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:17:18.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:17 smithi142 ceph-mon[112475]: pgmap v870: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:17:20.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:19 smithi152 ceph-mon[88186]: pgmap v871: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:17:20.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:19 smithi142 ceph-mon[112475]: pgmap v871: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:17:21.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:21 smithi142 ceph-mon[112475]: pgmap v872: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:17:22.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:21 smithi152 ceph-mon[88186]: pgmap v872: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:17:24.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:23 smithi152 ceph-mon[88186]: pgmap v873: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:17:24.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:23 smithi142 ceph-mon[112475]: pgmap v873: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:17:26.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:25 smithi152 ceph-mon[88186]: pgmap v874: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:17:26.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:25 smithi142 ceph-mon[112475]: pgmap v874: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:17:28.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:27 smithi152 ceph-mon[88186]: pgmap v875: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:17:28.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:27 smithi142 ceph-mon[112475]: pgmap v875: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:17:30.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:29 smithi152 ceph-mon[88186]: pgmap v876: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:17:30.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:29 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:17:30.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:29 smithi142 ceph-mon[112475]: pgmap v876: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:17:30.126 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:29 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:17:31.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:31 smithi142 ceph-mon[112475]: pgmap v877: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:17:32.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:31 smithi152 ceph-mon[88186]: pgmap v877: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:17:34.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:33 smithi152 ceph-mon[88186]: pgmap v878: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:17:34.050 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:33 smithi142 ceph-mon[112475]: pgmap v878: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:17:36.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:35 smithi152 ceph-mon[88186]: pgmap v879: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:17:36.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:35 smithi142 ceph-mon[112475]: pgmap v879: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:17:38.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:37 smithi152 ceph-mon[88186]: pgmap v880: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:17:38.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:37 smithi142 ceph-mon[112475]: pgmap v880: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:17:40.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:39 smithi152 ceph-mon[88186]: pgmap v881: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:17:40.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:39 smithi142 ceph-mon[112475]: pgmap v881: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:17:41.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:41 smithi142 ceph-mon[112475]: pgmap v882: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:17:42.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:41 smithi152 ceph-mon[88186]: pgmap v882: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:17:44.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:43 smithi152 ceph-mon[88186]: pgmap v883: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:17:44.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:43 smithi142 ceph-mon[112475]: pgmap v883: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:17:45.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:44 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:17:45.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:44 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:17:46.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:45 smithi152 ceph-mon[88186]: pgmap v884: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:17:46.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:45 smithi142 ceph-mon[112475]: pgmap v884: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:17:48.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:47 smithi152 ceph-mon[88186]: pgmap v885: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:17:48.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:47 smithi142 ceph-mon[112475]: pgmap v885: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:17:50.023 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:49 smithi152 ceph-mon[88186]: pgmap v886: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:17:50.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:49 smithi142 ceph-mon[112475]: pgmap v886: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:17:51.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:51 smithi142 ceph-mon[112475]: pgmap v887: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:17:52.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:51 smithi152 ceph-mon[88186]: pgmap v887: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:17:54.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:53 smithi152 ceph-mon[88186]: pgmap v888: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:17:54.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:53 smithi142 ceph-mon[112475]: pgmap v888: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:17:56.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:55 smithi152 ceph-mon[88186]: pgmap v889: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:17:56.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:55 smithi142 ceph-mon[112475]: pgmap v889: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:17:58.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:57 smithi152 ceph-mon[88186]: pgmap v890: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:17:58.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:57 smithi142 ceph-mon[112475]: pgmap v890: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:18:00.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:59 smithi152 ceph-mon[88186]: pgmap v891: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:18:00.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:17:59 smithi152 ceph-mon[88186]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:18:00.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:59 smithi142 ceph-mon[112475]: pgmap v891: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-29T05:18:00.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:17:59 smithi142 ceph-mon[112475]: from='mgr.34104 172.21.15.142:0/4230194641' entity='mgr.smithi142.swesrg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T05:18:01.875 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:18:01 smithi142 ceph-mon[112475]: pgmap v892: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:18:02.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:18:01 smithi152 ceph-mon[88186]: pgmap v892: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-29T05:18:04.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:18:03 smithi152 ceph-mon[88186]: pgmap v893: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:18:04.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:18:03 smithi142 ceph-mon[112475]: pgmap v893: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-29T05:18:06.045 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:18:05 smithi152 ceph-mon[88186]: pgmap v894: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:18:06.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:18:05 smithi142 ceph-mon[112475]: pgmap v894: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-29T05:18:08.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:18:07 smithi152 ceph-mon[88186]: pgmap v895: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:18:08.125 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:18:07 smithi142 ceph-mon[112475]: pgmap v895: 65 pgs: 65 active+clean; 270 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-29T05:18:09.151 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-03-29T05:18:09.152 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_8da23624bc17fe0b1f797227e72a0db0ce7821ba/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_8da23624bc17fe0b1f797227e72a0db0ce7821ba/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-29T05:18:09.153 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-29T05:18:09.163 INFO:tasks.cephadm:Teardown begin 2024-03-29T05:18:09.163 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_8da23624bc17fe0b1f797227e72a0db0ce7821ba/qa/tasks/cephadm.py", line 1807, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_8da23624bc17fe0b1f797227e72a0db0ce7821ba/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_8da23624bc17fe0b1f797227e72a0db0ce7821ba/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-29T05:18:09.165 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-29T05:18:09.193 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-29T05:18:09.230 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-29T05:18:09.230 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-29T05:18:09.249 DEBUG:teuthology.orchestra.run.smithi152:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-29T05:18:09.290 INFO:tasks.cephadm:Stopping all daemons... 2024-03-29T05:18:09.291 INFO:tasks.cephadm.mon.smithi142:Stopping mon.smithi142... 2024-03-29T05:18:09.291 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi142 2024-03-29T05:18:09.511 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:18:09 smithi142 systemd[1]: Stopping Ceph mon.smithi142 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T05:18:09.776 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:18:09 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[112471]: 2024-03-29T05:18:09.646+0000 7f6a504ba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi142 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T05:18:09.776 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:18:09 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142[112471]: 2024-03-29T05:18:09.646+0000 7f6a504ba700 -1 mon.smithi142@0(leader) e3 *** Got Signal Terminated *** 2024-03-29T05:18:10.101 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:18:09 smithi142 podman[162648]: 2024-03-29 05:18:09.828976935 +0000 UTC m=+0.347380214 container died 434f713a3fd23a14db6943020b23e47e6aba693f3c53a4cc888153bd3e00945c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=) 2024-03-29T05:18:10.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:18:10 smithi142 podman[162648]: 2024-03-29 05:18:10.230192627 +0000 UTC m=+0.748595911 container cleanup 434f713a3fd23a14db6943020b23e47e6aba693f3c53a4cc888153bd3e00945c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142, io.buildah.version=1.33.5, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-29T05:18:10.375 INFO:journalctl@ceph.mon.smithi142.smithi142.stdout:Mar 29 05:18:10 smithi142 bash[162648]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi142 2024-03-29T05:18:10.698 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi142.service' 2024-03-29T05:18:10.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T05:18:10.744 INFO:tasks.cephadm.mon.smithi142:Stopped mon.smithi142 2024-03-29T05:18:10.744 INFO:tasks.cephadm.mon.smithi152:Stopping mon.smithi152... 2024-03-29T05:18:10.744 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi152 2024-03-29T05:18:11.044 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:18:10 smithi152 systemd[1]: Stopping Ceph mon.smithi152 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T05:18:11.414 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:18:11 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152[88182]: 2024-03-29T05:18:11.051+0000 7f04b053f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi152 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T05:18:11.415 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:18:11 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152[88182]: 2024-03-29T05:18:11.051+0000 7f04b053f700 -1 mon.smithi152@1(peon) e3 *** Got Signal Terminated *** 2024-03-29T05:18:11.415 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:18:11 smithi152 podman[125740]: 2024-03-29 05:18:11.16398835 +0000 UTC m=+0.227017664 container died 4ad4708e8405d5b4877453023a469ee756500171220edd9bfc6a176829a9244e (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, RELEASE=squid-8da2362, io.buildah.version=1.33.5) 2024-03-29T05:18:11.794 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:18:11 smithi152 podman[125740]: 2024-03-29 05:18:11.610490361 +0000 UTC m=+0.673519664 container cleanup 4ad4708e8405d5b4877453023a469ee756500171220edd9bfc6a176829a9244e (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152, RELEASE=squid-8da2362, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-29T05:18:11.795 INFO:journalctl@ceph.mon.smithi152.smithi152.stdout:Mar 29 05:18:11 smithi152 bash[125740]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-mon-smithi152 2024-03-29T05:18:12.026 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@mon.smithi152.service' 2024-03-29T05:18:12.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T05:18:12.072 INFO:tasks.cephadm.mon.smithi152:Stopped mon.smithi152 2024-03-29T05:18:12.072 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-29T05:18:12.073 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.0 2024-03-29T05:18:12.375 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:12 smithi142 systemd[1]: Stopping Ceph osd.0 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T05:18:12.876 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:12 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0[124243]: 2024-03-29T05:18:12.425+0000 7fe62b64c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T05:18:12.876 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:12 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0[124243]: 2024-03-29T05:18:12.426+0000 7fe62b64c700 -1 osd.0 99 *** Got signal Terminated *** 2024-03-29T05:18:12.876 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:12 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0[124243]: 2024-03-29T05:18:12.426+0000 7fe62b64c700 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T05:18:18.223 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:17 smithi142 podman[162780]: 2024-03-29 05:18:17.91664637 +0000 UTC m=+5.657858632 container died 5c47cbf77b860de266dfc47f6c8fd807c632d329bd2c2a7de52986c0931d53a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-29T05:18:18.625 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:18 smithi142 podman[162780]: 2024-03-29 05:18:18.318370357 +0000 UTC m=+6.059582608 container cleanup 5c47cbf77b860de266dfc47f6c8fd807c632d329bd2c2a7de52986c0931d53a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0, RELEASE=squid-8da2362, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-29T05:18:18.625 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:18 smithi142 bash[162780]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0 2024-03-29T05:18:19.049 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:18 smithi142 podman[162798]: 2024-03-29 05:18:18.645108217 +0000 UTC m=+0.728026759 container remove 5c47cbf77b860de266dfc47f6c8fd807c632d329bd2c2a7de52986c0931d53a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, GIT_BRANCH=HEAD) 2024-03-29T05:18:19.369 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:19 smithi142 podman[162891]: 2024-03-29 05:18:18.948871091 +0000 UTC m=+0.026424906 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T05:18:19.369 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:19 smithi142 podman[162891]: 2024-03-29 05:18:19.054318686 +0000 UTC m=+0.131872477 container create 6c9b5aa3856a0ac9477d25d9278772c518a01529f49367bed17affc930385018 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.schema-version=1.0, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-29T05:18:19.369 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:19 smithi142 podman[162891]: 2024-03-29 05:18:19.249090856 +0000 UTC m=+0.326644666 container init 6c9b5aa3856a0ac9477d25d9278772c518a01529f49367bed17affc930385018 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2) 2024-03-29T05:18:19.369 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:19 smithi142 podman[162891]: 2024-03-29 05:18:19.254039353 +0000 UTC m=+0.331593151 container start 6c9b5aa3856a0ac9477d25d9278772c518a01529f49367bed17affc930385018 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8da2362, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-03-29T05:18:19.369 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:19 smithi142 podman[162891]: 2024-03-29 05:18:19.3657344 +0000 UTC m=+0.443288201 container attach 6c9b5aa3856a0ac9477d25d9278772c518a01529f49367bed17affc930385018 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-29T05:18:19.625 INFO:journalctl@ceph.osd.0.smithi142.stdout:Mar 29 05:18:19 smithi142 podman[162891]: 2024-03-29 05:18:19.584267944 +0000 UTC m=+0.661821754 container died 6c9b5aa3856a0ac9477d25d9278772c518a01529f49367bed17affc930385018 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-29T05:18:20.217 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.0.service' 2024-03-29T05:18:20.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T05:18:20.262 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-29T05:18:20.262 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-29T05:18:20.262 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.1 2024-03-29T05:18:20.501 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:20 smithi142 systemd[1]: Stopping Ceph osd.1 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T05:18:20.875 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:20 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[129707]: 2024-03-29T05:18:20.656+0000 7fd21078f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T05:18:20.876 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:20 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[129707]: 2024-03-29T05:18:20.656+0000 7fd21078f700 -1 osd.1 99 *** Got signal Terminated *** 2024-03-29T05:18:20.876 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:20 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1[129707]: 2024-03-29T05:18:20.656+0000 7fd21078f700 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T05:18:26.436 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:26 smithi142 podman[163001]: 2024-03-29 05:18:26.16260484 +0000 UTC m=+5.634776802 container died fadc1526c831c6a43697478b628fd39fc116361a09aeb388a9dd8ac14c1dc3fd (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8da2362, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-29T05:18:26.820 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:26 smithi142 podman[163001]: 2024-03-29 05:18:26.556615497 +0000 UTC m=+6.028787456 container cleanup fadc1526c831c6a43697478b628fd39fc116361a09aeb388a9dd8ac14c1dc3fd (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-8da2362, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-29T05:18:26.820 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:26 smithi142 bash[163001]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1 2024-03-29T05:18:27.125 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:26 smithi142 podman[163029]: 2024-03-29 05:18:26.816323867 +0000 UTC m=+0.650072062 container remove fadc1526c831c6a43697478b628fd39fc116361a09aeb388a9dd8ac14c1dc3fd (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1, RELEASE=squid-8da2362, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T05:18:27.498 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:27 smithi142 podman[163119]: 2024-03-29 05:18:27.094581514 +0000 UTC m=+0.020974789 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T05:18:27.499 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:27 smithi142 podman[163119]: 2024-03-29 05:18:27.225699442 +0000 UTC m=+0.152092707 container create c48b98144f1246fe4626c615de278bf5caee32730a4ba72494c4fbc19b7f8550 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-8da2362, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-29T05:18:27.499 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:27 smithi142 podman[163119]: 2024-03-29 05:18:27.42011453 +0000 UTC m=+0.346507795 container init c48b98144f1246fe4626c615de278bf5caee32730a4ba72494c4fbc19b7f8550 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, org.label-schema.schema-version=1.0) 2024-03-29T05:18:27.499 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:27 smithi142 podman[163119]: 2024-03-29 05:18:27.423895067 +0000 UTC m=+0.350288338 container start c48b98144f1246fe4626c615de278bf5caee32730a4ba72494c4fbc19b7f8550 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, RELEASE=squid-8da2362, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-29T05:18:27.499 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:27 smithi142 podman[163119]: 2024-03-29 05:18:27.494840844 +0000 UTC m=+0.421234119 container attach c48b98144f1246fe4626c615de278bf5caee32730a4ba72494c4fbc19b7f8550 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-29T05:18:27.876 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 29 05:18:27 smithi142 podman[163119]: 2024-03-29 05:18:27.735059085 +0000 UTC m=+0.661452342 container died c48b98144f1246fe4626c615de278bf5caee32730a4ba72494c4fbc19b7f8550 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-29T05:18:28.393 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.1.service' 2024-03-29T05:18:28.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T05:18:28.441 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-29T05:18:28.441 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-29T05:18:28.442 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.2 2024-03-29T05:18:28.780 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:28 smithi142 systemd[1]: Stopping Ceph osd.2 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T05:18:29.126 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:28 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2[135168]: 2024-03-29T05:18:28.776+0000 7fc578304700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T05:18:29.126 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:28 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2[135168]: 2024-03-29T05:18:28.776+0000 7fc578304700 -1 osd.2 99 *** Got signal Terminated *** 2024-03-29T05:18:29.126 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:28 smithi142 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2[135168]: 2024-03-29T05:18:28.776+0000 7fc578304700 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T05:18:34.498 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:34 smithi142 podman[163226]: 2024-03-29 05:18:34.226854483 +0000 UTC m=+5.574780499 container died ce765131932542e8de854311b3bf5b71cedd9f40e73ff065e3b64ccb6a925edb (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, RELEASE=squid-8da2362, ceph=True, org.label-schema.license=GPLv2) 2024-03-29T05:18:34.878 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:34 smithi142 podman[163226]: 2024-03-29 05:18:34.584733825 +0000 UTC m=+5.932659831 container cleanup ce765131932542e8de854311b3bf5b71cedd9f40e73ff065e3b64ccb6a925edb (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8da2362) 2024-03-29T05:18:34.878 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:34 smithi142 bash[163226]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2 2024-03-29T05:18:35.319 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:34 smithi142 podman[163243]: 2024-03-29 05:18:34.911165856 +0000 UTC m=+0.688196781 container remove ce765131932542e8de854311b3bf5b71cedd9f40e73ff065e3b64ccb6a925edb (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-8da2362, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, ceph=True, org.label-schema.license=GPLv2) 2024-03-29T05:18:35.625 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:35 smithi142 podman[163336]: 2024-03-29 05:18:35.220596372 +0000 UTC m=+0.021859846 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T05:18:35.625 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:35 smithi142 podman[163336]: 2024-03-29 05:18:35.320695626 +0000 UTC m=+0.121959090 container create d2f360d25cb9bbc494395db035ea2ad7b36d13e6d522445cd5ad119749bc3166 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-deactivate, RELEASE=squid-8da2362, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.build-date=20240326) 2024-03-29T05:18:35.626 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:35 smithi142 podman[163336]: 2024-03-29 05:18:35.514825259 +0000 UTC m=+0.316088739 container init d2f360d25cb9bbc494395db035ea2ad7b36d13e6d522445cd5ad119749bc3166 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, RELEASE=squid-8da2362, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, ceph=True) 2024-03-29T05:18:35.626 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:35 smithi142 podman[163336]: 2024-03-29 05:18:35.519106803 +0000 UTC m=+0.320370273 container start d2f360d25cb9bbc494395db035ea2ad7b36d13e6d522445cd5ad119749bc3166 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, RELEASE=squid-8da2362, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-29T05:18:36.114 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:35 smithi142 podman[163336]: 2024-03-29 05:18:35.642348559 +0000 UTC m=+0.443612021 container attach d2f360d25cb9bbc494395db035ea2ad7b36d13e6d522445cd5ad119749bc3166 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, RELEASE=squid-8da2362, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-29T05:18:36.114 INFO:journalctl@ceph.osd.2.smithi142.stdout:Mar 29 05:18:35 smithi142 podman[163336]: 2024-03-29 05:18:35.818574867 +0000 UTC m=+0.619838337 container died d2f360d25cb9bbc494395db035ea2ad7b36d13e6d522445cd5ad119749bc3166 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-8da2362, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2) 2024-03-29T05:18:36.393 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.2.service' 2024-03-29T05:18:36.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T05:18:36.436 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-29T05:18:36.437 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-29T05:18:36.437 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.3 2024-03-29T05:18:36.733 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:36 smithi152 systemd[1]: Stopping Ceph osd.3 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T05:18:37.044 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:36 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3[97643]: 2024-03-29T05:18:36.728+0000 7f7fe4eb3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T05:18:37.044 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:36 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3[97643]: 2024-03-29T05:18:36.728+0000 7f7fe4eb3700 -1 osd.3 99 *** Got signal Terminated *** 2024-03-29T05:18:37.044 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:36 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3[97643]: 2024-03-29T05:18:36.728+0000 7f7fe4eb3700 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T05:18:40.545 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:40 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:40.168+0000 7f864f157700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:15.461612+0000 front 2024-03-29T05:18:15.461678+0000 (oldest deadline 2024-03-29T05:18:39.561685+0000) 2024-03-29T05:18:40.545 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:40 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:40.326+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:41.544 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:41 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:41.371+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:41.544 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:41 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:41.132+0000 7f864f157700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:15.461612+0000 front 2024-03-29T05:18:15.461678+0000 (oldest deadline 2024-03-29T05:18:39.561685+0000) 2024-03-29T05:18:42.527 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:42 smithi152 podman[125863]: 2024-03-29 05:18:42.2684034 +0000 UTC m=+5.641403565 container died c41086b4f6bc25351259a93f8fd5101ed1c927fe0098eaee399c7ee8584b68f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3, CEPH_POINT_RELEASE=, RELEASE=squid-8da2362, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-29T05:18:42.527 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:42 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:42.115+0000 7f864f157700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:15.461612+0000 front 2024-03-29T05:18:15.461678+0000 (oldest deadline 2024-03-29T05:18:39.561685+0000) 2024-03-29T05:18:42.528 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:42 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:42.328+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:42.794 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:42 smithi152 podman[125863]: 2024-03-29 05:18:42.613619832 +0000 UTC m=+5.986619983 container cleanup c41086b4f6bc25351259a93f8fd5101ed1c927fe0098eaee399c7ee8584b68f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, RELEASE=squid-8da2362, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-03-29T05:18:42.795 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:42 smithi152 bash[125863]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3 2024-03-29T05:18:43.110 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:42 smithi152 podman[125880]: 2024-03-29 05:18:42.815015254 +0000 UTC m=+0.545715400 container remove c41086b4f6bc25351259a93f8fd5101ed1c927fe0098eaee399c7ee8584b68f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6) 2024-03-29T05:18:43.363 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:43 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:43.105+0000 7f864f157700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:15.461612+0000 front 2024-03-29T05:18:15.461678+0000 (oldest deadline 2024-03-29T05:18:39.561685+0000) 2024-03-29T05:18:43.363 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:43 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:43.314+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:43.363 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:43 smithi152 podman[125969]: 2024-03-29 05:18:43.128681677 +0000 UTC m=+0.017550221 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T05:18:43.363 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:43 smithi152 podman[125969]: 2024-03-29 05:18:43.249376498 +0000 UTC m=+0.138245044 container create 1ccc1a80e0378b709b52f4cc0ed352ac278d3ea45f665ef068965d33b8f70682 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6) 2024-03-29T05:18:43.759 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:43 smithi152 podman[125969]: 2024-03-29 05:18:43.4609366 +0000 UTC m=+0.349805147 container init 1ccc1a80e0378b709b52f4cc0ed352ac278d3ea45f665ef068965d33b8f70682 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-29T05:18:43.759 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:43 smithi152 podman[125969]: 2024-03-29 05:18:43.464395635 +0000 UTC m=+0.353264188 container start 1ccc1a80e0378b709b52f4cc0ed352ac278d3ea45f665ef068965d33b8f70682 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True) 2024-03-29T05:18:43.760 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:43 smithi152 podman[125969]: 2024-03-29 05:18:43.560740072 +0000 UTC m=+0.449608626 container attach 1ccc1a80e0378b709b52f4cc0ed352ac278d3ea45f665ef068965d33b8f70682 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-deactivate, ceph=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T05:18:44.044 INFO:journalctl@ceph.osd.3.smithi152.stdout:Mar 29 05:18:43 smithi152 podman[125969]: 2024-03-29 05:18:43.75599474 +0000 UTC m=+0.644863299 container died 1ccc1a80e0378b709b52f4cc0ed352ac278d3ea45f665ef068965d33b8f70682 (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-29T05:18:44.376 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:44 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:44.296+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:44.377 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:44 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:44.146+0000 7f864f157700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:15.461612+0000 front 2024-03-29T05:18:15.461678+0000 (oldest deadline 2024-03-29T05:18:39.561685+0000) 2024-03-29T05:18:44.396 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.3.service' 2024-03-29T05:18:44.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T05:18:44.441 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-29T05:18:44.441 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-29T05:18:44.441 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.4 2024-03-29T05:18:44.771 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:44 smithi152 systemd[1]: Stopping Ceph osd.4 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T05:18:45.044 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:44 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:44.767+0000 7f8652362700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T05:18:45.044 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:44 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:44.767+0000 7f8652362700 -1 osd.4 99 *** Got signal Terminated *** 2024-03-29T05:18:45.044 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:44 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:44.767+0000 7f8652362700 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T05:18:45.544 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:45 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:45.125+0000 7f864f157700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:15.461612+0000 front 2024-03-29T05:18:15.461678+0000 (oldest deadline 2024-03-29T05:18:39.561685+0000) 2024-03-29T05:18:45.545 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:45 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:45.315+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:46.544 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:46 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:46.331+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:46.545 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:46 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:46.133+0000 7f864f157700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:15.461612+0000 front 2024-03-29T05:18:15.461678+0000 (oldest deadline 2024-03-29T05:18:39.561685+0000) 2024-03-29T05:18:47.544 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:47 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:47.155+0000 7f864f157700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:15.461612+0000 front 2024-03-29T05:18:15.461678+0000 (oldest deadline 2024-03-29T05:18:39.561685+0000) 2024-03-29T05:18:47.544 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:47 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:47.342+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:48.544 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:48 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:48.151+0000 7f864f157700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:15.461612+0000 front 2024-03-29T05:18:15.461678+0000 (oldest deadline 2024-03-29T05:18:39.561685+0000) 2024-03-29T05:18:48.545 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:48 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:48.375+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:48.545 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:48 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:48.375+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6814 osd.1 since back 2024-03-29T05:18:22.209804+0000 front 2024-03-29T05:18:22.209861+0000 (oldest deadline 2024-03-29T05:18:47.509905+0000) 2024-03-29T05:18:49.544 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:49 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4[103017]: 2024-03-29T05:18:49.169+0000 7f864f157700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:15.461612+0000 front 2024-03-29T05:18:15.461678+0000 (oldest deadline 2024-03-29T05:18:39.561685+0000) 2024-03-29T05:18:49.545 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:49 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:49.391+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:49.545 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:49 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:49.391+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6814 osd.1 since back 2024-03-29T05:18:22.209804+0000 front 2024-03-29T05:18:22.209861+0000 (oldest deadline 2024-03-29T05:18:47.509905+0000) 2024-03-29T05:18:50.388 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:50 smithi152 podman[126075]: 2024-03-29 05:18:50.110301387 +0000 UTC m=+5.445634261 container died eca640a33744335e0e28848261b5f5014ee010ded7484201dc7e3362b974607e (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8da2362, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.vendor=CentOS) 2024-03-29T05:18:50.652 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:50 smithi152 podman[126075]: 2024-03-29 05:18:50.484554918 +0000 UTC m=+5.819887786 container cleanup eca640a33744335e0e28848261b5f5014ee010ded7484201dc7e3362b974607e (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4, ceph=True, org.label-schema.build-date=20240326, RELEASE=squid-8da2362, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-29T05:18:50.652 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:50 smithi152 bash[126075]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4 2024-03-29T05:18:50.652 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:50 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:50.383+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:50.652 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:50 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:50.383+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6814 osd.1 since back 2024-03-29T05:18:22.209804+0000 front 2024-03-29T05:18:22.209861+0000 (oldest deadline 2024-03-29T05:18:47.509905+0000) 2024-03-29T05:18:51.046 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:50 smithi152 podman[126092]: 2024-03-29 05:18:50.76137829 +0000 UTC m=+0.651490482 container remove eca640a33744335e0e28848261b5f5014ee010ded7484201dc7e3362b974607e (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, GIT_BRANCH=HEAD) 2024-03-29T05:18:51.405 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:51 smithi152 podman[126187]: 2024-03-29 05:18:51.095723318 +0000 UTC m=+0.113301299 container create a57a021016635eddbc1a8d5e7899f4c97243e0330aeb5cd0e0ff35644dfa9c5a (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8da2362, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-29T05:18:51.405 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:51 smithi152 podman[126187]: 2024-03-29 05:18:51.000300401 +0000 UTC m=+0.017878360 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T05:18:51.405 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:51 smithi152 podman[126187]: 2024-03-29 05:18:51.340351421 +0000 UTC m=+0.357929369 container init a57a021016635eddbc1a8d5e7899f4c97243e0330aeb5cd0e0ff35644dfa9c5a (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-29T05:18:51.405 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:51 smithi152 podman[126187]: 2024-03-29 05:18:51.343623136 +0000 UTC m=+0.361201083 container start a57a021016635eddbc1a8d5e7899f4c97243e0330aeb5cd0e0ff35644dfa9c5a (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-8da2362, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-03-29T05:18:51.794 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:51 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:51.401+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:51.795 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:51 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:51.401+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6814 osd.1 since back 2024-03-29T05:18:22.209804+0000 front 2024-03-29T05:18:22.209861+0000 (oldest deadline 2024-03-29T05:18:47.509905+0000) 2024-03-29T05:18:51.795 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:51 smithi152 podman[126187]: 2024-03-29 05:18:51.431474706 +0000 UTC m=+0.449052656 container attach a57a021016635eddbc1a8d5e7899f4c97243e0330aeb5cd0e0ff35644dfa9c5a (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-8da2362, GIT_CLEAN=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-29T05:18:51.795 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:51 smithi152 podman[126187]: 2024-03-29 05:18:51.633736315 +0000 UTC m=+0.651314255 container died a57a021016635eddbc1a8d5e7899f4c97243e0330aeb5cd0e0ff35644dfa9c5a (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240326) 2024-03-29T05:18:52.186 INFO:journalctl@ceph.osd.4.smithi152.stdout:Mar 29 05:18:52 smithi152 podman[126213]: 2024-03-29 05:18:52.160385148 +0000 UTC m=+0.521519757 container remove a57a021016635eddbc1a8d5e7899f4c97243e0330aeb5cd0e0ff35644dfa9c5a (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8da2362, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-29T05:18:52.191 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.4.service' 2024-03-29T05:18:52.233 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T05:18:52.233 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-29T05:18:52.233 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-29T05:18:52.234 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.5 2024-03-29T05:18:52.544 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:52 smithi152 systemd[1]: Stopping Ceph osd.5 for 9f96302c-ed85-11ee-95d0-87774f69a715... 2024-03-29T05:18:52.544 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:52 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:52.413+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:52.545 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:52 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:52.413+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6814 osd.1 since back 2024-03-29T05:18:22.209804+0000 front 2024-03-29T05:18:22.209861+0000 (oldest deadline 2024-03-29T05:18:47.509905+0000) 2024-03-29T05:18:53.044 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:52 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:52.582+0000 7f8c8acfe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T05:18:53.044 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:52 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:52.582+0000 7f8c8acfe700 -1 osd.5 99 *** Got signal Terminated *** 2024-03-29T05:18:53.044 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:52 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:52.582+0000 7f8c8acfe700 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T05:18:53.794 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:53 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:53.440+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:53.794 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:53 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:53.440+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6814 osd.1 since back 2024-03-29T05:18:22.209804+0000 front 2024-03-29T05:18:22.209861+0000 (oldest deadline 2024-03-29T05:18:47.509905+0000) 2024-03-29T05:18:54.794 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:54 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:54.434+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:54.794 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:54 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:54.434+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6814 osd.1 since back 2024-03-29T05:18:22.209804+0000 front 2024-03-29T05:18:22.209861+0000 (oldest deadline 2024-03-29T05:18:47.509905+0000) 2024-03-29T05:18:55.677 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:55 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:55.399+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:55.677 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:55 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:55.399+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6814 osd.1 since back 2024-03-29T05:18:22.209804+0000 front 2024-03-29T05:18:22.209861+0000 (oldest deadline 2024-03-29T05:18:47.509905+0000) 2024-03-29T05:18:56.794 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:56 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:56.402+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:56.794 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:56 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:56.402+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6814 osd.1 since back 2024-03-29T05:18:22.209804+0000 front 2024-03-29T05:18:22.209861+0000 (oldest deadline 2024-03-29T05:18:47.509905+0000) 2024-03-29T05:18:56.795 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:56 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:56.402+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6822 osd.2 since back 2024-03-29T05:18:31.510943+0000 front 2024-03-29T05:18:31.510971+0000 (oldest deadline 2024-03-29T05:18:55.610919+0000) 2024-03-29T05:18:57.794 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:57 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:57.438+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6806 osd.0 since back 2024-03-29T05:18:12.908636+0000 front 2024-03-29T05:18:12.908612+0000 (oldest deadline 2024-03-29T05:18:39.309121+0000) 2024-03-29T05:18:57.794 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:57 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:57.438+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6814 osd.1 since back 2024-03-29T05:18:22.209804+0000 front 2024-03-29T05:18:22.209861+0000 (oldest deadline 2024-03-29T05:18:47.509905+0000) 2024-03-29T05:18:57.795 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:57 smithi152 ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5[108222]: 2024-03-29T05:18:57.438+0000 7f8c83aeb700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.142:6822 osd.2 since back 2024-03-29T05:18:31.510943+0000 front 2024-03-29T05:18:31.510971+0000 (oldest deadline 2024-03-29T05:18:55.610919+0000) 2024-03-29T05:18:58.360 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:58 smithi152 podman[126291]: 2024-03-29 05:18:58.08575103 +0000 UTC m=+5.637805617 container died dfaff5665341bf2d23dc830135c62ee4d3b2c8b6c5f81cc50d1e1db8d1628fac (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5, GIT_BRANCH=HEAD, RELEASE=squid-8da2362, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326) 2024-03-29T05:18:58.726 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:58 smithi152 podman[126291]: 2024-03-29 05:18:58.455104287 +0000 UTC m=+6.007158870 container cleanup dfaff5665341bf2d23dc830135c62ee4d3b2c8b6c5f81cc50d1e1db8d1628fac (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-8da2362, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-29T05:18:58.727 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:58 smithi152 bash[126291]: ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5 2024-03-29T05:18:59.022 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:58 smithi152 podman[126309]: 2024-03-29 05:18:58.723159683 +0000 UTC m=+0.632215406 container remove dfaff5665341bf2d23dc830135c62ee4d3b2c8b6c5f81cc50d1e1db8d1628fac (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-29T05:18:59.294 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:59 smithi152 podman[126400]: 2024-03-29 05:18:58.921737162 +0000 UTC m=+0.017817630 image pull 54112f57da446f72e18ea6d8bc84acd8c183ef52b079778a6eb8b11550032956 quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 2024-03-29T05:18:59.294 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:59 smithi152 podman[126400]: 2024-03-29 05:18:59.024112004 +0000 UTC m=+0.120192506 container create cd7777301f9db648de6387d8d2666884079f1e74bef43100995d3ce122bbb33c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, RELEASE=squid-8da2362, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-29T05:18:59.295 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:59 smithi152 podman[126400]: 2024-03-29 05:18:59.23578185 +0000 UTC m=+0.331862320 container init cd7777301f9db648de6387d8d2666884079f1e74bef43100995d3ce122bbb33c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8da2362, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-29T05:18:59.295 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:59 smithi152 podman[126400]: 2024-03-29 05:18:59.239267473 +0000 UTC m=+0.335347934 container start cd7777301f9db648de6387d8d2666884079f1e74bef43100995d3ce122bbb33c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8da2362, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-29T05:18:59.795 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:59 smithi152 podman[126400]: 2024-03-29 05:18:59.318583452 +0000 UTC m=+0.414663916 container attach cd7777301f9db648de6387d8d2666884079f1e74bef43100995d3ce122bbb33c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-8da2362, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-29T05:18:59.795 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 29 05:18:59 smithi152 podman[126400]: 2024-03-29 05:18:59.529800237 +0000 UTC m=+0.625880698 container died cd7777301f9db648de6387d8d2666884079f1e74bef43100995d3ce122bbb33c (image=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28, name=ceph-9f96302c-ed85-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=17c757dce51698639f1922a504f4809ecb6725a6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8da2362) 2024-03-29T05:19:00.125 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f96302c-ed85-11ee-95d0-87774f69a715@osd.5.service' 2024-03-29T05:19:00.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T05:19:00.167 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-29T05:19:00.168 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-29T05:19:00.365 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T05:19:04.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi142.stderr:ceph-fuse[77265]: fuse finished with error 0 and tester_r 0 2024-03-29T05:19:23.819 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-29T05:19:23.978 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T05:19:40.335 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-29T05:19:40.372 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-29T05:19:40.402 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-29T05:19:40.404 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119/remote/smithi142/crash 2024-03-29T05:19:40.405 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/crash -- . 2024-03-29T05:19:40.441 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-29T05:19:40.441 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-03-29T05:19:40.443 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119/remote/smithi152/crash 2024-03-29T05:19:40.444 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/crash -- . 2024-03-29T05:19:40.470 INFO:teuthology.orchestra.run.smithi152.stderr:tar: /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-29T05:19:40.470 INFO:teuthology.orchestra.run.smithi152.stderr:tar: Error is not recoverable: exiting now 2024-03-29T05:19:40.471 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-29T05:19:40.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-29T05:19:40.516 INFO:tasks.cephadm:Compressing logs... 2024-03-29T05:19:40.516 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-29T05:19:40.559 DEBUG:teuthology.orchestra.run.smithi152:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-29T05:19:40.584 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-29T05:19:40.584 INFO:teuthology.orchestra.run.smithi142.stderr:: No such file or directory 2024-03-29T05:19:40.585 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-29T05:19:40.585 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mgr.smithi142.swesrg.log 2024-03-29T05:19:40.585 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-29T05:19:40.590 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.1.log 2024-03-29T05:19:40.590 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mgr.smithi142.swesrg.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.0.log 2024-03-29T05:19:40.590 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.audit.log: /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-29T05:19:40.590 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-client.ceph-exporter.smithi142.log 2024-03-29T05:19:40.590 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.log 2024-03-29T05:19:40.590 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-client.ceph-exporter.smithi142.log: 85.9% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-29T05:19:40.590 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mon.smithi142.log 2024-03-29T05:19:40.596 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.log: 94.4% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-client.ceph-exporter.smithi142.log.gz 2024-03-29T05:19:40.596 INFO:teuthology.orchestra.run.smithi142.stderr: 91.4% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.audit.log.gzgzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.2.log 2024-03-29T05:19:40.596 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-03-29T05:19:40.596 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mon.smithi142.log: 89.3% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-29T05:19:40.596 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mds.cephfs.smithi142.bmkjzf.log 2024-03-29T05:19:40.597 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-29T05:19:40.598 INFO:teuthology.orchestra.run.smithi152.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-29T05:19:40.598 INFO:teuthology.orchestra.run.smithi152.stderr:: No such file or directory 2024-03-29T05:19:40.598 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-03-29T05:19:40.598 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mgr.smithi152.usoolh.log 2024-03-29T05:19:40.598 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-29T05:19:40.598 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mon.smithi152.log 2024-03-29T05:19:40.598 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.4.log 2024-03-29T05:19:40.598 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mgr.smithi152.usoolh.log: /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mon.smithi152.log: /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.5.log 2024-03-29T05:19:40.598 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mds.cephfs.smithi152.hiilum.log 2024-03-29T05:19:40.598 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.3.log 2024-03-29T05:19:40.598 INFO:teuthology.orchestra.run.smithi152.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-29T05:19:40.599 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mds.cephfs.smithi152.hiilum.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-29T05:19:40.599 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.3.log: 91.5% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-29T05:19:40.599 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.log 2024-03-29T05:19:40.599 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.cephadm.log: 85.8% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-29T05:19:40.599 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-29T05:19:40.600 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.log: 89.3% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-29T05:19:40.600 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mds.cephfs.smithi152.uhebsi.log 2024-03-29T05:19:40.609 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mds.cephfs.smithi142.bmkjzf.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mds.cephfs.smithi142.wgmndr.log 2024-03-29T05:19:40.610 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-volume.log: 90.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-29T05:19:40.611 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-client.ceph-exporter.smithi152.log 2024-03-29T05:19:40.629 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mds.cephfs.smithi142.wgmndr.log: 91.0% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-29T05:19:40.631 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mds.cephfs.smithi152.uhebsi.log: /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-client.ceph-exporter.smithi152.log: 91.0% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-29T05:19:40.632 INFO:teuthology.orchestra.run.smithi152.stderr: 94.4% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-client.ceph-exporter.smithi152.log.gz 2024-03-29T05:19:40.641 INFO:teuthology.orchestra.run.smithi152.stderr: 89.7% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mgr.smithi152.usoolh.log.gz 2024-03-29T05:19:41.055 INFO:teuthology.orchestra.run.smithi152.stderr: 92.5% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mon.smithi152.log.gz 2024-03-29T05:19:41.139 INFO:teuthology.orchestra.run.smithi142.stderr: 89.1% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mgr.smithi142.swesrg.log.gz 2024-03-29T05:19:42.095 INFO:teuthology.orchestra.run.smithi142.stderr: 90.3% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mon.smithi142.log.gz 2024-03-29T05:19:45.875 INFO:teuthology.orchestra.run.smithi142.stderr: 93.2% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.2.log.gz 2024-03-29T05:19:47.103 INFO:teuthology.orchestra.run.smithi152.stderr: 93.5% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.3.log.gz 2024-03-29T05:19:47.634 INFO:teuthology.orchestra.run.smithi152.stderr: 93.2% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.4.log.gz 2024-03-29T05:19:48.137 INFO:teuthology.orchestra.run.smithi142.stderr: 93.5% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.0.log.gz 2024-03-29T05:19:48.956 INFO:teuthology.orchestra.run.smithi142.stderr: 93.5% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.1.log.gz 2024-03-29T05:19:49.657 INFO:teuthology.orchestra.run.smithi152.stderr: 93.2% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-osd.5.log.gz 2024-03-29T05:19:50.150 INFO:teuthology.orchestra.run.smithi152.stderr: 95.1% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mds.cephfs.smithi152.hiilum.log.gz 2024-03-29T05:19:50.718 INFO:teuthology.orchestra.run.smithi142.stderr: 95.1% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mds.cephfs.smithi142.wgmndr.log.gz 2024-03-29T05:19:55.387 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-03-29T05:19:55.387 INFO:teuthology.orchestra.run.smithi152.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-03-29T05:19:57.509 INFO:teuthology.orchestra.run.smithi142.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-29T05:20:27.098 INFO:teuthology.orchestra.run.smithi152.stderr: 93.6% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mds.cephfs.smithi152.uhebsi.log.gz 2024-03-29T05:20:27.099 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-03-29T05:20:27.099 INFO:teuthology.orchestra.run.smithi152.stderr:real 0m46.527s 2024-03-29T05:20:27.099 INFO:teuthology.orchestra.run.smithi152.stderr:user 1m31.046s 2024-03-29T05:20:27.099 INFO:teuthology.orchestra.run.smithi152.stderr:sys 0m2.826s 2024-03-29T05:20:30.567 INFO:teuthology.orchestra.run.smithi142.stderr: 93.4% -- replaced with /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/ceph-mds.cephfs.smithi142.bmkjzf.log.gz 2024-03-29T05:20:30.569 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-03-29T05:20:30.569 INFO:teuthology.orchestra.run.smithi142.stderr:real 0m49.998s 2024-03-29T05:20:30.569 INFO:teuthology.orchestra.run.smithi142.stderr:user 1m36.843s 2024-03-29T05:20:30.569 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m3.302s 2024-03-29T05:20:30.569 INFO:tasks.cephadm:Archiving logs... 2024-03-29T05:20:30.570 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119/remote/smithi142/log 2024-03-29T05:20:30.571 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T05:20:43.944 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119/remote/smithi152/log 2024-03-29T05:20:43.944 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T05:20:56.695 INFO:tasks.cephadm:Removing cluster... 2024-03-29T05:20:56.695 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 --force 2024-03-29T05:20:56.853 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T05:20:57.213 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f96302c-ed85-11ee-95d0-87774f69a715 --force 2024-03-29T05:20:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: 9f96302c-ed85-11ee-95d0-87774f69a715 2024-03-29T05:20:57.746 INFO:tasks.cephadm:Removing cephadm ... 2024-03-29T05:20:57.746 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-29T05:20:57.763 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-29T05:20:57.779 INFO:tasks.cephadm:Teardown complete 2024-03-29T05:20:57.779 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-29T05:20:57.791 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_8da23624bc17fe0b1f797227e72a0db0ce7821ba/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_8da23624bc17fe0b1f797227e72a0db0ce7821ba/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-29T05:20:57.791 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-29T05:20:57.792 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T05:20:57.805 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T05:20:57.895 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-29T05:20:57.907 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-29T05:20:57.907 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T05:20:57.910 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T05:20:57.922 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-03-29T05:20:57.923 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-03-29T05:20:57.926 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T05:20:57.926 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-03-29T05:20:57.926 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 16 -341us[ -349us] +/- 78ms 2024-03-29T05:20:57.926 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv02.front.sepia.ceph.com 3 6 377 16 -1661us[-1670us] +/- 57ms 2024-03-29T05:20:57.926 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +2665us[+2665us] +/- 81ms 2024-03-29T05:20:57.926 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T05:20:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T05:20:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-03-29T05:20:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 15 -343us[ -359us] +/- 78ms 2024-03-29T05:20:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 -1665us[-1665us] +/- 57ms 2024-03-29T05:20:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:^* hv03.front.sepia.ceph.com 3 7 377 14 +2698us[+2681us] +/- 81ms 2024-03-29T05:20:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T05:20:57.929 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-29T05:20:57.940 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-29T05:20:57.941 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-29T05:20:57.967 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-29T05:20:58.054 DEBUG:teuthology.orchestra.run.smithi152:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-29T05:20:58.134 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T05:20:58.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T05:20:58.166 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T05:20:58.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T05:20:58.197 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-29T05:20:58.209 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-29T05:20:58.235 INFO:teuthology.task.internal:Duration was 3315.841529 seconds 2024-03-29T05:20:58.235 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-29T05:20:58.257 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-29T05:20:58.258 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T05:20:58.260 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T05:20:58.301 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T05:20:58.305 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T05:20:58.667 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-29T05:20:58.667 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-03-29T05:20:58.667 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T05:20:58.728 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2024-03-29T05:20:58.728 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T05:20:58.757 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-29T05:20:58.757 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T05:20:58.772 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T05:20:58.827 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-29T05:20:58.827 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T05:20:58.841 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T05:20:59.758 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-29T05:20:59.770 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-29T05:20:59.770 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T05:20:59.798 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T05:20:59.827 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-29T05:20:59.841 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T05:20:59.843 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T05:20:59.866 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-03-29T05:20:59.895 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2024-03-29T05:20:59.942 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T05:20:59.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T05:20:59.956 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T05:20:59.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T05:20:59.997 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-29T05:21:00.010 INFO:teuthology.task.internal:Transferring archived files... 2024-03-29T05:21:00.011 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119/remote/smithi142 2024-03-29T05:21:00.011 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T05:21:00.069 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119/remote/smithi152 2024-03-29T05:21:00.069 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T05:21:00.113 INFO:teuthology.task.internal:Removing archive directory... 2024-03-29T05:21:00.114 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T05:21:00.116 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T05:21:00.163 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-29T05:21:00.177 INFO:teuthology.task.internal:Not uploading archives. 2024-03-29T05:21:00.177 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-29T05:21:00.201 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-29T05:21:00.201 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T05:21:00.203 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T05:21:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: 262166 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 29 05:21 /home/ubuntu/cephtest 2024-03-29T05:21:00.215 INFO:teuthology.orchestra.run.smithi142.stdout: 556121 4 d--------- 2 ubuntu ubuntu 4096 Mar 29 04:42 /home/ubuntu/cephtest/mnt.0 2024-03-29T05:21:00.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T05:21:00.217 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-03-29T05:21:00.217 INFO:teuthology.orchestra.run.smithi142.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-03-29T05:21:00.217 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_8da23624bc17fe0b1f797227e72a0db0ce7821ba/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_8da23624bc17fe0b1f797227e72a0db0ce7821ba/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi142 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-03-29T05:21:00.217 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-29T05:21:00.231 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-29T05:21:00.270 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-29T05:21:00.270 INFO:teuthology.nuke:Checking targets against current locks 2024-03-29T05:21:00.297 DEBUG:teuthology.nuke:shortname: smithi142 2024-03-29T05:21:00.297 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T05:21:00.323 DEBUG:teuthology.nuke:shortname: smithi152 2024-03-29T05:21:00.323 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T05:21:00.327 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 04:17:19.785464', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKhuqw5ByIsVOWv4Bv0KqDWPRK5wkqtiEp0lF0GjNFCvvdH54FCOKPyJTmRhpWF4JlXASZzBBNDsjgxKMNjvZKw='} 2024-03-29T05:21:00.354 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 04:17:19.783753', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHXTdImcwoR3lXy6Keid7Lvx4jVNBDVmUGWuWyLP+wnSrYDEGyLhxMDcmu6M+EGBk1piqUuihrtjh4gZsJTn1BU='} 2024-03-29T05:21:00.383 INFO:teuthology.orchestra.console.smithi142:Power off 2024-03-29T05:21:00.384 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T05:21:00.407 DEBUG:teuthology.orchestra.console.smithi142:power off output: Chassis Power Control: Down/Off 2024-03-29T05:21:00.407 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T05:21:00.423 INFO:teuthology.orchestra.console.smithi152:Power off 2024-03-29T05:21:00.424 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T05:21:00.433 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-03-29T05:21:00.448 DEBUG:teuthology.orchestra.console.smithi152:power off output: Chassis Power Control: Down/Off 2024-03-29T05:21:00.448 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T05:21:00.471 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-03-29T05:21:04.435 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T05:21:04.472 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T05:21:04.545 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-03-29T05:21:04.582 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-03-29T05:21:08.550 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T05:21:08.583 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T05:21:08.661 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is off 2024-03-29T05:21:08.693 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-03-29T05:21:08.763 INFO:teuthology.orchestra.console.smithi142:Power off completed 2024-03-29T05:21:12.694 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T05:21:12.805 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is off 2024-03-29T05:21:12.906 INFO:teuthology.orchestra.console.smithi152:Power off completed 2024-03-29T05:21:13.011 INFO:teuthology.run:Summary data: description: upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 3315.8415286540985 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-03-29T05:21:13.012 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T05:21:13.110 INFO:teuthology.run:FAIL